|
|
@@ -0,0 +1,391 @@
|
|
|
+# v2.0 Storage Engine — Phase C Execution Plan
|
|
|
+
|
|
|
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. Each task ends with a buildable, testable, committable state.
|
|
|
+>
|
|
|
+> **Authoritative companions** (read these first):
|
|
|
+> - Architecture: `docs/superpowers/specs/2026-05-15-v2.0-storage-engine-architecture.md`
|
|
|
+> - Spike: `docs/superpowers/spikes/2026-05-15-storage-engine-spike.md`
|
|
|
+> - Phase C roadmap stub: `docs/superpowers/plans/2026-05-15-v2.0-storage-engine-phase-c.md`
|
|
|
+
|
|
|
+**Goal:** Replace the v1.x in-memory storage (MemoryStore + persistence subsystem) with an LMDB-backed substrate while preserving every v1.x feature on the gRPC wire surface. Ship as v2.0.0.
|
|
|
+
|
|
|
+**Architecture (in one sentence):** Each user collection becomes one LMDB sub-database (`<collection>` for documents) plus optional sidecar sub-dbs (`_vectors_<collection>`, `_history_<collection>`), with `_meta` / `_collections` / `_files` / `_views` for system state. Document ops are batched into single-writer LMDB transactions; reads use ad-hoc reader txns. Replication WAL stays at the document-op level (decoupled from substrate).
|
|
|
+
|
|
|
+**Naming rule (operator-mandated):** every metadata field on documents and every internal sub-database name starts with `_`. Already enforced for `_id`, `_created_at`, `_updated_at`, `_vector`, `_event`. v2.0 inherits and extends.
|
|
|
+
|
|
|
+**Tech Stack:** C++20, LMDB 0.9.31 (libsmartbotic-db-* runtime dep `liblmdb0`, build dep `liblmdb-dev`), yyjson (Phase A), doc_binary (Phase B), nlohmann::json, spdlog, jemalloc, LZ4, gRPC/Protobuf.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Plan structure
|
|
|
+
|
|
|
+The work is grouped into **stages**. Each stage produces a buildable, committable state. Stages are ordered so the v1.x → v2.0 boundary stays clean: build the new substrate, prove it independently, then integrate.
|
|
|
+
|
|
|
+1. **Stage 1 — LMDB substrate scaffolding.** New `service/src/storage/` directory with env + transaction wrappers and unit tests. Production binary still uses v1.x MemoryStore; new code is dead until Stage 4.
|
|
|
+2. **Stage 2 — Document storage layer.** `DocumentStore` interface + LMDB implementation. Tested standalone.
|
|
|
+3. **Stage 3 — Migration tool.** `--migrate-from-v1` mode that reads v1.x snapshots + WAL and populates a fresh LMDB env. Tested with synthetic v1.x data dirs.
|
|
|
+4. **Stage 4 — gRPC integration.** Swap handlers to use `DocumentStore`. Eviction machinery, recovery modes, snapshot/WAL files all deleted in this stage.
|
|
|
+5. **Stage 5 — Vector + history + files + views.** Sub-db wiring for the remaining v1.x features.
|
|
|
+6. **Stage 6 — Replication preservation.** Validate WAL streaming still works.
|
|
|
+7. **Stage 7 — Migrations RPC.** Update the migration runner to call the new storage layer.
|
|
|
+8. **Stage 8 — Eviction + recovery cleanup.** Confirm-and-delete the v1.x machinery declared dead in Stage 4.
|
|
|
+9. **Stage 9 — Performance gates + soak.** Multi-producer benchmark, RSS-under-cgroup, crash injection.
|
|
|
+10. **Stage 10 — Release.** VERSION bump, CLAUDE.md, deb build, operator-gated rollout.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 1 — LMDB substrate scaffolding
|
|
|
+
|
|
|
+**Files to create:**
|
|
|
+- `service/src/storage/lmdb_env.hpp` — RAII wrapper over `MDB_env*` with `mdb_env_create` / `mdb_env_set_mapsize` / `mdb_env_set_maxdbs` / `mdb_env_open` baked in. Exposes `begin_write()` / `begin_read()` returning RAII `WriteTxn` / `ReadTxn`.
|
|
|
+- `service/src/storage/lmdb_env.cpp` — implementation; the only file that includes `<lmdb.h>`.
|
|
|
+- `service/src/storage/lmdb_txn.hpp` — `WriteTxn` / `ReadTxn` types; commit-on-destruction with explicit `abort()` opt-out.
|
|
|
+- `service/src/storage/lmdb_dbi.hpp` — opens a named sub-database within a txn; convenience methods `put` / `get` / `del` / `cursor`.
|
|
|
+- `tests/test_lmdb_env.cpp` — tests env create/open, mapsize enforcement, multiple sub-db open, basic put/get round-trip, txn abort vs commit semantics.
|
|
|
+
|
|
|
+**Files to modify:**
|
|
|
+- `service/CMakeLists.txt` — add `pkg_check_modules(LMDB REQUIRED lmdb)` (or find_path/find_library fallback), append `${LMDB_LIBRARIES}` + `${LMDB_INCLUDE_DIRS}` to the service target. Add `src/storage/lmdb_env.cpp` to `DATABASE_SERVICE_SOURCES`.
|
|
|
+- `tests/CMakeLists.txt` — register `test_lmdb_env` mirroring `test_doc_binary` pattern (file-scope LMDB resolution near the top, target wiring, CTest registration).
|
|
|
+- `packaging/Dockerfile.base` — append `liblmdb-dev \` to apt-get install; bump base-image stamp `v4` → `v5`.
|
|
|
+- `packaging/deb/templates/control.server` — append `, liblmdb0` to `Depends:`.
|
|
|
+
|
|
|
+### Task 1.1 — Build deps + env stub
|
|
|
+
|
|
|
+- [ ] Add `liblmdb-dev` to `Dockerfile.base`, bump stamp to `v5`.
|
|
|
+- [ ] Add `, liblmdb0` to `control.server` Depends.
|
|
|
+- [ ] Wire LMDB into `service/CMakeLists.txt` via `pkg_check_modules` or `find_path`/`find_library` (LMDB doesn't ship pkg-config on Debian 13).
|
|
|
+- [ ] Build verifies (link succeeds, no source uses LMDB yet).
|
|
|
+- [ ] Commit: `build: add lmdb build + runtime dependency (v2.0 scaffolding)`.
|
|
|
+
|
|
|
+### Task 1.2 — `LmdbEnv` RAII wrapper
|
|
|
+
|
|
|
+- [ ] Write `tests/test_lmdb_env.cpp` with failing tests for env creation, mapsize ≥ on-disk-size enforcement, opening multiple sub-dbs.
|
|
|
+- [ ] Implement `service/src/storage/lmdb_env.{hpp,cpp}` to pass tests.
|
|
|
+- [ ] Build and run tests: GREEN.
|
|
|
+- [ ] Commit: `feat(storage): LmdbEnv RAII wrapper`.
|
|
|
+
|
|
|
+### Task 1.3 — Transaction wrappers
|
|
|
+
|
|
|
+- [ ] Extend `tests/test_lmdb_env.cpp` with write/read txn round-trip tests (commit, abort, reader-during-write).
|
|
|
+- [ ] Implement `service/src/storage/lmdb_txn.hpp` with RAII commit-on-success / abort-on-exception.
|
|
|
+- [ ] Implement `service/src/storage/lmdb_dbi.hpp` with `put`/`get`/`del`/cursor primitives.
|
|
|
+- [ ] Build and test: GREEN.
|
|
|
+- [ ] Commit: `feat(storage): WriteTxn + ReadTxn + LmdbDbi primitives`.
|
|
|
+
|
|
|
+**Stage 1 done state:** new `storage/` module exists, has unit tests, doesn't yet touch the service binary's hot path. The whole stage is a "library addition" PR-shape — fully reviewable in isolation.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 2 — Document storage layer
|
|
|
+
|
|
|
+**Files to create:**
|
|
|
+- `service/src/storage/document_store.hpp` — `DocumentStore` interface: `put(collection, id, doc)`, `get(collection, id)`, `del(collection, id)`, `scan(collection, filter, sort, limit, offset)`, `count(collection)`.
|
|
|
+- `service/src/storage/document_store_lmdb.hpp/.cpp` — `LmdbDocumentStore` implementation. Wraps the v1.x `MemoryStore` interface enough to be a drop-in once Stage 4 lands.
|
|
|
+- `tests/test_document_store.cpp` — exercises the interface against synthetic docs. Includes filter/sort/projection regression tests against v1.x expected outputs.
|
|
|
+
|
|
|
+**Files to modify:**
|
|
|
+- `service/src/storage/document_store_lmdb.cpp` — uses `doc_binary::to_json_text` for encode, `parse_to_nlohmann` for decode (preserves Phase A/B work).
|
|
|
+
|
|
|
+### Task 2.1 — `DocumentStore` interface + tests
|
|
|
+
|
|
|
+- [ ] Write `tests/test_document_store.cpp` with failing tests: put/get/del round-trip, scan with filter on `_id`, scan with empty result, count, collection isolation (writes to collection A don't appear in collection B).
|
|
|
+- [ ] Define `DocumentStore` interface (pure-virtual or template).
|
|
|
+- [ ] Build verifies the interface.
|
|
|
+- [ ] Commit: `feat(storage): DocumentStore interface + failing tests`.
|
|
|
+
|
|
|
+### Task 2.2 — LMDB-backed implementation
|
|
|
+
|
|
|
+- [ ] Implement `LmdbDocumentStore` against `LmdbEnv`. One sub-db per collection (lazy-created on first write).
|
|
|
+- [ ] Tests pass.
|
|
|
+- [ ] Commit: `feat(storage): LmdbDocumentStore implementation`.
|
|
|
+
|
|
|
+### Task 2.3 — Filter + sort + projection parity tests
|
|
|
+
|
|
|
+- [ ] Port the existing v1.x filter/sort/projection coverage from `test_views.cpp` / `test_timestamp_precision.cpp` to exercise `LmdbDocumentStore`. Same input → same output as v1.x.
|
|
|
+- [ ] Tests pass.
|
|
|
+- [ ] Commit: `test(storage): port filter+sort+projection coverage to DocumentStore`.
|
|
|
+
|
|
|
+**Stage 2 done state:** `LmdbDocumentStore` is a working drop-in for `MemoryStore`'s document interface. Still not wired into the gRPC handlers; production binary still uses v1.x storage.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 3 — Migration tool
|
|
|
+
|
|
|
+**Files to create:**
|
|
|
+- `service/src/storage/migrate_v1_to_v2.hpp/.cpp` — one-shot migration: read v1.x `snapshots/` (latest), replay v1.x `wal/` after the snapshot, populate fresh LMDB env. Writes `_meta.schema_version = 2` on success.
|
|
|
+- `tests/test_migrate_v1_to_v2.cpp` — generates a synthetic v1.x data dir using the v1.x snapshot/WAL writers, runs migration, verifies round-trip equivalence (every doc in v1.x dir appears in v2.0 env with same `_id`, same content, same metadata).
|
|
|
+- `tests/load_test/migration_v1_to_v2.cpp` — realistic-scale: populate a v1.x dir with 1M docs (`load_test_crash_insert`-style), migrate, verify counts and a random sample.
|
|
|
+
|
|
|
+**Files to modify:**
|
|
|
+- `service/src/main.cpp` — recognize `--migrate-from-v1` CLI flag (and a `--no-auto-migrate` opt-out per the ADR). On startup, detect `snapshots/` + missing `_meta.schema_version` → run migration before opening for gRPC.
|
|
|
+
|
|
|
+### Task 3.1 — Migration core
|
|
|
+
|
|
|
+- [ ] Write `tests/test_migrate_v1_to_v2.cpp` with failing tests: empty v1.x dir → empty v2.0 env; one collection with N docs → same N docs in v2.0; deleted-in-WAL docs are absent in v2.0.
|
|
|
+- [ ] Implement `migrate_v1_to_v2`: read snapshot (via existing `service/src/persistence/snapshot.cpp` loader), then replay WAL (via existing `wal.cpp` reader), accumulate into a `LmdbDocumentStore`.
|
|
|
+- [ ] Tests pass.
|
|
|
+- [ ] Commit: `feat(storage): v1.x → v2.0 migration core`.
|
|
|
+
|
|
|
+### Task 3.2 — Auto-detect + CLI flags
|
|
|
+
|
|
|
+- [ ] Add `--migrate-from-v1` and `--no-auto-migrate` flags to `main.cpp` argv parsing.
|
|
|
+- [ ] Wire auto-detect: if `snapshots/` dir exists and `_meta.schema_version` is unset or < 2, run migration (unless `--no-auto-migrate`).
|
|
|
+- [ ] `sd_notify(STATUS="Migrating from v1.x: <docs_migrated>/<docs_total>")` updates during migration.
|
|
|
+- [ ] Build verifies; integration test in `tests/load_test/migration_v1_to_v2.cpp` writes a real v1.x data dir, boots a v2.0 binary, observes migration completing.
|
|
|
+- [ ] Commit: `feat: auto-migrate v1.x data dirs on boot`.
|
|
|
+
|
|
|
+### Task 3.3 — Vector migration
|
|
|
+
|
|
|
+- [ ] Extend the migration to extract `_vector` fields during the v1→v2 copy and write to `_vectors_<collection>` sub-db. JSON doc retains `_vector` for wire-format round-trip.
|
|
|
+- [ ] Test: a v1.x dir with vector collections migrates with all vectors readable from the new sub-db.
|
|
|
+- [ ] Commit: `feat(storage): migrate vector fields to _vectors_<collection> sub-db`.
|
|
|
+
|
|
|
+### Task 3.4 — Migration safety nets
|
|
|
+
|
|
|
+- [ ] On migration failure, do NOT overwrite v1.x snapshots/wal. Service refuses to start with operator-actionable error pointing at the v1.9.5 backup at `/var/backups/smartbotic-database/pre-upgrade-*-from-1.x/`.
|
|
|
+- [ ] On migration success, write `_meta.schema_version = 2`; subsequent boots see this and skip migration.
|
|
|
+- [ ] On migration in-progress (process killed mid-way), `_meta.schema_version` remains unset and the next boot reruns migration (idempotent — clears the partial LMDB env first).
|
|
|
+- [ ] Crash-injection test: kill -9 during migration, restart, verify clean re-migration.
|
|
|
+- [ ] Commit: `feat(storage): migration safety nets (failure, restart, idempotence)`.
|
|
|
+
|
|
|
+**Stage 3 done state:** v1.x data dirs migrate cleanly to v2.0 LMDB envs. Production binary still uses v1.x for live serving; v2.0 substrate is parallel.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 4 — gRPC integration (the big swap)
|
|
|
+
|
|
|
+**Files to modify:**
|
|
|
+- `service/src/database_service.cpp` — replace `std::unique_ptr<MemoryStore> store_` with `std::unique_ptr<DocumentStore> store_`. Boot path opens `LmdbEnv` instead of starting `PersistenceManager`.
|
|
|
+- `service/src/database_grpc_impl.cpp` — every handler that called `store_->insert(...)` / `update(...)` / `find(...)` continues to do so against the new interface. Most handler bodies don't change shape because Stage 2 made `LmdbDocumentStore` match `MemoryStore`'s public interface.
|
|
|
+- `service/src/main.cpp` — boot order: parse args → detect+migrate (Stage 3) → open LMDB env → construct `LmdbDocumentStore` → start gRPC.
|
|
|
+
|
|
|
+**Files to delete (in this stage; verify Stage 8 nothing depends on them):**
|
|
|
+- `service/src/memory_store.cpp/.hpp` — replaced.
|
|
|
+- `service/src/persistence/persistence_manager.cpp/.hpp` — replaced.
|
|
|
+- `service/src/persistence/snapshot.cpp/.hpp` — gone (LMDB has its own checkpoint).
|
|
|
+- `service/src/persistence/wal.cpp/.hpp` — gone (LMDB has its own journal). **But** the v1.x WAL reader code path is needed by Stage 3's migration; either keep it under `service/src/storage/v1_compat/` or inline the reader logic into `migrate_v1_to_v2.cpp`.
|
|
|
+
|
|
|
+### Task 4.1 — Swap `store_` type
|
|
|
+
|
|
|
+- [ ] Replace `MemoryStore` references with `DocumentStore` in `database_service.cpp`.
|
|
|
+- [ ] Build will fail at every call site that uses MemoryStore-specific methods not in the DocumentStore interface — track these as Task 4.2's inventory.
|
|
|
+- [ ] Commit: `feat: swap database_service to DocumentStore interface (WIP, build will break)`.
|
|
|
+
|
|
|
+### Task 4.2 — Migrate handler call sites
|
|
|
+
|
|
|
+- [ ] For each handler in `database_grpc_impl.cpp`, port to the new interface. Most are trivial (interface preserved). Edge cases: anything that touched eviction, WAL fallback for evicted docs, or recovery-mode logic — those features are gone in v2.0 (see ADR).
|
|
|
+- [ ] Tests in `tests/load_test/` (load_test_mixed, load_test_integrity, etc.) all pass.
|
|
|
+- [ ] Commit: `feat: migrate gRPC handlers to LmdbDocumentStore`.
|
|
|
+
|
|
|
+### Task 4.3 — Delete v1.x storage code
|
|
|
+
|
|
|
+- [ ] Remove `service/src/memory_store.{cpp,hpp}`, `persistence/persistence_manager.*`, `persistence/snapshot.*`, `persistence/wal.*` (or keep wal.cpp's reader under `storage/v1_compat/` for the migration tool).
|
|
|
+- [ ] Remove their entries from `service/CMakeLists.txt`'s `DATABASE_SERVICE_SOURCES`.
|
|
|
+- [ ] Remove eviction-specific test files (`test_eviction.cpp` — coverage subsumed by `test_lmdb_env` reader-cap tests if needed).
|
|
|
+- [ ] Build cleanly; all remaining tests pass.
|
|
|
+- [ ] Commit: `chore: remove v1.x MemoryStore + persistence (subsumed by LMDB substrate)`.
|
|
|
+
|
|
|
+**Stage 4 done state:** the production binary now uses LMDB end-to-end. v1.x storage code is gone. gRPC wire surface unchanged.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 5 — Vector, history, files, views
|
|
|
+
|
|
|
+### Task 5.1 — Vector sub-db integration
|
|
|
+
|
|
|
+- [ ] On Insert/Update/Patch with `_vector` field present and collection has `vector_dimension > 0`: write to `_vectors_<collection>` in the same txn as the doc.
|
|
|
+- [ ] On Delete: also delete from `_vectors_<collection>`.
|
|
|
+- [ ] `SimilaritySearch` handler opens `_vectors_<collection>` read txn, cursor-iterates, runs SIMD cosine (AVX2/SSE4.1 paths from v1.x preserved unchanged), returns top-N.
|
|
|
+- [ ] Test: `test_vector_storage.cpp` migrates to the new layout; all existing assertions pass.
|
|
|
+- [ ] Commit: `feat(vector): _vectors_<collection> sub-db wiring + SIMD search`.
|
|
|
+
|
|
|
+### Task 5.2 — History store rewrite
|
|
|
+
|
|
|
+- [ ] `_history_<collection>` sub-db keyed by `<doc_id>:<version>` storing JSON-text `DocumentVersion`. Only created when collection's `maxVersions > 0`.
|
|
|
+- [ ] On Update/Delete: emit history entry inside the same txn.
|
|
|
+- [ ] `GetVersion` / `ListVersions` handlers read from the sub-db.
|
|
|
+- [ ] Old `service/src/persistence/history_store.cpp` is deleted; its tests port to LMDB.
|
|
|
+- [ ] Commit: `feat(history): _history_<collection> sub-db replaces .hlog files`.
|
|
|
+
|
|
|
+### Task 5.3 — File store metadata in `_files`
|
|
|
+
|
|
|
+- [ ] `_files` sub-db stores file metadata (id → metadata + blob_ref); blob bytes stay in `files/` (unchanged from v1.x).
|
|
|
+- [ ] `UploadFile` / `DownloadFile` / file-CRUD handlers updated.
|
|
|
+- [ ] Existing file-store tests pass.
|
|
|
+- [ ] Commit: `feat(files): _files sub-db for metadata; blobs unchanged`.
|
|
|
+
|
|
|
+### Task 5.4 — Views in `_views`
|
|
|
+
|
|
|
+- [ ] `_views` sub-db stores view definitions (name → JSON-text View). Existing view CRUD handlers ported.
|
|
|
+- [ ] `test_views.cpp` ports to the new layout.
|
|
|
+- [ ] Commit: `feat(views): _views sub-db replaces in-memory view manager state`.
|
|
|
+
|
|
|
+**Stage 5 done state:** every v1.x feature on the gRPC surface works against LMDB. No functional regressions.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 6 — Replication preservation
|
|
|
+
|
|
|
+The ADR keeps WAL-streaming replication; only the local persistence layer changed. This stage validates that decoupling.
|
|
|
+
|
|
|
+### Task 6.1 — Replication WAL = LMDB-emitted record stream
|
|
|
+
|
|
|
+- [ ] Define a per-write WAL record stream — for each LMDB write txn, emit a sequence of document-op records (PUT/DELETE/VEC_PUT/VEC_DELETE) into the replication WAL ring buffer. This is the *replication* WAL — separate from LMDB's internal journal.
|
|
|
+- [ ] Followers consume the same record format as v1.x. Apply path unchanged.
|
|
|
+- [ ] `load_test_replication` passes.
|
|
|
+- [ ] Commit: `feat(replication): emit document-op records into replication WAL on LMDB commit`.
|
|
|
+
|
|
|
+### Task 6.2 — Replica boot from leader stream
|
|
|
+
|
|
|
+- [ ] Replica boot: open empty LMDB env, request snapshot+WAL stream from leader, apply records into LMDB.
|
|
|
+- [ ] `load_test_replica_eviction` passes — eviction no longer exists in v2.0, but the test exercises the boot path and apply-during-pressure scenario; rename / adapt as needed.
|
|
|
+- [ ] Commit: `feat(replication): replica bootstrap into LMDB`.
|
|
|
+
|
|
|
+**Stage 6 done state:** leader + follower run on LMDB; same wire protocol as v1.x.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 7 — Migrations RPC
|
|
|
+
|
|
|
+### Task 7.1 — Migration runner on LMDB
|
|
|
+
|
|
|
+- [ ] `service/src/migrations/migration_runner.cpp` ported to call `DocumentStore` methods instead of `MemoryStore`.
|
|
|
+- [ ] JSON migration files on disk (shadowman, callerai) unchanged.
|
|
|
+- [ ] Existing migration tests pass.
|
|
|
+- [ ] Commit: `feat(migrations): port migration runner to DocumentStore`.
|
|
|
+
|
|
|
+### Task 7.2 — `_meta.schema_version` migration log
|
|
|
+
|
|
|
+- [ ] Track applied migrations under `_meta.migrations_applied` (JSON-text array of migration IDs).
|
|
|
+- [ ] On boot, run pending migrations idempotently (same semantic as v1.x's "applied migrations" set, just persisted in `_meta`).
|
|
|
+- [ ] Commit: `feat(migrations): persist applied-migrations log in _meta sub-db`.
|
|
|
+
|
|
|
+**Stage 7 done state:** migrations RPC works against LMDB; existing consumer scripts unaffected.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 8 — Eviction + recovery cleanup
|
|
|
+
|
|
|
+Mostly already deleted in Stage 4. This stage is the **audit**: confirm nothing lingers, deprecation warnings, doc updates.
|
|
|
+
|
|
|
+### Task 8.1 — Eviction config deprecation
|
|
|
+
|
|
|
+- [ ] Remove `max_memory_mb`, `eviction_chunk_size`, `eviction_chunk_pause_ms`, `hot_write_floor_ms`, `memory_priority` from config schema. Emit a deprecation log line for one release if the operator's config still sets them.
|
|
|
+- [ ] Rename `max_memory_mb` → `buffer_pool_size_mb` in config schema with a back-compat shim that maps the old key.
|
|
|
+- [ ] Commit: `chore(config): deprecate eviction knobs; rename max_memory_mb to buffer_pool_size_mb`.
|
|
|
+
|
|
|
+### Task 8.2 — Recovery mode deprecation
|
|
|
+
|
|
|
+- [ ] Remove `recovery.mode` config handling. Emit a deprecation log line if set. Remove the `recovery.lock` codepath entirely.
|
|
|
+- [ ] `--recovery-mode` and `--force-readwrite` CLI flags become no-ops with a "deprecated, ignored" warning.
|
|
|
+- [ ] Commit: `chore(recovery): remove v1.x tiered-recovery modes; LMDB recovery is built-in`.
|
|
|
+
|
|
|
+### Task 8.3 — Memory-pressure events
|
|
|
+
|
|
|
+- [ ] Decide whether `MEMORY_PRESSURE_HIGH` / `MEMORY_EVICTION_BURST` events stay on the event bus. Probably yes — the substrate change doesn't eliminate the *concept* of pressure, just changes who manages it (OS page cache now). Wire them to LMDB cache miss-rate or to RSS-near-cgroup-limit signals.
|
|
|
+- [ ] Commit: `feat(events): MEMORY_PRESSURE_HIGH wired to LMDB cache pressure signal`.
|
|
|
+
|
|
|
+**Stage 8 done state:** all v1.x machinery declared dead in the ADR is gone or formally deprecated.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 9 — Performance gates + soak
|
|
|
+
|
|
|
+### Task 9.1 — Multi-producer write benchmark
|
|
|
+
|
|
|
+- [ ] `tests/bench_v2_write.cpp` runs N producer threads each emitting K puts/sec sustained. Measure p50/p99 commit latency, single-writer slot contention.
|
|
|
+- [ ] Gate: sustained ≥ Zoe-projected write rate (5k docs/sec) with p99 commit < 50 ms.
|
|
|
+- [ ] Commit: `test(bench): multi-producer write benchmark for v2.0`.
|
|
|
+
|
|
|
+### Task 9.2 — RSS under cgroup constraint
|
|
|
+
|
|
|
+- [ ] `tests/bench_v2_rss.cpp` loads a populated env into a process running under a 256 MB / 512 MB / 1 GB cgroup, exercises a hot/cold access pattern, reports RSS plateau.
|
|
|
+- [ ] Gate: RSS plateaus at cgroup limit (not OOM, not unbounded).
|
|
|
+- [ ] Commit: `test(bench): RSS-under-cgroup benchmark for v2.0`.
|
|
|
+
|
|
|
+### Task 9.3 — Crash injection
|
|
|
+
|
|
|
+- [ ] `tests/load_test/crash_injection_v2.cpp` writes N docs, kills the process mid-commit, restarts, verifies LMDB recovery is clean and no doc loss occurred (modulo the last uncommitted txn).
|
|
|
+- [ ] Gate: zero doc loss from any committed txn; LMDB recovery completes in < 1s on Zoe-shape data.
|
|
|
+- [ ] Commit: `test(crash): v2.0 crash-injection coverage`.
|
|
|
+
|
|
|
+### Task 9.4 — 24-hour soak
|
|
|
+
|
|
|
+- [ ] Real-world workload simulation against a Zoe-shape copy of production data. Sustained gRPC traffic for 24h. Watch RSS, write tput, error counts.
|
|
|
+- [ ] Gate: zero crashes, RSS plateau < cgroup limit, sustained tput ≥ v1.11 baseline.
|
|
|
+- [ ] Operator-supervised, not subagent-runnable.
|
|
|
+- [ ] Commit (post-soak): `test(soak): v2.0 24h soak baseline captured`.
|
|
|
+
|
|
|
+**Stage 9 done state:** v2.0 has measured numbers backing the ADR's claims. Risks (single-writer ceiling, bounded RSS) are validated or escalated.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Stage 10 — Release
|
|
|
+
|
|
|
+### Task 10.1 — VERSION + CLAUDE.md
|
|
|
+
|
|
|
+- [ ] Bump `VERSION` from `1.11.0` to `2.0.0`.
|
|
|
+- [ ] Add CLAUDE.md Key Features bullets:
|
|
|
+ - LMDB-backed storage substrate
|
|
|
+ - Auto-migrate v1.x data dirs
|
|
|
+ - Removed: in-memory MemoryStore, snapshot files, eviction machinery, tiered recovery modes
|
|
|
+ - Operator runbook for `buffer_pool_size_mb` resize, hot-copy via `mdb_env_copy2`, repair procedures
|
|
|
+- [ ] Commit: `release(v2.0.0): LMDB storage substrate (phase C)`.
|
|
|
+
|
|
|
+### Task 10.2 — deb build
|
|
|
+
|
|
|
+- [ ] `./packaging/build.sh --local` produces v2.0.0 debs.
|
|
|
+- [ ] Verify `liblmdb0` in `Depends:` and `ldd` shows `liblmdb.so.0` linked.
|
|
|
+- [ ] Verify the deb upgrade path: install v1.11.0 deb, populate a data dir, install v2.0.0 deb, observe auto-migration on first boot.
|
|
|
+- [ ] Commit: (none — packaging changes already in Stage 1).
|
|
|
+
|
|
|
+### Task 10.3 — Operator-gated rollout
|
|
|
+
|
|
|
+- [ ] Push branch.
|
|
|
+- [ ] Operator decision on `./packaging/build.sh --sync --suite trixie`.
|
|
|
+- [ ] Operator-supervised Zoe upgrade (v1.11.0 → v2.0.0); v1.9.5 backup-before-upgrade fires.
|
|
|
+- [ ] Smoke: gRPC works, data is intact, replication healthy, RSS bounded.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Out of scope (deferred to v2.1+)
|
|
|
+
|
|
|
+- Secondary user-defined indexes.
|
|
|
+- Online schema evolution (add column without rewriting docs).
|
|
|
+- Distributed clustering beyond v1.x's leader+follower.
|
|
|
+- LMDB writer-pool sharding (single-writer for v2.0).
|
|
|
+- Adaptive `buffer_pool_size_mb` auto-tune.
|
|
|
+- New gRPC RPCs.
|
|
|
+- Vector index variants (IVF, HNSW). v2.0 keeps the v1.x linear-scan SIMD path.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Success criteria
|
|
|
+
|
|
|
+1. Every `tests/test_*.cpp` and `tests/load_test/*` from v1.11 passes against the v2.0 binary (after the eviction-specific tests are removed/adapted per Stage 4 / Stage 8).
|
|
|
+2. A v1.11.0 data dir auto-migrates to v2.0.0 on first boot with no manual operator action (modulo `--no-auto-migrate` opt-out).
|
|
|
+3. Multi-producer write benchmark sustains ≥ 5k docs/sec with p99 commit < 50 ms.
|
|
|
+4. RSS under a 512 MB cgroup plateaus below the limit on a 5+ GB dataset.
|
|
|
+5. 24h soak completes with zero crashes and RSS bounded.
|
|
|
+6. v1.9.5 backup-before-upgrade still fires on v1.x → v2.0 upgrade (verified in throwaway VM).
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Open questions retired during implementation
|
|
|
+
|
|
|
+These were called out in the ADR's "What this ADR does NOT cover" section. Track resolutions in their own commits:
|
|
|
+
|
|
|
+- Exact LMDB env flag set (Task 1.2).
|
|
|
+- Write-batch sizing default (Stage 4 + tuned in Stage 9).
|
|
|
+- Writer queue ordering (Stage 4).
|
|
|
+- Operator runbook (Task 10.1).
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Risks (in priority order)
|
|
|
+
|
|
|
+1. **Single-writer ceiling under sustained Zoe write rate.** Mitigated by Task 9.1's benchmark gate; if exceeded, fall back to RocksDB candidate (would need its own spike and partial re-do of Stage 1-2).
|
|
|
+2. **Migration tool latency on large data dirs.** Operator-visible — `sd_notify(STATUS=...)` reports progress. If migration of Zoe-scale data exceeds 10 minutes, gate-block and consider parallelising.
|
|
|
+3. **LMDB mapsize set too low.** Operator-actionable — config knob with safe default (50% RAM). Document the resize procedure in CLAUDE.md.
|
|
|
+4. **Vector layout migration corner cases.** Mitigated by Task 3.3's vector-migration tests.
|
|
|
+5. **Replication WAL format change** (if needed). Followers built against v2.0 read the v2.0 format; legacy v1.x followers cannot follow a v2.0 leader. Operator must upgrade all replicas in lockstep — already implicit in the "no intermediate-release deployment" stance.
|
