docs: add v1.10/v1.11/v2.0 phase plans for storage engine rewrite

- 2026-05-15-v1.10-yyjson-phase-a.md — full TDD plan, 10 tasks,
  17 parse-site swaps, parse-time bench.
- 2026-05-15-v1.11-binary-docs-phase-b.md — 9 tasks; Task 1 is the
  binary-format brainstorm/decision gate (BSON vs custom tape vs
  yyjson mut_doc); subsequent tasks describe format-agnostic contracts
  the chosen format must satisfy.
- 2026-05-15-v2.0-storage-engine-phase-c.md — skeleton with only the
  build-vs-buy spike (Task 1) and the architecture brainstorm (Task 2);
  Tasks 3+ explicitly deferred to a follow-up execution plan authored
  after the spike outcome.

Other plan files in docs/superpowers/plans/ remain untracked pending
operator review.

docs/superpowers/plans/2026-05-15-v1.10-yyjson-phase-a.md

@@ -0,0 +1,775 @@
+# yyjson Hot-Path Swap (Phase A → v1.10.0) Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Replace `nlohmann::json::parse` with yyjson at every hot-path callsite (WAL replay, snapshot deserialize, history reads, gRPC request bodies, replication apply) while keeping `nlohmann::json` as the in-memory document representation. Phase A is a parse-speed win — not a memory win. Phase B is where memory drops.
+
+**Architecture:** Add yyjson as a build + runtime dep. Introduce one bridge helper (`parse_to_nlohmann(string_view) -> nlohmann::json`) that uses yyjson under the hood and rebuilds an `nlohmann::json` tree from the yyjson DOM. Every existing `nlohmann::json::parse(s)` callsite is rewritten to call this helper. The bridge is the *only* yyjson surface in the codebase for v1.10 — Phase B will later let yyjson buffers live longer and back binary doc storage.
+
+**Tech Stack:** C++20, yyjson 0.9+ (libyyjson-dev on Debian 13), nlohmann::json (unchanged), CMake/pkg-config, existing CTest suite.
+
+---
+
+## Context
+
+**The 17 parse callsites Phase A touches** (verified via `rg -n 'json::parse'` against the v1.9.5 tree):
+
+| File | Lines | What it parses |
+|------|-------|----------------|
+| `service/src/persistence/wal.cpp` | 233, 249 | WAL replay: doc data + collection options |
+| `service/src/persistence/snapshot.cpp` | 820, 929, 963 | Snapshot deserialize: collection opts, document data, version history |
+| `service/src/persistence/history_store.cpp` | 124 | Per-collection history (.hlog) version read |
+| `service/src/database_service.cpp` | 722 | Replication apply path |
+| `service/src/database_grpc_impl.cpp` | 109, 187, 222, 307, 362, 592, 708, 1288, 1344, 1397 | Insert / Update / Patch / Find / Subscribe / similarity-search / filter values |
+
+**Backward compatibility:** Wire and on-disk JSON bytes are unchanged. The swap is local to the parser implementation. A v1.10 binary must read v1.9.x snapshots/WAL byte-for-byte identically; tests verify this.
+
+**Why yyjson:** ~3-5× faster than nlohmann's parser on the kind of documents Zoe stores (modest nesting, mostly strings + ints). License is MIT. Header + tiny static lib (~30 KB). Already packaged for Debian 13 (`libyyjson-dev` / `libyyjson0`).
+
+**Build infrastructure baseline (verified):**
+- `packaging/Dockerfile.base` line 10 has the apt-get list; tag is `smartbotic-db-build-base:debian13:v3` (we bump to `v4`).
+- `service/CMakeLists.txt` uses `pkg_check_modules` for LZ4 and jemalloc — we follow the same pattern for yyjson.
+- `packaging/deb/templates/control.server` `Depends:` line: append `libyyjson0`.
+
+---
+
+## File Structure
+
+### Files to create
+
+- `service/src/json_parse.hpp` — bridge helper declaration: `nlohmann::json parse_to_nlohmann(std::string_view)` + `parse_to_nlohmann(const char*, size_t)`.
+- `service/src/json_parse.cpp` — yyjson-backed implementation. The ONLY file that includes `<yyjson.h>`.
+- `tests/test_json_parse.cpp` — unit tests for the bridge against nlohmann::json's own parser on the same inputs.
+- `tests/bench_json_parse.cpp` — micro-benchmark, std::chrono based, run by hand to capture the parse-time delta on a Zoe-shape snapshot.
+
+### Files to modify
+
+- `packaging/Dockerfile.base` — append `libyyjson-dev` to the apt-get install list, bump the base-image tag echoed at the bottom from `v3` to `v4`.
+- `packaging/deb/templates/control.server` — append `, libyyjson0` to the `Depends:` line.
+- `service/CMakeLists.txt` — add `pkg_check_modules(yyjson REQUIRED yyjson)` and link `${yyjson_LIBRARIES}` + include `${yyjson_INCLUDE_DIRS}`.
+- `tests/CMakeLists.txt` — register `test_json_parse` and `bench_json_parse`.
+- `service/src/persistence/wal.cpp` — 2 parse-site swaps.
+- `service/src/persistence/snapshot.cpp` — 3 parse-site swaps.
+- `service/src/persistence/history_store.cpp` — 1 parse-site swap.
+- `service/src/database_service.cpp` — 1 parse-site swap.
+- `service/src/database_grpc_impl.cpp` — 10 parse-site swaps.
+- `VERSION` — bump 1.9.5 → 1.10.0.
+- `CLAUDE.md` — add Key Features bullet documenting the yyjson swap.
+
+### Files NOT to touch
+
+- `service/src/document.hpp` — `Document::data` stays `nlohmann::json` for Phase A. Phase B is where this changes.
+- `client/` — client library does not parse JSON in any hot path; the proto bindings are already binary.
+- `proto/database.proto` — wire format unchanged.
+
+---
+
+## Tasks
+
+### Task 1 — Add yyjson build + runtime dependencies
+
+**Files:**
+- Modify: `packaging/Dockerfile.base:10-31`
+- Modify: `packaging/deb/templates/control.server:18` (the `Depends:` line)
+- Modify: `service/CMakeLists.txt:7-10` (where `pkg_check_modules` for LZ4 is declared)
+- Modify: `service/CMakeLists.txt:59-90` (where `target_link_libraries` clauses live)
+
+- [ ] **Step 1: Add libyyjson-dev to Dockerfile.base.** Open `packaging/Dockerfile.base` and append `libyyjson-dev \` to the apt-get install list (right after `libfmt-dev \`). Bump the echo at the bottom from `v3` to `v4`:
+
+```dockerfile
+    libfmt-dev \
+    libyyjson-dev \
+    dpkg-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# ... existing ENV/WORKDIR lines unchanged ...
+
+RUN echo "smartbotic-db-build-base:debian13:v4" > /etc/smartbotic-db-build-base
+```
+
+- [ ] **Step 2: Rebuild the Docker base image.**
+
+Run: `./packaging/build.sh --rebuild-base`
+Expected: builds image tagged `smartbotic-db-build-base:debian13:v4` with no errors.
+
+- [ ] **Step 3: Add libyyjson0 to runtime deb dependencies.** In `packaging/deb/templates/control.server`, append `, libyyjson0` to the `Depends:` line:
+
+```
+Depends: libsmartbotic-db-client (= {{VERSION}}), libssl3t64, liblz4-1, libsystemd0, libjemalloc2, libyyjson0
+```
+
+- [ ] **Step 4: Wire yyjson into CMake.** In `service/CMakeLists.txt`, locate the `find_package(PkgConfig)` block (line 7) and add a `pkg_check_modules` call:
+
+```cmake
+find_package(PkgConfig)
+if(PkgConfig_FOUND)
+    pkg_check_modules(LZ4 QUIET liblz4)
+    pkg_check_modules(yyjson REQUIRED yyjson)
+endif()
+```
+
+Then in the `target_link_libraries` / `target_include_directories` section, add after the jemalloc clause:
+
+```cmake
+target_include_directories(smartbotic-database PRIVATE ${yyjson_INCLUDE_DIRS})
+target_link_libraries(smartbotic-database PRIVATE ${yyjson_LIBRARIES})
+```
+
+- [ ] **Step 5: Verify the local build still works.**
+
+Run: `cmake -B build -G Ninja && cmake --build build -j$(nproc)`
+Expected: PASS. No new compile or link errors. yyjson is now linked but unused — that's fine for this task.
+
+- [ ] **Step 6: Commit.**
+
+```bash
+git add packaging/Dockerfile.base packaging/deb/templates/control.server service/CMakeLists.txt
+git commit -m "build: add yyjson build + runtime dependency (phase A scaffolding)"
+```
+
+---
+
+### Task 2 — Write the yyjson → nlohmann bridge helper
+
+**Files:**
+- Create: `service/src/json_parse.hpp`
+- Create: `service/src/json_parse.cpp`
+- Create: `tests/test_json_parse.cpp`
+- Modify: `tests/CMakeLists.txt` (register the new test executable)
+- Modify: `service/CMakeLists.txt` (add `json_parse.cpp` to the source list)
+
+- [ ] **Step 1: Write the failing test.** Create `tests/test_json_parse.cpp`:
+
+```cpp
+#include <cassert>
+#include <cstring>
+#include <iostream>
+#include <string>
+#include <nlohmann/json.hpp>
+#include "json_parse.hpp"
+
+using nlohmann::json;
+using smartbotic::db::parse_to_nlohmann;
+
+namespace {
+
+void check(bool cond, const char* msg) {
+    if (!cond) {
+        std::cerr << "FAIL: " << msg << "\n";
+        std::abort();
+    }
+}
+
+void test_empty_object() {
+    auto j = parse_to_nlohmann("{}");
+    check(j.is_object() && j.empty(), "empty object");
+}
+
+void test_empty_array() {
+    auto j = parse_to_nlohmann("[]");
+    check(j.is_array() && j.empty(), "empty array");
+}
+
+void test_primitives() {
+    check(parse_to_nlohmann("true").get<bool>() == true, "true");
+    check(parse_to_nlohmann("false").get<bool>() == false, "false");
+    check(parse_to_nlohmann("null").is_null(), "null");
+    check(parse_to_nlohmann("42").get<int64_t>() == 42, "int");
+    check(parse_to_nlohmann("-7").get<int64_t>() == -7, "negative int");
+    check(parse_to_nlohmann("3.14").get<double>() == 3.14, "double");
+    check(parse_to_nlohmann("\"hello\"").get<std::string>() == "hello", "string");
+}
+
+void test_nested() {
+    std::string s = R"({"a":1,"b":[2,3,{"c":"x"}],"d":null,"e":true})";
+    auto y = parse_to_nlohmann(s);
+    auto n = json::parse(s);
+    check(y == n, "nested equality with nlohmann");
+}
+
+void test_unicode_string() {
+    std::string s = R"({"name":"Árvíztűrő tükörfúrógép","emoji":"🚀"})";
+    auto y = parse_to_nlohmann(s);
+    auto n = json::parse(s);
+    check(y == n, "unicode equality");
+}
+
+void test_large_int() {
+    auto j = parse_to_nlohmann("9223372036854775807");
+    check(j.get<int64_t>() == 9223372036854775807LL, "int64 max");
+}
+
+void test_string_view_overload() {
+    std::string s = "{\"k\":1}";
+    auto j = parse_to_nlohmann(std::string_view(s));
+    check(j["k"].get<int>() == 1, "string_view overload");
+}
+
+void test_invalid_json_throws() {
+    bool threw = false;
+    try {
+        (void)parse_to_nlohmann("{not json");
+    } catch (const std::exception&) {
+        threw = true;
+    }
+    check(threw, "invalid json throws");
+}
+
+void test_corpus_equivalence() {
+    // Sample of doc shapes the production service sees.
+    const char* corpus[] = {
+        R"({"_id":"abc","name":"x","tags":["a","b"],"count":0})",
+        R"({"_id":"xyz","_vector":[0.1,-0.2,0.3,0.4],"meta":{"k":"v"}})",
+        R"({"_id":"e1","_event":"call.ended","at":1731628800123})",
+        R"([])",
+        R"([1,2,3,4,5,6,7,8,9,10])",
+        R"({"deeply":{"nested":{"object":{"is":{"fine":true}}}}})",
+    };
+    for (const char* s : corpus) {
+        check(parse_to_nlohmann(s) == json::parse(s), s);
+    }
+}
+
+}  // namespace
+
+int main() {
+    test_empty_object();
+    test_empty_array();
+    test_primitives();
+    test_nested();
+    test_unicode_string();
+    test_large_int();
+    test_string_view_overload();
+    test_invalid_json_throws();
+    test_corpus_equivalence();
+    std::cout << "test_json_parse: all passed\n";
+    return 0;
+}
+```
+
+- [ ] **Step 2: Register the test in CMake.** In `tests/CMakeLists.txt`, add (matching the style of `test_vector_storage`):
+
+```cmake
+add_executable(test_json_parse test_json_parse.cpp ${CMAKE_SOURCE_DIR}/service/src/json_parse.cpp)
+target_include_directories(test_json_parse PRIVATE
+    ${CMAKE_SOURCE_DIR}/service/src
+    ${yyjson_INCLUDE_DIRS})
+target_link_libraries(test_json_parse PRIVATE nlohmann_json::nlohmann_json ${yyjson_LIBRARIES})
+add_test(NAME test_json_parse COMMAND test_json_parse)
+```
+
+- [ ] **Step 3: Add `json_parse.cpp` to the service binary's source list** in `service/CMakeLists.txt` wherever the service source list is declared (search for any other `src/` .cpp file added to `smartbotic-database` and add `src/json_parse.cpp` alongside it).
+
+- [ ] **Step 4: Run test to verify it fails to *link* (helper not yet implemented).**
+
+Run: `cmake --build build -j$(nproc) --target test_json_parse`
+Expected: FAIL with `undefined reference to smartbotic::db::parse_to_nlohmann`.
+
+- [ ] **Step 5: Write the header.** Create `service/src/json_parse.hpp`:
+
+```cpp
+#pragma once
+
+#include <string_view>
+#include <nlohmann/json.hpp>
+
+namespace smartbotic::db {
+
+// Parses a JSON document using yyjson (read-only DOM) and materialises an
+// nlohmann::json tree from it. Drop-in replacement for nlohmann::json::parse
+// at hot paths where parse speed matters but we still want nlohmann semantics
+// downstream. Throws nlohmann::json::parse_error on malformed input so existing
+// catch sites keep working.
+nlohmann::json parse_to_nlohmann(std::string_view bytes);
+nlohmann::json parse_to_nlohmann(const char* data, size_t length);
+
+}  // namespace smartbotic::db
+```
+
+- [ ] **Step 6: Write the implementation.** Create `service/src/json_parse.cpp`:
+
+```cpp
+#include "json_parse.hpp"
+
+#include <yyjson.h>
+
+#include <stdexcept>
+#include <string>
+
+namespace smartbotic::db {
+
+namespace {
+
+nlohmann::json convert(yyjson_val* v) {
+    if (!v) {
+        return nullptr;
+    }
+    switch (yyjson_get_type(v)) {
+        case YYJSON_TYPE_NULL:
+            return nullptr;
+        case YYJSON_TYPE_BOOL:
+            return yyjson_get_bool(v);
+        case YYJSON_TYPE_NUM:
+            switch (yyjson_get_subtype(v)) {
+                case YYJSON_SUBTYPE_UINT:
+                    return yyjson_get_uint(v);
+                case YYJSON_SUBTYPE_SINT:
+                    return yyjson_get_sint(v);
+                case YYJSON_SUBTYPE_REAL:
+                default:
+                    return yyjson_get_real(v);
+            }
+        case YYJSON_TYPE_STR:
+            return std::string(yyjson_get_str(v), yyjson_get_len(v));
+        case YYJSON_TYPE_ARR: {
+            nlohmann::json out = nlohmann::json::array();
+            size_t idx, max;
+            yyjson_val* elem;
+            yyjson_arr_foreach(v, idx, max, elem) {
+                out.push_back(convert(elem));
+            }
+            return out;
+        }
+        case YYJSON_TYPE_OBJ: {
+            nlohmann::json out = nlohmann::json::object();
+            size_t idx, max;
+            yyjson_val* key;
+            yyjson_val* val;
+            yyjson_obj_foreach(v, idx, max, key, val) {
+                out[std::string(yyjson_get_str(key), yyjson_get_len(key))] = convert(val);
+            }
+            return out;
+        }
+        default:
+            return nullptr;
+    }
+}
+
+}  // namespace
+
+nlohmann::json parse_to_nlohmann(const char* data, size_t length) {
+    if (!data || length == 0) {
+        throw nlohmann::json::parse_error::create(
+            101, 0, "empty input", nullptr);
+    }
+    yyjson_read_err err{};
+    yyjson_doc* doc = yyjson_read_opts(
+        const_cast<char*>(data), length,
+        YYJSON_READ_NOFLAG, nullptr, &err);
+    if (!doc) {
+        std::string msg = "yyjson parse error: ";
+        msg += err.msg ? err.msg : "unknown";
+        throw nlohmann::json::parse_error::create(
+            101, static_cast<size_t>(err.pos), msg, nullptr);
+    }
+    try {
+        nlohmann::json out = convert(yyjson_doc_get_root(doc));
+        yyjson_doc_free(doc);
+        return out;
+    } catch (...) {
+        yyjson_doc_free(doc);
+        throw;
+    }
+}
+
+nlohmann::json parse_to_nlohmann(std::string_view bytes) {
+    return parse_to_nlohmann(bytes.data(), bytes.size());
+}
+
+}  // namespace smartbotic::db
+```
+
+- [ ] **Step 7: Run the test.**
+
+Run: `cmake --build build -j$(nproc) --target test_json_parse && ./build/tests/test_json_parse`
+Expected: `test_json_parse: all passed`.
+
+- [ ] **Step 8: Commit.**
+
+```bash
+git add service/src/json_parse.hpp service/src/json_parse.cpp tests/test_json_parse.cpp tests/CMakeLists.txt service/CMakeLists.txt
+git commit -m "feat(json): add yyjson-backed parse_to_nlohmann bridge helper"
+```
+
+---
+
+### Task 3 — Swap WAL replay parse sites
+
+**Files:**
+- Modify: `service/src/persistence/wal.cpp:233,249`
+
+- [ ] **Step 1: Add the include.** At the top of `service/src/persistence/wal.cpp`, near the existing `#include <nlohmann/json.hpp>`, add:
+
+```cpp
+#include "../json_parse.hpp"
+```
+
+- [ ] **Step 2: Swap the two parse calls.** Replace `nlohmann::json::parse(dataStr)` at line 233 with `smartbotic::db::parse_to_nlohmann(dataStr)`, and similarly at line 249 for `nlohmann::json::parse(optsStr)`.
+
+- [ ] **Step 3: Build the service.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: PASS.
+
+- [ ] **Step 4: Run the snapshot durability test.**
+
+Run: `./build/tests/test_snapshot_durability`
+Expected: PASS. This test exercises WAL replay against real on-disk bytes — proves the swap is binary-equivalent.
+
+- [ ] **Step 5: Run the WAL load test.**
+
+Run: `cd tests/load_test && ./load_test_crash_insert && ./load_test_crash_verify`
+Expected: both report success (no document loss, no checksum mismatch).
+
+- [ ] **Step 6: Commit.**
+
+```bash
+git add service/src/persistence/wal.cpp
+git commit -m "perf(wal): use yyjson for replay parses (phase A)"
+```
+
+---
+
+### Task 4 — Swap snapshot deserialize parse sites
+
+**Files:**
+- Modify: `service/src/persistence/snapshot.cpp:820,929,963`
+
+- [ ] **Step 1: Add the include.** At the top of `service/src/persistence/snapshot.cpp`, add:
+
+```cpp
+#include "../json_parse.hpp"
+```
+
+- [ ] **Step 2: Swap all three sites.**
+  - Line 820 — `nlohmann::json::parse(optionsJson)` → `smartbotic::db::parse_to_nlohmann(optionsJson)`
+  - Line 929 — `nlohmann::json::parse(docJson)` → `smartbotic::db::parse_to_nlohmann(docJson)`
+  - Line 963 — `nlohmann::json::parse(verJson)` → `smartbotic::db::parse_to_nlohmann(verJson)`
+
+- [ ] **Step 3: Build.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: PASS.
+
+- [ ] **Step 4: Run the snapshot durability test.**
+
+Run: `./build/tests/test_snapshot_durability`
+Expected: PASS. Covers atomic snapshot write + read-back equivalence.
+
+- [ ] **Step 5: Run vector + views tests** (snapshot covers their serialization too).
+
+Run: `./build/tests/test_vector_storage && ./build/tests/test_views`
+Expected: both PASS.
+
+- [ ] **Step 6: Commit.**
+
+```bash
+git add service/src/persistence/snapshot.cpp
+git commit -m "perf(snapshot): use yyjson for deserialize parses (phase A)"
+```
+
+---
+
+### Task 5 — Swap history_store version-read parse
+
+**Files:**
+- Modify: `service/src/persistence/history_store.cpp:124`
+
+- [ ] **Step 1: Add the include + swap the call.** Add `#include "../json_parse.hpp"` near the existing nlohmann include, then change line 124 from `nlohmann::json::parse(verJson)` to `smartbotic::db::parse_to_nlohmann(verJson)`.
+
+- [ ] **Step 2: Build.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: PASS.
+
+- [ ] **Step 3: Run the existing test suite.** History-store coverage rides on `test_snapshot_durability` and `test_eviction` (eviction restores from history when a doc is evicted-then-read).
+
+Run: `./build/tests/test_snapshot_durability && ./build/tests/test_eviction`
+Expected: both PASS.
+
+- [ ] **Step 4: Commit.**
+
+```bash
+git add service/src/persistence/history_store.cpp
+git commit -m "perf(history): use yyjson for hlog version-read parse (phase A)"
+```
+
+---
+
+### Task 6 — Swap replication apply parse
+
+**Files:**
+- Modify: `service/src/database_service.cpp:722`
+
+- [ ] **Step 1: Add the include + swap.** Add `#include "json_parse.hpp"` (same directory; relative path is just `"json_parse.hpp"`). Change line 722 from `nlohmann::json::parse(entry.data())` to `smartbotic::db::parse_to_nlohmann(entry.data())`.
+
+- [ ] **Step 2: Build.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: PASS.
+
+- [ ] **Step 3: Run the replication load test.**
+
+Run: `cd tests/load_test && ./load_test_replication`
+Expected: leader and follower converge on identical state; no parse errors logged.
+
+- [ ] **Step 4: Run the replica eviction load test.** This was the regression vector in 1.8.0 — re-run it as belt-and-braces.
+
+Run: `cd tests/load_test && ./load_test_replica_eviction`
+Expected: PASS.
+
+- [ ] **Step 5: Commit.**
+
+```bash
+git add service/src/database_service.cpp
+git commit -m "perf(replication): use yyjson for replicated entry parse (phase A)"
+```
+
+---
+
+### Task 7 — Swap gRPC handler parse sites
+
+**Files:**
+- Modify: `service/src/database_grpc_impl.cpp:109,187,222,307,362,592,708,1288,1344,1397`
+
+All ten sites are identical-shape rewrites: `nlohmann::json::parse(<expr>)` → `smartbotic::db::parse_to_nlohmann(<expr>)`. Listed individually so the reviewer can scan the diff per-RPC.
+
+- [ ] **Step 1: Add the include** at the top of `service/src/database_grpc_impl.cpp`:
+
+```cpp
+#include "json_parse.hpp"
+```
+
+- [ ] **Step 2: Swap line 109** (Insert handler — request data parse).
+- [ ] **Step 3: Swap line 187** (Insert handler — proto doc data parse).
+- [ ] **Step 4: Swap line 222** (Update handler — request data parse).
+- [ ] **Step 5: Swap line 307** (Patch handler — patch body parse).
+- [ ] **Step 6: Swap line 362** (Find handler — filter-value path).
+- [ ] **Step 7: Swap line 592** (Subscribe handler — initial-state parse).
+- [ ] **Step 8: Swap line 708** (Similarity-search handler — proto doc parse).
+- [ ] **Step 9: Swap line 1288** (Generic doc-from-proto parse helper).
+- [ ] **Step 10: Swap line 1344** (Filter `.value` parse).
+- [ ] **Step 11: Swap line 1397** (Nested filter `.value` parse).
+
+- [ ] **Step 12: Build.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: PASS.
+
+- [ ] **Step 13: Run the integration load tests.** These exercise the gRPC handlers end-to-end.
+
+Run: `cd tests/load_test && ./load_test && ./load_test_mixed && ./load_test_integrity && ./load_test_pinned`
+Expected: all PASS, no parse errors logged.
+
+- [ ] **Step 14: Run views + timestamp-precision tests** (cover Find/Filter handlers).
+
+Run: `./build/tests/test_views && ./build/tests/test_timestamp_precision`
+Expected: PASS.
+
+- [ ] **Step 15: Commit.**
+
+```bash
+git add service/src/database_grpc_impl.cpp
+git commit -m "perf(grpc): use yyjson for handler body parses (phase A)"
+```
+
+---
+
+### Task 8 — Parse-time micro-benchmark
+
+**Files:**
+- Create: `tests/bench_json_parse.cpp`
+- Modify: `tests/CMakeLists.txt`
+
+Establish a number for the operator: how much faster is yyjson on Zoe-shape data? Not a regression gate — informational, captured in the commit message.
+
+- [ ] **Step 1: Write the benchmark.** Create `tests/bench_json_parse.cpp`:
+
+```cpp
+#include <chrono>
+#include <fstream>
+#include <iostream>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include <nlohmann/json.hpp>
+#include "json_parse.hpp"
+
+int main(int argc, char** argv) {
+    if (argc < 2) {
+        std::cerr << "usage: bench_json_parse <path-to-json-corpus>\n"
+                  << "  corpus = one JSON document per line\n";
+        return 1;
+    }
+    std::ifstream f(argv[1]);
+    std::vector<std::string> docs;
+    std::string line;
+    while (std::getline(f, line)) {
+        if (!line.empty()) docs.push_back(line);
+    }
+    if (docs.empty()) {
+        std::cerr << "empty corpus\n";
+        return 1;
+    }
+
+    constexpr int kIters = 5;
+    auto bench = [&](const char* name, auto fn) {
+        double best = 1e18;
+        for (int i = 0; i < kIters; ++i) {
+            auto t0 = std::chrono::steady_clock::now();
+            size_t sink = 0;
+            for (const auto& s : docs) {
+                auto j = fn(s);
+                sink += j.size();
+            }
+            auto t1 = std::chrono::steady_clock::now();
+            double ms = std::chrono::duration<double, std::milli>(t1 - t0).count();
+            best = std::min(best, ms);
+            std::cout << name << " iter " << i << ": " << ms << " ms  (sink=" << sink << ")\n";
+        }
+        return best;
+    };
+
+    double nl = bench("nlohmann", [](const std::string& s) {
+        return nlohmann::json::parse(s);
+    });
+    double yy = bench("yyjson  ", [](const std::string& s) {
+        return smartbotic::db::parse_to_nlohmann(s);
+    });
+
+    std::cout << "\nbest nlohmann: " << nl << " ms\n";
+    std::cout << "best yyjson:   " << yy << " ms\n";
+    std::cout << "speedup:       " << (nl / yy) << "x\n";
+    return 0;
+}
+```
+
+- [ ] **Step 2: Register the benchmark in `tests/CMakeLists.txt`.** Match the test_json_parse pattern but DO NOT `add_test` it — this is a manual bench, not a regression gate:
+
+```cmake
+add_executable(bench_json_parse bench_json_parse.cpp ${CMAKE_SOURCE_DIR}/service/src/json_parse.cpp)
+target_include_directories(bench_json_parse PRIVATE
+    ${CMAKE_SOURCE_DIR}/service/src
+    ${yyjson_INCLUDE_DIRS})
+target_link_libraries(bench_json_parse PRIVATE nlohmann_json::nlohmann_json ${yyjson_LIBRARIES})
+```
+
+- [ ] **Step 3: Build the bench binary.**
+
+Run: `cmake --build build -j$(nproc) --target bench_json_parse`
+Expected: PASS.
+
+- [ ] **Step 4: Capture a Zoe-shape corpus.** Use any existing seeded test data — `tests/load_test/` writes JSON via the client; dump 10k docs to a file (one per line) using `smartbotic-db-cli export <collection>` against a populated local instance. If no instance is handy, generate synthetic shapes:
+
+```bash
+python3 -c "
+import json, random, sys
+for i in range(10000):
+    print(json.dumps({
+        '_id': f'doc{i}',
+        'name': f'name-{i}',
+        'tags': [f't{j}' for j in range(random.randint(0,5))],
+        'meta': {'k': i, 'flag': bool(i%2)},
+        'vec': [random.random() for _ in range(8)],
+    }))
+" > /tmp/bench-corpus.jsonl
+```
+
+- [ ] **Step 5: Run the benchmark.**
+
+Run: `./build/tests/bench_json_parse /tmp/bench-corpus.jsonl`
+Expected: yyjson at least 2× faster than nlohmann on this corpus. Record the numbers.
+
+- [ ] **Step 6: Commit the bench source with measured numbers in the commit body.**
+
+```bash
+git add tests/bench_json_parse.cpp tests/CMakeLists.txt
+git commit -m "$(cat <<'EOF'
+test: add bench_json_parse for phase A parse-time delta
+
+Measured on 10k synthetic Zoe-shape docs:
+  nlohmann: <N> ms
+  yyjson:   <N> ms
+  speedup:  <N>x
+EOF
+)"
+```
+
+---
+
+### Task 9 — Bump VERSION, update CLAUDE.md, build deb
+
+**Files:**
+- Modify: `VERSION`
+- Modify: `CLAUDE.md`
+
+- [ ] **Step 1: Bump VERSION.** Replace `1.9.5` with `1.10.0` in `/data/smartbotic-database/VERSION`.
+
+- [ ] **Step 2: Add a CLAUDE.md bullet under "Key Features"** (insert after the auto-backup bullet):
+
+> **yyjson hot-path parsing (v1.10.0+)** — every JSON parse on WAL replay, snapshot deserialize, history reads, gRPC request bodies, and replication apply now goes through `smartbotic::db::parse_to_nlohmann` (yyjson-backed). In-memory representation stays `nlohmann::json` until Phase B. Wire and on-disk JSON bytes unchanged.
+
+- [ ] **Step 3: Local deb build.**
+
+Run: `./packaging/build.sh --local`
+Expected: produces `dist/smartbotic-database_1.10.0-1_amd64.deb` and the three sibling debs.
+
+- [ ] **Step 4: Verify the deb declares libyyjson0 dependency.**
+
+Run: `dpkg-deb -I dist/smartbotic-database_1.10.0-1_amd64.deb | grep -i depends`
+Expected: `Depends:` line contains `libyyjson0`.
+
+- [ ] **Step 5: Verify the binary actually links yyjson.**
+
+Run: `dpkg-deb -x dist/smartbotic-database_1.10.0-1_amd64.deb /tmp/deb-check && ldd /tmp/deb-check/usr/sbin/smartbotic-database | grep -i yyjson`
+Expected: `libyyjson.so.0 => /usr/lib/x86_64-linux-gnu/libyyjson.so.0`.
+
+- [ ] **Step 6: Commit.**
+
+```bash
+git add VERSION CLAUDE.md
+git commit -m "release(v1.10.0): yyjson hot-path swap (phase A)"
+```
+
+---
+
+### Task 10 — Operator-gated rollout
+
+**Files:** none (process step)
+
+- [ ] **Step 1: Push the branch.** Operator decides timing — do NOT auto-push.
+
+```bash
+git push origin feat/v1.9.5-backup    # branch name stays even though it now carries v1.10 work
+```
+
+- [ ] **Step 2: Operator decision on sync.** `./packaging/build.sh --sync --suite trixie` publishes to repository.smartbotics.ai. Per CLAUDE.md and prior pattern, this requires explicit operator authorization — do NOT run it unprompted.
+
+- [ ] **Step 3: Zoe upgrade.** Same gate — only on explicit operator command. v1.9.5's auto-backup preinst will fire on the upgrade and snapshot the data dir to `/var/backups/smartbotic-database/pre-upgrade-*-from-1.9.5/`, which is exactly the rollback safety net Phase A inherits from the v1.9.5 work.
+
+---
+
+## Out of scope (Phase A)
+
+- **Binary in-memory document storage.** That is Phase B. Phase A's `Document::data` is still `nlohmann::json` — yyjson is a parser only.
+- **WAL/snapshot binary format.** Wire and on-disk bytes are byte-for-byte unchanged in v1.10.
+- **Serialisation (dump).** `nlohmann::json::dump()` callsites are untouched. yyjson's writer is fine but parse is the hot path; writer swap can wait.
+- **Migration tooling.** No `--migrate-from-v1` mode in Phase A — v1.10 reads v1.9.x data dirs natively.
+
+---
+
+## Success criteria
+
+1. All 17 `nlohmann::json::parse` callsites in the hot-path file list are gone (verified by `rg -n 'nlohmann::json::parse' service/src` returning zero hits, OR only hits in files explicitly listed as out-of-scope cold-path code).
+2. All existing tests pass: `test_snapshot_durability`, `test_eviction`, `test_views`, `test_vector_storage`, `test_timestamp_precision`, `test_config_dropins`.
+3. All load tests pass: `load_test`, `load_test_mixed`, `load_test_integrity`, `load_test_pinned`, `load_test_crash_insert/verify`, `load_test_replication`, `load_test_replica_eviction`.
+4. `bench_json_parse` reports ≥ 2× speedup over nlohmann on Zoe-shape corpus.
+5. Local deb builds, declares `libyyjson0` dep, and `ldd` confirms the runtime link.
+6. A clean v1.9.5 data dir boots cleanly on the v1.10.0 binary (manual smoke test in a throwaway VM, identical to the v1.9.5 deb's upgrade smoke test).
+
+---
+
+## Open questions
+
+None — Phase A is mechanical. Phase B's open question (binary format choice) lives in the Phase B plan.

docs/superpowers/plans/2026-05-15-v1.11-binary-docs-phase-b.md

@@ -0,0 +1,611 @@
+# Binary Lazy Document Storage (Phase B → v1.11.0) Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+>
+> **HARD GATE:** Task 1 is a brainstorming-and-decision task that MUST land first. Tasks 2-9 depend on the format decision Task 1 produces. Do NOT skip ahead — the encoder, the accessor surface, and the callsite migration all key off the chosen format.
+
+**Goal:** Change `Document::data` from a heap-allocated `nlohmann::json` AST to a compact binary buffer with a lazy parse-on-demand accessor. This is the phase that actually drops RSS on Zoe-shape workloads. WAL/snapshot bytes on disk stay JSON text — the on-disk format break is deferred to Phase C.
+
+**Architecture:** Documents in memory hold a `std::vector<uint8_t>` (binary-encoded JSON in the chosen format from Task 1) plus an optional cached `nlohmann::json` view materialised on first field access. Reads that need exactly one field decode just that field (fast). Reads that materialise the whole tree pay a one-shot decode + cache. Writes serialise `nlohmann::json` → binary at the boundary. WAL replay and snapshot deserialize parse JSON text from disk straight into binary (the bridge from Phase A already gets us yyjson; we add a yyjson→binary path here).
+
+**Tech Stack:** C++20, yyjson (from Phase A), one of {libbson, custom tape format, yyjson mut_doc} — chosen in Task 1, nlohmann::json (kept for the lazy view), existing CTest + load tests.
+
+---
+
+## Context
+
+**Why memory drops here, not in Phase A:** `nlohmann::json` is a discriminated-union AST. Every object node is a `std::map<std::string, json>`; every string is a `std::string`; every leaf carries a tag byte and (on libstdc++) an SSO buffer or a heap pointer. A modest 5-field document occupies ~600-1200 bytes on the heap. The same document re-encoded as BSON or a packed tape is ~80-200 bytes. Multiplied by Zoe-scale doc counts, that's where the GB-scale RSS lives.
+
+**Why we keep `nlohmann::json` semantics at the boundary:** ~44 callsites in `service/src/` (8 files) touch `doc.data["field"]`, `doc.data.contains("x")`, `doc.data.dump()`, etc. Rewriting all of them as part of Phase B's blast radius is feasible; rewriting them *and* introducing a new accessor pattern *and* swapping the storage format in one step is asking for trouble. The lazy accessor returns an `nlohmann::json` view so callsites that need full-AST semantics keep working; only hot single-field reads get a direct fast-path.
+
+**Wire compatibility:** gRPC `Document.data` stays a JSON-text string on the wire. The encode/decode happens at the gRPC boundary (which Phase A already touches). v1.10 clients see no change.
+
+**Snapshot/WAL compatibility:** v1.11's snapshot format is identical to v1.10 — documents serialise to JSON text on disk. A v1.10 binary can read v1.11 snapshots, and vice versa. Operator policy ("no intermediate-release upgrades") means we don't *require* this, but it's free and worth keeping.
+
+**Files in scope (verified):**
+
+| Path | Callsites | Notes |
+|------|-----------|-------|
+| `service/src/document.hpp` | 1 | The type definition itself — the load-bearing change |
+| `service/src/memory_store.cpp` | 17 | Most affected file; reads + writes the store |
+| `service/src/database_grpc_impl.cpp` | 8 | gRPC boundary — encode/decode happens here |
+| `service/src/migrations/migration_runner.cpp` | 7 | Migration ops touch doc.data |
+| `service/src/replication/conflict_resolver.cpp` | 5 | Conflict merge needs full-tree semantics |
+| `service/src/views/view_manager.cpp` | 2 | Projection / filter evaluation |
+| `service/src/encryption/encryption_manager.cpp` | 2 | Field-level encrypt/decrypt |
+| `service/src/config/collection_config_manager.cpp` | 2 | Reads collection options docs |
+| `tests/` | ~30-40 sites | Most tests construct docs via `Document::fromJson` so the blast radius is smaller than the raw count suggests |
+
+---
+
+## File Structure
+
+### Files to create
+
+- `docs/superpowers/specs/2026-05-15-binary-doc-format-decision.md` — the Task 1 deliverable. Produced by brainstorming. Specifies the chosen format, encoder/decoder contract, performance + size projections, and the rejected alternatives with reasons.
+- `service/src/doc_binary.hpp` — the chosen format's encoder/decoder. Header declares `encode(const nlohmann::json&) -> std::vector<uint8_t>`, `decode(span<const uint8_t>) -> nlohmann::json`, and a field-fast-path `get_field(span<const uint8_t>, string_view) -> std::optional<nlohmann::json>`.
+- `service/src/doc_binary.cpp` — implementation. The ONLY file that knows about the chosen format's wire bytes.
+- `tests/test_doc_binary.cpp` — round-trip tests, field-fast-path tests, malformed-input tests.
+- `tests/bench_doc_memory.cpp` — measures RSS delta on a fixed corpus before/after the swap.
+
+### Files to modify
+
+- `service/src/document.hpp` — `data` field type changes; new accessor methods added.
+- `service/src/memory_store.{hpp,cpp}` — all 17 callsites + hot-read paths.
+- `service/src/database_grpc_impl.cpp` — 8 callsites + the proto-↔-internal boundary.
+- `service/src/migrations/migration_runner.cpp` — 7 callsites.
+- `service/src/replication/conflict_resolver.cpp` — 5 callsites.
+- `service/src/views/view_manager.cpp` — 2 callsites.
+- `service/src/encryption/encryption_manager.cpp` — 2 callsites.
+- `service/src/config/collection_config_manager.cpp` — 2 callsites.
+- `service/src/persistence/snapshot.cpp` — serialiser still emits JSON text; deserialiser now goes JSON-text → binary directly (skips intermediate nlohmann tree on the hot path).
+- `service/src/persistence/wal.cpp` — same pattern: parse-and-encode on the way in, decode-and-emit on the way out.
+- `service/src/persistence/history_store.cpp` — same.
+- `VERSION` — 1.10.0 → 1.11.0.
+- `CLAUDE.md` — Key Features bullet for binary in-memory docs.
+
+### Files NOT to touch
+
+- `proto/database.proto` — wire format unchanged.
+- `client/` — clients see no change.
+- `tests/` snapshot fixtures — on-disk JSON-text is unchanged.
+
+---
+
+## Tasks
+
+### Task 1 — Brainstorm and decide the binary format
+
+**Files:**
+- Create: `docs/superpowers/specs/2026-05-15-binary-doc-format-decision.md`
+
+This is a brainstorm task, not a code task. Use `superpowers:brainstorming` skill. Operator is in the loop.
+
+- [ ] **Step 1: Invoke the brainstorming skill** with the specific scope: "Choose the in-memory binary document format for v1.11. Three candidates pre-identified in the v2.0 roadmap: BSON, custom packed-JSON tape, yyjson mut_doc. The decision must address: (a) memory footprint per doc on Zoe-shape data, (b) field-access cost, (c) round-trip fidelity with nlohmann::json, (d) library dependency cost, (e) long-term ownership cost (do we own the spec)."
+
+- [ ] **Step 2: Produce the decision document** at `docs/superpowers/specs/2026-05-15-binary-doc-format-decision.md`. Required sections:
+  - **Choice** — one of BSON / custom tape / yyjson mut_doc.
+  - **Rationale** — why this one wins on the five axes above.
+  - **Rejected alternatives** — what the other two are, why they lose. Include real numbers where possible (size benchmark on a 100-doc Zoe-shape sample).
+  - **Encoder/decoder contract** — function signatures, error behaviour on malformed input, what types round-trip exactly and what types lose precision (e.g., does the format preserve int64 vs double distinction?).
+  - **Field-fast-path semantics** — can the format support `get_field(buf, "name")` without decoding the whole doc? If not, the lazy cache is the only fast-path and that's fine.
+  - **Memory model** — does the buffer own its strings, or are they slices into the buffer? Affects `string_view` safety downstream.
+  - **Library dependency** — exact apt package name (if BSON: `libbson-dev`/`libbson-1.0-0`; if yyjson mut_doc: already shipped in Phase A; if custom tape: none).
+
+- [ ] **Step 3: Operator review gate.** Per the brainstorming skill's "User Review Gate", do NOT proceed to Task 2 until the operator signs off on the decision document.
+
+- [ ] **Step 4: Commit the decision doc.**
+
+```bash
+git add docs/superpowers/specs/2026-05-15-binary-doc-format-decision.md
+git commit -m "spec: decide v1.11 binary doc format ($CHOICE)"
+```
+
+---
+
+### Task 2 — Write the encoder/decoder with TDD
+
+**Files:**
+- Create: `service/src/doc_binary.hpp`
+- Create: `service/src/doc_binary.cpp`
+- Create: `tests/test_doc_binary.cpp`
+- Modify: `tests/CMakeLists.txt`
+- Modify: `service/CMakeLists.txt`
+
+The exact byte layout, the `encode`/`decode` internals, and the field-fast-path implementation follow the decision document from Task 1. Below is the *contract*-level test design, which is stable regardless of format choice.
+
+- [ ] **Step 1: Write the failing test.** Create `tests/test_doc_binary.cpp`:
+
+```cpp
+#include <cassert>
+#include <iostream>
+#include <nlohmann/json.hpp>
+#include "doc_binary.hpp"
+
+using nlohmann::json;
+using smartbotic::db::doc_binary::encode;
+using smartbotic::db::doc_binary::decode;
+using smartbotic::db::doc_binary::get_field;
+
+namespace {
+
+void check(bool cond, const char* msg) {
+    if (!cond) {
+        std::cerr << "FAIL: " << msg << "\n";
+        std::abort();
+    }
+}
+
+void roundtrip(const json& j) {
+    auto buf = encode(j);
+    auto back = decode(buf);
+    check(back == j, "roundtrip");
+}
+
+void test_primitives_roundtrip() {
+    roundtrip(json(nullptr));
+    roundtrip(json(true));
+    roundtrip(json(false));
+    roundtrip(json(0));
+    roundtrip(json(42));
+    roundtrip(json(-7));
+    roundtrip(json(9223372036854775807LL));
+    roundtrip(json(3.14));
+    roundtrip(json("hello"));
+    roundtrip(json(""));
+}
+
+void test_collection_roundtrip() {
+    roundtrip(json::array());
+    roundtrip(json::object());
+    roundtrip(json::array({1, 2, 3, "four", true, nullptr}));
+    roundtrip(json({{"k", "v"}, {"n", 1}, {"nested", {{"a", 1}}}}));
+}
+
+void test_zoe_shape_roundtrip() {
+    json zoe = {
+        {"_id", "doc-1"},
+        {"_created_at", 1731628800123LL},
+        {"name", "Zoe"},
+        {"tags", json::array({"hot", "pinned"})},
+        {"_vector", json::array({0.1, -0.2, 0.3, 0.4})},
+        {"meta", {{"flag", true}, {"score", 0.93}}},
+    };
+    roundtrip(zoe);
+}
+
+void test_unicode_roundtrip() {
+    roundtrip(json({{"hu", "Árvíztűrő tükörfúrógép"}, {"emoji", "🚀"}}));
+}
+
+void test_field_fast_path_present() {
+    json j = {{"name", "Zoe"}, {"count", 42}};
+    auto buf = encode(j);
+    auto name = get_field(buf, "name");
+    check(name.has_value() && name->get<std::string>() == "Zoe", "fast-path name");
+    auto count = get_field(buf, "count");
+    check(count.has_value() && count->get<int>() == 42, "fast-path count");
+}
+
+void test_field_fast_path_missing() {
+    json j = {{"name", "Zoe"}};
+    auto buf = encode(j);
+    auto v = get_field(buf, "missing");
+    check(!v.has_value(), "fast-path returns nullopt for missing");
+}
+
+void test_decode_corrupt_input_throws() {
+    std::vector<uint8_t> garbage = {0xff, 0xfe, 0xfd, 0xfc};
+    bool threw = false;
+    try {
+        (void)decode(garbage);
+    } catch (const std::exception&) {
+        threw = true;
+    }
+    check(threw, "garbage decode throws");
+}
+
+void test_int_vs_double_preserved() {
+    // The decision doc must specify whether int64 vs double are distinguishable
+    // after roundtrip. If yes, this test asserts that. If no, this test asserts
+    // the documented coercion behaviour. Adjust per the decision doc.
+    json i = 42;
+    json d = 42.0;
+    auto bi = encode(i);
+    auto bd = encode(d);
+    check(decode(bi).is_number_integer(), "int stays int");
+    check(decode(bd).is_number_float(), "double stays double");
+}
+
+}  // namespace
+
+int main() {
+    test_primitives_roundtrip();
+    test_collection_roundtrip();
+    test_zoe_shape_roundtrip();
+    test_unicode_roundtrip();
+    test_field_fast_path_present();
+    test_field_fast_path_missing();
+    test_decode_corrupt_input_throws();
+    test_int_vs_double_preserved();
+    std::cout << "test_doc_binary: all passed\n";
+    return 0;
+}
+```
+
+- [ ] **Step 2: Register the test** in `tests/CMakeLists.txt` (mirror the test_json_parse pattern).
+
+- [ ] **Step 3: Run the test, watch it fail to link.**
+
+Run: `cmake --build build -j$(nproc) --target test_doc_binary`
+Expected: FAIL — `encode`/`decode`/`get_field` undefined.
+
+- [ ] **Step 4: Implement `doc_binary.hpp`** per the Task 1 decision document. Header skeleton (format-agnostic):
+
+```cpp
+#pragma once
+
+#include <cstdint>
+#include <optional>
+#include <span>
+#include <vector>
+#include <string_view>
+
+#include <nlohmann/json.hpp>
+
+namespace smartbotic::db::doc_binary {
+
+// Encode an nlohmann::json tree into the binary on-heap format chosen in
+// docs/superpowers/specs/2026-05-15-binary-doc-format-decision.md.
+std::vector<uint8_t> encode(const nlohmann::json& j);
+
+// Decode a binary doc into a fresh nlohmann::json tree. Throws on corrupt
+// input. Cost is proportional to the doc size — for single-field reads,
+// prefer get_field.
+nlohmann::json decode(std::span<const uint8_t> buf);
+
+// Extract a single top-level field without decoding the whole doc.
+// Returns nullopt if the field is absent. Throws on corrupt input.
+// Format-specific: if the chosen format does not support O(field-size)
+// extraction, this still works but internally decodes the whole doc.
+std::optional<nlohmann::json> get_field(std::span<const uint8_t> buf,
+                                        std::string_view field_name);
+
+}  // namespace smartbotic::db::doc_binary
+```
+
+- [ ] **Step 5: Implement `doc_binary.cpp`** per the Task 1 decision document's encoder/decoder contract. The exact byte layout is defined there; the contract is fixed by the tests above. Include only the format-specific library headers (e.g., `<bson.h>` or `<yyjson.h>`).
+
+- [ ] **Step 6: Wire any new lib dependency into CMake / Dockerfile / control.server** if Task 1 chose a format that needs a new package (e.g., BSON). Skip this step if Task 1 chose yyjson mut_doc or custom tape (no new dep).
+
+- [ ] **Step 7: Run the test.**
+
+Run: `cmake --build build -j$(nproc) --target test_doc_binary && ./build/tests/test_doc_binary`
+Expected: `test_doc_binary: all passed`.
+
+- [ ] **Step 8: Commit.**
+
+```bash
+git add service/src/doc_binary.hpp service/src/doc_binary.cpp tests/test_doc_binary.cpp tests/CMakeLists.txt service/CMakeLists.txt
+git commit -m "feat(doc_binary): encode/decode/get_field for in-memory binary docs"
+```
+
+---
+
+### Task 3 — Change `Document::data` type and add lazy view accessor
+
+**Files:**
+- Modify: `service/src/document.hpp`
+
+This is the big-blast-radius commit. Every consumer of `Document::data` breaks at compile time — by design. We fix them one file at a time in Tasks 4-7.
+
+- [ ] **Step 1: Replace the type and add accessors.** Edit `service/src/document.hpp`:
+
+```cpp
+struct Document {
+    std::string id;
+
+    // v1.11.0+ — data lives as a binary buffer; an nlohmann::json view is
+    // materialised lazily on first field access and cached.
+    std::vector<uint8_t> data_binary;
+    mutable std::optional<nlohmann::json> data_view;
+
+    // Lazy accessor — preserves the old `doc.data` callsite ergonomics.
+    // Materialises the full tree on first call; subsequent calls return the
+    // cached view in O(1).
+    const nlohmann::json& data() const;
+    nlohmann::json& mutable_data();
+
+    // Replace the binary payload from a nlohmann::json tree. Invalidates the
+    // cached view.
+    void set_data(const nlohmann::json& j);
+
+    // Fast-path single-field read. Avoids materialising the full tree.
+    std::optional<nlohmann::json> field(std::string_view name) const;
+
+    // ... existing fields (timestamps, version, _vector, etc.) unchanged ...
+};
+```
+
+Implementation goes in `document.hpp` if it's header-only currently, or in a new `document.cpp` if not. Match the existing pattern.
+
+- [ ] **Step 2: Same treatment for `DocumentVersion`** (line 110+). It also holds `nlohmann::json data` and is used by the history store.
+
+- [ ] **Step 3: Build, observe the breakage.**
+
+Run: `cmake --build build -j$(nproc)`
+Expected: FAIL with many `'data' is a private member` or `no member named 'data' in 'Document'` errors. This is the inventory of callsites to migrate in subsequent tasks.
+
+- [ ] **Step 4: Commit the type change only.** The compile is broken between Tasks 3 and 4 — that's intentional and short-lived. Use a WIP commit and squash later, or accept a non-bisectable commit here.
+
+```bash
+git add service/src/document.hpp service/src/document.cpp  # if created
+git commit -m "feat(document): swap Document::data to binary buffer + lazy view (WIP, callsites follow)"
+```
+
+---
+
+### Task 4 — Migrate `memory_store.cpp` callsites
+
+**Files:**
+- Modify: `service/src/memory_store.cpp` — 17 callsites
+
+- [ ] **Step 1: Walk the compile errors.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database 2>&1 | grep memory_store.cpp | head -40`
+Expected: enumerates the 17 callsites that need updating.
+
+- [ ] **Step 2: Rewrite each callsite** per these patterns:
+  - Read access `doc.data["field"]` → `doc.field("field").value_or(nlohmann::json(nullptr))` for fast-path single-field reads; or `doc.data()["field"]` for paths that already pay full-tree decode.
+  - Write access `doc.data["field"] = v` → load `doc.mutable_data()`, mutate, then call `doc.set_data(doc.data())` to re-encode. Acceptable for the rare write paths; the binary format trades write speed for read speed.
+  - `doc.data.dump()` → `doc.data().dump()` (full-tree materialise, fine for log/error paths).
+  - `doc.data.contains("x")` → `doc.field("x").has_value()` for fast-path; `doc.data().contains("x")` for paths already in a full-tree context.
+
+- [ ] **Step 3: Build memory_store object.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: PASS for memory_store.cpp; subsequent compile errors are in other files (still expected).
+
+- [ ] **Step 4: Commit.**
+
+```bash
+git add service/src/memory_store.cpp service/src/memory_store.hpp
+git commit -m "feat(memory_store): migrate to Document binary data accessors (phase B)"
+```
+
+---
+
+### Task 5 — Migrate gRPC and migration runner callsites
+
+**Files:**
+- Modify: `service/src/database_grpc_impl.cpp` — 8 callsites
+- Modify: `service/src/migrations/migration_runner.cpp` — 7 callsites
+
+Same migration pattern as Task 4. Done together because these are the next-most-touched files.
+
+- [ ] **Step 1: Rewrite `database_grpc_impl.cpp` callsites** using the same patterns from Task 4. Pay extra attention to the proto-↔-internal boundary: incoming `protoDoc.data()` (a JSON string) goes through `parse_to_nlohmann` then `set_data`; outgoing responses do `doc.data().dump()` or use the field-fast-path.
+
+- [ ] **Step 2: Rewrite `migration_runner.cpp` callsites.** Migrations are mostly batch mutators — they read the full tree, modify it, and write it back. So most migrate to `doc.mutable_data()` + `doc.set_data(...)` rather than fast-path field reads.
+
+- [ ] **Step 3: Build.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: errors now confined to conflict_resolver / views / encryption / config_manager.
+
+- [ ] **Step 4: Commit.**
+
+```bash
+git add service/src/database_grpc_impl.cpp service/src/migrations/migration_runner.cpp
+git commit -m "feat(grpc,migrations): migrate Document.data callsites (phase B)"
+```
+
+---
+
+### Task 6 — Migrate remaining service-side callsites
+
+**Files:**
+- Modify: `service/src/replication/conflict_resolver.cpp` — 5 callsites
+- Modify: `service/src/views/view_manager.cpp` — 2 callsites
+- Modify: `service/src/encryption/encryption_manager.cpp` — 2 callsites
+- Modify: `service/src/config/collection_config_manager.cpp` — 2 callsites
+
+- [ ] **Step 1: Migrate each file** using the Task 4 patterns. Note for `conflict_resolver.cpp`: conflict merge logic needs full-tree access on both sides, so use `doc.mutable_data()` rather than field-fast-path.
+
+- [ ] **Step 2: Migrate view_manager.cpp.** Filter / projection evaluation already walks the tree, so `doc.data()` (full-tree) is fine; field-fast-path is a future optimization.
+
+- [ ] **Step 3: Migrate encryption_manager.cpp.** Field-level encrypt/decrypt is a perfect fast-path candidate — only the encrypted fields are touched. Use `doc.field(name)` and `doc.mutable_data()[name] = ciphertext`.
+
+- [ ] **Step 4: Migrate collection_config_manager.cpp.**
+
+- [ ] **Step 5: Build the full binary.**
+
+Run: `cmake --build build -j$(nproc) --target smartbotic-database`
+Expected: PASS. All callsites migrated.
+
+- [ ] **Step 6: Commit.**
+
+```bash
+git add service/src/replication/conflict_resolver.cpp service/src/views/view_manager.cpp service/src/encryption/encryption_manager.cpp service/src/config/collection_config_manager.cpp
+git commit -m "feat: migrate remaining Document.data callsites (phase B)"
+```
+
+---
+
+### Task 7 — Migrate persistence layer (WAL / snapshot / history)
+
+**Files:**
+- Modify: `service/src/persistence/wal.cpp`
+- Modify: `service/src/persistence/snapshot.cpp`
+- Modify: `service/src/persistence/history_store.cpp`
+
+These already use `parse_to_nlohmann` from Phase A. Now they wrap that in `doc_binary::encode` on the way *into* the store, and call `doc.data().dump()` on the way *out* (still JSON-text on disk).
+
+- [ ] **Step 1: WAL replay.** After parsing JSON text → `nlohmann::json`, call `doc_binary::encode(j)` and assign to `doc.data_binary` directly (skip the lazy cache).
+
+- [ ] **Step 2: Snapshot deserialize.** Same pattern — parse text → encode binary → assign.
+
+- [ ] **Step 3: Snapshot serialize.** When writing snapshot bytes, call `doc.data().dump()` for the document body (full-tree materialise is fine on the write path; snapshot writes are infrequent).
+
+- [ ] **Step 4: WAL append.** Same as snapshot serialize — `doc.data().dump()` on the write side.
+
+- [ ] **Step 5: History store read + write.** Same patterns applied to `DocumentVersion`.
+
+- [ ] **Step 6: Build, run all unit tests.**
+
+Run: `cmake --build build -j$(nproc) && ctest --test-dir build --output-on-failure`
+Expected: all tests PASS.
+
+- [ ] **Step 7: Run all load tests.**
+
+Run: `cd tests/load_test && ./load_test && ./load_test_mixed && ./load_test_integrity && ./load_test_pinned && ./load_test_crash_insert && ./load_test_crash_verify && ./load_test_replication && ./load_test_replica_eviction`
+Expected: all PASS.
+
+- [ ] **Step 8: Commit.**
+
+```bash
+git add service/src/persistence/wal.cpp service/src/persistence/snapshot.cpp service/src/persistence/history_store.cpp
+git commit -m "feat(persistence): binary-encode docs after parse, dump on serialize (phase B)"
+```
+
+---
+
+### Task 8 — Memory benchmark + smoke test
+
+**Files:**
+- Create: `tests/bench_doc_memory.cpp`
+- Modify: `tests/CMakeLists.txt`
+
+The whole point of Phase B is RSS reduction. Measure it.
+
+- [ ] **Step 1: Write the bench.** Load N synthetic Zoe-shape docs into a `MemoryStore` (or a stand-in that uses `Document` directly), read `/proc/self/status` for `VmRSS`, report. Compare against a baseline run pinned to the v1.10 binary.
+
+```cpp
+#include <fstream>
+#include <iostream>
+#include <random>
+#include <string>
+#include <vector>
+
+#include <nlohmann/json.hpp>
+#include "doc_binary.hpp"
+#include "document.hpp"
+
+size_t rss_kb() {
+    std::ifstream f("/proc/self/status");
+    std::string line;
+    while (std::getline(f, line)) {
+        if (line.rfind("VmRSS:", 0) == 0) {
+            return std::stoul(line.substr(6));
+        }
+    }
+    return 0;
+}
+
+int main(int argc, char** argv) {
+    size_t n = (argc > 1) ? std::stoul(argv[1]) : 100000;
+    std::cout << "loading " << n << " docs...\n";
+    std::vector<smartbotic::db::Document> store;
+    store.reserve(n);
+    std::mt19937 rng(42);
+    for (size_t i = 0; i < n; ++i) {
+        nlohmann::json j = {
+            {"_id", "doc-" + std::to_string(i)},
+            {"name", "name-" + std::to_string(i)},
+            {"tags", nlohmann::json::array({"a", "b", "c"})},
+            {"_vector", nlohmann::json::array({0.1, 0.2, 0.3, 0.4})},
+            {"meta", {{"k", static_cast<int>(rng() % 1000)}}},
+        };
+        smartbotic::db::Document d;
+        d.id = "doc-" + std::to_string(i);
+        d.set_data(j);
+        store.push_back(std::move(d));
+    }
+    std::cout << "RSS after load: " << rss_kb() << " kB\n";
+    std::cout << "store.size(): " << store.size() << "\n";
+    // Per-doc bytes:
+    std::cout << "approx per-doc kB: " << (rss_kb() / n) << "\n";
+    return 0;
+}
+```
+
+- [ ] **Step 2: Register and build.**
+
+Run: `cmake --build build -j$(nproc) --target bench_doc_memory`
+Expected: PASS.
+
+- [ ] **Step 3: Run against v1.11 binary.**
+
+Run: `./build/tests/bench_doc_memory 100000`
+Expected: per-doc kB drops noticeably vs v1.10 (target: ≥ 2× reduction on Zoe-shape data; the decision doc from Task 1 will have a more precise target).
+
+- [ ] **Step 4: Commit with measured numbers in the commit body.**
+
+```bash
+git add tests/bench_doc_memory.cpp tests/CMakeLists.txt
+git commit -m "$(cat <<'EOF'
+test: add bench_doc_memory for phase B RSS verification
+
+Measured at 100k docs on Zoe-shape synthetic corpus:
+  v1.10 (nlohmann::json in-memory): <N> MB total / <N> kB per doc
+  v1.11 ($CHOICE binary):            <N> MB total / <N> kB per doc
+  reduction:                         <N>x
+EOF
+)"
+```
+
+---
+
+### Task 9 — Version bump + deb build + soak
+
+**Files:**
+- Modify: `VERSION`
+- Modify: `CLAUDE.md`
+
+- [ ] **Step 1: Bump VERSION** to `1.11.0`.
+
+- [ ] **Step 2: CLAUDE.md bullet** under Key Features:
+
+> **Binary lazy document storage (v1.11.0+)** — `Document::data` is stored in memory as a $CHOICE-encoded binary buffer with a lazy `nlohmann::json` view materialised on first full-tree access. Single-field reads use `doc.field(name)` and skip the full-tree decode. WAL/snapshot bytes on disk remain JSON text; only in-memory representation changed.
+
+- [ ] **Step 3: Local deb build + verify deps.**
+
+Run: `./packaging/build.sh --local && dpkg-deb -I dist/smartbotic-database_1.11.0-1_amd64.deb | grep -i depends`
+Expected: includes any new runtime dep added in Task 2's library-wiring step.
+
+- [ ] **Step 4: Soak.** Boot the v1.11 binary against a copy of Zoe's production data dir in a throwaway VM. Watch RSS over a 24-hour soak with synthetic gRPC traffic. Expected: RSS plateau is materially below the v1.10 plateau on the same workload.
+
+- [ ] **Step 5: Commit.**
+
+```bash
+git add VERSION CLAUDE.md
+git commit -m "release(v1.11.0): binary lazy in-memory documents (phase B)"
+```
+
+- [ ] **Step 6: Operator-gated push / sync / Zoe upgrade.** Same gates as Phase A.
+
+---
+
+## Out of scope (Phase B)
+
+- **On-disk binary format.** WAL/snapshot still write JSON text. The on-disk break is Phase C.
+- **Page-based storage.** Phase C.
+- **Removing nlohmann::json entirely.** The lazy view keeps nlohmann as the public AST surface; this is intentional. Phase C might shrink the nlohmann footprint further, but it stays.
+- **Wire format change.** gRPC `Document.data` stays JSON-text on the wire forever (or until v2.0+ ships a new RPC).
+
+---
+
+## Success criteria
+
+1. `Document::data_binary` is a `std::vector<uint8_t>` and the entire codebase compiles + links against it.
+2. All unit tests pass: `test_json_parse`, `test_doc_binary`, `test_snapshot_durability`, `test_eviction`, `test_views`, `test_vector_storage`, `test_timestamp_precision`, `test_config_dropins`.
+3. All load tests pass.
+4. `bench_doc_memory` reports ≥ 2× per-doc kB reduction vs v1.10 on Zoe-shape data (refine target per Task 1's decision doc).
+5. A v1.10 binary can read v1.11's snapshots and WAL (and vice versa) — manual round-trip test in a throwaway VM.
+6. 24-hour soak against Zoe-shape traffic shows RSS plateau below v1.10's plateau.
+
+---
+
+## Open questions
+
+- **Task 1 is the open question.** Format choice is unresolved at plan-write time; brainstorming session produces the decision document that unblocks Task 2+.
+- **Migration UX for fast-path adoption** — should we add a clang-tidy / grep gate that forbids new `doc.data()["field"]` callsites in hot files, nudging contributors toward `doc.field(name)`? Defer to post-v1.11 if it becomes a problem.

docs/superpowers/plans/2026-05-15-v2.0-storage-engine-phase-c.md

@@ -0,0 +1,176 @@
+# Page-Based Storage Engine (Phase C → v2.0) Implementation Plan — SKELETON
+
+> **For agentic workers:** This plan is INTENTIONALLY INCOMPLETE. Only Tasks 1 and 2 are executable now. Tasks 3+ depend on the spike outcome from Task 1 and the architecture brainstorm in Task 2, and CANNOT be specified without those decisions. Do NOT attempt to write or execute Tasks 3+ until Task 2 produces a follow-up plan file.
+>
+> When Task 2 completes, it MUST author `docs/superpowers/plans/2026-05-15-v2.0-storage-engine-phase-c-execution.md` with the full TDD-style task breakdown. That follow-up plan is what subagent-driven-development will execute.
+
+**Goal:** Replace the "load everything into memory" v1.x architecture with a bounded-memory disk-backed page storage engine. RSS is governed by `buffer_pool_size_mb`, not by total dataset size. This is the "act like MySQL/InnoDB" rewrite that lets Smartbotic Database run a dataset much larger than RAM.
+
+**Architecture:** TBD by spike. Three candidate architectures are pre-identified in the v2.0 design doc (`docs/2026-05-15-v2.0-storage-engine-design.md`):
+
+1. **Homegrown** — 16 KB slotted pages, B+ tree primary index, LRU buffer pool, page-level WAL redo. We own everything. ~3 months engineering. No new runtime deps.
+2. **RocksDB-backed** — LSM-tree handles page management; we keep document/vector/index semantics on top of RocksDB column families. ~3 weeks engineering. Ongoing operational dep on RocksDB.
+3. **LMDB-backed** — mmap'd B+ tree, single-writer concurrency. ~2 weeks engineering. Simpler than RocksDB but with write-concurrency limits that may matter at Zoe's write rate.
+
+The spike (Task 1) picks one. Without that decision, the implementation plan cannot be written.
+
+**Tech Stack:** TBD by spike. The non-negotiables — C++20, gRPC/Protobuf wire format unchanged, nlohmann::json kept as the document AST surface, yyjson (from Phase A) as the parser — survive any choice.
+
+---
+
+## Context
+
+**What changes in v2.0:**
+
+- **On-disk layout:** `/var/lib/smartbotic-database/pages/` (or backend-equivalent — RocksDB's `*.sst`, LMDB's `data.mdb`). The current `snapshots/`, `wal/`, `history/` tree from v1.x is gone after one-shot migration.
+- **WAL format:** page-level redo records replace document-level append. Format break — v1.x WAL is unreadable by v2.0 directly; the migration tool reads it.
+- **Snapshot format:** page checkpoints replace whole-store LZ4 dumps. Another format break.
+- **Eviction:** the v1.7.0 chunked eviction machinery (`hot_write_floor_ms`, `eviction_chunk_size`, four pressure levels, evicted-stub WAL fallback, `docWalSeq_`) is GONE. The buffer pool's LRU replacement is the only eviction mechanism. `max_memory_mb` → `buffer_pool_size_mb`.
+- **Recovery:** the v1.7.0 tiered recovery (snapshot → snapshot_fallback → WAL-only → best_effort → force_empty) collapses to standard checkpoint-and-redo. `recovery.mode` config keeps the same surface for operator continuity but maps to the new engine's terms.
+
+**Operator constraint (load-bearing):** "I will not upgrade services which use the database just when the new v2 finished." This means v2.0 is a single-jump release from v1.9.5 (or whatever the latest v1.x is when the spike completes). The `--migrate-from-v1` one-shot mode is REQUIRED — the operator needs `apt upgrade smartbotic-database` to:
+
+1. Stop the service (v1.9.5 preinst takes the auto-backup).
+2. Install v2.0.
+3. On first boot, detect a v1.x data dir and run the one-shot migration into the new engine's format.
+4. Start serving.
+
+If migration fails, the operator restores from the v1.9.5 backup and reinstalls v1.x — that's the rollback path. v1.9.5's backup-before-upgrade is the load-bearing safety net here; treat it as a hard requirement.
+
+**Cross-cutting changes in v2.0 (from the design doc):**
+- New `service/src/storage/` directory tree replaces `service/src/persistence/`.
+- `MemoryStore` is renamed (e.g., `DocumentStore` or `PageStore`) and gains a buffer pool.
+- All replication paths re-routed through the new WAL.
+- Vector storage (currently parallel float arrays alongside docs) needs a v2.0 home — page-based or alongside the index. Spike must address this.
+- Field-level encryption stays at the AES-256-GCM layer; only the storage substrate changes.
+
+---
+
+## Tasks
+
+### Task 1 — Build-vs-buy spike (1 week)
+
+**Files:**
+- Create: `docs/superpowers/spikes/2026-05-15-storage-engine-spike.md` — the spike writeup.
+- Create: `service/spike/` — throwaway prototype code, gitignored or committed on a `spike/v2-storage` branch (operator's call).
+
+The spike is research, not production code. Its only deliverable is the writeup that justifies the architecture choice.
+
+- [ ] **Step 1: Define the spike scope.** Prototype the *minimum* path that exercises each candidate's failure modes:
+  - Load 1M Zoe-shape docs into the backend.
+  - Run a sustained 5k write/s + 50k read/s workload.
+  - Crash the process and verify recovery time + data integrity.
+  - Run a SimilaritySearch over a vector subset.
+
+- [ ] **Step 2: Prototype Option 3 first (LMDB)** — smallest scope, fastest to falsify or validate. Measure: write throughput ceiling under single-writer constraint, mmap RSS behaviour, recovery time.
+
+- [ ] **Step 3: Prototype Option 2 (RocksDB)** — if Option 3's write-concurrency ceiling is below Zoe's projected sustained write rate, RocksDB is the alternative. Measure: compaction overhead, RSS plateau, recovery time, ops experience (compaction tuning, blob storage for large docs).
+
+- [ ] **Step 4: Prototype Option 1 (homegrown) only if both 2 and 3 fail to meet requirements.** Document the requirement gap that justifies a 3-month build over a 2-3 week buy. Be honest — the homegrown option is the heaviest commitment and should require strong evidence to choose.
+
+- [ ] **Step 5: Write the spike report.** Required sections:
+  - **Scope** — what was measured, what wasn't.
+  - **Results** — numbers per candidate: write tps, read tps, recovery time, RSS curve, on-disk size, vector search latency.
+  - **Operational cost** — what's the ops burden of each choice over 5 years (compaction tuning for RocksDB, single-writer for LMDB, total ownership for homegrown).
+  - **Decision** — one of {homegrown, RocksDB, LMDB} with the reasoning anchored in numbers.
+  - **Rejected alternatives** — what we ruled out and why.
+  - **Risks** — the top 3 things that could derail the chosen path.
+  - **Migration sketch** — high-level mapping of v1.x data → v2.0 backend (the detail goes in Task 2's plan).
+
+- [ ] **Step 6: Operator review gate.** The spike report needs operator sign-off before Task 2. Per the brainstorming skill's review-gate pattern.
+
+- [ ] **Step 7: Commit the spike report** (keep the prototype code on its own branch).
+
+```bash
+git checkout main
+git add docs/superpowers/spikes/2026-05-15-storage-engine-spike.md
+git commit -m "spike: v2.0 storage engine build-vs-buy ($DECISION)"
+```
+
+---
+
+### Task 2 — Architecture brainstorm + execution plan authoring
+
+**Files:**
+- Create: `docs/superpowers/specs/2026-05-15-v2.0-storage-engine-architecture.md` — the architecture decision record.
+- Create: `docs/superpowers/plans/2026-05-15-v2.0-storage-engine-phase-c-execution.md` — the full execution plan.
+
+Use `superpowers:brainstorming` to drive Step 1; use `superpowers:writing-plans` for Step 4.
+
+- [ ] **Step 1: Run a brainstorming session** scoped to the v2.0 architecture given the spike outcome. Key questions to drive the session:
+  - Buffer pool size policy — fixed at config, adaptive to system memory, both?
+  - Index strategy — what indexes ship in v2.0 (primary key + per-collection BTrees + vector index)? Which are deferred to v2.1?
+  - Migration UX — operator runs it explicitly, or auto-runs on first boot detecting a v1.x data dir? What does failure look like?
+  - Replication — does the v1.x WAL-streaming replication model survive, or does v2.0 replicate at the page level (Postgres physical replication style)? This is a big architecture call.
+  - Vector storage — co-located with docs in pages, or in a separate index?
+  - Schema/index migrations — v1.x's JSON migrations RPC needs a v2.0 equivalent.
+
+- [ ] **Step 2: Author the architecture decision record** with the brainstorm outcome.
+
+- [ ] **Step 3: Operator review gate** on the architecture doc.
+
+- [ ] **Step 4: Author the v2.0 execution plan** at `docs/superpowers/plans/2026-05-15-v2.0-storage-engine-phase-c-execution.md`. This plan is full-detail (every TDD step, every file path, every command) at the same level as the Phase A plan in this same directory. Required sections:
+  - File-structure decomposition (what gets created in `service/src/storage/`, what gets deleted from `service/src/persistence/`, etc.)
+  - Migration tool task breakdown
+  - WAL format spec
+  - Snapshot/checkpoint format spec
+  - Buffer pool implementation tasks
+  - Index re-implementation tasks (primary index, vector index)
+  - Replication migration tasks
+  - Eviction-machinery deletion tasks (remove `hot_write_floor_ms`, evicted-stub WAL fallback, `docWalSeq_`, four-level pressure events from v1.7.0)
+  - Performance / soak gates
+  - VERSION bump to 2.0.0
+  - Operator-facing release notes
+
+- [ ] **Step 5: Operator review** of the execution plan.
+
+- [ ] **Step 6: Commit both docs.**
+
+```bash
+git add docs/superpowers/specs/2026-05-15-v2.0-storage-engine-architecture.md docs/superpowers/plans/2026-05-15-v2.0-storage-engine-phase-c-execution.md
+git commit -m "spec+plan: v2.0 storage engine architecture and execution plan"
+```
+
+---
+
+### Tasks 3+ — Execution
+
+**File:** see `docs/superpowers/plans/2026-05-15-v2.0-storage-engine-phase-c-execution.md` (authored by Task 2).
+
+The detailed implementation tasks live in the execution plan. They CANNOT be written here because:
+
+1. The set of files to create/modify depends on the spike outcome (homegrown vs RocksDB vs LMDB).
+2. The TDD test design depends on the chosen storage primitives.
+3. The migration tool's structure depends on the source-to-target schema mapping decided in Task 2.
+
+Writing Tasks 3+ here without those decisions would produce exactly the kind of placeholder-laden plan that `writing-plans` warns against. Tasks 1 and 2 exist precisely to retire that uncertainty.
+
+---
+
+## Out of scope (Phase C)
+
+- **Distributed scaling.** v2.0 is single-node + replication, same as v1.x. Sharding / Raft / multi-leader is post-v2.0.
+- **Online schema migration without rewriting docs.** v2.0 keeps the JSON-document-with-optional-schema model.
+- **Query language change.** Filters, views, vector search keep the v1.x gRPC API surface.
+- **Anything not flowing from the spike outcome.** No speculative work.
+
+---
+
+## Success criteria
+
+1. The spike report exists, is reviewed, and names a chosen architecture with numerical justification.
+2. The architecture decision record exists and is reviewed.
+3. The execution plan exists, passes `writing-plans` self-review (no placeholders, no TBDs, every step has code/commands), and the operator approves it.
+4. Subagent-driven-development can execute the execution plan task-by-task with no ambiguity left in the plan.
+5. v2.0 ships with a working `--migrate-from-v1` mode that boots cleanly against a v1.9.x data dir (Zoe-shape, full size).
+6. v1.9.5 backup-before-upgrade still fires on v1.x → v2.0 upgrade — confirmed manually in the soak test.
+
+---
+
+## Open questions (resolved by Task 1 + Task 2)
+
+- Build vs buy (Task 1).
+- Buffer pool sizing policy (Task 2).
+- Replication model — WAL streaming vs page-level (Task 2).
+- Vector index placement (Task 2).
+- Migration UX (Task 2).