# yyjson Hot-Path Swap (Phase A → v1.10.0) Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Replace `nlohmann::json::parse` with yyjson at every hot-path callsite (WAL replay, snapshot deserialize, history reads, gRPC request bodies, replication apply) while keeping `nlohmann::json` as the in-memory document representation. Phase A is a parse-speed win — not a memory win. Phase B is where memory drops. **Architecture:** Add yyjson as a build + runtime dep. Introduce one bridge helper (`parse_to_nlohmann(string_view) -> nlohmann::json`) that uses yyjson under the hood and rebuilds an `nlohmann::json` tree from the yyjson DOM. Every existing `nlohmann::json::parse(s)` callsite is rewritten to call this helper. The bridge is the *only* yyjson surface in the codebase for v1.10 — Phase B will later let yyjson buffers live longer and back binary doc storage. **Tech Stack:** C++20, yyjson 0.9+ (libyyjson-dev on Debian 13), nlohmann::json (unchanged), CMake/pkg-config, existing CTest suite. --- ## Context **The 17 parse callsites Phase A touches** (verified via `rg -n 'json::parse'` against the v1.9.5 tree): | File | Lines | What it parses | |------|-------|----------------| | `service/src/persistence/wal.cpp` | 233, 249 | WAL replay: doc data + collection options | | `service/src/persistence/snapshot.cpp` | 820, 929, 963 | Snapshot deserialize: collection opts, document data, version history | | `service/src/persistence/history_store.cpp` | 124 | Per-collection history (.hlog) version read | | `service/src/database_service.cpp` | 722 | Replication apply path | | `service/src/database_grpc_impl.cpp` | 109, 187, 222, 307, 362, 592, 708, 1288, 1344, 1397 | Insert / Update / Patch / Find / Subscribe / similarity-search / filter values | **Backward compatibility:** Wire and on-disk JSON bytes are unchanged. The swap is local to the parser implementation. A v1.10 binary must read v1.9.x snapshots/WAL byte-for-byte identically; tests verify this. **Why yyjson:** ~3-5× faster than nlohmann's parser on the kind of documents Zoe stores (modest nesting, mostly strings + ints). License is MIT. Header + tiny static lib (~30 KB). Already packaged for Debian 13 (`libyyjson-dev` / `libyyjson0`). **Build infrastructure baseline (verified):** - `packaging/Dockerfile.base` line 10 has the apt-get list; tag is `smartbotic-db-build-base:debian13:v3` (we bump to `v4`). - `service/CMakeLists.txt` uses `pkg_check_modules` for LZ4 and jemalloc — we follow the same pattern for yyjson. - `packaging/deb/templates/control.server` `Depends:` line: append `libyyjson0`. --- ## File Structure ### Files to create - `service/src/json_parse.hpp` — bridge helper declaration: `nlohmann::json parse_to_nlohmann(std::string_view)` + `parse_to_nlohmann(const char*, size_t)`. - `service/src/json_parse.cpp` — yyjson-backed implementation. The ONLY file that includes ``. - `tests/test_json_parse.cpp` — unit tests for the bridge against nlohmann::json's own parser on the same inputs. - `tests/bench_json_parse.cpp` — micro-benchmark, std::chrono based, run by hand to capture the parse-time delta on a Zoe-shape snapshot. ### Files to modify - `packaging/Dockerfile.base` — append `libyyjson-dev` to the apt-get install list, bump the base-image tag echoed at the bottom from `v3` to `v4`. - `packaging/deb/templates/control.server` — append `, libyyjson0` to the `Depends:` line. - `service/CMakeLists.txt` — add `pkg_check_modules(yyjson REQUIRED yyjson)` and link `${yyjson_LIBRARIES}` + include `${yyjson_INCLUDE_DIRS}`. - `tests/CMakeLists.txt` — register `test_json_parse` and `bench_json_parse`. - `service/src/persistence/wal.cpp` — 2 parse-site swaps. - `service/src/persistence/snapshot.cpp` — 3 parse-site swaps. - `service/src/persistence/history_store.cpp` — 1 parse-site swap. - `service/src/database_service.cpp` — 1 parse-site swap. - `service/src/database_grpc_impl.cpp` — 10 parse-site swaps. - `VERSION` — bump 1.9.5 → 1.10.0. - `CLAUDE.md` — add Key Features bullet documenting the yyjson swap. ### Files NOT to touch - `service/src/document.hpp` — `Document::data` stays `nlohmann::json` for Phase A. Phase B is where this changes. - `client/` — client library does not parse JSON in any hot path; the proto bindings are already binary. - `proto/database.proto` — wire format unchanged. --- ## Tasks ### Task 1 — Add yyjson build + runtime dependencies **Files:** - Modify: `packaging/Dockerfile.base:10-31` - Modify: `packaging/deb/templates/control.server:18` (the `Depends:` line) - Modify: `service/CMakeLists.txt:7-10` (where `pkg_check_modules` for LZ4 is declared) - Modify: `service/CMakeLists.txt:59-90` (where `target_link_libraries` clauses live) - [ ] **Step 1: Add libyyjson-dev to Dockerfile.base.** Open `packaging/Dockerfile.base` and append `libyyjson-dev \` to the apt-get install list (right after `libfmt-dev \`). Bump the echo at the bottom from `v3` to `v4`: ```dockerfile libfmt-dev \ libyyjson-dev \ dpkg-dev \ && rm -rf /var/lib/apt/lists/* # ... existing ENV/WORKDIR lines unchanged ... RUN echo "smartbotic-db-build-base:debian13:v4" > /etc/smartbotic-db-build-base ``` - [ ] **Step 2: Rebuild the Docker base image.** Run: `./packaging/build.sh --rebuild-base` Expected: builds image tagged `smartbotic-db-build-base:debian13:v4` with no errors. - [ ] **Step 3: Add libyyjson0 to runtime deb dependencies.** In `packaging/deb/templates/control.server`, append `, libyyjson0` to the `Depends:` line: ``` Depends: libsmartbotic-db-client (= {{VERSION}}), libssl3t64, liblz4-1, libsystemd0, libjemalloc2, libyyjson0 ``` - [ ] **Step 4: Wire yyjson into CMake.** In `service/CMakeLists.txt`, locate the `find_package(PkgConfig)` block (line 7) and add a `pkg_check_modules` call: ```cmake find_package(PkgConfig) if(PkgConfig_FOUND) pkg_check_modules(LZ4 QUIET liblz4) pkg_check_modules(yyjson REQUIRED yyjson) endif() ``` Then in the `target_link_libraries` / `target_include_directories` section, add after the jemalloc clause: ```cmake target_include_directories(smartbotic-database PRIVATE ${yyjson_INCLUDE_DIRS}) target_link_libraries(smartbotic-database PRIVATE ${yyjson_LIBRARIES}) ``` - [ ] **Step 5: Verify the local build still works.** Run: `cmake -B build -G Ninja && cmake --build build -j$(nproc)` Expected: PASS. No new compile or link errors. yyjson is now linked but unused — that's fine for this task. - [ ] **Step 6: Commit.** ```bash git add packaging/Dockerfile.base packaging/deb/templates/control.server service/CMakeLists.txt git commit -m "build: add yyjson build + runtime dependency (phase A scaffolding)" ``` --- ### Task 2 — Write the yyjson → nlohmann bridge helper **Files:** - Create: `service/src/json_parse.hpp` - Create: `service/src/json_parse.cpp` - Create: `tests/test_json_parse.cpp` - Modify: `tests/CMakeLists.txt` (register the new test executable) - Modify: `service/CMakeLists.txt` (add `json_parse.cpp` to the source list) - [ ] **Step 1: Write the failing test.** Create `tests/test_json_parse.cpp`: ```cpp #include #include #include #include #include #include "json_parse.hpp" using nlohmann::json; using smartbotic::db::parse_to_nlohmann; namespace { void check(bool cond, const char* msg) { if (!cond) { std::cerr << "FAIL: " << msg << "\n"; std::abort(); } } void test_empty_object() { auto j = parse_to_nlohmann("{}"); check(j.is_object() && j.empty(), "empty object"); } void test_empty_array() { auto j = parse_to_nlohmann("[]"); check(j.is_array() && j.empty(), "empty array"); } void test_primitives() { check(parse_to_nlohmann("true").get() == true, "true"); check(parse_to_nlohmann("false").get() == false, "false"); check(parse_to_nlohmann("null").is_null(), "null"); check(parse_to_nlohmann("42").get() == 42, "int"); check(parse_to_nlohmann("-7").get() == -7, "negative int"); check(parse_to_nlohmann("3.14").get() == 3.14, "double"); check(parse_to_nlohmann("\"hello\"").get() == "hello", "string"); } void test_nested() { std::string s = R"({"a":1,"b":[2,3,{"c":"x"}],"d":null,"e":true})"; auto y = parse_to_nlohmann(s); auto n = json::parse(s); check(y == n, "nested equality with nlohmann"); } void test_unicode_string() { std::string s = R"({"name":"Árvíztűrő tükörfúrógép","emoji":"🚀"})"; auto y = parse_to_nlohmann(s); auto n = json::parse(s); check(y == n, "unicode equality"); } void test_large_int() { auto j = parse_to_nlohmann("9223372036854775807"); check(j.get() == 9223372036854775807LL, "int64 max"); } void test_string_view_overload() { std::string s = "{\"k\":1}"; auto j = parse_to_nlohmann(std::string_view(s)); check(j["k"].get() == 1, "string_view overload"); } void test_invalid_json_throws() { bool threw = false; try { (void)parse_to_nlohmann("{not json"); } catch (const std::exception&) { threw = true; } check(threw, "invalid json throws"); } void test_corpus_equivalence() { // Sample of doc shapes the production service sees. const char* corpus[] = { R"({"_id":"abc","name":"x","tags":["a","b"],"count":0})", R"({"_id":"xyz","_vector":[0.1,-0.2,0.3,0.4],"meta":{"k":"v"}})", R"({"_id":"e1","_event":"call.ended","at":1731628800123})", R"([])", R"([1,2,3,4,5,6,7,8,9,10])", R"({"deeply":{"nested":{"object":{"is":{"fine":true}}}}})", }; for (const char* s : corpus) { check(parse_to_nlohmann(s) == json::parse(s), s); } } } // namespace int main() { test_empty_object(); test_empty_array(); test_primitives(); test_nested(); test_unicode_string(); test_large_int(); test_string_view_overload(); test_invalid_json_throws(); test_corpus_equivalence(); std::cout << "test_json_parse: all passed\n"; return 0; } ``` - [ ] **Step 2: Register the test in CMake.** In `tests/CMakeLists.txt`, add (matching the style of `test_vector_storage`): ```cmake add_executable(test_json_parse test_json_parse.cpp ${CMAKE_SOURCE_DIR}/service/src/json_parse.cpp) target_include_directories(test_json_parse PRIVATE ${CMAKE_SOURCE_DIR}/service/src ${yyjson_INCLUDE_DIRS}) target_link_libraries(test_json_parse PRIVATE nlohmann_json::nlohmann_json ${yyjson_LIBRARIES}) add_test(NAME test_json_parse COMMAND test_json_parse) ``` - [ ] **Step 3: Add `json_parse.cpp` to the service binary's source list** in `service/CMakeLists.txt` wherever the service source list is declared (search for any other `src/` .cpp file added to `smartbotic-database` and add `src/json_parse.cpp` alongside it). - [ ] **Step 4: Run test to verify it fails to *link* (helper not yet implemented).** Run: `cmake --build build -j$(nproc) --target test_json_parse` Expected: FAIL with `undefined reference to smartbotic::db::parse_to_nlohmann`. - [ ] **Step 5: Write the header.** Create `service/src/json_parse.hpp`: ```cpp #pragma once #include #include namespace smartbotic::db { // Parses a JSON document using yyjson (read-only DOM) and materialises an // nlohmann::json tree from it. Drop-in replacement for nlohmann::json::parse // at hot paths where parse speed matters but we still want nlohmann semantics // downstream. Throws nlohmann::json::parse_error on malformed input so existing // catch sites keep working. nlohmann::json parse_to_nlohmann(std::string_view bytes); nlohmann::json parse_to_nlohmann(const char* data, size_t length); } // namespace smartbotic::db ``` - [ ] **Step 6: Write the implementation.** Create `service/src/json_parse.cpp`: ```cpp #include "json_parse.hpp" #include #include #include namespace smartbotic::db { namespace { nlohmann::json convert(yyjson_val* v) { if (!v) { return nullptr; } switch (yyjson_get_type(v)) { case YYJSON_TYPE_NULL: return nullptr; case YYJSON_TYPE_BOOL: return yyjson_get_bool(v); case YYJSON_TYPE_NUM: switch (yyjson_get_subtype(v)) { case YYJSON_SUBTYPE_UINT: return yyjson_get_uint(v); case YYJSON_SUBTYPE_SINT: return yyjson_get_sint(v); case YYJSON_SUBTYPE_REAL: default: return yyjson_get_real(v); } case YYJSON_TYPE_STR: return std::string(yyjson_get_str(v), yyjson_get_len(v)); case YYJSON_TYPE_ARR: { nlohmann::json out = nlohmann::json::array(); size_t idx, max; yyjson_val* elem; yyjson_arr_foreach(v, idx, max, elem) { out.push_back(convert(elem)); } return out; } case YYJSON_TYPE_OBJ: { nlohmann::json out = nlohmann::json::object(); size_t idx, max; yyjson_val* key; yyjson_val* val; yyjson_obj_foreach(v, idx, max, key, val) { out[std::string(yyjson_get_str(key), yyjson_get_len(key))] = convert(val); } return out; } default: return nullptr; } } } // namespace nlohmann::json parse_to_nlohmann(const char* data, size_t length) { if (!data || length == 0) { throw nlohmann::json::parse_error::create( 101, 0, "empty input", nullptr); } yyjson_read_err err{}; yyjson_doc* doc = yyjson_read_opts( const_cast(data), length, YYJSON_READ_NOFLAG, nullptr, &err); if (!doc) { std::string msg = "yyjson parse error: "; msg += err.msg ? err.msg : "unknown"; throw nlohmann::json::parse_error::create( 101, static_cast(err.pos), msg, nullptr); } try { nlohmann::json out = convert(yyjson_doc_get_root(doc)); yyjson_doc_free(doc); return out; } catch (...) { yyjson_doc_free(doc); throw; } } nlohmann::json parse_to_nlohmann(std::string_view bytes) { return parse_to_nlohmann(bytes.data(), bytes.size()); } } // namespace smartbotic::db ``` - [ ] **Step 7: Run the test.** Run: `cmake --build build -j$(nproc) --target test_json_parse && ./build/tests/test_json_parse` Expected: `test_json_parse: all passed`. - [ ] **Step 8: Commit.** ```bash git add service/src/json_parse.hpp service/src/json_parse.cpp tests/test_json_parse.cpp tests/CMakeLists.txt service/CMakeLists.txt git commit -m "feat(json): add yyjson-backed parse_to_nlohmann bridge helper" ``` --- ### Task 3 — Swap WAL replay parse sites **Files:** - Modify: `service/src/persistence/wal.cpp:233,249` - [ ] **Step 1: Add the include.** At the top of `service/src/persistence/wal.cpp`, near the existing `#include `, add: ```cpp #include "../json_parse.hpp" ``` - [ ] **Step 2: Swap the two parse calls.** Replace `nlohmann::json::parse(dataStr)` at line 233 with `smartbotic::db::parse_to_nlohmann(dataStr)`, and similarly at line 249 for `nlohmann::json::parse(optsStr)`. - [ ] **Step 3: Build the service.** Run: `cmake --build build -j$(nproc) --target smartbotic-database` Expected: PASS. - [ ] **Step 4: Run the snapshot durability test.** Run: `./build/tests/test_snapshot_durability` Expected: PASS. This test exercises WAL replay against real on-disk bytes — proves the swap is binary-equivalent. - [ ] **Step 5: Run the WAL load test.** Run: `cd tests/load_test && ./load_test_crash_insert && ./load_test_crash_verify` Expected: both report success (no document loss, no checksum mismatch). - [ ] **Step 6: Commit.** ```bash git add service/src/persistence/wal.cpp git commit -m "perf(wal): use yyjson for replay parses (phase A)" ``` --- ### Task 4 — Swap snapshot deserialize parse sites **Files:** - Modify: `service/src/persistence/snapshot.cpp:820,929,963` - [ ] **Step 1: Add the include.** At the top of `service/src/persistence/snapshot.cpp`, add: ```cpp #include "../json_parse.hpp" ``` - [ ] **Step 2: Swap all three sites.** - Line 820 — `nlohmann::json::parse(optionsJson)` → `smartbotic::db::parse_to_nlohmann(optionsJson)` - Line 929 — `nlohmann::json::parse(docJson)` → `smartbotic::db::parse_to_nlohmann(docJson)` - Line 963 — `nlohmann::json::parse(verJson)` → `smartbotic::db::parse_to_nlohmann(verJson)` - [ ] **Step 3: Build.** Run: `cmake --build build -j$(nproc) --target smartbotic-database` Expected: PASS. - [ ] **Step 4: Run the snapshot durability test.** Run: `./build/tests/test_snapshot_durability` Expected: PASS. Covers atomic snapshot write + read-back equivalence. - [ ] **Step 5: Run vector + views tests** (snapshot covers their serialization too). Run: `./build/tests/test_vector_storage && ./build/tests/test_views` Expected: both PASS. - [ ] **Step 6: Commit.** ```bash git add service/src/persistence/snapshot.cpp git commit -m "perf(snapshot): use yyjson for deserialize parses (phase A)" ``` --- ### Task 5 — Swap history_store version-read parse **Files:** - Modify: `service/src/persistence/history_store.cpp:124` - [ ] **Step 1: Add the include + swap the call.** Add `#include "../json_parse.hpp"` near the existing nlohmann include, then change line 124 from `nlohmann::json::parse(verJson)` to `smartbotic::db::parse_to_nlohmann(verJson)`. - [ ] **Step 2: Build.** Run: `cmake --build build -j$(nproc) --target smartbotic-database` Expected: PASS. - [ ] **Step 3: Run the existing test suite.** History-store coverage rides on `test_snapshot_durability` and `test_eviction` (eviction restores from history when a doc is evicted-then-read). Run: `./build/tests/test_snapshot_durability && ./build/tests/test_eviction` Expected: both PASS. - [ ] **Step 4: Commit.** ```bash git add service/src/persistence/history_store.cpp git commit -m "perf(history): use yyjson for hlog version-read parse (phase A)" ``` --- ### Task 6 — Swap replication apply parse **Files:** - Modify: `service/src/database_service.cpp:722` - [ ] **Step 1: Add the include + swap.** Add `#include "json_parse.hpp"` (same directory; relative path is just `"json_parse.hpp"`). Change line 722 from `nlohmann::json::parse(entry.data())` to `smartbotic::db::parse_to_nlohmann(entry.data())`. - [ ] **Step 2: Build.** Run: `cmake --build build -j$(nproc) --target smartbotic-database` Expected: PASS. - [ ] **Step 3: Run the replication load test.** Run: `cd tests/load_test && ./load_test_replication` Expected: leader and follower converge on identical state; no parse errors logged. - [ ] **Step 4: Run the replica eviction load test.** This was the regression vector in 1.8.0 — re-run it as belt-and-braces. Run: `cd tests/load_test && ./load_test_replica_eviction` Expected: PASS. - [ ] **Step 5: Commit.** ```bash git add service/src/database_service.cpp git commit -m "perf(replication): use yyjson for replicated entry parse (phase A)" ``` --- ### Task 7 — Swap gRPC handler parse sites **Files:** - Modify: `service/src/database_grpc_impl.cpp:109,187,222,307,362,592,708,1288,1344,1397` All ten sites are identical-shape rewrites: `nlohmann::json::parse()` → `smartbotic::db::parse_to_nlohmann()`. Listed individually so the reviewer can scan the diff per-RPC. - [ ] **Step 1: Add the include** at the top of `service/src/database_grpc_impl.cpp`: ```cpp #include "json_parse.hpp" ``` - [ ] **Step 2: Swap line 109** (Insert handler — request data parse). - [ ] **Step 3: Swap line 187** (Insert handler — proto doc data parse). - [ ] **Step 4: Swap line 222** (Update handler — request data parse). - [ ] **Step 5: Swap line 307** (Patch handler — patch body parse). - [ ] **Step 6: Swap line 362** (Find handler — filter-value path). - [ ] **Step 7: Swap line 592** (Subscribe handler — initial-state parse). - [ ] **Step 8: Swap line 708** (Similarity-search handler — proto doc parse). - [ ] **Step 9: Swap line 1288** (Generic doc-from-proto parse helper). - [ ] **Step 10: Swap line 1344** (Filter `.value` parse). - [ ] **Step 11: Swap line 1397** (Nested filter `.value` parse). - [ ] **Step 12: Build.** Run: `cmake --build build -j$(nproc) --target smartbotic-database` Expected: PASS. - [ ] **Step 13: Run the integration load tests.** These exercise the gRPC handlers end-to-end. Run: `cd tests/load_test && ./load_test && ./load_test_mixed && ./load_test_integrity && ./load_test_pinned` Expected: all PASS, no parse errors logged. - [ ] **Step 14: Run views + timestamp-precision tests** (cover Find/Filter handlers). Run: `./build/tests/test_views && ./build/tests/test_timestamp_precision` Expected: PASS. - [ ] **Step 15: Commit.** ```bash git add service/src/database_grpc_impl.cpp git commit -m "perf(grpc): use yyjson for handler body parses (phase A)" ``` --- ### Task 8 — Parse-time micro-benchmark **Files:** - Create: `tests/bench_json_parse.cpp` - Modify: `tests/CMakeLists.txt` Establish a number for the operator: how much faster is yyjson on Zoe-shape data? Not a regression gate — informational, captured in the commit message. - [ ] **Step 1: Write the benchmark.** Create `tests/bench_json_parse.cpp`: ```cpp #include #include #include #include #include #include #include #include "json_parse.hpp" int main(int argc, char** argv) { if (argc < 2) { std::cerr << "usage: bench_json_parse \n" << " corpus = one JSON document per line\n"; return 1; } std::ifstream f(argv[1]); std::vector docs; std::string line; while (std::getline(f, line)) { if (!line.empty()) docs.push_back(line); } if (docs.empty()) { std::cerr << "empty corpus\n"; return 1; } constexpr int kIters = 5; auto bench = [&](const char* name, auto fn) { double best = 1e18; for (int i = 0; i < kIters; ++i) { auto t0 = std::chrono::steady_clock::now(); size_t sink = 0; for (const auto& s : docs) { auto j = fn(s); sink += j.size(); } auto t1 = std::chrono::steady_clock::now(); double ms = std::chrono::duration(t1 - t0).count(); best = std::min(best, ms); std::cout << name << " iter " << i << ": " << ms << " ms (sink=" << sink << ")\n"; } return best; }; double nl = bench("nlohmann", [](const std::string& s) { return nlohmann::json::parse(s); }); double yy = bench("yyjson ", [](const std::string& s) { return smartbotic::db::parse_to_nlohmann(s); }); std::cout << "\nbest nlohmann: " << nl << " ms\n"; std::cout << "best yyjson: " << yy << " ms\n"; std::cout << "speedup: " << (nl / yy) << "x\n"; return 0; } ``` - [ ] **Step 2: Register the benchmark in `tests/CMakeLists.txt`.** Match the test_json_parse pattern but DO NOT `add_test` it — this is a manual bench, not a regression gate: ```cmake add_executable(bench_json_parse bench_json_parse.cpp ${CMAKE_SOURCE_DIR}/service/src/json_parse.cpp) target_include_directories(bench_json_parse PRIVATE ${CMAKE_SOURCE_DIR}/service/src ${yyjson_INCLUDE_DIRS}) target_link_libraries(bench_json_parse PRIVATE nlohmann_json::nlohmann_json ${yyjson_LIBRARIES}) ``` - [ ] **Step 3: Build the bench binary.** Run: `cmake --build build -j$(nproc) --target bench_json_parse` Expected: PASS. - [ ] **Step 4: Capture a Zoe-shape corpus.** Use any existing seeded test data — `tests/load_test/` writes JSON via the client; dump 10k docs to a file (one per line) using `smartbotic-db-cli export ` against a populated local instance. If no instance is handy, generate synthetic shapes: ```bash python3 -c " import json, random, sys for i in range(10000): print(json.dumps({ '_id': f'doc{i}', 'name': f'name-{i}', 'tags': [f't{j}' for j in range(random.randint(0,5))], 'meta': {'k': i, 'flag': bool(i%2)}, 'vec': [random.random() for _ in range(8)], })) " > /tmp/bench-corpus.jsonl ``` - [ ] **Step 5: Run the benchmark.** Run: `./build/tests/bench_json_parse /tmp/bench-corpus.jsonl` Expected: yyjson at least 2× faster than nlohmann on this corpus. Record the numbers. - [ ] **Step 6: Commit the bench source with measured numbers in the commit body.** ```bash git add tests/bench_json_parse.cpp tests/CMakeLists.txt git commit -m "$(cat <<'EOF' test: add bench_json_parse for phase A parse-time delta Measured on 10k synthetic Zoe-shape docs: nlohmann: ms yyjson: ms speedup: x EOF )" ``` --- ### Task 9 — Bump VERSION, update CLAUDE.md, build deb **Files:** - Modify: `VERSION` - Modify: `CLAUDE.md` - [ ] **Step 1: Bump VERSION.** Replace `1.9.5` with `1.10.0` in `/data/smartbotic-database/VERSION`. - [ ] **Step 2: Add a CLAUDE.md bullet under "Key Features"** (insert after the auto-backup bullet): > **yyjson hot-path parsing (v1.10.0+)** — every JSON parse on WAL replay, snapshot deserialize, history reads, gRPC request bodies, and replication apply now goes through `smartbotic::db::parse_to_nlohmann` (yyjson-backed). In-memory representation stays `nlohmann::json` until Phase B. Wire and on-disk JSON bytes unchanged. - [ ] **Step 3: Local deb build.** Run: `./packaging/build.sh --local` Expected: produces `dist/smartbotic-database_1.10.0-1_amd64.deb` and the three sibling debs. - [ ] **Step 4: Verify the deb declares libyyjson0 dependency.** Run: `dpkg-deb -I dist/smartbotic-database_1.10.0-1_amd64.deb | grep -i depends` Expected: `Depends:` line contains `libyyjson0`. - [ ] **Step 5: Verify the binary actually links yyjson.** Run: `dpkg-deb -x dist/smartbotic-database_1.10.0-1_amd64.deb /tmp/deb-check && ldd /tmp/deb-check/usr/sbin/smartbotic-database | grep -i yyjson` Expected: `libyyjson.so.0 => /usr/lib/x86_64-linux-gnu/libyyjson.so.0`. - [ ] **Step 6: Commit.** ```bash git add VERSION CLAUDE.md git commit -m "release(v1.10.0): yyjson hot-path swap (phase A)" ``` --- ### Task 10 — Operator-gated rollout **Files:** none (process step) - [ ] **Step 1: Push the branch.** Operator decides timing — do NOT auto-push. ```bash git push origin feat/v1.9.5-backup # branch name stays even though it now carries v1.10 work ``` - [ ] **Step 2: Operator decision on sync.** `./packaging/build.sh --sync --suite trixie` publishes to repository.smartbotics.ai. Per CLAUDE.md and prior pattern, this requires explicit operator authorization — do NOT run it unprompted. - [ ] **Step 3: Zoe upgrade.** Same gate — only on explicit operator command. v1.9.5's auto-backup preinst will fire on the upgrade and snapshot the data dir to `/var/backups/smartbotic-database/pre-upgrade-*-from-1.9.5/`, which is exactly the rollback safety net Phase A inherits from the v1.9.5 work. --- ## Out of scope (Phase A) - **Binary in-memory document storage.** That is Phase B. Phase A's `Document::data` is still `nlohmann::json` — yyjson is a parser only. - **WAL/snapshot binary format.** Wire and on-disk bytes are byte-for-byte unchanged in v1.10. - **Serialisation (dump).** `nlohmann::json::dump()` callsites are untouched. yyjson's writer is fine but parse is the hot path; writer swap can wait. - **Migration tooling.** No `--migrate-from-v1` mode in Phase A — v1.10 reads v1.9.x data dirs natively. --- ## Success criteria 1. All 17 `nlohmann::json::parse` callsites in the hot-path file list are gone (verified by `rg -n 'nlohmann::json::parse' service/src` returning zero hits, OR only hits in files explicitly listed as out-of-scope cold-path code). 2. All existing tests pass: `test_snapshot_durability`, `test_eviction`, `test_views`, `test_vector_storage`, `test_timestamp_precision`, `test_config_dropins`. 3. All load tests pass: `load_test`, `load_test_mixed`, `load_test_integrity`, `load_test_pinned`, `load_test_crash_insert/verify`, `load_test_replication`, `load_test_replica_eviction`. 4. `bench_json_parse` reports ≥ 2× speedup over nlohmann on Zoe-shape corpus. 5. Local deb builds, declares `libyyjson0` dep, and `ldd` confirms the runtime link. 6. A clean v1.9.5 data dir boots cleanly on the v1.10.0 binary (manual smoke test in a throwaway VM, identical to the v1.9.5 deb's upgrade smoke test). --- ## Open questions None — Phase A is mechanical. Phase B's open question (binary format choice) lives in the Phase B plan.