Prechádzať zdrojové kódy

release(v2.2.0): default ns timestamps + update() docstring

Two shadowman-audit follow-ups bundled. One is a default-behavior flip
for fresh collections; the other is a docstring change. Both are
backed by the v2.x-no-backward-compat stance.

## Breaking change

`CollectionCfg::timestampPrecision` default flipped from `"ms"` to
`"ns"`. Closes audit item B5.

- Affects: collections created via `Client::createCollection()` WITHOUT
  an explicit `configureCollection()` follow-up. They now stamp
  `_created_at` / `_updated_at` in nanoseconds.
- NOT affected: collections that already have an explicit config record
  on disk (every fleet collection that ran `configureCollection` at any
  point). Their stored precision wins.
- Mixed-precision rows in the same collection (a pre-flip ms row next
  to a post-flip ns row) are read-correct via the 10^15 magnitude
  threshold the codebase has carried since per-collection precision
  shipped. Operators who want clean rows should run
  `migrateCollectionTimestamps`.

Operators who need ms can still set it explicitly:

    CollectionConfig cfg;
    cfg.set_timestamp_precision("ms");
    client.configureCollection("legacy_collection", cfg);

## Doc change

`Client::update()` docstring rewritten to clarify it OVERWRITES the
doc (every field not in `data` is dropped), and points at `patch()` for
partial mutations. Reflects audit item B6 — shadowman's 71 update() vs.
3 patch() callsites caused the v2.8.6→v2.8.8 conversation_tool_state
split race; the patch() approach is the server-side atomic answer.

## Tests

`test_timestamp_precision.cpp` asserts the new default (`"ns"` for
unknown collections, `"ns"` before configure in the cache-reflects
test). Full suite 12/12 green.
fszontagh 2 mesiacov pred
rodič
commit
2858f622ea

+ 1 - 0
CLAUDE.md

@@ -41,6 +41,7 @@ cmake --build build -j$(nproc) && ./build/tests/test_vector_storage
 - **Auto-backup before deb upgrade** — `apt upgrade smartbotic-database` runs a `preinst` hook that snapshots `/var/lib/smartbotic-database/` to `/var/backups/smartbotic-database/pre-upgrade-<ts>-from-<version>/` via `cp -a` after the old `prerm` stops the service. Retention `3`, 1.2× free-space precheck on the backup volume (aborts the upgrade if short). Opt out with `SMARTBOTIC_DB_SKIP_BACKUP=1 apt upgrade`. Tunables: `SMARTBOTIC_DB_BACKUP_RETENTION`, `SMARTBOTIC_DB_BACKUP_ROOT`.
 - **yyjson hot-path parsing (v1.10.0+)** — every JSON parse on WAL replay, snapshot deserialize, history reads, gRPC request bodies, and replication apply goes through `smartbotic::db::parse_to_nlohmann` (yyjson-backed) instead of `nlohmann::json::parse`. Measured 2.80× speedup on Zoe-shape corpora (136ms→49ms on 10k docs). In-memory representation stays `nlohmann::json` until Phase B. Wire and on-disk JSON bytes unchanged. New runtime dep: `libyyjson0`. Bench: `./build/tests/bench_json_parse <corpus.jsonl>`.
 - **Binary-backed Document storage (v1.11.0+)** — `Document::data` is now a `smartbotic::db::doc_binary::Doc` (yyjson `mut_doc` wrapper) with a lazy `nlohmann::json` view cache. New accessor surface: `doc.field(name)` is the O(field-count) fast-path that reads one top-level field without materialising the full tree (the dominant read pattern), `doc.data()` is the lazy full-tree accessor, `doc.set_data(j)` re-encodes from an nlohmann tree. Wire and on-disk JSON bytes unchanged. **Honest note:** per-doc heap footprint is ~1.5% smaller than v1.10 (1618→1594 bytes on Zoe-shape) — the win is field-access CPU + Phase C scaffolding, not RSS. See `docs/superpowers/specs/2026-05-15-binary-doc-format-decision.md` post-implementation amendment for the bench numbers. Bench: `./build/tests/bench_doc_memory --mode=<nlohmann|binary> [N]`.
+- **v2.2.0 — default timestamp precision flipped to `"ns"`.** BREAKING for fresh collections. `CollectionCfg::timestampPrecision` default went from `"ms"` to `"ns"` so rapid-write collections (chat messages, metrics events, tool-call audits) get nanosecond stamps and strict ordering without needing an explicit `configureCollection()` call. Closes shadowman audit B5. Operators who need ms can still set it explicitly. Existing collections that already have a stored config record keep their value — only collections created post-upgrade WITHOUT an explicit `configureCollection()` inherit the new default. Mixed-precision rows in the same collection are read-correct via the 10^15 magnitude auto-detect; operators wanting clean rows should run `migrateCollectionTimestamps`. **`update()` docstring** updated to clarify: it OVERWRITES the doc; use `patch()` for partial mutations. Reflects shadowman audit B6.
 - **v2.1.1 cleanup — no defensive fallback on LMDB throw.** Removed the `catch → fall back to MemoryStore` paths in Exists / Get / Find / SimilaritySearch. A live LMDB exception now returns gRPC `INTERNAL` to the client instead of silently being masked by MemoryStore. The gate-closed fallback (`mirror_healthy_=false`, system collection, env unopened) stays — that's correctness, since MemoryStore can legitimately be ahead of LMDB when a mirror write has failed. **Answers shadowman audit F3:** `usedWalFallback` / `walMatchCount` / `memoryMatchCount` / `memorySearchMicros` / `walSearchMicros` are guaranteed zero on the LMDB-served path. Any non-zero value is a real signal — it means the read fell back to MemoryStore (mirror unhealthy). Operators can treat `walMatchCount > 0` as a paging/SLA alert.
 - **v2.1 audit answers (shadowman alignment).** F1 — v1.x auto-migration: stays in-binary through v2.x, extracts to separate `smartbotic-database-migrate-v1` deb in v3.0. F2 — `setReadOnly` / `lock()` / `unlock()` are dual-audience: operator-side via `smartbotic-db-cli unlock` and client-side via `Client::lock()` for self-defensive migrations. F3 — see above. F4 — `migrateCollectionTimestamps` cost benchmark deferred until shadowman starts D.1 (we haven't seen real traffic on the path yet); back-of-envelope ~1ms per 1k docs on Zoe-shape data, online-safe up to ~1M docs.
 - **Client API cleanup (v2.1.0)** — BREAKING. Removed the pre-v1.6 `smartbotic::database::QueryOptions` struct from `client/include/smartbotic/database/document.hpp` (dead code; the EQ-only `vector<pair<string, json>>` shape that predated FilterOp). Removed the no-options `Client::find(collection)` overload — pass `QueryOptions{}` explicitly. Added ergonomic factories on `Client::Filter`: `Filter::eq` / `ne` / `gt` / `gte` / `lt` / `lte` / `in` / `contains` / `exists` / `regex` / `search`. Callsites now read like the predicate they're asking: `find("users", {.filters = {Filter::gt("age", 18), Filter::in("role", {"admin", "owner"})}})`. SOVERSION stays at 2 (matches the v2.0.0 deb; no soname bump needed) — consumers must rebuild against v2.1 headers, but the soname is identical. `libsmartbotic-db-client (>= 2.1.0)` should be the floor in consumer deb deps.

+ 1 - 1
VERSION

@@ -1 +1 @@
-2.1.1
+2.2.0

+ 10 - 1
client/include/smartbotic/database/client.hpp

@@ -71,7 +71,16 @@ public:
     [[nodiscard]] std::optional<nlohmann::json> get(const std::string& collection, const std::string& id);
 
     /**
-     * Update a document.
+     * Replace a whole document. This OVERWRITES the existing doc — every
+     * field not in `data` is dropped.
+     *
+     * For partial mutations ("set field X to Y, leave everything else"),
+     * use `patch()` instead. v2.x consolidated the "read, mutate one
+     * field, write back" pattern onto `patch()` because that pattern is
+     * server-side atomic; doing it client-side with `update()` races
+     * with any other writer touching the same doc and was responsible
+     * for the v2.8.6→v2.8.8 conversation_tool_state split.
+     *
      * @param actor User ID performing the operation (for audit trail)
      */
     bool update(const std::string& collection, const std::string& id, const nlohmann::json& data,

+ 2 - 1
service/src/config/collection_config_manager.cpp

@@ -17,7 +17,8 @@ nlohmann::json toJson(const CollectionCfg& cfg) {
 
 CollectionCfg fromJson(const nlohmann::json& j) {
     CollectionCfg c;
-    c.timestampPrecision = j.value("timestamp_precision", std::string("ms"));
+    // v2.2 — default flipped to "ns" (see collection_config_manager.hpp).
+    c.timestampPrecision = j.value("timestamp_precision", std::string("ns"));
     return c;
 }
 

+ 8 - 2
service/src/config/collection_config_manager.hpp

@@ -16,8 +16,14 @@ class MemoryStore;
  * Mirrors the CollectionConfig proto message.
  */
 struct CollectionCfg {
-    // "ms" (default) or "ns"
-    std::string timestampPrecision = "ms";
+    // v2.2 — default flipped from "ms" to "ns". Rapid-write collections
+    // (chat messages, metrics events, tool-call audits) had ordering
+    // collisions when two writes landed in the same millisecond; ns gives
+    // strict-ordering guarantees out of the box. Operators who need ms
+    // can still set it explicitly via configureCollection(). Existing
+    // collections that already have a stored config keep their value;
+    // only collections without an explicit config inherit the new default.
+    std::string timestampPrecision = "ns";
 };
 
 /**

+ 1 - 1
service/src/database_grpc_impl.cpp

@@ -1850,7 +1850,7 @@ grpc::Status DatabaseGrpcImpl::ConfigureCollection(
         CollectionCfg cfg;
         cfg.timestampPrecision = request->config().timestamp_precision();
         if (cfg.timestampPrecision.empty()) {
-            cfg.timestampPrecision = "ms";  // default
+            cfg.timestampPrecision = "ns";  // v2.2 — default flipped from "ms"
         }
 
         // Idempotent: setConfig handles create-or-update

+ 8 - 6
tests/test_timestamp_precision.cpp

@@ -160,17 +160,17 @@ void test_cache_reflects_configure_immediately() {
     CollectionConfigManager mgr(store);
     store.setConfigManager(&mgr);
 
-    // Before configure: default ms, not explicit.
-    assert(mgr.configFor("docs_cache").timestampPrecision == "ms");
+    // Before configure: default ns (v2.2), not explicit.
+    assert(mgr.configFor("docs_cache").timestampPrecision == "ns");
     assert(!mgr.hasExplicitConfig("docs_cache"));
 
     CollectionCfg cfg;
-    cfg.timestampPrecision = "ns";
+    cfg.timestampPrecision = "ms";
     std::string err;
     assert(mgr.setConfig("docs_cache", cfg, err));
 
-    // After configure: ns, immediately — no reloadFromStore() required.
-    assert(mgr.configFor("docs_cache").timestampPrecision == "ns");
+    // After configure: ms, immediately — no reloadFromStore() required.
+    assert(mgr.configFor("docs_cache").timestampPrecision == "ms");
     assert(mgr.hasExplicitConfig("docs_cache"));
 
     store.stop();
@@ -184,7 +184,9 @@ void test_default_for_unknown_collection() {
     store.setConfigManager(&mgr);
 
     auto cfg = mgr.configFor("never_configured_collection");
-    assert(cfg.timestampPrecision == "ms");
+    // v2.2 — default flipped from "ms" to "ns" so rapid-write collections
+    // get nanosecond stamps without an explicit configureCollection() call.
+    assert(cfg.timestampPrecision == "ns");
     assert(!mgr.hasExplicitConfig("never_configured_collection"));
 
     store.stop();