Переглянути джерело

docs: v2.4.5 collection namespacing + versioning switch; fix stale ms default

Three gaps, one of them a factual error predating this release:

- integration-guide claimed timestamps default to milliseconds. v2.2.0 flipped
  the default to ns; the prose, the code sample and the hasCollectionConfig
  example were all wrong. Also notes that Client::CollectionConfig still
  defaults the field to "ms", so Cfg{} sends an explicit ms rather than
  meaning "server default".
- Documented collections-and-projects with the same shape as the existing
  views-and-projects section, including the 2.3.0-2.4.4 breakage and what does
  and does not get fixed by upgrading (vectorDimension is immutable, so an
  affected vector collection must be recreated).
- Documented the versioning switch and partial-update semantics, and why
  versioning_enabled is 'optional' in the proto.

README feature list was missing LMDB, multi-project namespaces, views,
timestamp precision and TLS/auth. load_test README now covers
test_client_namespacing.sh plus the two e2e scripts that were already there
but undocumented.
fszontagh 1 місяць тому
батько
коміт
44bb2baafe
3 змінених файлів з 181 додано та 15 видалено
  1. 20 2
      README.md
  2. 124 13
      docs/integration-guide.md
  3. 37 0
      tests/load_test/README.md

+ 20 - 2
README.md

@@ -4,8 +4,12 @@ A standalone key-value document database with gRPC API, designed for microservic
 
 ## Features
 
-- **Document Storage** - JSON document store with collections
-- **Version History** - Per-document version tracking with restore by version or date
+- **Document Storage** - JSON document store with collections, backed by LMDB (2.0+)
+- **Multi-project Namespaces** - One instance hosts many `project` namespaces, addressed as `<project>:<collection>` (2.3+)
+- **Version History** - Per-document version tracking with restore by version or date; switchable per collection (2.4.5+)
+- **Views** - Read-only named projections over collections with baked-in filters and default sort (1.5+)
+- **Per-collection Timestamp Precision** - `ms` or `ns` stamping, configurable per collection (2.2+)
+- **TLS + Bearer Auth** - Per-listener TLS and token auth; one process, multiple listeners (2.4+)
 - **Health Check** - gRPC `HealthCheck()` RPC for service status verification
 - **Replication** - Multi-master replication via `DatabaseReplication` gRPC service
 - **Persistence** - WAL (Write-Ahead Log) + snapshots with LZ4 compression
@@ -190,6 +194,20 @@ uint64_t restored = client.restoreVersion("users", docId, 1, "admin");
 
 History depth is configurable per-collection via `max_versions` (0 = unlimited). History survives restarts through snapshots and WAL replay.
 
+Since 2.4.5, history can also be turned **off** per collection — on a collection
+that already exists — via `configureCollection()`:
+
+```cpp
+smartbotic::database::Client::CollectionConfig cfg;
+cfg.timestampPrecision = "";     // leave precision unchanged
+cfg.versioningEnabled  = false;  // stop recording new versions
+client.configureCollection("metrics_events", cfg);
+```
+
+Disabling stops new versions; history already written stays readable, so the
+switch is reversible. This differs from `max_versions = 0`, which means
+"keep unlimited". See the [integration guide](docs/integration-guide.md) for details.
+
 ## gRPC API
 
 ### DatabaseService

+ 124 - 13
docs/integration-guide.md

@@ -370,33 +370,98 @@ Per-collection runtime settings. Currently supports timestamp precision; extensi
 
 #### Timestamp precision
 
-Every document carries `_created_at` and `_updated_at` — int64 timestamps since epoch. By default these are **milliseconds**. Collections that receive rapid-fire writes (LLM streaming chunks, agent tool loops) often see ties at ms resolution that make `ORDER BY _created_at` ambiguous. Configure those collections for **nanoseconds** to get unique, strictly ordered timestamps.
+Every document carries `_created_at` and `_updated_at` — int64 timestamps since epoch.
+
+**Since 2.2.0 the default is nanoseconds (`ns`).** It was milliseconds up to
+2.1.x. The flip was made because collections that receive rapid-fire writes
+(LLM streaming chunks, agent tool loops) saw ties at ms resolution that made
+`ORDER BY _created_at` ambiguous; ns gives unique, strictly ordered timestamps
+out of the box. Collections that already had a stored config keep their value —
+only collections created after the upgrade *without* an explicit
+`configureCollection()` inherit the new default.
 
 ```cpp
 using Cfg = smartbotic::database::Client::CollectionConfig;
 
-// Default: ms (no configuration needed)
-auto id = db.insert("logs", {{"msg", "..."}});
-// doc["_created_at"] is ~13 digits, milliseconds since epoch
+// Default since 2.2.0: ns (no configuration needed)
+auto id = db.insert("llm_tokens", {{"token", "hello"}});
+// doc["_created_at"] is ~19 digits, nanoseconds since epoch
 
-// Switch a collection to nanoseconds
-db.configureCollection("llm_tokens", Cfg{"ns"});
+// Opt a collection back to milliseconds
+db.configureCollection("logs", Cfg{"ms"});
 
-// Future inserts into llm_tokens get ns-precision timestamps
-auto id2 = db.insert("llm_tokens", {{"token", "hello"}});
-// doc["_created_at"] is ~19 digits, nanoseconds since epoch
+// Future inserts into logs get ms-precision timestamps
+auto id2 = db.insert("logs", {{"msg", "..."}});
+// doc["_created_at"] is ~13 digits, milliseconds since epoch
 ```
 
+> **Note:** `Client::CollectionConfig::timestampPrecision` still defaults to
+> `"ms"` in the header, so `Cfg{}` sends an explicit `"ms"` rather than meaning
+> "server default". Pass the precision you want explicitly, or leave the string
+> empty to mean "leave unchanged" (2.4.5+, see partial updates below).
+
 #### Reading the config
 
 ```cpp
-auto cfg = db.getCollectionConfig("llm_tokens");
-std::cout << cfg.timestampPrecision << std::endl;  // "ns"
+auto cfg = db.getCollectionConfig("logs");
+std::cout << cfg.timestampPrecision << std::endl;  // "ms"
+
+bool explicit_set = db.hasCollectionConfig("logs");        // true
+bool default_only = db.hasCollectionConfig("llm_tokens");  // false (still ns default)
+```
+
+#### Version history on/off (2.4.5+)
+
+Version history can be switched per collection, **on a collection that already
+exists and already holds data**:
+
+```cpp
+Cfg off;
+off.timestampPrecision = "";      // empty = leave precision unchanged
+off.versioningEnabled  = false;   // stop recording new versions
+db.configureCollection("metrics_events", off);
+
+// ... later, turn it back on
+Cfg on;
+on.timestampPrecision = "";
+on.versioningEnabled  = true;
+db.configureCollection("metrics_events", on);
+```
+
+Semantics:
+
+- **Reversible.** Disabling stops *new* versions being recorded. History already
+  on disk stays readable via `getVersionHistory()` / `getDocumentVersion()`, so
+  re-enabling simply resumes appending.
+- **Not the same as `maxVersions = 0`,** which means "keep unlimited history".
+  `versioningEnabled = false` means "keep none".
+- **Durable.** The setting lives in the `_collection_meta` system collection, so
+  it survives restarts through the normal WAL + snapshot path.
+- Takes effect on the next write.
 
-bool explicit_set = db.hasCollectionConfig("llm_tokens");  // true
-bool default_only = db.hasCollectionConfig("logs");        // false (still ms default)
+#### Partial updates (2.4.5+)
+
+`configureCollection()` is a **partial update**. Fields you leave unset are
+preserved, not reset:
+
+- empty `timestampPrecision` → precision unchanged
+- unset `versioningEnabled` (`std::nullopt`) → versioning unchanged
+
+```cpp
+Cfg justPrecision;
+justPrecision.timestampPrecision = "ms";   // versioningEnabled stays nullopt
+db.configureCollection("logs", justPrecision);
+// -> precision becomes ms; versioning is left exactly as it was
 ```
 
+On `getCollectionConfig()`, `versioningEnabled` is always populated with the
+collection's effective value rather than the "leave unchanged" sentinel.
+
+> Before 2.4.5 this RPC replaced the whole config. That was harmless while
+> precision was the only field, but it is why `versioning_enabled` is declared
+> `optional` in the proto: a plain proto3 bool defaults to `false`, so any
+> precision-only call would otherwise have silently switched versioning **off**.
+
 #### What configureCollection does NOT do
 
 **It does not convert existing timestamps.** Flipping a collection from `ms` to `ns` only affects *future* inserts. Pre-existing docs keep whatever precision they had. This keeps the config flip atomic and reversible.
@@ -690,6 +755,52 @@ if (info) {
 db.dropCollection("old_data");
 ```
 
+#### Collections and projects (2.4.5+)
+
+Collection names are namespaced with the client's `Config::project`, exactly
+like documents. You keep passing bare names; the client qualifies them on the
+wire and un-qualifies them coming back, so `listCollections()` round-trips:
+
+```cpp
+Client::Config cfg;
+cfg.project = "acme";
+Client db(cfg);
+
+db.createCollection("users");              // -> acme:users
+db.insert("users", {{"name", "Ada"}});     // -> acme:users
+auto info = db.getCollectionInfo("users"); // -> acme:users
+db.listCollections();                      // -> { "users", ... }
+```
+
+> **Fixed in 2.4.5 — upgrade if you call `createCollection`.** On 2.3.0-2.4.4
+> `createCollection`, `dropCollection` and `getCollectionInfo` sent the **bare**
+> name while `insert`/`get`/`find` sent the project-qualified one. Because the
+> client qualifies against `Config::project` unconditionally — including
+> `default` — **every** project was affected. One `createCollection("things")`
+> plus one `insert("things")` created *two* collections: the bare one received
+> the options and stayed empty, while `<project>:things` was created implicitly
+> by the insert with **defaults**. No error was raised.
+>
+> Concretely, on affected versions:
+> - `vectorDimension` never reached the collection holding the data, so
+>   `similaritySearch()` returned no hits.
+> - `encrypted = true` silently did not apply.
+> - `dropCollection()` returned `true` while every document remained readable —
+>   it dropped the empty phantom.
+>
+> **After upgrading:** leftover empty phantom collections are harmless litter,
+> but options that were swallowed do **not** retroactively apply.
+> `createCollection` on a now-correctly-resolved existing collection returns
+> `false` and only updates `maxVersions`. Since `vectorDimension` is **immutable
+> after creation**, a collection that needs vector search must be recreated:
+> create a new collection with the right `vectorDimension`, copy the documents
+> across, then drop the old one.
+
+Note that `listCollections()` currently returns every project's collections —
+names outside your own project stay visibly qualified rather than being
+flattened into yours. Unlike `ListViewsRequest`, `ListCollectionsRequest` has no
+project filter yet.
+
 ### Vector Similarity Search
 
 Requires a collection created with `vectorDimension > 0`.

+ 37 - 0
tests/load_test/README.md

@@ -448,3 +448,40 @@ And in the follower server log:
 
 This confirms replication worked (5000 stubs exist on the follower and their IDs match what the leader inserted) — but the evicted-stub recovery path has nothing to page in from.
 
+
+---
+
+## Test 9 — Client/server boundary: namespacing + versioning (`test_client_namespacing.sh`)
+
+    ./test_client_namespacing.sh
+
+17 assertions driving a real `Client` over gRPC with `Config::project` set to a
+**non-default** project. Boots its own server on port 9011 and tears it down.
+
+This exists because the unit suite structurally cannot catch this class of bug.
+Two shipped namespacing breaks — v2.4.2's `createView` and v2.4.5's
+`createCollection` / `dropCollection` / `getCollectionInfo` — were both
+invisible to `ctest`, because `tests/test_views.cpp` only exercises
+`applyProjection()` in-process and nothing drove the client/server boundary with
+a project set. These bugs are only observable on the wire.
+
+Covers:
+
+- **Part 1 — collection management is project-namespaced.** `createCollection`
+  does not create a phantom; `listCollections()` round-trips bare names;
+  `getCollectionInfo` counts the caller's documents; `vectorDimension` reaches
+  the collection that holds the data (asserted via a real `similaritySearch`);
+  `dropCollection` actually deletes.
+- **Part 2 — versioning switch.** On by default; disabling stops new versions
+  while leaving existing history readable; a precision-only
+  `configureCollection` does **not** clobber the versioning setting (the proto3
+  field-presence trap); re-enabling resumes recording.
+
+## Also present, not yet documented above
+
+- `test_views_multiproject.sh` — 17 checks over two projects: projection, view
+  name reuse across projects, cross-project isolation, `NOT_FOUND` on a name
+  that is neither view nor collection.
+- `test_v24_tls_auth.sh` — boots plaintext + TLS/auth listeners and drives 4
+  scenarios: plaintext-local OK, TLS+token OK, TLS+wrong-token
+  `UNAUTHENTICATED`, TLS+no-token `UNAUTHENTICATED`.