|
|
@@ -334,6 +334,99 @@ The `op` field in `where` filters accepts either a string name (`EQ`, `NE`, `GT`
|
|
|
|
|
|
The view's filters act as a **security enforcement boundary**. A consumer querying `active_admins` literally cannot see inactive users — the filter is applied server-side, after authentication, before projection. This is how SQL views with `WHERE` clauses work, and how row-level security works in most databases. Any consumer filter is ANDed on top, narrowing further.
|
|
|
|
|
|
+### Collection Configuration
|
|
|
+
|
|
|
+Per-collection runtime settings. Currently supports timestamp precision; extensible for future knobs.
|
|
|
+
|
|
|
+#### Timestamp precision
|
|
|
+
|
|
|
+Every document carries `_created_at` and `_updated_at` — int64 timestamps since epoch. By default these are **milliseconds**. Collections that receive rapid-fire writes (LLM streaming chunks, agent tool loops) often see ties at ms resolution that make `ORDER BY _created_at` ambiguous. Configure those collections for **nanoseconds** to get unique, strictly ordered timestamps.
|
|
|
+
|
|
|
+```cpp
|
|
|
+using Cfg = smartbotic::database::Client::CollectionConfig;
|
|
|
+
|
|
|
+// Default: ms (no configuration needed)
|
|
|
+auto id = db.insert("logs", {{"msg", "..."}});
|
|
|
+// doc["_created_at"] is ~13 digits, milliseconds since epoch
|
|
|
+
|
|
|
+// Switch a collection to nanoseconds
|
|
|
+db.configureCollection("llm_tokens", Cfg{"ns"});
|
|
|
+
|
|
|
+// Future inserts into llm_tokens get ns-precision timestamps
|
|
|
+auto id2 = db.insert("llm_tokens", {{"token", "hello"}});
|
|
|
+// doc["_created_at"] is ~19 digits, nanoseconds since epoch
|
|
|
+```
|
|
|
+
|
|
|
+#### Reading the config
|
|
|
+
|
|
|
+```cpp
|
|
|
+auto cfg = db.getCollectionConfig("llm_tokens");
|
|
|
+std::cout << cfg.timestampPrecision << std::endl; // "ns"
|
|
|
+
|
|
|
+bool explicit_set = db.hasCollectionConfig("llm_tokens"); // true
|
|
|
+bool default_only = db.hasCollectionConfig("logs"); // false (still ms default)
|
|
|
+```
|
|
|
+
|
|
|
+#### What configureCollection does NOT do
|
|
|
+
|
|
|
+**It does not convert existing timestamps.** Flipping a collection from `ms` to `ns` only affects *future* inserts. Pre-existing docs keep whatever precision they had. This keeps the config flip atomic and reversible.
|
|
|
+
|
|
|
+Consequences of mixed precision within one collection:
|
|
|
+- `int64` ordering within the collection is still consistent (ms values are always < 10^15, ns values are always > 10^15, so ns docs sort after ms docs). Queries don't break.
|
|
|
+- But the ns-labeled docs appear to be in the future relative to the ms-labeled docs unless you convert.
|
|
|
+
|
|
|
+#### Migrating existing timestamps
|
|
|
+
|
|
|
+When you're ready to convert legacy data, run the migration helper during a planned maintenance window:
|
|
|
+
|
|
|
+```cpp
|
|
|
+auto result = db.migrateCollectionTimestamps("llm_tokens",
|
|
|
+ /*fromPrecision=*/"ms",
|
|
|
+ /*toPrecision=*/"ns");
|
|
|
+if (result.success) {
|
|
|
+ std::cout << "migrated " << result.rowsMigrated
|
|
|
+ << " rows, skipped " << result.rowsSkipped << std::endl;
|
|
|
+} else {
|
|
|
+ std::cerr << "failed: " << result.error << std::endl;
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+Semantics:
|
|
|
+- `ms → ns`: multiply `_created_at` / `_updated_at` by 10^6
|
|
|
+- `ns → ms`: divide by 10^6 (truncation — loses sub-ms precision)
|
|
|
+- **Idempotent and resumable.** Rows that are already in the target range are detected via a 10^15 threshold and skipped. If the migration is interrupted (network drop, deadline), rerun it — already-converted rows will count as `rows_skipped`.
|
|
|
+- `from == to` is a no-op success.
|
|
|
+- **Metadata-only** — document version numbers are NOT incremented, WAL is not appended, history is not updated. This is an infrastructure rewrite, not a user-visible change.
|
|
|
+- Deadline: the client's RPC timeout for this call is 10 minutes — large collections may take a while.
|
|
|
+
|
|
|
+#### Recommended workflow for switching precision on production data
|
|
|
+
|
|
|
+1. Schedule a maintenance window (writes against the collection may pause briefly if needed).
|
|
|
+2. Call `configureCollection(name, {ns})` — atomic config flip; future writes immediately use ns.
|
|
|
+3. Call `migrateCollectionTimestamps(name, "ms", "ns")` — converts existing docs.
|
|
|
+4. Verify with a sample read that `_created_at` values are now in the ns range.
|
|
|
+5. Resume normal traffic.
|
|
|
+
|
|
|
+Between steps 2 and 3, new writes are ns and old docs are ms — this is temporary but normal. Ordering still works because the 10^15 threshold cleanly separates the two ranges.
|
|
|
+
|
|
|
+#### Via migrations
|
|
|
+
|
|
|
+To set timestamp precision declaratively, call `db.configureCollection()` from your consumer project's startup code. The `configure_collection` migration operation will be available in a future version.
|
|
|
+
|
|
|
+```json
|
|
|
+{
|
|
|
+ "version": "021",
|
|
|
+ "name": "llm_tokens_ns_precision",
|
|
|
+ "operations": [
|
|
|
+ {
|
|
|
+ "type": "configure_collection",
|
|
|
+ "collection": "llm_tokens",
|
|
|
+ "config": { "timestamp_precision": "ns" }
|
|
|
+ }
|
|
|
+ ]
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
### Querying
|
|
|
|
|
|
```cpp
|