Strona główna Odkrywaj Pomoc
Zaloguj się
fszontagh
/
smartbotic-database
1
0
Forkuj 0
Pliki Problemy 0 Oczekujące zmiany 0 Wiki
Drzewo: 41c858a419
Gałęzie Tagi
smartbotic-data...
/
docs
/
superpowers
/
plans
/
2026-05-16-v2.3-multi-project.md

2026-05-16-v2.3-multi-project.md 6.4 KB
Historia Czysty

v2.3 multi-project plan

Goal: every smartbotic-database instance hosts multiple projects, each with its own namespace of collections. Backward-compatible by construction — a v2.2 client that doesn't address a project transparently uses "default", which auto-migrates from the existing <dataDir>/env/ on upgrade.

Settled design (from operator decisions today)

Axis Decision
Namespace name project
Wire shape Collection name encodes project: "my_app:users" parses to ("my_app","users"); bare "users" → ("default","users")
Separator : (single, only one allowed)
Storage layout One LMDB env per project at <dataDir>/projects/<name>/env/
Upgrade path First v2.3 boot moves <dataDir>/env/ → <dataDir>/projects/default/env/ atomically
Toggle / first-boot mode None. Multi-project is always on. v2.2 clients without explicit project see identical behaviour via the default project.
Project codebase rename Deferred. v2.x arc is supposed to freeze surface area.

Parser semantics locked in by tests/test_project_addressing.cpp (23 assertions). Strict on edge cases — leading/trailing/double : all throw std::invalid_argument.

Reserved names

  • default: always present, addressable. Idempotent on create.
  • _-prefix: reserved for system use (matches collection-name rules).
  • Length: 1..63 chars, ^[a-zA-Z_][a-zA-Z0-9_-]{0,62}$.

Stage breakdown

Each stage ends at a buildable, testable, committable state.

Stage A — Foundation: parser + addressing types ✅ DONE

  • service/src/project_addressing.hpp — parseProjectCollection, isValidProjectName, constants.
  • tests/test_project_addressing.cpp — 23 assertions cover the wire-form edge cases + the name-validation regex boundary.

Stage B — Per-project storage layout

  • New ProjectStore type wrapping (LmdbEnv, LmdbDocumentStore) per project.
  • DatabaseService holds std::map<std::string, std::unique_ptr<ProjectStore>> instead of single env_/doc_store_.
  • Lazy-open semantics: first write to project <X> creates <dataDir>/projects/<X>/env/ and registers the store.
  • Eager-open at boot: scan <dataDir>/projects/*/env/, open each.
  • Default project always exists — created at boot if absent.

Files touched: database_service.cpp/.hpp, possibly a new storage/project_store.hpp. Test: new tests/test_project_store.cpp exercising open/lazy-create/list.

Stage C — v2.2 → v2.3 storage migration

  • On boot: if <dataDir>/env/ exists AND <dataDir>/projects/default/env/ does not, atomically rename the directory.
  • Idempotent: if both exist (shouldn't, but defensively) refuse to start with a clear error.
  • Logs the move at INFO level for operator visibility.

Files touched: database_service.cpp (boot path). Test: extend test_project_store.cpp with a v2.2-layout fixture.

Stage D — Handler routing through project

  • Every gRPC handler that receives collection parses via parseProjectCollection and dispatches against the resolved project's store. Same change ~12 sites in database_grpc_impl.cpp.
  • The MemoryStore-side mirror gets the qualified name "project:collection" as its in-memory key, so concurrent reads see consistent state. The LMDB-side mirror calls the resolved project's doc_store_->put.

Files touched: database_grpc_impl.cpp (every handler), memory_store.cpp (qualified-key handling), database_service.cpp (mirror routing).

Stage E — Client API

  • Add Client::Config::project field (default "default"). When set, the Client prepends project + ":" to every bare collection name in outgoing requests. Explicit "other:users" overrides the sticky default.
  • Documented examples in client.hpp.
  • Per-call form client.find("other:users", ...) always works without setting Config::project.

Files touched: client/include/smartbotic/database/client.hpp, client/src/client.cpp. Tests in the existing client tests (extend or add a new file).

Stage F — Project CRUD RPCs

  • ListProjects() — returns the set of open project names. Filesystem scan-backed; trivial.
  • CreateProject(name) — validates name, ensures <dataDir>/projects/<name>/env/ exists. Idempotent. Useful so admins can pre-create empty projects before the first write.
  • DropProject(name) — refuses if name == "default"; closes the env and rm -rfs the dir. Operator-only.

Files touched: proto/database.proto (3 new RPCs), service implementations, client wrappers.

Stage G — Tests, docs, release

  • test_project_addressing ✅
  • test_project_store — boot, lazy-create, list, drop, default-project always present.
  • test_v22_to_v23_migration — synthetic v2.2 dataDir, boot, verify the env moves to projects/default/env/.
  • Integration test that hits two projects from two Client instances concurrently and verifies isolation.
  • CLAUDE.md update with the v2.3 entry + back-compat statement.
  • VERSION bump to 2.3.0.
  • Local + zeus + apt repo publish.

Out of scope for v2.3

  • Per-project memory budgets / quotas (could come in v2.3.x).
  • Per-project ACLs / auth (no auth in the wire today).
  • Project rename (project codebase rename — deferred).
  • Listing projects from the CLI tool (smartbotic-db-cli projects) — can wait for v2.3.x.

Open questions

  1. MemoryStore: per-project or global with qualified keys? v2.3 plan currently says global+qualified. Cheaper, simpler, no per-project eviction tuning. Right call for v2.3 since MemoryStore is being decommissioned in v2.4 anyway.
  2. Per-project mapsize override in config? Add a projects.<name>.mapsize_mb schema, or just take a global default? Global default for v2.3; per-project tuning in v2.3.x if real pressure shows up.
  3. Operator visibility for "you typed : in your collection name in single-project mode"? Now moot since there's no single-project mode — but if a v2.2 client accidentally sends "my_app:users", the server will route to project "my_app", NOT to a collection named literally "my_app:users". Could be surprising. Mitigation: every v2.2 client should NOT be sending : in collection names today, so the surprise window is "nobody actually does this." OK to ignore.

Estimated effort

Stages A-G total: ~1-2 days of focused work. The biggest two are Stage D (handler routing — many sites, all formulaic) and Stage F (proto + RPC wiring). Stage B + C are the meatiest design pieces.