Quellcode durchsuchen

feat(storage): open LmdbEnv in DatabaseService alongside MemoryStore

Stage 4 scaffold: DatabaseService now opens an LmdbEnv at <dataDir>/env/
(same path the v1.x auto-migration tool writes to) during setupComponents
and constructs an LmdbDocumentStore over it. Both stores coexist — gRPC
handlers still route through MemoryStore in this commit; subsequent Stage
4 / 5 commits migrate handlers one group at a time.

Failure to open the env is logged-and-tolerated for now: doc_store_ stays
null and the service boots on v1.x. Once handlers start depending on
doc_store_ this will be upgraded to a hard failure.

Member declaration order: env_ before doc_store_ so destruction is LIFO
(doc_store_ destroyed before env_, matching its non-owning reference into
env_).

Build green, all existing tests pass (test_document_store, test_lmdb_env,
test_views, test_timestamp_precision, test_vector_storage,
test_config_dropins, test_doc_binary, test_json_parse, test_eviction,
test_snapshot_durability, test_migrate_v1_to_v2).
fszontagh vor 2 Monaten
Ursprung
Commit
9ad6677500
2 geänderte Dateien mit 72 neuen und 0 gelöschten Zeilen
  1. 49 0
      service/src/database_service.cpp
  2. 23 0
      service/src/database_service.hpp

+ 49 - 0
service/src/database_service.cpp

@@ -1,5 +1,8 @@
 #include "database_service.hpp"
 #include "json_parse.hpp"
+#include "storage/document_store.hpp"
+#include "storage/document_store_lmdb.hpp"
+#include "storage/lmdb_env.hpp"
 
 #include <grpcpp/grpcpp.h>
 #include <grpcpp/resource_quota.h>
@@ -505,6 +508,52 @@ DatabaseService::Config DatabaseService::parseConfig(const nlohmann::json& json)
 }
 
 void DatabaseService::setupComponents() {
+    // v2.0 storage substrate (Stage 4 scaffold).
+    //
+    // Open the LMDB environment at <dataDir>/env/ — same path the v1.x
+    // auto-migration tool in main.cpp writes to. By the time setupComponents
+    // runs, either:
+    //   (a) the env already exists and contains migrated v2.0 data, or
+    //   (b) this is a fresh deployment and the env is empty.
+    // Both cases are fine: LmdbDocumentStore opens sub-dbs lazily on first
+    // put/read, so an empty env is a valid steady state.
+    //
+    // map_size_bytes / max_dbs match the values main.cpp uses for migration
+    // so that opening the same env from both call sites is consistent.
+    //
+    // Failure to open the env is currently logged-and-tolerated: Stage 4
+    // does not yet route any handlers through doc_store_, so a missing or
+    // unopenable env should not kill the service. Stage 5+ may upgrade this
+    // to a hard failure once handlers depend on the store.
+    {
+        smartbotic::db::storage::LmdbEnvOpts envOpts;
+        envOpts.path = (config_.dataDirectory / "env").string();
+        envOpts.map_size_bytes = 4ULL << 30;  // 4 GiB initial; LMDB grows lazily.
+        envOpts.max_dbs = 1024;
+
+        std::error_code ec;
+        std::filesystem::create_directories(envOpts.path, ec);
+        if (ec) {
+            spdlog::warn("v2.0 storage: failed to create env directory '{}': {} -- "
+                         "doc_store_ will be left null; Stage 4 still routes through "
+                         "MemoryStore so the service continues to boot",
+                         envOpts.path, ec.message());
+        } else {
+            try {
+                env_ = std::make_unique<smartbotic::db::storage::LmdbEnv>(envOpts);
+                doc_store_ = std::make_unique<smartbotic::db::storage::LmdbDocumentStore>(*env_);
+                spdlog::info("v2.0 storage: LMDB env opened at {} (mapsize={}MB, max_dbs={})",
+                             envOpts.path, envOpts.map_size_bytes >> 20, envOpts.max_dbs);
+            } catch (const std::exception& e) {
+                spdlog::warn("v2.0 storage: failed to open LMDB env at '{}': {} -- "
+                             "doc_store_ will be left null; service continues with v1.x",
+                             envOpts.path, e.what());
+                env_.reset();
+                doc_store_.reset();
+            }
+        }
+    }
+
     // Create memory store with eviction config
     MemoryStore::Config storeConfig;
     storeConfig.nodeId = config_.nodeId;

+ 23 - 0
service/src/database_service.hpp

@@ -13,6 +13,15 @@
 #include "views/view_manager.hpp"
 #include "config/collection_config_manager.hpp"
 
+// v2.0 storage substrate (Stage 4) — DocumentStore + LmdbEnv live alongside
+// the v1.x MemoryStore during the Phase C transition. Stage 4 opens both;
+// later stages migrate handlers one group at a time, deleting v1.x paths
+// once they go cold.
+namespace smartbotic::db::storage {
+class LmdbEnv;
+class DocumentStore;
+}
+
 #include <nlohmann/json.hpp>
 
 #include <atomic>
@@ -198,6 +207,20 @@ private:
 
     // Components
     std::unique_ptr<MemoryStore> store_;
+
+    // v2.0 storage substrate (Stage 4 scaffold).
+    //
+    // The LMDB env is opened at `<dataDir>/env/` — the same path the v1.x
+    // auto-migration tool writes to. Stage 4 lands these as members so the
+    // service has the v2.0 store in hand but does not yet route gRPC
+    // handlers through it; that swap happens incrementally in Stage 4 / 5.
+    //
+    // Ownership: env_ owns the MDB_env*; doc_store_ holds a non-owning
+    // reference into env_. Declaration order matters — doc_store_ must be
+    // destroyed before env_.
+    std::unique_ptr<smartbotic::db::storage::LmdbEnv> env_;
+    std::unique_ptr<smartbotic::db::storage::DocumentStore> doc_store_;
+
     std::unique_ptr<PersistenceManager> persistence_;
     std::unique_ptr<HistoryStore> history_store_;  // v1.9.0 — disk-resident history
     std::unique_ptr<EncryptionManager> encryption_;