瀏覽代碼

docs: add vector storage docs, .gitignore, project CLAUDE.md

README gains a Vector Storage & Similarity Search section with usage
example and notes, plus the SimilaritySearch RPC entry and an updated
migration example. Adds standard .gitignore (build, IDE, *.key, cores)
and a project-level CLAUDE.md with build/test commands and architecture
overview.
fszontagh 3 月之前
父節點
當前提交
06a8b0969d
共有 3 個文件被更改,包括 114 次插入1 次删除
  1. 23 0
      .gitignore
  2. 40 0
      CLAUDE.md
  3. 51 1
      README.md

+ 23 - 0
.gitignore

@@ -0,0 +1,23 @@
+# Build
+build/
+cmake-build-*/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+.cache/
+compile_commands.json
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Encryption keys
+*.key
+
+# Core dumps
+core
+core.*

+ 40 - 0
CLAUDE.md

@@ -0,0 +1,40 @@
+# Smartbotic Database
+
+## Build
+
+```bash
+cmake -B build -G Ninja
+cmake --build build -j$(nproc)
+```
+
+## Test
+
+```bash
+cmake --build build -j$(nproc) && ./build/tests/test_vector_storage
+```
+
+## Architecture
+
+- `proto/database.proto` — gRPC service and message definitions
+- `service/src/` — Database server implementation
+  - `memory_store.hpp/.cpp` — In-memory document + vector storage
+  - `document.hpp` — Document and CollectionOptions types
+  - `database_grpc_impl.cpp` — gRPC request handlers
+  - `persistence/` — WAL, snapshots, persistence manager
+  - `migrations/` — Migration runner
+- `client/` — C++ client library (`smartbotic-db-client`)
+- `tests/` — Integration tests
+
+## Key Features
+
+- JSON document store with collections, version history, field-level encryption
+- gRPC API with replication, events, file storage, migrations
+- **Vector Storage** — Embedding vectors stored alongside documents with SIMD-accelerated (AVX2/SSE4.1) cosine similarity search via `SimilaritySearch` RPC. Vectors use `_vector` document field, stored in parallel float arrays, persisted through WAL (`VEC_PUT`/`VEC_DELETE`) and snapshots (v3 format). Collection `vector_dimension` option is immutable after creation.
+
+## Conventions
+
+- C++20
+- nlohmann/json for JSON handling
+- spdlog for logging
+- LZ4 for compression
+- gRPC/Protobuf for API

+ 51 - 1
README.md

@@ -12,6 +12,7 @@ A standalone key-value document database with gRPC API, designed for microservic
 - **Encryption** - Field-level AES-256-GCM encryption for sensitive data
 - **Events** - Pub/sub event subscription for real-time updates
 - **File Storage** - Binary file management with streaming support
+- **Vector Storage** - Embedding vector storage with SIMD-accelerated cosine similarity search
 - **Migrations** - Declarative JSON-based schema migrations
 
 ## Architecture
@@ -119,6 +120,36 @@ auto [data, metadata] = client.downloadFile(fileId);
 client.deleteFile(fileId);
 ```
 
+### Vector Storage & Similarity Search
+
+Store embedding vectors alongside documents and perform cosine similarity search — no external dependencies required.
+
+```cpp
+// Create a vector-enabled collection (dimension must be fixed at creation)
+client.createCollection("memories", 0, false, 0, 384);  // 384-dim embeddings
+
+// Insert documents with _vector field
+nlohmann::json doc = {
+    {"id", "mem-1"},
+    {"content", "The user prefers dark mode"},
+    {"_vector", embedding}  // std::vector<float> of size 384
+};
+client.put("memories", doc);
+
+// Similarity search — returns top-K results sorted by cosine similarity
+auto results = client.similaritySearch("memories", queryEmbedding, /*topK=*/5, /*minScore=*/0.7f);
+for (const auto& r : results) {
+    std::cout << r.id << " (score=" << r.score << "): "
+              << r.data["content"] << std::endl;
+}
+```
+
+Notes:
+- `_vector` is extracted and stored separately — it is **not** returned in `Get()` or `Query()` responses
+- Dimension is validated on every insert/update — mismatches are rejected
+- Documents without `_vector` are stored normally but excluded from similarity search
+- SIMD-accelerated (AVX2/SSE4.1) brute-force scan, suitable for up to ~100K vectors
+
 ### Subscribing to Events
 
 ```cpp
@@ -179,6 +210,7 @@ History depth is configurable per-collection via `max_versions` (0 = unlimited).
 | `UploadFile` | Upload binary file (streaming) |
 | `DownloadFile` | Download binary file (streaming) |
 | `DeleteFile` | Delete file by ID |
+| `SimilaritySearch` | Cosine similarity search over document vectors |
 | `GetFileMetadata` | Get file metadata |
 
 ### DatabaseReplication
@@ -277,10 +309,28 @@ Create JSON migration files in the migrations directory. Files are processed in
 }
 ```
 
-**002_seed_admin.json:**
+**002_create_memories.json:**
 ```json
 {
   "version": "002",
+  "name": "create_memories",
+  "description": "Create vector-enabled memories collection",
+  "operations": [
+    {
+      "type": "create_collection",
+      "collection": "memories",
+      "options": {
+        "vector_dimension": 4096
+      }
+    }
+  ]
+}
+```
+
+**003_seed_admin.json:**
+```json
+{
+  "version": "003",
   "name": "seed_admin",
   "description": "Create default admin user",
   "operations": [