# Smartbotic Database

## Build

```bash
cmake -B build -G Ninja
cmake --build build -j$(nproc)
```

## Test

```bash
cmake --build build -j$(nproc) && ./build/tests/test_vector_storage
```

## Architecture

- `proto/database.proto` — gRPC service and message definitions
- `service/src/` — Database server implementation
  - `memory_store.hpp/.cpp` — In-memory document + vector storage
  - `document.hpp` — Document and CollectionOptions types
  - `database_grpc_impl.cpp` — gRPC request handlers
  - `persistence/` — WAL, snapshots, persistence manager
  - `migrations/` — Migration runner
- `client/` — C++ client library (`smartbotic-db-client`)
- `tests/` — Integration tests

## Key Features

- JSON document store with collections, version history, field-level encryption
- gRPC API with replication, events, file storage, migrations
- **Vector Storage** — Embedding vectors stored alongside documents with SIMD-accelerated (AVX2/SSE4.1) cosine similarity search via `SimilaritySearch` RPC. Vectors use `_vector` document field, stored in parallel float arrays, persisted through WAL (`VEC_PUT`/`VEC_DELETE`) and snapshots (v3 format). Collection `vector_dimension` option is immutable after creation.
- **Atomic Partial Updates** — `PatchDocument` RPC merges fields into existing documents atomically (server-side, within collection lock). Client `patch()` method. `update()` uses automatic optimistic locking with retry. See `docs/integration-guide.md` for concurrency patterns.
- **Views** — Read-only named projections over collections. Include/exclude field lists (dot-notation for nested paths), baked-in filters with all operators (EQ/NE/GT/GTE/LT/LTE/IN/CONTAINS/EXISTS/REGEX/SEARCH) AND-merged with caller filters, optional default sort that caller can override. Stored in `_views` system collection, created via `create_view` migration op or `createView` RPC. Writes on view names are rejected.

## Packaging

Canonical source for all smartbotic-database Debian packages.
Replaces the old `shadowman-database` and `callerai-storage` packages.

### Build modes

```bash
# Production (Docker, Debian 13)
./packaging/build.sh                          # Build 4 .debs
./packaging/build.sh --rebuild-base           # Rebuild Docker base image
./packaging/build.sh --sync --suite trixie    # Build + publish to repository.smartbotics.ai

# Local development (native, current system)
./packaging/build.sh --local                  # Build 4 .debs
./packaging/build.sh --local --install        # Build + install locally
```

### Packages

| Package | Contents | Deployed where |
|---------|----------|---------------|
| `smartbotic-database` | Server binary + systemd + config | Production DB servers |
| `libsmartbotic-db-client` | Shared `.so` (runtime) | All production machines |
| `libsmartbotic-db-client-dev` | Headers + linker symlink + `.proto` + cmake config | Docker build images, dev workstations |
| `smartbotic-db-cli` | CLI tool | Anywhere (optional) |

### Local dev with installed packages

After `./packaging/build.sh --local --install`, consumer projects can use:

```cmake
find_package(smartbotic-db-client REQUIRED)
target_link_libraries(myapp PRIVATE smartbotic::db-client)
```

The submodule fallback still works for projects that haven't switched (`BUILD_SHARED_LIBS=OFF`).

### Consumers

| Project | Depends on | Min version | Migrations dir |
|---------|-----------|-------------|----------------|
| shadowman-cpp | `smartbotic-database (>= 1.2.0)` | 1.2.0 | `/opt/shadowman/share/shadowman/migrations/json` |
| callerai | `smartbotic-database (>= 1.2.0)` | 1.2.0 | `/etc/callerai/migrations` |

### Version policy

Bump `VERSION` for API/protocol changes. Deb revision (`-N`) for packaging-only changes.

## Conventions

- C++20
- nlohmann/json for JSON handling
- spdlog for logging
- LZ4 for compression
- gRPC/Protobuf for API