|
@@ -0,0 +1,3154 @@
|
|
|
|
|
+# smartbotic-vectorapi Backend Implementation Plan
|
|
|
|
|
+
|
|
|
|
|
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
|
+
|
|
|
|
|
+**Goal:** Build the C++20 REST API backend that fronts `smartbotic-database` over gRPC — bearer/cookie auth, admin-managed collections, JSON CRUD, RAG vector store + similarity search, OpenAI embedding generation, DB-backed hot-reloaded settings, and self-describing `openapi.json`/`llms.txt`.
|
|
|
|
|
+
|
|
|
|
|
+**Architecture:** A single static core library (`vectorapi_core`) holds all logic; the `smartbotic-vectorapi` binary and the test executables link it. HTTP is cpp-httplib (sync, OpenSSL). The DB is reached through the installed `libsmartbotic-db-client` (`smartbotic::db-client`). Pure logic (config, settings parsing, filters, errors, embeddings parse, auth/session) is unit-tested with GoogleTest; DB-touching and HTTP code is covered by integration tests against the locally-installed `smartbotic-database` on `localhost:9004` (tests `GTEST_SKIP()` when it is unreachable). Runtime settings live in an encrypted `_vectorapi_settings` DB collection and are hot-reloaded via the client's `subscribe()` event stream with an atomic snapshot swap.
|
|
|
|
|
+
|
|
|
|
|
+**Tech Stack:** C++20, CMake (FetchContent for cpp-httplib v0.46.0), `libsmartbotic-db-client` 2.2.1, nlohmann/json, spdlog, OpenSSL, GoogleTest 1.17, optional libsystemd (`sd_notify`).
|
|
|
|
|
+
|
|
|
|
|
+**Scope note:** This is plan 1 of 3. Plan 2 = React web UI; plan 3 = Debian packaging (`build.sh`, `.deb`, systemd). The web UI dev server proxies `/api` to this backend, so this plan delivers a fully usable API on its own.
|
|
|
|
|
+
|
|
|
|
|
+**Conventions:**
|
|
|
|
|
+- Namespace `svapi`. Header guard `#pragma once`.
|
|
|
|
|
+- Handlers `throw svapi::ApiError(...)`; a global httplib exception handler formats the JSON error + status.
|
|
|
|
|
+- Tests live in `tests/`; each `test_<unit>.cpp` is registered with `gtest_discover_tests`.
|
|
|
|
|
+- Commit after every green task.
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## File structure
|
|
|
|
|
+
|
|
|
|
|
+```
|
|
|
|
|
+smartbotic-vectorapi/
|
|
|
|
|
+├── CMakeLists.txt # root: project, deps, options, install
|
|
|
|
|
+├── cmake/Dependencies.cmake # FetchContent cpp-httplib (OpenSSL on)
|
|
|
|
|
+├── cmake/version.hpp.in # configured -> generated/svapi/version.hpp
|
|
|
|
|
+├── src/
|
|
|
|
|
+│ ├── CMakeLists.txt # vectorapi_core lib + binary
|
|
|
|
|
+│ ├── main.cpp # wiring, signals, watchdog, sd_notify
|
|
|
|
|
+│ ├── config.{hpp,cpp} # bootstrap config (file + defaults)
|
|
|
|
|
+│ ├── errors.{hpp,cpp} # ApiError, ErrCode, httpStatus, errorBody
|
|
|
|
|
+│ ├── json_http.{hpp,cpp} # bodyJson(req), sendJson(res,...)
|
|
|
|
|
+│ ├── filters.{hpp,cpp} # parseFilter / opFromSlug -> client Filter
|
|
|
|
|
+│ ├── db_gateway.{hpp,cpp} # owns smartbotic::database::Client
|
|
|
|
|
+│ ├── settings.{hpp,cpp} # Settings struct + parse + key resolve
|
|
|
|
|
+│ ├── settings_store.{hpp,cpp} # DB-backed load/bootstrap/watch/snapshot
|
|
|
|
|
+│ ├── collection_registry.{hpp,cpp} # _vectorapi_collections metadata
|
|
|
|
|
+│ ├── embeddings.{hpp,cpp} # OpenAI /v1/embeddings client + parse
|
|
|
|
|
+│ ├── auth.{hpp,cpp} # SessionStore, bearer/cookie helpers
|
|
|
|
|
+│ ├── server.{hpp,cpp} # ApiServer, ServerDeps, routing, auth, CORS
|
|
|
|
|
+│ └── handlers/
|
|
|
|
|
+│ ├── meta.cpp # /healthz /readyz /openapi.json /llms.txt
|
|
|
|
|
+│ ├── collections.cpp # collection CRUD
|
|
|
|
|
+│ ├── documents.cpp # JSON doc CRUD + find
|
|
|
|
|
+│ ├── vectors.cpp # vector store + similarity search
|
|
|
|
|
+│ ├── stats.cpp # /api/v1/stats
|
|
|
|
|
+│ └── webui_auth.cpp # /ui/login /ui/logout
|
|
|
|
|
+├── api/{openapi.json,llms.txt} # served + shipped in deb
|
|
|
|
|
+├── config/config.json # minimal bootstrap conffile
|
|
|
|
|
+└── tests/
|
|
|
|
|
+ ├── CMakeLists.txt
|
|
|
|
|
+ ├── db_test_util.hpp # connect-or-skip fixture helper
|
|
|
|
|
+ ├── test_config.cpp
|
|
|
|
|
+ ├── test_errors.cpp
|
|
|
|
|
+ ├── test_filters.cpp
|
|
|
|
|
+ ├── test_settings.cpp
|
|
|
|
|
+ ├── test_embeddings.cpp
|
|
|
|
|
+ ├── test_auth.cpp
|
|
|
|
|
+ ├── test_registry.cpp # integration (DB)
|
|
|
|
|
+ ├── test_settings_store.cpp # integration (DB)
|
|
|
|
|
+ └── test_api_integration.cpp # integration (server + DB + mock OpenAI)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 1: Project skeleton, CMake, version, trivial test
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `CMakeLists.txt`, `cmake/Dependencies.cmake`, `cmake/version.hpp.in`
|
|
|
|
|
+- Create: `src/CMakeLists.txt`, `src/main.cpp`
|
|
|
|
|
+- Create: `config/config.json`
|
|
|
|
|
+- Create: `tests/CMakeLists.txt`, `tests/test_smoke.cpp`
|
|
|
|
|
+
|
|
|
|
|
+Prerequisite system packages (already present on this host, listed for clean machines):
|
|
|
|
|
+`cmake ninja-build g++ libsmartbotic-db-client-dev nlohmann-json3-dev libspdlog-dev libssl-dev libgtest-dev libsystemd-dev pkg-config`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Root `CMakeLists.txt`**
|
|
|
|
|
+
|
|
|
|
|
+```cmake
|
|
|
|
|
+cmake_minimum_required(VERSION 3.20)
|
|
|
|
|
+file(READ "${CMAKE_CURRENT_SOURCE_DIR}/VERSION" SVAPI_VERSION)
|
|
|
|
|
+string(STRIP "${SVAPI_VERSION}" SVAPI_VERSION)
|
|
|
|
|
+project(smartbotic-vectorapi VERSION ${SVAPI_VERSION} LANGUAGES CXX)
|
|
|
|
|
+
|
|
|
|
|
+set(CMAKE_CXX_STANDARD 20)
|
|
|
|
|
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
|
|
|
|
|
+set(CMAKE_CXX_EXTENSIONS OFF)
|
|
|
|
|
+set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
|
|
|
|
|
+if(NOT CMAKE_BUILD_TYPE)
|
|
|
|
|
+ set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Build type" FORCE)
|
|
|
|
|
+endif()
|
|
|
|
|
+
|
|
|
|
|
+option(BUILD_TESTS "Build tests" ON)
|
|
|
|
|
+option(BUILD_WEBUI "Build the React web UI via npm" ON)
|
|
|
|
|
+
|
|
|
|
|
+include(GNUInstallDirs)
|
|
|
|
|
+list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
|
|
|
|
|
+include(Dependencies)
|
|
|
|
|
+
|
|
|
|
|
+find_package(smartbotic-db-client REQUIRED)
|
|
|
|
|
+find_package(nlohmann_json REQUIRED)
|
|
|
|
|
+find_package(spdlog REQUIRED)
|
|
|
|
|
+find_package(OpenSSL REQUIRED)
|
|
|
|
|
+
|
|
|
|
|
+# Optional systemd (sd_notify)
|
|
|
|
|
+find_package(PkgConfig)
|
|
|
|
|
+if(PkgConfig_FOUND)
|
|
|
|
|
+ pkg_check_modules(SYSTEMD IMPORTED_TARGET libsystemd)
|
|
|
|
|
+endif()
|
|
|
|
|
+
|
|
|
|
|
+# git commit for version banner
|
|
|
|
|
+execute_process(COMMAND git rev-parse --short HEAD
|
|
|
|
|
+ WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
|
|
|
|
|
+ OUTPUT_VARIABLE SVAPI_GIT_COMMIT OUTPUT_STRIP_TRAILING_WHITESPACE
|
|
|
|
|
+ ERROR_QUIET)
|
|
|
|
|
+if(NOT SVAPI_GIT_COMMIT)
|
|
|
|
|
+ set(SVAPI_GIT_COMMIT "unknown")
|
|
|
|
|
+endif()
|
|
|
|
|
+configure_file(${CMAKE_SOURCE_DIR}/cmake/version.hpp.in
|
|
|
|
|
+ ${CMAKE_BINARY_DIR}/generated/svapi/version.hpp @ONLY)
|
|
|
|
|
+
|
|
|
|
|
+add_subdirectory(src)
|
|
|
|
|
+
|
|
|
|
|
+if(BUILD_TESTS)
|
|
|
|
|
+ enable_testing()
|
|
|
|
|
+ add_subdirectory(tests)
|
|
|
|
|
+endif()
|
|
|
|
|
+
|
|
|
|
|
+if(BUILD_WEBUI)
|
|
|
|
|
+ include(WebUI OPTIONAL) # provided by plan 2; absent is fine
|
|
|
|
|
+endif()
|
|
|
|
|
+
|
|
|
|
|
+# Install: config conffile, api docs
|
|
|
|
|
+install(FILES ${CMAKE_SOURCE_DIR}/config/config.json
|
|
|
|
|
+ DESTINATION ${CMAKE_INSTALL_SYSCONFDIR}/smartbotic-vectorapi)
|
|
|
|
|
+install(FILES ${CMAKE_SOURCE_DIR}/api/openapi.json
|
|
|
|
|
+ ${CMAKE_SOURCE_DIR}/api/llms.txt
|
|
|
|
|
+ DESTINATION ${CMAKE_INSTALL_DATADIR}/smartbotic-vectorapi)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: `cmake/Dependencies.cmake`**
|
|
|
|
|
+
|
|
|
|
|
+```cmake
|
|
|
|
|
+include(FetchContent)
|
|
|
|
|
+set(HTTPLIB_REQUIRE_OPENSSL ON CACHE BOOL "" FORCE)
|
|
|
|
|
+set(HTTPLIB_USE_OPENSSL_IF_AVAILABLE ON CACHE BOOL "" FORCE)
|
|
|
|
|
+FetchContent_Declare(httplib
|
|
|
|
|
+ GIT_REPOSITORY https://github.com/yhirose/cpp-httplib.git
|
|
|
|
|
+ GIT_TAG v0.46.0)
|
|
|
|
|
+FetchContent_MakeAvailable(httplib)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: `cmake/version.hpp.in`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+inline constexpr const char* VERSION = "@SVAPI_VERSION@";
|
|
|
|
|
+inline constexpr const char* GIT_COMMIT = "@SVAPI_GIT_COMMIT@";
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: `src/CMakeLists.txt`** (core lib has no sources yet except a placeholder; we add files as tasks land)
|
|
|
|
|
+
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_library(vectorapi_core STATIC
|
|
|
|
|
+ config.cpp
|
|
|
|
|
+)
|
|
|
|
|
+target_include_directories(vectorapi_core PUBLIC
|
|
|
|
|
+ ${CMAKE_CURRENT_SOURCE_DIR}
|
|
|
|
|
+ ${CMAKE_BINARY_DIR}/generated)
|
|
|
|
|
+target_link_libraries(vectorapi_core PUBLIC
|
|
|
|
|
+ smartbotic::db-client
|
|
|
|
|
+ httplib::httplib
|
|
|
|
|
+ nlohmann_json::nlohmann_json
|
|
|
|
|
+ spdlog::spdlog
|
|
|
|
|
+ OpenSSL::SSL OpenSSL::Crypto)
|
|
|
|
|
+target_compile_definitions(vectorapi_core PUBLIC CPPHTTPLIB_OPENSSL_SUPPORT)
|
|
|
|
|
+
|
|
|
|
|
+add_executable(smartbotic-vectorapi main.cpp)
|
|
|
|
|
+target_link_libraries(smartbotic-vectorapi PRIVATE
|
|
|
|
|
+ vectorapi_core
|
|
|
|
|
+ $<$<BOOL:${SYSTEMD_FOUND}>:PkgConfig::SYSTEMD>)
|
|
|
|
|
+if(SYSTEMD_FOUND)
|
|
|
|
|
+ target_compile_definitions(smartbotic-vectorapi PRIVATE HAVE_SYSTEMD)
|
|
|
|
|
+endif()
|
|
|
|
|
+install(TARGETS smartbotic-vectorapi RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> Note: `config.cpp` is created in Task 2. For Task 1, create a 1-line stub `src/config.cpp` containing only `// placeholder` so the library compiles, OR start the lib with `main.cpp` only. Use the stub: create `src/config.cpp` with `// placeholder until Task 2`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: `src/main.cpp` (version stub)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <svapi/version.hpp>
|
|
|
|
|
+#include <cstring>
|
|
|
|
|
+#include <iostream>
|
|
|
|
|
+
|
|
|
|
|
+int main(int argc, char** argv) {
|
|
|
|
|
+ for (int i = 1; i < argc; ++i) {
|
|
|
|
|
+ if (std::strcmp(argv[i], "--version") == 0 || std::strcmp(argv[i], "-v") == 0) {
|
|
|
|
|
+ std::cout << "smartbotic-vectorapi " << svapi::VERSION
|
|
|
|
|
+ << " (" << svapi::GIT_COMMIT << ")\n";
|
|
|
|
|
+ return 0;
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ std::cout << "smartbotic-vectorapi " << svapi::VERSION << " — not yet wired (see plan)\n";
|
|
|
|
|
+ return 0;
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: `config/config.json`**
|
|
|
|
|
+
|
|
|
|
|
+```json
|
|
|
|
|
+{
|
|
|
|
|
+ "log_level": "info",
|
|
|
|
|
+ "http": { "bind_address": "0.0.0.0", "port": 8080 },
|
|
|
|
|
+ "database": { "address": "localhost:9004" }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7: `tests/CMakeLists.txt`**
|
|
|
|
|
+
|
|
|
|
|
+```cmake
|
|
|
|
|
+find_package(GTest REQUIRED)
|
|
|
|
|
+include(GoogleTest)
|
|
|
|
|
+
|
|
|
|
|
+add_executable(test_smoke test_smoke.cpp)
|
|
|
|
|
+target_link_libraries(test_smoke PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_smoke)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8: `tests/test_smoke.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include <svapi/version.hpp>
|
|
|
|
|
+
|
|
|
|
|
+TEST(Smoke, VersionIsNonEmpty) {
|
|
|
|
|
+ EXPECT_GT(std::string(svapi::VERSION).size(), 0u);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 9: Configure + build**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake -B build -G Ninja -DBUILD_WEBUI=OFF && cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: builds `smartbotic-vectorapi` and `test_smoke`, linking `smartbotic::db-client` cleanly.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 10: Run smoke test + version**
|
|
|
|
|
+
|
|
|
|
|
+Run: `ctest --test-dir build --output-on-failure && ./build/src/smartbotic-vectorapi --version`
|
|
|
|
|
+Expected: 1 test passes; version prints `smartbotic-vectorapi 0.1.0 (<sha>)`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 11: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add CMakeLists.txt cmake src config tests
|
|
|
|
|
+git commit -m "build: project skeleton, cpp-httplib v0.46.0, db-client link, smoke test"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 2: Config module (bootstrap)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/config.hpp`, replace placeholder `src/config.cpp`
|
|
|
|
|
+- Test: `tests/test_config.cpp`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_config.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "config.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using svapi::Config;
|
|
|
|
|
+
|
|
|
|
|
+TEST(Config, DefaultsWhenEmpty) {
|
|
|
|
|
+ Config c = Config::fromJson(nlohmann::json::object());
|
|
|
|
|
+ EXPECT_EQ(c.logLevel, "info");
|
|
|
|
|
+ EXPECT_EQ(c.httpBind, "0.0.0.0");
|
|
|
|
|
+ EXPECT_EQ(c.httpPort, 8080);
|
|
|
|
|
+ EXPECT_EQ(c.dbAddress, "localhost:9004");
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Config, ParsesProvidedValues) {
|
|
|
|
|
+ auto j = nlohmann::json::parse(R"({
|
|
|
|
|
+ "log_level":"debug",
|
|
|
|
|
+ "http":{"bind_address":"127.0.0.1","port":9090},
|
|
|
|
|
+ "database":{"address":"db:9004"}
|
|
|
|
|
+ })");
|
|
|
|
|
+ Config c = Config::fromJson(j);
|
|
|
|
|
+ EXPECT_EQ(c.logLevel, "debug");
|
|
|
|
|
+ EXPECT_EQ(c.httpBind, "127.0.0.1");
|
|
|
|
|
+ EXPECT_EQ(c.httpPort, 9090);
|
|
|
|
|
+ EXPECT_EQ(c.dbAddress, "db:9004");
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Config, IgnoresUnknownKeys) {
|
|
|
|
|
+ auto j = nlohmann::json::parse(R"({"nonsense":true,"http":{"port":1234}})");
|
|
|
|
|
+ Config c = Config::fromJson(j);
|
|
|
|
|
+ EXPECT_EQ(c.httpPort, 1234);
|
|
|
|
|
+ EXPECT_EQ(c.httpBind, "0.0.0.0");
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_config test_config.cpp)
|
|
|
|
|
+target_link_libraries(test_config PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_config)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL to compile — `config.hpp` has no `Config::fromJson`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/config.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <cstdint>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <nlohmann/json.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+struct Config {
|
|
|
|
|
+ std::string logLevel = "info";
|
|
|
|
|
+ std::string httpBind = "0.0.0.0";
|
|
|
|
|
+ uint16_t httpPort = 8080;
|
|
|
|
|
+ std::string dbAddress = "localhost:9004";
|
|
|
|
|
+ std::string webuiDir = "/usr/share/smartbotic-vectorapi/webui";
|
|
|
|
|
+
|
|
|
|
|
+ static Config fromJson(const nlohmann::json& j);
|
|
|
|
|
+ static Config load(const std::string& path); // reads file then fromJson
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/config.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "config.hpp"
|
|
|
|
|
+#include <fstream>
|
|
|
|
|
+#include <stdexcept>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+Config Config::fromJson(const nlohmann::json& j) {
|
|
|
|
|
+ Config c;
|
|
|
|
|
+ c.logLevel = j.value("log_level", c.logLevel);
|
|
|
|
|
+ if (j.contains("http") && j["http"].is_object()) {
|
|
|
|
|
+ const auto& h = j["http"];
|
|
|
|
|
+ c.httpBind = h.value("bind_address", c.httpBind);
|
|
|
|
|
+ c.httpPort = h.value("port", c.httpPort);
|
|
|
|
|
+ }
|
|
|
|
|
+ if (j.contains("database") && j["database"].is_object()) {
|
|
|
|
|
+ c.dbAddress = j["database"].value("address", c.dbAddress);
|
|
|
|
|
+ }
|
|
|
|
|
+ return c;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+Config Config::load(const std::string& path) {
|
|
|
|
|
+ std::ifstream f(path);
|
|
|
|
|
+ if (!f) throw std::runtime_error("cannot open config file: " + path);
|
|
|
|
|
+ nlohmann::json j;
|
|
|
|
|
+ f >> j;
|
|
|
|
|
+ return fromJson(j);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R Config --output-on-failure`
|
|
|
|
|
+Expected: 3 Config tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/config.hpp src/config.cpp tests/test_config.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(config): minimal bootstrap config with defaults and file load"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 3: Error model + JSON/HTTP helpers
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/errors.hpp`, `src/errors.cpp`, `src/json_http.hpp`, `src/json_http.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `errors.cpp json_http.cpp` to `vectorapi_core`)
|
|
|
|
|
+- Test: `tests/test_errors.cpp`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_errors.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+
|
|
|
|
|
+TEST(Errors, StatusMapping) {
|
|
|
|
|
+ EXPECT_EQ(httpStatus(ErrCode::BadRequest), 400);
|
|
|
|
|
+ EXPECT_EQ(httpStatus(ErrCode::Unauthorized), 401);
|
|
|
|
|
+ EXPECT_EQ(httpStatus(ErrCode::NotFound), 404);
|
|
|
|
|
+ EXPECT_EQ(httpStatus(ErrCode::Unprocessable), 422);
|
|
|
|
|
+ EXPECT_EQ(httpStatus(ErrCode::Unavailable), 503);
|
|
|
|
|
+ EXPECT_EQ(httpStatus(ErrCode::Internal), 500);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Errors, BodyShape) {
|
|
|
|
|
+ auto j = errorBody("not_found", "no such collection");
|
|
|
|
|
+ EXPECT_EQ(j["error"]["code"], "not_found");
|
|
|
|
|
+ EXPECT_EQ(j["error"]["message"], "no such collection");
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Errors, ApiErrorCarriesFields) {
|
|
|
|
|
+ ApiError e(ErrCode::NotFound, "not_found", "gone");
|
|
|
|
|
+ EXPECT_EQ(e.code, ErrCode::NotFound);
|
|
|
|
|
+ EXPECT_EQ(e.slug, "not_found");
|
|
|
|
|
+ EXPECT_STREQ(e.what(), "gone");
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_errors test_errors.cpp)
|
|
|
|
|
+target_link_libraries(test_errors PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_errors)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `errors.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/errors.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <stdexcept>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <nlohmann/json.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+enum class ErrCode { BadRequest, Unauthorized, NotFound, Unprocessable, Unavailable, Internal };
|
|
|
|
|
+
|
|
|
|
|
+int httpStatus(ErrCode code);
|
|
|
|
|
+nlohmann::json errorBody(const std::string& code, const std::string& message);
|
|
|
|
|
+
|
|
|
|
|
+struct ApiError : std::runtime_error {
|
|
|
|
|
+ ErrCode code;
|
|
|
|
|
+ std::string slug;
|
|
|
|
|
+ ApiError(ErrCode c, std::string slug_, const std::string& msg)
|
|
|
|
|
+ : std::runtime_error(msg), code(c), slug(std::move(slug_)) {}
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/errors.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+int httpStatus(ErrCode code) {
|
|
|
|
|
+ switch (code) {
|
|
|
|
|
+ case ErrCode::BadRequest: return 400;
|
|
|
|
|
+ case ErrCode::Unauthorized: return 401;
|
|
|
|
|
+ case ErrCode::NotFound: return 404;
|
|
|
|
|
+ case ErrCode::Unprocessable: return 422;
|
|
|
|
|
+ case ErrCode::Unavailable: return 503;
|
|
|
|
|
+ case ErrCode::Internal: return 500;
|
|
|
|
|
+ }
|
|
|
|
|
+ return 500;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+nlohmann::json errorBody(const std::string& code, const std::string& message) {
|
|
|
|
|
+ return nlohmann::json{{"error", {{"code", code}, {"message", message}}}};
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Write `src/json_http.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <httplib.h>
|
|
|
|
|
+#include <nlohmann/json.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+// Parse a request body as JSON; throws ApiError(BadRequest) on failure.
|
|
|
|
|
+nlohmann::json bodyJson(const httplib::Request& req);
|
|
|
|
|
+
|
|
|
|
|
+// Serialize and send a JSON response with the given status.
|
|
|
|
|
+void sendJson(httplib::Response& res, int status, const nlohmann::json& body);
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Write `src/json_http.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+nlohmann::json bodyJson(const httplib::Request& req) {
|
|
|
|
|
+ if (req.body.empty()) return nlohmann::json::object();
|
|
|
|
|
+ try {
|
|
|
|
|
+ return nlohmann::json::parse(req.body);
|
|
|
|
|
+ } catch (const std::exception& e) {
|
|
|
|
|
+ throw ApiError(ErrCode::BadRequest, "invalid_json",
|
|
|
|
|
+ std::string("request body is not valid JSON: ") + e.what());
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void sendJson(httplib::Response& res, int status, const nlohmann::json& body) {
|
|
|
|
|
+ res.status = status;
|
|
|
|
|
+ res.set_content(body.dump(), "application/json");
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7: Add sources to `src/CMakeLists.txt`**
|
|
|
|
|
+
|
|
|
|
|
+Change the `vectorapi_core` source list to:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_library(vectorapi_core STATIC
|
|
|
|
|
+ config.cpp
|
|
|
|
|
+ errors.cpp
|
|
|
|
|
+ json_http.cpp
|
|
|
|
|
+)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8: Build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R Errors --output-on-failure`
|
|
|
|
|
+Expected: 3 Errors tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 9: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/errors.hpp src/errors.cpp src/json_http.hpp src/json_http.cpp src/CMakeLists.txt tests/test_errors.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(errors): ApiError + status mapping + JSON request/response helpers"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 4: Filter parsing (`field:op:value` -> client Filter)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/filters.hpp`, `src/filters.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `filters.cpp`)
|
|
|
|
|
+- Test: `tests/test_filters.cpp`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_filters.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "filters.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using svapi::parseFilter;
|
|
|
|
|
+using FilterOp = smartbotic::database::Client::FilterOp;
|
|
|
|
|
+
|
|
|
|
|
+TEST(Filters, ParsesNumericGt) {
|
|
|
|
|
+ auto f = parseFilter("age:gt:18");
|
|
|
|
|
+ EXPECT_EQ(f.field, "age");
|
|
|
|
|
+ EXPECT_EQ(f.op, FilterOp::GT);
|
|
|
|
|
+ EXPECT_EQ(f.value, 18); // parsed as JSON number
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Filters, ParsesStringEq) {
|
|
|
|
|
+ auto f = parseFilter("status:eq:active");
|
|
|
|
|
+ EXPECT_EQ(f.field, "status");
|
|
|
|
|
+ EXPECT_EQ(f.op, FilterOp::EQ);
|
|
|
|
|
+ EXPECT_EQ(f.value, "active"); // non-JSON -> string
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Filters, ValueMayContainColons) {
|
|
|
|
|
+ auto f = parseFilter("url:eq:http://x:8080/y");
|
|
|
|
|
+ EXPECT_EQ(f.field, "url");
|
|
|
|
|
+ EXPECT_EQ(f.value, "http://x:8080/y");
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Filters, UnknownOpThrows) {
|
|
|
|
|
+ EXPECT_THROW(parseFilter("a:wat:1"), svapi::ApiError);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Filters, MalformedThrows) {
|
|
|
|
|
+ EXPECT_THROW(parseFilter("nope"), svapi::ApiError);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_filters test_filters.cpp)
|
|
|
|
|
+target_link_libraries(test_filters PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_filters)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `filters.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/filters.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <smartbotic/database/client.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+// Map an operator slug ("eq","ne","gt","gte","lt","lte","in","contains",
|
|
|
|
|
+// "exists","regex","search") to the client FilterOp. Throws ApiError(BadRequest).
|
|
|
|
|
+smartbotic::database::Client::FilterOp opFromSlug(const std::string& slug);
|
|
|
|
|
+
|
|
|
|
|
+// Parse "field:op:value". The value is everything after the second ':' and is
|
|
|
|
|
+// parsed as JSON when possible, else treated as a string. Throws
|
|
|
|
|
+// ApiError(BadRequest) on malformed input or unknown op.
|
|
|
|
|
+smartbotic::database::Client::Filter parseFilter(const std::string& spec);
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/filters.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "filters.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include <unordered_map>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+using Op = smartbotic::database::Client::FilterOp;
|
|
|
|
|
+
|
|
|
|
|
+Op opFromSlug(const std::string& slug) {
|
|
|
|
|
+ static const std::unordered_map<std::string, Op> m = {
|
|
|
|
|
+ {"eq", Op::EQ}, {"ne", Op::NE}, {"gt", Op::GT}, {"gte", Op::GTE},
|
|
|
|
|
+ {"lt", Op::LT}, {"lte", Op::LTE}, {"in", Op::IN}, {"contains", Op::CONTAINS},
|
|
|
|
|
+ {"exists", Op::EXISTS}, {"regex", Op::REGEX}, {"search", Op::SEARCH},
|
|
|
|
|
+ };
|
|
|
|
|
+ auto it = m.find(slug);
|
|
|
|
|
+ if (it == m.end())
|
|
|
|
|
+ throw ApiError(ErrCode::BadRequest, "bad_filter", "unknown filter operator: " + slug);
|
|
|
|
|
+ return it->second;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+smartbotic::database::Client::Filter parseFilter(const std::string& spec) {
|
|
|
|
|
+ auto p1 = spec.find(':');
|
|
|
|
|
+ if (p1 == std::string::npos)
|
|
|
|
|
+ throw ApiError(ErrCode::BadRequest, "bad_filter", "filter must be field:op:value: " + spec);
|
|
|
|
|
+ auto p2 = spec.find(':', p1 + 1);
|
|
|
|
|
+ if (p2 == std::string::npos)
|
|
|
|
|
+ throw ApiError(ErrCode::BadRequest, "bad_filter", "filter must be field:op:value: " + spec);
|
|
|
|
|
+
|
|
|
|
|
+ std::string field = spec.substr(0, p1);
|
|
|
|
|
+ std::string opSlug = spec.substr(p1 + 1, p2 - p1 - 1);
|
|
|
|
|
+ std::string rawValue = spec.substr(p2 + 1);
|
|
|
|
|
+
|
|
|
|
|
+ nlohmann::json value;
|
|
|
|
|
+ try {
|
|
|
|
|
+ value = nlohmann::json::parse(rawValue);
|
|
|
|
|
+ } catch (...) {
|
|
|
|
|
+ value = rawValue; // fall back to string
|
|
|
|
|
+ }
|
|
|
|
|
+ return {field, opFromSlug(opSlug), value};
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Add `filters.cpp` to `src/CMakeLists.txt`** (append to the `vectorapi_core` list), then build + run
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R Filters --output-on-failure`
|
|
|
|
|
+Expected: 5 Filters tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/filters.hpp src/filters.cpp src/CMakeLists.txt tests/test_filters.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(filters): parse field:op:value query filters into client Filter"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 5: DbGateway (owns the gRPC client)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/db_gateway.hpp`, `src/db_gateway.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `db_gateway.cpp`)
|
|
|
|
|
+- Create: `tests/db_test_util.hpp`
|
|
|
|
|
+- Test: `tests/test_db_gateway.cpp` (integration; skips without DB)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write `tests/db_test_util.hpp`** (shared connect-or-skip helper)
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include "db_gateway.hpp"
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include <cstdlib>
|
|
|
|
|
+#include <memory>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi::testutil {
|
|
|
|
|
+
|
|
|
|
|
+// Address of a running smartbotic-database; override with SVAPI_TEST_DB.
|
|
|
|
|
+inline std::string testDbAddress() {
|
|
|
|
|
+ const char* env = std::getenv("SVAPI_TEST_DB");
|
|
|
|
|
+ return env ? env : "localhost:9004";
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+// Returns a connected gateway, or nullptr if the DB is unreachable.
|
|
|
|
|
+inline std::unique_ptr<DbGateway> connectOrNull() {
|
|
|
|
|
+ auto g = std::make_unique<DbGateway>(testDbAddress());
|
|
|
|
|
+ if (!g->connect() || !g->healthy()) return nullptr;
|
|
|
|
|
+ return g;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+// Unique-ish collection name for test isolation (no Date/random available
|
|
|
|
|
+// concerns here — these run on a real clock).
|
|
|
|
|
+inline std::string tmpCollection(const std::string& base) {
|
|
|
|
|
+ return "_svapi_test_" + base;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi::testutil
|
|
|
|
|
+
|
|
|
|
|
+#define SVAPI_REQUIRE_DB(varname) \
|
|
|
|
|
+ auto varname = ::svapi::testutil::connectOrNull(); \
|
|
|
|
|
+ if (!varname) GTEST_SKIP() << "smartbotic-database not reachable at " \
|
|
|
|
|
+ << ::svapi::testutil::testDbAddress()
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Write the failing test (`tests/test_db_gateway.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "db_test_util.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+
|
|
|
|
|
+TEST(DbGateway, ConnectsAndReportsHealthy) {
|
|
|
|
|
+ SVAPI_REQUIRE_DB(db);
|
|
|
|
|
+ EXPECT_TRUE(db->healthy());
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(DbGateway, RoundTripsADocument) {
|
|
|
|
|
+ SVAPI_REQUIRE_DB(db);
|
|
|
|
|
+ const std::string coll = testutil::tmpCollection("gw");
|
|
|
|
|
+ db->client().createCollection(coll);
|
|
|
|
|
+ auto id = db->client().insert(coll, nlohmann::json{{"hello", "world"}});
|
|
|
|
|
+ ASSERT_FALSE(id.empty());
|
|
|
|
|
+ auto got = db->client().get(coll, id);
|
|
|
|
|
+ ASSERT_TRUE(got.has_value());
|
|
|
|
|
+ EXPECT_EQ((*got)["hello"], "world");
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_db_gateway test_db_gateway.cpp)
|
|
|
|
|
+target_link_libraries(test_db_gateway PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_db_gateway)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `db_gateway.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/db_gateway.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <cstdint>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <smartbotic/database/client.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+// Owns the single gRPC client to smartbotic-database. Thin wrapper: callers use
|
|
|
|
|
+// client() for DB operations; the wrapper centralizes construction, connection,
|
|
|
|
|
+// and a health probe.
|
|
|
|
|
+class DbGateway {
|
|
|
|
|
+public:
|
|
|
|
|
+ explicit DbGateway(std::string address, uint32_t timeoutMs = 5000);
|
|
|
|
|
+
|
|
|
|
|
+ bool connect(); // establishes the channel; safe to retry
|
|
|
|
|
+ bool healthy(); // HealthCheck RPC
|
|
|
|
|
+
|
|
|
|
|
+ smartbotic::database::Client& client() { return client_; }
|
|
|
|
|
+
|
|
|
|
|
+private:
|
|
|
|
|
+ smartbotic::database::Client client_;
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Write `src/db_gateway.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "db_gateway.hpp"
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+namespace {
|
|
|
|
|
+smartbotic::database::Client::Config makeConfig(std::string address, uint32_t timeoutMs) {
|
|
|
|
|
+ smartbotic::database::Client::Config cfg;
|
|
|
|
|
+ cfg.address = std::move(address);
|
|
|
|
|
+ cfg.timeoutMs = timeoutMs;
|
|
|
|
|
+ return cfg;
|
|
|
|
|
+}
|
|
|
|
|
+} // namespace
|
|
|
|
|
+
|
|
|
|
|
+DbGateway::DbGateway(std::string address, uint32_t timeoutMs)
|
|
|
|
|
+ : client_(makeConfig(std::move(address), timeoutMs)) {}
|
|
|
|
|
+
|
|
|
|
|
+bool DbGateway::connect() { return client_.connect(); }
|
|
|
|
|
+
|
|
|
|
|
+bool DbGateway::healthy() { return client_.healthCheck(); }
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Add `db_gateway.cpp` to `src/CMakeLists.txt`, build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R DbGateway --output-on-failure`
|
|
|
|
|
+Expected: with the local DB up, 2 tests PASS; otherwise SKIPPED (not failed).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/db_gateway.hpp src/db_gateway.cpp src/CMakeLists.txt tests/db_test_util.hpp tests/test_db_gateway.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(db): DbGateway wrapping smartbotic-database client + integration test"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 6: Settings struct + parse + API-key resolution
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/settings.hpp`, `src/settings.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `settings.cpp`)
|
|
|
|
|
+- Test: `tests/test_settings.cpp`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_settings.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "settings.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+
|
|
|
|
|
+TEST(Settings, DefaultsAndRoundTrip) {
|
|
|
|
|
+ Settings s = Settings::fromJson(nlohmann::json::object());
|
|
|
|
|
+ EXPECT_EQ(s.openaiApiBase, "https://api.openai.com");
|
|
|
|
|
+ EXPECT_EQ(s.defaultEmbeddingModel, "text-embedding-3-small");
|
|
|
|
|
+ EXPECT_EQ(s.sessionTtlMinutes, 720u);
|
|
|
|
|
+ EXPECT_TRUE(s.webuiEnabled);
|
|
|
|
|
+ ASSERT_EQ(s.corsOrigins.size(), 1u);
|
|
|
|
|
+ EXPECT_EQ(s.corsOrigins[0], "*");
|
|
|
|
|
+
|
|
|
|
|
+ Settings r = Settings::fromJson(s.toJson());
|
|
|
|
|
+ EXPECT_EQ(r.toJson(), s.toJson());
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Settings, ParsesValues) {
|
|
|
|
|
+ auto j = nlohmann::json::parse(R"({
|
|
|
|
|
+ "api_key":"abc","openai_api_base":"http://x","openai_api_key":"sk",
|
|
|
|
|
+ "default_embedding_model":"m","cors_origins":["a","b"],
|
|
|
|
|
+ "session_ttl_minutes":60,"webui_enabled":false
|
|
|
|
|
+ })");
|
|
|
|
|
+ Settings s = Settings::fromJson(j);
|
|
|
|
|
+ EXPECT_EQ(s.apiKey, "abc");
|
|
|
|
|
+ EXPECT_EQ(s.openaiApiKey, "sk");
|
|
|
|
|
+ EXPECT_EQ(s.corsOrigins.size(), 2u);
|
|
|
|
|
+ EXPECT_EQ(s.sessionTtlMinutes, 60u);
|
|
|
|
|
+ EXPECT_FALSE(s.webuiEnabled);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Settings, GenerateApiKeyIsHexAndUnique) {
|
|
|
|
|
+ std::string a = generateApiKey();
|
|
|
|
|
+ std::string b = generateApiKey();
|
|
|
|
|
+ EXPECT_GE(a.size(), 32u);
|
|
|
|
|
+ EXPECT_NE(a, b);
|
|
|
|
|
+ for (char c : a) EXPECT_TRUE(std::isxdigit(static_cast<unsigned char>(c)));
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Settings, ResolveKeyPrefersStored) {
|
|
|
|
|
+ auto r = resolveApiKey(/*stored*/ "stored", /*env*/ "envkey");
|
|
|
|
|
+ EXPECT_EQ(r.value, "stored");
|
|
|
|
|
+ EXPECT_FALSE(r.mustPersist);
|
|
|
|
|
+ EXPECT_FALSE(r.mustLog);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Settings, ResolveKeyUsesEnvWhenNoStored) {
|
|
|
|
|
+ auto r = resolveApiKey(/*stored*/ "", /*env*/ "envkey");
|
|
|
|
|
+ EXPECT_EQ(r.value, "envkey");
|
|
|
|
|
+ EXPECT_TRUE(r.mustPersist);
|
|
|
|
|
+ EXPECT_FALSE(r.mustLog);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Settings, ResolveKeyGeneratesWhenNothing) {
|
|
|
|
|
+ auto r = resolveApiKey(/*stored*/ "", /*env*/ "");
|
|
|
|
|
+ EXPECT_GE(r.value.size(), 32u);
|
|
|
|
|
+ EXPECT_TRUE(r.mustPersist);
|
|
|
|
|
+ EXPECT_TRUE(r.mustLog);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_settings test_settings.cpp)
|
|
|
|
|
+target_link_libraries(test_settings PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_settings)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `settings.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/settings.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <cstdint>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <vector>
|
|
|
|
|
+#include <nlohmann/json.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+struct Settings {
|
|
|
|
|
+ std::string apiKey;
|
|
|
|
|
+ std::string openaiApiBase = "https://api.openai.com";
|
|
|
|
|
+ std::string openaiApiKey;
|
|
|
|
|
+ std::string defaultEmbeddingModel = "text-embedding-3-small";
|
|
|
|
|
+ std::vector<std::string> corsOrigins = {"*"};
|
|
|
|
|
+ uint32_t sessionTtlMinutes = 720;
|
|
|
|
|
+ bool webuiEnabled = true;
|
|
|
|
|
+
|
|
|
|
|
+ static Settings fromJson(const nlohmann::json& j);
|
|
|
|
|
+ nlohmann::json toJson() const;
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+// 24 random bytes as 48 lowercase hex chars (OpenSSL RAND_bytes).
|
|
|
|
|
+std::string generateApiKey();
|
|
|
|
|
+
|
|
|
|
|
+struct ResolvedKey {
|
|
|
|
|
+ std::string value;
|
|
|
|
|
+ bool mustPersist = false; // write back to the DB
|
|
|
|
|
+ bool mustLog = false; // log once (only when freshly generated)
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+// stored wins; else env (persist, no log); else generate (persist + log once).
|
|
|
|
|
+ResolvedKey resolveApiKey(const std::string& stored, const std::string& env);
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/settings.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "settings.hpp"
|
|
|
|
|
+#include <openssl/rand.h>
|
|
|
|
|
+#include <array>
|
|
|
|
|
+#include <stdexcept>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+Settings Settings::fromJson(const nlohmann::json& j) {
|
|
|
|
|
+ Settings s;
|
|
|
|
|
+ s.apiKey = j.value("api_key", s.apiKey);
|
|
|
|
|
+ s.openaiApiBase = j.value("openai_api_base", s.openaiApiBase);
|
|
|
|
|
+ s.openaiApiKey = j.value("openai_api_key", s.openaiApiKey);
|
|
|
|
|
+ s.defaultEmbeddingModel = j.value("default_embedding_model", s.defaultEmbeddingModel);
|
|
|
|
|
+ if (j.contains("cors_origins") && j["cors_origins"].is_array())
|
|
|
|
|
+ s.corsOrigins = j["cors_origins"].get<std::vector<std::string>>();
|
|
|
|
|
+ s.sessionTtlMinutes = j.value("session_ttl_minutes", s.sessionTtlMinutes);
|
|
|
|
|
+ s.webuiEnabled = j.value("webui_enabled", s.webuiEnabled);
|
|
|
|
|
+ return s;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+nlohmann::json Settings::toJson() const {
|
|
|
|
|
+ return {
|
|
|
|
|
+ {"api_key", apiKey},
|
|
|
|
|
+ {"openai_api_base", openaiApiBase},
|
|
|
|
|
+ {"openai_api_key", openaiApiKey},
|
|
|
|
|
+ {"default_embedding_model", defaultEmbeddingModel},
|
|
|
|
|
+ {"cors_origins", corsOrigins},
|
|
|
|
|
+ {"session_ttl_minutes", sessionTtlMinutes},
|
|
|
|
|
+ {"webui_enabled", webuiEnabled},
|
|
|
|
|
+ };
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+std::string generateApiKey() {
|
|
|
|
|
+ std::array<unsigned char, 24> buf{};
|
|
|
|
|
+ if (RAND_bytes(buf.data(), static_cast<int>(buf.size())) != 1)
|
|
|
|
|
+ throw std::runtime_error("RAND_bytes failed");
|
|
|
|
|
+ static const char* hex = "0123456789abcdef";
|
|
|
|
|
+ std::string out;
|
|
|
|
|
+ out.reserve(buf.size() * 2);
|
|
|
|
|
+ for (unsigned char b : buf) { out.push_back(hex[b >> 4]); out.push_back(hex[b & 0xF]); }
|
|
|
|
|
+ return out;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+ResolvedKey resolveApiKey(const std::string& stored, const std::string& env) {
|
|
|
|
|
+ if (!stored.empty()) return {stored, false, false};
|
|
|
|
|
+ if (!env.empty()) return {env, true, false};
|
|
|
|
|
+ return {generateApiKey(), true, true};
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Add `settings.cpp` to `src/CMakeLists.txt`, build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R Settings --output-on-failure`
|
|
|
|
|
+Expected: all Settings tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/settings.hpp src/settings.cpp src/CMakeLists.txt tests/test_settings.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(settings): Settings struct, key generation, bootstrap key resolution"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 7: SettingsStore (DB-backed bootstrap, snapshot, hot reload)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/settings_store.hpp`, `src/settings_store.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `settings_store.cpp`)
|
|
|
|
|
+- Test: `tests/test_settings_store.cpp` (integration)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_settings_store.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "db_test_util.hpp"
|
|
|
|
|
+#include "settings_store.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+
|
|
|
|
|
+TEST(SettingsStore, BootstrapGeneratesAndPersistsKey) {
|
|
|
|
|
+ SVAPI_REQUIRE_DB(db);
|
|
|
|
|
+ const std::string coll = testutil::tmpCollection("settings_a");
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+
|
|
|
|
|
+ SettingsStore store(*db, coll, "current");
|
|
|
|
|
+ store.bootstrap(/*envApiKey*/ "", /*envOpenAiKey*/ "");
|
|
|
|
|
+ auto snap = store.snapshot();
|
|
|
|
|
+ ASSERT_TRUE(snap != nullptr);
|
|
|
|
|
+ EXPECT_GE(snap->apiKey.size(), 32u);
|
|
|
|
|
+
|
|
|
|
|
+ // A second store reading the same collection sees the persisted key.
|
|
|
|
|
+ SettingsStore store2(*db, coll, "current");
|
|
|
|
|
+ store2.bootstrap("", "");
|
|
|
|
|
+ EXPECT_EQ(store2.snapshot()->apiKey, snap->apiKey);
|
|
|
|
|
+
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(SettingsStore, SaveThenReloadReflectsChange) {
|
|
|
|
|
+ SVAPI_REQUIRE_DB(db);
|
|
|
|
|
+ const std::string coll = testutil::tmpCollection("settings_b");
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+
|
|
|
|
|
+ SettingsStore store(*db, coll, "current");
|
|
|
|
|
+ store.bootstrap("", "");
|
|
|
|
|
+ Settings s = *store.snapshot();
|
|
|
|
|
+ s.defaultEmbeddingModel = "text-embedding-3-large";
|
|
|
|
|
+ EXPECT_TRUE(store.save(s));
|
|
|
|
|
+ EXPECT_EQ(store.snapshot()->defaultEmbeddingModel, "text-embedding-3-large");
|
|
|
|
|
+
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(SettingsStore, EnvSeedsKeyWhenNoneStored) {
|
|
|
|
|
+ SVAPI_REQUIRE_DB(db);
|
|
|
|
|
+ const std::string coll = testutil::tmpCollection("settings_c");
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+
|
|
|
|
|
+ SettingsStore store(*db, coll, "current");
|
|
|
|
|
+ store.bootstrap(/*envApiKey*/ "seeded-key", /*envOpenAiKey*/ "sk-test");
|
|
|
|
|
+ EXPECT_EQ(store.snapshot()->apiKey, "seeded-key");
|
|
|
|
|
+ EXPECT_EQ(store.snapshot()->openaiApiKey, "sk-test");
|
|
|
|
|
+
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_settings_store test_settings_store.cpp)
|
|
|
|
|
+target_link_libraries(test_settings_store PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_settings_store)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `settings_store.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/settings_store.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include "db_gateway.hpp"
|
|
|
|
|
+#include "settings.hpp"
|
|
|
|
|
+#include <memory>
|
|
|
|
|
+#include <mutex>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+// Holds runtime settings in an (encrypted) DB collection and serves an
|
|
|
|
|
+// immutable, atomically-swapped snapshot. Hot-reloads on DB change events.
|
|
|
|
|
+class SettingsStore {
|
|
|
|
|
+public:
|
|
|
|
|
+ SettingsStore(DbGateway& db, std::string collection = "_vectorapi_settings",
|
|
|
|
|
+ std::string docId = "current");
|
|
|
|
|
+
|
|
|
|
|
+ // Ensure the collection exists, resolve the API key (stored > env > generated;
|
|
|
|
|
+ // logs once if generated), seed the OpenAI key from env if empty, persist if
|
|
|
|
|
+ // needed, then publish the first snapshot.
|
|
|
|
|
+ void bootstrap(const std::string& envApiKey, const std::string& envOpenAiKey);
|
|
|
|
|
+
|
|
|
|
|
+ std::shared_ptr<const Settings> snapshot() const; // never null after bootstrap
|
|
|
|
|
+ void reloadFromDb(); // re-read + swap snapshot
|
|
|
|
|
+ bool save(const Settings& s); // persist + reload
|
|
|
|
|
+ void startWatch(); // subscribe -> reload on change
|
|
|
|
|
+
|
|
|
|
|
+private:
|
|
|
|
|
+ void store(const Settings& s);
|
|
|
|
|
+ void setSnapshot(Settings s);
|
|
|
|
|
+
|
|
|
|
|
+ DbGateway& db_;
|
|
|
|
|
+ std::string coll_, docId_;
|
|
|
|
|
+ mutable std::mutex m_;
|
|
|
|
|
+ std::shared_ptr<const Settings> snap_;
|
|
|
|
|
+ std::shared_ptr<void> sub_;
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/settings_store.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "settings_store.hpp"
|
|
|
|
|
+#include <spdlog/spdlog.h>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+SettingsStore::SettingsStore(DbGateway& db, std::string collection, std::string docId)
|
|
|
|
|
+ : db_(db), coll_(std::move(collection)), docId_(std::move(docId)),
|
|
|
|
|
+ snap_(std::make_shared<const Settings>()) {}
|
|
|
|
|
+
|
|
|
|
|
+void SettingsStore::bootstrap(const std::string& envApiKey, const std::string& envOpenAiKey) {
|
|
|
|
|
+ if (!db_.client().getCollectionInfo(coll_).has_value())
|
|
|
|
|
+ db_.client().createCollection(coll_, 0, /*encrypted*/ true, 0, 0);
|
|
|
|
|
+
|
|
|
|
|
+ Settings s;
|
|
|
|
|
+ if (auto doc = db_.client().get(coll_, docId_)) s = Settings::fromJson(*doc);
|
|
|
|
|
+
|
|
|
|
|
+ ResolvedKey rk = resolveApiKey(s.apiKey, envApiKey);
|
|
|
|
|
+ s.apiKey = rk.value;
|
|
|
|
|
+ if (s.openaiApiKey.empty() && !envOpenAiKey.empty()) s.openaiApiKey = envOpenAiKey;
|
|
|
|
|
+
|
|
|
|
|
+ if (rk.mustPersist || !db_.client().exists(coll_, docId_)) store(s);
|
|
|
|
|
+ if (rk.mustLog)
|
|
|
|
|
+ spdlog::warn("Generated initial API key: {} — store it now; this is shown only once.", s.apiKey);
|
|
|
|
|
+
|
|
|
|
|
+ setSnapshot(s);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void SettingsStore::store(const Settings& s) {
|
|
|
|
|
+ db_.client().upsert(coll_, s.toJson(), docId_);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void SettingsStore::setSnapshot(Settings s) {
|
|
|
|
|
+ auto p = std::make_shared<const Settings>(std::move(s));
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(m_);
|
|
|
|
|
+ snap_ = p;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+std::shared_ptr<const Settings> SettingsStore::snapshot() const {
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(m_);
|
|
|
|
|
+ return snap_;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void SettingsStore::reloadFromDb() {
|
|
|
|
|
+ if (auto doc = db_.client().get(coll_, docId_)) setSnapshot(Settings::fromJson(*doc));
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+bool SettingsStore::save(const Settings& s) {
|
|
|
|
|
+ store(s);
|
|
|
|
|
+ reloadFromDb();
|
|
|
|
|
+ return true;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void SettingsStore::startWatch() {
|
|
|
|
|
+ sub_ = db_.client().subscribe(
|
|
|
|
|
+ {coll_},
|
|
|
|
|
+ [this](const std::string&, const std::string&, const std::string&,
|
|
|
|
|
+ const std::optional<nlohmann::json>&) {
|
|
|
|
|
+ try { reloadFromDb(); }
|
|
|
|
|
+ catch (const std::exception& e) { spdlog::warn("settings reload failed: {}", e.what()); }
|
|
|
|
|
+ });
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Add `settings_store.cpp` to `src/CMakeLists.txt`, build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R SettingsStore --output-on-failure`
|
|
|
|
|
+Expected: 3 tests PASS (or SKIP without DB).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/settings_store.hpp src/settings_store.cpp src/CMakeLists.txt tests/test_settings_store.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(settings): DB-backed SettingsStore with bootstrap, snapshot, hot reload"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 8: CollectionRegistry (per-collection metadata)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/collection_registry.hpp`, `src/collection_registry.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `collection_registry.cpp`)
|
|
|
|
|
+- Test: `tests/test_registry.cpp` (unit for meta + integration for ops)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_registry.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "db_test_util.hpp"
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+
|
|
|
|
|
+TEST(CollectionMeta, JsonRoundTrip) {
|
|
|
|
|
+ CollectionMeta m{"docs", "vector", 1536, "text-embedding-3-small", 123};
|
|
|
|
|
+ CollectionMeta r = CollectionMeta::fromJson(m.toJson());
|
|
|
|
|
+ EXPECT_EQ(r.name, "docs");
|
|
|
|
|
+ EXPECT_EQ(r.kind, "vector");
|
|
|
|
|
+ EXPECT_EQ(r.vectorDimension, 1536u);
|
|
|
|
|
+ EXPECT_EQ(r.embeddingModel, "text-embedding-3-small");
|
|
|
|
|
+ EXPECT_EQ(r.createdAt, 123u);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(CollectionRegistry, AddGetListRemove) {
|
|
|
|
|
+ SVAPI_REQUIRE_DB(db);
|
|
|
|
|
+ const std::string coll = testutil::tmpCollection("reg");
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+
|
|
|
|
|
+ CollectionRegistry reg(*db, coll);
|
|
|
|
|
+ reg.ensure();
|
|
|
|
|
+ EXPECT_TRUE(reg.add({"a", "json", 0, "", 0}));
|
|
|
|
|
+ EXPECT_TRUE(reg.add({"b", "vector", 8, "m", 0}));
|
|
|
|
|
+
|
|
|
|
|
+ auto a = reg.get("a");
|
|
|
|
|
+ ASSERT_TRUE(a.has_value());
|
|
|
|
|
+ EXPECT_EQ(a->kind, "json");
|
|
|
|
|
+
|
|
|
|
|
+ auto all = reg.list();
|
|
|
|
|
+ EXPECT_EQ(all.size(), 2u);
|
|
|
|
|
+
|
|
|
|
|
+ EXPECT_TRUE(reg.remove("a"));
|
|
|
|
|
+ EXPECT_FALSE(reg.get("a").has_value());
|
|
|
|
|
+
|
|
|
|
|
+ db->client().dropCollection(coll);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_registry test_registry.cpp)
|
|
|
|
|
+target_link_libraries(test_registry PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_registry)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `collection_registry.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/collection_registry.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include "db_gateway.hpp"
|
|
|
|
|
+#include <cstdint>
|
|
|
|
|
+#include <optional>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <vector>
|
|
|
|
|
+#include <nlohmann/json.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+struct CollectionMeta {
|
|
|
|
|
+ std::string name;
|
|
|
|
|
+ std::string kind; // "vector" | "json"
|
|
|
|
|
+ uint32_t vectorDimension = 0;
|
|
|
|
|
+ std::string embeddingModel; // empty => use settings default at embed time
|
|
|
|
|
+ uint64_t createdAt = 0;
|
|
|
|
|
+
|
|
|
|
|
+ static CollectionMeta fromJson(const nlohmann::json& j);
|
|
|
|
|
+ nlohmann::json toJson() const;
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+// Stores one document per managed collection in `_vectorapi_collections`
|
|
|
|
|
+// (document id == collection name).
|
|
|
|
|
+class CollectionRegistry {
|
|
|
|
|
+public:
|
|
|
|
|
+ CollectionRegistry(DbGateway& db, std::string collection = "_vectorapi_collections");
|
|
|
|
|
+ void ensure(); // create the registry collection
|
|
|
|
|
+ bool add(const CollectionMeta& meta); // upsert; stamps createdAt if 0
|
|
|
|
|
+ std::optional<CollectionMeta> get(const std::string& name);
|
|
|
|
|
+ std::vector<CollectionMeta> list();
|
|
|
|
|
+ bool remove(const std::string& name);
|
|
|
|
|
+
|
|
|
|
|
+private:
|
|
|
|
|
+ DbGateway& db_;
|
|
|
|
|
+ std::string coll_;
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/collection_registry.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+#include <ctime>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+CollectionMeta CollectionMeta::fromJson(const nlohmann::json& j) {
|
|
|
|
|
+ CollectionMeta m;
|
|
|
|
|
+ m.name = j.value("name", "");
|
|
|
|
|
+ m.kind = j.value("kind", "json");
|
|
|
|
|
+ m.vectorDimension = j.value("vector_dimension", 0u);
|
|
|
|
|
+ m.embeddingModel = j.value("embedding_model", "");
|
|
|
|
|
+ m.createdAt = j.value("created_at", 0ull);
|
|
|
|
|
+ return m;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+nlohmann::json CollectionMeta::toJson() const {
|
|
|
|
|
+ return {
|
|
|
|
|
+ {"name", name},
|
|
|
|
|
+ {"kind", kind},
|
|
|
|
|
+ {"vector_dimension", vectorDimension},
|
|
|
|
|
+ {"embedding_model", embeddingModel},
|
|
|
|
|
+ {"created_at", createdAt},
|
|
|
|
|
+ };
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+CollectionRegistry::CollectionRegistry(DbGateway& db, std::string collection)
|
|
|
|
|
+ : db_(db), coll_(std::move(collection)) {}
|
|
|
|
|
+
|
|
|
|
|
+void CollectionRegistry::ensure() {
|
|
|
|
|
+ if (!db_.client().getCollectionInfo(coll_).has_value())
|
|
|
|
|
+ db_.client().createCollection(coll_, 0, /*encrypted*/ false, 0, 0);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+bool CollectionRegistry::add(const CollectionMeta& meta) {
|
|
|
|
|
+ CollectionMeta m = meta;
|
|
|
|
|
+ if (m.createdAt == 0) m.createdAt = static_cast<uint64_t>(std::time(nullptr));
|
|
|
|
|
+ auto [id, isNew] = db_.client().upsert(coll_, m.toJson(), m.name);
|
|
|
|
|
+ return !id.empty();
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+std::optional<CollectionMeta> CollectionRegistry::get(const std::string& name) {
|
|
|
|
|
+ if (auto doc = db_.client().get(coll_, name)) return CollectionMeta::fromJson(*doc);
|
|
|
|
|
+ return std::nullopt;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+std::vector<CollectionMeta> CollectionRegistry::list() {
|
|
|
|
|
+ std::vector<CollectionMeta> out;
|
|
|
|
|
+ smartbotic::database::Client::QueryOptions opts;
|
|
|
|
|
+ opts.limit = 10000;
|
|
|
|
|
+ for (const auto& doc : db_.client().find(coll_, opts)) out.push_back(CollectionMeta::fromJson(doc));
|
|
|
|
|
+ return out;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+bool CollectionRegistry::remove(const std::string& name) {
|
|
|
|
|
+ return db_.client().remove(coll_, name);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Add `collection_registry.cpp` to `src/CMakeLists.txt`, build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "CollectionMeta|CollectionRegistry" --output-on-failure`
|
|
|
|
|
+Expected: meta test PASS; registry test PASS (or SKIP without DB).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/collection_registry.hpp src/collection_registry.cpp src/CMakeLists.txt tests/test_registry.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(registry): per-collection metadata store (_vectorapi_collections)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 9: Embeddings (OpenAI client + response parse)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/embeddings.hpp`, `src/embeddings.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `embeddings.cpp`)
|
|
|
|
|
+- Test: `tests/test_embeddings.cpp`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_embeddings.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "embeddings.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include <httplib.h>
|
|
|
|
|
+#include <thread>
|
|
|
|
|
+
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+
|
|
|
|
|
+TEST(Embeddings, ParsesWellFormedResponse) {
|
|
|
|
|
+ auto body = nlohmann::json::parse(R"({"data":[{"embedding":[0.5,1.0,1.5]}]})");
|
|
|
|
|
+ auto v = parseEmbeddingResponse(body);
|
|
|
|
|
+ ASSERT_EQ(v.size(), 3u);
|
|
|
|
|
+ EXPECT_FLOAT_EQ(v[0], 0.5f);
|
|
|
|
|
+ EXPECT_FLOAT_EQ(v[2], 1.5f);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Embeddings, MalformedThrows) {
|
|
|
|
|
+ EXPECT_THROW(parseEmbeddingResponse(nlohmann::json::parse(R"({"data":[]})")), ApiError);
|
|
|
|
|
+ EXPECT_THROW(parseEmbeddingResponse(nlohmann::json::object()), ApiError);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Embeddings, ClientHitsMockServer) {
|
|
|
|
|
+ httplib::Server mock;
|
|
|
|
|
+ mock.Post("/v1/embeddings", [](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ auto j = nlohmann::json::parse(req.body);
|
|
|
|
|
+ EXPECT_EQ(j["model"], "m");
|
|
|
|
|
+ EXPECT_EQ(j["input"], "hello");
|
|
|
|
|
+ res.set_content(R"({"data":[{"embedding":[1,2,3,4]}]})", "application/json");
|
|
|
|
|
+ });
|
|
|
|
|
+ int port = mock.bind_to_any_port("127.0.0.1");
|
|
|
|
|
+ std::thread t([&] { mock.listen_after_bind(); });
|
|
|
|
|
+
|
|
|
|
|
+ EmbeddingClient cli("http://127.0.0.1:" + std::to_string(port), "test-key");
|
|
|
|
|
+ auto v = cli.embed("m", "hello");
|
|
|
|
|
+ EXPECT_EQ(v.size(), 4u);
|
|
|
|
|
+ EXPECT_FLOAT_EQ(v[3], 4.0f);
|
|
|
|
|
+
|
|
|
|
|
+ mock.stop();
|
|
|
|
|
+ t.join();
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_embeddings test_embeddings.cpp)
|
|
|
|
|
+target_link_libraries(test_embeddings PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_embeddings)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `embeddings.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/embeddings.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <vector>
|
|
|
|
|
+#include <nlohmann/json.hpp>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+// Extract data[0].embedding as floats. Throws ApiError(Unprocessable) if absent.
|
|
|
|
|
+std::vector<float> parseEmbeddingResponse(const nlohmann::json& body);
|
|
|
|
|
+
|
|
|
|
|
+// Minimal OpenAI-style embeddings client (POST {apiBase}/v1/embeddings).
|
|
|
|
|
+class EmbeddingClient {
|
|
|
|
|
+public:
|
|
|
|
|
+ EmbeddingClient(std::string apiBase, std::string apiKey);
|
|
|
|
|
+ std::vector<float> embed(const std::string& model, const std::string& text);
|
|
|
|
|
+
|
|
|
|
|
+private:
|
|
|
|
|
+ std::string apiBase_;
|
|
|
|
|
+ std::string apiKey_;
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/embeddings.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "embeddings.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include <httplib.h>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+std::vector<float> parseEmbeddingResponse(const nlohmann::json& body) {
|
|
|
|
|
+ if (!body.contains("data") || !body["data"].is_array() || body["data"].empty())
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "embedding_failed", "embedding response missing data[]");
|
|
|
|
|
+ const auto& emb = body["data"][0].value("embedding", nlohmann::json());
|
|
|
|
|
+ if (!emb.is_array() || emb.empty())
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "embedding_failed", "embedding response missing embedding[]");
|
|
|
|
|
+ std::vector<float> out;
|
|
|
|
|
+ out.reserve(emb.size());
|
|
|
|
|
+ for (const auto& v : emb) out.push_back(v.get<float>());
|
|
|
|
|
+ return out;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+EmbeddingClient::EmbeddingClient(std::string apiBase, std::string apiKey)
|
|
|
|
|
+ : apiBase_(std::move(apiBase)), apiKey_(std::move(apiKey)) {}
|
|
|
|
|
+
|
|
|
|
|
+std::vector<float> EmbeddingClient::embed(const std::string& model, const std::string& text) {
|
|
|
|
|
+ httplib::Client cli(apiBase_);
|
|
|
|
|
+ cli.set_connection_timeout(10);
|
|
|
|
|
+ cli.set_read_timeout(30);
|
|
|
|
|
+ cli.enable_server_certificate_verification(true);
|
|
|
|
|
+ if (!apiKey_.empty()) cli.set_bearer_token_auth(apiKey_);
|
|
|
|
|
+
|
|
|
|
|
+ nlohmann::json req{{"model", model}, {"input", text}};
|
|
|
|
|
+ auto res = cli.Post("/v1/embeddings", req.dump(), "application/json");
|
|
|
|
|
+ if (!res)
|
|
|
|
|
+ throw ApiError(ErrCode::Unavailable, "embedding_unreachable", "could not reach embedding endpoint");
|
|
|
|
|
+ if (res->status != 200)
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "embedding_failed",
|
|
|
|
|
+ "embedding endpoint returned HTTP " + std::to_string(res->status));
|
|
|
|
|
+ return parseEmbeddingResponse(nlohmann::json::parse(res->body));
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Add `embeddings.cpp` to `src/CMakeLists.txt`, build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R Embeddings --output-on-failure`
|
|
|
|
|
+Expected: 3 Embeddings tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/embeddings.hpp src/embeddings.cpp src/CMakeLists.txt tests/test_embeddings.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(embeddings): OpenAI /v1/embeddings client + response parsing"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 10: Auth (SessionStore + bearer/cookie helpers)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/auth.hpp`, `src/auth.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `auth.cpp`)
|
|
|
|
|
+- Test: `tests/test_auth.cpp`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write the failing test (`tests/test_auth.cpp`)**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "auth.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+using clock = std::chrono::system_clock;
|
|
|
|
|
+
|
|
|
|
|
+TEST(Auth, ParsesBearer) {
|
|
|
|
|
+ EXPECT_EQ(bearerToken("Bearer abc123").value(), "abc123");
|
|
|
|
|
+ EXPECT_FALSE(bearerToken("Basic xyz").has_value());
|
|
|
|
|
+ EXPECT_FALSE(bearerToken("").has_value());
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Auth, ParsesCookie) {
|
|
|
|
|
+ EXPECT_EQ(cookieValue("a=1; svapi_session=tok; b=2", "svapi_session").value(), "tok");
|
|
|
|
|
+ EXPECT_EQ(cookieValue("svapi_session=only", "svapi_session").value(), "only");
|
|
|
|
|
+ EXPECT_FALSE(cookieValue("a=1; b=2", "svapi_session").has_value());
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Auth, SessionLifecycle) {
|
|
|
|
|
+ SessionStore store(std::chrono::minutes(10));
|
|
|
|
|
+ auto now = clock::now();
|
|
|
|
|
+ std::string tok = store.create(now);
|
|
|
|
|
+ EXPECT_TRUE(store.valid(tok, now));
|
|
|
|
|
+ EXPECT_FALSE(store.valid("nonexistent", now));
|
|
|
|
|
+ EXPECT_FALSE(store.valid(tok, now + std::chrono::minutes(11))); // expired
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST(Auth, RemoveInvalidates) {
|
|
|
|
|
+ SessionStore store(std::chrono::minutes(10));
|
|
|
|
|
+ auto now = clock::now();
|
|
|
|
|
+ std::string tok = store.create(now);
|
|
|
|
|
+ store.remove(tok);
|
|
|
|
|
+ EXPECT_FALSE(store.valid(tok, now));
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+Add to `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_auth test_auth.cpp)
|
|
|
|
|
+target_link_libraries(test_auth PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+gtest_discover_tests(test_auth)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: FAIL — `auth.hpp` missing.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/auth.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <chrono>
|
|
|
|
|
+#include <mutex>
|
|
|
|
|
+#include <optional>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <unordered_map>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+// "Bearer <tok>" -> tok (case-sensitive scheme). nullopt otherwise.
|
|
|
|
|
+std::optional<std::string> bearerToken(const std::string& authHeader);
|
|
|
|
|
+
|
|
|
|
|
+// Value of cookie `name` from a Cookie header ("a=1; name=val; b=2").
|
|
|
|
|
+std::optional<std::string> cookieValue(const std::string& cookieHeader, const std::string& name);
|
|
|
|
|
+
|
|
|
|
|
+// 16 random bytes as 32 lowercase hex chars.
|
|
|
|
|
+std::string randomToken();
|
|
|
|
|
+
|
|
|
|
|
+// In-memory session set with TTL. Restart drops all sessions (acceptable).
|
|
|
|
|
+class SessionStore {
|
|
|
|
|
+public:
|
|
|
|
|
+ explicit SessionStore(std::chrono::minutes ttl);
|
|
|
|
|
+ std::string create(std::chrono::system_clock::time_point now);
|
|
|
|
|
+ bool valid(const std::string& token, std::chrono::system_clock::time_point now);
|
|
|
|
|
+ void remove(const std::string& token);
|
|
|
|
|
+ void setTtl(std::chrono::minutes ttl);
|
|
|
|
|
+
|
|
|
|
|
+private:
|
|
|
|
|
+ std::mutex m_;
|
|
|
|
|
+ std::chrono::minutes ttl_;
|
|
|
|
|
+ std::unordered_map<std::string, std::chrono::system_clock::time_point> sessions_; // token -> expiry
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/auth.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "auth.hpp"
|
|
|
|
|
+#include <openssl/rand.h>
|
|
|
|
|
+#include <array>
|
|
|
|
|
+#include <stdexcept>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+std::optional<std::string> bearerToken(const std::string& authHeader) {
|
|
|
|
|
+ constexpr const char* kPrefix = "Bearer ";
|
|
|
|
|
+ constexpr size_t kLen = 7;
|
|
|
|
|
+ if (authHeader.size() <= kLen || authHeader.compare(0, kLen, kPrefix) != 0)
|
|
|
|
|
+ return std::nullopt;
|
|
|
|
|
+ return authHeader.substr(kLen);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+std::optional<std::string> cookieValue(const std::string& cookieHeader, const std::string& name) {
|
|
|
|
|
+ size_t pos = 0;
|
|
|
|
|
+ while (pos < cookieHeader.size()) {
|
|
|
|
|
+ size_t semi = cookieHeader.find(';', pos);
|
|
|
|
|
+ std::string pair = cookieHeader.substr(pos, semi == std::string::npos ? std::string::npos : semi - pos);
|
|
|
|
|
+ size_t start = pair.find_first_not_of(" \t");
|
|
|
|
|
+ if (start != std::string::npos) pair = pair.substr(start);
|
|
|
|
|
+ size_t eq = pair.find('=');
|
|
|
|
|
+ if (eq != std::string::npos && pair.substr(0, eq) == name)
|
|
|
|
|
+ return pair.substr(eq + 1);
|
|
|
|
|
+ if (semi == std::string::npos) break;
|
|
|
|
|
+ pos = semi + 1;
|
|
|
|
|
+ }
|
|
|
|
|
+ return std::nullopt;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+std::string randomToken() {
|
|
|
|
|
+ std::array<unsigned char, 16> buf{};
|
|
|
|
|
+ if (RAND_bytes(buf.data(), static_cast<int>(buf.size())) != 1)
|
|
|
|
|
+ throw std::runtime_error("RAND_bytes failed");
|
|
|
|
|
+ static const char* hex = "0123456789abcdef";
|
|
|
|
|
+ std::string out;
|
|
|
|
|
+ out.reserve(32);
|
|
|
|
|
+ for (unsigned char b : buf) { out.push_back(hex[b >> 4]); out.push_back(hex[b & 0xF]); }
|
|
|
|
|
+ return out;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+SessionStore::SessionStore(std::chrono::minutes ttl) : ttl_(ttl) {}
|
|
|
|
|
+
|
|
|
|
|
+std::string SessionStore::create(std::chrono::system_clock::time_point now) {
|
|
|
|
|
+ std::string tok = randomToken();
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(m_);
|
|
|
|
|
+ sessions_[tok] = now + ttl_;
|
|
|
|
|
+ return tok;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+bool SessionStore::valid(const std::string& token, std::chrono::system_clock::time_point now) {
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(m_);
|
|
|
|
|
+ auto it = sessions_.find(token);
|
|
|
|
|
+ if (it == sessions_.end()) return false;
|
|
|
|
|
+ if (now >= it->second) { sessions_.erase(it); return false; }
|
|
|
|
|
+ return true;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void SessionStore::remove(const std::string& token) {
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(m_);
|
|
|
|
|
+ sessions_.erase(token);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void SessionStore::setTtl(std::chrono::minutes ttl) {
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(m_);
|
|
|
|
|
+ ttl_ = ttl;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Add `auth.cpp` to `src/CMakeLists.txt`, build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R Auth --output-on-failure`
|
|
|
|
|
+Expected: 4 Auth tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/auth.hpp src/auth.cpp src/CMakeLists.txt tests/test_auth.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(auth): SessionStore + bearer/cookie parsing"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 11: ApiServer scaffolding + meta routes + auth middleware + CORS
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `src/server.hpp`, `src/server.cpp`, `src/handlers/meta.cpp`
|
|
|
|
|
+- Modify: `src/CMakeLists.txt` (add `server.cpp handlers/meta.cpp`)
|
|
|
|
|
+- Create: `api/openapi.json` (minimal stub now; fully authored in Task 17), `api/llms.txt` (stub now)
|
|
|
|
|
+- Create: `tests/api_fixture.hpp`, `tests/test_api_integration.cpp`
|
|
|
|
|
+
|
|
|
|
|
+Handlers are free functions `registerXxxRoutes(ApiServer&)` so each lives in its
|
|
|
|
|
+own translation unit. Auth is enforced once in a pre-routing handler for `/api/*`;
|
|
|
|
|
+handlers throw `ApiError` and a global exception handler formats it.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Create minimal `api/openapi.json` + `api/llms.txt` stubs**
|
|
|
|
|
+
|
|
|
|
|
+`api/openapi.json`:
|
|
|
|
|
+```json
|
|
|
|
|
+{ "openapi": "3.1.0", "info": { "title": "smartbotic-vectorapi", "version": "0.1.0" }, "paths": {} }
|
|
|
|
|
+```
|
|
|
|
|
+`api/llms.txt`:
|
|
|
|
|
+```text
|
|
|
|
|
+# smartbotic-vectorapi
|
|
|
|
|
+
|
|
|
|
|
+> REST API fronting smartbotic-database for RAG vectors and JSON documents.
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Write `src/server.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include "auth.hpp"
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+#include "db_gateway.hpp"
|
|
|
|
|
+#include "settings_store.hpp"
|
|
|
|
|
+#include <cstdint>
|
|
|
|
|
+#include <httplib.h>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+struct ServerDeps {
|
|
|
|
|
+ DbGateway& db;
|
|
|
|
|
+ SettingsStore& settings;
|
|
|
|
|
+ CollectionRegistry& registry;
|
|
|
|
|
+ SessionStore& sessions;
|
|
|
|
|
+ std::string webuiDir; // static SPA root ("" disables static serving)
|
|
|
|
|
+ std::string shareDir; // directory holding openapi.json + llms.txt
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+class ApiServer {
|
|
|
|
|
+public:
|
|
|
|
|
+ explicit ApiServer(ServerDeps deps);
|
|
|
|
|
+ void registerRoutes(); // call once before binding
|
|
|
|
|
+
|
|
|
|
|
+ int bindToAnyPort(const std::string& host); // returns chosen port (tests)
|
|
|
|
|
+ void listenAfterBind(); // blocking
|
|
|
|
|
+ bool listen(const std::string& host, uint16_t port); // blocking
|
|
|
|
|
+ void stop();
|
|
|
|
|
+
|
|
|
|
|
+ httplib::Server& raw() { return svr_; }
|
|
|
|
|
+ ServerDeps& deps() { return d_; }
|
|
|
|
|
+
|
|
|
|
|
+private:
|
|
|
|
|
+ ServerDeps d_;
|
|
|
|
|
+ httplib::Server svr_;
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+// Auth predicate: bearer == settings.apiKey OR a valid session cookie.
|
|
|
|
|
+bool authorize(const httplib::Request& req, const Settings& s, SessionStore& sessions);
|
|
|
|
|
+
|
|
|
|
|
+// Route registration (each defined in handlers/*.cpp).
|
|
|
|
|
+void registerMetaRoutes(ApiServer&);
|
|
|
|
|
+void registerWebuiAuthRoutes(ApiServer&);
|
|
|
|
|
+void registerCollectionRoutes(ApiServer&);
|
|
|
|
|
+void registerDocumentRoutes(ApiServer&);
|
|
|
|
|
+void registerVectorRoutes(ApiServer&);
|
|
|
|
|
+void registerStatsRoutes(ApiServer&);
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write `src/server.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include <chrono>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+ApiServer::ApiServer(ServerDeps deps) : d_(std::move(deps)) {}
|
|
|
|
|
+
|
|
|
|
|
+bool authorize(const httplib::Request& req, const Settings& s, SessionStore& sessions) {
|
|
|
|
|
+ if (auto it = req.headers.find("Authorization"); it != req.headers.end()) {
|
|
|
|
|
+ if (auto tok = bearerToken(it->second); tok && !s.apiKey.empty() && *tok == s.apiKey)
|
|
|
|
|
+ return true;
|
|
|
|
|
+ }
|
|
|
|
|
+ if (auto it = req.headers.find("Cookie"); it != req.headers.end()) {
|
|
|
|
|
+ if (auto c = cookieValue(it->second, "svapi_session");
|
|
|
|
|
+ c && sessions.valid(*c, std::chrono::system_clock::now()))
|
|
|
|
|
+ return true;
|
|
|
|
|
+ }
|
|
|
|
|
+ return false;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+void ApiServer::registerRoutes() {
|
|
|
|
|
+ svr_.set_exception_handler([](const httplib::Request&, httplib::Response& res, std::exception_ptr ep) {
|
|
|
|
|
+ try { std::rethrow_exception(ep); }
|
|
|
|
|
+ catch (const ApiError& e) { sendJson(res, httpStatus(e.code), errorBody(e.slug, e.what())); }
|
|
|
|
|
+ catch (const std::exception& e) { sendJson(res, 500, errorBody("internal", e.what())); }
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr_.set_pre_routing_handler([this](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ if (req.method != "OPTIONS" && req.path.rfind("/api/", 0) == 0) {
|
|
|
|
|
+ auto snap = d_.settings.snapshot();
|
|
|
|
|
+ if (!authorize(req, *snap, d_.sessions)) {
|
|
|
|
|
+ sendJson(res, 401, errorBody("unauthorized", "missing or invalid credentials"));
|
|
|
|
|
+ return httplib::Server::HandlerResponse::Handled;
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ return httplib::Server::HandlerResponse::Unhandled;
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr_.set_post_routing_handler([this](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ auto snap = d_.settings.snapshot();
|
|
|
|
|
+ const std::string origin = snap->corsOrigins.empty() ? "*" : snap->corsOrigins.front();
|
|
|
|
|
+ res.set_header("Access-Control-Allow-Origin", origin);
|
|
|
|
|
+ res.set_header("Access-Control-Allow-Headers", "Authorization, Content-Type");
|
|
|
|
|
+ res.set_header("Access-Control-Allow-Methods", "GET, POST, PUT, PATCH, DELETE, OPTIONS");
|
|
|
|
|
+ });
|
|
|
|
|
+ svr_.Options(R"(.*)", [](const httplib::Request&, httplib::Response& res) { res.status = 204; });
|
|
|
|
|
+
|
|
|
|
|
+ registerMetaRoutes(*this);
|
|
|
|
|
+ registerWebuiAuthRoutes(*this);
|
|
|
|
|
+ registerCollectionRoutes(*this);
|
|
|
|
|
+ registerDocumentRoutes(*this);
|
|
|
|
|
+ registerVectorRoutes(*this);
|
|
|
|
|
+ registerStatsRoutes(*this);
|
|
|
|
|
+
|
|
|
|
|
+ if (!d_.webuiDir.empty()) svr_.set_mount_point("/", d_.webuiDir);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+int ApiServer::bindToAnyPort(const std::string& host) { return svr_.bind_to_any_port(host); }
|
|
|
|
|
+void ApiServer::listenAfterBind() { svr_.listen_after_bind(); }
|
|
|
|
|
+bool ApiServer::listen(const std::string& host, uint16_t port) { return svr_.listen(host, port); }
|
|
|
|
|
+void ApiServer::stop() { svr_.stop(); }
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> `meta.cpp` and `webui_auth.cpp` are implemented in this task (Steps 4–5). The
|
|
|
|
|
+> four remaining handler files are defined in later tasks (12–15); to compile
|
|
|
|
|
+> Task 11, add **empty stub definitions** now, each replaced later with the real
|
|
|
|
|
+> body. Create exactly these four files with these exact function names:
|
|
|
|
|
+> ```cpp
|
|
|
|
|
+> // src/handlers/collections.cpp
|
|
|
|
|
+> #include "server.hpp"
|
|
|
|
|
+> namespace svapi { void registerCollectionRoutes(ApiServer&) {} }
|
|
|
|
|
+> ```
|
|
|
|
|
+> ```cpp
|
|
|
|
|
+> // src/handlers/documents.cpp
|
|
|
|
|
+> #include "server.hpp"
|
|
|
|
|
+> namespace svapi { void registerDocumentRoutes(ApiServer&) {} }
|
|
|
|
|
+> ```
|
|
|
|
|
+> ```cpp
|
|
|
|
|
+> // src/handlers/vectors.cpp
|
|
|
|
|
+> #include "server.hpp"
|
|
|
|
|
+> namespace svapi { void registerVectorRoutes(ApiServer&) {} }
|
|
|
|
|
+> ```
|
|
|
|
|
+> ```cpp
|
|
|
|
|
+> // src/handlers/stats.cpp
|
|
|
|
|
+> #include "server.hpp"
|
|
|
|
|
+> namespace svapi { void registerStatsRoutes(ApiServer&) {} }
|
|
|
|
|
+> ```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write `src/handlers/meta.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+#include <fstream>
|
|
|
|
|
+#include <sstream>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+namespace {
|
|
|
|
|
+std::string readFile(const std::string& path) {
|
|
|
|
|
+ std::ifstream f(path);
|
|
|
|
|
+ std::stringstream ss;
|
|
|
|
|
+ ss << f.rdbuf();
|
|
|
|
|
+ return ss.str();
|
|
|
|
|
+}
|
|
|
|
|
+} // namespace
|
|
|
|
|
+
|
|
|
|
|
+void registerMetaRoutes(ApiServer& s) {
|
|
|
|
|
+ auto& svr = s.raw();
|
|
|
|
|
+ DbGateway* db = &s.deps().db;
|
|
|
|
|
+ const std::string share = s.deps().shareDir;
|
|
|
|
|
+
|
|
|
|
|
+ svr.Get("/healthz", [](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ sendJson(res, 200, {{"status", "ok"}});
|
|
|
|
|
+ });
|
|
|
|
|
+ svr.Get("/readyz", [db](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ bool ok = db->healthy();
|
|
|
|
|
+ sendJson(res, ok ? 200 : 503, {{"ready", ok}});
|
|
|
|
|
+ });
|
|
|
|
|
+ svr.Get("/openapi.json", [share](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ auto body = readFile(share + "/openapi.json");
|
|
|
|
|
+ if (body.empty()) { sendJson(res, 404, errorBody("not_found", "openapi.json not found")); return; }
|
|
|
|
|
+ res.set_content(body, "application/json");
|
|
|
|
|
+ });
|
|
|
|
|
+ svr.Get("/llms.txt", [share](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ auto body = readFile(share + "/llms.txt");
|
|
|
|
|
+ if (body.empty()) { sendJson(res, 404, errorBody("not_found", "llms.txt not found")); return; }
|
|
|
|
|
+ res.set_content(body, "text/plain; charset=utf-8");
|
|
|
|
|
+ });
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Write `src/handlers/webui_auth.cpp`** (login/logout — needed for the full flow; safe to implement now)
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "auth.hpp"
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+#include <chrono>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+void registerWebuiAuthRoutes(ApiServer& s) {
|
|
|
|
|
+ auto& svr = s.raw();
|
|
|
|
|
+ ServerDeps* d = &s.deps();
|
|
|
|
|
+
|
|
|
|
|
+ svr.Post("/ui/login", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ auto body = bodyJson(req);
|
|
|
|
|
+ std::string key = body.value("key", "");
|
|
|
|
|
+ auto snap = d->settings.snapshot();
|
|
|
|
|
+ if (snap->apiKey.empty() || key != snap->apiKey)
|
|
|
|
|
+ throw ApiError(ErrCode::Unauthorized, "unauthorized", "invalid key");
|
|
|
|
|
+ d->sessions.setTtl(std::chrono::minutes(snap->sessionTtlMinutes));
|
|
|
|
|
+ std::string tok = d->sessions.create(std::chrono::system_clock::now());
|
|
|
|
|
+ res.set_header("Set-Cookie",
|
|
|
|
|
+ "svapi_session=" + tok + "; HttpOnly; SameSite=Strict; Path=/; Max-Age=" +
|
|
|
|
|
+ std::to_string(snap->sessionTtlMinutes * 60));
|
|
|
|
|
+ sendJson(res, 200, {{"ok", true}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr.Post("/ui/logout", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ if (auto it = req.headers.find("Cookie"); it != req.headers.end())
|
|
|
|
|
+ if (auto c = cookieValue(it->second, "svapi_session")) d->sessions.remove(*c);
|
|
|
|
|
+ res.set_header("Set-Cookie", "svapi_session=; HttpOnly; SameSite=Strict; Path=/; Max-Age=0");
|
|
|
|
|
+ sendJson(res, 200, {{"ok", true}});
|
|
|
|
|
+ });
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Write `tests/api_fixture.hpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include "db_test_util.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include <httplib.h>
|
|
|
|
|
+#include <chrono>
|
|
|
|
|
+#include <memory>
|
|
|
|
|
+#include <thread>
|
|
|
|
|
+#include <vector>
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi::testutil {
|
|
|
|
|
+
|
|
|
|
|
+// Spins up an ApiServer (random port) against the live DB plus a mock OpenAI
|
|
|
|
|
+// embeddings server. Skips all tests in the suite if the DB is unreachable.
|
|
|
|
|
+class ApiFixture : public ::testing::Test {
|
|
|
|
|
+protected:
|
|
|
|
|
+ std::unique_ptr<DbGateway> db_;
|
|
|
|
|
+ std::unique_ptr<CollectionRegistry> registry_;
|
|
|
|
|
+ std::unique_ptr<SettingsStore> settings_;
|
|
|
|
|
+ std::unique_ptr<SessionStore> sessions_;
|
|
|
|
|
+ std::unique_ptr<ApiServer> server_;
|
|
|
|
|
+ std::thread serverThread_;
|
|
|
|
|
+ int port_ = 0;
|
|
|
|
|
+ std::string apiKey_;
|
|
|
|
|
+
|
|
|
|
|
+ httplib::Server mockOpenAi_;
|
|
|
|
|
+ std::thread mockThread_;
|
|
|
|
|
+ int mockPort_ = 0;
|
|
|
|
|
+ int mockDim_ = 4;
|
|
|
|
|
+
|
|
|
|
|
+ std::string settingsColl_, registryColl_;
|
|
|
|
|
+ std::vector<std::string> createdCollections_;
|
|
|
|
|
+
|
|
|
|
|
+ void SetUp() override {
|
|
|
|
|
+ db_ = connectOrNull();
|
|
|
|
|
+ if (!db_) GTEST_SKIP() << "smartbotic-database not reachable";
|
|
|
|
|
+
|
|
|
|
|
+ settingsColl_ = tmpCollection("api_settings");
|
|
|
|
|
+ registryColl_ = tmpCollection("api_registry");
|
|
|
|
|
+ db_->client().dropCollection(settingsColl_);
|
|
|
|
|
+ db_->client().dropCollection(registryColl_);
|
|
|
|
|
+
|
|
|
|
|
+ registry_ = std::make_unique<CollectionRegistry>(*db_, registryColl_);
|
|
|
|
|
+ registry_->ensure();
|
|
|
|
|
+ settings_ = std::make_unique<SettingsStore>(*db_, settingsColl_, "current");
|
|
|
|
|
+ settings_->bootstrap("", "");
|
|
|
|
|
+ apiKey_ = settings_->snapshot()->apiKey;
|
|
|
|
|
+ sessions_ = std::make_unique<SessionStore>(std::chrono::minutes(60));
|
|
|
|
|
+
|
|
|
|
|
+ mockOpenAi_.Post("/v1/embeddings", [this](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ nlohmann::json emb = nlohmann::json::array();
|
|
|
|
|
+ for (int i = 0; i < mockDim_; ++i) emb.push_back(0.1f * (i + 1));
|
|
|
|
|
+ res.set_content(nlohmann::json{{"data", {{{"embedding", emb}}}}}.dump(), "application/json");
|
|
|
|
|
+ });
|
|
|
|
|
+ mockPort_ = mockOpenAi_.bind_to_any_port("127.0.0.1");
|
|
|
|
|
+ mockThread_ = std::thread([this] { mockOpenAi_.listen_after_bind(); });
|
|
|
|
|
+
|
|
|
|
|
+ Settings s = *settings_->snapshot();
|
|
|
|
|
+ s.openaiApiBase = "http://127.0.0.1:" + std::to_string(mockPort_);
|
|
|
|
|
+ s.openaiApiKey = "test";
|
|
|
|
|
+ settings_->save(s);
|
|
|
|
|
+
|
|
|
|
|
+ ServerDeps deps{*db_, *settings_, *registry_, *sessions_, "", apiDocsDir()};
|
|
|
|
|
+ server_ = std::make_unique<ApiServer>(std::move(deps));
|
|
|
|
|
+ server_->registerRoutes();
|
|
|
|
|
+ port_ = server_->bindToAnyPort("127.0.0.1");
|
|
|
|
|
+ serverThread_ = std::thread([this] { server_->listenAfterBind(); });
|
|
|
|
|
+
|
|
|
|
|
+ httplib::Client probe("127.0.0.1", port_);
|
|
|
|
|
+ for (int i = 0; i < 200; ++i) {
|
|
|
|
|
+ if (probe.Get("/healthz")) break;
|
|
|
|
|
+ std::this_thread::sleep_for(std::chrono::milliseconds(10));
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ void TearDown() override {
|
|
|
|
|
+ if (server_) server_->stop();
|
|
|
|
|
+ if (serverThread_.joinable()) serverThread_.join();
|
|
|
|
|
+ mockOpenAi_.stop();
|
|
|
|
|
+ if (mockThread_.joinable()) mockThread_.join();
|
|
|
|
|
+ if (db_) {
|
|
|
|
|
+ db_->client().dropCollection(settingsColl_);
|
|
|
|
|
+ db_->client().dropCollection(registryColl_);
|
|
|
|
|
+ for (const auto& c : createdCollections_) db_->client().dropCollection(c);
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ // Authenticated client (bearer).
|
|
|
|
|
+ httplib::Client http() {
|
|
|
|
|
+ httplib::Client c("127.0.0.1", port_);
|
|
|
|
|
+ c.set_default_headers({{"Authorization", "Bearer " + apiKey_}});
|
|
|
|
|
+ return c;
|
|
|
|
|
+ }
|
|
|
|
|
+ // Unauthenticated client.
|
|
|
|
|
+ httplib::Client httpNoAuth() { return httplib::Client("127.0.0.1", port_); }
|
|
|
|
|
+
|
|
|
|
|
+ static std::string apiDocsDir() { return SVAPI_API_DOCS_DIR; }
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi::testutil
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7: Write `tests/test_api_integration.cpp`** (meta + auth coverage; extended in later tasks)
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "api_fixture.hpp"
|
|
|
|
|
+
|
|
|
|
|
+using svapi::testutil::ApiFixture;
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, HealthzIsPublicAndOk) {
|
|
|
|
|
+ auto r = httpNoAuth().Get("/healthz");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ EXPECT_EQ(r->status, 200);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, ReadyzReportsDbUp) {
|
|
|
|
|
+ auto r = httpNoAuth().Get("/readyz");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ EXPECT_EQ(r->status, 200);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, OpenApiAndLlmsArePublic) {
|
|
|
|
|
+ auto a = httpNoAuth().Get("/openapi.json");
|
|
|
|
|
+ ASSERT_TRUE(a);
|
|
|
|
|
+ EXPECT_EQ(a->status, 200);
|
|
|
|
|
+ auto b = httpNoAuth().Get("/llms.txt");
|
|
|
|
|
+ ASSERT_TRUE(b);
|
|
|
|
|
+ EXPECT_EQ(b->status, 200);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, ApiRequiresAuth) {
|
|
|
|
|
+ auto r = httpNoAuth().Get("/api/v1/collections");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ EXPECT_EQ(r->status, 401);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, LoginIssuesCookieThatAuthorizes) {
|
|
|
|
|
+ auto login = httpNoAuth().Post("/ui/login",
|
|
|
|
|
+ nlohmann::json{{"key", apiKey_}}.dump(), "application/json");
|
|
|
|
|
+ ASSERT_TRUE(login);
|
|
|
|
|
+ EXPECT_EQ(login->status, 200);
|
|
|
|
|
+ ASSERT_TRUE(login->has_header("Set-Cookie"));
|
|
|
|
|
+ std::string setCookie = login->get_header_value("Set-Cookie");
|
|
|
|
|
+ std::string cookie = setCookie.substr(0, setCookie.find(';'));
|
|
|
|
|
+
|
|
|
|
|
+ httplib::Client c("127.0.0.1", port_);
|
|
|
|
|
+ c.set_default_headers({{"Cookie", cookie}});
|
|
|
|
|
+ auto r = c.Get("/api/v1/collections");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ EXPECT_NE(r->status, 401); // cookie authorized (200 once collections land)
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8: Add sources + test target to CMake**
|
|
|
|
|
+
|
|
|
|
|
+In `src/CMakeLists.txt`, add to `vectorapi_core`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+ server.cpp
|
|
|
|
|
+ handlers/meta.cpp
|
|
|
|
|
+ handlers/webui_auth.cpp
|
|
|
|
|
+ handlers/collections.cpp
|
|
|
|
|
+ handlers/documents.cpp
|
|
|
|
|
+ handlers/vectors.cpp
|
|
|
|
|
+ handlers/stats.cpp
|
|
|
|
|
+```
|
|
|
|
|
+In `tests/CMakeLists.txt`:
|
|
|
|
|
+```cmake
|
|
|
|
|
+add_executable(test_api_integration test_api_integration.cpp)
|
|
|
|
|
+target_link_libraries(test_api_integration PRIVATE vectorapi_core GTest::gtest_main)
|
|
|
|
|
+target_compile_definitions(test_api_integration PRIVATE
|
|
|
|
|
+ SVAPI_API_DOCS_DIR="${CMAKE_SOURCE_DIR}/api")
|
|
|
|
|
+gtest_discover_tests(test_api_integration)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 9: Build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R ApiFixture --output-on-failure`
|
|
|
|
|
+Expected: meta/auth tests PASS (or SKIP without DB). `LoginIssuesCookie...` may return 404 on the final GET until Task 12 — adjust the assertion to `EXPECT_NE(r->status, 401)` (already written that way).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 10: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/server.hpp src/server.cpp src/handlers api/openapi.json api/llms.txt tests/api_fixture.hpp tests/test_api_integration.cpp src/CMakeLists.txt tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(server): ApiServer, auth middleware, CORS, meta routes, login/logout, integration fixture"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 12: Collection handlers (admin-managed CRUD)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `src/handlers/collections.cpp` (replace the stub)
|
|
|
|
|
+- Modify: `tests/test_api_integration.cpp` (append tests)
|
|
|
|
|
+
|
|
|
|
|
+Endpoints: `POST /api/v1/collections`, `GET /api/v1/collections`,
|
|
|
|
|
+`GET /api/v1/collections/{name}`, `DELETE /api/v1/collections/{name}`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Append failing tests to `tests/test_api_integration.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+TEST_F(ApiFixture, CreateListGetDeleteJsonCollection) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ std::string name = "_svapi_test_users";
|
|
|
|
|
+ createdCollections_.push_back(name);
|
|
|
|
|
+
|
|
|
|
|
+ auto created = c.Post("/api/v1/collections",
|
|
|
|
|
+ nlohmann::json{{"name", name}, {"kind", "json"}}.dump(), "application/json");
|
|
|
|
|
+ ASSERT_TRUE(created);
|
|
|
|
|
+ EXPECT_EQ(created->status, 201);
|
|
|
|
|
+
|
|
|
|
|
+ auto list = c.Get("/api/v1/collections");
|
|
|
|
|
+ ASSERT_TRUE(list);
|
|
|
|
|
+ EXPECT_EQ(list->status, 200);
|
|
|
|
|
+ auto arr = nlohmann::json::parse(list->body);
|
|
|
|
|
+ bool found = false;
|
|
|
|
|
+ for (auto& e : arr["collections"]) if (e["name"] == name) found = true;
|
|
|
|
|
+ EXPECT_TRUE(found);
|
|
|
|
|
+
|
|
|
|
|
+ auto got = c.Get(("/api/v1/collections/" + name).c_str());
|
|
|
|
|
+ ASSERT_TRUE(got);
|
|
|
|
|
+ EXPECT_EQ(got->status, 200);
|
|
|
|
|
+ EXPECT_EQ(nlohmann::json::parse(got->body)["kind"], "json");
|
|
|
|
|
+
|
|
|
|
|
+ auto del = c.Delete(("/api/v1/collections/" + name).c_str());
|
|
|
|
|
+ ASSERT_TRUE(del);
|
|
|
|
|
+ EXPECT_EQ(del->status, 200);
|
|
|
|
|
+
|
|
|
|
|
+ auto gone = c.Get(("/api/v1/collections/" + name).c_str());
|
|
|
|
|
+ ASSERT_TRUE(gone);
|
|
|
|
|
+ EXPECT_EQ(gone->status, 404);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, CreateVectorCollectionRequiresDimension) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ auto bad = c.Post("/api/v1/collections",
|
|
|
|
|
+ nlohmann::json{{"name", "_svapi_test_novec"}, {"kind", "vector"}}.dump(), "application/json");
|
|
|
|
|
+ ASSERT_TRUE(bad);
|
|
|
|
|
+ EXPECT_EQ(bad->status, 422);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture.Create" --output-on-failure`
|
|
|
|
|
+Expected: FAIL — handlers return 404 (stub registers nothing).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Replace `src/handlers/collections.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+void registerCollectionRoutes(ApiServer& s) {
|
|
|
|
|
+ auto& svr = s.raw();
|
|
|
|
|
+ ServerDeps* d = &s.deps();
|
|
|
|
|
+
|
|
|
|
|
+ // Create
|
|
|
|
|
+ svr.Post("/api/v1/collections", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ auto body = bodyJson(req);
|
|
|
|
|
+ std::string name = body.value("name", "");
|
|
|
|
|
+ std::string kind = body.value("kind", "json");
|
|
|
|
|
+ if (name.empty())
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "name is required");
|
|
|
|
|
+ if (name.rfind('_', 0) == 0)
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "name may not start with '_'");
|
|
|
|
|
+ if (kind != "json" && kind != "vector")
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "kind must be 'json' or 'vector'");
|
|
|
|
|
+
|
|
|
|
|
+ uint32_t dim = 0;
|
|
|
|
|
+ std::string model;
|
|
|
|
|
+ if (kind == "vector") {
|
|
|
|
|
+ if (!body.contains("vector_dimension"))
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation",
|
|
|
|
|
+ "vector_dimension is required for vector collections");
|
|
|
|
|
+ dim = body.value("vector_dimension", 0u);
|
|
|
|
|
+ if (dim == 0)
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "vector_dimension must be > 0");
|
|
|
|
|
+ model = body.value("embedding_model", "");
|
|
|
|
|
+ }
|
|
|
|
|
+ if (d->registry.get(name).has_value())
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "exists", "collection already exists");
|
|
|
|
|
+
|
|
|
|
|
+ if (!d->db.client().createCollection(name, 0, false, 0, dim))
|
|
|
|
|
+ throw ApiError(ErrCode::Unavailable, "db_error", "failed to create collection");
|
|
|
|
|
+ d->registry.add({name, kind, dim, model, 0});
|
|
|
|
|
+ sendJson(res, 201, {{"name", name}, {"kind", kind}, {"vector_dimension", dim}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ // List
|
|
|
|
|
+ svr.Get("/api/v1/collections", [d](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ nlohmann::json out = nlohmann::json::array();
|
|
|
|
|
+ for (const auto& m : d->registry.list()) {
|
|
|
|
|
+ nlohmann::json e = m.toJson();
|
|
|
|
|
+ if (auto info = d->db.client().getCollectionInfo(m.name)) {
|
|
|
|
|
+ e["document_count"] = info->documentCount;
|
|
|
|
|
+ e["size_bytes"] = info->sizeBytes;
|
|
|
|
|
+ }
|
|
|
|
|
+ out.push_back(e);
|
|
|
|
|
+ }
|
|
|
|
|
+ sendJson(res, 200, {{"collections", out}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ // Get one
|
|
|
|
|
+ svr.Get(R"(/api/v1/collections/([^/]+))", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string name = req.matches[1];
|
|
|
|
|
+ auto m = d->registry.get(name);
|
|
|
|
|
+ if (!m) throw ApiError(ErrCode::NotFound, "not_found", "no such collection");
|
|
|
|
|
+ nlohmann::json e = m->toJson();
|
|
|
|
|
+ if (auto info = d->db.client().getCollectionInfo(name)) {
|
|
|
|
|
+ e["document_count"] = info->documentCount;
|
|
|
|
|
+ e["size_bytes"] = info->sizeBytes;
|
|
|
|
|
+ }
|
|
|
|
|
+ sendJson(res, 200, e);
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ // Delete
|
|
|
|
|
+ svr.Delete(R"(/api/v1/collections/([^/]+))", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string name = req.matches[1];
|
|
|
|
|
+ if (!d->registry.get(name)) throw ApiError(ErrCode::NotFound, "not_found", "no such collection");
|
|
|
|
|
+ d->db.client().dropCollection(name);
|
|
|
|
|
+ d->registry.remove(name);
|
|
|
|
|
+ sendJson(res, 200, {{"deleted", name}});
|
|
|
|
|
+ });
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture" --output-on-failure`
|
|
|
|
|
+Expected: collection tests PASS (the earlier `LoginIssuesCookie...` final GET now returns 200).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/handlers/collections.cpp tests/test_api_integration.cpp
|
|
|
|
|
+git commit -m "feat(api): collection create/list/get/delete handlers"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 13: Document handlers (JSON CRUD + find)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `src/handlers/documents.cpp` (replace the stub)
|
|
|
|
|
+- Modify: `tests/test_api_integration.cpp` (append tests)
|
|
|
|
|
+
|
|
|
|
|
+Endpoints under `/api/v1/collections/{name}/documents`. httplib matches the full
|
|
|
|
|
+path, so `/documents` (find) and `/documents/{id}` (get one) do not collide.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Append failing tests to `tests/test_api_integration.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+TEST_F(ApiFixture, DocumentCrudAndFind) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ std::string coll = "_svapi_test_docs";
|
|
|
|
|
+ createdCollections_.push_back(coll);
|
|
|
|
|
+ ASSERT_EQ(c.Post("/api/v1/collections",
|
|
|
|
|
+ nlohmann::json{{"name", coll}, {"kind", "json"}}.dump(), "application/json")->status, 201);
|
|
|
|
|
+
|
|
|
|
|
+ // insert
|
|
|
|
|
+ auto ins = c.Post(("/api/v1/collections/" + coll + "/documents").c_str(),
|
|
|
|
|
+ nlohmann::json{{"data", {{"name", "Alice"}, {"age", 30}}}}.dump(), "application/json");
|
|
|
|
|
+ ASSERT_TRUE(ins);
|
|
|
|
|
+ ASSERT_EQ(ins->status, 201);
|
|
|
|
|
+ std::string id = nlohmann::json::parse(ins->body)["id"];
|
|
|
|
|
+ ASSERT_FALSE(id.empty());
|
|
|
|
|
+
|
|
|
|
|
+ // get
|
|
|
|
|
+ auto got = c.Get(("/api/v1/collections/" + coll + "/documents/" + id).c_str());
|
|
|
|
|
+ ASSERT_EQ(got->status, 200);
|
|
|
|
|
+ EXPECT_EQ(nlohmann::json::parse(got->body)["name"], "Alice");
|
|
|
|
|
+
|
|
|
|
|
+ // patch
|
|
|
|
|
+ auto pat = c.Patch(("/api/v1/collections/" + coll + "/documents/" + id).c_str(),
|
|
|
|
|
+ nlohmann::json{{"age", 31}}.dump(), "application/json");
|
|
|
|
|
+ ASSERT_EQ(pat->status, 200);
|
|
|
|
|
+ EXPECT_EQ(nlohmann::json::parse(c.Get(("/api/v1/collections/" + coll + "/documents/" + id).c_str())->body)["age"], 31);
|
|
|
|
|
+
|
|
|
|
|
+ // find with filter
|
|
|
|
|
+ auto found = c.Get(("/api/v1/collections/" + coll + "/documents?filter=age:gte:31&limit=10").c_str());
|
|
|
|
|
+ ASSERT_EQ(found->status, 200);
|
|
|
|
|
+ EXPECT_GE(nlohmann::json::parse(found->body)["count"].get<int>(), 1);
|
|
|
|
|
+
|
|
|
|
|
+ // delete
|
|
|
|
|
+ ASSERT_EQ(c.Delete(("/api/v1/collections/" + coll + "/documents/" + id).c_str())->status, 200);
|
|
|
|
|
+ EXPECT_EQ(c.Get(("/api/v1/collections/" + coll + "/documents/" + id).c_str())->status, 404);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, DocumentOnMissingCollectionIs404) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ auto r = c.Get("/api/v1/collections/_svapi_test_nope/documents/x");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ EXPECT_EQ(r->status, 404);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture.Document" --output-on-failure`
|
|
|
|
|
+Expected: FAIL — documents stub registers no routes (404 on insert).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Replace `src/handlers/documents.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include "filters.hpp"
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+namespace {
|
|
|
|
|
+void requireCollection(ServerDeps* d, const std::string& name) {
|
|
|
|
|
+ if (!d->registry.get(name)) throw ApiError(ErrCode::NotFound, "not_found", "no such collection");
|
|
|
|
|
+}
|
|
|
|
|
+} // namespace
|
|
|
|
|
+
|
|
|
|
|
+void registerDocumentRoutes(ApiServer& s) {
|
|
|
|
|
+ auto& svr = s.raw();
|
|
|
|
|
+ ServerDeps* d = &s.deps();
|
|
|
|
|
+
|
|
|
|
|
+ svr.Post(R"(/api/v1/collections/([^/]+)/documents)", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string coll = req.matches[1];
|
|
|
|
|
+ requireCollection(d, coll);
|
|
|
|
|
+ auto body = bodyJson(req);
|
|
|
|
|
+ std::string id = body.value("id", "");
|
|
|
|
|
+ nlohmann::json data = body.contains("data") ? body["data"] : body;
|
|
|
|
|
+ if (data.is_object() && data.contains("id")) data.erase("id");
|
|
|
|
|
+ std::string newId = d->db.client().insert(coll, data, id);
|
|
|
|
|
+ if (newId.empty()) throw ApiError(ErrCode::Unavailable, "db_error", "insert failed");
|
|
|
|
|
+ sendJson(res, 201, {{"id", newId}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr.Get(R"(/api/v1/collections/([^/]+)/documents/([^/]+))", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string coll = req.matches[1], id = req.matches[2];
|
|
|
|
|
+ requireCollection(d, coll);
|
|
|
|
|
+ auto doc = d->db.client().get(coll, id);
|
|
|
|
|
+ if (!doc) throw ApiError(ErrCode::NotFound, "not_found", "no such document");
|
|
|
|
|
+ sendJson(res, 200, *doc);
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr.Get(R"(/api/v1/collections/([^/]+)/documents)", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string coll = req.matches[1];
|
|
|
|
|
+ requireCollection(d, coll);
|
|
|
|
|
+ smartbotic::database::Client::QueryOptions opts;
|
|
|
|
|
+ auto range = req.params.equal_range("filter");
|
|
|
|
|
+ for (auto it = range.first; it != range.second; ++it) opts.filters.push_back(parseFilter(it->second));
|
|
|
|
|
+ if (req.has_param("limit")) opts.limit = static_cast<uint32_t>(std::stoul(req.get_param_value("limit")));
|
|
|
|
|
+ if (req.has_param("offset")) opts.offset = static_cast<uint32_t>(std::stoul(req.get_param_value("offset")));
|
|
|
|
|
+ if (req.has_param("sort")) opts.sortField = req.get_param_value("sort");
|
|
|
|
|
+ if (req.has_param("desc")) opts.sortDescending = (req.get_param_value("desc") == "true");
|
|
|
|
|
+ auto docs = d->db.client().find(coll, opts);
|
|
|
|
|
+ sendJson(res, 200, {{"documents", docs}, {"count", docs.size()}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr.Patch(R"(/api/v1/collections/([^/]+)/documents/([^/]+))", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string coll = req.matches[1], id = req.matches[2];
|
|
|
|
|
+ requireCollection(d, coll);
|
|
|
|
|
+ if (!d->db.client().exists(coll, id)) throw ApiError(ErrCode::NotFound, "not_found", "no such document");
|
|
|
|
|
+ uint64_t v = d->db.client().patch(coll, id, bodyJson(req));
|
|
|
|
|
+ if (v == 0) throw ApiError(ErrCode::Unavailable, "db_error", "patch failed");
|
|
|
|
|
+ sendJson(res, 200, {{"id", id}, {"version", v}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr.Put(R"(/api/v1/collections/([^/]+)/documents/([^/]+))", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string coll = req.matches[1], id = req.matches[2];
|
|
|
|
|
+ requireCollection(d, coll);
|
|
|
|
|
+ if (!d->db.client().exists(coll, id)) throw ApiError(ErrCode::NotFound, "not_found", "no such document");
|
|
|
|
|
+ if (!d->db.client().update(coll, id, bodyJson(req)))
|
|
|
|
|
+ throw ApiError(ErrCode::Unavailable, "db_error", "update failed");
|
|
|
|
|
+ sendJson(res, 200, {{"id", id}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr.Delete(R"(/api/v1/collections/([^/]+)/documents/([^/]+))", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ std::string coll = req.matches[1], id = req.matches[2];
|
|
|
|
|
+ requireCollection(d, coll);
|
|
|
|
|
+ if (!d->db.client().remove(coll, id)) throw ApiError(ErrCode::NotFound, "not_found", "no such document");
|
|
|
|
|
+ sendJson(res, 200, {{"deleted", id}});
|
|
|
|
|
+ });
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture.Document" --output-on-failure`
|
|
|
|
|
+Expected: document tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/handlers/documents.cpp tests/test_api_integration.cpp
|
|
|
|
|
+git commit -m "feat(api): JSON document CRUD + filtered find handlers"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 14: Vector + search handlers (with embedding generation)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `src/handlers/vectors.cpp` (replace the stub)
|
|
|
|
|
+- Modify: `tests/test_api_integration.cpp` (append tests)
|
|
|
|
|
+
|
|
|
|
|
+The fixture's mock OpenAI returns a `mockDim_`-length (4) embedding, so vector
|
|
|
|
|
+collections in these tests use dimension 4.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Append failing tests to `tests/test_api_integration.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+TEST_F(ApiFixture, StoreVectorFromTextThenSearch) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ std::string coll = "_svapi_test_rag";
|
|
|
|
|
+ createdCollections_.push_back(coll);
|
|
|
|
|
+ ASSERT_EQ(c.Post("/api/v1/collections",
|
|
|
|
|
+ nlohmann::json{{"name", coll}, {"kind", "vector"}, {"vector_dimension", mockDim_}}.dump(),
|
|
|
|
|
+ "application/json")->status, 201);
|
|
|
|
|
+
|
|
|
|
|
+ // store from text -> embedding generated by mock OpenAI
|
|
|
|
|
+ auto store = c.Post(("/api/v1/collections/" + coll + "/vectors").c_str(),
|
|
|
|
|
+ nlohmann::json{{"text", "the user likes dark mode"}, {"metadata", {{"src", "n8n"}}}}.dump(),
|
|
|
|
|
+ "application/json");
|
|
|
|
|
+ ASSERT_TRUE(store);
|
|
|
|
|
+ ASSERT_EQ(store->status, 201);
|
|
|
|
|
+
|
|
|
|
|
+ // search by text
|
|
|
|
|
+ auto search = c.Post(("/api/v1/collections/" + coll + "/search").c_str(),
|
|
|
|
|
+ nlohmann::json{{"query_text", "dark mode"}, {"top_k", 5}}.dump(), "application/json");
|
|
|
|
|
+ ASSERT_TRUE(search);
|
|
|
|
|
+ ASSERT_EQ(search->status, 200);
|
|
|
|
|
+ auto results = nlohmann::json::parse(search->body)["results"];
|
|
|
|
|
+ EXPECT_GE(results.size(), 1u);
|
|
|
|
|
+ EXPECT_TRUE(results[0].contains("score"));
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, StoreVectorWithWrongDimensionIs422) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ std::string coll = "_svapi_test_rag2";
|
|
|
|
|
+ createdCollections_.push_back(coll);
|
|
|
|
|
+ ASSERT_EQ(c.Post("/api/v1/collections",
|
|
|
|
|
+ nlohmann::json{{"name", coll}, {"kind", "vector"}, {"vector_dimension", mockDim_}}.dump(),
|
|
|
|
|
+ "application/json")->status, 201);
|
|
|
|
|
+
|
|
|
|
|
+ auto bad = c.Post(("/api/v1/collections/" + coll + "/vectors").c_str(),
|
|
|
|
|
+ nlohmann::json{{"vector", {1.0, 2.0}}}.dump(), "application/json"); // dim 2 != 4
|
|
|
|
|
+ ASSERT_TRUE(bad);
|
|
|
|
|
+ EXPECT_EQ(bad->status, 422);
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+TEST_F(ApiFixture, VectorEndpointOnJsonCollectionIs422) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ std::string coll = "_svapi_test_plain";
|
|
|
|
|
+ createdCollections_.push_back(coll);
|
|
|
|
|
+ ASSERT_EQ(c.Post("/api/v1/collections",
|
|
|
|
|
+ nlohmann::json{{"name", coll}, {"kind", "json"}}.dump(), "application/json")->status, 201);
|
|
|
|
|
+ auto r = c.Post(("/api/v1/collections/" + coll + "/vectors").c_str(),
|
|
|
|
|
+ nlohmann::json{{"vector", {1, 2, 3, 4}}}.dump(), "application/json");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ EXPECT_EQ(r->status, 422);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture.StoreVector|ApiFixture.Vector" --output-on-failure`
|
|
|
|
|
+Expected: FAIL — vectors stub registers no routes.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Replace `src/handlers/vectors.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+#include "embeddings.hpp"
|
|
|
|
|
+#include "errors.hpp"
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+namespace {
|
|
|
|
|
+CollectionMeta requireVectorCollection(ServerDeps* d, const std::string& name) {
|
|
|
|
|
+ auto m = d->registry.get(name);
|
|
|
|
|
+ if (!m) throw ApiError(ErrCode::NotFound, "not_found", "no such collection");
|
|
|
|
|
+ if (m->kind != "vector")
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "not_vector", "collection is not a vector collection");
|
|
|
|
|
+ return *m;
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+// Resolve a vector from {vector} or {text} (generating via OpenAI), validating
|
|
|
|
|
+// against the collection's fixed dimension.
|
|
|
|
|
+std::vector<float> resolveVector(ServerDeps* d, const CollectionMeta& meta,
|
|
|
|
|
+ const nlohmann::json& body,
|
|
|
|
|
+ const char* vecField, const char* textField) {
|
|
|
|
|
+ std::vector<float> v;
|
|
|
|
|
+ if (body.contains(vecField) && body[vecField].is_array()) {
|
|
|
|
|
+ v = body[vecField].get<std::vector<float>>();
|
|
|
|
|
+ } else if (body.contains(textField) && body[textField].is_string()) {
|
|
|
|
|
+ auto snap = d->settings.snapshot();
|
|
|
|
|
+ std::string model = meta.embeddingModel.empty() ? snap->defaultEmbeddingModel : meta.embeddingModel;
|
|
|
|
|
+ EmbeddingClient emb(snap->openaiApiBase, snap->openaiApiKey);
|
|
|
|
|
+ v = emb.embed(model, body[textField].get<std::string>());
|
|
|
|
|
+ } else {
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation",
|
|
|
|
|
+ std::string("provide '") + vecField + "' or '" + textField + "'");
|
|
|
|
|
+ }
|
|
|
|
|
+ if (v.size() != meta.vectorDimension)
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "dimension_mismatch",
|
|
|
|
|
+ "expected dimension " + std::to_string(meta.vectorDimension) +
|
|
|
|
|
+ ", got " + std::to_string(v.size()));
|
|
|
|
|
+ return v;
|
|
|
|
|
+}
|
|
|
|
|
+} // namespace
|
|
|
|
|
+
|
|
|
|
|
+void registerVectorRoutes(ApiServer& s) {
|
|
|
|
|
+ auto& svr = s.raw();
|
|
|
|
|
+ ServerDeps* d = &s.deps();
|
|
|
|
|
+
|
|
|
|
|
+ svr.Post(R"(/api/v1/collections/([^/]+)/vectors)", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ CollectionMeta meta = requireVectorCollection(d, req.matches[1]);
|
|
|
|
|
+ auto body = bodyJson(req);
|
|
|
|
|
+ std::vector<float> vec = resolveVector(d, meta, body, "vector", "text");
|
|
|
|
|
+
|
|
|
|
|
+ nlohmann::json doc = body.value("metadata", nlohmann::json::object());
|
|
|
|
|
+ if (!doc.is_object()) doc = nlohmann::json::object();
|
|
|
|
|
+ if (body.contains("text")) doc["text"] = body["text"];
|
|
|
|
|
+ doc["_vector"] = vec;
|
|
|
|
|
+ std::string newId = d->db.client().insert(std::string(req.matches[1]), doc, body.value("id", ""));
|
|
|
|
|
+ if (newId.empty()) throw ApiError(ErrCode::Unavailable, "db_error", "insert failed");
|
|
|
|
|
+ sendJson(res, 201, {{"id", newId}});
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ svr.Post(R"(/api/v1/collections/([^/]+)/search)", [d](const httplib::Request& req, httplib::Response& res) {
|
|
|
|
|
+ CollectionMeta meta = requireVectorCollection(d, req.matches[1]);
|
|
|
|
|
+ auto body = bodyJson(req);
|
|
|
|
|
+ std::vector<float> q = resolveVector(d, meta, body, "query_vector", "query_text");
|
|
|
|
|
+ uint32_t topK = body.value("top_k", 5u);
|
|
|
|
|
+ float minScore = body.value("min_score", 0.0f);
|
|
|
|
|
+
|
|
|
|
|
+ auto results = d->db.client().similaritySearch(std::string(req.matches[1]), q, topK, minScore);
|
|
|
|
|
+ nlohmann::json arr = nlohmann::json::array();
|
|
|
|
|
+ for (const auto& r : results)
|
|
|
|
|
+ arr.push_back({{"id", r.id}, {"score", r.score}, {"data", r.data}});
|
|
|
|
|
+ sendJson(res, 200, {{"results", arr}});
|
|
|
|
|
+ });
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture" --output-on-failure`
|
|
|
|
|
+Expected: vector/search tests PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/handlers/vectors.cpp tests/test_api_integration.cpp
|
|
|
|
|
+git commit -m "feat(api): vector store + similarity search with OpenAI embedding generation"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 15: Stats handler
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `src/handlers/stats.cpp` (replace the stub)
|
|
|
|
|
+- Modify: `tests/test_api_integration.cpp` (append a test)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Append failing test to `tests/test_api_integration.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+TEST_F(ApiFixture, StatsReturnsCounts) {
|
|
|
|
|
+ auto c = http();
|
|
|
|
|
+ auto r = c.Get("/api/v1/stats");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ ASSERT_EQ(r->status, 200);
|
|
|
|
|
+ auto j = nlohmann::json::parse(r->body);
|
|
|
|
|
+ EXPECT_TRUE(j.contains("total_documents"));
|
|
|
|
|
+ EXPECT_TRUE(j.contains("memory_pressure_level"));
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture.Stats" --output-on-failure`
|
|
|
|
|
+Expected: FAIL — stats stub returns 404.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Replace `src/handlers/stats.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+#include "json_http.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+
|
|
|
|
|
+void registerStatsRoutes(ApiServer& s) {
|
|
|
|
|
+ auto& svr = s.raw();
|
|
|
|
|
+ ServerDeps* d = &s.deps();
|
|
|
|
|
+
|
|
|
|
|
+ svr.Get("/api/v1/stats", [d](const httplib::Request&, httplib::Response& res) {
|
|
|
|
|
+ nlohmann::json out;
|
|
|
|
|
+ if (auto st = d->db.client().getStats()) {
|
|
|
|
|
+ out["total_documents"] = st->totalDocuments;
|
|
|
|
|
+ out["total_collections"] = st->totalCollections;
|
|
|
|
|
+ out["memory_used_bytes"] = st->memoryUsedBytes;
|
|
|
|
|
+ out["insert_count"] = st->insertCount;
|
|
|
|
|
+ out["update_count"] = st->updateCount;
|
|
|
|
|
+ out["delete_count"] = st->deleteCount;
|
|
|
|
|
+ out["query_count"] = st->queryCount;
|
|
|
|
|
+ } else {
|
|
|
|
|
+ out["total_documents"] = 0;
|
|
|
|
|
+ }
|
|
|
|
|
+ auto mem = d->db.client().getMemoryStats();
|
|
|
|
|
+ out["memory_pressure_level"] = mem.pressureLevel;
|
|
|
|
|
+ out["memory_pressure_percent"] = mem.pressurePercent;
|
|
|
|
|
+ out["managed_collections"] = d->registry.list().size();
|
|
|
|
|
+ sendJson(res, 200, out);
|
|
|
|
|
+ });
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+} // namespace svapi
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Build + run**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture.Stats" --output-on-failure`
|
|
|
|
|
+Expected: PASS.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/handlers/stats.cpp tests/test_api_integration.cpp
|
|
|
|
|
+git commit -m "feat(api): /api/v1/stats handler (DB + memory stats)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 16: main.cpp full wiring (config, connect, bootstrap, serve, signals)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `src/main.cpp` (replace the version stub)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Replace `src/main.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "auth.hpp"
|
|
|
|
|
+#include "collection_registry.hpp"
|
|
|
|
|
+#include "config.hpp"
|
|
|
|
|
+#include "db_gateway.hpp"
|
|
|
|
|
+#include "server.hpp"
|
|
|
|
|
+#include "settings_store.hpp"
|
|
|
|
|
+#include <svapi/version.hpp>
|
|
|
|
|
+
|
|
|
|
|
+#include <spdlog/spdlog.h>
|
|
|
|
|
+#ifdef HAVE_SYSTEMD
|
|
|
|
|
+#include <systemd/sd-daemon.h>
|
|
|
|
|
+#else
|
|
|
|
|
+#define sd_notify(u, s) do {} while (0)
|
|
|
|
|
+#endif
|
|
|
|
|
+
|
|
|
|
|
+#include <atomic>
|
|
|
|
|
+#include <chrono>
|
|
|
|
|
+#include <csignal>
|
|
|
|
|
+#include <cstdlib>
|
|
|
|
|
+#include <cstring>
|
|
|
|
|
+#include <iostream>
|
|
|
|
|
+#include <thread>
|
|
|
|
|
+
|
|
|
|
|
+namespace {
|
|
|
|
|
+std::atomic<bool> g_running{true};
|
|
|
|
|
+void onSignal(int) { g_running = false; }
|
|
|
|
|
+std::string envOr(const char* key, const std::string& dflt = "") {
|
|
|
|
|
+ const char* v = std::getenv(key);
|
|
|
|
|
+ return v ? std::string(v) : dflt;
|
|
|
|
|
+}
|
|
|
|
|
+} // namespace
|
|
|
|
|
+
|
|
|
|
|
+int main(int argc, char** argv) {
|
|
|
|
|
+ std::string configPath = "/etc/smartbotic-vectorapi/config.json";
|
|
|
|
|
+ std::string webuiDirOverride;
|
|
|
|
|
+ std::string shareDir = "/usr/share/smartbotic-vectorapi";
|
|
|
|
|
+
|
|
|
|
|
+ for (int i = 1; i < argc; ++i) {
|
|
|
|
|
+ if (std::strcmp(argv[i], "--config") == 0 && i + 1 < argc) configPath = argv[++i];
|
|
|
|
|
+ else if (std::strcmp(argv[i], "--webui-dir") == 0 && i + 1 < argc) webuiDirOverride = argv[++i];
|
|
|
|
|
+ else if (std::strcmp(argv[i], "--share-dir") == 0 && i + 1 < argc) shareDir = argv[++i];
|
|
|
|
|
+ else if (std::strcmp(argv[i], "--version") == 0 || std::strcmp(argv[i], "-v") == 0) {
|
|
|
|
|
+ std::cout << "smartbotic-vectorapi " << svapi::VERSION << " (" << svapi::GIT_COMMIT << ")\n";
|
|
|
|
|
+ return 0;
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+
|
|
|
|
|
+ svapi::Config cfg;
|
|
|
|
|
+ try { cfg = svapi::Config::load(configPath); }
|
|
|
|
|
+ catch (const std::exception& e) { std::cerr << "config error: " << e.what() << "\n"; return 1; }
|
|
|
|
|
+ spdlog::set_level(spdlog::level::from_str(cfg.logLevel));
|
|
|
|
|
+ if (!webuiDirOverride.empty()) cfg.webuiDir = webuiDirOverride;
|
|
|
|
|
+
|
|
|
|
|
+ std::signal(SIGINT, onSignal);
|
|
|
|
|
+ std::signal(SIGTERM, onSignal);
|
|
|
|
|
+ std::signal(SIGPIPE, SIG_IGN);
|
|
|
|
|
+
|
|
|
|
|
+ svapi::DbGateway db(cfg.dbAddress);
|
|
|
|
|
+ spdlog::info("connecting to smartbotic-database at {}", cfg.dbAddress);
|
|
|
|
|
+ for (int i = 0; i < 30 && !db.connect(); ++i) std::this_thread::sleep_for(std::chrono::seconds(1));
|
|
|
|
|
+ if (!db.healthy()) { spdlog::error("database not reachable at {}", cfg.dbAddress); return 1; }
|
|
|
|
|
+
|
|
|
|
|
+ svapi::CollectionRegistry registry(db);
|
|
|
|
|
+ registry.ensure();
|
|
|
|
|
+
|
|
|
|
|
+ svapi::SettingsStore settings(db);
|
|
|
|
|
+ settings.bootstrap(envOr("SMARTBOTIC_VECTORAPI_KEY"), envOr("OPENAI_API_KEY"));
|
|
|
|
|
+ settings.startWatch();
|
|
|
|
|
+
|
|
|
|
|
+ svapi::SessionStore sessions(std::chrono::minutes(settings.snapshot()->sessionTtlMinutes));
|
|
|
|
|
+
|
|
|
|
|
+ svapi::ServerDeps deps{db, settings, registry, sessions, cfg.webuiDir, shareDir};
|
|
|
|
|
+ svapi::ApiServer server(std::move(deps));
|
|
|
|
|
+ server.registerRoutes();
|
|
|
|
|
+
|
|
|
|
|
+ std::thread http([&] {
|
|
|
|
|
+ spdlog::info("listening on {}:{}", cfg.httpBind, cfg.httpPort);
|
|
|
|
|
+ if (!server.listen(cfg.httpBind, cfg.httpPort)) {
|
|
|
|
|
+ spdlog::error("failed to bind {}:{}", cfg.httpBind, cfg.httpPort);
|
|
|
|
|
+ g_running = false;
|
|
|
|
|
+ }
|
|
|
|
|
+ });
|
|
|
|
|
+
|
|
|
|
|
+ sd_notify(0, "READY=1");
|
|
|
|
|
+ while (g_running) {
|
|
|
|
|
+ sd_notify(0, "WATCHDOG=1");
|
|
|
|
|
+ std::this_thread::sleep_for(std::chrono::milliseconds(200));
|
|
|
|
|
+ }
|
|
|
|
|
+ sd_notify(0, "STOPPING=1");
|
|
|
|
|
+ server.stop();
|
|
|
|
|
+ if (http.joinable()) http.join();
|
|
|
|
|
+ return 0;
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Build**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)"`
|
|
|
|
|
+Expected: builds clean.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Manual end-to-end smoke (DB must be running)**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# Run with a writable share dir pointing at the repo's api/ files:
|
|
|
|
|
+./build/src/smartbotic-vectorapi --config config/config.json --share-dir "$PWD/api" &
|
|
|
|
|
+SVAPI_PID=$!
|
|
|
|
|
+sleep 1
|
|
|
|
|
+# Grab the auto-generated key from the log line, or set SMARTBOTIC_VECTORAPI_KEY beforehand.
|
|
|
|
|
+curl -s localhost:8080/healthz; echo
|
|
|
|
|
+curl -s localhost:8080/openapi.json | head -c 60; echo
|
|
|
|
|
+kill $SVAPI_PID
|
|
|
|
|
+```
|
|
|
|
|
+Expected: `{"status":"ok"}` and the start of the OpenAPI JSON. (For authed calls, restart with `SMARTBOTIC_VECTORAPI_KEY=test` and use `-H "Authorization: Bearer test"`.)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/main.cpp
|
|
|
|
|
+git commit -m "feat(main): full service wiring — config, DB connect, settings bootstrap, serve, signals"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 17: Author `openapi.json` and `llms.txt`
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `api/openapi.json` (full content)
|
|
|
|
|
+- Modify: `api/llms.txt` (full content)
|
|
|
|
|
+- Modify: `tests/test_api_integration.cpp` (append a parse/path assertion)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Append failing test to `tests/test_api_integration.cpp`**
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+TEST_F(ApiFixture, OpenApiHasCollectionsPath) {
|
|
|
|
|
+ auto r = httpNoAuth().Get("/openapi.json");
|
|
|
|
|
+ ASSERT_TRUE(r);
|
|
|
|
|
+ ASSERT_EQ(r->status, 200);
|
|
|
|
|
+ auto j = nlohmann::json::parse(r->body); // must be valid JSON
|
|
|
|
|
+ EXPECT_EQ(j["openapi"], "3.1.0");
|
|
|
|
|
+ ASSERT_TRUE(j.contains("paths"));
|
|
|
|
|
+ EXPECT_TRUE(j["paths"].contains("/api/v1/collections"));
|
|
|
|
|
+ EXPECT_TRUE(j["paths"].contains("/api/v1/collections/{name}/search"));
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run to verify it fails**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiFixture.OpenApiHas" --output-on-failure`
|
|
|
|
|
+Expected: FAIL — the stub openapi.json has no paths.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Write the full `api/openapi.json`**
|
|
|
|
|
+
|
|
|
|
|
+```json
|
|
|
|
|
+{
|
|
|
|
|
+ "openapi": "3.1.0",
|
|
|
|
|
+ "info": {
|
|
|
|
|
+ "title": "smartbotic-vectorapi",
|
|
|
|
|
+ "version": "0.1.0",
|
|
|
|
|
+ "description": "General-purpose REST API over smartbotic-database for RAG vectors and JSON documents. n8n is the primary client, but any HTTP client may use it."
|
|
|
|
|
+ },
|
|
|
|
|
+ "servers": [{ "url": "/" }],
|
|
|
|
|
+ "components": {
|
|
|
|
|
+ "securitySchemes": {
|
|
|
|
|
+ "bearerAuth": { "type": "http", "scheme": "bearer" }
|
|
|
|
|
+ },
|
|
|
|
|
+ "schemas": {
|
|
|
|
|
+ "Error": {
|
|
|
|
|
+ "type": "object",
|
|
|
|
|
+ "properties": {
|
|
|
|
|
+ "error": {
|
|
|
|
|
+ "type": "object",
|
|
|
|
|
+ "properties": { "code": { "type": "string" }, "message": { "type": "string" } }
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ },
|
|
|
|
|
+ "security": [{ "bearerAuth": [] }],
|
|
|
|
|
+ "paths": {
|
|
|
|
|
+ "/healthz": {
|
|
|
|
|
+ "get": { "summary": "Liveness probe", "security": [], "responses": { "200": { "description": "alive" } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/readyz": {
|
|
|
|
|
+ "get": { "summary": "Readiness probe (DB connected)", "security": [], "responses": { "200": { "description": "ready" }, "503": { "description": "not ready" } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/api/v1/collections": {
|
|
|
|
|
+ "get": { "summary": "List managed collections", "responses": { "200": { "description": "list of collections with stats" } } },
|
|
|
|
|
+ "post": {
|
|
|
|
|
+ "summary": "Create a collection",
|
|
|
|
|
+ "requestBody": {
|
|
|
|
|
+ "required": true,
|
|
|
|
|
+ "content": { "application/json": { "schema": {
|
|
|
|
|
+ "type": "object",
|
|
|
|
|
+ "required": ["name", "kind"],
|
|
|
|
|
+ "properties": {
|
|
|
|
|
+ "name": { "type": "string", "description": "may not start with '_'" },
|
|
|
|
|
+ "kind": { "type": "string", "enum": ["json", "vector"] },
|
|
|
|
|
+ "vector_dimension": { "type": "integer", "description": "required when kind=vector" },
|
|
|
|
|
+ "embedding_model": { "type": "string", "description": "OpenAI model for text->vector (vector kind)" }
|
|
|
|
|
+ }
|
|
|
|
|
+ } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "responses": { "201": { "description": "created" }, "422": { "description": "validation error" } }
|
|
|
|
|
+ }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/api/v1/collections/{name}": {
|
|
|
|
|
+ "parameters": [{ "name": "name", "in": "path", "required": true, "schema": { "type": "string" } }],
|
|
|
|
|
+ "get": { "summary": "Get collection info + stats", "responses": { "200": { "description": "info" }, "404": { "description": "not found" } } },
|
|
|
|
|
+ "delete": { "summary": "Drop a collection", "responses": { "200": { "description": "deleted" }, "404": { "description": "not found" } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/api/v1/collections/{name}/documents": {
|
|
|
|
|
+ "parameters": [{ "name": "name", "in": "path", "required": true, "schema": { "type": "string" } }],
|
|
|
|
|
+ "get": {
|
|
|
|
|
+ "summary": "Find documents",
|
|
|
|
|
+ "parameters": [
|
|
|
|
|
+ { "name": "filter", "in": "query", "description": "repeatable field:op:value (op in eq,ne,gt,gte,lt,lte,in,contains,exists,regex,search)", "schema": { "type": "string" } },
|
|
|
|
|
+ { "name": "limit", "in": "query", "schema": { "type": "integer" } },
|
|
|
|
|
+ { "name": "offset", "in": "query", "schema": { "type": "integer" } },
|
|
|
|
|
+ { "name": "sort", "in": "query", "schema": { "type": "string" } },
|
|
|
|
|
+ { "name": "desc", "in": "query", "schema": { "type": "boolean" } }
|
|
|
|
|
+ ],
|
|
|
|
|
+ "responses": { "200": { "description": "matching documents" } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "post": { "summary": "Insert a JSON document", "responses": { "201": { "description": "created (returns id)" } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/api/v1/collections/{name}/documents/{id}": {
|
|
|
|
|
+ "parameters": [
|
|
|
|
|
+ { "name": "name", "in": "path", "required": true, "schema": { "type": "string" } },
|
|
|
|
|
+ { "name": "id", "in": "path", "required": true, "schema": { "type": "string" } }
|
|
|
|
|
+ ],
|
|
|
|
|
+ "get": { "summary": "Get a document", "responses": { "200": { "description": "document" }, "404": { "description": "not found" } } },
|
|
|
|
|
+ "put": { "summary": "Replace a document", "responses": { "200": { "description": "replaced" } } },
|
|
|
|
|
+ "patch": { "summary": "Merge fields into a document", "responses": { "200": { "description": "patched (returns version)" } } },
|
|
|
|
|
+ "delete": { "summary": "Delete a document", "responses": { "200": { "description": "deleted" } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/api/v1/collections/{name}/vectors": {
|
|
|
|
|
+ "parameters": [{ "name": "name", "in": "path", "required": true, "schema": { "type": "string" } }],
|
|
|
|
|
+ "post": {
|
|
|
|
|
+ "summary": "Store a RAG item (provide vector, or text to embed)",
|
|
|
|
|
+ "requestBody": {
|
|
|
|
|
+ "required": true,
|
|
|
|
|
+ "content": { "application/json": { "schema": {
|
|
|
|
|
+ "type": "object",
|
|
|
|
|
+ "properties": {
|
|
|
|
|
+ "id": { "type": "string" },
|
|
|
|
|
+ "text": { "type": "string", "description": "embedded via OpenAI when no vector is given" },
|
|
|
|
|
+ "vector": { "type": "array", "items": { "type": "number" } },
|
|
|
|
|
+ "metadata": { "type": "object" }
|
|
|
|
|
+ }
|
|
|
|
|
+ } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "responses": { "201": { "description": "stored (returns id)" }, "422": { "description": "dimension mismatch / validation" } }
|
|
|
|
|
+ }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/api/v1/collections/{name}/search": {
|
|
|
|
|
+ "parameters": [{ "name": "name", "in": "path", "required": true, "schema": { "type": "string" } }],
|
|
|
|
|
+ "post": {
|
|
|
|
|
+ "summary": "Similarity search (provide query_vector, or query_text to embed)",
|
|
|
|
|
+ "requestBody": {
|
|
|
|
|
+ "required": true,
|
|
|
|
|
+ "content": { "application/json": { "schema": {
|
|
|
|
|
+ "type": "object",
|
|
|
|
|
+ "properties": {
|
|
|
|
|
+ "query_text": { "type": "string" },
|
|
|
|
|
+ "query_vector": { "type": "array", "items": { "type": "number" } },
|
|
|
|
|
+ "top_k": { "type": "integer", "default": 5 },
|
|
|
|
|
+ "min_score": { "type": "number", "default": 0.0 }
|
|
|
|
|
+ }
|
|
|
|
|
+ } } }
|
|
|
|
|
+ },
|
|
|
|
|
+ "responses": { "200": { "description": "ranked results [{id, score, data}]" } }
|
|
|
|
|
+ }
|
|
|
|
|
+ },
|
|
|
|
|
+ "/api/v1/stats": {
|
|
|
|
|
+ "get": { "summary": "Database + memory statistics", "responses": { "200": { "description": "stats" } } }
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Write the full `api/llms.txt`**
|
|
|
|
|
+
|
|
|
|
|
+```text
|
|
|
|
|
+# smartbotic-vectorapi
|
|
|
|
|
+
|
|
|
|
|
+> A general-purpose REST API in front of smartbotic-database. Stores RAG embedding
|
|
|
|
|
+> vectors (with cosine similarity search) and arbitrary JSON documents. n8n is the
|
|
|
|
|
+> primary client, but any HTTP client may use it. Auth: send `Authorization: Bearer <API_KEY>`.
|
|
|
|
|
+
|
|
|
|
|
+## Conventions
|
|
|
|
|
+
|
|
|
|
|
+- Base path: `/api/v1`. All `/api/v1/*` routes require the bearer key.
|
|
|
|
|
+- Collections are admin-managed: create one before writing to it. Names may not start with `_`.
|
|
|
|
|
+- A collection is either `json` (documents) or `vector` (RAG; fixed `vector_dimension`).
|
|
|
|
|
+- Errors: `{ "error": { "code": "<slug>", "message": "<text>" } }` with standard HTTP status.
|
|
|
|
|
+- Full machine-readable spec: `/openapi.json`.
|
|
|
|
|
+
|
|
|
|
|
+## Collections
|
|
|
|
|
+
|
|
|
|
|
+- `POST /api/v1/collections` — create. Body: `{name, kind:"json"|"vector", vector_dimension?, embedding_model?}`.
|
|
|
|
|
+- `GET /api/v1/collections` — list with document counts and sizes.
|
|
|
|
|
+- `GET /api/v1/collections/{name}` — info for one collection.
|
|
|
|
|
+- `DELETE /api/v1/collections/{name}` — drop it.
|
|
|
|
|
+
|
|
|
|
|
+## JSON documents
|
|
|
|
|
+
|
|
|
|
|
+- `POST /api/v1/collections/{name}/documents` — insert. Body is the document, or `{id?, data:{...}}`. Returns `{id}`.
|
|
|
|
|
+- `GET /api/v1/collections/{name}/documents/{id}` — fetch one.
|
|
|
|
|
+- `GET /api/v1/collections/{name}/documents?filter=field:op:value&limit=&offset=&sort=&desc=` — find. `op` ∈ eq,ne,gt,gte,lt,lte,in,contains,exists,regex,search. `filter` is repeatable (AND).
|
|
|
|
|
+- `PUT /api/v1/collections/{name}/documents/{id}` — replace whole document.
|
|
|
|
|
+- `PATCH /api/v1/collections/{name}/documents/{id}` — merge fields.
|
|
|
|
|
+- `DELETE /api/v1/collections/{name}/documents/{id}` — delete.
|
|
|
|
|
+
|
|
|
|
|
+## RAG vectors
|
|
|
|
|
+
|
|
|
|
|
+- `POST /api/v1/collections/{name}/vectors` — store. Body: `{id?, text?, vector?, metadata?}`. Provide `vector` to store as-is, or `text` to have it embedded (OpenAI) using the collection's model. The vector length must equal the collection's `vector_dimension`.
|
|
|
|
|
+- `POST /api/v1/collections/{name}/search` — retrieve. Body: `{query_text?|query_vector?, top_k=5, min_score=0.0}`. Returns `{results:[{id, score, data}]}` ordered by descending cosine similarity.
|
|
|
|
|
+
|
|
|
|
|
+## Ops
|
|
|
|
|
+
|
|
|
|
|
+- `GET /api/v1/stats` — document/collection counts, memory pressure.
|
|
|
|
|
+- `GET /healthz` — liveness (no auth). `GET /readyz` — readiness (no auth).
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Build + run + full suite**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build --output-on-failure`
|
|
|
|
|
+Expected: all unit tests PASS; integration tests PASS (or SKIP if the DB is down).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add api/openapi.json api/llms.txt tests/test_api_integration.cpp
|
|
|
|
|
+git commit -m "docs(api): author openapi.json + llms.txt for client/agent discovery"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Backend definition of done
|
|
|
|
|
+
|
|
|
|
|
+- `cmake -B build -G Ninja -DBUILD_WEBUI=OFF && cmake --build build` succeeds.
|
|
|
|
|
+- `ctest --test-dir build` is green (integration tests skip cleanly without a DB).
|
|
|
|
|
+- The binary serves `/healthz`, `/openapi.json`, `/llms.txt`, the full `/api/v1` surface, and `/ui/login`.
|
|
|
|
|
+- First run with no stored key logs the generated key exactly once; `SMARTBOTIC_VECTORAPI_KEY` / `OPENAI_API_KEY` seed on first boot.
|
|
|
|
|
+- Changing settings in the DB hot-reloads (verify by editing `_vectorapi_settings` and observing a new snapshot without restart).
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Self-Review
|
|
|
|
|
+
|
|
|
|
|
+**1. Spec coverage** (against `2026-05-31-smartbotic-vectorapi-design.md`):
|
|
|
|
|
+
|
|
|
|
|
+| Spec section | Task(s) |
|
|
|
|
|
+|---|---|
|
|
|
|
|
+| §3 components: config | Task 2 |
|
|
|
|
|
+| §3 db_gateway | Task 5 |
|
|
|
|
|
+| §3 settings_store + §4.2 hot reload + §4.3 bootstrap | Tasks 6, 7, 16 |
|
|
|
|
|
+| §3 collection_registry + §5 data model | Task 8 |
|
|
|
|
|
+| §3 embeddings + §1 OpenAI generation | Task 9, 14 |
|
|
|
|
|
+| §3 auth + §5 bearer/cookie | Task 10, 11 |
|
|
|
|
|
+| §6.1 collections | Task 12 |
|
|
|
|
|
+| §6.2 documents CRUD + find | Task 13 |
|
|
|
|
|
+| §6.3 vectors + search | Task 14 |
|
|
|
|
|
+| §6.4 stats/health/meta + §10 openapi/llms | Tasks 11, 15, 17 |
|
|
|
|
|
+| §6.5 webui login/logout | Task 11 |
|
|
|
|
|
+| §6.6 error mapping | Task 3 (+ used everywhere) |
|
|
|
|
|
+| §1 client-agnostic / CORS | Task 11 (CORS), Task 17 (docs) |
|
|
|
|
|
+| §11 testing | every task (unit + integration) |
|
|
|
|
|
+
|
|
|
|
|
+Not in this plan (deferred to plans 2 & 3, as designed): React web UI (§7), CMake `WebUI.cmake`/`npm` (§8), packaging/systemd/`.deb` (§9). The root `CMakeLists.txt` already references `WebUI.cmake` with `OPTIONAL` so its absence does not break the backend build.
|
|
|
|
|
+
|
|
|
|
|
+**2. Placeholder scan:** Every code/test step contains complete code. The only intentional stubs are the four handler files in Task 11 (collections/documents/vectors/stats), each replaced in Tasks 12–15; `webui_auth.cpp` and `meta.cpp` are implemented in Task 11.
|
|
|
|
|
+
|
|
|
|
|
+**3. Type consistency:** `Config` (`logLevel/httpBind/httpPort/dbAddress/webuiDir`), `Settings` fields, `DbGateway::client()`, `ServerDeps{db,settings,registry,sessions,webuiDir,shareDir}`, `ApiServer` methods, and the `registerMetaRoutes/registerWebuiAuthRoutes/registerCollectionRoutes/registerDocumentRoutes/registerVectorRoutes/registerStatsRoutes` names are used identically in `server.cpp`, the handler files, the fixture, and `main.cpp`. `CollectionMeta` and `parseFilter`/`opFromSlug` signatures match across tasks.
|
|
|
|
|
+
|
|
|
|
|
+
|