|
@@ -0,0 +1,752 @@
|
|
|
|
|
+# Capability-Scoped API Keys — Implementation Plan
|
|
|
|
|
+
|
|
|
|
|
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
|
+
|
|
|
|
|
+**Goal:** Add capability-scoped ("publishable") API keys so a backend-less browser app can call the API directly with a credential that is harmless when extracted.
|
|
|
|
|
+
|
|
|
|
|
+**Architecture:** Extend `ApiKey` with an optional `scope` (collection×operation rules + origin allowlist + per-minute rate limit + expiry + per-rule CAPTCHA flag). A new `requireCapability()` authz helper replaces `requireProjectAccess()` at the document/vector/search routes. Keys without a scope behave exactly as today (fully backward-compatible). A process-singleton `RateLimiter` and a free `verifyCaptcha()` keep `ServerDeps` wiring untouched.
|
|
|
|
|
+
|
|
|
|
|
+**Tech Stack:** C++20, nlohmann/json, cpp-httplib (OpenSSL), spdlog, GoogleTest. Namespace `svapi`. Quoted includes, flat headers.
|
|
|
|
|
+
|
|
|
|
|
+**Spec:** `docs/superpowers/specs/2026-06-15-scoped-api-keys-design.md`
|
|
|
|
|
+
|
|
|
|
|
+**Design calls locked:** `PUT`/`PATCH` both map to op `Update`. Rate limiter is in-process / single-instance (fine for single-host rag001).
|
|
|
|
|
+
|
|
|
|
|
+**Build/test commands (use throughout):**
|
|
|
|
|
+```bash
|
|
|
|
|
+cmake -B build -G Ninja -DBUILD_TESTS=ON -DBUILD_WEBUI=OFF -S . && cmake --build build -j"$(nproc)"
|
|
|
|
|
+ctest --test-dir build --output-on-failure -R <regex>
|
|
|
|
|
+```
|
|
|
|
|
+Integration tests `GTEST_SKIP()` when the DB on `localhost:9004` is unreachable.
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## File structure
|
|
|
|
|
+
|
|
|
|
|
+- `src/apikey.hpp` / `src/apikey.cpp` — **modify**: `KeyOp` enum, `KeyScopeRule`, `KeyScope` (+ `findRule`/`originAllowed`/`expired`), `ApiKey::scope`, JSON (de)serialization, `keyOpFromString`/`keyOpToString`.
|
|
|
|
|
+- `src/errors.hpp` / `src/errors.cpp` — **modify**: add `ErrCode::TooManyRequests` → 429.
|
|
|
|
|
+- `src/rate_limiter.hpp` / `src/rate_limiter.cpp` — **create**: process-singleton fixed-window limiter.
|
|
|
|
|
+- `src/captcha.hpp` / `src/captcha.cpp` — **create**: `verifyCaptcha(settings, token, ip)` (Phase 2).
|
|
|
|
|
+- `src/settings.hpp` / `src/settings.cpp` / `src/handlers/settings.cpp` — **modify**: captcha provider/secret/verify-url (Phase 2).
|
|
|
|
|
+- `src/server.hpp` / `src/server.cpp` — **modify**: `requireCapability()`, expiry in `resolveKey`, `Retry-After` on 429.
|
|
|
|
|
+- `src/handlers/documents.cpp` — **modify**: `scoped()` takes a `KeyOp`; per-route op.
|
|
|
|
|
+- `src/handlers/vectors.cpp` — **modify**: `requireVectorCollection()` takes a `KeyOp`; search→`Search`, vectors→`Insert`.
|
|
|
|
|
+- `src/handlers/keys.cpp` — **modify**: accept/validate `scope` on POST/PATCH.
|
|
|
|
|
+- `src/key_store.hpp` / `src/key_store.cpp` — **modify**: `create`/`update` carry `scope`.
|
|
|
|
|
+- `src/CMakeLists.txt` — **modify**: add `rate_limiter.cpp`, `captcha.cpp`.
|
|
|
|
|
+- `tests/test_apikey.cpp` (**create**), `tests/test_rate_limiter.cpp` (**create**), `tests/test_captcha.cpp` (**create**), `tests/test_api_integration.cpp` (**modify**), `tests/test_settings.cpp` (**modify**), `tests/CMakeLists.txt` (**modify**).
|
|
|
|
|
+- `api/openapi.json`, `api/llms.txt` — **modify**: `scope`, `429`, `X-Captcha-Token`, captcha settings.
|
|
|
|
|
+- `webui/src/types/index.ts`, `webui/src/pages/KeysAdmin.tsx` — **modify** (Phase 3).
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+# Phase 1 — MVP (scope model, enforcement, rate limit)
|
|
|
|
|
+
|
|
|
|
|
+## Task 1: KeyOp enum + scope structs + helpers
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/apikey.hpp`, `src/apikey.cpp`; Create `tests/test_apikey.cpp`; Modify `tests/CMakeLists.txt`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Write failing tests** — create `tests/test_apikey.cpp`:
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "apikey.hpp"
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+
|
|
|
|
|
+TEST(KeyOp, StringRoundTrip) {
|
|
|
|
|
+ EXPECT_EQ(keyOpToString(KeyOp::Search), "search");
|
|
|
|
|
+ EXPECT_EQ(*keyOpFromString("insert"), KeyOp::Insert);
|
|
|
|
|
+ EXPECT_FALSE(keyOpFromString("bogus").has_value());
|
|
|
|
|
+}
|
|
|
|
|
+TEST(KeyScope, ExactAndGlobRuleMatch) {
|
|
|
|
|
+ KeyScope s;
|
|
|
|
|
+ s.rules.push_back({"catalog_*", {KeyOp::Read, KeyOp::List}, false});
|
|
|
|
|
+ s.rules.push_back({"quotes", {KeyOp::Insert}, false});
|
|
|
|
|
+ EXPECT_NE(s.findRule("catalog_doors", KeyOp::Read), nullptr); // glob
|
|
|
|
|
+ EXPECT_EQ(s.findRule("catalog_doors", KeyOp::Insert), nullptr); // op not granted
|
|
|
|
|
+ EXPECT_NE(s.findRule("quotes", KeyOp::Insert), nullptr); // exact
|
|
|
|
|
+ EXPECT_EQ(s.findRule("designs", KeyOp::Read), nullptr); // no rule
|
|
|
|
|
+ EXPECT_EQ(s.findRule("catalogX", KeyOp::Read), nullptr); // glob is prefix, needs the underscore
|
|
|
|
|
+}
|
|
|
|
|
+TEST(KeyScope, OriginAllowlist) {
|
|
|
|
|
+ KeyScope s; EXPECT_TRUE(s.originAllowed("https://anything")); // empty = any
|
|
|
|
|
+ s.origins = {"https://konfig.megatoloajto.hu"};
|
|
|
|
|
+ EXPECT_TRUE(s.originAllowed("https://konfig.megatoloajto.hu"));
|
|
|
|
|
+ EXPECT_FALSE(s.originAllowed("https://evil.example"));
|
|
|
|
|
+ EXPECT_FALSE(s.originAllowed("")); // absent Origin fails closed
|
|
|
|
|
+}
|
|
|
|
|
+TEST(KeyScope, Expiry) {
|
|
|
|
|
+ KeyScope s; EXPECT_FALSE(s.expired(1000)); // 0 = never
|
|
|
|
|
+ s.expiresAt = 500; EXPECT_TRUE(s.expired(1000)); EXPECT_FALSE(s.expired(499));
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Register the test** — in `tests/CMakeLists.txt`, follow the existing `add_executable(... gtest_main)` + `gtest_discover_tests(...)` pattern used for `test_apikey`'s siblings (read the file; add `test_apikey` linking `vectorapi_core` and GTest exactly like `test_settings`).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Run — expect FAIL (compile error: KeyOp undefined)**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake -B build -G Ninja -DBUILD_TESTS=ON -DBUILD_WEBUI=OFF -S . && cmake --build build -j"$(nproc)" 2>&1 | tail -5`
|
|
|
|
|
+Expected: compile error referencing `KeyOp` / `keyOpFromString`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Implement in `src/apikey.hpp`** — add near the top of `namespace svapi`, before `struct ApiKey`:
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <optional>
|
|
|
|
|
+#include <set>
|
|
|
|
|
+// ...
|
|
|
|
|
+enum class KeyOp { Read, List, Search, Insert, Update, Delete };
|
|
|
|
|
+std::optional<KeyOp> keyOpFromString(const std::string& s);
|
|
|
|
|
+std::string keyOpToString(KeyOp op);
|
|
|
|
|
+
|
|
|
|
|
+struct KeyScopeRule {
|
|
|
|
|
+ std::string collection; // exact ("presets") or prefix glob ("catalog_*")
|
|
|
|
|
+ std::set<KeyOp> ops;
|
|
|
|
|
+ bool requireHumanToken = false; // CAPTCHA required for ops matched by this rule
|
|
|
|
|
+};
|
|
|
|
|
+
|
|
|
|
|
+struct KeyScope {
|
|
|
|
|
+ std::vector<KeyScopeRule> rules;
|
|
|
|
|
+ std::vector<std::string> origins; // empty = any origin allowed
|
|
|
|
|
+ uint32_t rateLimitPerMin = 0; // 0 = unlimited
|
|
|
|
|
+ uint64_t expiresAt = 0; // epoch seconds; 0 = never
|
|
|
|
|
+
|
|
|
|
|
+ const KeyScopeRule* findRule(const std::string& collection, KeyOp op) const;
|
|
|
|
|
+ bool originAllowed(const std::string& origin) const;
|
|
|
|
|
+ bool expired(uint64_t nowEpochSec) const;
|
|
|
|
|
+
|
|
|
|
|
+ static KeyScope fromJson(const nlohmann::json& j);
|
|
|
|
|
+ nlohmann::json toJson() const;
|
|
|
|
|
+};
|
|
|
|
|
+```
|
|
|
|
|
+Add `std::optional<KeyScope> scope;` as the last member of `struct ApiKey`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5: Implement in `src/apikey.cpp`** — add:
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <optional>
|
|
|
|
|
+
|
|
|
|
|
+std::optional<KeyOp> keyOpFromString(const std::string& s) {
|
|
|
|
|
+ if (s == "read") return KeyOp::Read;
|
|
|
|
|
+ if (s == "list") return KeyOp::List;
|
|
|
|
|
+ if (s == "search") return KeyOp::Search;
|
|
|
|
|
+ if (s == "insert") return KeyOp::Insert;
|
|
|
|
|
+ if (s == "update") return KeyOp::Update;
|
|
|
|
|
+ if (s == "delete") return KeyOp::Delete;
|
|
|
|
|
+ return std::nullopt;
|
|
|
|
|
+}
|
|
|
|
|
+std::string keyOpToString(KeyOp op) {
|
|
|
|
|
+ switch (op) {
|
|
|
|
|
+ case KeyOp::Read: return "read"; case KeyOp::List: return "list";
|
|
|
|
|
+ case KeyOp::Search: return "search"; case KeyOp::Insert: return "insert";
|
|
|
|
|
+ case KeyOp::Update: return "update"; case KeyOp::Delete: return "delete";
|
|
|
|
|
+ }
|
|
|
|
|
+ return "";
|
|
|
|
|
+}
|
|
|
|
|
+static bool collMatches(const std::string& pat, const std::string& coll) {
|
|
|
|
|
+ if (!pat.empty() && pat.back() == '*')
|
|
|
|
|
+ return coll.compare(0, pat.size() - 1, pat, 0, pat.size() - 1) == 0;
|
|
|
|
|
+ return pat == coll;
|
|
|
|
|
+}
|
|
|
|
|
+const KeyScopeRule* KeyScope::findRule(const std::string& collection, KeyOp op) const {
|
|
|
|
|
+ for (const auto& r : rules)
|
|
|
|
|
+ if (collMatches(r.collection, collection) && r.ops.count(op)) return &r;
|
|
|
|
|
+ return nullptr;
|
|
|
|
|
+}
|
|
|
|
|
+bool KeyScope::originAllowed(const std::string& origin) const {
|
|
|
|
|
+ if (origins.empty()) return true;
|
|
|
|
|
+ for (const auto& o : origins) if (o == origin) return true;
|
|
|
|
|
+ return false;
|
|
|
|
|
+}
|
|
|
|
|
+bool KeyScope::expired(uint64_t nowEpochSec) const {
|
|
|
|
|
+ return expiresAt != 0 && nowEpochSec > expiresAt;
|
|
|
|
|
+}
|
|
|
|
|
+KeyScope KeyScope::fromJson(const nlohmann::json& j) {
|
|
|
|
|
+ KeyScope s;
|
|
|
|
|
+ if (j.contains("rules") && j["rules"].is_array()) {
|
|
|
|
|
+ for (const auto& rj : j["rules"]) {
|
|
|
|
|
+ KeyScopeRule r;
|
|
|
|
|
+ r.collection = rj.value("collection", "");
|
|
|
|
|
+ r.requireHumanToken = rj.value("require_human_token", false);
|
|
|
|
|
+ if (rj.contains("ops") && rj["ops"].is_array())
|
|
|
|
|
+ for (const auto& oj : rj["ops"])
|
|
|
|
|
+ if (auto op = keyOpFromString(oj.get<std::string>())) r.ops.insert(*op);
|
|
|
|
|
+ s.rules.push_back(std::move(r));
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+ if (j.contains("origins") && j["origins"].is_array())
|
|
|
|
|
+ s.origins = j["origins"].get<std::vector<std::string>>();
|
|
|
|
|
+ s.rateLimitPerMin = j.value("rate_limit_per_min", 0u);
|
|
|
|
|
+ s.expiresAt = j.value("expires_at", 0ull);
|
|
|
|
|
+ return s;
|
|
|
|
|
+}
|
|
|
|
|
+nlohmann::json KeyScope::toJson() const {
|
|
|
|
|
+ nlohmann::json rules_j = nlohmann::json::array();
|
|
|
|
|
+ for (const auto& r : rules) {
|
|
|
|
|
+ nlohmann::json ops_j = nlohmann::json::array();
|
|
|
|
|
+ for (KeyOp op : r.ops) ops_j.push_back(keyOpToString(op));
|
|
|
|
|
+ rules_j.push_back({{"collection", r.collection}, {"ops", ops_j},
|
|
|
|
|
+ {"require_human_token", r.requireHumanToken}});
|
|
|
|
|
+ }
|
|
|
|
|
+ return {{"rules", rules_j}, {"origins", origins},
|
|
|
|
|
+ {"rate_limit_per_min", rateLimitPerMin}, {"expires_at", expiresAt}};
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6: Run — expect PASS**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build --output-on-failure -R "KeyOp|KeyScope"`
|
|
|
|
|
+Expected: all green.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/apikey.hpp src/apikey.cpp tests/test_apikey.cpp tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(auth): KeyOp + KeyScope model with glob/op/origin/expiry helpers"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 2: Persist scope on ApiKey (fromJson/toJson/toPublicJson)
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/apikey.cpp`; Modify `tests/test_apikey.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Add failing test** to `tests/test_apikey.cpp`:
|
|
|
|
|
+
|
|
|
|
|
+```cpp
|
|
|
|
|
+TEST(ApiKey, ScopeRoundTripsThroughJson) {
|
|
|
|
|
+ ApiKey k; k.id = "k1"; k.key = "secretvalue"; k.projects = {"megatoloajto"};
|
|
|
|
|
+ KeyScope s; s.rules.push_back({"quotes", {KeyOp::Insert}, true});
|
|
|
|
|
+ s.origins = {"https://x"}; s.rateLimitPerMin = 120; s.expiresAt = 0;
|
|
|
|
|
+ k.scope = s;
|
|
|
|
|
+ ApiKey back = ApiKey::fromJson(k.toJson());
|
|
|
|
|
+ ASSERT_TRUE(back.scope.has_value());
|
|
|
|
|
+ EXPECT_NE(back.scope->findRule("quotes", KeyOp::Insert), nullptr);
|
|
|
|
|
+ EXPECT_TRUE(back.scope->findRule("quotes", KeyOp::Insert)->requireHumanToken);
|
|
|
|
|
+ EXPECT_EQ(back.scope->rateLimitPerMin, 120u);
|
|
|
|
|
+ // public json exposes scope but never the secret
|
|
|
|
|
+ auto pub = k.toPublicJson();
|
|
|
|
|
+ EXPECT_TRUE(pub.contains("scope"));
|
|
|
|
|
+ EXPECT_FALSE(pub.contains("key"));
|
|
|
|
|
+}
|
|
|
|
|
+TEST(ApiKey, NoScopeStaysAbsent) {
|
|
|
|
|
+ ApiKey k; k.id = "k2";
|
|
|
|
|
+ EXPECT_FALSE(ApiKey::fromJson(k.toJson()).scope.has_value());
|
|
|
|
|
+ EXPECT_FALSE(k.toPublicJson().contains("scope"));
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2: Run — expect FAIL**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cmake --build build -j"$(nproc)" && ctest --test-dir build -R "ApiKey" --output-on-failure`
|
|
|
|
|
+Expected: FAIL (scope not in JSON).
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3: Implement** — in `src/apikey.cpp`, edit the three JSON functions:
|
|
|
|
|
+
|
|
|
|
|
+In `fromJson`, before `return k;`:
|
|
|
|
|
+```cpp
|
|
|
|
|
+ if (j.contains("scope") && j["scope"].is_object()) k.scope = KeyScope::fromJson(j["scope"]);
|
|
|
|
|
+```
|
|
|
|
|
+In `toJson`, build then conditionally add scope:
|
|
|
|
|
+```cpp
|
|
|
|
|
+nlohmann::json ApiKey::toJson() const {
|
|
|
|
|
+ nlohmann::json j = {{"key_id", id}, {"key", key}, {"label", label}, {"projects", projects},
|
|
|
|
|
+ {"admin", admin}, {"created_at", createdAt}};
|
|
|
|
|
+ if (scope) j["scope"] = scope->toJson();
|
|
|
|
|
+ return j;
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+In `toPublicJson`, similarly:
|
|
|
|
|
+```cpp
|
|
|
|
|
+nlohmann::json ApiKey::toPublicJson() const {
|
|
|
|
|
+ nlohmann::json j = {{"id", id}, {"label", label}, {"projects", projects}, {"admin", admin},
|
|
|
|
|
+ {"created_at", createdAt},
|
|
|
|
|
+ {"key_prefix", key.substr(0, std::min<size_t>(8, key.size()))}};
|
|
|
|
|
+ if (scope) j["scope"] = scope->toJson();
|
|
|
|
|
+ return j;
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4: Run — expect PASS**; **Step 5: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/apikey.cpp tests/test_apikey.cpp
|
|
|
|
|
+git commit -m "feat(auth): (de)serialize ApiKey.scope; expose in public json"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 3: Add 429 TooManyRequests
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/errors.hpp`, `src/errors.cpp`, `tests/test_errors.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Failing test** — add to `tests/test_errors.cpp`:
|
|
|
|
|
+```cpp
|
|
|
|
|
+TEST(Errors, TooManyRequestsMapsTo429) {
|
|
|
|
|
+ EXPECT_EQ(svapi::httpStatus(svapi::ErrCode::TooManyRequests), 429);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 2: Run — expect FAIL** (`TooManyRequests` undefined).
|
|
|
|
|
+- [ ] **Step 3: Implement** — in `src/errors.hpp` add `TooManyRequests` to the enum:
|
|
|
|
|
+```cpp
|
|
|
|
|
+enum class ErrCode { BadRequest, Unauthorized, Forbidden, NotFound, Unprocessable, TooManyRequests, Unavailable, Internal };
|
|
|
|
|
+```
|
|
|
|
|
+In `src/errors.cpp` `httpStatus`, add the case `case ErrCode::TooManyRequests: return 429;`.
|
|
|
|
|
+- [ ] **Step 4: Run — expect PASS**; **Step 5: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/errors.hpp src/errors.cpp tests/test_errors.cpp
|
|
|
|
|
+git commit -m "feat(errors): add 429 TooManyRequests"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 4: RateLimiter (fixed window per minute)
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Create `src/rate_limiter.hpp`, `src/rate_limiter.cpp`, `tests/test_rate_limiter.cpp`; Modify `src/CMakeLists.txt`, `tests/CMakeLists.txt`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Failing test** — `tests/test_rate_limiter.cpp`:
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "rate_limiter.hpp"
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+TEST(RateLimiter, AllowsUpToLimitThenBlocks) {
|
|
|
|
|
+ RateLimiter rl; int64_t t = 1000; rl.setClockForTesting([&]{ return t; });
|
|
|
|
|
+ EXPECT_TRUE(rl.allow("k", 2));
|
|
|
|
|
+ EXPECT_TRUE(rl.allow("k", 2));
|
|
|
|
|
+ EXPECT_FALSE(rl.allow("k", 2)); // 3rd in same minute blocked
|
|
|
|
|
+ t += 60; // next window
|
|
|
|
|
+ EXPECT_TRUE(rl.allow("k", 2)); // resets
|
|
|
|
|
+}
|
|
|
|
|
+TEST(RateLimiter, ZeroMeansUnlimited) {
|
|
|
|
|
+ RateLimiter rl; for (int i=0;i<1000;i++) EXPECT_TRUE(rl.allow("k", 0));
|
|
|
|
|
+}
|
|
|
|
|
+TEST(RateLimiter, KeysAreIndependent) {
|
|
|
|
|
+ RateLimiter rl; int64_t t=1000; rl.setClockForTesting([&]{ return t; });
|
|
|
|
|
+ EXPECT_TRUE(rl.allow("a",1)); EXPECT_FALSE(rl.allow("a",1));
|
|
|
|
|
+ EXPECT_TRUE(rl.allow("b",1));
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 2: Register** `test_rate_limiter` in `tests/CMakeLists.txt` (same pattern as Task 1).
|
|
|
|
|
+- [ ] **Step 3: Run — expect FAIL** (header missing).
|
|
|
|
|
+- [ ] **Step 4: Implement** — `src/rate_limiter.hpp`:
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <cstdint>
|
|
|
|
|
+#include <functional>
|
|
|
|
|
+#include <mutex>
|
|
|
|
|
+#include <string>
|
|
|
|
|
+#include <unordered_map>
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+class RateLimiter {
|
|
|
|
|
+public:
|
|
|
|
|
+ RateLimiter();
|
|
|
|
|
+ bool allow(const std::string& keyId, uint32_t limitPerMin); // true if under limit; 0 = unlimited
|
|
|
|
|
+ void setClockForTesting(std::function<int64_t()> fn);
|
|
|
|
|
+private:
|
|
|
|
|
+ struct Window { int64_t minute = -1; uint32_t count = 0; };
|
|
|
|
|
+ std::unordered_map<std::string, Window> windows_;
|
|
|
|
|
+ std::function<int64_t()> now_; // epoch seconds
|
|
|
|
|
+ std::mutex mu_;
|
|
|
|
|
+};
|
|
|
|
|
+RateLimiter& rateLimiter(); // process singleton
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+`src/rate_limiter.cpp`:
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "rate_limiter.hpp"
|
|
|
|
|
+#include <chrono>
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+RateLimiter::RateLimiter()
|
|
|
|
|
+ : now_([]{ return std::chrono::duration_cast<std::chrono::seconds>(
|
|
|
|
|
+ std::chrono::system_clock::now().time_since_epoch()).count(); }) {}
|
|
|
|
|
+bool RateLimiter::allow(const std::string& keyId, uint32_t limitPerMin) {
|
|
|
|
|
+ if (limitPerMin == 0) return true;
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(mu_);
|
|
|
|
|
+ int64_t minute = now_() / 60;
|
|
|
|
|
+ auto& w = windows_[keyId];
|
|
|
|
|
+ if (w.minute != minute) { w.minute = minute; w.count = 0; }
|
|
|
|
|
+ if (w.count >= limitPerMin) return false;
|
|
|
|
|
+ ++w.count;
|
|
|
|
|
+ return true;
|
|
|
|
|
+}
|
|
|
|
|
+void RateLimiter::setClockForTesting(std::function<int64_t()> fn) {
|
|
|
|
|
+ std::lock_guard<std::mutex> lk(mu_); now_ = std::move(fn);
|
|
|
|
|
+}
|
|
|
|
|
+RateLimiter& rateLimiter() { static RateLimiter inst; return inst; }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 5: Add to build** — in `src/CMakeLists.txt` add `rate_limiter.cpp` to the `vectorapi_core` source list (after `embedding_cache.cpp`).
|
|
|
|
|
+- [ ] **Step 6: Run — expect PASS**; **Step 7: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/rate_limiter.hpp src/rate_limiter.cpp tests/test_rate_limiter.cpp src/CMakeLists.txt tests/CMakeLists.txt
|
|
|
|
|
+git commit -m "feat(auth): per-key fixed-window RateLimiter (singleton)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 5: Enforce expiry at authentication
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/server.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** In `src/server.cpp`, in `resolveKey(...)`, after a key is found and before returning it, drop expired scoped keys. Replace the two `return k;` paths so both go through an expiry gate. Concretely, wrap lookups:
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <chrono> // ensure present
|
|
|
|
|
+// helper local to this TU:
|
|
|
|
|
+static bool keyExpired(const ApiKey& k) {
|
|
|
|
|
+ if (!k.scope) return false;
|
|
|
|
|
+ auto now = (uint64_t)std::chrono::duration_cast<std::chrono::seconds>(
|
|
|
|
|
+ std::chrono::system_clock::now().time_since_epoch()).count();
|
|
|
|
|
+ return k.scope->expired(now);
|
|
|
|
|
+}
|
|
|
|
|
+std::optional<ApiKey> resolveKey(ServerDeps& d, const httplib::Request& req) {
|
|
|
|
|
+ if (auto it = req.headers.find("Authorization"); it != req.headers.end())
|
|
|
|
|
+ if (auto tok = bearerToken(it->second))
|
|
|
|
|
+ if (auto k = d.keys.lookup(*tok)) { if (!keyExpired(*k)) return k; return std::nullopt; }
|
|
|
|
|
+ if (auto it = req.headers.find("Cookie"); it != req.headers.end())
|
|
|
|
|
+ if (auto c = cookieValue(it->second, "svapi_session"))
|
|
|
|
|
+ if (auto keyVal = d.sessions.keyFor(*c, std::chrono::system_clock::now()))
|
|
|
|
|
+ if (auto k = d.keys.lookup(*keyVal)) { if (!keyExpired(*k)) return k; return std::nullopt; }
|
|
|
|
|
+ return std::nullopt;
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 2: Build** — `cmake --build build -j"$(nproc)" 2>&1 | tail -3` (expected: clean; integration coverage lands in Task 10).
|
|
|
|
|
+- [ ] **Step 3: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/server.cpp
|
|
|
|
|
+git commit -m "feat(auth): expired scoped keys fail authentication (401)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 6: requireCapability (project + scope + origin + rate-limit)
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/server.hpp`, `src/server.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** Declare in `src/server.hpp` next to the other auth helpers:
|
|
|
|
|
+```cpp
|
|
|
|
|
+void requireCapability(ServerDeps& d, const ApiKey& k, const httplib::Request& req,
|
|
|
|
|
+ const std::string& project, const std::string& collection, KeyOp op);
|
|
|
|
|
+```
|
|
|
|
|
+(Ensure `#include "apikey.hpp"` is present in server.hpp.)
|
|
|
|
|
+- [ ] **Step 2:** Implement in `src/server.cpp` (after `requireProjectAccess`). Include `"rate_limiter.hpp"`. CAPTCHA block is added in Phase 2 (Task 13) — leave the marked TODO comment only as a code anchor, not a logic gap.
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "rate_limiter.hpp"
|
|
|
|
|
+void requireCapability(ServerDeps& d, const ApiKey& k, const httplib::Request& req,
|
|
|
|
|
+ const std::string& project, const std::string& collection, KeyOp op) {
|
|
|
|
|
+ requireProjectAccess(k, project);
|
|
|
|
|
+ if (!k.scope) return; // legacy full-access key
|
|
|
|
|
+ const KeyScope& sc = *k.scope;
|
|
|
|
|
+ const KeyScopeRule* rule = sc.findRule(collection, op);
|
|
|
|
|
+ if (!rule)
|
|
|
|
|
+ throw ApiError(ErrCode::Forbidden, "forbidden", "key scope does not permit this operation");
|
|
|
|
|
+ std::string origin;
|
|
|
|
|
+ if (auto it = req.headers.find("Origin"); it != req.headers.end()) origin = it->second;
|
|
|
|
|
+ if (!sc.originAllowed(origin))
|
|
|
|
|
+ throw ApiError(ErrCode::Forbidden, "forbidden", "origin not allowed for this key");
|
|
|
|
|
+ if (!rateLimiter().allow(k.id, sc.rateLimitPerMin))
|
|
|
|
|
+ throw ApiError(ErrCode::TooManyRequests, "rate_limited", "rate limit exceeded");
|
|
|
|
|
+ // Phase 2 inserts the CAPTCHA check for rule->requireHumanToken here.
|
|
|
|
|
+ (void)d;
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 3:** Add `Retry-After` for 429 — in `registerRoutes()`'s exception handler, change the `ApiError` catch:
|
|
|
|
|
+```cpp
|
|
|
|
|
+catch (const ApiError& e) {
|
|
|
|
|
+ if (e.code == ErrCode::TooManyRequests) res.set_header("Retry-After", "60");
|
|
|
|
|
+ sendJson(res, httpStatus(e.code), errorBody(e.slug, e.what()));
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 4: Build** — `cmake --build build -j"$(nproc)" 2>&1 | tail -3` (expected: clean).
|
|
|
|
|
+- [ ] **Step 5: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/server.hpp src/server.cpp
|
|
|
|
|
+git commit -m "feat(auth): requireCapability (scope rule + origin + rate-limit) + Retry-After"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 7: Wire enforcement into the document routes
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/handlers/documents.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** Change the `scoped()` helper to take and enforce an op:
|
|
|
|
|
+```cpp
|
|
|
|
|
+std::string scoped(ServerDeps* d, const httplib::Request& req, int projIdx, int collIdx, KeyOp op) {
|
|
|
|
|
+ ApiKey k = requireKey(*d, req);
|
|
|
|
|
+ std::string project = req.matches[projIdx], name = req.matches[collIdx];
|
|
|
|
|
+ requireCapability(*d, k, req, project, name, op);
|
|
|
|
|
+ if (!d->registry.get(project, name)) throw ApiError(ErrCode::NotFound, "not_found", "no such collection");
|
|
|
|
|
+ return qualify(project, name);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 2:** Update each route's `scoped(...)` call with the op from the table:
|
|
|
|
|
+ - POST `/documents` → `scoped(d, req, 1, 2, KeyOp::Insert)`
|
|
|
|
|
+ - GET `/documents/{id}` → `KeyOp::Read`
|
|
|
|
|
+ - GET `/documents` (list) → `KeyOp::List`
|
|
|
|
|
+ - PATCH `/documents/{id}` → `KeyOp::Update`
|
|
|
|
|
+ - PUT `/documents/{id}` → `KeyOp::Update`
|
|
|
|
|
+ - DELETE `/documents/{id}` → `KeyOp::Delete`
|
|
|
|
|
+- [ ] **Step 3: Build** — `cmake --build build -j"$(nproc)" 2>&1 | tail -3`. **Step 4: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/handlers/documents.cpp
|
|
|
|
|
+git commit -m "feat(auth): enforce capability scope on document routes"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 8: Wire enforcement into vector/search routes
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/handlers/vectors.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** Change `requireVectorCollection(...)` to accept and enforce an op:
|
|
|
|
|
+```cpp
|
|
|
|
|
+CollectionMeta requireVectorCollection(ServerDeps* d, const httplib::Request& req,
|
|
|
|
|
+ std::string& projectOut, std::string& nameOut, KeyOp op) {
|
|
|
|
|
+ ApiKey k = requireKey(*d, req);
|
|
|
|
|
+ projectOut = req.matches[1]; nameOut = req.matches[2];
|
|
|
|
|
+ requireCapability(*d, k, req, projectOut, nameOut, op);
|
|
|
|
|
+ auto m = d->registry.get(projectOut, nameOut);
|
|
|
|
|
+ if (!m) throw ApiError(ErrCode::NotFound, "not_found", "no such collection");
|
|
|
|
|
+ if (m->kind != "vector") throw ApiError(ErrCode::Unprocessable, "not_vector", "collection is not a vector collection");
|
|
|
|
|
+ return *m;
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 2:** At the call sites: the `/vectors` POST handler passes `KeyOp::Insert`; the `/search` POST handler passes `KeyOp::Search`.
|
|
|
|
|
+- [ ] **Step 3: Build** — clean. **Step 4: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/handlers/vectors.cpp
|
|
|
|
|
+git commit -m "feat(auth): enforce capability scope on vector insert + search"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 9: Accept & validate scope on key create/patch
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/key_store.hpp`, `src/key_store.cpp`, `src/handlers/keys.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** Extend `KeyStore::create`/`update` signatures to carry scope:
|
|
|
|
|
+```cpp
|
|
|
|
|
+// key_store.hpp
|
|
|
|
|
+ApiKey create(const std::string& label, std::vector<std::string> projects, bool admin,
|
|
|
|
|
+ std::optional<KeyScope> scope = std::nullopt);
|
|
|
|
|
+bool update(const std::string& id, const std::optional<std::string>& label,
|
|
|
|
|
+ const std::optional<std::vector<std::string>>& projects,
|
|
|
|
|
+ const std::optional<bool>& admin,
|
|
|
|
|
+ const std::optional<std::optional<KeyScope>>& scope = std::nullopt);
|
|
|
|
|
+```
|
|
|
|
|
+In `key_store.cpp`, set `k.scope = scope;` on create; on update, when the outer optional is engaged, assign `target.scope = *scope;` (inner `std::nullopt` clears the scope). Follow the existing field-update pattern in that file.
|
|
|
|
|
+- [ ] **Step 2:** In `src/handlers/keys.cpp`, add a validation helper + parse in POST and PATCH:
|
|
|
|
|
+```cpp
|
|
|
|
|
+static std::optional<KeyScope> parseScope(const nlohmann::json& body, bool admin) {
|
|
|
|
|
+ if (!body.contains("scope")) return std::nullopt;
|
|
|
|
|
+ if (admin) throw ApiError(ErrCode::Unprocessable, "validation", "admin keys cannot be scoped");
|
|
|
|
|
+ if (!body["scope"].is_object()) throw ApiError(ErrCode::Unprocessable, "validation", "scope must be an object");
|
|
|
|
|
+ const auto& sj = body["scope"];
|
|
|
|
|
+ if (!sj.contains("rules") || !sj["rules"].is_array() || sj["rules"].empty())
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "scope.rules must be a non-empty array");
|
|
|
|
|
+ for (const auto& r : sj["rules"]) {
|
|
|
|
|
+ if (r.value("collection", "").empty())
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "each rule needs a collection");
|
|
|
|
|
+ if (!r.contains("ops") || !r["ops"].is_array() || r["ops"].empty())
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "each rule needs ops");
|
|
|
|
|
+ for (const auto& o : r["ops"])
|
|
|
|
|
+ if (!o.is_string() || !keyOpFromString(o.get<std::string>()))
|
|
|
|
|
+ throw ApiError(ErrCode::Unprocessable, "validation", "unknown op in scope");
|
|
|
|
|
+ }
|
|
|
|
|
+ return KeyScope::fromJson(sj);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+POST handler: after computing `admin`, `auto scope = parseScope(body, admin);` then `d->keys.create(label, projects, admin, scope)`.
|
|
|
|
|
+PATCH handler: if `body.contains("scope")`, compute the effective admin (the patched value if present, else the target's current admin) and pass `std::optional<std::optional<KeyScope>>{ parseScope(body, effectiveAdmin) }` to `update`; otherwise pass `std::nullopt` (leave unchanged).
|
|
|
|
|
+- [ ] **Step 3: Build** — clean. **Step 4: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/key_store.hpp src/key_store.cpp src/handlers/keys.cpp
|
|
|
|
|
+git commit -m "feat(keys): accept and validate scope on create/patch"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 10: Integration tests for scoped enforcement
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `tests/test_api_integration.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** Add tests using the existing `ApiFixture` (read it first for how it creates keys/collections and issues requests; reuse that helper style). Cover, with a scoped key granting only `{designs:[read,insert]}` + `{quotes:[insert]}`, origins `["https://app.test"]`, rate_limit 2/min:
|
|
|
|
|
+```cpp
|
|
|
|
|
+// pseudostructure — adapt to ApiFixture's request helpers:
|
|
|
|
|
+TEST_F(ApiFixture, ScopedKeyAllowsGrantedInsertDeniesUngranted) {
|
|
|
|
|
+ // create vector/json collections "designs" and "quotes" and "secret_other"
|
|
|
|
|
+ // create scoped key as above
|
|
|
|
|
+ // POST designs/documents with Origin: https://app.test -> 201
|
|
|
|
|
+ // GET designs/documents (list) -> 403 (no 'list')
|
|
|
|
|
+ // POST quotes/documents -> 201
|
|
|
|
|
+ // GET quotes/documents/<id> -> 403 (no 'read' on quotes)
|
|
|
|
|
+ // any op on secret_other -> 403 (no rule)
|
|
|
|
|
+}
|
|
|
|
|
+TEST_F(ApiFixture, ScopedKeyOriginAndRateLimit) {
|
|
|
|
|
+ // POST designs/documents with Origin: https://evil.test -> 403
|
|
|
|
|
+ // with allowed origin: 1st,2nd -> 201 ; 3rd within minute -> 429
|
|
|
|
|
+}
|
|
|
|
|
+TEST_F(ApiFixture, LegacyKeyUnaffectedByScopeChecks) {
|
|
|
|
|
+ // the fixture's normal (unscoped) key still does full CRUD -> 200/201
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+Implement the bodies concretely against the fixture's actual request API (e.g. `post(path, body, headers)` returning status + json). If the fixture lacks a header-passing request helper, add a minimal overload rather than skipping the Origin/429 assertions.
|
|
|
|
|
+- [ ] **Step 2: Run**
|
|
|
|
|
+```bash
|
|
|
|
|
+cmake --build build -j"$(nproc)" && ctest --test-dir build --output-on-failure -R "ApiFixture"
|
|
|
|
|
+```
|
|
|
|
|
+Expected: PASS (or GTEST_SKIP if DB down — note it explicitly).
|
|
|
|
|
+- [ ] **Step 3: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add tests/test_api_integration.cpp
|
|
|
|
|
+git commit -m "test(auth): scoped-key allow/deny, origin, and 429 enforcement"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 11: Docs sync (scope + 429)
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `api/openapi.json`, `api/llms.txt`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** In `api/openapi.json`: add an optional `scope` object to the `POST /api/v1/keys` and `PATCH /api/v1/keys/{id}` request bodies and to `ApiKeyPublic`; add a `429` response (`Too Many Requests`) to the `/search`, `/vectors`, and `/documents` operations. Keep edits surgical; verify `python3 -c "import json; json.load(open('api/openapi.json'))"`.
|
|
|
|
|
+- [ ] **Step 2:** In `api/llms.txt`: under "## Keys", document that a non-admin key may carry a `scope` (`rules[{collection,ops,require_human_token}]`, `origins`, `rate_limit_per_min`, `expires_at`) and that scoped keys are safe to embed in untrusted clients; note `429` on rate-limit.
|
|
|
|
|
+- [ ] **Step 3: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add api/openapi.json api/llms.txt
|
|
|
|
|
+git commit -m "docs(api): document key scope + 429"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+# Phase 2 — CAPTCHA on gated operations
|
|
|
|
|
+
|
|
|
|
|
+## Task 12: Captcha settings
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `src/settings.hpp`, `src/settings.cpp`, `src/handlers/settings.cpp`, `tests/test_settings.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Failing test** in `tests/test_settings.cpp`: defaults empty; round-trip `captcha_provider`/`captcha_verify_url` through fromJson/toJson; `captcha_secret` round-trips through fromJson/toJson.
|
|
|
|
|
+- [ ] **Step 2: Run — FAIL.**
|
|
|
|
|
+- [ ] **Step 3: Implement** — add to `struct Settings`:
|
|
|
|
|
+```cpp
|
|
|
|
|
+std::string captchaProvider; // "" disabled | "turnstile" | "hcaptcha"
|
|
|
|
|
+std::string captchaSecret; // masked in GET like openai_api_key
|
|
|
|
|
+std::string captchaVerifyUrl; // full verify endpoint URL
|
|
|
|
|
+```
|
|
|
|
|
+Wire `fromJson`/`toJson` (keys `captcha_provider`, `captcha_secret`, `captcha_verify_url`) and the PUT handler (merge all three; for `captcha_secret` only overwrite when a non-empty string is supplied, mirroring `openai_api_key`). In `GET /api/v1/settings`, erase `captcha_secret` and add `captcha_secret_set` (bool), mirroring the existing `openai_api_key` masking.
|
|
|
|
|
+- [ ] **Step 4: Run — PASS.** **Step 5: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/settings.hpp src/settings.cpp src/handlers/settings.cpp tests/test_settings.cpp
|
|
|
|
|
+git commit -m "feat(settings): captcha provider/secret/verify-url (secret masked)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 13: verifyCaptcha + wire into requireCapability
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Create `src/captcha.hpp`, `src/captcha.cpp`, `tests/test_captcha.cpp`; Modify `src/CMakeLists.txt`, `tests/CMakeLists.txt`, `src/server.cpp`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1: Failing test** — `tests/test_captcha.cpp` spins an in-process `httplib::Server` mock (pattern: `Embeddings.ClientHitsMockServer`) that returns `{"success":true}` for token `"good"` and `{"success":false}` otherwise; point `Settings.captchaVerifyUrl` at it.
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include <gtest/gtest.h>
|
|
|
|
|
+#include "captcha.hpp"
|
|
|
|
|
+#include "settings.hpp"
|
|
|
|
|
+#include <httplib.h>
|
|
|
|
|
+#include <thread>
|
|
|
|
|
+using namespace svapi;
|
|
|
|
|
+TEST(Captcha, VerifiesAgainstProvider) {
|
|
|
|
|
+ httplib::Server mock;
|
|
|
|
|
+ mock.Post("/verify", [](const httplib::Request& req, httplib::Response& res){
|
|
|
|
|
+ bool ok = req.body.find("good") != std::string::npos;
|
|
|
|
|
+ res.set_content(ok ? R"({"success":true})" : R"({"success":false})", "application/json");
|
|
|
|
|
+ });
|
|
|
|
|
+ int port = mock.bind_to_any_port("127.0.0.1");
|
|
|
|
|
+ std::thread t([&]{ mock.listen_after_bind(); });
|
|
|
|
|
+ while (!mock.is_running()) std::this_thread::sleep_for(std::chrono::milliseconds(5));
|
|
|
|
|
+ Settings s; s.captchaProvider = "turnstile";
|
|
|
|
|
+ s.captchaVerifyUrl = "http://127.0.0.1:" + std::to_string(port) + "/verify";
|
|
|
|
|
+ s.captchaSecret = "sek";
|
|
|
|
|
+ EXPECT_TRUE(verifyCaptcha(s, "good-token", "1.2.3.4"));
|
|
|
|
|
+ EXPECT_FALSE(verifyCaptcha(s, "bad-token", "1.2.3.4"));
|
|
|
|
|
+ EXPECT_FALSE(verifyCaptcha(s, "", "1.2.3.4")); // missing token
|
|
|
|
|
+ Settings off; EXPECT_FALSE(verifyCaptcha(off, "good", "")); // provider disabled => fail closed
|
|
|
|
|
+ mock.stop(); t.join();
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 2: Register** `test_captcha` in `tests/CMakeLists.txt`. **Run — FAIL.**
|
|
|
|
|
+- [ ] **Step 3: Implement** — `src/captcha.hpp`:
|
|
|
|
|
+```cpp
|
|
|
|
|
+#pragma once
|
|
|
|
|
+#include <string>
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+struct Settings;
|
|
|
|
|
+// Verify a CAPTCHA token. Returns false (fail-closed) on disabled provider,
|
|
|
|
|
+// empty token, network error, or a non-success provider response.
|
|
|
|
|
+bool verifyCaptcha(const Settings& s, const std::string& token, const std::string& remoteIp);
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+`src/captcha.cpp` (reuse `splitEmbeddingBase` shape for origin/path; do a plain httplib POST with `response`/`secret`/`remoteip` form-encoded; parse JSON `success`):
|
|
|
|
|
+```cpp
|
|
|
|
|
+#include "captcha.hpp"
|
|
|
|
|
+#include "settings.hpp"
|
|
|
|
|
+#include <httplib.h>
|
|
|
|
|
+#include <nlohmann/json.hpp>
|
|
|
|
|
+namespace svapi {
|
|
|
|
|
+bool verifyCaptcha(const Settings& s, const std::string& token, const std::string& remoteIp) {
|
|
|
|
|
+ if (s.captchaProvider.empty() || token.empty() || s.captchaVerifyUrl.empty()) return false;
|
|
|
|
|
+ std::string url = s.captchaVerifyUrl;
|
|
|
|
|
+ while (!url.empty() && url.back() == '/') url.pop_back();
|
|
|
|
|
+ auto schemeEnd = url.find("://");
|
|
|
|
|
+ std::size_t from = (schemeEnd == std::string::npos) ? 0 : schemeEnd + 3;
|
|
|
|
|
+ auto slash = url.find('/', from);
|
|
|
|
|
+ std::string origin = (slash == std::string::npos) ? url : url.substr(0, slash);
|
|
|
|
|
+ std::string path = (slash == std::string::npos) ? "/" : url.substr(slash);
|
|
|
|
|
+ httplib::Client cli(origin);
|
|
|
|
|
+ cli.set_connection_timeout(5); cli.set_read_timeout(10);
|
|
|
|
|
+ cli.enable_server_certificate_verification(true);
|
|
|
|
|
+ httplib::Params params{{"secret", s.captchaSecret}, {"response", token}, {"remoteip", remoteIp}};
|
|
|
|
|
+ auto res = cli.Post(path, params);
|
|
|
|
|
+ if (!res || res->status != 200) return false;
|
|
|
|
|
+ try { return nlohmann::json::parse(res->body).value("success", false); }
|
|
|
|
|
+ catch (...) { return false; }
|
|
|
|
|
+}
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 4: Add to build** — `captcha.cpp` to `src/CMakeLists.txt`.
|
|
|
|
|
+- [ ] **Step 5: Wire into `requireCapability`** — in `src/server.cpp` replace the Phase-1 anchor comment with:
|
|
|
|
|
+```cpp
|
|
|
|
|
+ if (rule->requireHumanToken) {
|
|
|
|
|
+ std::string token;
|
|
|
|
|
+ if (auto it = req.headers.find("X-Captcha-Token"); it != req.headers.end()) token = it->second;
|
|
|
|
|
+ auto snap = d.settings.snapshot();
|
|
|
|
|
+ if (!verifyCaptcha(*snap, token, req.remote_addr))
|
|
|
|
|
+ throw ApiError(ErrCode::Forbidden, "captcha_required", "human verification required");
|
|
|
|
|
+ }
|
|
|
|
|
+```
|
|
|
|
|
+Add `#include "captcha.hpp"` to `src/server.cpp`; remove the now-unneeded `(void)d;`.
|
|
|
|
|
+- [ ] **Step 6: Run — PASS** (`-R "Captcha"`). **Step 7: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/captcha.hpp src/captcha.cpp tests/test_captcha.cpp src/CMakeLists.txt tests/CMakeLists.txt src/server.cpp
|
|
|
|
|
+git commit -m "feat(auth): CAPTCHA verification on require_human_token operations"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+## Task 14: Integration test for CAPTCHA gate + docs
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `tests/test_api_integration.cpp`, `api/openapi.json`, `api/llms.txt`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** Add an `ApiFixture` test: scoped key with `{quotes:[insert], require_human_token:true}`, settings `captcha_*` pointed at an in-process mock; POST `quotes/documents` without `X-Captcha-Token` → 403; with a token the mock accepts → 201.
|
|
|
|
|
+- [ ] **Step 2:** Docs: in `api/openapi.json` add the `X-Captcha-Token` header parameter to the `POST /documents` + `/vectors` operations and the `captcha_*` fields to `PUT /api/v1/settings`; in `api/llms.txt` note `require_human_token` + the `X-Captcha-Token` header. Validate JSON.
|
|
|
|
|
+- [ ] **Step 3: Run + Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+cmake --build build -j"$(nproc)" && ctest --test-dir build --output-on-failure -R "ApiFixture"
|
|
|
|
|
+git add tests/test_api_integration.cpp api/openapi.json api/llms.txt
|
|
|
|
|
+git commit -m "test+docs(auth): CAPTCHA-gated insert"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+# Phase 3 — webui scoped-key builder
|
|
|
|
|
+
|
|
|
|
|
+## Task 15: KeysAdmin scoped-key UI
|
|
|
|
|
+
|
|
|
|
|
+**Files:** Modify `webui/src/types/index.ts`, `webui/src/pages/KeysAdmin.tsx`.
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1:** In `types/index.ts`, add a `scope` shape to the key type: `{ rules: {collection:string; ops:string[]; require_human_token?:boolean}[]; origins:string[]; rate_limit_per_min:number; expires_at:number }` (optional). No TS `enum`, no `any` (`erasableSyntaxOnly`).
|
|
|
|
|
+- [ ] **Step 2:** In `KeysAdmin.tsx` (read it first to match the existing create-key form/state pattern), add an optional "Scoped (publishable) key" section to the create form: a toggle that, when on, sets `admin=false` and reveals rule rows (collection text + op checkboxes read/list/search/insert/update/delete + a require-human-token checkbox), an origins input (comma-separated), a rate-limit number, and an expiry. Send `scope` in the create payload only when the toggle is on. Display each key's scope summary in the list.
|
|
|
|
|
+- [ ] **Step 3: Build**
|
|
|
|
|
+```bash
|
|
|
|
|
+cd webui && npm run build # tsc + vite; must be clean
|
|
|
|
|
+```
|
|
|
|
|
+- [ ] **Step 4: Commit**
|
|
|
|
|
+```bash
|
|
|
|
|
+git add webui/src/types/index.ts webui/src/pages/KeysAdmin.tsx
|
|
|
|
|
+git commit -m "feat(webui): scoped (publishable) key builder in KeysAdmin"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Provisioning note (post-merge, run once against prod)
|
|
|
|
|
+
|
|
|
|
|
+Create the project + the publishable key for the configurator (admin key required):
|
|
|
|
|
+```bash
|
|
|
|
|
+# POST /api/v1/projects {"name":"megatoloajto"}
|
|
|
|
|
+# POST /api/v1/keys -> the scoped key from the spec's "concrete megatoloajto key" section
|
|
|
|
|
+# add konfig.megatoloajto.hu to settings.cors_origins
|
|
|
|
|
+```
|
|
|
|
|
+This is operational, not part of the build.
|