docs(spec): capability-scoped API keys design (publishable keys + rate-limit + CAPTCHA)

docs/superpowers/specs/2026-06-15-scoped-api-keys-design.md

@@ -0,0 +1,223 @@
+# Design: capability-scoped API keys (safe-to-expose "publishable" keys)
+
+**Date:** 2026-06-15
+**Status:** approved for implementation (MVP + CAPTCHA in one spec)
+**Motivation:** a backend-less browser SPA (`/data/furniture_designer`, the
+Mega-Tolóajtó configurator) must call this API directly. Today the bearer token
+*is* a full-project read+write API key — embedding it in client JS exposes a
+powerful, long-lived credential. We need a credential that is **safe to expose**:
+narrow enough that extracting it from the browser causes negligible harm.
+
+## Problem
+
+- `Authorization: Bearer <token>` is the raw API key (`server.cpp resolveKey`),
+  granting read **and** write on every project in `key.projects` with no expiry,
+  rate limit, or origin restriction.
+- The SPA needs, against project `megatoloajto`: **read** `catalog_*` / `presets`
+  / `settings`; **read-by-id + insert** `designs` (doc id = share secret);
+  **insert-only** `quotes` (lead capture — contains PII); later **search**
+  `kb_assistant`.
+- Hard truth: a public client cannot hide a secret. The fix is not obfuscation
+  and not public-key crypto (a public client can't hold a private key secretly) —
+  it is **capability-scoping** the key so it is harmless when extracted, plus
+  **rate-limiting** and (for write-spam) **CAPTCHA**.
+
+## Goals
+
+- A key can be restricted to a subset of collections × operations within its
+  granted project(s), with an origin allowlist, an expiry, and a per-key rate
+  limit. Optionally require a verified human (CAPTCHA) token on chosen operations.
+- 100% backward compatible: a key with no `scope` behaves exactly as today.
+- All enforcement native C++ (no scripting engine), reusable for any future
+  public / third-party client.
+
+## Non-goals
+
+- No embedded JS runtime (QuickJS) — a scripting engine on the auth boundary is
+  unjustified surface for a fixed policy.
+- No end-user accounts / OAuth (the configurator is anonymous lead-capture).
+- No distributed/shared rate-limit store — in-process, single-instance (rag001 is
+  one host). A shared store is a future extension.
+- No external edge proxy — the API enforces scope itself; the client talks to it
+  directly with a harmless key.
+
+## Data model
+
+Extend `ApiKey` (`src/apikey.hpp`) with an optional scope. Stored in the
+`vectorapi_keys` doc body (via `toJson`); surfaced by `toPublicJson` (not secret).
+
+```cpp
+enum class KeyOp { Read, List, Search, Insert, Update, Delete };   // documents/vectors/search ops
+
+struct KeyScopeRule {
+    std::string collection;            // exact ("presets") or prefix glob ("catalog_*")
+    std::set<KeyOp> ops;
+    bool requireHumanToken = false;    // CAPTCHA required for ops matched by this rule
+};
+
+struct KeyScope {
+    std::vector<KeyScopeRule> rules;
+    std::vector<std::string> origins;  // allowlist; empty = any origin
+    uint32_t rateLimitPerMin = 0;      // 0 = unlimited
+    uint64_t expiresAt       = 0;      // epoch seconds; 0 = never
+};
+
+struct ApiKey {
+    // ... existing: id, key, label, projects, admin, createdAt ...
+    std::optional<KeyScope> scope;     // absent ⇒ legacy full-access behavior
+};
+```
+
+JSON shape (create/patch body and `toPublicJson`):
+```json
+"scope": {
+  "rules": [
+    { "collection": "catalog_*", "ops": ["read","list"] },
+    { "collection": "designs",   "ops": ["read","insert"] },
+    { "collection": "quotes",    "ops": ["insert"], "require_human_token": true }
+  ],
+  "origins": ["https://konfig.megatoloajto.hu"],
+  "rate_limit_per_min": 120,
+  "expires_at": 0
+}
+```
+
+A scoped key is implicitly **non-admin**; setting both `admin:true` and `scope`
+is rejected at create/patch (422).
+
+## Operations ↔ routes (enforcement points)
+
+| Route (handler) | Op |
+|---|---|
+| `POST …/collections/{c}/search` (vectors.cpp) | `search` |
+| `POST …/collections/{c}/vectors` (vectors.cpp) | `insert` |
+| `POST …/collections/{c}/documents` (documents.cpp) | `insert` |
+| `GET …/collections/{c}/documents` find/list (documents.cpp) | `list` |
+| `GET …/collections/{c}/documents/{id}` (documents.cpp) | `read` |
+| `PUT …/documents/{id}` (documents.cpp) | `update` |
+| `PATCH …/documents/{id}` (documents.cpp) | `update` |
+| `DELETE …/documents/{id}` (documents.cpp) | `delete` |
+
+Collection-management (`/collections` create/list/delete), project, key, settings,
+and stats routes remain **admin-only** and are never granted to a scoped key.
+(`read` = get-by-id only; withholding `list` means a key cannot enumerate a
+collection — e.g. `quotes` insert-only with no read/list means leads are never
+exfiltrable, and `designs` read-by-id requires already knowing the share id.)
+
+## Enforcement
+
+New helper in `server.cpp` (alongside `requireProjectAccess`):
+
+```cpp
+void requireCapability(ServerDeps& d, const ApiKey& k, const httplib::Request& req,
+                       const std::string& project, const std::string& collection, KeyOp op);
+```
+Order of checks:
+1. `requireProjectAccess(k, project)` — existing project grant.
+2. If `!k.scope` → return (legacy full-access). Otherwise:
+3. **Expiry** is enforced earlier, at auth time — see below. (Re-checking here is cheap and defensive.)
+4. **Rule match**: find a rule whose `collection` matches (exact or prefix-glob)
+   and whose `ops` contains `op`. None → `403 forbidden`.
+5. **Origin**: if `scope.origins` non-empty, the request's `Origin` header must
+   equal one of them; else `403`. (Browser-only signal — defense in depth.)
+6. **Rate limit**: consume one token from the per-key bucket; empty → `429`.
+7. **CAPTCHA**: if the matched rule has `requireHumanToken`, verify the
+   `X-Captcha-Token` request header with the configured provider; failure → `403`.
+
+Handlers replace their `requireProjectAccess(k, project)` call with
+`requireCapability(d, k, req, project, collection, op)` at the points in the table.
+Legacy keys (no scope) are unaffected — steps 3–7 are skipped.
+
+**Expiry at auth time:** `resolveKey` returns `nullopt` for a key whose
+`scope.expiresAt != 0 && now > expiresAt`, so an expired key fails authentication
+uniformly (`401`) on every route, including the pre-routing gate.
+
+## Rate limiting
+
+New `src/rate_limiter.{hpp,cpp}`: a process-singleton, mutex-guarded
+fixed-window-per-minute counter keyed by **key id**. `allow(keyId, limitPerMin)`
+returns false when the window is exhausted; the handler then throws the new
+`ErrCode::TooManyRequests` → HTTP **429** (add to `errors.hpp` + `httpStatus`),
+with a `Retry-After` header. `limitPerMin == 0` ⇒ unlimited (always allow).
+In-process / single-instance (documented assumption).
+
+## CAPTCHA verification
+
+- Settings additions (`settings.{hpp,cpp}` + PUT handler, hot-reload):
+  `captchaProvider` (`""`=disabled | `"turnstile"` | `"hcaptcha"`),
+  `captchaSecret` (write-only, masked in GET like `openai_api_key`),
+  `captchaVerifyUrl` (default per provider, overridable).
+- A rule with `requireHumanToken:true` requires the request to carry
+  `X-Captcha-Token: <token>`. The server POSTs `{secret, response:token,
+  remoteip}` to the verify URL (outbound httplib client) and requires
+  `{"success": true}`. Missing/invalid/failed verify → `403 captcha_required`.
+  If `captchaProvider` is empty but a rule demands a token, fail closed (`403`,
+  logged) rather than silently allowing.
+- Single-use tokens ⇒ no caching. Adds one outbound round-trip to gated inserts
+  (acceptable for lead submission).
+
+## Key management API
+
+- `POST /api/v1/keys` and `PATCH /api/v1/keys/{id}` accept an optional `scope`
+  object (admin-only, as today). Validation (→ 422): reject `admin:true` + `scope`
+  together; every `ops` entry must be a known op; `rules` non-empty with non-empty
+  `collection`; numeric fields ≥ 0; `origins` are strings.
+- `toPublicJson` includes `scope`. The `vectorapi_keys` doc stores it.
+
+## The concrete `megatoloajto` public key
+
+```json
+POST /api/v1/keys
+{ "label": "megatoloajto-public-web", "projects": ["megatoloajto"], "admin": false,
+  "scope": {
+    "rules": [
+      { "collection": "catalog_*", "ops": ["read","list"] },
+      { "collection": "presets",   "ops": ["read","list"] },
+      { "collection": "settings",  "ops": ["read"] },
+      { "collection": "designs",   "ops": ["read","insert"] },
+      { "collection": "quotes",    "ops": ["insert"], "require_human_token": true },
+      { "collection": "kb_assistant", "ops": ["search"] }
+    ],
+    "origins": ["https://konfig.megatoloajto.hu"],
+    "rate_limit_per_min": 120, "expires_at": 0
+  } }
+```
+Also requires `konfig.megatoloajto.hu` in the global `cors_origins` setting so the
+browser's cross-origin fetch is permitted (CORS is separate from scope authz).
+
+## Security analysis / residual risks (state honestly)
+
+- A scoped key is still a **shared, exposed** credential; `Origin` is spoofable
+  off-browser, so **rate-limit + CAPTCHA do the real abuse prevention**, not
+  origin-pinning.
+- The app's `settings` collection doc must hold **no secrets** (a read-scoped
+  public key exposes it). Per the ROADMAP it's pricing/flags — confirm.
+- `designs` are readable by id to anyone with the share link (capability-URL
+  model, same exposure as today's shareable URL); never grant `list` on it.
+- In-process rate limit ⇒ per-instance; fine for single-host rag001.
+
+## Testing
+
+- Unit: scope JSON round-trip; glob + exact collection match; op gating
+  (allow/deny matrix); expiry (expired ⇒ auth fails); rate limiter
+  (window exhaustion + reset); CAPTCHA verify against a mock provider
+  (success/fail/missing-token/provider-disabled-but-required).
+- Integration (`test_api_integration.cpp`, mock upstreams): a scoped key —
+  allowed op succeeds; denied op → 403; denied collection → 403; bad Origin →
+  403; over-limit → 429; `quotes` insert without token → 403, with valid token →
+  201; legacy (unscoped) key unaffected.
+
+## Docs / UI sync
+
+- `api/openapi.json` + `api/llms.txt`: document the `scope` object on
+  create/patch keys, the new `429`, the `X-Captcha-Token` header, and the
+  CAPTCHA settings fields.
+- `webui` `KeysAdmin.tsx`: a scoped-key builder (collection/op rows, origins,
+  rate-limit, expiry, CAPTCHA toggle). May be split into a follow-on task.
+
+## Phasing within implementation
+
+1. **MVP:** `scope` model + `requireCapability` + route wiring + expiry +
+   origin allowlist + rate limiter (`429`) + key create/patch + tests + docs.
+2. **CAPTCHA:** `require_human_token` + settings + outbound verify + tests + docs.
+3. **webui** scoped-key builder.