smartbotic-vectorapi/api/llms.txt

# smartbotic-vectorapi

> General-purpose REST API over smartbotic-database. **Multi-project**: every
> collection and document lives under a named project supplied as a URL path
> segment (e.g. `/api/v1/projects/{project}/collections`). **Multi-key**: each
> API key carries a list of project grants (MySQL-style; `["*"]` for all); admin
> keys additionally manage projects and keys. Auth: `Authorization: Bearer <KEY>`.
> Machine-readable spec: `/openapi.json` (OpenAPI 3.1).

## Projects

Requires a valid key. Admin-only operations are noted.

- `GET  /api/v1/projects` - List projects the key may access (admin sees all).
- `POST /api/v1/projects` `{name}` - Create a project (admin).
- `GET  /api/v1/projects/{project}` - Get project info (collection/document counts).
- `DELETE /api/v1/projects/{project}` - Drop a project (admin; the `default` project cannot be dropped).

## Keys (admin)

All key-management endpoints require an admin key.

- `GET  /api/v1/keys` - List keys. Secret value is masked; returns `id`, `key_prefix`, `label`, `projects`, `admin`, `created_at`, and `scope` (if present). Use `id` to reference the key in PATCH/DELETE.
- `POST /api/v1/keys` `{label, projects:[], admin?, scope?}` - Generate a new key. Returns the secret key value **once** (store it immediately) plus the stable `id`. Pass a `scope` object (see below) to create a capability-scoped key.
- `PATCH /api/v1/keys/{id}` `{label?, projects?, admin?, scope?}` - Update grants for an existing key, identified by its `id` (not the secret). Pass `scope: null` to clear an existing scope.
- `DELETE /api/v1/keys/{id}` - Revoke a key by `id`. Active sessions using it are immediately invalidated.

### Capability-scoped keys

A non-admin key may carry a `scope` object that restricts what it can do. Scoped keys are designed to be safely embedded in untrusted clients (browser frontends, mobile apps, n8n workflows). A scoped key cannot manage collections or projects regardless of its project grants.

`scope` fields:

- `rules` (required, non-empty array) - Each rule specifies a `collection` name and the permitted `ops` array (`read`, `list`, `search`, `insert`, `update`, `delete`). A request succeeds only if a matching `{collection, op}` rule exists. A rule may also set `require_human_token: true` (see CAPTCHA below).
- `origins` (array of strings, default empty) - Allowed `Origin` header values. When non-empty, requests without a matching `Origin` header are rejected with 403 (**fails closed** - no origin header means denied). Use this to pin a key to a specific domain.
- `rate_limit_per_min` (integer, default 0 = unlimited) - Per-key request rate limit. When the limit is exceeded the server returns **429** with a `Retry-After: 60` header.
- `expires_at` (integer, default 0 = never) - Unix epoch seconds after which the key is treated as expired; auth returns 401.

### CAPTCHA-gated operations

A scope rule may set `require_human_token: true`. When this flag is active, the client must supply an `X-Captcha-Token` request header containing the CAPTCHA challenge response. The server verifies the token via the configured provider endpoint and returns **403** `captcha_required` if the token is absent, invalid, or if no provider is configured. Configure the provider via the settings API: `captcha_provider` (`"turnstile"` or `"hcaptcha"`), `captcha_secret` (server-side secret), and `captcha_verify_url` (full verify endpoint URL).

Example scope (insert-only, origin-pinned, rate-limited, expiring):
```json
{
  "rules": [{"collection": "designs", "ops": ["insert"]}],
  "origins": ["https://app.example.com"],
  "rate_limit_per_min": 60,
  "expires_at": 1893456000
}
```

## Collections

Project-scoped. A key must be granted the project.

- `POST   /api/v1/projects/{project}/collections` `{name, kind:"json"|"vector", vector_dimension?, embedding_model?}` - Create a collection. `vector_dimension` is required for `kind=vector`. `embedding_model` defaults to the server's `default_embedding_model` setting.
- `GET    /api/v1/projects/{project}/collections` - List collections with metadata (incl. `document_count`, `size_bytes`). Query params (all applied server-side): `q` (case-insensitive name substring filter), `sort` (`name`|`kind`|`documents`|`size`|`created_at`, default `name`), `desc` (bool).
- `GET    /api/v1/projects/{project}/collections/{name}` - Get collection metadata.
- `DELETE /api/v1/projects/{project}/collections/{name}` - Drop a collection and all its documents.

Collection names may not start with `_` or the reserved prefix `vectorapi_`.

## JSON documents

Arbitrary JSON CRUD under a `kind=json` collection.

- `POST  /api/v1/projects/{project}/collections/{name}/documents` `{data:{...}}` - Insert a document. Returns `{id}`. Optional `?ttl_seconds=N` query param sets an expiry; absent means no expiry.
- `GET   /api/v1/projects/{project}/collections/{name}/documents/{id}` - Fetch a document by ID.
- `PUT   /api/v1/projects/{project}/collections/{name}/documents/{id}` - Replace a document (full upsert).
- `PATCH /api/v1/projects/{project}/collections/{name}/documents/{id}` - Partially update a document (merge fields). Optional `?ttl_seconds=N`: **omitted leaves the existing expiry untouched**; `ttl_seconds=0` **clears** it (document becomes permanent) - do not assume 0 means "no change".
- `DELETE /api/v1/projects/{project}/collections/{name}/documents/{id}` - Delete a document. Returns **409** `relation_restricted` (with `error.details.impacts`) if a restrict relation has children referencing it; see delete-impact below to preview this first.
- `GET   /api/v1/projects/{project}/collections/{name}/documents` - Find documents.
- `GET   /api/v1/projects/{project}/collections/{name}/documents/{id}/delete-impact` - Preview what deleting this document would affect, without deleting it: `{would_be_blocked, impacts:[{relation, child_collection, child_field, on_delete, child_count, sample_child_ids, blocks}]}`. Read-only; follows the collection's read grant (a scoped key with a read rule may call it).

**Filter grammar**: `field:op:value`, repeatable and ANDed. On documents it is the `?filter=` query param; on vector search it is the `filters` body array. The same grammar, operators and semantics apply to both, because the database evaluates them.

Supported ops: `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `in`, `contains`, `exists`, `regex`, `search`.

- `in` - value is a JSON array, e.g. `colour:in:["red","yellow"]`.
- `contains` - **array membership**, not substring: it matches when the field is an array holding the value. `colour:contains:yellow` does NOT match the string `"yellow"`.
- `search` - substring match on a string field, e.g. `colour:search:ell` matches `"yellow"`.
- `regex` - regular-expression match.
- `exists` - `field:exists:true` / `field:exists:false`.

The value is parsed as JSON when it parses (numbers, booleans, arrays), otherwise it is taken as a string. Everything after the second `:` is the value, so a value may itself contain `:`. A malformed spec or unknown operator is rejected with **400** `bad_filter`.

Example: `?filter=age:gte:30&filter=name:eq:Alice`.

Additional query params: `limit` (default 20), `offset` (default 0), `sort` (field name), `desc` (bool).

## Indexes and facets

Project-scoped, under a collection. Index create/drop require a full (non-scoped) key with project access - scoped keys cannot manage indexes.

- `POST   /api/v1/projects/{project}/collections/{name}/indexes` `{field, unique?}` - Declare an index on `field`, backfilling existing rows so it's usable immediately. Idempotent: re-declaring reports `already_existed: true`. With `unique: true`, refuses with **409** `duplicate_values` (`error.details.duplicate_examples`, up to five colliding document ids) if the field already holds duplicate values.
- `GET    /api/v1/projects/{project}/collections/{name}/indexes` - List indexes: `{indexes:[{field, distinct_values, entries, unique}]}`.
- `DELETE /api/v1/projects/{project}/collections/{name}/indexes/{field}` - Drop an index.
- `GET    /api/v1/projects/{project}/collections/{name}/indexes/{field}/values` - Distinct values with counts: `{values:[{value, count}]}`. Index-dependent by design (not an error): returns an empty list for a field with no index and for array-valued fields, whose index keys are elements rather than values. Query params: `limit` (1-1000, default 100), `order` (`asc`|`desc`, default `asc`; use `order=desc&limit=1` for the maximum). Follows the collection's read grant.

## Relations (admin)

Relation names are per-project and **unqualified**: requests and responses always use the bare name; the internal `project:` prefix is never exposed. All relation endpoints require an admin key.

- `POST   /api/v1/projects/{project}/relations` `{name, child, child_field, parent, on_delete?, validate_on_write?}` - Create a relation: deleting a `parent` document affects `child` documents whose `child_field` references it. `on_delete` is one of `restrict` (default), `cascade`, `set_null`, `no_action`; anything else is 422.
- `GET    /api/v1/projects/{project}/relations` - List relations in the project.
- `GET    /api/v1/projects/{project}/relations/{name}` - Get a relation by name.
- `DELETE /api/v1/projects/{project}/relations/{name}` - Drop a relation by name.
- `PUT    /api/v1/projects/{project}/collections/{name}/relations-enforced` `{enforced}` - Enable/disable relation enforcement on a collection. **`relations_enforced` is read on the collection you are deleting FROM (the PARENT side of the relation), not on the child collection holding the reference.** Disabling it on the child has no effect on deletes issued against the parent; set it on the parent to let its deletes bypass restrict relations.

## RAG vectors

Requires a `kind=vector` collection. Vectors are stored with cosine-similarity indexing.

- `POST /api/v1/projects/{project}/collections/{name}/vectors` `{id?, text?, vector?, metadata?}` - Store a vector. Supply either `text` (the server embeds it, resolving the model as: the collection's `embedding_model` pin, else the project's embedding override, else the global default) or a pre-computed `vector` array. `vector` dimension must match the collection's `vector_dimension`. `metadata` is an arbitrary JSON object stored alongside the vector.
- `POST /api/v1/projects/{project}/collections/{name}/search` `{query_text?, query_vector?, top_k?, min_score?, filters?}` - Cosine-similarity search. Supply either `query_text` or `query_vector`; neither is **422**. `top_k` caps the results, **default 5**. `min_score` (0-1) drops low-confidence matches, default 0. `filters` is an array of `field:op:value` strings applied to the vector's metadata, using the grammar and operators above - metadata is stored at the top level of the document, so a vector written with `metadata:{"colour":"red"}` is filtered as `colour:eq:red`.

  Filtering is evaluated by the database, then intersected with the similarity ranking, so scores are identical with and without a filter and `min_score` means the same thing either way. The scan is bounded at 10000 candidates: within a collection of that size a filtered search returns exactly the true top-k, and beyond it a filter matching only very low-ranked vectors may return fewer.

  **Latency tip:** `query_text` triggers a synchronous server-side embedding call to the configured provider, which usually dominates request time (seconds for large models). Cosine search itself is sub-millisecond. If your client issues many searches (e.g. an LLM/n8n agent doing RAG), embed the query once on your side and pass `query_vector` to skip server-side embedding entirely.

  **Cache bypass:** Send `Cache-Control: no-store` on a `/vectors` or `/search` request to skip the server-side embedding cache - the text is re-embedded fresh and the result is not stored in the cache.

## Settings (admin)

All settings endpoints require an admin key.

- `GET  /api/v1/settings` - Return current settings as JSON. `openai_api_key` is masked: the field is absent and `openai_api_key_set` (bool) indicates whether a key is configured.
- `PUT  /api/v1/settings` `{openai_api_base?, openai_api_key?, default_embedding_model?, cors_origins?, session_ttl_minutes?, webui_enabled?, default_project?, embedding_connect_timeout_sec?, embedding_read_timeout_sec?, embedding_cache_size?, embedding_cache_ttl_sec?, embedding_cache_max_bytes?, embedding_cache_normalize?, embedding_client_pool_size?, captcha_provider?, captcha_secret?, captcha_verify_url?}` - Merge the supplied fields into the current settings and persist. Omit `openai_api_key` or `captcha_secret` (or pass an empty string) to keep the existing secret. Changes are hot-reloaded immediately; cache settings are applied instantly via `reconfigure()`.

## Project embedding settings

A project can point its embeddings at its own provider endpoint, key and model, which is how per-customer usage gets metered at the provider. These routes require a key granted the project (admin keys reach every project); capability-scoped publishable keys are refused.

Precedence for one embedding call: the collection's own `embedding_model` wins, then the project override, then the global default. The endpoint and key are the project override when set, otherwise the global ones.

- `GET  /api/v1/projects/{project}/settings/embeddings` - Return `{effective, inherited, overrides}`. Each carries `openai_api_base`, `default_embedding_model` and `openai_api_key_set` (bool). The API key itself is never returned. In `overrides`, an empty string means the field is inherited.
- `PUT  /api/v1/projects/{project}/settings/embeddings` `{openai_api_base?, openai_api_key?, default_embedding_model?}` - Omit a field to leave it unchanged; send `null` to clear the override and inherit the global value. An empty string is rejected with **422**. Setting `openai_api_base` without an `openai_api_key` for the same project is rejected with **422** `embedding_base_requires_key`, so the global credential is never sent to a foreign endpoint. A project key without a project base URL is allowed (own key, default endpoint).
- `DELETE /api/v1/projects/{project}/settings/embeddings` - Drop every override; the project inherits the global settings again.

## Stats

- `GET /api/v1/projects/{project}/stats` - Per-project stats (collections, document counts). Accessible to any key granted the project.
- `GET /api/v1/stats` - Server-wide stats including `memory_pressure_level`, an `embedding` object with cache telemetry (`cache_size`, `cache_capacity`, `cache_hits`, `cache_misses`, `cache_hit_ratio`, `cache_bytes`), and `db_client_version`/`db_client_commit` (the smartbotic-database client library the dynamic linker actually bound at process startup, not necessarily what the binary was compiled against (2.4.x and 2.11.x share a SONAME, so a mismatched library can load silently and only crash on a later restart)). Admin only.

## Admin

- `GET /api/v1/admin/readonly` - Get the server-wide (not per-project) read-only lock status: `{readonly, scope:"server", reason?}`. Reflects the database's own authoritative state - it can enter read-only by itself after a degraded/failed recovery, not only via the PUT below, so `readonly:true` does not necessarily mean an operator locked it; `reason` (DB-defined free text) is present only when non-empty. While locked, writes are rejected by the database and surfaced by vectorapi as **503**. Admin only.
- `PUT /api/v1/admin/readonly` `{readonly}` - Set the server-wide read-only lock. Admin only.

## Ops

Public (no auth required):

- `GET /healthz` - Liveness probe; always 200 while the process is running.
- `GET /readyz`  - Readiness probe; 200 when the database is reachable, 503 otherwise.
- `GET /openapi.json` - OpenAPI 3.1 machine-readable spec.
- `GET /llms.txt`     - This file.
- `GET /docs`         - Rendered API reference page (RapiDoc over the spec above).

Session login (used by the web UI):

- `POST /ui/login` `{key}` - Exchange an API key for an HttpOnly session cookie (`svapi_session`). The cookie may be used in place of the `Authorization: Bearer` header for all `/api/v1/*` endpoints.
- `POST /ui/logout` - Invalidate the session and clear the cookie.
- `GET  /ui/session` - Return `{ok, admin, projects}` for the caller's current session (cookie or bearer), or **401** when there is none. Sessions are held in memory, so a server restart invalidates every cookie; a client must call this before trusting any locally stored "logged in" state.