feat(v2.8.0): unfreeze post-creation settings; fix two silent-truncation bugs

Came out of auditing "what is actually left?" against the code instead of the
session narrative. That corrected two of my own claims and found a regression I
had introduced.

ListFiles.total_count reported the PAGE, not the total. The policy work set it
to files_size() to avoid leaking counts of files a principal cannot see; with
limit=5 on an 865-file project it reported 5, so paging file lists was
impossible. Now: the store's own total when no project is secured (free, and the
case for every deployment today), a recount over the whole matching set when one
is. has_more derives from the same number.

ViewManager::loadFromStore truncated at 100 views - a bare Query, and
Query::limit defaults to 100. View 101+ silently did not exist and the v2.4.2
re-key migration never reached it. Same trap as PolicyManager; limit=0 returns
nothing, not everything. test_views.cpp could never have caught this because it
only exercises applyProjection() and never builds a MemoryStore, so
test_view_manager_paging.cpp now does: 250 views, reload, spot-checks either
side of the old boundary. 7 of its assertions fail without the fix.

TTL is changeable after creation on all four axes that were frozen:
- optional ttl_seconds on UpdateRequest and PatchDocumentRequest. Presence is
  load-bearing: absent leaves the expiry alone, 0 CLEARS it, N sets N seconds
  from now. Client::patchWithTtl is an ABI-safe overload.
- defaultTtlSeconds on an existing collection, via createCollection. Always safe
  since it only affects future inserts, and 0 is a real value rather than
  "unset".
- SetFileTtl RPC + FileManager::setExpiryIn for an already-stored file.
  Project-scoped, so a leaked id cannot change someone else's retention, and an
  already-expired file cannot be resurrected by extending it.
- per-fileType retention at runtime via setDefaultTtl, persisted to
  retention.json with write-and-rename and layered over the config baseline so a
  restart keeps it.

vector_dimension recovery. It was unreachable after creation - createCollection
updated only maxVersions, and alterCollection can set it but has no RPC and zero
call sites - which permanently stranded collections created through the
pre-2.4.5 createCollection bug. Now settable only while the collection holds no
vectors, which is safe because extractVector() returns early when the dimension
is 0, so no vector of a conflicting dimension can exist. Refused once vectors
exist. Existing documents keep _vector embedded and are not indexed
retroactively.

ListCollections is project-scoped, mirroring ListViewsRequest. A collection name
is information about someone else's schema even when its contents are
unreachable.

Build provenance works now. BUILD_GIT_COMMIT was computed and passed as a
--build-arg since v2.4 with the ARG declared, but unlike BUILD_VERSION it was
never promoted to an ENV, so nothing consumed it and packages carried no
provenance. --version now prints "2.8.0 (commit 51f6b6f)". My earlier claim that
the CLAUDE.md note was stale was wrong - the note was right.

ctest 19/19, namespacing 44/44, policy 19/19, views and TLS/auth green.

CMakeLists.txt

@@ -7,6 +7,28 @@ cmake_minimum_required(VERSION 3.20)
 file(READ "${CMAKE_CURRENT_SOURCE_DIR}/VERSION" SMARTBOTIC_DB_VERSION)
 string(STRIP "${SMARTBOTIC_DB_VERSION}" SMARTBOTIC_DB_VERSION)
 
+# v2.8.0 — build provenance. BUILD_GIT_COMMIT was computed by build.sh and passed
+# as a Docker --build-arg since v2.4, and Dockerfile.build declared the ARG, but
+# nothing ever consumed it: unlike BUILD_VERSION it was never promoted to an ENV,
+# so no commit reached the binary or the package. `dpkg -s` showed a version and
+# nothing else, which is the same triage gap that made --version useless for the
+# whole 2.x line before v2.4.1.
+#
+# Prefer the environment (set by the Docker build), fall back to asking git, and
+# accept "unknown" rather than failing a build outside a checkout.
+if(DEFINED ENV{BUILD_GIT_COMMIT})
+    set(SMARTBOTIC_DB_GIT_COMMIT "$ENV{BUILD_GIT_COMMIT}")
+else()
+    execute_process(
+        COMMAND git -C "${CMAKE_CURRENT_SOURCE_DIR}" rev-parse --short HEAD
+        OUTPUT_VARIABLE SMARTBOTIC_DB_GIT_COMMIT
+        OUTPUT_STRIP_TRAILING_WHITESPACE
+        ERROR_QUIET)
+endif()
+if(NOT SMARTBOTIC_DB_GIT_COMMIT)
+    set(SMARTBOTIC_DB_GIT_COMMIT "unknown")
+endif()
+
 # Detect standalone vs embedded mode
 if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
     # Building as standalone project

client/include/smartbotic/database/client.hpp

@@ -175,6 +175,25 @@ public:
      * @param fields JSON object with fields to merge
      * @return new version number, or 0 on failure
      */
+    /**
+     * Patch, and change the document's expiry at the same time.
+     *
+     * v2.8.0. `ttlSeconds` of 0 CLEARS the expiry (the document becomes
+     * permanent); any other value sets it that many seconds from now. Use the
+     * three-argument patch() to leave the expiry untouched.
+     *
+     * Before 2.8.0 a TTL could only be set on insert/upsert, so changing one
+     * meant rewriting the whole document. Note the related fix in the same
+     * release: update() used to WIPE a document's expiry, because it carried
+     * createdAt and createdBy across from the stored document but not expiresAt.
+     *
+     * An OVERLOAD rather than a new parameter on the existing signature, so the
+     * exported symbol callers already link against is untouched.
+     */
+    uint64_t patchWithTtl(const std::string& collection, const std::string& id,
+                          const nlohmann::json& patch, uint32_t ttlSeconds,
+                          const std::string& actor = "");
+
     uint64_t patch(const std::string& collection, const std::string& id,
                    const nlohmann::json& fields, const std::string& actor = "");
 
@@ -821,6 +840,20 @@ public:
     [[nodiscard]] std::vector<uint8_t> downloadFile(const std::string& id);
     [[nodiscard]] std::optional<FileRecord> getFileInfo(const std::string& id);
     bool deleteFile(const std::string& id);
+    /**
+     * Set or clear the expiry of a file that is already stored.
+     *
+     * v2.8.0. A file's TTL was previously fixed at upload. `ttlSeconds` of 0
+     * clears the expiry so the file is kept indefinitely; any other value expires
+     * it that many seconds from now.
+     *
+     * Returns false if no such file exists in this client's project. An id owned
+     * by another project reads as absent, so it cannot be used to extend or
+     * shorten someone else's retention. An already-expired file cannot be
+     * resurrected by extending it.
+     */
+    bool setFileTtl(const std::string& id, uint32_t ttlSeconds);
+
     [[nodiscard]] FileListResult listFiles(const std::string& file_type = "",
                                             const std::string& related_id = "",
                                             uint32_t limit = 100, uint32_t offset = 0,

client/src/client.cpp

@@ -390,9 +390,13 @@ public:
     }
 
     uint64_t patch(const std::string& collection, const std::string& id,
-                   const nlohmann::json& fields, const std::string& actor) {
+                   const nlohmann::json& fields, const std::string& actor,
+                   std::optional<uint32_t> ttlSeconds = std::nullopt) {
         smartbotic::databasepb::PatchDocumentRequest request;
         request.set_collection(qualify(collection));
+        // v2.8.0 — presence distinguishes "leave the expiry alone" (absent) from
+        // "clear it" (0) from "set it" (N).
+        if (ttlSeconds.has_value()) request.set_ttl_seconds(*ttlSeconds);
         request.set_id(id);
         request.set_patch_json(fields.dump());
         if (!actor.empty()) {
@@ -898,6 +902,9 @@ public:
 
     std::vector<std::string> listCollections() {
         smartbotic::databasepb::ListCollectionsRequest request;
+        // v2.8.0 — scope the listing to this client's project, so a workspace
+        // never sees another workspace's collection names.
+        request.set_project(config_.project);
 
         smartbotic::databasepb::ListCollectionsResponse response;
         grpc::ClientContext context;
@@ -1776,6 +1783,24 @@ public:
         return response.deleted();
     }
 
+    bool setFileTtl(const std::string& id, uint32_t ttlSeconds) {
+        smartbotic::databasepb::SetFileTtlRequest request;
+        request.set_id(id);
+        request.set_project(config_.project);
+        request.set_ttl_seconds(ttlSeconds);
+
+        smartbotic::databasepb::SetFileTtlResponse response;
+        grpc::ClientContext context;
+        setDeadline(context);
+
+        auto status = stub_->SetFileTtl(&context, request, &response);
+        if (!status.ok()) {
+            spdlog::error("Client::setFileTtl failed: {}", status.error_message());
+            return false;
+        }
+        return response.updated();
+    }
+
     Client::FileListResult listFiles(const std::string& file_type,
                                       const std::string& related_id,
                                       uint32_t limit, uint32_t offset,
@@ -1903,6 +1928,12 @@ uint64_t Client::patch(const std::string& collection, const std::string& id,
     return impl_->patch(collection, id, fields, actor);
 }
 
+uint64_t Client::patchWithTtl(const std::string& collection, const std::string& id,
+                              const nlohmann::json& fields, uint32_t ttlSeconds,
+                              const std::string& actor) {
+    return impl_->patch(collection, id, fields, actor, ttlSeconds);
+}
+
 std::pair<std::string, bool> Client::upsert(const std::string& collection, const nlohmann::json& data,
                                                     const std::string& id, uint32_t ttlSeconds,
                                                     const std::string& actor) {
@@ -2106,6 +2137,10 @@ bool Client::deleteFile(const std::string& id) {
     return impl_->deleteFile(id);
 }
 
+bool Client::setFileTtl(const std::string& id, uint32_t ttlSeconds) {
+    return impl_->setFileTtl(id, ttlSeconds);
+}
+
 Client::FileListResult Client::listFiles(const std::string& file_type,
                                           const std::string& related_id,
                                           uint32_t limit, uint32_t offset,

packaging/Dockerfile.build

@@ -14,6 +14,10 @@ ARG BUILD_VERSION=0.0.0
 ARG BUILD_DEB_REVISION=1
 ARG BUILD_JOBS=16
 ARG BUILD_GIT_COMMIT=unknown
+# v2.8.0 — promote it to an ENV so the CMake build can bake it into the binary.
+# It was declared here since v2.4 and passed by build.sh, but never promoted, so
+# nothing consumed it and packages recorded no provenance.
+ENV BUILD_GIT_COMMIT=${BUILD_GIT_COMMIT}
 ENV DEBIAN_FRONTEND=noninteractive
 ENV BUILD_VERSION=${BUILD_VERSION}
 ENV BUILD_DEB_REVISION=${BUILD_DEB_REVISION}

proto/database.proto

@@ -81,6 +81,7 @@ service DatabaseService {
     rpc DeleteFile(DeleteFileRequest) returns (DeleteFileResponse);
     rpc GetFileInfo(GetFileInfoRequest) returns (FileInfo);
     rpc ListFiles(ListFilesRequest) returns (ListFilesResponse);
+    rpc SetFileTtl(SetFileTtlRequest) returns (SetFileTtlResponse);
 
     // Event subscription
     rpc Subscribe(SubscribeRequest) returns (stream DatabaseEvent);
@@ -156,6 +157,18 @@ message UpdateRequest {
     bytes data = 3;
     uint64 expected_version = 4;  // 0 = no version check (force update)
     string actor = 5;             // User ID performing the operation (for audit)
+
+    // v2.8.0 — change the document's expiry as part of the update.
+    //
+    // `optional` gives three distinct meanings, which a plain uint32 cannot:
+    //   absent  -> leave the existing expiry alone (the default, and what an
+    //              update did implicitly once it stopped WIPING the expiry)
+    //   0       -> clear the expiry: the document becomes permanent
+    //   N       -> expire N seconds from now
+    //
+    // Before this, ttl_seconds existed only on Insert/Upsert/BatchInsert, so
+    // changing a TTL meant rewriting the whole document via upsert.
+    optional uint32 ttl_seconds = 6;
 }
 
 message UpdateResponse {
@@ -192,6 +205,10 @@ message PatchDocumentRequest {
     string id = 2;
     bytes patch_json = 3;             // JSON fields to merge into existing document
     string actor = 4;                 // User ID performing the operation (for audit)
+
+    // v2.8.0 — same three-way semantics as UpdateRequest.ttl_seconds: absent
+    // leaves the expiry alone, 0 clears it, N sets N seconds from now.
+    optional uint32 ttl_seconds = 5;
 }
 
 message PatchDocumentResponse {
@@ -467,6 +484,12 @@ message DropCollectionResponse {
 }
 
 message ListCollectionsRequest {
+    // v2.8.0 — restrict the listing to one project namespace, mirroring
+    // ListViewsRequest. Empty lists every project (operator/CLI use); clients
+    // always set it, so a workspace never sees another workspace's collection
+    // names. A collection name is information about the shape of someone else's
+    // data even when its contents are unreachable.
+    string project = 1;
 }
 
 message ListCollectionsResponse {
@@ -618,6 +641,22 @@ message ListFilesRequest {
     string project = 7;              // v2.6.0 — empty means "default"
 }
 
+// v2.8.0 — change the expiry of a file that is already stored. A file's TTL was
+// otherwise fixed at upload.
+message SetFileTtlRequest {
+    string id = 1;
+    string project = 2;               // empty means "default"
+    // Absent is invalid here - the call exists to change the expiry, so it must
+    // say what to. 0 clears the expiry (keep indefinitely); N expires the file N
+    // seconds from now.
+    uint32 ttl_seconds = 3;
+}
+
+message SetFileTtlResponse {
+    bool updated = 1;                 // false if no such file in that project
+    uint64 expires_at = 2;            // the resulting absolute expiry, 0 = never
+}
+
 message ListFilesResponse {
     repeated FileInfo files = 1;
     uint64 total_count = 2;

service/CMakeLists.txt

@@ -84,6 +84,7 @@ target_include_directories(smartbotic-database PRIVATE
 # drift from the deb version again (it reported a hardcoded 1.0.0 for all of 2.x).
 target_compile_definitions(smartbotic-database PRIVATE
     SMARTBOTIC_DB_VERSION_STRING="${SMARTBOTIC_DB_VERSION}"
+    SMARTBOTIC_DB_GIT_COMMIT_STRING="${SMARTBOTIC_DB_GIT_COMMIT}"
 )
 
 # Link dependencies

service/src/database_grpc_impl.cpp

@@ -108,6 +108,21 @@ DatabaseGrpcImpl::DatabaseGrpcImpl(
 // function rather than a check per handler.
 // -------------------------------------------------------------------------
 
+
+// v2.8.0 — translate a request's optional ttl_seconds into an absolute expiry
+// override. Absent means "leave the stored expiry alone"; 0 means "clear it";
+// N means "N seconds from now". Presence is what makes the three cases
+// distinguishable, which a plain uint32 could not do.
+template <typename Req>
+static std::optional<uint64_t> expiryOverrideFrom(const Req& request) {
+    if (!request.has_ttl_seconds()) return std::nullopt;
+    if (request.ttl_seconds() == 0) return uint64_t{0};
+    const auto now = std::chrono::duration_cast<std::chrono::milliseconds>(
+        std::chrono::system_clock::now().time_since_epoch()).count();
+    return static_cast<uint64_t>(now) +
+           static_cast<uint64_t>(request.ttl_seconds()) * 1000ULL;
+}
+
 grpc::Status DatabaseGrpcImpl::gate(const grpc::ServerContext* context,
                                      const std::string& qualified,
                                      smartbotic::database::Access access,
@@ -544,7 +559,8 @@ grpc::Status DatabaseGrpcImpl::Update(
                 request->collection(),
                 request->id(),
                 doc,
-                request->expected_version()
+                request->expected_version(),
+                expiryOverrideFrom(*request)
             );
             if (!success) {
                 response->set_success(false);
@@ -621,7 +637,8 @@ grpc::Status DatabaseGrpcImpl::PatchDocument(
             request->collection(),
             request->id(),
             patch,
-            request->actor()
+            request->actor(),
+            expiryOverrideFrom(*request)
         );
 
         if (newVersion == 0) {
@@ -1609,14 +1626,28 @@ grpc::Status DatabaseGrpcImpl::DropCollection(
 
 grpc::Status DatabaseGrpcImpl::ListCollections(
     grpc::ServerContext* context,
-    const pb::ListCollectionsRequest* /*request*/,
+    const pb::ListCollectionsRequest* request,
     pb::ListCollectionsResponse* response
 ) {
     auto names = store_.listCollections();
-    // v2.8.0 — filter per entry rather than refusing the call. The mere
-    // existence of a collection is information: an enumeration that shows names
-    // a principal cannot read leaks the shape of the dataset.
+
+    // v2.8.0 — restrict to one project when the caller names one, mirroring
+    // ListViews. Empty means every project, which is the operator/CLI case.
+    const std::string& wantProject = request->project();
+
+    // Filter per entry rather than refusing the call. The mere existence of a
+    // collection is information: an enumeration that shows names a principal
+    // cannot read leaks the shape of the dataset.
     for (const auto& name : names) {
+        if (!wantProject.empty()) {
+            std::string owner;
+            try {
+                owner = smartbotic::database::resolveCollection(name).project;
+            } catch (const std::exception&) {
+                continue;   // unparseable name cannot be attributed; omit it
+            }
+            if (owner != wantProject) continue;
+        }
         smartbotic::database::Decision ldec;
         if (auto st = gate(context, name, smartbotic::database::Access::Read, ldec);
             !st.ok()) {
@@ -1966,6 +1997,49 @@ grpc::Status DatabaseGrpcImpl::GetFileInfo(
     return grpc::Status::OK;
 }
 
+grpc::Status DatabaseGrpcImpl::SetFileTtl(
+    grpc::ServerContext* context,
+    const pb::SetFileTtlRequest* request,
+    pb::SetFileTtlResponse* response
+) {
+    if (service_.isReadOnly()) {
+        return readOnlyStatus("SetFileTtl", service_);
+    }
+
+    // Changing retention is a WRITE, gated on the record's own file type - which
+    // is only known after the lookup. A denial is reported as "not updated"
+    // rather than distinguished from a missing file.
+    auto info = files_.getFileInfoIn(request->project(), request->id());
+    if (!info) {
+        response->set_updated(false);
+        return grpc::Status::OK;
+    }
+    {
+        smartbotic::database::Decision fdec;
+        if (auto st = gateFile(context, info->project, info->fileType,
+                               smartbotic::database::Access::Write, fdec);
+            !st.ok()) {
+            response->set_updated(false);
+            return grpc::Status::OK;
+        }
+    }
+
+    // Relative in, absolute stored - so a later read or a restart cannot extend
+    // the file's life. 0 clears the expiry.
+    uint64_t expiresAt = 0;
+    if (request->ttl_seconds() > 0) {
+        const auto now = std::chrono::duration_cast<std::chrono::milliseconds>(
+            std::chrono::system_clock::now().time_since_epoch()).count();
+        expiresAt = static_cast<uint64_t>(now) +
+                    static_cast<uint64_t>(request->ttl_seconds()) * 1000ULL;
+    }
+
+    const bool ok = files_.setExpiryIn(request->project(), request->id(), expiresAt);
+    response->set_updated(ok);
+    response->set_expires_at(ok ? expiresAt : 0);
+    return grpc::Status::OK;
+}
+
 grpc::Status DatabaseGrpcImpl::ListFiles(
     grpc::ServerContext* context,
     const pb::ListFilesRequest* request,
@@ -1979,8 +2053,6 @@ grpc::Status DatabaseGrpcImpl::ListFiles(
         request->checksum(), request->name()
     );
 
-    response->set_has_more(result.hasMore);
-
     for (const auto& info : result.files) {
         // v2.8.0 — filter per entry rather than refusing the whole listing, so a
         // principal granted one file type still gets a usable result. total_count
@@ -2007,10 +2079,42 @@ grpc::Status DatabaseGrpcImpl::ListFiles(
         file->set_expires_at(info.expiresAt);
     }
 
-    // v2.8.0 — total_count must reflect what the caller may actually see, not
-    // what the store holds. Reporting the unfiltered total would leak the number
-    // of files in file types this principal has no grant for.
-    response->set_total_count(static_cast<uint64_t>(response->files_size()));
+    // v2.8.0 — total_count must reflect what the caller may actually SEE, but it
+    // must still be a TOTAL, not the size of this page.
+    //
+    // The first cut set it to files_size(), which is the page: with limit=5 on a
+    // 865-file project it reported 5, so a client could not page at all. Fixed
+    // by distinguishing the two cases:
+    //
+    //   * nothing secured (the default, and every pre-2.7.0 deployment): no
+    //     entry was filtered, so the store's own total is exactly right and
+    //     costs nothing.
+    //   * a project IS secured: recount over the whole matching set, because a
+    //     filtered total cannot be derived from one page. Reporting the
+    //     unfiltered total instead would leak how many files exist in file types
+    //     this principal has no grant for.
+    if (!policy_manager_.anyProjectSecured()) {
+        response->set_total_count(result.totalCount);
+        response->set_has_more(
+            static_cast<uint64_t>(request->offset()) + response->files_size() <
+            result.totalCount);
+    } else {
+        const auto all = files_.listFiles(request->project(), request->file_type(),
+                                          request->related_id(),
+                                          /*limit=*/UINT32_MAX, /*offset=*/0,
+                                          request->checksum(), request->name());
+        uint64_t visible = 0;
+        for (const auto& info : all.files) {
+            smartbotic::database::Decision fdec;
+            if (gateFile(context, info.project, info.fileType,
+                         smartbotic::database::Access::Read, fdec).ok()) {
+                ++visible;
+            }
+        }
+        response->set_total_count(visible);
+        response->set_has_more(
+            static_cast<uint64_t>(request->offset()) + response->files_size() < visible);
+    }
     return grpc::Status::OK;
 }
 

service/src/database_grpc_impl.hpp

@@ -308,6 +308,12 @@ public:
         pb::FileInfo* response
     ) override;
 
+    grpc::Status SetFileTtl(
+        grpc::ServerContext* context,
+        const pb::SetFileTtlRequest* request,
+        pb::SetFileTtlResponse* response
+    ) override;
+
     grpc::Status ListFiles(
         grpc::ServerContext* context,
         const pb::ListFilesRequest* request,

service/src/files/file_manager.cpp

@@ -28,6 +28,8 @@ void FileManager::start() {
     // v2.8.0 — start the expiry sweeper. `cleanupIntervalSec` has been in
     // Config from the start but nothing ever read it, so expired files (and
     // their blobs) were never reclaimed.
+    loadRetentionOverrides();
+
     running_.store(true, std::memory_order_release);
     cleanup_thread_ = std::thread([this] { cleanupLoop(); });
 
@@ -54,21 +56,122 @@ uint64_t now_ms() {
 }
 }  // namespace
 
+bool FileManager::setExpiryIn(const std::string& project, const std::string& id,
+                               uint64_t expiresAt) {
+    // Scoped lookup: another project's record must look absent, so an id that
+    // leaks cannot be used to change someone else's retention. Note this uses the
+    // expiry-FILTERED accessor deliberately - an already-expired file is gone as
+    // far as callers are concerned, so it cannot be resurrected by extending it.
+    if (!getFileInfoIn(project, id)) return false;
+
+    auto rp = recordPath(id);
+    try {
+        nlohmann::json meta;
+        {
+            std::ifstream in(rp);
+            if (!in) return false;
+            in >> meta;
+        }
+        meta["expiresAt"] = expiresAt;
+        std::ofstream out(rp);
+        if (!out) return false;
+        out << meta.dump(2);
+    } catch (const nlohmann::json::exception&) {
+        return false;
+    }
+    spdlog::info("File {} expiry set to {} (0 = keep indefinitely)", id, expiresAt);
+    return true;
+}
+
 uint32_t FileManager::defaultTtlFor(const std::string& project,
                                      const std::string& fileType) const {
-    const auto& m = config_.defaultTtlSecondsByType;
-    if (m.empty()) return 0;
     const std::string proj = project.empty() ? "default" : project;
     // Most specific first, so a project can override a global policy and a
-    // single type can override its project's blanket rule.
+    // single type can override its project's blanket rule. At each level a
+    // runtime override wins over the config baseline, so an operator can revise
+    // retention without a restart.
+    std::unordered_map<std::string, uint32_t> overrides;
+    {
+        std::lock_guard lock(retention_mutex_);
+        overrides = retention_overrides_;
+    }
     for (const std::string& key : {proj + ":" + fileType, fileType,
                                    proj + ":*", std::string("*")}) {
-        auto it = m.find(key);
-        if (it != m.end()) return it->second;
+        auto ov = overrides.find(key);
+        if (ov != overrides.end()) return ov->second;
+        auto it = config_.defaultTtlSecondsByType.find(key);
+        if (it != config_.defaultTtlSecondsByType.end()) return it->second;
     }
     return 0;
 }
 
+bool FileManager::setDefaultTtl(const std::string& key, uint32_t seconds) {
+    if (key.empty()) return false;
+    {
+        std::lock_guard lock(retention_mutex_);
+        retention_overrides_[key] = seconds;
+    }
+    const bool saved = saveRetentionOverrides();
+    spdlog::info("File retention policy: '{}' -> {}s{} (applies to files stored "
+                 "from now on; existing files keep the expiry they were given)",
+                 key, seconds, saved ? "" : " [NOT PERSISTED]");
+    return saved;
+}
+
+std::unordered_map<std::string, uint32_t> FileManager::retentionPolicy() const {
+    auto merged = config_.defaultTtlSecondsByType;
+    std::lock_guard lock(retention_mutex_);
+    for (const auto& [k, v] : retention_overrides_) merged[k] = v;
+    return merged;
+}
+
+void FileManager::loadRetentionOverrides() {
+    const auto path = config_.filesDir / "retention.json";
+    if (!std::filesystem::exists(path)) return;
+    try {
+        nlohmann::json j;
+        std::ifstream in(path);
+        in >> j;
+        if (!j.is_object()) return;
+        std::lock_guard lock(retention_mutex_);
+        retention_overrides_.clear();
+        for (const auto& [k, v] : j.items()) {
+            if (v.is_number_unsigned()) retention_overrides_[k] = v.get<uint32_t>();
+        }
+        spdlog::info("File retention: loaded {} runtime override(s) from {}",
+                     retention_overrides_.size(), path.string());
+    } catch (const std::exception& e) {
+        // A corrupt overrides file must not stop the service; the config
+        // baseline still applies.
+        spdlog::warn("File retention: could not read {}: {} - using config only",
+                     path.string(), e.what());
+    }
+}
+
+bool FileManager::saveRetentionOverrides() const {
+    const auto path = config_.filesDir / "retention.json";
+    try {
+        nlohmann::json j = nlohmann::json::object();
+        {
+            std::lock_guard lock(retention_mutex_);
+            for (const auto& [k, v] : retention_overrides_) j[k] = v;
+        }
+        // Write-and-rename so a crash mid-write cannot leave a truncated policy.
+        const auto tmp = path.string() + ".tmp";
+        {
+            std::ofstream out(tmp);
+            if (!out) return false;
+            out << j.dump(2);
+            if (!out) return false;
+        }
+        std::error_code ec;
+        std::filesystem::rename(tmp, path, ec);
+        return !ec;
+    } catch (const std::exception&) {
+        return false;
+    }
+}
+
 bool FileManager::isExpired(const FileInfo& info, uint64_t nowMs) {
     if (info.expiresAt == 0) return false;   // never expires
     if (nowMs == 0) nowMs = now_ms();

service/src/files/file_manager.hpp

@@ -142,6 +142,34 @@ public:
     [[nodiscard]] uint32_t defaultTtlFor(const std::string& project,
                                          const std::string& fileType) const;
 
+    // v2.8.0 — change retention policy at runtime, and persist it.
+    //
+    // The config key `storage.files.default_ttl_seconds` sets the policy at boot,
+    // but retention is exactly the sort of thing an operator revises without
+    // wanting a restart. Overrides set here are written to `retention.json`
+    // alongside the blobs and reloaded on the next boot, layered OVER the config -
+    // so config stays the declared baseline and runtime changes stick.
+    //
+    // `key` uses the same matching vocabulary as the config:
+    //   "<project>:<fileType>" | "<fileType>" | "<project>:*" | "*"
+    // `seconds` of 0 means "keep indefinitely" for anything matching that key.
+    bool setDefaultTtl(const std::string& key, uint32_t seconds);
+
+    // Current effective policy, config plus runtime overrides.
+    [[nodiscard]] std::unordered_map<std::string, uint32_t> retentionPolicy() const;
+
+    // v2.8.0 — set or clear the expiry of a file that is already stored.
+    //
+    // `expiresAt` is absolute ms; 0 clears the expiry so the file is kept
+    // indefinitely. Project-scoped: a record owned by another project is treated
+    // as absent, exactly like every other caller-facing accessor, so a leaked id
+    // cannot be used to extend or shorten someone else's retention.
+    //
+    // Returns false if the file does not exist in that project. Rewrites the
+    // record in place; the blob is untouched.
+    bool setExpiryIn(const std::string& project, const std::string& id,
+                     uint64_t expiresAt);
+
     // v2.8.0 — true if `info` has an expiry in the past.
     [[nodiscard]] static bool isExpired(const FileInfo& info, uint64_t nowMs = 0);
 
@@ -167,6 +195,13 @@ private:
     std::condition_variable cleanup_cv_;
     void cleanupLoop();
 
+    // v2.8.0 — runtime retention overrides, layered over Config. Persisted to
+    // <filesDir>/retention.json so a restart keeps them.
+    mutable std::mutex retention_mutex_;
+    std::unordered_map<std::string, uint32_t> retention_overrides_;
+    void loadRetentionOverrides();
+    bool saveRetentionOverrides() const;
+
     // Project-blind and expiry-blind. The sweeper needs to SEE expired records
     // in order to delete them, while every caller-facing accessor must treat
     // them as already gone.

service/src/main.cpp

@@ -116,8 +116,19 @@ void printUsage(const char* programName) {
 #define SMARTBOTIC_DB_VERSION_STRING "unknown"
 #endif
 
+#ifndef SMARTBOTIC_DB_GIT_COMMIT_STRING
+// Mirrors the SMARTBOTIC_DB_VERSION_STRING fallback: report the absence rather
+// than baking in a literal that could drift.
+#define SMARTBOTIC_DB_GIT_COMMIT_STRING "unknown"
+#endif
+
 void printVersion() {
-    std::cout << "smartbotic-database version " << SMARTBOTIC_DB_VERSION_STRING << "\n"
+    // v2.8.0 — report the build commit too. It was computed and passed as a
+    // Docker build-arg since v2.4 but never consumed, so packages carried no
+    // provenance and `--version` could not tell two builds of the same version
+    // apart.
+    std::cout << "smartbotic-database version " << SMARTBOTIC_DB_VERSION_STRING
+              << " (commit " << SMARTBOTIC_DB_GIT_COMMIT_STRING << ")\n"
               << "Smartbotic Database Service\n"
               << std::endl;
 }

service/src/memory_store.cpp

@@ -93,6 +93,67 @@ bool MemoryStore::createCollection(const std::string& name, const CollectionOpti
 
     auto it = collections_.find(name);
     if (it != collections_.end()) {
+        // v2.8.0 — allow vectorDimension to be set on an EXISTING collection,
+        // but ONLY while it holds no vectors.
+        //
+        // vectorDimension was otherwise unreachable after creation:
+        // createCollection updated maxVersions and nothing else, and
+        // alterCollection can set it but has no RPC and no call sites. That
+        // stranded every collection created through the pre-2.4.5 createCollection
+        // bug, where the options went to a phantom collection and the real one
+        // got defaults - similaritySearch on those returns nothing, permanently.
+        //
+        // Safe because extractVector() returns early when the dimension is 0, so
+        // such a collection has stored no vectors: there is nothing of a
+        // conflicting dimension to invalidate. Refused once vectors exist, since
+        // changing the dimension under them would make the stored data
+        // unreadable by the search kernel.
+        //
+        // Documents already inserted keep `_vector` embedded in their data and
+        // are NOT retroactively extracted - this fixes subsequent writes, not the
+        // history. Re-write those documents to index them.
+        if (options.vectorDimension > 0 &&
+            it->second->options.vectorDimension == 0) {
+            if (it->second->vectors.empty()) {
+                it->second->options.vectorDimension = options.vectorDimension;
+                it->second->updatedAt = currentTimeMs();
+                spdlog::warn("Set vector_dimension={} on existing collection '{}' "
+                             "(it held no vectors). Documents already stored keep "
+                             "_vector embedded in their data and are not indexed "
+                             "retroactively - rewrite them to index them.",
+                             options.vectorDimension, name);
+            } else {
+                spdlog::warn("Refusing to set vector_dimension on '{}': it already "
+                             "holds {} vector(s), and changing the dimension would "
+                             "make them unreadable by the search kernel.",
+                             name, it->second->vectors.size());
+            }
+        } else if (options.vectorDimension > 0 &&
+                   it->second->options.vectorDimension != options.vectorDimension) {
+            spdlog::warn("Ignoring vector_dimension={} for '{}': it is already {} "
+                         "and is immutable once set.",
+                         options.vectorDimension, name,
+                         it->second->options.vectorDimension);
+        }
+
+        // v2.8.0 — defaultTtlSeconds is changeable at any time, unlike
+        // vectorDimension. It only affects documents inserted AFTER the change
+        // (insert/upsert/setAdd/restore consult it), so there is no stored state
+        // it can invalidate. Retention policy is exactly the kind of setting an
+        // operator revises later, and before this it was frozen at creation.
+        //
+        // 0 is a real value meaning "no default expiry", so it is applied like
+        // any other rather than being treated as "unset".
+        if (it->second->options.defaultTtlSeconds != options.defaultTtlSeconds) {
+            spdlog::info("Updated default_ttl_seconds for collection '{}': {} -> {} "
+                         "(applies to documents inserted from now on; existing "
+                         "documents keep the expiry they were written with)",
+                         name, it->second->options.defaultTtlSeconds,
+                         options.defaultTtlSeconds);
+            it->second->options.defaultTtlSeconds = options.defaultTtlSeconds;
+            it->second->updatedAt = currentTimeMs();
+        }
+
         // Update maxVersions on existing collection if specified
         if (options.maxVersions > 0 && it->second->options.maxVersions != options.maxVersions) {
             it->second->options.maxVersions = options.maxVersions;
@@ -710,7 +771,8 @@ bool MemoryStore::exists(const std::string& collection, const std::string& id) c
 }
 
 bool MemoryStore::updateIfVersion(const std::string& collection, const std::string& id,
-                                   const Document& doc, uint64_t expectedVersion) {
+                                   const Document& doc, uint64_t expectedVersion,
+                                   std::optional<uint64_t> expiryOverride) {
     CollectionData* coll = getOrCreateCollection(collection);
 
     // v1.7.0 T6 quiesce: mark before taking the write lock.
@@ -760,7 +822,10 @@ bool MemoryStore::updateIfVersion(const std::string& collection, const std::stri
     //
     // A non-zero incoming expiresAt still wins, so a caller CAN deliberately
     // change or extend a TTL by supplying one.
-    if (updated.expiresAt == 0) {
+    if (expiryOverride.has_value()) {
+        // v2.8.0 — the caller asked to change the expiry explicitly. 0 clears it.
+        updated.expiresAt = *expiryOverride;
+    } else if (updated.expiresAt == 0) {
         updated.expiresAt = it->second.expiresAt;
     }
 
@@ -823,7 +888,8 @@ bool MemoryStore::updateIfVersion(const std::string& collection, const std::stri
 }
 
 uint64_t MemoryStore::patchDocument(const std::string& collection, const std::string& id,
-                                     const nlohmann::json& patch, const std::string& actor) {
+                                     const nlohmann::json& patch, const std::string& actor,
+                                     std::optional<uint64_t> expiryOverride) {
     CollectionData* coll = getOrCreateCollection(collection);
 
     // v1.7.0 T6 quiesce: mark before taking the write lock.
@@ -869,6 +935,13 @@ uint64_t MemoryStore::patchDocument(const std::string& collection, const std::st
         it->second.updatedBy = actor;
     }
 
+    // v2.8.0 — an explicit expiry from the caller replaces the stored one. 0
+    // clears it. Without an override the stored value is kept, which is what
+    // patch always did.
+    if (expiryOverride.has_value()) {
+        it->second.expiresAt = *expiryOverride;
+    }
+
     // Add to new expiration index
     if (it->second.expiresAt > 0) {
         addToExpirationIndex(*coll, id, it->second.expiresAt);

service/src/memory_store.hpp

@@ -275,8 +275,13 @@ public:
      * @param expectedVersion Expected version of the document
      * @return true if updated, false if version mismatch or not found
      */
+    // v2.8.0 — `expiryOverride` changes the document's expiry as part of the
+    // write. Absent (the default) preserves whatever the stored document had,
+    // which is what stopped an update from silently making a TTL'd document
+    // permanent. A value of 0 clears the expiry; anything else sets it.
     bool updateIfVersion(const std::string& collection, const std::string& id,
-                         const Document& doc, uint64_t expectedVersion);
+                         const Document& doc, uint64_t expectedVersion,
+                         std::optional<uint64_t> expiryOverride = std::nullopt);
 
     /**
      * Atomically merge fields into an existing document.
@@ -286,7 +291,8 @@ public:
      * @return new version number, or 0 if document not found
      */
     uint64_t patchDocument(const std::string& collection, const std::string& id,
-                           const nlohmann::json& patch, const std::string& actor = "");
+                           const nlohmann::json& patch, const std::string& actor = "",
+                           std::optional<uint64_t> expiryOverride = std::nullopt);
 
     /**
      * Directly update a document's createdAt / updatedAt metadata fields

service/src/views/view_manager.cpp

@@ -87,10 +87,29 @@ void ViewManager::loadFromStore() {
     // Pre-v2.4.2 rows carry a bare name (views predate multi-project and were
     // stored globally). Re-key those into the default project on load, and
     // rewrite the row so the migration happens once rather than on every boot.
-    Query q;
-    auto result = store_.find(SYSTEM_COLLECTION, q);
+    // v2.8.0 — page explicitly. Query::limit DEFAULTS TO 100, so a bare Query
+    // silently loaded only the first 100 views: view 101 did not exist as far as
+    // the server was concerned, and the re-key migration below would never reach
+    // it either. Note limit=0 does NOT mean "everything" - find() computes
+    // `end = min(offset + limit, size)`, so 0 returns nothing. The same trap was
+    // fixed in PolicyManager::loadFromStore.
+    std::vector<Document> rows;
+    {
+        constexpr uint32_t kPage = 500;
+        uint32_t offset = 0;
+        while (true) {
+            Query q;
+            q.limit = kPage;
+            q.offset = offset;
+            auto page = store_.find(SYSTEM_COLLECTION, q);
+            for (auto& d : page.documents) rows.push_back(std::move(d));
+            if (page.documents.size() < kPage) break;
+            offset += kPage;
+        }
+    }
+
     size_t migrated = 0;
-    for (const auto& doc : result.documents) {
+    for (const auto& doc : rows) {
         ViewInfo v = fromJson(doc.data());
         if (v.name.empty()) continue;
 

tests/CMakeLists.txt

@@ -655,3 +655,37 @@ endif()
 find_package(Threads REQUIRED)
 target_link_libraries(test_document_ttl PRIVATE Threads::Threads)
 add_test(NAME test_document_ttl COMMAND test_document_ttl)
+
+# v2.8.0 — ViewManager persistence. test_views only exercises applyProjection()
+# in-process, so nothing covered loadFromStore's paging: a bare Query stops at
+# 100 views and silently loses the rest.
+add_executable(test_view_manager_paging
+    test_view_manager_paging.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/views/view_manager.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/views/projection.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/memory_store.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/config/collection_config_manager.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/persistence/history_store.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/persistence/wal.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/json_parse.cpp
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src/doc_binary.cpp
+)
+target_include_directories(test_view_manager_paging PRIVATE
+    ${CMAKE_CURRENT_SOURCE_DIR}/../service/src
+    ${yyjson_INCLUDE_DIRS}
+)
+target_link_libraries(test_view_manager_paging PRIVATE ${yyjson_LIBRARIES})
+if(TARGET nlohmann_json::nlohmann_json)
+    target_link_libraries(test_view_manager_paging PRIVATE nlohmann_json::nlohmann_json)
+else()
+    target_include_directories(test_view_manager_paging PRIVATE ${NLOHMANN_JSON_INCLUDE_DIRS})
+endif()
+if(TARGET spdlog::spdlog)
+    target_link_libraries(test_view_manager_paging PRIVATE spdlog::spdlog)
+else()
+    target_link_libraries(test_view_manager_paging PRIVATE ${SPDLOG_LIBRARIES})
+    target_include_directories(test_view_manager_paging PRIVATE ${SPDLOG_INCLUDE_DIRS})
+endif()
+find_package(Threads REQUIRED)
+target_link_libraries(test_view_manager_paging PRIVATE Threads::Threads)
+add_test(NAME test_view_manager_paging COMMAND test_view_manager_paging)

tests/load_test/test_client_namespacing.cpp

@@ -149,6 +149,70 @@ int main() {
            "an empty subscription does NOT receive another project's events");
     }
 
+    // ---- Part 5: ListFiles total_count must be a TOTAL, not the page size.
+    // The first policy-filtering cut set it to files_size(), so limit=5 on an
+    // 865-file project reported 5 and paging was impossible.
+    {
+        Client::FileUploadMeta fm3;
+        fm3.file_type = "pagecheck";
+        for (int i = 0; i < 7; ++i) {
+            fm3.name = "pg" + std::to_string(i) + ".bin";
+            c.uploadFile({static_cast<uint8_t>(i)}, fm3);
+        }
+        auto page = c.listFiles("pagecheck", "", 3, 0);
+        ck(page.files.size() == 3, "a page returns `limit` files");
+        ck(page.total_count == 7,
+           "total_count is the TOTAL, not the page size");
+        ck(page.has_more, "has_more true when more remain");
+
+        auto last = c.listFiles("pagecheck", "", 3, 6);
+        ck(last.files.size() == 1, "final page returns the remainder");
+        ck(last.total_count == 7, "total_count stable across pages");
+        ck(!last.has_more, "has_more false on the last page");
+    }
+
+    // ---- Part 6 (v2.8.0): listCollections is project-scoped.
+    // It previously returned every project's collection names, so a workspace
+    // could enumerate another workspace's schema even with its data unreachable.
+    {
+        Client peer({.address="127.0.0.1:9011", .project="proj_iso"});
+        peer.connect();
+        peer.createCollection("peer_only");
+        peer.insert("peer_only", nlohmann::json{{"x", 1}});
+
+        auto mine = c.listCollections();
+        bool leaked = false;
+        for (const auto& n : mine) {
+            if (n == "peer_only" || n == "proj_iso:peer_only") leaked = true;
+        }
+        ck(!leaked, "listCollections does not show another project's collections");
+
+        auto theirs = peer.listCollections();
+        bool sees_own = false;
+        for (const auto& n : theirs) if (n == "peer_only") sees_own = true;
+        ck(sees_own, "and a project does see its own, under the bare name");
+    }
+
+    // ---- Part 7 (v2.8.0): a stored file's TTL can be changed over the wire.
+    {
+        Client::FileUploadMeta fm4;
+        fm4.name = "ttlfile.bin";
+        fm4.file_type = "document";
+        auto up4 = c.uploadFile({1,2,3}, fm4);
+        auto before = c.getFileInfo(up4.id);
+        ck(before && before->project == "proj1", "uploaded into our project");
+
+        ck(c.setFileTtl(up4.id, 3600), "setFileTtl succeeds on our own file");
+        ck(c.setFileTtl(up4.id, 0), "and can clear the expiry again");
+
+        Client peer2({.address="127.0.0.1:9011", .project="proj_ttl_peer"});
+        peer2.connect();
+        ck(!peer2.setFileTtl(up4.id, 60),
+           "another project cannot change our file's retention");
+        ck(!peer2.setFileTtl("no-such-id", 60), "nor invent one");
+        c.deleteFile(up4.id);
+    }
+
     std::cout << "\npassed=" << pass << " failed=" << fail << "\n";
     return fail==0?0:1;
 }

tests/test_document_ttl.cpp

@@ -46,6 +46,14 @@ struct Fixture {
     ~Fixture() { store.stop(); }
 };
 
+Document mkDocIn(const std::string& coll, const std::string& id) {
+    Document d;
+    d.id = id;
+    d.collection = coll;
+    d.set_data(nlohmann::json{{"n", 1}});
+    return d;
+}
+
 Document mkDoc(const std::string& id, uint64_t expiresAt = 0) {
     Document d;
     d.id = id;
@@ -127,6 +135,118 @@ void test_patch_preserves_expiry() {
     check(after && after->data().value("n", 0) == 5, "and applies the patch");
 }
 
+// v2.8.0 — an expiry can now be changed or cleared without rewriting the whole
+// document. Before this, ttl_seconds existed only on insert/upsert, so the only
+// way to alter a TTL was a full upsert.
+void test_expiry_override_on_update() {
+    Fixture f;
+    const uint64_t original = future_ms(60);
+    f.store.insert("things", mkDoc("a", original));
+    auto stored = f.store.get("things", "a");
+
+    // Absent override -> preserve (the default).
+    Document next = mkDoc("a");
+    f.store.updateIfVersion("things", "a", next, stored->version);
+    check(f.store.get("things", "a")->expiresAt == original,
+          "no override preserves the existing expiry");
+
+    // Explicit new expiry -> replace.
+    const uint64_t later = future_ms(7200);
+    stored = f.store.get("things", "a");
+    f.store.updateIfVersion("things", "a", mkDoc("a"), stored->version, later);
+    check(f.store.get("things", "a")->expiresAt == later,
+          "an override replaces the expiry");
+
+    // Explicit 0 -> clear, making the document permanent.
+    stored = f.store.get("things", "a");
+    f.store.updateIfVersion("things", "a", mkDoc("a"), stored->version, uint64_t{0});
+    check(f.store.get("things", "a")->expiresAt == 0,
+          "an override of 0 CLEARS the expiry - this is how a document is made "
+          "permanent, and is why the field needs presence semantics on the wire");
+}
+
+void test_expiry_override_on_patch() {
+    Fixture f;
+    const uint64_t original = future_ms(60);
+    f.store.insert("things", mkDoc("a", original));
+
+    f.store.patchDocument("things", "a", nlohmann::json{{"n", 2}}, "");
+    check(f.store.get("things", "a")->expiresAt == original,
+          "patch without an override preserves the expiry");
+
+    const uint64_t later = future_ms(3600);
+    f.store.patchDocument("things", "a", nlohmann::json{{"n", 3}}, "", later);
+    check(f.store.get("things", "a")->expiresAt == later,
+          "patch with an override replaces the expiry");
+    check(f.store.get("things", "a")->data().value("n", 0) == 3,
+          "and still applies the patch");
+
+    f.store.patchDocument("things", "a", nlohmann::json{{"n", 4}}, "", uint64_t{0});
+    check(f.store.get("things", "a")->expiresAt == 0,
+          "patch with an override of 0 clears the expiry");
+}
+
+// v2.8.0 — collection settings that were frozen at creation.
+//
+// createCollection on an existing collection used to update ONLY maxVersions, so
+// defaultTtlSeconds could never be revised and vectorDimension was unreachable
+// entirely (alterCollection can set it but has no RPC and no call sites). That
+// stranded collections created through the pre-2.4.5 createCollection bug, where
+// the options went to a phantom collection and the real one got defaults.
+void test_collection_default_ttl_is_changeable() {
+    Fixture f;
+    CollectionOptions o;
+    o.defaultTtlSeconds = 0;
+    f.store.createCollection("ttlcoll", o);
+
+    f.store.insert("ttlcoll", mkDocIn("ttlcoll", "before"));
+    check(f.store.get("ttlcoll", "before")->expiresAt == 0,
+          "inserted with no default: no expiry");
+
+    // Revise the policy.
+    o.defaultTtlSeconds = 3600;
+    f.store.createCollection("ttlcoll", o);      // existing collection
+    f.store.insert("ttlcoll", mkDocIn("ttlcoll", "after"));
+    check(f.store.get("ttlcoll", "after")->expiresAt > 0,
+          "a document inserted after the change inherits the new default");
+    check(f.store.get("ttlcoll", "before")->expiresAt == 0,
+          "and existing documents keep what they were written with - the change "
+          "is not retroactive");
+
+    // And back to none.
+    o.defaultTtlSeconds = 0;
+    f.store.createCollection("ttlcoll", o);
+    f.store.insert("ttlcoll", mkDocIn("ttlcoll", "third"));
+    check(f.store.get("ttlcoll", "third")->expiresAt == 0,
+          "clearing the default works too - 0 is a real value, not 'unset'");
+}
+
+void test_vector_dimension_recovery() {
+    Fixture f;
+    CollectionOptions o;
+    o.vectorDimension = 0;                       // the broken-createCollection state
+    f.store.createCollection("vec", o);
+
+    // Settable while the collection holds no vectors.
+    o.vectorDimension = 4;
+    f.store.createCollection("vec", o);
+    auto info = f.store.getCollectionInfo("vec");
+    check(info && info->options.vectorDimension == 4,
+          "vector_dimension can be set on a collection that holds no vectors");
+
+    // Now it holds one, so the dimension must be refused rather than changed.
+    Document d = mkDocIn("vec", "v1");
+    d.set_data(nlohmann::json{{"_vector", {1.0, 0.0, 0.0, 0.0}}});
+    f.store.insert("vec", d);
+
+    o.vectorDimension = 8;
+    f.store.createCollection("vec", o);
+    auto after = f.store.getCollectionInfo("vec");
+    check(after && after->options.vectorDimension == 4,
+          "once vectors exist the dimension is immutable - changing it would make "
+          "the stored vectors unreadable by the search kernel");
+}
+
 }  // namespace
 
 int main() {
@@ -136,6 +256,10 @@ int main() {
     test_update_can_change_expiry_when_given_one();
     test_update_of_a_document_with_no_expiry_stays_permanent();
     test_patch_preserves_expiry();
+    test_expiry_override_on_update();
+    test_expiry_override_on_patch();
+    test_collection_default_ttl_is_changeable();
+    test_vector_dimension_recovery();
     std::cout << "passed: " << g_pass << ", failed: " << g_fail << "\n";
     return g_fail == 0 ? 0 : 1;
 }

tests/test_file_project_scope.cpp

@@ -126,6 +126,28 @@ struct TmpFm {
     }
 };
 
+// Like TmpFm but leaves the directory behind, so a second FileManager can be
+// opened over it - which is what a service restart amounts to.
+struct TmpFmKeep {
+    std::string dir;
+    std::unique_ptr<FileManager> fm;
+    explicit TmpFmKeep(std::unordered_map<std::string, uint32_t> ttlByType) {
+        static std::atomic<int> counter{0};
+        dir = "/tmp/sbdb-file-retention-test-" + std::to_string(::getpid()) + "-" +
+              std::to_string(counter.fetch_add(1));
+        std::error_code ec;
+        fs::remove_all(dir, ec);
+        fs::create_directories(dir);
+        FileManager::Config cfg;
+        cfg.filesDir = dir;
+        cfg.defaultTtlSecondsByType = std::move(ttlByType);
+        fm = std::make_unique<FileManager>(cfg);
+        fm->start();
+    }
+    ~TmpFmKeep() { if (fm) fm->stop(); }
+    FileManager* operator->() { return fm.get(); }
+};
+
 FileManager::FileInfo mk(const std::string& name,
                          const std::string& project,
                          const std::string& fileType = "document") {
@@ -410,6 +432,97 @@ void test_no_config_means_no_expiry() {
           "with no retention configured nothing expires - the default default");
 }
 
+// v2.8.0 — a stored file's expiry can be changed or cleared. It was previously
+// fixed at upload, so the only way to alter retention was to re-upload.
+void test_set_expiry_on_a_stored_file() {
+    TmpFm fm;
+    auto r = fm->storeFile({1, 2, 3}, mk("f.bin", "acme"));
+    check(fm->getFileInfo(r.id)->expiresAt == 0, "uploaded with no expiry");
+
+    const uint64_t soon = now_ms_test() + 60000;
+    check(fm->setExpiryIn("acme", r.id, soon), "expiry can be set after upload");
+    check(fm->getFileInfo(r.id)->expiresAt == soon, "and it persisted");
+
+    // Clearing it keeps the file indefinitely.
+    check(fm->setExpiryIn("acme", r.id, 0), "expiry can be cleared");
+    check(fm->getFileInfo(r.id)->expiresAt == 0, "0 means keep indefinitely");
+    check(fm->purgeExpired() == 0, "and the sweeper leaves it alone");
+}
+
+void test_set_expiry_is_project_scoped() {
+    TmpFm fm;
+    auto r = fm->storeFile({4, 5}, mk("owned.bin", "acme"));
+    check(!fm->setExpiryIn("other", r.id, now_ms_test() + 1000),
+          "another project cannot change this file's retention");
+    check(fm->getFileInfo(r.id)->expiresAt == 0, "and the expiry is untouched");
+}
+
+void test_expired_file_cannot_be_resurrected() {
+    TmpFm fm;
+    auto info = mk("dead.bin", "acme");
+    info.expiresAt = now_ms_test() - 1000;
+    auto r = fm->storeFile({6}, info);
+
+    check(!fm->setExpiryIn("acme", r.id, now_ms_test() + 60000),
+          "an already-expired file cannot be given a new lease - it is gone as far "
+          "as callers are concerned, so extending it would resurrect it");
+    check(fm->purgeExpired() == 1, "and the sweeper still reclaims it");
+}
+
+// v2.8.0 — retention policy is changeable at runtime and survives a restart.
+// The config key sets the baseline at boot; overrides layer over it and persist
+// to retention.json, so revising retention does not need a restart.
+void test_runtime_retention_override() {
+    TmpFm fm({{"generated", 3600}});
+    check(fm->defaultTtlFor("acme", "generated") == 3600, "config baseline applies");
+
+    check(fm->setDefaultTtl("generated", 60), "an override can be set at runtime");
+    check(fm->defaultTtlFor("acme", "generated") == 60,
+          "and it wins over the config baseline");
+
+    check(fm->setDefaultTtl("acme:generated", 10), "a more specific key can be set");
+    check(fm->defaultTtlFor("acme", "generated") == 10,
+          "and the most specific override wins");
+    check(fm->defaultTtlFor("other", "generated") == 60,
+          "while a peer project still gets the less specific one");
+
+    // 0 means keep indefinitely, and must beat a non-zero config baseline.
+    check(fm->setDefaultTtl("acme:generated", 0), "an override of 0 can be set");
+    check(fm->defaultTtlFor("acme", "generated") == 0,
+          "0 means keep indefinitely, overriding the config's 3600");
+
+    // A newly stored file picks up the effective policy.
+    auto r = fm->storeFile({1}, mk("g.bin", "acme", "generated"));
+    check(fm->getFileInfo(r.id)->expiresAt == 0,
+          "a file stored under a 0 override gets no expiry");
+}
+
+void test_runtime_retention_survives_restart() {
+    // Same directory, two FileManager lifetimes - what a service restart is.
+    std::string dir;
+    {
+        TmpFmKeep fm({{"generated", 3600}});
+        dir = fm.dir;
+        fm->setDefaultTtl("generated", 42);
+        check(fm->defaultTtlFor("acme", "generated") == 42, "override set");
+    }
+    {
+        FileManager::Config cfg;
+        cfg.filesDir = dir;
+        cfg.defaultTtlSecondsByType = {{"generated", 3600}};
+        FileManager again(cfg);
+        again.start();
+        check(again.defaultTtlFor("acme", "generated") == 42,
+              "the override was persisted to retention.json and reloaded - the "
+              "config baseline does not silently win back");
+        auto pol = again.retentionPolicy();
+        check(pol["generated"] == 42, "retentionPolicy reports the effective value");
+        again.stop();
+    }
+    std::error_code ec;
+    fs::remove_all(dir, ec);
+}
+
 }  // namespace
 
 int main() {
@@ -433,6 +546,11 @@ int main() {
     test_explicit_expiry_overrides_the_default();
     test_never_expires_opts_out_of_the_default();
     test_no_config_means_no_expiry();
+    test_set_expiry_on_a_stored_file();
+    test_set_expiry_is_project_scoped();
+    test_expired_file_cannot_be_resurrected();
+    test_runtime_retention_override();
+    test_runtime_retention_survives_restart();
 
     std::cout << "passed: " << g_pass << ", failed: " << g_fail << "\n";
     return g_fail == 0 ? 0 : 1;

tests/test_view_manager_paging.cpp

@@ -0,0 +1,101 @@
+// v2.8.0 — ViewManager must load EVERY view, not the first 100.
+//
+// loadFromStore() read the `_views` collection with a bare `Query`, and
+// Query::limit defaults to 100. So on an installation with more than 100 views,
+// view 101 onwards silently did not exist: not in the cache, not resolvable by
+// name, and skipped by the v2.4.2 re-key migration that runs in the same loop.
+//
+// Note limit=0 is NOT "unlimited" here - find() computes
+// `end = min(offset + limit, size)`, so 0 returns nothing at all. That is the
+// same trap that made PolicyManager load zero policies on its first cut, which
+// is why both now page explicitly.
+//
+// tests/test_views.cpp cannot catch this: it only exercises applyProjection() and
+// never constructs a MemoryStore, so nothing there touches persistence.
+
+#include <iostream>
+#include <string>
+
+#include <nlohmann/json.hpp>
+
+#include "memory_store.hpp"
+#include "views/view_manager.hpp"
+
+using namespace smartbotic::database;
+
+namespace {
+
+int g_pass = 0;
+int g_fail = 0;
+
+void check(bool cond, const std::string& msg) {
+    if (cond) { ++g_pass; }
+    else { ++g_fail; std::cerr << "FAIL: " << msg << "\n"; }
+}
+
+struct Fixture {
+    MemoryStore store;
+    Fixture() : store(MemoryStore::Config{}) {
+        store.start();
+        CollectionOptions o;
+        store.createCollection("users", o);
+    }
+    ~Fixture() { store.stop(); }
+};
+
+ViewInfo mkView(const std::string& name) {
+    ViewInfo v;
+    v.name = "default:" + name;
+    v.collection = "default:users";
+    v.include = {"name"};
+    return v;
+}
+
+void test_more_than_one_page_of_views_survives_reload() {
+    Fixture f;
+    ViewManager vm(f.store);
+    vm.loadFromStore();
+
+    // 250 views: two full pages plus a partial one.
+    constexpr int kCount = 250;
+    std::string err;
+    int created = 0;
+    for (int i = 0; i < kCount; ++i) {
+        char buf[32];
+        std::snprintf(buf, sizeof(buf), "v%03d", i);
+        if (vm.createView(mkView(buf), err)) ++created;
+    }
+    check(created == kCount, "all views created");
+    check(vm.listViews().size() == static_cast<size_t>(kCount),
+          "all views present before reload");
+
+    // Reload from the same store, as a restart would.
+    ViewManager fresh(f.store);
+    fresh.loadFromStore();
+
+    check(fresh.listViews().size() == static_cast<size_t>(kCount),
+          "every view survives reload - a bare Query would have stopped at 100");
+
+    // Spot-check names either side of the old 100 boundary, since that is where
+    // the truncation bit.
+    check(fresh.isView("default:v000"), "the first view resolves");
+    check(fresh.isView("default:v099"), "the last view of page one resolves");
+    check(fresh.isView("default:v100"),
+          "view 100 resolves - this is the one the truncation lost");
+    check(fresh.isView("default:v249"), "the final view resolves");
+    check(!fresh.isView("default:v250"), "a view that was never created does not");
+
+    auto got = fresh.getView("default:v100");
+    check(got.has_value(), "view 100 is retrievable");
+    check(got && got->collection == "default:users",
+          "and its definition round-tripped intact");
+}
+
+}  // namespace
+
+int main() {
+    std::cout << "=== test_view_manager_paging ===\n";
+    test_more_than_one_page_of_views_survives_reload();
+    std::cout << "passed: " << g_pass << ", failed: " << g_fail << "\n";
+    return g_fail == 0 ? 0 : 1;
+}