smartbotic-automation Debian Packaging Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Ship /data/smartbotic as a Debian 13 .deb package named smartbotic-automation, replacing the in-repo src/database/ with the upstream smartbotic-database daemon (talked to via libsmartbotic-db-client).

Architecture: Three phases: (A) code migration — drop src/database/, rewrite lib/storage/ as a thin adapter over smartbotic::database::Client; (B) packaging — Docker-based two-stage Debian 13 build mirroring /data/shadowman-cpp/packaging/; (C) smoke test + (D) repo publish via /data/smartbotics-deb-repo/.

Tech Stack: C++20, gRPC (via upstream client only), libsmartbotic-db-client, CMake + Ninja, Docker BuildKit, Debian 13 (trixie), systemd, dpkg-deb.

Spec: docs/superpowers/specs/2026-05-09-smartbotic-automation-debian-packaging-design.md

File Structure Map

Created

Path	Purpose
`VERSION`	Single-line version string (`1.0.0`)
`packaging/Dockerfile.base`	Debian 13 + build deps + `libsmartbotic-db-client-dev`
`packaging/Dockerfile.build`	Compile C++ + WebUI, assemble `.deb` payload, run `dpkg-deb`
`packaging/build.sh`	Orchestrator (mirrors `/data/shadowman-cpp/packaging/build.sh`)
`packaging/smartbotics-repo.gpg`	Public key for SmartBotics APT repo (copied from deb repo)
`packaging/scripts/create-deb.sh`	Inside-container packager: stages files, runs `dpkg-deb --build`
`packaging/deb/templates/control.smartbotic-automation`	Debian control file (with `{{VERSION}}` / `{{INSTALLED_SIZE}}` placeholders)
`packaging/deb/scripts/preinst`	No-op pre-install hook
`packaging/deb/scripts/postinst`	Create user/group, dirs, daemon-reload, enable units
`packaging/deb/scripts/prerm`	Stop + (on remove) disable units
`packaging/deb/scripts/postrm`	Purge user/group + data/log dirs
`packaging/deb/systemd/smartbotic-automation-webserver.service`	systemd unit
`packaging/deb/systemd/smartbotic-automation-runner.service`	systemd unit
`packaging/deb/config/webserver.json`	Default `/etc/smartbotic-automation/webserver.json`
`packaging/deb/config/runner.json`	Default `/etc/smartbotic-automation/runner.json`
`packaging/deb/config/env`	Default empty `/etc/smartbotic-automation/env`

Modified

Path	Change
`CMakeLists.txt`	Drop `smartbotic-database` target + `src/database/` sources; add `find_package(smartbotic-db-client)`; link `lib/storage` against upstream client; drop `proto/storage.proto` from generated set
`lib/storage/storage_client.hpp`	Drop gRPC + internal proto includes; keep public API stable
`lib/storage/storage_client.cpp`	Replace gRPC stub calls with `smartbotic::database::Client` calls

Deleted

Path	Reason
`src/database/` (entire directory)	Replaced by upstream `smartbotic-database` daemon
`proto/storage.proto`	Replaced by upstream's `database.proto` (hidden inside client lib)

Phase A — Code Migration

Task A1: Confirm `smartbotic-migrate-nodes` has no direct DB dependency

Files: Read-only — src/tools/migrate_nodes.cpp

[ ] Step 1: Grep for storage/proto usage

grep -nE 'storage_client|StorageClient|smartbotic::storage|proto::Storage|<smartbotic/database' \
/data/smartbotic/src/tools/migrate_nodes.cpp

Expected: no matches. The tool is a pure HTTP client (uses <curl/curl.h> and <nlohmann/json.hpp> only). If matches appear, STOP and add the migration of this tool to Phase A before continuing.

[ ] Step 2: Verify CMake target deps

grep -n 'smartbotic-migrate-nodes\|migrate_nodes' /data/smartbotic/CMakeLists.txt

Expected: target smartbotic-migrate-nodes links only against CURL::libcurl, nlohmann_json::nlohmann_json, smartbotic_logging, smartbotic_common. No smartbotic_storage or smartbotic_proto link. If it links to those, document the additional refactor needed and add a Task A6½ to fix.

Task A2: Install `libsmartbotic-db-client-dev` for native dev build

Files: None (host setup)

[ ] Step 1: Check if already installed

dpkg -l libsmartbotic-db-client-dev 2>&1 | grep '^ii' || echo "not installed"

If already installed, skip to Task A3.

[ ] Step 2: Install if missing

sudo apt-get update && sudo apt-get install -y libsmartbotic-db-client-dev

Expected: pulls in the dev headers under /usr/include/smartbotic/database/ and CMake config files under /usr/lib/x86_64-linux-gnu/cmake/ (or pkg-config under /usr/lib/x86_64-linux-gnu/pkgconfig/).

[ ] Step 3: Locate the install path of the client

dpkg -L libsmartbotic-db-client-dev | grep -E '(client\.hpp|smartbotic-db-client.*\.cmake|smartbotic-db-client\.pc)$'

Record the CMake target name or pkg-config name in your shell history — Task A3 will need it. (Likely either smartbotic::db-client CMake target or smartbotic-db-client pkg-config name.)

Task A3: Add `libsmartbotic-db-client` to `CMakeLists.txt`

Files:

Modify: /data/smartbotic/CMakeLists.txt:80-100 (around the smartbotic_storage target)
[ ] Step 1: Add a find_package block

Insert near the other find_package calls (top of CMakeLists, before smartbotic_storage definition). Use a fallback chain so we work whether the upstream ships CMake config or only pkg-config:

# Upstream smartbotic-database client library — provides target smartbotic::db-client
find_package(smartbotic-db-client CONFIG REQUIRED)

(Verified locally: /usr/lib/x86_64-linux-gnu/cmake/smartbotic-db-client/smartbotic-db-clientConfig.cmake exports smartbotic::db-client as a SHARED IMPORTED target; INTERFACE_LINK_LIBRARIES brings in gRPC::grpc++ and protobuf::libprotobuf automatically. No pkg-config fallback is shipped, so we do not need one.)

[ ] Step 2: Configure & verify

cd /data/smartbotic
rm -rf build
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Debug

Expected: configuration succeeds. The smartbotic-db-client package is found. (Build will still pass at this stage because nothing links against it yet.)

[ ] Step 3: Commit

git add CMakeLists.txt
git commit -m "build: discover libsmartbotic-db-client via CMake/pkg-config"

Task A4: Rewrite `lib/storage/storage_client.hpp`

Files:

Rewrite: /data/smartbotic/lib/storage/storage_client.hpp (entire file, 115 lines → ~110 lines)

[ ] Step 1: Replace the file contents

#pragma once

#include <memory>
#include <string>
#include <vector>

#include <nlohmann/json.hpp>

#include "common/error.hpp"

namespace smartbotic::storage {

// Storage client configuration
struct StorageClientConfig {
std::string address = "localhost:9010";  // upstream smartbotic-database default in our deployments
int timeout_ms = 5000;
int max_message_size_mb = 64;            // kept for back-compat; ignored by adapter
};

// Query options - field projection and multi-sort kept for source compatibility,
// but the upstream client supports only a single sort field. The adapter will
// log a warning and use only the first entry of `sorts`. Field projection is
// silently ignored by upstream; callers must filter in-process if needed.
struct QueryOptions {
std::vector<std::pair<std::string, nlohmann::json>> filters;
std::vector<std::pair<std::string, bool>> sorts;  // field, ascending
std::vector<std::string> fields;
int32_t page = 1;
int32_t page_size = 100;
};

struct QueryResult {
std::vector<nlohmann::json> documents;
int64_t total_count = 0;
bool has_more = false;
};

struct VersioningOptions {
bool enabled = false;
int32_t max_versions = 0;
int64_t version_ttl_ms = 0;
bool keep_on_delete = false;
};

struct DocumentVersionInfo {
std::string id;
std::string document_id;
int64_t version = 0;
nlohmann::json data;
int64_t created_at = 0;
};

struct VersionListResult {
std::vector<DocumentVersionInfo> versions;
int64_t total_count = 0;
bool has_more = false;
};

// Adapter over smartbotic::database::Client. Public surface unchanged from the
// pre-1.0 internal-DB version; PIMPL hides the upstream client.
class StorageClient {
public:
explicit StorageClient(const StorageClientConfig& config);
~StorageClient();

StorageClient(const StorageClient&) = delete;
StorageClient& operator=(const StorageClient&) = delete;
StorageClient(StorageClient&&) noexcept;
StorageClient& operator=(StorageClient&&) noexcept;

common::Result<nlohmann::json> get(const std::string& collection, const std::string& id);

common::Result<std::string> insert(const std::string& collection,
                                   const nlohmann::json& data,
                                   const std::string& id = "",
                                   int64_t ttl_ms = 0);

common::Result<int64_t> update(const std::string& collection,
                               const std::string& id,
                               const nlohmann::json& data,
                               int64_t expected_version = 0,
                               bool partial = false);

common::Result<void> remove(const std::string& collection,
                            const std::string& id,
                            int64_t expected_version = 0);

common::Result<QueryResult> query(const std::string& collection);
common::Result<QueryResult> query(const std::string& collection, const QueryOptions& options);

common::Result<void> createCollection(const std::string& name,
                                      const nlohmann::json& schema = nlohmann::json{},
                                      int64_t default_ttl_ms = 0,
                                      const VersioningOptions* versioning = nullptr);

common::Result<void> dropCollection(const std::string& name);

std::vector<std::string> listCollections();

common::Result<nlohmann::json> getVersion(const std::string& collection,
                                          const std::string& id,
                                          int64_t version);

common::Result<VersionListResult> listVersions(const std::string& collection,
                                               const std::string& id,
                                               int32_t limit = 100,
                                               int32_t offset = 0);

bool isConnected() const;

private:
class Impl;
std::unique_ptr<Impl> impl_;
};

} // namespace smartbotic::storage

[ ] Step 2: Sanity-check with grep

grep -nE 'grpcpp|proto::Storage|storage\.grpc\.pb' /data/smartbotic/lib/storage/storage_client.hpp

Expected: zero matches (header is now grpc/proto-free).

Task A5: Rewrite `lib/storage/storage_client.cpp`

Files:

Rewrite: /data/smartbotic/lib/storage/storage_client.cpp (entire file, 347 lines → ~280 lines)

[ ] Step 1: Replace the file contents

#include "storage/storage_client.hpp"

#include <chrono>
#include <smartbotic/database/client.hpp>

#include "logging/logger.hpp"

namespace smartbotic::storage {

namespace dbc = smartbotic::database;
using common::Error;
using common::Result;

namespace {

// Round milliseconds up to whole seconds. 0 -> 0 (no expiry).
uint32_t msToSec(int64_t ms) {
if (ms <= 0) return 0;
return static_cast<uint32_t>((ms + 999) / 1000);
}

// Translate our string-encoded operator to upstream FilterOp.
// Returns FilterOp::EQ for unknown ops (with a warning log).
dbc::Client::FilterOp translateOp(const std::string& op) {
if (op == "eq" || op.empty()) return dbc::Client::FilterOp::EQ;
if (op == "ne") return dbc::Client::FilterOp::NE;
if (op == "gt") return dbc::Client::FilterOp::GT;
if (op == "gte") return dbc::Client::FilterOp::GTE;
if (op == "lt") return dbc::Client::FilterOp::LT;
if (op == "lte") return dbc::Client::FilterOp::LTE;
if (op == "in") return dbc::Client::FilterOp::IN;
if (op == "contains") return dbc::Client::FilterOp::CONTAINS;
if (op == "exists") return dbc::Client::FilterOp::EXISTS;
if (op == "regex") return dbc::Client::FilterOp::REGEX;
if (op == "search") return dbc::Client::FilterOp::SEARCH;
LOG_WARN("Storage: unknown filter op '{}', defaulting to EQ", op);
return dbc::Client::FilterOp::EQ;
}

// Decode a {field, value} pair into an upstream Filter. The value can be a
// raw JSON value (implicit EQ) or {"op": "...", "value": ...}.
dbc::Client::Filter buildFilter(const std::string& field, const nlohmann::json& v) {
dbc::Client::Filter f;
f.field = field;
if (v.is_object() && v.contains("op")) {
    f.op = translateOp(v.value("op", "eq"));
    f.value = v.value("value", nlohmann::json{});
} else {
    f.op = dbc::Client::FilterOp::EQ;
    f.value = v;
}
return f;
}

} // namespace

class StorageClient::Impl {
public:
explicit Impl(const StorageClientConfig& cfg) {
    dbc::Client::Config config;
    config.address = cfg.address;
    config.timeoutMs = static_cast<uint32_t>(cfg.timeout_ms);
    client_ = std::make_unique<dbc::Client>(std::move(config));
    connected_ = client_->connect();
    if (!connected_) {
        LOG_WARN("StorageClient: connect() to {} returned false", cfg.address);
    }
}

std::unique_ptr<dbc::Client> client_;
bool connected_ = false;
};

StorageClient::StorageClient(const StorageClientConfig& config)
: impl_(std::make_unique<Impl>(config)) {}

StorageClient::~StorageClient() = default;
StorageClient::StorageClient(StorageClient&&) noexcept = default;
StorageClient& StorageClient::operator=(StorageClient&&) noexcept = default;

bool StorageClient::isConnected() const {
return impl_ && impl_->client_ && impl_->client_->isConnected();
}

Result<nlohmann::json> StorageClient::get(const std::string& collection, const std::string& id) {
auto opt = impl_->client_->get(collection, id);
if (!opt) {
    return Error{6002, "Document not found: " + collection + "/" + id};
}
return *opt;
}

Result<std::string> StorageClient::insert(const std::string& collection,
                                      const nlohmann::json& data,
                                      const std::string& id,
                                      int64_t ttl_ms) {
auto new_id = impl_->client_->insert(collection, data, id, msToSec(ttl_ms));
if (new_id.empty()) {
    return Error{6000, "Insert failed: " + collection};
}
return new_id;
}

Result<int64_t> StorageClient::update(const std::string& collection,
                                  const std::string& id,
                                  const nlohmann::json& data,
                                  int64_t expected_version,
                                  bool partial) {
if (partial) {
    uint64_t v = impl_->client_->patch(collection, id, data);
    if (v == 0) return Error{6000, "Patch failed: " + collection + "/" + id};
    return static_cast<int64_t>(v);
}
if (expected_version != 0) {
    bool ok = impl_->client_->updateIfVersion(
        collection, id, data, static_cast<uint64_t>(expected_version));
    if (!ok) return Error{6004, "Version conflict: " + collection + "/" + id};
    return expected_version + 1;
}
bool ok = impl_->client_->update(collection, id, data);
if (!ok) return Error{6000, "Update failed: " + collection + "/" + id};
return 0;  // unknown new version when version not requested
}

Result<void> StorageClient::remove(const std::string& collection,
                               const std::string& id,
                               int64_t expected_version) {
(void)expected_version;  // upstream remove() is unconditional; document this
bool ok = impl_->client_->remove(collection, id);
if (!ok) return Error{6002, "Remove failed: " + collection + "/" + id};
return {};
}

Result<QueryResult> StorageClient::query(const std::string& collection) {
QueryOptions opts;
return query(collection, opts);
}

Result<QueryResult> StorageClient::query(const std::string& collection, const QueryOptions& opts) {
dbc::Client::QueryOptions up;
for (const auto& [field, val] : opts.filters) {
    up.filters.push_back(buildFilter(field, val));
}
if (!opts.sorts.empty()) {
    if (opts.sorts.size() > 1) {
        LOG_WARN("Storage: upstream client supports only one sort field; using first");
    }
    up.sortField = opts.sorts.front().first;
    up.sortDescending = !opts.sorts.front().second;
}
if (!opts.fields.empty()) {
    LOG_WARN("Storage: field projection not supported by upstream client; returning all fields");
}
up.limit = static_cast<uint32_t>(opts.page_size);
up.offset = static_cast<uint32_t>(std::max(0, (opts.page - 1) * opts.page_size));

auto fr = impl_->client_->findWithMetrics(collection, up);
QueryResult out;
out.documents = std::move(fr.documents);
out.total_count = static_cast<int64_t>(fr.totalCount);
out.has_more = fr.hasMore;
return out;
}

Result<void> StorageClient::createCollection(const std::string& name,
                                         const nlohmann::json& schema,
                                         int64_t default_ttl_ms,
                                         const VersioningOptions* versioning) {
if (!schema.is_null() && !schema.empty()) {
    LOG_WARN("Storage: createCollection 'schema' parameter is ignored by upstream client (no server-side schema validation in 1.7.x)");
}
uint32_t maxVersions = 0;
if (versioning && versioning->enabled) {
    maxVersions = static_cast<uint32_t>(versioning->max_versions);
    if (versioning->version_ttl_ms != 0 || versioning->keep_on_delete) {
        LOG_WARN("Storage: createCollection version_ttl_ms / keep_on_delete are ignored — upstream 1.7.x createCollection only takes maxVersions");
    }
}
bool ok = impl_->client_->createCollection(
    name,
    msToSec(default_ttl_ms),
    /*encrypted=*/false,
    maxVersions,
    /*vectorDimension=*/0);
if (!ok) return Error{6001, "createCollection failed: " + name};
return {};
}

Result<void> StorageClient::dropCollection(const std::string& name) {
bool ok = impl_->client_->dropCollection(name);
if (!ok) return Error{6001, "dropCollection failed: " + name};
return {};
}

std::vector<std::string> StorageClient::listCollections() {
return impl_->client_->listCollections();
}

Result<nlohmann::json> StorageClient::getVersion(const std::string& collection,
                                             const std::string& id,
                                             int64_t version) {
auto entry = impl_->client_->getDocumentVersion(collection, id, static_cast<uint64_t>(version));
if (!entry) return Error{6010, "Version not found"};
return entry->data;
}

Result<VersionListResult> StorageClient::listVersions(const std::string& collection,
                                                  const std::string& id,
                                                  int32_t limit,
                                                  int32_t offset) {
auto h = impl_->client_->getVersionHistory(collection, id,
                                           static_cast<uint32_t>(limit),
                                           static_cast<uint32_t>(offset));
VersionListResult out;
out.total_count = static_cast<int64_t>(h.totalCount);
out.has_more = (offset + static_cast<int32_t>(h.versions.size())) <
               static_cast<int32_t>(h.totalCount);
for (auto& v : h.versions) {
    DocumentVersionInfo info;
    info.id = id;
    info.document_id = id;
    info.version = static_cast<int64_t>(v.version);
    info.data = std::move(v.data);
    info.created_at = static_cast<int64_t>(v.timestamp);
    out.versions.push_back(std::move(info));
}
return out;
}

} // namespace smartbotic::storage

NOTE: the exact field names on dbc::Client::CollectionOptions (schema, defaultTtlSeconds, versioning.maxVersions, etc.) come from <smartbotic/database/client.hpp>. If they differ, fix at compile time — the structure is the same, only the spelling may shift. Look at /usr/include/smartbotic/database/client.hpp after Task A2 to confirm.

Task A6: Update `lib/storage` CMake target to link upstream client

Files:

Modify: /data/smartbotic/CMakeLists.txt:57-75 (the smartbotic_storage target block)
[ ] Step 1: Replace the storage target block

Find the current block (around lines 57-75). Replace it with:

# Storage client library — adapter over upstream smartbotic-database client
add_library(smartbotic_storage STATIC
    lib/storage/storage_client.cpp
)
target_include_directories(smartbotic_storage PUBLIC
    ${CMAKE_CURRENT_SOURCE_DIR}/lib
)
target_link_libraries(smartbotic_storage PUBLIC
    smartbotic_common
    smartbotic_logging
    nlohmann_json::nlohmann_json
    smartbotic::db-client
)

Note: smartbotic_proto is no longer linked here. The storage client no longer needs the local proto files.

[ ] Step 2: Verify build

cd /data/smartbotic
cmake --build build -j$(nproc) --target smartbotic_storage

Expected: target compiles. If linker complains about a different target name (e.g., smartbotic-db-client::smartbotic-db-client), update the link line and Task A3's find_package block to match.

Task A7: Build webserver + runner against the new adapter

Files: None (build-only)

[ ] Step 1: Build dependent binaries

cd /data/smartbotic
cmake --build build -j$(nproc) --target smartbotic-webserver smartbotic-runner smartbotic-migrate-nodes

Expected: all three targets build clean. The 10 caller files in src/webserver/ should compile without changes because the public adapter API is preserved.

If a caller file breaks: it's relying on a method, return type, or struct field that we changed. Fix at the call site (do not re-add the old internal proto). Most likely culprits:

Code expecting a specific gRPC Status value — handle the Result<T> Error.code instead.
Code reading the now-unused max_message_size_mb — leave it; it's still present.

[ ] Step 2: Commit Phase A part 1

git add CMakeLists.txt lib/storage/storage_client.hpp lib/storage/storage_client.cpp
git commit -m "feat(storage): adapt lib/storage to upstream smartbotic-db-client

Replace gRPC stubs over our internal proto::StorageService with calls into
smartbotic::database::Client (libsmartbotic-db-client). Public API of
StorageClient is unchanged; PIMPL hides the upstream client. Field projection
and multi-sort warnings emitted; TTL converted ms->s with round-up."

Task A8: Remove the `smartbotic-database` executable target

Files:

Modify: /data/smartbotic/CMakeLists.txt:140-180 (the add_executable(smartbotic-database ...) block and any associated target_link_libraries / install rules)
[ ] Step 1: Delete the target block

Locate the section beginning with add_executable(smartbotic-database. Delete it entirely, including its target_link_libraries, target_include_directories, and any install(TARGETS smartbotic-database ...) lines.

[ ] Step 2: Verify config

cd /data/smartbotic
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Debug

Expected: smartbotic-database no longer appears in the target list. (Build dir keeps a stale binary until cleaned — fine.)

[ ] Step 3: Remove src/database/
```
git rm -r src/database/
```

[ ] Step 4: Build remaining targets

rm -rf build
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j$(nproc)

Expected: build/smartbotic-webserver, build/smartbotic-runner, build/smartbotic-migrate-nodes produced. No smartbotic-database.

Task A9: Remove `proto/storage.proto` and its CMake reference

Files:

Delete: /data/smartbotic/proto/storage.proto
Modify: /data/smartbotic/CMakeLists.txt (the proto file list, likely under a set(PROTO_FILES ...) or similar)

[ ] Step 1: Find proto list

grep -n 'storage\.proto\|PROTO_FILES\|protobuf_generate' /data/smartbotic/CMakeLists.txt

[ ] Step 2: Remove proto/storage.proto from the proto sources

Edit the matching line(s) to drop storage.proto. The other protos (common, credentials, runner, workflow) remain — they are not DB-related.

[ ] Step 3: Delete the file
```
git rm proto/storage.proto
```

[ ] Step 4: Reconfigure & rebuild

rm -rf build
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j$(nproc)

Expected: clean build. No references to storage.grpc.pb.h or storage.pb.h anywhere.

[ ] Step 5: Verify

grep -rn 'storage\.grpc\.pb\|storage\.pb\.h\|proto::StorageService\|proto::Document' \
/data/smartbotic --include='*.cpp' --include='*.hpp' \
--exclude-dir=build --exclude-dir=build-asan

Expected: zero matches.

[ ] Step 6: Commit Phase A part 2

git add -A CMakeLists.txt proto src
git commit -m "feat: drop in-repo smartbotic-database service and storage.proto

Remove the entire src/database/ tree and proto/storage.proto. Going forward
all storage goes through libsmartbotic-db-client against the upstream
smartbotic-database daemon, configured via webserver.json / runner.json
'database_address'."

Task A10: Native smoke test against a live upstream `smartbotic-database`

Files: None (runtime test)

Step 1: Start a temporary upstream DB on an alternate port

The host already has smartbotic-database 1.7.1-1 listening on its default port (in use by another project). Run a second instance from the just-installed binary on port 9099:

mkdir -p /tmp/sb-test-db
/usr/bin/smartbotic-database --grpc-port 9099 --data-dir /tmp/sb-test-db &
SB_DB_PID=$!
sleep 2

If the binary's flags differ, point a config file at --config. Adjust based on what /usr/bin/smartbotic-database --help shows.

[ ] Step 2: Run our webserver against it

cd /data/smartbotic
DATABASE_ADDRESS=localhost:9099 \
./build/smartbotic-webserver config/webserver.json &
WS_PID=$!
sleep 3

[ ] Step 3: Hit health + auth endpoints

curl -s http://localhost:8090/api/v1/healthz
TOKEN=$(curl -s http://localhost:8090/api/v1/auth/login \
-H 'Content-Type: application/json' \
-d '{"username":"admin","password":"admin"}' | jq -r '.accessToken')
test -n "$TOKEN" && echo "auth OK" || echo "auth FAILED"

Expected: 200 OK from healthz and a non-empty $TOKEN.

[ ] Step 4: Migrate nodes and list them

curl -s -X POST http://localhost:8090/api/v1/nodes/migrate \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"nodesPath":"./nodes"}'
curl -s http://localhost:8090/api/v1/nodes \
-H "Authorization: Bearer $TOKEN" | jq '.nodes | length'

Expected: a positive node count (matches what nodes/ contains).

[ ] Step 5: Tear down

kill $WS_PID || true
kill $SB_DB_PID || true
rm -rf /tmp/sb-test-db

If any step failed, fix and rerun before continuing to Phase B.

Phase B — Packaging

Task B1: Create `VERSION` file

Files:

Create: /data/smartbotic/VERSION

[ ] Step 1: Write the file

echo "1.0.0" > /data/smartbotic/VERSION
git add VERSION

(Plain text, single line, no trailing newline issue — printf if you want strict.)

Task B2: Create `packaging/` directory tree

Files:

Create: directory layout under /data/smartbotic/packaging/

[ ] Step 1: Create directories

cd /data/smartbotic
mkdir -p packaging/deb/templates \
     packaging/deb/scripts \
     packaging/deb/systemd \
     packaging/deb/config \
     packaging/scripts

[ ] Step 2: Copy the SmartBotics APT public key

cp /data/smartbotics-deb-repo/smartbotics-repo.gpg packaging/smartbotics-repo.gpg

Task B3: Write `packaging/Dockerfile.base`

Files:

Create: /data/smartbotic/packaging/Dockerfile.base

[ ] Step 1: Write the file

# smartbotic-automation Build Base Image
# Pre-installed Debian 13 build dependencies for fast iterative builds.
#
# Build:
#   docker buildx build -f packaging/Dockerfile.base \
#     --build-arg REPO_PASS=<password> \
#     -t smartbotic-automation-build-base:debian13 .

FROM debian:trixie

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \
&& rm -rf /var/lib/apt/lists/*

# SmartBotics APT repository (for libsmartbotic-db-client-dev)
ARG REPO_USER=callerai
ARG REPO_PASS
COPY packaging/smartbotics-repo.gpg /usr/share/keyrings/smartbotics-repo.gpg
RUN echo "deb [signed-by=/usr/share/keyrings/smartbotics-repo.gpg] https://repository.smartbotics.ai trixie main" \
    > /etc/apt/sources.list.d/smartbotics.list \
&& printf "machine repository.smartbotics.ai\nlogin %s\npassword %s\n" \
    "$REPO_USER" "$REPO_PASS" > /etc/apt/auth.conf.d/smartbotics.conf \
&& chmod 600 /etc/apt/auth.conf.d/smartbotics.conf

RUN apt-get update && apt-get install -y --no-install-recommends \
    # Build tools
    build-essential cmake ninja-build git pkg-config ccache dpkg-dev \
    # Compression / crypto / uuid
    libssl-dev zlib1g-dev libbrotli-dev uuid-dev \
    # gRPC + protobuf
    protobuf-compiler libprotobuf-dev libgrpc++-dev protobuf-compiler-grpc \
    # JSON + logging + fmt
    nlohmann-json3-dev libspdlog-dev libfmt-dev \
    # bcrypt / curl
    libbcrypt-dev libcurl4-openssl-dev \
    # WebUI build (Node.js + npm)
    nodejs npm \
    # Upstream smartbotic-database client (the whole point)
    libsmartbotic-db-client-dev \
&& rm -rf /var/lib/apt/lists/* \
&& echo "smartbotic-automation:debian13:$(date -u +%Y-%m-%dT%H:%M:%SZ)" > /etc/smartbotic-automation-build-base

NOTE: if libbcrypt-dev is not packaged in trixie, drop that line and rely on the in-tree libbcrypt_lib.a that the project already vendors (visible in your local build/).

Task B4: Write `packaging/Dockerfile.build`

Files:

Create: /data/smartbotic/packaging/Dockerfile.build

[ ] Step 1: Write the file

# syntax=docker/dockerfile:1.4
# smartbotic-automation Package Builder
# Compiles the C++ binaries + WebUI bundle, then assembles a single .deb.
#
# Build:
#   docker buildx build -f packaging/Dockerfile.build \
#     --build-arg BASE_IMAGE=smartbotic-automation-build-base:debian13 \
#     --build-arg BUILD_VERSION=1.0.0 \
#     --target packages \
#     --output "type=local,dest=dist/debian13/" .

ARG BASE_IMAGE=debian:trixie
FROM ${BASE_IMAGE} AS builder

ARG BUILD_VERSION=0.0.0
ARG BUILD_DEB_REVISION=1
ARG BUILD_JOBS=8
ARG BUILD_GIT_COMMIT=unknown
ENV DEBIAN_FRONTEND=noninteractive
ENV BUILD_VERSION=${BUILD_VERSION}
ENV BUILD_DEB_REVISION=${BUILD_DEB_REVISION}
ENV BUILD_JOBS=${BUILD_JOBS}

ENV CCACHE_DIR=/ccache
ENV CCACHE_MAXSIZE=5G
ENV PATH="/usr/lib/ccache:${PATH}"

# Fall back to fresh dep install if the base image isn't our prebuilt one
RUN if [ ! -f /etc/smartbotic-automation-build-base ]; then \
    apt-get update && apt-get install -y --no-install-recommends \
        build-essential cmake ninja-build git pkg-config ccache dpkg-dev \
        libssl-dev zlib1g-dev libbrotli-dev uuid-dev \
        protobuf-compiler libprotobuf-dev libgrpc++-dev protobuf-compiler-grpc \
        nlohmann-json3-dev libspdlog-dev libfmt-dev libcurl4-openssl-dev \
        nodejs npm libsmartbotic-db-client-dev \
    && rm -rf /var/lib/apt/lists/*; \
else echo "Using pre-built base: $(cat /etc/smartbotic-automation-build-base)"; fi

WORKDIR /build

# WebUI build (cached separately from C++)
COPY webui/package.json webui/package-lock.json ./webui/
RUN --mount=type=cache,target=/npm-cache,id=smartbotic-automation-npm-cache \
cd webui && npm ci --cache /npm-cache
COPY VERSION ./
COPY webui/ ./webui/
RUN cd webui && npm run build

# C++ build
COPY CMakeLists.txt ./
COPY cmake/ ./cmake/
COPY proto/ ./proto/
COPY lib/ ./lib/
COPY src/ ./src/
COPY config/ ./config/
COPY nodes/ ./nodes/
COPY packaging/ ./packaging/

RUN --mount=type=cache,target=/ccache,id=smartbotic-automation-ccache \
cmake -B build -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_C_COMPILER_LAUNCHER=ccache \
    -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
&& cmake --build build --parallel ${BUILD_JOBS} \
&& (ccache --show-stats || true)

# ----- Package assembly stage -----
FROM builder AS packager
RUN mkdir -p /packages
RUN chmod +x /build/packaging/scripts/create-deb.sh \
&& OUTPUT_DIR=/packages /build/packaging/scripts/create-deb.sh

# ----- Final export stage -----
FROM scratch AS packages
COPY --from=packager /packages/*.deb /

Task B5: Write `packaging/scripts/create-deb.sh`

Files:

Create: /data/smartbotic/packaging/scripts/create-deb.sh

[ ] Step 1: Write the file

#!/usr/bin/env bash
# Assemble the smartbotic-automation .deb tree and run dpkg-deb.
# Runs INSIDE the Docker build container after `cmake --build`.
set -euo pipefail

PROJECT_DIR="${PROJECT_DIR:-/build}"
OUTPUT_DIR="${OUTPUT_DIR:-/packages}"
VERSION="${BUILD_VERSION:-$(cat "$PROJECT_DIR/VERSION")}"
REVISION="${BUILD_DEB_REVISION:-1}"
ARCH="amd64"
NAME="smartbotic-automation"
PKG_VERSION="${VERSION}-${REVISION}"

PKG_ROOT="$(mktemp -d)"
trap "rm -rf $PKG_ROOT" EXIT

# Directory layout inside the .deb
install -d "$PKG_ROOT/DEBIAN"
install -d "$PKG_ROOT/usr/bin"
install -d "$PKG_ROOT/usr/share/$NAME/webui"
install -d "$PKG_ROOT/usr/share/$NAME/nodes"
install -d "$PKG_ROOT/etc/$NAME"
install -d "$PKG_ROOT/lib/systemd/system"
install -d "$PKG_ROOT/var/lib/$NAME"
install -d "$PKG_ROOT/var/log/$NAME"

# Binaries
install -m 0755 "$PROJECT_DIR/build/smartbotic-webserver"     "$PKG_ROOT/usr/bin/"
install -m 0755 "$PROJECT_DIR/build/smartbotic-runner"        "$PKG_ROOT/usr/bin/"
install -m 0755 "$PROJECT_DIR/build/smartbotic-migrate-nodes" "$PKG_ROOT/usr/bin/"

# WebUI bundle
cp -a "$PROJECT_DIR/webui/dist/." "$PKG_ROOT/usr/share/$NAME/webui/"

# Default workflow nodes
cp -a "$PROJECT_DIR/nodes/." "$PKG_ROOT/usr/share/$NAME/nodes/"

# Default configs (will become conffiles via DEBIAN/conffiles)
install -m 0644 "$PROJECT_DIR/packaging/deb/config/webserver.json" "$PKG_ROOT/etc/$NAME/webserver.json"
install -m 0644 "$PROJECT_DIR/packaging/deb/config/runner.json"    "$PKG_ROOT/etc/$NAME/runner.json"
install -m 0644 "$PROJECT_DIR/packaging/deb/config/env"            "$PKG_ROOT/etc/$NAME/env"

# Systemd units
install -m 0644 "$PROJECT_DIR/packaging/deb/systemd/$NAME-webserver.service" \
            "$PKG_ROOT/lib/systemd/system/"
install -m 0644 "$PROJECT_DIR/packaging/deb/systemd/$NAME-runner.service" \
            "$PKG_ROOT/lib/systemd/system/"

# Maintainer scripts
for script in preinst postinst prerm postrm; do
install -m 0755 "$PROJECT_DIR/packaging/deb/scripts/$script" "$PKG_ROOT/DEBIAN/$script"
done

# Conffiles
cat > "$PKG_ROOT/DEBIAN/conffiles" <<EOF
/etc/$NAME/webserver.json
/etc/$NAME/runner.json
/etc/$NAME/env
EOF

# Render control file from template
INSTALLED_SIZE=$(du -sk "$PKG_ROOT" --exclude=DEBIAN | cut -f1)
sed -e "s/{{VERSION}}/${PKG_VERSION}/g" \
-e "s/{{INSTALLED_SIZE}}/${INSTALLED_SIZE}/g" \
"$PROJECT_DIR/packaging/deb/templates/control.smartbotic-automation" \
> "$PKG_ROOT/DEBIAN/control"

# Build the .deb
mkdir -p "$OUTPUT_DIR"
DEB_FILE="$OUTPUT_DIR/${NAME}_${PKG_VERSION}_${ARCH}.deb"
dpkg-deb --build --root-owner-group "$PKG_ROOT" "$DEB_FILE"
echo "Built: $DEB_FILE"
ls -lh "$DEB_FILE"

[ ] Step 2: Make executable

chmod +x /data/smartbotic/packaging/scripts/create-deb.sh

Task B6: Write `packaging/deb/templates/control.smartbotic-automation`

Files:

Create: /data/smartbotic/packaging/deb/templates/control.smartbotic-automation

[ ] Step 1: Write the file

Package: smartbotic-automation
Version: {{VERSION}}
Architecture: amd64
Maintainer: Ferenc Szontagh <ferenc.szontagh@smartbotics.ai>
Description: SmartBotic Automation — workflow execution platform
HTTP/WebSocket API server, workflow runner, and built-in JavaScript node
set for the SmartBotic automation platform. Persists workflows, executions,
credentials and runtime state in the upstream smartbotic-database daemon
(via libsmartbotic-db-client). Includes a React WebUI bundle.
Proprietary software by SmartBotics Kft.
Homepage: https://smartbotics.ai
Section: contrib/utils
Priority: optional
License: proprietary
Depends: smartbotic-database (>= 1.7.5), libsmartbotic-db-client (>= 1.7.5), libssl3t64, libprotobuf32t64, libgrpc++1.51t64, libspdlog1.15, libfmt10, libuuid1, libcurl4t64, zlib1g, libbrotli1, nodejs
Installed-Size: {{INSTALLED_SIZE}}

NOTE: the exact tXX suffix on Debian library packages depends on the time64 transition state of trixie at build time. Verify with apt-cache search libssl3 inside the base image; adjust if the suffix is just libssl3 or libssl3t64. The libcurl4t64 line may need to drop to libcurl4 if your trixie snapshot predates the transition.

Task B7: Write systemd units

Files:

Create: /data/smartbotic/packaging/deb/systemd/smartbotic-automation-webserver.service
Create: /data/smartbotic/packaging/deb/systemd/smartbotic-automation-runner.service

[ ] Step 1: Write the webserver unit

[Unit]
Description=SmartBotic Automation — Webserver (HTTP/WS API)
Documentation=https://smartbotics.ai
After=network-online.target smartbotic-database.service
Wants=network-online.target smartbotic-database.service

[Service]
Type=simple
User=smartbotic-automation
Group=smartbotic-automation
ExecStart=/usr/bin/smartbotic-webserver --config /etc/smartbotic-automation/webserver.json
WorkingDirectory=/var/lib/smartbotic-automation
EnvironmentFile=-/etc/smartbotic-automation/env
Restart=on-failure
RestartSec=5
TimeoutStopSec=15
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

[ ] Step 2: Write the runner unit

[Unit]
Description=SmartBotic Automation — Runner (workflow execution)
Documentation=https://smartbotics.ai
After=network-online.target smartbotic-automation-webserver.service
Wants=network-online.target smartbotic-automation-webserver.service

[Service]
Type=simple
User=smartbotic-automation
Group=smartbotic-automation
ExecStart=/usr/bin/smartbotic-runner --config /etc/smartbotic-automation/runner.json
WorkingDirectory=/var/lib/smartbotic-automation
EnvironmentFile=-/etc/smartbotic-automation/env
Restart=on-failure
RestartSec=5
TimeoutStopSec=15
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Task B8: Write default config files

Files:

Create: /data/smartbotic/packaging/deb/config/webserver.json
Create: /data/smartbotic/packaging/deb/config/runner.json
Create: /data/smartbotic/packaging/deb/config/env

[ ] Step 1: Write webserver.json

{
"http_port": 8090,
"node_sync_port": 9012,
"credential_service_port": 9013,
"static_files_path": "${WEBUI_PATH:/usr/share/smartbotic-automation/webui}",
"database_address": "${DATABASE_ADDRESS:localhost:9010}",
"runners": {
"load_balancing": "least-connections",
"heartbeat_timeout_sec": 30,
"offline_removal_sec": 60
},
"auth": {
"jwt_secret": "${JWT_SECRET:CHANGE_ME_BEFORE_PRODUCTION}",
"access_token_lifetime_sec": 900,
"refresh_token_lifetime_sec": 604800
},
"credentials": {
"master_key": "${CREDENTIALS_MASTER_KEY:CHANGE_ME_BEFORE_PRODUCTION}",
"pbkdf2_iterations": 100000
},
"cors": {
"enabled": true,
"origins": ["*"]
},
"logging": {
"level": "${LOG_LEVEL:info}",
"file": "${LOG_FILE:/var/log/smartbotic-automation/webserver.log}"
}
}

[ ] Step 2: Write runner.json

{
"grpc_port": 9011,
"runner_id": "${RUNNER_ID:runner-1}",
"webserver_address": "${WEBSERVER_ADDRESS:localhost:8090}",
"node_sync_address": "${NODE_SYNC_ADDRESS:localhost:9012}",
"credential_service_address": "${CREDENTIAL_SERVICE_ADDRESS:localhost:9013}",
"database_address": "${DATABASE_ADDRESS:localhost:9010}",
"max_message_size_mb": 64,
"node_sync": {
"enabled": true,
"reconnect_interval_ms": 5000
},
"registration": {
"heartbeat_interval_sec": 10,
"max_concurrent_executions": 10
},
"execution": {
"default_timeout_ms": 60000,
"max_memory_per_script_mb": 64
},
"logging": {
"level": "${LOG_LEVEL:info}",
"file": "${LOG_FILE:/var/log/smartbotic-automation/runner.log}"
}
}

[ ] Step 3: Write env (empty by default; admins set DATABASE_ADDRESS, secrets here)

# /etc/smartbotic-automation/env
# Override config values with environment variables here.
# Example:
# DATABASE_ADDRESS=localhost:9010
# JWT_SECRET=...
# CREDENTIALS_MASTER_KEY=...
# LOG_LEVEL=info

Task B9: Write maintainer scripts

Files:

Create: /data/smartbotic/packaging/deb/scripts/preinst
Create: /data/smartbotic/packaging/deb/scripts/postinst
Create: /data/smartbotic/packaging/deb/scripts/prerm
Create: /data/smartbotic/packaging/deb/scripts/postrm

[ ] Step 1: Write preinst

#!/bin/bash
# preinst for smartbotic-automation — minimal pre-install checks
set -e

case "$1" in
install|upgrade)
    # No-op; postinst handles user/group + dirs
    ;;
esac

exit 0

[ ] Step 2: Write postinst

#!/bin/bash
# postinst for smartbotic-automation
set -e

SM_USER="smartbotic-automation"
SM_GROUP="smartbotic-automation"
SM_HOME="/home/smartbotic-automation"
SERVICES="smartbotic-automation-webserver smartbotic-automation-runner"

case "$1" in
configure)
    # System group + user (idempotent)
    if ! getent group "$SM_GROUP" >/dev/null 2>&1; then
        addgroup --system "$SM_GROUP"
    fi
    if ! getent passwd "$SM_USER" >/dev/null 2>&1; then
        adduser --system --ingroup "$SM_GROUP" --home "$SM_HOME" \
            --shell /usr/sbin/nologin \
            --gecos "SmartBotic Automation Service User" "$SM_USER"
    fi

    install -d -o "$SM_USER" -g "$SM_GROUP" -m 0750 "$SM_HOME"
    install -d -o "$SM_USER" -g "$SM_GROUP" -m 0750 /var/lib/smartbotic-automation
    install -d -o "$SM_USER" -g "$SM_GROUP" -m 0750 /var/log/smartbotic-automation
    install -d -o root      -g "$SM_GROUP" -m 0750 /etc/smartbotic-automation

    # systemd
    if [ -d /run/systemd/system ]; then
        systemctl daemon-reload
    fi

    if [ -z "$2" ]; then
        # First install: enable but do NOT start (admin must configure
        # database_address first; the upstream DB port is deployment-specific).
        for svc in $SERVICES; do
            deb-systemd-helper enable "$svc.service" >/dev/null || true
        done
        echo
        echo "smartbotic-automation: services are enabled but not started."
        echo "  1. Edit /etc/smartbotic-automation/webserver.json (set database_address)"
        echo "  2. Edit /etc/smartbotic-automation/runner.json (set database_address)"
        echo "  3. systemctl start smartbotic-automation-webserver smartbotic-automation-runner"
        echo
    else
        # Upgrade path: try-restart any active units
        for svc in $SERVICES; do
            deb-systemd-invoke try-restart "$svc.service" >/dev/null 2>&1 || true
        done
    fi
    ;;
esac

#DEBHELPER#
exit 0

NOTE: the #DEBHELPER# marker is harmless if we're not using debhelper — dpkg-deb --build ignores it. Leaving it in keeps the script compatible if we later switch to a debian/ directory layout.

[ ] Step 3: Write prerm

#!/bin/bash
# prerm for smartbotic-automation
set -e

SERVICES="smartbotic-automation-webserver smartbotic-automation-runner"

case "$1" in
remove|deconfigure)
    if [ -d /run/systemd/system ]; then
        for svc in $SERVICES; do
            deb-systemd-invoke stop "$svc.service" >/dev/null 2>&1 || true
            deb-systemd-helper disable "$svc.service" >/dev/null 2>&1 || true
        done
    fi
    ;;
upgrade|failed-upgrade)
    if [ -d /run/systemd/system ]; then
        for svc in $SERVICES; do
            deb-systemd-invoke stop "$svc.service" >/dev/null 2>&1 || true
        done
    fi
    ;;
esac

exit 0

[ ] Step 4: Write postrm

#!/bin/bash
# postrm for smartbotic-automation
set -e

SM_USER="smartbotic-automation"
SM_GROUP="smartbotic-automation"

case "$1" in
purge)
    rm -rf /var/lib/smartbotic-automation /var/log/smartbotic-automation
    if getent passwd "$SM_USER" >/dev/null 2>&1; then
        deluser --system "$SM_USER" >/dev/null 2>&1 || true
    fi
    if getent group "$SM_GROUP" >/dev/null 2>&1; then
        delgroup --system "$SM_GROUP" >/dev/null 2>&1 || true
    fi
    ;;
remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear)
    if [ -d /run/systemd/system ]; then
        systemctl daemon-reload >/dev/null 2>&1 || true
    fi
    ;;
esac

exit 0

[ ] Step 5: Make all scripts executable

chmod +x /data/smartbotic/packaging/deb/scripts/{preinst,postinst,prerm,postrm}

Task B10: Write `packaging/build.sh`

Files:

Create: /data/smartbotic/packaging/build.sh

[ ] Step 1: Write the file

#!/usr/bin/env bash
# smartbotic-automation production build orchestrator.
# Modeled on /data/shadowman-cpp/packaging/build.sh.
#
# Usage:
#   packaging/build.sh                          # Build with version from VERSION
#   packaging/build.sh 1.2.0                    # Override version
#   packaging/build.sh --rebuild-base           # Force rebuild of base image
#   packaging/build.sh --sync --suite trixie    # Build + publish to APT repo
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
cd "$PROJECT_DIR"

BASE_IMAGE_NAME="smartbotic-automation-build-base:debian13"
OUTPUT_DIR="${PROJECT_DIR}/dist/debian13"
BUILD_JOBS="${BUILD_JOBS:-$(nproc)}"
DEB_REPO_DIR="${DEB_REPO_DIR:-/data/smartbotics-deb-repo}"

RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; BLUE='\033[0;34m'; NC='\033[0m'
log_info()    { echo -e "${BLUE}[INFO]${NC} $1"; }
log_success() { echo -e "${GREEN}[OK]${NC} $1"; }
log_warn()    { echo -e "${YELLOW}[WARN]${NC} $1"; }
log_error()   { echo -e "${RED}[ERROR]${NC} $1"; }

REBUILD_BASE=false
NO_CACHE=false
BUILD_REPO=false
SYNC_REPO=false
REPO_SUITE=""
VERSION=""
DEB_REVISION="1"

while [[ $# -gt 0 ]]; do
case $1 in
    --rebuild-base)  REBUILD_BASE=true; shift ;;
    --no-cache)      NO_CACHE=true; shift ;;
    --deb-revision)  DEB_REVISION="$2"; shift 2 ;;
    --repo)          BUILD_REPO=true; shift ;;
    --suite)         REPO_SUITE="$2"; shift 2 ;;
    --sync)          SYNC_REPO=true; BUILD_REPO=true; shift ;;
    --help|-h)
        cat <<EOF
smartbotic-automation Production Build Script

Usage: $0 [OPTIONS] [VERSION]

Options:
--rebuild-base       Force rebuild the base image
--no-cache           Build without using BuildKit caches
--deb-revision N     Debian revision suffix (default: 1)
--repo               Generate APT repo from built packages
--suite NAME         Distribution suite for repo (e.g., trixie)
--sync               Generate repo and sync to repository.smartbotics.ai

Environment:
BUILD_JOBS           Parallel build jobs (default: nproc)
DEB_REPO_DIR         Path to smartbotics-deb-repo (default: /data/smartbotics-deb-repo)
EOF
        exit 0 ;;
    *) VERSION="$1"; shift ;;
esac
done

if [ -z "$VERSION" ]; then
VERSION="$(tr -d '[:space:]' < VERSION 2>/dev/null || true)"
if [ -z "$VERSION" ]; then
    log_error "Could not read version from VERSION file"; exit 1
fi
fi

if ! command -v docker &>/dev/null; then
log_error "Docker not found in PATH"; exit 1
fi

base_image_exists() { docker image inspect "$BASE_IMAGE_NAME" &>/dev/null; }
build_base_image() {
log_info "Building base image: $BASE_IMAGE_NAME"
local repo_pass=""
if [ -f "${DEB_REPO_DIR}/.env" ]; then
    repo_pass=$(grep -oP 'RepoPass:\s*\K.*' "${DEB_REPO_DIR}/.env" | tr -d '[:space:]')
fi
if [ -z "$repo_pass" ]; then
    log_error "Could not read RepoPass from ${DEB_REPO_DIR}/.env"; exit 1
fi
docker buildx build \
    -f packaging/Dockerfile.base \
    --build-arg REPO_PASS="$repo_pass" \
    -t "$BASE_IMAGE_NAME" \
    .
log_success "Base image built"
}

if $REBUILD_BASE; then
build_base_image
elif base_image_exists; then
log_success "Base image cached: $BASE_IMAGE_NAME"
else
build_base_image
fi

GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo unknown)

rm -rf "$OUTPUT_DIR"; mkdir -p "$OUTPUT_DIR"

BUILD_ARGS=(
-f packaging/Dockerfile.build
--build-arg BASE_IMAGE="$BASE_IMAGE_NAME"
--build-arg BUILD_VERSION="$VERSION"
--build-arg BUILD_DEB_REVISION="$DEB_REVISION"
--build-arg BUILD_GIT_COMMIT="$GIT_COMMIT"
--build-arg BUILD_JOBS="$BUILD_JOBS"
--target packages
--output "type=local,dest=$OUTPUT_DIR"
)
$NO_CACHE && BUILD_ARGS+=(--no-cache)

log_info "Building smartbotic-automation ${VERSION}-${DEB_REVISION} (commit ${GIT_COMMIT})"
docker buildx build "${BUILD_ARGS[@]}" .

log_success "Built packages:"
ls -lh "$OUTPUT_DIR"/*.deb

if $BUILD_REPO; then
if [ -z "$REPO_SUITE" ]; then
    log_error "--suite is required with --repo"; exit 1
fi
if [ ! -d "$DEB_REPO_DIR" ]; then
    log_error "Repository dir not found: $DEB_REPO_DIR"; exit 1
fi
"$DEB_REPO_DIR/scripts/add-packages.sh" "$OUTPUT_DIR"
"$DEB_REPO_DIR/scripts/create-repo.sh" --suite "$REPO_SUITE"
fi

if $SYNC_REPO; then
"$DEB_REPO_DIR/scripts/sync-repo.sh"
fi

log_success "Done."

[ ] Step 2: Make executable

chmod +x /data/smartbotic/packaging/build.sh

Task B11: Build the base image

Files: None (Docker)

[ ] Step 1: Run base build

cd /data/smartbotic
./packaging/build.sh --rebuild-base 2>&1 | tail -40

Expected: ends with [OK] Base image built. Verify with:

docker image inspect smartbotic-automation-build-base:debian13 | jq '.[0].Size' \
    | awk '{ printf "Base image size: %.0f MB\n", $1/1024/1024 }'

If apt fails to find libsmartbotic-db-client-dev, check that:

The repo password from .env resolved (RepoPass: line is present)
The trixie suite has been published in /data/smartbotics-deb-repo/repo/dists/trixie/

Task B12: Build the `.deb`

Files: None (Docker output to dist/debian13/)

[ ] Step 1: Build

cd /data/smartbotic
./packaging/build.sh 2>&1 | tail -30

Expected: a single smartbotic-automation_1.0.0-1_amd64.deb in dist/debian13/.

[ ] Step 2: Inspect the package metadata

dpkg-deb -I dist/debian13/smartbotic-automation_1.0.0-1_amd64.deb

Expected: lists Maintainer, Depends including smartbotic-database (>= 1.7.5) and libsmartbotic-db-client (>= 1.7.5), sane Installed-Size.

[ ] Step 3: Inspect file list

dpkg-deb -c dist/debian13/smartbotic-automation_1.0.0-1_amd64.deb | head -40

Expected: /usr/bin/smartbotic-{webserver,runner,migrate-nodes}, /usr/share/smartbotic-automation/webui/..., /etc/smartbotic-automation/..., systemd units, owned root:root mode-correct.

[ ] Step 4: Lint with lintian (best effort)

lintian dist/debian13/smartbotic-automation_1.0.0-1_amd64.deb 2>&1 | head -20 || true

lintian warnings are not fatal — fix any errors (E:) and any warnings about systemd units, postinst exit codes, or missing depends. Skip purely cosmetic warnings (no manpage, etc.).

Task B13: Commit Phase B

Files: None (git)

[ ] Step 1: Stage and commit

cd /data/smartbotic
git add VERSION packaging/
git commit -m "build: add Debian 13 packaging for smartbotic-automation

Single .deb mirroring shadowman-cpp packaging style. Two-stage Docker build
(Dockerfile.base + Dockerfile.build) targeting trixie. build.sh wraps
buildx and integrates with /data/smartbotics-deb-repo for --repo / --sync.
Installs binaries to /usr/bin, configs to /etc/smartbotic-automation,
data to /var/lib/smartbotic-automation, with systemd units for webserver
and runner."

Phase C — Smoke Test

Task C1: Spin up a fresh Debian 13 container

Files: None (host)

[ ] Step 1: Start container with the .deb mounted

docker run -it --rm \
-v /data/smartbotic/dist/debian13:/debs:ro \
-v /data/smartbotics-deb-repo/smartbotics-repo.gpg:/etc/apt/keyrings/sb.gpg:ro \
--name sb-auto-test \
debian:trixie bash

Inside the container, the next steps run.

[ ] Step 2: Configure the SmartBotics APT repo (for the smartbotic-database dep)

apt-get update && apt-get install -y curl ca-certificates
echo "deb [signed-by=/etc/apt/keyrings/sb.gpg] https://repository.smartbotics.ai trixie main" \
> /etc/apt/sources.list.d/smartbotics.list
# Use the repo creds from /data/smartbotics-deb-repo/.env
printf "machine repository.smartbotics.ai\nlogin %s\npassword %s\n" \
callerai "$(grep -oP 'RepoPass:\s*\K.*' /etc/apt/auth.conf.d/smartbotics.conf || echo)" \
> /etc/apt/auth.conf.d/smartbotics.conf
apt-get update

(Adjust if you mount the password file differently.)

[ ] Step 3: Install the upstream DB then our package

apt-get install -y /debs/smartbotic-automation_1.0.0-1_amd64.deb

Expected: APT pulls smartbotic-database, libsmartbotic-db-client, nodejs, etc. as dependencies. Postinst runs, prints the "configure database_address" message, exits 0.

Task C2: Configure and start

Files: Inside container — /etc/smartbotic-automation/{webserver,runner}.json

[ ] Step 1: Start upstream smartbotic-database

systemctl start smartbotic-database.service
systemctl status smartbotic-database --no-pager | head -10
ss -tlnp | grep smartbotic-database  # confirm a port

Note the port (likely 9010). If different, update both config files' database_address.

[ ] Step 2: Start our services

systemctl start smartbotic-automation-webserver smartbotic-automation-runner
systemctl status smartbotic-automation-webserver --no-pager | head -15
systemctl status smartbotic-automation-runner --no-pager | head -15

Expected: both active (running). If the runner exits because it can't reach the webserver, restart in order with a sleep 2 between.

Task C3: Functional smoke check

Files: None

[ ] Step 1: Health + auth

curl -s http://localhost:8090/api/v1/healthz
TOKEN=$(curl -s http://localhost:8090/api/v1/auth/login \
-H 'Content-Type: application/json' \
-d '{"username":"admin","password":"admin"}' | jq -r '.accessToken')
echo "TOKEN length: ${#TOKEN}"

Expected: healthz returns 200; token is non-empty (>50 chars).

[ ] Step 2: Migrate nodes

curl -s -X POST http://localhost:8090/api/v1/nodes/migrate \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"nodesPath":"/usr/share/smartbotic-automation/nodes"}' | jq
curl -s http://localhost:8090/api/v1/nodes \
-H "Authorization: Bearer $TOKEN" | jq '.nodes | length'

Expected: a positive node count.

[ ] Step 3: Verify WebUI is served

curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8090/
curl -s http://localhost:8090/ | grep -oE '<title>[^<]*</title>' | head -1

Expected: 200 and a <title> containing "SmartBotic" or similar.

[ ] Step 4: Stop, then test purge

systemctl stop smartbotic-automation-webserver smartbotic-automation-runner
apt-get purge -y smartbotic-automation
test -d /var/lib/smartbotic-automation && echo "FAIL: data dir still present" || echo "OK: data dir purged"
test -d /etc/smartbotic-automation && echo "INFO: config dir kept (conffile dpkg behavior)" || true
getent passwd smartbotic-automation && echo "FAIL: user still present" || echo "OK: user purged"

Expected: data dir gone, user gone, config dir may remain (dpkg conffile semantics).

[ ] Step 5: Exit container
```
exit
```

If any step failed, fix in source, rebuild via Phase B Task B12, retry Phase C from scratch.

Phase D — Repository Publish

Task D1: Build with `--sync`

Files: None

[ ] Step 1: Confirm repo-side state is clean

ls -lh /data/smartbotic/dist/debian13/*.deb
ls /data/smartbotics-deb-repo/dist/ 2>/dev/null

If /data/smartbotics-deb-repo/dist/ already has a stale smartbotic-automation_*.deb, remove it first:

rm -f /data/smartbotics-deb-repo/dist/smartbotic-automation_*.deb

[ ] Step 2: Run sync

cd /data/smartbotic
./packaging/build.sh --sync --suite trixie 2>&1 | tail -40

Expected: build completes, add-packages.sh copies into dist/, create-repo.sh --suite trixie regenerates repo/dists/trixie/, sync-repo.sh uploads to repository.smartbotics.ai.

Task D2: Verify from a clean trixie container

Files: None

[ ] Step 1: Pull and install over the network

docker run --rm debian:trixie bash -c '
apt-get update && apt-get install -y curl ca-certificates gnupg
curl -fsSL https://repository.smartbotics.ai/smartbotics-repo.gpg \
    | gpg --dearmor -o /usr/share/keyrings/smartbotics-repo.gpg
echo "deb [signed-by=/usr/share/keyrings/smartbotics-repo.gpg] https://repository.smartbotics.ai trixie main" \
    > /etc/apt/sources.list.d/smartbotics.list
apt-get update 2>&1 | grep -E "smartbotic|Err|^E:" || true
apt-cache policy smartbotic-automation
'

(If the repo is auth-protected, supply credentials via auth.conf.d.)

Expected: apt-cache policy smartbotic-automation shows the version 1.0.0-1 with https://repository.smartbotics.ai trixie/main as candidate source.

Step 2: Final commit (if any tweaks were needed)

If Phase C or D revealed issues that required source edits:

cd /data/smartbotic
git add -A
git commit -m "fix(packaging): <whatever was tweaked>"
./packaging/build.sh --sync --suite trixie  # re-publish

Otherwise nothing to commit.

Self-Review Checklist (run before handoff)

Spec §3 (code migration) — covered by tasks A1, A4, A5, A6, A7, A8, A9
Spec §4 (file layout) — covered by Task B5 (create-deb.sh stages exactly the spec'd paths)
Spec §4.1 (systemd) — covered by Task B7
Spec §5 (deps) — covered by Task B6
Spec §6 (maintainer scripts) — covered by Task B9
Spec §7 (packaging/ layout) — covered by Tasks B2-B10
Spec §8 (APT repo deploy) — covered by Tasks D1, D2
Spec §9 (no local cleanup) — confirmed in design; Phase C explicitly uses a separate container
Spec §10 decisions 1-5 — all reflected: drop src/database/ (A8-9), single smartbotic-automation package (B6), /usr/bin install (B5), unrelated 1.7.1 instance left alone (test in container), enable-but-don't-start (B9 postinst)
Spec §11 (out of scope) — no tasks for repackaging upstream DB, no cross-distro builds, no data migration; correct

2026-05-09-smartbotic-automation-debian-packaging.md 59 KB 文件历史 原始文件

smartbotic-automation Debian Packaging Implementation Plan

File Structure Map

Created

Modified

Deleted

Phase A — Code Migration

Task A1: Confirm smartbotic-migrate-nodes has no direct DB dependency

Task A2: Install libsmartbotic-db-client-dev for native dev build

Task A3: Add libsmartbotic-db-client to CMakeLists.txt

Task A4: Rewrite lib/storage/storage_client.hpp

Task A5: Rewrite lib/storage/storage_client.cpp

Task A6: Update lib/storage CMake target to link upstream client

Task A7: Build webserver + runner against the new adapter

Task A8: Remove the smartbotic-database executable target

Task A9: Remove proto/storage.proto and its CMake reference

Task A10: Native smoke test against a live upstream smartbotic-database

Phase B — Packaging

Task B1: Create VERSION file

Task B2: Create packaging/ directory tree

Task B3: Write packaging/Dockerfile.base

Task B4: Write packaging/Dockerfile.build

Task B5: Write packaging/scripts/create-deb.sh

Task B6: Write packaging/deb/templates/control.smartbotic-automation

Task B7: Write systemd units

Task B8: Write default config files

Task B9: Write maintainer scripts

Task B10: Write packaging/build.sh

Task B11: Build the base image

Task B12: Build the .deb

Task B13: Commit Phase B

Phase C — Smoke Test

Task C1: Spin up a fresh Debian 13 container

Task C2: Configure and start

Task C3: Functional smoke check

Phase D — Repository Publish

Task D1: Build with --sync

Task D2: Verify from a clean trixie container

Self-Review Checklist (run before handoff)

2026-05-09-smartbotic-automation-debian-packaging.md 59 KB

文件历史原始文件

Task A1: Confirm `smartbotic-migrate-nodes` has no direct DB dependency

Task A2: Install `libsmartbotic-db-client-dev` for native dev build

Task A3: Add `libsmartbotic-db-client` to `CMakeLists.txt`

Task A4: Rewrite `lib/storage/storage_client.hpp`

Task A5: Rewrite `lib/storage/storage_client.cpp`

Task A6: Update `lib/storage` CMake target to link upstream client

Task A8: Remove the `smartbotic-database` executable target

Task A9: Remove `proto/storage.proto` and its CMake reference

Task A10: Native smoke test against a live upstream `smartbotic-database`

Task B1: Create `VERSION` file

Task B2: Create `packaging/` directory tree

Task B3: Write `packaging/Dockerfile.base`

Task B4: Write `packaging/Dockerfile.build`

Task B5: Write `packaging/scripts/create-deb.sh`

Task B6: Write `packaging/deb/templates/control.smartbotic-automation`

Task B10: Write `packaging/build.sh`

Task B12: Build the `.deb`

Task D1: Build with `--sync`