smartbotic-automation: Debian 13 packaging + upstream DB migration

Date: 2026-05-09 Status: Design — pending user approval Owner: Ferenc Szontagh

1. Goal

Ship the contents of /data/smartbotic as a single Debian 13 (trixie) .deb package named smartbotic-automation, distributed through the SmartBotics APT repository at https://repository.smartbotics.ai.

As part of this work, drop the in-repo src/database/ service entirely and have our remaining services (smartbotic-webserver, smartbotic-runner, smartbotic-migrate-nodes) talk to the standalone upstream smartbotic-database daemon via its libsmartbotic-db-client library (upstream repo: https://git.smartbotics.ai/fszontagh/smartbotic-database, current version 1.7.5).

Packaging style follows /data/shadowman-cpp/packaging/ and /tmp/smartbotic-database-upstream/packaging/ — Docker-based two-stage build targeting Debian 13, with build.sh --sync integration into /data/smartbotics-deb-repo/.

2. Identity & metadata

Field	Value
Package name	`smartbotic-automation`
Version source	`/data/smartbotic/VERSION` (new file, initial value `1.0.0` — matches `project(SmartBotic VERSION 1.0.0 …)` in `CMakeLists.txt`)
Architecture	`amd64`
Maintainer	`Ferenc Szontagh <ferenc.szontagh@smartbotics.ai>`
Section	`contrib/utils`
Priority	`optional`
License	proprietary (`Proprietary software by SmartBotics Kft.`)
Homepage	`https://smartbotics.ai`
Target distro	Debian 13 (`trixie`)

3. Code migration: drop internal DB, adopt upstream client

3.1 Removals

src/database/ (entire directory: database_service.{cpp,hpp}, memory_store.{cpp,hpp}, document.hpp, main.cpp, persistence/{wal,snapshot}.{cpp,hpp})
proto/storage.proto (replaced by upstream's database.proto, which we do not need to ship — upstream client hides gRPC behind PIMPL)
The smartbotic-database CMake executable target (currently at CMakeLists.txt:152)
The current lib/storage/storage_client.{cpp,hpp} implementations (file paths kept; contents replaced — see §3.2)

3.2 `lib/storage/` adapter

Keep the existing public interface so that the 10 caller files do not need to change:

src/webserver/webserver_service.{hpp,cpp}
src/webserver/runners/runner_registry.{hpp,cpp}
src/webserver/nodes/node_store.{hpp,cpp}
src/webserver/api/database_controller.cpp
src/webserver/api/workflow_group_controller.cpp
src/webserver/api/webhook_controller.{hpp,cpp}

Internally, lib/storage/storage_client.cpp becomes a thin adapter over smartbotic::database::Client (<smartbotic/database/client.hpp> from libsmartbotic-db-client-dev):

Adapter API (kept stable)	Upstream call
`Result<json> get(coll, id)`	wraps `std::optional<json> Client::get(...)`
`Result<string> insert(coll, data, id, ttl_ms)`	`Client::insert(...)` (note: upstream takes `ttlSeconds` — we convert)
`Result<int64> update(coll, id, data, expected_version, partial)`	dispatch by flags: `partial=false, expected_version=0` → `Client::update`; `partial=false, expected_version!=0` → `Client::updateIfVersion`; `partial=true` → `Client::patch`
`Result<void> remove(coll, id, expected_version)`	`Client::remove` / `Client::removeIfVersion`
`QueryResult query(coll, options)`	`Client::find(...)` (translate filter/sort enums)

Translation rules:

TTL: caller passes ms; upstream takes seconds. Adapter divides by 1000, rounding up; documents the loss-of-precision in a one-line comment.
Connection address: adapter StorageClientConfig::address default changes from localhost:9001 to localhost:9004 (upstream default), and is config-overridable through webserver.json / runner.json.
Versioning options (VersioningOptions, DocumentVersionInfo, VersionListResult) map directly onto upstream's version-history API.

If a method on our existing surface has no upstream equivalent (none expected on a quick read of client.hpp), it gets removed and any caller adapted at the same time.

3.3 CMake changes

Remove src/database/* source list and the add_executable(smartbotic-database ...) block.
Replace internal proto/storage.proto references in lib/storage/'s target with a find_package(libsmartbotic-db-client REQUIRED) (or pkg_check_modules fallback). target_link_libraries(smartbotic_storage PUBLIC smartbotic::database::client) (exact target name to be confirmed when reading the installed CMake config).
proto/credentials.proto, proto/runner.proto, proto/workflow.proto, proto/common.proto remain — they are between webserver and runner, not the DB.

3.4 `smartbotic-migrate-nodes`

Verify (in implementation phase) that this binary only talks to the webserver via HTTP /api/v1/nodes/migrate and has no direct DB dependency. If it does have a direct DB dependency, port it to the new client at the same time.

4. Installed file layout (system-wide)

/usr/bin/smartbotic-webserver
/usr/bin/smartbotic-runner
/usr/bin/smartbotic-migrate-nodes

/usr/share/smartbotic-automation/webui/         # built React app (`webui/dist/`)
/usr/share/smartbotic-automation/nodes/         # default workflow node JS modules

/etc/smartbotic-automation/webserver.json       # conffile
/etc/smartbotic-automation/runner.json          # conffile
/etc/smartbotic-automation/env                  # optional EnvironmentFile (empty default, conffile)

/var/lib/smartbotic-automation/                 # mode 0750, owned smartbotic-automation:smartbotic-automation
/var/log/smartbotic-automation/                 # mode 0750, owned smartbotic-automation:smartbotic-automation

/lib/systemd/system/smartbotic-automation-webserver.service
/lib/systemd/system/smartbotic-automation-runner.service

Both default config files point database.address at localhost:9004. Admin overrides per-deployment.

4.1 systemd units

smartbotic-automation-webserver.service:

[Unit]
Description=SmartBotic Automation — Webserver (HTTP/WS API)
After=network.target smartbotic-database.service
Wants=smartbotic-database.service

[Service]
Type=simple
User=smartbotic-automation
Group=smartbotic-automation
ExecStart=/usr/bin/smartbotic-webserver --config /etc/smartbotic-automation/webserver.json
WorkingDirectory=/var/lib/smartbotic-automation
EnvironmentFile=-/etc/smartbotic-automation/env
Restart=on-failure
RestartSec=5
TimeoutStopSec=15
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

smartbotic-automation-runner.service is structurally identical, with ExecStart=/usr/bin/smartbotic-runner --config /etc/smartbotic-automation/runner.json and After=network.target smartbotic-automation-webserver.service.

The unit Wants smartbotic-database.service — that upstream daemon is a Depends: of our package. The admin is responsible for ensuring it listens on the port webserver.json points at (default 9004; in test environments where the host already has another smartbotic-database instance, run ours on an alternate port and override).

5. Dependencies

Depends:
  smartbotic-database (>= 1.7.5),
  libsmartbotic-db-client (>= 1.7.5),
  libssl3t64,
  libprotobuf32t64,
  libgrpc++1.51t64,
  libspdlog1.15,
  libfmt10,
  libuuid1,
  zlib1g,
  nodejs

Build-Depends:
  libsmartbotic-db-client-dev (>= 1.7.5),
  libgrpc++-dev,
  libprotobuf-dev,
  protobuf-compiler,
  protobuf-compiler-grpc,
  libssl-dev,
  libspdlog-dev,
  libfmt-dev,
  libuuid-dev,
  nlohmann-json3-dev,
  nodejs,
  npm,
  cmake (>= 3.20),
  ninja-build,
  g++ (>= 12)

(The exact versioned names match Debian 13 / trixie package names — the t64 suffix is the time64 transition. We mirror ShadowMan's list and trim to what we actually link.)

6. Maintainer scripts

6.1 `preinst`

No-op for install and upgrade (mirrors ShadowMan).

6.2 `postinst configure`

Create system group + user (idempotent), home /home/smartbotic-automation, shell /usr/sbin/nologin.
install -d -o smartbotic-automation -g smartbotic-automation -m 0750 on:
- /home/smartbotic-automation
- /var/lib/smartbotic-automation
- /var/log/smartbotic-automation
- /etc/smartbotic-automation (only if missing)
systemctl daemon-reload
First install: deb-systemd-helper enable smartbotic-automation-webserver.service smartbotic-automation-runner.service — but do not start them. Reason: the operator must first verify that smartbotic-database is reachable at the configured port (default 9004) before starting our services. A NEWS.Debian or echo at install time will tell the admin to run systemctl start smartbotic-automation-webserver smartbotic-automation-runner once the DB is ready.
Upgrade: deb-systemd-invoke try-restart for any unit that was already active.

6.3 `prerm remove` / `prerm upgrade`

deb-systemd-invoke stop both units. On prerm remove: also deb-systemd-helper disable them.

6.4 `postrm purge`

deluser --system smartbotic-automation (idempotent)
delgroup --system smartbotic-automation (idempotent)
rm -rf /var/lib/smartbotic-automation /var/log/smartbotic-automation
Leave /etc/smartbotic-automation/ to dpkg's normal conffile handling.

7. `packaging/` directory layout (added to `/data/smartbotic`)

packaging/
  Dockerfile.base                 # Debian 13 + build deps + libsmartbotic-db-client-dev
  Dockerfile.build                # COPY source, build C++ + npm run build, dpkg-deb assembly
  build.sh                        # orchestrator
  smartbotics-repo.gpg            # public APT key (copied from /data/smartbotics-deb-repo/)
  deb/
    templates/
      control.smartbotic-automation
    scripts/
      preinst
      postinst
      prerm
      postrm
    systemd/
      smartbotic-automation-webserver.service
      smartbotic-automation-runner.service
    config/
      webserver.json
      runner.json
      env

7.1 `Dockerfile.base`

Mirrors ShadowMan's Dockerfile.base:

FROM debian:trixie
Configure SmartBotics APT repo with REPO_USER / REPO_PASS build args (used to install libsmartbotic-db-client-dev)
Install build toolchain: build-essential cmake ninja-build git pkg-config ccache libssl-dev libprotobuf-dev libgrpc++-dev protobuf-compiler-grpc nlohmann-json3-dev libspdlog-dev libfmt-dev libuuid-dev zlib1g-dev nodejs npm libsmartbotic-db-client-dev

Image tag: smartbotic-automation-build-base:debian13.

7.2 `Dockerfile.build`

FROM smartbotic-automation-build-base:debian13
Mount source as /src, build dir /build, output dir /out
cmake -G Ninja -B /build -S /src -DCMAKE_BUILD_TYPE=Release
cmake --build /build -j${BUILD_JOBS}
npm --prefix /src/webui ci && npm --prefix /src/webui run build
bash /src/packaging/scripts/assemble-deb.sh /build /src/webui/dist /out — writes the DEBIAN/ payload tree, runs dpkg-deb --build
Final .deb lands in /out/

7.3 `build.sh` flag surface (matches ShadowMan + upstream)

[VERSION]
--rebuild-base
--no-cache
--deb-revision N        (default 1)
--repo                  (run /data/smartbotics-deb-repo/scripts/{add-packages,create-repo --suite})
--suite NAME            (default trixie)
--sync                  (--repo + sync-repo.sh)
--help / -h

Env: BUILD_JOBS (default nproc), DEB_REPO_DIR (default /data/smartbotics-deb-repo).

Outputs land in /data/smartbotic/dist/debian13/smartbotic-automation_<version>-<rev>_amd64.deb.

8. APT repository deployment

build.sh --sync --suite trixie chains:

${DEB_REPO_DIR}/scripts/add-packages.sh dist/debian13/
${DEB_REPO_DIR}/scripts/create-repo.sh --suite trixie
${DEB_REPO_DIR}/scripts/sync-repo.sh

End state: apt install smartbotic-automation works on a trixie host that has the SmartBotics repo configured.

9. Local-instance cleanup (this host)

Confirmed during exploration:

Nothing built from /data/smartbotic is currently running as a service or process.
The system service smartbotic-database.service running /usr/bin/smartbotic-database (installed package smartbotic-database 1.7.1-1) belongs to a different project — left intact per user direction.

No cleanup action needed on this host. Testing of the new package will happen on a separate VM or with a side-by-side smartbotic-database instance running on an alternate port.

10. Decisions explicitly approved by user

DB strategy: delete src/database/, depend on upstream smartbotic-database package, talk to it via libsmartbotic-db-client.
Package layout: single smartbotic-automation package (no split).
Install prefix: system-wide (/usr/bin/, /usr/share/smartbotic-automation/, /etc/smartbotic-automation/).
Local instance: leave the unrelated smartbotic-database 1.7.1-1 system service alone.
Auto-start policy on first install: enable units, but do not start — admin must point config at a reachable smartbotic-database and start manually (the upstream DB port is deployment-specific).

11. Out of scope

Repackaging the upstream smartbotic-database itself.
Any frontend / webui/ feature work beyond npm run build.
Cross-distro builds (only trixie is targeted).
Migrating the existing /data/smartbotic smartbotic-database data (WAL, snapshots) — operators run a fresh upstream instance for their deployment.

12. Implementation phases (for the plan)

Phase A — code migration: remove src/database/, rewrite lib/storage/, fix CMake, ensure native build (cmake --build build) succeeds.
Phase B — packaging skeleton: add packaging/ directory, write Dockerfile.base, Dockerfile.build, build.sh, control template, maintainer scripts, systemd units, default configs, VERSION file. Confirm Docker build produces a .deb.
Phase C — install / smoke test: install the .deb in a throwaway container or VM, start the upstream smartbotic-database on the configured port, start smartbotic-automation-webserver, hit /api/v1/healthz, confirm node migration and basic workflow works.
Phase D — repo publish: build.sh --sync --suite trixie. Verify apt update && apt install smartbotic-automation works from a clean trixie host.

2026-05-09-smartbotic-automation-debian-packaging-design.md 14 KB 히스토리 Raw

smartbotic-automation: Debian 13 packaging + upstream DB migration

1. Goal

2. Identity & metadata

3. Code migration: drop internal DB, adopt upstream client

3.1 Removals

3.2 lib/storage/ adapter

3.3 CMake changes

3.4 smartbotic-migrate-nodes

4. Installed file layout (system-wide)

4.1 systemd units

5. Dependencies

6. Maintainer scripts

6.1 preinst

6.2 postinst configure

6.3 prerm remove / prerm upgrade

6.4 postrm purge

7. packaging/ directory layout (added to /data/smartbotic)

7.1 Dockerfile.base

7.2 Dockerfile.build

7.3 build.sh flag surface (matches ShadowMan + upstream)