# smartbotic-automation: Debian 13 packaging + upstream DB migration

**Date:** 2026-05-09
**Status:** Design — pending user approval
**Owner:** Ferenc Szontagh

## 1. Goal

Ship the contents of `/data/smartbotic` as a single Debian 13 (`trixie`) `.deb` package named `smartbotic-automation`, distributed through the SmartBotics APT repository at `https://repository.smartbotics.ai`.

As part of this work, drop the in-repo `src/database/` service entirely and have our remaining services (`smartbotic-webserver`, `smartbotic-runner`, `smartbotic-migrate-nodes`) talk to the standalone upstream `smartbotic-database` daemon via its `libsmartbotic-db-client` library (upstream repo: <https://git.smartbotics.ai/fszontagh/smartbotic-database>, current version `1.7.5`).

Packaging style follows `/data/shadowman-cpp/packaging/` and `/tmp/smartbotic-database-upstream/packaging/` — Docker-based two-stage build targeting Debian 13, with `build.sh --sync` integration into `/data/smartbotics-deb-repo/`.

## 2. Identity & metadata

| Field | Value |
| --- | --- |
| Package name | `smartbotic-automation` |
| Version source | `/data/smartbotic/VERSION` (new file, initial value `1.0.0` — matches `project(SmartBotic VERSION 1.0.0 …)` in `CMakeLists.txt`) |
| Architecture | `amd64` |
| Maintainer | `Ferenc Szontagh <ferenc.szontagh@smartbotics.ai>` |
| Section | `contrib/utils` |
| Priority | `optional` |
| License | proprietary (`Proprietary software by SmartBotics Kft.`) |
| Homepage | `https://smartbotics.ai` |
| Target distro | Debian 13 (`trixie`) |

## 3. Code migration: drop internal DB, adopt upstream client

### 3.1 Removals

- `src/database/` (entire directory: `database_service.{cpp,hpp}`, `memory_store.{cpp,hpp}`, `document.hpp`, `main.cpp`, `persistence/{wal,snapshot}.{cpp,hpp}`)
- `proto/storage.proto` (replaced by upstream's `database.proto`, which we do not need to ship — upstream client hides gRPC behind PIMPL)
- The `smartbotic-database` CMake executable target (currently at `CMakeLists.txt:152`)
- The current `lib/storage/storage_client.{cpp,hpp}` implementations (file paths kept; contents replaced — see §3.2)

### 3.2 `lib/storage/` adapter

Keep the existing public interface so that the 10 caller files do not need to change:

- `src/webserver/webserver_service.{hpp,cpp}`
- `src/webserver/runners/runner_registry.{hpp,cpp}`
- `src/webserver/nodes/node_store.{hpp,cpp}`
- `src/webserver/api/database_controller.cpp`
- `src/webserver/api/workflow_group_controller.cpp`
- `src/webserver/api/webhook_controller.{hpp,cpp}`

Internally, `lib/storage/storage_client.cpp` becomes a thin adapter over `smartbotic::database::Client` (`<smartbotic/database/client.hpp>` from `libsmartbotic-db-client-dev`):

| Adapter API (kept stable) | Upstream call |
| --- | --- |
| `Result<json> get(coll, id)` | wraps `std::optional<json> Client::get(...)` |
| `Result<string> insert(coll, data, id, ttl_ms)` | `Client::insert(...)` (note: upstream takes `ttlSeconds` — we convert) |
| `Result<int64> update(coll, id, data, expected_version, partial)` | dispatch by flags: `partial=false, expected_version=0` → `Client::update`; `partial=false, expected_version!=0` → `Client::updateIfVersion`; `partial=true` → `Client::patch` |
| `Result<void> remove(coll, id, expected_version)` | `Client::remove` / `Client::removeIfVersion` |
| `QueryResult query(coll, options)` | `Client::find(...)` (translate filter/sort enums) |

Translation rules:
- TTL: caller passes ms; upstream takes seconds. Adapter divides by 1000, rounding up; documents the loss-of-precision in a one-line comment.
- Connection address: adapter `StorageClientConfig::address` default changes from `localhost:9001` to `localhost:9004` (upstream default), and is config-overridable through `webserver.json` / `runner.json`.
- Versioning options (`VersioningOptions`, `DocumentVersionInfo`, `VersionListResult`) map directly onto upstream's version-history API.

If a method on our existing surface has no upstream equivalent (none expected on a quick read of `client.hpp`), it gets removed and any caller adapted at the same time.

### 3.3 CMake changes

- Remove `src/database/*` source list and the `add_executable(smartbotic-database ...)` block.
- Replace internal `proto/storage.proto` references in `lib/storage/`'s target with a `find_package(libsmartbotic-db-client REQUIRED)` (or `pkg_check_modules` fallback). `target_link_libraries(smartbotic_storage PUBLIC smartbotic::database::client)` (exact target name to be confirmed when reading the installed CMake config).
- `proto/credentials.proto`, `proto/runner.proto`, `proto/workflow.proto`, `proto/common.proto` remain — they are between webserver and runner, not the DB.

### 3.4 `smartbotic-migrate-nodes`

Verify (in implementation phase) that this binary only talks to the webserver via HTTP `/api/v1/nodes/migrate` and has no direct DB dependency. If it does have a direct DB dependency, port it to the new client at the same time.

## 4. Installed file layout (system-wide)

```
/usr/bin/smartbotic-webserver
/usr/bin/smartbotic-runner
/usr/bin/smartbotic-migrate-nodes

/usr/share/smartbotic-automation/webui/         # built React app (`webui/dist/`)
/usr/share/smartbotic-automation/nodes/         # default workflow node JS modules

/etc/smartbotic-automation/webserver.json       # conffile
/etc/smartbotic-automation/runner.json          # conffile
/etc/smartbotic-automation/env                  # optional EnvironmentFile (empty default, conffile)

/var/lib/smartbotic-automation/                 # mode 0750, owned smartbotic-automation:smartbotic-automation
/var/log/smartbotic-automation/                 # mode 0750, owned smartbotic-automation:smartbotic-automation

/lib/systemd/system/smartbotic-automation-webserver.service
/lib/systemd/system/smartbotic-automation-runner.service
```

Both default config files point `database.address` at `localhost:9004`. Admin overrides per-deployment.

### 4.1 systemd units

`smartbotic-automation-webserver.service`:

```ini
[Unit]
Description=SmartBotic Automation — Webserver (HTTP/WS API)
After=network.target smartbotic-database.service
Wants=smartbotic-database.service

[Service]
Type=simple
User=smartbotic-automation
Group=smartbotic-automation
ExecStart=/usr/bin/smartbotic-webserver --config /etc/smartbotic-automation/webserver.json
WorkingDirectory=/var/lib/smartbotic-automation
EnvironmentFile=-/etc/smartbotic-automation/env
Restart=on-failure
RestartSec=5
TimeoutStopSec=15
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

`smartbotic-automation-runner.service` is structurally identical, with `ExecStart=/usr/bin/smartbotic-runner --config /etc/smartbotic-automation/runner.json` and `After=network.target smartbotic-automation-webserver.service`.

The unit *Wants* `smartbotic-database.service` — that upstream daemon is a `Depends:` of our package. The admin is responsible for ensuring it listens on the port `webserver.json` points at (default `9004`; in test environments where the host already has another `smartbotic-database` instance, run ours on an alternate port and override).

## 5. Dependencies

```
Depends:
  smartbotic-database (>= 1.7.5),
  libsmartbotic-db-client (>= 1.7.5),
  libssl3t64,
  libprotobuf32t64,
  libgrpc++1.51t64,
  libspdlog1.15,
  libfmt10,
  libuuid1,
  zlib1g,
  nodejs

Build-Depends:
  libsmartbotic-db-client-dev (>= 1.7.5),
  libgrpc++-dev,
  libprotobuf-dev,
  protobuf-compiler,
  protobuf-compiler-grpc,
  libssl-dev,
  libspdlog-dev,
  libfmt-dev,
  libuuid-dev,
  nlohmann-json3-dev,
  nodejs,
  npm,
  cmake (>= 3.20),
  ninja-build,
  g++ (>= 12)
```

(The exact versioned names match Debian 13 / `trixie` package names — the `t64` suffix is the time64 transition. We mirror ShadowMan's list and trim to what we actually link.)

## 6. Maintainer scripts

### 6.1 `preinst`

No-op for `install` and `upgrade` (mirrors ShadowMan).

### 6.2 `postinst configure`

1. Create system group + user (idempotent), home `/home/smartbotic-automation`, shell `/usr/sbin/nologin`.
2. `install -d -o smartbotic-automation -g smartbotic-automation -m 0750` on:
   - `/home/smartbotic-automation`
   - `/var/lib/smartbotic-automation`
   - `/var/log/smartbotic-automation`
   - `/etc/smartbotic-automation` (only if missing)
3. `systemctl daemon-reload`
4. **First install:** `deb-systemd-helper enable smartbotic-automation-webserver.service smartbotic-automation-runner.service` — **but do not start them.** Reason: the operator must first verify that `smartbotic-database` is reachable at the configured port (default `9004`) before starting our services. A `NEWS.Debian` or echo at install time will tell the admin to run `systemctl start smartbotic-automation-webserver smartbotic-automation-runner` once the DB is ready.
5. **Upgrade:** `deb-systemd-invoke try-restart` for any unit that was already active.

### 6.3 `prerm remove` / `prerm upgrade`

`deb-systemd-invoke stop` both units. On `prerm remove`: also `deb-systemd-helper disable` them.

### 6.4 `postrm purge`

- `deluser --system smartbotic-automation` (idempotent)
- `delgroup --system smartbotic-automation` (idempotent)
- `rm -rf /var/lib/smartbotic-automation /var/log/smartbotic-automation`
- Leave `/etc/smartbotic-automation/` to dpkg's normal conffile handling.

## 7. `packaging/` directory layout (added to `/data/smartbotic`)

```
packaging/
  Dockerfile.base                 # Debian 13 + build deps + libsmartbotic-db-client-dev
  Dockerfile.build                # COPY source, build C++ + npm run build, dpkg-deb assembly
  build.sh                        # orchestrator
  smartbotics-repo.gpg            # public APT key (copied from /data/smartbotics-deb-repo/)
  deb/
    templates/
      control.smartbotic-automation
    scripts/
      preinst
      postinst
      prerm
      postrm
    systemd/
      smartbotic-automation-webserver.service
      smartbotic-automation-runner.service
    config/
      webserver.json
      runner.json
      env
```

### 7.1 `Dockerfile.base`

Mirrors ShadowMan's `Dockerfile.base`:

- `FROM debian:trixie`
- Configure SmartBotics APT repo with `REPO_USER` / `REPO_PASS` build args (used to install `libsmartbotic-db-client-dev`)
- Install build toolchain: `build-essential cmake ninja-build git pkg-config ccache libssl-dev libprotobuf-dev libgrpc++-dev protobuf-compiler-grpc nlohmann-json3-dev libspdlog-dev libfmt-dev libuuid-dev zlib1g-dev nodejs npm libsmartbotic-db-client-dev`

Image tag: `smartbotic-automation-build-base:debian13`.

### 7.2 `Dockerfile.build`

- `FROM smartbotic-automation-build-base:debian13`
- Mount source as `/src`, build dir `/build`, output dir `/out`
- `cmake -G Ninja -B /build -S /src -DCMAKE_BUILD_TYPE=Release`
- `cmake --build /build -j${BUILD_JOBS}`
- `npm --prefix /src/webui ci && npm --prefix /src/webui run build`
- `bash /src/packaging/scripts/assemble-deb.sh /build /src/webui/dist /out` — writes the `DEBIAN/` payload tree, runs `dpkg-deb --build`
- Final `.deb` lands in `/out/`

### 7.3 `build.sh` flag surface (matches ShadowMan + upstream)

```
[VERSION]
--rebuild-base
--no-cache
--deb-revision N        (default 1)
--repo                  (run /data/smartbotics-deb-repo/scripts/{add-packages,create-repo --suite})
--suite NAME            (default trixie)
--sync                  (--repo + sync-repo.sh)
--help / -h
```

Env: `BUILD_JOBS` (default `nproc`), `DEB_REPO_DIR` (default `/data/smartbotics-deb-repo`).

Outputs land in `/data/smartbotic/dist/debian13/smartbotic-automation_<version>-<rev>_amd64.deb`.

## 8. APT repository deployment

`build.sh --sync --suite trixie` chains:

1. `${DEB_REPO_DIR}/scripts/add-packages.sh dist/debian13/`
2. `${DEB_REPO_DIR}/scripts/create-repo.sh --suite trixie`
3. `${DEB_REPO_DIR}/scripts/sync-repo.sh`

End state: `apt install smartbotic-automation` works on a `trixie` host that has the SmartBotics repo configured.

## 9. Local-instance cleanup (this host)

Confirmed during exploration:

- Nothing built from `/data/smartbotic` is currently running as a service or process.
- The system service `smartbotic-database.service` running `/usr/bin/smartbotic-database` (installed package `smartbotic-database 1.7.1-1`) belongs to a different project — left intact per user direction.

No cleanup action needed on this host. Testing of the new package will happen on a separate VM or with a side-by-side smartbotic-database instance running on an alternate port.

## 10. Decisions explicitly approved by user

1. **DB strategy:** delete `src/database/`, depend on upstream `smartbotic-database` package, talk to it via `libsmartbotic-db-client`.
2. **Package layout:** single `smartbotic-automation` package (no split).
3. **Install prefix:** system-wide (`/usr/bin/`, `/usr/share/smartbotic-automation/`, `/etc/smartbotic-automation/`).
4. **Local instance:** leave the unrelated `smartbotic-database 1.7.1-1` system service alone.
5. **Auto-start policy on first install:** `enable` units, but **do not start** — admin must point config at a reachable `smartbotic-database` and start manually (the upstream DB port is deployment-specific).

## 11. Out of scope

- Repackaging the upstream `smartbotic-database` itself.
- Any frontend / `webui/` feature work beyond `npm run build`.
- Cross-distro builds (only `trixie` is targeted).
- Migrating the existing `/data/smartbotic` `smartbotic-database` *data* (WAL, snapshots) — operators run a fresh upstream instance for their deployment.

## 12. Implementation phases (for the plan)

1. **Phase A — code migration**: remove `src/database/`, rewrite `lib/storage/`, fix CMake, ensure native build (`cmake --build build`) succeeds.
2. **Phase B — packaging skeleton**: add `packaging/` directory, write `Dockerfile.base`, `Dockerfile.build`, `build.sh`, control template, maintainer scripts, systemd units, default configs, `VERSION` file. Confirm Docker build produces a `.deb`.
3. **Phase C — install / smoke test**: install the `.deb` in a throwaway container or VM, start the upstream `smartbotic-database` on the configured port, start `smartbotic-automation-webserver`, hit `/api/v1/healthz`, confirm node migration and basic workflow works.
4. **Phase D — repo publish**: `build.sh --sync --suite trixie`. Verify `apt update && apt install smartbotic-automation` works from a clean `trixie` host.