fix: give the containers the trust the host used to provide implicitly

The SD.cpp service on mulan is self-signed. On zeus its certificate was in the
system store, so every client trusted it without anything saying so; a container
has a clean store and the two sdcpp cases failed with "SSL peer certificate or
SSH remote key was not OK".

The app's own trusted-certificate store was not the answer. It works, but it is
opt-in per workflow (settings.certificateIds), which is right for a certificate
belonging to one integration and wrong for reaching a service this whole
deployment depends on - it would have to be repeated in every workflow that
happens to touch SD.cpp. So the trust lives in the deployment: deploy/zeus/ca/
holds the extra certificates and build-bundle.sh splices them onto the runtime
image's own CA set.

The bundle is mounted OVER /etc/ssl/certs/ca-certificates.crt rather than
beside it, which is the only thing that works here and is worth writing down.
The runner never sets CURLOPT_CAINFO, so libcurl falls back to its compiled-in
default - that exact path. libcurl reads neither SSL_CERT_FILE (OpenSSL's) nor
CURL_CA_BUNDLE (the curl command-line tool's), so pointing either at a bundle
elsewhere looks correct and changes nothing. Both were tried first and neither
moved the result.

The bundle is read out of the image, not off the host: the containers trust
Debian's CA set and zeus is Void, so splicing the host's roots in would hand
them a different set than they were built with.

.gitignore

@@ -62,3 +62,7 @@ dist/
 
 # Playwright MCP session output
 .playwright-mcp/
+
+# Generated by deploy/zeus/ca/build-bundle.sh from the runtime image's own CA
+# set plus the certificates beside it - not a source file.
+deploy/zeus/ca/bundle.crt

deploy/zeus/ca/README.md

@@ -0,0 +1,24 @@
+# Extra CA certificates for the containers
+
+Certificates this deployment must trust that a public CA does not vouch for.
+Currently just mulan's SD.cpp service, which is self-signed.
+
+`build-bundle.sh` concatenates the runtime image's own CA bundle with
+everything here and writes `bundle.crt`, which compose mounts and points
+`SSL_CERT_FILE` at. Run it again after changing this directory or rebuilding
+the image on a newer Debian.
+
+## Why not the app's own trusted-certificate store?
+
+That exists and works, but it is **opt-in per workflow**
+(`workflow.settings.certificateIds`) - which is right for a certificate that
+belongs to one integration. Reaching mulan's SD.cpp is infrastructure for this
+whole deployment, so it belongs here rather than repeated in every workflow
+that happens to touch it. Before the move to containers this was zeus's own
+system store doing the same job.
+
+## Why not bake it into the image?
+
+The image should not carry one host's certificate - it would have to be
+rebuilt to add or rotate one, and the same image should run in a deployment
+that has never heard of mulan.

deploy/zeus/ca/build-bundle.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+# Combine the runtime image's CA bundle with the extra certificates here.
+#
+# Read out of the image rather than off the host: the container trusts Debian's
+# CA set, and zeus is Void - splicing the host's bundle in would hand the
+# containers a different set of roots than the image was built with.
+set -eu
+cd "$(dirname "$0")"
+IMAGE="${IMAGE:-smartbotic-automation:current}"
+
+docker run --rm --entrypoint cat "$IMAGE" /etc/ssl/certs/ca-certificates.crt > bundle.crt
+for cert in *.crt; do
+    [ "$cert" = "bundle.crt" ] && continue
+    printf '\n# %s\n' "$cert" >> bundle.crt
+    cat "$cert" >> bundle.crt
+done
+echo "bundle.crt: $(grep -c 'BEGIN CERTIFICATE' bundle.crt) certificates"

deploy/zeus/ca/mulan-sdcpp.crt

@@ -0,0 +1,19 @@
+-----BEGIN CERTIFICATE-----
+MIIDHzCCAgegAwIBAgIUEo/Z/caw9NvKTnnrTTMNqhkcxBswDQYJKoZIhvcNAQEL
+BQAwEDEOMAwGA1UEAwwFbXVsYW4wHhcNMjYwODE0MDgxMDM4WhcNMjgxMTE2MDgx
+MDM4WjAQMQ4wDAYDVQQDDAVtdWxhbjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
+AQoCggEBAJHHAoPguNjvu1Oh2eY0NLc8mNOfKAgN1WMT3cmoFJXcvwYyNf/pCgYz
+c3y+xdU2OoZTORgfzMRGs9shlBizdO+ovE6Sh2A1K4EJPfKaLiYm+dGFu1RD6nS5
+kHQc4RMlLokodrbUYXQGe6El9El+HqA4tWIOWuGRkat0KnpJP/d4LcNpy1WfZV6+
+l6zEEdPTGUYFYc9un9ABZ44mU/J3ZEKwRTF59B0fMSgD4McBg6lOzdVNjK9uJw2f
+nyCcl0t4gjaFqAuqZUmcss52g6hxgzZw/qsoBHWVbYr+T0WctFAfTlX9rpzp9iPZ
+ahVjWULrix6ECUvviGJLIAfUs+2WXx8CAwEAAaNxMG8wIQYDVR0RBBowGIIJbG9j
+YWxob3N0ggVtdWxhbocEfwAAATAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDATBgNV
+HSUEDDAKBggrBgEFBQcDATAdBgNVHQ4EFgQU0I6yMVkHregZi29/ZOY9CM2s9ZQw
+DQYJKoZIhvcNAQELBQADggEBAEFwbrvGyUeNhj37Gue5gEmHGe6RRabg9Lx2gqFa
+B7aezXp5uAMdAXj/FiQsSv7bJHRJXMM0vsNhpXq2PvVGnomwuD5/DyFOm/j0YQgE
+wIajvSZolfRiRJOIgqh/eTawN1r7GfIgSL/pcode8JvJ9PHsB/eMCOJPjA5QMLch
+5uFNKruaJev9uQBEznk4QKGcQ2wUWAF0tWMvRj5f5oIQ5MWAQmy15T01u6YYYBsu
+ScJv/kLg82Rd/g5DajSOhg/NWR5sTZbmKRYEgHVm+btbRg4p6Um+xTjuOkx5T1Jz
+qIeu0Qg9rLAys1nTZCfFIyIkEo6uPW4biVD0abKEdpCFQvg=
+-----END CERTIFICATE-----

deploy/zeus/docker-compose.services.yml

@@ -36,6 +36,10 @@ services:
       LOG_LEVEL: info
       DATABASE_ADDRESS: host.docker.internal:9004
       DATABASE_PROJECT: smartbotic-automation
+      # For anything that talks to OpenSSL directly. libcurl ignores it, which
+      # is why the bundle is also mounted over curl's default CA path below -
+      # see the volume comment.
+      SSL_CERT_FILE: /etc/ssl/certs/ca-certificates.crt
     volumes:
       # Read-only: it carries the credentials master key and the JWT secret,
       # and nothing should be rewriting it from inside a container.
@@ -43,6 +47,16 @@ services:
       # Bind-mounted rather than used from the image so a node can be edited and
       # re-migrated without a rebuild, which is how they are worked on today.
       - /data/dev/smartbotics/smartbotic/nodes:/usr/share/smartbotic-automation/nodes:ro
+      # Debian's CA set plus the extra certificates in ca/ - currently mulan's
+      # self-signed SD.cpp cert. Regenerate with ca/build-bundle.sh.
+      #
+      # Mounted OVER curl's default CA file rather than beside it. The runner
+      # never sets CURLOPT_CAINFO, so libcurl uses its compiled-in default,
+      # which on Debian is exactly this path - and libcurl reads neither
+      # SSL_CERT_FILE (that is OpenSSL's) nor CURL_CA_BUNDLE (that is the curl
+      # command-line tool's). Pointing an environment variable at a bundle
+      # somewhere else looks like it should work and silently does nothing.
+      - /data/dev/smartbotics/smartbotic/deploy/zeus/ca/bundle.crt:/etc/ssl/certs/ca-certificates.crt:ro
     healthcheck:
       # bash's /dev/tcp, because a slim image carries no curl or wget.
       #
@@ -83,12 +97,26 @@ services:
       # "localhost:9011", which inside a container means the webserver's own
       # container - it registers, reports online, and every dispatch vanishes.
       ADVERTISE_ADDRESS: smartbotic-runner:9011
+      # For anything that talks to OpenSSL directly. libcurl ignores it, which
+      # is why the bundle is also mounted over curl's default CA path below -
+      # see the volume comment.
+      SSL_CERT_FILE: /etc/ssl/certs/ca-certificates.crt
     volumes:
       - /data/dev/smartbotics/smartbotic/config:/var/lib/smartbotic/config:ro
       - /data/dev/smartbotics/smartbotic/nodes:/usr/share/smartbotic-automation/nodes:ro
       # Written by imap-extract-attachments. A bind mount, so the 230 files
       # carried over from mulan stay where the host can see them.
       - /data/dev/smartbotics/smartbotic/data:/var/lib/smartbotic/data
+      # Debian's CA set plus the extra certificates in ca/ - currently mulan's
+      # self-signed SD.cpp cert. Regenerate with ca/build-bundle.sh.
+      #
+      # Mounted OVER curl's default CA file rather than beside it. The runner
+      # never sets CURLOPT_CAINFO, so libcurl uses its compiled-in default,
+      # which on Debian is exactly this path - and libcurl reads neither
+      # SSL_CERT_FILE (that is OpenSSL's) nor CURL_CA_BUNDLE (that is the curl
+      # command-line tool's). Pointing an environment variable at a bundle
+      # somewhere else looks like it should work and silently does nothing.
+      - /data/dev/smartbotics/smartbotic/deploy/zeus/ca/bundle.crt:/etc/ssl/certs/ca-certificates.crt:ro
 
 networks:
   smartbotic: