[Unit] Description=Smartbotic Database Service After=network.target [Service] # v1.7.5: Type=notify so dependents block at startup until the service is # *actually* ready (recovery complete + migrations applied + gRPC listening), # not just "process spawned." Pre-1.7.5 this was Type=simple, which made # `After=smartbotic-database.service` on consumers ineffective: tools, # gateway, server etc. would launch immediately and silently fall back to # defaults when their startup DB reads timed out against an in-recovery # database. The service code already calls `sd_notify(READY=1)` at the # correct point in `service/src/main.cpp` after `service.start()`; with # Type=notify systemd actually waits for that signal before considering # the unit active and unblocking dependents. Type=notify NotifyAccess=main # v1.8.1: Cap glibc per-thread arenas to two. With Type=notify + threads # in eviction / WAL sync / snapshot / gRPC / replication, glibc allocates # `8 × num_cpus` arenas by default and small frees go on per-thread # free-lists that never `madvise(MADV_DONTNEED)`. On Zoe (22 threads) the # committed heap drifted 6.5 GB above the 5 GB the DB itself was tracking # — a fragmentation gap large enough to OOM-kill the process every ~3h # even though `estimatedMemoryBytes_` was correct. Two arenas is enough # for the writer + reader thread split; the small extra contention is # negligible vs. the RSS savings (measured 7.97 → 5.57 GB on Zoe). # Operators can override per-deployment by setting MALLOC_ARENA_MAX # elsewhere; the binary also calls `mallopt(M_ARENA_MAX, 2)` at startup # as a fallback for non-systemd deploys. Environment="MALLOC_ARENA_MAX=2" # Recovery on a large dataset (700k+ docs, multi-MB WAL) routinely takes # 60–90s. Default systemd start timeout (90s) is too tight; bump to 5 min. # The service emits `EXTEND_TIMEOUT_USEC=` during long recovery anyway, but # the static cap is the safety net. TimeoutStartSec=300 User=smartbotic-db Group=smartbotic-db ExecStart=/usr/bin/smartbotic-database --config /etc/smartbotic-database/config.json WorkingDirectory=/var/lib/smartbotic-database Restart=on-failure RestartSec=3 TimeoutStopSec=30 LimitNOFILE=65536 [Install] WantedBy=multi-user.target