Bug: smartbotic-database 1.7.5 RSS exceeds `max_memory_mb` cap and is OOM-killed every ~3 hours on Zoe

Status

Open. No fix yet. Workaround (lower budget + cgroup ceiling) not yet applied — we want a real fix first.
Version observed: smartbotic-database 1.7.5-1 (deb), installed on Debian 13 (trixie).
Reporter: Zoe production instance (zoe.shadowman.fsociety.hu).
Severity: High — every kill takes the whole ShadowMan stack with it for the duration of recovery. ~8 kills in the last 24 h.

Summary

The DB process grows past its configured max_memory_mb budget and keeps climbing until the kernel OOM-killer reaps it. The internal eviction logic is firing — estimatedMemoryBytes_ gets reduced — but actual RSS does not. This means allocations are leaking out of an accounting path that the budget tracks, OR are happening in code paths the budget never tracked.

Evidence

Host

Zoe: 11 GB total RAM, 4.6 GB swap. Single-host (no replication).
Several services share the host (shadowman-server, shadowman-gateway, shadowman-llm, shadowman-tools, shadowman-stt, shadowman-tts, shadowman-cron, smartbotic-database, redis-server). Database is by far the largest.

Config

/etc/smartbotic-database/conf.d/99-local.json (operator override):

{ "storage": { "memory": {
  "max_memory_mb": 6144,
  "eviction_threshold_percent": 90,
  "eviction_target_percent": 75
}}}

Defaults from base + 50-shadowman.json for everything else, notably:

max_file_size_mb: 500
grpc.max_receive_message_size_mb: 100
grpc.max_send_message_size_mb: 100
grpc.resource_quota_memory_mb: 256
snapshot_interval_sec: 3600
encryption.enabled: true

Observed process behaviour

After a fresh start it climbs to 8.4 GB RSS in ~30 minutes (about 4 minutes per 1 GB) and stabilises briefly, then crosses ~9 GB and the kernel kills it.

Recent OOM samples from dmesg -T --level=err,warn (every line is smartbotic-data as the victim — the named "invoker" is just whoever was unlucky enough to allocate next):

Mon May 11 23:41:29 — killed pid 441106  anon-rss:11757612 kB  total-vm:21405132 kB
Tue May 12 01:29:19 — killed pid 452374  anon-rss:11789832 kB  total-vm:20891196 kB
Tue May 12 04:06:31 — killed pid 455113  anon-rss:11753480 kB  total-vm:20529072 kB
Tue May 12 07:05:14 — killed pid 458822  anon-rss:11783236 kB  total-vm:20211244 kB
Tue May 12 09:02:12 — killed pid 468259  anon-rss:11739336 kB  total-vm:21097712 kB
Tue May 12 12:01:57 — killed pid 472711  anon-rss:11599740 kB  total-vm:19878288 kB
Tue May 12 15:02:59 — killed pid 487257  anon-rss:11631612 kB  total-vm:19682928 kB
Wed May 13 00:07:19 — killed pid 531382  anon-rss:11750160 kB  total-vm:19053096 kB

Pattern: anon-rss clamps to ~11.7 GB right before kill (essentially "all of RAM"), total-vm ~20 GB. Memory grows linearly until the kernel intervenes — there is no plateau anywhere near 6 GB (the configured cap) or 7.6 GB (85% emergency threshold).

Eviction is firing — RSS doesn't care

estimatedMemoryBytes_ accounting code is correct on the document path (every insert/replace/delete updates the atomic counter via estimateDocumentSize).
Eviction triggers above 90% and targets 75%, per config, so the budgeted memory cycles between 4.6 GB and 5.5 GB. But RSS keeps climbing through that whole range and beyond.

This is the smoking gun: the leak is in untracked allocations, or in heap fragmentation that prevents pages from being returned to the OS after eviction frees them.

Suspects (ranked by my prior)

1. glibc allocator fragmentation after eviction bursts (most likely, easiest to verify)

v1.7.0 introduced "chunked eviction" — evicting up to 137k documents in a single burst is on the record (50-shadowman.json mentions a 2026-04-22 incident).
Each evicted document frees many small nlohmann::json nodes (numbers, strings, object/array nodes). glibc's malloc uses per-thread arenas, and small-allocation frees often don't return pages to the OS — they go on the thread-local free list. RSS stays high; the next alloc reuses those pages, so it doesn't grow infinitely, but it never shrinks either.
Hypothesis: every eviction burst leaks ~50–200 MB of effectively-unreturnable heap. Run for 30 minutes → 1–2 bursts → RSS reaches 8 GB and never comes back.
Verification:
- Add a periodic malloc_trim(0) call after each eviction burst (or every N minutes) — see if RSS drops.
- Set MALLOC_ARENA_MAX=2 in the systemd unit's Environment= — limits arena count, reduces fragmentation.
- Switch the allocator to mimalloc or jemalloc (LD_PRELOAD= in the unit) — both are dramatically better at returning pages after fragmentation. This is a one-line change to verify.
If swapping to jemalloc/mimalloc resolves it, the right ship is either keeping it as the recommended allocator (document in README + ship via deb dependency) or invoke malloc_trim from the eviction path.

2. Snapshot serialization peak

snapshot_interval_sec: 3600. Every hour the entire state gets serialized.
If the snapshot path serializes-to-buffer-then-writes (i.e. holds the full snapshot in RAM in addition to the live state), a 5 GB live set needs ~10 GB during the dump.
Verification:
- Inspect service/src/persistence/ — look for whether snapshots stream to disk incrementally or buffer the whole thing in memory.
- Time the OOMs against snapshot intervals — if they cluster near :00 (hourly), this is it.
- Two of the eight kills are at :02:12 and :01:57 past the hour — suggestive but not conclusive. Add a snapshot-start / snapshot-end log line and correlate.

3. File streaming buffers

max_file_size_mb: 500. Each in-flight PutFile could allocate up to 500 MB if implemented as buffer-in-memory.
grpc.max_concurrent_file_streams: 10 → worst case 5 GB simultaneous.
Verification: look at how service/src/files/ (or wherever the file-stream RPCs live) handles streamed chunks. Are they accumulated into a contiguous buffer? Or streamed to disk as they arrive?

4. Replication outbox / migration apply buffers

Replication is disabled on Zoe (replication.enabled: false), so the outbox code shouldn't be active. Mark this low-priority but worth a quick grep.
Migrations run at startup only — wouldn't explain steady-state growth.

5. gRPC channel/stream lifecycle leaks

v1.7.0 (14eccea — "client retry writes on DEADLINE_EXCEEDED/RESOURCE_EXHAUSTED/UNAVAILABLE") added retry logic. If retried-then-cancelled streams don't release their gRPC buffers, that could trickle.
Verification: check the gRPC reflection metrics; grpcpp_sync_ser appears as an OOM-killer invoker in the dmesg log (the gRPC server's worker pool name), which means gRPC was actively allocating at kill time.

6. Encryption pipeline working buffers

encryption.enabled: true. The AES-256-GCM path likely staging-buffers each write before persisting. If those buffers don't shrink between writes (e.g. cached per-thread), heavy write workloads could pile pages.

How to reproduce locally

Bring up smartbotic-database 1.7.5 with the same conf as Zoe (max_memory_mb: 6144).
Start RES=$(pgrep smartbotic-data); while sleep 30; do ps -o rss= -p $RES; done > /tmp/rss.log to log RSS every 30 s.
Drive a write-heavy synthetic workload — ideally the existing tests/load/ harness, scaled to the same document count Zoe runs.
Watch RSS grow past max_memory_mb.

If reproducing on a smaller config is fine, drop max_memory_mb to 512 locally and watch whether RSS exceeds 1 GB. The leak should be proportional.

Files to read first

service/src/memory_store.{hpp,cpp} — the budget + eviction logic. estimateDocumentSize (line ~2390) is the only path that updates estimatedMemoryBytes_. Anything allocating outside MemoryStore::insertImpl/replaceImpl/deleteImpl paths is untracked.
service/src/persistence/* — snapshot + WAL handlers. Look for buffered serialization.
service/src/files/* — file-streaming RPCs.
service/src/database_service.cpp:363 (config.maxMemoryMb = memory.value(...)) — confirms the config-binding path.

Acceptance criteria

The fix is acceptable when all of these hold:

On the reproduction workload above, RSS stays within 1.25 × max_memory_mb for at least 8 hours of continuous load (we allow 25% headroom for allocator overhead, but not 100%).
Eviction events still fire correctly and estimatedMemoryBytes_ tracks them.
The snapshot path does not double-allocate the live state (or, if it does, the doc explicitly says so and the budget accounts for it).
A regression test under tests/load/ reproduces the original failure on a known-bad commit and passes on the fix commit.
CHANGELOG.md (or commit message) documents what the leak was and why the fix works.

Ship target

New version: 1.7.6 if this is a bugfix only, 1.8.0 if the fix involves shipping an allocator change or new env vars.
VERSION file bump + git tag v1.7.6 (or v1.8.0).
Build deb via the existing packaging path under packaging/. The repo machinery follows the same add-packages.sh → create-repo.sh --suite trixie → sync-repo.sh flow used by shadowman-cpp.
ShadowMan-cpp's external/smartbotic-database/VERSION doesn't pin a hard version, so a new deb on the FTP repo is enough; on Zoe apt install --only-upgrade smartbotic-database should take it.

Investigation order (cheapest first)

30-minute test: Environment=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 in the unit, restart, watch RSS. If RSS now stays under 6 GB, ship that as the immediate mitigation while you investigate (1).
If jemalloc doesn't help: instrument the snapshot path with start/end timing + RSS log lines. Correlate with OOMs.
If snapshots aren't it: instrument the file-stream RPC entry/exit with allocation count.
Only after ruling out the above, hunt for genuine leaks via heaptrack/massif (each takes hours).

Out of scope for this fix

Replication memory behaviour (Zoe doesn't replicate).
Recovery memory peaks (we've shipped fixes for those in 1.7.2–1.7.4 — they manifest only at startup; this leak is steady-state).
View / FilterOp memory cost — single-user instance, low query rate; not the bottleneck.

Update 2026-05-13: idle leak measured

Stopped all shadowman-* services on Zoe, kept smartbotic-database running, sampled RSS every 60 s for 11 minutes (zero clients connected, no gRPC traffic):

RSS (KB)   CPU%
6399968    59.9   t=0
6401836    58.7
6406924    57.7
6409024    56.7
6412340    55.7
6413276    54.7
6420368    53.6
6420920    52.6
6421788    51.6
6425260    50.7
6426356    49.8
6427176    48.9   t=11

Idle leak rate: ~2.4 MB/min (27 MB over 11 minutes). The CPU decline reflects the eviction loop catching up post-emergency-pressure; the memory growth is monotonic with zero client load.

Strong implication: this isn't an access-pattern problem — it's a real allocator-level leak in a background thread (likely the eviction loop itself, the WAL fsync thread, or the snapshot thread). Anything that allocates without going through estimateDocumentSize (which is what updates estimatedMemoryBytes_) will leak invisibly to the budget.

We also found that the docs-on-disk picture is much smaller than the tracked working set:

Latest snapshot is 224 MB lz4-compressed (~500-700 MB raw JSON)
WAL is 169 MB
DB reports 4982 MB tracked

The gap is dominated by:

estimateDocumentSize's 2.5x heap multiplier (memory_store.cpp:2402) — applied to live JSON nodes.
Version history. Frequently-updated docs (e.g. conversation_context_usage updated every LLM turn) accumulate hundreds of DocumentVersion entries in versionHistory[docId] — each one a full copy of the doc data, also counted with the 2.5x multiplier. A single conversation with 200 turns × ~5 KB context_usage doc × 2.5 ≈ 2.5 MB per doc; multiply by 10 conversations + similar churn on conversation_tool_state, conversation_active_skills, metrics_rollup_*, etc. and you get GBs of version history in RAM.

Shadowman-cpp is shipping migration 062 (alter_collection setting max_versions: 1-3 on hot-update collections) which will collapse this. But it's a workaround — the real question for smartbotic-database is whether unbounded max_versions=0 should still be the default. A sensible default of 5 or 10 (with explicit unlimited for collections that opt in) would prevent this category of footgun.

Suggested additions to investigation order

1a. Before any heap-profiling work, measure idle leak per-thread: gdb attach the live process, capture info threads + per-thread stacks, see which thread is allocating. The eviction loop (MemoryStore::expirationLoop?) and the WAL/snapshot threads are the suspects.

1b. Add a periodic malloc_info(0, stderr) dump (or use mallctl stats if jemalloc) to compare arena usage between a freshly-started process and one that's been idle for an hour. Confirms whether the leak is in malloc arena fragmentation or in genuinely-unreleased allocations.

Update 2026-05-13 09:35: glibc fragmentation confirmed as dominant cause

After deploying v1.8.0 on Zoe the workload pattern produced this:

Before MALLOC_ARENA_MAX:        After MALLOC_ARENA_MAX=2:
VmData: 11.55 GB                VmData:  5.70 GB
VmRSS:   7.97 GB                VmRSS:   5.57 GB
VmSwap:  3.27 GB                VmSwap:  0 KB
VmHWM:  10.77 GB                VmHWM:   8.29 GB
Tracked: 4995 MB                Tracked: ~4995 MB (same workload)

Setting MALLOC_ARENA_MAX=2 via the systemd unit closed the 6.5 GB gap between tracked memory and committed heap to ~500 MB. The DB's tracker was correct; the rest was glibc per-thread arena fragmentation. With 22 threads, each thread gets its own arena (by default 8 × num_cpus), small frees go on per-thread freelists and never make it back to madvise(MADV_DONTNEED). The evictDocument history-pruning fix from v1.8.0 IS valid (and necessary), but its effect was masked on Zoe because eviction never fired — the workload kept memory under the 85% threshold, so the leak surface that triggered was fragmentation, not unreleased history.

Recommendation for v1.8.x:

Document MALLOC_ARENA_MAX=2 as a recommended environment variable in the deb's systemd unit (or set it directly in the service file). Operators won't discover this on their own and we lost a working day to it.
Consider linking against jemalloc or mimalloc instead — both return memory to the OS far more aggressively than glibc and remove the env-var requirement. mimalloc in particular has measured 30-50% RSS reductions for similar high-thread-count C++ servers.
The Linux-only mallopt(M_ARENA_MAX, 2) programmatic equivalent in main.cpp would make this self-contained without env vars.

The boot-time parallel snapshot deserialize from v1.8.0 also worked beautifully — DB went from systemd start to READY in <1 second on Zoe's 5 GB snapshot (sequential pre-v1.8 would have been ~30 s).

Update 2026-05-13 10:05: v1.8.0 + MALLOC_ARENA_MAX=2 still leaks ~160 MB/min

After running on Zoe for 30 minutes under real workload (single user, normal LLM/tool traffic — nothing heavy), v1.8.0 with MALLOC_ARENA_MAX=2 produced this growth pattern:

Internal tracker (Memory check log line):
  09:44 — 4988 MB estimated, 1854705 docs, 0 evicted
  09:54 — 4994 MB estimated, 1858821 docs, 0 evicted
  10:03 — 4999 MB estimated, 1862015 docs, 0 evicted   (+11 MB tracked / 19 min)

Process VM (from /proc/$pid/status):
  After restart (~09:34):  VmRSS=5.57 GB  VmData=5.70 GB   ← matches tracker
  30 minutes later:        VmRSS=10.3 GB  VmData=10.5 GB   ← +4.8 GB over 30 min

The leak is real, untracked by estimatedMemoryBytes_, and reproduces consistently. ~160 MB/min under modest workload. Eviction never fires (DB stays at 69% reported pressure) so the v1.8.0 evictDocument fix never engages.

Doc-count growth is only ~250-600/min (mostly metrics_events + small rollups), accounting for ~10-15 MB/min of legitimate tracked growth. The remaining ~145 MB/min is allocations the tracker doesn't see. Candidates from earlier suspect list (re-ranked now that we have this data):

gRPC request/response arena pools — top candidate. With shadowman-server, gateway, tools, llm all maintaining open bidi-streams (SubscribeEvents, conversation_history streaming, agent task output streaming, voice audio), every stream has its own arena that grows under load. C++ gRPC has known issues with arena retention; bounded buffer-pool reset is non-trivial.
WAL serialization buffers — every insert/update builds a WAL record. If the WAL writer's per-thread buffer grows-but-doesn't-shrink, this scales with write traffic. The 250-600 doc/min write rate matches the ~150 MB/min leak rate.
Snapshot dump in progress? — we're getting hourly snapshots. If the dump path holds a copy of every document in a per-collection buffer before lz4'ing it to disk, peak heap during snapshot ≈ 5 GB tracked + 5 GB working buffer = 10 GB. But that should drop back after the dump completes (~10 s of work). If it doesn't drop back, the buffer isn't being released.

What we shipped on the operator side as containment

/etc/systemd/system/smartbotic-database.service.d/arena-cap.conf on Zoe:

[Service]
Environment=MALLOC_ARENA_MAX=2
MemoryMax=9G
MemoryHigh=7G
RuntimeMaxSec=5400        ; 90-min watchdog — DB restarts cleanly before VmData reaches MemoryMax

This bounds the damage but is a hack — clients see a 3-5 s outage every 90 minutes when the watchdog fires. Acceptable while upstream debugs but not the long-term answer.

Suggested next steps for v1.8.x

Heaptrack the DB under steady-state load. heaptrack /usr/bin/smartbotic-database --config /etc/smartbotic-database/config.json, drive realistic write traffic for 30 min, then heaptrack_print to find the top allocator paths with no matching frees. Should immediately surface the second leak — far cheaper than guessing.
If heaptrack output points at gRPC arenas, the fix is to recycle the arena per request (grpc::Arena has Reset()) instead of letting it grow with the channel.
If it points at snapshot/WAL, the fix is to stream-write rather than buffer-then-write.

The good news from this datapoint: the v1.8.0 changes ARE working as advertised — the boot-time parallel deserialize halved snapshot recovery time, and the eviction history-prune is correct (we just haven't been able to trigger eviction). The remaining leak is a different bug that pre-dates the v1.8.0 work.

Update 2026-05-13 12:07: v1.9.0 verified on Zoe — bug fixed, suggestion for auto-trim

v1.9.0 deployed on Zoe. The off-heap history architecture closed the tracker gap to zero:

Pre-1.9.0 baseline (1.8.2):
  Tracker:       5017 MB
  VmRSS:         6.26 GB
  VmData:        8.31 GB
  Growth/min:    ~3.5 MB (under modest load)

Post-1.9.0 (boot done, 11 min of light traffic):
  Tracker:       1396 MB     ← was 5017 MB (-3.6 GB, version history off heap)
  VmRSS:         6.26 GB     ← unchanged (the gap moved from "leak" to "arena retention")
  VmData:        6.41 GB
  Growth/min:    0 MB on tracker, 2.7 MB/min on VmData

After SIGUSR2 (malloc_trim from v1.8.3):
  VmRSS:         3.51 GB     ← dropped 2.76 GB in one call
  System free:   5.1 GB      ← was 2.5 GB
  Tracker:       1.4 GB
  Gap (Vm-track):  ~2.1 GB   ← reasonable: binary + gRPC channels + thread stacks + page tables

So:

The architectural fix works. Tracker is now flat (0 MB/min on the tracker) and matches reality once arena pages are returned.
Boot-time history migration leaves ~2.8 GB of arena pages on glibc's free list that don't auto-trim. The pages are reusable but show up as RSS, which is the operator-visible metric the kernel uses for OOM scoring.
The 2.7 MB/min VmData growth is virtual size climbing, not RSS — that's normal arena allocation behavior and self-bounded.

Suggestion for v1.9.1 (small, high-impact)

Call malloc_trim(0) automatically once at the end of MemoryStore::loadSnapshot (or wherever boot-time deserialize completes). The pages are freeable; they're just sitting on the per-thread free-list because glibc doesn't proactively check thresholds during quiet periods. One-line addition next to the existing mallopt(M_ARENA_MAX, 2) from v1.8.1. Saves operators ~3 GB of RSS post-boot, makes the OOM-killer score much friendlier on tight VMs.

Optional: trim periodically (every 5 minutes? every snapshot completion?) for ongoing arena hygiene. The SIGUSR2 handler from v1.8.3 already proves the operation is safe under load. Production allocators like jemalloc / mimalloc do this automatically.

Operator-side state

My /etc/systemd/system/smartbotic-database.service.d/zoe-leak-watchdog.conf removed — 1.9.0 makes RuntimeMaxSec unnecessary.
shadowman-cpp migration 062 (max_versions caps) still applied and still correct — they bound the on-disk .hlog file growth, which is the new constraint.
Zoe's history dir is 1.6 GB across 52 collections after migration. The largest single file is metrics_rollup_monthly.hlog at 22 MB.

Update 2026-05-13 13:34: 14-min sustained stress test on v1.9.1

Setup: app named "DB Stress Probe" with a 5-field cron schedule (* * * * *) that on each tick inserts 200 rows into probe_events, upserts probe_stats (single doc with max_versions: 5), and trims probe_events to the latest 1000 rows. After bumping apps_max_collection_rows from 500 to 100000 so the trim/insert ratio is realistic, ran 14 minutes / 9 successful cron ticks.

                    start      end        delta     rate
Tracker (MB):       1408       1409       +1        0.07 MB/min
VmRSS (MB):         6112       6115       +3        0.19 MB/min
VmData (MB):        6275       6277       +2        0.17 MB/min
Docs:               1,873,653  1,890,558  +16,905   ~1200/min
Swap:               0          0          —         —
Pressure:           19%        19%        —         normal
CPU:                7.3%       5.2%       decreasing —

The architecture decoupling is complete. Write churn flows through WAL + hlog files on disk; memory is bounded by max_memory_mb and doesn't grow with workload. Compared to the pre-fix baseline (~200+ MB/min under load on 1.7.5), this is a >1000× improvement at steady-state.

The auto-trim from v1.9.1 doesn't fire here because no snapshot completed during the window (hourly snapshots; window was 13:19–13:34, next snapshot at ~13:53). When the snapshot lands, RSS will likely drop a few hundred MB closer to the tracker — observed previously on this run that snapshot trims drop RSS by 500-3000 MB depending on prior activity.

Final operator state on Zoe (no workarounds remain)

$ ls /etc/systemd/system/smartbotic-database.service.d/
# (empty — drop-in dir doesn't exist)

$ apt-cache policy smartbotic-database
  Installed: 1.9.1-1
  Candidate: 1.9.1-1

$ cat /etc/smartbotic-database/conf.d/99-local.json
{
  "storage": {
    "memory": {
      "max_memory_mb": 7168,
      "eviction_threshold_percent": 85,
      "eviction_target_percent": 70
    }
  }
}

The 7 GB cap is the only customization, and even that is generous — the actual working set under load is ~1.4 GB tracked / ~6.3 GB resident. Could safely drop to 2-3 GB if you want to test eviction-under-pressure behavior; currently eviction stays dormant which is fine.

Update 2026-05-13 15:05: post-boot arena retention recurs without periodic trim

v1.9.1's malloc_trim(0) at end of loadSnapshot/createSnapshot is necessary but not sufficient on the Zoe workload pattern:

DB restart at ~14:42 (triggered by shadowman-common dpkg upgrade)
~23 min later:
  Tracker:       1422 MB
  VmRSS:         9.31 GB     ← 7.9 GB of glibc arena pages
  System avail:  1.6 GB      ← squeezing the rest of the box

After manual SIGUSR2 (malloc_trim):
  VmRSS:         3.60 GB     ← -5.7 GB freed
  System avail:  7.0 GB

The auto-trim only fires on snapshot boundaries (hourly). Between a fresh start and the first snapshot, the DB accumulates GBs of arena pages from:

Initial snapshot deserialize (handled — trim fires at end of loadSnapshot)
WAL replay AFTER snapshot deserialize (NOT handled — replayWal allocates and frees a lot of small Document JSON nodes)
Initial migration runner pass (NOT handled)
First N minutes of normal write traffic before any heavy compaction completes

Suggestion for v1.9.2 (small, one place to add):

Trim periodically — easiest insertion is in MemoryStore's minute-tick "Memory check" path that already logs Memory check: X MB estimated (Y%, pressure=Z), N docs, M evicted. Add a malloc_trim(0) every Nth tick (5? 10?). Behind the same #if defined(__GLIBC__) guard as the existing trims.
Alternative: trim once at the end of replayWal (the post-snapshot WAL fast-forward in startup). Cheaper because it's still bounded to the boot path, but doesn't help with mid-life arena bloat.

This is the only thing operators on tight VMs still trip on — the actual leak is fixed.

Contact

If anything's unclear, the live evidence is on shadowman-zoe: dmesg -T --level=err,warn | grep -iE 'oom|killed process' shows the kill history, and systemctl status smartbotic-database shows the current RSS via Memory:. shadowman-cpp's CLAUDE.md has the Zoe SSH context (creds in /data/shadowman-cpp/.env).

BUG-memory-leak-zoe.md 25 KB History Raw

Bug: smartbotic-database 1.7.5 RSS exceeds max_memory_mb cap and is OOM-killed every ~3 hours on Zoe