|
|
@@ -0,0 +1,119 @@
|
|
|
+# ComfyUI anime generation from analysed images
|
|
|
+
|
|
|
+**Date:** 2026-08-03
|
|
|
+**Status:** Design - approved
|
|
|
+**Owner:** Ferenc Szontagh
|
|
|
+
|
|
|
+## 1. Goal
|
|
|
+
|
|
|
+Extend the 35photo workflow so that every image flagged NSFW is additionally
|
|
|
+rendered as an anime-style image by the local ComfyUI instance at
|
|
|
+`http://127.0.0.1:8188`, using the Z-Image Turbo model, and stored with its
|
|
|
+metadata.
|
|
|
+
|
|
|
+## 2. Constraints established during exploration
|
|
|
+
|
|
|
+- ComfyUI 0.28.0, RTX 3060 (12.5 GB total). ComfyUI already holds ~7.2 GB of
|
|
|
+ cached models; ~3.3 GB free. `z_image_turbo_bf16.safetensors` is ~11.5 GB, so
|
|
|
+ ComfyUI must evict and will still partially offload. Generations will be slow.
|
|
|
+ Use `z_image_turbo_bf16` regardless - no Q8/GGUF substitution.
|
|
|
+- Z-Image Turbo is text-to-image only. `TextEncodeZImageOmni` exists in this
|
|
|
+ install but targets the Omni checkpoint, which is not present, so
|
|
|
+ image-conditioned generation is not available. The prompt produced by the
|
|
|
+ vision step is therefore the source of the generated image.
|
|
|
+- No base64 image-loader node exists. Not needed here, since generation is
|
|
|
+ text-to-image, but it rules out feeding source pixels without the
|
|
|
+ `/upload/image` endpoint.
|
|
|
+- Model components present: `z_image_turbo_bf16.safetensors` (UNETLoader),
|
|
|
+ `qwen_3_4b.safetensors` (CLIPLoader), `ae.safetensors` (VAELoader).
|
|
|
+- The upstream database exposes a file API (`uploadFile`, `downloadFile`,
|
|
|
+ `getFileInfo`, `deleteFile`, `listFiles`) with checksums, dedupe and
|
|
|
+ ref-counting. `lib/storage/storage_client.hpp` exposes none of it.
|
|
|
+
|
|
|
+## 3. Part A - expose database file storage
|
|
|
+
|
|
|
+The document API already flows adapter -> script engine -> node. Files follow the
|
|
|
+same path.
|
|
|
+
|
|
|
+`lib/storage/storage_client.{hpp,cpp}`:
|
|
|
+
|
|
|
+| Method | Upstream call |
|
|
|
+| --- | --- |
|
|
|
+| `Result<FileUploadInfo> uploadFile(bytes, FileMeta)` | `Client::uploadFile` |
|
|
|
+| `Result<std::vector<uint8_t>> downloadFile(id)` | `Client::downloadFile` |
|
|
|
+| `Result<FileInfo> getFileInfo(id)` | `Client::getFileInfo` |
|
|
|
+| `Result<void> deleteFile(id)` | `Client::deleteFile` |
|
|
|
+
|
|
|
+`src/runner/engine/script_engine.cpp` exposes:
|
|
|
+
|
|
|
+- `smartbotic.storage.uploadFile(base64, meta)` -> `{success, id, size, checksum, deduplicated}`
|
|
|
+- `smartbotic.storage.downloadFile(id)` -> `{success, data (base64), mimeType}`
|
|
|
+- `smartbotic.storage.getFileInfo(id)` -> `{success, ...FileRecord fields}`
|
|
|
+
|
|
|
+Base64 is the transport at the JS boundary, matching how `http-request` already
|
|
|
+carries binary through `file.data`. The C++ side decodes once; documents never
|
|
|
+carry image bytes.
|
|
|
+
|
|
|
+Rationale: storing a ~1.5 MB PNG as base64 inside a document costs ~133% of the
|
|
|
+image size per record and defeats dedupe. The file store is purpose-built for
|
|
|
+this and unblocks every future binary node.
|
|
|
+
|
|
|
+## 4. Part B - generic `comfyui-prompt` node
|
|
|
+
|
|
|
+`nodes/ai/comfyui-prompt.js`. Task-agnostic, following the `ollama-chat`
|
|
|
+precedent: nothing anime- or Z-Image-specific lives in the code.
|
|
|
+
|
|
|
+**Config:** `baseUrl`, `workflowJson` (ComfyUI API-format graph, with
|
|
|
+`{{placeholder}}` substitution), `outputNodeId`, `pollIntervalMs`, `timeoutMs`,
|
|
|
+`retryCount`, `retryDelayMs`, `retryMaxDelayMs`, `skipOnError`.
|
|
|
+
|
|
|
+**Behaviour:** substitute placeholders -> `POST /prompt` -> poll
|
|
|
+`/history/{prompt_id}` until the entry appears -> read the output image
|
|
|
+descriptor from `outputNodeId` -> fetch bytes via `/view` -> return base64.
|
|
|
+
|
|
|
+**Output:** `{success, error, imageBase64, filename, mimeType, width, height,
|
|
|
+promptId, attempts}`.
|
|
|
+
|
|
|
+**Error handling:** mirrors `ollama-chat`. Exponential backoff between attempts;
|
|
|
+`skipOnError` returns `success: false` rather than aborting the loop.
|
|
|
+
|
|
|
+## 5. Part C - workflow wiring
|
|
|
+
|
|
|
+Only NSFW-flagged images proceed, so generation is rare and GPU pressure stays
|
|
|
+bounded.
|
|
|
+
|
|
|
+```
|
|
|
+Store If NSFW -> ComfyUI Generate -> Store Anime File
|
|
|
+```
|
|
|
+
|
|
|
+- **ComfyUI Generate:** `comfyui-prompt` configured with a Z-Image Turbo
|
|
|
+ text-to-image graph. Positive prompt is `anime style, ` prefixed to the
|
|
|
+ `prompt` field the safety gate already produces (which has the adult
|
|
|
+ descriptor enforced). Sampler settings follow the model's guidance: `cfg 1.0`,
|
|
|
+ low step count for Turbo. Both are exposed in the graph config for tuning.
|
|
|
+- **Store Anime File:** a `code` node that calls
|
|
|
+ `smartbotic.storage.uploadFile` with `file_type: "generated"`, then inserts a
|
|
|
+ document into the `anime_images` collection holding `fileId`, `sourceUrl`,
|
|
|
+ `sourceRecordId`, `prompt`, `model` and sampler parameters.
|
|
|
+
|
|
|
+`anime_images` must be granted `read-write` in the workflow's
|
|
|
+`settings.storagePermissions`, since storage access is deny-by-default.
|
|
|
+
|
|
|
+## 6. Testing
|
|
|
+
|
|
|
+The NSFW branch has never fired on this feed, so the path cannot be validated by
|
|
|
+a normal run. Testing proceeds in this order:
|
|
|
+
|
|
|
+1. Part A verified directly: upload a known file, download it, compare
|
|
|
+ checksums, confirm `deduplicated` on a second identical upload.
|
|
|
+2. Part B verified standalone: a scratch workflow with a fixed prompt, confirming
|
|
|
+ an image comes back and lands in `anime_images`.
|
|
|
+3. Part C verified by temporarily relaxing the store condition, exactly as the
|
|
|
+ NSFW store path was verified, then restoring it.
|
|
|
+
|
|
|
+## 7. Out of scope
|
|
|
+
|
|
|
+- Image-conditioned (img2img) generation - not supported by the available
|
|
|
+ checkpoint.
|
|
|
+- Substituting a smaller Z-Image variant to fit VRAM.
|
|
|
+- Backfilling anime images for previously stored records.
|
