Ver código fonte

fix: make http-request and imap-extract-attachments TTL fallbacks agree with their schema defaults

Fix round 2, resolving the TTL question the project owner held back from
round 1.

Before this branch, nodes/core/http-request.js read
config.downloadTtlHours || 0 and nodes/imap/imap-extract-attachments.js
read config.storageTtlHours || 0, while both schemas declare default: 24.
A workflow whose config omitted the key stored downloads and attachments
forever, because the || fallback won and the schema default never
arrived. Now that defaults are applied (this branch), those same
workflows get 24 and the data expires after a day.

The owner decided to KEEP the 24-hour expiry - the schema is the intended
behaviour, and unbounded storage growth is exactly what the default was
written to prevent. This commit makes the code agree with that decision
instead of contradicting it:

- nodes/core/http-request.js: config.downloadTtlHours ?? 24
- nodes/imap/imap-extract-attachments.js: config.storageTtlHours ?? 24

Nullish coalescing rather than ||, so an explicit 0 - documented in both
schemas as "never expire" - still survives. Each file only reads its TTL
key in one place, so there was nothing else to make consistent.

Both TTL field descriptions now state the default is 24 hours and that 0
means never expire. docs/nodes.md gains a paragraph under Storage
(Database) warning that stored downloads and extracted attachments now
expire after 24 hours by default, and that a workflow wanting them kept
must set the TTL field to 0 explicitly - this is the behaviour change an
existing workflow owner needs to know about.

Added one fixture per node proving the explicit-zero case survives:
- tests/nodes/config-defaults-ttl-zero-http-request.json
- tests/nodes/config-defaults-ttl-zero-imap-extract-attachments.json

Both set the TTL field to 0 and assert the node's storage path completes
and returns a "storage" object, proving smartbotic.storage.insert is
reached and does not reject the call with an explicit-zero config. This
is the limit of what the test harness can prove: the storage client's
JS binding does not expose TTL or expiry back through storage.get() or
storage.query(), and the document metadata returned (_created_at,
_updated_at, _version, etc.) carries no TTL field either - confirmed by
probing a document inserted with a short TTL, whose fields matched a
document inserted with a large TTL exactly. So neither fixture can
distinguish "TTL landed as 0" from "TTL silently became 24" by reading
storage back; that guarantee rests on the ?? 24 change itself being
correct, which is a one-line, directly-readable diff at each read site.
Waiting out a real 24-hour expiry to observe the difference is not
practical for this suite.

Full fixture suite: 42/42 passing (the 40 from round 1 plus these 2).
Re-confirmed "Loaded 2 scheduled workflows" including wf_520f6f05,
unmodified; its imap-trigger has since fired several scheduled polls in
the running webserver log, visible confirmation the round-1 fix is
holding in the background while this round's changes were verified.

No rebuild was needed - both changed node files are JavaScript, hot-
reloaded by the already-running services.
fszontagh 1 mês atrás
pai
commit
ccdc442fc1

+ 9 - 0
docs/nodes.md

@@ -424,6 +424,15 @@ smartbotic.storage.update('collection', 'document-id', { key: 'new-value' });
 smartbotic.storage.delete('collection', 'document-id');
 ```
 
+`http-request` (Store Download) and `imap-extract-attachments` (Store in
+Database) both write through `smartbotic.storage.insert` with a TTL. Both
+default that TTL field to 24 hours, so downloaded files and extracted
+attachments stored by those nodes now expire and are auto-deleted a day
+after they're stored, unless the workflow sets the TTL field to `0`
+explicitly - `0` means keep forever. Any workflow that was relying on
+permanent storage under an unset TTL field needs that `0` set explicitly,
+or it will start losing data a day later.
+
 ### Filesystem
 
 ```javascript

+ 2 - 2
nodes/core/http-request.js

@@ -81,7 +81,7 @@ const configSchema = {
       type: 'number',
       title: 'TTL (hours)',
       default: 24,
-      description: 'Auto-delete stored file after hours (0 = never)'
+      description: 'Auto-delete stored file after this many hours. Default is 24 hours; set to 0 to keep it forever.'
     }
   },
   required: ['url']
@@ -335,7 +335,7 @@ async function execute(config, input, context) {
       // Store in database if configured
       if (config.storeDownload) {
         const collection = config.downloadCollection || 'downloads';
-        const ttlHours = config.downloadTtlHours || 0;
+        const ttlHours = config.downloadTtlHours ?? 24;
         const ttlMs = ttlHours > 0 ? ttlHours * 60 * 60 * 1000 : 0;
 
         const checksum = contentHash;

+ 2 - 2
nodes/imap/imap-extract-attachments.js

@@ -58,7 +58,7 @@ const configSchema = {
             type: 'number',
             title: 'TTL (hours)',
             default: 24,
-            description: 'Auto-delete stored files after hours (0 = never)'
+            description: 'Auto-delete stored files after this many hours. Default is 24 hours; set to 0 to keep them forever.'
         },
         binaryStoragePath: {
             type: 'string',
@@ -439,7 +439,7 @@ module.exports = {
         // Store metadata in database if configured (without binary data)
         if (config.storeInDatabase && attachments.length > 0) {
             const collection = config.storageCollection || 'email_attachments';
-            const ttlHours = config.storageTtlHours || 0;
+            const ttlHours = config.storageTtlHours ?? 24;
             const ttlMs = ttlHours > 0 ? ttlHours * 60 * 60 * 1000 : 0;
 
             for (const att of attachments) {

+ 20 - 0
tests/nodes/config-defaults-ttl-zero-http-request.json

@@ -0,0 +1,20 @@
+{
+  "name": "verify-config-defaults-ttl-zero-http-request",
+  "settings": {
+    "storagePermissions": {
+      "collections": { "ttl0_download_probe": "read-write" }
+    }
+  },
+  "nodes": [
+    {"id": "n1", "name": "Trigger", "type": "click-trigger", "position": {"x": 0, "y": 0}, "config": {}},
+    {"id": "dl", "name": "Download", "type": "http-request", "position": {"x": 0, "y": 100},
+     "config": {"method": "GET", "url": "http://localhost:8090/index.html", "responseMode": "binary",
+       "storeDownload": true, "downloadCollection": "ttl0_download_probe", "downloadTtlHours": 0}}
+  ],
+  "connections": [
+    {"sourceNodeId": "n1", "sourceOutput": "main", "targetNodeId": "dl", "targetInput": "data"}
+  ],
+  "expect": {
+    "dl": {"status": "completed", "output": {"storage": {"collection": "ttl0_download_probe"}}}
+  }
+}

+ 23 - 0
tests/nodes/config-defaults-ttl-zero-imap-extract-attachments.json

@@ -0,0 +1,23 @@
+{
+  "name": "verify-config-defaults-ttl-zero-imap-extract-attachments",
+  "settings": {
+    "storagePermissions": {
+      "collections": { "ttl0_attach_probe": "read-write" }
+    }
+  },
+  "nodes": [
+    {"id": "n1", "name": "Trigger", "type": "click-trigger", "position": {"x": 0, "y": 0}, "config": {}},
+    {"id": "mail", "name": "Mail", "type": "code", "position": {"x": 0, "y": 100},
+     "config": {"code": "var boundary = 'BOUNDARY123'; var b64 = smartbotic.utils.base64Encode('hello world'); var raw = 'Content-Type: multipart/mixed; boundary=\"' + boundary + '\"\\n\\n' + '--' + boundary + '\\n' + 'Content-Type: text/plain\\n\\n' + 'body text\\n' + '--' + boundary + '\\n' + 'Content-Type: text/plain; name=\"test.txt\"\\n' + 'Content-Disposition: attachment; filename=\"test.txt\"\\n' + 'Content-Transfer-Encoding: base64\\n\\n' + b64 + '\\n' + '--' + boundary + '--'; return { raw: raw };"}},
+    {"id": "extract", "name": "Extract", "type": "imap-extract-attachments", "position": {"x": 0, "y": 200},
+     "config": {"emailSource": "{{data.result.raw}}", "deduplicateByHash": false, "storeInDatabase": true,
+       "storageCollection": "ttl0_attach_probe", "storageTtlHours": 0}}
+  ],
+  "connections": [
+    {"sourceNodeId": "n1", "sourceOutput": "main", "targetNodeId": "mail", "targetInput": "data"},
+    {"sourceNodeId": "mail", "sourceOutput": "main", "targetNodeId": "extract", "targetInput": "data"}
+  ],
+  "expect": {
+    "extract": {"status": "completed", "output": {"count": 1}}
+  }
+}