smartbotic/docs/nodes.md

Creating Workflow Nodes

This guide explains how to create custom workflow nodes for SmartBotic.

Node File Structure

Nodes are JavaScript modules located in nodes/<category>/ directories. Each node file must export:

• configSchema - JSON Schema for node configuration
• inputSchema - JSON Schema for input data
• outputSchema - JSON Schema for output data
• execute - Async function that performs the node's action

Basic Template

/**
 * @node my-node-id
 * @name My Node Name
 * @category my-category
 * @version 1.0.0
 * @description What this node does
 * @icon icon-name
 */

const configSchema = {
    type: 'object',
    properties: {
        myOption: {
            type: 'string',
            title: 'My Option',
            description: 'Description of this option',
            default: 'default-value'
        }
    },
    required: ['myOption']
};

const inputSchema = {
    type: 'object',
    properties: {
        data: {
            type: 'any',
            description: 'Input data'
        }
    }
};

const outputSchema = {
    type: 'object',
    properties: {
        result: {
            type: 'string',
            description: 'Output result'
        }
    }
};

module.exports = {
    configSchema,
    inputSchema,
    outputSchema,

    async execute(config, input, context) {
        // Your node logic here
        return {
            result: 'success'
        };
    }
};

JSDoc Metadata Tags

The comment block at the top of the file defines node metadata:

Tag	Required	Description
@node	Yes	Unique node identifier (kebab-case)
@name	Yes	Display name shown in UI
@category	Yes	Category for grouping (e.g., http, data, email)
@version	Yes	Semantic version (e.g., 1.0.0)
@description	Yes	Brief description of node functionality. Must be a single line - the parser captures only the first line of a multi-line @description
@icon	No	Lucide icon name (e.g., globe, mail, database)
@trigger	No	Add this tag if the node is a trigger (starts workflows)

Configuration Schema

The configSchema defines what options users can configure in the node editor.

A top-level property's default is applied automatically whenever the stored config omits that key - both when the workflow is saved and again before the node executes - so execute() can rely on the key being present even for a workflow saved before the field existed. A default is resolved through the same expression evaluation as any other stored config value, so a default: containing {{ }} will be evaluated as an expression rather than kept as a literal string. No current node relies on this; keep it in mind if you're the first to try it.

Supported Field Types

// String field
myString: {
    type: 'string',
    title: 'Label',
    description: 'Help text',
    default: 'default value'
}

// Number field
myNumber: {
    type: 'number',
    title: 'Count',
    default: 10
}

// Boolean field
myBoolean: {
    type: 'boolean',
    title: 'Enable Feature',
    default: false
}

// Dropdown (enum)
myChoice: {
    type: 'string',
    title: 'Select Option',
    enum: ['option1', 'option2', 'option3'],
    default: 'option1'
}

// Object field
myHeaders: {
    type: 'object',
    title: 'Headers',
    additionalProperties: { type: 'string' }
}

Dynamic Options (Credentials)

To create a credential selector:

credentialId: {
    type: 'string',
    title: 'Credential',
    description: 'Select authentication credential',
    dynamicOptions: {
        source: 'credentials',
        filter: { type: 'bearer' }  // Filter by credential type
    }
}

Available credential types: bearer, api_key, basic, imap

Custom Outputs

Define multiple output ports:

const outputs = [
    { name: 'success', displayName: 'Success', type: 'object', color: '#10b981' },
    { name: 'error', displayName: 'Error', type: 'object', color: '#ef4444' }
];

module.exports = {
    configSchema,
    inputSchema,
    outputSchema,
    outputs,
    execute
};

Outputs That Come From Config

A node whose output count depends on how it is configured declares dynamicOutputs beside its static outputs:

const outputs = [
    { name: 'fallback', displayName: 'Fallback', type: 'any', color: '#6b7280' }
];

const dynamicOutputs = {
    from: 'rules',
    namePrefix: 'case',
    labelFrom: 'label',
    color: '#3b82f6'
};

module.exports = { configSchema, inputSchema, outputSchema, outputs, dynamicOutputs, execute };

For each entry in the placed node's config.rules, the editor draws a port named case0, case1 and so on, labelled from that entry's label field, followed by the static outputs. Port names are positional, so an edge survives editing a rule's label or value. Deleting a rule drops the edges hanging off the port that went with it.

At run time the node routes with _activeBranch, exactly as a fixed-port node does - the branch name is just computed:

return { _activeBranch: 'case' + i, ['case' + i]: data };

Pausing for an Answer

A node pauses its execution by returning a _pause marker, the way a branching node returns _activeBranch:

return {
    token: token,
    reason: 'Approve the refund',
    expiresAt: Date.now() + 86400000,
    _pause: { token: token, reason: 'Approve the refund', expiresAt: expiresAt }
};

The engine stops the walk, stores the execution as waiting with everything computed so far, and returns. Nothing downstream runs. The marker is stripped from the stored output, so a reader sees the request rather than the mechanism.

Answering it continues the run:

POST /api/v1/executions/{id}/resume
{ "token": "...", "approved": true, "data": { "note": "looks fine" } }

The paused node's output becomes that payload, and the walk continues from there. Nodes that already ran are not run again. The workflow is rebuilt from the snapshot stored with the execution, not from the workflow as it stands now, because it may have been edited while the approval waited.

GET /api/v1/executions/pending lists executions waiting for an answer. It deliberately omits the token: listing is a weaker permission than approving.

Two limits are deliberate. A pause inside a Loop body cannot be resumed, because loop iteration state is not part of the stored execution, so a node must refuse to pause there rather than record something unanswerable. And a webhook cannot wait for an approval - the HTTP request is still open and its deadline is 35 seconds - so a webhook-triggered workflow that pauses returns a 202 with { "executionId": "...", "status": "waiting" } rather than blocking for an answer that has nowhere to arrive on this connection. Answer it the same way as any other paused execution, with POST /api/v1/executions/{id}/resume.

Answering a Webhook

A webhook-triggered workflow returns the last node's output as JSON by default. To control the response, return a _webhookResponse marker from any node:

return {
    _webhookResponse: {
        status: 201,
        headers: { 'Content-Type': 'application/json' },
        body: { id: created.id }
    }
};

Any node may set it, not only the last one to run, so adding a node to the end of a workflow cannot silently change what its API returns. If several set it, the last one wins and the runner logs that it happened.

Multiple Inputs

A node that joins two branches names its inputs:

const inputs = [
    { name: 'input1', displayName: 'Input 1', type: 'any', required: false },
    { name: 'input2', displayName: 'Input 2', type: 'any', required: false }
];

module.exports = { configSchema, inputSchema, outputSchema, inputs, execute };

Each named input becomes its own target handle, and arrives in execute under that name - input.input1, input.input2. A node that declares no inputs keeps the single data handle it has always had.

Note the difference between a node that sets _activeBranch and one that does not: with the marker, exactly one output carries data and everything downstream of the others is skipped. Without it, every named output the node returns is live at once, which is how filter sends kept and discarded items down two paths in the same run.

Conventions These Nodes Follow

Four rules emerged while building the Tier 1 set. New nodes should follow them.

Fail loudly rather than drop data. A node that cannot do what was asked throws, with a message naming what it received. set-fields throws when keep-all mode is handed an array instead of an object; its dotted paths refuse to overwrite a value they would otherwise destroy; merge throws when an item lacks the key it was told to combine on. The alternative - returning an empty object, or bucketing everything under the string undefined - produces a workflow that keeps running and quietly produces wrong data.

Guard a configurable output name against your own reserved keys. A node that returns fixed keys alongside a user-named field must reject a name that would collide, throwing early:

if (outputField === 'count' || outputField === 'groups' || (groupBy && outputField === 'key')) {
    throw new Error('Aggregate: outputField cannot be "' + outputField +
        '", which is a reserved output name for this node. Pick another name.');
}

aggregate, sort-limit-dedupe and datetime all need this. template and json do not, because they return exactly one key and there is nothing to collide with - do not add the guard where it protects nothing. When a node builds its reserved-key list dynamically, as aggregate does for key (only reserved once groupBy is set, since that is the only time it is written), check every mode the node has, not just the always-on keys.

Config values arrive already evaluated. The engine resolves {{...}} in node config before execute() runs, and a config string that is exactly one expression keeps its native type. So a field typed as a string in the schema can legitimately hold an array at run time, which is why several nodes begin with if (Array.isArray(inputField)). That branch is not dead code. Nodes must not re-interpolate config values. template is the one deliberate exception: its templateSource: 'input' mode reads template text out of the input DATA at run time, which the engine never walks, so that text still has its {{...}} placeholders intact when execute() sees it, and calling smartbotic.utils.interpolate on it is genuine work, not a re-interpolation of something the engine already resolved.

Read paths with the shared helper. Use smartbotic.utils.getFieldValue(data, path) rather than writing a private path walker. Two older nodes, if-condition and loop, still carry their own divergent copies - loop silently skips a leading data. segment and if-condition does not - and that inconsistency is exactly what the shared helper exists to stop spreading.

Available APIs

Inside execute(), you have access to the smartbotic global object:

Logging

smartbotic.log.debug('Debug message');
smartbotic.log.info('Info message');
smartbotic.log.warn('Warning message');
smartbotic.log.error('Error message');

HTTP Requests

const response = smartbotic.http.request({
    method: 'GET',  // GET, POST, PUT, PATCH, DELETE
    url: 'https://api.example.com/data',
    headers: { 'Authorization': 'Bearer token' },
    body: JSON.stringify({ key: 'value' }),
    timeout: 30000,
    followRedirects: true
});

// Response: { status, headers, data, dataBase64 }

Sending and receiving binary

body is a text field. It is read as a UTF-8 string, so any byte sequence that is not valid UTF-8 is replaced on the way out - an image sent that way uploads successfully and arrives corrupt. There are two binary-safe paths instead.

For an API that takes a bare binary payload, such as an S3 PUT or the AT Protocol blob upload, use bodyBase64. It is decoded to raw bytes and sent as the whole body, and it wins over body when both are given:

const file = smartbotic.storage.downloadFile(fileId);

smartbotic.http.request({
    method: 'POST',
    url: 'https://bsky.social/xrpc/com.atproto.repo.uploadBlob',
    headers: { 'Content-Type': 'image/jpeg' },
    bodyBase64: file.data
});

For an API that expects a form upload, use files with formData, which builds a multipart body - see nodes/integration/telegram-send.js.

On the way back, data is also UTF-8 text and cannot carry binary. Read dataBase64 for image or file downloads.

One trap worth knowing: utils.base64Encode encodes a JavaScript string as UTF-8, so it is not a byte-preserving round trip for arbitrary bytes you build in JavaScript with String.fromCharCode. Anything at or above 0x80 becomes two bytes. Binary should come from the file store, where it never passes through a JavaScript string, rather than being assembled by hand.

Storage (Database)

// Insert document
const result = smartbotic.storage.insert('collection', { key: 'value' });
// Returns: { success, id }

// Get document
const doc = smartbotic.storage.get('collection', 'document-id');
// Returns: { found, document }

// Query documents
const results = smartbotic.storage.query('collection', { field: 'value' });
// Returns: { success, documents }

// Update document
smartbotic.storage.update('collection', 'document-id', { key: 'new-value' });

// Delete document
smartbotic.storage.delete('collection', 'document-id');

http-request (Store Download) and imap-extract-attachments (Store in Database) both write through smartbotic.storage.insert with a TTL. Both default that TTL field to 24 hours, so downloaded files and extracted attachments stored by those nodes now expire and are auto-deleted a day after they're stored, unless the workflow sets the TTL field to 0 explicitly - 0 means keep forever. Any workflow that was relying on permanent storage under an unset TTL field needs that 0 set explicitly, or it will start losing data a day later.

Filesystem

// Read a file
const read = smartbotic.fs.readFile('/tmp/example.txt', 'base64');
// { success, data } - data is base64-encoded when the encoding argument is 'base64'

// Write a file
smartbotic.fs.writeFile('/tmp/example.txt', base64Data, 'base64');

// Check existence
const there = smartbotic.fs.exists('/tmp/example.txt');

// Stat a file
const info = smartbotic.fs.stat('/tmp/example.txt');
// { success, size, mtime, isDirectory } - mtime is milliseconds since the epoch

// Create a directory
smartbotic.fs.mkdir('/tmp/example-dir');

// Delete a file
smartbotic.fs.unlink('/tmp/example.txt');

// List a directory, one level
const listing = smartbotic.fs.readdir('/var/spool/incoming');
// { success: true, entries: [{ name, path, size, modifiedAt, isDirectory }] }

The fs API applies no path restrictions. Every call takes an arbitrary absolute path and acts on it with the runner process's own permissions, the same trust level a code node or process.exec already has.

Credentials

const auth = smartbotic.credentials.get(credentialId);
if (auth.success) {
    // auth.headerName = 'Authorization' or 'X-API-Key'
    // auth.headerValue = 'Bearer <token>' or '<api-key>'
    headers[auth.headerName] = auth.headerValue;
}

Sending Email

const result = smartbotic.smtp.send({
    credentialId: config.credentialId,  // an smtp credential, or the imap one for the same mailbox
    to: ['someone@example.com'],        // a single address or an array
    cc: [],
    bcc: [],
    subject: 'Subject line',
    body: 'Message text',
    html: false,                        // true sends the body as text/html
    replyTo: '',
    from: '',                           // overrides the sender on the credential
    fromName: '',
    attachments: [
        { filename: 'note.txt', mimeType: 'text/plain', contentBase64: '...' }
    ]
});
// result.messageId - the Message-ID the mail went out with
// result.accepted  - how many addresses it was submitted for

An SMTP credential holds host, port, username, password, security (starttls, ssl or none), and optionally from_address and from_name. An IMAP credential is accepted in its place: the same account is reached on the submission port with STARTTLS, so a mailbox stored for reading mail does not have to be entered again to send it.

Utilities

// Generate UUID
const id = smartbotic.utils.uuid();

// Sleep (pause execution)
smartbotic.utils.sleep(1000);  // milliseconds, max 300000 (5 min)

// Template interpolation
const result = smartbotic.utils.interpolate('Hello {{name}}', { name: 'World' });

// Base64 encoding/decoding
const encoded = smartbotic.utils.base64Encode('data');
const decoded = smartbotic.utils.base64Decode(encoded);

// SHA256 hash
const hash = smartbotic.utils.sha256('data');  // Returns hex string

// Object utilities
const picked = smartbotic.utils.pick(obj, ['key1', 'key2']);
const omitted = smartbotic.utils.omit(obj, ['unwantedKey']);

// Read a nested value by dotted path
const city = smartbotic.utils.getFieldValue(input, 'data.user.address.city');
const second = smartbotic.utils.getFieldValue(input, 'data.items[1].id');
const howMany = smartbotic.utils.getFieldValue(input, 'data.items.length');

Missing paths return undefined. Arrays are reached with key[0] or a bare numeric segment, and .length works on an array. The path is taken literally, with no special handling of a leading data. segment.

Important Guidelines

Indentation

Use 4 spaces for indentation (consistent with existing nodes).

Avoid Inline Comments

Do NOT use // comments inside schema definitions. The parser may incorrectly interpret them:

// BAD - comment may break parsing
const configSchema = {
    type: 'object',
    properties: {
        url: {
            type: 'string',
            default: 'http://localhost:3000'  // This is the default  <- AVOID
        }
    }
};

// GOOD - no inline comments in schema
const configSchema = {
    type: 'object',
    properties: {
        url: {
            type: 'string',
            default: 'http://localhost:3000'
        }
    }
};

String Quotes

Use single quotes for string values in schemas. The parser converts them to JSON:

// GOOD
title: 'My Title'

// AVOID - embedded quotes can cause issues
description: 'Use "quotes" carefully'  // May break parsing

Error Handling

IMPORTANT: Throw errors instead of returning success: false. This ensures the workflow fails properly:

async execute(config, input, context) {
    // Validate required inputs
    if (!config.requiredField) {
        throw new Error('Required field is missing');
    }

    // Make API call
    const response = smartbotic.http.request({
        method: 'GET',
        url: config.url,
        headers: headers
    });

    // Check response status - throw on error
    if (response.status < 200 || response.status >= 300) {
        const errorMsg = typeof response.data === 'string'
            ? response.data
            : JSON.stringify(response.data);
        throw new Error(`API error (${response.status}): ${errorMsg}`);
    }

    // Return data on success (no try/catch needed for most cases)
    return {
        data: response.data
    };
}

Do NOT catch errors just to return { success: false } - let them propagate so the workflow fails.

Migrating Nodes to Database

After creating or modifying node files, migrate them to the running database:

# Get authentication token
TOKEN=$(curl -s http://localhost:8090/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "admin"}' | jq -r '.accessToken')

# Migrate nodes
curl -X POST http://localhost:8090/api/v1/nodes/migrate \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"nodesPath": "./nodes"}'

Runners automatically receive node updates via gRPC streaming.

Example: HTTP Download Node

/**
 * @node download-file
 * @name Download File
 * @category http
 * @version 1.0.0
 * @description Download a file from URL
 * @icon download
 */

const configSchema = {
    type: 'object',
    properties: {
        url: {
            type: 'string',
            title: 'URL',
            description: 'URL to download from'
        },
        timeout: {
            type: 'number',
            title: 'Timeout (ms)',
            default: 30000
        }
    },
    required: ['url']
};

const inputSchema = {
    type: 'object',
    properties: {
        url: { type: 'string', description: 'Override URL from input' }
    }
};

const outputSchema = {
    type: 'object',
    properties: {
        success: { type: 'boolean' },
        data: { type: 'string' },
        contentType: { type: 'string' }
    }
};

module.exports = {
    configSchema,
    inputSchema,
    outputSchema,

    async execute(config, input, context) {
        const url = input.url || config.url;

        smartbotic.log.info(`Downloading from ${url}`);

        try {
            const response = smartbotic.http.request({
                method: 'GET',
                url: url,
                timeout: config.timeout
            });

            if (response.status < 200 || response.status >= 300) {
                throw new Error(`HTTP ${response.status}`);
            }

            return {
                success: true,
                data: response.data,
                contentType: response.headers['content-type'] || ''
            };
        } catch (error) {
            smartbotic.log.error(`Download failed: ${error.message}`);
            return {
                success: false,
                error: error.message
            };
        }
    }
};

Trigger Nodes

Trigger nodes start workflow executions. Add @trigger to the JSDoc:

/**
 * @node my-trigger
 * @name My Trigger
 * @category triggers
 * @version 1.0.0
 * @description Triggers workflow on event
 * @icon zap
 * @trigger
 */

Triggers typically don't have input schemas (they generate initial data).

Chat nodes for OpenAI-compatible providers

OpenRouter, Together, Groq, DeepSeek, Mistral, xAI, Fireworks, Perplexity, DeepInfra and OpenAI itself all accept the same chat request and return the same reply. One implementation covers all of them, in scripts/gen-openai-compatible-nodes.py, which writes one node per provider. A node cannot require another node's file, so the code is generated rather than shared at runtime - edit the generator, run it, commit the diff.

Adding a provider is a row in the PROVIDERS table: its id, display name, base URL, a sensible default model, and where its keys come from. Set no_model_list when the provider publishes no /models endpoint, and the model field becomes a plain box rather than an empty dropdown.

Each node registers a credential type of its own, so a key for one provider is never offered to another - and any of them will still take a plain bearer credential, because the shape is the same.

The base URL is a setting, so any of these nodes also reaches a proxy, a gateway or a self-hosted service that speaks the same API.

Retrying an HTTP request

smartbotic.http.request takes retry options, so a node does not need its own loop and its own sleep:

smartbotic.http.request({
  method: 'GET', url: feedUrl, timeout: 30000,
  retries: 2,               // extra attempts, 0 (off) unless asked for
  retryDelayMs: 2000,       // doubles each attempt
  retryMaxDelayMs: 30000,   // and is capped here
  retryOnStatus: [429, 503] // defaults to 408, 425, 429, 500, 502, 503, 504
});

Retried: a timeout, a refused connection, a name that would not resolve, and the statuses above. Not retried: a 404 or a 401 - those are answers, and asking again does not change them. A Retry-After header from the server wins over the backoff, capped at retryMaxDelayMs, because being told to wait and then not waiting is how a rate limit becomes a ban.

It is off by default on purpose. A POST that timed out may already have been received at the far end, and repeating it would do the thing twice - so the caller opts in where repeating is safe.