# SmartBotic

A powerful workflow automation and execution platform built with a modern microservices architecture.

## Overview

SmartBotic is a distributed system for creating, managing, and executing complex workflows. It features a high-performance C++20 backend with gRPC communication, an in-memory database with persistence, and a modern React/TypeScript frontend for workflow design.

### Key Features

- **Microservices Architecture**: Three independent services (Database, WebServer, Runner) that scale independently
- **Visual Workflow Editor**: React-based UI with drag-and-drop workflow design using ReactFlow
- **JavaScript Node System**: Extensible workflow nodes written in JavaScript, executed in QuickJS
- **Real-time Updates**: WebSocket support for live workflow execution monitoring
- **Distributed Execution**: Multiple runner instances for parallel workflow execution
- **Persistent Storage**: In-memory database with Write-Ahead Log (WAL) and snapshot support
- **Hot Reload**: Node changes are automatically propagated to runners without restart
- **JWT Authentication**: Secure API access with token-based authentication
- **RESTful API**: Comprehensive HTTP API for all operations

## Architecture

SmartBotic consists of three microservices:

```
┌─────────────────┐
│   WebUI         │  React/TypeScript Frontend
│   (Port 3000)   │  Workflow Editor, Dashboard
└────────┬────────┘
         │ HTTP/WS
         ↓
┌─────────────────┐
│   WebServer     │  HTTP REST API, WebSocket
│   (Port 8090)   │  JWT Auth, Runner Registry
└────┬───────┬────┘
     │       │ gRPC
     ↓       ↓
┌─────────┐ ┌──────────┐
│Database │ │ Runner(s)│  Workflow Execution
│(Pt 9010)│ │(Pt 9011) │  QuickJS Engine
└─────────┘ └──────────┘
```

### Services

1. **Database Service** (`src/database/`)
   - In-memory key-value storage with gRPC API
   - Write-Ahead Log (WAL) for durability
   - Periodic snapshots for recovery
   - Prefix-based queries and watch support

2. **WebServer Service** (`src/webserver/`)
   - HTTP REST API for workflows, nodes, and executions
   - WebSocket endpoint for real-time updates
   - JWT-based authentication
   - Runner registry and load balancing
   - Static file serving for the frontend

3. **Runner Service** (`src/runner/`)
   - Executes workflows using QuickJS JavaScript engine
   - Supports hot-reload of node definitions
   - Concurrent execution with configurable limits
   - Context injection for node execution (credentials, logging, etc.)

## Prerequisites

### C++ Backend

- **Compiler**: GCC 11+ or Clang 14+ (C++20 support required)
- **CMake**: 3.20 or higher
- **Dependencies**:
  - gRPC and Protocol Buffers
  - OpenSSL
  - spdlog (logging)
  - nlohmann/json (JSON parsing)
  - cpp-httplib (HTTP server)
  - QuickJS (JavaScript engine)
  - bcrypt (password hashing)
  - libcurl

### Frontend

- **Node.js**: 18.x or higher
- **npm**: 9.x or higher

## Building

### Backend (C++)

```bash
# Create build directory
mkdir build && cd build

# Debug build
cmake -DCMAKE_BUILD_TYPE=Debug ..
make -j$(nproc)

# Release build (optimized)
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j$(nproc)

# With Address Sanitizer (for debugging memory issues)
cmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_ASAN=ON ..
make -j$(nproc)
```

Binaries will be created in the `build/` directory:
- `smartbotic-database`
- `smartbotic-webserver`
- `smartbotic-runner`

### Frontend (WebUI)

```bash
cd webui

# Install dependencies
npm install

# Development server (port 3000, proxies API to localhost:8090)
npm run dev

# Production build
npm run build

# Lint
npm run lint
```

The production build outputs to `webui/dist/` and is served by the WebServer service.

## Running

Services must be started in order due to dependencies:

### Manual Start

```bash
# 1. Start Database service (must be first)
./build/smartbotic-database

# 2. Start WebServer (depends on Database)
./build/smartbotic-webserver

# 3. Start Runner(s) (depends on Database and WebServer)
./build/smartbotic-runner
```

### Using systemd

```bash
# Start all services
systemctl --user start smartbotic.target

# Check status
systemctl --user status smartbotic-*

# View logs
journalctl --user -u smartbotic-* -f

# Stop all services
systemctl --user stop smartbotic.target
```

Unit files are provided in the `systemd/` directory.

### Default Ports

- **WebUI Dev Server**: 3000
- **WebServer HTTP API**: 8090
- **WebServer gRPC**: 9002
- **Database gRPC**: 9010
- **Runner gRPC**: 9011

### Default Credentials

- **Username**: `admin`
- **Password**: `admin`

⚠️ **Change these credentials in production!**

## Configuration

Services load JSON configuration files from the `config/` directory:

- `database.json` - Storage settings, WAL/snapshot intervals
- `webserver.json` - HTTP port, JWT settings, CORS, runner load balancing
- `runner.json` - Runner ID, max concurrent executions, node hot reload

Configuration supports environment variable substitution using `${VAR_NAME:default}` syntax.

### Example: `config/webserver.json`

```json
{
  "httpPort": 8090,
  "grpcPort": 9002,
  "databaseAddress": "localhost:9010",
  "jwtSecret": "${JWT_SECRET:your-secret-key-change-in-production}",
  "jwtExpiration": 3600,
  "staticFilesPath": "./webui/dist"
}
```

## Development

### Workflow Nodes

Workflow nodes are JavaScript modules located in the `nodes/` directory. Each node defines:
- Configuration schema (static settings)
- Input schema (data from previous nodes)
- Output schema (data passed to next nodes)
- Execute function (async JavaScript code)

**See [docs/nodes.md](docs/nodes.md) for a complete guide on creating nodes.**

#### Example Node

```javascript
// nodes/my-node.js
module.exports = {
  configSchema: {
    type: "object",
    properties: {
      message: { type: "string", default: "Hello" }
    }
  },
  inputSchema: {
    type: "object",
    properties: {
      name: { type: "string" }
    }
  },
  outputSchema: {
    type: "object",
    properties: {
      greeting: { type: "string" }
    }
  },
  execute: async (config, input, context) => {
    return {
      greeting: `${config.message}, ${input.name}!`
    };
  }
};
```

### Migrating Nodes to Database

After creating nodes in `nodes/`, migrate them to the running database:

```bash
# 1. Get JWT token
TOKEN=$(curl -s http://localhost:8090/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "admin"}' | jq -r '.accessToken')

# 2. Migrate nodes
curl -X POST http://localhost:8090/api/v1/nodes/migrate \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"nodesPath": "./nodes"}'

# 3. Verify
curl http://localhost:8090/api/v1/nodes \
  -H "Authorization: Bearer $TOKEN"
```

Runners automatically receive updates via gRPC streaming - no restart required!

### API Examples

```bash
# Login
curl -X POST http://localhost:8090/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "admin"}'

# List workflows
curl http://localhost:8090/api/v1/workflows \
  -H "Authorization: Bearer $TOKEN"

# Create workflow
curl -X POST http://localhost:8090/api/v1/workflows \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Workflow",
    "description": "Test workflow",
    "definition": {...}
  }'

# Execute workflow
curl -X POST http://localhost:8090/api/v1/workflows/{id}/execute \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input": {}}'
```

### WebSocket API

Connect to `ws://localhost:8090/api/v1/ws?token={JWT_TOKEN}` for real-time updates:

```javascript
const ws = new WebSocket(`ws://localhost:8090/api/v1/ws?token=${token}`);

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log('Update:', data);
};
```

## Project Structure

```
smartbotic/
├── build/              # Build output directory
├── cmake/              # CMake modules
├── config/             # Runtime configuration files
├── docs/               # Documentation
│   └── nodes.md        # Node development guide
├── lib/                # Shared C++ libraries
│   ├── common/         # Utilities (UUID, time, string)
│   ├── config/         # Configuration loader
│   ├── credentials/    # Credential management
│   ├── crypto/         # Encryption (AES-GCM)
│   ├── logging/        # Logging wrapper
│   └── storage/        # Database client
├── nodes/              # Workflow node definitions (JS)
├── proto/              # Protocol Buffer definitions
├── src/                # Microservice implementations
│   ├── database/       # Database service
│   ├── runner/         # Runner service
│   └── webserver/      # WebServer service
├── systemd/            # systemd unit files
├── webui/              # React frontend
│   ├── src/
│   │   ├── components/ # React components
│   │   ├── pages/      # Page components
│   │   ├── stores/     # Zustand state management
│   │   └── types/      # TypeScript types
│   └── dist/           # Production build output
├── CLAUDE.md           # AI assistant guidance
└── CMakeLists.txt      # Root CMake configuration
```

## C++ Standards

- **C++20** required (strict compliance, no extensions)
- Compiler warnings: `-Wall -Wextra -Wpedantic`
- Modern practices: RAII, smart pointers, move semantics
- Error handling: Exceptions for exceptional cases, error codes for expected failures

## Frontend Stack

- **React** 18.2 with TypeScript
- **State Management**: Zustand + React Query
- **UI**: TailwindCSS + Lucide icons
- **Workflow Editor**: ReactFlow
- **Build Tool**: Vite
- **Code Quality**: ESLint + TypeScript strict mode

## Testing

```bash
# C++ (if tests are configured)
cd build
ctest

# Frontend
cd webui
npm run lint
```

## Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Follow the existing code style and architecture
4. Ensure all services build successfully
5. Test your changes thoroughly
6. Commit your changes (`git commit -m 'feat: add amazing feature'`)
7. Push to the branch (`git push origin feature/amazing-feature`)
8. Open a Pull Request

### Commit Message Convention

Follow conventional commits:
- `feat:` - New feature
- `fix:` - Bug fix
- `docs:` - Documentation changes
- `refactor:` - Code refactoring
- `test:` - Adding tests
- `chore:` - Maintenance tasks

## Troubleshooting

### Services won't start

- Ensure ports 8090, 9002, 9010, 9011 are available
- Start services in order: Database → WebServer → Runner
- Check logs for detailed error messages

### Build errors

- Verify C++20 compiler support
- Install all required dependencies
- Clear build directory and rebuild: `rm -rf build && mkdir build && cd build && cmake ..`

### Frontend proxy issues

- Ensure WebServer is running on port 8090
- Check `webui/vite.config.ts` proxy configuration

### Node execution failures

- Verify node definitions have all required fields
- Check QuickJS syntax compatibility (ES2020)
- Review runner logs for JavaScript errors

## Performance

- **Database**: In-memory storage with microsecond latency
- **WebServer**: Async I/O with cpp-httplib
- **Runner**: Configurable concurrent execution limit
- **Frontend**: Code splitting and lazy loading

## Security

- JWT-based authentication with configurable expiration
- Password hashing with bcrypt
- Credential encryption with AES-256-GCM
- CORS configuration in WebServer
- Input validation on all API endpoints

⚠️ **Production Checklist**:
- [ ] Change default admin credentials
- [ ] Set strong JWT secret
- [ ] Configure HTTPS/TLS
- [ ] Enable firewall rules
- [ ] Regular security updates

## License

[License information not specified - please add LICENSE file]

## Support

For issues, questions, or contributions, please use the project's issue tracker.

---

**Built with ❤️ using C++20, React, and modern cloud-native practices.**