smartbotic-microbit/docs/DEPLOYMENT.md

Deployment Guide

This guide covers deploying Smartbotic Microbit in production environments.

Table of Contents

• System Requirements
• Installation Methods
• Configuration
• Service Management
• Nginx Setup
• Security Considerations
• Monitoring and Logs
• Backup and Recovery
• Troubleshooting
• Upgrading

System Requirements

Hardware Requirements

Minimum:

• CPU: 2 cores
• RAM: 2 GB
• Disk: 10 GB free space
• Network: Stable internet connection

Recommended:

• CPU: 4 cores
• RAM: 4 GB
• Disk: 20 GB SSD
• Network: 100 Mbps+

Software Requirements

Operating System:

• Debian 12 (Bookworm) - recommended
• Ubuntu 22.04 LTS or later
• Other Linux distributions (may require adjustments)

System Packages:

• systemd (for service management)
• Nginx or Apache (for reverse proxy)
• OpenSSL (for TLS)

Installation Methods

Method 1: Using Pre-built Packages (Recommended)

This is the easiest method for production deployment.

Step 1: Download Packages

Download the latest release packages:

• smartbotic-database-*.tar.gz

• smartbotic-microbit-*.tar.gz

  # Example download (replace with actual URLs)
  wget https://releases.smartbotics.ai/smartbotic-database-0.1.0-debian12.tar.gz
  wget https://releases.smartbotics.ai/smartbotic-microbit-0.1.0-debian12.tar.gz
  

Step 2: Run Installer

# Install both packages
sudo ./packaging/scripts/install.sh smartbotic-*.tar.gz

# Or for verbose output
sudo ./packaging/scripts/install.sh -v smartbotic-*.tar.gz

Installer Options:

• --prefix DIR: Install location (default: /opt/smartbotic)
• --confdir DIR: Config directory (default: /etc/smartbotic)
• --user USER: Service user (default: smartbotic)
• --no-systemd: Skip systemd service installation
• --dry-run: Show what would be done without making changes
• -v, --verbose: Show detailed output

Step 3: Directory Structure

After installation:

/opt/smartbotic/
├── bin/
│   ├── smartbotic-database
│   └── smartbotic-microbit
└── share/smartbotic/
    └── nginx/

/etc/smartbotic/
├── database.json
├── microbit.json
└── smartbotic.env.default

/var/lib/smartbotic/
├── database/
│   ├── storage/
│   └── migrations/
└── microbit/
    └── webui/

/var/log/smartbotic/
├── database.log
└── microbit.log

Method 2: Building from Source

For custom builds or development deployment, see DEVELOPMENT.md.

Configuration

Environment Configuration

Create the main environment file:

sudo cp /etc/smartbotic/smartbotic.env.default /etc/smartbotic/smartbotic.env
sudo chmod 640 /etc/smartbotic/smartbotic.env
sudo chown root:smartbotic /etc/smartbotic/smartbotic.env
sudo nano /etc/smartbotic/smartbotic.env

Required settings in /etc/smartbotic/smartbotic.env:

# JWT Secret (REQUIRED - generate a strong random secret)
JWT_SECRET="your-secret-key-change-this-to-random-string"

# Database connection
DB_ADDRESS="localhost:9004"

# SMTP Settings (for email notifications)
SMTP_HOST="smtp.example.com"
SMTP_PORT="587"
SMTP_USER="noreply@example.com"
SMTP_PASSWORD="smtp-password"
SMTP_FROM="noreply@example.com"

# CallerAI Integration (for AI assistants)
CALLERAI_API_URL="https://api.caller.ai"
CALLERAI_API_KEY="your-callerai-api-key"

# WebUI Path
WEBUI_PATH="/var/lib/smartbotic/microbit/webui"

Generating a secure JWT secret:

# Generate a random 64-character secret
openssl rand -hex 32

Database Configuration

Edit /etc/smartbotic/database.json if needed:

{
  "log_level": "info",
  "grpc": {
    "bind_address": "127.0.0.1",
    "port": 9004
  },
  "storage": {
    "path": "/var/lib/smartbotic/database/storage",
    "migrations_path": "/var/lib/smartbotic/database/migrations"
  }
}

Important:

• Keep bind_address as 127.0.0.1 for security (database should not be exposed)
• Ensure storage paths exist and are writable by smartbotic user

Microbit Service Configuration

Edit /etc/smartbotic/microbit.json:

{
  "log_level": "info",
  "database": {
    "rpc_address": "${DB_ADDRESS:localhost:9004}",
    "timeout_ms": 5000,
    "max_retries": 3
  },
  "http": {
    "bind_address": "127.0.0.1",
    "port": 8090,
    "static_files": {
      "enabled": true,
      "path": "${WEBUI_PATH:/var/lib/smartbotic/microbit/webui}"
    }
  },
  "auth": {
    "enabled": true,
    "jwt_secret": "${JWT_SECRET:change-me-in-production}",
    "access_token_lifetime_sec": 900,
    "refresh_token_lifetime_sec": 604800,
    "password_policy": {
      "min_length": 8,
      "require_number": true,
      "require_special": false
    }
  },
  "smtp": {
    "host": "${SMTP_HOST:}",
    "port": "${SMTP_PORT:587}",
    "username": "${SMTP_USER:}",
    "password": "${SMTP_PASSWORD:}",
    "from_address": "${SMTP_FROM:noreply@smartbotics.ai}",
    "from_name": "Smartbotic",
    "use_tls": true
  },
  "callerai": {
    "api_url": "${CALLERAI_API_URL:http://localhost:8080}",
    "api_key": "${CALLERAI_API_KEY:}",
    "timeout_sec": 30
  },
  "cors": {
    "enabled": true,
    "origins": ["*"],
    "methods": ["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"],
    "headers": ["Authorization", "Content-Type", "X-Requested-With"]
  },
  "rate_limit": {
    "enabled": true,
    "requests_per_minute": 200
  }
}

Production Settings:

• Set http.bind_address to 127.0.0.1 (use Nginx for external access)
• Adjust rate_limit.requests_per_minute based on expected traffic
• Configure cors.origins to specific domains for security

Service Management

Systemd Services

Two systemd services are installed:

1. smartbotic-database.service: Database service
2. smartbotic-microbit.service: Main application service

Enable Services

# Enable services to start on boot
sudo systemctl enable smartbotic-database
sudo systemctl enable smartbotic-microbit

Start Services

# Start database first
sudo systemctl start smartbotic-database

# Wait a moment, then start microbit
sleep 2
sudo systemctl start smartbotic-microbit

Or start both:

sudo systemctl start smartbotic-database smartbotic-microbit

Check Status

# Check service status
sudo systemctl status smartbotic-database
sudo systemctl status smartbotic-microbit

# View recent logs
sudo journalctl -u smartbotic-database -n 50 --no-pager
sudo journalctl -u smartbotic-microbit -n 50 --no-pager

# Follow logs in real-time
sudo journalctl -u smartbotic-microbit -f

Stop Services

sudo systemctl stop smartbotic-microbit
sudo systemctl stop smartbotic-database

Restart Services

# After configuration changes
sudo systemctl restart smartbotic-microbit

# Or restart both
sudo systemctl restart smartbotic-database smartbotic-microbit

Service Configuration Files

Service files are located at:

• /etc/systemd/system/smartbotic-database.service
• /etc/systemd/system/smartbotic-microbit.service

Example microbit service file:

[Unit]
Description=Smartbotic Microbit Service
After=network.target smartbotic-database.service
Requires=smartbotic-database.service

[Service]
Type=simple
User=smartbotic
Group=smartbotic
WorkingDirectory=/var/lib/smartbotic/microbit
EnvironmentFile=/etc/smartbotic/smartbotic.env
ExecStart=/opt/smartbotic/bin/smartbotic-microbit --config /etc/smartbotic/microbit.json
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Nginx Setup

Automatic Setup (Recommended)

Use the provided setup script:

# Install Nginx if not already installed
sudo apt install nginx certbot python3-certbot-nginx

# Run the setup script
sudo /opt/smartbotic/bin/smartbotic-setup-nginx \
  --domain smartbotic.example.com \
  --email admin@example.com

This script:

• Creates Nginx configuration
• Obtains Let's Encrypt SSL certificate
• Enables and reloads Nginx

Manual Nginx Configuration

Create /etc/nginx/sites-available/smartbotic:

# Redirect HTTP to HTTPS
server {
    listen 80;
    listen [::]:80;
    server_name smartbotic.example.com;

    location /.well-known/acme-challenge/ {
        root /var/www/html;
    }

    location / {
        return 301 https://$server_name$request_uri;
    }
}

# HTTPS server
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name smartbotic.example.com;

    # SSL certificates (use certbot to obtain)
    ssl_certificate /etc/letsencrypt/live/smartbotic.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/smartbotic.example.com/privkey.pem;

    # SSL configuration
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;

    # Logging
    access_log /var/log/nginx/smartbotic_access.log;
    error_log /var/log/nginx/smartbotic_error.log;

    # Proxy settings
    client_max_body_size 10M;

    # API endpoints
    location /api/ {
        proxy_pass http://127.0.0.1:8090;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }

    # Static files (WebUI)
    location / {
        proxy_pass http://127.0.0.1:8090;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Enable the site:

sudo ln -s /etc/nginx/sites-available/smartbotic /etc/nginx/sites-enabled/
sudo nginx -t  # Test configuration
sudo systemctl reload nginx

SSL Certificate with Let's Encrypt

# Install certbot
sudo apt install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d smartbotic.example.com --email admin@example.com --agree-tos

# Certbot will automatically configure Nginx for HTTPS
# Certificates auto-renew via cron

Security Considerations

Firewall Configuration

# Allow SSH, HTTP, and HTTPS
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https

# Enable firewall
sudo ufw enable

# Verify rules
sudo ufw status

Do NOT expose these ports:

• 9004 (database gRPC)
• 8090 (microbit HTTP - use Nginx proxy instead)

File Permissions

Ensure proper permissions:

# Configuration files (readable by service user)
sudo chown root:smartbotic /etc/smartbotic/*.json
sudo chmod 640 /etc/smartbotic/*.json
sudo chmod 640 /etc/smartbotic/smartbotic.env

# Data directories (writable by service user)
sudo chown -R smartbotic:smartbotic /var/lib/smartbotic
sudo chmod 750 /var/lib/smartbotic

# Log directories
sudo chown -R smartbotic:smartbotic /var/log/smartbotic
sudo chmod 750 /var/log/smartbotic

Secret Management

Important secrets to protect:

• JWT_SECRET: Used for token signing
• SMTP_PASSWORD: Email server password
• CALLERAI_API_KEY: CallerAI API key

Best practices:

• Use strong random values
• Never commit secrets to version control
• Rotate secrets periodically
• Use environment variables or secret management tools

Database Security

• Database listens only on localhost (127.0.0.1)
• No external access to database port
• Regular backups with encryption
• Access controlled by systemd user permissions

Monitoring and Logs

Log Locations

# Application logs
/var/log/smartbotic/database.log
/var/log/smartbotic/microbit.log

# Systemd journal
sudo journalctl -u smartbotic-database
sudo journalctl -u smartbotic-microbit

# Nginx logs
/var/log/nginx/smartbotic_access.log
/var/log/nginx/smartbotic_error.log

Log Rotation

Configure log rotation in /etc/logrotate.d/smartbotic:

/var/log/smartbotic/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 smartbotic smartbotic
    sharedscripts
    postrotate
        systemctl reload smartbotic-microbit > /dev/null 2>&1 || true
    endscript
}

Health Checks

Check service health:

# Check if services are running
sudo systemctl is-active smartbotic-database
sudo systemctl is-active smartbotic-microbit

# Test API endpoint
curl -f http://localhost:8090/api/v1/settings || echo "Service unhealthy"

# Check database connection
sudo journalctl -u smartbotic-microbit | grep "Connected to database"

Monitoring Tools (Optional)

Consider integrating:

• Prometheus: Metrics collection (future feature)
• Grafana: Metrics visualization
• Uptime Kuma: Uptime monitoring
• Fail2ban: Intrusion prevention

Backup and Recovery

Backup Strategy

What to backup:

1. Database storage: /var/lib/smartbotic/database/storage
2. Configuration: /etc/smartbotic/
3. Environment file: /etc/smartbotic/smartbotic.env

Backup script example:

#!/bin/bash
BACKUP_DIR="/backup/smartbotic"
DATE=$(date +%Y%m%d-%H%M%S)

# Stop services for consistent backup
sudo systemctl stop smartbotic-microbit
sudo systemctl stop smartbotic-database

# Create backup
sudo mkdir -p "$BACKUP_DIR"
sudo tar -czf "$BACKUP_DIR/smartbotic-$DATE.tar.gz" \
    /var/lib/smartbotic/database/storage \
    /etc/smartbotic/

# Restart services
sudo systemctl start smartbotic-database
sudo systemctl start smartbotic-microbit

# Clean old backups (keep last 30 days)
find "$BACKUP_DIR" -name "smartbotic-*.tar.gz" -mtime +30 -delete

Automated backups with cron:

# Add to crontab: daily backup at 2 AM
0 2 * * * /usr/local/bin/smartbotic-backup.sh

Recovery

To restore from backup:

# Stop services
sudo systemctl stop smartbotic-microbit smartbotic-database

# Restore files
sudo tar -xzf /backup/smartbotic/smartbotic-YYYYMMDD-HHMMSS.tar.gz -C /

# Fix permissions
sudo chown -R smartbotic:smartbotic /var/lib/smartbotic

# Start services
sudo systemctl start smartbotic-database smartbotic-microbit

Troubleshooting

Service Won't Start

Check logs:

sudo journalctl -u smartbotic-microbit -n 100 --no-pager

Common issues:

1. Database not running:

   sudo systemctl start smartbotic-database
   

2. Configuration error:

   # Validate JSON config
   jq . /etc/smartbotic/microbit.json
   

3. Port already in use:

   sudo netstat -tuln | grep 8090
   # Change port in config or stop conflicting service
   

4. Permission errors:

   sudo chown -R smartbotic:smartbotic /var/lib/smartbotic
   

Database Connection Failed

Check database service:

sudo systemctl status smartbotic-database
sudo journalctl -u smartbotic-database -n 50

Verify port is listening:

sudo netstat -tuln | grep 9004

WebUI Not Loading

Check static files:

ls -la /var/lib/smartbotic/microbit/webui/

Verify Nginx configuration:

sudo nginx -t
sudo systemctl status nginx

Email Not Sending

Test SMTP configuration:

# Use API endpoint
curl -X POST http://localhost:8090/api/v1/settings/test-smtp \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"to_email":"test@example.com"}'

Check logs for SMTP errors:

sudo journalctl -u smartbotic-microbit | grep -i smtp

Upgrading

Upgrade Process

1. Backup current installation:

   sudo /usr/local/bin/smartbotic-backup.sh
   

2. Download new packages:

   wget https://releases.smartbotics.ai/smartbotic-microbit-NEW_VERSION.tar.gz
   

3. Stop services:

   sudo systemctl stop smartbotic-microbit smartbotic-database
   

4. Install new packages:

   sudo ./packaging/scripts/install.sh smartbotic-*.tar.gz
   

5. Review configuration changes:

   # Check for new .new config files
   ls -la /etc/smartbotic/*.new
   
   # Merge changes if needed
   diff /etc/smartbotic/microbit.json /etc/smartbotic/microbit.json.new
   

6. Restart services:

   sudo systemctl start smartbotic-database smartbotic-microbit
   

7. Verify upgrade:

   sudo journalctl -u smartbotic-microbit -n 50
   curl -f http://localhost:8090/api/v1/settings
   

Rollback

If upgrade fails:

# Stop services
sudo systemctl stop smartbotic-microbit smartbotic-database

# Restore from backup
sudo tar -xzf /backup/smartbotic/smartbotic-YYYYMMDD.tar.gz -C /

# Start services
sudo systemctl start smartbotic-database smartbotic-microbit

Performance Tuning

System Limits

Increase file descriptor limits for the service user:

# Edit /etc/security/limits.conf
smartbotic soft nofile 65536
smartbotic hard nofile 65536

Database Tuning

For high-traffic deployments, consider:

• Dedicated database server
• SSD storage for database
• Increased memory allocation

Nginx Tuning

For high concurrency:

worker_processes auto;
worker_connections 2048;

Getting Help

• Documentation: Check other docs in this directory
• Logs: Always check logs first
• Support: support@smartbotics.ai
• GitHub Issues: Report bugs on the issue tracker

Additional Resources

• Architecture Documentation
• Development Guide
• API Reference
• Nginx Documentation
• Systemd Service Documentation