# 🔄 Workflow System - The Killer Feature

## What Are Workflows?

Workflows let you **chain multiple AI models, tools, and data sources** into automated pipelines. Think Zapier/n8n but for LLMs.

### Why This Matters

**Current State (Competitors):**
```
User → Single Prompt → One Model → Response
```

**Chat Switchboard Workflows:**
```
User Input → [Model A] → [Tool] → [Model B] → [Conditional Logic] → Final Output
             ↓ logs    ↓ saves    ↓ verifies   ↓ branches
```

## Real-World Examples

### 1. Research Assistant
**Problem:** Researching a topic requires multiple steps and models

```
┌─────────────┐
│ User Query  │
└──────┬──────┘
       │
       ↓
┌─────────────────────┐
│ 1. Web Search Tool  │ ← DuckDuckGo API
└──────┬──────────────┘
       │ (top 5 results)
       ↓
┌─────────────────────────┐
│ 2. GPT-4: Extract Facts │ ← Fast, cheap summarization
└──────┬──────────────────┘
       │ (structured data)
       ↓
┌──────────────────────────┐
│ 3. Claude: Verify Claims │ ← Accurate fact-checking
└──────┬───────────────────┘
       │ (verified facts)
       ↓
┌──────────────────────────┐
│ 4. Gemini: Write Report  │ ← Long-form generation
└──────┬───────────────────┘
       │ (final report)
       ↓
┌──────────────────────┐
│ 5. Save to KB Tool   │ ← Store for future reference
└──────────────────────┘
```

**Time Saved:** 30 minutes → 2 minutes  
**Cost Optimization:** Uses cheap models where possible

### 2. Code Review Pipeline
```
[GitHub PR] → [Fetch Code Tool] → [GPT-4: Find Bugs] → [Claude: Security Check] 
                                        ↓                        ↓
                                   [Save Issues]          [Create Report]
                                                              ↓
                                                    [Post to Channel Tool]
```

### 3. Multi-Model Consensus
**Problem:** Get the "best" answer by combining multiple AI perspectives

```
                    ┌─[GPT-4]─┐
                    │         │
[User Question] ────┼─[Claude]─┼──→ [Voting Logic] ──→ [Best Answer]
                    │         │
                    └─[Gemini]┘
```

**Use Case:** Medical advice, legal analysis, important decisions

### 4. Content Creation Factory
```
[Topic] → [GPT-4: Outline] → [Claude: Draft] → [Grammar Tool] → [SEO Tool] → [Publish]
              ↓                   ↓                ↓                ↓
         [Save Draft]        [Version 1]      [Version 2]     [Final]
```

### 5. Data Processing Pipeline
```
[CSV Upload] → [Python Tool: Parse] → [GPT-3.5: Categorize] → [Save to PostgreSQL]
                                              ↓
                                    [Generate Report Tool]
```

## Workflow Builder UI

### Visual Editor (Drag & Drop)

```
+------------------+     +------------------+     +------------------+
|   [Web Search]   |---->|   [GPT-4 Node]   |---->|   [Save Note]    |
|                  |     |                  |     |                  |
| Query: {input}   |     | Prompt: Summary  |     | Title: Research  |
+------------------+     +------------------+     +------------------+
        ↓                        ↓                        ↓
   [5 results]           [Markdown text]           [Note ID]
```

### Node Types

#### 1. Input Nodes
- **User Input** - Text, file, form
- **Trigger** - Schedule, webhook, event
- **Variable** - Constants, config

#### 2. Model Nodes
- **LLM Call** - Any configured model
- **Model Router** - Auto-select best model
- **Multi-Model** - Run parallel, compare

#### 3. Tool Nodes
- **Web Search**
- **Code Execution**
- **API Call**
- **Database Query**
- **File Operations**
- **Custom Extension Tools**

#### 4. Logic Nodes
- **Conditional** - If/else branching
- **Loop** - Iterate over data
- **Merge** - Combine results
- **Filter** - Data transformation

#### 5. Output Nodes
- **Response** - Return to user
- **Save** - Store in DB/KB
- **Webhook** - External notification
- **Channel Message** - Post to channel

## Workflow Definition (JSON)

```json
{
  "id": "research-assistant",
  "name": "Research Assistant",
  "version": "1.0.0",
  "description": "Search, summarize, verify, and save research",
  "nodes": [
    {
      "id": "input",
      "type": "input",
      "config": {
        "schema": {
          "query": {"type": "string", "required": true}
        }
      }
    },
    {
      "id": "search",
      "type": "tool",
      "tool": "web_search",
      "config": {
        "query": "{{input.query}}",
        "max_results": 5
      }
    },
    {
      "id": "summarize",
      "type": "llm",
      "model": "gpt-4o-mini",
      "config": {
        "prompt": "Summarize these search results:\n{{search.results}}",
        "max_tokens": 500
      }
    },
    {
      "id": "verify",
      "type": "llm",
      "model": "claude-3.5-sonnet",
      "config": {
        "prompt": "Verify the accuracy of:\n{{summarize.content}}",
        "max_tokens": 300
      }
    },
    {
      "id": "save",
      "type": "tool",
      "tool": "save_to_kb",
      "config": {
        "kb_id": "research",
        "title": "{{input.query}}",
        "content": "{{verify.content}}"
      }
    },
    {
      "id": "output",
      "type": "output",
      "config": {
        "response": "{{verify.content}}"
      }
    }
  ],
  "edges": [
    {"from": "input", "to": "search"},
    {"from": "search", "to": "summarize"},
    {"from": "summarize", "to": "verify"},
    {"from": "verify", "to": "save"},
    {"from": "save", "to": "output"}
  ]
}
```

## Execution Engine

### Go Workflow Runner

```go
// server/workflows/executor.go
package workflows

type Executor struct {
    workflow *Workflow
    context  map[string]interface{}
    llmClient *LLMClient
    toolRegistry *ToolRegistry
}

func (e *Executor) Run(input map[string]interface{}) (*Result, error) {
    e.context["input"] = input
    
    // Topological sort to execute DAG in order
    sorted := e.workflow.TopologicalSort()
    
    for _, node := range sorted {
        result, err := e.executeNode(node)
        if err != nil {
            return nil, err
        }
        e.context[node.ID] = result
    }
    
    return e.context["output"], nil
}

func (e *Executor) executeNode(node *Node) (interface{}, error) {
    switch node.Type {
    case "llm":
        return e.executeLLM(node)
    case "tool":
        return e.executeTool(node)
    case "conditional":
        return e.executeConditional(node)
    // ... other types
    }
}

func (e *Executor) executeLLM(node *Node) (interface{}, error) {
    // Resolve template variables
    prompt := e.resolveTemplate(node.Config["prompt"])
    model := node.Config["model"]
    
    // Call LLM
    response, err := e.llmClient.Chat(model, prompt)
    return response, err
}
```

### Variable Resolution

```go
// Template: "Summarize: {{search.results}}"
// Context: {"search": {"results": "..."}}
// Result: "Summarize: ..."

func (e *Executor) resolveTemplate(template string) string {
    re := regexp.MustCompile(`\{\{([^}]+)\}\}`)
    return re.ReplaceAllStringFunc(template, func(match string) string {
        path := match[2:len(match)-2] // Remove {{}}
        return e.getFromContext(path)
    })
}
```

## Database Schema (Workflows)

```sql
-- Workflow definitions
CREATE TABLE workflows (
    id UUID PRIMARY KEY,
    user_id UUID REFERENCES users(id),
    name VARCHAR(200),
    graph JSONB NOT NULL, -- Node/edge structure
    input_schema JSONB,
    output_schema JSONB,
    is_public BOOLEAN DEFAULT false,
    tags TEXT[]
);

-- Workflow executions (audit trail)
CREATE TABLE workflow_executions (
    id UUID PRIMARY KEY,
    workflow_id UUID REFERENCES workflows(id),
    input_data JSONB,
    output_data JSONB,
    status VARCHAR(20), -- running, completed, failed
    steps JSONB, -- [{node_id, input, output, duration}]
    total_cost_usd NUMERIC(10, 6),
    total_time_ms INTEGER,
    started_at TIMESTAMP,
    completed_at TIMESTAMP
);
```

## Workflow Marketplace

### Shareable Templates

```bash
# Export workflow
POST /api/workflows/{id}/export
→ workflow.json file

# Import workflow
POST /api/workflows/import
← Upload workflow.json

# Publish to marketplace
POST /api/marketplace/publish
{
  "workflow_id": "uuid",
  "category": "research",
  "price": 0, // Free or paid
  "license": "MIT"
}
```

### Categories
- 📊 Data Analysis
- 🔍 Research
- 📝 Content Creation
- 💻 Code Review
- 🎓 Education
- 💼 Business Intelligence
- 🔐 Security Audits

## Cost Optimization

### Smart Model Routing in Workflows

```javascript
// Workflow node: Auto-select cheapest capable model
{
  "id": "summarize",
  "type": "llm_auto",
  "config": {
    "task": "summarization",
    "max_cost": 0.01, // $0.01 max
    "min_quality": 0.8, // 80% quality threshold
    "prompt": "..."
  }
}

// Engine selects:
// GPT-4o-mini ($0.0001/token) ✅
// Not GPT-4 ($0.03/token) - too expensive
```

### Execution Cost Tracking

```sql
-- Track workflow costs
SELECT 
    w.name,
    COUNT(we.id) as executions,
    AVG(we.total_cost_usd) as avg_cost,
    SUM(we.total_cost_usd) as total_cost
FROM workflows w
JOIN workflow_executions we ON w.id = we.workflow_id
GROUP BY w.id
ORDER BY total_cost DESC;
```

## Advanced Features

### 1. Conditional Branching

```json
{
  "id": "quality_check",
  "type": "conditional",
  "config": {
    "condition": "{{verify.confidence}} > 0.8",
    "true_branch": "save",
    "false_branch": "human_review"
  }
}
```

### 2. Loops

```json
{
  "id": "process_batch",
  "type": "loop",
  "config": {
    "items": "{{input.files}}",
    "body": "extract_data_node",
    "merge": "combine_results_node"
  }
}
```

### 3. Parallel Execution

```json
{
  "id": "parallel_models",
  "type": "parallel",
  "config": {
    "branches": ["gpt4_node", "claude_node", "gemini_node"],
    "merge_strategy": "voting" // or "first", "all", "custom"
  }
}
```

### 4. Error Handling

```json
{
  "id": "api_call",
  "type": "tool",
  "config": {
    "tool": "external_api",
    "retry": 3,
    "timeout": 5000,
    "on_error": "fallback_node"
  }
}
```

## Security & Limits

### Sandboxing
- Code execution in isolated containers
- API rate limiting per workflow
- Cost caps per execution

### Permissions
```sql
-- Workflow ACL
CREATE TABLE workflow_permissions (
    workflow_id UUID REFERENCES workflows(id),
    user_id UUID REFERENCES users(id),
    permission VARCHAR(20), -- read, execute, edit, admin
    UNIQUE(workflow_id, user_id)
);
```

## API Endpoints

```
POST   /api/workflows                 # Create workflow
GET    /api/workflows                 # List user's workflows
GET    /api/workflows/{id}            # Get workflow
PUT    /api/workflows/{id}            # Update workflow
DELETE /api/workflows/{id}            # Delete workflow

POST   /api/workflows/{id}/execute    # Run workflow
GET    /api/workflows/{id}/executions # Execution history
GET    /api/executions/{id}           # Execution details
POST   /api/executions/{id}/stop      # Cancel execution

GET    /api/marketplace/workflows     # Browse public workflows
POST   /api/marketplace/install/{id}  # Install from marketplace
```

## Frontend UI (React/Vue Later, Start Simple)

### Workflow Editor Component

```javascript
// src/js/workflow-editor.js
class WorkflowEditor {
    constructor(canvas) {
        this.canvas = canvas;
        this.nodes = [];
        this.edges = [];
    }
    
    addNode(type, x, y) {
        const node = {
            id: generateId(),
            type: type,
            position: {x, y},
            config: {}
        };
        this.nodes.push(node);
        this.render();
    }
    
    connectNodes(from, to) {
        this.edges.push({from, to});
        this.render();
    }
    
    execute() {
        const workflow = this.toJSON();
        fetch('/api/workflows/execute', {
            method: 'POST',
            body: JSON.stringify(workflow)
        }).then(r => r.json()).then(result => {
            console.log('Workflow result:', result);
        });
    }
}
```

## Comparison with Alternatives

| Feature | Chat Switchboard | LangChain | n8n | Zapier |
|---------|-----------------|-----------|-----|--------|
| Visual Builder | ✅ | ❌ | ✅ | ✅ |
| Multi-Model | ✅ | Limited | ❌ | ❌ |
| Open Source | ✅ | ✅ | ✅ | ❌ |
| Self-Hosted | ✅ | N/A | ✅ | ❌ |
| Cost Tracking | ✅ | ❌ | ❌ | ✅ |
| LLM-First | ✅ | ✅ | ❌ | ❌ |
| No-Code | ✅ | ❌ | ✅ | ✅ |

**Unique Position:** LangChain for non-coders + n8n for LLMs

## Roadmap

### MVP (v1.0)
- ✅ Basic DAG execution
- ✅ LLM nodes (single model)
- ✅ Tool nodes (web search, code exec)
- ✅ Input/output nodes
- ✅ Simple UI (form-based, not visual yet)

### v1.1
- ⬜ Visual drag-drop editor
- ⬜ Conditional logic
- ⬜ Loops
- ⬜ Cost tracking

### v1.2
- ⬜ Template marketplace
- ⬜ Workflow sharing
- ⬜ Scheduled executions
- ⬜ Webhook triggers

### v2.0
- ⬜ Collaborative editing
- ⬜ Version control (Git-like)
- ⬜ A/B testing workflows
- ⬜ Analytics dashboard

---

## Get Started

```bash
# Create your first workflow
curl -X POST https://api.chatswitch.example.com/api/workflows \
  -H "Authorization: Bearer $TOKEN" \
  -d @examples/research-assistant.json

# Execute it
curl -X POST https://api.chatswitch.example.com/api/workflows/{id}/execute \
  -d '{"query": "Latest AI research"}'
```

**This is the feature that sets Chat Switchboard apart.**