Changeset 0.5.0 (#35)
This commit is contained in:
@@ -1,575 +0,0 @@
|
||||
# 🔄 Workflow System - The Killer Feature
|
||||
|
||||
## What Are Workflows?
|
||||
|
||||
Workflows let you **chain multiple AI models, tools, and data sources** into automated pipelines. Think Zapier/n8n but for LLMs.
|
||||
|
||||
### Why This Matters
|
||||
|
||||
**Current State (Competitors):**
|
||||
```
|
||||
User → Single Prompt → One Model → Response
|
||||
```
|
||||
|
||||
**Chat Switchboard Workflows:**
|
||||
```
|
||||
User Input → [Model A] → [Tool] → [Model B] → [Conditional Logic] → Final Output
|
||||
↓ logs ↓ saves ↓ verifies ↓ branches
|
||||
```
|
||||
|
||||
## Real-World Examples
|
||||
|
||||
### 1. Research Assistant
|
||||
**Problem:** Researching a topic requires multiple steps and models
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
│ User Query │
|
||||
└──────┬──────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────────┐
|
||||
│ 1. Web Search Tool │ ← DuckDuckGo API
|
||||
└──────┬──────────────┘
|
||||
│ (top 5 results)
|
||||
↓
|
||||
┌─────────────────────────┐
|
||||
│ 2. GPT-4: Extract Facts │ ← Fast, cheap summarization
|
||||
└──────┬──────────────────┘
|
||||
│ (structured data)
|
||||
↓
|
||||
┌──────────────────────────┐
|
||||
│ 3. Claude: Verify Claims │ ← Accurate fact-checking
|
||||
└──────┬───────────────────┘
|
||||
│ (verified facts)
|
||||
↓
|
||||
┌──────────────────────────┐
|
||||
│ 4. Gemini: Write Report │ ← Long-form generation
|
||||
└──────┬───────────────────┘
|
||||
│ (final report)
|
||||
↓
|
||||
┌──────────────────────┐
|
||||
│ 5. Save to KB Tool │ ← Store for future reference
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
**Time Saved:** 30 minutes → 2 minutes
|
||||
**Cost Optimization:** Uses cheap models where possible
|
||||
|
||||
### 2. Code Review Pipeline
|
||||
```
|
||||
[GitHub PR] → [Fetch Code Tool] → [GPT-4: Find Bugs] → [Claude: Security Check]
|
||||
↓ ↓
|
||||
[Save Issues] [Create Report]
|
||||
↓
|
||||
[Post to Channel Tool]
|
||||
```
|
||||
|
||||
### 3. Multi-Model Consensus
|
||||
**Problem:** Get the "best" answer by combining multiple AI perspectives
|
||||
|
||||
```
|
||||
┌─[GPT-4]─┐
|
||||
│ │
|
||||
[User Question] ────┼─[Claude]─┼──→ [Voting Logic] ──→ [Best Answer]
|
||||
│ │
|
||||
└─[Gemini]┘
|
||||
```
|
||||
|
||||
**Use Case:** Medical advice, legal analysis, important decisions
|
||||
|
||||
### 4. Content Creation Factory
|
||||
```
|
||||
[Topic] → [GPT-4: Outline] → [Claude: Draft] → [Grammar Tool] → [SEO Tool] → [Publish]
|
||||
↓ ↓ ↓ ↓
|
||||
[Save Draft] [Version 1] [Version 2] [Final]
|
||||
```
|
||||
|
||||
### 5. Data Processing Pipeline
|
||||
```
|
||||
[CSV Upload] → [Python Tool: Parse] → [GPT-3.5: Categorize] → [Save to PostgreSQL]
|
||||
↓
|
||||
[Generate Report Tool]
|
||||
```
|
||||
|
||||
## Workflow Builder UI
|
||||
|
||||
### Visual Editor (Drag & Drop)
|
||||
|
||||
```
|
||||
+------------------+ +------------------+ +------------------+
|
||||
| [Web Search] |---->| [GPT-4 Node] |---->| [Save Note] |
|
||||
| | | | | |
|
||||
| Query: {input} | | Prompt: Summary | | Title: Research |
|
||||
+------------------+ +------------------+ +------------------+
|
||||
↓ ↓ ↓
|
||||
[5 results] [Markdown text] [Note ID]
|
||||
```
|
||||
|
||||
### Node Types
|
||||
|
||||
#### 1. Input Nodes
|
||||
- **User Input** - Text, file, form
|
||||
- **Trigger** - Schedule, webhook, event
|
||||
- **Variable** - Constants, config
|
||||
|
||||
#### 2. Model Nodes
|
||||
- **LLM Call** - Any configured model
|
||||
- **Model Router** - Auto-select best model
|
||||
- **Multi-Model** - Run parallel, compare
|
||||
|
||||
#### 3. Tool Nodes
|
||||
- **Web Search**
|
||||
- **Code Execution**
|
||||
- **API Call**
|
||||
- **Database Query**
|
||||
- **File Operations**
|
||||
- **Custom Extension Tools**
|
||||
|
||||
#### 4. Logic Nodes
|
||||
- **Conditional** - If/else branching
|
||||
- **Loop** - Iterate over data
|
||||
- **Merge** - Combine results
|
||||
- **Filter** - Data transformation
|
||||
|
||||
#### 5. Output Nodes
|
||||
- **Response** - Return to user
|
||||
- **Save** - Store in DB/KB
|
||||
- **Webhook** - External notification
|
||||
- **Channel Message** - Post to channel
|
||||
|
||||
## Workflow Definition (JSON)
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "research-assistant",
|
||||
"name": "Research Assistant",
|
||||
"version": "1.0.0",
|
||||
"description": "Search, summarize, verify, and save research",
|
||||
"nodes": [
|
||||
{
|
||||
"id": "input",
|
||||
"type": "input",
|
||||
"config": {
|
||||
"schema": {
|
||||
"query": {"type": "string", "required": true}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "search",
|
||||
"type": "tool",
|
||||
"tool": "web_search",
|
||||
"config": {
|
||||
"query": "{{input.query}}",
|
||||
"max_results": 5
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "summarize",
|
||||
"type": "llm",
|
||||
"model": "gpt-4o-mini",
|
||||
"config": {
|
||||
"prompt": "Summarize these search results:\n{{search.results}}",
|
||||
"max_tokens": 500
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "verify",
|
||||
"type": "llm",
|
||||
"model": "claude-3.5-sonnet",
|
||||
"config": {
|
||||
"prompt": "Verify the accuracy of:\n{{summarize.content}}",
|
||||
"max_tokens": 300
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "save",
|
||||
"type": "tool",
|
||||
"tool": "save_to_kb",
|
||||
"config": {
|
||||
"kb_id": "research",
|
||||
"title": "{{input.query}}",
|
||||
"content": "{{verify.content}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "output",
|
||||
"type": "output",
|
||||
"config": {
|
||||
"response": "{{verify.content}}"
|
||||
}
|
||||
}
|
||||
],
|
||||
"edges": [
|
||||
{"from": "input", "to": "search"},
|
||||
{"from": "search", "to": "summarize"},
|
||||
{"from": "summarize", "to": "verify"},
|
||||
{"from": "verify", "to": "save"},
|
||||
{"from": "save", "to": "output"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Execution Engine
|
||||
|
||||
### Go Workflow Runner
|
||||
|
||||
```go
|
||||
// server/workflows/executor.go
|
||||
package workflows
|
||||
|
||||
type Executor struct {
|
||||
workflow *Workflow
|
||||
context map[string]interface{}
|
||||
llmClient *LLMClient
|
||||
toolRegistry *ToolRegistry
|
||||
}
|
||||
|
||||
func (e *Executor) Run(input map[string]interface{}) (*Result, error) {
|
||||
e.context["input"] = input
|
||||
|
||||
// Topological sort to execute DAG in order
|
||||
sorted := e.workflow.TopologicalSort()
|
||||
|
||||
for _, node := range sorted {
|
||||
result, err := e.executeNode(node)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e.context[node.ID] = result
|
||||
}
|
||||
|
||||
return e.context["output"], nil
|
||||
}
|
||||
|
||||
func (e *Executor) executeNode(node *Node) (interface{}, error) {
|
||||
switch node.Type {
|
||||
case "llm":
|
||||
return e.executeLLM(node)
|
||||
case "tool":
|
||||
return e.executeTool(node)
|
||||
case "conditional":
|
||||
return e.executeConditional(node)
|
||||
// ... other types
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Executor) executeLLM(node *Node) (interface{}, error) {
|
||||
// Resolve template variables
|
||||
prompt := e.resolveTemplate(node.Config["prompt"])
|
||||
model := node.Config["model"]
|
||||
|
||||
// Call LLM
|
||||
response, err := e.llmClient.Chat(model, prompt)
|
||||
return response, err
|
||||
}
|
||||
```
|
||||
|
||||
### Variable Resolution
|
||||
|
||||
```go
|
||||
// Template: "Summarize: {{search.results}}"
|
||||
// Context: {"search": {"results": "..."}}
|
||||
// Result: "Summarize: ..."
|
||||
|
||||
func (e *Executor) resolveTemplate(template string) string {
|
||||
re := regexp.MustCompile(`\{\{([^}]+)\}\}`)
|
||||
return re.ReplaceAllStringFunc(template, func(match string) string {
|
||||
path := match[2:len(match)-2] // Remove {{}}
|
||||
return e.getFromContext(path)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Database Schema (Workflows)
|
||||
|
||||
```sql
|
||||
-- Workflow definitions
|
||||
CREATE TABLE workflows (
|
||||
id UUID PRIMARY KEY,
|
||||
user_id UUID REFERENCES users(id),
|
||||
name VARCHAR(200),
|
||||
graph JSONB NOT NULL, -- Node/edge structure
|
||||
input_schema JSONB,
|
||||
output_schema JSONB,
|
||||
is_public BOOLEAN DEFAULT false,
|
||||
tags TEXT[]
|
||||
);
|
||||
|
||||
-- Workflow executions (audit trail)
|
||||
CREATE TABLE workflow_executions (
|
||||
id UUID PRIMARY KEY,
|
||||
workflow_id UUID REFERENCES workflows(id),
|
||||
input_data JSONB,
|
||||
output_data JSONB,
|
||||
status VARCHAR(20), -- running, completed, failed
|
||||
steps JSONB, -- [{node_id, input, output, duration}]
|
||||
total_cost_usd NUMERIC(10, 6),
|
||||
total_time_ms INTEGER,
|
||||
started_at TIMESTAMP,
|
||||
completed_at TIMESTAMP
|
||||
);
|
||||
```
|
||||
|
||||
## Workflow Marketplace
|
||||
|
||||
### Shareable Templates
|
||||
|
||||
```bash
|
||||
# Export workflow
|
||||
POST /api/workflows/{id}/export
|
||||
→ workflow.json file
|
||||
|
||||
# Import workflow
|
||||
POST /api/workflows/import
|
||||
← Upload workflow.json
|
||||
|
||||
# Publish to marketplace
|
||||
POST /api/marketplace/publish
|
||||
{
|
||||
"workflow_id": "uuid",
|
||||
"category": "research",
|
||||
"price": 0, // Free or paid
|
||||
"license": "MIT"
|
||||
}
|
||||
```
|
||||
|
||||
### Categories
|
||||
- 📊 Data Analysis
|
||||
- 🔍 Research
|
||||
- 📝 Content Creation
|
||||
- 💻 Code Review
|
||||
- 🎓 Education
|
||||
- 💼 Business Intelligence
|
||||
- 🔐 Security Audits
|
||||
|
||||
## Cost Optimization
|
||||
|
||||
### Smart Model Routing in Workflows
|
||||
|
||||
```javascript
|
||||
// Workflow node: Auto-select cheapest capable model
|
||||
{
|
||||
"id": "summarize",
|
||||
"type": "llm_auto",
|
||||
"config": {
|
||||
"task": "summarization",
|
||||
"max_cost": 0.01, // $0.01 max
|
||||
"min_quality": 0.8, // 80% quality threshold
|
||||
"prompt": "..."
|
||||
}
|
||||
}
|
||||
|
||||
// Engine selects:
|
||||
// GPT-4o-mini ($0.0001/token) ✅
|
||||
// Not GPT-4 ($0.03/token) - too expensive
|
||||
```
|
||||
|
||||
### Execution Cost Tracking
|
||||
|
||||
```sql
|
||||
-- Track workflow costs
|
||||
SELECT
|
||||
w.name,
|
||||
COUNT(we.id) as executions,
|
||||
AVG(we.total_cost_usd) as avg_cost,
|
||||
SUM(we.total_cost_usd) as total_cost
|
||||
FROM workflows w
|
||||
JOIN workflow_executions we ON w.id = we.workflow_id
|
||||
GROUP BY w.id
|
||||
ORDER BY total_cost DESC;
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### 1. Conditional Branching
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "quality_check",
|
||||
"type": "conditional",
|
||||
"config": {
|
||||
"condition": "{{verify.confidence}} > 0.8",
|
||||
"true_branch": "save",
|
||||
"false_branch": "human_review"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Loops
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "process_batch",
|
||||
"type": "loop",
|
||||
"config": {
|
||||
"items": "{{input.files}}",
|
||||
"body": "extract_data_node",
|
||||
"merge": "combine_results_node"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Parallel Execution
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "parallel_models",
|
||||
"type": "parallel",
|
||||
"config": {
|
||||
"branches": ["gpt4_node", "claude_node", "gemini_node"],
|
||||
"merge_strategy": "voting" // or "first", "all", "custom"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Error Handling
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "api_call",
|
||||
"type": "tool",
|
||||
"config": {
|
||||
"tool": "external_api",
|
||||
"retry": 3,
|
||||
"timeout": 5000,
|
||||
"on_error": "fallback_node"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Security & Limits
|
||||
|
||||
### Sandboxing
|
||||
- Code execution in isolated containers
|
||||
- API rate limiting per workflow
|
||||
- Cost caps per execution
|
||||
|
||||
### Permissions
|
||||
```sql
|
||||
-- Workflow ACL
|
||||
CREATE TABLE workflow_permissions (
|
||||
workflow_id UUID REFERENCES workflows(id),
|
||||
user_id UUID REFERENCES users(id),
|
||||
permission VARCHAR(20), -- read, execute, edit, admin
|
||||
UNIQUE(workflow_id, user_id)
|
||||
);
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
```
|
||||
POST /api/workflows # Create workflow
|
||||
GET /api/workflows # List user's workflows
|
||||
GET /api/workflows/{id} # Get workflow
|
||||
PUT /api/workflows/{id} # Update workflow
|
||||
DELETE /api/workflows/{id} # Delete workflow
|
||||
|
||||
POST /api/workflows/{id}/execute # Run workflow
|
||||
GET /api/workflows/{id}/executions # Execution history
|
||||
GET /api/executions/{id} # Execution details
|
||||
POST /api/executions/{id}/stop # Cancel execution
|
||||
|
||||
GET /api/marketplace/workflows # Browse public workflows
|
||||
POST /api/marketplace/install/{id} # Install from marketplace
|
||||
```
|
||||
|
||||
## Frontend UI (React/Vue Later, Start Simple)
|
||||
|
||||
### Workflow Editor Component
|
||||
|
||||
```javascript
|
||||
// src/js/workflow-editor.js
|
||||
class WorkflowEditor {
|
||||
constructor(canvas) {
|
||||
this.canvas = canvas;
|
||||
this.nodes = [];
|
||||
this.edges = [];
|
||||
}
|
||||
|
||||
addNode(type, x, y) {
|
||||
const node = {
|
||||
id: generateId(),
|
||||
type: type,
|
||||
position: {x, y},
|
||||
config: {}
|
||||
};
|
||||
this.nodes.push(node);
|
||||
this.render();
|
||||
}
|
||||
|
||||
connectNodes(from, to) {
|
||||
this.edges.push({from, to});
|
||||
this.render();
|
||||
}
|
||||
|
||||
execute() {
|
||||
const workflow = this.toJSON();
|
||||
fetch('/api/workflows/execute', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify(workflow)
|
||||
}).then(r => r.json()).then(result => {
|
||||
console.log('Workflow result:', result);
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Comparison with Alternatives
|
||||
|
||||
| Feature | Chat Switchboard | LangChain | n8n | Zapier |
|
||||
|---------|-----------------|-----------|-----|--------|
|
||||
| Visual Builder | ✅ | ❌ | ✅ | ✅ |
|
||||
| Multi-Model | ✅ | Limited | ❌ | ❌ |
|
||||
| Open Source | ✅ | ✅ | ✅ | ❌ |
|
||||
| Self-Hosted | ✅ | N/A | ✅ | ❌ |
|
||||
| Cost Tracking | ✅ | ❌ | ❌ | ✅ |
|
||||
| LLM-First | ✅ | ✅ | ❌ | ❌ |
|
||||
| No-Code | ✅ | ❌ | ✅ | ✅ |
|
||||
|
||||
**Unique Position:** LangChain for non-coders + n8n for LLMs
|
||||
|
||||
## Roadmap
|
||||
|
||||
### MVP (v1.0)
|
||||
- ✅ Basic DAG execution
|
||||
- ✅ LLM nodes (single model)
|
||||
- ✅ Tool nodes (web search, code exec)
|
||||
- ✅ Input/output nodes
|
||||
- ✅ Simple UI (form-based, not visual yet)
|
||||
|
||||
### v1.1
|
||||
- ⬜ Visual drag-drop editor
|
||||
- ⬜ Conditional logic
|
||||
- ⬜ Loops
|
||||
- ⬜ Cost tracking
|
||||
|
||||
### v1.2
|
||||
- ⬜ Template marketplace
|
||||
- ⬜ Workflow sharing
|
||||
- ⬜ Scheduled executions
|
||||
- ⬜ Webhook triggers
|
||||
|
||||
### v2.0
|
||||
- ⬜ Collaborative editing
|
||||
- ⬜ Version control (Git-like)
|
||||
- ⬜ A/B testing workflows
|
||||
- ⬜ Analytics dashboard
|
||||
|
||||
---
|
||||
|
||||
## Get Started
|
||||
|
||||
```bash
|
||||
# Create your first workflow
|
||||
curl -X POST https://api.chatswitch.example.com/api/workflows \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d @examples/research-assistant.json
|
||||
|
||||
# Execute it
|
||||
curl -X POST https://api.chatswitch.example.com/api/workflows/{id}/execute \
|
||||
-d '{"query": "Latest AI research"}'
|
||||
```
|
||||
|
||||
**This is the feature that sets Chat Switchboard apart.**
|
||||
Reference in New Issue
Block a user