Changeset 0.5.0 (#35)

2026-02-19 15:03:20 +00:00
parent a93a6b9635
commit 30d0c11219
65 changed files with 5345 additions and 8070 deletions
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -1,482 +1,180 @@
-# 🗺️ Chat Switchboard Roadmap
+# 🗺️ Chat Switchboard — Roadmap

-## Vision
+**See also:**
+- [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design (Notes, KBs, Tasks, Channels, Embeddings)
+- [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers, tools, surfaces)

-**Become the "VSCode of AI Interfaces"** - A minimal, extensible core with infinite capabilities through plugins.
+## Current State: v0.5.4

-## Competitive Position
+### ✅ Done

-| Feature | Chat Switchboard | Open WebUI | ChatGPT | Claude Desktop | LibreChat |
-|---------|-----------------|------------|---------|----------------|-----------|
-| **Multi-Model** | ✅ Auto-routing | ✅ Manual | ❌ | ❌ | ✅ |
-| **Plugin System** | ✅⭐ Dogfooded | 🟡 Limited | ❌ | ❌ | 🟡 |
-| **Offline Mode** | ✅ Full featured | ❌ | ❌ | 🟡 | ❌ |
-| **Workflows** | ✅⭐ Visual DAG | ❌ | ❌ | ❌ | ❌ |
-| **Channels** | ✅ With AI | ✅ | ❌ | ❌ | 🟡 |
-| **RAG/Knowledge** | ✅ pgvector | ✅ | 🟡 | 🟡 | ✅ |
-| **Self-Hosted** | ✅ Easy | ✅ Complex | ❌ | ❌ | ✅ |
-| **Open Source** | ✅ MIT | ✅ MIT | ❌ | ❌ | ✅ MIT |
-| **No Backend Required** | ✅⭐ Optional | ❌ | ❌ | ❌ | ❌ |
-| **Desktop App** | 🟡 Planned | ❌ | ✅ | ✅ | ❌ |
-| **Mobile App** | 🟡 PWA | 🟡 | ✅ | ✅ | 🟡 |
+**Backend (Go)**
+- [x] PostgreSQL schema + auto-migrations (go:embed, startup)
+- [x] JWT auth with refresh token rotation
+- [x] Chat CRUD + message persistence
+- [x] Streaming completion proxy (OpenAI + Anthropic + Venice + OpenRouter)
+- [x] Per-user + global API config management
+- [x] Admin endpoints (users, providers, models, settings, stats)
+- [x] Rate limiting, error middleware, request logging
+- [x] Health check endpoint (includes schema_version)
+- [x] EventBus with WebSocket hub
+- [x] Provider capability system (known models, heuristic detection, resolution chain)
+- [x] Dynamic max_tokens resolution (no more hardcoded 4096)
+- [x] User + admin provider model listing with capabilities

-**Unique Advantages:**
-1. **Workflows** - No competitor has visual AI orchestration
-2. **Plugin-First** - Core features ARE plugins (proves extensibility)
-3. **Dual-Mode** - Works offline or managed (privacy + collaboration)
-4. **Smart Routing** - Automatic model selection for cost/quality
-
-## Development Phases
-
-### ✅ Phase 0: Foundation (Current)
-**Status:** Complete  
-**Timeline:** Week 1-2
-
- [x] Frontend core (vanilla JS)
- [x] LocalStorage state management
- [x] OpenAI-compatible API client
- [x] Streaming responses
- [x] Chat history
+**Frontend (Vanilla JS)**
+- [x] Collapsible sidebar with time-grouped chat history
+- [x] User menu flyout (Settings, Admin, Debug, Sign Out)
+- [x] Model selector with capability badges (output, context, tools, vision, thinking)
+- [x] Client-side known model table with backend overlay
+- [x] Streaming SSE display with smart scroll
+- [x] Full markdown rendering (marked.js + DOMPurify)
+- [x] Thinking block display (`<think>`/`<thinking>` tags)
+- [x] Settings modal (profile, chat prefs, provider management)
+- [x] Admin modal (users, providers, models with cap badges, settings, stats)
+- [x] Debug modal (console intercept, network log, state inspector)
+- [x] EventBus client with exponential backoff + max retries
 - [x] Export (Markdown, JSON, Text)
- [x] Model switching
- [x] Standalone build
+- [x] Vendor libs with CDN fallback (SCIF-safe)
+- [x] 3-stage Dockerfile (Go build → vendor download → nginx)

-### 🚧 Phase 1: Backend Core
-**Status:** In Progress  
-**Timeline:** Week 3-6 (4 weeks)
+**CI/CD (Gitea Actions)**
+- [x] Three-env pipeline (dev/test/prod)
+- [x] Dev: upgrade test → schema validate → wipe → fresh install test
+- [x] Backend auto-migrates on startup (no CI migration step)
+- [x] Shared PG safety (_dev suffix guard on wipe)
+- [x] Post-deploy schema verification via /api/v1/health

-**Goals:**
- Full Go backend with PostgreSQL
- User authentication (JWT)
- WebSocket server for real-time
- Extension manager (load/manage plugins)
- Mode switching (unmanaged ↔ managed)
+**Architecture Design (v0.5.4)**
+- [x] ARCHITECTURE.md: core services spec (13 sections)
+- [x] EXTENSIONS.md: three-tier extension system spec
+- [x] Terminology: Roles, Teams, Channels, Message Tree, etc.
+- [x] "Everything is a channel" — chat is a channel with type:'direct'
+- [x] Conversation forking (message tree via parent_id)
+- [x] Auth strategy (builtin/mTLS/OIDC)
+- [x] Classification banners (CSS custom props)

-**Tasks:**
- [ ] Implement auth handlers (register, login, JWT)
- [ ] PostgreSQL models and migrations
- [ ] WebSocket hub for real-time messaging
- [ ] Extension discovery and lifecycle management
- [ ] API endpoints for chats, settings, users
- [ ] Frontend mode detection and switching
+### Implementation Phasing

-**Deliverables:**
- Users can create accounts
- Backend stores chat history
- WebSocket connections work
- At least 1 extension loads and runs
+**See ARCHITECTURE.md §10 for the authoritative implementation sequence.**
+Phases 1–8, from channel foundation through RBAC.

 ---

-### 🎯 Phase 2: Core Features as Plugins
-**Status:** Not Started  
-**Timeline:** Week 7-10 (4 weeks)
+## Phase 1: Real-Time Foundation

-**Goal:** Prove plugin architecture by implementing core features as extensions
+### 1.1 WebSocket Hub
+**Priority:** HIGH — blocks Channels, typing indicators, live updates

-**2.1 Chat Engine Plugin (Python)**
-```
-extensions/backend/chat-engine/
-├── extension.json
-├── main.py
-├── requirements.txt
-└── models/
-    ├── openai.py
-    ├── anthropic.py
-    └── ollama.py
-```
+- [ ] WebSocket upgrade endpoint (`/ws`)
+- [ ] Connection manager (register, unregister, broadcast)
+- [ ] Room-based pub/sub (per-chat, per-channel, global)
+- [ ] JWT auth on WebSocket handshake
+- [ ] Heartbeat / ping-pong keepalive
+- [ ] Reconnection handling (frontend)
+- [ ] Message types: `chat.message`, `chat.typing`, `user.presence`, `system.notify`

-**Tasks:**
- [ ] Multi-provider adapter pattern
- [ ] Streaming response handling
- [ ] Token counting and cost tracking
- [ ] Message history management
- [ ] System prompt injection
+### 1.2 Live Chat Updates
+- [ ] New messages pushed via WebSocket (not just SSE streaming)
+- [ ] Typing indicators
+- [ ] Chat list updates when other sessions create/delete chats
+- [ ] Online user presence

-**2.2 Channels Plugin (Go)**
-```
-extensions/backend/channels/
-├── extension.json
-├── main.go
-├── handlers/
-│   ├── channels.go
-│   └── messages.go
-└── websocket/
-    └── broadcaster.go
-```
+---

-**Tasks:**
- [ ] Channel CRUD operations
- [ ] Membership management
- [ ] Real-time message broadcasting
- [ ] @mention parsing (users + AI models)
- [ ] Threading support
+## Phase 2: Channels

-**2.3 Notes Plugin (Go)**
-```
-extensions/backend/notes/
-├── extension.json
-├── main.go
-└── handlers/
-    └── notes.go
-```
+### 2.1 Backend
+- [ ] Channel model (name, description, type: public/private/DM, creator)
+- [ ] Membership model (user_id, channel_id, role: owner/member)
+- [ ] Channel messages (extends message model with channel_id)
+- [ ] CRUD endpoints: `/api/v1/channels`, `/api/v1/channels/:id/messages`
+- [ ] @mention parsing — users and AI models
+- [ ] AI response triggered by @model-name mentions
+- [ ] WebSocket broadcast per channel room

-**Tasks:**
- [ ] Note CRUD
+### 2.2 Frontend
+- [ ] Channels section in sidebar (below chats, collapsible)
+- [ ] Channel creation modal (name, description, public/private)
+- [ ] Member management (invite, remove, role change)
+- [ ] Message display with user avatars and @mention highlighting
+- [ ] AI responses inline with user messages
+- [ ] Unread count badges
+
+---
+
+## Phase 3: Plugin System
+
+### 3.1 Architecture (see discussion below)
+- [ ] Plugin manifest format (`plugin.json`)
+- [ ] Plugin lifecycle (install, enable, disable, uninstall)
+- [ ] Plugin storage (per-plugin isolated DB namespace or key-value)
+- [ ] Hook system (pre-completion, post-completion, on-message, on-channel-message)
+- [ ] Admin UI for plugin management (install, configure, enable/disable)
+- [ ] Plugin API (what plugins can access: messages, user context, settings)
+
+### 3.2 Built-in Plugin: Web Search
+- [ ] Tool-use integration in completion flow
+- [ ] Search provider abstraction (DuckDuckGo, SearXNG, Brave)
+- [ ] Results injected into context
+
+### 3.3 Built-in Plugin: Token Counter
+- [ ] Per-message token estimation
+- [ ] Per-chat cost tracking
+- [ ] Provider-specific pricing data
+
+---
+
+## Phase 4: Notes & Knowledge Base
+
+### 4.1 Notes
+- [ ] Note model (title, content, folder, tags)
+- [ ] CRUD endpoints
 - [ ] Folder organization
- [ ] Tagging system
- [ ] Markdown rendering
- [ ] Full-text search
+- [ ] Full-text search (PostgreSQL `tsvector`)
+- [ ] Markdown editor in frontend
+- [ ] Link notes to chats

-**Deliverables:**
- Chat works via chat-engine extension
- Channels support user + AI conversations
- Notes can be created and organized
- All features disabled if extension not loaded
-
---
-
-### 🚀 Phase 3: RAG & Knowledge Bases
-**Status:** Not Started  
-**Timeline:** Week 11-13 (3 weeks)
-
-**Goal:** Advanced knowledge management with vector search
-
-**3.1 RAG Engine Plugin (Python)**
-```
-extensions/backend/rag-engine/
-├── extension.json
-├── main.py
-├── embeddings.py
-├── chunker.py
-└── requirements.txt
-    # Dependencies: langchain, pgvector, tiktoken
-```
-
-**Features:**
+### 4.2 RAG (Retrieval Augmented Generation)
 - [ ] Document upload (PDF, DOCX, TXT, MD)
- [ ] Smart chunking (recursive, semantic)
+- [ ] Chunking strategies (recursive, semantic)
 - [ ] Embedding generation (OpenAI, local models)
- [ ] Vector storage (pgvector)
- [ ] Semantic search
- [ ] Context injection in chat
-
-**3.2 Knowledge Base UI**
- [ ] KB management interface
- [ ] Drag-and-drop upload
- [ ] Document viewer
- [ ] Search results preview
- [ ] Chat integration (auto-search when enabled)
-
-**Deliverables:**
- Upload documents to KB
- Semantic search works
- Chat can reference KB automatically
- Cost tracking for embeddings
+- [ ] pgvector storage and similarity search
+- [ ] Context injection in completion flow
+- [ ] Per-KB toggle in chat settings

 ---

-### ⭐ Phase 4: Workflows (KILLER FEATURE)
-**Status:** Not Started  
-**Timeline:** Week 14-18 (5 weeks)
+## Phase 5: Polish

-**Goal:** Visual workflow builder for multi-step AI operations
-
-**4.1 Workflow Engine (Go + Python)**
-```
-extensions/backend/workflows/
-├── extension.json
-├── executor.go          # Go: DAG execution
-├── registry.py          # Python: Tool registry
-└── nodes/
-    ├── llm_node.go
-    ├── tool_node.go
-    ├── conditional_node.go
-    └── loop_node.go
-```
-
-**Features:**
- [ ] DAG execution engine
- [ ] Node type system (LLM, tool, logic, I/O)
- [ ] Variable resolution and templating
- [ ] Conditional branching
- [ ] Loops and iteration
- [ ] Error handling and retries
- [ ] Cost tracking per workflow
- [ ] Execution history and logs
-
-**4.2 Visual Workflow Builder (Frontend)**
- [ ] Drag-and-drop canvas
- [ ] Node palette (search and add)
- [ ] Connection drawing
- [ ] Node configuration panels
- [ ] Live execution preview
- [ ] Debug mode (step-through)
-
-**4.3 Workflow Templates**
- [ ] Research Assistant
- [ ] Code Review Pipeline
- [ ] Multi-Model Consensus
- [ ] Content Creation Factory
- [ ] Data Processing Pipeline
-
-**Deliverables:**
- Users can create workflows visually
- Workflows execute correctly
- Template marketplace basics
- At least 5 working templates
+- [ ] Chat search / filter in sidebar
+- [ ] Message editing
+- [ ] Chat folders / pinning
+- [ ] Bulk chat operations
+- [ ] Keyboard shortcuts (Ctrl+K command palette)
+- [ ] PWA manifest + offline shell
+- [ ] Performance: lazy load chat history, virtual scroll for long chats

 ---

-### 🔧 Phase 5: Developer Experience
-**Status:** Not Started  
-**Timeline:** Week 19-21 (3 weeks)
+## Future

-**Goal:** Make extension development effortless
-
-**5.1 Extension Templates**
- [ ] `_template-python` with FastAPI boilerplate
- [ ] `_template-go` with Gin boilerplate
- [ ] `_template-node` with Express boilerplate
- [ ] CLI tool: `chatswitch create-extension`
-
-**5.2 Extension Marketplace**
-```
-marketplace/
-├── backend/
-│   ├── api.go           # Publish, search, install
-│   └── storage/         # Extension packages
-└── frontend/
-    └── marketplace.html # Browse and install UI
-```
-
-**Features:**
- [ ] Publish extensions (tarball upload)
- [ ] Search and browse
- [ ] One-click install
- [ ] Ratings and reviews
- [ ] Version management
- [ ] Dependency resolution
-
-**5.3 Documentation & Examples**
- [ ] Complete API documentation
- [ ] 10+ example extensions
- [ ] Video tutorials
- [ ] Plugin development guide
- [ ] Contributing guidelines
-
-**Deliverables:**
- Easy extension scaffolding
- Working marketplace
- Comprehensive docs
+- Desktop app (Tauri)
+- Smart model routing (cost/quality/latency auto-selection)
+- Workflow builder (visual DAG for chaining models + tools)
+- Plugin marketplace
+- SSO/SAML
+- Audit logging
+- Multi-tenant SaaS mode

 ---

-### 💎 Phase 6: Polish & Launch Prep
-**Status:** Not Started  
-**Timeline:** Week 22-24 (3 weeks)
+## Plugin / Extension Architecture

-**6.1 Desktop App (Tauri)**
-```
-desktop/
-├── src-tauri/           # Rust backend
-│   ├── main.rs
-│   └── Cargo.toml
-└── src/                 # Frontend (reuse web)
-    └── index.html
-```
-
-**Features:**
- [ ] Native app wrapper
- [ ] System tray integration
- [ ] Keyboard shortcuts
- [ ] Native notifications
- [ ] Auto-updates
-
-**6.2 Mobile (PWA)**
- [ ] Responsive design
- [ ] Touch gestures
- [ ] Offline support
- [ ] Install prompt
-
-**6.3 Performance**
- [ ] Frontend bundle optimization
- [ ] Backend query optimization
- [ ] WebSocket connection pooling
- [ ] Redis caching
- [ ] CDN for static assets
-
-**6.4 Security Audit**
- [ ] SQL injection prevention
- [ ] XSS protection
- [ ] CSRF tokens
- [ ] Rate limiting
- [ ] API key encryption audit
- [ ] Extension sandboxing review
-
-**Deliverables:**
- Desktop app for Windows, Mac, Linux
- Mobile-friendly PWA
- Performance benchmarks
- Security audit report
-
---
-
-### 🎉 Phase 7: Launch
-**Timeline:** Week 25
-
-**7.1 Marketing**
- [ ] Landing page
- [ ] Demo video
- [ ] Blog post
- [ ] HackerNews/Reddit launch
- [ ] ProductHunt submission
-
-**7.2 Community**
- [ ] Discord server
- [ ] GitHub Discussions
- [ ] Documentation site
- [ ] Example workflows gallery
-
-**7.3 Infrastructure**
- [ ] Hosted demo instance
- [ ] CI/CD pipeline
- [ ] Monitoring (Prometheus, Grafana)
- [ ] Error tracking (Sentry)
-
-**Launch Checklist:**
- [ ] All core features working
- [ ] 10+ bundled extensions
- [ ] Desktop app released
- [ ] Documentation complete
- [ ] Demo video published
- [ ] Community channels live
-
---
-
-## Post-Launch Roadmap
-
-### v1.1 - Enterprise Features
-**Timeline:** Month 2-3
-
- [ ] SSO/SAML authentication
- [ ] Team management
- [ ] Role-based access control (RBAC)
- [ ] Audit logs
- [ ] Usage analytics dashboard
- [ ] Billing integration (Stripe)
-
-### v1.2 - Advanced Workflows
-**Timeline:** Month 4-5
-
- [ ] Workflow version control (Git-like)
- [ ] Collaborative editing
- [ ] A/B testing workflows
- [ ] Scheduled executions (cron)
- [ ] Webhook triggers
- [ ] API-first workflow execution
-
-### v1.3 - AI Enhancements
-**Timeline:** Month 6-7
-
- [ ] Fine-tuned model hosting
- [ ] Custom embedding models
- [ ] Multi-modal support (images, audio)
- [ ] Voice chat
- [ ] Real-time translation
-
-### v2.0 - Platform Evolution
-**Timeline:** Month 8-12
-
- [ ] Plugin marketplace monetization
- [ ] White-label solution
- [ ] Multi-tenant SaaS mode
- [ ] Advanced analytics (cost optimization, usage insights)
- [ ] Mobile native apps (React Native)
-
---
-
-## Success Metrics
-
-### User Metrics
- **Week 1:** 100 GitHub stars
- **Month 1:** 1,000 active users
- **Month 3:** 5,000 active users, 50 community extensions
- **Month 6:** 20,000 active users, 200 community extensions
-
-### Technical Metrics
- **Plugin Load Time:** < 2s
- **API Response Time:** < 100ms (p95)
- **WebSocket Latency:** < 50ms
- **Uptime:** 99.9%
-
-### Business Metrics (If Applicable)
- **Free Tier:** Unlimited local use
- **Hosted Basic:** $10/user/month
- **Hosted Pro:** $25/user/month (workflows, advanced features)
- **Enterprise:** Custom pricing
-
---
-
-## Risk Mitigation
-
-### Technical Risks
-| Risk | Mitigation |
-|------|-----------|
-| Extension security vulnerabilities | Sandboxing, code review, security audit |
-| WebSocket scaling issues | Redis pub/sub, horizontal scaling |
-| Database performance | Query optimization, indexing, caching |
-| Third-party API rate limits | Retry logic, fallback chains, caching |
-
-### Product Risks
-| Risk | Mitigation |
-|------|-----------|
-| Competitors copy workflows feature | Speed to market, open-source advantage |
-| Low plugin adoption | High-quality templates, good DX |
-| Users prefer monolithic apps | Prove extensibility with core features |
-
-### Market Risks
-| Risk | Mitigation |
-|------|-----------|
-| AI providers release similar tools | Self-hosted option, multi-provider support |
-| Regulation changes | Privacy-first design, compliance features |
-
---
-
-## How to Contribute
-
-### For Developers
-1. Pick an issue from GitHub
-2. Fork the repo
-3. Create a feature branch
-4. Submit a PR
-
-**High Priority:**
- Implement Phase 1 backend tasks
- Create example extensions
- Write documentation
-
-### For Designers
- UI/UX improvements
- Workflow builder mockups
- Landing page design
-
-### For Writers
- Blog posts
- Tutorials
- Documentation improvements
-
-### For Users
- Bug reports
- Feature requests
- Extension ideas
-
---
-
-## Questions?
-
- **GitHub Issues:** Bug reports, feature requests
- **GitHub Discussions:** General questions, ideas
- **Discord:** Real-time chat (coming soon)
-
-**Let's build the future of AI interfaces together! 🚀**
+**Moved to dedicated documents:**
+- [EXTENSIONS.md](EXTENSIONS.md) — Full extension system spec with three tiers
+  (Browser JS, Starlark sandbox, Sidecar containers), manifest format,
+  browser tool bridge, surface/mode system, and implementation roadmap.
+- [ARCHITECTURE.md](ARCHITECTURE.md) — Core backend services that extensions
+  build on: Notes, Knowledge Bases, Tasks, Channels, Embeddings, and
+  Folders/Projects. Includes data models, sequencing, and admin controls.