# Roadmap — Chat Switchboard **See also:** - [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design, store layer, scope model - [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers, manifests, browser tool bridge, surfaces/modes, model roles) **Versioning (pre-1.0):** `0..` — hotfixes use quad: `0.x.y.z` No compatibility guarantees before 1.0. --- ## Dependency Graph Features have real dependencies. This ordering respects them. ``` v0.9.x Stability + Quick UX Wins │ v0.9.3 Content Visibility & Block Controls │ v0.9.4 API Key Encryption + Vault │ v0.10.0 Model Roles (utility + embedding) + Usage Tracking │ v0.10.2 Summarize & Continue (first utility role consumer) │ v0.10.3 Frontend Refactor (no features, code health) │ v0.10.4 Model Type Pipeline + Role Fixes │ v0.10.5 UI Primitives + Extension Surfaces │ ┌───────┴──────────────┐ │ │ v0.11.0 Extension v0.12.0 File Handling Foundation + Vision (browser tier) │ │ │ ├── Custom renderers ├── Doc gen outputs ├── Image gen tool ├── KB documents ├── STT/TTS │ │ │ ├───────────────────────┤ │ │ v0.13.0 Web Search + Tools Expansion │ v0.14.0 Knowledge Bases (embedding role + file storage + pgvector) │ v0.15.0 Compaction (utility role + background job) │ v0.16.0 @mention Routing + Multi-model │ v0.17.0 Extension Surfaces + Modes │ v0.18.0 Smart Routing (policy on model roles) │ v0.19.0 Multi-Participant Channels + Presence │ v0.20.0 Auth Strategy (mTLS/OIDC) + Full RBAC │ ┌───────┴──────────────┐ │ │ v0.21.0 Workflow v0.22.0 Tasks / Engine Autonomous Agents (team-owned, (service channels, staged processes, scheduler, unattended human + AI collab) execution) ``` --- ## ✅ v0.9.0 — Schema Consolidation + BYOK The "rewrite" release: 21 migrations collapsed to 1, store layer abstraction, and the model pipeline made real for end users. **Schema & Architecture** - [x] Consolidated schema (21 migrations → single 001_initial.sql) - [x] Store layer abstraction (all DB via typed interfaces, no raw SQL in handlers) - [x] Persona-as-trust-boundary model (replaces old presets concept) - [x] Scope model (global/team/personal) for configs, personas, models - [x] Backward-compatible API routes (v0.8 → v0.9 field aliases) **Capabilities System** - [x] Two-tier resolution: catalog (provider API, per-provider) → heuristic inference - [x] Venice provider: full capability mapping incl. `optimizedForCode` - [x] `ResolveIntrinsic` with gap-fill merging (catalog authoritative, heuristic fills gaps) - [x] Preset capability inheritance via `GetByModelIDAny` fallback **BYOK (Bring Your Own Key)** - [x] Auto-fetch models on provider creation - [x] User-facing refresh endpoint + "Refresh Models" button - [x] Personal models in selector with `scope=personal` badge - [x] Composite model IDs (`configId:modelId`) prevent cross-provider collisions **Bug Fixes (0.9.0.x)** - [x] API key storage (`json:"-"` tag silently dropped keys) - [x] NULL model_default scan crash (sql.NullString) - [x] Nil slice → JSON null (make([]T, 0)) - [x] Frontend error swallowing **Testing** - [x] Journey-based integration tests (API-only, 4-actor × 3-provider matrix) - [x] Live Venice API integration test - [x] Frontend JS test suite (107 tests, 27 suites) --- ## ✅ v0.9.1 — Capability Architecture + Docs - [x] Removed static known model table (same model ≠ same capabilities across providers) - [x] Resolution chain: catalog → heuristic only, no hardcoded table - [x] Frontend KNOWN_MODELS removed — backend is sole source of truth - [x] Heuristic patterns expanded (qwen3, gpt-5, grok, kimi, minimax, glm-5, gemma-3) - [x] All providers updated to use `InferCapabilities()` directly - [x] EXTENSIONS.md recovered and updated with Appendix A (Custom Renderers) and Appendix B (Model Roles) - [x] ROADMAP.md, CHANGELOG.md, README.md comprehensive rewrite --- ## ✅ v0.9.2 — Quick UX Wins + Hardening Polish that doesn't need new infrastructure. Ship fast, stabilize. **UX** - [x] Collapsible code blocks (long outputs get `
` wrapper) - [x] HTML preview for generated content (sandboxed iframe toggle) - [x] Token count estimate in input area (before send) - [x] "Conversation is getting long" warning (context % thresholds) - [x] New switchboard panel favicon (animated SVG + PNG + ICO) **Hardening** - [x] Proxy interception detection (Content-Type checks, actionable error messages) - [x] Environment injection to frontend (`window.__ENV__`) - [x] Enhanced debug diagnostics (NET:PROXY log type, Content-Type capture, env info) - [x] Team admin audit scoping (see only their team's audit entries) --- ## ✅ v0.9.3 — Content Visibility & Block Controls User control over what's expanded/collapsed in the message flow. Thinking blocks, tool calls, and code blocks all get consistent collapse controls instead of being hidden or auto-formatted without user agency. **Code Blocks** - [x] Collapse/expand toggle button on code block header (alongside Copy/Preview) - [x] Auto-collapse at threshold (>15 lines), user can always toggle - [x] Smooth max-height transition instead of `
` wrapping **Thinking Blocks** - [x] Always render thinking blocks (never hide from DOM) - [x] `showThinking` setting controls default open/closed (default: false), not visibility - [x] User can always click to expand regardless of setting - [x] `reasoning_content` delta field support for models that stream thinking separately (Grok, etc.) - [x] Reasoning accumulated during streaming, wrapped in `` tags before persistence **Tool Calls** - [x] Persist tool call metadata in stored messages (`tool_calls` JSONB column) - [x] Render tool calls on history load (not just during streaming) - [x] Collapsed by default: "🔧 tool_name → done" with toggle for full I/O - [x] Notes tool: "📝 View note" link that navigates to Notes panel - [x] Tool result rendering fix (operator precedence bug showing "undefined results") **Notes Panel → Side Panel** - [x] Copy button on note cards (same pattern as code block copy) - [x] Unified side panel replaces inline HTML preview and notes modal - [x] Tabbed interface: Preview and Notes tabs - [x] Desktop: slides in from right (480px), chat area shrinks via flexbox - [x] Mobile: full-width overlay at z-index 200 - [x] Escape key integration (generation → command palette → side panel → modals) **Streaming Refactor** - [x] Extracted `streamWithToolLoop()` into `stream_loop.go` as single canonical streaming implementation - [x] Both `streamCompletion` (normal chat) and `Regenerate` call shared function — only persistence differs - [x] Regenerate handler now has full feature parity: tools, reasoning, tool_calls persistence - [x] Fixed nil `json.RawMessage` causing PostgreSQL JSONB rejection on regen persistence **Regenerate Fix** - [x] Regen streams in-place (truncates display to parent before streaming) - [x] No more "collapse to original branch" after completion **Favicon** - [x] Animated SVG favicon during generation (cascading opacity pulse on 4 dots) **Admin Scope** - [x] Admin models endpoint returns only global-scope models (not user BYOK or team models) - [x] Bulk visibility toggle scoped to global providers only --- ## ✅ v0.9.4 — API Key Encryption + Vault Two-tier encryption with a trust boundary between organizational keys (admin- accessible) and personal BYOK keys (user-only). Personal keys are protected by a per-user encryption key (UEK) that the platform admin cannot recover without the user's passphrase. ### Threat Model | Threat | Global/Team keys | Personal BYOK keys | |--------|-----------------|---------------------| | DB backup theft | ✅ Protected (env-var AES) | ✅ Protected (UEK AES) | | SQL injection exfil | ✅ Protected | ✅ Protected | | Admin reads DB directly | ⚠️ Admin can decrypt (owns env var) | ✅ Admin cannot decrypt (lacks user passphrase) | | Admin modifies server code | ⚠️ Out of scope (runtime access) | ⚠️ Out of scope (runtime access) | ### Crypto Design **Two encryption paths, one mechanism (AES-256-GCM):** 1. **Global/Team keys** — encrypted with a key derived from `ENCRYPTION_KEY` env var. Admin-owned, admin-recoverable. Simple. 2. **Personal keys** — encrypted with a per-user **User Encryption Key (UEK)**. The UEK is a random 256-bit key generated at account creation, itself encrypted with a key derived from the user's password via Argon2id. ``` Password → Argon2id(salt) → Password-Derived Key (PDK) PDK → AES-256-GCM-Decrypt(encrypted_uek) → UEK (plaintext, session-only) UEK → AES-256-GCM-Decrypt(api_key_enc) → API key (plaintext, per-request) ``` The UEK lives in server memory for the duration of the session (alongside the JWT). It is never written to disk, logs, or the database in plaintext. ### Schema Changes (migration) ```sql -- Users: vault support ALTER TABLE users ADD COLUMN encrypted_uek BYTEA; ALTER TABLE users ADD COLUMN uek_salt BYTEA; -- Argon2id salt ALTER TABLE users ADD COLUMN uek_nonce BYTEA; -- GCM nonce for UEK wrap ALTER TABLE users ADD COLUMN vault_set BOOLEAN NOT NULL DEFAULT false; -- API configs: encrypted key storage ALTER TABLE api_configs ADD COLUMN api_key_enc BYTEA; ALTER TABLE api_configs ADD COLUMN key_nonce BYTEA; ALTER TABLE api_configs ADD COLUMN key_scope TEXT NOT NULL DEFAULT 'global'; -- key_scope: 'global' | 'team' | 'personal' -- global/team: decrypt with env-var-derived key -- personal: decrypt with user's UEK -- Team providers: same pattern ALTER TABLE team_providers ADD COLUMN api_key_enc BYTEA; ALTER TABLE team_providers ADD COLUMN key_nonce BYTEA; ``` Migration backfills: encrypt existing plaintext keys using the env var key (all existing keys are global/team scope). Drop plaintext `api_key` column after backfill. If `ENCRYPTION_KEY` is not set, migration refuses to run (fail-safe). ### Auth Mode Integration **Builtin auth** — password serves double duty: authentication + UEK derivation. No additional UX. On login, derive PDK → unwrap UEK → cache in session. User never knows the vault exists. **mTLS / OIDC** (v0.20.0) — authentication is external (cert/token). Vault passphrase is conditionally required: ``` External auth → auto-authenticated ├── Personal providers DISABLED → straight to app, no prompt └── Personal providers ENABLED ├── First visit → "Set a vault passphrase to protect your API keys" └── Returning → "Unlock your vault" (passphrase field, pre-filled username) ``` Terminology: **"vault passphrase"** for mTLS/OIDC users (distinct from their non-existent account password). Builtin auth users never see this term. ### Backend Implementation **`server/crypto/vault.go`** — pure encryption/decryption, no DB awareness: - `DeriveKey(password, salt) → key` (Argon2id, 64MB memory, 3 iterations) - `GenerateUEK() → (uek, error)` (crypto/rand, 32 bytes) - `WrapUEK(uek, pdk) → (ciphertext, nonce, error)` - `UnwrapUEK(ciphertext, nonce, pdk) → (uek, error)` - `Encrypt(plaintext, key) → (ciphertext, nonce, error)` - `Decrypt(ciphertext, nonce, key) → (plaintext, error)` - `DeriveEnvKey(envVar) → key` (HKDF-SHA256 with domain separation context) **Session UEK caching** — UEK stored in a sync.Map keyed by user ID, evicted on logout / token expiry. Completion handler retrieves UEK from cache to decrypt personal provider keys at request time. **Provider resolution** — `key_scope` determines decryption path: ```go func (s *Store) DecryptAPIKey(config ApiConfig, uekCache *sync.Map) (string, error) { switch config.KeyScope { case "global", "team": return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, envDerivedKey) case "personal": uek, ok := uekCache.Load(config.UserID) if !ok { return "", ErrVaultLocked } return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, uek.([]byte)) } } ``` ### Edge Cases | Scenario | Behavior | |----------|----------| | **User changes password** (builtin) | Re-derive PDK, re-encrypt UEK with new PDK. Personal API keys unchanged (keyed to UEK, not password). | | **Admin resets user password** | Old UEK unrecoverable. Personal keys destroyed. Admin UI warns before confirming. | | **mTLS user forgets passphrase** | Self-service reset: destroys encrypted personal keys. UX: "This will erase your stored API keys." | | **Admin disables personal providers** | Keys stay encrypted in DB (inert). Re-enable later → keys accessible if user remembers passphrase. | | **`ENCRYPTION_KEY` not set** | Startup refuses to run if encrypted keys exist in DB. Fresh install: keys stored plaintext with deprecation warning in logs + admin panel. | | **`ENCRYPTION_KEY` rotated** | Admin CLI tool: `switchboard vault rekey` — re-encrypts all global/team keys with new env key. Personal keys unaffected (keyed to UEK). | ### Frontend Changes - Provider management (Settings → Providers): no change to UX for builtin auth - mTLS/OIDC splash: conditional vault passphrase prompt (new component) - Admin → Users → Reset Password: warning dialog about personal key destruction - Admin → Settings: indicator for `ENCRYPTION_KEY` status (set / not set) ### Checklist **Crypto core** - [x] `server/crypto/vault.go` — Argon2id derivation, AES-256-GCM wrap/unwrap - [x] `server/crypto/vault_test.go` — round-trip, wrong-password, corruption (11 tests) - [x] Env-var key derivation + validation on startup (HKDF-SHA256 with domain separation) - [x] `server/crypto/cache.go` — UEK cache with copy-on-read/write, memory zeroing - [x] `server/crypto/resolver.go` — KeyResolver routes decrypt/encrypt by key_scope - [x] `server/crypto/backfill.go` — startup migration for plaintext → encrypted keys **Schema + migration** - [x] Migration 003_vault.sql: encrypted columns, key_scope backfill - [x] `ENCRYPTION_KEY` enforcement (refuse to start if keys exist but no env var) - [x] Unencrypted fallback when `ENCRYPTION_KEY` not set (raw bytes, graceful degradation) **Auth integration** - [x] UEK generation on user creation (builtin auth) - [x] UEK unwrap on login, cache in session - [x] UEK eviction on logout (memory zeroing) - [ ] UEK re-wrap on password change _(deferred — Settings handler)_ - [ ] UEK destruction on admin password reset with confirmation _(deferred)_ - [ ] Vault passphrase prompt for mTLS/OIDC _(deferred — v0.20.0 dependency)_ **Provider key lifecycle** - [x] Encrypt on provider create/update (all 6 write paths: admin global, personal, team) - [x] Decrypt on completion request (all 4 read paths: completion, admin fetch, personal fetch, team fetch) - [x] `ErrVaultLocked` handling (prompt user to unlock) **Admin tooling** - [ ] `switchboard vault rekey` CLI command _(deferred)_ - [ ] Admin UI: encryption status indicator _(deferred)_ - [x] Admin UI: password reset warning (v0.10.0 — vault destruction dialog) **CI/CD** - [x] `ENCRYPTION_KEY` Gitea secret → k8s `switchboard-encryption` secret sync - [x] Backend k8s manifest: optional `secretKeyRef` for `ENCRYPTION_KEY` - [x] Pipeline version bump to v0.9.4 **Security fix (bonus)** - [x] DOMPurify: strict `ALLOWED_TAGS` allowlist (was permissive `ADD_TAGS` default) - [x] Think-block placeholder `\n\n` padding (prevents fence fusion with ``) - [x] Unclosed code fence auto-close before `marked.parse` (streaming protection) --- ## ✅ v0.10.0 — Model Roles + Usage Tracking + Vault Debt Three tracks of infrastructure that everything downstream depends on. **Vault Debt (Security Fixes)** - [x] UEK re-wrap on user password change (prevents permanent key loss) - [x] UEK destruction on admin password reset (vault + personal BYOK keys deleted) - [x] Admin UI: password reset confirmation with vault destruction warning - [x] Audit logging for vault destruction events **Model Roles** (see [EXTENSIONS.md Appendix B](EXTENSIONS.md#appendix-b-model-roles)) - [x] `model_roles` in global_settings: named slots (utility, embedding, generation) - [x] Each role: primary provider+model, optional fallback provider+model - [x] Team-level role overrides (team.settings JSONB) - [x] `Provider.Embed()` interface — OpenAI/Venice/OpenRouter implement, Anthropic returns `ErrNotSupported` - [x] `roles.Resolver` package: `Complete()` + `Embed()` with automatic fallback - [x] Admin API: `GET/PUT /admin/roles/:role`, `POST /admin/roles/:role/test` - [x] Team API: `GET/PUT/DELETE /teams/:id/roles/:role` - [x] Admin UI: role configuration tab (select provider + model per role, test fire) **Usage Tracking** - [x] Streaming token capture fix — OpenAI: `stream_options.include_usage` + parse usage chunk; Anthropic: parse `message_start`/`message_delta` usage events - [x] Cache token fields across all types (`CompletionResponse`, `StreamEvent`, `streamResult`) - [x] `usage_log` table: channel_id, user_id, model_id, provider_config_id, role, token counts (prompt, completion, cache_creation, cache_read), cost, timestamp - [x] `model_pricing` table: provider+model, input/output/cache per-M, source (catalog/manual) - [x] Cost calculated at insert time (baked, not query-time) - [x] BYOK exclusion: admin views filter `provider_scope != 'personal'`, BYOK gets separate endpoint - [x] Model pricing from catalog sync (won't overwrite manual overrides) - [x] Admin API: `GET /admin/usage`, `GET /admin/usage/users/:id`, `GET /admin/usage/teams/:id` - [x] Admin API: `GET/PUT/DELETE /admin/pricing` - [x] User API: `GET /usage` (personal usage summary) - [x] Admin UI: usage dashboard (period selector, group-by, totals cards, breakdown table, pricing table) - [x] Team admin UI: usage dashboard for team-owned providers (`GET /teams/:teamId/usage`) - [x] Token accumulation across multi-tool-call iterations in `stream_loop.go` **Schema (migration 004)** - [x] `usage_log` with indexes on user, created_at, provider, model, scope - [x] `model_pricing` with unique constraint on (provider_config_id, model_id) - [x] `model_roles` seed in global_settings --- ## ✅ v0.10.1 — Polish + UX Spit and polish release: admin system prompt injection, team management modal extraction, persona tab, side panel enhancements, code download, and critical OpenAI streaming usage race fix. - [x] Admin system prompt (global, injected before user/preset prompts) - [x] Personas tab (moved from Models, policy-gated) - [x] Team Management modal (extracted from Settings, tabbed, multi-team picker) - [x] Preview pane: clear button, fullscreen toggle, resizable - [x] Code block: download button (infers extension from language tag) - [x] Personal Usage scoped to BYOK only - [x] OpenAI streaming usage race fix (deferred Done event until usage chunk) - [x] Usage logging zero-token guard removal --- ## v0.10.2 — Summarize & Continue + Role Cleanup First consumer of the utility model role. User-triggered conversation compaction — not automatic (that's v0.15.0). Depends on: utility model role (v0.10.0). **Summarize & Continue** - [x] "Summarize and continue" button in context warning bar (≥75% context) - [x] `POST /channels/:id/summarize` — calls utility role with conversation history - [x] Summary inserted as tree node with `metadata.type = "summary"` + boundary metadata - [x] `loadConversation` recognizes summary boundaries: replaces pre-summary messages with summary system message - [x] Multiple summaries stack (re-summarize after context fills again) - [x] Fork-aware: summary is a tree node, branches have independent boundaries - [x] Original messages preserved: "show full history" toggle reveals collapsed messages - [x] Usage logged against utility role with cost tracking **BYOK Role Overrides** - [x] Personal role resolution chain: personal → team → global - [x] User Settings → Model Roles tab (BYOK providers only) - [x] Stored in `user.settings` JSONB (`model_roles` key, same shape as team overrides) - [x] Only utility and embedding roles (generation removed) **Generation Role Removal** - [x] `RoleGeneration` removed from `ValidRoles` (was "generation" for image/media) - [x] Removed from admin Roles tab UI - [x] Image gen will be extension-managed (v0.11.0), not a role slot **Utility Rate Limiting** - [x] `utility_rate_limit` global setting (default: 20 calls/hour/user, 0 = unlimited) - [x] Check against `usage_log` before executing org-funded utility calls - [x] BYOK calls exempt (user's own key, user's own cost) - [x] 429 response with clear message when exceeded --- ## v0.10.3 — Frontend Refactor ✅ Zero features. Split the two monolith frontend files (app.js 2,940 lines, ui.js 2,582 lines) into 13 domain-scoped files averaging ~544 lines each. Vanilla JS, no modules, no build step, no function renaming. See [REFACTOR-0.10.3.md](REFACTOR-0.10.3.md) for full plan. - [x] Extract `ui-format.js` — markdown, code blocks, `esc()`, helpers - [x] Extract `ui-settings.js` — settings tabs, teams, providers, user prefs - [x] Extract `ui-admin.js` — admin modal tabs, all admin rendering - [x] Extract `tokens.js` — context tracking, token estimation - [x] Extract `notes.js` — notes panel, editor, multi-select - [x] Extract `chat.js` — chat ops, send, regen, edit, branch, summarize - [x] Extract `settings-handlers.js` — settings save, provider CRUD, cmd palette - [x] Extract `admin-handlers.js` — admin actions, presets, team management - [x] Slim `app.js` to orchestrator — state, init, boot, auth, listener dispatch - [x] Update `sw.js` SHELL_FILES, `index.html` script tags - [x] All 159+ frontend tests pass, `node --check` on all files --- ## v0.10.4 — Model Type Pipeline + Role Fixes ✅ Provider APIs report model types (chat, embedding, image) — now captured end-to-end through sync → catalog → resolver → frontend. Role dropdowns filter models by type instead of showing everything. - [x] Migration `005_model_type.sql` — `model_catalog.model_type` column - [x] `providers.Model.Type` — captured from provider wire response - [x] Venice: reads `type` field, normalizes (`text`→`chat`, `embedding`→`embedding`) - [x] OpenAI: wire struct extended with optional `type` for compatible APIs - [x] `CatalogSyncEntry.ModelType` → `CatalogEntry.ModelType` → `UserModel.ModelType` - [x] Frontend `model_type` mapped through `fetchModels()` - [x] Admin role dropdowns: filter by `model_type` matching role - [x] User role dropdowns: filter by `model_type` matching role - [x] `adminSaveRole()` reloads UI after save (was showing "Saved" without refresh) --- ## v0.10.5 — UI Primitives + Extension Surfaces ✅ Consolidate duplicated UI patterns into shared primitives with extension-ready interfaces. Pre-extension infrastructure: the primitives we build now become the `ctx.*` extension API surface in v0.11.0 with no rewrite needed. - [x] `Providers` registry — single source of truth for types, labels, endpoints - [x] `Roles` registry — names, type filters, hints - [x] `renderCapBadges()` — consolidates 3 badge builders (compact + detailed modes) - [x] `renderProviderForm()` — replaces 3 form implementations, dual-mode create/edit - [x] `renderProviderList()` — replaces 3 list renderers, event delegation - [x] `renderRoleConfig()` — replaces 2 role UIs + 4 handlers - [x] `renderUsageDashboard()` — replaces 3 usage renderers (compact + full) - [x] Admin provider form gets endpoint auto-fill (was missing) - [x] Team provider form consolidated from 2 separate forms to 1 dual-mode - [x] Removed ~360 lines of duplicated code + ~40 lines of static HTML - [x] `showConfirm()` — styled confirm dialog replacing all 16 native `confirm()` calls - [x] `.popup-menu` shared CSS + `createPopupMenu()` primitive for consistent menus - [x] Sidebar collapse icon always visible (dimmed), brightens on hover --- ## v0.11.0 — Extension Foundation (Browser Tier) The platform play. See [EXTENSIONS.md](EXTENSIONS.md) for full spec. **Core Loader** (EXTENSIONS.md §4, §7) - [ ] `extensions.js` — loader, registry, scoped context (~200 lines) - [ ] `Extensions.register()` API with permission enforcement - [ ] Manifest format parser and validator - [ ] `extensions` + `extension_user_settings` tables - [ ] Admin endpoints: install, uninstall, enable/disable - [ ] Asset serving: `/api/v1/extensions/:id/assets/*path` - [ ] Extension settings UI in Settings modal **Browser Tool Bridge** (EXTENSIONS.md §5) - [ ] `ctx.tools.handle()` API for browser tool registration - [ ] Tool schema collection: merge built-in + extension tools - [ ] `tool.call.*` / `tool.result.*` WebSocket routing - [ ] 30-second timeout with LLM error recovery - [ ] Tool execution via EventBus `WaitFor` pattern **Custom Renderer API** (EXTENSIONS.md Appendix A) - [ ] `ctx.renderers.register()` — pattern + render function - [ ] Renderer pipeline in message display (after markdown, before DOM insert) - [ ] First-party renderers: collapsible code, HTML preview, mermaid, LaTeX (migrate core v0.9.2 implementations to extension format) **First Extensions** - [ ] Image generation tool (browser or sidecar, uses generation model role) - [ ] STT/TTS (browser extension, Web Speech API — personal use case) --- ## v0.12.0 — File Handling + Vision File input into chat — table stakes for serious use. **Storage Backend** - [ ] S3-compatible API (MinIO, Ceph RGW, AWS — anything with S3 API) - [ ] Local PVC fallback (single-node / dev) - [ ] Admin config: backend selection, endpoint, credentials, size limits - [ ] Reused by KBs (v0.14.0) and compaction snapshots (v0.15.0) **Chat Integration** - [ ] `attachments` table (metadata in PG, blobs in object store) - [ ] Image/file upload in chat messages - [ ] Multimodal message assembly for vision-capable models - [ ] Text extraction on upload (PDF, DOCX, TXT → tsvector) - [ ] Paste-to-upload (clipboard image support) - [ ] Document generation output storage (extension → attachment → download) --- ## v0.13.0 — Web Search + Tools Expansion Built-in tools that use the tool framework shipped in v0.7.x. - [ ] `web_search` tool: search provider abstraction (SearXNG, Brave, DuckDuckGo) - [ ] `url_fetch` tool: retrieve and extract content from URLs - [ ] Sidecar or direct HTTP from backend (configurable) - [ ] Results injected into context - [ ] "Research X and save to my notes" workflow --- ## v0.14.0 — Knowledge Bases Depends on: embedding model role (v0.10.0), file storage (v0.12.0). - [ ] Embedding pipeline: chunking (recursive, semantic), generation via embedding role - [ ] `pgvector` storage and similarity search - [ ] `knowledge_bases` table with team scoping - [ ] KB document storage via v0.12.0 storage backend - [ ] `kb_search` tool (uses existing tool framework) - [ ] Context injection in completion flow - [ ] Notes embedded too (once pipeline exists) - [ ] Per-channel KB toggle --- ## v0.15.0 — Compaction Depends on: utility model role (v0.10.0). - [ ] Auto-compaction service: background job that calls utility model to summarize - [ ] Channel-scoped: triggers when context exceeds threshold - [ ] Summaries stored as system messages - [ ] Configurable: per-channel opt-in/out, summary model override - [ ] Admin controls for resource limits --- ## v0.16.0 — @mention Routing + Multi-model The channel schema already supports multiple models. This adds routing. - [ ] @mention parsing in messages (users and AI models) - [ ] Resolve mentions against `channel_models` - [ ] Multi-model channels: fire completions per mentioned model - [ ] "Add a model" UI per channel - [ ] Enables: second opinions, cross-model conversations --- ## v0.17.0 — Extension Surfaces + Modes Depends on: extension foundation (v0.11.0). See [EXTENSIONS.md §6](EXTENSIONS.md#6-surfaces-modes). - [ ] Surface registration and activation API - [ ] Mode selector in sidebar (below brand, above chat history) - [ ] `ctx.ui.replace()` / `ctx.ui.restore()` for region management - [ ] `surface.activated` / `surface.deactivated` events - [ ] Editor mode (extension: file tree + code editor + AI chat split pane) - [ ] Article mode (extension: outline + rich text + AI assistant panel) --- ## v0.18.0 — Smart Routing Depends on: model roles (v0.10.0), usage tracking (v0.10.0). Rules-based routing engine — policy, not ML. - [ ] Admin routing policies: "cheapest model with required capabilities", "prefer private providers, fallback to cloud", "for team X, use Y" - [ ] Fallback chains: primary provider down → next with same model - [ ] Cost-aware: factor pricing into routing decisions - [ ] Latency-aware: track response times per provider, prefer faster --- ## v0.19.0 — Multi-Participant Channels + Presence The channel foundation for workflows and real-time collaboration. Today channels are single-user direct chats. This release makes channels a shared space that multiple actors can inhabit. Depends on: extension surfaces (v0.17.0), WebSocket infrastructure (already exists). **Channel Participants** - [ ] `channel_participants` table: `channel_id`, `participant_type` (user, session, persona), `participant_id`, `role` (owner, member, observer), `joined_at` - [ ] Channel access queries rewritten: `WHERE user_id = $1` → participant-based lookup - [ ] Backward compatible: existing single-user channels auto-have one participant (owner) - [ ] Channel `type` column: `direct` (current), `group` (multi-user), `workflow` (staged) **Presence + Real-time** - [ ] Typing indicators and presence (who's online, who's in this channel) - [ ] Participant join/leave events via WebSocket - [ ] Co-editing for extension surfaces (editor mode, article mode) - [ ] Cursor sharing, selection highlighting **Channel-Scoped Notes** - [ ] Optional `channel_id` FK on notes (NULL = personal notebook, set = channel-attached) - [ ] Notes created by tool calls within a channel auto-attach to that channel - [ ] Channel notes visible to all participants (not just creator) --- ## v0.20.0 — Auth Strategy (mTLS/OIDC) + Full RBAC Enterprise auth modes on top of the teams foundation. Depends on: vault passphrase support from v0.9.3 (conditional unlock for personal providers under external auth). - [ ] `AUTH_MODE` env var: `builtin` | `mtls` | `oidc` - [ ] All three resolve to the same internal user model - [ ] `auth_source` + `external_id` columns on users table - [ ] mTLS: header trust, auto-provision from cert DN - [ ] OIDC: Keycloak/Okta token validation, claim extraction, role mapping - [ ] Per-source auto-activate policy: auto_activate, default_team, default_role - [ ] Vault passphrase prompt for mTLS/OIDC users (v0.9.3 conditional unlock) - [ ] Fine-grained permissions: model access, KB write, task create, token budgets **Anonymous / Session Participants** - [ ] Session-based identity for unauthenticated visitors (workflow intake) - [ ] mTLS: cert CN/fingerprint as stable session identity (no `users` row required) - [ ] Non-mTLS: ephemeral session token (cookie or URL token) - [ ] Session participants can send messages and trigger tool calls within workflow channels - [ ] Session identity visible to assigned team members (cert DN, display name, or "Anonymous #N") --- ## v0.21.0 — Workflow Engine Team-owned, stage-based process execution. The channel is the runtime, personas drive each stage, and the existing tool/notes infrastructure handles structured data collection. See [ARCHITECTURE.md — Workflow Architecture](ARCHITECTURE.md#workflow-architecture-future--v0210) for the conceptual model. Depends on: multi-participant channels (v0.19.0), anonymous identity (v0.20.0). **Workflow Definitions** - [ ] `workflows` table: `team_id` (owner), `name`, `description`, `is_active`, `entry_mode` (public_link, team_internal, api) - [ ] `workflow_stages` table: `workflow_id`, `ordinal`, `persona_id`, `assignment_team_id`, `form_template` (JSONB, structured note schema), `transition_rules` (JSONB) - [ ] Team-admin CRUD: create/edit/delete workflows and stages - [ ] Workflow versioning: edits create new version, active instances continue on their version **Workflow Instances (Channels)** - [ ] Workflow-typed channels: `type='workflow'`, `workflow_id`, `current_stage`, `stage_version` - [ ] Entry point: public link generates workflow channel, adds anonymous participant, binds stage 1 persona - [ ] Stage transitions: triggered by AI (tool call), team member (button), or rule (form complete) - [ ] On transition: swap active persona, notify assignment team, update channel metadata **AI Intake** - [ ] Stage persona drives structured collection via system prompt - [ ] Form template → persona system prompt injection ("collect these fields: ...") - [ ] Tool calls create channel-scoped notes as form responses - [ ] AI determines readiness: "I have everything needed" → trigger transition **Assignment + Queue** - [ ] Assignment queue: unassigned workflow channels visible to team members - [ ] Claim model: team member claims channel (becomes participant with `member` role) - [ ] Round-robin / rule-based auto-assignment (optional, per-stage config) - [ ] Assignment notifications via WebSocket + optional webhook **Team Member Collaboration** - [ ] Assigned member sees full history (AI intake + artifacts + notes) - [ ] Persona remains active — assists both visitor and team member - [ ] Member can trigger stage transitions, add notes, invoke tools - [ ] Member can reassign to different team member or escalate to different team **Frontend** - [ ] Workflow builder UI in team admin panel (stage list, persona picker, form template editor) - [ ] Queue view: sidebar section showing unassigned workflow channels for user's teams - [ ] Channel header: workflow stage indicator, transition controls - [ ] Anonymous visitor view: minimal UI, persona-driven conversation (public link entry) --- ## v0.22.0 — Tasks / Autonomous Agents Unattended execution — workflows without a human in the loop. Depends on: workflow engine (v0.21.0). - [ ] Scheduler + task runner (cron-like triggers for workflow instantiation) - [ ] `task_create` tool (AI can spawn sub-workflows) - [ ] `type: 'service'` channels: workflow instances with no human participants - [ ] Execution budgets: max tokens, max tool calls, max wall-clock per stage - [ ] Admin controls for resource limits and kill switches - [ ] Completion webhooks (notify external systems when workflow completes) --- ## TBD (unscheduled — real features, no immediate need) Items that are real but don't yet have a version assignment. Pull left based on need. **Extension System — Server Tiers** - Starlark runtime integration (Tier 1 — server sandbox) - Sidecar HTTP tool protocol (Tier 2 — container isolation) - Server-side tool execution in completion handler **Desktop + Mobile** - Desktop app (Tauri) - Full PWA with offline capability - Mobile-optimized layouts (beyond current responsive) **Data + Portability** - Bulk export/import (account data, conversations, settings) - ChatGPT/other tool import - GDPR-style "download my data" - Backup/restore CronJob manifests **UX / Multi-Seat** - Per-chat model/preset persistence (server-side). Currently a localStorage bandaid (v0.10.2) that doesn't roam across devices. Proper fix: store `last_selector_id` in `channels.settings` JSONB on each send. On chat selection, resolve from `channel.settings.last_selector_id` → match against available models/presets → fall back to `channel.model` → global default. Eliminates device-local state entirely. Needs `updateChannel` PATCH on completion success (single extra column merge, no migration). **Platform** - Rate limiting per user/team/tier (token budgets) - Provider health monitoring + key rotation - Multi-tenant SaaS mode - Plugin/extension marketplace - Virtual scroll for long conversations - SQLite backend option (single-user / dev)