740 lines
32 KiB
Markdown
740 lines
32 KiB
Markdown
# Roadmap — Chat Switchboard
|
||
|
||
**See also:**
|
||
- [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design, store layer, scope model
|
||
- [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers,
|
||
manifests, browser tool bridge, surfaces/modes, model roles)
|
||
|
||
**Versioning (pre-1.0):** `0.<major>.<minor>` — hotfixes use quad: `0.x.y.z`
|
||
No compatibility guarantees before 1.0.
|
||
|
||
---
|
||
|
||
## Dependency Graph
|
||
|
||
Features have real dependencies. This ordering respects them.
|
||
|
||
```
|
||
v0.9.x Stability + Quick UX Wins
|
||
│
|
||
v0.9.3 Content Visibility & Block Controls
|
||
│
|
||
v0.9.4 API Key Encryption + Vault
|
||
│
|
||
v0.10.0 Model Roles (utility + embedding) + Usage Tracking
|
||
│
|
||
v0.10.2 Summarize & Continue (first utility role consumer)
|
||
│
|
||
┌───────┴──────────────┐
|
||
│ │
|
||
v0.11.0 Extension v0.12.0 File Handling
|
||
Foundation + Vision
|
||
(browser tier) │
|
||
│ │
|
||
├── Custom renderers ├── Doc gen outputs
|
||
├── Image gen tool ├── KB documents
|
||
├── STT/TTS │
|
||
│ │
|
||
├───────────────────────┤
|
||
│ │
|
||
v0.13.0 Web Search + Tools Expansion
|
||
│
|
||
v0.14.0 Knowledge Bases (embedding role + file storage + pgvector)
|
||
│
|
||
v0.15.0 Compaction (utility role + background job)
|
||
│
|
||
v0.16.0 @mention Routing + Multi-model
|
||
│
|
||
v0.17.0 Extension Surfaces + Modes
|
||
│
|
||
v0.18.0 Smart Routing (policy on model roles)
|
||
│
|
||
v0.19.0 Multi-Participant Channels + Presence
|
||
│
|
||
v0.20.0 Auth Strategy (mTLS/OIDC) + Full RBAC
|
||
│
|
||
┌───────┴──────────────┐
|
||
│ │
|
||
v0.21.0 Workflow v0.22.0 Tasks /
|
||
Engine Autonomous Agents
|
||
(team-owned, (service channels,
|
||
staged processes, scheduler, unattended
|
||
human + AI collab) execution)
|
||
```
|
||
|
||
---
|
||
|
||
## ✅ v0.9.0 — Schema Consolidation + BYOK
|
||
|
||
The "rewrite" release: 21 migrations collapsed to 1, store layer abstraction,
|
||
and the model pipeline made real for end users.
|
||
|
||
**Schema & Architecture**
|
||
- [x] Consolidated schema (21 migrations → single 001_initial.sql)
|
||
- [x] Store layer abstraction (all DB via typed interfaces, no raw SQL in handlers)
|
||
- [x] Persona-as-trust-boundary model (replaces old presets concept)
|
||
- [x] Scope model (global/team/personal) for configs, personas, models
|
||
- [x] Backward-compatible API routes (v0.8 → v0.9 field aliases)
|
||
|
||
**Capabilities System**
|
||
- [x] Two-tier resolution: catalog (provider API, per-provider) → heuristic inference
|
||
- [x] Venice provider: full capability mapping incl. `optimizedForCode`
|
||
- [x] `ResolveIntrinsic` with gap-fill merging (catalog authoritative, heuristic fills gaps)
|
||
- [x] Preset capability inheritance via `GetByModelIDAny` fallback
|
||
|
||
**BYOK (Bring Your Own Key)**
|
||
- [x] Auto-fetch models on provider creation
|
||
- [x] User-facing refresh endpoint + "Refresh Models" button
|
||
- [x] Personal models in selector with `scope=personal` badge
|
||
- [x] Composite model IDs (`configId:modelId`) prevent cross-provider collisions
|
||
|
||
**Bug Fixes (0.9.0.x)**
|
||
- [x] API key storage (`json:"-"` tag silently dropped keys)
|
||
- [x] NULL model_default scan crash (sql.NullString)
|
||
- [x] Nil slice → JSON null (make([]T, 0))
|
||
- [x] Frontend error swallowing
|
||
|
||
**Testing**
|
||
- [x] Journey-based integration tests (API-only, 4-actor × 3-provider matrix)
|
||
- [x] Live Venice API integration test
|
||
- [x] Frontend JS test suite (107 tests, 27 suites)
|
||
|
||
---
|
||
|
||
## ✅ v0.9.1 — Capability Architecture + Docs
|
||
|
||
- [x] Removed static known model table (same model ≠ same capabilities across providers)
|
||
- [x] Resolution chain: catalog → heuristic only, no hardcoded table
|
||
- [x] Frontend KNOWN_MODELS removed — backend is sole source of truth
|
||
- [x] Heuristic patterns expanded (qwen3, gpt-5, grok, kimi, minimax, glm-5, gemma-3)
|
||
- [x] All providers updated to use `InferCapabilities()` directly
|
||
- [x] EXTENSIONS.md recovered and updated with Appendix A (Custom Renderers)
|
||
and Appendix B (Model Roles)
|
||
- [x] ROADMAP.md, CHANGELOG.md, README.md comprehensive rewrite
|
||
|
||
---
|
||
|
||
## ✅ v0.9.2 — Quick UX Wins + Hardening
|
||
|
||
Polish that doesn't need new infrastructure. Ship fast, stabilize.
|
||
|
||
**UX**
|
||
- [x] Collapsible code blocks (long outputs get `<details>` wrapper)
|
||
- [x] HTML preview for generated content (sandboxed iframe toggle)
|
||
- [x] Token count estimate in input area (before send)
|
||
- [x] "Conversation is getting long" warning (context % thresholds)
|
||
- [x] New switchboard panel favicon (animated SVG + PNG + ICO)
|
||
|
||
**Hardening**
|
||
- [x] Proxy interception detection (Content-Type checks, actionable error messages)
|
||
- [x] Environment injection to frontend (`window.__ENV__`)
|
||
- [x] Enhanced debug diagnostics (NET:PROXY log type, Content-Type capture, env info)
|
||
- [x] Team admin audit scoping (see only their team's audit entries)
|
||
|
||
---
|
||
|
||
## ✅ v0.9.3 — Content Visibility & Block Controls
|
||
|
||
User control over what's expanded/collapsed in the message flow. Thinking
|
||
blocks, tool calls, and code blocks all get consistent collapse controls
|
||
instead of being hidden or auto-formatted without user agency.
|
||
|
||
**Code Blocks**
|
||
- [x] Collapse/expand toggle button on code block header (alongside Copy/Preview)
|
||
- [x] Auto-collapse at threshold (>15 lines), user can always toggle
|
||
- [x] Smooth max-height transition instead of `<details>` wrapping
|
||
|
||
**Thinking Blocks**
|
||
- [x] Always render thinking blocks (never hide from DOM)
|
||
- [x] `showThinking` setting controls default open/closed (default: false), not visibility
|
||
- [x] User can always click to expand regardless of setting
|
||
- [x] `reasoning_content` delta field support for models that stream thinking separately (Grok, etc.)
|
||
- [x] Reasoning accumulated during streaming, wrapped in `<think>` tags before persistence
|
||
|
||
**Tool Calls**
|
||
- [x] Persist tool call metadata in stored messages (`tool_calls` JSONB column)
|
||
- [x] Render tool calls on history load (not just during streaming)
|
||
- [x] Collapsed by default: "🔧 tool_name → done" with toggle for full I/O
|
||
- [x] Notes tool: "📝 View note" link that navigates to Notes panel
|
||
- [x] Tool result rendering fix (operator precedence bug showing "undefined results")
|
||
|
||
**Notes Panel → Side Panel**
|
||
- [x] Copy button on note cards (same pattern as code block copy)
|
||
- [x] Unified side panel replaces inline HTML preview and notes modal
|
||
- [x] Tabbed interface: Preview and Notes tabs
|
||
- [x] Desktop: slides in from right (480px), chat area shrinks via flexbox
|
||
- [x] Mobile: full-width overlay at z-index 200
|
||
- [x] Escape key integration (generation → command palette → side panel → modals)
|
||
|
||
**Streaming Refactor**
|
||
- [x] Extracted `streamWithToolLoop()` into `stream_loop.go` as single canonical streaming implementation
|
||
- [x] Both `streamCompletion` (normal chat) and `Regenerate` call shared function — only persistence differs
|
||
- [x] Regenerate handler now has full feature parity: tools, reasoning, tool_calls persistence
|
||
- [x] Fixed nil `json.RawMessage` causing PostgreSQL JSONB rejection on regen persistence
|
||
|
||
**Regenerate Fix**
|
||
- [x] Regen streams in-place (truncates display to parent before streaming)
|
||
- [x] No more "collapse to original branch" after completion
|
||
|
||
**Favicon**
|
||
- [x] Animated SVG favicon during generation (cascading opacity pulse on 4 dots)
|
||
|
||
**Admin Scope**
|
||
- [x] Admin models endpoint returns only global-scope models (not user BYOK or team models)
|
||
- [x] Bulk visibility toggle scoped to global providers only
|
||
|
||
---
|
||
|
||
## ✅ v0.9.4 — API Key Encryption + Vault
|
||
|
||
Two-tier encryption with a trust boundary between organizational keys (admin-
|
||
accessible) and personal BYOK keys (user-only). Personal keys are protected
|
||
by a per-user encryption key (UEK) that the platform admin cannot recover
|
||
without the user's passphrase.
|
||
|
||
### Threat Model
|
||
|
||
| Threat | Global/Team keys | Personal BYOK keys |
|
||
|--------|-----------------|---------------------|
|
||
| DB backup theft | ✅ Protected (env-var AES) | ✅ Protected (UEK AES) |
|
||
| SQL injection exfil | ✅ Protected | ✅ Protected |
|
||
| Admin reads DB directly | ⚠️ Admin can decrypt (owns env var) | ✅ Admin cannot decrypt (lacks user passphrase) |
|
||
| Admin modifies server code | ⚠️ Out of scope (runtime access) | ⚠️ Out of scope (runtime access) |
|
||
|
||
### Crypto Design
|
||
|
||
**Two encryption paths, one mechanism (AES-256-GCM):**
|
||
|
||
1. **Global/Team keys** — encrypted with a key derived from `ENCRYPTION_KEY`
|
||
env var. Admin-owned, admin-recoverable. Simple.
|
||
|
||
2. **Personal keys** — encrypted with a per-user **User Encryption Key (UEK)**.
|
||
The UEK is a random 256-bit key generated at account creation, itself
|
||
encrypted with a key derived from the user's password via Argon2id.
|
||
|
||
```
|
||
Password → Argon2id(salt) → Password-Derived Key (PDK)
|
||
PDK → AES-256-GCM-Decrypt(encrypted_uek) → UEK (plaintext, session-only)
|
||
UEK → AES-256-GCM-Decrypt(api_key_enc) → API key (plaintext, per-request)
|
||
```
|
||
|
||
The UEK lives in server memory for the duration of the session (alongside the
|
||
JWT). It is never written to disk, logs, or the database in plaintext.
|
||
|
||
### Schema Changes (migration)
|
||
|
||
```sql
|
||
-- Users: vault support
|
||
ALTER TABLE users ADD COLUMN encrypted_uek BYTEA;
|
||
ALTER TABLE users ADD COLUMN uek_salt BYTEA; -- Argon2id salt
|
||
ALTER TABLE users ADD COLUMN uek_nonce BYTEA; -- GCM nonce for UEK wrap
|
||
ALTER TABLE users ADD COLUMN vault_set BOOLEAN NOT NULL DEFAULT false;
|
||
|
||
-- API configs: encrypted key storage
|
||
ALTER TABLE api_configs ADD COLUMN api_key_enc BYTEA;
|
||
ALTER TABLE api_configs ADD COLUMN key_nonce BYTEA;
|
||
ALTER TABLE api_configs ADD COLUMN key_scope TEXT NOT NULL DEFAULT 'global';
|
||
-- key_scope: 'global' | 'team' | 'personal'
|
||
-- global/team: decrypt with env-var-derived key
|
||
-- personal: decrypt with user's UEK
|
||
|
||
-- Team providers: same pattern
|
||
ALTER TABLE team_providers ADD COLUMN api_key_enc BYTEA;
|
||
ALTER TABLE team_providers ADD COLUMN key_nonce BYTEA;
|
||
```
|
||
|
||
Migration backfills: encrypt existing plaintext keys using the env var key
|
||
(all existing keys are global/team scope). Drop plaintext `api_key` column
|
||
after backfill. If `ENCRYPTION_KEY` is not set, migration refuses to run
|
||
(fail-safe).
|
||
|
||
### Auth Mode Integration
|
||
|
||
**Builtin auth** — password serves double duty: authentication + UEK
|
||
derivation. No additional UX. On login, derive PDK → unwrap UEK → cache
|
||
in session. User never knows the vault exists.
|
||
|
||
**mTLS / OIDC** (v0.20.0) — authentication is external (cert/token).
|
||
Vault passphrase is conditionally required:
|
||
|
||
```
|
||
External auth → auto-authenticated
|
||
├── Personal providers DISABLED → straight to app, no prompt
|
||
└── Personal providers ENABLED
|
||
├── First visit → "Set a vault passphrase to protect your API keys"
|
||
└── Returning → "Unlock your vault" (passphrase field, pre-filled username)
|
||
```
|
||
|
||
Terminology: **"vault passphrase"** for mTLS/OIDC users (distinct from their
|
||
non-existent account password). Builtin auth users never see this term.
|
||
|
||
### Backend Implementation
|
||
|
||
**`server/crypto/vault.go`** — pure encryption/decryption, no DB awareness:
|
||
- `DeriveKey(password, salt) → key` (Argon2id, 64MB memory, 3 iterations)
|
||
- `GenerateUEK() → (uek, error)` (crypto/rand, 32 bytes)
|
||
- `WrapUEK(uek, pdk) → (ciphertext, nonce, error)`
|
||
- `UnwrapUEK(ciphertext, nonce, pdk) → (uek, error)`
|
||
- `Encrypt(plaintext, key) → (ciphertext, nonce, error)`
|
||
- `Decrypt(ciphertext, nonce, key) → (plaintext, error)`
|
||
- `DeriveEnvKey(envVar) → key` (HKDF-SHA256 with domain separation context)
|
||
|
||
**Session UEK caching** — UEK stored in a sync.Map keyed by user ID, evicted
|
||
on logout / token expiry. Completion handler retrieves UEK from cache to
|
||
decrypt personal provider keys at request time.
|
||
|
||
**Provider resolution** — `key_scope` determines decryption path:
|
||
```go
|
||
func (s *Store) DecryptAPIKey(config ApiConfig, uekCache *sync.Map) (string, error) {
|
||
switch config.KeyScope {
|
||
case "global", "team":
|
||
return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, envDerivedKey)
|
||
case "personal":
|
||
uek, ok := uekCache.Load(config.UserID)
|
||
if !ok { return "", ErrVaultLocked }
|
||
return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, uek.([]byte))
|
||
}
|
||
}
|
||
```
|
||
|
||
### Edge Cases
|
||
|
||
| Scenario | Behavior |
|
||
|----------|----------|
|
||
| **User changes password** (builtin) | Re-derive PDK, re-encrypt UEK with new PDK. Personal API keys unchanged (keyed to UEK, not password). |
|
||
| **Admin resets user password** | Old UEK unrecoverable. Personal keys destroyed. Admin UI warns before confirming. |
|
||
| **mTLS user forgets passphrase** | Self-service reset: destroys encrypted personal keys. UX: "This will erase your stored API keys." |
|
||
| **Admin disables personal providers** | Keys stay encrypted in DB (inert). Re-enable later → keys accessible if user remembers passphrase. |
|
||
| **`ENCRYPTION_KEY` not set** | Startup refuses to run if encrypted keys exist in DB. Fresh install: keys stored plaintext with deprecation warning in logs + admin panel. |
|
||
| **`ENCRYPTION_KEY` rotated** | Admin CLI tool: `switchboard vault rekey` — re-encrypts all global/team keys with new env key. Personal keys unaffected (keyed to UEK). |
|
||
|
||
### Frontend Changes
|
||
|
||
- Provider management (Settings → Providers): no change to UX for builtin auth
|
||
- mTLS/OIDC splash: conditional vault passphrase prompt (new component)
|
||
- Admin → Users → Reset Password: warning dialog about personal key destruction
|
||
- Admin → Settings: indicator for `ENCRYPTION_KEY` status (set / not set)
|
||
|
||
### Checklist
|
||
|
||
**Crypto core**
|
||
- [x] `server/crypto/vault.go` — Argon2id derivation, AES-256-GCM wrap/unwrap
|
||
- [x] `server/crypto/vault_test.go` — round-trip, wrong-password, corruption (11 tests)
|
||
- [x] Env-var key derivation + validation on startup (HKDF-SHA256 with domain separation)
|
||
- [x] `server/crypto/cache.go` — UEK cache with copy-on-read/write, memory zeroing
|
||
- [x] `server/crypto/resolver.go` — KeyResolver routes decrypt/encrypt by key_scope
|
||
- [x] `server/crypto/backfill.go` — startup migration for plaintext → encrypted keys
|
||
|
||
**Schema + migration**
|
||
- [x] Migration 003_vault.sql: encrypted columns, key_scope backfill
|
||
- [x] `ENCRYPTION_KEY` enforcement (refuse to start if keys exist but no env var)
|
||
- [x] Unencrypted fallback when `ENCRYPTION_KEY` not set (raw bytes, graceful degradation)
|
||
|
||
**Auth integration**
|
||
- [x] UEK generation on user creation (builtin auth)
|
||
- [x] UEK unwrap on login, cache in session
|
||
- [x] UEK eviction on logout (memory zeroing)
|
||
- [ ] UEK re-wrap on password change _(deferred — Settings handler)_
|
||
- [ ] UEK destruction on admin password reset with confirmation _(deferred)_
|
||
- [ ] Vault passphrase prompt for mTLS/OIDC _(deferred — v0.20.0 dependency)_
|
||
|
||
**Provider key lifecycle**
|
||
- [x] Encrypt on provider create/update (all 6 write paths: admin global, personal, team)
|
||
- [x] Decrypt on completion request (all 4 read paths: completion, admin fetch, personal fetch, team fetch)
|
||
- [x] `ErrVaultLocked` handling (prompt user to unlock)
|
||
|
||
**Admin tooling**
|
||
- [ ] `switchboard vault rekey` CLI command _(deferred)_
|
||
- [ ] Admin UI: encryption status indicator _(deferred)_
|
||
- [x] Admin UI: password reset warning (v0.10.0 — vault destruction dialog)
|
||
|
||
**CI/CD**
|
||
- [x] `ENCRYPTION_KEY` Gitea secret → k8s `switchboard-encryption` secret sync
|
||
- [x] Backend k8s manifest: optional `secretKeyRef` for `ENCRYPTION_KEY`
|
||
- [x] Pipeline version bump to v0.9.4
|
||
|
||
**Security fix (bonus)**
|
||
- [x] DOMPurify: strict `ALLOWED_TAGS` allowlist (was permissive `ADD_TAGS` default)
|
||
- [x] Think-block placeholder `\n\n` padding (prevents fence fusion with `</think>`)
|
||
- [x] Unclosed code fence auto-close before `marked.parse` (streaming protection)
|
||
|
||
---
|
||
|
||
## ✅ v0.10.0 — Model Roles + Usage Tracking + Vault Debt
|
||
|
||
Three tracks of infrastructure that everything downstream depends on.
|
||
|
||
**Vault Debt (Security Fixes)**
|
||
- [x] UEK re-wrap on user password change (prevents permanent key loss)
|
||
- [x] UEK destruction on admin password reset (vault + personal BYOK keys deleted)
|
||
- [x] Admin UI: password reset confirmation with vault destruction warning
|
||
- [x] Audit logging for vault destruction events
|
||
|
||
**Model Roles** (see [EXTENSIONS.md Appendix B](EXTENSIONS.md#appendix-b-model-roles))
|
||
- [x] `model_roles` in global_settings: named slots (utility, embedding, generation)
|
||
- [x] Each role: primary provider+model, optional fallback provider+model
|
||
- [x] Team-level role overrides (team.settings JSONB)
|
||
- [x] `Provider.Embed()` interface — OpenAI/Venice/OpenRouter implement, Anthropic returns `ErrNotSupported`
|
||
- [x] `roles.Resolver` package: `Complete()` + `Embed()` with automatic fallback
|
||
- [x] Admin API: `GET/PUT /admin/roles/:role`, `POST /admin/roles/:role/test`
|
||
- [x] Team API: `GET/PUT/DELETE /teams/:id/roles/:role`
|
||
- [x] Admin UI: role configuration tab (select provider + model per role, test fire)
|
||
|
||
**Usage Tracking**
|
||
- [x] Streaming token capture fix — OpenAI: `stream_options.include_usage` + parse usage chunk;
|
||
Anthropic: parse `message_start`/`message_delta` usage events
|
||
- [x] Cache token fields across all types (`CompletionResponse`, `StreamEvent`, `streamResult`)
|
||
- [x] `usage_log` table: channel_id, user_id, model_id, provider_config_id,
|
||
role, token counts (prompt, completion, cache_creation, cache_read), cost, timestamp
|
||
- [x] `model_pricing` table: provider+model, input/output/cache per-M, source (catalog/manual)
|
||
- [x] Cost calculated at insert time (baked, not query-time)
|
||
- [x] BYOK exclusion: admin views filter `provider_scope != 'personal'`, BYOK gets separate endpoint
|
||
- [x] Model pricing from catalog sync (won't overwrite manual overrides)
|
||
- [x] Admin API: `GET /admin/usage`, `GET /admin/usage/users/:id`, `GET /admin/usage/teams/:id`
|
||
- [x] Admin API: `GET/PUT/DELETE /admin/pricing`
|
||
- [x] User API: `GET /usage` (personal usage summary)
|
||
- [x] Admin UI: usage dashboard (period selector, group-by, totals cards, breakdown table, pricing table)
|
||
- [x] Team admin UI: usage dashboard for team-owned providers (`GET /teams/:teamId/usage`)
|
||
- [x] Token accumulation across multi-tool-call iterations in `stream_loop.go`
|
||
|
||
**Schema (migration 004)**
|
||
- [x] `usage_log` with indexes on user, created_at, provider, model, scope
|
||
- [x] `model_pricing` with unique constraint on (provider_config_id, model_id)
|
||
- [x] `model_roles` seed in global_settings
|
||
|
||
---
|
||
|
||
## ✅ v0.10.1 — Polish + UX
|
||
|
||
Spit and polish release: admin system prompt injection, team management modal
|
||
extraction, persona tab, side panel enhancements, code download, and critical
|
||
OpenAI streaming usage race fix.
|
||
|
||
- [x] Admin system prompt (global, injected before user/preset prompts)
|
||
- [x] Personas tab (moved from Models, policy-gated)
|
||
- [x] Team Management modal (extracted from Settings, tabbed, multi-team picker)
|
||
- [x] Preview pane: clear button, fullscreen toggle, resizable
|
||
- [x] Code block: download button (infers extension from language tag)
|
||
- [x] Personal Usage scoped to BYOK only
|
||
- [x] OpenAI streaming usage race fix (deferred Done event until usage chunk)
|
||
- [x] Usage logging zero-token guard removal
|
||
|
||
---
|
||
|
||
## v0.10.2 — Summarize & Continue + Role Cleanup
|
||
|
||
First consumer of the utility model role. User-triggered conversation
|
||
compaction — not automatic (that's v0.15.0).
|
||
|
||
Depends on: utility model role (v0.10.0).
|
||
|
||
**Summarize & Continue**
|
||
- [x] "Summarize and continue" button in context warning bar (≥75% context)
|
||
- [x] `POST /channels/:id/summarize` — calls utility role with conversation history
|
||
- [x] Summary inserted as tree node with `metadata.type = "summary"` + boundary metadata
|
||
- [x] `loadConversation` recognizes summary boundaries: replaces pre-summary messages with summary system message
|
||
- [x] Multiple summaries stack (re-summarize after context fills again)
|
||
- [x] Fork-aware: summary is a tree node, branches have independent boundaries
|
||
- [x] Original messages preserved: "show full history" toggle reveals collapsed messages
|
||
- [x] Usage logged against utility role with cost tracking
|
||
|
||
**BYOK Role Overrides**
|
||
- [x] Personal role resolution chain: personal → team → global
|
||
- [x] User Settings → Model Roles tab (BYOK providers only)
|
||
- [x] Stored in `user.settings` JSONB (`model_roles` key, same shape as team overrides)
|
||
- [x] Only utility and embedding roles (generation removed)
|
||
|
||
**Generation Role Removal**
|
||
- [x] `RoleGeneration` removed from `ValidRoles` (was "generation" for image/media)
|
||
- [x] Removed from admin Roles tab UI
|
||
- [x] Image gen will be extension-managed (v0.11.0), not a role slot
|
||
|
||
**Utility Rate Limiting**
|
||
- [x] `utility_rate_limit` global setting (default: 20 calls/hour/user, 0 = unlimited)
|
||
- [x] Check against `usage_log` before executing org-funded utility calls
|
||
- [x] BYOK calls exempt (user's own key, user's own cost)
|
||
- [x] 429 response with clear message when exceeded
|
||
|
||
---
|
||
|
||
## v0.11.0 — Extension Foundation (Browser Tier)
|
||
|
||
The platform play. See [EXTENSIONS.md](EXTENSIONS.md) for full spec.
|
||
|
||
**Core Loader** (EXTENSIONS.md §4, §7)
|
||
- [ ] `extensions.js` — loader, registry, scoped context (~200 lines)
|
||
- [ ] `Extensions.register()` API with permission enforcement
|
||
- [ ] Manifest format parser and validator
|
||
- [ ] `extensions` + `extension_user_settings` tables
|
||
- [ ] Admin endpoints: install, uninstall, enable/disable
|
||
- [ ] Asset serving: `/api/v1/extensions/:id/assets/*path`
|
||
- [ ] Extension settings UI in Settings modal
|
||
|
||
**Browser Tool Bridge** (EXTENSIONS.md §5)
|
||
- [ ] `ctx.tools.handle()` API for browser tool registration
|
||
- [ ] Tool schema collection: merge built-in + extension tools
|
||
- [ ] `tool.call.*` / `tool.result.*` WebSocket routing
|
||
- [ ] 30-second timeout with LLM error recovery
|
||
- [ ] Tool execution via EventBus `WaitFor` pattern
|
||
|
||
**Custom Renderer API** (EXTENSIONS.md Appendix A)
|
||
- [ ] `ctx.renderers.register()` — pattern + render function
|
||
- [ ] Renderer pipeline in message display (after markdown, before DOM insert)
|
||
- [ ] First-party renderers: collapsible code, HTML preview, mermaid, LaTeX
|
||
(migrate core v0.9.2 implementations to extension format)
|
||
|
||
**First Extensions**
|
||
- [ ] Image generation tool (browser or sidecar, uses generation model role)
|
||
- [ ] STT/TTS (browser extension, Web Speech API — personal use case)
|
||
|
||
---
|
||
|
||
## v0.12.0 — File Handling + Vision
|
||
|
||
File input into chat — table stakes for serious use.
|
||
|
||
**Storage Backend**
|
||
- [ ] S3-compatible API (MinIO, Ceph RGW, AWS — anything with S3 API)
|
||
- [ ] Local PVC fallback (single-node / dev)
|
||
- [ ] Admin config: backend selection, endpoint, credentials, size limits
|
||
- [ ] Reused by KBs (v0.14.0) and compaction snapshots (v0.15.0)
|
||
|
||
**Chat Integration**
|
||
- [ ] `attachments` table (metadata in PG, blobs in object store)
|
||
- [ ] Image/file upload in chat messages
|
||
- [ ] Multimodal message assembly for vision-capable models
|
||
- [ ] Text extraction on upload (PDF, DOCX, TXT → tsvector)
|
||
- [ ] Paste-to-upload (clipboard image support)
|
||
- [ ] Document generation output storage (extension → attachment → download)
|
||
|
||
---
|
||
|
||
## v0.13.0 — Web Search + Tools Expansion
|
||
|
||
Built-in tools that use the tool framework shipped in v0.7.x.
|
||
|
||
- [ ] `web_search` tool: search provider abstraction (SearXNG, Brave, DuckDuckGo)
|
||
- [ ] `url_fetch` tool: retrieve and extract content from URLs
|
||
- [ ] Sidecar or direct HTTP from backend (configurable)
|
||
- [ ] Results injected into context
|
||
- [ ] "Research X and save to my notes" workflow
|
||
|
||
---
|
||
|
||
## v0.14.0 — Knowledge Bases
|
||
|
||
Depends on: embedding model role (v0.10.0), file storage (v0.12.0).
|
||
|
||
- [ ] Embedding pipeline: chunking (recursive, semantic), generation via embedding role
|
||
- [ ] `pgvector` storage and similarity search
|
||
- [ ] `knowledge_bases` table with team scoping
|
||
- [ ] KB document storage via v0.12.0 storage backend
|
||
- [ ] `kb_search` tool (uses existing tool framework)
|
||
- [ ] Context injection in completion flow
|
||
- [ ] Notes embedded too (once pipeline exists)
|
||
- [ ] Per-channel KB toggle
|
||
|
||
---
|
||
|
||
## v0.15.0 — Compaction
|
||
|
||
Depends on: utility model role (v0.10.0).
|
||
|
||
- [ ] Auto-compaction service: background job that calls utility model to summarize
|
||
- [ ] Channel-scoped: triggers when context exceeds threshold
|
||
- [ ] Summaries stored as system messages
|
||
- [ ] Configurable: per-channel opt-in/out, summary model override
|
||
- [ ] Admin controls for resource limits
|
||
|
||
---
|
||
|
||
## v0.16.0 — @mention Routing + Multi-model
|
||
|
||
The channel schema already supports multiple models. This adds routing.
|
||
|
||
- [ ] @mention parsing in messages (users and AI models)
|
||
- [ ] Resolve mentions against `channel_models`
|
||
- [ ] Multi-model channels: fire completions per mentioned model
|
||
- [ ] "Add a model" UI per channel
|
||
- [ ] Enables: second opinions, cross-model conversations
|
||
|
||
---
|
||
|
||
## v0.17.0 — Extension Surfaces + Modes
|
||
|
||
Depends on: extension foundation (v0.11.0).
|
||
See [EXTENSIONS.md §6](EXTENSIONS.md#6-surfaces-modes).
|
||
|
||
- [ ] Surface registration and activation API
|
||
- [ ] Mode selector in sidebar (below brand, above chat history)
|
||
- [ ] `ctx.ui.replace()` / `ctx.ui.restore()` for region management
|
||
- [ ] `surface.activated` / `surface.deactivated` events
|
||
- [ ] Editor mode (extension: file tree + code editor + AI chat split pane)
|
||
- [ ] Article mode (extension: outline + rich text + AI assistant panel)
|
||
|
||
---
|
||
|
||
## v0.18.0 — Smart Routing
|
||
|
||
Depends on: model roles (v0.10.0), usage tracking (v0.10.0).
|
||
Rules-based routing engine — policy, not ML.
|
||
|
||
- [ ] Admin routing policies: "cheapest model with required capabilities",
|
||
"prefer private providers, fallback to cloud", "for team X, use Y"
|
||
- [ ] Fallback chains: primary provider down → next with same model
|
||
- [ ] Cost-aware: factor pricing into routing decisions
|
||
- [ ] Latency-aware: track response times per provider, prefer faster
|
||
|
||
---
|
||
|
||
## v0.19.0 — Multi-Participant Channels + Presence
|
||
|
||
The channel foundation for workflows and real-time collaboration. Today
|
||
channels are single-user direct chats. This release makes channels a
|
||
shared space that multiple actors can inhabit.
|
||
|
||
Depends on: extension surfaces (v0.17.0), WebSocket infrastructure (already exists).
|
||
|
||
**Channel Participants**
|
||
- [ ] `channel_participants` table: `channel_id`, `participant_type` (user, session, persona), `participant_id`, `role` (owner, member, observer), `joined_at`
|
||
- [ ] Channel access queries rewritten: `WHERE user_id = $1` → participant-based lookup
|
||
- [ ] Backward compatible: existing single-user channels auto-have one participant (owner)
|
||
- [ ] Channel `type` column: `direct` (current), `group` (multi-user), `workflow` (staged)
|
||
|
||
**Presence + Real-time**
|
||
- [ ] Typing indicators and presence (who's online, who's in this channel)
|
||
- [ ] Participant join/leave events via WebSocket
|
||
- [ ] Co-editing for extension surfaces (editor mode, article mode)
|
||
- [ ] Cursor sharing, selection highlighting
|
||
|
||
**Channel-Scoped Notes**
|
||
- [ ] Optional `channel_id` FK on notes (NULL = personal notebook, set = channel-attached)
|
||
- [ ] Notes created by tool calls within a channel auto-attach to that channel
|
||
- [ ] Channel notes visible to all participants (not just creator)
|
||
|
||
---
|
||
|
||
## v0.20.0 — Auth Strategy (mTLS/OIDC) + Full RBAC
|
||
|
||
Enterprise auth modes on top of the teams foundation.
|
||
Depends on: vault passphrase support from v0.9.3 (conditional unlock for
|
||
personal providers under external auth).
|
||
|
||
- [ ] `AUTH_MODE` env var: `builtin` | `mtls` | `oidc`
|
||
- [ ] All three resolve to the same internal user model
|
||
- [ ] `auth_source` + `external_id` columns on users table
|
||
- [ ] mTLS: header trust, auto-provision from cert DN
|
||
- [ ] OIDC: Keycloak/Okta token validation, claim extraction, role mapping
|
||
- [ ] Per-source auto-activate policy: auto_activate, default_team, default_role
|
||
- [ ] Vault passphrase prompt for mTLS/OIDC users (v0.9.3 conditional unlock)
|
||
- [ ] Fine-grained permissions: model access, KB write, task create, token budgets
|
||
|
||
**Anonymous / Session Participants**
|
||
- [ ] Session-based identity for unauthenticated visitors (workflow intake)
|
||
- [ ] mTLS: cert CN/fingerprint as stable session identity (no `users` row required)
|
||
- [ ] Non-mTLS: ephemeral session token (cookie or URL token)
|
||
- [ ] Session participants can send messages and trigger tool calls within workflow channels
|
||
- [ ] Session identity visible to assigned team members (cert DN, display name, or "Anonymous #N")
|
||
|
||
---
|
||
|
||
## v0.21.0 — Workflow Engine
|
||
|
||
Team-owned, stage-based process execution. The channel is the runtime,
|
||
personas drive each stage, and the existing tool/notes infrastructure
|
||
handles structured data collection. See [ARCHITECTURE.md — Workflow
|
||
Architecture](ARCHITECTURE.md#workflow-architecture-future--v0210) for
|
||
the conceptual model.
|
||
|
||
Depends on: multi-participant channels (v0.19.0), anonymous identity (v0.20.0).
|
||
|
||
**Workflow Definitions**
|
||
- [ ] `workflows` table: `team_id` (owner), `name`, `description`, `is_active`, `entry_mode` (public_link, team_internal, api)
|
||
- [ ] `workflow_stages` table: `workflow_id`, `ordinal`, `persona_id`, `assignment_team_id`, `form_template` (JSONB, structured note schema), `transition_rules` (JSONB)
|
||
- [ ] Team-admin CRUD: create/edit/delete workflows and stages
|
||
- [ ] Workflow versioning: edits create new version, active instances continue on their version
|
||
|
||
**Workflow Instances (Channels)**
|
||
- [ ] Workflow-typed channels: `type='workflow'`, `workflow_id`, `current_stage`, `stage_version`
|
||
- [ ] Entry point: public link generates workflow channel, adds anonymous participant, binds stage 1 persona
|
||
- [ ] Stage transitions: triggered by AI (tool call), team member (button), or rule (form complete)
|
||
- [ ] On transition: swap active persona, notify assignment team, update channel metadata
|
||
|
||
**AI Intake**
|
||
- [ ] Stage persona drives structured collection via system prompt
|
||
- [ ] Form template → persona system prompt injection ("collect these fields: ...")
|
||
- [ ] Tool calls create channel-scoped notes as form responses
|
||
- [ ] AI determines readiness: "I have everything needed" → trigger transition
|
||
|
||
**Assignment + Queue**
|
||
- [ ] Assignment queue: unassigned workflow channels visible to team members
|
||
- [ ] Claim model: team member claims channel (becomes participant with `member` role)
|
||
- [ ] Round-robin / rule-based auto-assignment (optional, per-stage config)
|
||
- [ ] Assignment notifications via WebSocket + optional webhook
|
||
|
||
**Team Member Collaboration**
|
||
- [ ] Assigned member sees full history (AI intake + artifacts + notes)
|
||
- [ ] Persona remains active — assists both visitor and team member
|
||
- [ ] Member can trigger stage transitions, add notes, invoke tools
|
||
- [ ] Member can reassign to different team member or escalate to different team
|
||
|
||
**Frontend**
|
||
- [ ] Workflow builder UI in team admin panel (stage list, persona picker, form template editor)
|
||
- [ ] Queue view: sidebar section showing unassigned workflow channels for user's teams
|
||
- [ ] Channel header: workflow stage indicator, transition controls
|
||
- [ ] Anonymous visitor view: minimal UI, persona-driven conversation (public link entry)
|
||
|
||
---
|
||
|
||
## v0.22.0 — Tasks / Autonomous Agents
|
||
|
||
Unattended execution — workflows without a human in the loop.
|
||
Depends on: workflow engine (v0.21.0).
|
||
|
||
- [ ] Scheduler + task runner (cron-like triggers for workflow instantiation)
|
||
- [ ] `task_create` tool (AI can spawn sub-workflows)
|
||
- [ ] `type: 'service'` channels: workflow instances with no human participants
|
||
- [ ] Execution budgets: max tokens, max tool calls, max wall-clock per stage
|
||
- [ ] Admin controls for resource limits and kill switches
|
||
- [ ] Completion webhooks (notify external systems when workflow completes)
|
||
|
||
---
|
||
|
||
## TBD (unscheduled — real features, no immediate need)
|
||
|
||
Items that are real but don't yet have a version assignment. Pull left
|
||
based on need.
|
||
|
||
**Extension System — Server Tiers**
|
||
- Starlark runtime integration (Tier 1 — server sandbox)
|
||
- Sidecar HTTP tool protocol (Tier 2 — container isolation)
|
||
- Server-side tool execution in completion handler
|
||
|
||
**Desktop + Mobile**
|
||
- Desktop app (Tauri)
|
||
- Full PWA with offline capability
|
||
- Mobile-optimized layouts (beyond current responsive)
|
||
|
||
**Data + Portability**
|
||
- Bulk export/import (account data, conversations, settings)
|
||
- ChatGPT/other tool import
|
||
- GDPR-style "download my data"
|
||
- Backup/restore CronJob manifests
|
||
|
||
**UX / Multi-Seat**
|
||
- Per-chat model/preset persistence (server-side). Currently a localStorage
|
||
bandaid (v0.10.2) that doesn't roam across devices. Proper fix: store
|
||
`last_selector_id` in `channels.settings` JSONB on each send. On chat
|
||
selection, resolve from `channel.settings.last_selector_id` → match against
|
||
available models/presets → fall back to `channel.model` → global default.
|
||
Eliminates device-local state entirely. Needs `updateChannel` PATCH on
|
||
completion success (single extra column merge, no migration).
|
||
|
||
**Platform**
|
||
- Rate limiting per user/team/tier (token budgets)
|
||
- Provider health monitoring + key rotation
|
||
- Multi-tenant SaaS mode
|
||
- Plugin/extension marketplace
|
||
- Virtual scroll for long conversations
|
||
- SQLite backend option (single-user / dev)
|