830 lines
38 KiB
Markdown
830 lines
38 KiB
Markdown
# Roadmap — Chat Switchboard
|
||
|
||
**See also:**
|
||
- [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design, store layer, scope model
|
||
- [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers,
|
||
manifests, browser tool bridge, surfaces/modes, model roles)
|
||
|
||
**Versioning (pre-1.0):** `0.<major>.<minor>` — hotfixes use quad: `0.x.y.z`
|
||
No compatibility guarantees before 1.0.
|
||
|
||
---
|
||
|
||
## Dependency Graph
|
||
|
||
Features have real dependencies. This ordering respects them.
|
||
|
||
```
|
||
v0.9.x Stability + Quick UX Wins ✅
|
||
│
|
||
v0.9.3 Content Visibility & Block Controls ✅
|
||
│
|
||
v0.9.4 API Key Encryption + Vault ✅
|
||
│
|
||
v0.10.0 Model Roles (utility + embedding) ✅
|
||
+ Usage Tracking
|
||
│
|
||
v0.10.2 Summarize & Continue ✅
|
||
│
|
||
v0.10.3 Frontend Refactor ✅
|
||
│
|
||
v0.10.4 Model Type Pipeline + Role Fixes ✅
|
||
│
|
||
v0.10.5 UI Primitives + Extension Surfaces ✅
|
||
│
|
||
v0.11.0 Extension Foundation (browser tier) ✅
|
||
│
|
||
┌───────┴──────────────┐
|
||
│ │
|
||
v0.12.0 File Handling v0.13.0 Web Search
|
||
+ Vision + Tools Expansion
|
||
│
|
||
v0.14.0 Knowledge Bases (embedding role + file storage + pgvector)
|
||
│
|
||
v0.15.0 Compaction (utility role + background job)
|
||
│
|
||
v0.16.0 @mention Routing + Multi-model
|
||
│
|
||
v0.17.0 Extension Surfaces + Modes
|
||
│
|
||
v0.18.0 Smart Routing (policy on model roles)
|
||
│
|
||
v0.19.0 Multi-Participant Channels + Presence
|
||
│
|
||
v0.20.0 Auth Strategy (mTLS/OIDC) + Full RBAC
|
||
│
|
||
┌───────┴──────────────┐
|
||
│ │
|
||
v0.21.0 Workflow v0.22.0 Tasks /
|
||
Engine Autonomous Agents
|
||
(team-owned, (service channels,
|
||
staged processes, scheduler, unattended
|
||
human + AI collab) execution)
|
||
```
|
||
|
||
---
|
||
|
||
## ✅ v0.9.0 — Schema Consolidation + BYOK
|
||
|
||
The "rewrite" release: 21 migrations collapsed to 1, store layer abstraction,
|
||
and the model pipeline made real for end users.
|
||
|
||
**Schema & Architecture**
|
||
- [x] Consolidated schema (21 migrations → single 001_initial.sql)
|
||
- [x] Store layer abstraction (all DB via typed interfaces, no raw SQL in handlers)
|
||
- [x] Persona-as-trust-boundary model (replaces old presets concept)
|
||
- [x] Scope model (global/team/personal) for configs, personas, models
|
||
- [x] Backward-compatible API routes (v0.8 → v0.9 field aliases)
|
||
|
||
**Capabilities System**
|
||
- [x] Two-tier resolution: catalog (provider API, per-provider) → heuristic inference
|
||
- [x] Venice provider: full capability mapping incl. `optimizedForCode`
|
||
- [x] `ResolveIntrinsic` with gap-fill merging (catalog authoritative, heuristic fills gaps)
|
||
- [x] Preset capability inheritance via `GetByModelIDAny` fallback
|
||
|
||
**BYOK (Bring Your Own Key)**
|
||
- [x] Auto-fetch models on provider creation
|
||
- [x] User-facing refresh endpoint + "Refresh Models" button
|
||
- [x] Personal models in selector with `scope=personal` badge
|
||
- [x] Composite model IDs (`configId:modelId`) prevent cross-provider collisions
|
||
|
||
**Bug Fixes (0.9.0.x)**
|
||
- [x] API key storage (`json:"-"` tag silently dropped keys)
|
||
- [x] NULL model_default scan crash (sql.NullString)
|
||
- [x] Nil slice → JSON null (make([]T, 0))
|
||
- [x] Frontend error swallowing
|
||
|
||
**Testing**
|
||
- [x] Journey-based integration tests (API-only, 4-actor × 3-provider matrix)
|
||
- [x] Live Venice API integration test
|
||
- [x] Frontend JS test suite (107 tests, 27 suites)
|
||
|
||
---
|
||
|
||
## ✅ v0.9.1 — Capability Architecture + Docs
|
||
|
||
- [x] Removed static known model table (same model ≠ same capabilities across providers)
|
||
- [x] Resolution chain: catalog → heuristic only, no hardcoded table
|
||
- [x] Frontend KNOWN_MODELS removed — backend is sole source of truth
|
||
- [x] Heuristic patterns expanded (qwen3, gpt-5, grok, kimi, minimax, glm-5, gemma-3)
|
||
- [x] All providers updated to use `InferCapabilities()` directly
|
||
- [x] EXTENSIONS.md recovered and updated with Appendix A (Custom Renderers)
|
||
and Appendix B (Model Roles)
|
||
- [x] ROADMAP.md, CHANGELOG.md, README.md comprehensive rewrite
|
||
|
||
---
|
||
|
||
## ✅ v0.9.2 — Quick UX Wins + Hardening
|
||
|
||
Polish that doesn't need new infrastructure. Ship fast, stabilize.
|
||
|
||
**UX**
|
||
- [x] Collapsible code blocks (long outputs get `<details>` wrapper)
|
||
- [x] HTML preview for generated content (sandboxed iframe toggle)
|
||
- [x] Token count estimate in input area (before send)
|
||
- [x] "Conversation is getting long" warning (context % thresholds)
|
||
- [x] New switchboard panel favicon (animated SVG + PNG + ICO)
|
||
|
||
**Hardening**
|
||
- [x] Proxy interception detection (Content-Type checks, actionable error messages)
|
||
- [x] Environment injection to frontend (`window.__ENV__`)
|
||
- [x] Enhanced debug diagnostics (NET:PROXY log type, Content-Type capture, env info)
|
||
- [x] Team admin audit scoping (see only their team's audit entries)
|
||
|
||
---
|
||
|
||
## ✅ v0.9.3 — Content Visibility & Block Controls
|
||
|
||
User control over what's expanded/collapsed in the message flow. Thinking
|
||
blocks, tool calls, and code blocks all get consistent collapse controls
|
||
instead of being hidden or auto-formatted without user agency.
|
||
|
||
**Code Blocks**
|
||
- [x] Collapse/expand toggle button on code block header (alongside Copy/Preview)
|
||
- [x] Auto-collapse at threshold (>15 lines), user can always toggle
|
||
- [x] Smooth max-height transition instead of `<details>` wrapping
|
||
|
||
**Thinking Blocks**
|
||
- [x] Always render thinking blocks (never hide from DOM)
|
||
- [x] `showThinking` setting controls default open/closed (default: false), not visibility
|
||
- [x] User can always click to expand regardless of setting
|
||
- [x] `reasoning_content` delta field support for models that stream thinking separately (Grok, etc.)
|
||
- [x] Reasoning accumulated during streaming, wrapped in `<think>` tags before persistence
|
||
|
||
**Tool Calls**
|
||
- [x] Persist tool call metadata in stored messages (`tool_calls` JSONB column)
|
||
- [x] Render tool calls on history load (not just during streaming)
|
||
- [x] Collapsed by default: "🔧 tool_name → done" with toggle for full I/O
|
||
- [x] Notes tool: "📝 View note" link that navigates to Notes panel
|
||
- [x] Tool result rendering fix (operator precedence bug showing "undefined results")
|
||
|
||
**Notes Panel → Side Panel**
|
||
- [x] Copy button on note cards (same pattern as code block copy)
|
||
- [x] Unified side panel replaces inline HTML preview and notes modal
|
||
- [x] Tabbed interface: Preview and Notes tabs
|
||
- [x] Desktop: slides in from right (480px), chat area shrinks via flexbox
|
||
- [x] Mobile: full-width overlay at z-index 200
|
||
- [x] Escape key integration (generation → command palette → side panel → modals)
|
||
|
||
**Streaming Refactor**
|
||
- [x] Extracted `streamWithToolLoop()` into `stream_loop.go` as single canonical streaming implementation
|
||
- [x] Both `streamCompletion` (normal chat) and `Regenerate` call shared function — only persistence differs
|
||
- [x] Regenerate handler now has full feature parity: tools, reasoning, tool_calls persistence
|
||
- [x] Fixed nil `json.RawMessage` causing PostgreSQL JSONB rejection on regen persistence
|
||
|
||
**Regenerate Fix**
|
||
- [x] Regen streams in-place (truncates display to parent before streaming)
|
||
- [x] No more "collapse to original branch" after completion
|
||
|
||
**Favicon**
|
||
- [x] Animated SVG favicon during generation (cascading opacity pulse on 4 dots)
|
||
|
||
**Admin Scope**
|
||
- [x] Admin models endpoint returns only global-scope models (not user BYOK or team models)
|
||
- [x] Bulk visibility toggle scoped to global providers only
|
||
|
||
---
|
||
|
||
## ✅ v0.9.4 — API Key Encryption + Vault
|
||
|
||
Two-tier encryption with a trust boundary between organizational keys (admin-
|
||
accessible) and personal BYOK keys (user-only). Personal keys are protected
|
||
by a per-user encryption key (UEK) that the platform admin cannot recover
|
||
without the user's passphrase.
|
||
|
||
### Threat Model
|
||
|
||
| Threat | Global/Team keys | Personal BYOK keys |
|
||
|--------|-----------------|---------------------|
|
||
| DB backup theft | ✅ Protected (env-var AES) | ✅ Protected (UEK AES) |
|
||
| SQL injection exfil | ✅ Protected | ✅ Protected |
|
||
| Admin reads DB directly | ⚠️ Admin can decrypt (owns env var) | ✅ Admin cannot decrypt (lacks user passphrase) |
|
||
| Admin modifies server code | ⚠️ Out of scope (runtime access) | ⚠️ Out of scope (runtime access) |
|
||
|
||
### Crypto Design
|
||
|
||
**Two encryption paths, one mechanism (AES-256-GCM):**
|
||
|
||
1. **Global/Team keys** — encrypted with a key derived from `ENCRYPTION_KEY`
|
||
env var. Admin-owned, admin-recoverable. Simple.
|
||
|
||
2. **Personal keys** — encrypted with a per-user **User Encryption Key (UEK)**.
|
||
The UEK is a random 256-bit key generated at account creation, itself
|
||
encrypted with a key derived from the user's password via Argon2id.
|
||
|
||
```
|
||
Password → Argon2id(salt) → Password-Derived Key (PDK)
|
||
PDK → AES-256-GCM-Decrypt(encrypted_uek) → UEK (plaintext, session-only)
|
||
UEK → AES-256-GCM-Decrypt(api_key_enc) → API key (plaintext, per-request)
|
||
```
|
||
|
||
The UEK lives in server memory for the duration of the session (alongside the
|
||
JWT). It is never written to disk, logs, or the database in plaintext.
|
||
|
||
### Schema Changes (migration)
|
||
|
||
```sql
|
||
-- Users: vault support
|
||
ALTER TABLE users ADD COLUMN encrypted_uek BYTEA;
|
||
ALTER TABLE users ADD COLUMN uek_salt BYTEA; -- Argon2id salt
|
||
ALTER TABLE users ADD COLUMN uek_nonce BYTEA; -- GCM nonce for UEK wrap
|
||
ALTER TABLE users ADD COLUMN vault_set BOOLEAN NOT NULL DEFAULT false;
|
||
|
||
-- API configs: encrypted key storage
|
||
ALTER TABLE api_configs ADD COLUMN api_key_enc BYTEA;
|
||
ALTER TABLE api_configs ADD COLUMN key_nonce BYTEA;
|
||
ALTER TABLE api_configs ADD COLUMN key_scope TEXT NOT NULL DEFAULT 'global';
|
||
-- key_scope: 'global' | 'team' | 'personal'
|
||
-- global/team: decrypt with env-var-derived key
|
||
-- personal: decrypt with user's UEK
|
||
|
||
-- Team providers: same pattern
|
||
ALTER TABLE team_providers ADD COLUMN api_key_enc BYTEA;
|
||
ALTER TABLE team_providers ADD COLUMN key_nonce BYTEA;
|
||
```
|
||
|
||
Migration backfills: encrypt existing plaintext keys using the env var key
|
||
(all existing keys are global/team scope). Drop plaintext `api_key` column
|
||
after backfill. If `ENCRYPTION_KEY` is not set, migration refuses to run
|
||
(fail-safe).
|
||
|
||
### Auth Mode Integration
|
||
|
||
**Builtin auth** — password serves double duty: authentication + UEK
|
||
derivation. No additional UX. On login, derive PDK → unwrap UEK → cache
|
||
in session. User never knows the vault exists.
|
||
|
||
**mTLS / OIDC** (v0.20.0) — authentication is external (cert/token).
|
||
Vault passphrase is conditionally required:
|
||
|
||
```
|
||
External auth → auto-authenticated
|
||
├── Personal providers DISABLED → straight to app, no prompt
|
||
└── Personal providers ENABLED
|
||
├── First visit → "Set a vault passphrase to protect your API keys"
|
||
└── Returning → "Unlock your vault" (passphrase field, pre-filled username)
|
||
```
|
||
|
||
Terminology: **"vault passphrase"** for mTLS/OIDC users (distinct from their
|
||
non-existent account password). Builtin auth users never see this term.
|
||
|
||
### Backend Implementation
|
||
|
||
**`server/crypto/vault.go`** — pure encryption/decryption, no DB awareness:
|
||
- `DeriveKey(password, salt) → key` (Argon2id, 64MB memory, 3 iterations)
|
||
- `GenerateUEK() → (uek, error)` (crypto/rand, 32 bytes)
|
||
- `WrapUEK(uek, pdk) → (ciphertext, nonce, error)`
|
||
- `UnwrapUEK(ciphertext, nonce, pdk) → (uek, error)`
|
||
- `Encrypt(plaintext, key) → (ciphertext, nonce, error)`
|
||
- `Decrypt(ciphertext, nonce, key) → (plaintext, error)`
|
||
- `DeriveEnvKey(envVar) → key` (HKDF-SHA256 with domain separation context)
|
||
|
||
**Session UEK caching** — UEK stored in a sync.Map keyed by user ID, evicted
|
||
on logout / token expiry. Completion handler retrieves UEK from cache to
|
||
decrypt personal provider keys at request time.
|
||
|
||
**Provider resolution** — `key_scope` determines decryption path:
|
||
```go
|
||
func (s *Store) DecryptAPIKey(config ApiConfig, uekCache *sync.Map) (string, error) {
|
||
switch config.KeyScope {
|
||
case "global", "team":
|
||
return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, envDerivedKey)
|
||
case "personal":
|
||
uek, ok := uekCache.Load(config.UserID)
|
||
if !ok { return "", ErrVaultLocked }
|
||
return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, uek.([]byte))
|
||
}
|
||
}
|
||
```
|
||
|
||
### Edge Cases
|
||
|
||
| Scenario | Behavior |
|
||
|----------|----------|
|
||
| **User changes password** (builtin) | Re-derive PDK, re-encrypt UEK with new PDK. Personal API keys unchanged (keyed to UEK, not password). |
|
||
| **Admin resets user password** | Old UEK unrecoverable. Personal keys destroyed. Admin UI warns before confirming. |
|
||
| **mTLS user forgets passphrase** | Self-service reset: destroys encrypted personal keys. UX: "This will erase your stored API keys." |
|
||
| **Admin disables personal providers** | Keys stay encrypted in DB (inert). Re-enable later → keys accessible if user remembers passphrase. |
|
||
| **`ENCRYPTION_KEY` not set** | Startup refuses to run if encrypted keys exist in DB. Fresh install: keys stored plaintext with deprecation warning in logs + admin panel. |
|
||
| **`ENCRYPTION_KEY` rotated** | Admin CLI tool: `switchboard vault rekey` — re-encrypts all global/team keys with new env key. Personal keys unaffected (keyed to UEK). |
|
||
|
||
### Frontend Changes
|
||
|
||
- Provider management (Settings → Providers): no change to UX for builtin auth
|
||
- mTLS/OIDC splash: conditional vault passphrase prompt (new component)
|
||
- Admin → Users → Reset Password: warning dialog about personal key destruction
|
||
- Admin → Settings: indicator for `ENCRYPTION_KEY` status (set / not set)
|
||
|
||
### Checklist
|
||
|
||
**Crypto core**
|
||
- [x] `server/crypto/vault.go` — Argon2id derivation, AES-256-GCM wrap/unwrap
|
||
- [x] `server/crypto/vault_test.go` — round-trip, wrong-password, corruption (11 tests)
|
||
- [x] Env-var key derivation + validation on startup (HKDF-SHA256 with domain separation)
|
||
- [x] `server/crypto/cache.go` — UEK cache with copy-on-read/write, memory zeroing
|
||
- [x] `server/crypto/resolver.go` — KeyResolver routes decrypt/encrypt by key_scope
|
||
- [x] `server/crypto/backfill.go` — startup migration for plaintext → encrypted keys
|
||
|
||
**Schema + migration**
|
||
- [x] Migration 003_vault.sql: encrypted columns, key_scope backfill
|
||
- [x] `ENCRYPTION_KEY` enforcement (refuse to start if keys exist but no env var)
|
||
- [x] Unencrypted fallback when `ENCRYPTION_KEY` not set (raw bytes, graceful degradation)
|
||
|
||
**Auth integration**
|
||
- [x] UEK generation on user creation (builtin auth)
|
||
- [x] UEK unwrap on login, cache in session
|
||
- [x] UEK eviction on logout (memory zeroing)
|
||
- [ ] UEK re-wrap on password change _(deferred — Settings handler)_
|
||
- [ ] UEK destruction on admin password reset with confirmation _(deferred)_
|
||
- [ ] Vault passphrase prompt for mTLS/OIDC _(deferred — v0.20.0 dependency)_
|
||
|
||
**Provider key lifecycle**
|
||
- [x] Encrypt on provider create/update (all 6 write paths: admin global, personal, team)
|
||
- [x] Decrypt on completion request (all 4 read paths: completion, admin fetch, personal fetch, team fetch)
|
||
- [x] `ErrVaultLocked` handling (prompt user to unlock)
|
||
|
||
**Admin tooling**
|
||
- [ ] `switchboard vault rekey` CLI command _(deferred)_
|
||
- [ ] Admin UI: encryption status indicator _(deferred)_
|
||
- [x] Admin UI: password reset warning (v0.10.0 — vault destruction dialog)
|
||
|
||
**CI/CD**
|
||
- [x] `ENCRYPTION_KEY` Gitea secret → k8s `switchboard-encryption` secret sync
|
||
- [x] Backend k8s manifest: optional `secretKeyRef` for `ENCRYPTION_KEY`
|
||
- [x] Pipeline version bump to v0.9.4
|
||
|
||
**Security fix (bonus)**
|
||
- [x] DOMPurify: strict `ALLOWED_TAGS` allowlist (was permissive `ADD_TAGS` default)
|
||
- [x] Think-block placeholder `\n\n` padding (prevents fence fusion with `</think>`)
|
||
- [x] Unclosed code fence auto-close before `marked.parse` (streaming protection)
|
||
|
||
---
|
||
|
||
## ✅ v0.10.0 — Model Roles + Usage Tracking + Vault Debt
|
||
|
||
Three tracks of infrastructure that everything downstream depends on.
|
||
|
||
**Vault Debt (Security Fixes)**
|
||
- [x] UEK re-wrap on user password change (prevents permanent key loss)
|
||
- [x] UEK destruction on admin password reset (vault + personal BYOK keys deleted)
|
||
- [x] Admin UI: password reset confirmation with vault destruction warning
|
||
- [x] Audit logging for vault destruction events
|
||
|
||
**Model Roles** (see [EXTENSIONS.md Appendix B](EXTENSIONS.md#appendix-b-model-roles))
|
||
- [x] `model_roles` in global_settings: named slots (utility, embedding, generation)
|
||
- [x] Each role: primary provider+model, optional fallback provider+model
|
||
- [x] Team-level role overrides (team.settings JSONB)
|
||
- [x] `Provider.Embed()` interface — OpenAI/Venice/OpenRouter implement, Anthropic returns `ErrNotSupported`
|
||
- [x] `roles.Resolver` package: `Complete()` + `Embed()` with automatic fallback
|
||
- [x] Admin API: `GET/PUT /admin/roles/:role`, `POST /admin/roles/:role/test`
|
||
- [x] Team API: `GET/PUT/DELETE /teams/:id/roles/:role`
|
||
- [x] Admin UI: role configuration tab (select provider + model per role, test fire)
|
||
|
||
**Usage Tracking**
|
||
- [x] Streaming token capture fix — OpenAI: `stream_options.include_usage` + parse usage chunk;
|
||
Anthropic: parse `message_start`/`message_delta` usage events
|
||
- [x] Cache token fields across all types (`CompletionResponse`, `StreamEvent`, `streamResult`)
|
||
- [x] `usage_log` table: channel_id, user_id, model_id, provider_config_id,
|
||
role, token counts (prompt, completion, cache_creation, cache_read), cost, timestamp
|
||
- [x] `model_pricing` table: provider+model, input/output/cache per-M, source (catalog/manual)
|
||
- [x] Cost calculated at insert time (baked, not query-time)
|
||
- [x] BYOK exclusion: admin views filter `provider_scope != 'personal'`, BYOK gets separate endpoint
|
||
- [x] Model pricing from catalog sync (won't overwrite manual overrides)
|
||
- [x] Admin API: `GET /admin/usage`, `GET /admin/usage/users/:id`, `GET /admin/usage/teams/:id`
|
||
- [x] Admin API: `GET/PUT/DELETE /admin/pricing`
|
||
- [x] User API: `GET /usage` (personal usage summary)
|
||
- [x] Admin UI: usage dashboard (period selector, group-by, totals cards, breakdown table, pricing table)
|
||
- [x] Team admin UI: usage dashboard for team-owned providers (`GET /teams/:teamId/usage`)
|
||
- [x] Token accumulation across multi-tool-call iterations in `stream_loop.go`
|
||
|
||
**Schema (migration 004)**
|
||
- [x] `usage_log` with indexes on user, created_at, provider, model, scope
|
||
- [x] `model_pricing` with unique constraint on (provider_config_id, model_id)
|
||
- [x] `model_roles` seed in global_settings
|
||
|
||
---
|
||
|
||
## ✅ v0.10.1 — Polish + UX
|
||
|
||
Spit and polish release: admin system prompt injection, team management modal
|
||
extraction, persona tab, side panel enhancements, code download, and critical
|
||
OpenAI streaming usage race fix.
|
||
|
||
- [x] Admin system prompt (global, injected before user/preset prompts)
|
||
- [x] Personas tab (moved from Models, policy-gated)
|
||
- [x] Team Management modal (extracted from Settings, tabbed, multi-team picker)
|
||
- [x] Preview pane: clear button, fullscreen toggle, resizable
|
||
- [x] Code block: download button (infers extension from language tag)
|
||
- [x] Personal Usage scoped to BYOK only
|
||
- [x] OpenAI streaming usage race fix (deferred Done event until usage chunk)
|
||
- [x] Usage logging zero-token guard removal
|
||
|
||
---
|
||
|
||
## v0.10.2 — Summarize & Continue + Role Cleanup
|
||
|
||
First consumer of the utility model role. User-triggered conversation
|
||
compaction — not automatic (that's v0.15.0).
|
||
|
||
Depends on: utility model role (v0.10.0).
|
||
|
||
**Summarize & Continue**
|
||
- [x] "Summarize and continue" button in context warning bar (≥75% context)
|
||
- [x] `POST /channels/:id/summarize` — calls utility role with conversation history
|
||
- [x] Summary inserted as tree node with `metadata.type = "summary"` + boundary metadata
|
||
- [x] `loadConversation` recognizes summary boundaries: replaces pre-summary messages with summary system message
|
||
- [x] Multiple summaries stack (re-summarize after context fills again)
|
||
- [x] Fork-aware: summary is a tree node, branches have independent boundaries
|
||
- [x] Original messages preserved: "show full history" toggle reveals collapsed messages
|
||
- [x] Usage logged against utility role with cost tracking
|
||
|
||
**BYOK Role Overrides**
|
||
- [x] Personal role resolution chain: personal → team → global
|
||
- [x] User Settings → Model Roles tab (BYOK providers only)
|
||
- [x] Stored in `user.settings` JSONB (`model_roles` key, same shape as team overrides)
|
||
- [x] Only utility and embedding roles (generation removed)
|
||
|
||
**Generation Role Removal**
|
||
- [x] `RoleGeneration` removed from `ValidRoles` (was "generation" for image/media)
|
||
- [x] Removed from admin Roles tab UI
|
||
- [x] Image gen will be extension-managed (v0.11.0), not a role slot
|
||
|
||
**Utility Rate Limiting**
|
||
- [x] `utility_rate_limit` global setting (default: 20 calls/hour/user, 0 = unlimited)
|
||
- [x] Check against `usage_log` before executing org-funded utility calls
|
||
- [x] BYOK calls exempt (user's own key, user's own cost)
|
||
- [x] 429 response with clear message when exceeded
|
||
|
||
---
|
||
|
||
## v0.10.3 — Frontend Refactor ✅
|
||
|
||
Zero features. Split the two monolith frontend files (app.js 2,940 lines,
|
||
ui.js 2,582 lines) into 13 domain-scoped files averaging ~544 lines each.
|
||
Vanilla JS, no modules, no build step, no function renaming.
|
||
|
||
See [REFACTOR-0.10.3.md](REFACTOR-0.10.3.md) for full plan.
|
||
|
||
- [x] Extract `ui-format.js` — markdown, code blocks, `esc()`, helpers
|
||
- [x] Extract `ui-settings.js` — settings tabs, teams, providers, user prefs
|
||
- [x] Extract `ui-admin.js` — admin modal tabs, all admin rendering
|
||
- [x] Extract `tokens.js` — context tracking, token estimation
|
||
- [x] Extract `notes.js` — notes panel, editor, multi-select
|
||
- [x] Extract `chat.js` — chat ops, send, regen, edit, branch, summarize
|
||
- [x] Extract `settings-handlers.js` — settings save, provider CRUD, cmd palette
|
||
- [x] Extract `admin-handlers.js` — admin actions, presets, team management
|
||
- [x] Slim `app.js` to orchestrator — state, init, boot, auth, listener dispatch
|
||
- [x] Update `sw.js` SHELL_FILES, `index.html` script tags
|
||
- [x] All 159+ frontend tests pass, `node --check` on all files
|
||
|
||
---
|
||
|
||
## v0.10.4 — Model Type Pipeline + Role Fixes ✅
|
||
|
||
Provider APIs report model types (chat, embedding, image) — now captured
|
||
end-to-end through sync → catalog → resolver → frontend. Role dropdowns
|
||
filter models by type instead of showing everything.
|
||
|
||
- [x] Migration `005_model_type.sql` — `model_catalog.model_type` column
|
||
- [x] `providers.Model.Type` — captured from provider wire response
|
||
- [x] Venice: reads `type` field, normalizes (`text`→`chat`, `embedding`→`embedding`)
|
||
- [x] OpenAI: wire struct extended with optional `type` for compatible APIs
|
||
- [x] `CatalogSyncEntry.ModelType` → `CatalogEntry.ModelType` → `UserModel.ModelType`
|
||
- [x] Frontend `model_type` mapped through `fetchModels()`
|
||
- [x] Admin role dropdowns: filter by `model_type` matching role
|
||
- [x] User role dropdowns: filter by `model_type` matching role
|
||
- [x] `adminSaveRole()` reloads UI after save (was showing "Saved" without refresh)
|
||
|
||
---
|
||
|
||
## v0.10.5 — UI Primitives + Extension Surfaces ✅
|
||
|
||
Consolidate duplicated UI patterns into shared primitives with extension-ready
|
||
interfaces. Pre-extension infrastructure: the primitives we build now become
|
||
the `ctx.*` extension API surface in v0.11.0 with no rewrite needed.
|
||
|
||
- [x] `Providers` registry — single source of truth for types, labels, endpoints
|
||
- [x] `Roles` registry — names, type filters, hints
|
||
- [x] `renderCapBadges()` — consolidates 3 badge builders (compact + detailed modes)
|
||
- [x] `renderProviderForm()` — replaces 3 form implementations, dual-mode create/edit
|
||
- [x] `renderProviderList()` — replaces 3 list renderers, event delegation
|
||
- [x] `renderRoleConfig()` — replaces 2 role UIs + 4 handlers
|
||
- [x] `renderUsageDashboard()` — replaces 3 usage renderers (compact + full)
|
||
- [x] Admin provider form gets endpoint auto-fill (was missing)
|
||
- [x] Team provider form consolidated from 2 separate forms to 1 dual-mode
|
||
- [x] Removed ~360 lines of duplicated code + ~40 lines of static HTML
|
||
- [x] `showConfirm()` — styled confirm dialog replacing all 16 native `confirm()` calls
|
||
- [x] `.popup-menu` shared CSS + `createPopupMenu()` primitive for consistent menus
|
||
- [x] Sidebar collapse icon always visible (dimmed), brightens on hover
|
||
|
||
---
|
||
|
||
## ✅ v0.11.0 — Extension Foundation (Browser Tier)
|
||
|
||
The platform play. See [EXTENSIONS.md](EXTENSIONS.md) for full spec.
|
||
|
||
**Core Loader** (EXTENSIONS.md §4, §7)
|
||
- [x] `extensions.js` — loader, registry, scoped context, renderer pipeline (~400 lines)
|
||
- [x] `Extensions.register()` API with permission-aware scoped contexts
|
||
- [x] Manifest format: `_script` inline or `entry` file, permissions, settings schema
|
||
- [x] `extensions` + `extension_user_settings` tables (migration 006)
|
||
- [x] Admin endpoints: CRUD, enable/disable, asset serving
|
||
- [x] `/api/v1/extensions/:id/assets/*path` — public (no auth, `<script>` tag compatible)
|
||
- [x] Auto-seeder: scans `extensions/builtin/*/` on startup, upserts manifests
|
||
- [x] Admin Extensions tab with Edit button (view/modify manifest + script inline)
|
||
|
||
**Browser Tool Bridge** (EXTENSIONS.md §5)
|
||
- [x] `ctx.tools.handle()` API for browser tool registration
|
||
- [x] Tool schema collection: merge server + extension tools in completions
|
||
- [x] `tool.call.*` / `tool.result.*` WebSocket routing via EventBus hub
|
||
- [x] 30-second timeout with error recovery
|
||
- [x] `Events.waitFor()` pattern for request/response tool calls
|
||
|
||
**Custom Renderer API** (EXTENSIONS.md Appendix A)
|
||
- [x] `ctx.renderers.register()` — block, post, and inline types with pattern/match
|
||
- [x] Renderer pipeline: block renderers intercept code fences, post renderers process DOM
|
||
- [x] Case-insensitive language matching (LLMs capitalize fence tags)
|
||
- [x] Nested ```` ```markdown ```` fence unwrapping (LLMs wrap responses in markdown fences)
|
||
- [x] Extension-owned styles via `_injectStyles()` / `destroy()` lifecycle
|
||
|
||
**Server Tools** (auto-registered via `init()`)
|
||
- [x] `datetime` — current date/time/timezone/day/unix/ISO-week (models hallucinate dates)
|
||
- [x] `calculator` — recursive-descent evaluator: arithmetic, 17 functions, constants
|
||
|
||
**Built-in Browser Extensions** (6 total, self-contained with own CSS)
|
||
- [x] **Mermaid** — block + post renderer, SVG diagrams, dark mode, local vendor + CDN fallback
|
||
- [x] **KaTeX** — block renderer (` ```latex/math/tex ```) + post renderer (inline `$...$` / `$$...$$`)
|
||
- [x] **CSV Table** — block renderer, RFC 4180 parser, sortable columns, numeric-aware sorting
|
||
- [x] **Diff Viewer** — block renderer, red/green syntax highlighting, stats badge
|
||
- [x] **JS Sandbox** — browser tool (`js_eval`), sandboxed iframe, 10s timeout, console capture
|
||
- [x] **Regex Tester** — browser tool (`regex_test`), multi-input matching, named groups
|
||
|
||
**Bugfixes**
|
||
- [x] WebSocket token field mismatch (`tokens.access` → `tokens.accessToken`)
|
||
- [x] Extension asset 401 (moved asset route to public group — `<script>` tags can't send auth headers)
|
||
- [x] `runExtensionPostRender is not defined` (stale SW cache, added `typeof` guard)
|
||
- [x] Extension init order (moved `loadAll()`/`initAll()` before `loadChats()`)
|
||
- [x] K8s Ingress WebSocket routing (`pathType: Exact` → `Prefix` for Traefik longest-prefix-wins)
|
||
- [x] Notes missing post-render call (mermaid/KaTeX wouldn't render in note view)
|
||
|
||
**Diagnostics**
|
||
- [x] Test 5: Browser extensions (loaded count, active/inactive, renderers, tool handlers)
|
||
- [x] Test 6: Service Worker cache (registration, scope, state, cache names, entry counts)
|
||
- [x] Purge Cache button in Debug Log modal (deletes all SW caches, unregisters workers)
|
||
|
||
---
|
||
|
||
## v0.12.0 — File Handling + Vision
|
||
|
||
File input into chat — table stakes for serious use.
|
||
|
||
**Storage Backend**
|
||
- [ ] S3-compatible API (MinIO, Ceph RGW, AWS — anything with S3 API)
|
||
- [ ] Local PVC fallback (single-node / dev)
|
||
- [ ] Admin config: backend selection, endpoint, credentials, size limits
|
||
- [ ] Reused by KBs (v0.14.0) and compaction snapshots (v0.15.0)
|
||
|
||
**Chat Integration**
|
||
- [ ] `attachments` table (metadata in PG, blobs in object store)
|
||
- [ ] Image/file upload in chat messages
|
||
- [ ] Multimodal message assembly for vision-capable models
|
||
- [ ] Text extraction on upload (PDF, DOCX, TXT → tsvector)
|
||
- [ ] Paste-to-upload (clipboard image support)
|
||
- [ ] Document generation output storage (extension → attachment → download)
|
||
|
||
---
|
||
|
||
## v0.13.0 — Web Search + Tools Expansion
|
||
|
||
Built-in tools that extend the tool framework shipped in v0.11.0.
|
||
|
||
- [ ] `web_search` tool: search provider abstraction (SearXNG, Brave, DuckDuckGo)
|
||
- [ ] `url_fetch` tool: retrieve and extract content from URLs
|
||
- [ ] Sidecar or direct HTTP from backend (configurable)
|
||
- [ ] Results injected into context
|
||
- [ ] "Research X and save to my notes" workflow
|
||
|
||
---
|
||
|
||
## v0.14.0 — Knowledge Bases
|
||
|
||
Depends on: embedding model role (v0.10.0), file storage (v0.12.0).
|
||
|
||
- [ ] Embedding pipeline: chunking (recursive, semantic), generation via embedding role
|
||
- [ ] `pgvector` storage and similarity search
|
||
- [ ] `knowledge_bases` table with team scoping
|
||
- [ ] KB document storage via v0.12.0 storage backend
|
||
- [ ] `kb_search` tool (uses existing tool framework)
|
||
- [ ] Context injection in completion flow
|
||
- [ ] Notes embedded too (once pipeline exists)
|
||
- [ ] Per-channel KB toggle
|
||
|
||
---
|
||
|
||
## v0.15.0 — Compaction
|
||
|
||
Depends on: utility model role (v0.10.0).
|
||
|
||
- [ ] Auto-compaction service: background job that calls utility model to summarize
|
||
- [ ] Channel-scoped: triggers when context exceeds threshold
|
||
- [ ] Summaries stored as system messages
|
||
- [ ] Configurable: per-channel opt-in/out, summary model override
|
||
- [ ] Admin controls for resource limits
|
||
|
||
---
|
||
|
||
## v0.16.0 — @mention Routing + Multi-model
|
||
|
||
The channel schema already supports multiple models. This adds routing.
|
||
|
||
- [ ] @mention parsing in messages (users and AI models)
|
||
- [ ] Resolve mentions against `channel_models`
|
||
- [ ] Multi-model channels: fire completions per mentioned model
|
||
- [ ] "Add a model" UI per channel
|
||
- [ ] Enables: second opinions, cross-model conversations
|
||
|
||
---
|
||
|
||
## v0.17.0 — Extension Surfaces + Modes
|
||
|
||
Depends on: extension foundation (v0.11.0).
|
||
See [EXTENSIONS.md §6](EXTENSIONS.md#6-surfaces-modes).
|
||
|
||
- [ ] Surface registration and activation API
|
||
- [ ] Mode selector in sidebar (below brand, above chat history)
|
||
- [ ] `ctx.ui.replace()` / `ctx.ui.restore()` for region management
|
||
- [ ] `surface.activated` / `surface.deactivated` events
|
||
- [ ] Editor mode (extension: file tree + code editor + AI chat split pane)
|
||
- [ ] Article mode (extension: outline + rich text + AI assistant panel)
|
||
|
||
---
|
||
|
||
## v0.18.0 — Smart Routing
|
||
|
||
Depends on: model roles (v0.10.0), usage tracking (v0.10.0).
|
||
Rules-based routing engine — policy, not ML.
|
||
|
||
- [ ] Admin routing policies: "cheapest model with required capabilities",
|
||
"prefer private providers, fallback to cloud", "for team X, use Y"
|
||
- [ ] Fallback chains: primary provider down → next with same model
|
||
- [ ] Cost-aware: factor pricing into routing decisions
|
||
- [ ] Latency-aware: track response times per provider, prefer faster
|
||
|
||
---
|
||
|
||
## v0.19.0 — Multi-Participant Channels + Presence
|
||
|
||
The channel foundation for workflows and real-time collaboration. Today
|
||
channels are single-user direct chats. This release makes channels a
|
||
shared space that multiple actors can inhabit.
|
||
|
||
Depends on: extension surfaces (v0.17.0), WebSocket infrastructure (already exists).
|
||
|
||
**Channel Participants**
|
||
- [ ] `channel_participants` table: `channel_id`, `participant_type` (user, session, persona), `participant_id`, `role` (owner, member, observer), `joined_at`
|
||
- [ ] Channel access queries rewritten: `WHERE user_id = $1` → participant-based lookup
|
||
- [ ] Backward compatible: existing single-user channels auto-have one participant (owner)
|
||
- [ ] Channel `type` column: `direct` (current), `group` (multi-user), `workflow` (staged)
|
||
|
||
**Presence + Real-time**
|
||
- [ ] Typing indicators and presence (who's online, who's in this channel)
|
||
- [ ] Participant join/leave events via WebSocket
|
||
- [ ] Co-editing for extension surfaces (editor mode, article mode)
|
||
- [ ] Cursor sharing, selection highlighting
|
||
|
||
**Channel-Scoped Notes**
|
||
- [ ] Optional `channel_id` FK on notes (NULL = personal notebook, set = channel-attached)
|
||
- [ ] Notes created by tool calls within a channel auto-attach to that channel
|
||
- [ ] Channel notes visible to all participants (not just creator)
|
||
|
||
---
|
||
|
||
## v0.20.0 — Auth Strategy (mTLS/OIDC) + Full RBAC
|
||
|
||
Enterprise auth modes on top of the teams foundation.
|
||
Depends on: vault passphrase support from v0.9.3 (conditional unlock for
|
||
personal providers under external auth).
|
||
|
||
- [ ] `AUTH_MODE` env var: `builtin` | `mtls` | `oidc`
|
||
- [ ] All three resolve to the same internal user model
|
||
- [ ] `auth_source` + `external_id` columns on users table
|
||
- [ ] mTLS: header trust, auto-provision from cert DN
|
||
- [ ] OIDC: Keycloak/Okta token validation, claim extraction, role mapping
|
||
- [ ] Per-source auto-activate policy: auto_activate, default_team, default_role
|
||
- [ ] Vault passphrase prompt for mTLS/OIDC users (v0.9.3 conditional unlock)
|
||
- [ ] Fine-grained permissions: model access, KB write, task create, token budgets
|
||
|
||
**Anonymous / Session Participants**
|
||
- [ ] Session-based identity for unauthenticated visitors (workflow intake)
|
||
- [ ] mTLS: cert CN/fingerprint as stable session identity (no `users` row required)
|
||
- [ ] Non-mTLS: ephemeral session token (cookie or URL token)
|
||
- [ ] Session participants can send messages and trigger tool calls within workflow channels
|
||
- [ ] Session identity visible to assigned team members (cert DN, display name, or "Anonymous #N")
|
||
|
||
---
|
||
|
||
## v0.21.0 — Workflow Engine
|
||
|
||
Team-owned, stage-based process execution. The channel is the runtime,
|
||
personas drive each stage, and the existing tool/notes infrastructure
|
||
handles structured data collection. See [ARCHITECTURE.md — Workflow
|
||
Architecture](ARCHITECTURE.md#workflow-architecture-future--v0210) for
|
||
the conceptual model.
|
||
|
||
Depends on: multi-participant channels (v0.19.0), anonymous identity (v0.20.0).
|
||
|
||
**Workflow Definitions**
|
||
- [ ] `workflows` table: `team_id` (owner), `name`, `description`, `is_active`, `entry_mode` (public_link, team_internal, api)
|
||
- [ ] `workflow_stages` table: `workflow_id`, `ordinal`, `persona_id`, `assignment_team_id`, `form_template` (JSONB, structured note schema), `transition_rules` (JSONB)
|
||
- [ ] Team-admin CRUD: create/edit/delete workflows and stages
|
||
- [ ] Workflow versioning: edits create new version, active instances continue on their version
|
||
|
||
**Workflow Instances (Channels)**
|
||
- [ ] Workflow-typed channels: `type='workflow'`, `workflow_id`, `current_stage`, `stage_version`
|
||
- [ ] Entry point: public link generates workflow channel, adds anonymous participant, binds stage 1 persona
|
||
- [ ] Stage transitions: triggered by AI (tool call), team member (button), or rule (form complete)
|
||
- [ ] On transition: swap active persona, notify assignment team, update channel metadata
|
||
|
||
**AI Intake**
|
||
- [ ] Stage persona drives structured collection via system prompt
|
||
- [ ] Form template → persona system prompt injection ("collect these fields: ...")
|
||
- [ ] Tool calls create channel-scoped notes as form responses
|
||
- [ ] AI determines readiness: "I have everything needed" → trigger transition
|
||
|
||
**Assignment + Queue**
|
||
- [ ] Assignment queue: unassigned workflow channels visible to team members
|
||
- [ ] Claim model: team member claims channel (becomes participant with `member` role)
|
||
- [ ] Round-robin / rule-based auto-assignment (optional, per-stage config)
|
||
- [ ] Assignment notifications via WebSocket + optional webhook
|
||
|
||
**Team Member Collaboration**
|
||
- [ ] Assigned member sees full history (AI intake + artifacts + notes)
|
||
- [ ] Persona remains active — assists both visitor and team member
|
||
- [ ] Member can trigger stage transitions, add notes, invoke tools
|
||
- [ ] Member can reassign to different team member or escalate to different team
|
||
|
||
**Frontend**
|
||
- [ ] Workflow builder UI in team admin panel (stage list, persona picker, form template editor)
|
||
- [ ] Queue view: sidebar section showing unassigned workflow channels for user's teams
|
||
- [ ] Channel header: workflow stage indicator, transition controls
|
||
- [ ] Anonymous visitor view: minimal UI, persona-driven conversation (public link entry)
|
||
|
||
---
|
||
|
||
## v0.22.0 — Tasks / Autonomous Agents
|
||
|
||
Unattended execution — workflows without a human in the loop.
|
||
Depends on: workflow engine (v0.21.0).
|
||
|
||
- [ ] Scheduler + task runner (cron-like triggers for workflow instantiation)
|
||
- [ ] `task_create` tool (AI can spawn sub-workflows)
|
||
- [ ] `type: 'service'` channels: workflow instances with no human participants
|
||
- [ ] Execution budgets: max tokens, max tool calls, max wall-clock per stage
|
||
- [ ] Admin controls for resource limits and kill switches
|
||
- [ ] Completion webhooks (notify external systems when workflow completes)
|
||
|
||
---
|
||
|
||
## TBD (unscheduled — real features, no immediate need)
|
||
|
||
Items that are real but don't yet have a version assignment. Pull left
|
||
based on need.
|
||
|
||
**Extension System — Additional Extensions**
|
||
- Image generation tool (browser or sidecar, provider-agnostic)
|
||
- STT/TTS (browser extension, Web Speech API — personal use case)
|
||
- Code execution sandbox (server-side, container isolation)
|
||
|
||
**Extension System — Server Tiers**
|
||
- Starlark runtime integration (Tier 1 — server sandbox)
|
||
- Sidecar HTTP tool protocol (Tier 2 — container isolation)
|
||
- Server-side tool execution in completion handler
|
||
|
||
**Desktop + Mobile**
|
||
- Desktop app (Tauri)
|
||
- Full PWA with offline capability
|
||
- Mobile-optimized layouts (beyond current responsive)
|
||
|
||
**Data + Portability**
|
||
- Bulk export/import (account data, conversations, settings)
|
||
- ChatGPT/other tool import
|
||
- GDPR-style "download my data"
|
||
- Backup/restore CronJob manifests
|
||
|
||
**UX / Multi-Seat**
|
||
- Per-chat model/preset persistence (server-side). Currently a localStorage
|
||
bandaid (v0.10.2) that doesn't roam across devices. Proper fix: store
|
||
`last_selector_id` in `channels.settings` JSONB on each send. On chat
|
||
selection, resolve from `channel.settings.last_selector_id` → match against
|
||
available models/presets → fall back to `channel.model` → global default.
|
||
Eliminates device-local state entirely. Needs `updateChannel` PATCH on
|
||
completion success (single extra column merge, no migration).
|
||
|
||
**Platform**
|
||
- Rate limiting per user/team/tier (token budgets)
|
||
- Provider health monitoring + key rotation
|
||
- Multi-tenant SaaS mode
|
||
- Plugin/extension marketplace
|
||
- Virtual scroll for long conversations
|
||
- SQLite backend option (single-user / dev)
|