Archived

This repository has been archived on 2026-04-03. You can view files and clone it. You cannot open issues or pull requests or push a commit.

Files

xcaliber 90021157e6 Changeset 0.9.3 (#53 )

2026-02-23 23:55:22 +00:00

22 KiB

Raw Blame History

Roadmap — Chat Switchboard

See also:

ARCHITECTURE.md — Core services design, store layer, scope model
EXTENSIONS.md — Extension system spec (Browser/Starlark/Sidecar tiers, manifests, browser tool bridge, surfaces/modes, model roles)

Versioning (pre-1.0): 0.<major>.<minor> — hotfixes use quad: 0.x.y.z No compatibility guarantees before 1.0.

Dependency Graph

Features have real dependencies. This ordering respects them.

v0.9.x  Stability + Quick UX Wins
          │
v0.9.3  Content Visibility & Block Controls
          │
v0.9.4  API Key Encryption + Vault
          │
v0.10.0 Model Roles (utility + embedding) + Usage Tracking
          │
v0.10.1 Summarize & Continue (first utility role consumer)
          │
  ┌───────┴──────────────┐
  │                      │
v0.11.0 Extension      v0.12.0 File Handling
  Foundation              + Vision
  (browser tier)          │
  │                       │
  ├── Custom renderers    ├── Doc gen outputs
  ├── Image gen tool      ├── KB documents
  ├── STT/TTS             │
  │                       │
  ├───────────────────────┤
  │                       │
v0.13.0 Web Search + Tools Expansion
          │
v0.14.0 Knowledge Bases (embedding role + file storage + pgvector)
          │
v0.15.0 Compaction (utility role + background job)
          │
v0.16.0 @mention Routing + Multi-model
          │
v0.17.0 Extension Surfaces + Modes
          │
v0.18.0 Smart Routing (policy on model roles)
          │
v0.19.0 Live Collaboration (presence, co-editing)
          │
v0.20.0 Auth Strategy (mTLS/OIDC) + Full RBAC
          │
v0.21.0 Tasks / Autonomous Agents

✅ v0.9.0 — Schema Consolidation + BYOK

The "rewrite" release: 21 migrations collapsed to 1, store layer abstraction, and the model pipeline made real for end users.

Schema & Architecture

Consolidated schema (21 migrations → single 001_initial.sql)
Store layer abstraction (all DB via typed interfaces, no raw SQL in handlers)
Persona-as-trust-boundary model (replaces old presets concept)
Scope model (global/team/personal) for configs, personas, models
Backward-compatible API routes (v0.8 → v0.9 field aliases)

Capabilities System

Two-tier resolution: catalog (provider API, per-provider) → heuristic inference
Venice provider: full capability mapping incl. optimizedForCode
ResolveIntrinsic with gap-fill merging (catalog authoritative, heuristic fills gaps)
Preset capability inheritance via GetByModelIDAny fallback

BYOK (Bring Your Own Key)

Auto-fetch models on provider creation
User-facing refresh endpoint + "Refresh Models" button
Personal models in selector with scope=personal badge
Composite model IDs (configId:modelId) prevent cross-provider collisions

Bug Fixes (0.9.0.x)

API key storage (json:"-" tag silently dropped keys)
NULL model_default scan crash (sql.NullString)
Nil slice → JSON null (make([]T, 0))
Frontend error swallowing

Testing

Journey-based integration tests (API-only, 4-actor × 3-provider matrix)
Live Venice API integration test
Frontend JS test suite (107 tests, 27 suites)

✅ v0.9.1 — Capability Architecture + Docs

Removed static known model table (same model ≠ same capabilities across providers)
Resolution chain: catalog → heuristic only, no hardcoded table
Frontend KNOWN_MODELS removed — backend is sole source of truth
Heuristic patterns expanded (qwen3, gpt-5, grok, kimi, minimax, glm-5, gemma-3)
All providers updated to use InferCapabilities() directly
EXTENSIONS.md recovered and updated with Appendix A (Custom Renderers) and Appendix B (Model Roles)
ROADMAP.md, CHANGELOG.md, README.md comprehensive rewrite

v0.9.2 — Quick UX Wins + Hardening

Polish that doesn't need new infrastructure. Ship fast, stabilize.

Collapsible code blocks (long outputs get <details> wrapper)
HTML preview for generated content (sandboxed iframe toggle)
Token count estimate in input area (before send)
"Conversation is getting long" warning (context % thresholds)
New switchboard panel favicon (animated SVG + PNG + ICO)

Hardening

Proxy interception detection (Content-Type checks, actionable error messages)
Environment injection to frontend (window.__ENV__)
Enhanced debug diagnostics (NET:PROXY log type, Content-Type capture, env info)
Team admin audit scoping (see only their team's audit entries)

v0.9.3 — Content Visibility & Block Controls

User control over what's expanded/collapsed in the message flow. Thinking blocks, tool calls, and code blocks all get consistent collapse controls instead of being hidden or auto-formatted without user agency.

Code Blocks

Collapse/expand toggle button on code block header (alongside Copy/Preview)
Auto-collapse at threshold (>15 lines), user can always toggle
Smooth max-height transition instead of <details> wrapping

Thinking Blocks

Always render thinking blocks (never hide from DOM)
showThinking setting controls default open/closed, not visibility
User can always click to expand regardless of setting

Tool Calls

Persist tool call metadata in stored messages
Render tool calls on history load (not just during streaming)
Collapsed by default: "🔧 tool_name → done" with toggle for full I/O
Notes tool: "📝 View note" link that navigates to Notes panel

Notes Panel

Copy button on note cards (same pattern as code block copy)

v0.9.4 — API Key Encryption + Vault

Two-tier encryption with a trust boundary between organizational keys (admin- accessible) and personal BYOK keys (user-only). Personal keys are protected by a per-user encryption key (UEK) that the platform admin cannot recover without the user's passphrase.

Threat Model

Threat	Global/Team keys	Personal BYOK keys
DB backup theft	✅ Protected (env-var AES)	✅ Protected (UEK AES)
SQL injection exfil	✅ Protected	✅ Protected
Admin reads DB directly	⚠️ Admin can decrypt (owns env var)	✅ Admin cannot decrypt (lacks user passphrase)
Admin modifies server code	⚠️ Out of scope (runtime access)	⚠️ Out of scope (runtime access)

Crypto Design

Two encryption paths, one mechanism (AES-256-GCM):

Global/Team keys — encrypted with a key derived from ENCRYPTION_KEY env var. Admin-owned, admin-recoverable. Simple.
Personal keys — encrypted with a per-user User Encryption Key (UEK). The UEK is a random 256-bit key generated at account creation, itself encrypted with a key derived from the user's password via Argon2id.

Password → Argon2id(salt) → Password-Derived Key (PDK)
PDK → AES-256-GCM-Decrypt(encrypted_uek) → UEK (plaintext, session-only)
UEK → AES-256-GCM-Decrypt(api_key_enc) → API key (plaintext, per-request)

The UEK lives in server memory for the duration of the session (alongside the JWT). It is never written to disk, logs, or the database in plaintext.

Schema Changes (migration)

-- Users: vault support
ALTER TABLE users ADD COLUMN encrypted_uek  BYTEA;
ALTER TABLE users ADD COLUMN uek_salt       BYTEA;    -- Argon2id salt
ALTER TABLE users ADD COLUMN uek_nonce      BYTEA;    -- GCM nonce for UEK wrap
ALTER TABLE users ADD COLUMN vault_set      BOOLEAN NOT NULL DEFAULT false;

-- API configs: encrypted key storage
ALTER TABLE api_configs ADD COLUMN api_key_enc  BYTEA;
ALTER TABLE api_configs ADD COLUMN key_nonce    BYTEA;
ALTER TABLE api_configs ADD COLUMN key_scope    TEXT NOT NULL DEFAULT 'global';
-- key_scope: 'global' | 'team' | 'personal'
-- global/team: decrypt with env-var-derived key
-- personal:    decrypt with user's UEK

-- Team providers: same pattern
ALTER TABLE team_providers ADD COLUMN api_key_enc  BYTEA;
ALTER TABLE team_providers ADD COLUMN key_nonce    BYTEA;

Migration backfills: encrypt existing plaintext keys using the env var key (all existing keys are global/team scope). Drop plaintext api_key column after backfill. If ENCRYPTION_KEY is not set, migration refuses to run (fail-safe).

Auth Mode Integration

Builtin auth — password serves double duty: authentication + UEK derivation. No additional UX. On login, derive PDK → unwrap UEK → cache in session. User never knows the vault exists.

mTLS / OIDC (v0.20.0) — authentication is external (cert/token). Vault passphrase is conditionally required:

External auth → auto-authenticated
  ├── Personal providers DISABLED → straight to app, no prompt
  └── Personal providers ENABLED
        ├── First visit  → "Set a vault passphrase to protect your API keys"
        └── Returning    → "Unlock your vault" (passphrase field, pre-filled username)

Terminology: "vault passphrase" for mTLS/OIDC users (distinct from their non-existent account password). Builtin auth users never see this term.

Backend Implementation

server/crypto/vault.go — pure encryption/decryption, no DB awareness:

DeriveKey(password, salt) → key (Argon2id, 64MB memory, 3 iterations)
GenerateUEK() → (uek, error) (crypto/rand, 32 bytes)
WrapUEK(uek, pdk) → (ciphertext, nonce, error)
UnwrapUEK(ciphertext, nonce, pdk) → (uek, error)
Encrypt(plaintext, key) → (ciphertext, nonce, error)
Decrypt(ciphertext, nonce, key) → (plaintext, error)
DeriveEnvKey(envVar) → key (SHA-256 of env var, simple deterministic)

Session UEK caching — UEK stored in a sync.Map keyed by user ID, evicted on logout / token expiry. Completion handler retrieves UEK from cache to decrypt personal provider keys at request time.

Provider resolution — key_scope determines decryption path:

func (s *Store) DecryptAPIKey(config ApiConfig, uekCache *sync.Map) (string, error) {
    switch config.KeyScope {
    case "global", "team":
        return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, envDerivedKey)
    case "personal":
        uek, ok := uekCache.Load(config.UserID)
        if !ok { return "", ErrVaultLocked }
        return crypto.Decrypt(config.APIKeyEnc, config.KeyNonce, uek.([]byte))
    }
}

Edge Cases

Scenario	Behavior
User changes password (builtin)	Re-derive PDK, re-encrypt UEK with new PDK. Personal API keys unchanged (keyed to UEK, not password).
Admin resets user password	Old UEK unrecoverable. Personal keys destroyed. Admin UI warns before confirming.
mTLS user forgets passphrase	Self-service reset: destroys encrypted personal keys. UX: "This will erase your stored API keys."
Admin disables personal providers	Keys stay encrypted in DB (inert). Re-enable later → keys accessible if user remembers passphrase.
`ENCRYPTION_KEY` not set	Startup refuses to run if encrypted keys exist in DB. Fresh install: keys stored plaintext with deprecation warning in logs + admin panel.
`ENCRYPTION_KEY` rotated	Admin CLI tool: `switchboard vault rekey` — re-encrypts all global/team keys with new env key. Personal keys unaffected (keyed to UEK).

Frontend Changes

Provider management (Settings → Providers): no change to UX for builtin auth
mTLS/OIDC splash: conditional vault passphrase prompt (new component)
Admin → Users → Reset Password: warning dialog about personal key destruction
Admin → Settings: indicator for ENCRYPTION_KEY status (set / not set)

Checklist

Crypto core

server/crypto/vault.go — Argon2id derivation, AES-256-GCM wrap/unwrap
server/crypto/vault_test.go — round-trip, wrong-password, corruption
Env-var key derivation + validation on startup

Schema + migration

Migration: add encrypted columns, backfill existing keys, drop plaintext
ENCRYPTION_KEY enforcement (refuse to start if keys exist but no env var)

Auth integration

UEK generation on user creation (builtin auth)
UEK unwrap on login, cache in session
UEK re-wrap on password change
UEK destruction on admin password reset (with confirmation)
Vault passphrase prompt for mTLS/OIDC (when personal providers enabled)

Provider key lifecycle

Encrypt on provider create/update
Decrypt on completion request (per key_scope)
ErrVaultLocked handling (prompt user to unlock)

Admin tooling

switchboard vault rekey CLI command
Admin UI: encryption status indicator
Admin UI: password reset warning

Two pieces of infrastructure that everything downstream depends on.

Model Roles (see EXTENSIONS.md Appendix B)

model_roles in global_settings: named slots (utility, embedding, generation)
Each role: primary provider+model, optional fallback provider+model
Team-level role overrides (team.settings JSONB)
Backend API: roles.Complete(ctx, "utility", messages) with automatic fallback
Backend API: roles.Embed(ctx, "embedding", text) with automatic fallback
Admin UI: role configuration (select provider + model per role)

Usage Tracking

Capture from provider responses: prompt_tokens, completion_tokens, cache_creation_tokens, cache_read_tokens
usage_log table: channel_id, user_id, model_id, provider_id, role (nullable — "utility", "embedding", or NULL for user chat), token counts, timestamp
Model pricing from catalog sync (Venice/OpenAI report pricing)
Admin override pricing (per M tokens, decimal)
Cost calculated at query time (join usage × pricing)
Per-user and per-team usage dashboard in admin panel

v0.10.1 — Summarize & Continue

First consumer of the utility model role. User-triggered conversation compaction — not automatic (that's v0.15.0).

"Summarize and continue" button (appears alongside context length warning)
Calls utility role to summarize conversation history
Summary inserted as system message, earlier messages hidden from context
Original messages preserved in DB (viewable via "show full history" toggle)
Respects team-level role overrides for utility model selection

v0.11.0 — Extension Foundation (Browser Tier)

The platform play. See EXTENSIONS.md for full spec.

Core Loader (EXTENSIONS.md §4, §7)

extensions.js — loader, registry, scoped context (~200 lines)
Extensions.register() API with permission enforcement
Manifest format parser and validator
extensions + extension_user_settings tables
Admin endpoints: install, uninstall, enable/disable
Asset serving: /api/v1/extensions/:id/assets/*path
Extension settings UI in Settings modal

Browser Tool Bridge (EXTENSIONS.md §5)

ctx.tools.handle() API for browser tool registration
Tool schema collection: merge built-in + extension tools
tool.call.* / tool.result.* WebSocket routing
30-second timeout with LLM error recovery
Tool execution via EventBus WaitFor pattern

Custom Renderer API (EXTENSIONS.md Appendix A)

ctx.renderers.register() — pattern + render function
Renderer pipeline in message display (after markdown, before DOM insert)
First-party renderers: collapsible code, HTML preview, mermaid, LaTeX (migrate core v0.9.2 implementations to extension format)

First Extensions

Image generation tool (browser or sidecar, uses generation model role)
STT/TTS (browser extension, Web Speech API — personal use case)

v0.12.0 — File Handling + Vision

File input into chat — table stakes for serious use.

Storage Backend

S3-compatible API (MinIO, Ceph RGW, AWS — anything with S3 API)
Local PVC fallback (single-node / dev)
Admin config: backend selection, endpoint, credentials, size limits
Reused by KBs (v0.14.0) and compaction snapshots (v0.15.0)

Chat Integration

attachments table (metadata in PG, blobs in object store)
Image/file upload in chat messages
Multimodal message assembly for vision-capable models
Text extraction on upload (PDF, DOCX, TXT → tsvector)
Paste-to-upload (clipboard image support)
Document generation output storage (extension → attachment → download)

v0.13.0 — Web Search + Tools Expansion

Built-in tools that use the tool framework shipped in v0.7.x.

web_search tool: search provider abstraction (SearXNG, Brave, DuckDuckGo)
url_fetch tool: retrieve and extract content from URLs
Sidecar or direct HTTP from backend (configurable)
Results injected into context
"Research X and save to my notes" workflow

v0.14.0 — Knowledge Bases

Depends on: embedding model role (v0.10.0), file storage (v0.12.0).

Embedding pipeline: chunking (recursive, semantic), generation via embedding role
pgvector storage and similarity search
knowledge_bases table with team scoping
KB document storage via v0.12.0 storage backend
kb_search tool (uses existing tool framework)
Context injection in completion flow
Notes embedded too (once pipeline exists)
Per-channel KB toggle

v0.15.0 — Compaction

Depends on: utility model role (v0.10.0).

Auto-compaction service: background job that calls utility model to summarize
Channel-scoped: triggers when context exceeds threshold
Summaries stored as system messages
Configurable: per-channel opt-in/out, summary model override
Admin controls for resource limits

v0.16.0 — @mention Routing + Multi-model

The channel schema already supports multiple models. This adds routing.

@mention parsing in messages (users and AI models)
Resolve mentions against channel_models
Multi-model channels: fire completions per mentioned model
"Add a model" UI per channel
Enables: second opinions, cross-model conversations

v0.17.0 — Extension Surfaces + Modes

Depends on: extension foundation (v0.11.0). See EXTENSIONS.md §6.

Surface registration and activation API
Mode selector in sidebar (below brand, above chat history)
ctx.ui.replace() / ctx.ui.restore() for region management
surface.activated / surface.deactivated events
Editor mode (extension: file tree + code editor + AI chat split pane)
Article mode (extension: outline + rich text + AI assistant panel)

v0.18.0 — Smart Routing

Depends on: model roles (v0.10.0), usage tracking (v0.10.0). Rules-based routing engine — policy, not ML.

Admin routing policies: "cheapest model with required capabilities", "prefer private providers, fallback to cloud", "for team X, use Y"
Fallback chains: primary provider down → next with same model
Cost-aware: factor pricing into routing decisions
Latency-aware: track response times per provider, prefer faster

v0.19.0 — Live Collaboration

Depends on: extension surfaces (v0.17.0), WebSocket infrastructure (already exists).

Typing indicators and presence (who's online, who's in this channel)
Co-editing for extension surfaces (editor mode, article mode)
Enables: IDE extension with pair programming, collaborative doc editing
Cursor sharing, selection highlighting

v0.20.0 — Auth Strategy (mTLS/OIDC) + Full RBAC

Enterprise auth modes on top of the teams foundation. Depends on: vault passphrase support from v0.9.3 (conditional unlock for personal providers under external auth).

AUTH_MODE env var: builtin | mtls | oidc
All three resolve to the same internal user model
auth_source + external_id columns on users table
mTLS: header trust, auto-provision from cert DN
OIDC: Keycloak/Okta token validation, claim extraction, role mapping
Per-source auto-activate policy: auto_activate, default_team, default_role
Vault passphrase prompt for mTLS/OIDC users (v0.9.3 conditional unlock)
Fine-grained permissions: model access, KB write, task create, token budgets

v0.21.0 — Tasks / Autonomous Agents

The capstone: everything combined into autonomous workflows.

Scheduler + task runner
task_create tool
type: 'service' channels with no human members
Depends on: completion handler, tool execution, notes, web search
Admin controls for resource limits, execution budgets

TBD (unscheduled — real features, no immediate need)

Items that are real but don't yet have a version assignment. Pull left based on need.

Extension System — Server Tiers

Starlark runtime integration (Tier 1 — server sandbox)
Sidecar HTTP tool protocol (Tier 2 — container isolation)
Server-side tool execution in completion handler

Desktop + Mobile

Desktop app (Tauri)
Full PWA with offline capability
Mobile-optimized layouts (beyond current responsive)

Data + Portability

Bulk export/import (account data, conversations, settings)
ChatGPT/other tool import
GDPR-style "download my data"
Backup/restore CronJob manifests

Platform

Rate limiting per user/team/tier (token budgets)
Provider health monitoring + key rotation
Multi-tenant SaaS mode
Workflow builder (visual DAG for chaining models + tools)
Plugin/extension marketplace
Virtual scroll for long conversations
SQLite backend option (single-user / dev)

22 KiB Raw Blame History Unescape Escape