# Roadmap — Chat Switchboard **See also:** - [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design, store layer, scope model - [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers, manifests, browser tool bridge, surfaces/modes, model roles) - [EXTENSION-SURFACES.md](EXTENSION-SURFACES.md) — Extension surface authoring guide (manifest format, platform API, CSS properties, install workflow) - [CHANGELOG.md](../CHANGELOG.md) — Detailed release notes for all completed versions **Versioning (pre-1.0):** `0..` — hotfixes use quad: `0.x.y.z` No compatibility guarantees before 1.0. --- ## Dependency Graph Features have real dependencies. This ordering respects them. ``` v0.9.x–v0.27.5 Foundation → Extensions → Surfaces → Auth ✅ → Workflows → Tasks → Team Tasks (see CHANGELOG.md for full history) │ v0.28.0 Platform Polish ├─ v0.28.1 Surfaces ICD audit ✅ ├─ v0.28.2 ICD audit — all domains ✅ ├─ v0.28.3 ICD close-out + FE decomp prep ├─ v0.28.4 Security tier (red team) ├─ v0.28.5 Infrastructure │ (virtual scroll, KB auto-inject, │ Helm chart, provider model prefs, │ git credentials UI) └─ v0.28.6 Frontend SDK │ v0.29.0 Starlark Sandbox (eval loop, permissions, admin UI) │ v0.29.1 API Extensions (Starlark route handlers, outbound HTTP, requires_provider) │ v0.29.2 DB Extensions (namespaced tables, scoped db module, declarative schema in manifest) │ v0.30.0 Surface Packages (full-stack .surface archives: UI + API + schema + settings) │ v0.31.0 Editor Surface Package (E2E proof: rebuild editor as installable surface, zero platform special-casing) ``` --- ## v0.28.0 — Platform Polish Audit arc, frontend decomposition, security, and infrastructure improvements. ### v0.28.1 — Surfaces ICD Audit ✅ - [x] ICD `surfaces.md` corrected (6 discrepancies: field name, archive format, response shape) - [x] Surface ID slug validation + `extractableRelPath` install hardening - [x] 19 E2E surface CRUD tests in ICD runner (install, enable/disable, delete, error paths) - [x] 22 handler-level tests + 14 store-level tests (PG + SQLite) - [x] CI timeout 8m → 12m for PG integration tests ### v0.28.2 — ICD Audit: All Domains ✅ Full audit of every ICD document against code. Goal: ICD becomes the single source of truth. After this version, the workflow is ICD-first — update the contract before changing code. **Methodology:** Read ICD → grep all source → trace route → handler → store (PG + SQLite) → tests → enforce `{"data": [...]}` envelope convention → fix ICD → fix code → fix runner → CI green. Final pass picks up straggling failures across all domains. **Final score: 469/469 (100%)** **Completed audits:** *Notifications (cs0):* - [x] Notifications ICD audit: object shape, query params, response envelopes, WS events - [x] Notification type enum sync: remove aspirational types, add implemented types (`kb.ready`, `kb.error`, `grant.changed`, `task.budget_exceeded`) - [x] Implement `memory.extracted` notification (hook in memory extractor) - [x] Implement `user.mentioned` persisted notification (was WS-only) - [x] Implement `workflow.claimed` persisted notification (was WS-only) - [x] Remove dead `NotifTypeProjectInvite` constant - [x] `GET /notifications/preferences` — envelope fix (`missing key "preferences"`) *Knowledge (cs1–cs4):* - [x] `knowledge.md` corrected: KB object shape (6 field mismatches), search envelope (`data` not `results`), search result fields, status progression (`extracting` step), file type support (text-only, not binary), auth annotations - [x] Test harness: add `RequirePermission` on `POST /knowledge-bases` and `POST /:id/documents`, add missing `GET /:id/documents/:docId/status` and `DELETE /:id/documents/:docId` routes - [x] New tests: document status polling, document delete, update-empty-body 400, permission denial for non-privileged user - [x] P0 fix: `SetDiscoverable` authorization — `loadAndAuthorize` + owner/admin check, audit log, cross-user security tests (cs2) - [x] `ListDiscoverableKBs` response normalization — use `toKBResponse()`, shape assertion test (cs3) - [x] Dead code removal: `ListGlobal`/`ListForTeam` on KnowledgeBaseStore — interface + PG + SQLite (cs4) - [x] Move `CreateKB` team role check from raw `database.DB.QueryRow` to `stores.Teams.IsTeamAdmin`; remove `database` import (cs4) - [x] Team-scoped KB creation test: member denied, team admin succeeds (cs4) *Profile (cs5):* - [x] `profile.md` corrected: avatar API (was `multipart/form-data`, actually JSON base64), response shapes for all 7 endpoints, auth annotations, field table - [x] P0 fix: `GET /settings` returns bare object → wrapped in `{"settings": {...}}` - [x] `profileResponse` shape: add `last_login_at` field + dialect-safe time scan (`database.ST()` / `database.SNT()`), COALESCE guard on settings column - [x] Integration test harness: register all 7 profile/settings routes (was only `GET /profile`), remove stale `/avatar` route aliases - [x] New `profile_test.go`: GET profile shape, PUT profile (display_name, email, duplicate email 409), GET/PUT settings envelope, password change (success, wrong current 401, too short 400), avatar delete *Workspaces (cs6):* - [x] `workspaces.md` corrected: workspace object shape (added `indexing_enabled`, `git_*` fields, `total_bytes` not `storage_bytes`, `owner_type` full enum), file entry shape (`is_directory`/`size_bytes` not `is_dir`/`size`), git status full shape, git log envelope (`{"data": [...]}`), git commit request (`paths` not `files`), archive format query param, auth annotations, all response shapes - [x] P0 fix: `GET /workspaces` returns bare array → `{"data": [...]}` - [x] P0 fix: `GET /git-credentials` returns bare array → `{"data": [...]}` - [x] Fix: `GET .../git/log` returns bare array → `{"data": [...]}` + nil slice guard - [x] New `workspace_test.go`: List empty envelope, list with data + shape, root_path not exposed, user isolation, GET by ID shape, not found 404, forbidden 403, git-credentials empty envelope, auth required (11 tests) *Projects (cs7):* - [x] `projects.md` full rewrite: 6 categories of drift (ghost fields, missing fields, request shapes, association objects, `omitempty` on computed counts) - [x] 14 new Go tests: CRUD shapes, 201 status, envelope, admin list, auth, isolation - [x] P0 nil guard: `ListByProject` files `null` → `[]` - [x] P0: `ListTeamProviderModels` missing `Type` field — Venice 400 fix - [x] ICD runner projects rewritten 9 → 19 tests - [x] ICD runner tier-providers: SSE parser, model exclusion, ID fallback fixes ### v0.28.3 — ICD Close-out + Frontend Decomposition Prep ICD straggler sweep (rolled in from v0.28.2 tail) and frontend decomposition groundwork. **ICD close-out (cs0):** - [x] `websocket.md` full rewrite: event envelope field (`event` not `type`), payload shapes for 11 event types, routing table from code, room model documented as planned-not-implemented - [x] `auth.md` corrected: login response shape (added `token_type`/`expires_in`, removed phantom fields), register admin-approval path - [x] `enums.md` corrected: added missing `queued` task run status - [x] Presence status gap documented (DB allows `away`, runtime emits only `online`/`offline`) - [x] VERSION, CHANGELOG, ROADMAP updated **WebSocket delivery fix (cs1):** - [x] `workflow.claimed`, `workflow.advanced`, `workflow.completed` use room-scoped `Bus.Publish()` but rooms are never joined — events never reach WebSocket clients. Convert to `SendToUser()` per channel participant (same pattern as `message.created`, `workflow.assigned`). **Frontend decomposition:** - [x] JS dependency audit: `docs/JS-DEPENDENCY-AUDIT.md` — full map of 47 files, 431 globals, cross-file call graph, script load order per surface - [x] IIFE extraction: all 47 JS files wrapped, ~168 functions privatized, zero implicit globals remain. Explicit `window.*` exports on every file. - [x] Cross-file coupling fix: `events.js` no longer reads `_storageKey` from `api.js` — uses `API.accessToken` instead - [ ] ICD runner gains test tiers: `crud`, `envelope`, `security`, `sdk` - [ ] Runner coverage target: 100% of ICD-documented endpoints have at least one test - [x] Phase 2: onclick → addEventListener migration (143 dynamic → 5 unconvertible, 163 data-action delegated via `_uiDispatch` on document.body) - [x] Phase 3: Action registry (`sb.js`) — 248 `window.*` exports replaced with `sb.register()`/`sb.ns()`. `sb.resolve()` centralized dispatch. `sb.call()` template bridge ready for Go template migration. - [ ] Phase 3b: Go template onclick → `sb.call()` migration (~50 handlers) - [ ] Phase 4: ES module conversion (remove IIFE wrappers, add import/export) ### v0.28.4 — Security Tier (ICD Runner Red Team) New `security` tier in ICD test runner. Multi-user fixtures already exist. **Auth boundary:** - [ ] JWT revocation: disable user via admin API, verify existing token is rejected immediately. **This will likely expose a real bug** — JWT middleware validates signature + expiry only, does not check `is_active` in DB. Fix: add `is_active` check in auth middleware (DB lookup, cached with short TTL). - [ ] Token reuse after `POST /auth/logout` — revoked refresh token must fail - [ ] Expired token rejection (craft a token with past expiry) - [ ] Role escalation: modify JWT claims client-side, attempt admin endpoints - [ ] User ID substitution: swap `user_id` in request bodies for another user's ID **Cross-tenant data access:** - [ ] User A's token → GET user B's notes, memories, workspaces, BYOK configs, task runs - [ ] Team member → access other team's providers, personas, tasks - [ ] Non-participant → read channel messages, files **Input validation:** - [ ] Path traversal in surface install (covered by cs12, verify E2E) - [ ] SQL injection in search endpoints (notes, users, channels) - [ ] XSS payloads in channel titles, note content, persona names (round-trip) - [ ] Oversized request bodies, malformed JSON **Session security:** - [ ] Visitor session → access channels outside bound workflow - [ ] Visitor → escalate to authenticated user via crafted headers - [ ] Cross-visitor isolation: visitor A's cookie cannot read visitor B's messages ### v0.28.5 — Infrastructure - [ ] Virtual scroll for long conversations (prerequisite for heavy task output channels) - [ ] KB auto-injection: top-K chunk prepend, context budget aware, per-channel toggle - [ ] Helm chart (replaces raw k8s manifests, `helm install switchboard ./chart`) - [ ] Per-provider model preferences — finalize: make `provider_config_id` required on `PUT /models/preferences` (fixes NULL-in-UNIQUE dedup bug), migrate `/models/enabled` and `/models/preferences` to `{"data": [...]}` envelope, add integration test coverage - [ ] Git credentials settings UI: vault-encrypted per-user credentials, SSH key management, per-workspace remote config. Table exists (`git_credentials`), vault pattern exists — needs settings section and CRUD handler. - [ ] `system.announcement` notification type: admin broadcast endpoint (`POST /admin/notifications/broadcast`), fan-out to all active users via `NotifyMany`, admin UI for composing announcements ### v0.28.6 — Frontend SDK `switchboard-sdk.js` — composition layer over existing globals. Surface authors consume a single coherent API instead of hunting through 15 JS files. - [ ] `Switchboard.init({ mount })` — idempotent boot: tokens, profile, theme, events, user menu - [ ] `sw.user`, `sw.isAdmin` — resolved identity - [ ] `sw.api.get()` / `sw.api.post()` — authenticated REST, no token/base-path management - [ ] `sw.on(event, fn)` — WebSocket subscription (no `Events` global knowledge) - [ ] `sw.chat(container, opts)` — drop-in ChatPane (wraps `ChatPane.create()`) - [ ] `sw.notes(container, opts)` — drop-in notes component - [ ] `sw.toast()`, `sw.confirm()` — UI primitives - [ ] `sw.theme.current`, `sw.theme.on('change', fn)` — theme queries - [ ] ICD runner gains `sdk` test tier: validates init, component mounting, event hooks - [ ] Absorbs cs15 UserMenu band-aid — universal hydration moves into `init()` **Tier 2 — Medium value (pull into any v0.28.x):** - [ ] Memory compaction: summarize old memories, confidence decay, prune low-confidence - [ ] `capability_match` routing policy ("cheapest model with tool_calling") - [ ] New provider types registrable via config file (OpenAI-compatible + custom schema) --- ## v0.29.0 — Starlark Sandbox + Permission Model Server-side extension runtime. Prove the eval loop, permission pipeline, and admin UI before adding capabilities. Depends on: v0.28.0 (platform polish), extension infrastructure (v0.11.0). - [ ] `go.starlark.net` integration (eval loop, timeout, memory ceiling) - [ ] Permission model: manifest declarations, admin grant/revoke, DB schema (`extension_permissions` table: extension_id, permission, granted, granted_by) - [ ] Runtime enforcement: sandbox loader checks granted permissions, injects only approved modules. Denied permissions → clean error, not silent no-op. - [ ] Admin UI: permission review on install, per-extension grant/revoke toggle, audit log of permission changes - [ ] Extension lifecycle states: `install → pending_review → approved → active` - [ ] Initial modules: `secrets` (vault-backed per-extension), `notifications` (emit to users) - [ ] Migration: `extension_permissions` table - [ ] ICD: update `extensions.md` with Starlark-specific endpoints --- ## v0.29.1 — API Extensions Starlark route handlers. Surfaces can serve custom JSON endpoints. Depends on: v0.29.0 (sandbox + permissions). - [ ] `api_routes` manifest key: `[{"method": "GET", "path": "/data", "handler": "handlers.list"}]` - [ ] Routes mounted at `/s/{surface_id}/api/...`, auth context injected - [ ] Starlark request/response primitives (headers, body, status) - [ ] `http` outbound module: allowlist/blocklist per extension - [ ] `requires_provider` manifest key: extensions declare provider capability needs (e.g. `"image_generation"`). Platform resolves via existing provider chain (BYOK → team → global → routing policy). Starlark `provider` module makes calls on behalf of extension — raw API keys never exposed to sandbox. - [ ] Server-side tool execution in completion handler (Starlark tools run server-side, browser tools run client-side — unified tool schema) - [ ] Multi-file asset routing: `ServeExtensionAsset` path-based lookup (replaces inline `_script` for all requests) - [ ] Deduplicate tool schema extraction: unify `ListBrowserToolSchemas` and `ListTools` into shared helper handling both browser and server-side tools --- ## v0.29.2 — DB Extensions Namespaced tables for extension data. Create-only (no migrations yet). Depends on: v0.29.1 (API extensions — handlers need somewhere to persist). - [ ] Namespaced tables: `ext_{surface_id}_*` prefix enforced by platform - [ ] Declarative schema in manifest: `"tables": [{"name": "items", "columns": [...]}]` Platform generates dialect-correct DDL (PG + SQLite) - [ ] `db` Starlark module: `query()`, `exec()` scoped to extension tables + declared views - [ ] Views as read contract: extension declares views over platform tables with explicit column allowlist — never direct access to platform tables - [ ] SQL validation: `db` module rejects references to tables outside namespace - [ ] Schema creation on install, drop on uninstall --- ## v0.30.0 — Surface Packages The `.surface` archive becomes the full-stack packaging unit. Depends on: v0.29.2 (DB extensions). - [ ] Install creates schema + mounts routes + serves assets in one operation - [ ] Schema versioning: `"schema_version": N` in manifest - [ ] Schema migrations: `"migrations": [{"from": 1, "to": 2, "sql": "..."}]` - [ ] Settings extension point: surfaces declare settings sections in manifest, platform settings surface renders them alongside core sections - [ ] Export/import format for sharing surfaces across instances - [ ] Surface marketplace (discovery, not hosting — instances pull from URLs) --- ## v0.31.0 — Editor Surface Package Rebuild the editor as an installable surface package. Zero platform special-casing — same `POST /admin/surfaces/install`, same manifest, same `/s/:slug` route as any third-party surface. If `pages.go` or `main.go` need changes to support it, the platform abstraction is wrong, not the editor. Validates the full v0.29.x–v0.30.0 stack E2E. Depends on: v0.30.0 (surface packages with settings extension point). **Platform primitives consumed (not owned):** - CM6 (core UI primitive — chat input, notes, code blocks all use it) - Workspace tools (`workspace_ls`, `workspace_read`, `workspace_write`) - Git tools + credentials (v0.28.0 — core settings, vault-encrypted) - Provider resolution via `requires_provider` (v0.29.1) **Surface package delivers:** - [ ] Editor `.surface` archive: manifest, JS, CSS, Starlark handlers - [ ] Built separately, installed via admin API — follows exact same patterns as any other surface - [ ] Editor settings section (via v0.30.0 settings extension point): keybindings (vim/emacs/standard), font size, editor theme - [ ] Editor state persistence via `ext_editor_*` tables: open tabs, cursor positions, split layout, per-workspace config - [ ] File tree, tab bar, multi-pane layout — all via surface JS, consuming platform CSS custom properties - [ ] Markdown preview (improved renderer, shared with notes surface) - [ ] Remove editor from core: delete `surface-editor` template, `"editor"` manifest in `pages.go`, `"editor"` data loader, `editor-surface.js` --- ## TBD (unscheduled — real features, no immediate need) Items that are real but don't yet have a version assignment. Pull left based on need. **Surfaces + Editor** - Article drag-to-reorder outline sections - Article-specific AI tools: suggest_outline, expand_section, check_citations - Mobile: mode selector collapses to hamburger/bottom nav - Surface IDE: built-in surface for building surfaces (Go template editor, JS/CSS editor, live preview in sandboxed region) - Project-bound surface/pane defaults: project config specifies which panes are available and default layout **Workspace + Git** - Integration tests: clone, commit, push/pull cycle (requires git binary in CI) **Provider Health + Routing** - Latency-aware routing: track response time percentiles, prefer faster providers - Cost-aware routing with budget ceiling per request - Provider profile editor: key-value config per provider type, preview of effective settings - Fallback chain visualizer: drag-to-reorder provider priority per model family **Extensions** - Image generation tool (browser or sidecar, provider-agnostic) - STT/TTS (browser extension, Web Speech API) - Code execution sandbox (server-side, container isolation) - Sidecar HTTP tool protocol (Tier 2 — container isolation) **Desktop + Mobile** - Desktop app (Tauri) - Full PWA with offline capability - Mobile-optimized layouts (beyond current responsive) **Data + Portability** - Bulk export/import (account data, conversations, settings) - ChatGPT/other tool import - GDPR-style "download my data" - Backup/restore CronJob manifests - Admin settings team/user export (user export blocked by vault-encrypted BYOK keys; team export needs merge-vs-replace semantics) **UX / Multi-Seat** - "Group" scope badge on model selector / KB list: access-source annotation so users see _why_ they have access (global, team, group grant) - Multi-tenant SaaS mode - Plugin/extension marketplace **Projects** - `/p/:id` — shared project view - Project-specific files: full project-level upload (own endpoint, storage path) - Project templates: create from predefined configurations - Project creation dialog: replace `prompt()` with proper modal - Admin-level project management: cross-instance visibility, ownership reassign - Sub-projects / nested hierarchy **Knowledge Bases** - Hybrid search: combine vector similarity with full-text `tsvector`, re-rank - Semantic chunking: embedding-based boundary detection for smarter splits - HNSW index: better query performance than IVFFlat for large datasets - Web scraping source: ingest URLs as KB documents (extends url_fetch) - Scheduled re-indexing: periodic rebuild when source documents update **Memory** - Memory export/import: portable memory format across instances - Cross-persona memory sharing (opt-in) - Memory analytics: dashboard showing what Personas are learning, growth trends **Platform** - Rate limiting per user/team/tier (token budgets beyond current group budgets) --- ## Pre-1.0 — Code Quality Sweeps Codebase-wide refactors required before 1.0. Not tied to a feature version — pull left when touching the affected code, or schedule a dedicated pass. **`SELECT *` → Explicit Column Lists** All store implementations use `SELECT * FROM ` with positional `rows.Scan()`. If a migration adds a column, every scan breaks silently at runtime (wrong column mapped to wrong field). Every `scanOne`/`scanMany` helper needs explicit column lists. Scope: every store file in `store/postgres/` and `store/sqlite/`. **Raw SQL Hunt** Several handlers execute raw SQL via `database.DB.Exec` / `database.TestDB.Exec` instead of going through the store interface. These bypass the dialect abstraction and are a source of PG/SQLite divergence bugs. Known instances: - `handlers/channels.go` — inline channel queries with `pq.Array` - `handlers/workflows.go` — inline unique constraint string matching - `handlers/participants.go` — `isDuplicateErr` string matching (should use `database.IsUniqueViolation`) - `knowledge/` handlers — direct `database.DB.ExecContext` for storage key updates Full sweep: grep for `database.DB.Exec`, `database.TestDB.Exec`, `DB.QueryRow` outside of `store/` packages. Move to store methods or use existing dialect-safe helpers.