# Roadmap — Chat Switchboard

**See also:**
- [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design, store layer, scope model
- [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers,
  manifests, browser tool bridge, surfaces/modes, model roles)
- [EXTENSION-SURFACES.md](EXTENSION-SURFACES.md) — Extension surface authoring guide
  (manifest format, platform API, CSS properties, install workflow)
- [CHANGELOG.md](../CHANGELOG.md) — Detailed release notes for all completed versions

**Versioning (pre-1.0):** `0.<major>.<minor>` — hotfixes use quad: `0.x.y.z`
No compatibility guarantees before 1.0.

---

## Dependency Graph

Features have real dependencies. This ordering respects them.

```
v0.9.x–v0.27.5  Foundation → Extensions → Surfaces → Auth     ✅
                 → Workflows → Tasks → Team Tasks
                 (see CHANGELOG.md for full history)
                        │
                v0.28.0 Platform Polish
                  ├─ v0.28.1  Surfaces ICD audit        ✅
                  ├─ v0.28.2  ICD audit — all domains
                  ├─ v0.28.3  FE decomp + SDK prep
                  ├─ v0.28.4  Security tier (red team)
                  ├─ v0.28.5  Infrastructure
                  │   (virtual scroll, KB auto-inject,
                  │    Helm chart, provider model prefs,
                  │    git credentials UI)
                  └─ v0.28.6  Frontend SDK
                        │
                v0.29.0 Starlark Sandbox
                  (eval loop, permissions, admin UI)
                        │
                v0.29.1 API Extensions
                  (Starlark route handlers,
                   outbound HTTP, requires_provider)
                        │
                v0.29.2 DB Extensions
                  (namespaced tables, scoped db module,
                   declarative schema in manifest)
                        │
                v0.30.0 Surface Packages
                  (full-stack .surface archives:
                   UI + API + schema + settings)
                        │
                v0.31.0 Editor Surface Package
                  (E2E proof: rebuild editor as
                   installable surface, zero
                   platform special-casing)
```

---

## v0.28.0 — Platform Polish

Audit arc, frontend decomposition, security, and infrastructure improvements.

### v0.28.1 — Surfaces ICD Audit ✅
- [x] ICD `surfaces.md` corrected (6 discrepancies: field name, archive format, response shape)
- [x] Surface ID slug validation + `extractableRelPath` install hardening
- [x] 19 E2E surface CRUD tests in ICD runner (install, enable/disable, delete, error paths)
- [x] 22 handler-level tests + 14 store-level tests (PG + SQLite)
- [x] CI timeout 8m → 12m for PG integration tests

### v0.28.2 — ICD Audit: All Domains
Full audit of every ICD document against code. Goal: ICD becomes the single
source of truth. After this version, the workflow is ICD-first — update the
contract before changing code.

**Methodology:** Read ICD → grep all source → trace route → handler → store
(PG + SQLite) → tests → enforce `{"data": [...]}` envelope convention → fix
ICD → fix code → fix runner → CI green. Final pass picks up straggling
failures across all domains.

**Completed audits:**

*Notifications (cs0):*
- [x] Notifications ICD audit: object shape, query params, response envelopes, WS events
- [x] Notification type enum sync: remove aspirational types, add implemented types
      (`kb.ready`, `kb.error`, `grant.changed`, `task.budget_exceeded`)
- [x] Implement `memory.extracted` notification (hook in memory extractor)
- [x] Implement `user.mentioned` persisted notification (was WS-only)
- [x] Implement `workflow.claimed` persisted notification (was WS-only)
- [x] Remove dead `NotifTypeProjectInvite` constant
- [x] `GET /notifications/preferences` — envelope fix (`missing key "preferences"`)

*Knowledge (cs1–cs4):*
- [x] `knowledge.md` corrected: KB object shape (6 field mismatches), search envelope
      (`data` not `results`), search result fields, status progression (`extracting`
      step), file type support (text-only, not binary), auth annotations
- [x] Test harness: add `RequirePermission` on `POST /knowledge-bases` and
      `POST /:id/documents`, add missing `GET /:id/documents/:docId/status` and
      `DELETE /:id/documents/:docId` routes
- [x] New tests: document status polling, document delete, update-empty-body 400,
      permission denial for non-privileged user
- [x] P0 fix: `SetDiscoverable` authorization — `loadAndAuthorize` + owner/admin
      check, audit log, cross-user security tests (cs2)
- [x] `ListDiscoverableKBs` response normalization — use `toKBResponse()`,
      shape assertion test (cs3)
- [x] Dead code removal: `ListGlobal`/`ListForTeam` on KnowledgeBaseStore —
      interface + PG + SQLite (cs4)
- [x] Move `CreateKB` team role check from raw `database.DB.QueryRow` to
      `stores.Teams.IsTeamAdmin`; remove `database` import (cs4)
- [x] Team-scoped KB creation test: member denied, team admin succeeds (cs4)

*Profile (cs5):*
- [x] `profile.md` corrected: avatar API (was `multipart/form-data`, actually JSON
      base64), response shapes for all 7 endpoints, auth annotations, field table
- [x] P0 fix: `GET /settings` returns bare object → wrapped in `{"settings": {...}}`
- [x] `profileResponse` shape: add `last_login_at` field + dialect-safe time scan
      (`database.ST()` / `database.SNT()`), COALESCE guard on settings column
- [x] Integration test harness: register all 7 profile/settings routes (was only
      `GET /profile`), remove stale `/avatar` route aliases
- [x] New `profile_test.go`: GET profile shape, PUT profile (display_name, email,
      duplicate email 409), GET/PUT settings envelope, password change (success,
      wrong current 401, too short 400), avatar delete

*Workspaces (cs6):*
- [x] `workspaces.md` corrected: workspace object shape (added `indexing_enabled`,
      `git_*` fields, `total_bytes` not `storage_bytes`, `owner_type` full enum),
      file entry shape (`is_directory`/`size_bytes` not `is_dir`/`size`), git status
      full shape, git log envelope (`{"data": [...]}`), git commit request (`paths`
      not `files`), archive format query param, auth annotations, all response shapes
- [x] P0 fix: `GET /workspaces` returns bare array → `{"data": [...]}`
- [x] P0 fix: `GET /git-credentials` returns bare array → `{"data": [...]}`
- [x] Fix: `GET .../git/log` returns bare array → `{"data": [...]}` + nil slice guard
- [x] New `workspace_test.go`: List empty envelope, list with data + shape, root_path
      not exposed, user isolation, GET by ID shape, not found 404, forbidden 403,
      git-credentials empty envelope, auth required (11 tests)

**Remaining audits:**

| Priority | ICD | Known failures | Notes |
|----------|-----|----------------|-------|
| 1 | `projects.md` | 0 | Passed clean, needs full trace |
| 2 | `websocket.md` | — | Document event shapes only (not testable in runner) |

- [ ] Notification handler + store tests (PG + SQLite)
- [ ] WebSocket event contract audit: document `message.created`, `workflow.advanced`,
      `presence.changed`, `typing`, `workflow.assigned` event shapes in ICD
- [ ] Final straggler pass: run full ICD runner, fix any remaining failures

### v0.28.3 — Frontend Decomposition + ICD Audit SDK Prep
Frontend JS decomposition and ICD runner hardening. Prepares the codebase
for the SDK layer by untangling the current 15-file global soup.

- [ ] JS dependency audit: map every `window.*` global, cross-file call, init order
- [ ] Extract shared primitives: toast, confirm, theme, API client into importable modules
- [ ] ICD runner gains test tiers: `crud`, `envelope`, `security`, `sdk`
- [ ] Runner coverage target: 100% of ICD-documented endpoints have at least one test
- [ ] Document JS module graph and init sequence

### v0.28.4 — Security Tier (ICD Runner Red Team)
New `security` tier in ICD test runner. Multi-user fixtures already exist.

**Auth boundary:**
- [ ] JWT revocation: disable user via admin API, verify existing token is rejected
      immediately. **This will likely expose a real bug** — JWT middleware validates
      signature + expiry only, does not check `is_active` in DB. Fix: add `is_active`
      check in auth middleware (DB lookup, cached with short TTL).
- [ ] Token reuse after `POST /auth/logout` — revoked refresh token must fail
- [ ] Expired token rejection (craft a token with past expiry)
- [ ] Role escalation: modify JWT claims client-side, attempt admin endpoints
- [ ] User ID substitution: swap `user_id` in request bodies for another user's ID

**Cross-tenant data access:**
- [ ] User A's token → GET user B's notes, memories, workspaces, BYOK configs, task runs
- [ ] Team member → access other team's providers, personas, tasks
- [ ] Non-participant → read channel messages, files

**Input validation:**
- [ ] Path traversal in surface install (covered by cs12, verify E2E)
- [ ] SQL injection in search endpoints (notes, users, channels)
- [ ] XSS payloads in channel titles, note content, persona names (round-trip)
- [ ] Oversized request bodies, malformed JSON

**Session security:**
- [ ] Visitor session → access channels outside bound workflow
- [ ] Visitor → escalate to authenticated user via crafted headers
- [ ] Cross-visitor isolation: visitor A's cookie cannot read visitor B's messages

### v0.28.5 — Infrastructure
- [ ] Virtual scroll for long conversations (prerequisite for heavy task output channels)
- [ ] KB auto-injection: top-K chunk prepend, context budget aware, per-channel toggle
- [ ] Helm chart (replaces raw k8s manifests, `helm install switchboard ./chart`)
- [ ] Per-provider model preferences — finalize: make `provider_config_id` required on
  `PUT /models/preferences` (fixes NULL-in-UNIQUE dedup bug), migrate `/models/enabled`
  and `/models/preferences` to `{"data": [...]}` envelope, add integration test coverage
- [ ] Git credentials settings UI: vault-encrypted per-user credentials, SSH key
  management, per-workspace remote config. Table exists (`git_credentials`), vault
  pattern exists — needs settings section and CRUD handler.
- [ ] `system.announcement` notification type: admin broadcast endpoint
  (`POST /admin/notifications/broadcast`), fan-out to all active users via
  `NotifyMany`, admin UI for composing announcements

### v0.28.6 — Frontend SDK
`switchboard-sdk.js` — composition layer over existing globals. Surface
authors consume a single coherent API instead of hunting through 15 JS files.

- [ ] `Switchboard.init({ mount })` — idempotent boot: tokens, profile, theme, events, user menu
- [ ] `sw.user`, `sw.isAdmin` — resolved identity
- [ ] `sw.api.get()` / `sw.api.post()` — authenticated REST, no token/base-path management
- [ ] `sw.on(event, fn)` — WebSocket subscription (no `Events` global knowledge)
- [ ] `sw.chat(container, opts)` — drop-in ChatPane (wraps `ChatPane.create()`)
- [ ] `sw.notes(container, opts)` — drop-in notes component
- [ ] `sw.toast()`, `sw.confirm()` — UI primitives
- [ ] `sw.theme.current`, `sw.theme.on('change', fn)` — theme queries
- [ ] ICD runner gains `sdk` test tier: validates init, component mounting, event hooks
- [ ] Absorbs cs15 UserMenu band-aid — universal hydration moves into `init()`

**Tier 2 — Medium value (pull into any v0.28.x):**
- [ ] Memory compaction: summarize old memories, confidence decay, prune low-confidence
- [ ] `capability_match` routing policy ("cheapest model with tool_calling")
- [ ] New provider types registrable via config file (OpenAI-compatible + custom schema)

---

## v0.29.0 — Starlark Sandbox + Permission Model

Server-side extension runtime. Prove the eval loop, permission pipeline,
and admin UI before adding capabilities.

Depends on: v0.28.0 (platform polish), extension infrastructure (v0.11.0).

- [ ] `go.starlark.net` integration (eval loop, timeout, memory ceiling)
- [ ] Permission model: manifest declarations, admin grant/revoke, DB schema
      (`extension_permissions` table: extension_id, permission, granted, granted_by)
- [ ] Runtime enforcement: sandbox loader checks granted permissions, injects
      only approved modules. Denied permissions → clean error, not silent no-op.
- [ ] Admin UI: permission review on install, per-extension grant/revoke toggle,
      audit log of permission changes
- [ ] Extension lifecycle states: `install → pending_review → approved → active`
- [ ] Initial modules: `secrets` (vault-backed per-extension), `notifications`
      (emit to users)
- [ ] Migration: `extension_permissions` table
- [ ] ICD: update `extensions.md` with Starlark-specific endpoints

---

## v0.29.1 — API Extensions

Starlark route handlers. Surfaces can serve custom JSON endpoints.

Depends on: v0.29.0 (sandbox + permissions).

- [ ] `api_routes` manifest key: `[{"method": "GET", "path": "/data", "handler": "handlers.list"}]`
- [ ] Routes mounted at `/s/{surface_id}/api/...`, auth context injected
- [ ] Starlark request/response primitives (headers, body, status)
- [ ] `http` outbound module: allowlist/blocklist per extension
- [ ] `requires_provider` manifest key: extensions declare provider capability
      needs (e.g. `"image_generation"`). Platform resolves via existing provider
      chain (BYOK → team → global → routing policy). Starlark `provider` module
      makes calls on behalf of extension — raw API keys never exposed to sandbox.
- [ ] Server-side tool execution in completion handler (Starlark tools run
      server-side, browser tools run client-side — unified tool schema)
- [ ] Multi-file asset routing: `ServeExtensionAsset` path-based lookup
      (replaces inline `_script` for all requests)
- [ ] Deduplicate tool schema extraction: unify `ListBrowserToolSchemas` and
      `ListTools` into shared helper handling both browser and server-side tools

---

## v0.29.2 — DB Extensions

Namespaced tables for extension data. Create-only (no migrations yet).

Depends on: v0.29.1 (API extensions — handlers need somewhere to persist).

- [ ] Namespaced tables: `ext_{surface_id}_*` prefix enforced by platform
- [ ] Declarative schema in manifest: `"tables": [{"name": "items", "columns": [...]}]`
      Platform generates dialect-correct DDL (PG + SQLite)
- [ ] `db` Starlark module: `query()`, `exec()` scoped to extension tables + declared views
- [ ] Views as read contract: extension declares views over platform tables with
      explicit column allowlist — never direct access to platform tables
- [ ] SQL validation: `db` module rejects references to tables outside namespace
- [ ] Schema creation on install, drop on uninstall

---

## v0.30.0 — Surface Packages

The `.surface` archive becomes the full-stack packaging unit.

Depends on: v0.29.2 (DB extensions).

- [ ] Install creates schema + mounts routes + serves assets in one operation
- [ ] Schema versioning: `"schema_version": N` in manifest
- [ ] Schema migrations: `"migrations": [{"from": 1, "to": 2, "sql": "..."}]`
- [ ] Settings extension point: surfaces declare settings sections in manifest,
      platform settings surface renders them alongside core sections
- [ ] Export/import format for sharing surfaces across instances
- [ ] Surface marketplace (discovery, not hosting — instances pull from URLs)

---

## v0.31.0 — Editor Surface Package

Rebuild the editor as an installable surface package. Zero platform
special-casing — same `POST /admin/surfaces/install`, same manifest,
same `/s/:slug` route as any third-party surface. If `pages.go` or
`main.go` need changes to support it, the platform abstraction is
wrong, not the editor.

Validates the full v0.29.x–v0.30.0 stack E2E.

Depends on: v0.30.0 (surface packages with settings extension point).

**Platform primitives consumed (not owned):**
- CM6 (core UI primitive — chat input, notes, code blocks all use it)
- Workspace tools (`workspace_ls`, `workspace_read`, `workspace_write`)
- Git tools + credentials (v0.28.0 — core settings, vault-encrypted)
- Provider resolution via `requires_provider` (v0.29.1)

**Surface package delivers:**
- [ ] Editor `.surface` archive: manifest, JS, CSS, Starlark handlers
- [ ] Built separately, installed via admin API — follows exact same
      patterns as any other surface
- [ ] Editor settings section (via v0.30.0 settings extension point):
      keybindings (vim/emacs/standard), font size, editor theme
- [ ] Editor state persistence via `ext_editor_*` tables: open tabs,
      cursor positions, split layout, per-workspace config
- [ ] File tree, tab bar, multi-pane layout — all via surface JS,
      consuming platform CSS custom properties
- [ ] Markdown preview (improved renderer, shared with notes surface)
- [ ] Remove editor from core: delete `surface-editor` template,
      `"editor"` manifest in `pages.go`, `"editor"` data loader,
      `editor-surface.js`

---

## TBD (unscheduled — real features, no immediate need)

Items that are real but don't yet have a version assignment. Pull left
based on need.

**Surfaces + Editor**
- Article drag-to-reorder outline sections
- Article-specific AI tools: suggest_outline, expand_section, check_citations
- Mobile: mode selector collapses to hamburger/bottom nav
- Surface IDE: built-in surface for building surfaces (Go template editor,
  JS/CSS editor, live preview in sandboxed region)
- Project-bound surface/pane defaults: project config specifies which panes
  are available and default layout

**Workspace + Git**
- Integration tests: clone, commit, push/pull cycle (requires git binary in CI)

**Provider Health + Routing**
- Latency-aware routing: track response time percentiles, prefer faster providers
- Cost-aware routing with budget ceiling per request
- Provider profile editor: key-value config per provider type, preview of effective settings
- Fallback chain visualizer: drag-to-reorder provider priority per model family

**Extensions**
- Image generation tool (browser or sidecar, provider-agnostic)
- STT/TTS (browser extension, Web Speech API)
- Code execution sandbox (server-side, container isolation)
- Sidecar HTTP tool protocol (Tier 2 — container isolation)

**Desktop + Mobile**
- Desktop app (Tauri)
- Full PWA with offline capability
- Mobile-optimized layouts (beyond current responsive)

**Data + Portability**
- Bulk export/import (account data, conversations, settings)
- ChatGPT/other tool import
- GDPR-style "download my data"
- Backup/restore CronJob manifests
- Admin settings team/user export (user export blocked by vault-encrypted
  BYOK keys; team export needs merge-vs-replace semantics)

**UX / Multi-Seat**
- "Group" scope badge on model selector / KB list: access-source annotation
  so users see _why_ they have access (global, team, group grant)
- Multi-tenant SaaS mode
- Plugin/extension marketplace

**Projects**
- `/p/:id` — shared project view
- Project-specific files: full project-level upload (own endpoint, storage path)
- Project templates: create from predefined configurations
- Project creation dialog: replace `prompt()` with proper modal
- Admin-level project management: cross-instance visibility, ownership reassign
- Sub-projects / nested hierarchy

**Knowledge Bases**
- Hybrid search: combine vector similarity with full-text `tsvector`, re-rank
- Semantic chunking: embedding-based boundary detection for smarter splits
- HNSW index: better query performance than IVFFlat for large datasets
- Web scraping source: ingest URLs as KB documents (extends url_fetch)
- Scheduled re-indexing: periodic rebuild when source documents update

**Memory**
- Memory export/import: portable memory format across instances
- Cross-persona memory sharing (opt-in)
- Memory analytics: dashboard showing what Personas are learning, growth trends

**Platform**
- Rate limiting per user/team/tier (token budgets beyond current group budgets)

---

## Pre-1.0 — Code Quality Sweeps

Codebase-wide refactors required before 1.0. Not tied to a feature version —
pull left when touching the affected code, or schedule a dedicated pass.

**`SELECT *` → Explicit Column Lists**
All store implementations use `SELECT * FROM <table>` with positional `rows.Scan()`.
If a migration adds a column, every scan breaks silently at runtime (wrong column
mapped to wrong field). Every `scanOne`/`scanMany` helper needs explicit column lists.
Scope: every store file in `store/postgres/` and `store/sqlite/`.

**Raw SQL Hunt**
Several handlers execute raw SQL via `database.DB.Exec` / `database.TestDB.Exec`
instead of going through the store interface. These bypass the dialect abstraction
and are a source of PG/SQLite divergence bugs. Known instances:
- `handlers/channels.go` — inline channel queries with `pq.Array`
- `handlers/workflows.go` — inline unique constraint string matching
- `handlers/participants.go` — `isDuplicateErr` string matching (should use `database.IsUniqueViolation`)
- `knowledge/` handlers — direct `database.DB.ExecContext` for storage key updates
Full sweep: grep for `database.DB.Exec`, `database.TestDB.Exec`, `DB.QueryRow` outside
of `store/` packages. Move to store methods or use existing dialect-safe helpers.
