802 lines
41 KiB
Markdown
802 lines
41 KiB
Markdown
# Roadmap — Chat Switchboard
|
||
|
||
**See also:**
|
||
- [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design, store layer, scope model
|
||
- [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers,
|
||
manifests, browser tool bridge, surfaces/modes, model roles)
|
||
- [EXTENSION-SURFACES.md](EXTENSION-SURFACES.md) — Extension surface authoring guide
|
||
(manifest format, platform API, CSS properties, install workflow)
|
||
- [CHANGELOG.md](../CHANGELOG.md) — Detailed release notes for all completed versions
|
||
|
||
**Versioning (pre-1.0):** `0.<major>.<minor>` — hotfixes use quad: `0.x.y.z`
|
||
No compatibility guarantees before 1.0.
|
||
|
||
---
|
||
|
||
## Dependency Graph
|
||
|
||
Features have real dependencies. This ordering respects them.
|
||
|
||
```
|
||
v0.9.x–v0.27.5 Foundation → Extensions → Surfaces → Auth ✅
|
||
→ Workflows → Tasks → Team Tasks
|
||
(see CHANGELOG.md for full history)
|
||
│
|
||
v0.28.0 Platform Polish
|
||
├─ v0.28.1 Surfaces ICD audit ✅
|
||
├─ v0.28.2 ICD audit — all domains ✅
|
||
├─ v0.28.3 ICD close-out + FE decomp ✅
|
||
├─ v0.28.4 Security tier (red team) ✅
|
||
├─ v0.28.5 Frontend SDK + Pipes ✅
|
||
│ (switchboard-sdk.js, pipe/filter
|
||
│ pipeline, component mounting)
|
||
├─ v0.28.6 Infrastructure
|
||
│ (virtual scroll, Helm, task webhook
|
||
│ UI, system tasks, model prefs)
|
||
└─ v0.28.7 Unified Packaging + Task RBAC
|
||
(.pkg archive, manifest.type,
|
||
task permission gate pre-Starlark)
|
||
│
|
||
v0.29.0 Starlark Sandbox
|
||
(eval loop, permissions, admin UI
|
||
— adds starlark capability to .pkg)
|
||
│
|
||
v0.29.1 API Extensions
|
||
(Starlark route handlers,
|
||
outbound HTTP, requires_provider)
|
||
│
|
||
v0.29.2 DB Extensions
|
||
(namespaced tables, scoped db module,
|
||
declarative schema in manifest)
|
||
│
|
||
v0.29.3 Workflow Forms
|
||
(form_template → real UI, LLM optional,
|
||
.star validation, structured data)
|
||
│
|
||
v0.30.0 Package Lifecycle
|
||
(schema versioning, migrations,
|
||
settings extension point,
|
||
export/import, marketplace)
|
||
│
|
||
v0.30.1 SDK Adoption
|
||
(sw.notes factory, core surface
|
||
migration, fix-once-fix-everywhere)
|
||
│
|
||
v0.30.2 Workflow Packages
|
||
(stage surfaces, custom UI per stage,
|
||
team admin workflow builder)
|
||
│
|
||
v0.31.0 Editor Package
|
||
(E2E proof: rebuild editor as
|
||
installable .pkg, zero
|
||
platform special-casing)
|
||
```
|
||
|
||
---
|
||
|
||
## v0.28.0 — Platform Polish
|
||
|
||
Audit arc, frontend decomposition, security, and infrastructure improvements.
|
||
|
||
### v0.28.1 — Surfaces ICD Audit ✅
|
||
- [x] ICD `surfaces.md` corrected (6 discrepancies: field name, archive format, response shape)
|
||
- [x] Surface ID slug validation + `extractableRelPath` install hardening
|
||
- [x] 19 E2E surface CRUD tests in ICD runner (install, enable/disable, delete, error paths)
|
||
- [x] 22 handler-level tests + 14 store-level tests (PG + SQLite)
|
||
- [x] CI timeout 8m → 12m for PG integration tests
|
||
|
||
### v0.28.2 — ICD Audit: All Domains ✅
|
||
Full audit of every ICD document against code. Goal: ICD becomes the single
|
||
source of truth. After this version, the workflow is ICD-first — update the
|
||
contract before changing code.
|
||
|
||
**Methodology:** Read ICD → grep all source → trace route → handler → store
|
||
(PG + SQLite) → tests → enforce `{"data": [...]}` envelope convention → fix
|
||
ICD → fix code → fix runner → CI green. Final pass picks up straggling
|
||
failures across all domains.
|
||
|
||
**Final score: 469/469 (100%)**
|
||
|
||
**Completed audits:**
|
||
|
||
*Notifications (cs0):*
|
||
- [x] Notifications ICD audit: object shape, query params, response envelopes, WS events
|
||
- [x] Notification type enum sync: remove aspirational types, add implemented types
|
||
(`kb.ready`, `kb.error`, `grant.changed`, `task.budget_exceeded`)
|
||
- [x] Implement `memory.extracted` notification (hook in memory extractor)
|
||
- [x] Implement `user.mentioned` persisted notification (was WS-only)
|
||
- [x] Implement `workflow.claimed` persisted notification (was WS-only)
|
||
- [x] Remove dead `NotifTypeProjectInvite` constant
|
||
- [x] `GET /notifications/preferences` — envelope fix (`missing key "preferences"`)
|
||
|
||
*Knowledge (cs1–cs4):*
|
||
- [x] `knowledge.md` corrected: KB object shape (6 field mismatches), search envelope
|
||
(`data` not `results`), search result fields, status progression (`extracting`
|
||
step), file type support (text-only, not binary), auth annotations
|
||
- [x] Test harness: add `RequirePermission` on `POST /knowledge-bases` and
|
||
`POST /:id/documents`, add missing `GET /:id/documents/:docId/status` and
|
||
`DELETE /:id/documents/:docId` routes
|
||
- [x] New tests: document status polling, document delete, update-empty-body 400,
|
||
permission denial for non-privileged user
|
||
- [x] P0 fix: `SetDiscoverable` authorization — `loadAndAuthorize` + owner/admin
|
||
check, audit log, cross-user security tests (cs2)
|
||
- [x] `ListDiscoverableKBs` response normalization — use `toKBResponse()`,
|
||
shape assertion test (cs3)
|
||
- [x] Dead code removal: `ListGlobal`/`ListForTeam` on KnowledgeBaseStore —
|
||
interface + PG + SQLite (cs4)
|
||
- [x] Move `CreateKB` team role check from raw `database.DB.QueryRow` to
|
||
`stores.Teams.IsTeamAdmin`; remove `database` import (cs4)
|
||
- [x] Team-scoped KB creation test: member denied, team admin succeeds (cs4)
|
||
|
||
*Profile (cs5):*
|
||
- [x] `profile.md` corrected: avatar API (was `multipart/form-data`, actually JSON
|
||
base64), response shapes for all 7 endpoints, auth annotations, field table
|
||
- [x] P0 fix: `GET /settings` returns bare object → wrapped in `{"settings": {...}}`
|
||
- [x] `profileResponse` shape: add `last_login_at` field + dialect-safe time scan
|
||
(`database.ST()` / `database.SNT()`), COALESCE guard on settings column
|
||
- [x] Integration test harness: register all 7 profile/settings routes (was only
|
||
`GET /profile`), remove stale `/avatar` route aliases
|
||
- [x] New `profile_test.go`: GET profile shape, PUT profile (display_name, email,
|
||
duplicate email 409), GET/PUT settings envelope, password change (success,
|
||
wrong current 401, too short 400), avatar delete
|
||
|
||
*Workspaces (cs6):*
|
||
- [x] `workspaces.md` corrected: workspace object shape (added `indexing_enabled`,
|
||
`git_*` fields, `total_bytes` not `storage_bytes`, `owner_type` full enum),
|
||
file entry shape (`is_directory`/`size_bytes` not `is_dir`/`size`), git status
|
||
full shape, git log envelope (`{"data": [...]}`), git commit request (`paths`
|
||
not `files`), archive format query param, auth annotations, all response shapes
|
||
- [x] P0 fix: `GET /workspaces` returns bare array → `{"data": [...]}`
|
||
- [x] P0 fix: `GET /git-credentials` returns bare array → `{"data": [...]}`
|
||
- [x] Fix: `GET .../git/log` returns bare array → `{"data": [...]}` + nil slice guard
|
||
- [x] New `workspace_test.go`: List empty envelope, list with data + shape, root_path
|
||
not exposed, user isolation, GET by ID shape, not found 404, forbidden 403,
|
||
git-credentials empty envelope, auth required (11 tests)
|
||
|
||
*Projects (cs7):*
|
||
- [x] `projects.md` full rewrite: 6 categories of drift (ghost fields, missing fields,
|
||
request shapes, association objects, `omitempty` on computed counts)
|
||
- [x] 14 new Go tests: CRUD shapes, 201 status, envelope, admin list, auth, isolation
|
||
- [x] P0 nil guard: `ListByProject` files `null` → `[]`
|
||
- [x] P0: `ListTeamProviderModels` missing `Type` field — Venice 400 fix
|
||
- [x] ICD runner projects rewritten 9 → 19 tests
|
||
- [x] ICD runner tier-providers: SSE parser, model exclusion, ID fallback fixes
|
||
|
||
### v0.28.3 — ICD Close-out + Frontend Decomposition Prep
|
||
ICD straggler sweep (rolled in from v0.28.2 tail) and frontend
|
||
decomposition groundwork.
|
||
|
||
**ICD close-out (cs0):**
|
||
- [x] `websocket.md` full rewrite: event envelope field (`event` not `type`),
|
||
payload shapes for 11 event types, routing table from code, room model
|
||
documented as planned-not-implemented
|
||
- [x] `auth.md` corrected: login response shape (added `token_type`/`expires_in`,
|
||
removed phantom fields), register admin-approval path
|
||
- [x] `enums.md` corrected: added missing `queued` task run status
|
||
- [x] Presence status gap documented (DB allows `away`, runtime emits only
|
||
`online`/`offline`)
|
||
- [x] VERSION, CHANGELOG, ROADMAP updated
|
||
|
||
**WebSocket delivery fix (cs1):**
|
||
- [x] `workflow.claimed`, `workflow.advanced`, `workflow.completed` use room-scoped
|
||
`Bus.Publish()` but rooms are never joined — events never reach WebSocket
|
||
clients. Convert to `SendToUser()` per channel participant (same pattern as
|
||
`message.created`, `workflow.assigned`).
|
||
|
||
**Frontend decomposition:**
|
||
- [x] JS dependency audit: `docs/JS-DEPENDENCY-AUDIT.md` — full map of 47 files,
|
||
431 globals, cross-file call graph, script load order per surface
|
||
- [x] IIFE extraction: all 47 JS files wrapped, ~168 functions privatized, zero
|
||
implicit globals remain. Explicit `window.*` exports on every file.
|
||
- [x] Cross-file coupling fix: `events.js` no longer reads `_storageKey` from
|
||
`api.js` — uses `API.accessToken` instead
|
||
- [ ] ICD runner gains test tiers: `crud`, `envelope`, `security`, `sdk`
|
||
- [ ] Runner coverage target: 100% of ICD-documented endpoints have at least one test
|
||
- [x] Phase 2: onclick → addEventListener migration (143 dynamic → 5 unconvertible,
|
||
163 data-action delegated via `_uiDispatch` on document.body)
|
||
- [x] Phase 3: Action registry (`sb.js`) — 248 `window.*` exports replaced with
|
||
`sb.register()`/`sb.ns()`. `sb.resolve()` centralized dispatch.
|
||
`sb.call()` template bridge ready for Go template migration.
|
||
- [x] Phase 3b: Go template onclick → `sb.call()` migration (72 handlers
|
||
across 10 templates, 4 unconvertible inline DOM ops, 3 standalone
|
||
pages excluded)
|
||
- [x] Phase 4: ES module conversion — IIFE wrappers removed from all 47
|
||
files, `<script type="module">` across all templates. `sb.js` +
|
||
vendor stay classic. CI test harness fixed (sb.js in VM context).
|
||
- [ ] Phase 5 (future): `import`/`export` statements, remove `window[name]`
|
||
dual-write from `sb.register()`/`sb.ns()`
|
||
|
||
### v0.28.4 — Security Tier (ICD Runner Red Team)
|
||
New `security` tier in ICD test runner. 58 tests, 529/531 pass.
|
||
5 bugs found and fixed. `surfaces/` directory added with build script.
|
||
|
||
**Auth boundary:**
|
||
- [x] JWT revocation: disable user → existing token rejected immediately.
|
||
**Real bug found + fixed** — `UserStatusCache` (30 s TTL) added to
|
||
auth middleware; `is_active` checked on every request.
|
||
- [x] Token reuse after `POST /auth/logout` — revoked refresh token fails
|
||
- [x] Expired token rejection (tampered JWT, alg=none — both rejected)
|
||
- [x] Role escalation: modified JWT claims → 401. **Real bug found + fixed**
|
||
— role now resolved from DB, not JWT claims.
|
||
- [x] User ID substitution: `user_id` in request bodies ignored by handlers
|
||
|
||
**Cross-tenant data access:**
|
||
- [x] User A's token → GET user B's notes, memories, workspaces, BYOK configs, tasks — all 404/403
|
||
- [x] Team member → access other team's providers, personas, tasks — all 403
|
||
- [x] Non-participant → read channel messages, files — 403/404
|
||
|
||
**Input validation:**
|
||
- [x] Path traversal in surface install — rejected (400)
|
||
- [x] SQL injection in search endpoints (notes, users, channels) — 7 payloads, all parameterized
|
||
- [x] XSS payloads in channel titles, note content, persona names — stored safely, frontend sanitizes
|
||
- [x] Oversized request bodies, malformed JSON — handled (400)
|
||
- [x] Null byte in path params → 500 **fixed** — `ValidatePathParams()` middleware (400)
|
||
- [x] Oversized path params → 500 **fixed** — same middleware (400)
|
||
|
||
**Session security:**
|
||
- [x] Visitor session → access channels outside bound workflow — 401
|
||
- [x] Visitor → escalate to authenticated user via crafted headers — 401
|
||
- [ ] Cross-visitor isolation: enforced by `session_auth.go` channel
|
||
binding, but no E2E test (requires public_link workflow setup —
|
||
defer to visitor E2E milestone)
|
||
|
||
**Infrastructure:**
|
||
- [x] `CORS_ALLOWED_ORIGINS` env var — production restricts, dev allows `*`
|
||
- [x] `surfaces/` directory with `build.sh` → `dist/<n>.surface`
|
||
- [x] `hello-dashboard.surface` unpacked into `surfaces/hello-dashboard/`
|
||
|
||
### v0.28.5 — Frontend SDK + Pipes ✅
|
||
`switchboard-sdk.js` — composition layer over existing globals. Surface
|
||
authors consume a single coherent API instead of hunting through 15 JS files.
|
||
Pipe/filter pipeline formalizes the extension hook model into composable,
|
||
priority-ordered transform stages.
|
||
|
||
**SDK core:**
|
||
- [x] `Switchboard.init({ mount })` — idempotent boot: tokens, profile, theme, events, user menu
|
||
- [x] `sw.user`, `sw.isAdmin` — resolved identity
|
||
- [x] `sw.api.get()` / `sw.api.post()` — authenticated REST, no token/base-path management
|
||
- [x] `sw.on(event, fn)` — WebSocket subscription (no `Events` global knowledge)
|
||
- [x] `sw.chat(container, opts)` — drop-in ChatPane (wraps `ChatPane.create()`)
|
||
- [ ] `sw.notes(container, opts)` — stub shipped (warns); notes component
|
||
needs factory refactor before clean mounting (v0.30.1 SDK Adoption)
|
||
- [x] `sw.toast()`, `sw.confirm()` — UI primitives
|
||
- [x] `sw.theme.current`, `sw.theme.on('change', fn)` — theme queries
|
||
- [x] Absorbs cs15 UserMenu band-aid — universal hydration moves into `init()`
|
||
|
||
**Pipe/filter pipeline:**
|
||
|
||
Three stages, each a priority-ordered chain of transform functions.
|
||
Filters receive typed context, return (possibly modified) context.
|
||
Pipeline wires filters in priority order; any filter can halt the chain.
|
||
|
||
- [x] `sw.pipe.pre(priority, fn)` — **pre-send**: transform user message
|
||
before LLM request. Context: `{ message, channel, attachments, metadata }`.
|
||
Runs on both `sendMessage()` and `regenerateMessage()` with `regenerate` flag.
|
||
- [x] `sw.pipe.stream(priority, fn)` — **post-receive**: transform LLM
|
||
response stream chunks as they arrive (sync-only). Context: `{ chunk,
|
||
accumulated, channel, model }`.
|
||
- [x] `sw.pipe.render(priority, fn)` — **post-render**: transform rendered
|
||
HTML/markdown after DOMPurify. Context: `{ element, message, channel }`.
|
||
Replaces `runExtensionPostRender()` with compat shim for existing
|
||
`ctx.renderers.register()` extensions.
|
||
- [x] Pipeline execution engine: ordered dispatch, error isolation (one
|
||
filter throws → skip it, continue chain), timing telemetry per filter
|
||
- [x] `sw.pipe.list()` — introspection: list registered filters by stage
|
||
with priority, source, scope, and timing stats
|
||
- [x] Filter scoping: `{ scope: { channelType: [...] } }` restricts
|
||
execution to specific channel types with zero-overhead skip
|
||
- [ ] Manifest `"pipes"` key: declarative filter registration — deferred
|
||
to v0.28.7 (unified packaging, where manifest keys are extended)
|
||
|
||
**Testing:**
|
||
- [x] ICD runner gains `sdk` test tier: 36 tests — boot, identity, REST,
|
||
events, theme, pipe registration, execution, scoping, halt, error
|
||
isolation, compat shim, introspection
|
||
- [x] Pipe filter tests: priority ordering, halt semantics, error isolation,
|
||
scoped/unscoped execution
|
||
|
||
### v0.28.6 — Infrastructure
|
||
- [ ] Virtual scroll for long conversations (prerequisite for heavy task output channels)
|
||
- [ ] KB auto-injection: platform-registered pre-send pipe filter (`_kb-auto-inject`,
|
||
priority 5), top-K chunk prepend to `ctx.metadata.kb_context`, context budget aware,
|
||
per-channel toggle. First real validation of the v0.28.5 pipeline architecture.
|
||
- [ ] Helm chart (replaces raw k8s manifests, `helm install switchboard ./chart`)
|
||
- [ ] Per-provider model preferences — finalize: make `provider_config_id` required on
|
||
`PUT /models/preferences` (fixes NULL-in-UNIQUE dedup bug), migrate `/models/enabled`
|
||
and `/models/preferences` to `{"data": [...]}` envelope, add integration test coverage
|
||
- [ ] Git credentials settings UI: vault-encrypted per-user credentials, SSH key
|
||
management, per-workspace remote config. Table exists (`git_credentials`), vault
|
||
pattern exists — needs settings section and CRUD handler.
|
||
- [ ] `system.announcement` notification type: admin broadcast endpoint
|
||
(`POST /admin/notifications/broadcast`), fan-out to all active users via
|
||
`NotifyMany`, admin UI for composing announcements
|
||
- [ ] Task webhook trigger UI: schedule selector gains `webhook` option,
|
||
trigger URL displayed + copy button, outbound `webhook_url` + `webhook_secret`
|
||
fields in create/edit form, task-to-task chaining documentation in admin UI.
|
||
Backend fully implemented — this is pure admin UI work.
|
||
- [ ] System task type: `task_type: "system"` — built-in Go function registry
|
||
(`retention_sweep`, `memory_compact`, `session_cleanup`, `staleness_check`).
|
||
Admin creates task, picks function from dropdown, sets cron schedule. Executor
|
||
calls registered Go function instead of LLM completion. Replaces the current
|
||
goroutine-based background jobs with visible, configurable, auditable tasks.
|
||
Existing goroutines kept as fallback until system tasks are validated.
|
||
**Permanent track** — Go registry is not replaced by Starlark (v0.29.0).
|
||
Core platform ops must not break from bad user code. Admin-only by design
|
||
(no RBAC needed — hardcoded to admin role).
|
||
|
||
### v0.28.7 — Unified Packaging + Task RBAC
|
||
Single `.pkg` archive format for both surfaces and extensions. Manifest
|
||
`"type"` field (`surface`, `extension`, `full`) drives install behavior.
|
||
Format first, capabilities later — v0.29.x adds Starlark/DB/API capabilities
|
||
into packages that already install cleanly.
|
||
|
||
Task permission model gates task creation by type and scope. Retroactively
|
||
locks down `action` tasks (webhook relay = data exfiltration risk) and
|
||
pre-positions for `starlark` tasks in v0.29.0.
|
||
|
||
Depends on: v0.28.5 (SDK — pipe/filter registration is part of manifest contract).
|
||
|
||
**Archive format:**
|
||
- [ ] `.pkg` archive: zip containing `manifest.json` + assets (JS, CSS,
|
||
templates, icons). Same structure as `.surface` archives, extended
|
||
manifest schema
|
||
- [ ] `manifest.type` field: `surface` (routes + templates + data loader),
|
||
`extension` (hooks + tools + pipes, no own route), `full` (both).
|
||
Absent `type` defaults to `surface` for backward compat
|
||
- [ ] `manifest.pipes` key: declarative pipe filter registration (pre-send,
|
||
post-receive, post-render) with priority and entry function reference.
|
||
Wired by SDK on extension load
|
||
- [ ] `manifest.tools` key: LLM-callable tool declarations (existing schema,
|
||
now part of unified manifest)
|
||
- [ ] `manifest.hooks` key: EventBus subscriptions (existing schema from
|
||
EXTENSIONS.md, carried forward)
|
||
|
||
**Install infrastructure:**
|
||
- [ ] `POST /admin/packages/install` — unified install endpoint. Reads
|
||
`manifest.type`, validates type-specific requirements, wires subsystems.
|
||
Replaces `POST /admin/surfaces/install` (old route kept as alias)
|
||
- [ ] Validation branches by type: `surface` requires `routes`/`template`,
|
||
`extension` requires at least one of `hooks`/`tools`/`pipes`, `full`
|
||
requires both sets
|
||
- [ ] Admin UI: merge Surfaces + Extensions into single "Packages" section.
|
||
Type badge on each entry. Filter by type. Enable/disable granular
|
||
for `full` packages (disable hooks independently of routes)
|
||
- [ ] `packages` table (or rename `surfaces` → `packages` with migration):
|
||
adds `type` column, retains all existing surface columns
|
||
|
||
**Migration:**
|
||
- [ ] Existing `.surface` archives install unchanged (type defaults to
|
||
`surface`). No re-install required
|
||
- [ ] Existing `extensions` table rows (loose-JS browser extensions) get
|
||
synthetic package wrappers: migration generates `manifest.json` from
|
||
existing DB fields, archives JS entry file into `.pkg` format
|
||
- [ ] `build.sh` in `surfaces/` updated to produce `.pkg` files. Directory
|
||
optionally renamed to `packages/`
|
||
- [ ] ICD runner surface install tests updated to use new endpoint (or alias)
|
||
|
||
**Task RBAC:**
|
||
|
||
Permission gate on task creation by `task_type` and `scope`. Must ship
|
||
before Starlark (v0.29.0) — cannot allow server-side code execution
|
||
without a permission model. Also retroactively locks down `action` tasks.
|
||
|
||
- [ ] `task_permissions` table (or extend `extension_permissions` pattern):
|
||
maps `(user_id | team_id | role) → task_type → allowed`. Default
|
||
permissions seeded on migration:
|
||
- Admin: all types, all scopes
|
||
- Team admin: `prompt`, `workflow` (team scope). `action` denied by default
|
||
- User: `prompt`, `workflow` (personal scope). `action` denied by default
|
||
- `system` always admin-only (hardcoded, not in permissions table)
|
||
- [ ] Create/update task handlers check permission before accepting
|
||
`task_type`. Existing `action` tasks grandfathered (can run, cannot
|
||
be cloned or created without permission). Migration adds warning
|
||
notification to owners of existing `action` tasks
|
||
- [ ] Admin UI: task permission management — per-user and per-team overrides.
|
||
"Allow action tasks for Team X" toggle. Pre-positions `starlark`
|
||
permission type (hidden until v0.29.0 enables it)
|
||
- [ ] `POST /api/v1/tasks` and `PATCH /api/v1/tasks/:id` validate
|
||
`task_type` against caller's permissions. 403 on denied type
|
||
- [ ] ICD tests: permission-denied task creation, team admin boundaries,
|
||
admin override
|
||
|
||
**ICD + docs:**
|
||
- [ ] `packages.md` ICD: install, list, enable/disable, delete, type filtering
|
||
- [ ] `EXTENSIONS.md` updated: packaging section references `.pkg` format,
|
||
loose-JS model documented as legacy
|
||
- [ ] `tasks.md` ICD updated: permission model, per-type RBAC
|
||
- [ ] ICD runner gains `packaging` test tier: install surface-type, install
|
||
extension-type, install full-type, type validation, backward compat
|
||
|
||
**Tier 2 — Medium value (pull into any v0.28.x):**
|
||
- [ ] User-installable extension packages: RBAC-gated `POST /packages/install`
|
||
(non-admin), team-scoped or personal scope, permission review flow.
|
||
Depends on v0.28.7 packaging.
|
||
- [ ] Memory compaction: summarize old memories, confidence decay, prune low-confidence
|
||
- [ ] `capability_match` routing policy ("cheapest model with tool_calling")
|
||
- [ ] New provider types registrable via config file (OpenAI-compatible + custom schema)
|
||
|
||
---
|
||
|
||
## v0.29.0 — Starlark Sandbox + Permission Model
|
||
|
||
Server-side extension runtime. Prove the eval loop, permission pipeline,
|
||
and admin UI before adding capabilities.
|
||
|
||
Depends on: v0.28.0 (platform polish — specifically v0.28.7 unified packaging),
|
||
extension infrastructure (v0.11.0).
|
||
|
||
- [ ] `go.starlark.net` integration (eval loop, timeout, memory ceiling)
|
||
- [ ] Permission model: manifest declarations, admin grant/revoke, DB schema
|
||
(`extension_permissions` table: extension_id, permission, granted, granted_by)
|
||
- [ ] Runtime enforcement: sandbox loader checks granted permissions, injects
|
||
only approved modules. Denied permissions → clean error, not silent no-op.
|
||
- [ ] Admin UI: permission review on install, per-extension grant/revoke toggle,
|
||
audit log of permission changes
|
||
- [ ] Extension lifecycle states: `install → pending_review → approved → active`
|
||
- [ ] Initial modules: `secrets` (vault-backed per-extension), `notifications`
|
||
(emit to users)
|
||
- [ ] Migration: `extension_permissions` table
|
||
- [ ] ICD: update `extensions.md` with Starlark-specific endpoints
|
||
- [ ] `task_type: "starlark"`: task executor gains Starlark code path.
|
||
Task references a `.star` script (inline or from a `.pkg`), executor
|
||
runs it in the sandbox with task context (trigger payload, schedule
|
||
info, previous run data). No LLM, no webhook relay — actual computation.
|
||
RBAC gate from v0.28.7 enforced: `task.starlark` permission enabled
|
||
in `task_permissions` (hidden until this version). Admin-grantable
|
||
to team admins on a per-team basis.
|
||
|
||
**Two-track execution model (permanent):**
|
||
- `system` (v0.28.6): Go function registry — core platform ops
|
||
(`retention_sweep`, `memory_compact`, etc.). Cannot break from
|
||
bad user code. Admin-only, not editable, not versionable.
|
||
- `starlark`: custom admin/team admin tasks — nightly reports,
|
||
data quality checks, integration sync, cleanup scripts. Editable,
|
||
versionable, sandbox-isolated, permission-gated.
|
||
|
||
---
|
||
|
||
## v0.29.1 — API Extensions
|
||
|
||
Starlark route handlers. Surfaces can serve custom JSON endpoints.
|
||
|
||
Depends on: v0.29.0 (sandbox + permissions).
|
||
|
||
- [ ] `api_routes` manifest key: `[{"method": "GET", "path": "/data", "handler": "handlers.list"}]`
|
||
- [ ] Routes mounted at `/s/{surface_id}/api/...`, auth context injected
|
||
- [ ] Starlark request/response primitives (headers, body, status)
|
||
- [ ] `http` outbound module: allowlist/blocklist per extension
|
||
- [ ] `requires_provider` manifest key: extensions declare provider capability
|
||
needs (e.g. `"image_generation"`). Platform resolves via existing provider
|
||
chain (BYOK → team → global → routing policy). Starlark `provider` module
|
||
makes calls on behalf of extension — raw API keys never exposed to sandbox.
|
||
- [ ] Server-side tool execution in completion handler (Starlark tools run
|
||
server-side, browser tools run client-side — unified tool schema)
|
||
- [ ] Multi-file asset routing: `ServeExtensionAsset` path-based lookup
|
||
(replaces inline `_script` for all requests)
|
||
- [ ] Deduplicate tool schema extraction: unify `ListBrowserToolSchemas` and
|
||
`ListTools` into shared helper handling both browser and server-side tools
|
||
|
||
---
|
||
|
||
## v0.29.2 — DB Extensions
|
||
|
||
Namespaced tables for extension data. Create-only (no migrations yet).
|
||
|
||
Depends on: v0.29.1 (API extensions — handlers need somewhere to persist).
|
||
|
||
- [ ] Namespaced tables: `ext_{surface_id}_*` prefix enforced by platform
|
||
- [ ] Declarative schema in manifest: `"tables": [{"name": "items", "columns": [...]}]`
|
||
Platform generates dialect-correct DDL (PG + SQLite)
|
||
- [ ] `db` Starlark module: `query()`, `exec()` scoped to extension tables + declared views
|
||
- [ ] Views as read contract: extension declares views over platform tables with
|
||
explicit column allowlist — never direct access to platform tables
|
||
- [ ] SQL validation: `db` module rejects references to tables outside namespace
|
||
- [ ] Schema creation on install, drop on uninstall
|
||
|
||
---
|
||
|
||
## v0.29.3 — Workflow Forms
|
||
|
||
`form_template` renders as real UI. The LLM moves to the corner —
|
||
present if configured, but not required for data collection. Workflows
|
||
become structured processes, not guided conversations.
|
||
|
||
Depends on: v0.29.2 (DB extensions — structured form data persists in
|
||
`ext_workflow_*` tables, not just `stage_data` JSON blob), v0.29.0
|
||
(Starlark — `.star` validators run on form submission).
|
||
|
||
**Form rendering:**
|
||
- [ ] `form_template` schema: typed field definitions (`text`, `email`,
|
||
`select`, `number`, `date`, `textarea`, `checkbox`, `file`).
|
||
Validation rules per field (`required`, `pattern`, `min`/`max`).
|
||
Current freeform JSON → structured schema with backward compat
|
||
(old JSON treated as `textarea` field prompts for the LLM)
|
||
- [ ] Stage renders as form when `form_template` has typed fields.
|
||
HTML form generation from schema, client-side validation,
|
||
styled with platform CSS custom properties
|
||
- [ ] Form submission → `stage_data` merge (same `MergeWorkflowStageData`
|
||
path as chat-based data collection, but from structured fields
|
||
instead of LLM extraction)
|
||
- [ ] Stage advance on form submit: if `auto_transition: true` and all
|
||
required fields pass validation, advance without human operator
|
||
or LLM involvement
|
||
|
||
**LLM-optional stages:**
|
||
- [ ] Stage without `persona_id` renders form only — no chat pane, no
|
||
completion endpoint, pure data collection
|
||
- [ ] Stage with `persona_id` + `form_template` renders form + chat
|
||
side-by-side: user fills form, LLM assists (auto-fill suggestions,
|
||
field explanations, validation help)
|
||
- [ ] Stage with `persona_id` + no `form_template` — current behavior
|
||
(pure chat-based collection, no change)
|
||
|
||
**Starlark validation:**
|
||
- [ ] `validate` Starlark hook: manifest declares `.star` validator per
|
||
stage. Runs on form submit before advance. Returns field errors
|
||
or approval. Uses `db` module for cross-reference validation
|
||
(e.g., "is this email already in the system?")
|
||
- [ ] `on_submit` Starlark hook: post-validation side effects (create
|
||
records, send notifications, call APIs via `http` module)
|
||
|
||
**Visitor form entry:**
|
||
- [ ] Visitor entry point renders form for non-chat stages (branded
|
||
page with form, no chat widget). Session-scoped same as current
|
||
visitor auth model
|
||
- [ ] Progressive: visitor sees form → submits → next stage (may be
|
||
form or chat or operator review)
|
||
|
||
**Team admin UX:**
|
||
- [ ] Form builder: visual field editor in workflow admin (drag fields,
|
||
set types/validation, preview). Replaces raw JSON textarea for
|
||
`form_template`. Admin-only initially, team admin in v0.30.2.
|
||
|
||
---
|
||
|
||
## v0.30.0 — Package Lifecycle
|
||
|
||
Lifecycle sophistication for the `.pkg` format established in v0.28.7.
|
||
Schema management, settings integration, and ecosystem features.
|
||
|
||
Depends on: v0.29.2 (DB extensions — schema declarations must exist before
|
||
versioning and migrations make sense).
|
||
|
||
- [ ] Install creates schema + mounts routes + serves assets in one operation
|
||
(extends v0.28.7 install with v0.29.2 schema support)
|
||
- [ ] Schema versioning: `"schema_version": N` in manifest
|
||
- [ ] Schema migrations: `"migrations": [{"from": 1, "to": 2, "sql": "..."}]`
|
||
- [ ] Settings extension point: packages declare settings sections in manifest,
|
||
platform settings surface renders them alongside core sections
|
||
- [ ] Export/import format for sharing packages across instances
|
||
- [ ] Package marketplace (discovery, not hosting — instances pull from URLs)
|
||
|
||
---
|
||
|
||
## v0.30.1 — SDK Adoption
|
||
|
||
Migrate core surfaces to consume the SDK instead of raw globals.
|
||
Prerequisite for the Editor Package — if the platform's own surfaces
|
||
don't use `sw.*`, the SDK isn't validated for real.
|
||
|
||
The goal: a bug fix in `sw.notes()` or `sw.chat()` propagates to every
|
||
surface automatically. No more duplicated wiring, no more "fixed on
|
||
chat but broken on editor" class of bugs.
|
||
|
||
Depends on: v0.28.5 (SDK exists), v0.30.0 (package lifecycle — settings
|
||
extension point needed for surface-specific config).
|
||
|
||
**Component factories:**
|
||
- [ ] `sw.notes(container, opts)` — real factory: creates NoteEditor,
|
||
note list, graph panel. Extracted from `notes.js` monolith into
|
||
self-contained mountable component. `opts: { projectId, channelId,
|
||
onLink, standalone }`. Returns instance with `destroy()`.
|
||
- [ ] `sw.chat(container, opts)` — extend existing wrapper with pipe
|
||
integration (stream/render filters auto-wired), model selector
|
||
binding, file upload, typing indicator. Currently thin wrapper
|
||
around `ChatPane.create()` — needs to become the canonical path.
|
||
- [ ] `sw.panels(container, opts)` — PaneContainer composition via SDK.
|
||
Resize, tab management, drag handles. Fixes the "scaling bug lives
|
||
in two places" problem — one implementation, one fix.
|
||
|
||
**Core surface migration:**
|
||
- [ ] Chat surface (`app.js`, `chat.js`): boot via `sw = Switchboard.init()`,
|
||
replace direct `API.*` / `Events.*` / `UI.*` calls with `sw.*` in
|
||
new code paths. Existing code keeps working (globals still exist).
|
||
Progressive — not a rewrite, just new code uses the SDK.
|
||
- [ ] Editor surface (`editor-surface.js`): mount chat pane via `sw.chat()`,
|
||
mount notes via `sw.notes()`, consume `sw.theme.on('change')` for
|
||
CM6 theme sync. Removes duplicated panel resize/scale wiring.
|
||
- [ ] Notes surface: mount via `sw.notes()`, share implementation with
|
||
chat sidebar notes panel.
|
||
- [ ] Settings surface: consume `sw.api.*` and `sw.theme.*`.
|
||
|
||
**Validation:**
|
||
- [ ] Existing ICD runner tests still pass (surfaces behave identically)
|
||
- [ ] ICD runner `sdk` tier extended with component mount/destroy tests
|
||
- [ ] Manual: chat, editor, notes surfaces all function after migration
|
||
- [ ] Bug fix propagation test: fix applied to `sw.notes()` visible on
|
||
both chat sidebar and notes surface simultaneously
|
||
|
||
---
|
||
|
||
## v0.30.2 — Workflow Packages
|
||
|
||
Workflows become packageable. Each stage can serve its own surface —
|
||
form, dashboard, chat, or custom UI. Team admins build and manage
|
||
workflows through a visual builder, not JSON editing.
|
||
|
||
Depends on: v0.29.3 (workflow forms — form rendering exists),
|
||
v0.30.0 (package lifecycle — schema versioning, settings extension point),
|
||
v0.30.1 (SDK adoption — stage surfaces consume `sw.*`).
|
||
|
||
**Stage surfaces:**
|
||
- [ ] A workflow stage can reference a surface (built-in or from a `.pkg`).
|
||
Stage config gains `surface_id` field — platform mounts that surface
|
||
when the stage is active, with stage data injected as context
|
||
- [ ] Built-in stage surfaces: `form` (v0.29.3 form renderer), `chat`
|
||
(current behavior), `review` (operator sees collected data + approve/reject)
|
||
- [ ] Custom stage surfaces: a `.pkg` with `type: "workflow-stage"` provides
|
||
a surface that receives stage data and emits advance/reject signals
|
||
via `sw.workflow.advance(data)` / `sw.workflow.reject(reason)` SDK calls
|
||
- [ ] Stage transitions respect surface completion — surface signals "done"
|
||
with structured data, workflow engine merges and advances
|
||
|
||
**Workflow-as-package:**
|
||
- [ ] `.pkg` with `type: "workflow"`: bundles workflow definition, stage
|
||
surfaces, Starlark handlers, and assets in one installable archive.
|
||
`POST /admin/packages/install` creates the workflow + stages + surfaces
|
||
- [ ] Manifest `"workflow"` key: stages, transitions, form schemas, surface
|
||
refs, on_complete hooks. Replaces manual workflow creation via admin API
|
||
- [ ] Version pinning: installed workflow package tracks its package version.
|
||
Upgrade re-publishes with new version number
|
||
|
||
**Team admin workflow builder:**
|
||
- [ ] Visual stage editor: drag-to-reorder stages, per-stage config panel
|
||
(surface type, persona, form fields, transition rules, assignment)
|
||
- [ ] Form builder integrated per stage (extends v0.29.3 admin form builder)
|
||
- [ ] Preview: team admin can walk through the workflow as a visitor would,
|
||
seeing each stage surface in sequence
|
||
- [ ] Publish/version from the builder UI — no JSON, no API calls
|
||
- [ ] Team admin permission: `workflow.manage` — team admins can create,
|
||
edit, publish workflows for their team. Global admin can create
|
||
global workflows. Regular users can only enter workflows, not build them
|
||
|
||
**SDK extensions:**
|
||
- [ ] `sw.workflow` namespace: `advance(data)`, `reject(reason)`,
|
||
`getData()`, `getStage()`, `onTransition(fn)` — stage surfaces
|
||
interact with the workflow engine through the SDK, not raw API calls
|
||
|
||
---
|
||
|
||
## v0.31.0 — Editor Package
|
||
|
||
Rebuild the editor as an installable `.pkg` package. Zero platform
|
||
special-casing — same `POST /admin/packages/install`, same manifest,
|
||
same `/s/:slug` route as any third-party package. If `pages.go` or
|
||
`main.go` need changes to support it, the platform abstraction is
|
||
wrong, not the editor.
|
||
|
||
Validates the full v0.28.7–v0.30.2 stack E2E: unified packaging, Starlark
|
||
handlers, DB extensions, package lifecycle, SDK adoption, and workflow
|
||
packages all exercised by one real-world package.
|
||
|
||
Depends on: v0.30.2 (workflow packages — editor validates the full stack
|
||
including workflow stage surfaces).
|
||
|
||
**Platform primitives consumed (not owned):**
|
||
- CM6 (core UI primitive — chat input, notes, code blocks all use it)
|
||
- Workspace tools (`workspace_ls`, `workspace_read`, `workspace_write`)
|
||
- Git tools + credentials (v0.28.0 — core settings, vault-encrypted)
|
||
- Provider resolution via `requires_provider` (v0.29.1)
|
||
|
||
**Package delivers:**
|
||
- [ ] Editor `.pkg` archive (type: `full`): manifest, JS, CSS, Starlark handlers
|
||
- [ ] Built separately, installed via admin API — follows exact same
|
||
patterns as any other package
|
||
- [ ] Editor settings section (via v0.30.0 settings extension point):
|
||
keybindings (vim/emacs/standard), font size, editor theme
|
||
- [ ] Editor state persistence via `ext_editor_*` tables: open tabs,
|
||
cursor positions, split layout, per-workspace config
|
||
- [ ] File tree, tab bar, multi-pane layout — all via package JS,
|
||
consuming platform CSS custom properties
|
||
- [ ] Markdown preview (improved renderer, shared with notes surface)
|
||
- [ ] Remove editor from core: delete `surface-editor` template,
|
||
`"editor"` manifest in `pages.go`, `"editor"` data loader,
|
||
`editor-surface.js`
|
||
|
||
---
|
||
|
||
## TBD (unscheduled — real features, no immediate need)
|
||
|
||
Items that are real but don't yet have a version assignment. Pull left
|
||
based on need.
|
||
|
||
**Surfaces + Editor**
|
||
- Article drag-to-reorder outline sections
|
||
- Article-specific AI tools: suggest_outline, expand_section, check_citations
|
||
- Mobile: mode selector collapses to hamburger/bottom nav
|
||
- Surface IDE: built-in surface for building surfaces (Go template editor,
|
||
JS/CSS editor, live preview in sandboxed region)
|
||
- Project-bound surface/pane defaults: project config specifies which panes
|
||
are available and default layout
|
||
|
||
**Workspace + Git**
|
||
- Integration tests: clone, commit, push/pull cycle (requires git binary in CI)
|
||
|
||
**Provider Health + Routing**
|
||
- Latency-aware routing: track response time percentiles, prefer faster providers
|
||
- Cost-aware routing with budget ceiling per request
|
||
- Provider profile editor: key-value config per provider type, preview of effective settings
|
||
- Fallback chain visualizer: drag-to-reorder provider priority per model family
|
||
|
||
**Extensions**
|
||
- Image generation tool (browser or sidecar, provider-agnostic)
|
||
- STT/TTS (browser extension, Web Speech API)
|
||
- Code execution sandbox (server-side, container isolation)
|
||
- Sidecar HTTP tool protocol (Tier 2 — container isolation)
|
||
|
||
**Desktop + Mobile**
|
||
- Desktop app (Tauri)
|
||
- Full PWA with offline capability
|
||
- Mobile-optimized layouts (beyond current responsive)
|
||
|
||
**Data + Portability**
|
||
- Bulk export/import (account data, conversations, settings)
|
||
- ChatGPT/other tool import
|
||
- GDPR-style "download my data"
|
||
- Backup/restore CronJob manifests
|
||
- Admin settings team/user export (user export blocked by vault-encrypted
|
||
BYOK keys; team export needs merge-vs-replace semantics)
|
||
|
||
**UX / Multi-Seat**
|
||
- "Group" scope badge on model selector / KB list: access-source annotation
|
||
so users see _why_ they have access (global, team, group grant)
|
||
- Multi-tenant SaaS mode
|
||
- Plugin/extension marketplace
|
||
|
||
**Projects**
|
||
- `/p/:id` — shared project view
|
||
- Project-specific files: full project-level upload (own endpoint, storage path)
|
||
- Project templates: create from predefined configurations
|
||
- Project creation dialog: replace `prompt()` with proper modal
|
||
- Admin-level project management: cross-instance visibility, ownership reassign
|
||
- Sub-projects / nested hierarchy
|
||
|
||
**Knowledge Bases**
|
||
- Hybrid search: combine vector similarity with full-text `tsvector`, re-rank
|
||
- Semantic chunking: embedding-based boundary detection for smarter splits
|
||
- HNSW index: better query performance than IVFFlat for large datasets
|
||
- Web scraping source: ingest URLs as KB documents (extends url_fetch)
|
||
- Scheduled re-indexing: periodic rebuild when source documents update
|
||
|
||
**Memory**
|
||
- Memory export/import: portable memory format across instances
|
||
- Cross-persona memory sharing (opt-in)
|
||
- Memory analytics: dashboard showing what Personas are learning, growth trends
|
||
|
||
**Platform**
|
||
- Rate limiting per user/team/tier (token budgets beyond current group budgets)
|
||
|
||
---
|
||
|
||
## Pre-1.0 — Code Quality Sweeps
|
||
|
||
Codebase-wide refactors required before 1.0. Not tied to a feature version —
|
||
pull left when touching the affected code, or schedule a dedicated pass.
|
||
|
||
**`SELECT *` → Explicit Column Lists**
|
||
All store implementations use `SELECT * FROM <table>` with positional `rows.Scan()`.
|
||
If a migration adds a column, every scan breaks silently at runtime (wrong column
|
||
mapped to wrong field). Every `scanOne`/`scanMany` helper needs explicit column lists.
|
||
Scope: every store file in `store/postgres/` and `store/sqlite/`.
|
||
|
||
**Raw SQL Hunt**
|
||
Several handlers execute raw SQL via `database.DB.Exec` / `database.TestDB.Exec`
|
||
instead of going through the store interface. These bypass the dialect abstraction
|
||
and are a source of PG/SQLite divergence bugs. Known instances:
|
||
- `handlers/channels.go` — inline channel queries with `pq.Array`
|
||
- `handlers/workflows.go` — inline unique constraint string matching
|
||
- `handlers/participants.go` — `isDuplicateErr` string matching (should use `database.IsUniqueViolation`)
|
||
- `knowledge/` handlers — direct `database.DB.ExecContext` for storage key updates
|
||
Full sweep: grep for `database.DB.Exec`, `database.TestDB.Exec`, `DB.QueryRow` outside
|
||
of `store/` packages. Move to store methods or use existing dialect-safe helpers.
|