gobha
fcb998bff9
Co-authored-by: gobha <jasafpro@gmail.com> Co-committed-by: gobha <jasafpro@gmail.com>
818 lines
37 KiB
Markdown
818 lines
37 KiB
Markdown
# Roadmap — Chat Switchboard
|
||
|
||
**See also:**
|
||
- [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design, store layer, scope model
|
||
- [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers,
|
||
manifests, browser tool bridge, surfaces/modes, model roles)
|
||
- [EXTENSION-SURFACES.md](EXTENSION-SURFACES.md) — Extension surface authoring guide
|
||
(manifest format, platform API, CSS properties, install workflow)
|
||
- [CHANGELOG.md](../CHANGELOG.md) — Detailed release notes for all completed versions
|
||
|
||
**Versioning (pre-1.0):** `0.<major>.<minor>` — hotfixes use quad: `0.x.y.z`
|
||
No compatibility guarantees before 1.0.
|
||
|
||
---
|
||
|
||
## Dependency Graph
|
||
|
||
Two parallel implementation tracks converge at v0.50.0 (MVP). The
|
||
extension track builds the package ecosystem and workflow capabilities.
|
||
The operations track builds production readiness for multi-team
|
||
deployment. Neither blocks the other until the MVP gate.
|
||
|
||
```
|
||
v0.9.x–v0.28.7 Foundation through Platform Polish ✅
|
||
(see CHANGELOG.md for full history)
|
||
│
|
||
v0.28.8 ICD Green Board ✅
|
||
│
|
||
┌───────────────┼───────────────┐
|
||
│ │ │
|
||
Extension Track UI Rewrite Operations Track
|
||
│ Track │
|
||
v0.29.0 ✅ │ v0.32.0 ✅
|
||
v0.29.1 ✅ │ v0.33.0 ✅
|
||
v0.29.2 ✅ │ v0.34.0 ✅
|
||
v0.29.3 ✅ │ │
|
||
v0.30.0 ✅ │ │
|
||
v0.30.1 ✅ │ │
|
||
v0.30.2 ✅ │ │
|
||
v0.31.0 ✅ │ │
|
||
v0.31.1 ✅ │ │
|
||
v0.31.2 ✅ │ │
|
||
│ │ v0.35.0 Workflow Product
|
||
│ │ │
|
||
│ │ v0.36.0 Full OpenAPI ✅
|
||
│ │ │
|
||
│ v0.37.1 Perm Audit ✅ │
|
||
│ v0.37.2 Primitives ✅ │
|
||
│ v0.37.3 SDK │
|
||
│ v0.37.4 Shell │
|
||
│ v0.37.5–13 Surfaces ✅ │
|
||
│ v0.37.14–18 Surfaces │
|
||
│ │ │
|
||
══════╪════════════╪═════════════════╪══════
|
||
│ MVP v0.50.0 │
|
||
│ (all three tracks complete │
|
||
│ + mobile + deploy docs) │
|
||
══════╪════════════╪═════════════════╪══════
|
||
│
|
||
v0.50.0+ Rich Media + Beyond
|
||
(image gen, code sandbox,
|
||
STT/TTS, desktop app)
|
||
```
|
||
|
||
## Completed: v0.28.0 — Platform Polish
|
||
|
||
Audit arc, frontend decomposition, security, infrastructure. Eight
|
||
sub-versions, all complete. See CHANGELOG.md for detailed release notes.
|
||
|
||
| Version | Summary | Key Deliverables |
|
||
|---------|---------|------------------|
|
||
| v0.28.1 | Surfaces ICD Audit ✅ | 6 ICD fixes, 19 E2E tests, 36 Go tests |
|
||
| v0.28.2 | ICD Audit: All Domains ✅ | 469/469 (100%), full methodology documented |
|
||
| v0.28.3 | ICD Close-out + FE Decomp ✅ | WebSocket ICD rewrite, 47 JS files → ES modules, `sb.js` registry |
|
||
| v0.28.4 | Security Tier ✅ | 58 red-team tests, 5 real bugs found+fixed, JWT role from DB |
|
||
| v0.28.5 | Frontend SDK + Pipes ✅ | `switchboard-sdk.js`, 3-stage pipe/filter pipeline, 36 SDK tests |
|
||
| v0.28.6 | Infrastructure ✅ | Virtual scroll, Helm chart, system tasks, git keygen, broadcast |
|
||
| v0.28.7 | Unified Packaging ✅ | `.pkg` format, `packages` table, task RBAC, `task.starlark` gate |
|
||
|
||
**Deferred from v0.28.x (relocated with version pins):**
|
||
- `sw.notes()` factory → v0.30.1
|
||
- Phase 5 FE decomp (`import`/`export`) → v0.30.1
|
||
- Cross-visitor isolation E2E test → v0.29.3
|
||
- Admin UI task permission management → v0.29.0
|
||
- ICD runner packaging test tier → v0.29.0
|
||
|
||
---
|
||
|
||
## v0.28.8 — ICD Green Board ✅
|
||
|
||
Close out pre-existing ICD test failures and infrastructure issues
|
||
discovered during the green board push.
|
||
|
||
Depends on: v0.28.7 (unified packaging).
|
||
|
||
**Root cause analysis (discovered during deployment):**
|
||
|
||
The 502/503 cascade that failed 8–10 ICD tests across provider, BYOK,
|
||
and SDK tiers was caused by **OOMKilled** — the dev backend pod had a
|
||
256Mi memory limit. The Go process peaked at ~216Mi during the 569-test
|
||
ICD suite and exceeded 256Mi during SSE completion streams, triggering
|
||
exit code 137. Traefik returned `503 no available server` until the pod
|
||
restarted (~15s).
|
||
|
||
Memory profile (measured on cluster via `kubectl top`):
|
||
- 17Mi idle after fresh start
|
||
- 216Mi peak during full ICD suite (569 requests, SSE streams)
|
||
- 60Mi post-test (Go GC ran, heap arena retained)
|
||
- 34Mi settled (~3 min, OS reclaimed freed pages via MADV_DONTNEED)
|
||
- Zero leak — memory returns to baseline, flat steady state
|
||
|
||
**Infrastructure fixes:**
|
||
- [x] Backend memory limits: dev/test 128Mi/256Mi → 256Mi/512Mi
|
||
- [x] Batched `UpsertFromSync`: single transaction, prepared
|
||
`INSERT ON CONFLICT DO UPDATE` (300 round-trips → 2, PG + SQLite)
|
||
- [x] DB connection lifecycle: `SetConnMaxLifetime(5m)` +
|
||
`SetConnMaxIdleTime(1m)`
|
||
- [x] HTTP transport pool: `sync.Map` keyed by proxy config
|
||
- [x] Provider sync timeout: `context.WithTimeout(30s)`, 504 on timeout
|
||
- [x] ICD runner: `apiPostRetry`, environment-aware CORS test,
|
||
ticket exchange verification test
|
||
|
||
**Security transport:**
|
||
- [x] CORS startup warning, `GetAllowedOrigins()`, WebSocket `CheckOrigin`
|
||
- [x] WebSocket ticket exchange: `POST /api/v1/ws/ticket`, `WsAuth`
|
||
middleware, `TicketStore` with TTL reaper
|
||
- [x] `events.js` async ticket-first auth with legacy fallback
|
||
|
||
**Kubernetes:**
|
||
- [x] Traefik retry `Middleware` CRD + Helm template
|
||
- [x] RBAC for Gitea runner → `traefik.io` middleware resources
|
||
- [x] CI non-fatal middleware apply with post-success ingress annotation
|
||
|
||
---
|
||
|
||
## Extension Track
|
||
|
||
Sequential. Each version builds on the previous. Delivers the package
|
||
ecosystem, workflow capabilities, and SDK-based surface architecture.
|
||
|
||
### v0.29.0 — Starlark Sandbox + Permission Model ✅
|
||
|
||
Server-side extension runtime. Eval loop, permission pipeline,
|
||
pre-completion filter chain, and admin review workflow.
|
||
|
||
Depends on: v0.28.8.
|
||
|
||
**Phase 0 — Store cleanup (prerequisite): ✅**
|
||
- [x] Raw SQL hunt: all ~242 `database.DB.*` calls outside `store/`
|
||
migrated to store interface methods (CS0–CS7b, 12 changesets)
|
||
- [x] CI green on both PG and SQLite pipelines
|
||
- [x] ICD runner: 579/580 pass, 1 expected skip
|
||
- [x] Documented exception: `events/pg_broadcast.go` (`pg_notify`,
|
||
PG-only, no store abstraction needed)
|
||
|
||
**Phase 1 — Starlark runtime: ✅**
|
||
- [x] Pre-completion filter chain: composable `PreCompletionFilter`
|
||
interface + `Chain` registry. KB auto-inject refactored as first
|
||
built-in filter. Extension filters register at order 100+ (CS0)
|
||
- [x] `go.starlark.net` integration: sandboxed eval with step limits
|
||
(1M ops default), context timeout, captured print output,
|
||
disabled `load()`. `MakeModule` helper for Go→Starlark (CS1)
|
||
- [x] Permission model: `extension_permissions` table (in 016),
|
||
`status` column on `packages` (`active`/`pending_review`/
|
||
`suspended`). Manifest `"permissions"` array parsed on install.
|
||
Admin review, grant, revoke, grant-all endpoints (CS2)
|
||
- [x] Runtime enforcement: `Runner.buildModules()` injects only
|
||
granted modules into sandbox namespace (CS3)
|
||
- [x] Extension lifecycle: `install → pending_review → grant-all →
|
||
active`, revoke → `suspended`. Auto-transitions on grant/revoke.
|
||
- [x] Initial modules: `secrets` (GlobalConfig-backed, per-package
|
||
key-value store, admin CRUD), `notifications` (wraps
|
||
notification service, `send(user_id, title, body?, type?)`) (CS3)
|
||
- [x] `task_type: "starlark"`: executor `executeStarlark` loads
|
||
package by ID, calls `on_run()` entry point via runner.
|
||
RBAC gate `task.starlark` enforced. `system_function` field
|
||
holds package ID (CS4)
|
||
- [x] KB auto-injection: server-side pre-completion filter chain.
|
||
Reference implementation for the filter model Starlark
|
||
extensions mirror via `on_pre_completion(ctx)` (CS0+CS3)
|
||
- [x] Starlark filter discovery: `DiscoverStarlarkFilters` scans
|
||
active packages with `filters.pre_completion` grant (CS3)
|
||
- [x] ICD runner: `packaging` test tier — 18 tests covering
|
||
permission lifecycle + secrets CRUD (CS5)
|
||
|
||
### v0.29.1 — API Extensions ✅
|
||
|
||
Starlark route handlers. Surfaces serve custom JSON endpoints.
|
||
|
||
Depends on: v0.29.0.
|
||
|
||
- [x] `api_routes` manifest key, mounted at `/s/{id}/api/...`
|
||
- [x] Starlark request/response primitives
|
||
- [x] `http` outbound module with allowlist/blocklist
|
||
- [x] `requires_provider` manifest key (provider resolution via BYOK chain)
|
||
- [x] `capability_match` routing policy (cheapest model with required caps)
|
||
- [x] Config-file provider types (JSON, no code deploy)
|
||
|
||
**Deferred to v0.29.2:**
|
||
- Server-side tool execution in completion handler (requires tool
|
||
registry integration with sandbox; aligns with DB extensions scope)
|
||
|
||
### v0.29.2 — DB Extensions ✅
|
||
|
||
Namespaced tables for extension data. Structured API, not raw SQL.
|
||
Server-side tool execution (deferred from v0.29.1) included.
|
||
|
||
Depends on: v0.29.1.
|
||
|
||
- [x] `ext_{id}_*` tables, dialect-correct DDL (PG + SQLite)
|
||
- [x] `db` Starlark module: structured `query/insert/update/delete/list_tables/view`
|
||
(structured API instead of raw `exec()` — prevents SQL injection)
|
||
- [x] Views as read contract over platform tables (`ext_view_users`, `ext_view_channels`)
|
||
- [x] Schema creation on install, drop on uninstall
|
||
- [x] Server-side tool execution in completion handler (deferred from v0.29.1)
|
||
|
||
### v0.29.3 — Workflow Forms ✅
|
||
|
||
`form_template` renders as real UI. LLM is optional for data collection.
|
||
|
||
Depends on: v0.29.2, v0.29.0 (Starlark validators).
|
||
|
||
- [x] Typed `form_template` schema (`text`, `email`, `select`, `number`,
|
||
`date`, `textarea`, `checkbox`, `file`) with validation rules
|
||
- [x] Stage renders as form when `form_template` has typed fields
|
||
- [x] LLM-optional stages: form-only, form+chat, chat-only
|
||
- [x] Starlark `validate` / `on_submit` hooks
|
||
- [x] Visitor form entry (branded page, no chat widget)
|
||
- [x] Form builder in workflow admin (visual field editor)
|
||
- [x] Cross-visitor isolation E2E test (deferred from v0.28.4)
|
||
|
||
### v0.30.0 — Package Lifecycle ✅
|
||
|
||
Lifecycle sophistication for `.pkg` format.
|
||
|
||
Depends on: v0.29.2.
|
||
|
||
- [x] Schema versioning + migrations in manifest
|
||
- [x] Settings extension point (packages declare settings sections)
|
||
- [x] Export/import format for cross-instance sharing
|
||
- [x] Package marketplace (discovery, not hosting)
|
||
- [x] User-installable packages (RBAC-gated, team/personal scope)
|
||
|
||
### v0.30.1 — SDK Adoption ✅
|
||
|
||
Migrate core surfaces to `sw.*` SDK.
|
||
|
||
Depends on: v0.28.5, v0.30.0.
|
||
|
||
- [x] `sw.notes()`, `sw.chat()`, `sw.panels()` real factories
|
||
- [x] Core surface migration: chat, editor, notes, settings
|
||
- [x] Phase 5 FE decomp: `import`/`export` statements
|
||
- [x] Memory compaction: summarize, confidence decay, prune
|
||
|
||
### v0.30.2 — Workflow Packages ✅
|
||
|
||
Workflows as installable packages. Visual builder for team admins.
|
||
|
||
Depends on: v0.29.3, v0.30.0, v0.30.1.
|
||
|
||
- [x] Stage surfaces: form, chat, review, custom `.pkg`
|
||
- [x] `.pkg` type `"workflow"` bundles definition + surfaces + handlers
|
||
- [x] Team admin workflow builder (visual, no JSON editing)
|
||
- [x] `sw.workflow` SDK namespace
|
||
- [x] ICD test fixes (auth rate limiter burst 30→8, vault UEK pre-warm)
|
||
- [x] Starlark `workflow.*` module (get_definition, get_stage_data, advance, reject)
|
||
- [x] Workflow package export/import (.pkg round-trip)
|
||
- [x] E2E tests: export/import, surface_pkg_id persistence
|
||
|
||
### v0.31.0 — Editor Package + SDK Composability ✅
|
||
|
||
E2E proof: rebuild editor as installable `.pkg`. Zero platform
|
||
special-casing. Full SDK composability — every UI piece created
|
||
through `sw.*` factories, no component duplication.
|
||
|
||
Depends on: v0.30.2.
|
||
|
||
**Editor Package (CS0–CS2):**
|
||
- [x] Editor `.pkg` (type: `full`), settings via extension point
|
||
- [x] State persistence via localStorage (per-user, per-workspace UI state)
|
||
- [x] Remove editor from core (`surface-editor` template, data loader)
|
||
- [x] E2E tests: install, settings, export/import round-trip, core removal
|
||
- [x] Self-mounting components (`Component.mount()` is canonical entry point)
|
||
|
||
**SDK Composability (CS3–CS4):**
|
||
- [x] Delete `NoteEditor` — `NotePanel` is single canonical notes component
|
||
- [x] Platform scripts in `base.html` (chat-pane, note-panel, note-graph available to all surfaces)
|
||
- [x] `UserMenu.mount()` — self-contained, works in any container
|
||
- [x] `sw.userMenu()` — mount user menu anywhere, `flyout: 'up'|'down'`
|
||
- [x] `sw.chat()` uses `ChatPane.mount()` (not manual DOM + `.create()`)
|
||
- [x] `sw.fileTree()`, `sw.codeEditor()`, `sw.layout()` SDK wrappers
|
||
- [x] `sw.menu()`, `sw.dropdown()`, `sw.toolbar()`, `sw.tabs()` UI primitives
|
||
- [x] `--bg-elevated` / `--border-elevated` CSS tokens for floating panels
|
||
- [x] Editor package uses SDK exclusively (`sw.layout`, `sw.fileTree`, `sw.codeEditor`, `sw.chat`, `sw.notes`, `sw.userMenu`)
|
||
- [x] Nginx caching: JS/CSS use `must-revalidate` (not `immutable`) for dev reload safety
|
||
- [x] NotePanel: pagination, select mode, sticky selection bar, `note-panel-root` class (no PanelRegistry conflict)
|
||
|
||
**Known remaining (visual polish, not blocking):**
|
||
- [x] UserMenu flyout contrast on very dark backgrounds — fixed in v0.31.1 CS0+CS1
|
||
- [x] Editor chat pane duplicates streaming/model-selector logic — absorbed into ChatPane in v0.31.1 CS2
|
||
|
||
|
||
### v0.31.1 — SDK Exercise Surface ✅
|
||
|
||
Build a dashboard surface package exercising every `sw.*` primitive
|
||
with zero component CSS overrides. Fix 4 SDK bugs first.
|
||
|
||
Depends on: v0.31.0.
|
||
|
||
**SDK Bug Fixes (CS0–CS2):**
|
||
- [x] Flyout unification — 3 competing CSS systems consolidated into `.sw-menu-flyout` in primitives.css
|
||
- [x] Flyout positioning — `position:fixed` escape hatch for overflow-clipped extension surfaces
|
||
- [x] Menu unification — UserMenu uses `.sw-menu-flyout` + `data-position`, same as `sw.menu()`
|
||
- [x] ChatPane self-contained — standalone mode with streaming, model selector, history (editor slimmed ~220 lines)
|
||
|
||
**Dashboard Package (CS3–CS4):**
|
||
- [x] `packages/dashboard/` — type "full", route `/s/dashboard`
|
||
- [x] Exercises all 14 primitives: userMenu, menu, toolbar, tabs, dropdown, chat, notes, modal, confirm, toast, on/emit, api, user/isAdmin, theme
|
||
- [x] Layout-only CSS — zero references to SDK component classes
|
||
- [x] E2E tests: install, settings, export/import round-trip (7 tests)
|
||
|
||
### v0.31.2 — Team Workflow Self-Service ✅
|
||
|
||
Close the gap: team admins can create and manage workflows for their
|
||
team without requiring platform admin access. Clean public URLs via
|
||
team slugs (`/w/engineering/intake`).
|
||
|
||
Depends on: v0.31.1.
|
||
|
||
**CS0 — Backend: Team-Scoped Workflow Routes + Team Slugs:**
|
||
- [x] `/teams/:teamId/workflows` route group behind `RequireTeamAdmin`
|
||
- [x] CRUD: GET (list), POST (create, inject team_id), PATCH (update), DELETE
|
||
- [x] Stage CRUD: GET, POST, PUT, DELETE, PATCH reorder — scoped to team workflows
|
||
- [x] Publish: `POST /teams/:teamId/workflows/:id/publish`
|
||
- [x] Ownership guard: all mutating ops verify `workflow.team_id == :teamId`
|
||
- [x] Reuse existing `WorkflowHandler` methods — team ID injection, not new logic
|
||
- [x] Team `slug` column — auto-generated from name, UNIQUE, `GetBySlug` store method
|
||
- [x] Workflow entry + page renderer resolve scope via team slug (UUID fallback)
|
||
- [x] Auth rate limiter burst 8→5
|
||
|
||
**CS1 — Frontend: Workflows Tab in Team Admin:**
|
||
- [x] 9th tab "Workflows" in team admin modal
|
||
- [x] List team workflows with status badge (active/inactive)
|
||
- [x] Create/edit workflow form (name, slug, entry_mode, branding, retention)
|
||
- [x] Stage builder (add/reorder/delete stages, persona picker, history_mode)
|
||
- [x] Publish button with version display
|
||
- [x] Adapted from platform admin workflow UI (`_loadAdminWorkflows` reference)
|
||
- [x] Public URL display uses team slug (`/w/engineering/intake`)
|
||
- [x] ICD runner: team workflow CRUD tests, package settings fix, SSE reasoning_content fix
|
||
|
||
**Future (not blocking):**
|
||
- Team admin modal → full surface package (when tab count justifies it)
|
||
|
||
---
|
||
|
||
## Operations Track
|
||
|
||
Parallel to extension track. No Starlark dependency. Delivers
|
||
production readiness for the target multi-team deployment.
|
||
|
||
### v0.32.0 — Multi-Replica HA ✅
|
||
|
||
Run 2–3 backend replicas across nodes for node-level availability.
|
||
|
||
Depends on: v0.28.8.
|
||
|
||
**Design decision:** PG `SKIP LOCKED` replaces Kubernetes Lease-based
|
||
leader election. Every replica runs the scheduler poll loop; PG
|
||
serializes task claims atomically. No K8s API dependency, no Redis,
|
||
no new coordination infrastructure. See `docs/DESIGN-0.32.0.md`.
|
||
|
||
**What already works multi-replica:**
|
||
- REST API (stateless, JWT auth) ✅
|
||
- PG + S3 + CephFS (shared infrastructure) ✅
|
||
- `pg_broadcast` LISTEN/NOTIFY (cross-pod event bus) ✅
|
||
|
||
**Delivered (5 changesets):**
|
||
- [x] Task scheduler: `FOR UPDATE SKIP LOCKED` atomic claim replaces
|
||
`ListDue`. `CreateRunExclusive` conditional insert for
|
||
belt-and-suspenders uniqueness. Startup jitter (0–15s).
|
||
- [x] WebSocket cross-pod delivery: `PublishToUser` routes through
|
||
bus → `pg_broadcast` → remote pod. 18 `SendToUser` call sites
|
||
migrated. `tool.result.*` routing changed to `DirBoth` for
|
||
cross-pod `WaitFor`. `TargetUserID` field on `Event`.
|
||
- [x] Shared ticket store: `ws_tickets` PG table with 30s TTL,
|
||
atomic `DELETE ... RETURNING` validation. Replaces `sync.Map`.
|
||
- [x] Shared rate limiter: `rate_limit_counters` PG table with
|
||
fixed-window counters. Fail-open policy on DB errors.
|
||
- [x] Health probes: `/healthz/ready` (PG ping, 2s timeout),
|
||
`/healthz/live` (process alive). Helm: `replicaCount: 2`,
|
||
pod anti-affinity, readiness/liveness probes.
|
||
|
||
**Schema:** `020_ha.sql` — `ws_tickets`, `rate_limit_counters`.
|
||
|
||
### v0.33.0 — Observability ✅
|
||
|
||
Metrics, dashboards, alerting. Operate the platform without reading
|
||
Go source code.
|
||
|
||
Depends on: v0.32.0 (multi-replica metrics aggregation).
|
||
|
||
**Delivered (6 changesets):**
|
||
- [x] Structured logging (`slog`): `LOG_FORMAT=json|text`,
|
||
`LOG_LEVEL=debug|info|warn|error`. Request ID middleware
|
||
(`X-Request-Id`), correlation IDs through completion chain.
|
||
- [x] Prometheus `/metrics` endpoint: HTTP request counters/histograms,
|
||
WebSocket gauge, completion tokens/duration, provider status,
|
||
DB pool stats, task/sandbox execution counters.
|
||
- [x] OpenAPI 3.0.3 spec (hand-curated, core API groups) + Swagger UI
|
||
at `/api/docs` with system dark/light mode, WCAG AA contrast.
|
||
- [x] Grafana dashboard JSON: request rate, latency percentiles,
|
||
provider health, token usage, DB pool, Go runtime.
|
||
- [x] PrometheusRule alerting: OOM recovery, provider down, pool
|
||
exhaustion, high error rate, task failure spike.
|
||
- [x] Admin monitoring dashboard: provider health, 24h usage, DB pool,
|
||
WS connections, Go runtime stats, storage status, recent errors,
|
||
30s auto-refresh.
|
||
|
||
**Deferred to v0.36.0 (delivered):**
|
||
- Full OpenAPI spec coverage (all 20 ICD domains)
|
||
- Bearer token / mTLS auth documentation in spec
|
||
|
||
### v0.34.0 — Data Portability ✅
|
||
|
||
Export, import, backup, compliance.
|
||
|
||
Depends on: v0.29.2 (DB extensions — extension data in exports).
|
||
|
||
- [x] Bulk export/import: account data, conversations, settings, files
|
||
- [x] GDPR "download my data" + "delete my data" (cascade + audit trail)
|
||
- [x] ChatGPT/other tool import (conversation format mapping)
|
||
- [x] Backup/restore CronJob manifests for K8s
|
||
- [x] Admin export: team/user config (excludes vault-encrypted keys)
|
||
|
||
### v0.35.0 — Workflow Product
|
||
|
||
Bridge from workflow infrastructure to real business process automation.
|
||
Visitors see forms (not just chat), collected data flows through Starlark
|
||
enrichment, stages branch conditionally, and team members review
|
||
structured data — not chat transcripts.
|
||
|
||
Depends on: v0.31.2 (team workflow self-service), v0.29.0 (Starlark sandbox).
|
||
|
||
**Form Rendering Surface:**
|
||
- [x] Visitor form renderer — reads `form_template` from stage, renders
|
||
HTML inputs (text, email, select, date, textarea, checkbox, file),
|
||
validates client-side, submits via `/w/:id/form-submit`
|
||
- [x] Progressive form — multi-step within a single stage (fieldsets)
|
||
- [x] Conditional fields — show/hide based on previous answers
|
||
- [x] File upload in forms — attach to stage_data via storage API
|
||
- [x] Branded form page — uses workflow `branding` (colors, logo, tagline)
|
||
|
||
**Data Pipeline (between-stage processing):**
|
||
- [x] `on_advance` hook — Starlark entry point fires after each stage
|
||
transition, receives `stage_data`, can enrich/transform/reject
|
||
- [x] External data enrichment — Starlark `http.fetch` pulls from
|
||
external APIs, merges results into `stage_data` (reduce double entry)
|
||
- [x] Data validation rules — Starlark `on_validate` can enforce
|
||
cross-field business rules beyond per-field type checks
|
||
- [x] Stage data schema — typed `stage_data` with declared fields,
|
||
not opaque JSON blob. Enables structured review views
|
||
|
||
**Conditional Routing:**
|
||
- [x] Branch expressions — `transition_rules.condition` evaluated against
|
||
`stage_data` to select next stage (not always ordinal+1)
|
||
- [x] AI-triggered routing — persona calls `workflow_route` tool with
|
||
a target stage name based on conversation analysis
|
||
- [x] Escalation pattern — "if AI confidence < threshold, route to
|
||
human review stage" (help desk use case)
|
||
- [x] Loop stages — return to a previous stage for correction without
|
||
using reject (iterative data gathering)
|
||
|
||
**Structured Review:**
|
||
- [x] Assignment review view — team member sees `stage_data` as a
|
||
structured card/form, not just chat history
|
||
- [x] Approval/reject with comments — reviewer adds notes visible
|
||
to the next stage (not buried in chat)
|
||
- [x] Side-by-side view — chat history + structured data together
|
||
- [x] Bulk review — multiple assignments in a queue with keyboard nav
|
||
|
||
**Monitoring Dashboard:**
|
||
- [x] Active instances list — workflow name, current stage, assignee,
|
||
age, last activity
|
||
- [x] Stage funnel — how many instances at each stage, bottleneck detection
|
||
- [x] SLA timers — configurable per-stage, visible in review + dashboard
|
||
- [x] Stale instance alerts — notify team admins when instances age out
|
||
|
||
**Use case validation targets:**
|
||
- Help desk: AI + KB → escalation to human → resolution tracking
|
||
- Data intake: form → Starlark enrichment → human review → follow-up
|
||
- Onboarding: multi-stage form → conditional branching → team assignment
|
||
|
||
### v0.36.0 — Full OpenAPI Spec ✅
|
||
|
||
Complete OpenAPI 3.0.3 coverage for every API domain. Expand the
|
||
hand-curated spec from v0.33.0 (core groups only) to cover all 20 ICD
|
||
test domains. Document Bearer token auth and mTLS modes.
|
||
|
||
Depends on: v0.33.0 (Swagger UI + initial spec).
|
||
|
||
**API domains to document (matching ICD test suite):**
|
||
- [x] Admin (system settings, stats, storage, users, provider config)
|
||
- [x] Channels (full CRUD + membership, already partially covered)
|
||
- [x] Completions (streaming + non-streaming, already partially covered)
|
||
- [x] Extensions (package install, permissions, lifecycle)
|
||
- [x] Knowledge (KB articles, sources, search, embeddings)
|
||
- [x] Memory (conversation memory, compaction, search)
|
||
- [x] Models (provider models, BYOK, capability matching)
|
||
- [x] Notes (CRUD, graph links, search)
|
||
- [x] Notifications (list, mark read, preferences)
|
||
- [x] Personas (CRUD, system prompts, tool config)
|
||
- [x] Profile (user profile, preferences, avatar)
|
||
- [x] Projects (CRUD, membership, file uploads)
|
||
- [x] Surfaces (extension surfaces, mounting, lifecycle)
|
||
- [x] Tasks (scheduled tasks, execution history, Starlark tasks)
|
||
- [x] Teams (CRUD, membership, roles, slugs)
|
||
- [x] Team Workflows (team-scoped CRUD, stages, publish)
|
||
- [x] Workflows (platform-level CRUD, stages, instances, assignments)
|
||
- [x] Workspaces (CRUD, membership, settings)
|
||
- [x] Dashboard Package (package-specific API routes)
|
||
- [x] Editor Package (package-specific API routes)
|
||
|
||
**Auth documentation:**
|
||
- [x] Bearer JWT flow (login → token → `Authorization: Bearer <token>`)
|
||
- [x] mTLS mode (NPE-to-NPE, no Bearer needed)
|
||
- [x] API key mode (for service-to-service integrations)
|
||
- [x] Security scheme definitions in OpenAPI spec
|
||
|
||
**Quality:**
|
||
- [x] Request/response schemas with examples for every endpoint
|
||
- [x] Error response schemas (400, 401, 403, 404, 409, 422, 429, 500)
|
||
- [x] Pagination parameters documented consistently
|
||
- [x] WebSocket event schemas (connection, ticket, event types)
|
||
|
||
---
|
||
|
||
## UI Rewrite Track
|
||
|
||
Scorched earth rebuild of the frontend on Preact+htm. Three-layer
|
||
architecture (Primitives → SDK → Shell) replacing the imperative
|
||
DOM manipulation codebase. See `UI redesign.md` for full design doc.
|
||
|
||
Depends on: v0.36.0 (OpenAPI spec), v0.31.2 (SDK composability).
|
||
|
||
### v0.37.1 — Permission Audit & Enforcement ✅
|
||
|
||
Backend prerequisite for frontend RBAC. See `CHANGESET.md`.
|
||
|
||
- [x] `RequirePermission` middleware on 8 unprotected routes
|
||
- [x] `GET /profile/permissions` — self-service resolved permission set
|
||
- [x] 12 permission enforcement integration tests (deny → grant → allow)
|
||
- [x] Mirror route fixes in integration test harness
|
||
|
||
### v0.37.2 — Layer 0: UI Primitives + Shell Layout ✅
|
||
|
||
Preact+htm component library and application frame.
|
||
|
||
- [x] Vendor: preact.module.js, hooks.module.js, htm.module.js (~5KB total)
|
||
- [x] 14 primitive components: Button, Spinner, Avatar, FormField,
|
||
Tooltip, Banner, Toast, Dialog, Confirm, Prompt, Menu, Drawer,
|
||
Tabs, Dropdown
|
||
- [x] AppShell layout: fixed top/bottom banners (single config),
|
||
dismissible message bar, footer, surface viewport
|
||
- [x] `sw-primitives.css` + `sw-shell.css` — `sw-` prefixed, no
|
||
conflicts with existing styles
|
||
- [x] `dev.html` — interactive primitives gallery + shell controls
|
||
- [x] ES module loading with `window.preact/hooks/html` globals
|
||
|
||
### v0.37.3 — Layer 1: SDK ✅
|
||
|
||
Namespaced REST client, auth module, RBAC gate, event bus, pipe/filter
|
||
pipeline, theme control.
|
||
|
||
Depends on: v0.37.2.
|
||
|
||
- [x] `sw.api.{domain}.*` — 18 namespaced domain clients (auth,
|
||
channels, personas, knowledge, notes, projects, workspaces,
|
||
memory, models, providers, notifications, extensions, profile,
|
||
teams, workflows, tasks, surfaces, admin)
|
||
- [x] `sw.auth.*` — login, logout, refresh, session lifecycle,
|
||
permissions/teams/policies cache
|
||
- [x] `sw.can(perm)` — synchronous RBAC gate from cached permission set
|
||
- [x] `sw.on/off/emit` — event bus with wildcard patterns, WebSocket
|
||
connection with ticket-first auth
|
||
- [x] `sw.pipe.*` — carry forward pre-send/stream/render pipeline
|
||
- [x] `sw.theme.*` — theme control with FOUC prevention
|
||
|
||
### v0.37.4 — Layer 2: Shell ✅
|
||
|
||
Application frame with user menu, surface viewport, banners, toast/dialog
|
||
stacks. Temp auth bypass for visual validation.
|
||
|
||
Depends on: v0.37.3.
|
||
|
||
- [x] Root `<App>` component tree (AppShell + SurfaceViewport +
|
||
ToastContainer + DialogStack)
|
||
- [x] `UserMenu` — avatar + flyout (Settings, Admin, Team Admin,
|
||
Debug, Sign Out) with RBAC gating
|
||
- [x] `SurfaceViewport` — mount point for active surface
|
||
- [x] `ToastContainer` + `DialogStack` — global overlay management
|
||
- [x] `sw.userMenu(container, opts)` — imperative mount helper for
|
||
non-Preact surfaces
|
||
|
||
### v0.37.5 — Login + Settings Surfaces ✅
|
||
|
||
First surface rebuilds. Login page and Settings surface converted from
|
||
Go templates + imperative DOM JS to Preact component trees.
|
||
|
||
Depends on: v0.37.4.
|
||
|
||
- [x] Login surface — 5 components (Hero, LoginForm, RegisterForm,
|
||
SSOPanel, LoginSurface root)
|
||
- [x] Settings surface — framework with nav + section routing
|
||
- [x] 7 Preact sections: General, Appearance, Profile, Models,
|
||
Teams, Providers (stub), Personas (stub)
|
||
- [x] Bridge pattern for deferred sections (Tasks, Git Keys,
|
||
Workflows, Data & Privacy)
|
||
- [x] `sw.toast()` SDK wiring, nav gating (BYOK + Personas)
|
||
- [x] `sw-login.css` extracted from inline styles
|
||
|
||
**Known issues (tracked in CHANGELOG, resolve before v0.37.# tag):**
|
||
- BYOK sections not tested with policy enabled
|
||
- ~~Bridge sections~~ Resolved in v0.37.7 (all settings sections native Preact)
|
||
- ~~`ui-settings.js` + `settings-handlers.js`~~ Confirmed deleted, stale `sw.js`
|
||
references cleaned in v0.37.12
|
||
|
||
### v0.37.6–v0.37.18 — Remaining Surface Rebuilds
|
||
|
||
Each surface rebuilt as Preact component tree. Old JS deleted per surface.
|
||
|
||
| Version | Surface | Notes |
|
||
|---------|---------|-------|
|
||
| v0.37.6 | Admin | Largest section count, no real-time |
|
||
| v0.37.7 | Team Admin | Inherits from Admin patterns |
|
||
| v0.37.8 | ChatPane ✅ | Composable kit (9 files), standalone stripped from old chat-pane.js |
|
||
| v0.37.9 | NotesPane ✅ | Obsidian-grade kit (10 files), old notes code scorched (~1,630 lines) |
|
||
| v0.37.10 | Chat surface ✅ | Scorched earth: 21 files deleted (−12,841 net), Preact surface + ChatPane kit upgrades |
|
||
| v0.37.11 | Notes surface ✅ | Preact surface (7 JS + 1 CSS), sidebar folders/tags, UserMenu, template rewrite, Menu primitive fix |
|
||
| v0.37.12 | Scorched Earth II ✅ | 23 files deleted (~2,600 lines): 9 JS, 6 CSS, 8 admin templates; sw.js + base.html cleaned |
|
||
| v0.37.13 | Scorched Earth III ✅ | 5 JS deleted (−2,269 lines): ui-primitives, code-editor, file-tree, user-menu, app-state; SDK slimmed, Theme→Preact |
|
||
| v0.37.14 | Pane audit | Collaborative walkthrough: core features + theming checked |
|
||
| v0.37.15 | Workflow surfaces | Form renderer, review views (design-first) |
|
||
| v0.37.16 | Projects surface | Project views, note/channel associations (design-first) |
|
||
| v0.37.17 | Debug / model surface | Debug tooling, model configuration UI |
|
||
| v0.37.18 | Tag | Light mode CSS audit, dead code hunt, all features verified |
|
||
|
||
**v0.37.11 — Notes Surface ✅:**
|
||
|
||
Notes surface layout (composed kit into full-page Preact surface):
|
||
- [x] Sidebar with folder tree + tag browser (sidebar-folders.js, sidebar-tags.js)
|
||
- [x] Main content area with NotesPane standalone=false
|
||
- [x] Mobile-responsive layout (768px breakpoint, overlay sidebar)
|
||
- [x] UserMenu in sidebar footer (same as chat surface)
|
||
- [x] Template rewrite — crash catcher + single mount point pattern
|
||
- [x] Menu primitive CSS fix — `sw-primitives.css` added to base.html (cross-surface)
|
||
- [x] Avatar initials fix — single-word names show first letter only (cross-surface)
|
||
- [x] Old `.surface-notes*` CSS deleted from surfaces.css
|
||
|
||
NotesPane kit enhancements (deferred to v0.37.14 pane audit):
|
||
- [ ] Markdown toolbar — bold, italic, heading, link, code, list, checkbox
|
||
- [ ] Hover preview on wikilinks — floating card on `[[Title]]` hover
|
||
- [ ] Breadcrumb trail — navigation history through wikilinks
|
||
- [ ] Split view (editor + preview) — side-by-side live preview
|
||
- [ ] Note templates — "New from template" in toolbar
|
||
- [ ] Slash commands in editor — `/table`, `/code`, `/heading`, `/date`, `/link`
|
||
- [ ] Graph minimap — small overview with draggable viewport
|
||
- [ ] Resizable split panes
|
||
|
||
Polish (deferred to v0.37.18 tag):
|
||
- [ ] Light mode CSS variable audit for `sw-notes-pane.css`
|
||
- [ ] Smooth view transitions (fade/slide between list → reader → editor)
|
||
- [ ] Note word count goal (optional target with progress bar)
|
||
- [ ] Server-side favorites via note metadata (cross-device pinning)
|
||
|
||
**v0.37.12 — Scorched Earth II ✅:**
|
||
|
||
Second dead-code purge. All six core surfaces confirmed Preact-only — old
|
||
vanilla JS, CSS, and admin templates that survived v0.37.10 are now removed.
|
||
Service worker cache manifest overhauled (17 stale entries → clean).
|
||
|
||
Deleted (23 files):
|
||
- [x] 9 JS: `ui-format.js`, `virtual-scroll.js`, `model-selector.js`,
|
||
`drag-resize.js`, `pages-splash.js`, `task-sidebar.js`,
|
||
`workflow-api.js`, `workflow-queue.js`, `workflow-monitor.js`
|
||
- [x] 6 CSS: `splash.css`, `styles.css`, `persona-kb.css`,
|
||
`notification-prefs.css`, `memory.css`, `channel-models.css`
|
||
- [x] 8 admin templates: `server/pages/templates/admin/` directory
|
||
- [x] 7 script/link tags removed from `base.html` (includes
|
||
`marked.min.js` + `purify.min.js` — surfaces load their own copies)
|
||
- [x] `sw.js` SHELL_FILES array replaced with actual file inventory
|
||
- [x] `pages.go` admin template embed directive removed
|
||
|
||
Kept (11 JS files — extension surface + debug dependencies):
|
||
`sb.js`, `app-state.js`, `events.js`, `ui-primitives.js`, `user-menu.js`,
|
||
`file-tree.js`, `code-editor.js`, `switchboard-sdk.js`, `debug.js`,
|
||
`repl.js`, `workflow-surfaces.js`
|
||
|
||
**v0.37.13 — Scorched Earth III ✅:**
|
||
|
||
Third dead-code purge. 5 files deleted (−2,269 net lines). The 1,133-line
|
||
`ui-primitives.js` is gone; Theme management migrated to Preact SDK module.
|
||
SDK slimmed by removing 6 dead factory wrappers.
|
||
|
||
Deleted (5 files):
|
||
- [x] `ui-primitives.js` (1,133 LOC) — `esc`, component lifecycle, Providers,
|
||
Roles, modal/confirm/prompt, Theme, render helpers
|
||
- [x] `code-editor.js` (362 LOC) — tabbed CodeMirror 6 editor
|
||
- [x] `file-tree.js` (290 LOC) — workspace file browser
|
||
- [x] `user-menu.js` (206 LOC) — vanilla flyout menu
|
||
- [x] `app-state.js` (139 LOC) — global `window.App` state
|
||
- [x] 5 script tags removed from `base.html`
|
||
- [x] 5 entries removed from `sw.js` SHELL_FILES
|
||
- [x] `switchboard-sdk.js` slimmed: 6 dead wrappers removed, Theme→Preact SDK
|
||
|
||
Kept (6 JS files — extension + debug dependencies):
|
||
`sb.js`, `events.js`, `switchboard-sdk.js`, `debug.js`, `repl.js`,
|
||
`workflow-surfaces.js`
|
||
|
||
Cumulative scorched earth: 49 files, ~−17,710 lines (v0.37.10 + v0.37.12 + v0.37.13)
|
||
|
||
**v0.37.14 — Pane Audit:**
|
||
|
||
Collaborative walkthrough of core features and theming. Each pane and surface
|
||
checked for form, function, and visual consistency. Fixes applied in-place.
|
||
No new architecture — making everything work correctly.
|
||
|
||
- [ ] ChatPane: message rendering, streaming, model selector, markdown, code blocks
|
||
- [ ] NotesPane: list, editor, reader, graph, quick switcher, save-to-note
|
||
- [ ] Cross-pane: toast, dialog, user menu, keyboard shortcuts
|
||
- [ ] Theming: light/dark mode consistency audit across all surfaces
|
||
- [ ] Kit enhancement backlog triage (above items evaluated for inclusion)
|
||
|
||
**v0.37.15 — Projects Surface:**
|
||
|
||
Project views, note/channel associations, project-scoped navigation.
|
||
|
||
**v0.37.16 — Workflow Assignment:**
|
||
|
||
Workflow routing and assignment UX.
|
||
|
||
**v0.37.17 — Debug / Model Surface:**
|
||
|
||
Debug tooling, model configuration UI, provider diagnostics.
|
||
|
||
---
|
||
|
||
## MVP v0.50.0
|
||
|
||
**Gate:** deploy for 5–10 teams, ~50 users, 100+ anonymous visitors
|
||
on a 3-node cluster. An IT team can operate the platform without
|
||
reading Go source code. Team admins build workflows visually.
|
||
|
||
**Requires all of:**
|
||
- Extension track through v0.31.2 (full package ecosystem, visual
|
||
workflow builder, SDK-based surfaces, team workflow self-service)
|
||
- Operations track through v0.34.0 (multi-replica HA, observability,
|
||
data portability) ✅
|
||
- Workflow product v0.35.0 (form rendering, data pipeline, conditional
|
||
routing, structured review, monitoring dashboard)
|
||
- Full OpenAPI spec v0.36.0 (complete API documentation for all domains)
|
||
|
||
**Additionally requires:**
|
||
- [ ] Deployment guide: step-by-step for IT team (PG cluster,
|
||
S3/MinIO, CephFS/NFS, Helm install, TLS, OIDC)
|
||
- [ ] Admin guide: team setup, provider config, workflow creation,
|
||
package management, backup/restore
|
||
- [ ] Mobile-responsive layouts (proper mobile navigation,
|
||
touch-optimized chat, not just CSS responsive)
|
||
- [ ] Project creation dialog (replace `prompt()` with modal)
|
||
- [ ] Admin-level project management (cross-instance visibility)
|
||
|
||
---
|
||
|
||
## v0.50.0+ — Rich Media + Beyond
|
||
|
||
Post-MVP. Each item is a standalone `.pkg` or platform primitive.
|
||
No ordering constraints between items.
|
||
|
||
**Media + Generation**
|
||
- Image generation tool (browser or sidecar, provider-agnostic)
|
||
- STT input (browser extension, Web Speech API)
|
||
- TTS output (browser extension, provider API)
|
||
- Code execution sandbox (server-side, container isolation)
|
||
|
||
**Desktop + Mobile**
|
||
- Desktop app (Tauri — native wrapper around existing web UI)
|
||
- Full PWA with offline capability
|
||
|
||
**Platform**
|
||
- Multi-tenant SaaS mode (tenant isolation, billing, onboarding)
|
||
- Surface IDE (built-in surface for building surfaces)
|
||
- Sidecar HTTP tool protocol (container-isolated tool execution)
|
||
- Latency-aware provider routing (response time percentiles)
|
||
- Cost-aware routing with budget ceiling per request
|
||
- Fallback chain visualizer (drag-to-reorder provider priority)
|
||
|
||
**Knowledge + Memory**
|
||
- Hybrid KB search (vector + `tsvector` + re-rank)
|
||
- Semantic chunking (embedding-based boundary detection)
|
||
- HNSW index (replaces IVFFlat for large datasets)
|
||
- Web scraping KB source (ingest URLs via `url_fetch`)
|
||
- Scheduled re-indexing (periodic rebuild when sources update)
|
||
- Memory export/import (portable format across instances)
|
||
- Cross-persona memory sharing (opt-in)
|
||
- Memory analytics dashboard
|
||
|
||
**Projects**
|
||
- `/p/:id` shared project view
|
||
- Project-specific file uploads (own endpoint, storage path)
|
||
- Project templates (predefined configurations)
|
||
- Sub-projects / nested hierarchy
|
||
|
||
**UX**
|
||
- "Group" scope badge on model selector / KB list
|
||
- Article drag-to-reorder outline sections
|
||
- Article AI tools (suggest_outline, expand_section, check_citations)
|
||
- Rate limiting per user/team/tier (token budgets)
|