Archived

This repository has been archived on 2026-04-03. You can view files and clone it. You cannot open issues or pull requests or push a commit.

Files

xcaliber 3237d55e0c Changeset 0.28.7 (#193 )

2026-03-15 15:45:00 +00:00

43 KiB

Raw Blame History

Roadmap — Chat Switchboard

See also:

ARCHITECTURE.md — Core services design, store layer, scope model
EXTENSIONS.md — Extension system spec (Browser/Starlark/Sidecar tiers, manifests, browser tool bridge, surfaces/modes, model roles)
EXTENSION-SURFACES.md — Extension surface authoring guide (manifest format, platform API, CSS properties, install workflow)
CHANGELOG.md — Detailed release notes for all completed versions

Versioning (pre-1.0): 0.<major>.<minor> — hotfixes use quad: 0.x.y.z No compatibility guarantees before 1.0.

Dependency Graph

Features have real dependencies. This ordering respects them.

v0.9.x–v0.27.5  Foundation → Extensions → Surfaces → Auth     ✅
                 → Workflows → Tasks → Team Tasks
                 (see CHANGELOG.md for full history)
                        │
                v0.28.0 Platform Polish
                  ├─ v0.28.1  Surfaces ICD audit        ✅
                  ├─ v0.28.2  ICD audit — all domains   ✅
                  ├─ v0.28.3  ICD close-out + FE decomp   ✅
                  ├─ v0.28.4  Security tier (red team)    ✅
                  ├─ v0.28.5  Frontend SDK + Pipes      ✅
                  │   (switchboard-sdk.js, pipe/filter
                  │    pipeline, component mounting)
                  ├─ v0.28.6  Infrastructure            ✅
                  │   (virtual scroll, Helm, system tasks,
                  │    broadcast, git keygen, model prefs)
                  └─ v0.28.7  Unified Packaging + Task RBAC  ✅
                      (.pkg archive, manifest.type,
                       task permission gate pre-Starlark)
                        │
                  v0.28.8  ICD Green Board
                      (provider timeout resilience, CORS
                       lockdown, SDK base path fix, WS
                       ticket exchange)
                        │
                v0.29.0 Starlark Sandbox
                  (eval loop, permissions, admin UI
                   — adds starlark capability to .pkg)
                        │
                v0.29.1 API Extensions
                  (Starlark route handlers,
                   outbound HTTP, requires_provider)
                        │
                v0.29.2 DB Extensions
                  (namespaced tables, scoped db module,
                   declarative schema in manifest)
                        │
                v0.29.3 Workflow Forms
                  (form_template → real UI, LLM optional,
                   .star validation, structured data)
                        │
                v0.30.0 Package Lifecycle
                  (schema versioning, migrations,
                   settings extension point,
                   export/import, marketplace)
                        │
                v0.30.1 SDK Adoption
                  (sw.notes factory, core surface
                   migration, fix-once-fix-everywhere)
                        │
                v0.30.2 Workflow Packages
                  (stage surfaces, custom UI per stage,
                   team admin workflow builder)
                        │
                v0.31.0 Editor Package
                  (E2E proof: rebuild editor as
                   installable .pkg, zero
                   platform special-casing)

v0.28.0 — Platform Polish

Audit arc, frontend decomposition, security, and infrastructure improvements.

v0.28.1 — Surfaces ICD Audit ✅

ICD surfaces.md corrected (6 discrepancies: field name, archive format, response shape)
Surface ID slug validation + extractableRelPath install hardening
19 E2E surface CRUD tests in ICD runner (install, enable/disable, delete, error paths)
22 handler-level tests + 14 store-level tests (PG + SQLite)
CI timeout 8m → 12m for PG integration tests

v0.28.2 — ICD Audit: All Domains ✅

Full audit of every ICD document against code. Goal: ICD becomes the single source of truth. After this version, the workflow is ICD-first — update the contract before changing code.

Methodology: Read ICD → grep all source → trace route → handler → store (PG + SQLite) → tests → enforce {"data": [...]} envelope convention → fix ICD → fix code → fix runner → CI green. Final pass picks up straggling failures across all domains.

Final score: 469/469 (100%)

Completed audits:

Notifications (cs0):

Notifications ICD audit: object shape, query params, response envelopes, WS events
Notification type enum sync: remove aspirational types, add implemented types (kb.ready, kb.error, grant.changed, task.budget_exceeded)
Implement memory.extracted notification (hook in memory extractor)
Implement user.mentioned persisted notification (was WS-only)
Implement workflow.claimed persisted notification (was WS-only)
Remove dead NotifTypeProjectInvite constant
GET /notifications/preferences — envelope fix (missing key "preferences")

Knowledge (cs1–cs4):

knowledge.md corrected: KB object shape (6 field mismatches), search envelope (data not results), search result fields, status progression (extracting step), file type support (text-only, not binary), auth annotations
Test harness: add RequirePermission on POST /knowledge-bases and POST /:id/documents, add missing GET /:id/documents/:docId/status and DELETE /:id/documents/:docId routes
New tests: document status polling, document delete, update-empty-body 400, permission denial for non-privileged user
P0 fix: SetDiscoverable authorization — loadAndAuthorize + owner/admin check, audit log, cross-user security tests (cs2)
ListDiscoverableKBs response normalization — use toKBResponse(), shape assertion test (cs3)
Dead code removal: ListGlobal/ListForTeam on KnowledgeBaseStore — interface + PG + SQLite (cs4)
Move CreateKB team role check from raw database.DB.QueryRow to stores.Teams.IsTeamAdmin; remove database import (cs4)
Team-scoped KB creation test: member denied, team admin succeeds (cs4)

Profile (cs5):

profile.md corrected: avatar API (was multipart/form-data, actually JSON base64), response shapes for all 7 endpoints, auth annotations, field table
P0 fix: GET /settings returns bare object → wrapped in {"settings": {...}}
profileResponse shape: add last_login_at field + dialect-safe time scan (database.ST() / database.SNT()), COALESCE guard on settings column
Integration test harness: register all 7 profile/settings routes (was only GET /profile), remove stale /avatar route aliases
New profile_test.go: GET profile shape, PUT profile (display_name, email, duplicate email 409), GET/PUT settings envelope, password change (success, wrong current 401, too short 400), avatar delete

Workspaces (cs6):

workspaces.md corrected: workspace object shape (added indexing_enabled, git_* fields, total_bytes not storage_bytes, owner_type full enum), file entry shape (is_directory/size_bytes not is_dir/size), git status full shape, git log envelope ({"data": [...]}), git commit request (paths not files), archive format query param, auth annotations, all response shapes
P0 fix: GET /workspaces returns bare array → {"data": [...]}
P0 fix: GET /git-credentials returns bare array → {"data": [...]}
Fix: GET .../git/log returns bare array → {"data": [...]} + nil slice guard
New workspace_test.go: List empty envelope, list with data + shape, root_path not exposed, user isolation, GET by ID shape, not found 404, forbidden 403, git-credentials empty envelope, auth required (11 tests)

Projects (cs7):

projects.md full rewrite: 6 categories of drift (ghost fields, missing fields, request shapes, association objects, omitempty on computed counts)
14 new Go tests: CRUD shapes, 201 status, envelope, admin list, auth, isolation
P0 nil guard: ListByProject files null → []
P0: ListTeamProviderModels missing Type field — Venice 400 fix
ICD runner projects rewritten 9 → 19 tests
ICD runner tier-providers: SSE parser, model exclusion, ID fallback fixes

v0.28.3 — ICD Close-out + Frontend Decomposition Prep

ICD straggler sweep (rolled in from v0.28.2 tail) and frontend decomposition groundwork.

ICD close-out (cs0):

websocket.md full rewrite: event envelope field (event not type), payload shapes for 11 event types, routing table from code, room model documented as planned-not-implemented
auth.md corrected: login response shape (added token_type/expires_in, removed phantom fields), register admin-approval path
enums.md corrected: added missing queued task run status
Presence status gap documented (DB allows away, runtime emits only online/offline)
VERSION, CHANGELOG, ROADMAP updated

WebSocket delivery fix (cs1):

workflow.claimed, workflow.advanced, workflow.completed use room-scoped Bus.Publish() but rooms are never joined — events never reach WebSocket clients. Convert to SendToUser() per channel participant (same pattern as message.created, workflow.assigned).

Frontend decomposition:

JS dependency audit: docs/JS-DEPENDENCY-AUDIT.md — full map of 47 files, 431 globals, cross-file call graph, script load order per surface
IIFE extraction: all 47 JS files wrapped, ~168 functions privatized, zero implicit globals remain. Explicit window.* exports on every file.
Cross-file coupling fix: events.js no longer reads _storageKey from api.js — uses API.accessToken instead
ICD runner gains test tiers: crud, envelope, security, sdk
Runner coverage target: 100% of ICD-documented endpoints have at least one test
Phase 2: onclick → addEventListener migration (143 dynamic → 5 unconvertible, 163 data-action delegated via _uiDispatch on document.body)
Phase 3: Action registry (sb.js) — 248 window.* exports replaced with sb.register()/sb.ns(). sb.resolve() centralized dispatch. sb.call() template bridge ready for Go template migration.
Phase 3b: Go template onclick → sb.call() migration (72 handlers across 10 templates, 4 unconvertible inline DOM ops, 3 standalone pages excluded)
Phase 4: ES module conversion — IIFE wrappers removed from all 47 files, <script type="module"> across all templates. sb.js + vendor stay classic. CI test harness fixed (sb.js in VM context).
Phase 5 (future): import/export statements, remove window[name] dual-write from sb.register()/sb.ns()

v0.28.4 — Security Tier (ICD Runner Red Team)

New security tier in ICD test runner. 58 tests, 529/531 pass. 5 bugs found and fixed. surfaces/ directory added with build script.

Auth boundary:

JWT revocation: disable user → existing token rejected immediately. Real bug found + fixed — UserStatusCache (30 s TTL) added to auth middleware; is_active checked on every request.
Token reuse after POST /auth/logout — revoked refresh token fails
Expired token rejection (tampered JWT, alg=none — both rejected)
Role escalation: modified JWT claims → 401. Real bug found + fixed — role now resolved from DB, not JWT claims.
User ID substitution: user_id in request bodies ignored by handlers

Cross-tenant data access:

User A's token → GET user B's notes, memories, workspaces, BYOK configs, tasks — all 404/403
Team member → access other team's providers, personas, tasks — all 403
Non-participant → read channel messages, files — 403/404

Input validation:

Path traversal in surface install — rejected (400)
SQL injection in search endpoints (notes, users, channels) — 7 payloads, all parameterized
XSS payloads in channel titles, note content, persona names — stored safely, frontend sanitizes
Oversized request bodies, malformed JSON — handled (400)
Null byte in path params → 500 fixed — ValidatePathParams() middleware (400)
Oversized path params → 500 fixed — same middleware (400)

Session security:

Visitor session → access channels outside bound workflow — 401
Visitor → escalate to authenticated user via crafted headers — 401
Cross-visitor isolation: enforced by session_auth.go channel binding, but no E2E test (requires public_link workflow setup — defer to visitor E2E milestone)

Infrastructure:

CORS_ALLOWED_ORIGINS env var — production restricts, dev allows *
surfaces/ directory with build.sh → dist/<n>.surface
hello-dashboard.surface unpacked into surfaces/hello-dashboard/

v0.28.5 — Frontend SDK + Pipes ✅

switchboard-sdk.js — composition layer over existing globals. Surface authors consume a single coherent API instead of hunting through 15 JS files. Pipe/filter pipeline formalizes the extension hook model into composable, priority-ordered transform stages.

SDK core:

Switchboard.init({ mount }) — idempotent boot: tokens, profile, theme, events, user menu
sw.user, sw.isAdmin — resolved identity
sw.api.get() / sw.api.post() — authenticated REST, no token/base-path management
sw.on(event, fn) — WebSocket subscription (no Events global knowledge)
sw.chat(container, opts) — drop-in ChatPane (wraps ChatPane.create())
sw.notes(container, opts) — stub shipped (warns); notes component needs factory refactor before clean mounting (v0.30.1 SDK Adoption)
sw.toast(), sw.confirm() — UI primitives
sw.theme.current, sw.theme.on('change', fn) — theme queries
Absorbs cs15 UserMenu band-aid — universal hydration moves into init()

Pipe/filter pipeline:

Three stages, each a priority-ordered chain of transform functions. Filters receive typed context, return (possibly modified) context. Pipeline wires filters in priority order; any filter can halt the chain.

sw.pipe.pre(priority, fn) — pre-send: transform user message before LLM request. Context: { message, channel, attachments, metadata }. Runs on both sendMessage() and regenerateMessage() with regenerate flag.
sw.pipe.stream(priority, fn) — post-receive: transform LLM response stream chunks as they arrive (sync-only). Context: { chunk, accumulated, channel, model }.
sw.pipe.render(priority, fn) — post-render: transform rendered HTML/markdown after DOMPurify. Context: { element, message, channel }. Replaces runExtensionPostRender() with compat shim for existing ctx.renderers.register() extensions.
Pipeline execution engine: ordered dispatch, error isolation (one filter throws → skip it, continue chain), timing telemetry per filter
sw.pipe.list() — introspection: list registered filters by stage with priority, source, scope, and timing stats
Filter scoping: { scope: { channelType: [...] } } restricts execution to specific channel types with zero-overhead skip
Manifest "pipes" key: declarative filter registration — deferred to v0.28.7 (unified packaging, where manifest keys are extended)

Testing:

ICD runner gains sdk test tier: 36 tests — boot, identity, REST, events, theme, pipe registration, execution, scoping, halt, error isolation, compat shim, introspection
Pipe filter tests: priority ordering, halt semantics, error isolation, scoped/unscoped execution

v0.28.6 — Infrastructure ✅

Virtual scroll for long conversations (prerequisite for heavy task output channels)
Helm chart (replaces raw k8s manifests, helm install switchboard ./chart)
Per-provider model preferences — finalize: make provider_config_id required on PUT /models/preferences (fixes NULL-in-UNIQUE dedup bug), migrate /models/enabled and /models/preferences to {"data": [...]} envelope, add integration test coverage
Git credentials settings UI: server-side ED25519 key generation (private key vault-encrypted, public key exposed for git host), optional persona binding for commit attribution, settings section with generate/list/copy/delete
system.announcement notification type: admin broadcast endpoint (POST /admin/notifications/broadcast), fan-out to all active users via NotifyMany, admin UI for composing announcements
Task webhook trigger UI: schedule selector gains webhook option, trigger URL displayed + copy button, outbound webhook_url + webhook_secret fields in create/edit form, task-to-task chaining documentation in admin UI. Backend fully implemented — this is pure admin UI work.
System task type: task_type: "system" — built-in Go function registry (retention_sweep, health_prune, session_cleanup, staleness_check). Admin creates task, picks function from dropdown, sets cron schedule. Executor calls registered Go function instead of LLM completion. Replaces the current goroutine-based background jobs with visible, configurable, auditable tasks. Existing goroutines kept as fallback until system tasks are validated. Permanent track — Go registry is not replaced by Starlark (v0.29.0). Core platform ops must not break from bad user code. Admin-only by design (no RBAC needed — hardcoded to admin role).
KB auto-injection moved to v0.29.0 — built as the reference Go implementation of the server-side filter model alongside Starlark. Ensures the filter architecture is right and performant before user code runs in it.

v0.28.7 — Unified Packaging + Task RBAC

Single .pkg archive format for both surfaces and extensions. Manifest "type" field (surface, extension, full) drives install behavior. Format first, capabilities later — v0.29.x adds Starlark/DB/API capabilities into packages that already install cleanly.

Task permission model gates task creation by type and scope. Retroactively locks down action tasks (webhook relay = data exfiltration risk) and pre-positions for starlark tasks in v0.29.0.

Depends on: v0.28.5 (SDK — pipe/filter registration is part of manifest contract).

Archive format:

.pkg archive: zip containing manifest.json + assets (JS, CSS, templates, icons). Same structure as .surface archives, extended manifest schema
manifest.type field: surface (routes + templates + data loader), extension (hooks + tools + pipes, no own route), full (both). Absent type defaults to surface for backward compat
manifest.pipes key: declarative pipe filter registration (pre-send, post-receive, post-render) with priority and entry function reference. Wired by SDK on extension load
manifest.tools key: LLM-callable tool declarations (existing schema, now part of unified manifest)
manifest.hooks key: EventBus subscriptions (existing schema from EXTENSIONS.md, carried forward)

Install infrastructure:

POST /admin/packages/install — unified install endpoint. Reads manifest.type, validates type-specific requirements, wires subsystems. Replaces POST /admin/surfaces/install (old route kept as alias)
Validation branches by type: surface requires routes/template, extension requires at least one of hooks/tools/pipes, full requires both sets
Admin UI: merge Surfaces + Extensions into single "Packages" section. Type badge on each entry. Filter by type.
packages table: replaces both surface_registry and extensions. Text slug PKs, type/version/tier/scope/is_system columns.

Migration:

Existing .surface archives install unchanged (type defaults to surface). Re-install required after DB rebuild.
extensions table removed. Builtin extensions re-seed into packages table at startup via SeedBuiltinPackages().
build.sh in packages/ produces .pkg files. Directory renamed from surfaces/.
ICD runner surface install tests updated to use new endpoint (or alias)

Task RBAC:

Permission gate on task creation by task_type. Uses existing group permission model — no new task_permissions table. task_type is immutable on PATCH (not in TaskPatch struct).

task.starlark permission constant pre-positioned in AllPermissions. Handler rejects with 400 referencing v0.29.0.
task.action permission gate on create (existing, v0.28.0). system type hardcoded admin-only (existing, v0.28.6).
Go tests: starlark rejected, system non-admin denied, action without permission denied
Admin UI: task permission management — per-user and per-team overrides. "Allow action tasks for Team X" toggle. Pre-positions starlark permission type (hidden until v0.29.0 enables it)

ICD + docs:

packages.md ICD: install, list, enable/disable, delete, type filtering
extensions.md + surfaces.md redirected to packages.md
tasks.md ICD updated: permission matrix, task_type immutability
ICD runner gains packaging test tier: install surface-type, install extension-type, install full-type, type validation, backward compat

Tier 2 — Medium value (pull into any v0.28.x):

User-installable extension packages: RBAC-gated POST /packages/install (non-admin), team-scoped or personal scope, permission review flow. Depends on v0.28.7 packaging.
Memory compaction: summarize old memories, confidence decay, prune low-confidence
capability_match routing policy ("cheapest model with tool_calling")
New provider types registrable via config file (OpenAI-compatible + custom schema)

v0.28.8 — ICD Green Board

Close out pre-existing ICD test failures. Gate: 543/543 (100%) on the ICD runner before starting v0.29.0 feature work.

Depends on: v0.28.7 (unified packaging).

Provider tier — upstream timeout resilience (6 failures):

The provider test tier calls POST /admin/models/fetch to sync the model catalog from the upstream provider (Venice). When Venice is slow (>30s), the Gin handler times out, the reverse proxy returns 502, and every subsequent provider test fails (bulk enable, model list, team creation for provider scoping, BYOK creation).

Request timeout on outbound provider HTTP calls: http.Client with configurable timeout (default 30s). Currently uses http.DefaultClient with no timeout in providers/sync.go.
POST /admin/models/fetch returns 504 (not 502) on upstream timeout, with {"error": "upstream timeout: <provider>"}. Currently the reverse proxy manufactures the 502 from a broken connection.
ICD runner provider tier: retry models/fetch once on 502/504 before failing. Slow providers are a fact of life; the test should tolerate one retry.

SDK tier — base path in sw.api calls (4 failures):

sw.api.get('/api/v1/health') calls API._get('/api/v1/health') which prepends window.__BASE__. In the ICD runner context on a /dev base path deployment, __BASE__ is /dev, so the request goes to /dev/api/v1/health — correct. The 502s here are cascading from the same upstream provider timeout that poisons the proxy connection pool. These will resolve when the provider timeout fix lands. No SDK code change needed.

Confirm SDK 502s are gone after provider timeout fix (no independent fix required)

Security transport — CORS + WebSocket (2 failures):

Both are informational findings from the v0.28.4 security tier. They're real issues but low severity (P2).

CORS_ALLOWED_ORIGINS enforcement: currently * in all environments. Production should restrict to the actual domain. Add validation: if CORS_ALLOWED_ORIGINS is unset or *, log a warning at startup. Document in ARCHITECTURE.md.
WebSocket ticket exchange: replace ?token= query parameter (JWT visible in server logs, proxy logs, browser history) with a short-lived opaque ticket. POST /api/v1/ws/ticket returns a single-use token (UUID, 30s TTL, stored in memory/Redis). WebSocket connects with ?ticket= instead. Server validates and deletes on use. This is a protocol change — coordinate with frontend events.js WebSocket connection logic.

v0.29.0 — Starlark Sandbox + Permission Model

Server-side extension runtime. Prove the eval loop, permission pipeline, and admin UI before adding capabilities.

Depends on: v0.28.8 (ICD green board — clean test baseline before feature work), v0.28.7 (unified packaging), extension infrastructure (v0.11.0).

go.starlark.net integration (eval loop, timeout, memory ceiling)
Permission model: manifest declarations, admin grant/revoke, DB schema (extension_permissions table: extension_id, permission, granted, granted_by)
Runtime enforcement: sandbox loader checks granted permissions, injects only approved modules. Denied permissions → clean error, not silent no-op.
Admin UI: permission review on install, per-extension grant/revoke toggle, audit log of permission changes
Extension lifecycle states: install → pending_review → approved → active
Initial modules: secrets (vault-backed per-extension), notifications (emit to users)
Migration: extension_permissions table
ICD: update extensions.md with Starlark-specific endpoints
task_type: "starlark": task executor gains Starlark code path. Task references a .star script (inline or from a .pkg), executor runs it in the sandbox with task context (trigger payload, schedule info, previous run data). No LLM, no webhook relay — actual computation. RBAC gate from v0.28.7 enforced: task.starlark permission enabled in task_permissions (hidden until this version). Admin-grantable to team admins on a per-team basis.
```
**Two-track execution model (permanent):**
- `system` (v0.28.6): Go function registry — core platform ops
  (`retention_sweep`, `memory_compact`, etc.). Cannot break from
  bad user code. Admin-only, not editable, not versionable.
- `starlark`: custom admin/team admin tasks — nightly reports,
  data quality checks, integration sync, cleanup scripts. Editable,
  versionable, sandbox-isolated, permission-gated.
```
KB auto-injection: server-side pre-completion filter (Go built-in, not Starlark). Top-K chunk prepend from channel-bound KBs, context budget aware, per-channel toggle. Reference implementation of the server-side filter model that Starlark filters will mirror. Moved from v0.28.6 to validate alongside the Starlark filter architecture.

v0.29.1 — API Extensions

Starlark route handlers. Surfaces can serve custom JSON endpoints.

Depends on: v0.29.0 (sandbox + permissions).

api_routes manifest key: [{"method": "GET", "path": "/data", "handler": "handlers.list"}]
Routes mounted at /s/{surface_id}/api/..., auth context injected
Starlark request/response primitives (headers, body, status)
http outbound module: allowlist/blocklist per extension
requires_provider manifest key: extensions declare provider capability needs (e.g. "image_generation"). Platform resolves via existing provider chain (BYOK → team → global → routing policy). Starlark provider module makes calls on behalf of extension — raw API keys never exposed to sandbox.
Server-side tool execution in completion handler (Starlark tools run server-side, browser tools run client-side — unified tool schema)
Multi-file asset routing: ServeExtensionAsset path-based lookup (replaces inline _script for all requests)
Deduplicate tool schema extraction: unify ListBrowserToolSchemas and ListTools into shared helper handling both browser and server-side tools

v0.29.2 — DB Extensions

Namespaced tables for extension data. Create-only (no migrations yet).

Depends on: v0.29.1 (API extensions — handlers need somewhere to persist).

Namespaced tables: ext_{surface_id}_* prefix enforced by platform
Declarative schema in manifest: "tables": [{"name": "items", "columns": [...]}] Platform generates dialect-correct DDL (PG + SQLite)
db Starlark module: query(), exec() scoped to extension tables + declared views
Views as read contract: extension declares views over platform tables with explicit column allowlist — never direct access to platform tables
SQL validation: db module rejects references to tables outside namespace
Schema creation on install, drop on uninstall

v0.29.3 — Workflow Forms

form_template renders as real UI. The LLM moves to the corner — present if configured, but not required for data collection. Workflows become structured processes, not guided conversations.

Depends on: v0.29.2 (DB extensions — structured form data persists in ext_workflow_* tables, not just stage_data JSON blob), v0.29.0 (Starlark — .star validators run on form submission).

Form rendering:

form_template schema: typed field definitions (text, email, select, number, date, textarea, checkbox, file). Validation rules per field (required, pattern, min/max). Current freeform JSON → structured schema with backward compat (old JSON treated as textarea field prompts for the LLM)
Stage renders as form when form_template has typed fields. HTML form generation from schema, client-side validation, styled with platform CSS custom properties
Form submission → stage_data merge (same MergeWorkflowStageData path as chat-based data collection, but from structured fields instead of LLM extraction)
Stage advance on form submit: if auto_transition: true and all required fields pass validation, advance without human operator or LLM involvement

LLM-optional stages:

Stage without persona_id renders form only — no chat pane, no completion endpoint, pure data collection
Stage with persona_id + form_template renders form + chat side-by-side: user fills form, LLM assists (auto-fill suggestions, field explanations, validation help)
Stage with persona_id + no form_template — current behavior (pure chat-based collection, no change)

Starlark validation:

validate Starlark hook: manifest declares .star validator per stage. Runs on form submit before advance. Returns field errors or approval. Uses db module for cross-reference validation (e.g., "is this email already in the system?")
on_submit Starlark hook: post-validation side effects (create records, send notifications, call APIs via http module)

Visitor form entry:

Visitor entry point renders form for non-chat stages (branded page with form, no chat widget). Session-scoped same as current visitor auth model
Progressive: visitor sees form → submits → next stage (may be form or chat or operator review)

Team admin UX:

Form builder: visual field editor in workflow admin (drag fields, set types/validation, preview). Replaces raw JSON textarea for form_template. Admin-only initially, team admin in v0.30.2.

v0.30.0 — Package Lifecycle

Lifecycle sophistication for the .pkg format established in v0.28.7. Schema management, settings integration, and ecosystem features.

Depends on: v0.29.2 (DB extensions — schema declarations must exist before versioning and migrations make sense).

Install creates schema + mounts routes + serves assets in one operation (extends v0.28.7 install with v0.29.2 schema support)
Schema versioning: "schema_version": N in manifest
Schema migrations: "migrations": [{"from": 1, "to": 2, "sql": "..."}]
Settings extension point: packages declare settings sections in manifest, platform settings surface renders them alongside core sections
Export/import format for sharing packages across instances
Package marketplace (discovery, not hosting — instances pull from URLs)

v0.30.1 — SDK Adoption

Migrate core surfaces to consume the SDK instead of raw globals. Prerequisite for the Editor Package — if the platform's own surfaces don't use sw.*, the SDK isn't validated for real.

The goal: a bug fix in sw.notes() or sw.chat() propagates to every surface automatically. No more duplicated wiring, no more "fixed on chat but broken on editor" class of bugs.

Depends on: v0.28.5 (SDK exists), v0.30.0 (package lifecycle — settings extension point needed for surface-specific config).

Component factories:

sw.notes(container, opts) — real factory: creates NoteEditor, note list, graph panel. Extracted from notes.js monolith into self-contained mountable component. opts: { projectId, channelId, onLink, standalone }. Returns instance with destroy().
sw.chat(container, opts) — extend existing wrapper with pipe integration (stream/render filters auto-wired), model selector binding, file upload, typing indicator. Currently thin wrapper around ChatPane.create() — needs to become the canonical path.
sw.panels(container, opts) — PaneContainer composition via SDK. Resize, tab management, drag handles. Fixes the "scaling bug lives in two places" problem — one implementation, one fix.

Core surface migration:

Chat surface (app.js, chat.js): boot via sw = Switchboard.init(), replace direct API.* / Events.* / UI.* calls with sw.* in new code paths. Existing code keeps working (globals still exist). Progressive — not a rewrite, just new code uses the SDK.
Editor surface (editor-surface.js): mount chat pane via sw.chat(), mount notes via sw.notes(), consume sw.theme.on('change') for CM6 theme sync. Removes duplicated panel resize/scale wiring.
Notes surface: mount via sw.notes(), share implementation with chat sidebar notes panel.
Settings surface: consume sw.api.* and sw.theme.*.

Validation:

Existing ICD runner tests still pass (surfaces behave identically)
ICD runner sdk tier extended with component mount/destroy tests
Manual: chat, editor, notes surfaces all function after migration
Bug fix propagation test: fix applied to sw.notes() visible on both chat sidebar and notes surface simultaneously

v0.30.2 — Workflow Packages

Workflows become packageable. Each stage can serve its own surface — form, dashboard, chat, or custom UI. Team admins build and manage workflows through a visual builder, not JSON editing.

Depends on: v0.29.3 (workflow forms — form rendering exists), v0.30.0 (package lifecycle — schema versioning, settings extension point), v0.30.1 (SDK adoption — stage surfaces consume sw.*).

Stage surfaces:

A workflow stage can reference a surface (built-in or from a .pkg). Stage config gains surface_id field — platform mounts that surface when the stage is active, with stage data injected as context
Built-in stage surfaces: form (v0.29.3 form renderer), chat (current behavior), review (operator sees collected data + approve/reject)
Custom stage surfaces: a .pkg with type: "workflow-stage" provides a surface that receives stage data and emits advance/reject signals via sw.workflow.advance(data) / sw.workflow.reject(reason) SDK calls
Stage transitions respect surface completion — surface signals "done" with structured data, workflow engine merges and advances

Workflow-as-package:

.pkg with type: "workflow": bundles workflow definition, stage surfaces, Starlark handlers, and assets in one installable archive. POST /admin/packages/install creates the workflow + stages + surfaces
Manifest "workflow" key: stages, transitions, form schemas, surface refs, on_complete hooks. Replaces manual workflow creation via admin API
Version pinning: installed workflow package tracks its package version. Upgrade re-publishes with new version number

Team admin workflow builder:

Visual stage editor: drag-to-reorder stages, per-stage config panel (surface type, persona, form fields, transition rules, assignment)
Form builder integrated per stage (extends v0.29.3 admin form builder)
Preview: team admin can walk through the workflow as a visitor would, seeing each stage surface in sequence
Publish/version from the builder UI — no JSON, no API calls
Team admin permission: workflow.manage — team admins can create, edit, publish workflows for their team. Global admin can create global workflows. Regular users can only enter workflows, not build them

SDK extensions:

sw.workflow namespace: advance(data), reject(reason), getData(), getStage(), onTransition(fn) — stage surfaces interact with the workflow engine through the SDK, not raw API calls

v0.31.0 — Editor Package

Rebuild the editor as an installable .pkg package. Zero platform special-casing — same POST /admin/packages/install, same manifest, same /s/:slug route as any third-party package. If pages.go or main.go need changes to support it, the platform abstraction is wrong, not the editor.

Validates the full v0.28.7–v0.30.2 stack E2E: unified packaging, Starlark handlers, DB extensions, package lifecycle, SDK adoption, and workflow packages all exercised by one real-world package.

Depends on: v0.30.2 (workflow packages — editor validates the full stack including workflow stage surfaces).

Platform primitives consumed (not owned):

CM6 (core UI primitive — chat input, notes, code blocks all use it)
Workspace tools (workspace_ls, workspace_read, workspace_write)
Git tools + credentials (v0.28.0 — core settings, vault-encrypted)
Provider resolution via requires_provider (v0.29.1)

Package delivers:

Editor .pkg archive (type: full): manifest, JS, CSS, Starlark handlers
Built separately, installed via admin API — follows exact same patterns as any other package
Editor settings section (via v0.30.0 settings extension point): keybindings (vim/emacs/standard), font size, editor theme
Editor state persistence via ext_editor_* tables: open tabs, cursor positions, split layout, per-workspace config
File tree, tab bar, multi-pane layout — all via package JS, consuming platform CSS custom properties
Markdown preview (improved renderer, shared with notes surface)
Remove editor from core: delete surface-editor template, "editor" manifest in pages.go, "editor" data loader, editor-surface.js

TBD (unscheduled — real features, no immediate need)

Items that are real but don't yet have a version assignment. Pull left based on need.

Surfaces + Editor

Article drag-to-reorder outline sections
Article-specific AI tools: suggest_outline, expand_section, check_citations
Mobile: mode selector collapses to hamburger/bottom nav
Surface IDE: built-in surface for building surfaces (Go template editor, JS/CSS editor, live preview in sandboxed region)
Project-bound surface/pane defaults: project config specifies which panes are available and default layout

Workspace + Git

Integration tests: clone, commit, push/pull cycle (requires git binary in CI)

Provider Health + Routing

Latency-aware routing: track response time percentiles, prefer faster providers
Cost-aware routing with budget ceiling per request
Provider profile editor: key-value config per provider type, preview of effective settings
Fallback chain visualizer: drag-to-reorder provider priority per model family

Extensions

Image generation tool (browser or sidecar, provider-agnostic)
STT/TTS (browser extension, Web Speech API)
Code execution sandbox (server-side, container isolation)
Sidecar HTTP tool protocol (Tier 2 — container isolation)

Desktop + Mobile

Desktop app (Tauri)
Full PWA with offline capability
Mobile-optimized layouts (beyond current responsive)

Data + Portability

Bulk export/import (account data, conversations, settings)
ChatGPT/other tool import
GDPR-style "download my data"
Backup/restore CronJob manifests
Admin settings team/user export (user export blocked by vault-encrypted BYOK keys; team export needs merge-vs-replace semantics)

UX / Multi-Seat

"Group" scope badge on model selector / KB list: access-source annotation so users see why they have access (global, team, group grant)
Multi-tenant SaaS mode
Plugin/extension marketplace

Projects

/p/:id — shared project view
Project-specific files: full project-level upload (own endpoint, storage path)
Project templates: create from predefined configurations
Project creation dialog: replace prompt() with proper modal
Admin-level project management: cross-instance visibility, ownership reassign
Sub-projects / nested hierarchy

Knowledge Bases

Hybrid search: combine vector similarity with full-text tsvector, re-rank
Semantic chunking: embedding-based boundary detection for smarter splits
HNSW index: better query performance than IVFFlat for large datasets
Web scraping source: ingest URLs as KB documents (extends url_fetch)
Scheduled re-indexing: periodic rebuild when source documents update

Memory

Memory export/import: portable memory format across instances
Cross-persona memory sharing (opt-in)
Memory analytics: dashboard showing what Personas are learning, growth trends

Platform

Rate limiting per user/team/tier (token budgets beyond current group budgets)

Pre-1.0 — Code Quality Sweeps

Codebase-wide refactors required before 1.0. Not tied to a feature version — pull left when touching the affected code, or schedule a dedicated pass.

SELECT * → Explicit Column Lists All store implementations use SELECT * FROM <table> with positional rows.Scan(). If a migration adds a column, every scan breaks silently at runtime (wrong column mapped to wrong field). Every scanOne/scanMany helper needs explicit column lists. Scope: every store file in store/postgres/ and store/sqlite/.

Raw SQL Hunt Several handlers execute raw SQL via database.DB.Exec / database.TestDB.Exec instead of going through the store interface. These bypass the dialect abstraction and are a source of PG/SQLite divergence bugs. Known instances:

handlers/channels.go — inline channel queries with pq.Array
handlers/workflows.go — inline unique constraint string matching
handlers/participants.go — isDuplicateErr string matching (should use database.IsUniqueViolation)
knowledge/ handlers — direct database.DB.ExecContext for storage key updates Full sweep: grep for database.DB.Exec, database.TestDB.Exec, DB.QueryRow outside of store/ packages. Move to store methods or use existing dialect-safe helpers.

43 KiB Raw Blame History Unescape Escape