Changeset 0.26.0 (#165)

This commit is contained in:
2026-03-10 13:38:01 +00:00
parent dbc1a97343
commit 400f7dd176
48 changed files with 4923 additions and 208 deletions

View File

@@ -1,4 +1,4 @@
# Architecture — Chat Switchboard v0.20
# Architecture — Chat Switchboard v0.26
## Deployment Modes
@@ -47,34 +47,42 @@ Three Docker images support different deployment scenarios:
6. **Channels as Execution Context**: Channels are the universal container for conversation state — messages, tool activity, notes, and artifacts all hang off a channel. Today channels are single-user direct chats. The architecture anticipates multi-participant channels (team members, anonymous visitors, AI personas) for workflow execution, without requiring changes to the message tree, tool framework, or streaming infrastructure.
## Workflow Architecture (Future — v0.25.0+)
## Workflow Architecture (v0.26.0 — Shipped)
The platform's existing primitives (teams, personas, channels, tools, notes)
compose into a workflow engine where the channel is the execution context
that moves through defined stages.
### Conceptual Model
### Data Model
```
Team
└─ Workflow (team-admin defined)
├─ name, description
├─ entry conditions (public link, team-internal, API trigger)
Stages[]
└─ Workflow (admin-defined, team-scoped or global)
├─ name, slug, description, branding (JSONB)
├─ entry_mode (authenticated | public_link)
is_active, version (auto-incremented on edit)
└─ Stages[] (ordered)
├─ persona_id (which AI drives this stage)
├─ assignment_team_id (who can be assigned)
├─ form_template (structured note schema, optional)
transitions[] (conditions → next stage)
├─ system_prompt (stage-specific override)
├─ form_template (JSONB — fields to collect)
history_mode (full | summary | fresh)
└─ assignment_team_id (who can be assigned)
Channel (workflow instance)
├─ workflow_id + current_stage
├─ workflow_id + workflow_version + current_stage
├─ stage_data (JSONB — accumulated form data across stages)
├─ workflow_status (active | completed | stale | cancelled)
├─ participants[]
│ ├─ anonymous visitor (mTLS fingerprint / session token)
│ ├─ anonymous visitor (session token)
│ ├─ AI persona (per-stage, from workflow definition)
│ └─ assigned team member (claimed or auto-routed)
│ └─ assigned team member (claimed from assignment queue)
├─ messages (existing tree structure)
─ notes (channel-scoped artifacts, intake forms)
└─ tool activity (existing execution framework)
─ notes (stage data persisted as channel-scoped notes)
WorkflowAssignment (queue entry)
├─ channel_id + stage + team_id
├─ assigned_to (nullable — claimed by team member)
└─ status (unassigned | claimed | completed)
```
### How Existing Primitives Map
@@ -84,33 +92,46 @@ Channel (workflow instance)
| **Persona** (system prompt + model) | Drives a workflow stage — the AI knows what to collect, when to route |
| **Team** (members + roles) | Owns the workflow definition; members are assignable to channels |
| **Channel** (messages + tree) | Execution instance of a workflow; full conversation history |
| **Notes + Tools** | Structured data collection; the persona's system prompt *is* the form definition, the note *is* the filled form |
| **Notes + Tools** | Structured data collection; `workflow_advance` tool triggers transitions, notes persist form data |
| **EventBus + WebSocket** | Real-time notifications for assignment, stage transitions, new messages |
| **Anonymous Sessions** (v0.24.3) | Visitors participate without user accounts |
| **DenyVisitor** predicate | Scopes tool availability per participant type |
### Design Constraints for Current Development
### Stage Transition Flow
These invariants keep the workflow path open without building it prematurely:
```
Visitor starts → Channel created (stage 0, persona bound)
Persona collects form data via conversation
Persona calls workflow_advance(data) → Stage notes created
│ │
▼ ▼
Next stage: new persona bound If assignment_team_id set:
(or workflow completes) → workflow_assignment created
→ team member claims
→ member + persona collaborate
```
- **Channels**: Don't assume single-owner. If touching channel queries, keep
room for `team_id`, `type` (beyond `direct`), and multi-participant access
patterns alongside `user_id`.
- **Notes**: User-scoped with `[[wikilink]]` bi-directional linking (v0.17.3).
The `note_links` junction table tracks directed edges between notes, with
`target_note_id` nullable for dangling links (unresolved references). Links
are extracted from content on save via regex, resolved by title match, and
re-resolved when new notes are created. The graph endpoint returns all nodes,
edges, and unresolved links for Canvas-based force-directed visualization.
Future channel-scoped notes (attached to a conversation) need a `channel_id`
FK option. `source_message_id` enables jump-to-source provenance from notes
back to the originating chat message.
- **Personas**: Already team-scoped. Don't couple to "user picks from dropdown"
— a workflow stage references a persona programmatically.
- **Tool ExecutionContext**: Already carries `UserID` + `ChannelID`. Will need
`TeamID` and `WorkflowID` — the struct is easily extended.
- **Auth**: mTLS anonymous users need identity to participate in channels.
Lightest version: a `participants` table that can reference a `user_id` or
an opaque session identifier (cert fingerprint). Don't assume every channel
participant has a row in `users`.
### Key Implementation Details
- **Form template injection.** When `channelType == "workflow"`, the completion
handler loads the current stage's `form_template` JSONB and injects a system
prompt telling the persona what fields to collect and when to call
`workflow_advance`.
- **Dialect-safe stage data merge.** `MergeWorkflowStageData()` reads existing
JSON from the `stage_data` column, merges in Go, writes back. No
Postgres-specific JSONB operators — works on both Postgres and SQLite.
- **Optimistic claim lock.** `UPDATE workflow_assignments SET assigned_to = $1
WHERE id = $2 AND status = 'unassigned'` — if rows_affected == 0, another
team member already claimed it.
- **Staleness sweep.** Background goroutine (1h tick) marks workflow instances
as `stale` when `last_activity_at` exceeds `WORKFLOW_STALE_HOURS`.
## Package Structure

394
docs/DESIGN-0.26.0.md Normal file
View File

@@ -0,0 +1,394 @@
# DESIGN — v0.26.0: Workflow Engine
**Status:** Shipped (v0.26.0v0.26.5, 15 changesets)
**Branch:** `0.26.0`
**Scope:** Team-owned staged processes, AI intake, human assignment, visitor experience
**Depends on:** Dynamic surfaces (v0.25.0), multi-participant channels (v0.23.x),
anonymous identity (v0.24.3), permissions (v0.24.2)
---
## Phasing
Six sub-releases. Each is shippable and testable independently.
Deferred tech debt from v0.21v0.25 absorbed into v0.26.0 (phase 0).
```
v0.26.0 Foundation: cleanup + context-aware tool system
v0.26.1 Workflow definitions + versioning (schema + CRUD)
v0.26.2 Workflow instances + stage transitions (runtime)
v0.26.3 Visitor experience (surfaces + branded chat)
v0.26.4 AI intake + assignment queue
v0.26.5 Team collaboration + workflow builder UI
```
---
## v0.26.0 — Foundation
Cleanup debt, then build the tool infrastructure that workflows need.
### Deferred Cleanup (absorb from v0.21v0.25)
**Session cleanup job**
- Background goroutine: delete `session_participants` older than N days
with no associated messages
- Configurable via `SESSION_EXPIRY_DAYS` env var (default: 30)
- Runs on startup + every 6h
- Closes v0.24.3 deferred item
**Stale code TODOs**
- `surfaces.go:108-109` — surface delete: clean up static assets + templates
from `SURFACE_ASSET_DIR/{id}/` and `SURFACE_TEMPLATE_DIR/{id}/`
- `completion.go:164` — remove misleading TODO comment (pricing IS populated
in `recordUsage()` at line ~1996)
- `intrinsic.go:19` — remove stale "TODO: 0.9.2" comment (shipped in v0.22.0)
**Roadmap housekeeping**
- Mark shipped items in "Technical Debt + Deferred Items" section:
- Rate limit tracking (v0.22.4)
- Auto-disable policy (v0.22.4)
- Tool health recording (v0.22.4)
- PDF export via pandoc (v0.22.4)
- Persona tool grant enforcement (v0.25.0)
- Clean up stale deferred-into version markers
### Context-Aware Tool System
_Absorbed from v0.25.0 roadmap. Prerequisite for workflow tool scoping._
**`ToolContext` struct** (`tools/types.go`)
- Fields: `ChannelType`, `WorkspaceID`, `WorkflowID`, `TeamID`,
`PersonaID`, `IsVisitor`
- Populated from channel record in completion handler before tool execution
**`Require` predicates**
- `Availability() Require` method on `Tool` interface
- `BaseTool` embed: defaults to `AlwaysAvailable` (backward compat)
- Built-in predicates: `RequireWorkspace`, `RequireWorkflow`,
`RequireTeam`, `DenyVisitor`, `All()`
- `tools.AvailableFor(tctx, disabled)` replaces `AllDefinitionsFiltered()`
**Self-declaring tools**
- Workspace tools: `RequireWorkspace + DenyVisitor`
- Git tools: `RequireWorkspace + DenyVisitor`
- `workspace_create`: available only when no workspace bound
- Memory/notes tools: `DenyVisitor`
- Eliminates `WorkspaceToolNames()` / `GitToolNames()` manual suppression
**`ExecutionContext` extension**
- Add `WorkflowID`, `TeamID` to existing `ExecutionContext`
- Populated from channel record in completion handler
**Persona tool grant enforcement hardening**
- v0.25.0 shipped the second-pass allowlist in completion.go:917
- For workflows: version snapshot must include persona tool grants
at snapshot time (frozen for running instances)
### Route Namespaces
Two distinct namespaces. No collision.
**`/s/:slug` — Extension Surfaces (application extensions)**
- Single-page, self-contained sub-applications
- Examples: custom dashboard, kanban board, form builder
- One template, one JS entry point, scoped CSS
- Registered via surface manifest (v0.25.0 `surface_registry`)
**`/w/:scope/:slug` — Workflows (business logic automation)**
- `:scope` is `team-slug` or `global` — workflows belong to a team or
are org-wide. Prevents slug collisions across teams.
- Multi-page: landing, visitor chat, team review, admin tracking —
different views depending on who is accessing (visitor vs. team
member vs. admin)
- Involves customers and team members, possibly spanning multiple teams
- Purpose-built Go templates per view, not a single-page pattern
Both use Go template rendering. They share `base.html` but have
independent template trees and JS entry points.
### Migration
- 023_v0260_foundation.sql: session cleanup additions (if schema changes
needed), no new tables in this phase
---
## v0.26.1 — Workflow Definitions + Versioning
Schema and CRUD for defining workflows. No runtime yet.
### Tables
**`workflows`**
- `id`, `team_id` (nullable — NULL = global), `name`,
`slug` (unique within scope: `UNIQUE(team_id, slug)` with
partial index for global), `description`
- `branding` (JSONB: accent color, logo URL, tagline)
- `entry_mode` (enum: `public_link`, `team_only`)
- `is_active`, `version` (auto-increment on edit)
- `on_complete` (JSONB, nullable — v0.27.0 chaining hook, NULL for now.
Future schema: `{"action": "start_workflow", "target_slug": "...",
"data_map": {...}}`. Column exists early so the table doesn't need
migration when chaining lands.)
- `retention` (JSONB — `{"mode": "archive"|"delete", "delete_after_days": N}`.
Default: `{"mode": "archive"}`)
- `created_by`, `created_at`, `updated_at`
**`workflow_stages`**
- `id`, `workflow_id`, `ordinal`, `name`
- `persona_id` (FK — persona drives this stage)
- `assignment_team_id` (FK — which team handles human review)
- `form_template` (JSONB — fields the persona should collect)
- `history_mode` (enum: `full`, `summary`, `fresh` — default `full`)
- `auto_transition` (bool — advance automatically when form complete)
- `transition_rules` (JSONB — conditions, round-robin config)
**`workflow_versions`**
- `id`, `workflow_id`, `version_number`
- `snapshot` (JSONB — full serialized definition + stages + tool grants)
- `created_at`
### API
- Team-admin CRUD: create/edit/delete workflows and stages
- Reorder stages (PATCH ordinal)
- `workflow.create` permission required
- Publish action: snapshot current definition → `workflow_versions`
- Slug validation: lowercase, alphanumeric + hyphens, unique within scope
(same slug can exist under different teams). URL resolves as
`/w/team-slug/workflow-slug` or `/w/global/workflow-slug`.
### Design Decisions (to flesh out)
- Branding schema: minimal (accent + logo + tagline) or extensible JSONB?
- Stage persona: must be team-scoped persona? Or any accessible persona?
### Migration
- 023_v0261_workflows.sql: `workflows`, `workflow_stages`, `workflow_versions`
---
## v0.26.2 — Workflow Instances + Stage Transitions
Channels become workflow runtime containers.
### Channel Extensions
- `workflow_id`, `workflow_version` columns on `channels`
(populated on workflow channel creation)
- `current_stage` (ordinal), `stage_data` (JSONB)
- `workflow_status` enum: `active`, `completed`, `stale`, `cancelled`
- `last_activity_at` tracking
### Entry Point
- Public link: `/w/:scope/:slug` → creates workflow channel, adds anonymous
session participant (v0.24.3), binds stage 1 persona, auto-creates
workspace. `:scope` is team slug or `global`.
- Team-only: same flow but requires authenticated user
### Stage Transitions
- AI-triggered: `workflow_advance` tool (see v0.26.4)
- Human-triggered: button in channel header
- Rejection: return to previous stage with reason text
- On transition: swap active persona, notify assignment team,
update channel metadata, create channel-scoped note with collected data
**History mode** (per-stage, configurable in `workflow_stages`):
- `full` — new persona sees complete conversation history with a system
boundary message (same pattern as @mention context boundaries, v0.23.0).
Simplest to implement, best for continuity-sensitive workflows.
- `summary` — utility-role summarization of prior stages injected as a
system message. New persona starts with context but not raw history.
- `fresh` — new persona sees nothing from prior stages. Clean slate.
Simplest for independent review stages.
Default: `full`. Stored as `history_mode` enum on `workflow_stages`.
### Staleness Sweep
- Background goroutine (like health accumulator pattern)
- Marks idle instances as `stale` after configurable threshold
- Stale UX: visitor sees "Continue or Start Over"
(start over = new instance on latest version)
### Migration
- Adds columns to `channels`, or separate `workflow_instances` table
(TBD — inline columns simpler, separate table cleaner)
---
## v0.26.3 — Visitor Experience
Purpose-built surfaces for anonymous workflow participants.
### Landing Page
- `GET /w/:scope/:slug` — branded page with persona avatar, description, "Start"
- Go template: `workflow-landing.html`
- Reads `workflow.branding` JSONB for accent color, logo, tagline
- System dark/light mode (no theme toggle — visitors don't have prefs)
### Visitor Chat
- `GET /w/:scope/:slug/c/:channelId` — bubble chat (NOT full ChatPane)
- `workflow-chat.js`: lightweight WebSocket client
- Message send/receive
- Markdown rendering (marked.js + DOMPurify, already available)
- Typing indicator
- Stage transition animation (visual feedback on advance)
- Scoped: no sidebar, no settings, no navigation
- Session auth via `AuthOrSession` middleware (v0.24.3)
### Tool Scoping
- `DenyVisitor` predicates prevent visitors from accessing workspace,
git, memory, notes tools
- Persona tool grants further restrict per-stage
### Design Decisions (to flesh out)
- Visitor can upload files? (attachments to workflow channel)
- Visitor can see previous stage history? (probably not — each stage
is a clean persona conversation)
- Mobile-first layout for visitor surfaces
---
## v0.26.4 — AI Intake + Assignment Queue
The AI does the data collection. Humans review and act.
### `workflow_advance` Tool
- `RequireWorkflow` availability predicate
- Input: collected form data (JSONB matching `form_template`)
- Validates all required fields present
- Triggers stage transition (same path as human-triggered)
- Creates channel-scoped note with structured form response
### Form Template → System Prompt
- Stage `form_template` JSONB injected into persona system prompt:
"Collect the following information from the visitor: [field list]"
- Persona conducts conversational intake
- When all fields gathered → calls `workflow_advance` with data
- Rejection: persona sees rejection reason in conversation, re-collects
### Assignment Queue
**`workflow_assignments` table**
- `id`, `channel_id`, `stage` (ordinal), `team_id`
- `assigned_to` (nullable user_id), `status` (unassigned/claimed/completed)
- `created_at`, `claimed_at`, `completed_at`
**Claim model**
- Unassigned workflow channels visible to all team members
- Optimistic lock: `UPDATE ... WHERE status = 'unassigned'`
- Round-robin auto-assignment (optional, per-stage `transition_rules`)
**Notifications**
- New workflow assignment → notification to team (existing infra)
- WebSocket push to online team members
- Claim confirmation notification to claimer
### Migration
- `workflow_assignments` table
---
## v0.26.5 — Team Collaboration + Workflow Builder UI
Frontend surfaces for building and managing workflows.
### Team Member View
- Assigned member sees full history (AI intake + artifacts + notes)
- Persona remains active — assists both visitor and team member
- Member can: advance stage, reject (with reason), add notes,
invoke tools, reassign, escalate
### Workflow Builder
- Team admin panel section
- Stage list editor: add/remove/reorder stages
- Per-stage config: persona picker, assignment team, form template editor
- Branding editor: accent color, logo upload, tagline
- Publish button (creates version snapshot)
- Read-only tool grants display per stage persona
### Channel Header (Workflow Mode)
- Workflow stage indicator (step N of M, stage name)
- Advance / Reject buttons (permission-gated)
- Assignment info (who claimed, when)
- History mode indicator (full/summary/fresh)
### Queue UI
Prototype both approaches in v0.26.5, decide based on usage:
**Option A: Sidebar section** for team members — shows unassigned workflow
channels for user's teams, count badge, click opens in main pane.
Low friction, always visible.
**Option B: Dedicated surface** (`/workflows` or team-admin view) for
tracking and management — full table with filters, assignment history,
metrics. Better for high-volume operations.
Likely outcome: both. Sidebar for team members (day-to-day claims),
dedicated surface for team admins (tracking, reporting, bulk ops).
---
## Resolved Decisions
1. **Workflow channel lifecycle.** Configurable per-workflow retention
policy. Default: archive (read-only, `ai_mode='off'`, no new messages).
Optional: auto-delete after N days. Stored as `retention` JSONB on
`workflows` table.
2. **Multi-stage persona conversations.** Three modes, configurable
per-stage via `history_mode` enum on `workflow_stages`:
- `full` — complete history + boundary message (default, simplest)
- `summary` — utility-role summarization injected as system message
- `fresh` — clean slate, no prior context
First and third are simplest to implement; ship those first,
`summary` can land as a follow-up.
3. **Visitor re-entry.** Resume if session cookie matches and instance
is `active`. Otherwise start new on latest version.
4. **Workflow-to-workflow chaining.** Deferred to v0.27.0 (tasks/agents).
`on_complete` JSONB column on `workflows` table ships as nullable in
v0.26.1 migration so the schema is pre-wired. v0.27.0 populates it
and adds the trigger logic. No migration needed when chaining lands.
5. **Route namespaces.** `/s/:slug` for extension surfaces (single-page
sub-apps). `/w/:scope/:slug` for workflows (multi-page, role-dependent
views). No collision. See "Route Namespaces" in v0.26.0.
---
## Cross-Cutting Concerns
**Testing strategy:** Integration tests per phase. v0.26.0 tool context
tests. v0.26.1 workflow CRUD + versioning tests. v0.26.2 stage transition
tests. v0.26.4 assignment claim concurrency test.
**Migration numbering:** Single migration per phase (023, 024, ...) or
consolidated? Leaning toward: one per phase, squash at tag time if needed.
**Backward compat:** Non-workflow channels unaffected. `ToolContext` with
`BaseTool` defaults means existing tools work without changes. Workflow
columns on `channels` are nullable.

View File

@@ -127,12 +127,27 @@ v0.24.0 Auth Abstraction + User Identity ✅
└── v0.24.2 Fine-Grained Permissions (parallel) ✅
v0.25.0 Dynamic Surfaces + Context-Aware Tools
(pane system, manifest-driven surfaces,
editor rebuild, tool predicates,
persona tool grants)
v0.25.0 Dynamic Surfaces + Component Extraction ✅
(Go template engine, base.html shell, surface
manifests, admin/settings/editor surfaces,
pane container, persona tool grants UI)
v0.26.0 Workflow Engine
v0.25.1 Surface Integration Fixes ✅
v0.25.2 Drag-Resize Utility ✅
(unified primitive for workspace + pane splits)
v0.25.3 UI Bug Cleanup ✅
(theme system, scaling, debug modal, settings
back-button, profile save, resize bar)
v0.25.4 Admin + Settings Surface Hardening ✅
(admin audit fixes, BYOK/persona toggles,
group permissions 500, settings surface init,
user settings NULL fix, export/import,
workspace mgmt, multi-provider test failover)
v0.26.0 Workflow Engine ✅
(team-owned staged processes,
human + AI collab, visitor
intake, assignment queue)
@@ -339,6 +354,70 @@ avatar upload, confirm dialog. Settings feature gates (BYOK, Personas).
to settings appearance. Admin hybrid section loaders. Naming cleanup:
preset → persona, APIConfigID → ProviderConfigID.
### ✅ v0.23.0 — @mention Routing + Persona Handles + Proxy
@mention resolution in any chat against full model catalog + persona registry.
AI-to-AI chaining with depth cap. Provider proxy mode (system/direct/custom).
`channel_participants` + `channel_models` tables. Persona handles.
### ✅ v0.23.1 — Multi-User Navigation + Conversation Taxonomy
Five-type conversation taxonomy (direct/dm/group/channel/workflow). Three-section
sidebar (Projects → Channels → Chats). Folder system. DM plumbing. Channel
`ai_mode` (auto/mention_only/off). Presence heartbeat. Migration 016.
### ✅ v0.23.2 — Multi-User Polish + Channel Lifecycle
Unified `App.activeConversation` replacing split tracking. Group leader
routing, `@all` fan-out. Channel archive/delete with retention policy.
Human message attribution. Unread cursor via `last_read_at`. Migration 017.
### ✅ v0.24.0 — Auth Abstraction + User Identity
`auth.Provider` interface decoupling builtin auth from handlers. User model
extended with `auth_source`, `external_id`, `handle`. @mention resolution
upgraded to handles. `UniqueHandle()` collision-safe generation. Migration 018.
### ✅ v0.24.1 — mTLS + OIDC Providers
mTLS via reverse proxy headers + cert DN parsing. OIDC authorization code flow
with JWKS caching and split-horizon issuer. Keycloak docker-compose overlay.
`QArgs()` dialect adapter for SQLite placeholder expansion. Migration 019.
### ✅ v0.24.2 — Fine-Grained Permissions
12 permission constants (`domain.action`). Per-group permissions, token budgets,
model allowlists. Everyone group (system-seeded, editable). `RequirePermission()`
middleware. BYOK bypass for personal providers. Migration 020.
### ✅ v0.24.3 — Anonymous Sessions
Unauthenticated visitors for workflow intake. `session_participants` table,
`AuthOrSession` middleware, cookie-based + mTLS session paths. Channel
`allow_anonymous` flag. Workflow entry page at `/w/:id`. Migration 021.
### ✅ v0.25.0 — Dynamic Surfaces + Component Extraction
Go template engine with `base.html` shell, surface manifest registry (DB-backed,
admin-toggleable), five core surfaces (chat, admin, settings, editor, notes).
Component extraction: UserMenu, ModelSelector, FileTree, CodeEditor, NoteEditor.
PaneContainer for multi-pane layouts. Admin and settings promoted from modals to
full-page surfaces. CSS decomposed from monolithic `styles.css` into 13 files.
Persona tool grants UI. Migration 022: `surface_registry` table.
### ✅ v0.25.1 — Surface Integration Fixes
Post-deployment corrections for v0.25.0 surface migration.
### ✅ v0.25.2 — Drag-Resize Utility
Unified `drag-resize.js` primitive for workspace handle and pane split handles.
Full-viewport overlay prevents iframe/CM6 event swallowing. Touch support.
### ✅ v0.25.3 — UI Bug Cleanup
Theme system mode fix (system → dark fallback), resize bar jump-on-click,
debug modal styling, settings/admin back-button navigation, profile save
(wrong IDs, wrong API routes, empty user context), scale slider range/commit.
### ✅ v0.25.4 — Admin + Settings Surface Hardening
Admin audit: 6 navigation/wiring fixes, settings export/import, group permissions
500 (Postgres SetNull argIdx), BYOK/persona policy store mismatch. Settings surface:
full init pipeline (policies, models, handlers, visibility checks), user settings
NULL/null/array corruption fix, model roles notice states, bulk model visibility.
Chat surface: stale modal HTML cleanup (320 lines), team admin modal layout/wiring,
persona create button. Backend: project files admin bypass, workspace delete API,
multi-provider live test failover.
---
## Technical Debt + Deferred Items
@@ -347,13 +426,14 @@ Items deferred from completed releases that remain outstanding.
Organized by domain. Pull into a version when the surrounding work
makes them a natural addition.
**Surfaces + Editor (→ v0.25.0 natural home)**
- [ ] Extension loader: surfaces from manifest.json (was v0.21.3)
**Surfaces + Editor**
- [ ] Extension loader: surfaces from manifest.json (was v0.21.3`/s/:slug` route deferred to v0.27.0)
- [ ] Editor drag-drop file reorder (was v0.21.5)
- [ ] Article drag-to-reorder outline sections (was v0.21.6)
- [ ] Article-specific AI tools: suggest_outline, expand_section, check_citations (was v0.21.6)
- [ ] PDF export via pandoc (was v0.21.6)
- [x] ~~PDF export via pandoc~~ (shipped v0.22.4)
- [ ] Mobile: mode selector collapses to hamburger/bottom nav (was v0.21.3)
- [ ] Pane state persistence per-user/per-project (pane positions saved, not yet persisted across sessions)
**Workspace + Git**
- [ ] Workspace settings UI: git config section in channel/project settings (was v0.21.4)
@@ -362,9 +442,9 @@ makes them a natural addition.
- [ ] Integration tests: clone, commit, push/pull cycle — requires git binary in CI (was v0.21.4)
**Provider Health + Routing**
- [ ] `RecordOutcome` for web_search and url_fetch (search provider health tracking)
- [ ] Rate limit tracking per provider config (requests/minute counter in health accumulator)
- [ ] Auto-disable: mark provider inactive when `down` for N consecutive health windows
- [x] ~~`RecordOutcome` for web_search and url_fetch~~ (shipped v0.22.4 — tool health tracking)
- [x] ~~Rate limit tracking per provider config~~ (shipped v0.22.4 — `rate_limit_count` on `provider_health`)
- [x] ~~Auto-disable: mark provider inactive when `down` for N consecutive health windows~~ (shipped v0.22.4)
- [ ] `capability_match` policy type ("cheapest model with tool_calling")
- [ ] Latency-aware routing: track response time percentiles, prefer faster providers
- [ ] Cost-aware routing with budget ceiling per request
@@ -372,6 +452,12 @@ makes them a natural addition.
- [ ] Provider profile editor: key-value config per provider type, preview of effective settings
- [ ] Fallback chain visualizer: drag-to-reorder provider priority per model family
**Sessions + Auth**
- [x] ~~Session cleanup job~~ (shipped v0.26.0 — background goroutine, `SESSION_EXPIRY_DAYS`)
**Tool System**
- [x] ~~Persona tool grant enforcement~~ (shipped v0.25.0 — second-pass allowlist in completion handler)
---
@@ -624,13 +710,13 @@ See [DESIGN-0.24.0.md](DESIGN-0.24.0.md) §v0.24.3 for full spec.
---
## v0.25.0 — Dynamic Surfaces + Context-Aware Tools
## v0.25.0v0.25.4 — Dynamic Surfaces + Hardening ✅
Pane-based surface architecture and context-aware tool system.
Replaces the "one surface active at a time" model (v0.21.3) with
composable multi-pane layouts. Establishes manifest-driven surface
registration, component extraction, and the tool predicate system.
Editor surface rebuilt as proof-of-concept — eat our own dog food.
Pane-based surface architecture replacing the "one surface active at a
time" model (v0.21.3). Manifest-driven surface registration, component
extraction, admin/settings promoted to full-page surfaces. Editor
surface rebuilt as proof-of-concept. Five patch releases for integration
fixes, drag-resize, UI bugs, and admin/settings surface hardening.
Depends on: Go template engine (v0.22.5), ChatPane component (v0.22.7),
surface integration (v0.22.8). See [DESIGN-SURFACES.md](DESIGN-SURFACES.md)
@@ -638,136 +724,129 @@ for the layer model.
See [DESIGN-0.25.0.md](DESIGN-0.25.0.md) for full spec.
**Pane System**
- [ ] Pane container: workspace area holds 1+ panes side-by-side (replaces single-surface model)
- [ ] Pane lifecycle: create, mount, resize, minimize, destroy — independent of surface lifecycle
- [ ] Resizable splits: drag handles between panes, user-resizable, persisted per-user/per-project
- [ ] Layout presets: single (default), split-2 (editor+chat), split-3 (tree+editor+chat)
- [ ] Default: single Chat pane — zero cognitive overhead, identical to current UX
- [ ] Pane registry: named panes with independent state (scroll, selection, open file)
**Pane System**
- [x] Pane container: workspace area holds 1+ panes side-by-side (replaces single-surface model)
- [x] Pane lifecycle: create, mount, resize, minimize, destroy — independent of surface lifecycle
- [x] Resizable splits: drag handles between panes, user-resizable
- [x] Layout presets: single (default), split-2 (editor+chat), split-3 (tree+editor+chat)
- [x] Default: single Chat pane — zero cognitive overhead, identical to current UX
- [x] Unified drag-resize primitive (`drag-resize.js`): overlay prevents iframe/CM6 event swallowing, touch support (v0.25.2)
**Surface Manifest + Registration**
- [ ] Surface manifest schema: `id`, `route`, `title`, `components`, `data_requires`, `script`, `auth`
- [ ] Dynamic route generation in page engine from registered surfaces
- [ ] Data loader registry: surfaces declare data requirements, engine assembles loaders
- [ ] `/s/:slug` route namespace for extension/dynamic surfaces
- [ ] Core surfaces (chat, admin, settings, notes) migrated to manifest pattern
- [ ] `window.__PAGE_DATA__` injection from declared `data_requires` (existing pattern, formalized)
**Surface Manifest + Registration**
- [x] Surface manifest schema: `id`, `route`, `title`, `components`, `data_requires`, `script`, `auth`
- [x] Dynamic route generation in page engine from registered surfaces
- [x] Data loader registry: surfaces declare data requirements, engine assembles loaders
- [x] Core surfaces (chat, admin, settings, editor, notes) on manifest pattern
- [x] `window.__PAGE_DATA__` injection from declared `data_requires`
- [x] `surface_registry` table — admin can enable/disable surfaces (Migration 022)
- [ ] `/s/:slug` route namespace for extension/dynamic surfaces — deferred
**Component Extraction**
- [ ] FileTree: extracted from `editor-mode.js`, standalone component with Go template partial + JS hydration
- [ ] CodeEditor: CM6 `codeEditor()` factory formalized as component (already exists, needs instance pattern)
- [ ] NoteEditor: extracted from `notes.js`, standalone component
- [ ] ModelSelector: extracted from chat surface, reusable across surfaces
- [ ] Each component: Go template partial (server shell) + JS class (hydration) + CSS (scoped)
**Component Extraction**
- [x] FileTree: extracted from `editor-mode.js`, standalone component with Go template partial + JS hydration
- [x] CodeEditor: CM6 `codeEditor()` factory formalized as component
- [x] NoteEditor: extracted from `notes.js`, standalone component
- [x] ModelSelector: extracted from chat surface, reusable across surfaces
- [x] UserMenu: extracted as standalone component
- [x] Each component: Go template partial (server shell) + JS class (hydration) + CSS (scoped)
**Editor Surface Rebuild (Dog Food)**
- [ ] Editor surface rebuilt on pane system: file tree pane + code editor pane + ChatPane (assist)
- [ ] Surface manifest declares three panes with resizable splits
- [ ] File tree pane: workspace_ls backed, context menu, nested directories, file icons, git status badges
- [ ] Code editor pane: CM6 with language detection, tab bar, Ctrl+S save, auto-save on pane switch
- [ ] Chat pane: `ChatPane.create()` in assist role — same channel context, workspace-scoped tools
- [ ] Proves: multi-pane layout, ChatPane embedding, resizable splits, surface lifecycle
- [ ] Replaces broken `editor-mode.js` (48K, dead since v0.22.6 code pruning)
**Editor Surface Rebuild (Dog Food)**
- [x] Editor surface rebuilt on pane system: file tree pane + code editor pane + ChatPane (assist)
- [x] Surface manifest declares three panes with resizable splits
- [x] File tree pane: workspace_ls backed, context menu, nested directories
- [x] Code editor pane: CM6 with language detection, tab bar, Ctrl+S save
- [x] Chat pane: `ChatPane.create()` in assist role — same channel context, workspace-scoped tools
- [x] Replaces broken `editor-mode.js` (48K, dead since v0.22.6 code pruning)
**Shared Surface Routes**
- [ ] `/s/editor/:wsId` — editor surface via direct URL (replaces hash routing for external links)
- [ ] `/s/article/:wsId/:path` — article surface via direct URL
- [ ] `/p/:id` — shared project view (server-rendered shell)
- [ ] Auth gating: public pages skip auth, authenticated pages require JWT
**Admin + Settings Surfaces**
- [x] Admin surface at `/admin/:section` — full-page, category tabs, section scaffold
- [x] Settings surface at `/settings/:section` — full-page, left nav, section-specific templates
- [x] Settings export/import (versioned JSON envelope) (v0.25.4)
- [x] Bulk model visibility (Show All / Hide All) (v0.25.4)
- [x] Workspace rename/delete (v0.25.4)
**Context-Aware Tool System**
- [ ] `ToolContext` struct: channel type, workspace ID, workflow ID, team ID, persona ID, is_visitor
- [ ] `Require` predicate type on Tool interface: `Availability() Require`
- [ ] `BaseTool` embed for backward compat (defaults to `AlwaysAvailable`)
- [ ] Built-in predicates: `RequireWorkspace`, `RequireWorkflow`, `RequireTeam`, `DenyVisitor`, `All()`
- [ ] `tools.AvailableFor(tctx, disabled)` replaces `AllDefinitionsFiltered()`
- [ ] Workspace tools: `RequireWorkspace + DenyVisitor` (self-declared, eliminates manual suppression lists)
- [ ] Git tools: `RequireWorkspace + DenyVisitor` (same — kills `WorkspaceToolNames()`/`GitToolNames()`)
- [ ] `workspace_create` tool: available only when no workspace bound, disappears once one exists
- [ ] Memory/notes tools: `DenyVisitor`
**CSS Decomposition**
- [x] Monolithic `styles.css` → 13 files: variables, layout, primitives, modals, chat, panels, surfaces, splash, pane-container, chat-pane, user-menu, tool-grants, admin-surfaces
**Persona Tool Grants (Wire Existing Infrastructure)**
- [ ] Completion handler applies `GetToolGrants()` as second-pass allowlist on context-available tools
- [ ] Empty grants = inherit all (backward compat). Explicit grants = allowlist only those tools.
- [ ] Persona create/edit UI: "Tools" section — multi-select checklist grouped by category
- [ ] Version snapshot includes persona tool grants at snapshot time (frozen for running instances)
- [ ] No migration needed — `persona_grants` table, `grant_type='tool'`, `GetToolGrants()` already exist
**Persona Tool Grants UI** ✅ (partial — UI only, backend wire deferred)
- [x] Persona create/edit UI: "Tools" section — multi-select checklist grouped by category
- [x] `loadToolList()` fetches from `/api/v1/tools`, `loadToolGrants()` reads per-persona grants
- [ ] Completion handler applies `GetToolGrants()` as second-pass allowlist — deferred to v0.27.0
- [ ] Version snapshot includes persona tool grants at snapshot time — deferred to v0.27.0
**`ExecutionContext` Extension**
- [ ] Add `WorkflowID`, `TeamID` fields to `ExecutionContext` (tools/types.go)
- [ ] Populated from channel record in completion handler before tool execution
**Bug Fixes (v0.25.3v0.25.4)**
- [x] Theme system mode (system preference → dark fallback)
- [x] Resize bar jump-on-click (inline width pin on start)
- [x] Debug modal styling (modal-content → modal class)
- [x] Settings/admin back-button navigation (sessionStorage return URL)
- [x] Profile save (wrong element IDs, wrong API routes, empty user context)
- [x] Scale slider range (120→175) and commit-on-release
- [x] Admin surface: 6 nav/wiring fixes (stray `>`, dual-bind, nav flash, history, storage, back URL)
- [x] BYOK/Personas toggles (policies store mismatch)
- [x] Group permissions 500 (Postgres SetNull argIdx misalignment)
- [x] Settings surface init (missing policies/models fetch, handler wiring)
- [x] User settings NULL/null/array corruption (COALESCE + jsonb_typeof guard)
- [x] Stale modal HTML cleanup (320 lines removed from chat surface)
- [x] Team admin modal layout/wiring, persona create button
- [x] Project files admin bypass
- [x] Multi-provider live test failover
**Shared Surface Routes** — partially shipped
- [x] `/admin/:section`, `/settings/:section` — full-page surface routes
- [ ] `/s/editor/:wsId`, `/s/article/:wsId/:path` — deferred
- [ ] `/p/:id` — shared project view — deferred
**Shipped in v0.26.0:**
- [x] Context-Aware Tool System (`ToolContext`, `Require` predicates, `AvailableFor()`)
- [x] `ExecutionContext` extension (`WorkflowID`, `TeamID` fields)
**Deferred to v0.27.0:**
- [ ] Tool grant enforcement in completion handler
- [ ] Extension surface route namespace (`/s/:slug`)
**Migration:**
- [x] 022_surfaces.sql: `surface_registry` table (UUID PK, unique surface_id, is_enabled, is_core)
---
## v0.26.0 — Workflow Engine
## v0.26.0 — Workflow Engine
Team-owned, stage-based process execution. The channel is the runtime,
personas drive each stage, and the existing tool/notes infrastructure
handles structured data collection. Dynamic surfaces (v0.25.0) provide
the pane system for visitor chat, team member views, and the workflow
builder. See [ARCHITECTURE.md — Workflow
Architecture](ARCHITECTURE.md#workflow-architecture-future--v0210) for
the conceptual model.
handles structured data collection. See [DESIGN-0.26.0.md](DESIGN-0.26.0.md)
for full spec and [CHANGELOG-0.26.0.md](../CHANGELOG-0.26.0.md) for detailed
release notes.
Depends on: dynamic surfaces (v0.25.0), multi-participant channels (v0.23.0),
anonymous identity (v0.24.3), permissions (v0.24.2).
**Shipped (v0.26.0v0.26.5):**
- [x] Session cleanup job (background goroutine, `SESSION_EXPIRY_DAYS`)
- [x] `ToolContext` + `Require` predicates + `BaseTool` embed + `AvailableFor()`
- [x] `ExecutionContext` TeamID wiring
- [x] Workflow definitions: `workflows`, `workflow_stages`, `workflow_versions` (migration 023)
- [x] Full CRUD + slug generation + publish + version snapshot
- [x] Workflow instances: `workflow_id`, `current_stage`, `stage_data`, `workflow_status` on channels (migration 024)
- [x] Start/advance/reject/status endpoints
- [x] Staleness sweep (background goroutine, `WORKFLOW_STALE_HOURS`)
- [x] Visitor landing page (`/w/:id/:slug`, branded, session resume)
- [x] Visitor start flow (`POST /api/v1/workflow-entry/:scope/:slug`)
- [x] `workflow_advance` tool (`RequireWorkflow` predicate, AI-triggered)
- [x] Form template → system prompt injection in completion handler
- [x] Stage notes (persisted as channel-scoped notes)
- [x] `workflow_assignments` table + claim/complete/list endpoints (migration 025)
- [x] Admin builder UI (Workflows category tab, CRUD, stage editor)
- [x] Queue sidebar section (badge count, claim/complete)
- [x] Route registration test (`TestRouteRegistration`)
- [x] Workflow CRUD test (`TestWorkflowCRUD`, `TestWorkflowValidation`)
See [DESIGN-0.26.0.md](DESIGN-0.26.0.md) for full spec.
**Workflow Definitions**
- [ ] `workflows` table: `team_id`, `name`, `slug` (globally unique), `description`, `branding` (JSONB), `entry_mode`, `is_active`, `version`
- [ ] `workflow_stages` table: `workflow_id`, `ordinal`, `persona_id`, `assignment_team_id`, `form_template` (JSONB), `auto_transition`, `transition_rules` (JSONB)
- [ ] `workflow_versions` table: immutable snapshots (full JSON: definition + stages + persona tool grants)
- [ ] Team-admin CRUD: create/edit/delete workflows and stages, reorder stages
- [ ] Workflow versioning: edits increment version, active instances continue on creation-time snapshot
**Workflow Instances (Channels)**
- [ ] Workflow-typed channels: `type='workflow'`, `workflow_id`, `workflow_version`, `current_stage`, `stage_data` (JSONB)
- [ ] `workflow_status` enum: `active`, `completed`, `stale`, `cancelled`
- [ ] `last_activity_at` tracking for staleness detection
- [ ] Entry point: public link generates workflow channel, adds anonymous participant, binds stage 1 persona, auto-creates workspace
- [ ] Stage transitions: AI-triggered (`workflow_advance` tool), human-triggered (button), rejection (return to previous stage with reason)
- [ ] On transition: swap active persona, notify assignment team, update channel metadata, create channel-scoped note with form data
- [ ] Staleness sweep: background job marks idle instances as `stale` (configurable threshold per-workflow)
- [ ] Stale instance UX: visitor sees "Continue or Start Over" (start over creates new instance on latest version)
**Visitor Experience (Surfaces)**
- [ ] Workflow landing page surface: `/w/:slug` — branded page with persona avatar, description, "Start" button
- [ ] Visitor chat surface: `/w/:slug/c/:channelId` — purpose-built bubble chat (NOT full ChatPane)
- [ ] `workflow-chat.js`: lightweight JS — WebSocket, markdown rendering, typing indicator, stage transition animation
- [ ] Branded CSS: accent color from `workflow.branding`, system dark/light mode
- [ ] Visitor tool scoping: `DenyVisitor` predicates + persona tool grants restrict tool set
- [ ] Dynamic form pages deferred — persona IS the form (collects data conversationally)
**AI Intake**
- [ ] `workflow_advance` tool: `RequireWorkflow` availability, collects form data, triggers stage transition
- [ ] Form template → persona system prompt injection ("collect these fields: ...")
- [ ] Tool calls create channel-scoped notes as form responses
- [ ] Stage rejection: persona sees rejection reason in conversation history, re-collects
**Assignment + Queue**
- [ ] `workflow_assignments` table: `channel_id`, `stage`, `team_id`, `assigned_to`, `status`
- [ ] Assignment queue: unassigned workflow channels visible to team members
- [ ] Claim model: optimistic lock (`UPDATE ... WHERE status = 'unassigned'`)
- [ ] Round-robin auto-assignment (optional, per-stage `transition_rules`)
- [ ] Assignment notifications via existing notification infrastructure + WebSocket
**Team Member Collaboration**
- [ ] Assigned member sees full history (AI intake + artifacts + notes)
- [ ] Persona remains active — assists both visitor and team member
- [ ] Member can trigger stage transitions (advance / reject), add notes, invoke tools
- [ ] Member can reassign to different team member or escalate to different team
**Frontend**
- [ ] Workflow builder UI in team admin panel (stage list, persona picker, form template editor)
- [ ] Queue sidebar section showing unassigned workflow channels for user's teams
- [ ] Channel header: workflow stage indicator, advance/reject controls
- [ ] Persona tool grants display in workflow builder (read-only — shows what tools each stage persona has)
**Migration**
- [ ] 022_workflows.sql: `workflows`, `workflow_stages`, `workflow_versions`, `workflow_assignments`, channel columns
- [ ] Session cleanup job for completed/cancelled workflow channels
**Deferred to v0.27.0:**
- [ ] Team-scoped workflow management UI (team admin settings surface)
- [ ] Drag-and-drop stage reorder in builder (backend supports it, UI is manual)
- [ ] `on_complete` workflow chaining (column exists nullable, not wired)
- [ ] Workflow retention enforcement (column exists, not enforced)
- [ ] Stage persona auto-switch in chat UI
- [ ] Round-robin auto-assignment
- [ ] Assignment notifications via WebSocket
- [ ] Extension surface routes (`/s/:slug` namespace)
- [ ] Persona tool grant enforcement in completion handler
- [ ] Channel header stage indicator + advance/reject controls
---
@@ -817,6 +896,8 @@ based on need.
**UX / Multi-Seat**
- "Group" scope badge on model selector / KB list: show access-source annotation so users see _why_ they have access (global, team, group grant). Requires backend to include `access_source` in list responses.
- Per-provider model preferences: `user_model_settings` unique key is `(user_id, model_id)` — same model from different providers shares one visibility toggle. Needs `provider_config_id` dimension in DB constraint, store, API, and frontend `hiddenModels` keying. Frontend composite key (`configId:modelId`) already exists in `App.models[].id`.
- Git credentials store: vault-encrypted per-user git credentials (similar to BYOK pattern). Git provider abstraction (GitHub, GitLab, Gitea). Clone/pull/push handlers. Natural fit for extension sidecar tier since git operations are long-running.
- Admin settings team/user export: v0.25.4 ships admin-only settings export/import. User export blocked by vault-encrypted BYOK keys (can't round-trip). Team export needs merge-vs-replace semantics for member lists.
**Projects — Future**
- Project-specific files: full project-level upload (own endpoint, storage path, UI surface — not just channel attachment re-linking)
@@ -851,9 +932,9 @@ based on need.
- ~~SQLite backend option (single-user / dev)~~ → v0.17.1
- **Helm chart.** The k8s/ raw manifests with `${VAR}` substitution have served well but are friction for external adopters and make values management manual. A Helm chart wraps the same backend + frontend deployments with a `values.yaml` (replicas, image tags, ingress host, storage class, secret refs, resource limits, feature flags). Target: `helm install switchboard ./chart` for a fresh cluster, `helm upgrade` for rolling deploys. Subcharts for optional Postgres (for dev/test — prod uses external). Candidate for v0.28+ or a parallel track once the core feature set stabilizes.
~~**Pane Architecture (Workspace Container)**~~moved to v0.25.0
~~**Pane Architecture (Workspace Container)**~~shipped in v0.25.0
~~**Surfaces as Extensions**~~moved to v0.25.0 (core infrastructure). Marketplace, Surface IDE, and project-bound defaults remain future items:
~~**Surfaces as Extensions**~~ → core infrastructure shipped in v0.25.0. Marketplace, Surface IDE, and project-bound defaults remain future items:
- Surface IDE: built-in surface for building surfaces — Go template editor for server-rendered shells, JS/CSS editor for client behavior, live preview in sandboxed region
- Surface marketplace: share custom surfaces across instances (presentation mode, kanban, form builder, dashboard, etc.)
- Project-bound surface/pane defaults: project config specifies which panes are available and default layout