From bf6cf835800e7fcc4b59095c4cbd1756d7051b56 Mon Sep 17 00:00:00 2001 From: Jeffrey Smith Date: Fri, 27 Mar 2026 12:25:53 +0000 Subject: [PATCH] Docs: workflow redesign design doc for v0.2.6 Maps viable chat-switchboard v0.39.x concepts onto core's extension-first architecture. Classifies every component as trash/mod/add/keep. Key decisions: per-stage audience (team/public/system) replacing binary entry_mode, branch_rules for conditional routing and stage skipping, public surfaces as a kernel capability. Co-Authored-By: Claude Opus 4.6 --- docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md | 806 +++++++++++++++++++++++++ 1 file changed, 806 insertions(+) create mode 100644 docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md diff --git a/docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md b/docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md new file mode 100644 index 0000000..36d3886 --- /dev/null +++ b/docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md @@ -0,0 +1,806 @@ +# Workflow Redesign β€” v0.2.6 + +**Status:** Draft +**Branch:** `feat/workflow-redesign-v0.2.6` +**Date:** 2026-03-27 +**Context:** Mapping viable ideas from chat-switchboard v0.39.x onto core's extension-first architecture. + +--- + +## 1. Guiding Principles + +1. **Workflows are kernel.** Definitions, instances, the stage graph, form validation, and + assignment queues are platform primitives β€” not extensions. +2. **Execution is delegated.** How a stage renders or collects data is an extension concern. + The kernel says *what* a stage needs; a surface package provides the *how*. +3. **No chat assumptions.** The kernel never references personas, channels, or AI providers. + A chat extension can participate in workflows, but the workflow engine does not depend on it. +4. **Public access is a platform capability.** Anonymous/public sessions are not workflow-specific β€” + they are a kernel-level feature that any surface can opt into. Workflows *consume* public + access; they don't own it. See Β§15. +5. **Edit existing migrations.** Per the design-decisions log: no migration chains until the + schema is in production. New tables get new files; modified tables are edited in place. + +--- + +## 2. Disposition Index + +Every item below is classified: + +| Tag | Meaning | +|-----|---------| +| **πŸ—‘ TRASH** | Remove from core. Vestigial chat-switchboard concept that doesn't fit. | +| **✏️ MOD** | Exists in core today β€” modify in place. | +| **βž• ADD** | New to core. Requires new code/tables. | +| **βœ… KEEP** | Already correct in core. No changes needed. | + +--- + +## 3. Schema Changes + +### 3a. `007_workflows.sql` β€” edit in place + +#### `workflows` table + +| Column | Disposition | Action | +|--------|-------------|--------| +| `entry_mode` | ✏️ MOD | Current values: `public_link`, `team_only`. Keep the column but change its meaning β€” it becomes a workflow-level default/flag that says "this workflow has at least one public stage." The real visibility control moves to per-stage `audience`. See note below. | +| All other columns | βœ… KEEP | Clean. No chat coupling. | + +**Note on `entry_mode`:** Chat-switchboard treated this as a binary toggle β€” the whole +workflow was either public or not. Real workflows mix audiences: internal setup β†’ public +form β†’ internal review β†’ public confirmation. The per-stage `audience` field (see below) +is the source of truth. `entry_mode` on the workflow becomes a convenience flag: if any +stage has `audience = 'public'`, the workflow is public-entry eligible. This avoids a +full schema break β€” existing code that checks `entry_mode` still works, it just gets set +automatically when stages are saved. + +#### `workflow_stages` table + +| Column | Disposition | Action | +|--------|-------------|--------| +| `persona_id` | πŸ—‘ TRASH | Drop column. Personas are a chat-extension concept. If a stage needs an AI participant, the stage's `surface_pkg_id` extension handles that internally. The comment in the migration already acknowledges this ("personas are extensions now") but the column is still there. Remove it. | +| `stage_mode` CHECK | ✏️ MOD | Current: `form_only`, `form_chat`, `review`, `custom`. Replace with: `form`, `review`, `delegated`, `automated`. See Β§4 for definitions. `form_chat` is trash (chat coupling). `form_only` renames to `form`. `custom` renames to `delegated` (clearer intent). `automated` is new (no UI, Starlark-only). | +| `history_mode` | πŸ—‘ TRASH | Drop column. This controlled how much chat history a persona saw across stages. Core has no chat history. If a delegated surface needs context management, it reads `stage_data` β€” that's the contract. | +| `audience` | βž• ADD | `TEXT NOT NULL DEFAULT 'team'` with CHECK `('team', 'public', 'system')`. Controls who interacts with this stage. `team` = authenticated team members only (internal). `public` = anonymous visitors via entry token (the "public-ish" stage). `system` = no human interaction, automated only. A workflow can freely alternate: teamβ†’publicβ†’teamβ†’public. The kernel enforces this: public stages are accessible via token auth, team stages require JWT, system stages have no UI. | +| `stage_type` | βž• ADD | `TEXT NOT NULL DEFAULT 'simple'` with CHECK `('simple', 'dynamic', 'automated')`. Drives the graph engine: `simple` = declarative rules only, `dynamic` = Starlark hook resolves next stage, `automated` = no human UI, fires hook on entry and auto-advances. | +| `starlark_hook` | βž• ADD | `TEXT` (nullable). Package-qualified entry point for dynamic/automated stages. Format: `package_id:entry_point`. NULL for simple stages. | +| `branch_rules` | βž• ADD | `JSONB NOT NULL DEFAULT '[]'`. Replaces the overloaded `transition_rules` for simple stage branching. Array of `{field, op, value, target_stage}`. Evaluated before `starlark_hook`, before ordinal fallback. | +| `transition_rules` | ✏️ MOD | Rename to `stage_config`. This JSONB blob was doing double duty (routing rules + on_advance hooks + auto_assign). Routing moves to `branch_rules`. What remains is stage-specific config that the kernel or surface reads: `{on_advance: {package_id, entry_point}, auto_assign: "round_robin"}`. | +| `assignment_team_id` | βœ… KEEP | Clean. Kernel concept. | +| `form_template` | βœ… KEEP | Kernel concern β€” structured data schema. | +| `auto_transition` | βœ… KEEP | Kernel concern β€” skip human interaction. | +| `surface_pkg_id` | βœ… KEEP | The delegation pointer. Required for `delegated` mode stages. | +| `sla_seconds` | βœ… KEEP | Kernel concern β€” time budget per stage. | + +#### `workflow_versions` table + +| Column | Disposition | Notes | +|--------|-------------|-------| +| All existing columns | βœ… KEEP | Immutable snapshots. No changes needed. | + +### 3b. `007_workflows.sql` β€” add new tables (same file) + +#### βž• ADD `workflow_instances` + +Replaces the channel-based instance model from chat-switchboard. This is the execution record. + +```sql +CREATE TABLE IF NOT EXISTS workflow_instances ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + workflow_id UUID NOT NULL REFERENCES workflows(id) ON DELETE CASCADE, + workflow_version INTEGER NOT NULL, + current_stage INTEGER NOT NULL DEFAULT 0, + stage_data JSONB NOT NULL DEFAULT '{}', + status TEXT NOT NULL DEFAULT 'active' + CHECK (status IN ('active', 'completed', 'cancelled', 'stale', 'error')), + started_by UUID REFERENCES users(id) ON DELETE SET NULL, + entry_token TEXT, -- for public_link resumption + metadata JSONB NOT NULL DEFAULT '{}', + stage_entered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +CREATE INDEX IF NOT EXISTS idx_wf_instances_workflow + ON workflow_instances(workflow_id, status); +CREATE INDEX IF NOT EXISTS idx_wf_instances_status + ON workflow_instances(status) WHERE status = 'active'; +CREATE UNIQUE INDEX IF NOT EXISTS idx_wf_instances_token + ON workflow_instances(entry_token) WHERE entry_token IS NOT NULL; + +DROP TRIGGER IF EXISTS wf_instances_updated_at ON workflow_instances; +CREATE TRIGGER wf_instances_updated_at BEFORE UPDATE ON workflow_instances + FOR EACH ROW EXECUTE FUNCTION update_updated_at(); +``` + +**Rationale:** Chat-switchboard used `channels` as instances β€” that table carried message trees, +AI context windows, participant lists, and other chat concerns. Core needs a purpose-built table +that holds only workflow execution state: which version, which stage, accumulated data, lifecycle status. + +#### βž• ADD `workflow_assignments` + +Dropped during the fork ("channel-dependent, rebuild as needed" β€” see 007 header comment). +Now rebuilt as a kernel primitive. + +```sql +CREATE TABLE IF NOT EXISTS workflow_assignments ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + instance_id UUID NOT NULL REFERENCES workflow_instances(id) ON DELETE CASCADE, + stage INTEGER NOT NULL, + team_id UUID NOT NULL REFERENCES teams(id) ON DELETE CASCADE, + assigned_to UUID REFERENCES users(id) ON DELETE SET NULL, + status TEXT NOT NULL DEFAULT 'unassigned' + CHECK (status IN ('unassigned', 'claimed', 'completed', 'cancelled')), + review_data JSONB NOT NULL DEFAULT '{}', + claimed_at TIMESTAMPTZ, + completed_at TIMESTAMPTZ, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +CREATE INDEX IF NOT EXISTS idx_wf_assignments_instance + ON workflow_assignments(instance_id, stage); +CREATE INDEX IF NOT EXISTS idx_wf_assignments_team_status + ON workflow_assignments(team_id, status) WHERE status IN ('unassigned', 'claimed'); +CREATE INDEX IF NOT EXISTS idx_wf_assignments_user + ON workflow_assignments(assigned_to) WHERE assigned_to IS NOT NULL; +``` + +**Claim lock:** Same optimistic pattern from chat-switchboard: +`UPDATE ... WHERE id = $1 AND status = 'unassigned'` β€” if `rows_affected == 0`, already claimed. + +### 3c. New migration file: `012_workflow_instances.sql` + +Wait β€” per the principle, new *tables* get new files. But we're also editing 007. Decision: + +**Put the new tables in 007_workflows.sql.** The schema isn't in production. Keep all workflow +DDL in one file. When the schema *is* in production (post-MVP), new additions get numbered +migrations. This is consistent with the existing design-decisions log entry: +"No new migrations pre-MVP." + +--- + +## 4. Stage Mode Redesign + +### Current (trash/rename) + +| Current Mode | Disposition | Rationale | +|-------------|-------------|-----------| +| `form_only` | ✏️ MOD β†’ `form` | Rename. Drop the `_only` suffix β€” there's no `form_chat` to distinguish from anymore. | +| `form_chat` | πŸ—‘ TRASH | Chat coupling. If you want a form + AI conversation, use `delegated` with a chat surface package. | +| `review` | βœ… KEEP | Team member reviews accumulated data. Pure kernel concept. | +| `custom` | ✏️ MOD β†’ `delegated` | Rename for clarity. "Custom" is vague. "Delegated" makes the contract explicit: the kernel delegates stage execution to `surface_pkg_id`. | + +### New modes + +| Mode | Disposition | Description | +|------|-------------|-------------| +| `form` | ✏️ MOD (renamed) | Kernel renders `form_template`, validates submission, merges into `stage_data`. No extension needed. | +| `review` | βœ… KEEP | Assignment queue stage. Team member claims, reviews `stage_data`, adds `review_data`, completes. Kernel-rendered. | +| `delegated` | ✏️ MOD (renamed) | Kernel hands off to `surface_pkg_id`. The surface package receives the instance context (stage_data, form_template, metadata) and calls back to advance. This is where chat, AI, or any custom UX lives. | +| `automated` | βž• ADD | No UI. On stage entry, kernel fires `starlark_hook`, merges returned data into `stage_data`, and auto-advances. For enrichment, API calls, validation, routing decisions. | + +### Stage type vs stage mode + +These are orthogonal: + +- **`stage_type`** controls *how the next stage is chosen*: `simple` (declarative branch_rules), `dynamic` (Starlark hook returns target), `automated` (hook + auto-advance). +- **`stage_mode`** controls *how the stage executes*: `form`, `review`, `delegated`, `automated`. + +An `automated` stage_type with `automated` stage_mode is the common case for system stages, +but you can have a `dynamic` stage_type with `form` stage_mode (the form collects data, then a +Starlark hook decides where to go next based on the submission). + +--- + +## 5. Model Changes + +### `server/models/workflow.go` + +#### WorkflowStage struct β€” ✏️ MOD + +```go +// TRASH: Remove these fields +// PersonaID *string `json:"persona_id,omitempty"` +// HistoryMode string `json:"history_mode"` + +// MOD: Rename transition_rules β†’ stage_config +// TransitionRules json.RawMessage `json:"transition_rules"` +StageConfig json.RawMessage `json:"stage_config"` + +// ADD: New fields +Audience string `json:"audience"` // team | public | system +StageType string `json:"stage_type"` // simple | dynamic | automated +StarlarkHook *string `json:"starlark_hook,omitempty"` // package_id:entry_point +BranchRules json.RawMessage `json:"branch_rules"` // [{field, op, value, target_stage}] +``` + +#### Stage mode constants β€” ✏️ MOD + +```go +// TRASH +// StageModeFormOnly = "form_only" +// StageModeFormChat = "form_chat" + +// MOD (rename) +StageModeForm = "form" // was form_only +StageModeDelegated = "delegated" // was custom + +// KEEP +StageModeReview = "review" + +// ADD +StageModeAutomated = "automated" + +// ADD: Stage type constants +StageTypeSimple = "simple" +StageTypeDynamic = "dynamic" +StageTypeAutomated = "automated" + +// ADD: Audience constants +AudienceTeam = "team" +AudiencePublic = "public" +AudienceSystem = "system" +``` + +#### βž• ADD: WorkflowInstance model + +```go +type WorkflowInstance struct { + ID string `json:"id"` + WorkflowID string `json:"workflow_id"` + WorkflowVersion int `json:"workflow_version"` + CurrentStage int `json:"current_stage"` + StageData json.RawMessage `json:"stage_data"` + Status string `json:"status"` // active | completed | cancelled | stale | error + StartedBy *string `json:"started_by,omitempty"` + EntryToken *string `json:"entry_token,omitempty"` + Metadata json.RawMessage `json:"metadata"` + StageEnteredAt time.Time `json:"stage_entered_at"` + CreatedAt time.Time `json:"created_at"` + UpdatedAt time.Time `json:"updated_at"` +} +``` + +#### βž• ADD: WorkflowAssignment model + +```go +type WorkflowAssignment struct { + ID string `json:"id"` + InstanceID string `json:"instance_id"` + Stage int `json:"stage"` + TeamID string `json:"team_id"` + AssignedTo *string `json:"assigned_to,omitempty"` + Status string `json:"status"` // unassigned | claimed | completed | cancelled + ReviewData json.RawMessage `json:"review_data"` + ClaimedAt *time.Time `json:"claimed_at,omitempty"` + CompletedAt *time.Time `json:"completed_at,omitempty"` + CreatedAt time.Time `json:"created_at"` +} +``` + +#### TypedFormTemplate, FormField, FormValidation β€” βœ… KEEP + +All form types are clean kernel code. No changes. + +#### ValidateFormData β€” βœ… KEEP + +Pure data validation. No chat coupling. + +--- + +## 6. Store Interface Changes + +### `server/store/workflow_iface.go` β€” ✏️ MOD + βž• ADD + +Current interface is definition-only (CRUD workflows + stages + versions). +Add instance and assignment operations. + +```go +type WorkflowStore interface { + // ── Definition CRUD (KEEP β€” no changes) ────────── + Create(ctx context.Context, w *models.Workflow) error + GetByID(ctx context.Context, id string) (*models.Workflow, error) + GetBySlug(ctx context.Context, teamID *string, slug string) (*models.Workflow, error) + Update(ctx context.Context, id string, patch models.WorkflowPatch) error + Delete(ctx context.Context, id string) error + ListForTeam(ctx context.Context, teamID string) ([]models.Workflow, error) + ListGlobal(ctx context.Context) ([]models.Workflow, error) + + // ── Stage CRUD (KEEP β€” no changes) ─────────────── + CreateStage(ctx context.Context, s *models.WorkflowStage) error + ListStages(ctx context.Context, workflowID string) ([]models.WorkflowStage, error) + UpdateStage(ctx context.Context, s *models.WorkflowStage) error + DeleteStage(ctx context.Context, id string) error + ReorderStages(ctx context.Context, workflowID string, orderedIDs []string) error + + // ── Versioning (KEEP β€” no changes) ─────────────── + Publish(ctx context.Context, v *models.WorkflowVersion) error + GetVersion(ctx context.Context, workflowID string, versionNumber int) (*models.WorkflowVersion, error) + GetLatestVersion(ctx context.Context, workflowID string) (*models.WorkflowVersion, error) + + // ── Instances (ADD) ────────────────────────────── + CreateInstance(ctx context.Context, inst *models.WorkflowInstance) error + GetInstance(ctx context.Context, id string) (*models.WorkflowInstance, error) + GetInstanceByToken(ctx context.Context, token string) (*models.WorkflowInstance, error) + UpdateInstance(ctx context.Context, id string, patch models.InstancePatch) error + ListInstances(ctx context.Context, workflowID string, status string) ([]models.WorkflowInstance, error) + AdvanceStage(ctx context.Context, id string, nextStage int, mergeData json.RawMessage) error + CompleteInstance(ctx context.Context, id string) error + CancelInstance(ctx context.Context, id string) error + MarkStale(ctx context.Context, olderThan time.Duration) (int, error) + + // ── Assignments (ADD) ──────────────────────────── + CreateAssignment(ctx context.Context, a *models.WorkflowAssignment) error + ClaimAssignment(ctx context.Context, id string, userID string) error // optimistic lock + UnclaimAssignment(ctx context.Context, id string) error + CompleteAssignment(ctx context.Context, id string, reviewData json.RawMessage) error + CancelAssignment(ctx context.Context, id string) error + ListAssignmentsByTeam(ctx context.Context, teamID string, status string) ([]models.WorkflowAssignment, error) + ListAssignmentsByUser(ctx context.Context, userID string) ([]models.WorkflowAssignment, error) + ListAssignmentsByInstance(ctx context.Context, instanceID string) ([]models.WorkflowAssignment, error) +} +``` + +--- + +## 7. Handler Changes + +### `server/handlers/workflows.go` β€” ✏️ MOD + +**Stage CRUD updates** β€” reflect new field names in CreateStage/UpdateStage: +- Remove `persona_id` and `history_mode` from bind/validation. +- Add `stage_type`, `starlark_hook`, `branch_rules` to bind/validation. +- Rename `transition_rules` β†’ `stage_config` in bind/validation. +- Update `stage_mode` CHECK to new set: `form`, `review`, `delegated`, `automated`. +- Validate: `delegated` requires `surface_pkg_id`. `dynamic`/`automated` stage_type requires `starlark_hook`. + +### `server/handlers/workflow_hooks.go` β€” ✏️ MOD + +**Rename:** `channelID` parameter β†’ `instanceID` in `FireOnAdvanceHook` signature. +The hook context dict changes from `{channel_id, ...}` to `{instance_id, ...}`. + +The `jsonToStarlark` helper referenced on line 81 lives in +`server/handlers/starlark_helpers.go`. Consider consolidating with `goToStarlark` in +`server/triggers/event.go` β€” two independent Goβ†’Starlark converters is tech debt. + +### `server/handlers/workflow_team.go` β€” βœ… KEEP + +Team-scoped wrappers. Clean delegation pattern. No changes. + +### `server/handlers/workflow_packages.go` β€” ✏️ MOD + +Update `workflowPkgStage` struct: +- Remove `PersonaID`, `HistoryMode` fields. +- Add `StageType`, `StarlarkHook`, `BranchRules` fields. +- Rename `TransitionRules` β†’ `StageConfig`. + +Update `ExportWorkflowPackage` and `InstallWorkflowFromManifest` accordingly. + +### βž• ADD `server/handlers/workflow_instances.go` + +New handler file for instance lifecycle: + +``` +POST /api/v1/workflows/:id/start β†’ Start (create instance from latest published version) +GET /api/v1/workflows/instances/:iid β†’ GetInstance +POST /api/v1/workflows/instances/:iid/advance β†’ Advance (submit stage data, resolve next) +POST /api/v1/workflows/instances/:iid/cancel β†’ Cancel +GET /api/v1/workflows/:id/instances β†’ ListInstances (by workflow, optionally by status) +``` + +Public entry (for `public_link` workflows): +``` +POST /api/v1/workflows/entry/:slug β†’ StartPublic (no auth, generates entry_token) +GET /api/v1/workflows/entry/:token β†’ ResumePublic (retrieve instance by token) +POST /api/v1/workflows/entry/:token/advance β†’ AdvancePublic (submit data via token) +``` + +### βž• ADD `server/handlers/workflow_assignments.go` + +New handler file for the assignment queue: + +``` +GET /api/v1/teams/:teamId/assignments β†’ ListTeamAssignments (filterable by status) +POST /api/v1/teams/:teamId/assignments/:id/claim β†’ Claim +POST /api/v1/teams/:teamId/assignments/:id/unclaim β†’ Unclaim +POST /api/v1/teams/:teamId/assignments/:id/complete β†’ Complete (with review_data) +POST /api/v1/teams/:teamId/assignments/:id/cancel β†’ Cancel +GET /api/v1/assignments/mine β†’ ListMyAssignments (across teams) +``` + +--- + +## 8. Routing Engine Changes + +### `server/workflow/routing.go` β€” ✏️ MOD + +The existing `ResolveNextStage` evaluates `transition_rules.conditions[]`. This needs to +become a two-phase resolution that respects the new `stage_type`: + +``` +Phase 1: Evaluate branch_rules (declarative, for simple + dynamic types) +Phase 2: If no match AND stage_type == dynamic β†’ fire starlark_hook +Phase 3: Fallback β†’ currentStage + 1 +``` + +For `automated` stage_type, the engine fires `starlark_hook` on entry (not for routing β€” for +data enrichment), then routes via branch_rules or ordinal fallback. The routing call happens +*after* the hook returns. + +**Concrete changes:** +- `ResolveNextStage` signature: add `stageType string, starlarkHook *string, runner *sandbox.Runner` parameters. +- Extract branch_rules evaluation from the current `TransitionRulesWithConditions` (which reads from `transition_rules` JSON) into a dedicated function that reads from the new `branch_rules` field. +- `TransitionRulesWithConditions` struct β†’ rename to `StageConfig`, remove `Conditions` field (moved to branch_rules), keep `AutoAssign` and `OnAdvance`. + +### `server/workflow/` β€” βž• ADD `engine.go` + +New file: the stage execution engine. Orchestrates the advance lifecycle: + +``` +1. Load instance + version snapshot +2. Validate current stage allows advancement (status checks) +3. If current stage has form_template β†’ validate submitted data +4. Merge submitted data into stage_data +5. Fire on_advance hook (if configured in stage_config) +6. Resolve next stage (branch_rules β†’ starlark_hook β†’ ordinal) +7. If next stage is past last stage β†’ complete instance +8. If next stage is automated β†’ fire its hook, recurse to step 6 +9. If next stage has assignment_team_id β†’ create WorkflowAssignment +10. Update instance (current_stage, stage_data, stage_entered_at) +11. Emit bus events (workflow.advanced, workflow.assigned, workflow.completed) +``` + +### `server/workflow/` β€” βž• ADD `automated.go` + +Handles `automated` stage execution: fire the Starlark hook, merge returned data, and +auto-advance. Includes a cycle guard (max 10 consecutive automated stages) to prevent +infinite loops from misconfigured workflows. + +--- + +## 9. Event Bus Updates + +### `server/events/types.go` β€” ✏️ MOD + +Current workflow events are fine but incomplete. Add: + +| Event | Direction | Disposition | Trigger | +|-------|-----------|-------------|---------| +| `workflow.assigned` | DirToClient | βœ… KEEP | Assignment created | +| `workflow.claimed` | DirToClient | βœ… KEEP | Assignment claimed | +| `workflow.advanced` | DirToClient | βœ… KEEP | Stage transition | +| `workflow.completed` | DirToClient | βœ… KEEP | Instance completed | +| `workflow.cancelled` | DirToClient | βž• ADD | Instance cancelled | +| `workflow.started` | DirToClient | βž• ADD | Instance created | +| `workflow.sla.warning` | DirToClient | βž• ADD | SLA at 80% threshold | +| `workflow.sla.breached` | DirToClient | βž• ADD | SLA exceeded | +| `workflow.error` | DirLocal | βž• ADD | Hook execution failure | + +--- + +## 10. Starlark Module Updates + +### `server/sandbox/workflow_module.go` β€” ✏️ MOD + +The `BuildWorkflowModule` (in `workflow_module.go`, not `modules.go`) currently exists but +targets the old channel-based model. It currently only exposes `get_definition`. +Rebuild to expose instance-oriented operations: + +```python +# Available to extensions with workflow.read or workflow.write permission +workflow.get_instance(instance_id) # β†’ instance dict +workflow.get_stage_data(instance_id) # β†’ stage_data dict +workflow.set_stage_data(instance_id, data) # β†’ merge into stage_data +workflow.advance(instance_id) # β†’ trigger advance (write perm) +workflow.get_assignments(instance_id) # β†’ assignment list +``` + +The old `workflow_advance()` function that chat-switchboard's personas called is gone. +Extensions call `workflow.advance()` which delegates to the same engine as the HTTP handler. + +--- + +## 11. Salvage Map from chat-switchboard v0.39.x + +What was planned there, and what happens to each idea in core: + +### v0.39.0 β€” Stage Graph Engine + Form Builder + +| Component | Disposition | Core equivalent | +|-----------|-------------|-----------------| +| `stage_type` enum | βž• ADD | Adopted directly: `simple`, `dynamic`, `automated` | +| `BranchRule` struct | βž• ADD | Adopted as `branch_rules` JSONB on `workflow_stages` | +| Starlark hook integration | βž• ADD | `starlark_hook` column + engine integration | +| Graph engine (`server/engine/graph.go`) | βž• ADD | `server/workflow/engine.go` in core | +| Automated stage runner | βž• ADD | `server/workflow/automated.go` in core | +| Visual form builder (5 Preact components) | πŸ—‘ TRASH *for now* | The form builder was tightly coupled to chat-switchboard's stage editor surface. Core doesn't ship a workflow editor surface (that's an extension). The form *schema* is kernel; the form *builder UI* is a surface package concern. Revisit when a workflow-admin surface is built. | +| Stage type badges in UI | πŸ—‘ TRASH | Same β€” UI is extension territory. | + +### v0.39.1 β€” Stage Reorder + Bulk Ops + +| Component | Disposition | Core equivalent | +|-----------|-------------|-----------------| +| Drag-and-drop reorder | πŸ—‘ TRASH | UI concern. The kernel `ReorderStages` endpoint already exists. | +| Bulk assignment ops | βž• ADD (later) | Useful but not blocking. Defer to v0.2.7+. The individual claim/unclaim/cancel endpoints come first. | + +### v0.39.2 β€” SLA Alerting + +| Component | Disposition | Core equivalent | +|-----------|-------------|-----------------| +| Periodic SLA scanner | βž• ADD | Kernel scheduled goroutine (like the existing staleness sweep). Query active instances with `sla_seconds` configured, compute breach from `stage_entered_at`. | +| `sla_notified_at` on instances | βž• ADD | Add to `workflow_instances` schema. Prevents duplicate notifications. | +| Notifications on breach | βž• ADD | Use existing `notifications` table + bus event (`workflow.sla.breached`). | +| Webhook on breach | βž• ADD | Fire workflow's `webhook_url` if configured. | +| Pulsing badge animation | πŸ—‘ TRASH | UI concern for a surface package. | + +### v0.39.3 β€” Visitor Portal + +| Component | Disposition | Core equivalent | +|-----------|-------------|-----------------| +| Public entry endpoint | βž• ADD | `/api/v1/workflows/entry/:slug` β€” kernel handles anonymous session creation + token issuance. | +| Session resume via token | βž• ADD | `entry_token` on `workflow_instances` + `GetInstanceByToken` store method. | +| Branded entry page | πŸ—‘ TRASH | Surface package. The kernel stores `branding` JSONB β€” a portal surface reads it. | +| Progress indicator | πŸ—‘ TRASH | Surface package reads stage list, computes "step N of M" locally. | +| Email notifications | πŸ—‘ TRASH *for now* | Requires SMTP infrastructure. Defer to post-MVP or make it a connection-based extension (use ext_connections for SMTP config, fire via Starlark hook). | + +### v0.39.4 β€” Templates + Clone + +| Component | Disposition | Core equivalent | +|-----------|-------------|-----------------| +| Clone endpoint | βž• ADD | `POST /api/v1/workflows/:id/clone` β€” deep copy workflow + stages. Straightforward. | +| Built-in templates table | πŸ—‘ TRASH | Core doesn't ship opinionated templates. Templates are workflow packages (`.pkg` archives). A "template gallery" is a surface. | +| Template picker UI | πŸ—‘ TRASH | Surface concern. | + +### v0.39.5 β€” Analytics + +| Component | Disposition | Core equivalent | +|-----------|-------------|-----------------| +| Analytics query endpoint | βž• ADD (later) | `GET /api/v1/workflows/:id/analytics?period=7d` β€” kernel exposes raw metrics (throughput, cycle time, stage dwell, SLA compliance) derived from instance + assignment timestamps. Defer to v0.2.8+ or v0.3.x. | +| Analytics dashboard UI | πŸ—‘ TRASH | Surface package. | + +--- + +## 12. Implementation Order + +All items within v0.2.6. Ordered by dependency: + +### Phase A: Schema + Models + +1. ✏️ Edit `007_workflows.sql`: drop `persona_id`, `history_mode`; add `audience`, `stage_type`, `starlark_hook`, `branch_rules`; rename `transition_rules` β†’ `stage_config`; update `stage_mode` CHECK. +2. ✏️ Edit `007_workflows.sql` (same file): add `workflow_instances` and `workflow_assignments` tables. +3. ✏️ Edit `007_workflows.sql` (SQLite dialect): mirror changes. +4. ✏️ Update `server/models/workflow.go`: remove trashed fields, add new structs, update constants. + +### Phase B: Store Layer + +5. ✏️ Update `server/store/workflow_iface.go`: add instance + assignment methods. +6. βž• Implement Postgres store: `server/store/postgres/workflow_instances.go`, `workflow_assignments.go`. +7. βž• Implement SQLite store: `server/store/sqlite/workflow_instances.go`, `workflow_assignments.go`. + +### Phase C: Engine + +8. ✏️ Update `server/workflow/routing.go`: branch_rules evaluation, starlark_hook integration. +9. βž• Add `server/workflow/engine.go`: advance lifecycle orchestration. +10. βž• Add `server/workflow/automated.go`: automated stage execution + cycle guard. +11. ✏️ Update `server/handlers/workflow_hooks.go`: rename channel β†’ instance references. + +### Phase D: Handlers + Routes + +12. ✏️ Update `server/handlers/workflows.go`: stage CRUD field changes. +13. ✏️ Update `server/handlers/workflow_packages.go`: package struct field changes. +14. βž• Add `server/handlers/workflow_instances.go`: instance lifecycle endpoints. +15. βž• Add `server/handlers/workflow_assignments.go`: assignment queue endpoints. +16. ✏️ Wire new routes in `server/main.go`. + +### Phase E: Events + Starlark + +17. ✏️ Update `server/events/types.go`: add new workflow events. +18. ✏️ Update `server/sandbox/modules.go`: rebuild workflow Starlark module. + +### Phase F: Background Jobs + +19. βž• Add SLA scanner goroutine (check active instances, fire notifications + events on breach). +20. βž• Add staleness sweep for workflow instances (reuse pattern from chat-switchboard). + +### Phase G: Verification + +21. βž• Integration tests: instance lifecycle, stage advancement, branching, automated stages, assignments, SLA breach. +22. `go build ./...` clean. +23. Update ICD (OpenAPI spec) with new endpoints. +24. Update ROADMAP.md + CHANGELOG.md. + +--- + +## 13. What We're NOT Doing (Explicit Deferrals) + +| Item | Why | Revisit | +|------|-----|---------| +| Visual form builder UI | Surface concern, no workflow-admin surface yet | When a workflow-admin surface ships | +| Drag-and-drop stage reorder UI | Surface concern | Same | +| Email notifications | Needs SMTP infrastructure or connection-based ext | Post-MVP | +| Built-in workflow templates | Core doesn't ship opinionated content | Package gallery | +| Analytics dashboard | Nice-to-have, kernel endpoint can come first | v0.3.x | +| Bulk assignment operations | Individual ops first | v0.2.7+ | +| Workflow chaining (`on_complete`) | Already in schema, needs engine wiring | v0.2.7 | +| Visitor portal surface | Surface package, not kernel | Extension track | + +--- + +## 14. Migration Policy Reminder + +From the design-decisions log: + +> **No new migrations pre-MVP.** Edit existing migration SQL files in place. +> No migration chains until schema is in production. + +For this redesign: +- **`007_workflows.sql`**: Edit in place for column changes AND add new tables to the same file. +- **No `012_*.sql`**: Everything goes in 007. +- **Both dialects**: Postgres and SQLite must be kept in sync. + +The only time a new numbered migration file is created is when a truly new, +unrelated subsystem is added (like 011_triggers.sql was for the trigger system). +Workflow instances and assignments are part of the workflow subsystem β†’ they go in 007. + +--- + +## 15. Public Surfaces β€” Kernel Capability, Not Workflow Feature + +### The problem with workflow-owned public access + +Chat-switchboard treated public access as a binary workflow toggle: `entry_mode: public_link` +made the whole workflow public. But real workflows alternate audiences β€” internal setup β†’ +public-facing customer form β†’ internal review β†’ public confirmation page. And public access +isn't even a workflow-only need: feedback forms, status pages, and surveys all need anonymous +sessions without being workflows. + +### Per-stage audience (the v0.2.6 solution) + +The `audience` field on `workflow_stages` is the source of truth: + +| Audience | Who interacts | Auth required | Example | +|----------|--------------|---------------|---------| +| `team` | Authenticated team members | JWT | Internal triage, setup, review | +| `public` | Anonymous visitors | Entry token | Customer intake form, confirmation page | +| `system` | Nobody (automated) | N/A | API enrichment, routing decision | + +A single workflow freely alternates: + +``` +Stage 0: "Setup" audience=team (agent configures parameters) +Stage 1: "Customer Form" audience=public (customer fills out intake) +Stage 2: "Internal Review" audience=team (team reviews submission) +Stage 3: "Confirmation" audience=public (customer sees result) +``` + +The engine enforces boundaries: +- When the instance enters a `public` stage, the visitor's token grants access. Team members + can also see it (they have elevated access). +- When the instance enters a `team` stage, the visitor's view freezes. They see "being + processed" or whatever the portal surface shows. The token is still valid for resumption + when the next `public` stage arrives. +- When the instance enters a `system` stage, nobody interacts β€” the automated hook fires + and the engine advances. + +### The `entry_mode` column + +Stays on the `workflows` table but becomes a derived convenience flag. If any stage has +`audience = 'public'`, the workflow is public-entry eligible. The kernel can auto-compute +this when stages are saved, or treat it as an explicit admin toggle that gates whether +public stages are actually reachable. Either way, it's no longer the primary access control +mechanism β€” `audience` on each stage is. + +### Platform-level public access (v0.2.7+ concern) + +The broader vision: public access as a **package-level manifest capability** so any surface +(not just workflows) can serve anonymous visitors. A package declares `"public": true` in +its manifest, and the kernel mounts a public route namespace (`/p/:package_slug/*`) with +anonymous session middleware, token issuance, and rate limiting. Workflows consume this +capability β€” they don't own it. + +For v0.2.6, the workflow-specific public entry routes are sufficient: +``` +POST /api/v1/workflows/entry/:slug β†’ StartPublic +GET /api/v1/workflows/entry/:token β†’ ResumePublic +POST /api/v1/workflows/entry/:token/advance β†’ AdvancePublic +``` + +The anonymous session management (entry_token on `workflow_instances`, IP rate limiting) +lives in the workflow handler for now. Generalization to a platform-level `public_sessions` +table happens when non-workflow surfaces need it. + +### Visitor view contract + +When a public stage is active, the kernel API returns only what the visitor should see: +- The current stage's `form_template` (if mode is `form`) +- The stage name and ordinal (for progress indicators) +- Accumulated `stage_data` keys marked as `visitor_visible` (future: field-level ACL) +- The `branding` JSONB from the workflow + +Internal stages, team assignments, review data, and other instances' data are never +exposed through the token-authenticated endpoints. The surface package renders whatever +the kernel returns β€” it doesn't need to filter. + +--- + +## 16. Branch Rules β€” Routing, Skipping, and Convergence + +### How branch_rules work + +Each stage has a `branch_rules` JSONB array. On stage completion, the engine evaluates rules +top-to-bottom against accumulated `stage_data`. First match wins. No match β†’ ordinal + 1. + +```json +[ + {"field": "priority", "op": "eq", "value": "urgent", "target_stage": "Emergency Review"}, + {"field": "amount", "op": "gt", "value": 10000, "target_stage": "Manager Approval"}, + {"field": "type", "op": "in", "value": ["a","b"], "target_stage": "Fast Track"} +] +``` + +`target_stage` accepts either a **stage name** (case-insensitive) or a **numeric ordinal**. +The engine tries name resolution first, falls back to numeric. This is already implemented +in `routing.go` β†’ `resolveTarget()`. + +### Skipping stages + +Branch rules can target any stage, not just the next one. If a workflow has stages +0β†’1β†’2β†’3β†’4 and stage 1's rules say `target_stage: "Stage 4"`, stages 2 and 3 are skipped +for that instance. The engine doesn't care about gaps β€” it sets `current_stage` to whatever +the rule resolved to. + +Linear workflows (no branch_rules on any stage) advance 0β†’1β†’2β†’3β†’4 by the ordinal + 1 +fallback. Adding a single branch rule to any stage introduces conditional skipping without +changing the linear stages. + +### Convergence (diamond patterns) + +Two different branches can target the same downstream stage: + +``` +Stage 0: Intake Form + branch_rules: [ + {field: "region", op: "eq", value: "US", target_stage: "US Processing"}, + {field: "region", op: "eq", value: "EU", target_stage: "EU Processing"} + ] + +Stage 1: US Processing (ordinal 1) + branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}] + (always converge after processing) + +Stage 2: EU Processing (ordinal 2) + branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}] + (always converge after processing) + +Stage 3: Final Review (ordinal 3) + (both branches land here) +``` + +No special syntax needed. The routing engine doesn't track "which branch am I on" β€” it just +resolves the target and moves the instance there. The accumulated `stage_data` carries +everything downstream stages need to know about what happened upstream. + +### Backward jumps + +Branch rules can also target earlier stages (lower ordinals) for retry/correction loops. +The cycle guard (max 10 consecutive automated stages) in `automated.go` prevents infinite +loops for automated stages. For human-facing stages, backward jumps are intentional β€” the +human breaks the cycle by submitting different data. + +### Operators + +Full set, inherited from the existing `routing.go` implementation: + +| Op | Description | Example | +|----|-------------|---------| +| `eq` | Loose equality (string comparison) | `"status" eq "approved"` | +| `neq` | Not equal | `"status" neq "rejected"` | +| `gt` | Greater than (numeric) | `"amount" gt 10000` | +| `lt` | Less than (numeric) | `"amount" lt 100` | +| `gte` | Greater or equal | `"score" gte 80` | +| `lte` | Less or equal | `"score" lte 20` | +| `in` | Value in array | `"category" in ["a","b","c"]` | +| `contains` | Substring match | `"notes" contains "urgent"` | +| `exists` | Field present in stage_data | `"approval_date" exists` | +| `not_exists` | Field absent | `"rejection_reason" not_exists` |