Jeffrey Smith
bf6cf83580
All checks were successful
CI/CD / detect-changes (pull_request) Successful in 4s
CI/CD / test-frontend (pull_request) Has been skipped
CI/CD / test-sqlite (pull_request) Has been skipped
CI/CD / test-go-pg (pull_request) Has been skipped
CI/CD / build-and-deploy (pull_request) Has been skipped
Maps viable chat-switchboard v0.39.x concepts onto core's extension-first architecture. Classifies every component as trash/mod/add/keep. Key decisions: per-stage audience (team/public/system) replacing binary entry_mode, branch_rules for conditional routing and stage skipping, public surfaces as a kernel capability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
807 lines
39 KiB
Markdown
807 lines
39 KiB
Markdown
# Workflow Redesign — v0.2.6
|
||
|
||
**Status:** Draft
|
||
**Branch:** `feat/workflow-redesign-v0.2.6`
|
||
**Date:** 2026-03-27
|
||
**Context:** Mapping viable ideas from chat-switchboard v0.39.x onto core's extension-first architecture.
|
||
|
||
---
|
||
|
||
## 1. Guiding Principles
|
||
|
||
1. **Workflows are kernel.** Definitions, instances, the stage graph, form validation, and
|
||
assignment queues are platform primitives — not extensions.
|
||
2. **Execution is delegated.** How a stage renders or collects data is an extension concern.
|
||
The kernel says *what* a stage needs; a surface package provides the *how*.
|
||
3. **No chat assumptions.** The kernel never references personas, channels, or AI providers.
|
||
A chat extension can participate in workflows, but the workflow engine does not depend on it.
|
||
4. **Public access is a platform capability.** Anonymous/public sessions are not workflow-specific —
|
||
they are a kernel-level feature that any surface can opt into. Workflows *consume* public
|
||
access; they don't own it. See §15.
|
||
5. **Edit existing migrations.** Per the design-decisions log: no migration chains until the
|
||
schema is in production. New tables get new files; modified tables are edited in place.
|
||
|
||
---
|
||
|
||
## 2. Disposition Index
|
||
|
||
Every item below is classified:
|
||
|
||
| Tag | Meaning |
|
||
|-----|---------|
|
||
| **🗑 TRASH** | Remove from core. Vestigial chat-switchboard concept that doesn't fit. |
|
||
| **✏️ MOD** | Exists in core today — modify in place. |
|
||
| **➕ ADD** | New to core. Requires new code/tables. |
|
||
| **✅ KEEP** | Already correct in core. No changes needed. |
|
||
|
||
---
|
||
|
||
## 3. Schema Changes
|
||
|
||
### 3a. `007_workflows.sql` — edit in place
|
||
|
||
#### `workflows` table
|
||
|
||
| Column | Disposition | Action |
|
||
|--------|-------------|--------|
|
||
| `entry_mode` | ✏️ MOD | Current values: `public_link`, `team_only`. Keep the column but change its meaning — it becomes a workflow-level default/flag that says "this workflow has at least one public stage." The real visibility control moves to per-stage `audience`. See note below. |
|
||
| All other columns | ✅ KEEP | Clean. No chat coupling. |
|
||
|
||
**Note on `entry_mode`:** Chat-switchboard treated this as a binary toggle — the whole
|
||
workflow was either public or not. Real workflows mix audiences: internal setup → public
|
||
form → internal review → public confirmation. The per-stage `audience` field (see below)
|
||
is the source of truth. `entry_mode` on the workflow becomes a convenience flag: if any
|
||
stage has `audience = 'public'`, the workflow is public-entry eligible. This avoids a
|
||
full schema break — existing code that checks `entry_mode` still works, it just gets set
|
||
automatically when stages are saved.
|
||
|
||
#### `workflow_stages` table
|
||
|
||
| Column | Disposition | Action |
|
||
|--------|-------------|--------|
|
||
| `persona_id` | 🗑 TRASH | Drop column. Personas are a chat-extension concept. If a stage needs an AI participant, the stage's `surface_pkg_id` extension handles that internally. The comment in the migration already acknowledges this ("personas are extensions now") but the column is still there. Remove it. |
|
||
| `stage_mode` CHECK | ✏️ MOD | Current: `form_only`, `form_chat`, `review`, `custom`. Replace with: `form`, `review`, `delegated`, `automated`. See §4 for definitions. `form_chat` is trash (chat coupling). `form_only` renames to `form`. `custom` renames to `delegated` (clearer intent). `automated` is new (no UI, Starlark-only). |
|
||
| `history_mode` | 🗑 TRASH | Drop column. This controlled how much chat history a persona saw across stages. Core has no chat history. If a delegated surface needs context management, it reads `stage_data` — that's the contract. |
|
||
| `audience` | ➕ ADD | `TEXT NOT NULL DEFAULT 'team'` with CHECK `('team', 'public', 'system')`. Controls who interacts with this stage. `team` = authenticated team members only (internal). `public` = anonymous visitors via entry token (the "public-ish" stage). `system` = no human interaction, automated only. A workflow can freely alternate: team→public→team→public. The kernel enforces this: public stages are accessible via token auth, team stages require JWT, system stages have no UI. |
|
||
| `stage_type` | ➕ ADD | `TEXT NOT NULL DEFAULT 'simple'` with CHECK `('simple', 'dynamic', 'automated')`. Drives the graph engine: `simple` = declarative rules only, `dynamic` = Starlark hook resolves next stage, `automated` = no human UI, fires hook on entry and auto-advances. |
|
||
| `starlark_hook` | ➕ ADD | `TEXT` (nullable). Package-qualified entry point for dynamic/automated stages. Format: `package_id:entry_point`. NULL for simple stages. |
|
||
| `branch_rules` | ➕ ADD | `JSONB NOT NULL DEFAULT '[]'`. Replaces the overloaded `transition_rules` for simple stage branching. Array of `{field, op, value, target_stage}`. Evaluated before `starlark_hook`, before ordinal fallback. |
|
||
| `transition_rules` | ✏️ MOD | Rename to `stage_config`. This JSONB blob was doing double duty (routing rules + on_advance hooks + auto_assign). Routing moves to `branch_rules`. What remains is stage-specific config that the kernel or surface reads: `{on_advance: {package_id, entry_point}, auto_assign: "round_robin"}`. |
|
||
| `assignment_team_id` | ✅ KEEP | Clean. Kernel concept. |
|
||
| `form_template` | ✅ KEEP | Kernel concern — structured data schema. |
|
||
| `auto_transition` | ✅ KEEP | Kernel concern — skip human interaction. |
|
||
| `surface_pkg_id` | ✅ KEEP | The delegation pointer. Required for `delegated` mode stages. |
|
||
| `sla_seconds` | ✅ KEEP | Kernel concern — time budget per stage. |
|
||
|
||
#### `workflow_versions` table
|
||
|
||
| Column | Disposition | Notes |
|
||
|--------|-------------|-------|
|
||
| All existing columns | ✅ KEEP | Immutable snapshots. No changes needed. |
|
||
|
||
### 3b. `007_workflows.sql` — add new tables (same file)
|
||
|
||
#### ➕ ADD `workflow_instances`
|
||
|
||
Replaces the channel-based instance model from chat-switchboard. This is the execution record.
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS workflow_instances (
|
||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||
workflow_id UUID NOT NULL REFERENCES workflows(id) ON DELETE CASCADE,
|
||
workflow_version INTEGER NOT NULL,
|
||
current_stage INTEGER NOT NULL DEFAULT 0,
|
||
stage_data JSONB NOT NULL DEFAULT '{}',
|
||
status TEXT NOT NULL DEFAULT 'active'
|
||
CHECK (status IN ('active', 'completed', 'cancelled', 'stale', 'error')),
|
||
started_by UUID REFERENCES users(id) ON DELETE SET NULL,
|
||
entry_token TEXT, -- for public_link resumption
|
||
metadata JSONB NOT NULL DEFAULT '{}',
|
||
stage_entered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||
);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_wf_instances_workflow
|
||
ON workflow_instances(workflow_id, status);
|
||
CREATE INDEX IF NOT EXISTS idx_wf_instances_status
|
||
ON workflow_instances(status) WHERE status = 'active';
|
||
CREATE UNIQUE INDEX IF NOT EXISTS idx_wf_instances_token
|
||
ON workflow_instances(entry_token) WHERE entry_token IS NOT NULL;
|
||
|
||
DROP TRIGGER IF EXISTS wf_instances_updated_at ON workflow_instances;
|
||
CREATE TRIGGER wf_instances_updated_at BEFORE UPDATE ON workflow_instances
|
||
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
|
||
```
|
||
|
||
**Rationale:** Chat-switchboard used `channels` as instances — that table carried message trees,
|
||
AI context windows, participant lists, and other chat concerns. Core needs a purpose-built table
|
||
that holds only workflow execution state: which version, which stage, accumulated data, lifecycle status.
|
||
|
||
#### ➕ ADD `workflow_assignments`
|
||
|
||
Dropped during the fork ("channel-dependent, rebuild as needed" — see 007 header comment).
|
||
Now rebuilt as a kernel primitive.
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS workflow_assignments (
|
||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||
instance_id UUID NOT NULL REFERENCES workflow_instances(id) ON DELETE CASCADE,
|
||
stage INTEGER NOT NULL,
|
||
team_id UUID NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
|
||
assigned_to UUID REFERENCES users(id) ON DELETE SET NULL,
|
||
status TEXT NOT NULL DEFAULT 'unassigned'
|
||
CHECK (status IN ('unassigned', 'claimed', 'completed', 'cancelled')),
|
||
review_data JSONB NOT NULL DEFAULT '{}',
|
||
claimed_at TIMESTAMPTZ,
|
||
completed_at TIMESTAMPTZ,
|
||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||
);
|
||
|
||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_instance
|
||
ON workflow_assignments(instance_id, stage);
|
||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_team_status
|
||
ON workflow_assignments(team_id, status) WHERE status IN ('unassigned', 'claimed');
|
||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_user
|
||
ON workflow_assignments(assigned_to) WHERE assigned_to IS NOT NULL;
|
||
```
|
||
|
||
**Claim lock:** Same optimistic pattern from chat-switchboard:
|
||
`UPDATE ... WHERE id = $1 AND status = 'unassigned'` — if `rows_affected == 0`, already claimed.
|
||
|
||
### 3c. New migration file: `012_workflow_instances.sql`
|
||
|
||
Wait — per the principle, new *tables* get new files. But we're also editing 007. Decision:
|
||
|
||
**Put the new tables in 007_workflows.sql.** The schema isn't in production. Keep all workflow
|
||
DDL in one file. When the schema *is* in production (post-MVP), new additions get numbered
|
||
migrations. This is consistent with the existing design-decisions log entry:
|
||
"No new migrations pre-MVP."
|
||
|
||
---
|
||
|
||
## 4. Stage Mode Redesign
|
||
|
||
### Current (trash/rename)
|
||
|
||
| Current Mode | Disposition | Rationale |
|
||
|-------------|-------------|-----------|
|
||
| `form_only` | ✏️ MOD → `form` | Rename. Drop the `_only` suffix — there's no `form_chat` to distinguish from anymore. |
|
||
| `form_chat` | 🗑 TRASH | Chat coupling. If you want a form + AI conversation, use `delegated` with a chat surface package. |
|
||
| `review` | ✅ KEEP | Team member reviews accumulated data. Pure kernel concept. |
|
||
| `custom` | ✏️ MOD → `delegated` | Rename for clarity. "Custom" is vague. "Delegated" makes the contract explicit: the kernel delegates stage execution to `surface_pkg_id`. |
|
||
|
||
### New modes
|
||
|
||
| Mode | Disposition | Description |
|
||
|------|-------------|-------------|
|
||
| `form` | ✏️ MOD (renamed) | Kernel renders `form_template`, validates submission, merges into `stage_data`. No extension needed. |
|
||
| `review` | ✅ KEEP | Assignment queue stage. Team member claims, reviews `stage_data`, adds `review_data`, completes. Kernel-rendered. |
|
||
| `delegated` | ✏️ MOD (renamed) | Kernel hands off to `surface_pkg_id`. The surface package receives the instance context (stage_data, form_template, metadata) and calls back to advance. This is where chat, AI, or any custom UX lives. |
|
||
| `automated` | ➕ ADD | No UI. On stage entry, kernel fires `starlark_hook`, merges returned data into `stage_data`, and auto-advances. For enrichment, API calls, validation, routing decisions. |
|
||
|
||
### Stage type vs stage mode
|
||
|
||
These are orthogonal:
|
||
|
||
- **`stage_type`** controls *how the next stage is chosen*: `simple` (declarative branch_rules), `dynamic` (Starlark hook returns target), `automated` (hook + auto-advance).
|
||
- **`stage_mode`** controls *how the stage executes*: `form`, `review`, `delegated`, `automated`.
|
||
|
||
An `automated` stage_type with `automated` stage_mode is the common case for system stages,
|
||
but you can have a `dynamic` stage_type with `form` stage_mode (the form collects data, then a
|
||
Starlark hook decides where to go next based on the submission).
|
||
|
||
---
|
||
|
||
## 5. Model Changes
|
||
|
||
### `server/models/workflow.go`
|
||
|
||
#### WorkflowStage struct — ✏️ MOD
|
||
|
||
```go
|
||
// TRASH: Remove these fields
|
||
// PersonaID *string `json:"persona_id,omitempty"`
|
||
// HistoryMode string `json:"history_mode"`
|
||
|
||
// MOD: Rename transition_rules → stage_config
|
||
// TransitionRules json.RawMessage `json:"transition_rules"`
|
||
StageConfig json.RawMessage `json:"stage_config"`
|
||
|
||
// ADD: New fields
|
||
Audience string `json:"audience"` // team | public | system
|
||
StageType string `json:"stage_type"` // simple | dynamic | automated
|
||
StarlarkHook *string `json:"starlark_hook,omitempty"` // package_id:entry_point
|
||
BranchRules json.RawMessage `json:"branch_rules"` // [{field, op, value, target_stage}]
|
||
```
|
||
|
||
#### Stage mode constants — ✏️ MOD
|
||
|
||
```go
|
||
// TRASH
|
||
// StageModeFormOnly = "form_only"
|
||
// StageModeFormChat = "form_chat"
|
||
|
||
// MOD (rename)
|
||
StageModeForm = "form" // was form_only
|
||
StageModeDelegated = "delegated" // was custom
|
||
|
||
// KEEP
|
||
StageModeReview = "review"
|
||
|
||
// ADD
|
||
StageModeAutomated = "automated"
|
||
|
||
// ADD: Stage type constants
|
||
StageTypeSimple = "simple"
|
||
StageTypeDynamic = "dynamic"
|
||
StageTypeAutomated = "automated"
|
||
|
||
// ADD: Audience constants
|
||
AudienceTeam = "team"
|
||
AudiencePublic = "public"
|
||
AudienceSystem = "system"
|
||
```
|
||
|
||
#### ➕ ADD: WorkflowInstance model
|
||
|
||
```go
|
||
type WorkflowInstance struct {
|
||
ID string `json:"id"`
|
||
WorkflowID string `json:"workflow_id"`
|
||
WorkflowVersion int `json:"workflow_version"`
|
||
CurrentStage int `json:"current_stage"`
|
||
StageData json.RawMessage `json:"stage_data"`
|
||
Status string `json:"status"` // active | completed | cancelled | stale | error
|
||
StartedBy *string `json:"started_by,omitempty"`
|
||
EntryToken *string `json:"entry_token,omitempty"`
|
||
Metadata json.RawMessage `json:"metadata"`
|
||
StageEnteredAt time.Time `json:"stage_entered_at"`
|
||
CreatedAt time.Time `json:"created_at"`
|
||
UpdatedAt time.Time `json:"updated_at"`
|
||
}
|
||
```
|
||
|
||
#### ➕ ADD: WorkflowAssignment model
|
||
|
||
```go
|
||
type WorkflowAssignment struct {
|
||
ID string `json:"id"`
|
||
InstanceID string `json:"instance_id"`
|
||
Stage int `json:"stage"`
|
||
TeamID string `json:"team_id"`
|
||
AssignedTo *string `json:"assigned_to,omitempty"`
|
||
Status string `json:"status"` // unassigned | claimed | completed | cancelled
|
||
ReviewData json.RawMessage `json:"review_data"`
|
||
ClaimedAt *time.Time `json:"claimed_at,omitempty"`
|
||
CompletedAt *time.Time `json:"completed_at,omitempty"`
|
||
CreatedAt time.Time `json:"created_at"`
|
||
}
|
||
```
|
||
|
||
#### TypedFormTemplate, FormField, FormValidation — ✅ KEEP
|
||
|
||
All form types are clean kernel code. No changes.
|
||
|
||
#### ValidateFormData — ✅ KEEP
|
||
|
||
Pure data validation. No chat coupling.
|
||
|
||
---
|
||
|
||
## 6. Store Interface Changes
|
||
|
||
### `server/store/workflow_iface.go` — ✏️ MOD + ➕ ADD
|
||
|
||
Current interface is definition-only (CRUD workflows + stages + versions).
|
||
Add instance and assignment operations.
|
||
|
||
```go
|
||
type WorkflowStore interface {
|
||
// ── Definition CRUD (KEEP — no changes) ──────────
|
||
Create(ctx context.Context, w *models.Workflow) error
|
||
GetByID(ctx context.Context, id string) (*models.Workflow, error)
|
||
GetBySlug(ctx context.Context, teamID *string, slug string) (*models.Workflow, error)
|
||
Update(ctx context.Context, id string, patch models.WorkflowPatch) error
|
||
Delete(ctx context.Context, id string) error
|
||
ListForTeam(ctx context.Context, teamID string) ([]models.Workflow, error)
|
||
ListGlobal(ctx context.Context) ([]models.Workflow, error)
|
||
|
||
// ── Stage CRUD (KEEP — no changes) ───────────────
|
||
CreateStage(ctx context.Context, s *models.WorkflowStage) error
|
||
ListStages(ctx context.Context, workflowID string) ([]models.WorkflowStage, error)
|
||
UpdateStage(ctx context.Context, s *models.WorkflowStage) error
|
||
DeleteStage(ctx context.Context, id string) error
|
||
ReorderStages(ctx context.Context, workflowID string, orderedIDs []string) error
|
||
|
||
// ── Versioning (KEEP — no changes) ───────────────
|
||
Publish(ctx context.Context, v *models.WorkflowVersion) error
|
||
GetVersion(ctx context.Context, workflowID string, versionNumber int) (*models.WorkflowVersion, error)
|
||
GetLatestVersion(ctx context.Context, workflowID string) (*models.WorkflowVersion, error)
|
||
|
||
// ── Instances (ADD) ──────────────────────────────
|
||
CreateInstance(ctx context.Context, inst *models.WorkflowInstance) error
|
||
GetInstance(ctx context.Context, id string) (*models.WorkflowInstance, error)
|
||
GetInstanceByToken(ctx context.Context, token string) (*models.WorkflowInstance, error)
|
||
UpdateInstance(ctx context.Context, id string, patch models.InstancePatch) error
|
||
ListInstances(ctx context.Context, workflowID string, status string) ([]models.WorkflowInstance, error)
|
||
AdvanceStage(ctx context.Context, id string, nextStage int, mergeData json.RawMessage) error
|
||
CompleteInstance(ctx context.Context, id string) error
|
||
CancelInstance(ctx context.Context, id string) error
|
||
MarkStale(ctx context.Context, olderThan time.Duration) (int, error)
|
||
|
||
// ── Assignments (ADD) ────────────────────────────
|
||
CreateAssignment(ctx context.Context, a *models.WorkflowAssignment) error
|
||
ClaimAssignment(ctx context.Context, id string, userID string) error // optimistic lock
|
||
UnclaimAssignment(ctx context.Context, id string) error
|
||
CompleteAssignment(ctx context.Context, id string, reviewData json.RawMessage) error
|
||
CancelAssignment(ctx context.Context, id string) error
|
||
ListAssignmentsByTeam(ctx context.Context, teamID string, status string) ([]models.WorkflowAssignment, error)
|
||
ListAssignmentsByUser(ctx context.Context, userID string) ([]models.WorkflowAssignment, error)
|
||
ListAssignmentsByInstance(ctx context.Context, instanceID string) ([]models.WorkflowAssignment, error)
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 7. Handler Changes
|
||
|
||
### `server/handlers/workflows.go` — ✏️ MOD
|
||
|
||
**Stage CRUD updates** — reflect new field names in CreateStage/UpdateStage:
|
||
- Remove `persona_id` and `history_mode` from bind/validation.
|
||
- Add `stage_type`, `starlark_hook`, `branch_rules` to bind/validation.
|
||
- Rename `transition_rules` → `stage_config` in bind/validation.
|
||
- Update `stage_mode` CHECK to new set: `form`, `review`, `delegated`, `automated`.
|
||
- Validate: `delegated` requires `surface_pkg_id`. `dynamic`/`automated` stage_type requires `starlark_hook`.
|
||
|
||
### `server/handlers/workflow_hooks.go` — ✏️ MOD
|
||
|
||
**Rename:** `channelID` parameter → `instanceID` in `FireOnAdvanceHook` signature.
|
||
The hook context dict changes from `{channel_id, ...}` to `{instance_id, ...}`.
|
||
|
||
The `jsonToStarlark` helper referenced on line 81 lives in
|
||
`server/handlers/starlark_helpers.go`. Consider consolidating with `goToStarlark` in
|
||
`server/triggers/event.go` — two independent Go→Starlark converters is tech debt.
|
||
|
||
### `server/handlers/workflow_team.go` — ✅ KEEP
|
||
|
||
Team-scoped wrappers. Clean delegation pattern. No changes.
|
||
|
||
### `server/handlers/workflow_packages.go` — ✏️ MOD
|
||
|
||
Update `workflowPkgStage` struct:
|
||
- Remove `PersonaID`, `HistoryMode` fields.
|
||
- Add `StageType`, `StarlarkHook`, `BranchRules` fields.
|
||
- Rename `TransitionRules` → `StageConfig`.
|
||
|
||
Update `ExportWorkflowPackage` and `InstallWorkflowFromManifest` accordingly.
|
||
|
||
### ➕ ADD `server/handlers/workflow_instances.go`
|
||
|
||
New handler file for instance lifecycle:
|
||
|
||
```
|
||
POST /api/v1/workflows/:id/start → Start (create instance from latest published version)
|
||
GET /api/v1/workflows/instances/:iid → GetInstance
|
||
POST /api/v1/workflows/instances/:iid/advance → Advance (submit stage data, resolve next)
|
||
POST /api/v1/workflows/instances/:iid/cancel → Cancel
|
||
GET /api/v1/workflows/:id/instances → ListInstances (by workflow, optionally by status)
|
||
```
|
||
|
||
Public entry (for `public_link` workflows):
|
||
```
|
||
POST /api/v1/workflows/entry/:slug → StartPublic (no auth, generates entry_token)
|
||
GET /api/v1/workflows/entry/:token → ResumePublic (retrieve instance by token)
|
||
POST /api/v1/workflows/entry/:token/advance → AdvancePublic (submit data via token)
|
||
```
|
||
|
||
### ➕ ADD `server/handlers/workflow_assignments.go`
|
||
|
||
New handler file for the assignment queue:
|
||
|
||
```
|
||
GET /api/v1/teams/:teamId/assignments → ListTeamAssignments (filterable by status)
|
||
POST /api/v1/teams/:teamId/assignments/:id/claim → Claim
|
||
POST /api/v1/teams/:teamId/assignments/:id/unclaim → Unclaim
|
||
POST /api/v1/teams/:teamId/assignments/:id/complete → Complete (with review_data)
|
||
POST /api/v1/teams/:teamId/assignments/:id/cancel → Cancel
|
||
GET /api/v1/assignments/mine → ListMyAssignments (across teams)
|
||
```
|
||
|
||
---
|
||
|
||
## 8. Routing Engine Changes
|
||
|
||
### `server/workflow/routing.go` — ✏️ MOD
|
||
|
||
The existing `ResolveNextStage` evaluates `transition_rules.conditions[]`. This needs to
|
||
become a two-phase resolution that respects the new `stage_type`:
|
||
|
||
```
|
||
Phase 1: Evaluate branch_rules (declarative, for simple + dynamic types)
|
||
Phase 2: If no match AND stage_type == dynamic → fire starlark_hook
|
||
Phase 3: Fallback → currentStage + 1
|
||
```
|
||
|
||
For `automated` stage_type, the engine fires `starlark_hook` on entry (not for routing — for
|
||
data enrichment), then routes via branch_rules or ordinal fallback. The routing call happens
|
||
*after* the hook returns.
|
||
|
||
**Concrete changes:**
|
||
- `ResolveNextStage` signature: add `stageType string, starlarkHook *string, runner *sandbox.Runner` parameters.
|
||
- Extract branch_rules evaluation from the current `TransitionRulesWithConditions` (which reads from `transition_rules` JSON) into a dedicated function that reads from the new `branch_rules` field.
|
||
- `TransitionRulesWithConditions` struct → rename to `StageConfig`, remove `Conditions` field (moved to branch_rules), keep `AutoAssign` and `OnAdvance`.
|
||
|
||
### `server/workflow/` — ➕ ADD `engine.go`
|
||
|
||
New file: the stage execution engine. Orchestrates the advance lifecycle:
|
||
|
||
```
|
||
1. Load instance + version snapshot
|
||
2. Validate current stage allows advancement (status checks)
|
||
3. If current stage has form_template → validate submitted data
|
||
4. Merge submitted data into stage_data
|
||
5. Fire on_advance hook (if configured in stage_config)
|
||
6. Resolve next stage (branch_rules → starlark_hook → ordinal)
|
||
7. If next stage is past last stage → complete instance
|
||
8. If next stage is automated → fire its hook, recurse to step 6
|
||
9. If next stage has assignment_team_id → create WorkflowAssignment
|
||
10. Update instance (current_stage, stage_data, stage_entered_at)
|
||
11. Emit bus events (workflow.advanced, workflow.assigned, workflow.completed)
|
||
```
|
||
|
||
### `server/workflow/` — ➕ ADD `automated.go`
|
||
|
||
Handles `automated` stage execution: fire the Starlark hook, merge returned data, and
|
||
auto-advance. Includes a cycle guard (max 10 consecutive automated stages) to prevent
|
||
infinite loops from misconfigured workflows.
|
||
|
||
---
|
||
|
||
## 9. Event Bus Updates
|
||
|
||
### `server/events/types.go` — ✏️ MOD
|
||
|
||
Current workflow events are fine but incomplete. Add:
|
||
|
||
| Event | Direction | Disposition | Trigger |
|
||
|-------|-----------|-------------|---------|
|
||
| `workflow.assigned` | DirToClient | ✅ KEEP | Assignment created |
|
||
| `workflow.claimed` | DirToClient | ✅ KEEP | Assignment claimed |
|
||
| `workflow.advanced` | DirToClient | ✅ KEEP | Stage transition |
|
||
| `workflow.completed` | DirToClient | ✅ KEEP | Instance completed |
|
||
| `workflow.cancelled` | DirToClient | ➕ ADD | Instance cancelled |
|
||
| `workflow.started` | DirToClient | ➕ ADD | Instance created |
|
||
| `workflow.sla.warning` | DirToClient | ➕ ADD | SLA at 80% threshold |
|
||
| `workflow.sla.breached` | DirToClient | ➕ ADD | SLA exceeded |
|
||
| `workflow.error` | DirLocal | ➕ ADD | Hook execution failure |
|
||
|
||
---
|
||
|
||
## 10. Starlark Module Updates
|
||
|
||
### `server/sandbox/workflow_module.go` — ✏️ MOD
|
||
|
||
The `BuildWorkflowModule` (in `workflow_module.go`, not `modules.go`) currently exists but
|
||
targets the old channel-based model. It currently only exposes `get_definition`.
|
||
Rebuild to expose instance-oriented operations:
|
||
|
||
```python
|
||
# Available to extensions with workflow.read or workflow.write permission
|
||
workflow.get_instance(instance_id) # → instance dict
|
||
workflow.get_stage_data(instance_id) # → stage_data dict
|
||
workflow.set_stage_data(instance_id, data) # → merge into stage_data
|
||
workflow.advance(instance_id) # → trigger advance (write perm)
|
||
workflow.get_assignments(instance_id) # → assignment list
|
||
```
|
||
|
||
The old `workflow_advance()` function that chat-switchboard's personas called is gone.
|
||
Extensions call `workflow.advance()` which delegates to the same engine as the HTTP handler.
|
||
|
||
---
|
||
|
||
## 11. Salvage Map from chat-switchboard v0.39.x
|
||
|
||
What was planned there, and what happens to each idea in core:
|
||
|
||
### v0.39.0 — Stage Graph Engine + Form Builder
|
||
|
||
| Component | Disposition | Core equivalent |
|
||
|-----------|-------------|-----------------|
|
||
| `stage_type` enum | ➕ ADD | Adopted directly: `simple`, `dynamic`, `automated` |
|
||
| `BranchRule` struct | ➕ ADD | Adopted as `branch_rules` JSONB on `workflow_stages` |
|
||
| Starlark hook integration | ➕ ADD | `starlark_hook` column + engine integration |
|
||
| Graph engine (`server/engine/graph.go`) | ➕ ADD | `server/workflow/engine.go` in core |
|
||
| Automated stage runner | ➕ ADD | `server/workflow/automated.go` in core |
|
||
| Visual form builder (5 Preact components) | 🗑 TRASH *for now* | The form builder was tightly coupled to chat-switchboard's stage editor surface. Core doesn't ship a workflow editor surface (that's an extension). The form *schema* is kernel; the form *builder UI* is a surface package concern. Revisit when a workflow-admin surface is built. |
|
||
| Stage type badges in UI | 🗑 TRASH | Same — UI is extension territory. |
|
||
|
||
### v0.39.1 — Stage Reorder + Bulk Ops
|
||
|
||
| Component | Disposition | Core equivalent |
|
||
|-----------|-------------|-----------------|
|
||
| Drag-and-drop reorder | 🗑 TRASH | UI concern. The kernel `ReorderStages` endpoint already exists. |
|
||
| Bulk assignment ops | ➕ ADD (later) | Useful but not blocking. Defer to v0.2.7+. The individual claim/unclaim/cancel endpoints come first. |
|
||
|
||
### v0.39.2 — SLA Alerting
|
||
|
||
| Component | Disposition | Core equivalent |
|
||
|-----------|-------------|-----------------|
|
||
| Periodic SLA scanner | ➕ ADD | Kernel scheduled goroutine (like the existing staleness sweep). Query active instances with `sla_seconds` configured, compute breach from `stage_entered_at`. |
|
||
| `sla_notified_at` on instances | ➕ ADD | Add to `workflow_instances` schema. Prevents duplicate notifications. |
|
||
| Notifications on breach | ➕ ADD | Use existing `notifications` table + bus event (`workflow.sla.breached`). |
|
||
| Webhook on breach | ➕ ADD | Fire workflow's `webhook_url` if configured. |
|
||
| Pulsing badge animation | 🗑 TRASH | UI concern for a surface package. |
|
||
|
||
### v0.39.3 — Visitor Portal
|
||
|
||
| Component | Disposition | Core equivalent |
|
||
|-----------|-------------|-----------------|
|
||
| Public entry endpoint | ➕ ADD | `/api/v1/workflows/entry/:slug` — kernel handles anonymous session creation + token issuance. |
|
||
| Session resume via token | ➕ ADD | `entry_token` on `workflow_instances` + `GetInstanceByToken` store method. |
|
||
| Branded entry page | 🗑 TRASH | Surface package. The kernel stores `branding` JSONB — a portal surface reads it. |
|
||
| Progress indicator | 🗑 TRASH | Surface package reads stage list, computes "step N of M" locally. |
|
||
| Email notifications | 🗑 TRASH *for now* | Requires SMTP infrastructure. Defer to post-MVP or make it a connection-based extension (use ext_connections for SMTP config, fire via Starlark hook). |
|
||
|
||
### v0.39.4 — Templates + Clone
|
||
|
||
| Component | Disposition | Core equivalent |
|
||
|-----------|-------------|-----------------|
|
||
| Clone endpoint | ➕ ADD | `POST /api/v1/workflows/:id/clone` — deep copy workflow + stages. Straightforward. |
|
||
| Built-in templates table | 🗑 TRASH | Core doesn't ship opinionated templates. Templates are workflow packages (`.pkg` archives). A "template gallery" is a surface. |
|
||
| Template picker UI | 🗑 TRASH | Surface concern. |
|
||
|
||
### v0.39.5 — Analytics
|
||
|
||
| Component | Disposition | Core equivalent |
|
||
|-----------|-------------|-----------------|
|
||
| Analytics query endpoint | ➕ ADD (later) | `GET /api/v1/workflows/:id/analytics?period=7d` — kernel exposes raw metrics (throughput, cycle time, stage dwell, SLA compliance) derived from instance + assignment timestamps. Defer to v0.2.8+ or v0.3.x. |
|
||
| Analytics dashboard UI | 🗑 TRASH | Surface package. |
|
||
|
||
---
|
||
|
||
## 12. Implementation Order
|
||
|
||
All items within v0.2.6. Ordered by dependency:
|
||
|
||
### Phase A: Schema + Models
|
||
|
||
1. ✏️ Edit `007_workflows.sql`: drop `persona_id`, `history_mode`; add `audience`, `stage_type`, `starlark_hook`, `branch_rules`; rename `transition_rules` → `stage_config`; update `stage_mode` CHECK.
|
||
2. ✏️ Edit `007_workflows.sql` (same file): add `workflow_instances` and `workflow_assignments` tables.
|
||
3. ✏️ Edit `007_workflows.sql` (SQLite dialect): mirror changes.
|
||
4. ✏️ Update `server/models/workflow.go`: remove trashed fields, add new structs, update constants.
|
||
|
||
### Phase B: Store Layer
|
||
|
||
5. ✏️ Update `server/store/workflow_iface.go`: add instance + assignment methods.
|
||
6. ➕ Implement Postgres store: `server/store/postgres/workflow_instances.go`, `workflow_assignments.go`.
|
||
7. ➕ Implement SQLite store: `server/store/sqlite/workflow_instances.go`, `workflow_assignments.go`.
|
||
|
||
### Phase C: Engine
|
||
|
||
8. ✏️ Update `server/workflow/routing.go`: branch_rules evaluation, starlark_hook integration.
|
||
9. ➕ Add `server/workflow/engine.go`: advance lifecycle orchestration.
|
||
10. ➕ Add `server/workflow/automated.go`: automated stage execution + cycle guard.
|
||
11. ✏️ Update `server/handlers/workflow_hooks.go`: rename channel → instance references.
|
||
|
||
### Phase D: Handlers + Routes
|
||
|
||
12. ✏️ Update `server/handlers/workflows.go`: stage CRUD field changes.
|
||
13. ✏️ Update `server/handlers/workflow_packages.go`: package struct field changes.
|
||
14. ➕ Add `server/handlers/workflow_instances.go`: instance lifecycle endpoints.
|
||
15. ➕ Add `server/handlers/workflow_assignments.go`: assignment queue endpoints.
|
||
16. ✏️ Wire new routes in `server/main.go`.
|
||
|
||
### Phase E: Events + Starlark
|
||
|
||
17. ✏️ Update `server/events/types.go`: add new workflow events.
|
||
18. ✏️ Update `server/sandbox/modules.go`: rebuild workflow Starlark module.
|
||
|
||
### Phase F: Background Jobs
|
||
|
||
19. ➕ Add SLA scanner goroutine (check active instances, fire notifications + events on breach).
|
||
20. ➕ Add staleness sweep for workflow instances (reuse pattern from chat-switchboard).
|
||
|
||
### Phase G: Verification
|
||
|
||
21. ➕ Integration tests: instance lifecycle, stage advancement, branching, automated stages, assignments, SLA breach.
|
||
22. `go build ./...` clean.
|
||
23. Update ICD (OpenAPI spec) with new endpoints.
|
||
24. Update ROADMAP.md + CHANGELOG.md.
|
||
|
||
---
|
||
|
||
## 13. What We're NOT Doing (Explicit Deferrals)
|
||
|
||
| Item | Why | Revisit |
|
||
|------|-----|---------|
|
||
| Visual form builder UI | Surface concern, no workflow-admin surface yet | When a workflow-admin surface ships |
|
||
| Drag-and-drop stage reorder UI | Surface concern | Same |
|
||
| Email notifications | Needs SMTP infrastructure or connection-based ext | Post-MVP |
|
||
| Built-in workflow templates | Core doesn't ship opinionated content | Package gallery |
|
||
| Analytics dashboard | Nice-to-have, kernel endpoint can come first | v0.3.x |
|
||
| Bulk assignment operations | Individual ops first | v0.2.7+ |
|
||
| Workflow chaining (`on_complete`) | Already in schema, needs engine wiring | v0.2.7 |
|
||
| Visitor portal surface | Surface package, not kernel | Extension track |
|
||
|
||
---
|
||
|
||
## 14. Migration Policy Reminder
|
||
|
||
From the design-decisions log:
|
||
|
||
> **No new migrations pre-MVP.** Edit existing migration SQL files in place.
|
||
> No migration chains until schema is in production.
|
||
|
||
For this redesign:
|
||
- **`007_workflows.sql`**: Edit in place for column changes AND add new tables to the same file.
|
||
- **No `012_*.sql`**: Everything goes in 007.
|
||
- **Both dialects**: Postgres and SQLite must be kept in sync.
|
||
|
||
The only time a new numbered migration file is created is when a truly new,
|
||
unrelated subsystem is added (like 011_triggers.sql was for the trigger system).
|
||
Workflow instances and assignments are part of the workflow subsystem → they go in 007.
|
||
|
||
---
|
||
|
||
## 15. Public Surfaces — Kernel Capability, Not Workflow Feature
|
||
|
||
### The problem with workflow-owned public access
|
||
|
||
Chat-switchboard treated public access as a binary workflow toggle: `entry_mode: public_link`
|
||
made the whole workflow public. But real workflows alternate audiences — internal setup →
|
||
public-facing customer form → internal review → public confirmation page. And public access
|
||
isn't even a workflow-only need: feedback forms, status pages, and surveys all need anonymous
|
||
sessions without being workflows.
|
||
|
||
### Per-stage audience (the v0.2.6 solution)
|
||
|
||
The `audience` field on `workflow_stages` is the source of truth:
|
||
|
||
| Audience | Who interacts | Auth required | Example |
|
||
|----------|--------------|---------------|---------|
|
||
| `team` | Authenticated team members | JWT | Internal triage, setup, review |
|
||
| `public` | Anonymous visitors | Entry token | Customer intake form, confirmation page |
|
||
| `system` | Nobody (automated) | N/A | API enrichment, routing decision |
|
||
|
||
A single workflow freely alternates:
|
||
|
||
```
|
||
Stage 0: "Setup" audience=team (agent configures parameters)
|
||
Stage 1: "Customer Form" audience=public (customer fills out intake)
|
||
Stage 2: "Internal Review" audience=team (team reviews submission)
|
||
Stage 3: "Confirmation" audience=public (customer sees result)
|
||
```
|
||
|
||
The engine enforces boundaries:
|
||
- When the instance enters a `public` stage, the visitor's token grants access. Team members
|
||
can also see it (they have elevated access).
|
||
- When the instance enters a `team` stage, the visitor's view freezes. They see "being
|
||
processed" or whatever the portal surface shows. The token is still valid for resumption
|
||
when the next `public` stage arrives.
|
||
- When the instance enters a `system` stage, nobody interacts — the automated hook fires
|
||
and the engine advances.
|
||
|
||
### The `entry_mode` column
|
||
|
||
Stays on the `workflows` table but becomes a derived convenience flag. If any stage has
|
||
`audience = 'public'`, the workflow is public-entry eligible. The kernel can auto-compute
|
||
this when stages are saved, or treat it as an explicit admin toggle that gates whether
|
||
public stages are actually reachable. Either way, it's no longer the primary access control
|
||
mechanism — `audience` on each stage is.
|
||
|
||
### Platform-level public access (v0.2.7+ concern)
|
||
|
||
The broader vision: public access as a **package-level manifest capability** so any surface
|
||
(not just workflows) can serve anonymous visitors. A package declares `"public": true` in
|
||
its manifest, and the kernel mounts a public route namespace (`/p/:package_slug/*`) with
|
||
anonymous session middleware, token issuance, and rate limiting. Workflows consume this
|
||
capability — they don't own it.
|
||
|
||
For v0.2.6, the workflow-specific public entry routes are sufficient:
|
||
```
|
||
POST /api/v1/workflows/entry/:slug → StartPublic
|
||
GET /api/v1/workflows/entry/:token → ResumePublic
|
||
POST /api/v1/workflows/entry/:token/advance → AdvancePublic
|
||
```
|
||
|
||
The anonymous session management (entry_token on `workflow_instances`, IP rate limiting)
|
||
lives in the workflow handler for now. Generalization to a platform-level `public_sessions`
|
||
table happens when non-workflow surfaces need it.
|
||
|
||
### Visitor view contract
|
||
|
||
When a public stage is active, the kernel API returns only what the visitor should see:
|
||
- The current stage's `form_template` (if mode is `form`)
|
||
- The stage name and ordinal (for progress indicators)
|
||
- Accumulated `stage_data` keys marked as `visitor_visible` (future: field-level ACL)
|
||
- The `branding` JSONB from the workflow
|
||
|
||
Internal stages, team assignments, review data, and other instances' data are never
|
||
exposed through the token-authenticated endpoints. The surface package renders whatever
|
||
the kernel returns — it doesn't need to filter.
|
||
|
||
---
|
||
|
||
## 16. Branch Rules — Routing, Skipping, and Convergence
|
||
|
||
### How branch_rules work
|
||
|
||
Each stage has a `branch_rules` JSONB array. On stage completion, the engine evaluates rules
|
||
top-to-bottom against accumulated `stage_data`. First match wins. No match → ordinal + 1.
|
||
|
||
```json
|
||
[
|
||
{"field": "priority", "op": "eq", "value": "urgent", "target_stage": "Emergency Review"},
|
||
{"field": "amount", "op": "gt", "value": 10000, "target_stage": "Manager Approval"},
|
||
{"field": "type", "op": "in", "value": ["a","b"], "target_stage": "Fast Track"}
|
||
]
|
||
```
|
||
|
||
`target_stage` accepts either a **stage name** (case-insensitive) or a **numeric ordinal**.
|
||
The engine tries name resolution first, falls back to numeric. This is already implemented
|
||
in `routing.go` → `resolveTarget()`.
|
||
|
||
### Skipping stages
|
||
|
||
Branch rules can target any stage, not just the next one. If a workflow has stages
|
||
0→1→2→3→4 and stage 1's rules say `target_stage: "Stage 4"`, stages 2 and 3 are skipped
|
||
for that instance. The engine doesn't care about gaps — it sets `current_stage` to whatever
|
||
the rule resolved to.
|
||
|
||
Linear workflows (no branch_rules on any stage) advance 0→1→2→3→4 by the ordinal + 1
|
||
fallback. Adding a single branch rule to any stage introduces conditional skipping without
|
||
changing the linear stages.
|
||
|
||
### Convergence (diamond patterns)
|
||
|
||
Two different branches can target the same downstream stage:
|
||
|
||
```
|
||
Stage 0: Intake Form
|
||
branch_rules: [
|
||
{field: "region", op: "eq", value: "US", target_stage: "US Processing"},
|
||
{field: "region", op: "eq", value: "EU", target_stage: "EU Processing"}
|
||
]
|
||
|
||
Stage 1: US Processing (ordinal 1)
|
||
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
|
||
(always converge after processing)
|
||
|
||
Stage 2: EU Processing (ordinal 2)
|
||
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
|
||
(always converge after processing)
|
||
|
||
Stage 3: Final Review (ordinal 3)
|
||
(both branches land here)
|
||
```
|
||
|
||
No special syntax needed. The routing engine doesn't track "which branch am I on" — it just
|
||
resolves the target and moves the instance there. The accumulated `stage_data` carries
|
||
everything downstream stages need to know about what happened upstream.
|
||
|
||
### Backward jumps
|
||
|
||
Branch rules can also target earlier stages (lower ordinals) for retry/correction loops.
|
||
The cycle guard (max 10 consecutive automated stages) in `automated.go` prevents infinite
|
||
loops for automated stages. For human-facing stages, backward jumps are intentional — the
|
||
human breaks the cycle by submitting different data.
|
||
|
||
### Operators
|
||
|
||
Full set, inherited from the existing `routing.go` implementation:
|
||
|
||
| Op | Description | Example |
|
||
|----|-------------|---------|
|
||
| `eq` | Loose equality (string comparison) | `"status" eq "approved"` |
|
||
| `neq` | Not equal | `"status" neq "rejected"` |
|
||
| `gt` | Greater than (numeric) | `"amount" gt 10000` |
|
||
| `lt` | Less than (numeric) | `"amount" lt 100` |
|
||
| `gte` | Greater or equal | `"score" gte 80` |
|
||
| `lte` | Less or equal | `"score" lte 20` |
|
||
| `in` | Value in array | `"category" in ["a","b","c"]` |
|
||
| `contains` | Substring match | `"notes" contains "urgent"` |
|
||
| `exists` | Field present in stage_data | `"approval_date" exists` |
|
||
| `not_exists` | Field absent | `"rejection_reason" not_exists` |
|