Docs: workflow redesign design doc for v0.2.6
All checks were successful
CI/CD / detect-changes (pull_request) Successful in 4s
CI/CD / test-frontend (pull_request) Has been skipped
CI/CD / test-sqlite (pull_request) Has been skipped
CI/CD / test-go-pg (pull_request) Has been skipped
CI/CD / build-and-deploy (pull_request) Has been skipped
All checks were successful
CI/CD / detect-changes (pull_request) Successful in 4s
CI/CD / test-frontend (pull_request) Has been skipped
CI/CD / test-sqlite (pull_request) Has been skipped
CI/CD / test-go-pg (pull_request) Has been skipped
CI/CD / build-and-deploy (pull_request) Has been skipped
Maps viable chat-switchboard v0.39.x concepts onto core's extension-first architecture. Classifies every component as trash/mod/add/keep. Key decisions: per-stage audience (team/public/system) replacing binary entry_mode, branch_rules for conditional routing and stage skipping, public surfaces as a kernel capability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
806
docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md
Normal file
806
docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md
Normal file
@@ -0,0 +1,806 @@
|
||||
# Workflow Redesign — v0.2.6
|
||||
|
||||
**Status:** Draft
|
||||
**Branch:** `feat/workflow-redesign-v0.2.6`
|
||||
**Date:** 2026-03-27
|
||||
**Context:** Mapping viable ideas from chat-switchboard v0.39.x onto core's extension-first architecture.
|
||||
|
||||
---
|
||||
|
||||
## 1. Guiding Principles
|
||||
|
||||
1. **Workflows are kernel.** Definitions, instances, the stage graph, form validation, and
|
||||
assignment queues are platform primitives — not extensions.
|
||||
2. **Execution is delegated.** How a stage renders or collects data is an extension concern.
|
||||
The kernel says *what* a stage needs; a surface package provides the *how*.
|
||||
3. **No chat assumptions.** The kernel never references personas, channels, or AI providers.
|
||||
A chat extension can participate in workflows, but the workflow engine does not depend on it.
|
||||
4. **Public access is a platform capability.** Anonymous/public sessions are not workflow-specific —
|
||||
they are a kernel-level feature that any surface can opt into. Workflows *consume* public
|
||||
access; they don't own it. See §15.
|
||||
5. **Edit existing migrations.** Per the design-decisions log: no migration chains until the
|
||||
schema is in production. New tables get new files; modified tables are edited in place.
|
||||
|
||||
---
|
||||
|
||||
## 2. Disposition Index
|
||||
|
||||
Every item below is classified:
|
||||
|
||||
| Tag | Meaning |
|
||||
|-----|---------|
|
||||
| **🗑 TRASH** | Remove from core. Vestigial chat-switchboard concept that doesn't fit. |
|
||||
| **✏️ MOD** | Exists in core today — modify in place. |
|
||||
| **➕ ADD** | New to core. Requires new code/tables. |
|
||||
| **✅ KEEP** | Already correct in core. No changes needed. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Schema Changes
|
||||
|
||||
### 3a. `007_workflows.sql` — edit in place
|
||||
|
||||
#### `workflows` table
|
||||
|
||||
| Column | Disposition | Action |
|
||||
|--------|-------------|--------|
|
||||
| `entry_mode` | ✏️ MOD | Current values: `public_link`, `team_only`. Keep the column but change its meaning — it becomes a workflow-level default/flag that says "this workflow has at least one public stage." The real visibility control moves to per-stage `audience`. See note below. |
|
||||
| All other columns | ✅ KEEP | Clean. No chat coupling. |
|
||||
|
||||
**Note on `entry_mode`:** Chat-switchboard treated this as a binary toggle — the whole
|
||||
workflow was either public or not. Real workflows mix audiences: internal setup → public
|
||||
form → internal review → public confirmation. The per-stage `audience` field (see below)
|
||||
is the source of truth. `entry_mode` on the workflow becomes a convenience flag: if any
|
||||
stage has `audience = 'public'`, the workflow is public-entry eligible. This avoids a
|
||||
full schema break — existing code that checks `entry_mode` still works, it just gets set
|
||||
automatically when stages are saved.
|
||||
|
||||
#### `workflow_stages` table
|
||||
|
||||
| Column | Disposition | Action |
|
||||
|--------|-------------|--------|
|
||||
| `persona_id` | 🗑 TRASH | Drop column. Personas are a chat-extension concept. If a stage needs an AI participant, the stage's `surface_pkg_id` extension handles that internally. The comment in the migration already acknowledges this ("personas are extensions now") but the column is still there. Remove it. |
|
||||
| `stage_mode` CHECK | ✏️ MOD | Current: `form_only`, `form_chat`, `review`, `custom`. Replace with: `form`, `review`, `delegated`, `automated`. See §4 for definitions. `form_chat` is trash (chat coupling). `form_only` renames to `form`. `custom` renames to `delegated` (clearer intent). `automated` is new (no UI, Starlark-only). |
|
||||
| `history_mode` | 🗑 TRASH | Drop column. This controlled how much chat history a persona saw across stages. Core has no chat history. If a delegated surface needs context management, it reads `stage_data` — that's the contract. |
|
||||
| `audience` | ➕ ADD | `TEXT NOT NULL DEFAULT 'team'` with CHECK `('team', 'public', 'system')`. Controls who interacts with this stage. `team` = authenticated team members only (internal). `public` = anonymous visitors via entry token (the "public-ish" stage). `system` = no human interaction, automated only. A workflow can freely alternate: team→public→team→public. The kernel enforces this: public stages are accessible via token auth, team stages require JWT, system stages have no UI. |
|
||||
| `stage_type` | ➕ ADD | `TEXT NOT NULL DEFAULT 'simple'` with CHECK `('simple', 'dynamic', 'automated')`. Drives the graph engine: `simple` = declarative rules only, `dynamic` = Starlark hook resolves next stage, `automated` = no human UI, fires hook on entry and auto-advances. |
|
||||
| `starlark_hook` | ➕ ADD | `TEXT` (nullable). Package-qualified entry point for dynamic/automated stages. Format: `package_id:entry_point`. NULL for simple stages. |
|
||||
| `branch_rules` | ➕ ADD | `JSONB NOT NULL DEFAULT '[]'`. Replaces the overloaded `transition_rules` for simple stage branching. Array of `{field, op, value, target_stage}`. Evaluated before `starlark_hook`, before ordinal fallback. |
|
||||
| `transition_rules` | ✏️ MOD | Rename to `stage_config`. This JSONB blob was doing double duty (routing rules + on_advance hooks + auto_assign). Routing moves to `branch_rules`. What remains is stage-specific config that the kernel or surface reads: `{on_advance: {package_id, entry_point}, auto_assign: "round_robin"}`. |
|
||||
| `assignment_team_id` | ✅ KEEP | Clean. Kernel concept. |
|
||||
| `form_template` | ✅ KEEP | Kernel concern — structured data schema. |
|
||||
| `auto_transition` | ✅ KEEP | Kernel concern — skip human interaction. |
|
||||
| `surface_pkg_id` | ✅ KEEP | The delegation pointer. Required for `delegated` mode stages. |
|
||||
| `sla_seconds` | ✅ KEEP | Kernel concern — time budget per stage. |
|
||||
|
||||
#### `workflow_versions` table
|
||||
|
||||
| Column | Disposition | Notes |
|
||||
|--------|-------------|-------|
|
||||
| All existing columns | ✅ KEEP | Immutable snapshots. No changes needed. |
|
||||
|
||||
### 3b. `007_workflows.sql` — add new tables (same file)
|
||||
|
||||
#### ➕ ADD `workflow_instances`
|
||||
|
||||
Replaces the channel-based instance model from chat-switchboard. This is the execution record.
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS workflow_instances (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
workflow_id UUID NOT NULL REFERENCES workflows(id) ON DELETE CASCADE,
|
||||
workflow_version INTEGER NOT NULL,
|
||||
current_stage INTEGER NOT NULL DEFAULT 0,
|
||||
stage_data JSONB NOT NULL DEFAULT '{}',
|
||||
status TEXT NOT NULL DEFAULT 'active'
|
||||
CHECK (status IN ('active', 'completed', 'cancelled', 'stale', 'error')),
|
||||
started_by UUID REFERENCES users(id) ON DELETE SET NULL,
|
||||
entry_token TEXT, -- for public_link resumption
|
||||
metadata JSONB NOT NULL DEFAULT '{}',
|
||||
stage_entered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_instances_workflow
|
||||
ON workflow_instances(workflow_id, status);
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_instances_status
|
||||
ON workflow_instances(status) WHERE status = 'active';
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_wf_instances_token
|
||||
ON workflow_instances(entry_token) WHERE entry_token IS NOT NULL;
|
||||
|
||||
DROP TRIGGER IF EXISTS wf_instances_updated_at ON workflow_instances;
|
||||
CREATE TRIGGER wf_instances_updated_at BEFORE UPDATE ON workflow_instances
|
||||
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
|
||||
```
|
||||
|
||||
**Rationale:** Chat-switchboard used `channels` as instances — that table carried message trees,
|
||||
AI context windows, participant lists, and other chat concerns. Core needs a purpose-built table
|
||||
that holds only workflow execution state: which version, which stage, accumulated data, lifecycle status.
|
||||
|
||||
#### ➕ ADD `workflow_assignments`
|
||||
|
||||
Dropped during the fork ("channel-dependent, rebuild as needed" — see 007 header comment).
|
||||
Now rebuilt as a kernel primitive.
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS workflow_assignments (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
instance_id UUID NOT NULL REFERENCES workflow_instances(id) ON DELETE CASCADE,
|
||||
stage INTEGER NOT NULL,
|
||||
team_id UUID NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
|
||||
assigned_to UUID REFERENCES users(id) ON DELETE SET NULL,
|
||||
status TEXT NOT NULL DEFAULT 'unassigned'
|
||||
CHECK (status IN ('unassigned', 'claimed', 'completed', 'cancelled')),
|
||||
review_data JSONB NOT NULL DEFAULT '{}',
|
||||
claimed_at TIMESTAMPTZ,
|
||||
completed_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_instance
|
||||
ON workflow_assignments(instance_id, stage);
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_team_status
|
||||
ON workflow_assignments(team_id, status) WHERE status IN ('unassigned', 'claimed');
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_user
|
||||
ON workflow_assignments(assigned_to) WHERE assigned_to IS NOT NULL;
|
||||
```
|
||||
|
||||
**Claim lock:** Same optimistic pattern from chat-switchboard:
|
||||
`UPDATE ... WHERE id = $1 AND status = 'unassigned'` — if `rows_affected == 0`, already claimed.
|
||||
|
||||
### 3c. New migration file: `012_workflow_instances.sql`
|
||||
|
||||
Wait — per the principle, new *tables* get new files. But we're also editing 007. Decision:
|
||||
|
||||
**Put the new tables in 007_workflows.sql.** The schema isn't in production. Keep all workflow
|
||||
DDL in one file. When the schema *is* in production (post-MVP), new additions get numbered
|
||||
migrations. This is consistent with the existing design-decisions log entry:
|
||||
"No new migrations pre-MVP."
|
||||
|
||||
---
|
||||
|
||||
## 4. Stage Mode Redesign
|
||||
|
||||
### Current (trash/rename)
|
||||
|
||||
| Current Mode | Disposition | Rationale |
|
||||
|-------------|-------------|-----------|
|
||||
| `form_only` | ✏️ MOD → `form` | Rename. Drop the `_only` suffix — there's no `form_chat` to distinguish from anymore. |
|
||||
| `form_chat` | 🗑 TRASH | Chat coupling. If you want a form + AI conversation, use `delegated` with a chat surface package. |
|
||||
| `review` | ✅ KEEP | Team member reviews accumulated data. Pure kernel concept. |
|
||||
| `custom` | ✏️ MOD → `delegated` | Rename for clarity. "Custom" is vague. "Delegated" makes the contract explicit: the kernel delegates stage execution to `surface_pkg_id`. |
|
||||
|
||||
### New modes
|
||||
|
||||
| Mode | Disposition | Description |
|
||||
|------|-------------|-------------|
|
||||
| `form` | ✏️ MOD (renamed) | Kernel renders `form_template`, validates submission, merges into `stage_data`. No extension needed. |
|
||||
| `review` | ✅ KEEP | Assignment queue stage. Team member claims, reviews `stage_data`, adds `review_data`, completes. Kernel-rendered. |
|
||||
| `delegated` | ✏️ MOD (renamed) | Kernel hands off to `surface_pkg_id`. The surface package receives the instance context (stage_data, form_template, metadata) and calls back to advance. This is where chat, AI, or any custom UX lives. |
|
||||
| `automated` | ➕ ADD | No UI. On stage entry, kernel fires `starlark_hook`, merges returned data into `stage_data`, and auto-advances. For enrichment, API calls, validation, routing decisions. |
|
||||
|
||||
### Stage type vs stage mode
|
||||
|
||||
These are orthogonal:
|
||||
|
||||
- **`stage_type`** controls *how the next stage is chosen*: `simple` (declarative branch_rules), `dynamic` (Starlark hook returns target), `automated` (hook + auto-advance).
|
||||
- **`stage_mode`** controls *how the stage executes*: `form`, `review`, `delegated`, `automated`.
|
||||
|
||||
An `automated` stage_type with `automated` stage_mode is the common case for system stages,
|
||||
but you can have a `dynamic` stage_type with `form` stage_mode (the form collects data, then a
|
||||
Starlark hook decides where to go next based on the submission).
|
||||
|
||||
---
|
||||
|
||||
## 5. Model Changes
|
||||
|
||||
### `server/models/workflow.go`
|
||||
|
||||
#### WorkflowStage struct — ✏️ MOD
|
||||
|
||||
```go
|
||||
// TRASH: Remove these fields
|
||||
// PersonaID *string `json:"persona_id,omitempty"`
|
||||
// HistoryMode string `json:"history_mode"`
|
||||
|
||||
// MOD: Rename transition_rules → stage_config
|
||||
// TransitionRules json.RawMessage `json:"transition_rules"`
|
||||
StageConfig json.RawMessage `json:"stage_config"`
|
||||
|
||||
// ADD: New fields
|
||||
Audience string `json:"audience"` // team | public | system
|
||||
StageType string `json:"stage_type"` // simple | dynamic | automated
|
||||
StarlarkHook *string `json:"starlark_hook,omitempty"` // package_id:entry_point
|
||||
BranchRules json.RawMessage `json:"branch_rules"` // [{field, op, value, target_stage}]
|
||||
```
|
||||
|
||||
#### Stage mode constants — ✏️ MOD
|
||||
|
||||
```go
|
||||
// TRASH
|
||||
// StageModeFormOnly = "form_only"
|
||||
// StageModeFormChat = "form_chat"
|
||||
|
||||
// MOD (rename)
|
||||
StageModeForm = "form" // was form_only
|
||||
StageModeDelegated = "delegated" // was custom
|
||||
|
||||
// KEEP
|
||||
StageModeReview = "review"
|
||||
|
||||
// ADD
|
||||
StageModeAutomated = "automated"
|
||||
|
||||
// ADD: Stage type constants
|
||||
StageTypeSimple = "simple"
|
||||
StageTypeDynamic = "dynamic"
|
||||
StageTypeAutomated = "automated"
|
||||
|
||||
// ADD: Audience constants
|
||||
AudienceTeam = "team"
|
||||
AudiencePublic = "public"
|
||||
AudienceSystem = "system"
|
||||
```
|
||||
|
||||
#### ➕ ADD: WorkflowInstance model
|
||||
|
||||
```go
|
||||
type WorkflowInstance struct {
|
||||
ID string `json:"id"`
|
||||
WorkflowID string `json:"workflow_id"`
|
||||
WorkflowVersion int `json:"workflow_version"`
|
||||
CurrentStage int `json:"current_stage"`
|
||||
StageData json.RawMessage `json:"stage_data"`
|
||||
Status string `json:"status"` // active | completed | cancelled | stale | error
|
||||
StartedBy *string `json:"started_by,omitempty"`
|
||||
EntryToken *string `json:"entry_token,omitempty"`
|
||||
Metadata json.RawMessage `json:"metadata"`
|
||||
StageEnteredAt time.Time `json:"stage_entered_at"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
```
|
||||
|
||||
#### ➕ ADD: WorkflowAssignment model
|
||||
|
||||
```go
|
||||
type WorkflowAssignment struct {
|
||||
ID string `json:"id"`
|
||||
InstanceID string `json:"instance_id"`
|
||||
Stage int `json:"stage"`
|
||||
TeamID string `json:"team_id"`
|
||||
AssignedTo *string `json:"assigned_to,omitempty"`
|
||||
Status string `json:"status"` // unassigned | claimed | completed | cancelled
|
||||
ReviewData json.RawMessage `json:"review_data"`
|
||||
ClaimedAt *time.Time `json:"claimed_at,omitempty"`
|
||||
CompletedAt *time.Time `json:"completed_at,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
```
|
||||
|
||||
#### TypedFormTemplate, FormField, FormValidation — ✅ KEEP
|
||||
|
||||
All form types are clean kernel code. No changes.
|
||||
|
||||
#### ValidateFormData — ✅ KEEP
|
||||
|
||||
Pure data validation. No chat coupling.
|
||||
|
||||
---
|
||||
|
||||
## 6. Store Interface Changes
|
||||
|
||||
### `server/store/workflow_iface.go` — ✏️ MOD + ➕ ADD
|
||||
|
||||
Current interface is definition-only (CRUD workflows + stages + versions).
|
||||
Add instance and assignment operations.
|
||||
|
||||
```go
|
||||
type WorkflowStore interface {
|
||||
// ── Definition CRUD (KEEP — no changes) ──────────
|
||||
Create(ctx context.Context, w *models.Workflow) error
|
||||
GetByID(ctx context.Context, id string) (*models.Workflow, error)
|
||||
GetBySlug(ctx context.Context, teamID *string, slug string) (*models.Workflow, error)
|
||||
Update(ctx context.Context, id string, patch models.WorkflowPatch) error
|
||||
Delete(ctx context.Context, id string) error
|
||||
ListForTeam(ctx context.Context, teamID string) ([]models.Workflow, error)
|
||||
ListGlobal(ctx context.Context) ([]models.Workflow, error)
|
||||
|
||||
// ── Stage CRUD (KEEP — no changes) ───────────────
|
||||
CreateStage(ctx context.Context, s *models.WorkflowStage) error
|
||||
ListStages(ctx context.Context, workflowID string) ([]models.WorkflowStage, error)
|
||||
UpdateStage(ctx context.Context, s *models.WorkflowStage) error
|
||||
DeleteStage(ctx context.Context, id string) error
|
||||
ReorderStages(ctx context.Context, workflowID string, orderedIDs []string) error
|
||||
|
||||
// ── Versioning (KEEP — no changes) ───────────────
|
||||
Publish(ctx context.Context, v *models.WorkflowVersion) error
|
||||
GetVersion(ctx context.Context, workflowID string, versionNumber int) (*models.WorkflowVersion, error)
|
||||
GetLatestVersion(ctx context.Context, workflowID string) (*models.WorkflowVersion, error)
|
||||
|
||||
// ── Instances (ADD) ──────────────────────────────
|
||||
CreateInstance(ctx context.Context, inst *models.WorkflowInstance) error
|
||||
GetInstance(ctx context.Context, id string) (*models.WorkflowInstance, error)
|
||||
GetInstanceByToken(ctx context.Context, token string) (*models.WorkflowInstance, error)
|
||||
UpdateInstance(ctx context.Context, id string, patch models.InstancePatch) error
|
||||
ListInstances(ctx context.Context, workflowID string, status string) ([]models.WorkflowInstance, error)
|
||||
AdvanceStage(ctx context.Context, id string, nextStage int, mergeData json.RawMessage) error
|
||||
CompleteInstance(ctx context.Context, id string) error
|
||||
CancelInstance(ctx context.Context, id string) error
|
||||
MarkStale(ctx context.Context, olderThan time.Duration) (int, error)
|
||||
|
||||
// ── Assignments (ADD) ────────────────────────────
|
||||
CreateAssignment(ctx context.Context, a *models.WorkflowAssignment) error
|
||||
ClaimAssignment(ctx context.Context, id string, userID string) error // optimistic lock
|
||||
UnclaimAssignment(ctx context.Context, id string) error
|
||||
CompleteAssignment(ctx context.Context, id string, reviewData json.RawMessage) error
|
||||
CancelAssignment(ctx context.Context, id string) error
|
||||
ListAssignmentsByTeam(ctx context.Context, teamID string, status string) ([]models.WorkflowAssignment, error)
|
||||
ListAssignmentsByUser(ctx context.Context, userID string) ([]models.WorkflowAssignment, error)
|
||||
ListAssignmentsByInstance(ctx context.Context, instanceID string) ([]models.WorkflowAssignment, error)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Handler Changes
|
||||
|
||||
### `server/handlers/workflows.go` — ✏️ MOD
|
||||
|
||||
**Stage CRUD updates** — reflect new field names in CreateStage/UpdateStage:
|
||||
- Remove `persona_id` and `history_mode` from bind/validation.
|
||||
- Add `stage_type`, `starlark_hook`, `branch_rules` to bind/validation.
|
||||
- Rename `transition_rules` → `stage_config` in bind/validation.
|
||||
- Update `stage_mode` CHECK to new set: `form`, `review`, `delegated`, `automated`.
|
||||
- Validate: `delegated` requires `surface_pkg_id`. `dynamic`/`automated` stage_type requires `starlark_hook`.
|
||||
|
||||
### `server/handlers/workflow_hooks.go` — ✏️ MOD
|
||||
|
||||
**Rename:** `channelID` parameter → `instanceID` in `FireOnAdvanceHook` signature.
|
||||
The hook context dict changes from `{channel_id, ...}` to `{instance_id, ...}`.
|
||||
|
||||
The `jsonToStarlark` helper referenced on line 81 lives in
|
||||
`server/handlers/starlark_helpers.go`. Consider consolidating with `goToStarlark` in
|
||||
`server/triggers/event.go` — two independent Go→Starlark converters is tech debt.
|
||||
|
||||
### `server/handlers/workflow_team.go` — ✅ KEEP
|
||||
|
||||
Team-scoped wrappers. Clean delegation pattern. No changes.
|
||||
|
||||
### `server/handlers/workflow_packages.go` — ✏️ MOD
|
||||
|
||||
Update `workflowPkgStage` struct:
|
||||
- Remove `PersonaID`, `HistoryMode` fields.
|
||||
- Add `StageType`, `StarlarkHook`, `BranchRules` fields.
|
||||
- Rename `TransitionRules` → `StageConfig`.
|
||||
|
||||
Update `ExportWorkflowPackage` and `InstallWorkflowFromManifest` accordingly.
|
||||
|
||||
### ➕ ADD `server/handlers/workflow_instances.go`
|
||||
|
||||
New handler file for instance lifecycle:
|
||||
|
||||
```
|
||||
POST /api/v1/workflows/:id/start → Start (create instance from latest published version)
|
||||
GET /api/v1/workflows/instances/:iid → GetInstance
|
||||
POST /api/v1/workflows/instances/:iid/advance → Advance (submit stage data, resolve next)
|
||||
POST /api/v1/workflows/instances/:iid/cancel → Cancel
|
||||
GET /api/v1/workflows/:id/instances → ListInstances (by workflow, optionally by status)
|
||||
```
|
||||
|
||||
Public entry (for `public_link` workflows):
|
||||
```
|
||||
POST /api/v1/workflows/entry/:slug → StartPublic (no auth, generates entry_token)
|
||||
GET /api/v1/workflows/entry/:token → ResumePublic (retrieve instance by token)
|
||||
POST /api/v1/workflows/entry/:token/advance → AdvancePublic (submit data via token)
|
||||
```
|
||||
|
||||
### ➕ ADD `server/handlers/workflow_assignments.go`
|
||||
|
||||
New handler file for the assignment queue:
|
||||
|
||||
```
|
||||
GET /api/v1/teams/:teamId/assignments → ListTeamAssignments (filterable by status)
|
||||
POST /api/v1/teams/:teamId/assignments/:id/claim → Claim
|
||||
POST /api/v1/teams/:teamId/assignments/:id/unclaim → Unclaim
|
||||
POST /api/v1/teams/:teamId/assignments/:id/complete → Complete (with review_data)
|
||||
POST /api/v1/teams/:teamId/assignments/:id/cancel → Cancel
|
||||
GET /api/v1/assignments/mine → ListMyAssignments (across teams)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Routing Engine Changes
|
||||
|
||||
### `server/workflow/routing.go` — ✏️ MOD
|
||||
|
||||
The existing `ResolveNextStage` evaluates `transition_rules.conditions[]`. This needs to
|
||||
become a two-phase resolution that respects the new `stage_type`:
|
||||
|
||||
```
|
||||
Phase 1: Evaluate branch_rules (declarative, for simple + dynamic types)
|
||||
Phase 2: If no match AND stage_type == dynamic → fire starlark_hook
|
||||
Phase 3: Fallback → currentStage + 1
|
||||
```
|
||||
|
||||
For `automated` stage_type, the engine fires `starlark_hook` on entry (not for routing — for
|
||||
data enrichment), then routes via branch_rules or ordinal fallback. The routing call happens
|
||||
*after* the hook returns.
|
||||
|
||||
**Concrete changes:**
|
||||
- `ResolveNextStage` signature: add `stageType string, starlarkHook *string, runner *sandbox.Runner` parameters.
|
||||
- Extract branch_rules evaluation from the current `TransitionRulesWithConditions` (which reads from `transition_rules` JSON) into a dedicated function that reads from the new `branch_rules` field.
|
||||
- `TransitionRulesWithConditions` struct → rename to `StageConfig`, remove `Conditions` field (moved to branch_rules), keep `AutoAssign` and `OnAdvance`.
|
||||
|
||||
### `server/workflow/` — ➕ ADD `engine.go`
|
||||
|
||||
New file: the stage execution engine. Orchestrates the advance lifecycle:
|
||||
|
||||
```
|
||||
1. Load instance + version snapshot
|
||||
2. Validate current stage allows advancement (status checks)
|
||||
3. If current stage has form_template → validate submitted data
|
||||
4. Merge submitted data into stage_data
|
||||
5. Fire on_advance hook (if configured in stage_config)
|
||||
6. Resolve next stage (branch_rules → starlark_hook → ordinal)
|
||||
7. If next stage is past last stage → complete instance
|
||||
8. If next stage is automated → fire its hook, recurse to step 6
|
||||
9. If next stage has assignment_team_id → create WorkflowAssignment
|
||||
10. Update instance (current_stage, stage_data, stage_entered_at)
|
||||
11. Emit bus events (workflow.advanced, workflow.assigned, workflow.completed)
|
||||
```
|
||||
|
||||
### `server/workflow/` — ➕ ADD `automated.go`
|
||||
|
||||
Handles `automated` stage execution: fire the Starlark hook, merge returned data, and
|
||||
auto-advance. Includes a cycle guard (max 10 consecutive automated stages) to prevent
|
||||
infinite loops from misconfigured workflows.
|
||||
|
||||
---
|
||||
|
||||
## 9. Event Bus Updates
|
||||
|
||||
### `server/events/types.go` — ✏️ MOD
|
||||
|
||||
Current workflow events are fine but incomplete. Add:
|
||||
|
||||
| Event | Direction | Disposition | Trigger |
|
||||
|-------|-----------|-------------|---------|
|
||||
| `workflow.assigned` | DirToClient | ✅ KEEP | Assignment created |
|
||||
| `workflow.claimed` | DirToClient | ✅ KEEP | Assignment claimed |
|
||||
| `workflow.advanced` | DirToClient | ✅ KEEP | Stage transition |
|
||||
| `workflow.completed` | DirToClient | ✅ KEEP | Instance completed |
|
||||
| `workflow.cancelled` | DirToClient | ➕ ADD | Instance cancelled |
|
||||
| `workflow.started` | DirToClient | ➕ ADD | Instance created |
|
||||
| `workflow.sla.warning` | DirToClient | ➕ ADD | SLA at 80% threshold |
|
||||
| `workflow.sla.breached` | DirToClient | ➕ ADD | SLA exceeded |
|
||||
| `workflow.error` | DirLocal | ➕ ADD | Hook execution failure |
|
||||
|
||||
---
|
||||
|
||||
## 10. Starlark Module Updates
|
||||
|
||||
### `server/sandbox/workflow_module.go` — ✏️ MOD
|
||||
|
||||
The `BuildWorkflowModule` (in `workflow_module.go`, not `modules.go`) currently exists but
|
||||
targets the old channel-based model. It currently only exposes `get_definition`.
|
||||
Rebuild to expose instance-oriented operations:
|
||||
|
||||
```python
|
||||
# Available to extensions with workflow.read or workflow.write permission
|
||||
workflow.get_instance(instance_id) # → instance dict
|
||||
workflow.get_stage_data(instance_id) # → stage_data dict
|
||||
workflow.set_stage_data(instance_id, data) # → merge into stage_data
|
||||
workflow.advance(instance_id) # → trigger advance (write perm)
|
||||
workflow.get_assignments(instance_id) # → assignment list
|
||||
```
|
||||
|
||||
The old `workflow_advance()` function that chat-switchboard's personas called is gone.
|
||||
Extensions call `workflow.advance()` which delegates to the same engine as the HTTP handler.
|
||||
|
||||
---
|
||||
|
||||
## 11. Salvage Map from chat-switchboard v0.39.x
|
||||
|
||||
What was planned there, and what happens to each idea in core:
|
||||
|
||||
### v0.39.0 — Stage Graph Engine + Form Builder
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| `stage_type` enum | ➕ ADD | Adopted directly: `simple`, `dynamic`, `automated` |
|
||||
| `BranchRule` struct | ➕ ADD | Adopted as `branch_rules` JSONB on `workflow_stages` |
|
||||
| Starlark hook integration | ➕ ADD | `starlark_hook` column + engine integration |
|
||||
| Graph engine (`server/engine/graph.go`) | ➕ ADD | `server/workflow/engine.go` in core |
|
||||
| Automated stage runner | ➕ ADD | `server/workflow/automated.go` in core |
|
||||
| Visual form builder (5 Preact components) | 🗑 TRASH *for now* | The form builder was tightly coupled to chat-switchboard's stage editor surface. Core doesn't ship a workflow editor surface (that's an extension). The form *schema* is kernel; the form *builder UI* is a surface package concern. Revisit when a workflow-admin surface is built. |
|
||||
| Stage type badges in UI | 🗑 TRASH | Same — UI is extension territory. |
|
||||
|
||||
### v0.39.1 — Stage Reorder + Bulk Ops
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Drag-and-drop reorder | 🗑 TRASH | UI concern. The kernel `ReorderStages` endpoint already exists. |
|
||||
| Bulk assignment ops | ➕ ADD (later) | Useful but not blocking. Defer to v0.2.7+. The individual claim/unclaim/cancel endpoints come first. |
|
||||
|
||||
### v0.39.2 — SLA Alerting
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Periodic SLA scanner | ➕ ADD | Kernel scheduled goroutine (like the existing staleness sweep). Query active instances with `sla_seconds` configured, compute breach from `stage_entered_at`. |
|
||||
| `sla_notified_at` on instances | ➕ ADD | Add to `workflow_instances` schema. Prevents duplicate notifications. |
|
||||
| Notifications on breach | ➕ ADD | Use existing `notifications` table + bus event (`workflow.sla.breached`). |
|
||||
| Webhook on breach | ➕ ADD | Fire workflow's `webhook_url` if configured. |
|
||||
| Pulsing badge animation | 🗑 TRASH | UI concern for a surface package. |
|
||||
|
||||
### v0.39.3 — Visitor Portal
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Public entry endpoint | ➕ ADD | `/api/v1/workflows/entry/:slug` — kernel handles anonymous session creation + token issuance. |
|
||||
| Session resume via token | ➕ ADD | `entry_token` on `workflow_instances` + `GetInstanceByToken` store method. |
|
||||
| Branded entry page | 🗑 TRASH | Surface package. The kernel stores `branding` JSONB — a portal surface reads it. |
|
||||
| Progress indicator | 🗑 TRASH | Surface package reads stage list, computes "step N of M" locally. |
|
||||
| Email notifications | 🗑 TRASH *for now* | Requires SMTP infrastructure. Defer to post-MVP or make it a connection-based extension (use ext_connections for SMTP config, fire via Starlark hook). |
|
||||
|
||||
### v0.39.4 — Templates + Clone
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Clone endpoint | ➕ ADD | `POST /api/v1/workflows/:id/clone` — deep copy workflow + stages. Straightforward. |
|
||||
| Built-in templates table | 🗑 TRASH | Core doesn't ship opinionated templates. Templates are workflow packages (`.pkg` archives). A "template gallery" is a surface. |
|
||||
| Template picker UI | 🗑 TRASH | Surface concern. |
|
||||
|
||||
### v0.39.5 — Analytics
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Analytics query endpoint | ➕ ADD (later) | `GET /api/v1/workflows/:id/analytics?period=7d` — kernel exposes raw metrics (throughput, cycle time, stage dwell, SLA compliance) derived from instance + assignment timestamps. Defer to v0.2.8+ or v0.3.x. |
|
||||
| Analytics dashboard UI | 🗑 TRASH | Surface package. |
|
||||
|
||||
---
|
||||
|
||||
## 12. Implementation Order
|
||||
|
||||
All items within v0.2.6. Ordered by dependency:
|
||||
|
||||
### Phase A: Schema + Models
|
||||
|
||||
1. ✏️ Edit `007_workflows.sql`: drop `persona_id`, `history_mode`; add `audience`, `stage_type`, `starlark_hook`, `branch_rules`; rename `transition_rules` → `stage_config`; update `stage_mode` CHECK.
|
||||
2. ✏️ Edit `007_workflows.sql` (same file): add `workflow_instances` and `workflow_assignments` tables.
|
||||
3. ✏️ Edit `007_workflows.sql` (SQLite dialect): mirror changes.
|
||||
4. ✏️ Update `server/models/workflow.go`: remove trashed fields, add new structs, update constants.
|
||||
|
||||
### Phase B: Store Layer
|
||||
|
||||
5. ✏️ Update `server/store/workflow_iface.go`: add instance + assignment methods.
|
||||
6. ➕ Implement Postgres store: `server/store/postgres/workflow_instances.go`, `workflow_assignments.go`.
|
||||
7. ➕ Implement SQLite store: `server/store/sqlite/workflow_instances.go`, `workflow_assignments.go`.
|
||||
|
||||
### Phase C: Engine
|
||||
|
||||
8. ✏️ Update `server/workflow/routing.go`: branch_rules evaluation, starlark_hook integration.
|
||||
9. ➕ Add `server/workflow/engine.go`: advance lifecycle orchestration.
|
||||
10. ➕ Add `server/workflow/automated.go`: automated stage execution + cycle guard.
|
||||
11. ✏️ Update `server/handlers/workflow_hooks.go`: rename channel → instance references.
|
||||
|
||||
### Phase D: Handlers + Routes
|
||||
|
||||
12. ✏️ Update `server/handlers/workflows.go`: stage CRUD field changes.
|
||||
13. ✏️ Update `server/handlers/workflow_packages.go`: package struct field changes.
|
||||
14. ➕ Add `server/handlers/workflow_instances.go`: instance lifecycle endpoints.
|
||||
15. ➕ Add `server/handlers/workflow_assignments.go`: assignment queue endpoints.
|
||||
16. ✏️ Wire new routes in `server/main.go`.
|
||||
|
||||
### Phase E: Events + Starlark
|
||||
|
||||
17. ✏️ Update `server/events/types.go`: add new workflow events.
|
||||
18. ✏️ Update `server/sandbox/modules.go`: rebuild workflow Starlark module.
|
||||
|
||||
### Phase F: Background Jobs
|
||||
|
||||
19. ➕ Add SLA scanner goroutine (check active instances, fire notifications + events on breach).
|
||||
20. ➕ Add staleness sweep for workflow instances (reuse pattern from chat-switchboard).
|
||||
|
||||
### Phase G: Verification
|
||||
|
||||
21. ➕ Integration tests: instance lifecycle, stage advancement, branching, automated stages, assignments, SLA breach.
|
||||
22. `go build ./...` clean.
|
||||
23. Update ICD (OpenAPI spec) with new endpoints.
|
||||
24. Update ROADMAP.md + CHANGELOG.md.
|
||||
|
||||
---
|
||||
|
||||
## 13. What We're NOT Doing (Explicit Deferrals)
|
||||
|
||||
| Item | Why | Revisit |
|
||||
|------|-----|---------|
|
||||
| Visual form builder UI | Surface concern, no workflow-admin surface yet | When a workflow-admin surface ships |
|
||||
| Drag-and-drop stage reorder UI | Surface concern | Same |
|
||||
| Email notifications | Needs SMTP infrastructure or connection-based ext | Post-MVP |
|
||||
| Built-in workflow templates | Core doesn't ship opinionated content | Package gallery |
|
||||
| Analytics dashboard | Nice-to-have, kernel endpoint can come first | v0.3.x |
|
||||
| Bulk assignment operations | Individual ops first | v0.2.7+ |
|
||||
| Workflow chaining (`on_complete`) | Already in schema, needs engine wiring | v0.2.7 |
|
||||
| Visitor portal surface | Surface package, not kernel | Extension track |
|
||||
|
||||
---
|
||||
|
||||
## 14. Migration Policy Reminder
|
||||
|
||||
From the design-decisions log:
|
||||
|
||||
> **No new migrations pre-MVP.** Edit existing migration SQL files in place.
|
||||
> No migration chains until schema is in production.
|
||||
|
||||
For this redesign:
|
||||
- **`007_workflows.sql`**: Edit in place for column changes AND add new tables to the same file.
|
||||
- **No `012_*.sql`**: Everything goes in 007.
|
||||
- **Both dialects**: Postgres and SQLite must be kept in sync.
|
||||
|
||||
The only time a new numbered migration file is created is when a truly new,
|
||||
unrelated subsystem is added (like 011_triggers.sql was for the trigger system).
|
||||
Workflow instances and assignments are part of the workflow subsystem → they go in 007.
|
||||
|
||||
---
|
||||
|
||||
## 15. Public Surfaces — Kernel Capability, Not Workflow Feature
|
||||
|
||||
### The problem with workflow-owned public access
|
||||
|
||||
Chat-switchboard treated public access as a binary workflow toggle: `entry_mode: public_link`
|
||||
made the whole workflow public. But real workflows alternate audiences — internal setup →
|
||||
public-facing customer form → internal review → public confirmation page. And public access
|
||||
isn't even a workflow-only need: feedback forms, status pages, and surveys all need anonymous
|
||||
sessions without being workflows.
|
||||
|
||||
### Per-stage audience (the v0.2.6 solution)
|
||||
|
||||
The `audience` field on `workflow_stages` is the source of truth:
|
||||
|
||||
| Audience | Who interacts | Auth required | Example |
|
||||
|----------|--------------|---------------|---------|
|
||||
| `team` | Authenticated team members | JWT | Internal triage, setup, review |
|
||||
| `public` | Anonymous visitors | Entry token | Customer intake form, confirmation page |
|
||||
| `system` | Nobody (automated) | N/A | API enrichment, routing decision |
|
||||
|
||||
A single workflow freely alternates:
|
||||
|
||||
```
|
||||
Stage 0: "Setup" audience=team (agent configures parameters)
|
||||
Stage 1: "Customer Form" audience=public (customer fills out intake)
|
||||
Stage 2: "Internal Review" audience=team (team reviews submission)
|
||||
Stage 3: "Confirmation" audience=public (customer sees result)
|
||||
```
|
||||
|
||||
The engine enforces boundaries:
|
||||
- When the instance enters a `public` stage, the visitor's token grants access. Team members
|
||||
can also see it (they have elevated access).
|
||||
- When the instance enters a `team` stage, the visitor's view freezes. They see "being
|
||||
processed" or whatever the portal surface shows. The token is still valid for resumption
|
||||
when the next `public` stage arrives.
|
||||
- When the instance enters a `system` stage, nobody interacts — the automated hook fires
|
||||
and the engine advances.
|
||||
|
||||
### The `entry_mode` column
|
||||
|
||||
Stays on the `workflows` table but becomes a derived convenience flag. If any stage has
|
||||
`audience = 'public'`, the workflow is public-entry eligible. The kernel can auto-compute
|
||||
this when stages are saved, or treat it as an explicit admin toggle that gates whether
|
||||
public stages are actually reachable. Either way, it's no longer the primary access control
|
||||
mechanism — `audience` on each stage is.
|
||||
|
||||
### Platform-level public access (v0.2.7+ concern)
|
||||
|
||||
The broader vision: public access as a **package-level manifest capability** so any surface
|
||||
(not just workflows) can serve anonymous visitors. A package declares `"public": true` in
|
||||
its manifest, and the kernel mounts a public route namespace (`/p/:package_slug/*`) with
|
||||
anonymous session middleware, token issuance, and rate limiting. Workflows consume this
|
||||
capability — they don't own it.
|
||||
|
||||
For v0.2.6, the workflow-specific public entry routes are sufficient:
|
||||
```
|
||||
POST /api/v1/workflows/entry/:slug → StartPublic
|
||||
GET /api/v1/workflows/entry/:token → ResumePublic
|
||||
POST /api/v1/workflows/entry/:token/advance → AdvancePublic
|
||||
```
|
||||
|
||||
The anonymous session management (entry_token on `workflow_instances`, IP rate limiting)
|
||||
lives in the workflow handler for now. Generalization to a platform-level `public_sessions`
|
||||
table happens when non-workflow surfaces need it.
|
||||
|
||||
### Visitor view contract
|
||||
|
||||
When a public stage is active, the kernel API returns only what the visitor should see:
|
||||
- The current stage's `form_template` (if mode is `form`)
|
||||
- The stage name and ordinal (for progress indicators)
|
||||
- Accumulated `stage_data` keys marked as `visitor_visible` (future: field-level ACL)
|
||||
- The `branding` JSONB from the workflow
|
||||
|
||||
Internal stages, team assignments, review data, and other instances' data are never
|
||||
exposed through the token-authenticated endpoints. The surface package renders whatever
|
||||
the kernel returns — it doesn't need to filter.
|
||||
|
||||
---
|
||||
|
||||
## 16. Branch Rules — Routing, Skipping, and Convergence
|
||||
|
||||
### How branch_rules work
|
||||
|
||||
Each stage has a `branch_rules` JSONB array. On stage completion, the engine evaluates rules
|
||||
top-to-bottom against accumulated `stage_data`. First match wins. No match → ordinal + 1.
|
||||
|
||||
```json
|
||||
[
|
||||
{"field": "priority", "op": "eq", "value": "urgent", "target_stage": "Emergency Review"},
|
||||
{"field": "amount", "op": "gt", "value": 10000, "target_stage": "Manager Approval"},
|
||||
{"field": "type", "op": "in", "value": ["a","b"], "target_stage": "Fast Track"}
|
||||
]
|
||||
```
|
||||
|
||||
`target_stage` accepts either a **stage name** (case-insensitive) or a **numeric ordinal**.
|
||||
The engine tries name resolution first, falls back to numeric. This is already implemented
|
||||
in `routing.go` → `resolveTarget()`.
|
||||
|
||||
### Skipping stages
|
||||
|
||||
Branch rules can target any stage, not just the next one. If a workflow has stages
|
||||
0→1→2→3→4 and stage 1's rules say `target_stage: "Stage 4"`, stages 2 and 3 are skipped
|
||||
for that instance. The engine doesn't care about gaps — it sets `current_stage` to whatever
|
||||
the rule resolved to.
|
||||
|
||||
Linear workflows (no branch_rules on any stage) advance 0→1→2→3→4 by the ordinal + 1
|
||||
fallback. Adding a single branch rule to any stage introduces conditional skipping without
|
||||
changing the linear stages.
|
||||
|
||||
### Convergence (diamond patterns)
|
||||
|
||||
Two different branches can target the same downstream stage:
|
||||
|
||||
```
|
||||
Stage 0: Intake Form
|
||||
branch_rules: [
|
||||
{field: "region", op: "eq", value: "US", target_stage: "US Processing"},
|
||||
{field: "region", op: "eq", value: "EU", target_stage: "EU Processing"}
|
||||
]
|
||||
|
||||
Stage 1: US Processing (ordinal 1)
|
||||
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
|
||||
(always converge after processing)
|
||||
|
||||
Stage 2: EU Processing (ordinal 2)
|
||||
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
|
||||
(always converge after processing)
|
||||
|
||||
Stage 3: Final Review (ordinal 3)
|
||||
(both branches land here)
|
||||
```
|
||||
|
||||
No special syntax needed. The routing engine doesn't track "which branch am I on" — it just
|
||||
resolves the target and moves the instance there. The accumulated `stage_data` carries
|
||||
everything downstream stages need to know about what happened upstream.
|
||||
|
||||
### Backward jumps
|
||||
|
||||
Branch rules can also target earlier stages (lower ordinals) for retry/correction loops.
|
||||
The cycle guard (max 10 consecutive automated stages) in `automated.go` prevents infinite
|
||||
loops for automated stages. For human-facing stages, backward jumps are intentional — the
|
||||
human breaks the cycle by submitting different data.
|
||||
|
||||
### Operators
|
||||
|
||||
Full set, inherited from the existing `routing.go` implementation:
|
||||
|
||||
| Op | Description | Example |
|
||||
|----|-------------|---------|
|
||||
| `eq` | Loose equality (string comparison) | `"status" eq "approved"` |
|
||||
| `neq` | Not equal | `"status" neq "rejected"` |
|
||||
| `gt` | Greater than (numeric) | `"amount" gt 10000` |
|
||||
| `lt` | Less than (numeric) | `"amount" lt 100` |
|
||||
| `gte` | Greater or equal | `"score" gte 80` |
|
||||
| `lte` | Less or equal | `"score" lte 20` |
|
||||
| `in` | Value in array | `"category" in ["a","b","c"]` |
|
||||
| `contains` | Substring match | `"notes" contains "urgent"` |
|
||||
| `exists` | Field present in stage_data | `"approval_date" exists` |
|
||||
| `not_exists` | Field absent | `"rejection_reason" not_exists` |
|
||||
Reference in New Issue
Block a user