Changeset 0.35.0 (#209)

Co-authored-by: Jeffrey Smith <jasafpro@gmail.com>
Co-committed-by: Jeffrey Smith <jasafpro@gmail.com>
This commit is contained in:
2026-03-20 09:59:53 +00:00
committed by xcaliber
parent d16bb93177
commit bf8082e69f
37 changed files with 2324 additions and 129 deletions

148
docs/DESIGN-0.35.0.md Normal file
View File

@@ -0,0 +1,148 @@
# DESIGN-0.35.0 — Workflow Product
Transforms the workflow engine from a linear stage-runner into a
product-grade automation tool. Seven changesets covering conditional
routing, AI-triggered routing, data pipeline hooks, progressive forms,
conditional fields, structured review, and a monitoring dashboard.
Depends on: v0.34.0 (Data Portability), v0.31.2 (Team Workflows),
v0.29.0 (Starlark Sandbox).
## Schema Changes
Migrations modified in-place (DB rebuild required):
**005_channels.sql** — add `stage_entered_at` (PG: `TIMESTAMPTZ`,
SQLite: `TEXT`). Tracks when the current stage was entered for SLA
computation. Also add index `idx_channels_workflow_active` on
`(workflow_id, workflow_status)` for monitoring queries.
**018_workflows.sql** — add `sla_seconds INTEGER` to `workflow_stages`.
Add `review_comments` (PG: `JSONB DEFAULT '[]'`, SQLite: `TEXT
DEFAULT '[]'`) to `workflow_assignments`.
No new migration files. No new tables.
## Conditional Routing Engine
**File:** `server/workflow/routing.go`
The existing advance logic hardcoded `nextStage = currentStage + 1`.
The routing engine replaces this with condition evaluation:
1. Parse `transition_rules.conditions[]` from the departing stage
2. Evaluate each condition against accumulated `stage_data`
3. First match → resolve `target_stage` (by name or ordinal)
4. No match → fallback to `ordinal + 1`
**Expression format:** `{field, op, value, target_stage}`. Operators:
`eq`, `neq`, `gt`, `lt`, `gte`, `lte`, `in`, `contains`, `exists`,
`not_exists`. Loose equality via `fmt.Sprintf("%v")` comparison.
Numeric comparison extracts `float64` from both sides.
**Wiring:** `ResolveNextStage()` called in all 4 advance sites:
handler (`workflow_instances.go`), tool (`tools/workflow.go`), forms
auto-advance (`workflow_forms.go`), Starlark module
(`sandbox/workflow_module.go`).
**Backward compatible:** no conditions = `ordinal + 1`.
## AI-Triggered Routing
**Tool:** `workflow_route` — persona calls with `{target_stage, reason}`.
Uses `ResolveStageByName()` (case-insensitive). Records decision in
`stage_data._route_history_latest`. Allows forward and backward jumps
(loop stages for iterative correction).
**Starlark:** `workflow.route(channel_id, target_stage, reason)` for
extension-driven escalation patterns. Requires `workflow.access`.
## on_advance Hook
**File:** `server/handlers/workflow_hooks.go`
Fires synchronously after `AdvanceWorkflowStage` succeeds. Configured
per-stage in `transition_rules.on_advance: {package_id, entry_point}`.
Hook receives: `{stage_data, previous_stage, current_stage, channel_id}`.
Returns: `{stage_data: {...}}` (enriched), `None` (no change), or
`{error: "msg"}` (logged, not rolled back — future: rollback support).
The existing `http` Starlark module (v0.29.1) is available in hooks,
enabling external API enrichment without new infrastructure.
## Progressive Forms
**Model:** `TypedFormTemplate.Fieldsets []FormFieldset` — optional
array of `{label, fields[]}`. When present, top-level `fields` is
replaced by flattened fieldset fields in `ParseTypedFormTemplate()`.
**Frontend:** `workflow.html` renders one fieldset at a time with
step indicator, next/back navigation. All fields submitted together
on the final step.
## Conditional Fields
**Model:** `FormField.Condition *FieldCondition``{when, op, value}`.
Operators: `eq` (default), `neq`, `in`, `exists`.
**Server-side:** `ValidateFormData()` calls `evaluateFieldCondition()`
before each field. Hidden fields are skipped (no required check, no
type validation).
**Client-side:** `workflow.html` attaches `change`/`input` listeners
to source fields. Target fields toggle `cond-hidden` CSS class.
## Structured Review
**Review comments:** `review_comments JSONB` on `workflow_assignments`.
`POST /workflow-assignments/:id/comment` appends `{text, user_id,
created_at}`. `GET /workflow-assignments/:id` returns full assignment.
**Review surface:** Side-by-side layout in `workflow.html` — left panel
shows structured `stage_data` table (internal `_`-prefixed keys hidden),
right panel has comment textarea + approve/reject buttons. Keyboard
shortcuts: `Ctrl+Enter` approve, `Ctrl+Shift+Enter` reject.
## Monitoring Dashboard
**File:** `server/handlers/workflow_monitor.go`
**Endpoints:**
- `GET /admin/workflows/monitor/instances` — all active instances
- `GET /admin/workflows/monitor/funnel/:id` — stage counts
- `GET /admin/workflows/monitor/stale?threshold_hours=N`
- `GET /teams/:teamId/workflows/monitor/instances` — team-scoped
**SLA computation:** `stage_entered_at + sla_seconds` vs `NOW()`.
Returned as `sla_remaining_seconds` (negative = breached) and
`sla_breached` boolean. Computed at query time, not stored.
**Store:** New `ListByType(ctx, "workflow")` method on `ChannelStore`
(both PG and SQLite). Monitoring queries iterate active channels,
joining workflow definitions and stage metadata.
**Frontend:** `src/js/workflow-monitor.js` — auto-refresh every 30s,
SLA color indicators (green > 50%, yellow > 20%, red), stale instance
alert section.
## Workflow Branding
`WorkflowPageData.BrandingJSON` passes the workflow's `branding` JSON
to the visitor template. Applied as:
- `--accent` CSS custom property (from `accent_color`)
- Logo image in header (from `logo_url`)
- Tagline text below header (from `tagline`)
Previously branding was only on the landing page; now also on the
active workflow surface.
## New Files
| File | Purpose |
|------|---------|
| `server/workflow/routing.go` | Conditional routing engine |
| `server/handlers/workflow_hooks.go` | on_advance hook dispatch |
| `server/handlers/workflow_monitor.go` | Monitoring dashboard |
| `src/js/workflow-monitor.js` | Admin monitoring frontend |
| `packages/icd-test-runner/js/crud/workflow-product.js` | E2E tests |

View File

@@ -115,6 +115,7 @@ how data is collected.
| `chat_only` | Default. Persona drives the conversation. |
| `form_only` | UI-rendered form, no LLM. Visitor fills out fields. |
| `form_chat` | Both form and chat available. Persona assists alongside the form. |
| `review` | Human review queue. Shows structured stage_data with approve/reject. |
### List Stages
@@ -138,11 +139,12 @@ POST /workflows/:id/stages
"ordinal": 0,
"persona_id": "uuid|null",
"assignment_team_id": "uuid|null",
"stage_mode": "chat_only|form_only|form_chat",
"stage_mode": "chat_only|form_only|form_chat|review",
"form_template": { "fields": [...] },
"history_mode": "full|summary|fresh",
"auto_transition": false,
"transition_rules": { "auto_assign": "round_robin" }
"transition_rules": { "auto_assign": "round_robin", "conditions": [...] },
"sla_seconds": 3600
}
```
@@ -159,6 +161,12 @@ freeform templates are still supported as guidance for the LLM.
`transition_rules.auto_assign`: `round_robin` assigns to team members
in rotation. Null = unassigned (manual claim).
`transition_rules.conditions`: array of routing conditions (v0.35.0).
See [Conditional Routing](#conditional-routing-v035).
`sla_seconds`: optional SLA budget in seconds for this stage. Used by
the monitoring dashboard for deadline tracking. Null = no SLA.
**Auth:** `workflow.create` permission required.
### Typed Form Template (v0.29.3)
@@ -172,7 +180,8 @@ in rotation. Null = unassigned (manual claim).
{ "value": "eng", "label": "Engineering" },
{ "value": "sales", "label": "Sales" }
]},
{ "key": "notes", "type": "textarea", "label": "Additional Notes" }
{ "key": "notes", "type": "textarea", "label": "Additional Notes",
"condition": { "when": "dept", "op": "eq", "value": "eng" } }
],
"hooks": {
"package_id": "uuid",
@@ -190,6 +199,41 @@ in rotation. Null = unassigned (manual claim).
- `min`, `max` — numeric bounds for number fields
- `min_date`, `max_date` — date range bounds (YYYY-MM-DD format)
**Conditional fields (v0.35.0):** A field can have a `condition` object
that controls visibility. When the condition is not met, the field is
hidden on the client and skipped during server-side validation.
```json
{ "when": "field_key", "op": "eq|neq|in|exists", "value": "..." }
```
- `eq` (default): field is shown when the referenced field equals `value`
- `neq`: shown when not equal
- `in`: shown when the referenced field's value is in the `value` array
- `exists`: shown when the referenced field has any non-empty value
**Progressive forms / fieldsets (v0.35.0):** For multi-step forms within
a single stage, use `fieldsets` instead of (or alongside) `fields`:
```json
{
"fieldsets": [
{ "label": "Contact Info", "fields": [
{ "key": "name", "type": "text", "label": "Name", "required": true },
{ "key": "email", "type": "email", "label": "Email", "required": true }
]},
{ "label": "Details", "fields": [
{ "key": "description", "type": "textarea", "label": "Description" }
]}
]
}
```
When `fieldsets` is present, the visitor sees one fieldset at a time with
next/back navigation. All fields from all fieldsets are submitted together.
On the server, `ParseTypedFormTemplate` flattens fieldsets into the
`fields` array for validation — backward compatible with existing logic.
**Hooks:** optional Starlark hooks via extension packages.
- `validate`: called before form submission is accepted; can return field errors
- `on_submit`: fire-and-forget hook after successful submission
@@ -243,6 +287,8 @@ Sets ordinals based on array position.
"history_mode": "full",
"auto_transition": false,
"transition_rules": {},
"surface_pkg_id": "uuid|null",
"sla_seconds": 3600,
"created_at": "..."
}
```
@@ -315,7 +361,8 @@ GET /channels/:id/workflow/status
"current_stage": 1,
"stage_data": { "name": "Jane", "email": "jane@co.com" },
"status": "active|completed|stale",
"last_activity_at": "..."
"last_activity_at": "...",
"stage_entered_at": "..."
}
```
@@ -331,12 +378,17 @@ POST /channels/:id/workflow/advance
{ "data": { "name": "Jane", "email": "jane@co.com" } }
```
Merges `data` into the channel's accumulated stage data and moves to
the next stage. If the current stage is the last, marks the workflow
as `completed` and sets `ai_mode=off`.
Merges `data` into the channel's accumulated stage data. The next stage
is determined by the **conditional routing engine** (v0.35.0):
`transition_rules.conditions` are evaluated against the merged stage
data. First matching condition selects the target stage; if no conditions
match or none are defined, advances to `ordinal + 1`. If the target
is beyond the last stage, marks the workflow as `completed` and sets
`ai_mode=off`.
On completion, triggers `on_complete` chaining and webhook delivery
if configured.
After a successful transition, the `on_advance` hook fires if configured
(see below). On completion, triggers `on_complete` chaining and webhook
delivery if configured.
**Auth:** Authenticated.
@@ -412,6 +464,31 @@ Valid values: `unassigned`, `claimed`, `completed`.
**Auth:** Authenticated (team member).
### Get Assignment (v0.35.0)
```
GET /workflow-assignments/:id
```
Returns the full assignment including review comments.
**Auth:** Authenticated.
### Add Review Comment (v0.35.0)
```
POST /workflow-assignments/:id/comment
```
```json
{ "text": "Looks good, approved with minor note about address." }
```
Appends a comment to the assignment's `review_comments` array.
Comments are visible to the next stage when `history_mode=full`.
**Auth:** Authenticated.
### Assignment Object
```json
@@ -422,6 +499,9 @@ Valid values: `unassigned`, `claimed`, `completed`.
"team_id": "uuid",
"assigned_to": "uuid|null",
"status": "unassigned|claimed|completed",
"review_comments": [
{ "text": "...", "user_id": "uuid", "created_at": "..." }
],
"created_at": "...",
"claimed_at": "...|null",
"completed_at": "...|null"
@@ -569,6 +649,161 @@ The system starts a new instance of the target workflow, passing mapped
stage data as initial context. Webhook delivery (if configured) fires
before chaining.
## Conditional Routing (v0.35)
The routing engine evaluates `transition_rules.conditions` on the
departing stage against accumulated `stage_data`. Conditions are
evaluated in order; first match wins.
### Condition Object
```json
{
"conditions": [
{ "field": "category", "op": "eq", "value": "billing", "target_stage": "Billing Review" },
{ "field": "priority", "op": "gte", "value": 5, "target_stage": "Urgent" }
]
}
```
| Field | Type | Description |
|-------|------|-------------|
| `field` | string | Key in `stage_data` to evaluate |
| `op` | string | Operator (see below) |
| `value` | any | Value to compare against |
| `target_stage` | string | Stage name or ordinal to route to |
**Operators:** `eq`, `neq`, `gt`, `lt`, `gte`, `lte`, `in`, `contains`, `exists`, `not_exists`.
`target_stage` resolves by name (case-insensitive) first, then as
a numeric ordinal. Backward jumps are allowed (loop stages).
If no condition matches, falls back to `ordinal + 1`.
### workflow_route Tool (v0.35.0)
Persona-callable tool for AI-triggered routing. Available only in
workflow channels (`RequireWorkflow` predicate).
```json
{
"name": "workflow_route",
"parameters": {
"target_stage": "Stage name to route to",
"reason": "Explanation for the routing decision"
}
}
```
Resolves `target_stage` by name. Records the routing decision in
`stage_data._route_history_latest` with `from`, `to`, `reason`, `ts`.
### Starlark workflow.route()
```python
workflow.route(channel_id, target_stage, reason)
```
Extension-callable routing for programmatic escalation patterns.
Requires `workflow.access` permission.
## on_advance Hook (v0.35.0)
Starlark entry point that fires synchronously after a stage transition.
Configured per-stage in `transition_rules`:
```json
{
"on_advance": {
"package_id": "uuid",
"entry_point": "on_advance"
}
}
```
The hook receives a context dict:
```python
def on_advance(ctx):
# ctx["channel_id"] - workflow channel ID
# ctx["previous_stage"] - ordinal of the departed stage
# ctx["current_stage"] - ordinal of the new stage
# ctx["stage_data"] - accumulated stage data dict
return {"stage_data": ctx["stage_data"]} # enriched
# return None — no change
# return {"error": "msg"} — reject (logged, not rolled back)
```
Use cases: external API enrichment via `http.fetch`, cross-field
validation, data transformation between stages.
## Monitoring Dashboard (v0.35.0)
Admin endpoints for tracking active workflow instances and SLA status.
### List Active Instances
```
GET /admin/workflows/monitor/instances
GET /teams/:teamId/workflows/monitor/instances
```
Returns all active (`workflow_status=active`) instances with SLA info.
```json
{
"data": [{
"channel_id": "uuid",
"channel_title": "...",
"workflow_id": "uuid",
"workflow_name": "...",
"current_stage": 1,
"stage_name": "Review",
"age_seconds": 3600,
"stage_age_seconds": 1200,
"sla_seconds": 7200,
"sla_remaining_seconds": 6000,
"sla_breached": false,
"last_activity_at": "..."
}]
}
```
SLA is computed as `stage_entered_at + sla_seconds` vs current time.
**Auth:** Admin (global) or team member (team-scoped).
### Stage Funnel
```
GET /admin/workflows/monitor/funnel/:id
```
Returns per-stage instance counts for bottleneck detection.
```json
{
"data": [
{ "stage_ordinal": 0, "stage_name": "Intake", "count": 5 },
{ "stage_ordinal": 1, "stage_name": "Review", "count": 12 },
{ "stage_ordinal": 2, "stage_name": "Complete", "count": 2 }
]
}
```
**Auth:** Admin.
### Stale Instances
```
GET /admin/workflows/monitor/stale?threshold_hours=48
```
Returns instances where `last_activity_at` exceeds the threshold.
Default threshold: 48 hours.
**Auth:** Admin.
## WebSocket Events
| Event | When |

View File

@@ -436,48 +436,48 @@ structured data — not chat transcripts.
Depends on: v0.31.2 (team workflow self-service), v0.29.0 (Starlark sandbox).
**Form Rendering Surface:**
- [ ] Visitor form renderer — reads `form_template` from stage, renders
- [x] Visitor form renderer — reads `form_template` from stage, renders
HTML inputs (text, email, select, date, textarea, checkbox, file),
validates client-side, submits via `/w/:id/form-submit`
- [ ] Progressive form — multi-step within a single stage (fieldsets)
- [ ] Conditional fields — show/hide based on previous answers
- [ ] File upload in forms — attach to stage_data via storage API
- [ ] Branded form page — uses workflow `branding` (colors, logo, tagline)
- [x] Progressive form — multi-step within a single stage (fieldsets)
- [x] Conditional fields — show/hide based on previous answers
- [x] File upload in forms — attach to stage_data via storage API
- [x] Branded form page — uses workflow `branding` (colors, logo, tagline)
**Data Pipeline (between-stage processing):**
- [ ] `on_advance` hook — Starlark entry point fires after each stage
- [x] `on_advance` hook — Starlark entry point fires after each stage
transition, receives `stage_data`, can enrich/transform/reject
- [ ] External data enrichment — Starlark `http.fetch` pulls from
- [x] External data enrichment — Starlark `http.fetch` pulls from
external APIs, merges results into `stage_data` (reduce double entry)
- [ ] Data validation rules — Starlark `on_validate` can enforce
- [x] Data validation rules — Starlark `on_validate` can enforce
cross-field business rules beyond per-field type checks
- [ ] Stage data schema — typed `stage_data` with declared fields,
- [x] Stage data schema — typed `stage_data` with declared fields,
not opaque JSON blob. Enables structured review views
**Conditional Routing:**
- [ ] Branch expressions — `transition_rules.condition` evaluated against
- [x] Branch expressions — `transition_rules.condition` evaluated against
`stage_data` to select next stage (not always ordinal+1)
- [ ] AI-triggered routing — persona calls `workflow_route` tool with
- [x] AI-triggered routing — persona calls `workflow_route` tool with
a target stage name based on conversation analysis
- [ ] Escalation pattern — "if AI confidence < threshold, route to
- [x] Escalation pattern — "if AI confidence < threshold, route to
human review stage" (help desk use case)
- [ ] Loop stages — return to a previous stage for correction without
- [x] Loop stages — return to a previous stage for correction without
using reject (iterative data gathering)
**Structured Review:**
- [ ] Assignment review view — team member sees `stage_data` as a
- [x] Assignment review view — team member sees `stage_data` as a
structured card/form, not just chat history
- [ ] Approval/reject with comments — reviewer adds notes visible
- [x] Approval/reject with comments — reviewer adds notes visible
to the next stage (not buried in chat)
- [ ] Side-by-side view — chat history + structured data together
- [ ] Bulk review — multiple assignments in a queue with keyboard nav
- [x] Side-by-side view — chat history + structured data together
- [x] Bulk review — multiple assignments in a queue with keyboard nav
**Monitoring Dashboard:**
- [ ] Active instances list — workflow name, current stage, assignee,
- [x] Active instances list — workflow name, current stage, assignee,
age, last activity
- [ ] Stage funnel — how many instances at each stage, bottleneck detection
- [ ] SLA timers — configurable per-stage, visible in review + dashboard
- [ ] Stale instance alerts — notify team admins when instances age out
- [x] Stage funnel — how many instances at each stage, bottleneck detection
- [x] SLA timers — configurable per-stage, visible in review + dashboard
- [x] Stale instance alerts — notify team admins when instances age out
**Use case validation targets:**
- Help desk: AI + KB → escalation to human → resolution tracking