core/docs/Workflow/ROADMAP-0.39.md

# ROADMAP-0.39 — Workflow Product Maturity

Everything deferred from v0.37.15 plus what the MVP gate actually
requires: "Team admins build workflows **visually**."

v0.37.15 delivers the plumbing — cancel, unclaim, reassign, stage CRUD
in the FE, queue views. v0.38.x delivers the extension infrastructure
(multi-file Starlark, connections, libraries) that dynamic workflows
depend on. This series builds the product on top of both foundations.

**Two workflow camps:**
- **Simple workflows** — static stages, fixed forms, declarative rules.
  Team admins author these visually.
- **Dynamic workflows** — Starlark-driven branching, API-backed
  pre-screening, tailored data collection based on customer context.
  Developers author these for team admins to deploy.

A single workflow graph can mix simple, dynamic, and automated stages.

---

## Stage Graph Architecture

Three node types in the workflow graph:

| Node Type | UI | Author | Example |
|-----------|-----|--------|---------|
| **Simple** | Declarative form, fixed fields, built-in renderer | Team admin (form builder) | "Collect name, email, issue type" |
| **Dynamic** | Starlark hook returns form config/routing at runtime | Developer | "Check CRM, skip fields we already know, route by customer tier" |
| **Automated** | No UI — Starlark execution only | Developer | "Call external API, transform data, pre-screen eligibility" |

**Branching:** Layered approach — declarative rules for simple cases
(`if customer.type == enterprise → stage A`), optional Starlark hook
for complex branching. Progressive complexity: team admins use rules,
developers add hooks when needed.

**Visitor surface:** Standalone portal (branded URL, no chat chrome).
The portal renders whatever the current stage dictates — it doesn't
care whether the stage config came from static definition or Starlark.

### Example Mixed Workflow

```
[Pre-screen (automated: CRM lookup via Starlark)]
        │
[Branch: known customer?]
   yes → [Verify Info (simple: pre-filled form)]
   no  → [Full Intake (dynamic: Starlark picks fields)]
        │
[Review (simple stage)]
        │
[Provision (automated: external API call)]
```

---

## Series Overview

```
v0.37.15  Workflow Ownership & Lifecycle (plumbing)
v0.37.16–18  Projects, Workspaces, Debug surfaces
v0.37.19     Tag ("UI Complete")
    │
v0.38.0–.4   Extension Continuation (Starlark, connections, libraries, git-board)
    │
  ── 0.38 tag ──
    │
v0.39.0   Stage Graph Engine + Form Builder  ← graph execution + visual editing
v0.39.1   Stage Reorder + Bulk Ops          ← queue management at scale
v0.39.2   SLA Alerting                      ← stale queues self-report
v0.39.3   Visitor Portal                    ← standalone branded experience
v0.39.4   Workflow Templates + Clone        ← reuse across teams
v0.39.5   Analytics Dashboard               ← throughput, bottlenecks, SLA compliance
    │
v0.40.x   Polish (bug hunt, dead code, stabilize)
    │
  ─ ─ ─  MVP v0.50.0 gate  ─ ─ ─
```

**Milestone gates:**

| Milestone | Gate | Meaning |
|-----------|------|---------|
| v0.37 tag | **UI Complete** | Every platform capability has a surface; no features require raw API calls |
| v0.38 tag | **Extensible** | Extensions can load modules, manage connections, and compose libraries; git-board proves it |
| v0.39.0 | **Self-service** | Team admins build simple workflows visually; developers can add dynamic stages |
| v0.39.2 | **Operational** | Stale work surfaces automatically; team admins can manage queues without monitoring dashboards |
| v0.39.5 | **Measurable** | Team leads can answer "how long does stage X take" and "where are we bottlenecked" |

---

## v0.39.0 — Stage Graph Engine + Form Builder

**The foundational version.** Two deliverables:

### 1. Stage Graph Engine

The workflow execution engine gains awareness of node types. Each stage
in the workflow definition includes a `stage_type` field:

```go
// Stage model additions:
type StageType string

const (
    StageSimple    StageType = "simple"    // declarative form, built-in renderer
    StageDynamic   StageType = "dynamic"   // Starlark hook returns form/routing
    StageAutomated StageType = "automated" // no UI, Starlark only
)

// Branching rule (declarative):
type BranchRule struct {
    Field    string `json:"field"`    // form field key or context var
    Op       string `json:"op"`       // eq, neq, gt, lt, contains, exists
    Value    any    `json:"value"`    // comparison value
    NextStage string `json:"next_stage"` // stage ID to route to
}

// Stage additions:
type Stage struct {
    // ... existing fields ...
    StageType    StageType    `json:"stage_type"`
    BranchRules  []BranchRule `json:"branch_rules,omitempty"`  // declarative
    StarlarkHook string       `json:"starlark_hook,omitempty"` // package:function
}
```

The engine evaluates branch rules first; if no rule matches and a
Starlark hook is configured, it calls the hook. Default fallback is
next ordinal (existing behavior).

For automated stages, the engine calls the Starlark hook immediately
on entry and auto-advances when it returns.

### 2. Form Builder

**The single biggest gap.** `form_template` is currently raw JSON in a
textarea. The MVP gate says "build workflows visually" — this is the
make-or-break.

A visual form builder embedded in the stage editor (E2 from the .15
design). Not a standalone surface — it's a panel that opens when you
click "Edit Form" on a `simple` or `form_chat` stage.

### Fields supported (matching existing TypedFormTemplate)

`text`, `email`, `select`, `number`, `date`, `textarea`, `checkbox`,
`file`. Each with the validation rules already defined in
`models/workflow.go` (min/max length, pattern, min/max value, etc).

### JSX Exemplar

```jsx
/**
 * FormBuilder — visual editor for stage form_template
 *
 * Opens as a slide-out panel from StageEditor.
 * Produces a TypedFormTemplate JSON blob on save.
 *
 * Features:
 *   - Add/remove/reorder fields
 *   - Field type picker with type-specific validation config
 *   - Fieldset grouping (progressive multi-step forms)
 *   - Conditional visibility (when/op/value)
 *   - Live preview pane
 */
function FormBuilder({ initial, onSave, onCancel }) {
  const [fields, setFields] = useState(initial?.fields || []);
  const [fieldsets, setFieldsets] = useState(initial?.fieldsets || []);
  const [useFieldsets, setUseFieldsets] = useState((initial?.fieldsets?.length || 0) > 0);
  const [preview, setPreview] = useState(false);

  function addField(targetFieldset) {
    const field = {
      key: `field_${Date.now()}`,
      type: 'text',
      label: '',
      required: false,
    };
    if (useFieldsets && targetFieldset !== undefined) {
      setFieldsets(fs => fs.map((f, i) =>
        i === targetFieldset ? { ...f, fields: [...f.fields, field] } : f
      ));
    } else {
      setFields(f => [...f, field]);
    }
  }

  function save() {
    const template = useFieldsets
      ? { fieldsets, hooks: initial?.hooks || null }
      : { fields, hooks: initial?.hooks || null };
    onSave(template);
  }

  return (
    <div className="form-builder">
      <div className="form-builder__header">
        <h4>Form Builder</h4>
        <label className="toggle-label">
          <input type="checkbox" checked={useFieldsets}
                 onChange={e => setUseFieldsets(e.target.checked)} />
          Multi-step (fieldsets)
        </label>
        <button className="btn-small" onClick={() => setPreview(!preview)}>
          {preview ? 'Edit' : 'Preview'}
        </button>
      </div>

      {preview ? (
        <FormPreview fields={fields} fieldsets={useFieldsets ? fieldsets : null} />
      ) : (
        <div className="form-builder__canvas">
          {useFieldsets ? (
            <FieldsetEditor fieldsets={fieldsets} setFieldsets={setFieldsets} />
          ) : (
            <FieldList fields={fields} setFields={setFields} />
          )}
          <button className="btn-small" onClick={() => addField()}>
            + Add Field
          </button>
        </div>
      )}

      <div className="form-builder__footer">
        <button className="btn-small btn-primary" onClick={save}>Save Form</button>
        <button className="btn-small" onClick={onCancel}>Cancel</button>
      </div>
    </div>
  );
}

/**
 * FieldEditor — single field configuration row
 *
 * Inline editing of key, type, label, required, validation, condition.
 * Type-specific validation options appear contextually.
 */
function FieldEditor({ field, onChange, onRemove, allFields }) {
  function set(k, v) { onChange({ ...field, [k]: v }); }

  return (
    <div className="field-editor">
      <div className="form-row">
        <div className="form-group" style={{ flex: 2 }}>
          <label>Label</label>
          <input value={field.label} onChange={e => set('label', e.target.value)} />
        </div>
        <div className="form-group" style={{ flex: 1 }}>
          <label>Key</label>
          <input value={field.key} onChange={e => set('key', e.target.value)}
                 placeholder="field_name" />
        </div>
        <div className="form-group" style={{ flex: 1 }}>
          <label>Type</label>
          <select value={field.type} onChange={e => set('type', e.target.value)}>
            <option value="text">Text</option>
            <option value="textarea">Textarea</option>
            <option value="email">Email</option>
            <option value="number">Number</option>
            <option value="date">Date</option>
            <option value="select">Select</option>
            <option value="checkbox">Checkbox</option>
            <option value="file">File</option>
          </select>
        </div>
      </div>

      <div className="form-row">
        <label className="toggle-label">
          <input type="checkbox" checked={field.required}
                 onChange={e => set('required', e.target.checked)} />
          Required
        </label>

        {/* Type-specific validation — only render what applies */}
        {(field.type === 'text' || field.type === 'textarea') && (
          <ValidationText validation={field.validation}
                          onChange={v => set('validation', v)} />
        )}
        {field.type === 'number' && (
          <ValidationNumber validation={field.validation}
                            onChange={v => set('validation', v)} />
        )}
        {field.type === 'select' && (
          <OptionsEditor options={field.options || []}
                         onChange={o => set('options', o)} />
        )}
      </div>

      {/* Conditional visibility */}
      <ConditionEditor condition={field.condition}
                       onChange={c => set('condition', c)}
                       allFields={allFields} />

      <button className="btn-small btn-danger" onClick={onRemove}>Remove</button>
    </div>
  );
}
```

### Deliverables

| File | What |
|------|------|
| `server/models/workflow.go` | `StageType`, `BranchRule` types; stage model additions |
| `server/engine/graph.go` | Graph execution: branch rules → Starlark hook → ordinal fallback |
| `server/engine/automated.go` | Automated stage runner (call hook, auto-advance) |
| `src/js/sw/components/form-builder/index.js` | FormBuilder root |
| `src/js/sw/components/form-builder/field-editor.js` | FieldEditor, type-specific validators |
| `src/js/sw/components/form-builder/fieldset-editor.js` | Fieldset grouping |
| `src/js/sw/components/form-builder/condition-editor.js` | Conditional visibility |
| `src/js/sw/components/form-builder/preview.js` | Live form preview |
| `src/css/form-builder.css` | Styles |
| Integration into StageEditor (E2 from .15) | "Edit Form" button opens builder; stage type picker |

---

## v0.39.1 — Stage Reorder + Bulk Ops

### Stage drag-and-drop

Replace button-based reorder with drag-and-drop in the stage editor.
No external library — use the HTML5 Drag and Drop API with
`draggable`, `ondragstart`, `ondragover`, `ondrop`.

The reorder PATCH endpoint already exists:
`PATCH /api/v1/teams/:teamId/workflows/:wfId/stages/reorder`

### Bulk assignment operations

Team admins managing 20+ assignments need batch actions.

```jsx
/**
 * BulkAssignmentBar — selection-based bulk actions
 *
 * Appears above the queue when 1+ assignments are selected.
 * Actions: bulk cancel, bulk reassign (to specific user).
 */
function BulkAssignmentBar({ selected, teamId, onAction }) {
  const [reassignTo, setReassignTo] = useState('');

  async function bulkCancel() {
    const ok = await sw.confirm(`Cancel ${selected.length} assignments?`, true);
    if (!ok) return;
    await Promise.all(selected.map(id => sw.api.workflowAssignments.cancel(id)));
    sw.toast(`${selected.length} cancelled`, 'success');
    onAction();
  }

  async function bulkReassign() {
    if (!reassignTo) return;
    await Promise.all(selected.map(id =>
      sw.api.workflowAssignments.reassign(id, reassignTo)
    ));
    sw.toast(`${selected.length} reassigned`, 'success');
    onAction();
  }

  return (
    <div className="bulk-bar">
      <span>{selected.length} selected</span>
      <button className="btn-small btn-danger" onClick={bulkCancel}>Cancel All</button>
      <div className="bulk-bar__reassign">
        <TeamMemberPicker teamId={teamId} value={reassignTo}
                          onChange={setReassignTo} />
        <button className="btn-small" onClick={bulkReassign}
                disabled={!reassignTo}>
          Reassign
        </button>
      </div>
    </div>
  );
}
```

### BE changes

None. Bulk ops are client-side `Promise.all` over existing single
endpoints. If performance becomes an issue at scale (50+ assignments),
add a `POST /api/v1/workflow-assignments/bulk` endpoint in a future
patch. YAGNI for now.

---

## v0.39.2 — SLA Alerting

The monitor already computes `sla_breached`. This version makes it
proactive instead of requiring someone to check the dashboard.

### Scheduled SLA scanner

A new periodic task (Go scheduler, not Starlark) that runs every N
minutes:

1. Query active workflow channels with `sla_seconds` configured
2. Compute breach status from `stage_entered_at`
3. For newly breached instances (not yet notified):
   - Create notification for the assigned team members
   - Emit `workflow.sla_breached` WS event
   - If webhook_url configured, fire webhook

### Data changes

Add `sla_notified_at` (nullable timestamp) to the channels table.
The scanner only fires notifications when `sla_notified_at IS NULL`
and the SLA is breached, then sets the timestamp. Prevents duplicate
alerts.

### FE changes

The queue items (E1, E4) already render `sla_breached` badges. Add:
- Pulsing animation on breached badges
- Toast on `workflow.sla_breached` WS event
- Notification bell integration (already wired in .15 via
  `notifications.NotifyWorkflowClaimed` pattern)

### Deliverables

| File | What |
|------|------|
| `server/scheduler/sla_scanner.go` | Periodic SLA check |
| `server/notifications/workflow_sla.go` | SLA notification builder |
| Migration: `sla_notified_at` column | Both PG + SQLite |
| FE: badge animation CSS | `src/css/sw-primitives.css` |

---

## v0.39.3 — Visitor Portal

The visitor currently gets a raw workflow surface embedded in the
app shell. This version delivers a **standalone branded portal** —
a dedicated URL with no chat chrome, optimized for the visitor
experience.

### Scope

1. **Standalone portal surface** — `/w/:id/:slug` renders a
   full-page portal outside the app shell. No sidebar, no user menu,
   no chat chrome. The portal renders the current stage's form
   (simple or dynamic) and handles stage transitions.

2. **Branded entry page** — the workflow landing page already supports
   branding (`accent_color`, `logo_url`, `tagline`). Extend with:
   - Custom welcome text (markdown)
   - Background image/gradient
   - "Powered by" toggle (hide/show Switchboard branding)

3. **Progress indicator** — visitor sees "Step 1 of N" for their
   visible stages only. Internal team stages are invisible. The
   progress bar counts only stages where the visitor is the audience.

4. **Email notifications** — optional. When configured on the workflow:
   - Collect email at stage 0 (or extract from form data)
   - Send "request received" confirmation
   - Send "your request is complete" when workflow completes
   - No notifications for internal stage transitions

5. **Session resume** — visitor who closes the browser can return to
   `/w/:id/:slug` and resume where they left off. Already partially
   implemented via `sb_session` cookie. Polish the UX: show "Welcome
   back, pick up where you left off" instead of starting fresh.

### BE changes

```go
// Workflow model additions:
type WorkflowBranding struct {
    AccentColor   string `json:"accent_color"`
    LogoURL       string `json:"logo_url"`
    Tagline       string `json:"tagline"`
    WelcomeText   string `json:"welcome_text"`    // NEW: markdown
    BackgroundCSS string `json:"background_css"`   // NEW: gradient or image URL
    HidePoweredBy bool   `json:"hide_powered_by"`  // NEW
}

// Email notification config (in workflow settings):
type WorkflowEmailConfig struct {
    Enabled       bool   `json:"enabled"`
    FromName      string `json:"from_name"`
    OnSubmit      string `json:"on_submit_template"`   // markdown template
    OnComplete    string `json:"on_complete_template"`  // markdown template
}
```

Email sending requires an SMTP config at the platform level
(admin settings). If not configured, email toggle is disabled in
the workflow editor with a hint.

### JSX Exemplar

```jsx
/**
 * VisitorProgress — step indicator for visitor-facing stages
 *
 * Mount: workflow portal, above the stage content
 * Only counts stages where audience = visitor.
 */
function VisitorProgress({ stages, currentStage }) {
  const visitorStages = stages.filter(s => !s.assignment_team_id);
  const visitorIndex = visitorStages.findIndex(s => s.ordinal === currentStage);
  const total = visitorStages.length;

  if (total <= 1) return null;

  return (
    <div className="visitor-progress">
      {visitorStages.map((s, i) => (
        <div key={s.id}
             className={`visitor-progress__step ${
               i < visitorIndex ? 'done' :
               i === visitorIndex ? 'active' : ''
             }`}>
          <div className="visitor-progress__dot">{i < visitorIndex ? '✓' : i + 1}</div>
          <span className="visitor-progress__label">{s.name}</span>
        </div>
      ))}
    </div>
  );
}
```

---

## v0.39.4 — Workflow Templates + Clone

Team admins shouldn't rebuild common patterns from scratch. Two
features:

### Clone workflow

"Duplicate" button on the workflow editor. Creates a new workflow with
the same stages, form templates, branch rules, and Starlark hooks.
New slug, new name (appended "- Copy"). No versions carried over
(must re-publish).

```
POST /api/v1/teams/:teamId/workflows/:id/clone
→ 201 { ...newWorkflow }
```

Pure BE operation — deep-copies workflow + stages, generates new IDs.

### Built-in templates

Seed a `workflow_templates` table with common patterns covering both
simple and mixed workflows:

| Template | Stages | Type |
|----------|--------|------|
| Customer Intake | form → review | Simple |
| IT Help Desk | LLM chat → triage → resolution | Simple |
| Approval Chain | form → approve → approve → notify | Simple |
| Smart Intake | automated (CRM lookup) → dynamic (tailored form) → review | Mixed |
| Onboarding | form → team A → team B → automated (provision) | Mixed |

"New from template" in the team-admin workflow creation flow.
Templates are read-only platform data, not user-editable. They
serve as starting points — clone and customize.

### FE changes

- "Duplicate" button in workflow editor header
- "New from Template" option in workflow creation flow
- Template picker modal with descriptions and type badges

---

## v0.39.5 — Analytics Dashboard

Team leads need to answer: how long does each stage take, where are
we bottlenecked, what's our SLA compliance rate.

### Data source

No new data collection. Everything is derivable from existing columns:

- `channels.created_at` — instance start time
- `channels.stage_entered_at` — current stage entry
- `channels.workflow_status` — terminal state
- `workflow_assignments.created_at / claimed_at / completed_at` — assignment lifecycle timestamps

### Metrics

| Metric | Derivation |
|--------|------------|
| Throughput | Count of `completed` instances per time period |
| Average cycle time | `completed_at - created_at` across instances |
| Stage dwell time | `next_stage_entered_at - stage_entered_at` (requires computing from history) |
| SLA compliance | `breached / total` per stage with SLA configured |
| Assignment claim time | `claimed_at - created_at` on assignments |
| Queue depth over time | Snapshot of unassigned count (requires periodic sampling or derive from events) |

### JSX Exemplar

```jsx
/**
 * WorkflowAnalytics — team-admin analytics tab
 *
 * Mount: team-admin workflows surface, "Analytics" tab
 * Data: GET /api/v1/teams/:teamId/workflows/analytics
 *        ?workflow_id=...&period=7d|30d|90d
 */
function WorkflowAnalytics({ teamId }) {
  const [data, setData] = useState(null);
  const [workflowId, setWorkflowId] = useState('');
  const [period, setPeriod] = useState('30d');

  // ... load, filter controls ...

  return (
    <div className="wf-analytics">
      <div className="wf-analytics__filters">
        <WorkflowPicker teamId={teamId} value={workflowId} onChange={setWorkflowId} />
        <PeriodPicker value={period} onChange={setPeriod} />
      </div>

      <div className="wf-analytics__cards">
        <StatCard label="Completed" value={data.completed_count} />
        <StatCard label="Avg Cycle Time" value={formatDuration(data.avg_cycle_seconds)} />
        <StatCard label="SLA Compliance" value={`${data.sla_compliance_pct}%`}
                  variant={data.sla_compliance_pct < 80 ? 'danger' : 'success'} />
        <StatCard label="Avg Claim Time" value={formatDuration(data.avg_claim_seconds)} />
      </div>

      {/* Stage funnel with dwell times — uses existing funnel endpoint + dwell data */}
      <StageFunnel stages={data.stage_metrics} />
    </div>
  );
}
```

### BE changes

New endpoint:
```
GET /api/v1/teams/:teamId/workflows/analytics
    ?workflow_id=...&period=7d|30d|90d
```

Aggregation query over channels + assignments tables. No new tables.
Consider materialized views or periodic snapshot if query cost is
too high on large datasets (unlikely at 50-user scale).

---

## Changeset Budget

| Version | Estimated CS | Heaviest piece |
|---------|-------------|----------------|
| v0.39.0 | 5 | Graph engine + form builder (10+ files) |
| v0.39.1 | 2 | Drag-and-drop is fiddly but small |
| v0.39.2 | 3 | SLA scanner + notification plumbing |
| v0.39.3 | 4 | Standalone portal + branding + email + progress |
| v0.39.4 | 2 | Clone is one BE endpoint + FE button |
| v0.39.5 | 3 | Analytics query + chart components |
| **Total** | **~19** | |

---

## Risk Notes

**v0.39.0 (Graph Engine + Form Builder) is the long pole.** Two
major deliverables in one version. If this slips, the MVP gate phrase
"build workflows visually" fails. Prioritize this over everything
else in the series. If time is tight, ship the graph engine with
simple stages only (dynamic/automated as stubs) and a simplified
form builder (no fieldsets, no conditional visibility), then add
dynamic/automated execution and form builder polish in a patch.

**v0.39.3 (Email) depends on platform SMTP.** If SMTP isn't
configured, the feature is invisible. Consider making the email
config a prerequisite admin step with a setup wizard, or defer email
to v0.39.5 and keep .3 focused on the standalone portal + branding
+ progress.

**v0.39.5 (Analytics) can be cut.** If the series is running long,
analytics is the most deferrable item — it's nice-to-have, not
gate-blocking. The existing monitor + funnel endpoints cover the
critical "is anything stuck" question.

**Dynamic stage authoring DX.** Developers need clear docs and
examples for writing Starlark hooks. The git-board rewrite (v0.38.4)
should establish patterns, but workflow-specific examples are needed.
Consider a "dynamic stage cookbook" as part of v0.39.0.