From 76c32be4398f2173b91fd5f63f567fa205db2309 Mon Sep 17 00:00:00 2001 From: Jeffrey Smith Date: Fri, 3 Apr 2026 11:32:59 +0000 Subject: [PATCH] Roadmap: trim completed history, insert v0.9.x workflow redesign MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completed v0.6.x–v0.8.x sections collapsed to summary tables — detail lives in CHANGELOG.md. New v0.9.x series: workflow system redesign (converter consolidation, team roles, typed forms promotion, stage_mode collapse, full r/w Starlark module, multi-surface manifest). Old 0.9.x reference extensions shift to 0.10.x, sidecar tier to 0.11.x. New design doc: docs/DESIGN-workflow-redesign.md Co-Authored-By: Claude Opus 4.6 (1M context) --- ROADMAP.md | 657 +++++++------------------------ docs/DESIGN-workflow-redesign.md | 126 ++++++ 2 files changed, 271 insertions(+), 512 deletions(-) create mode 100644 docs/DESIGN-workflow-redesign.md diff --git a/ROADMAP.md b/ROADMAP.md index a0fa1bf..1d19ea0 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,6 @@ # Armature — Roadmap -## Current: v0.8.5 — Extension Composability (Kernel Complete) +## Current: v0.9.x — Workflow Redesign Self-hosted extensible platform kernel. Auth, identity, packages, Starlark sandbox, storage, realtime, and ops are kernel primitives. Everything else @@ -9,565 +9,213 @@ is an extension. **Kernel capabilities:** Auth (builtin/mTLS/OIDC) · Users/teams/groups/RBAC · Surfaces/extensions/libraries/workflows · Starlark sandbox (capability-gated) · Object storage (PVC/S3) + ext_data tables · WebSocket hub + realtime pub/sub · -Audit log · Notifications · Scheduled tasks · Cluster registry + HA - -**Completed history:** v0.2.x–v0.5.x fully documented in `CHANGELOG.md`. -Highlights: RBAC + settings cascade, event bus + triggers, SDK stabilization, -workflow engine (multi-stage, team roles, signoff gate, public entry, SLA), -package distribution, Notes surface (CM6, folders, tags, backlinks, graph), -realtime primitive, Chat surface (chat-core library + surface + polish), -upgrade test harness, cluster registry + HA. +Audit log · Notifications · Scheduled tasks · Cluster registry + HA · +Extension composability (slots/contributes/cross-package calls) --- -## v0.6.x — Completed (MVP + Hardening) - -All v0.6.x work is shipped and documented in `CHANGELOG.md`. Summary: - -| Version | Title | Key Deliverables | -|---------|-------|-----------------| -| v0.6.0 | Cluster Registry + HA | PG-backed node registry, heartbeat sweep, LISTEN/NOTIFY routing, self-eviction | -| v0.6.1 | Backup/Restore + Docs | `.swb` archive format, server-side backups, docs surface + 5 guides | -| v0.6.2 | Docs Polish + OpenAPI | Dark mode fix, topbar nav, `api_schema` manifest field, dynamic spec builder | -| v0.6.3 | Dead Code Sweep | Registry install fix, dead Go/JS/HTML deletion, narrowed default bundle | -| v0.6.4 | Admin Health/Metrics | Cluster dashboard merged into Admin tab, block renderer `requires` removed | -| v0.6.5 | Renderer Pipeline | `sw.renderers.register()` kernel primitive, unified markdown, docs rewrite | -| v0.6.6 | Final Hardening | Dependency auto-activation, `ValidateManifest()`, OIDC nonce, ICD/SDK update | -| v0.6.7 | Native mTLS | `TLS_MODE` config, `MTLSNativeProvider`, node-to-node mTLS, `armature-ca.sh` | -| v0.6.8 | Cookie Fix + UI Roadmap | Cookie SameSite fix, UI hardening roadmap published | -| v0.6.9 | Session Lifetime Config | Admin-configurable TTLs, idle timeout, "keep me logged in" | -| v0.6.10 | Viewport Foundation | Single layout model, CSS zoom, 100dvh, dead shell deprecated | -| v0.6.11 | CSS Deduplication | Old primitive system retired, one class per concept | -| v0.6.12 | Extension CSS Isolation | Prefix enforcement via linter, all 12 in-tree packages migrated | -| v0.6.13 | Responsive & Spacing | Spacing token scale (4px grid), tablet breakpoint | -| v0.6.14 | Visual Polish | Stale fallback colors purged, fonts self-hosted, radius tokens | -| v0.6.15 | User Display Audit | Batch user resolve API, `sw.users` SDK module | -| v0.6.16 | Usability Survey Gate | Four audit scripts, contrast/touch-target fixes | -| v0.6.17 | Bug Fixes & Welcome | Notes folder fix, team member add fix, welcome auto-disable, zero default bundle | -| v0.6.18 | CI Bundle Wiring | `BUNDLED_PACKAGES` env var wired into Gitea CI pipeline | - ---- - -## v0.7.x — Test Infrastructure + Quality Gate - -The v0.6.x series built the kernel. v0.7.x makes it provably correct. - -A full surface audit (docs/AUDIT-surfaces.md) found 7 cross-surface issues -and 18 surface-specific issues across Settings, Admin, Team Admin, and Docs. -Only Docs is properly built. The fix is three phases: (1) establish a shell -contract and bring all four primary surfaces to parity, (2) build a runner -framework for end-to-end browser tests, (3) automate those runners headlessly -in CI. - -Design docs: -- `docs/DESIGN-shell-contract.md` — two-slot topbar, three navigation patterns, surface migrations -- `docs/DESIGN-surface-runners.md` — runner framework, requires declarations, headless E2E -- `docs/AUDIT-surfaces.md` — full audit findings - -### v0.7.0 — Shell Contract + Surface Audit + Rebrand Cleanup - -Design doc: `docs/DESIGN-shell-contract.md` - -**Shell Infrastructure** - -| Step | Status | Description | -|------|--------|-------------| -| Shell topbar (two-slot model) | done | Kernel injects topbar into `surface-extension` template. Two named slots: **left** (defaults to manifest title) and **center** (`flex: 1`, for tabs/pickers/search). Home link, notification bell, and user menu always present. | -| Topbar customization API | done | `sw.shell.topbar.setLeft(vnode)` overrides left slot. `sw.shell.topbar.setSlot(vnode)` sets center slot. `sw.shell.topbar.setTitle(str)` shorthand for text-only left. `sw.shell.topbar.hide()` / `.show()` for full-bleed surfaces. | -| Kernel tab CSS | done | `.sw-topbar__tabs` and `.sw-topbar__tab` classes for consistent tab styling in the center slot. Surfaces use these for free or style their own slot content. | -| Notification read broadcast | done | Backend emits `notification.read` and `notification.all_read` WS events. Bell listens for `.created`, `.read`, `.all_read`. Cross-surface sync without refetch. | -| User menu reactivity | done | Emit `package.changed` (install/uninstall/enable/disable) and `auth.changed` (role/membership) WS events. UserMenu listens and re-fetches surface list. Most impactful single fix. | -| Shell announcement global dismiss | done | Dismissed state persisted to localStorage keyed by content hash. Dismiss once, dismissed everywhere. | - -**Surface Migrations** - -| Step | Status | Description | -|------|--------|-------------| -| Settings → Pattern B (flat tabs) | done | Delete custom topbar + sidebar nav. 6 sections become flat tabs in topbar center slot. Content full-width. Fix Teams section: add team admin link, role display, leave action. Remove sessionStorage return URL logic. | -| Admin → Pattern C (category tabs + sidebar) | done | Delete custom `admin-topbar`. `setLeft()` for favicon + "Administration". `setSlot()` for category tabs (People / Workflows / System / Monitoring). Admin sidebar (sub-navigation) unchanged — surface-owned, below the topbar. Bell + user menu come free from shell. Delete bespoke CatIcon renderer if using standard SVGs. | -| Team Admin → Pattern B (flat tabs) | done | Delete custom topbar + sidebar nav. 5 sections (Members / Connections / Workflows / Settings / Activity — Groups removed) become flat tabs. Content full-width. `setTitle()` for team-specific name. Remove sessionStorage return URL. Fix signoff user display (`user_id` → `sw.users.displayName()`). | -| Team Admin: remove Groups tab | done | 37-line dead-end. Read-only "No groups" with no create/docs/link. Remove until team-scoped group management is properly designed. | -| Docs → Pattern A (default) | done | Delete explicit Topbar import. Shell topbar auto-renders with manifest title. Docs sidebar (document list) is in content area, unaffected. | - -**Error Handling + UX Pass** - -| Step | Status | Description | -|------|--------|-------------| -| Inline error states | done | Replace `catch { toast }` with inline error + retry on all list endpoints. New `.sw-inline-error` CSS primitive. Systematic pass across Settings, Admin, Team Admin. | -| Empty state guidance | done | Every "No X" message gets one-line explanation + primary action (create button or doc link). Admin Groups, Workflows, Teams; Team Admin Workflows; Settings Notifications. | - -**Bug Fixes** - -| Step | Status | Description | -|------|--------|-------------| -| evil-chat cleanup | done | ICD security tier: `finally` cleanup block + tighten `409` assertion. | -| Workflow demo error surfacing | done | Replace silent `catch` with inline error + retry. | -| Hello dashboard removal | done | Delete `packages/hello-dashboard/`. | - -**Rebrand** - -| Step | Status | Description | -|------|--------|-------------| -| Light-mode icon SVG | done | New `favicon-light.svg` — square icon, transparent bg, dark node fills. Rename current `favicon-light.svg` (wordmark) to `wordmark.svg`. | -| Dark-mode wordmark SVG | done | New `wordmark-dark.svg` — light text for dark backgrounds. | -| Light-mode raster assets | done | `favicon-light-32.png`, `favicon-light-256.png`. | -| PWA manifest description | done | "Self-hosted extension platform — build, compose, and run extensions." | -| REBRAND-SPEC.md | | Land into `docs/`. Find/replace patterns, validation checklist, asset inventory. | -| base.html favicon swap | done | Verify theme swap works with new square light icon. | - -**Tests** - -| Step | Status | Description | -|------|--------|-------------| -| Shell topbar renders for extensions | done | Home, left slot, center slot, bell, user menu present. | -| Topbar API (setLeft, setSlot, hide) | done | Custom content renders. Hide removes topbar. | -| All 4 surfaces use shell topbar | done | No double topbars. Each pattern (A/B/C) renders correctly. | -| User menu reactive to package install | | Install → menu updates without reload. | -| Notification bell cross-surface sync | | Dismiss on Notes → clears on Chat. | -| Inline error on API failure | | Error + retry shown, not empty list. | - -### v0.7.1 — Surface Runner Framework - -Design doc: `docs/DESIGN-surface-runners.md` - -| Step | Status | Description | -|------|--------|-------------| -| Runner framework (`sw.testing`) | done | SDK module: suite/test/assert, lifecycle hooks, structured JSON results. | -| `requires` declarations | done | Runner manifests declare dependencies. Missing packages → clean skip. | -| Cleanup enforcement | done | `s.track(type, id)` auto-deletes in `afterAll`. No leaked state. | -| Warning tier | done | pass / fail / warning. No silent catch swallowing. | -| ICD runner migration | done | Refactor to `sw.testing`. Test logic preserved. | -| SDK runner migration | done | Refactor to `sw.testing`. Domain suites preserved. | -| Runner registry surface | done | `/s/test-runners` — list runners, run-all, results dashboard. | - -### v0.7.2 — Package Runners + CI Gate - -| Step | Status | Description | -|------|--------|-------------| -| Notes runner | done | `requires: ["notes"]`. 3 suites (crud, folders, tags-search), 12 tests. | -| Chat runner | done | `requires: ["chat", "chat-core"]`. 2 suites (conversations, messaging), 9 tests. | -| Schedules runner | done | `requires: ["schedules"]`. 1 suite (crud), 5 tests. | -| Workflow runner | done | `requires: ["content-approval"]`. 1 suite (lifecycle), 5 tests. | -| Renderer runner | done | `requires: ["mermaid-renderer"]`. 1 suite (contract), 4 tests. | -| Runner result API | done | `POST/GET /api/v1/admin/test-runners/results`. In-memory store, 4 Go tests. | -| CI integration | done | `test-runners` stage in Gitea CI. Playwright driver, `wait-for-healthy.sh`. | -| CI DinD networking fix | done | Resolve container IP via `docker inspect` — DinD port mapping doesn't expose to runner localhost. | - -### v0.7.3 — Extension Shell Migration - -Chat, Notes, and Schedules still use the old `sw.shell.Topbar` component, -producing a double topbar (shell-injected + surface-owned). Migrate all -three to the v0.7.0 shell contract (`sw.shell.topbar.setTitle/setSlot`), -same pattern as the kernel surface migrations. - -| Step | Status | Description | -|------|--------|-------------| -| Chat → shell topbar | done | Delete ``, use `setTitle('Chat')` + `setSlot()` for thread title/people button. | -| Notes → shell topbar | done | Delete ``, use `setTitle('Notes')` + `setSlot()` for action buttons. | -| Schedules → shell topbar | done | Delete ``, use `setTitle('Schedules')` + `setSlot()` for count/new button. | -| Update package runner tests | done | Shell-topbar test suite in each runner — assert no legacy Topbar, uses shell API. | - -### v0.7.4 — Documentation + Deferred Surface Work - -| Step | Status | Description | -|------|--------|-------------| -| Docs category grouping | done | Backend `Category` field on doc entries. Frontend groups sidebar by category with headings. Four categories: Getting Started, Platform, Extension Development, Operations. | -| Permissions & Groups guide | done | RBAC model, 7 permission slugs, system/custom groups, settings cascade, extension permissions. | -| Workflows user guide | done | Entry modes, stages, team roles, signoff gates, SLA, public forms, branch rules, Starlark hooks. | -| Starlark Reference | done | Sandbox constraints, 10 modules with function signatures, permission gates, example hook script. | -| Frontend JS Guide | done | Preact+htm runtime, 16 SDK modules with API reference, shell topbar patterns, CSS contract. | -| Extension config_section docs | done | Manifest schema, backend discovery, frontend contract, example component. Added to Extension Guide. | -| Docs content refresh | done | All 10 user-facing docs reviewed for v0.7.x accuracy: rebrand volume names, stale CSS vars, TLS_MODE env, self-hosted font notes, Architecture frontend section. | -| Team Admin Workflows split | done | 722-line `workflows.js` split into 3 modules: `workflows.js` (router+CRUD), `workflow-editor.js` (editor+stages), `workflow-monitor.js` (assignments+monitor+signoff). | -| Stale CSS variable fix | done | `--bg-2` references in `sw-shell.css` and `sw-primitives.css` replaced with `--bg-secondary`. | - -### v0.7.5 — Headless E2E + CI Gate - -| Step | Status | Description | -|------|--------|-------------| -| CI gating redesign | done | Fix path rules: `VERSION`/`scripts/*` deploy-only, `docs/*` no longer skips deploy, test-runners trigger on any code change. | -| E2E smoke test harness | done | `ci/e2e-smoke-test.sh` + `ci/e2e-smoke-driver.js` — Playwright visits every surface, asserts topbar, no JS errors, home link. | -| Screenshot-on-failure | done | Full-page screenshot + console log saved as CI artifacts. | -| CI pipeline integration | done | Re-enable `test-runners` stage, add `e2e-smoke` stage. Failure blocks merge. | -| Design docs landed | done | `docs/DESIGN-storage-primitives.md`, `docs/DESIGN-extension-composability.md`. Roadmap expanded through v1.0. | - -### v0.7.6 — Code Hygiene + Test Coverage - -Codebase analysis (April 2026) found 9 code smells from the chat gut and -a test coverage gap in the store layer. Fix before v0.8.x kernel expansion. - -**Critical Fixes** - -| Step | Status | Description | -|------|--------|-------------| -| Remove channels from allowedViews | done | `db_module.go` — `ext_view_channels` does not exist. `db.view("channels")` crashes. | -| Fix workflow.html dead routes | done | Template calls `/api/v1/channels/{id}/workflow/*` — routes moved to `/api/v1/public/workflows/`. Update template to match. Also fixed `RenderWorkflow` handler to use route param as entry token. | - -**Dead Code Removal** - -| Step | Status | Description | -|------|--------|-------------| -| Delete SeedTestChannel() | done | `database/testhelper.go` — inserts into nonexistent channels table. | -| Remove RunContext.ChannelID | done | `sandbox/runner.go` — vestigial field, unused by any module. | -| Remove webhook.ChannelID | done | `webhook/webhook.go` — vestigial struct field from chat era. | -| Fix stale comments | done | `storage.go` key format, `prometheus.go` route pattern, `workflow_module.go` param name — reference dead `/channels` paths. | - -**Migration Hygiene** - -| Step | Status | Description | -|------|--------|-------------| -| Align SQLite migration numbering | done | SQLite 013 placeholder + renumber test_runner_type to 014. Compat rename in migrate.go. | -| Document missing 008 | done | Comment in both 009 files explaining the gap (merged into 007). | - -**Test Gap Closure** - -| Step | Status | Description | -|------|--------|-------------| -| store/ unit tests | done | Moved to v0.7.9. 17 SQLite store tests: workflow CRUD, stages, instances, lifecycle, tokens, users, groups. | -| workflow/ routing unit tests | done | 82 tests: ResolveNextStage, ResolveStageByName, ParseStageConfig, evaluateCondition (all operators). | -| middleware/ unit tests | done | 10 tests: RequirePermission, RequireAdmin, permission caching, RateLimiter (allow/deny/fail-open). | -| InstallPackage decomposition | done | Moved to v0.7.9. Split into 7 phases: receiveUpload, parseAndValidateArchive, extractPackageAssets, registerPackage, applySchemaAndPermissions, resolveDependencies, checkCapabilities. | - -**Bug Fixes** - -| Step | Status | Description | -|------|--------|-------------| -| Remove Admin Storage tab | done | Dead weight under System — backend endpoint preserved for future Monitoring use. | -| Fix backup download | done | `sw.auth.token()` → `sw.auth._getToken()`. Both "Download Backup" and server backup download buttons now work. | - -### v0.7.7 — API Tokens + Extension Permissions - -Personal access tokens (PATs) for programmatic API access, plus -extension-declared user permissions for backend RBAC enforcement. -Unblocks: E2E test auth, CI automation, extension permission gating, -sidecar auth (v0.10.x). - -**API Tokens (PATs)** - -| Step | Status | Description | -|------|--------|-------------| -| `api_tokens` table + migration | done | `id`, `user_id`, `name`, `token_hash`, `permissions` (JSON array), `expires_at`, `last_used_at`, `created_at`. PG + SQLite. | -| Token store interface | done | `Create`, `GetByHash`, `ListForUser`, `Revoke`, `CleanExpired`, `UpdateLastUsed`. | -| Token generation endpoint | done | `POST /api/v1/auth/tokens` — name, permissions (subset of user's), expires_at. Returns token once (plain text), stored as SHA-256 hash. | -| Token management endpoints | done | `GET /api/v1/auth/tokens` (list mine), `DELETE /api/v1/auth/tokens/:id` (revoke). | -| Admin token endpoint | done | `POST /api/v1/admin/tokens` — creates token for any user. Explicit admin action, audit logged. | -| Auth middleware update | done | Accept `Bearer arm_pat_...` tokens alongside JWTs. Resolve user + scoped permissions from token. PAT requests skip refresh token flow. | -| Permission scoping | done | Token permissions ⊆ creating user's resolved permissions at creation time. Validated on create. If user loses a permission later, token retains it until revoked (git model). | -| Settings UI | done | Settings → API Tokens tab. Create, list, revoke. Show permissions, expiry, last used. Token value shown once on creation. | -| Admin UI | done | Admin → People → user detail → "Create API Token" action. | -| E2E test migration | done | Replace `ci/e2e-smoke-test.sh` login flow with PAT-based auth. Seed token via `ARMATURE_BOOTSTRAP_PAT=true`. | - -**Extension-Declared User Permissions** - -| Step | Status | Description | -|------|--------|-------------| -| `user_permissions` manifest field | done | Extensions declare permissions users need: `"user_permissions": ["image-gen.use", "image-gen.admin"]`. | -| Dynamic permission registry | done | On install: merge into valid permission set. On uninstall: remove. `AllPermissions` becomes `KernelPermissions + ExtensionPermissions`. | -| Group UI update | done | Admin → Groups → permission checkboxes include extension-registered permissions, grouped by source package. | -| `permissions` Starlark module | done | `permissions.check(user_id, "image-gen.use")` → resolves user's groups, returns bool. New permission: none required (read-only check against kernel data). | -| `gate_permission` manifest field | done | Optional. If set, `ext_api.go` checks this permission before calling `on_request`. Extension doesn't execute for unauthorized users. | -| `req["permissions"]` in request dict | done | `ext_api.go` resolves user permissions and includes them in the request dict. Extensions can check inline without the module. | - -### v0.7.8 — Bug Fixes & Admin Gaps - -| Step | Status | Description | -|------|--------|-------------| -| Workflow landing page Start | done | Added `StartBySlug` handler + `/api/v1/workflow-entry/:scope/:slug` route. Landing page Start button now resolves scope/slug → workflow ID and creates instance. | -| Workflow delete guard | done | Admin `DELETE /api/v1/workflows/:id` rejects team-scoped workflows (403). Team workflows must use team endpoint. Prevents accidental global template deletion. | -| Package button cleanup | done | Delete button hidden for `bundled` packages via `IMMUTABLE_SOURCES` guard, matching Export/Update. | -| Package export fetch | done | Export uses `fetch()` with auth token instead of `window.open()`. Handles errors, shows toast on missing assets. | -| Settings CSS fix | done | Bottom padding increased to `--sp-12` on `.admin-settings-form`. Save button no longer cut off. | -| Admin stage builder + links | done | Shared `StageForm` component between admin and team-admin. Public entry URL with copy button in admin workflow editor. | - -### v0.7.9 — Workflow Independence Audit - -Workflows must work end-to-end without chat or any 3rd-party package -dependency. Chat, notifications, and other packages should *enhance* -workflows but never be required for core operation. - -**RenderWorkflow Fix (critical)** - -| Step | Status | Description | -|------|--------|-------------| -| Fix RenderWorkflow() handler | done | Was a stub (always `stageMode = "custom"`, no data loaded). Now loads instance by token/ID, resolves current stage, populates all template data. | -| Rename WorkflowPageData fields | done | `ChannelID` → `EntryToken`, `ChannelTitle` → `WorkflowTitle`, `ChannelDescription` → `WorkflowDescription`. Vestigial chat-era names removed. | -| Align stage mode values | done | Template now uses Go constants (`form`, `review`, `delegated`, `automated`) instead of legacy names (`form_only`, `form_chat`). | -| Remove dead chat UI | done | Chat CSS, split layout, `sendMessage()` function, and fallback chat branch all removed from `workflow.html`. Replaced with clean form/review/unsupported modes. | -| Fix landing page mode mapping | done | `workflow-landing.html` conditionals updated to match Go stage mode constants. | -| Update OpenAPI spec | done | `stage_mode` enum changed from `[custom, form_only, form_chat, review]` to `[form, review, delegated, automated]`. | - -**Verification & Testing** - -| Step | Status | Description | -|------|--------|-------------| -| Admin/team-admin UI audit | done | All workflow CRUD, monitoring, stage builder verified clean — zero chat/chat-core references. | -| E2E workflow test | done | `ci/e2e-workflow-nochat.sh` — creates workflow via API, adds form+review stages, starts instance via public entry, verifies landing + execution pages. | -| Store unit tests (SQLite) | done | 17 tests: workflow CRUD, stages, instances, lifecycle (advance/complete/cancel/stale), team scope, API tokens, users, groups. Carried from v0.7.6. | -| InstallPackage decomposition | done | Split 400-line handler into 7 private methods: `receiveUpload`, `parseAndValidateArchive`, `extractPackageAssets`, `registerPackage`, `applySchemaAndPermissions`, `resolveDependencies`, `checkCapabilities`. Carried from v0.7.6. | - -**Bug Fixes (found during verification)** - -| Step | Status | Description | -|------|--------|-------------| -| Entry token resolution | done | `RenderWorkflow()` passed route param (instance ID) as entry token. JS then sent instance ID to the advance API which expects entry token. Fixed: resolve actual token from `inst.EntryToken` + check `?token=` query param. | -| Fieldset submit guard | done | `submitForm()` checked `FORM_TPL.fields` but not `.fieldsets` — progressive forms silently no-op'd on submit. Fixed: accept either. | -| Stage advance status check | done | Template JS checked `result.status === 'advanced'` but API returns instance status (`active`/`completed`). Fixed: on 200 OK with `active` status, reload to render next stage. | - -**Discovered issues (deferred)** - -The audit uncovered deeper workflow UX gaps that are out of scope for v0.7.9: - -1. **Public→authenticated stage handoff** — After a public visitor completes their stages, the workflow advances to an authenticated stage (e.g. "classify"). The visitor sees the next stage's form + an auth error instead of a "thank you, we'll take it from here" message. The page should detect audience mismatch and show a completion/waiting screen. -2. **Team member pickup UI** — No surface exists for team members to see workflow instances assigned to their team and claim/work on them. Currently only visible in team-admin monitoring. -3. **Assignment flow** — No mechanism for team admins to assign a specific instance to a specific team member. The `workflow_assignments` table exists but has no admin UI for manual assignment. - -These will be addressed in a future version (see v0.7.10 below). - -### v0.7.10 — Workflow Handoff + Assignment UI - -Addresses the three UX gaps found during the v0.7.9 independence audit. -The workflow engine and data model already support these flows — the -missing pieces are page-level audience detection and team-facing UI. - -| Step | Status | Description | -|------|--------|-------------| -| Public stage completion screen | done | `RenderWorkflow()` detects audience mismatch (team/system stage + unauthenticated visitor) and renders "Submitted Successfully" screen with reference ID. | -| Team workflow inbox | done | Enhanced `AssignmentsTab` in team-admin: claim/unclaim/work actions, time-ago display, WS live updates. Fixed vestigial `channel_id` references. | -| Manual assignment UI | done | "Assign" button on unassigned rows with team member dropdown. New `POST /api/v1/assignments/:id/assign` endpoint (requires `workflow.create` permission). | -| Assignment notifications | done | Engine calls `notifyAssignment()` on assignment creation — notifies specific user or all team members via notification system. | -| Team workflow instances endpoint | done | `GET /api/v1/teams/:teamId/workflow-instances` + cancel endpoint. `ListInstancesByTeam` store method (SQLite + PG). SDK methods added. | -| API response enrichment | done | `ListByTeam` and `ListMine` handlers enrich assignments with `workflow_name`, `stage_name`, `sla_breached` by joining instance → workflow → version snapshot. Cached per-request. Same for `ListTeamInstances` with `instanceView` struct. | -| Team middleware fix | done | `RequireTeamAdmin` / `RequireTeamMember` system-admin bypass was dead code (`c.Get("role")` never set). Fixed to check `PermSurfaceAdminAccess` via `resolveAndCachePerms`. | -| SDK gap closure | done | Added `workflowAssignments` domain (claim/unclaim/complete/cancel/assign/mine), `teams.assignments`, `teams.workflowInstances`, `teams.cancelWorkflowInstance`. Fixed dead `workflows.cancel` route. | -| E2E multi-stage test | done | `ci/e2e-workflow-handoff.sh`: public form → team review → claim → complete. Verifies audience mismatch screen, assignment creation, full lifecycle. | - -### v0.7.11 — Query & HTTP Ergonomics (was v0.7.10) - -Four additions to existing sandbox modules. No new files beyond tests, -no schema changes. Motivated by real-world extension porting feedback: -complex SQL decomposition and synchronous HTTP fan-out are the two -remaining friction points for extensions migrating from native Go/Python. - -| Step | Status | Description | -|------|--------|-------------| -| `db.count()` | done | `db.count(table, filters={})` → int. `SELECT count(*) FROM ext_{pkg}_{table} WHERE ...`. Uses existing `starlarkFiltersToSQL` builder. Permission: `db.read`. | -| `db.aggregate()` | done | `db.aggregate(table, column, op, filters={})` → single value. `op` ∈ {`count`, `sum`, `avg`, `min`, `max`}. Column name validated same as filter keys. Returns int/float/None. Permission: `db.read`. | -| `db.query_batch()` | done | `db.query_batch(queries)` → list of result sets. Each query spec is a dict with `table` (required), `filters`, `order`, `limit`, `before`, `after`, `search_like` (all optional, same as `db.query`). Loops extracted `buildSelectQuery` helper internally. Permission: `db.read`. | -| `http.batch()` | done | `http.batch(requests)` → list of response dicts (ordered). Each request dict: `method`, `url`, `body`, `headers`. Concurrent dispatch via goroutines capped at 10, `sync.WaitGroup` collection. Reuses existing `executeHTTPRequest` plumbing (SSRF checks, domain allowlist, body cap, timeout). Individual failures return error response dicts, don't abort batch. Permission: `api.http`. | -| Tests | done | 21 new tests (14 db, 7 http). `db.count`/`db.aggregate` test empty table, filtered, type coercion, invalid op/column. `db.query_batch` test mixed specs, empty list, cap, missing table. `http.batch` test blocked domains, partial failure, mixed results, input validation. | - -### v0.7.12 — Concurrent Execution Primitive (was v0.7.11) - -New `batch` sandbox module. Enables extensions to parallelize arbitrary -Starlark callables — including library function calls via `lib.require()`. -The kernel manages concurrency (multiple `starlark.Thread` instances in -goroutines); extensions stay single-threaded and synchronous from the -author's perspective. - -Key insight: library exports loaded via `lib.require()` are frozen structs -(immutable, safe to share across threads). Each concurrent branch gets -fresh module instances (`db`, `http`, etc.) — same pattern as -`buildRestrictedModules` in `triggers/schedule.go`. - -| Step | Status | Description | -|------|--------|-------------| -| Design doc | done | `docs/DESIGN-batch-exec.md`. Thread isolation model, module construction, error semantics (partial results on failure), permission model, concurrency cap. | -| `batch.exec()` | done | `batch.exec(callables)` → `(results, errors)` tuple. List of Starlark callables (lambdas, function refs). Each runs in its own `starlark.Thread` with fresh modules. Concurrent goroutines capped at 8. Ordered results. Errors are per-callable (None on success, string on failure). Permission: `batch.exec` (new). New file: `sandbox/batch_module.go`. | -| Runner wiring | done | `buildModulesWithLibCtx` creates `batch` module when `batch.exec` permission granted. Module receives runner reference for per-branch module construction. | -| Tests | done | 12 unit tests: parallel execution ordering, partial failure, timeout per-branch, parent cancellation, cap enforcement, empty list, non-callable, permission gating, frozen struct sharing, nested batch.exec, default timeout, timeout clamping. All pass with `-race`. | +## Completed — v0.6.x through v0.8.x + +All completed work is documented in `CHANGELOG.md`. + +### v0.6.x — MVP + Hardening + +| Version | Title | +|---------|-------| +| v0.6.0 | Cluster Registry + HA | +| v0.6.1 | Backup/Restore + Docs | +| v0.6.2 | Docs Polish + Dynamic OpenAPI | +| v0.6.3 | Dead Code Sweep + Registry Fix | +| v0.6.4 | Admin Health/Metrics + Cluster Merge | +| v0.6.5 | Renderer Pipeline + Docs Rewrite | +| v0.6.6 | Final Hardening | +| v0.6.7 | Native mTLS | +| v0.6.8 | Rebrand + Cookie Fix | +| v0.6.9 | Session Lifetime Config | +| v0.6.10 | Viewport Foundation | +| v0.6.11 | CSS Deduplication | +| v0.6.12 | Extension CSS Isolation | +| v0.6.13 | Responsive & Spacing | +| v0.6.14 | Visual Polish | +| v0.6.15 | User Display Audit | +| v0.6.16 | Usability Survey Gate | +| v0.6.17 | Bug Fixes & Welcome | +| v0.6.18 | CI Bundle Wiring | + +### v0.7.x — Test Infrastructure + Quality Gate + +| Version | Title | +|---------|-------| +| v0.7.0 | Shell Contract + Surface Audit + Rebrand | +| v0.7.1 | Surface Runner Framework | +| v0.7.2 | Package Runners + CI Gate | +| v0.7.3 | Extension Shell Migration | +| v0.7.4 | Documentation + Surface Work | +| v0.7.5 | Headless E2E + CI Gate | +| v0.7.6 | Code Hygiene + Test Coverage | +| v0.7.7 | API Tokens + Extension Permissions | +| v0.7.8 | Bug Fixes & Admin Gaps | +| v0.7.9 | Workflow Independence Audit | +| v0.7.10 | Workflow Handoff + Assignment UI | +| v0.7.11 | Query & HTTP Ergonomics | +| v0.7.12 | Concurrent Execution Primitive | + +### v0.8.x — Storage Primitives + Composability (Kernel Complete) + +| Version | Title | +|---------|-------| +| v0.8.0 | `files` Module | +| v0.8.1 | `workspace` Module | +| v0.8.2 | Capability Negotiation | +| v0.8.3 | Vector Column Type | +| v0.8.4 | Documentation Refresh + Surface Sizing Fix | +| v0.8.5 | Extension Composability | --- ## Planned -### v0.8.x — Storage Primitives +### v0.9.x — Workflow Redesign -Design doc: `docs/DESIGN-storage-primitives.md` +Design doc: `docs/DESIGN-workflow-redesign.md` -The last major kernel expansion. Completes the primitive set that the -entire extension ecosystem builds on. After v0.8.x, the kernel is -feature-complete for 1.0 — all new capabilities are extensions. +The workflow system (~7,600 lines, 25+ files) was built incrementally and +has accumulated redundant concepts, buried primitives, and a read-only +Starlark module. This series cleans up the debt and promotes reusable +primitives before building reference extensions on top. -**v0.8.0 — `files` Module** ✅ +**v0.9.0 — Starlark Converter Consolidation + Snapshot Cleanup** -Bridge `ObjectStore` (PVC/S3) into the Starlark sandbox. Extension-scoped -key namespacing (`ext/{packageID}/`). Permissions: `files.read`, -`files.write`. Metadata companions stored as sibling objects. +Three files contain near-identical Go↔Starlark converters; three copies +of the snapshot parser exist. Consolidate into `sandbox/convert.go` and +one exported snapshot function. Standardize on wrapped snapshot format. -New: `sandbox/files_module.go`. Modified: runner wiring, permission -constants. `ObjectStore` interface gained `List()` method. 16 new tests. +**v0.9.1 — Team User Roles** -**v0.8.1 — `workspace` Module** ✅ +Promote the buried role system to a kernel primitive. New +`team_user_roles` table (many-to-many). Manifest `requires_roles` field. +Team admin UI for role assignment. Kernel middleware `RequireRole()`. +Starlark SDK: `teams.get_member_roles()`, `teams.has_role()`. -Managed disk directories for tools that need a real filesystem (git, -compilers, media tools). Extension-scoped path namespacing -(`{WORKSPACE_ROOT}/{packageID}/{name}/`). Five builtins: `create`, -`path`, `list`, `delete`, `usage`. Permission: `workspace.manage`. -Quota enforcement via `WORKSPACE_QUOTA_MB`. Path traversal and symlink -escape protection. +**v0.9.2 — Package Adoption + Roles** -New: `sandbox/workspace_module.go`. Modified: runner wiring, permission -constants, config (`WORKSPACE_ROOT`, `WORKSPACE_QUOTA_MB`), main.go -startup. 16 new tests. +`scope: adoptable` manifest field. When a team adopts an adoptable +package, the package's `requires_roles` auto-populate into the team's +role slots. Replaces `AdoptTeamWorkflow` clone mechanism. -**v0.8.2 — Capability Negotiation** ✅ +**v0.9.3 — Typed Forms → SDK Primitive** -Extensions declare environment requirements in manifest (`capabilities.required`, -`capabilities.optional`). Kernel validates at install time — required caps -reject with 422, optional caps log a warning. Runtime query via -`settings.has_capability()`. Admin endpoint: `GET /admin/capabilities`. +Extract `TypedFormTemplate`, `FormField`, `FormFieldset`, etc. from +`models/workflow.go` into a `forms` package. FE SDK: `sw.forms.render()` +and `sw.forms.validate()`. Starlark: `forms.validate()`. Any package +can declare forms, not just workflow stages. -Detected capabilities: `pgvector`, `workspace`, `object_storage`, `s3`, -`postgres`. +**v0.9.4 — Deprecate `stage_type`, Collapse `stage_mode`** -New: `handlers/capabilities.go`, `handlers/capabilities_test.go`, -`sandbox/settings_module_test.go`. Modified: install handler, bundled -handler, settings module, runner, main.go. 18 new tests. +`stage_type` (simple/dynamic/automated) is redundant with `starlark_hook` +presence. Remove from new manifests, keep parsing for backward compat. +Collapse `stage_mode` from 4 to 3 values: form / delegated / automated. +`review` was just `delegated` with signoff — signoff is independent of +rendering mode. -**v0.8.3 — Vector Column Type** ✅ +**v0.9.5 — Full Read/Write Workflow Starlark Module** -`"vector(N)"` in `db_tables` manifest. Maps to native `vector(N)` on -PG+pgvector, `JSONB` on PG without, `TEXT` on SQLite. New `db.query_similar()` -with dual-path dispatch (native operator vs Go-side brute force). Auto-created -HNSW index on pgvector. `starlarkToGoValue` extended with list→JSON. +Add `workflow.start()`, `workflow.advance()`, `workflow.cancel()`, +`workflow.submit_signoff()` to the Starlark module. Extract engine +interface to break circular import. Unlocks fully automated workflow +orchestration from Starlark hooks and extensions. -New: `docs/DESIGN-vector-column.md`. Modified: `ext_db_schema.go`, -`db_module.go`, `runner.go`, `extensions.go`, `main.go`. 13 new tests. +**v0.9.6 — Conditional Routing → SDK Primitive** -**v0.8.4 — Documentation Refresh + Surface Sizing Fix** ✅ +Expose `routing.evaluate(rules, data)` as a Starlark SDK function. +Branch rules become a reusable decision engine for any extension. -Docs pass covering v0.7.5–v0.8.3 additions. STARLARK-REFERENCE.md gains -`workspace` module, `permissions` module, `settings.has_capability()`. -EXTENSION-GUIDE.md gains `capabilities` manifest, `vector(N)` column type, -`user_permissions`, `gate_permission`, updated permissions list. Surface -sizing fix: `.surface-inner` converted to flex column layout so the shell -topbar and surface container share vertical space correctly — eliminates -44px scroll cutoff on all surfaces. +**v0.9.7 — Multi-Surface Manifest + Route Resolution** -**v0.8.5 — Extension Composability** ✅ +Packages can declare multiple surface routes with independent access +controls. Kernel resolves routes per-page. Enables mixed public/auth +pages in a single package. -Design doc: `docs/DESIGN-extension-composability.md` +**v0.9.8 — Surface Access via Roles** -Manifest declarations for `slots` (host surfaces declare injection points) -and `contributes` (extensions declare UI contributions to other packages' -slots). `lib.require()` relaxed from library-only to any package with -`exports` — enables full packages (with surfaces, settings, UI) to also -expose callable backend functions. `sw.slots.renderAll()` SDK helper. -Admin slot aggregation endpoint. Uninstall orphan warnings. - -The composability primitive that enables: LLM tool registration, UI -contribution (toolbar buttons, image actions), and cross-extension -function calls. No new tables, migrations, or permissions. - -Modified: `sandbox/lib_module.go`, `handlers/extensions.go`, -`handlers/packages.go`, `src/js/sw/sdk/slots.js`, `server/main.go`. -New: `handlers/admin_slots.go`. Docs: PACKAGE-FORMAT, EXTENSION-GUIDE, -STARLARK-REFERENCE updated. +Wire team roles (v0.9.1) into surface access declarations: +`access: role:approver`. Kernel middleware checks role membership. +Completes the workflow→package access story. --- -### v0.9.x — Reference Extensions +### v0.10.x — Reference Extensions The kernel is complete. This series proves the platform by shipping first-party extensions that exercise every primitive. These are **extensions, not kernel code** — they ship as `.pkg` files and can be uninstalled. -**v0.9.0 — `vector-store` Library** +**v0.10.0 — `vector-store` Library** -Library extension: document ingestion, chunk storage, embedding persistence, -semantic similarity search. Three-tier vector strategy (pgvector → JSONB -fallback → TEXT fallback). Consumes: `db.write`, `files.read`. +Document ingestion, chunk storage, embedding persistence, semantic +similarity search. Consumes: `db.write`, `files.read`. -**v0.9.1 — `llm-bridge` Library** +**v0.10.1 — `llm-bridge` Library** -Library extension: model abstraction, tool-use routing, provider BYOK via -connections. Exposes `complete()`, `embed()`, `classify()`, `register_tool()` -to consumer extensions. Tool extensions (image-gen, code-exec, web-search) -register at load time — LLM discovers available tools automatically. +Model abstraction, tool-use routing, provider BYOK via connections. +Exposes `complete()`, `embed()`, `classify()`, `register_tool()`. Consumes: `connections.read`, `api.http`, `db.write`. -**v0.9.2 — `file-share` Extension** +**v0.10.2 — `file-share` Extension** -File upload via `api_routes`, storage via `files` module, download links, -optional team/group ACLs via resource grants. First extension to exercise -the full `files` primitive end-to-end. +File upload, storage via `files` module, download links, team/group +ACLs via resource grants. -**v0.9.3 — `code-workspace` Extension** +**v0.10.3 — `code-workspace` Extension** -Managed code repositories via `workspace` module. Git clone/pull via -`api.http` to Gitea/GitHub API. File browser surface. Structural indexing -(AST-aware) for code navigation. Consumes: `workspace.manage`, `files.write`, -`api.http`. +Managed code repos via `workspace` module. Git operations, file browser +surface, structural indexing. Consumes: `workspace.manage`, `files.write`. -**v0.9.4 — `image-gen` + `image-edit` Extensions** +**v0.10.4 — `image-gen` + `image-edit` Extensions** -Image generation via external APIs (DALL-E, Stable Diffusion). Registers -as LLM tool with `llm-bridge`. Contributes regen/edit buttons to -`chat:image-actions` slot. Image-edit extends with inpaint, outpaint, -upscale, and style transfer — each contributing action buttons to the -same slot independently. First full demonstration of the composability -pattern: three extensions contributing UI to a fourth's surface. +Image generation/editing via external APIs. Composability demo: +contributes to `chat:image-actions` slot. -**v0.9.5 — Chat System** +**v0.10.5 — Chat System** -Generic 1-to-N human messaging. `chat-core` library (conversation/message -CRUD) + `chat` surface. Declares `chat:message-actions`, `chat:image-actions`, -`chat:composer-tools` slots for extension contribution. LLM participation -as a follow-on consuming `llm-bridge`. Consumes: `db.write`, -`realtime.publish`, `files.write` (attachments). +Generic 1-to-N messaging with slots. `chat-core` library + `chat` +surface. Declares `chat:message-actions`, `chat:image-actions`, +`chat:composer-tools` slots. Consumes: `db.write`, `realtime.publish`, +`files.write`. --- -### v0.10.x — Sidecar Tier + Polish +### v0.11.x — Sidecar Tier + Polish -**v0.10.0 — Sidecar Tier** +**v0.11.0 — Sidecar Tier** -Autonomous out-of-process extensions for workloads that can't run in -Starlark: ML inference, media transcoding, language servers, local git -operations. The kernel does NOT manage container lifecycle — sidecars -are independent processes (Docker, systemd, bare binary) that connect -inward to the kernel, following the cluster registry pattern: +Out-of-process extensions for workloads that can't run in Starlark (ML +inference, media transcoding, language servers, local git). Sidecars are +independent processes that connect inward to the kernel, following the +cluster registry pattern. Authentication via mTLS or registration tokens. -- Sidecar boots with cluster config (instance URLs, credentials). -- Connects and self-registers, declaring capabilities ("sklearn", - "gpu", "ffmpeg", "sentence-transformers"). -- Goes inert until an admin authorizes (same flow as extension permissions). -- Kernel dispatches work; sidecar returns results. -- Heartbeat keeps registration alive; missed heartbeat = eviction. - -Communication channel TBD (WebSocket over existing event bus, gRPC, or -HTTP callback model — design doc required). Authentication via mTLS -certs or registration tokens. - -This model eliminates k8s RBAC dependency for local deployments. The -kernel stays thin — it accepts registrations and dispatches work. The -sidecar brings its own compute and manages its own lifecycle. - -**v0.10.1 — Native Dialog Audit** +**v0.11.1 — Native Dialog Audit** Replace `prompt()`/`confirm()`/`alert()` with `sw.dialog` SDK primitives. -Accessible, themed, promise-based. Small kernel SDK change that improves -every surface. -**v0.10.2 — Stability + Migration Tooling** +**v0.11.2 — Stability + Migration Tooling** -Proper versioned migrations (post-MVP discipline). `armature migrate` -CLI command. Backup/restore validation against reference dataset. -Pre-1.0 schema freeze. +Versioned migrations, `armature migrate` CLI, backup/restore validation, +pre-1.0 schema freeze. --- ### v1.0.0 — Stable Release -The kernel API surface is frozen. Extensions are the product. +Kernel API surface frozen. Extensions are the product. Gate criteria: - All kernel Starlark modules documented with examples - All `api_routes` covered by OpenAPI spec - Upgrade path tested from v0.8.0 → v1.0.0 -- At least 3 reference extensions shipped and tested (vector-store, llm-bridge, chat) +- At least 3 reference extensions shipped (vector-store, llm-bridge, chat) - At least 1 cross-extension composability demo (image-gen → chat:image-actions) - Headless E2E green on PG + SQLite - `armature-ca.sh` + mTLS deployment guide @@ -613,29 +261,14 @@ These are candidates, not commitments. Each requires a design doc. | Single Docker image | Go binary + assets + migrations. | | Admin → RBAC group | Grant check replaces role check. | | Settings cascade | Scope auth + `user_overridable`. Two orthogonal axes. | -| No new migrations pre-MVP | Proper versioned migrations post-MVP. | | Chat as extension, not kernel | Zero kernel awareness. Proves extensibility thesis. | | PG as consensus layer | UNLOGGED node_registry + LISTEN/NOTIFY. No etcd/Consul/Redis. | | Two trigger tiers | Extension-declared (full sandbox) vs user ad-hoc (restricted). | -| Builtin package rationale | Must enhance kernel surfaces or demonstrate platform capabilities. | -| Two-slot topbar model | Left slot (title/branding) + center slot (`flex: 1`, tabs/pickers). Two named slots cover every navigation pattern: simple title (Pattern A), flat tabs full-width (Pattern B), category tabs + surface-owned sidebar (Pattern C). Surfaces without sub-items get full content width; surfaces with hierarchical navigation add their own sidebar below the topbar. Shell provides the stage; surface decides the theater. | -| All four primary surfaces migrate to shell topbar | Admin was "keep custom topbar" initially. The two-slot model makes it unnecessary — category tabs fit in the center slot, sidebar is surface-owned below. One topbar implementation replaces four. Bell + user menu + reactivity come free on every surface. | -| Settings / Team Admin → flat tabs (Pattern B) | Both had thin sidebars (~140px) that consumed width without justification. 5–6 sections fit cleanly in topbar tabs. Full-width content is a better use of space for these surfaces. | -| Team Admin Groups removed | 37-line read-only dead-end. Admin Groups has full CRUD. Restore when properly designed. | -| Docs is the reference surface | Only surface with shell Topbar, bell, user menu, inline errors. Others converge. | -| Surface runners over expanding ICD | Different test tier, different failure class. | -| Headless E2E via Playwright | Runners produce structured JSON; Playwright navigates and reads output. | -| `db` module is the structured store | No separate KV primitive. Extensions declare tables — infrastructure exists. | -| `files` module rides ObjectStore | No new kernel tables. Metadata as companion objects. Works on PVC and S3 identically. | -| `workspace` for real filesystem | Flat blob store can't serve git/compilers/ffmpeg. Managed disk paths are the explicit escape hatch. | +| Two-slot topbar model | Left slot (title/branding) + center slot (`flex: 1`, tabs/pickers). | +| `db` module is the structured store | No separate KV primitive. Extensions declare tables. | +| `files` module rides ObjectStore | No new kernel tables. Metadata as companion objects. | +| `workspace` for real filesystem | Flat blob store can't serve git/compilers/ffmpeg. Managed disk paths. | | Capability negotiation at install | Fail loud with actionable message, not silently at runtime. | -| Vector column with three-tier fallback | Works everywhere, works fast with pgvector. Documentation > hard gates. | -| Docs refresh before composability | 8 versions of module additions (v0.7.5–v0.8.3) without a docs pass. Ship accurate docs before adding more API surface. | -| Sidecar deferred to v0.10.x | HTTP module covers external APIs. Local compute is a v0.10.x concern. Sidecars connect inward (like cluster nodes), not spawned outward (no k8s RBAC needed). | -| No second scripting runtime | Starlark for glue, HTTP for external compute, sidecars for local compute. Three tiers with clear boundaries. Lua/WASM middle tier rejected — Turing-complete enough to be dangerous, not isolated enough to be trustworthy. | -| Reference extensions prove the platform | First-party `.pkg` files, not kernel code. Uninstallable. Dogfooding. | -| Composability via slots + exports, not kernel | `sw.slots` + `sw.actions` + `lib.require()` are the composition primitives. Manifest declarations add admin visibility. No inter-extension message bus — too complex, too fragile. | -| Gut cleanup before kernel expansion | Codebase analysis found 9 smells from the chat gut. Fix before adding new primitives — dead views and broken templates compound when new code builds on them. | -| PATs over OAuth client credentials | PATs are simpler (one table, one middleware check), cover all use cases (E2E tests, CI, sidecars, user automation), and follow the git hosting convention users already understand. OAuth adds client registration, token exchange, scopes — complexity without benefit for a self-hosted platform. | -| Extension-declared user permissions | Extensions register custom permissions in the kernel's group system. Admin assigns them to groups. Backend enforces via `permissions.check()` or `gate_permission`. Frontend hides via `sw.can()`. The kernel owns the permission registry; extensions contribute to it. | -| Admin token as explicit action | Following Venice.ai pattern — admin token creation is a separate endpoint, audit logged, requires `surface.admin.access`. No ambiguity about the privilege level. | \ No newline at end of file +| Vector column with three-tier fallback | Works everywhere, works fast with pgvector. | +| Sidecar deferred to v0.11.x | HTTP module covers external APIs. Sidecars connect inward (no k8s RBAC needed). | +| Workflow redesign before reference extensions | Clean up debt and promote primitives before building on top. | diff --git a/docs/DESIGN-workflow-redesign.md b/docs/DESIGN-workflow-redesign.md new file mode 100644 index 0000000..e2afc8b --- /dev/null +++ b/docs/DESIGN-workflow-redesign.md @@ -0,0 +1,126 @@ +# DESIGN — Workflow System Redesign (0.9.x) + +**Version:** v0.9.0 +**Status:** Draft +**Author:** Jeff / Claude session 2026-04-03 + +--- + +## Problem + +The workflow system spans ~7,600 lines across 25+ files and was built +incrementally from v0.3.x through v0.7.10. It works, but several +concepts are redundant, primitives are buried, and the Starlark module +is read-only. Before shipping reference extensions (v0.10.x) that +build on workflows, the system needs cleanup and promotion of reusable +primitives. + +## Audit Summary + +Full audit in `/config/Downloads/AUDIT-workflow-0.9.md`. Classification: + +### KEEP — Core primitives that survive + +| Component | File(s) | Lines | Rationale | +|-----------|---------|-------|-----------| +| Engine core | `workflow/engine.go` | ~400 | Stateless state machine — Start, Advance, Cancel, version-pinned execution | +| Conditional routing | `workflow/routing.go` | ~500 | Pure functions, 8 operators, well-tested (497 lines of tests) | +| Automated processing | `workflow/automated.go` | ~300 | Starlark hook execution, cycle guard (max 10) | +| Scanner | `workflow/scanner.go` | ~200 | Background SLA + staleness checks, 5-minute interval | +| Signoff system | engine.go + handlers | ~400 | Multi-party approve/reject with quorum, role-gated | +| Assignment queue | handlers | ~300 | Claim/unclaim/complete/cancel lifecycle | +| Version snapshots | store + models | ~200 | Immutable snapshots, version-pinned instances | +| Typed form system | `models/workflow.go` | ~300 | 8 field types, fieldsets, conditional visibility | +| Store interface | `store/workflow_iface.go` | 61 methods | Complete lifecycle coverage | + +### OBE — Superseded by new design + +| Concept | Superseded by | +|---------|---------------| +| Per-stage `audience` field | Multi-page packages with mixed access declarations | +| `entry_mode` (public_link/team_only) | Package-level `scope: adoptable` + per-page access | +| `stage_mode` proliferation (4 values) | Collapse to 3: form / delegated / automated | +| `stage_type` (simple/dynamic/automated) | Redundant with `starlark_hook` presence check | +| `AdoptTeamWorkflow` clone | Package adoption model | +| `ExportWorkflowPackage` endpoint | Standard package export | + +### PROMOTE — Buried features to expose as kernel primitives + +1. **Roles → kernel primitive**: `team_user_roles` table, manifest `requires_roles`, + team admin UI, kernel middleware, Starlark SDK +2. **Typed forms → SDK primitive**: Move from workflow models to `forms` package, + FE SDK `sw.forms.render()` / `sw.forms.validate()` +3. **Conditional routing → SDK primitive**: Starlark `routing.evaluate(rules, data)` +4. **Workflow Starlark module → full read/write**: `workflow.start()`, `.advance()`, + `.cancel()`, `.submit_signoff()` + +--- + +## Technical Debt + +### Starlark Converter Duplication + +Three files contain nearly identical Go↔Starlark conversion functions: + +| File | Functions | +|------|-----------| +| `workflow/automated.go` | `goToStarlark`, `starlarkDictToMap`, `starlarkToGo` | +| `handlers/workflow_hooks.go` | `starlarkDictToMap`, `starlarkToGo`, `jsonToStarlark` | +| `sandbox/workflow_module.go` | `goValToStarlark` | + +**Fix:** Consolidate into `sandbox/convert.go`. + +### Snapshot Format Inconsistency + +Two formats exist (wrapped `{stages, workflow}` vs legacy flat array). +Three copies of the parser across engine, instance handlers, and +assignment handlers. + +**Fix:** Consolidate into one exported function. Standardize on wrapped format. + +--- + +## Sequencing + +| Version | Feature | Dependencies | +|---------|---------|-------------| +| **v0.9.0** | Starlark converter consolidation + snapshot format cleanup | None | +| **v0.9.1** | `team_user_roles` table + management API + admin UI | None | +| **v0.9.2** | `scope: adoptable` manifest field + adoption flow with role auto-populate | v0.9.1 | +| **v0.9.3** | Promote typed forms to SDK primitive (`sw.forms`) | None | +| **v0.9.4** | Deprecate `stage_type`, collapse `stage_mode` to 3 values | None | +| **v0.9.5** | Full read/write workflow Starlark module | v0.9.0 | +| **v0.9.6** | Conditional routing as SDK primitive | v0.9.5 | +| **v0.9.7** | Multi-surface manifest + kernel route resolution | None | +| **v0.9.8** | Wire roles into surface access (`access: role:X`) + kernel middleware | v0.9.1, v0.9.7 | + +Each step is CI-green independently. The first four are the high-value items. + +--- + +## Files Affected (by component) + +### Converter Consolidation (v0.9.0) +- New: `sandbox/convert.go` +- Modified: `workflow/automated.go`, `handlers/workflow_hooks.go`, `sandbox/workflow_module.go` + +### Team Roles (v0.9.1) +- New: migration (PG + SQLite), `store/team_roles.go`, `handlers/team_roles.go` +- Modified: manifest validation, adoption flow, team admin UI + +### Typed Forms (v0.9.3) +- New: `forms/` package (extracted from `models/workflow.go`) +- New: `sw.forms` SDK module +- Modified: workflow engine to import from `forms/` instead of inline + +### Workflow Starlark Module (v0.9.5) +- Modified: `sandbox/workflow_module.go` — add start/advance/cancel/signoff +- Modified: `workflow/engine.go` — extract interface to break circular import + +--- + +## Non-Goals + +- Rewriting the workflow engine. The state machine is correct and stays. +- Adding a visual workflow builder. That's an extension concern, not kernel. +- Multi-tenancy changes. Team scoping works as designed.