diff --git a/CHANGELOG.md b/CHANGELOG.md index b22eae7..432a94a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,62 @@ # Changelog +## [0.28.5] — 2026-03-14 + +### Summary + +Frontend SDK + pipe/filter pipeline. `switchboard-sdk.js` provides a +single-object composition layer (`sw`) over the 15+ platform globals. +Three-stage pipe/filter pipeline (pre-send, post-receive stream, +post-render) formalizes the ad-hoc extension hook model into composable, +priority-ordered, scope-aware transform chains. Backward-compatible +compat shim for existing `ctx.renderers.register()` extensions. ICD +runner gains `sdk` test tier (36 tests). + +### New + +- **`switchboard-sdk.js`** — SDK core. `Switchboard.init()` returns + `sw` singleton with identity (`sw.user`, `sw.isAdmin`), REST client + (`sw.api.get/post/put/del/stream`), events (`sw.on/once/off/emit`), + theme (`sw.theme.current/mode/set/on`), UI primitives + (`sw.toast/confirm/modal`), and component wrappers (`sw.chat()`). +- **Pipe/filter pipeline** — three composable stages: + - `sw.pipe.pre(priority, fn, opts)` — pre-send: transform user message + before LLM request. Runs on both `sendMessage()` and `regenerateMessage()`. + Context includes `regenerate` flag and `regenerateMessageId`. + - `sw.pipe.stream(priority, fn, opts)` — post-receive: transform each + SSE chunk during streaming. Sync-only for performance. + - `sw.pipe.render(priority, fn, opts)` — post-render: transform rendered + DOM after DOMPurify. Replaces `runExtensionPostRender()` delegation. +- **Filter scoping** — filters declare optional `scope: { channelType: [...] }` + to restrict execution to specific channel types. Scoped-out filters are + skipped with zero overhead. +- **Pipeline introspection** — `sw.pipe.list()` returns all registered + filters by stage with priority, source, scope, call count, avgMs, errors. +- **Extension compat shim** — `ctx.renderers.register(name, { type: 'post' })` + auto-shims into `sw.pipe.render()`. Existing block renderers (mermaid, + katex, csv, diff) continue working through the unchanged `formatMessage()` + path. Shimmed renderers appear in `sw.pipe.list()`. +- **ICD runner `sdk` tier** — 36 tests: boot, identity, REST client, + events, theme, pipe registration, execution ordering, scoping, halt + semantics, error isolation, compat shim, introspection. + +### Changed + +- `base.html` — SDK script tag added after component scripts. Universal + init block simplified to `Switchboard.init()` call. UserMenu band-aid + (cs15) absorbed into SDK `_hydrateUserMenu()`. +- `app.js` — `startApp()` calls `Switchboard.init()` (idempotent). +- `chat.js` — `sendMessage()` and `regenerateMessage()` run pre-send + pipe. New `_buildPreSendContext()` helper. +- `ui-core.js` — `streamResponse()` SSE loop runs stream pipe per chunk. +- `ui-format.js` — `runExtensionPostRender()` delegates to render pipe + with fallback to `Extensions.runPostRenderers()`. +- `extensions.js` — `_registerRenderer()` shims `type: 'post'` renderers + into `sw.pipe.render()`. +- ICD runner version bumped to 0.28.5.0, `sdk` tier added. + + + ## [0.28.4] — 2026-03-14 ### Summary diff --git a/VERSION b/VERSION index 097bc93..16b6bce 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.28.4 +0.28.5 diff --git a/docs/DESIGN-0.28.5.md b/docs/DESIGN-0.28.5.md new file mode 100644 index 0000000..4838db6 --- /dev/null +++ b/docs/DESIGN-0.28.5.md @@ -0,0 +1,754 @@ +# DESIGN — v0.28.5: Frontend SDK + Pipe/Filter Pipeline + +**Status:** Draft +**Scope:** `switchboard-sdk.js`, pipe/filter pipeline, extension compat shims, ICD `sdk` test tier +**Depends on:** v0.28.4 (security tier), v0.28.3 (frontend decomposition — `sb.js`, IIFE→module, `sb.call()`) +**Informs:** v0.28.7 (unified packaging — manifest `"pipes"` key), v0.29.0+ (Starlark pipeline stages) +**Current version:** 0.28.4 + +--- + +## Overview + +The platform has 47 JS files, 15+ globals (`API`, `Events`, `UI`, `App`, +`Extensions`, `ChatPane`, `Theme`, etc.), and a partially-decomposed +frontend (Phase 4 ES modules, `sb.register()`/`sb.ns()` action registry). +Surface authors and extension authors must hunt through multiple files, +understand load order, and hold implicit coupling in their heads. + +v0.28.5 delivers two things: + +1. **`switchboard-sdk.js` (the SDK)** — a single-object composition layer + (`sw`) over existing globals. One import, one init call, coherent API. + Does not rewrite internals; wraps and delegates. + +2. **Pipe/filter pipeline** — three composable, priority-ordered transform + stages that formalize the ad-hoc extension hooks into a real pipeline: + pre-send → post-receive (stream) → post-render. Extensions register + filters; the SDK wires them. Replaces `ctx.renderers.register()` and + the implicit hook points in `chat.js` / `ui-core.js` / `ui-format.js`. + +--- + +## Design Principles + +**Wrap, don't rewrite.** The SDK delegates to `API`, `Events`, `UI`, `App`, +`Theme`, `Extensions`, `ChatPane`. It does not duplicate their logic. If +`API.streamCompletion()` changes, the SDK's `sw.api.post()` still works +because it calls through. This means v0.28.5 doesn't risk destabilizing +the 531 passing CI tests. + +**The pipeline is the extension model.** The existing six hooks in +DESIGN-SURFACES.md (Stream Processing, Post-Render, Tool Bridge, etc.) +are replaced by three pipe stages plus the existing tool bridge. The old +`ctx.renderers.register()` becomes a compat shim that registers into +`sw.pipe.render()`. New extensions use `sw.pipe.*` directly. + +**Filters are pure transforms.** A filter receives a context object and +returns it (possibly modified). Side effects are allowed (logging, metrics) +but the contract is input→output. This makes filters testable in isolation. + +**Priority ordering is explicit.** Every filter declares a numeric priority. +Lower runs first. Ties break by registration order. The SDK enforces this +and exposes introspection (`sw.pipe.list()`). + +**Error isolation is mandatory.** One filter throwing must not break the +chain. The SDK catches, logs, records the failure in telemetry, and +continues with the next filter. A filter can explicitly halt the chain +by returning `null` (pre-send: message suppressed) or a `{ halt: true }` +signal. + +--- + +## SDK API Reference + +### Boot + +```js +// Idempotent. Safe to call multiple times (returns same instance). +// Typically called by the surface boot script after DOMContentLoaded. +const sw = Switchboard.init({ mount: '#app' }); +``` + +`init()` performs: +1. Token load (`API.loadTokens()`) +2. Profile resolution (`API.getProfile()`) +3. Theme init (`Theme.init()`) +4. EventBus connection (`Events.connect()`) +5. Pipeline engine init (empty filter chains) +6. UserMenu hydration (absorbs cs15 band-aid) +7. Emits `sw:ready` event on `document` + +If already initialized, returns the existing `sw` instance. The `mount` +option is reserved for future component auto-mounting but is not used in +v0.28.5 (surfaces manage their own DOM). + +### Identity + +```js +sw.user // { id, username, display_name, email, role, avatar } +sw.isAdmin // boolean (shorthand for sw.user.role === 'admin') +``` + +Read-only. Resolved during `init()`. Updates on `profile.updated` event. + +Delegates to: `API.user`, `API.isAdmin`. + +### REST Client + +```js +const channels = await sw.api.get('/api/v1/channels'); +const created = await sw.api.post('/api/v1/channels', { title, model }); +const updated = await sw.api.put('/api/v1/channels/123', { title }); +const deleted = await sw.api.del('/api/v1/channels/123'); + +// Streaming (returns raw Response for SSE consumption) +const resp = await sw.api.stream('/api/v1/chat/completions', body, signal); +``` + +All methods auto-prefix `BASE_PATH`, inject auth headers, handle 401 +refresh, and parse JSON. `sw.api.stream()` returns the raw `Response` +for SSE reading (same contract as `API.streamCompletion()` but generic). + +Delegates to: `API._get()`, `API._post()`, `API._put()`, `API._delete()`, +`API._fetch()`. + +### Events + +```js +const unsub = sw.on('chat.message.*', (payload, meta) => { ... }); +sw.once('workflow.completed', (payload) => { ... }); +sw.emit('custom.event', { data: 42 }); +sw.off('chat.message.*', handler); +``` + +Thin passthrough to `Events.on/once/off/emit`. No behavior change. +The SDK does not add any event namespacing — extensions already namespace +by convention (`ext.{id}.{event}`). + +Delegates to: `Events`. + +### Components + +```js +// Drop-in ChatPane — creates the component in the container +const pane = sw.chat(document.getElementById('myChat'), { + channelId: 'abc123', + standalone: true, +}); +pane.renderMessages(messages); +pane.destroy(); + +// Drop-in notes panel +const notes = sw.notes(document.getElementById('myNotes'), { + projectId: 'proj-1', +}); +``` + +`sw.chat()` wraps `ChatPane.create()` with SDK-aware defaults (auto-wires +pipe filters into the pane's stream/render path). `sw.notes()` wraps the +notes panel initialization from `notes.js`. + +Delegates to: `ChatPane.create()`, notes init functions. + +### UI Primitives + +```js +sw.toast('Saved', 'success'); // UI.toast() +const ok = await sw.confirm('Delete?'); // showConfirm() +sw.modal.open(contentEl); // openModal() +sw.modal.close(); // closeModal() +``` + +Delegates to: `UI.toast()`, `showConfirm()`, `openModal()`, `closeModal()`. + +### Theme + +```js +sw.theme.current // 'dark' | 'light' (resolved, never 'system') +sw.theme.mode // 'dark' | 'light' | 'system' (user preference) +sw.theme.set('dark'); // Theme.set() +sw.theme.on('change', (resolved) => { ... }); +``` + +`sw.theme.on('change', fn)` subscribes to the `theme.changed` EventBus +event and fires with the resolved theme string. + +Delegates to: `Theme.get()`, `Theme.resolved()`, `Theme.set()`, `Events.on('theme.changed', ...)`. + +--- + +## Pipe/Filter Pipeline + +### Architecture + +``` +User types message + │ + ▼ +┌─────────────────┐ +│ PRE-SEND chain │ sw.pipe.pre(priority, fn) +│ filter₁ → ··· → filterₙ +└────────┬────────┘ + │ context: { message, channel, attachments, model, persona, metadata } + ▼ + API.streamCompletion() + │ + ▼ SSE chunks +┌─────────────────┐ +│ STREAM chain │ sw.pipe.stream(priority, fn) +│ filter₁ → ··· → filterₙ (per chunk) +└────────┬────────┘ + │ context: { chunk, accumulated, channel, model, tokens } + ▼ + formatMessage() → DOMPurify + │ + ▼ rendered HTML +┌─────────────────┐ +│ RENDER chain │ sw.pipe.render(priority, fn) +│ filter₁ → ··· → filterₙ +└────────┬────────┘ + │ context: { element, message, channel } + ▼ + DOM inserted +``` + +### Registration API + +```js +// Pre-send: transform message before it hits the server +sw.pipe.pre(10, (ctx) => { + // Inject KB context + ctx.metadata.kb_chunks = getRelevantChunks(ctx.message); + return ctx; +}); + +// Pre-send with scope: only run on workflow channels +sw.pipe.pre(20, (ctx) => { + // Auto-tag workflow messages with stage metadata + ctx.metadata.workflow_stage = ctx.channel.stage; + return ctx; +}, { scope: { channelType: ['workflow'] } }); + +// Post-receive: transform each SSE chunk during streaming +sw.pipe.stream(50, (ctx) => { + // Token counting + tokenCounter.add(ctx.chunk); + ctx.tokens = tokenCounter.total; + return ctx; +}); + +// Post-render: transform rendered DOM +sw.pipe.render(50, (ctx) => { + // Replace mermaid code blocks with rendered diagrams + ctx.element.querySelectorAll('pre code.language-mermaid').forEach(el => { + renderMermaid(el); + }); + return ctx; +}); + +// Post-render scoped to DMs only +sw.pipe.render(60, (ctx) => { + // Highlight @mentions differently in DMs + highlightDmMentions(ctx.element); + return ctx; +}, { scope: { channelType: ['dm'] } }); + +// Introspection +sw.pipe.list(); +// → { pre: [{priority, source, scope, avgMs}], stream: [...], render: [...] } +``` + +### Filter Contract + +```typescript +// Type definitions (for documentation — runtime is plain JS) + +interface PreSendContext { + message: string; // User's text (mutable) + channel: { id: string; type: string; title: string }; + attachments: string[]; // File IDs + model: string; // Selected model ID + persona: string | null; // Persona ID if active + metadata: Record; // Extensible metadata bag + regenerate: boolean; // true if this is a regeneration + regenerateMessageId: string | null; // message ID being regenerated +} + +interface StreamContext { + chunk: string; // Current SSE delta text + accumulated: string; // Full response so far + channel: { id: string; type: string }; + model: string; + event: string; // SSE event type (content, tool_use, etc.) + tokens: { input: number; output: number }; // Running count +} + +interface RenderContext { + element: HTMLElement; // The message container DOM node (mutable) + message: { role: string; content: string; id?: string }; + channel: { id: string; type: string }; +} + +// Filter scope — restricts which channels a filter runs on. +// Omit or pass null for "all channels" (default). +interface FilterScope { + channelType?: string[]; // ['direct','dm','group','channel','workflow','service'] +} + +// Registration options (third argument to sw.pipe.pre/stream/render) +interface FilterOpts { + scope?: FilterScope; // Channel type restriction + source?: string; // Label for introspection (defaults to extension ID or 'platform') +} + +// All filters are synchronous. Return context to continue, null to halt. +type PipeFilter = (ctx: T) => T | null; +``` + +### Halt Semantics + +**Pre-send:** Returning `null` suppresses the message entirely. The SDK +calls `UI.toast('Message blocked by extension', 'warning')` and does not +call `API.streamCompletion()`. Use case: content policy filter. + +**Stream:** Returning `null` drops the current chunk (not displayed). The +accumulated buffer is unchanged. Use case: real-time content filter that +strips sensitive data from the stream. + +**Render:** Returning `null` skips remaining render filters for this +message. The DOM stays as-is after the last successful filter. Unusual +but available for guard-rail extensions. + +### Timing Telemetry + +The pipeline engine records per-filter execution time (monotonic clock). +Available via `sw.pipe.list()` and emitted on `pipe.stats` EventBus event +every 60 seconds (if any filters ran). Format: + +```json +{ + "pre": [{ "source": "kb-inject", "priority": 10, "calls": 47, "avgMs": 2.3, "errors": 0 }], + "stream": [{ "source": "token-counter", "priority": 50, "calls": 12840, "avgMs": 0.1, "errors": 0 }], + "render": [{ "source": "mermaid", "priority": 50, "calls": 47, "avgMs": 15.7, "errors": 1 }] +} +``` + +### Filter Scoping + +Filters declare an optional `scope` at registration time. The pipeline +engine checks scope before invoking a filter — scoped-out filters are +skipped with zero overhead (no function call, no timing entry). + +```js +// Only run on workflow and service channels +sw.pipe.pre(10, myFilter, { scope: { channelType: ['workflow', 'service'] } }); + +// Only run on direct (1:1 AI) chats +sw.pipe.stream(50, tokenCounter, { scope: { channelType: ['direct'] } }); + +// No scope = runs on everything (default) +sw.pipe.render(50, mermaidRenderer); +``` + +**Channel types** (from the `channels` table `type` column): +`direct`, `dm`, `group`, `channel`, `workflow`, `service`. + +**Scope evaluation:** Before each filter call, the engine reads +`ctx.channel.type` and checks against the filter's `scope.channelType` +array. If the type is not in the array, the filter is skipped. If the +filter has no scope (or `scope` is `null`), it always runs. + +**Introspection:** `sw.pipe.list()` includes the `scope` object for +each filter so extensions can inspect which filters apply to their +channel type. + +**Manifest mapping (v0.28.7):** The `"pipes"` manifest key will support +the same `scope` object: + +```json +{ + "pipes": [ + { + "stage": "pre", + "priority": 10, + "handler": "filters.kbInject", + "scope": { "channelType": ["direct", "dm", "group"] } + } + ] +} +``` + +--- + +## Integration Points + +### Pre-send: `chat.js` → `sendMessage()` + `regenerateMessage()` + +**`sendMessage()` — current flow (chat.js line ~659):** +``` +text = ChatInput.getValue() +→ build body +→ API.streamCompletion(channelId, text, model, ...) +``` + +**`sendMessage()` — new flow:** +``` +text = ChatInput.getValue() +→ build PreSendContext { message, channel, ..., regenerate: false } +→ sw.pipe._runPre(context) // returns modified context or null +→ if null: toast + return +→ API.streamCompletion(ctx.channel.id, ctx.message, ctx.model, ...) +``` + +**`regenerateMessage()` — current flow (chat.js line ~835):** +``` +messageId from clicked button +→ API.streamRegenerate(channelId, messageId, model, ...) +``` + +**`regenerateMessage()` — new flow:** +``` +messageId from clicked button +→ build PreSendContext { message: '', channel, ..., regenerate: true, regenerateMessageId: messageId } +→ sw.pipe._runPre(context) +→ if null: toast + return +→ API.streamRegenerate(ctx.channel.id, messageId, ctx.model, ...) +``` + +Both paths share the same filter chain. The `regenerate` flag and +`regenerateMessageId` let filters distinguish — a content policy filter +runs on both, a KB inject filter might skip regeneration (the original +context is already in the conversation), a logging filter records both. + +The `sendMessage()` function gains ~10 lines. `regenerateMessage()` gains +~8 lines. Extracted helper: `_buildPreSendContext(opts)` constructs the +context object for both callers. + +**Metadata passthrough:** The `metadata` bag on `PreSendContext` maps to +the completion request body. Pre-send filters can inject KB chunks, +persona instructions, or custom fields that the backend completion +handler reads from the request. KB auto-inject (v0.28.6) will be the +first platform-level filter using this mechanism. + +### Post-receive (stream): `ui-core.js` → `UI.streamResponse()` + +Current flow (ui-core.js line ~1060): +``` +SSE reader loop: + parse chunk → extract delta + content += delta + activeContentEl.innerHTML = formatMessage(content) +``` + +New flow: +``` +SSE reader loop: + parse chunk → extract delta + streamCtx = { chunk: delta, accumulated: content, ... } + streamCtx = sw.pipe._runStream(streamCtx) + if streamCtx is null: continue (chunk dropped) + content = streamCtx.accumulated // filter may have mutated + activeContentEl.innerHTML = formatMessage(content) +``` + +The pipe runs synchronously per chunk (filters that need async should +buffer internally). The `accumulated` field lets filters see the full +response so far (for context-dependent transforms like citation +injection that need to see paragraph boundaries). + +**Performance:** Stream filters run on every SSE chunk (potentially +hundreds per response). The telemetry tracks per-filter avgMs. Filters +exceeding 5ms average get a console warning. The pipeline engine skips +disabled filters without allocation. + +### Post-render: `ui-format.js` + `ui-core.js` + +Current flow: +``` +formatMessage(content) // marked.js + DOMPurify +→ Extensions.runBlockRenderers() // inside formatMessage, per code block +→ el.innerHTML = html +→ runExtensionPostRender(el) // Extensions.runPostRenderers() +``` + +New flow: +``` +formatMessage(content) // unchanged +→ el.innerHTML = html +→ renderCtx = { element: el, message, channel } +→ sw.pipe._runRender(renderCtx) // subsumes runExtensionPostRender +``` + +The render pipe replaces both `Extensions.runBlockRenderers()` and +`Extensions.runPostRenderers()`. The distinction between "block" and +"post" renderer types collapses into a single render stage — filters +operate on the full message DOM and can target code blocks, inline +elements, or the entire container. + +**Block renderer compat:** Extensions that register block renderers via +the old `ctx.renderers.register(name, { type: 'block', ... })` API get +automatically shimmed into `sw.pipe.render()` filters. The shim wraps +the old `(lang, code, container)` signature into the new +`(renderCtx) => ...` contract. Priority is preserved. + +--- + +## Backward Compatibility + +### Extension Context Shim + +The existing `Extensions._buildContext()` returns a `ctx` object with +`ctx.renderers.register()`, `ctx.tools.handle()`, `ctx.events.*`, +`ctx.ui.*`, `ctx.storage.*`, and `ctx.api.*`. + +All of these continue to work. The shim layer: + +| Old API | Shim behavior | +|---------|--------------| +| `ctx.renderers.register(name, { type: 'block', ... })` | Wraps into `sw.pipe.render()` filter with matching priority | +| `ctx.renderers.register(name, { type: 'post', ... })` | Wraps into `sw.pipe.render()` filter | +| `ctx.renderers.register(name, { type: 'inline', ... })` | Wraps into `sw.pipe.render()` filter (inline → DOM manipulation) | +| `ctx.tools.handle(name, fn)` | Unchanged — tool bridge is orthogonal to pipes | +| `ctx.events.*` | Unchanged — passthrough to `Events` | +| `ctx.ui.*` | Unchanged — passthrough to `UI` / `showConfirm` / `Theme` | +| `ctx.storage.*` | Unchanged — scoped `localStorage` | +| `ctx.api.*` | Unchanged — passthrough to `API._fetch` | +| `ctx.surfaces.*` | Unchanged (already stub/warn for removed features) | + +The shim is invisible to existing extensions. They register the same way; +the pipeline engine sees their filters alongside new `sw.pipe.*` filters +and orders everything by priority. + +### Global Availability + +`sb.js` continues to work. `window.API`, `window.Events`, `window.UI`, +etc. remain available. The SDK is additive — it does not remove globals +or break direct access. Phase 5 (ES module `import`/`export` migration) +is when globals start going away, and the SDK becomes the only entry point. + +### Load Order + +``` +base.html: + sb.js ← action registry (unchanged) + app-state.js ← App, fetchModels (unchanged) + api.js ← API (unchanged) + events.js ← Events (unchanged) + ui-primitives.js ← Theme, showConfirm, etc. (unchanged) + ui-format.js ← formatMessage (patched: render pipe integration) + ui-core.js ← UI (patched: stream pipe integration) + switchboard-sdk.js ← NEW: Switchboard object, pipe engine + extensions.js ← Extensions (patched: renderer shim) + ... surface-specific JS ... + chat.js ← sendMessage (patched: pre-send pipe) + app.js ← init (patched: calls Switchboard.init) +``` + +`switchboard-sdk.js` loads after all primitives/globals it wraps and +before `extensions.js` (so the shim layer is ready when extensions +register renderers). `app.js` `init()` calls `Switchboard.init()` as +part of its boot sequence; the SDK handles idempotency. + +--- + +## File Inventory + +### New Files + +| File | Purpose | +|------|---------| +| `src/js/switchboard-sdk.js` | SDK object, pipe engine, compat shims | +| `surfaces/icd-test-runner/js/tier-sdk.js` | ICD runner `sdk` test tier | +| `docs/DESIGN-0.28.5.md` | This document (archived after merge) | + +### Modified Files + +| File | Change | +|------|--------| +| `src/js/chat.js` | `sendMessage()`: pre-send pipe insertion | +| `src/js/ui-core.js` | `streamResponse()`: stream pipe insertion | +| `src/js/ui-format.js` | `runExtensionPostRender()`: delegates to render pipe | +| `src/js/extensions.js` | `_registerRenderer()`: shim into `sw.pipe.render()` | +| `src/js/app.js` | `init()`: call `Switchboard.init()` during boot | +| `src/templates/base.html` | Add ` + {{/* v0.28.5: SDK — composition layer over globals. Must load after all + primitives/components it wraps, before extensions.js. */}} + - {{/* ── Universal init: theme + appearance for ALL surfaces ── */}} + {{/* ── Universal init: SDK boot for ALL surfaces ── */}} {{end}} - {{/* ── v0.28.0: Universal UserMenu hydration ────────────────── - Runs after all surface-specific scripts. Creates and binds the - UserMenu component if no surface script did it already. - - Guard: chat surface excluded — it uses UI.toggleUserMenu() with - richer behavior (team admin refresh). Chat migration to UserMenu - component is tracked as tech debt (Pre-1.0 sweep). - */}} - + {{/* v0.28.5: UserMenu hydration moved into Switchboard.init() (switchboard-sdk.js). + The SDK handles create/bind/setUser for all non-chat surfaces automatically. */}} {{/* ── Debug Log Modal (all surfaces) ───── */}}