# DESIGN — v0.28.5: Frontend SDK + Pipe/Filter Pipeline **Status:** Draft **Scope:** `switchboard-sdk.js`, pipe/filter pipeline, extension compat shims, ICD `sdk` test tier **Depends on:** v0.28.4 (security tier), v0.28.3 (frontend decomposition — `sb.js`, IIFE→module, `sb.call()`) **Informs:** v0.28.7 (unified packaging — manifest `"pipes"` key), v0.29.0+ (Starlark pipeline stages) **Current version:** 0.28.4 --- ## Overview The platform has 47 JS files, 15+ globals (`API`, `Events`, `UI`, `App`, `Extensions`, `ChatPane`, `Theme`, etc.), and a partially-decomposed frontend (Phase 4 ES modules, `sb.register()`/`sb.ns()` action registry). Surface authors and extension authors must hunt through multiple files, understand load order, and hold implicit coupling in their heads. v0.28.5 delivers two things: 1. **`switchboard-sdk.js` (the SDK)** — a single-object composition layer (`sw`) over existing globals. One import, one init call, coherent API. Does not rewrite internals; wraps and delegates. 2. **Pipe/filter pipeline** — three composable, priority-ordered transform stages that formalize the ad-hoc extension hooks into a real pipeline: pre-send → post-receive (stream) → post-render. Extensions register filters; the SDK wires them. Replaces `ctx.renderers.register()` and the implicit hook points in `chat.js` / `ui-core.js` / `ui-format.js`. --- ## Design Principles **Wrap, don't rewrite.** The SDK delegates to `API`, `Events`, `UI`, `App`, `Theme`, `Extensions`, `ChatPane`. It does not duplicate their logic. If `API.streamCompletion()` changes, the SDK's `sw.api.post()` still works because it calls through. This means v0.28.5 doesn't risk destabilizing the 531 passing CI tests. **The pipeline is the extension model.** The existing six hooks in DESIGN-SURFACES.md (Stream Processing, Post-Render, Tool Bridge, etc.) are replaced by three pipe stages plus the existing tool bridge. The old `ctx.renderers.register()` becomes a compat shim that registers into `sw.pipe.render()`. New extensions use `sw.pipe.*` directly. **Filters are pure transforms.** A filter receives a context object and returns it (possibly modified). Side effects are allowed (logging, metrics) but the contract is input→output. This makes filters testable in isolation. **Priority ordering is explicit.** Every filter declares a numeric priority. Lower runs first. Ties break by registration order. The SDK enforces this and exposes introspection (`sw.pipe.list()`). **Error isolation is mandatory.** One filter throwing must not break the chain. The SDK catches, logs, records the failure in telemetry, and continues with the next filter. A filter can explicitly halt the chain by returning `null` (pre-send: message suppressed) or a `{ halt: true }` signal. --- ## SDK API Reference ### Boot ```js // Idempotent. Safe to call multiple times (returns same instance). // Typically called by the surface boot script after DOMContentLoaded. const sw = Switchboard.init({ mount: '#app' }); ``` `init()` performs: 1. Token load (`API.loadTokens()`) 2. Profile resolution (`API.getProfile()`) 3. Theme init (`Theme.init()`) 4. EventBus connection (`Events.connect()`) 5. Pipeline engine init (empty filter chains) 6. UserMenu hydration (absorbs cs15 band-aid) 7. Emits `sw:ready` event on `document` If already initialized, returns the existing `sw` instance. The `mount` option is reserved for future component auto-mounting but is not used in v0.28.5 (surfaces manage their own DOM). ### Identity ```js sw.user // { id, username, display_name, email, role, avatar } sw.isAdmin // boolean (shorthand for sw.user.role === 'admin') ``` Read-only. Resolved during `init()`. Updates on `profile.updated` event. Delegates to: `API.user`, `API.isAdmin`. ### REST Client ```js const channels = await sw.api.get('/api/v1/channels'); const created = await sw.api.post('/api/v1/channels', { title, model }); const updated = await sw.api.put('/api/v1/channels/123', { title }); const deleted = await sw.api.del('/api/v1/channels/123'); // Streaming (returns raw Response for SSE consumption) const resp = await sw.api.stream('/api/v1/chat/completions', body, signal); ``` All methods auto-prefix `BASE_PATH`, inject auth headers, handle 401 refresh, and parse JSON. `sw.api.stream()` returns the raw `Response` for SSE reading (same contract as `API.streamCompletion()` but generic). Delegates to: `API._get()`, `API._post()`, `API._put()`, `API._delete()`, `API._fetch()`. ### Events ```js const unsub = sw.on('chat.message.*', (payload, meta) => { ... }); sw.once('workflow.completed', (payload) => { ... }); sw.emit('custom.event', { data: 42 }); sw.off('chat.message.*', handler); ``` Thin passthrough to `Events.on/once/off/emit`. No behavior change. The SDK does not add any event namespacing — extensions already namespace by convention (`ext.{id}.{event}`). Delegates to: `Events`. ### Components ```js // Drop-in ChatPane — creates the component in the container const pane = sw.chat(document.getElementById('myChat'), { channelId: 'abc123', standalone: true, }); pane.renderMessages(messages); pane.destroy(); // Drop-in notes panel const notes = sw.notes(document.getElementById('myNotes'), { projectId: 'proj-1', }); ``` `sw.chat()` wraps `ChatPane.create()` with SDK-aware defaults (auto-wires pipe filters into the pane's stream/render path). `sw.notes()` wraps the notes panel initialization from `notes.js`. Delegates to: `ChatPane.create()`, notes init functions. ### UI Primitives ```js sw.toast('Saved', 'success'); // UI.toast() const ok = await sw.confirm('Delete?'); // showConfirm() sw.modal.open(contentEl); // openModal() sw.modal.close(); // closeModal() ``` Delegates to: `UI.toast()`, `showConfirm()`, `openModal()`, `closeModal()`. ### Theme ```js sw.theme.current // 'dark' | 'light' (resolved, never 'system') sw.theme.mode // 'dark' | 'light' | 'system' (user preference) sw.theme.set('dark'); // Theme.set() sw.theme.on('change', (resolved) => { ... }); ``` `sw.theme.on('change', fn)` subscribes to the `theme.changed` EventBus event and fires with the resolved theme string. Delegates to: `Theme.get()`, `Theme.resolved()`, `Theme.set()`, `Events.on('theme.changed', ...)`. --- ## Pipe/Filter Pipeline ### Architecture ``` User types message │ ▼ ┌─────────────────┐ │ PRE-SEND chain │ sw.pipe.pre(priority, fn) │ filter₁ → ··· → filterₙ └────────┬────────┘ │ context: { message, channel, attachments, model, persona, metadata } ▼ API.streamCompletion() │ ▼ SSE chunks ┌─────────────────┐ │ STREAM chain │ sw.pipe.stream(priority, fn) │ filter₁ → ··· → filterₙ (per chunk) └────────┬────────┘ │ context: { chunk, accumulated, channel, model, tokens } ▼ formatMessage() → DOMPurify │ ▼ rendered HTML ┌─────────────────┐ │ RENDER chain │ sw.pipe.render(priority, fn) │ filter₁ → ··· → filterₙ └────────┬────────┘ │ context: { element, message, channel } ▼ DOM inserted ``` ### Registration API ```js // Pre-send: transform message before it hits the server sw.pipe.pre(10, (ctx) => { // Inject KB context ctx.metadata.kb_chunks = getRelevantChunks(ctx.message); return ctx; }); // Pre-send with scope: only run on workflow channels sw.pipe.pre(20, (ctx) => { // Auto-tag workflow messages with stage metadata ctx.metadata.workflow_stage = ctx.channel.stage; return ctx; }, { scope: { channelType: ['workflow'] } }); // Post-receive: transform each SSE chunk during streaming sw.pipe.stream(50, (ctx) => { // Token counting tokenCounter.add(ctx.chunk); ctx.tokens = tokenCounter.total; return ctx; }); // Post-render: transform rendered DOM sw.pipe.render(50, (ctx) => { // Replace mermaid code blocks with rendered diagrams ctx.element.querySelectorAll('pre code.language-mermaid').forEach(el => { renderMermaid(el); }); return ctx; }); // Post-render scoped to DMs only sw.pipe.render(60, (ctx) => { // Highlight @mentions differently in DMs highlightDmMentions(ctx.element); return ctx; }, { scope: { channelType: ['dm'] } }); // Introspection sw.pipe.list(); // → { pre: [{priority, source, scope, avgMs}], stream: [...], render: [...] } ``` ### Filter Contract ```typescript // Type definitions (for documentation — runtime is plain JS) interface PreSendContext { message: string; // User's text (mutable) channel: { id: string; type: string; title: string }; attachments: string[]; // File IDs model: string; // Selected model ID persona: string | null; // Persona ID if active metadata: Record; // Extensible metadata bag regenerate: boolean; // true if this is a regeneration regenerateMessageId: string | null; // message ID being regenerated } interface StreamContext { chunk: string; // Current SSE delta text accumulated: string; // Full response so far channel: { id: string; type: string }; model: string; event: string; // SSE event type (content, tool_use, etc.) tokens: { input: number; output: number }; // Running count } interface RenderContext { element: HTMLElement; // The message container DOM node (mutable) message: { role: string; content: string; id?: string }; channel: { id: string; type: string }; } // Filter scope — restricts which channels a filter runs on. // Omit or pass null for "all channels" (default). interface FilterScope { channelType?: string[]; // ['direct','dm','group','channel','workflow','service'] } // Registration options (third argument to sw.pipe.pre/stream/render) interface FilterOpts { scope?: FilterScope; // Channel type restriction source?: string; // Label for introspection (defaults to extension ID or 'platform') } // All filters are synchronous. Return context to continue, null to halt. type PipeFilter = (ctx: T) => T | null; ``` ### Halt Semantics **Pre-send:** Returning `null` suppresses the message entirely. The SDK calls `UI.toast('Message blocked by extension', 'warning')` and does not call `API.streamCompletion()`. Use case: content policy filter. **Stream:** Returning `null` drops the current chunk (not displayed). The accumulated buffer is unchanged. Use case: real-time content filter that strips sensitive data from the stream. **Render:** Returning `null` skips remaining render filters for this message. The DOM stays as-is after the last successful filter. Unusual but available for guard-rail extensions. ### Timing Telemetry The pipeline engine records per-filter execution time (monotonic clock). Available via `sw.pipe.list()` and emitted on `pipe.stats` EventBus event every 60 seconds (if any filters ran). Format: ```json { "pre": [{ "source": "kb-inject", "priority": 10, "calls": 47, "avgMs": 2.3, "errors": 0 }], "stream": [{ "source": "token-counter", "priority": 50, "calls": 12840, "avgMs": 0.1, "errors": 0 }], "render": [{ "source": "mermaid", "priority": 50, "calls": 47, "avgMs": 15.7, "errors": 1 }] } ``` ### Filter Scoping Filters declare an optional `scope` at registration time. The pipeline engine checks scope before invoking a filter — scoped-out filters are skipped with zero overhead (no function call, no timing entry). ```js // Only run on workflow and service channels sw.pipe.pre(10, myFilter, { scope: { channelType: ['workflow', 'service'] } }); // Only run on direct (1:1 AI) chats sw.pipe.stream(50, tokenCounter, { scope: { channelType: ['direct'] } }); // No scope = runs on everything (default) sw.pipe.render(50, mermaidRenderer); ``` **Channel types** (from the `channels` table `type` column): `direct`, `dm`, `group`, `channel`, `workflow`, `service`. **Scope evaluation:** Before each filter call, the engine reads `ctx.channel.type` and checks against the filter's `scope.channelType` array. If the type is not in the array, the filter is skipped. If the filter has no scope (or `scope` is `null`), it always runs. **Introspection:** `sw.pipe.list()` includes the `scope` object for each filter so extensions can inspect which filters apply to their channel type. **Manifest mapping (v0.28.7):** The `"pipes"` manifest key will support the same `scope` object: ```json { "pipes": [ { "stage": "pre", "priority": 10, "handler": "filters.kbInject", "scope": { "channelType": ["direct", "dm", "group"] } } ] } ``` --- ## Integration Points ### Pre-send: `chat.js` → `sendMessage()` + `regenerateMessage()` **`sendMessage()` — current flow (chat.js line ~659):** ``` text = ChatInput.getValue() → build body → API.streamCompletion(channelId, text, model, ...) ``` **`sendMessage()` — new flow:** ``` text = ChatInput.getValue() → build PreSendContext { message, channel, ..., regenerate: false } → sw.pipe._runPre(context) // returns modified context or null → if null: toast + return → API.streamCompletion(ctx.channel.id, ctx.message, ctx.model, ...) ``` **`regenerateMessage()` — current flow (chat.js line ~835):** ``` messageId from clicked button → API.streamRegenerate(channelId, messageId, model, ...) ``` **`regenerateMessage()` — new flow:** ``` messageId from clicked button → build PreSendContext { message: '', channel, ..., regenerate: true, regenerateMessageId: messageId } → sw.pipe._runPre(context) → if null: toast + return → API.streamRegenerate(ctx.channel.id, messageId, ctx.model, ...) ``` Both paths share the same filter chain. The `regenerate` flag and `regenerateMessageId` let filters distinguish — a content policy filter runs on both, a KB inject filter might skip regeneration (the original context is already in the conversation), a logging filter records both. The `sendMessage()` function gains ~10 lines. `regenerateMessage()` gains ~8 lines. Extracted helper: `_buildPreSendContext(opts)` constructs the context object for both callers. **Metadata passthrough:** The `metadata` bag on `PreSendContext` maps to the completion request body. Pre-send filters can inject KB chunks, persona instructions, or custom fields that the backend completion handler reads from the request. KB auto-inject (v0.28.6) will be the first platform-level filter using this mechanism. ### Post-receive (stream): `ui-core.js` → `UI.streamResponse()` Current flow (ui-core.js line ~1060): ``` SSE reader loop: parse chunk → extract delta content += delta activeContentEl.innerHTML = formatMessage(content) ``` New flow: ``` SSE reader loop: parse chunk → extract delta streamCtx = { chunk: delta, accumulated: content, ... } streamCtx = sw.pipe._runStream(streamCtx) if streamCtx is null: continue (chunk dropped) content = streamCtx.accumulated // filter may have mutated activeContentEl.innerHTML = formatMessage(content) ``` The pipe runs synchronously per chunk (filters that need async should buffer internally). The `accumulated` field lets filters see the full response so far (for context-dependent transforms like citation injection that need to see paragraph boundaries). **Performance:** Stream filters run on every SSE chunk (potentially hundreds per response). The telemetry tracks per-filter avgMs. Filters exceeding 5ms average get a console warning. The pipeline engine skips disabled filters without allocation. ### Post-render: `ui-format.js` + `ui-core.js` Current flow: ``` formatMessage(content) // marked.js + DOMPurify → Extensions.runBlockRenderers() // inside formatMessage, per code block → el.innerHTML = html → runExtensionPostRender(el) // Extensions.runPostRenderers() ``` New flow: ``` formatMessage(content) // unchanged → el.innerHTML = html → renderCtx = { element: el, message, channel } → sw.pipe._runRender(renderCtx) // subsumes runExtensionPostRender ``` The render pipe replaces both `Extensions.runBlockRenderers()` and `Extensions.runPostRenderers()`. The distinction between "block" and "post" renderer types collapses into a single render stage — filters operate on the full message DOM and can target code blocks, inline elements, or the entire container. **Block renderer compat:** Extensions that register block renderers via the old `ctx.renderers.register(name, { type: 'block', ... })` API get automatically shimmed into `sw.pipe.render()` filters. The shim wraps the old `(lang, code, container)` signature into the new `(renderCtx) => ...` contract. Priority is preserved. --- ## Backward Compatibility ### Extension Context Shim The existing `Extensions._buildContext()` returns a `ctx` object with `ctx.renderers.register()`, `ctx.tools.handle()`, `ctx.events.*`, `ctx.ui.*`, `ctx.storage.*`, and `ctx.api.*`. All of these continue to work. The shim layer: | Old API | Shim behavior | |---------|--------------| | `ctx.renderers.register(name, { type: 'block', ... })` | Wraps into `sw.pipe.render()` filter with matching priority | | `ctx.renderers.register(name, { type: 'post', ... })` | Wraps into `sw.pipe.render()` filter | | `ctx.renderers.register(name, { type: 'inline', ... })` | Wraps into `sw.pipe.render()` filter (inline → DOM manipulation) | | `ctx.tools.handle(name, fn)` | Unchanged — tool bridge is orthogonal to pipes | | `ctx.events.*` | Unchanged — passthrough to `Events` | | `ctx.ui.*` | Unchanged — passthrough to `UI` / `showConfirm` / `Theme` | | `ctx.storage.*` | Unchanged — scoped `localStorage` | | `ctx.api.*` | Unchanged — passthrough to `API._fetch` | | `ctx.surfaces.*` | Unchanged (already stub/warn for removed features) | The shim is invisible to existing extensions. They register the same way; the pipeline engine sees their filters alongside new `sw.pipe.*` filters and orders everything by priority. ### Global Availability `sb.js` continues to work. `window.API`, `window.Events`, `window.UI`, etc. remain available. The SDK is additive — it does not remove globals or break direct access. Phase 5 (ES module `import`/`export` migration) is when globals start going away, and the SDK becomes the only entry point. ### Load Order ``` base.html: sb.js ← action registry (unchanged) app-state.js ← App, fetchModels (unchanged) api.js ← API (unchanged) events.js ← Events (unchanged) ui-primitives.js ← Theme, showConfirm, etc. (unchanged) ui-format.js ← formatMessage (patched: render pipe integration) ui-core.js ← UI (patched: stream pipe integration) switchboard-sdk.js ← NEW: Switchboard object, pipe engine extensions.js ← Extensions (patched: renderer shim) ... surface-specific JS ... chat.js ← sendMessage (patched: pre-send pipe) app.js ← init (patched: calls Switchboard.init) ``` `switchboard-sdk.js` loads after all primitives/globals it wraps and before `extensions.js` (so the shim layer is ready when extensions register renderers). `app.js` `init()` calls `Switchboard.init()` as part of its boot sequence; the SDK handles idempotency. --- ## File Inventory ### New Files | File | Purpose | |------|---------| | `src/js/switchboard-sdk.js` | SDK object, pipe engine, compat shims | | `surfaces/icd-test-runner/js/tier-sdk.js` | ICD runner `sdk` test tier | | `docs/DESIGN-0.28.5.md` | This document (archived after merge) | ### Modified Files | File | Change | |------|--------| | `src/js/chat.js` | `sendMessage()`: pre-send pipe insertion | | `src/js/ui-core.js` | `streamResponse()`: stream pipe insertion | | `src/js/ui-format.js` | `runExtensionPostRender()`: delegates to render pipe | | `src/js/extensions.js` | `_registerRenderer()`: shim into `sw.pipe.render()` | | `src/js/app.js` | `init()`: call `Switchboard.init()` during boot | | `src/templates/base.html` | Add `