This repository has been archived on 2026-04-03. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
core/EXTENSIONS.md
2026-02-21 21:59:38 +00:00

637 lines
21 KiB
Markdown

# Chat Switchboard — Extension System Specification
**Version:** 0.1 draft
**Status:** Design
**Applies to:** v0.7.x+
**Companion to:** ARCHITECTURE.md (core services + terminology)
---
## 1. Philosophy
Chat Switchboard is a substrate, not an application. The core provides:
authentication, provider routing, message persistence, an event bus, and a
rendering surface. Everything else — editing, writing, cluster management,
cost tracking, custom renderers — is an extension.
The goal is that someone with a problem and some JS (or Go, or Python) can
solve it without forking the project. The "modes" Jeff envisions — Editor,
Article, Chat, Cluster Manager — are just extensions that register surfaces,
tools, and event handlers. This project doesn't have to build all of them.
It just has to make them possible.
---
## 2. Extension Tiers
| | Tier 0: Browser | Tier 1: Starlark | Tier 2: Sidecar |
|---|---|---|---|
| **Runs in** | User's browser | Go server (embedded) | Separate container |
| **Language** | JavaScript | Starlark (Python subset) | Any |
| **Deployed by** | User or Admin push | Admin | Admin |
| **Latency** | Zero (client-side) | Low (in-process) | Network hop |
| **Can access** | DOM, EventBus, LocalStorage, user context | Message data, DB reads (sandboxed) | Anything (HTTP, filesystem, network) |
| **Cannot access** | Server internals, other users' data | Network, filesystem, raw SQL | N/A (full access) |
| **Trust model** | Same-origin; user-scoped or admin-pushed | Starlark sandbox (no I/O) | Container isolation |
| **Use cases** | UI, rendering, shortcuts, client tools, modes | Routing rules, message transforms, logging | RAG, external APIs, webhooks, heavy compute |
All three tiers communicate through the EventBus. A browser extension
publishes `tool.result.{callId}` the same way a sidecar does — the bus
doesn't care where the event originated.
---
## 3. Manifest Format
Every extension, regardless of tier, is described by a manifest:
```json
{
"id": "cost-tracker",
"name": "Cost Tracker",
"version": "1.0.0",
"tier": "browser",
"author": "jeff",
"description": "Real-time token counting and cost estimation",
"permissions": [
"events:chat.message.*",
"events:model.selected",
"dom:input-area",
"storage:local"
],
"entry": "cost-tracker.js",
"hooks": {
"chat.message.send": { "priority": 10, "async": false },
"chat.message.received": { "priority": 50, "async": true }
},
"tools": [],
"surfaces": [],
"settings": {
"showInline": {
"type": "boolean",
"label": "Show cost inline",
"default": true
}
}
}
```
### 3.1 Fields
- **id**: Unique identifier. Namespaced by convention (`jeff.cost-tracker`).
- **tier**: `browser` | `starlark` | `sidecar`
- **permissions**: What the extension needs access to. The loader enforces
these. Undeclared access is blocked (Tier 0 via proxy, Tier 1 via sandbox,
Tier 2 via API scoping).
- **entry**: For browser: JS file path. For starlark: `.star` file. For
sidecar: endpoint URL or Docker image.
- **hooks**: EventBus events this extension subscribes to, with priority
(lower = runs first) and whether the hook is async.
- **tools**: LLM-callable tools this extension provides (see §5).
- **surfaces**: UI surfaces this extension registers (see §6).
- **settings**: User-configurable options, rendered as a form in the
extension settings UI.
---
## 4. Browser Extensions (Tier 0)
### 4.1 Lifecycle
```
Install → Load → Init → Active → Disable → Unload
```
**Install**: Admin pushes manifest + JS to server, or user adds from
settings. Stored in `extensions` table with `tier = 'browser'`.
**Load**: On page load, after `events.js` but before `app.js`, the
extension loader injects `<script>` tags for all enabled browser extensions.
Load order respects declared dependencies.
**Init**: Each extension's entry script calls `Extensions.register()`:
```js
Extensions.register({
id: 'cost-tracker',
init(ctx) {
// ctx.events — scoped EventBus (only permitted events)
// ctx.storage — scoped localStorage wrapper
// ctx.settings — this extension's settings values
// ctx.ui — DOM injection points
this.ctx = ctx;
ctx.events.on('chat.message.send', (msg) => {
const est = this.estimateTokens(msg.content);
ctx.ui.inject('input-area', this.renderCost(est));
});
ctx.events.on('model.selected', ({ capabilities }) => {
this.pricing = this.lookupPricing(capabilities);
});
},
destroy() {
// Cleanup: remove DOM elements, unsubscribe events
},
estimateTokens(text) { /* ... */ },
renderCost(est) { /* ... */ },
lookupPricing(caps) { /* ... */ }
});
```
### 4.2 Context Object
The `ctx` object is the extension's API surface. It's scoped — an
extension that didn't declare `dom:sidebar` in permissions gets a `ctx.ui`
that throws on `inject('sidebar', ...)`.
```
ctx.events — EventBus subscribe/publish (filtered by permissions)
ctx.storage — localStorage namespace (extensions::{id}::*)
ctx.settings — Read-only settings values from manifest
ctx.ui — DOM injection into declared surfaces
ctx.api — Proxied fetch() to backend (auth headers injected)
ctx.model — Current model ID + resolved capabilities
ctx.user — Current user info (id, username, role)
```
### 4.3 Admin-Pushed vs User-Installed
- **Admin-pushed**: `extensions` table row with `is_system = true`. Loaded
for all users. Users cannot disable. JS served from
`/api/v1/extensions/{id}/assets/`.
- **User-installed**: Stored in user settings JSONB. Users can enable/disable.
JS loaded from same asset path or inline (for small scripts).
- Both use the same runtime. The difference is governance, not execution.
### 4.4 Security Model
Browser extensions run in the same origin. They can't be truly sandboxed
without iframes (which breaks DOM injection). The security model is:
1. **Permission declaration** — extensions declare what they need; the
loader enforces it via the `ctx` proxy.
2. **Admin review** — admin-pushed extensions are implicitly trusted.
User-installed extensions get a "this extension can access: ..." prompt.
3. **Event scoping** — extensions only see events they declared in
`permissions`. The scoped EventBus filters unpermitted subscriptions.
4. **CSP headers** — strict Content-Security-Policy prevents inline script
injection. Extension scripts are served from known paths only.
---
## 5. Browser-Defined Tools (The Bridge)
This is the critical innovation. ai-editor proved that LLM tools work in
browser JS. But in Chat Switchboard, the completion handler runs server-side.
The tool call originates from the LLM response, arrives at the Go backend,
and needs to execute in the user's browser.
### 5.1 The Flow
```
User sends message
→ Backend sends to LLM with tools[] from enabled extensions
→ LLM returns tool_call: { name: "read_file", args: {...} }
→ Backend sees tool_call, checks tool registry
→ Tool is tier:browser → Backend publishes via WebSocket:
event: tool.call.{callId}
data: { tool: "read_file", args: {...}, callId: "uuid" }
→ Browser extension receives event, executes tool
→ Extension publishes result via WebSocket:
event: tool.result.{callId}
data: { callId: "uuid", result: "file contents..." }
→ Backend receives result, feeds back to LLM as tool_result message
→ LLM continues with the result
→ Response streams to user
```
### 5.2 Tool Registration
Extensions declare tools in their manifest and implement them:
```json
{
"tools": [
{
"name": "estimate_cost",
"description": "Estimate the token cost of a given text",
"parameters": {
"type": "object",
"properties": {
"text": { "type": "string", "description": "Text to estimate" }
},
"required": ["text"]
},
"tier": "browser"
}
]
}
```
```js
Extensions.register({
id: 'cost-tracker',
init(ctx) {
ctx.tools.handle('estimate_cost', async (args) => {
const tokens = this.estimateTokens(args.text);
return { tokens, estimated_cost_usd: tokens * this.pricing.outputPerM / 1e6 };
});
}
});
```
### 5.3 Server-Side Tool Router
The completion handler maintains a tool registry. When building the tools[]
array for the LLM request:
```go
func (h *CompletionHandler) collectTools(userID string) []Tool {
var tools []Tool
// Tier 1: Starlark tools (server-side, execute inline)
tools = append(tools, h.starlarkTools()...)
// Tier 2: Sidecar tools (server-side, HTTP call)
tools = append(tools, h.sidecarTools()...)
// Tier 0: Browser tools (client-side, routed via WebSocket)
// Only included if user has WebSocket connected
if hub.IsConnected(userID) {
tools = append(tools, h.browserTools(userID)...)
}
return tools
}
```
When a tool_call arrives and the tool is `tier: browser`:
```go
case "browser":
callId := uuid.New().String()
// Publish to user's WebSocket
bus.Publish(events.Event{
Type: "tool.call." + callId,
Data: toolCallData,
Room: "user:" + userID,
})
// Wait for result (with timeout)
result, err := bus.WaitFor("tool.result." + callId, 30*time.Second)
```
### 5.4 Timeout and Fallback
Browser tools have a 30-second timeout. If the user's browser disconnects
or the tool fails, the backend sends a tool_result with an error message
to the LLM so it can recover gracefully. This is no different from how
ai-editor handles tool failures — the LLM gets an error and adapts.
### 5.5 Why This Matters
This means an extension author can write a tool in 20 lines of JavaScript
that the LLM can call. No Go code. No container. No deployment. The
cluster manager extension defines `kubectl_get`, `ceph_status`,
`node_drain` as browser tools. The editor extension defines `read_file`,
`write_file`, `search_replace`. The article extension defines
`fetch_source`, `check_citation`.
The LLM doesn't know or care where the tool executes. It sees a tool
schema, calls it, gets a result. The bus handles the routing.
---
## 6. Surfaces (Modes)
A "mode" is an extension that registers a **surface** — a UI region that
replaces or augments the default chat area. The core app provides injection
points:
```
┌─────────────────────────────────────────────┐
│ sidebar-top │ surface-header │
│ │ │
│ sidebar-nav │ │
│ (mode selector) │ surface-main │
│ │ (chat, editor, article, │
│ sidebar-content │ cluster, ...) │
│ (context panel) │ │
│ │ │
│ sidebar-bottom │ surface-footer │
│ │ (input area) │
└─────────────────────────────────────────────┘
```
### 6.1 Surface Registration
```json
{
"surfaces": [
{
"id": "editor",
"label": "Editor",
"icon": "code",
"regions": ["surface-main", "surface-footer", "sidebar-content"],
"default": false
}
]
}
```
```js
Extensions.register({
id: 'editor-mode',
init(ctx) {
ctx.surfaces.register('editor', {
activate() {
// Replace surface-main with editor UI
ctx.ui.replace('surface-main', this.renderEditor());
ctx.ui.replace('surface-footer', this.renderEditorInput());
ctx.ui.replace('sidebar-content', this.renderFileTree());
},
deactivate() {
// Restore defaults
ctx.ui.restore('surface-main');
ctx.ui.restore('surface-footer');
ctx.ui.restore('sidebar-content');
}
});
}
});
```
### 6.2 The Core Surfaces
Chat mode is just the default surface. It's not special — it's the surface
that's active when no extension surface is selected. In theory, even chat
mode could be extracted into an extension, but pragmatically it stays in
core because everything depends on it.
### 6.3 Mode Selector
When extensions register surfaces, a mode selector appears in the sidebar
(below the brand, above chat history). Clicking a mode calls `activate()`
on that surface and `deactivate()` on the current one.
The bus event `surface.activated` fires so other extensions can react.
The cost tracker might show different metrics in editor mode vs chat mode.
### 6.4 Jeff's Planned Modes
**Chat Mode** (core)
- What exists today, plus tool calling and richer message types.
- The "default surface" that everything else builds on.
**Editor Mode** (extension)
- Surfaces: file tree in sidebar, code editor in main, AI chat in a
split pane or overlay.
- Tools: `read_file`, `write_file`, `search_replace`, `run_command`,
`git_status`, `git_commit` — all browser tools backed by a git
provider API (GitHub, Gitea).
- This is ai-editor rebuilt properly: the tool definitions are JS,
the LLM calls happen through the standard completion handler, and
the file operations go through a git provider abstraction.
- The key difference from ai-editor: the completion handler is
server-side, so tool calls are logged, token-counted, and auditable.
**Article Mode** (extension)
- Surfaces: outline in sidebar, rich text editor in main, AI assistant
in a panel.
- Tools: `fetch_url`, `summarize_section`, `check_citation`,
`suggest_structure`.
- Think: a writing environment where the AI is a research assistant,
not a chatbot. The conversation is hidden; only the document matters.
**Cluster Manager Mode** (extension — or extensions plural)
- This one is interesting because it's actually 2-3 cooperating
extensions:
- `k8s-manager`: tools for `kubectl` operations, surfaces for
pod/deployment views.
- `ceph-manager`: tools for `ceph status`, OSD management, pool
operations.
- `node-manager`: tools for SSH commands, system metrics,
package management.
- These extensions talk to each other via the EventBus. When
`k8s-manager` detects a node is NotReady, it publishes
`cluster.node.unhealthy`. `node-manager` subscribes and surfaces
diagnostics. `ceph-manager` subscribes and checks if that node
had OSDs.
- The LLM sees ALL the tools from ALL active cluster extensions.
"Drain node-3, ensure Ceph rebalances, then cordon it" becomes
a multi-tool conversation where the LLM calls `kubectl_drain`,
then `ceph_osd_out`, then `kubectl_cordon` — each routed to the
appropriate extension.
- This is what you do manually with Claude Desktop today. The
extension system makes it one coherent interface.
---
## 7. Extension Loader Architecture
### 7.1 Load Order
```html
<!-- Core -->
<script src="js/events.js"></script>
<script src="js/extensions.js"></script> <!-- NEW: loader + registry -->
<!-- Extensions (injected by loader) -->
<script src="/api/v1/extensions/cost-tracker/assets/main.js"></script>
<script src="/api/v1/extensions/editor-mode/assets/main.js"></script>
<!-- App (runs after extensions registered) -->
<script src="js/api.js"></script>
<script src="js/ui.js"></script>
<script src="js/app.js"></script>
```
### 7.2 extensions.js (Core)
The extension loader and registry. ~200 lines. Responsibilities:
- Fetch enabled extensions list from `/api/v1/extensions?tier=browser`
- Inject script tags in dependency order
- Provide `Extensions.register()` API
- Build scoped `ctx` objects per extension (permission enforcement)
- Manage surface activation/deactivation
- Collect browser tool schemas for the completion handler
- Route `tool.call.*` events to the correct handler
- Provide `Extensions.list()`, `Extensions.get()`, `Extensions.settings()`
### 7.3 Backend Support
New tables (migration 006):
```sql
CREATE TABLE extensions (
-- already exists from 001, but needs columns:
tier VARCHAR(20) NOT NULL DEFAULT 'browser', -- browser, starlark, sidecar
manifest JSONB NOT NULL,
assets_path TEXT, -- filesystem path for browser JS
endpoint TEXT, -- URL for sidecar
script TEXT, -- inline Starlark source
installed_by UUID REFERENCES users(id),
is_system BOOLEAN DEFAULT false
);
CREATE TABLE extension_user_settings (
extension_id UUID REFERENCES extensions(id) ON DELETE CASCADE,
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
settings JSONB DEFAULT '{}',
is_enabled BOOLEAN DEFAULT true,
PRIMARY KEY (extension_id, user_id)
);
```
New endpoints:
```
GET /api/v1/extensions — list enabled for current user
GET /api/v1/extensions/:id/manifest — get manifest
GET /api/v1/extensions/:id/assets/*path — serve browser JS
POST /api/v1/extensions/:id/settings — update user settings
POST /api/v1/admin/extensions — install extension
DELETE /api/v1/admin/extensions/:id — uninstall
PUT /api/v1/admin/extensions/:id — update (enable/disable, config)
```
### 7.4 Tool Registry Integration
The completion handler's tool collection expands:
```go
func (h *CompletionHandler) collectTools(userID string, configID string) []ToolSchema {
var tools []ToolSchema
// 1. Model-native tools (if provider supports them)
caps := h.getModelCapabilities(model, configID)
if !caps.ToolCalling {
return nil // Model doesn't support tools, skip everything
}
// 2. Server-side extension tools (Starlark + Sidecar)
tools = append(tools, h.serverTools(userID)...)
// 3. Browser tools (only if WebSocket connected)
if h.hub.IsConnected(userID) {
tools = append(tools, h.browserTools(userID)...)
}
return tools
}
```
---
## 8. EventBus Integration
The routing table from events/types.go expands:
```go
var Routes = map[string]Direction{
// Core
"chat.message.*": DirBoth,
"user.presence": DirToClient,
// Tool execution
"tool.call.*": DirToClient, // Server → specific client
"tool.result.*": DirFromClient, // Client → server
// Surfaces
"surface.activated": DirLocal, // Client-only
"surface.deactivated": DirLocal,
// Extension lifecycle
"extension.loaded": DirLocal,
"extension.error": DirLocal,
// Cross-extension (cluster manager example)
"cluster.node.*": DirLocal, // Between browser extensions
"cluster.alert.*": DirBoth, // Could notify server too
}
```
Browser extensions use `Events.on()` and `Events.emit()` — the same API
the core app uses. Events with `DirBoth` or `DirFromClient` cross the
WebSocket. `DirLocal` events stay in the browser. This means cluster
manager extensions can coordinate locally at zero latency, and only
publish to the server when something needs persistence or notification.
---
## 9. Implementation Roadmap
### Phase A: Foundation
- [ ] `extensions.js` — loader, registry, scoped context
- [ ] `Extensions.register()` API with permission enforcement
- [ ] Manifest format parser and validator
- [ ] Admin endpoints for extension management
- [ ] Asset serving endpoint
- [ ] Extension settings UI in Settings modal
### Phase B: Browser Tools
- [ ] `ctx.tools.handle()` API for browser tool registration
- [ ] Tool schema collection in completion handler
- [ ] `tool.call.*` / `tool.result.*` WebSocket routing
- [ ] Timeout and error handling for browser tools
- [ ] Tool execution in EventBus `WaitFor` pattern
- [ ] Tool use display in chat messages
### Phase C: Surfaces
- [ ] Surface registration and activation API
- [ ] Mode selector in sidebar
- [ ] `ctx.ui.replace()` / `ctx.ui.restore()` for region management
- [ ] `surface.activated` / `surface.deactivated` events
### Phase D: First Extensions
- [ ] Cost tracker (browser, proof of concept)
- [ ] Slash commands (browser, message transforms)
- [ ] Custom renderers (browser, mermaid/latex/etc)
- [ ] Editor mode (browser, with git provider tools)
### Phase E: Server-Side Tiers
- [ ] Starlark runtime integration
- [ ] Sidecar HTTP tool protocol
- [ ] Server-side tool execution in completion handler
- [ ] Web search as sidecar extension
---
## 10. Design Principles
1. **Extensions are first-class.** The system is designed so that a mode
or feature implemented as an extension is indistinguishable from one
built into core. No second-class citizens.
2. **The EventBus is the spine.** Extensions don't import each other.
They publish and subscribe to events. This is how the cluster manager
extensions cooperate without knowing about each other at build time.
3. **Tools are location-transparent.** The LLM sees a tool schema. It
doesn't know if the tool runs in the browser, in a Starlark sandbox,
or in a container in another data center. The routing is the platform's
problem.
4. **Permissions are declared, not discovered.** An extension says what it
needs upfront. The loader enforces it. No ambient authority.
5. **The core stays small.** Auth, provider routing, message persistence,
event bus, extension loader. That's core. Everything else is an
extension — even if it ships with the project.
6. **Progressive capability.** A browser extension with zero tools and
zero surfaces is just an EventBus subscriber. Add a tool and the LLM
can call it. Add a surface and it becomes a mode. The same manifest
format scales from "show token count" to "full IDE."