689 lines
23 KiB
Markdown
689 lines
23 KiB
Markdown
# Chat Switchboard — Extension System Specification
|
|
|
|
**Version:** 0.2
|
|
**Status:** Design (pre-implementation)
|
|
**See also:** [ARCHITECTURE.md](ARCHITECTURE.md), [ROADMAP.md](ROADMAP.md)
|
|
|
|
---
|
|
|
|
## 1. Philosophy
|
|
|
|
Chat Switchboard is a substrate, not an application. The core provides:
|
|
authentication, provider routing, message persistence, an event bus, and a
|
|
rendering surface. Everything else — editing, writing, cluster management,
|
|
cost tracking, custom renderers — is an extension.
|
|
|
|
The goal is that someone with a problem and some JS (or Go, or Python) can
|
|
solve it without forking the project. The "modes" — Editor, Article, Chat,
|
|
Cluster Manager — are extensions that register surfaces, tools, and event
|
|
handlers. This project doesn't have to build all of them. It has to make
|
|
them possible.
|
|
|
|
---
|
|
|
|
## 2. Extension Tiers
|
|
|
|
| | Tier 0: Browser | Tier 1: Starlark | Tier 2: Sidecar |
|
|
|---|---|---|---|
|
|
| **Runs in** | User's browser | Go server (embedded) | Separate container |
|
|
| **Language** | JavaScript | Starlark (Python subset) | Any |
|
|
| **Deployed by** | User or Admin push | Admin | Admin |
|
|
| **Latency** | Zero (client-side) | Low (in-process) | Network hop |
|
|
| **Can access** | DOM, EventBus, user context | Message data, DB reads (sandboxed) | Anything (HTTP, filesystem, network) |
|
|
| **Cannot access** | Server internals, other users' data | Network, filesystem, raw SQL | N/A (full access) |
|
|
| **Trust model** | Same-origin; user-scoped or admin-pushed | Starlark sandbox (no I/O) | Container isolation |
|
|
| **Use cases** | UI, rendering, shortcuts, client tools, modes | Routing rules, message transforms, logging | RAG, external APIs, webhooks, heavy compute |
|
|
|
|
All three tiers communicate through the EventBus. A browser extension
|
|
publishes `tool.result.{callId}` the same way a sidecar does — the bus
|
|
doesn't care where the event originated.
|
|
|
|
**Admin-pushed vs User-installed (Tier 0):**
|
|
Admin-pushed extensions load for all users (like a managed browser
|
|
extension policy). User-installed extensions load from user settings
|
|
and only affect that user's session. Both use the same runtime, same
|
|
manifest format. The difference is governance, not execution.
|
|
|
|
---
|
|
|
|
## 3. Manifest Format
|
|
|
|
Every extension, regardless of tier, is described by a manifest:
|
|
|
|
```json
|
|
{
|
|
"id": "cost-tracker",
|
|
"name": "Cost Tracker",
|
|
"version": "1.0.0",
|
|
"tier": "browser",
|
|
"author": "jeff",
|
|
"description": "Real-time token counting and cost estimation",
|
|
|
|
"permissions": [
|
|
"events:chat.message.*",
|
|
"events:model.selected",
|
|
"dom:input-area",
|
|
"storage:local"
|
|
],
|
|
|
|
"entry": "cost-tracker.js",
|
|
|
|
"hooks": {
|
|
"chat.message.send": { "priority": 10, "async": false },
|
|
"chat.message.received": { "priority": 50, "async": true }
|
|
},
|
|
|
|
"tools": [],
|
|
"surfaces": [],
|
|
|
|
"settings": {
|
|
"showInline": {
|
|
"type": "boolean",
|
|
"label": "Show cost inline",
|
|
"default": true
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3.1 Fields
|
|
|
|
- **id**: Unique identifier. Namespaced by convention (`jeff.cost-tracker`).
|
|
- **tier**: `browser` | `starlark` | `sidecar`
|
|
- **permissions**: What the extension needs. The loader enforces these.
|
|
Undeclared access is blocked (Tier 0 via proxy, Tier 1 via sandbox,
|
|
Tier 2 via API scoping).
|
|
- **entry**: Browser: JS file path. Starlark: `.star` file. Sidecar:
|
|
endpoint URL or Docker image.
|
|
- **hooks**: EventBus events this extension subscribes to, with priority
|
|
(lower = runs first) and whether the hook is async.
|
|
- **tools**: LLM-callable tools this extension provides (see §5).
|
|
- **surfaces**: UI surfaces this extension registers (see §6).
|
|
- **settings**: User-configurable options, rendered as a form in the
|
|
extension settings UI.
|
|
|
|
---
|
|
|
|
## 4. Browser Extensions (Tier 0)
|
|
|
|
### 4.1 Lifecycle
|
|
|
|
```
|
|
Install → Load → Init → Active → Disable → Unload
|
|
```
|
|
|
|
**Install**: Admin pushes manifest + JS to server, or user adds from
|
|
settings. Stored in `extensions` table with `tier = 'browser'`.
|
|
|
|
**Load**: On page load, after `events.js` but before `app.js`, the
|
|
extension loader injects `<script>` tags for all enabled browser extensions.
|
|
Load order respects declared dependencies.
|
|
|
|
**Init**: Each extension's entry script calls `Extensions.register()`:
|
|
|
|
```js
|
|
Extensions.register({
|
|
id: 'cost-tracker',
|
|
|
|
init(ctx) {
|
|
// ctx.events — scoped EventBus (only permitted events)
|
|
// ctx.storage — scoped localStorage wrapper
|
|
// ctx.settings — this extension's settings values
|
|
// ctx.ui — DOM injection points
|
|
// ctx.api — proxied fetch() to backend (auth injected)
|
|
// ctx.model — current model ID + resolved capabilities
|
|
// ctx.user — current user info (id, username, role)
|
|
this.ctx = ctx;
|
|
|
|
ctx.events.on('chat.message.send', (msg) => {
|
|
const est = this.estimateTokens(msg.content);
|
|
ctx.ui.inject('input-area', this.renderCost(est));
|
|
});
|
|
},
|
|
|
|
destroy() {
|
|
// Cleanup: remove DOM elements, unsubscribe events
|
|
}
|
|
});
|
|
```
|
|
|
|
### 4.2 Context Object
|
|
|
|
The `ctx` object is the extension's API surface. It's scoped — an
|
|
extension that didn't declare `dom:sidebar` in permissions gets a `ctx.ui`
|
|
that throws on `inject('sidebar', ...)`.
|
|
|
|
```
|
|
ctx.events — EventBus subscribe/publish (filtered by permissions)
|
|
ctx.storage — localStorage namespace (extensions::{id}::*)
|
|
ctx.settings — Read-only settings values from manifest
|
|
ctx.ui — DOM injection into declared surfaces
|
|
ctx.api — Proxied fetch() to backend (auth headers injected)
|
|
ctx.model — Current model ID + resolved capabilities
|
|
ctx.user — Current user info (id, username, role)
|
|
```
|
|
|
|
### 4.3 Admin-Pushed vs User-Installed
|
|
|
|
- **Admin-pushed**: `extensions` row with `is_system = true`. Loaded for
|
|
all users. Users cannot disable.
|
|
- **User-installed**: Stored in user settings JSONB. Users can toggle.
|
|
- Both use the same runtime.
|
|
|
|
### 4.4 Security Model
|
|
|
|
Browser extensions run in the same origin. The security model is:
|
|
|
|
1. **Permission declaration** — extensions declare what they need; the
|
|
loader enforces it via the `ctx` proxy.
|
|
2. **Admin review** — admin-pushed extensions are implicitly trusted.
|
|
User-installed extensions get a "this extension can access: ..." prompt.
|
|
3. **Event scoping** — extensions only see events they declared in
|
|
`permissions`.
|
|
4. **CSP headers** — strict Content-Security-Policy prevents inline script
|
|
injection. Extension scripts are served from known paths only.
|
|
|
|
---
|
|
|
|
## 5. Browser-Defined Tools (The Bridge)
|
|
|
|
This is the critical innovation. The LLM tool call originates from the
|
|
provider response, arrives at the Go backend, and needs to execute in the
|
|
user's browser. The EventBus + WebSocket bridge makes this transparent.
|
|
|
|
### 5.1 The Flow
|
|
|
|
```
|
|
User sends message
|
|
→ Backend sends to LLM with tools[] from enabled extensions
|
|
→ LLM returns tool_call: { name: "read_file", args: {...} }
|
|
→ Backend checks tool registry → tool is tier:browser
|
|
→ Backend publishes via WebSocket:
|
|
event: tool.call.{callId}
|
|
data: { tool: "read_file", args: {...}, callId: "uuid" }
|
|
→ Browser extension receives event, executes tool handler
|
|
→ Extension publishes result via WebSocket:
|
|
event: tool.result.{callId}
|
|
data: { callId: "uuid", result: "file contents..." }
|
|
→ Backend receives result, feeds back to LLM as tool_result
|
|
→ LLM continues with the result
|
|
→ Response streams to user
|
|
```
|
|
|
|
### 5.2 Tool Registration
|
|
|
|
Extensions declare tools in their manifest and implement handlers:
|
|
|
|
```json
|
|
{
|
|
"tools": [
|
|
{
|
|
"name": "estimate_cost",
|
|
"description": "Estimate the token cost of a given text",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"text": { "type": "string", "description": "Text to estimate" }
|
|
},
|
|
"required": ["text"]
|
|
},
|
|
"tier": "browser"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
```js
|
|
Extensions.register({
|
|
id: 'cost-tracker',
|
|
init(ctx) {
|
|
ctx.tools.handle('estimate_cost', async (args) => {
|
|
const tokens = this.estimateTokens(args.text);
|
|
return { tokens, estimated_cost: tokens * this.pricing / 1e6 };
|
|
});
|
|
}
|
|
});
|
|
```
|
|
|
|
### 5.3 Server-Side Tool Router
|
|
|
|
The completion handler maintains a tool registry. When building the
|
|
`tools[]` array for the LLM request:
|
|
|
|
```go
|
|
func (h *CompletionHandler) collectTools(userID string) []ToolSchema {
|
|
var tools []ToolSchema
|
|
|
|
// Tier 1: Starlark tools (server-side, execute inline)
|
|
tools = append(tools, h.starlarkTools()...)
|
|
|
|
// Tier 2: Sidecar tools (server-side, HTTP call)
|
|
tools = append(tools, h.sidecarTools()...)
|
|
|
|
// Tier 0: Browser tools (client-side, routed via WebSocket)
|
|
if h.hub.IsConnected(userID) {
|
|
tools = append(tools, h.browserTools(userID)...)
|
|
}
|
|
|
|
return tools
|
|
}
|
|
```
|
|
|
|
When a `tool_call` arrives and the tool is `tier: browser`:
|
|
|
|
```go
|
|
case "browser":
|
|
callId := uuid.New().String()
|
|
bus.Publish(events.Event{
|
|
Type: "tool.call." + callId,
|
|
Data: toolCallData,
|
|
Room: "user:" + userID,
|
|
})
|
|
result, err := bus.WaitFor("tool.result." + callId, 30*time.Second)
|
|
```
|
|
|
|
### 5.4 Timeout and Fallback
|
|
|
|
Browser tools have a 30-second timeout. If the browser disconnects or the
|
|
tool fails, the backend sends a `tool_result` with an error message to the
|
|
LLM so it can recover gracefully.
|
|
|
|
### 5.5 Why This Matters
|
|
|
|
An extension author can write a tool in 20 lines of JavaScript that the
|
|
LLM can call. No Go code. No container. No deployment. The cluster manager
|
|
extension defines `kubectl_get`, `ceph_status`, `node_drain` as browser
|
|
tools. The editor extension defines `read_file`, `write_file`,
|
|
`search_replace`. The LLM doesn't know or care where the tool executes.
|
|
|
|
---
|
|
|
|
## 6. Surfaces (Modes)
|
|
|
|
A "mode" is an extension that registers a **surface** — a UI region that
|
|
replaces or augments the default chat area.
|
|
|
|
### 6.1 Surface Regions
|
|
|
|
```
|
|
┌─────────────────────────────────────────────┐
|
|
│ sidebar-top │ surface-header │
|
|
│ │ │
|
|
│ sidebar-nav │ │
|
|
│ (mode selector) │ surface-main │
|
|
│ │ (chat, editor, article, │
|
|
│ sidebar-content │ cluster, ...) │
|
|
│ (context panel) │ │
|
|
│ │ │
|
|
│ sidebar-bottom │ surface-footer │
|
|
│ │ (input area) │
|
|
└─────────────────────────────────────────────┘
|
|
```
|
|
|
|
### 6.2 Surface Registration
|
|
|
|
```json
|
|
{
|
|
"surfaces": [
|
|
{
|
|
"id": "editor",
|
|
"label": "Editor",
|
|
"icon": "code",
|
|
"regions": ["surface-main", "surface-footer", "sidebar-content"],
|
|
"default": false
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
```js
|
|
Extensions.register({
|
|
id: 'editor-mode',
|
|
init(ctx) {
|
|
ctx.surfaces.register('editor', {
|
|
activate() {
|
|
ctx.ui.replace('surface-main', this.renderEditor());
|
|
ctx.ui.replace('surface-footer', this.renderEditorInput());
|
|
ctx.ui.replace('sidebar-content', this.renderFileTree());
|
|
},
|
|
deactivate() {
|
|
ctx.ui.restore('surface-main');
|
|
ctx.ui.restore('surface-footer');
|
|
ctx.ui.restore('sidebar-content');
|
|
}
|
|
});
|
|
}
|
|
});
|
|
```
|
|
|
|
### 6.3 Mode Selector
|
|
|
|
When extensions register surfaces, a mode selector appears in the sidebar.
|
|
Clicking a mode calls `activate()` on that surface and `deactivate()` on
|
|
the current one. The bus event `surface.activated` fires so other
|
|
extensions can react.
|
|
|
|
### 6.4 Core Surface
|
|
|
|
Chat mode is the default surface. It's not special architecturally — it's
|
|
the surface that's active when no extension surface is selected. Pragmatically
|
|
it stays in core because everything depends on it.
|
|
|
|
### 6.5 Planned Modes
|
|
|
|
**Editor Mode** (extension)
|
|
- File tree in sidebar, code editor in main, AI chat in split/overlay.
|
|
- Tools: `read_file`, `write_file`, `search_replace`, `git_status`,
|
|
`git_commit` — browser tools backed by a git provider API.
|
|
- This is ai-editor rebuilt properly: tool calls are server-side, logged,
|
|
token-counted, auditable.
|
|
|
|
**Article Mode** (extension)
|
|
- Outline in sidebar, rich text editor in main, AI assistant in panel.
|
|
- Tools: `fetch_url`, `summarize_section`, `check_citation`.
|
|
- The conversation is hidden; only the document matters.
|
|
|
|
**Cluster Manager Mode** (2-3 cooperating extensions)
|
|
- `k8s-manager`: kubectl tools, pod/deployment views.
|
|
- `ceph-manager`: ceph status, OSD management.
|
|
- `node-manager`: SSH commands, system metrics.
|
|
- These talk to each other via EventBus. When `k8s-manager` detects a
|
|
node is NotReady, it publishes `cluster.node.unhealthy`. `ceph-manager`
|
|
subscribes and checks OSD placement.
|
|
- The LLM sees ALL tools from ALL active cluster extensions. "Drain node-3,
|
|
ensure Ceph rebalances, then cordon it" → `kubectl_drain`, `ceph_osd_out`,
|
|
`kubectl_cordon` — each routed to the appropriate extension.
|
|
|
|
---
|
|
|
|
## 7. Extension Loader Architecture
|
|
|
|
### 7.1 Load Order
|
|
|
|
```html
|
|
<!-- Core -->
|
|
<script src="js/events.js"></script>
|
|
<script src="js/extensions.js"></script> <!-- loader + registry -->
|
|
|
|
<!-- Extensions (injected by loader) -->
|
|
<script src="/api/v1/extensions/cost-tracker/assets/main.js"></script>
|
|
<script src="/api/v1/extensions/editor-mode/assets/main.js"></script>
|
|
|
|
<!-- App (runs after extensions registered) -->
|
|
<script src="js/api.js"></script>
|
|
<script src="js/ui.js"></script>
|
|
<script src="js/app.js"></script>
|
|
```
|
|
|
|
### 7.2 extensions.js (Core)
|
|
|
|
The extension loader and registry. ~200 lines. Responsibilities:
|
|
|
|
- Fetch enabled extensions from `/api/v1/extensions?tier=browser`
|
|
- Inject script tags in dependency order
|
|
- Provide `Extensions.register()` API
|
|
- Build scoped `ctx` objects per extension (permission enforcement)
|
|
- Manage surface activation/deactivation
|
|
- Collect browser tool schemas for the completion handler
|
|
- Route `tool.call.*` events to the correct handler
|
|
|
|
### 7.3 Backend Support
|
|
|
|
New tables (migration):
|
|
|
|
```sql
|
|
CREATE TABLE extensions (
|
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
ext_id VARCHAR(100) NOT NULL UNIQUE, -- manifest id
|
|
name VARCHAR(200) NOT NULL,
|
|
tier VARCHAR(20) NOT NULL DEFAULT 'browser',
|
|
manifest JSONB NOT NULL,
|
|
assets_path TEXT, -- filesystem path for browser JS
|
|
endpoint TEXT, -- URL for sidecar
|
|
script TEXT, -- inline Starlark source
|
|
installed_by UUID REFERENCES users(id),
|
|
is_system BOOLEAN DEFAULT false,
|
|
is_enabled BOOLEAN DEFAULT true,
|
|
scope VARCHAR(20) DEFAULT 'global', -- global, team, personal
|
|
team_id UUID REFERENCES teams(id),
|
|
created_at TIMESTAMPTZ DEFAULT now(),
|
|
updated_at TIMESTAMPTZ DEFAULT now()
|
|
);
|
|
|
|
CREATE TABLE extension_user_settings (
|
|
extension_id UUID REFERENCES extensions(id) ON DELETE CASCADE,
|
|
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
|
|
settings JSONB DEFAULT '{}',
|
|
is_enabled BOOLEAN DEFAULT true,
|
|
PRIMARY KEY (extension_id, user_id)
|
|
);
|
|
```
|
|
|
|
New endpoints:
|
|
|
|
```
|
|
GET /api/v1/extensions — list enabled for current user
|
|
GET /api/v1/extensions/:id/manifest — get manifest
|
|
GET /api/v1/extensions/:id/assets/*path — serve browser JS
|
|
POST /api/v1/extensions/:id/settings — update user settings
|
|
|
|
POST /api/v1/admin/extensions — install extension
|
|
DELETE /api/v1/admin/extensions/:id — uninstall
|
|
PUT /api/v1/admin/extensions/:id — enable/disable, config
|
|
```
|
|
|
|
---
|
|
|
|
## 8. EventBus Integration
|
|
|
|
The routing table from `events/types.go` expands:
|
|
|
|
```go
|
|
var Routes = map[string]Direction{
|
|
// Core
|
|
"chat.message.*": DirBoth,
|
|
"user.presence": DirToClient,
|
|
|
|
// Tool execution
|
|
"tool.call.*": DirToClient, // Server → specific client
|
|
"tool.result.*": DirFromClient, // Client → server
|
|
|
|
// Surfaces
|
|
"surface.activated": DirLocal, // Client-only
|
|
"surface.deactivated": DirLocal,
|
|
|
|
// Extension lifecycle
|
|
"extension.loaded": DirLocal,
|
|
"extension.error": DirLocal,
|
|
|
|
// Cross-extension (cluster manager example)
|
|
"cluster.node.*": DirLocal, // Between browser extensions
|
|
"cluster.alert.*": DirBoth, // Could notify server too
|
|
}
|
|
```
|
|
|
|
Browser extensions use `Events.on()` and `Events.emit()` — the same API
|
|
the core app uses. `DirLocal` events stay in the browser. `DirBoth` and
|
|
`DirFromClient` events cross the WebSocket.
|
|
|
|
---
|
|
|
|
## 9. Built-in Tools
|
|
|
|
These ship with core because multiple modes and services depend on them:
|
|
|
|
| Tool | Tier | Description |
|
|
|---|---|---|
|
|
| `web_search` | Sidecar | Search provider abstraction (SearXNG, Brave, DuckDuckGo) |
|
|
| `url_fetch` | Server | Retrieve and extract content from a URL |
|
|
| `note_create` | Server | Create a note with title, content, folder, tags |
|
|
| `note_search` | Server | Full-text search across user's notes |
|
|
| `note_update` | Server | Update note content |
|
|
| `note_list` | Server | List notes with optional folder filter |
|
|
| `kb_search` | Server | Semantic search across knowledge bases (future) |
|
|
| `task_create` | Server | Schedule a new task (future) |
|
|
|
|
Extension-provided tools (not core, expected early extensions):
|
|
- `read_file`, `write_file`, `search_replace` (Editor mode)
|
|
- `kubectl_get`, `ceph_status`, `node_ssh` (Cluster manager)
|
|
- `generate_image` (Image gen, browser or sidecar)
|
|
- `speak_text`, `transcribe_audio` (STT/TTS, browser via Web Speech API)
|
|
|
|
---
|
|
|
|
## 10. Design Principles
|
|
|
|
1. **Extensions are first-class.** A mode or feature implemented as an
|
|
extension is indistinguishable from one built into core. No second-class
|
|
citizens.
|
|
|
|
2. **The EventBus is the spine.** Extensions don't import each other.
|
|
They publish and subscribe to events. This is how cluster manager
|
|
extensions cooperate without knowing about each other at build time.
|
|
|
|
3. **Tools are location-transparent.** The LLM sees a tool schema. It
|
|
doesn't know if the tool runs in the browser, in a Starlark sandbox,
|
|
or in a container. The routing is the platform's problem.
|
|
|
|
4. **Permissions are declared, not discovered.** An extension says what it
|
|
needs upfront. The loader enforces it. No ambient authority.
|
|
|
|
5. **The core stays small.** Auth, provider routing, message persistence,
|
|
event bus, extension loader. That's core. Everything else is an
|
|
extension — even if it ships with the project.
|
|
|
|
6. **Progressive capability.** A browser extension with zero tools and
|
|
zero surfaces is just an EventBus subscriber. Add a tool and the LLM
|
|
can call it. Add a surface and it becomes a mode. The same manifest
|
|
format scales from "show token count" to "full IDE."
|
|
|
|
---
|
|
|
|
## Appendix A: Custom Renderers
|
|
|
|
Custom renderers are the simplest useful browser extension. They intercept
|
|
message content rendering to handle specific content types — no tools, no
|
|
surfaces, just a pattern match and a render function.
|
|
|
|
```js
|
|
Extensions.register({
|
|
id: 'collapsible-code',
|
|
init(ctx) {
|
|
ctx.renderers.register('collapsible-code', {
|
|
// Match fenced code blocks with >10 lines
|
|
pattern: /^```(\w+)?\n([\s\S]{10,}?)```$/gm,
|
|
render(match, container) {
|
|
const lang = match[1] || 'text';
|
|
const code = match[2];
|
|
const lines = code.split('\n');
|
|
const details = document.createElement('details');
|
|
details.innerHTML = `
|
|
<summary>${lang} — ${lines.length} lines</summary>
|
|
<pre><code class="language-${lang}">${escapeHtml(code)}</code></pre>
|
|
`;
|
|
container.replaceWith(details);
|
|
}
|
|
});
|
|
}
|
|
});
|
|
```
|
|
|
|
Expected first-party renderer extensions (ship with core or as official):
|
|
|
|
| Renderer | Matches | Renders |
|
|
|---|---|---|
|
|
| `collapsible-code` | Code blocks > N lines | `<details>` with expand/collapse |
|
|
| `html-preview` | ```html blocks | Sandboxed iframe with live preview |
|
|
| `mermaid` | ```mermaid blocks | SVG diagram via mermaid.js |
|
|
| `latex-math` | `$...$` and `$$...$$` | Rendered math via KaTeX |
|
|
| `doc-preview` | Generated HTML/PDF content | Preview panel with download link |
|
|
|
|
These are all ~30-50 line browser extensions. They don't need tool calling
|
|
or surfaces. They just pattern-match content in rendered messages.
|
|
|
|
---
|
|
|
|
## Appendix B: Model Roles
|
|
|
|
Extensions (and core services like compaction) may need to call LLMs for
|
|
their own purposes — summarization, embedding, classification — independent
|
|
of the user's selected chat model. **Model roles** are named slots that
|
|
resolve to a specific provider+model at runtime.
|
|
|
|
### B.1 Role Definition
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ Role │ Purpose │ Typical Model │
|
|
├──────────────────────────────────────────────────────────────┤
|
|
│ utility │ Summarization, routing │ gpt-4o-mini │
|
|
│ │ classification, triage │ deepseek-chat │
|
|
│ embedding │ Vector generation for │ text-embedding-3 │
|
|
│ │ KB search, note search │ nomic-embed-text │
|
|
└──────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
> **Note:** The `generation` role (image/media) was removed in v0.10.2.
|
|
> Image generation will be extension-managed with its own provider config
|
|
> and API surface, not a named role slot.
|
|
|
|
### B.2 Configuration
|
|
|
|
Admin configures roles in global settings, with optional fallback chain:
|
|
|
|
```json
|
|
{
|
|
"model_roles": {
|
|
"utility": {
|
|
"primary": { "provider_config_id": "abc", "model_id": "gpt-4o-mini" },
|
|
"fallback": { "provider_config_id": "def", "model_id": "deepseek-chat" }
|
|
},
|
|
"embedding": {
|
|
"primary": { "provider_config_id": "abc", "model_id": "text-embedding-3-small" },
|
|
"fallback": null
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Teams can override roles for their scope. Personal overrides TBD.
|
|
|
|
### B.3 Extension API
|
|
|
|
```js
|
|
// Browser extension requests a utility completion
|
|
const result = await ctx.models.complete('utility', {
|
|
messages: [{ role: 'user', content: 'Summarize this in 2 sentences: ...' }]
|
|
});
|
|
|
|
// Server-side: extension requests an embedding
|
|
vec, err := models.Embed(ctx, "embedding", "text to embed")
|
|
```
|
|
|
|
The caller doesn't know which model or provider is being used. The role
|
|
system resolves it, tries fallback if primary fails, and logs usage
|
|
against the role for cost tracking.
|
|
|
|
### B.4 Core Consumers
|
|
|
|
| Consumer | Role | Purpose |
|
|
|---|---|---|
|
|
| Compaction service | `utility` | Summarize long conversations |
|
|
| Smart routing | `utility` | Classify intent, pick best model |
|
|
| Summarize & Continue | `utility` | Conversation compaction (v0.10.2) |
|
|
| Knowledge bases | `embedding` | Generate vectors for similarity search |
|
|
| Note search (future) | `embedding` | Semantic search across notes |
|
|
| Image gen extension | _(extension-managed)_ | Uses own provider config, not a role |
|
|
|
|
### B.5 Relationship to Smart Routing
|
|
|
|
Model roles and smart routing are complementary:
|
|
|
|
- **Model roles**: "I need *a* cheap model for summarization" → resolves
|
|
to a specific provider+model based on admin config.
|
|
- **Smart routing**: "Route *this user's chat* to the best model" → policy
|
|
engine that picks based on capabilities, cost, latency, team rules.
|
|
|
|
Roles are for background/system tasks. Routing is for user-facing chat.
|
|
Both use the same provider infrastructure and cost tracking.
|