diff --git a/CHANGELOG.md b/CHANGELOG.md index 26a9e0b..9474f3f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5317 +1,57 @@ # Changelog -## [0.38.5.0] — 2026-03-25 +All notable changes to Switchboard Core are documented here. -### Summary +## [Unreleased] — v0.1.0 -Git-board rewrite — library composition capstone. Rewrites git-board from -a self-contained monolith into a library consumer of gitea-client, proving -the full extension architecture end-to-end: lib.require(), connections, -dependencies, multi-file Starlark, and connection type discovery. Includes -5 platform bug fixes found during composition testing. - -### Changed - -- **git-board v0.2.0:** Rewrote Starlark backend to delegate all Gitea API - work to gitea-client library via `lib.require("gitea-client")`. Removed - `api.http` permission, platform abstraction, and per-user token settings. - Added `connections.read` permission and `gitea-client >= 1.0.0` dependency. - Surface JS: replaced TokenSetup with ConnectionSetup screen. - (~130 lines, down from ~329) -- **gitea-client v1.0.1:** Added missing fields to `get_issues()` (created_at, - updated_at, html_url) and `get_prs()` (created_at, html_url, mergeable). - Re-exported loaded sub-module functions via explicit global assignment for - lib.require() compatibility. - -### Fixed (platform bugs) - -- **ext_api.go:** Route handler no longer requires `api.http` for all packages - with API routes. Consumer packages that delegate HTTP to libraries don't - need `api.http` themselves. -- **packages.go:** `InstallPackage` handler now calls `SyncManifestPermissions` - so permission rows are created on package install (was only in the legacy - `AdminInstallExtension` handler). -- **runner.go:** Package loader for `load()` sub-modules now injects the `json` - module into predeclared scope, matching ExecWithLoader behavior. -- **connection_resolver.go:** Added panic recovery around `vault.Decrypt` to - prevent goroutine crashes from invalid nonce lengths after vault key rotation. - -### Added - -- **SDK test runner:** `git-board.js` domain (14 tests) — validates multi-file - library install, connection type registration, consumer dependency, tool - registration, consumers list, uninstall protection, connection CRUD, full - cleanup cycle. - -## [0.38.2.0] — 2026-03-25 - -### Summary - -Library Packages. Reusable Starlark libraries that extensions can declare -as dependencies and load at runtime via `lib.require()`. Libraries export -functions, run with their own permission context, and are protected from -uninstall while active consumers exist. Includes a `test-tool` admin -endpoint for invoking extension tool calls without the AI chat loop. - -### Added - -- **Schema:** `ext_dependencies` table with `(consumer_id, library_id)` - composite PK, `version_spec` and `resolved_ver` columns. SQLite - migration also rebuilds `packages` table to add `'library'` to the - type CHECK constraint (SQLite cannot ALTER constraints). - (`server/database/migrations/023_ext_dependencies.sql`, - `server/database/migrations/sqlite/023_ext_dependencies.sql`) -- **Model:** `ExtDependency` struct with `ConsumerID`, `LibraryID`, - `VersionSpec`, `ResolvedVer`. - (`server/models/ext_dependency.go`) -- **Store:** `DependencyStore` interface — `Create`, `DeleteByConsumer`, - `ListByConsumer`, `ListByLibrary`, `ListAll`. Postgres and SQLite - implementations. - (`server/store/interfaces.go`, `postgres/ext_dependency.go`, - `sqlite/ext_dependency.go`) -- **Handlers:** Dependency admin endpoints: - - `GET /admin/packages/:id/dependencies` — list consumer's deps - - `GET /admin/packages/:id/consumers` — list library's consumers - - `GET /admin/dependencies` — full dependency graph - (`server/handlers/dependencies.go`) -- **Test-tool endpoint:** `POST /admin/packages/:id/test-tool` — invokes - a starlark extension's `on_tool_call` entry point directly for testing. - (`server/handlers/packages.go`) -- **Starlark module:** `lib.require(id)` — loads a declared library - dependency, executes its script with library-scoped permissions, - extracts manifest exports, freezes into a cached struct. Per-invocation - cache and cycle detection. - (`server/sandbox/lib_module.go`) -- **Install validation:** Library packages require `exports` array, - must not have `tools`, `pipes`, or `route`. Consumer `dependencies` - map validated against installed active libraries. Dependency records - created on install, cleaned up on consumer uninstall. Libraries with - active consumers return 409 on delete. - (`server/handlers/packages.go`) -- **SDK:** `sw.api.admin.packages.dependencies(id)`, - `sw.api.admin.packages.consumers(id)` domain methods. - (`src/js/sw/sdk/api-domains.js`) -- **ICD:** Extension dependencies section with all endpoint specs. - (`docs/ICD/extensions.md`) -- **OpenAPI:** Dependency endpoint schemas and `test-tool` endpoint. - (`server/static/openapi.yaml`) -- **SDK test runner:** `dependencies` domain — 8 tests covering library - install, dependency creation, consumer listing, uninstall protection, - cleanup, and library deletion. - (`packages/sdk-test-runner/js/domains/dependencies.js`) - -### Changed - -- **Runner:** `ExecPackage` creates a `libContext` per invocation and - passes it through `buildModulesWithLibCtx`. The `lib` module is always - injected (no permission required). Multi-file `packageLoader` reused - for library scripts. - (`server/sandbox/runner.go`) -- **Packages handler:** Install flow extended with dependency resolution - and `ext_dependencies` record creation. Delete flow checks for active - consumers before allowing library removal. - (`server/handlers/packages.go`) - -### Design Notes - -- `lib.require()` was originally `lib.load()` but `load` is a reserved - keyword in Starlark (used for `load("module.star", "symbol")`). Renamed - to `require` to avoid parser conflicts. -- Libraries run with their **own** permission context, not the consumer's. - A consumer without `api.http` can still call a library that uses - `http.get()` — the library's permissions are evaluated independently. -- The `test-tool` endpoint enables E2E testing of Starlark extensions - without a configured LLM provider. - -### Files - -- **8 created:** `lib_module.go`, `dependencies.go`, `ext_dependency.go` - (model), `ext_dependency.go` (pg), `ext_dependency.go` (sqlite), - `023_ext_dependencies.sql` (×2), `dependencies.js` (SDK runner) -- **11 modified:** `packages.go`, `runner.go`, `main.go`, `interfaces.go`, - `stores.go` (×2), `api-domains.js`, `openapi.yaml`, `extensions.md`, - `packages.js`, `main.js` (SDK runner) -- **+1,141 lines** (737 new files + 404 modified) - -## [0.38.1.0] — 2026-03-25 - -### Summary - -Extension Connections. Scoped credential management for extensions that -integrate with external services. Same scope/resolution pattern as LLM -providers (personal → team → global), sharable across packages. - -### Added - -- **Schema:** `ext_connections` table with `(type, scope, owner_id, name)` - unique constraint. Secret fields encrypted at rest using vault pattern. - (`server/database/migrations/022_ext_connections.sql`) -- **Store:** `ConnectionStore` interface with CRUD, scoped queries, and - resolution chain (personal → team → global). - (`server/store/interfaces.go`, `postgres/ext_connection.go`, `sqlite/ext_connection.go`) -- **Handlers:** Three scope tiers of REST endpoints: - - Personal: `GET/POST /connections`, `GET/PUT/DELETE /connections/:id`, `GET /connections/resolve` - - Team: `GET/POST /teams/:teamId/connections`, `PUT/DELETE /teams/:teamId/connections/:id` - - Admin: `GET/POST /admin/connections`, `PUT/DELETE /admin/connections/:id` - (`server/handlers/connections.go`, `team_connections.go`, `admin_connections.go`) -- **Starlark module:** `connections.get(type)`, `connections.get(type, name=)`, - `connections.list(type)` — gated by `connections.read` permission. - (`server/sandbox/connections_module.go`) -- **UI:** Connection management sections in Admin, Team Admin, and Settings - surfaces with dynamic form rendering from manifest field schemas. - (`src/js/sw/surfaces/admin/connections.js`, `team-admin/connections.js`, `settings/connections.js`) -- **SDK:** `sw.api.connections.*`, `sw.api.teams.{create,update,delete}Connection()`, - `sw.api.admin.connections.*` domain methods. - (`src/js/sw/sdk/api-domains.js`) -- **ICD:** Extension connections section with all endpoint specs. - (`docs/ICD/extensions.md`) -- **OpenAPI:** Connection endpoint schemas (personal, team, admin, resolve). - (`server/static/openapi.yaml`) -- **SDK test runner:** `connections` domain — CRUD + resolve chain tests. - (`packages/sdk-test-runner/js/domains/connections.js`) - -## [0.38.0.0] — 2026-03-25 - -### Summary - -Multi-file Starlark packages. Promotes Starlark scripts from a single -inline blob in the DB manifest to a proper file tree with `load()` support. -Prerequisite for library packages (v0.38.2) and large extensions. - -### Changed - -- **Extraction:** `extractableRelPath()` now extracts `star/` directories - and bare `*.star` files alongside `js/`, `css/`, `assets/`. Starlark - scripts land on disk at install time instead of being serialized into - the manifest JSONB. - (`server/handlers/packages.go`) -- **Sandbox:** New `ExecWithLoader()` accepts an optional `LoadFunc` for - resolving `load()` statements. `Exec()` is now a thin wrapper that - passes a nil loader (backward compatible). - (`server/sandbox/sandbox.go`) -- **Runner:** Disk-first script loading via `loadScript()` — reads from - `{packagesDir}/{pkgID}/script.star` (or custom `entry_point`), falls - back to legacy `_starlark_script` manifest field. Package-scoped - `packageLoader()` enables `load()` within a package's directory with - path traversal protection, `.star`-only enforcement, and circular - dependency detection. - (`server/sandbox/runner.go`) -- **Wiring:** `SetPackagesDir()` called at startup with - `cfg.StoragePath + "/packages"`. Build script includes `star/` in - archives. - (`server/main.go`, `packages/build.sh`) - -### Added - -- **Entry point validation:** Starlark-tier packages must include - `script.star` (or the manifest's `entry_point` value) in the archive. - Returns 400 if missing. -- **`entry_point` manifest field:** Optional override for the default - entry script name (`script.star`). -- **SDK test runner:** New `packages` domain with 6 tests covering - install validation (missing entry point, with star/, custom entry_point, - browser-tier bypass). - (`packages/sdk-test-runner/js/domains/packages.js`) +Forked from chat-switchboard v0.38.5. Gutted to a pure extension platform. ### Removed -- **`_starlark_script` injection:** The installer no longer reads - `script.star` from the archive and stuffs it into the manifest JSON. - Scripts are loaded from disk at runtime. Existing packages with - `_starlark_script` in their DB row continue to work (legacy fallback). - -### Documentation - -- **ICD:** Multi-file Starlark section added to `extensions.md` with - archive structure, `load()` constraints, and error codes. -- **OpenAPI:** Install endpoint description updated with entry point - validation and `star/` directory extraction. -- **Surfaces ICD:** Archive format updated to include `script.star` and - `star/` entries. - ---- - -## [0.37.19.0] — 2026-03-24 - -### Summary - -Tag release — "UI Complete." Closes all remaining code review items from -FE-REWRITE-REVIEW-0.37.14.md. Adds client-side RBAC gates (`sw.can()`) to -surface action buttons, migrates settings policy reads from `__PAGE_DATA__` -to `sw.auth.policies`, removes dead `__USER__` / `__PAGE_DATA__` template -injections, and moves shell-coupled `_getScale()` into the SDK. Light mode -CSS audit and dead code sweep confirm no outstanding issues. - -### Changed - -- **sw.can() RBAC gates (CR P2-1):** Channel/group creation buttons in chat - dashboard and sidebar gated behind `sw.can('channel.create')`. Persona - create/edit/delete gated behind `sw.can('persona.create')` / - `sw.can('persona.manage')`. Knowledge base create/upload/delete gated - behind `sw.can('kb.create')` / `sw.can('kb.write')`. - (`chat-workspace.js`, `sidebar.js`, `personas.js`, `knowledge.js`) -- **Settings → sw.auth.policies (CR P2-4):** Settings surface reads BYOK and - persona policies from `sw.auth.policies` (populated at SDK boot via - `/api/v1/profile/permissions`) instead of `window.__PAGE_DATA__`. Listens - for `auth.permissions.changed` event to react to policy updates. - (`settings/index.js`) -- **_getScale → sw.shell.getScale() (CR P3-3):** Scale detection for CSS zoom - compensation moved from sidebar-chats.js local function into SDK shell - namespace. Explicit coupling replaces implicit `#surfaceInner` DOM reach. - (`sdk/index.js`, `sidebar-chats.js`) -- **SDK version:** `sw._sdk` updated to `'0.37.19'`. - -### Removed - -- **__USER__ / __PAGE_DATA__ injection (CR P2-4):** Removed `window.__USER__` - and `window.__PAGE_DATA__` global injections from `base.html`. SDK boot - populates `sw.auth.user` and `sw.auth.policies` from API. Extension packages - updated to use SDK (`editor`, `hello-dashboard`, `icd-test-runner`). - (`base.html`, `loaders.go`, `EXTENSION-SURFACES.md`) -- **SettingsPageData policy fields:** `BYOKEnabled` and `UserPersonasEnabled` - removed from Go struct (no longer injected via template). (`loaders.go`) - -### Documentation - -- **CR P3-17:** Added comment documenting intentional raw DOM usage in notes - markdown wikilink processing (`markdown.js`). -- **CR P3-19:** Added comment documenting keyboard shortcut DOM presence check - pattern with future guidance (`use-notes.js`). -- **EXTENSION-SURFACES.md:** Updated `__USER__` references to `sw.auth.user`. - ---- - -## [0.37.18.0] — 2026-03-24 - -### Summary - -Debug/Model surface rebuild + workspace bridge. Debug modal converted from -673 lines of imperative DOM JS to a Preact component tree (CR P2-5). User -default workspace auto-created on first access. Tool outputs auto-save file -refs linked to assistant messages. "Save to workspace" action on chat code -blocks. Model selector shows provider health dots. Provider "Test Connection" -buttons added to both BYOK and admin UIs. - -### Added - -- **Debug modal Preact rebuild:** Engine singleton (`debugEngine`) with - subscribe() pattern for reactive updates. Component tree: ConsoleTab, - NetworkTab, StateTab, ReplTab, DebugBadge. `sw-debug.css` extracted from - `modals.css`. `debug.js` reduced from 673 to ~40 lines (thin bootstrap). - `repl.js` absorbed into `repl-tab.js`. (`src/js/sw/components/debug/`) -- **Default user workspace:** `GET /workspaces/default` auto-creates a - "My Files" workspace per user on first access. Idempotent. Route wired - before `:id` param. SDK: `sw.api.workspaces.getDefault()`. - (`server/handlers/workspaces.go`, `server/main.go`) -- **tool_output auto-save:** When `workspace_write` or `workspace_patch` - tools succeed during completion, a file ref with `origin=tool_output` is - recorded in the `files` table, linked to the assistant message via - `message_id`. (`server/handlers/tool_loop.go`, `completion.go`) -- **Origin filter on channel files:** `GET /channels/:id/files?origin=` - query param. SDK: `sw.api.channels.files(id, {origin})`. - (`server/handlers/files.go`) -- **Save-to-Workspace bridge:** Save button on assistant messages with code - blocks. Saves to user's default workspace with auto-generated filenames. - `guessExtension()` utility (37 language mappings). (`message-actions.js`, - `message-bubble.js`, `code-block.js`) -- **Model selector health dots:** Provider health status (green/yellow/red) - shown next to model names in chat selector. Admin-only, graceful degrade - for non-admin. (`model-selector.js`) -- **Provider test buttons:** "Test Connection" in Settings BYOK and "Test" - in Admin providers. Calls `fetchModels()` and shows success/failure toast. - (`settings/providers.js`, `admin/providers.js`) - -### Changed - -- `modals.css`: Debug-specific rules moved to `sw-debug.css` (~22 rules). -- `base.html`: 46-line debug modal HTML block replaced by `
`. -- `sw.js`: Service worker cache list updated (repl.js removed, sw-debug.css added). -- `LoopResult` struct: new `FileRefs` field for tool file output tracking. -- `channels.files` SDK method: now accepts optional opts for query params. - -### Removed - -- `src/js/repl.js` (527 lines) — absorbed into Preact `repl-tab.js`. - -### ICD / OpenAPI / SDK Test Runner - -- ICD: `GET /workspaces/default` documented; tool_output creation behavior updated. -- OpenAPI: `/api/v1/workspaces/default` endpoint added. -- SDK runner: 2 workspace tests (getDefault + idempotent), 1 channel files origin filter test. - -## [0.37.17.0] — 2026-03-24 - -### Summary - -Workspaces — workspace-first file storage for projects. Project files route -through the workspace FS subsystem (auto-created on first upload) instead of -the flat objStore path. Tree browser UI with nested folders, archive support -(zip/tar.gz upload and download), file actions (download, delete, mkdir), and -upload improvements (progress counter, archive detection, quota errors). - -Architecture decision: `files` table + objStore = ephemeral message attachments; -Workspace FS = persistent file collections (tree, dirs, archive, quota, indexing). -"Save to workspace" bridge from chat files deferred to v0.37.18. - -Also: extension track continuation scheduled as v0.39.x in roadmap, two new -design docs committed (multi-file Starlark, extension connections & libraries). - -### Added - -- **Workspace-backed project files:** `ensureProjectWorkspace()` auto-creates a - workspace (`owner_type=project`) on first file upload, sets `project.workspace_id`. - All project file operations route through workspace FS instead of objStore. - (`server/handlers/files.go`) -- **File download:** `GET /projects/:id/files/download?path=` streams raw file - content with `Content-Disposition: attachment`. (`server/handlers/files.go`) -- **File delete:** `DELETE /projects/:id/files?path=` with optional `recursive` - param for directories. (`server/handlers/files.go`) -- **Create directory:** `POST /projects/:id/files/mkdir` accepts path via JSON - body or query param. (`server/handlers/files.go`) -- **Archive upload:** `POST /projects/:id/archive/upload` extracts .zip/.tar.gz - into project workspace. Returns `files_extracted` count. - (`server/handlers/files.go`) -- **Archive download:** `GET /projects/:id/archive/download?format=zip` bundles - all project files into a zip/tar.gz. (`server/handlers/files.go`) -- **File browser UI:** Tree browser with nested folders (expand/collapse), - type icons, size display, on-hover delete button. Replaces the flat file list - from v0.37.16. (`src/js/sw/surfaces/projects/detail.js`) -- **File action bar:** "New folder" and "Download all" buttons above the file - tree when files exist. (`detail.js`) -- **Upload improvements:** Progress counter ("Uploading 3 of 7…"), archive - detection (`.zip`/`.tar.gz` → confirm extract dialog), specific error - messages (413 = quota exceeded). (`detail.js`) -- **SDK methods:** `deleteFile`, `mkdir`, `uploadArchive` added to - `sw.api.projects` domain. `files()` and `uploadFile()` updated to support - path param. (`src/js/sw/sdk/api-domains.js`) -- **File browser CSS:** 120 lines — file-row, file-chevron, file-icon, - file-name, file-meta, file-delete, dropzone, file-actions-bar. - (`src/css/sw-projects-surface.css`) -- **DESIGN-EXT-CONNECTIONS-LIBRARIES.md:** Extension connections (scoped - credentials) and library packages (shared code + data via `lib.load()`). - Three implementation phases. (`docs/`) -- **Extension track v0.39.x:** Roadmap updated — v0.39.0 multi-file Starlark, - v0.39.1 connections, v0.39.2 libraries, v0.39.3 full composition. - (`docs/ROADMAP.md`) - -### Changed - -- **Project file list response:** Now returns `WorkspaceFile` objects (with - `path`, `is_directory`, `content_type`, `size_bytes`) instead of `File` model - objects. `GET /projects/:id/files` supports `?path=` and `?recursive=` params. -- **FileHandler constructor:** Gains `SetWorkspaceFS()` method for workspace - injection. Project file routes use dedicated `projFileH` instance with wfs - attached. (`server/main.go`) -- **ICD updated:** `docs/ICD/projects.md` — expanded Project Files section with - all 7 endpoints, request/response schemas, error codes. -- **OpenAPI updated:** `server/static/openapi.yaml` — 5 new paths for project - file operations (download, delete, mkdir, archive upload/download). - -### Known Limitations - -- **User default workspace:** Not yet implemented. Personal "My Files" deferred - to v0.37.18. -- **Tool output persistence:** `origin=tool_output` auto-save for LLM-generated - artifacts deferred to v0.37.18. -- **Save to workspace bridge:** "Save" action to promote ephemeral chat files - to workspace storage deferred to v0.37.18. -- **Workspace unique constraint:** Race guard on concurrent workspace creation - relies on DB error handling (re-read on duplicate key). Migration 014 for - explicit `UNIQUE(owner_type, owner_id)` not yet added. - ---- - -## [0.37.16.0] — 2026-03-24 - -### Summary - -Projects Surface. Card-grid list + two-column detail view with inline ChatPane, -context menu (color/archive/delete), sidebar sections (description, instructions, -KBs, notes, files), star toggle, deep-linking, and search/sort. UserMenu in header -on both views. Backend was already complete (16 protected + 2 admin endpoints, -18 SDK methods); this changeset delivers the full Preact frontend. - -### Added - -- **Projects list view:** Card grid with search filter, activity/name sort, - "+ New project" via `sw.prompt` dialog, UserMenu in header left-of-title. - (`src/js/sw/surfaces/projects/list.js`, `src/css/sw-projects-surface.css`) -- **Projects detail view:** Two-column layout — left panel for conversations - (inline ChatPane, "+ New" channel creation, channel list with back nav), - right sidebar for description/instructions textareas with save, collapsible - Knowledge Bases/Notes/Files sections with "+" picker dropdowns. - (`src/js/sw/surfaces/projects/detail.js`) -- **Context menu:** "..." button opens dropdown with Rename, 8 color swatches, - Archive/Unarchive toggle, Delete with confirm dialog. All actions persist - via `PUT /api/v1/projects/:id`. (`detail.js`) -- **Star toggle:** Optimistic ☆↔★ toggle, persists in `settings.starred`. - (`detail.js`) -- **Deep-linking:** `/projects/:id` loads detail view directly via - `window.__PROJECT_ID__` template injection. Invalid IDs gracefully fall - back to list view. (`src/js/sw/surfaces/projects/index.js`, - `server/pages/loaders.go`, `server/pages/templates/surfaces/projects.html`) -- **Surface registration:** `/projects` route with `:id` param, nginx proxy, - docker-entrypoint FE route, conditional CSS/JS loading in `base.html`. - (`server/pages/pages.go`, `nginx.conf`, `docker-entrypoint-fe.sh`, - `server/pages/templates/base.html`) - -### Fixed - -- **Picker section expand:** `openPicker()` now auto-expands the collapsed - section when "+" is clicked, so the picker dropdown is visible without - manually toggling the chevron first. (`detail.js`) - -### Changed - -- **UserMenu in header (list view):** Moved from bottom footer to header row, - left of "Projects" title. Consistent with detail view placement. Removed - `.sw-projects__list-footer` CSS class. (`list.js`, `sw-projects-surface.css`) - -### Known Limitations - -- **File upload:** UI is wired (dropzone + click handler) but backend returns - "failed to save file metadata" — requires workspace storage layer. File - upload (archive upload/download) is required for MVP; deferred to v0.37.17 - (Workspaces). Git integration is optional/post-MVP. -- **Conversation names:** Channel list shows raw UUID instead of friendly name - (channels created as type `direct` have no name field). - ---- - -## [0.37.15.0] — 2026-03-23 - -### Summary - -Workflow Ownership & Lifecycle (CS1–CS5). Adds instance cancel with assignment -cascade, assignment unclaim/reassign/cancel operations, full FE API domain -wiring for stage CRUD and assignments, team-admin workflows rewrite with -tabbed layout (Workflows + Stage Editor | Assignments Queue | Instance Monitor), -and settings assignments view replacing the read-only workflow list. - -### Added - -- **Instance cancel:** `POST /channels/:id/workflow/cancel` sets - `workflow_status='cancelled'` and cascades to all open assignments. Auth: - instance owner, team admin, or global admin. Team-scoped variant at - `POST /teams/:teamId/workflows/monitor/instances/:channelId/cancel`. - (`server/handlers/workflow_instances.go`, `server/store/*/channel.go`) -- **Assignment unclaim:** `POST /workflow-assignments/:id/unclaim` returns - claimed assignment to unassigned. Auth: claimer or team admin. - (`server/handlers/workflow_assignments.go`, `server/store/*/workflows.go`) -- **Assignment reassign:** `POST /workflow-assignments/:id/reassign` changes - `assigned_to` on claimed assignment. Auth: team admin. - (`server/handlers/workflow_assignments.go`, `server/store/*/workflows.go`) -- **Assignment cancel:** `POST /workflow-assignments/:id/cancel` cancels a - single assignment. Auth: team admin. - (`server/handlers/workflow_assignments.go`, `server/store/*/workflows.go`) -- **FE API domains:** `workflowAssignments` domain (mine, claim, unclaim, - complete, reassign, cancel, comment). Team domain additions: assignments, - workflowInstances, cancelWorkflowInstance, stage CRUD (workflowStages, - createWorkflowStage, updateWorkflowStage, deleteWorkflowStage, - reorderWorkflowStages). (`src/js/sw/sdk/api-domains.js`) -- **Team-admin workflows tabs:** Tabbed layout (Workflows | Assignments | - Monitor). Workflows tab includes inline stage editor with add/edit/delete. - Assignments tab shows claimed + available queue with claim/release/complete - actions and WS live updates. Monitor tab shows active instances with cancel. - (`src/js/sw/surfaces/team-admin/workflows.js`) -- **Settings assignments:** Replaces read-only workflow list with personal - assignment queue across all teams. Shows claimed and available assignments - with claim/release/open/complete actions. - (`src/js/sw/surfaces/settings/workflows.js`) - -### Changed - -- Settings sidebar label "Workflows" → "Assignments". - (`src/js/sw/surfaces/settings/index.js`) - ---- - -## [0.37.14.23] — 2026-03-23 - -### Summary - -Extension surface user menu fix + roadmap workflow integration. The ICD Runner -(and all extension surfaces) now mount the Preact UserMenu via `sw.userMenu()` -instead of relying on the dead Go template hydration. - -### Fixed - -- **Extension surface "? User" label:** extension template included static - `{{template "user-menu"}}` which rendered placeholder HTML ("? User") since - the legacy hydration script was removed in v0.37.13. Removed the dead - template inclusion. (`server/pages/templates/surfaces/extension.html`) -- **Extension surface user menu click does nothing:** ICD Runner never called - `sw.userMenu()` to mount the Preact UserMenu component. Added topbar with - `sw.userMenu(topbar, { flyout: 'down' })` in `renderShell`. Also uses - `sw.auth.user` instead of dead `window.__USER__` for header display. - (`packages/icd-test-runner/js/ui.js`, `packages/icd-test-runner/js/main.js`) - -### Changed - -- **Roadmap:** added v0.37.15 reference to `DESIGN-0.37.15.md` workflow design - doc, added v0.38.x Workflow Product Maturity section referencing - `ROADMAP-0.38.md`. Trimmed completed version descriptions. - ---- - -## [0.37.14.22] — 2026-03-23 - -### Summary - -Thinking tags persistence + deleteMessage endpoint. Two features that close -out the v0.37.14 pane audit. Also fixes an S9 latent bug (messages paginated -response not unwrapped) and a SQLite NULL scan crash in GetByID. - -### Added - -- **deleteMessage server endpoint** — `DELETE /channels/:id/messages/:msgId` - with soft-delete, channel access check, and `message.deleted` WS broadcast - to other participants. (`server/handlers/messages.go`, `server/main.go`) -- **`message.deleted` event type** — registered as `DirToClient` for - cross-tab message removal. (`server/events/types.go`) -- **`deleteMessage` SDK method** — `rc.del(...)` in channels namespace. - (`src/js/sw/sdk/api-domains.js`) -- **`message.deleted` WS handler** — filters by active channel, removes - message from state. (`src/js/sw/components/chat-pane/use-chat.js`) -- **`onDelete` wired in ChatPane** — delete button now visible on all - messages. (`src/js/sw/components/chat-pane/index.js`) -- **`extractThinking` / `hydrateThinking` helpers** — parse `` blocks - from message content on the load path (switchChannel, loadMore, WS - message.created). (`src/js/sw/components/chat-pane/use-chat.js`) - -### Fixed - -- **Thinking tags lost on refresh:** `` content rendered correctly - during streaming (via SSE `reasoning_content`) but spilled as raw text - after page reload. Now extracted on the load path and rendered in the - collapsible `
` block. Frontend-only fix, no DB migration. -- **S9 messages unwrap bug:** messages endpoint returns paginated response - with 5 sibling keys, so `_unwrap` preserves the full object. `resp || []` - produced an object, not an array. Changed to `resp.data || resp || []`. -- **SQLite NULL scan crash:** `GetByID` fails on user messages where `model` - and `tokens_used` are NULL (struct uses `string`/`int`, not pointers). - Delete handler now uses a direct `EXISTS` query instead. - ---- - -## [0.37.14.21] — 2026-03-23 - -### Summary - -Double-unwrap cleanup + API envelope normalization. Eliminated 93 defensive -`resp.data || resp || []` patterns across the frontend by fixing `_unwrap` to -preserve sibling fields and normalizing all Go handlers to use `{data: [...]}` -standard envelopes. Three latent bugs fixed (roles, provider types, audit). - -### Changed - -- `src/js/sw/sdk/rest-client.js` — `_unwrap` now only strips `{data: [...]}` when - `data` is the sole key (`Object.keys(json).length === 1`). Responses with siblings - (e.g. `{data: [...], total: N}`) pass through intact. -- **15 Go handlers** normalized from named keys (`users`, `configs`, `models`, - `channels`, `participants`, `tools`, `folders`, `messages`, `values`, `packages`) - to standard `{data: [...]}` envelope: - - `server/handlers/admin.go` — ListUsers, ListGlobalConfigs, ListModelConfigs, - ListArchivedChannels - - `server/handlers/apiconfigs.go` — ListConfigs, ListModels - - `server/handlers/channel_models.go` — List, Add, Update, Delete - - `server/handlers/participants.go` — List, Add, Update, Remove - - `server/handlers/presence.go` — SearchUsers - - `server/handlers/completion.go` — ListTools - - `server/handlers/folders.go` — List - - `server/handlers/notes.go` — ListFolders - - `server/handlers/messages.go` — GetActivePath, SetCursor - - `server/handlers/teams.go` — ListAvailableModels - - `server/handlers/team_providers.go` — ListTeamProviderModels - - `server/handlers/packages.go` — UpdatePackageSettings - - `server/handlers/package_registry.go` — BrowseRegistry - - `server/handlers/roles.go` — ListRoles (bare map → wrapped in `{data: map}`) -- **~45 frontend files** — simplified all double-unwrap patterns: - - Sole-key endpoints (unwrapped): `resp || []` - - Sibling endpoints (not unwrapped): `resp.data || []`, `resp.total || 0` - - Roles map: `r.data || {}` + `Object.entries()` for iteration -- Go integration + live provider tests updated for new envelope keys -- Frontend contract + policy-gating tests updated - -### Fixed - -- **Admin → Roles always empty:** `ListRoles` returned a raw map; `r.roles` - was undefined on it. Now wrapped in `{data: map}`, frontend iterates via - `Object.entries(roles)`. -- **Admin → Provider Types dropdown always empty:** `GetProviderTypes` returned - `{data: types}`, `_unwrap` stripped it to bare array, `t.types` was undefined. - Now correctly uses `t || []`. -- **Admin → Audit entries always empty:** `_unwrap` stripped `{data, total}` to - just the array, losing `total`. Now preserved because of sibling-aware `_unwrap`. - -### Design Notes - -- **Envelope contract:** All list endpoints now return `{data: [...]}` or - `{data: [...], ...metadata}`. No more `{users: [...]}`, `{models: [...]}`, etc. -- **`_unwrap` contract:** sole-key `{data: [...]}` → bare array; - sibling keys present → full object (frontend accesses `.data` and siblings). -- **Editor package skipped:** `packages/editor/js/main.js` uses `API._get` which - bypasses `_unwrap`; defensive patterns intentionally preserved. - ---- - -## [0.37.14] — 2026-03-22 - -### Summary - -Scorched Earth IV + Pane Audit — fourth dead-code purge. 4 files deleted (~−1,672 lines). -The old global layer (`sb.js`, `events.js`, `switchboard-sdk.js`, `workflow-surfaces.js`) -is fully removed. Two survivors (`debug.js`, `repl.js`) cleaned of all old-layer dependencies -and now stand alone with direct `window.*` exports. - -### Removed - -- `src/js/sb.js` (133 LOC) — global action registry (`window.sb`); onclick handlers - in base.html rewritten to direct function calls -- `src/js/events.js` (361 LOC) — event bus + WebSocket bridge (`window.Events`); - WebSocket auth called deleted `API` global; new SDK (`sw/sdk/events.js`) is the - sole event system -- `src/js/switchboard-sdk.js` (~697 LOC) — composition layer (`window.Switchboard`); - `window.sw` was immediately overwritten by Preact SDK `boot()`; most methods - referenced deleted globals (`API`, `UI`) -- `src/js/workflow-surfaces.js` (481 LOC) — workflow stage surface registry; loaded - deleted `ui-primitives.js` as dependency (already broken at runtime since v0.37.13) -- `base.html`: removed 3 script tags, removed `Switchboard.init()` block - (including dead `handleLogout` referencing deleted `Events`/`API`) - -### Changed - -- `src/js/debug.js` — replaced `sb.ns()`/`sb.register()` with direct `window.*` - exports; replaced dead `API`/`App`/`Events`/`Extensions` state snapshot with - Preact SDK `window.sw` introspection; diagnostics use `sw.auth.token` and - `sw.events.connected`; fixed `copyDebugLog` referencing deleted `UI.toast` -- `src/js/repl.js` — replaced `sb.ns()` with `window.REPL`; admin gate uses - `window.sw.auth.user.role` instead of deleted `API.user.role`; injected globals - reduced to `sw` + `DebugLog`; removed dead event label tab-completion -- `server/pages/templates/base.html` — 11 `sb.call()` onclick handlers rewritten - to direct function calls -- `server/pages/templates/workflow.html` — broken custom surface loader replaced - with graceful error message -- `src/sw.js` — removed 3 entries from SHELL_FILES (sb.js, events.js, - switchboard-sdk.js) -- `packages/icd-test-runner/js/tier-sdk.js` — boot tests updated from - `Switchboard.init()` to Preact SDK `boot()`; identity tests use `sw.auth.user`; - theme test uses `sw.emit()`; removed extension compat shim test -- `src/js/sw/sdk/index.js` — SDK version marker updated to 0.37.14 - -### Design Notes - -- **2 old JS files survive:** `debug.js` (debug modal + diagnostics) and `repl.js` - (admin REPL console). Both are standalone with zero old-layer dependencies. - Scheduled for Preact rewrite at v0.37.17. -- **Cumulative scorched earth:** 53 files deleted, ~−19,382 lines across four - passes (v0.37.10, v0.37.12, v0.37.13, v0.37.14). - ---- - -## [0.37.13] — 2026-03-22 - -### Summary - -Scorched Earth III — third dead-code purge. 5 files deleted (−2,269 net lines). -The massive `ui-primitives.js` (1,133 LOC) is gone; its few remaining consumers -(`debug.js`, `switchboard-sdk.js`) received inlined replacements. Theme management -migrated from the global `Theme` object to the Preact SDK's `createTheme` module. -SDK slimmed by removing 6 dead factory wrappers. - -### Removed - -- `src/js/ui-primitives.js` (1,133 LOC) — `esc()`, `createComponentRegistry()`, - `componentMixin()`, `Providers`, `Roles`, `showConfirm()`, `showPrompt()`, - `openModal()`, `closeModal()`, `Theme`, render helpers. All consumers deleted - or inlined. -- `src/js/code-editor.js` (362 LOC) — tabbed CodeMirror 6 editor, zero consumers -- `src/js/file-tree.js` (290 LOC) — workspace file browser widget, zero consumers -- `src/js/user-menu.js` (206 LOC) — vanilla flyout menu; all surfaces use - Preact `sw/shell/user-menu.js` -- `src/js/app-state.js` (139 LOC) — global `window.App` state; Preact surfaces - use local state hooks; `debug.js` state snapshot already guards with `typeof` - -### Changed - -- `src/js/debug.js` — inlined `openModal()`, `closeModal()`, `showConfirm()`, - `_escHtml()` (~44 LOC) to replace dependency on deleted `ui-primitives.js` -- `src/js/switchboard-sdk.js` — removed 6 dead factory wrappers (`sw.notes()`, - `sw.panels()`, `sw.fileTree()`, `sw.codeEditor()`, `sw.layout()`, - `sw.userMenu()` + `_hydrateUserMenu()`); replaced global `Theme` with - `import { createTheme } from './sw/sdk/theme.js'`; inlined `esc()` and - `openModal`/`closeModal` helpers; net −213 LOC -- `server/pages/templates/base.html` — removed 5 script tags (`app-state.js`, - `ui-primitives.js`, `user-menu.js`, `file-tree.js`, `code-editor.js`); - consolidated removal comments -- `src/sw.js` — removed 5 entries from `SHELL_FILES` cache manifest - -### Design Notes - -- **6 old JS files survive:** `sb.js` (debug modal onclick handlers), - `events.js` (extension bridge), `switchboard-sdk.js` (extension compatibility), - `debug.js` + `repl.js` (active tooling), `workflow-surfaces.js` (deferred to - Preact rewrite). Targeted for removal as extensions migrate. -- **`user-menu.css` kept:** extension surface template still renders server-side - user-menu HTML; CSS provides styling for the placeholder. -- **Cumulative scorched earth:** 49 files deleted, ~−17,710 lines across three - passes (v0.37.10, v0.37.12, v0.37.13). - ---- - -## [0.37.12] — 2026-03-22 - -### Summary - -Scorched Earth II — second dead-code purge. 23 files deleted (~2,600 lines). -All six core surfaces are confirmed Preact-only; old vanilla JS, CSS, and -admin templates removed. Service worker cache manifest overhauled. - -### Removed - -- `src/js/ui-format.js` — old markdown renderer (replaced by `sw/components/chat-pane/markdown.js`) -- `src/js/virtual-scroll.js` — only consumer was deleted `chat.js` -- `src/js/model-selector.js` — Preact ChatPane has its own `ModelSelector` -- `src/js/drag-resize.js` — only consumer was deleted `PaneContainer` -- `src/js/pages-splash.js` — login is Preact since v0.37.5 -- `src/js/task-sidebar.js` — no template loads this -- `src/js/workflow-api.js` — new SDK has `sw.api.workflows.*` -- `src/js/workflow-queue.js` — references deleted `App.chats` -- `src/js/workflow-monitor.js` — ES module export, never imported -- `src/css/splash.css` — replaced by `sw-login.css` -- `src/css/styles.css` — 13-line decomposition comment from v0.22.9 -- `src/css/persona-kb.css` — old admin/settings UI -- `src/css/notification-prefs.css` — old settings UI -- `src/css/memory.css` — old admin/settings UI -- `src/css/channel-models.css` — old chat autocomplete -- `server/pages/templates/admin/` — 8 dead templates (overview, users, - models, providers, settings, teams, roles, routing); admin is Preact - since v0.37.6 - -### Changed - -- `server/pages/templates/base.html` — removed 7 script/link tags: - `splash.css`, `marked.min.js`, `purify.min.js`, `ui-format.js`, - `virtual-scroll.js`, `model-selector.js`, `drag-resize.js`. - `marked`/`purify` now loaded only by surfaces that need them (chat, notes). -- `src/sw.js` — replaced entire `SHELL_FILES` array; removed 17 stale - entries referencing files deleted in v0.37.10, added all `sw-*.css` files -- `server/pages/pages.go` — removed `templates/admin/*.html` from - `//go:embed` directive and `ParseFS` call - -### Design Notes - -- **11 old JS files survive:** `sb.js`, `app-state.js`, `events.js`, - `ui-primitives.js`, `user-menu.js`, `file-tree.js`, `code-editor.js`, - `switchboard-sdk.js`, `debug.js`, `repl.js`, `workflow-surfaces.js`. - These provide globals consumed by extension surfaces (`Events.on()`, - `Theme.resolved()`, `sw.fileTree()`) and the debug modal (`sb.call()`). - Targeted for removal when extensions migrate to Preact SDK. -- **`workflow-surfaces.js` deferred:** still loaded by `workflow.html` - standalone page; removed when workflow surfaces rebuilt (v0.37.13). - ---- - -## [0.37.11] — 2026-03-22 - -### Summary - -Notes surface built as full Preact surface, replacing the v0.37.9 template -mount. Sidebar with folder tree, tag cloud, search, and UserMenu. NotesPane -runs `standalone=false` — the surface sidebar owns filter controls. Cross-surface -fixes: Menu primitive CSS was never loaded (now added to base.html), Avatar -initials fixed for single-word names. - -### New - -- `src/js/sw/surfaces/notes/index.js` — NotesSurface root, self-mounts at `#notes-mount` -- `src/js/sw/surfaces/notes/use-workspace.js` — sidebar toggle + active note (sessionStorage) -- `src/js/sw/surfaces/notes/use-sidebar.js` — folders, tags, search, WS live updates -- `src/js/sw/surfaces/notes/sidebar.js` — search input, folder/tag sections, UserMenu footer -- `src/js/sw/surfaces/notes/sidebar-folders.js` — collapsible folder tree with counts -- `src/js/sw/surfaces/notes/sidebar-tags.js` — collapsible tag cloud with active state -- `src/js/sw/surfaces/notes/notes-workspace.js` — workspace header + NotesPane -- `src/css/sw-notes-surface.css` — surface layout (sidebar + workspace + mobile responsive) - -### Changed - -- `src/js/sw/components/notes-pane/index.js` — handleRef expanded with `setFolder`, - `setTagFilter`, `setSearchQuery`, `getFolder`, `getTagFilter`; threads `standalone` - prop to NoteList -- `src/js/sw/components/notes-pane/note-list.js` — `standalone=false` hides search row, - folder dropdown, tag cloud (surface sidebar owns these) -- `src/js/sw/primitives/avatar.js` — single-word names show first letter only - ("admin" → "A" not "AD"); multi-word unchanged ("John Doe" → "JD") -- `server/pages/templates/surfaces/notes.html` — full rewrite: crash catcher + - `#notes-mount` + Preact boot (mirrors chat.html pattern) -- `server/pages/templates/base.html` — added `sw-primitives.css` link (Menu primitive - was unstyled on all surfaces); added `css-notes` conditional -- `src/css/surfaces.css` — deleted `.surface-notes*` rules (9 lines, replaced by - `sw-notes-surface.css`) - -### Design Notes - -- **Sidebar ↔ NotesPane coordination:** write-only via handleRef. Sidebar sets - folder/tag/search on the kit; kit reloads its own list. No bidirectional sync - needed because `standalone=false` hides the kit's own filter controls. -- **Menu primitive bug:** `sw-primitives.css` was never loaded in `base.html`. - The Menu component (`sw-menu__item`) had no hover, no cursor, no background. - Chat surface masked this by using its own `.sw-chat-surface__context-menu` - styles. Fixed globally — all surfaces now get proper Menu styles. -- **Kit enhancements deferred:** markdown toolbar, wikilink hover preview, - breadcrumbs, split view, templates, slash commands, graph minimap all moved - to v0.37.14 pane audit. - -### Cumulative Layer Status - -| Layer | Version | Status | -|-------|---------|--------| -| Preact+htm vendor | v0.37.2 | ✅ | -| Layer 0: Primitives | v0.37.11 | ✅ (Menu CSS fix, Avatar fix) | -| Layer 1: SDK | v0.37.10 | ✅ (8 modules) | -| Layer 1.5: ChatPane | v0.37.10 | ✅ (13 files) | -| Layer 1.5: NotesPane | v0.37.11 | ✅ (10 files, standalone prop) | -| Layer 2: Shell | v0.37.4 | ✅ (5 components) | -| Surface: Login | v0.37.5 | ✅ | -| Surface: Settings | v0.37.7 | ✅ | -| Surface: Admin | v0.37.6 | ✅ | -| Surface: Team Admin | v0.37.7 | ✅ | -| Surface: Chat | v0.37.10 | ✅ | -| Surface: Notes | v0.37.11 | ✅ | - -## [0.37.10] — 2026-03-21 - -### Summary - -Scorched earth rebuild of the Chat surface. The old 16-file imperative SPA -(~11,500 lines of `app.js`, `chat.js`, `ui-core.js`, `projects-ui.js`, etc.) -is replaced by a focused Preact surface (8 files) composing the ChatPane kit. -ChatPane kit upgraded with error display, stop generation, regenerate, code -block copy, markdown toolbar, emoji picker, and message pagination. Features -that don't belong on the chat surface (projects, KB management, extensions, -notifications) are deferred — each earns its way back as its own surface or -shell-level feature. - -32 files changed. +774 / −13,615. **Net: −12,841 lines.** - -### New - -- **Chat Surface** (`src/js/sw/surfaces/chat/`) — 8 files: - - `ChatSurface` root — sidebar + workspace + ToastContainer + DialogStack - - `useWorkspace` — active conversation coordinator, sessionStorage restore - - `useSidebar` — channels, chats, folders data; search filter; WS live - updates; CRUD (rename, delete, move-to-folder, create folder) - - `Sidebar` — search bar, "New Chat" button, collapsible Channels section, - collapsible Chats section with folder grouping, context menus, desktop - collapse toggle - - `SidebarChannels` — DMs + named channels, unread badges - - `SidebarChats` — personal AI chats, folder grouping, context menus - (rename, delete, move to folder) - - `ChatWorkspace` — workspace header (sidebar toggle, tools button) + - ChatPane in `standalone=false` mode (surface manages navigation) - - `ToolsPopup` — scrollable category-based tool toggles with persistence, - `useTools` hook loads from API -- **ChatPane Kit enhancements** (4 new components in `chat-pane/`): - - `CodeBlock` — language badge + copy button, code block extraction from - rendered markdown HTML - - `MessageActions` — per-message toolbar: regenerate, copy text, delete. - Appears on hover/focus - - `MarkdownToolbar` — bold, italic, code, code block, link, bullet list, - numbered list, heading, blockquote. Wraps selection with markdown syntax. - Keyboard shortcuts: Ctrl+B, Ctrl+I, Ctrl+E - - `EmojiPicker` — categorized grid (7 categories), search, recently used - (localStorage). Inserts at cursor position -- **ChatPane Kit upgrades** (6 modified files): - - `index.js` — error banner rendered from `error` state, stop button wiring, - new kit re-exports - - `use-chat.js` — `regenerate(msgId)`, `deleteMessage(msgId)`, `loadMore()` - pagination, `hasMore`/`loadingMore` state - - `message-list.js` — "Load older messages" pagination button, scroll - position preservation on prepend, ARIA `role="log"`, regenerate/delete - wiring per message - - `message-bubble.js` — CodeBlock rendering with copy+language badge, - MessageActions toolbar (regenerate/copy/delete), relative timestamps - - `message-input.js` — MarkdownToolbar, EmojiPicker, stop button (replaces - send during streaming), ARIA labels, keyboard format shortcuts - - `markdown.js` — `marked.setOptions()` with GFM+breaks, task list checkbox - rendering via custom renderer -- **UserMenu auto-surface-filtering** — reads `body[data-surface]`, auto- - includes links to all other surfaces (Notes, extensions), filters out the - current surface. Default `handleSelect` navigates via `location.href`. - Every surface gets the same menu minus itself. -- **`sw-chat-surface.css`** — sidebar + workspace flex layout, responsive - mobile overlay, sidebar collapse, tools popup, context menus (~600 lines) - -### Deleted - -**16 JS files (~11,500 lines):** -- `src/js/app.js` (589) — SPA bootstrap, sidebar, init -- `src/js/api.js` (995) — REST client, auth, token management -- `src/js/chat.js` (1,344) — message rendering, streaming, CM6 input -- `src/js/chat-pane.js` (160) — legacy ChatPane wrapper -- `src/js/channel-models.js` (437) — channel model management UI -- `src/js/extensions.js` (506) — extension panel, package management -- `src/js/files.js` (857) — file upload/staging UI -- `src/js/knowledge-ui.js` (556) — knowledge base toggle popup -- `src/js/notifications.js` (431) — notification bell + dropdown -- `src/js/pages.js` (349) — admin page rendering -- `src/js/pane-container.js` (589) — split pane resize manager -- `src/js/panels.js` (454) — side panel registration + lifecycle -- `src/js/projects-ui.js` (2,090) — project sidebar section -- `src/js/tokens.js` (167) — token count display -- `src/js/tools-toggle.js` (245) — tools toggle popup -- `src/js/ui-core.js` (1,699) — core UI: sidebar, messages, context menus - -**5 CSS files (~1,650 lines):** -- `src/css/chat.css` (648) — old chat area, sidebar, workspace, input -- `src/css/chat-pane.css` — legacy chat pane styles -- `src/css/panels.css` — side panel styles -- `src/css/pane-container.css` — split pane layout -- `src/css/notifications.css` — notification bell + dropdown - -### Changed - -- `chat.html` — rewritten from ~387-line SPA scaffold to ~50-line Preact boot - (same pattern as settings/admin/team-admin surfaces) -- `base.html` — removed ` -``` - -Application files continue to load as ` - - - - ... platform scripts ... - ← YOUR SCRIPT - -``` - -The `#extension-mount` div uses `flex: 1` and `overflow: auto`, so it -fills all available vertical space. You can set its content to anything. - ---- - -## Admin API Reference - -Surfaces are managed via the admin API. All endpoints require admin -authentication. - -### Install a Surface - -``` -POST /api/v1/admin/surfaces/install -Content-Type: multipart/form-data - -file: <.surface or .zip file> -``` - -The archive must contain a `manifest.json` with `id` and `title`. -Static assets (`js/`, `css/`, `assets/`) are extracted to the server's -storage directory. The surface is registered in the database and -enabled immediately. - -**Response:** -```json -{ - "id": "my-dashboard", - "title": "My Dashboard", - "source": "extension", - "enabled": true -} -``` - -### List All Surfaces - -``` -GET /api/v1/admin/surfaces -``` - -Returns all surfaces (core + extension) with their enabled state. - -### Enable / Disable - -``` -PUT /api/v1/admin/surfaces/:id/enable -PUT /api/v1/admin/surfaces/:id/disable -``` - -Disabled surfaces redirect to the chat surface. The sidebar link is -hidden. Core surfaces (`chat`, `admin`) cannot be disabled. - -### Uninstall - -``` -DELETE /api/v1/admin/surfaces/:id -``` - -Removes the surface registration and cleans up extracted static assets. -Core surfaces cannot be deleted. - -### List Enabled (Non-Admin) - -``` -GET /api/v1/surfaces -``` - -Returns enabled surfaces with minimal info (id, title, route). Used by -the sidebar to render nav links. Available to all authenticated users. - ---- - -## Platform CSS Classes - -Your extension can use these CSS classes from the platform's -`primitives.css` without importing anything: - -### Buttons - -```html - - - - - -``` - -### Badges - -```html -Default -Success -Error -Warning -``` - -### Toast (via JS) - -```js -UI.toast('Message here', 'success'); // green -UI.toast('Message here', 'error'); // red -UI.toast('Message here', 'info'); // blue -UI.toast('Message here', 'warning'); // yellow -``` - ---- - -## Tips - -**Always wrap in an IIFE.** Your script shares the global scope with -platform scripts. Use `(function() { ... })();` to avoid collisions. - -**Use `esc()` for user-generated content.** The platform does not -provide a global escaping function. Define your own: - -```js -function esc(s) { - var el = document.createElement('span'); - el.textContent = s; - return el.innerHTML; -} -``` - -**Prepend `__BASE__` to internal links.** The platform may be deployed -under a path prefix (e.g. `/dev`): - -```js -var base = window.__BASE__ || ''; -link.href = base + '/settings/general'; -``` - -**Check for globals before using them.** If your surface might be -previewed outside the platform: - -```js -if (typeof API !== 'undefined' && API._get) { - // safe to make API calls -} -if (typeof UI !== 'undefined' && UI.toast) { - UI.toast('Works!', 'success'); -} -``` - -**Size limit:** Archive upload is capped at 50 MB. Only `js/`, `css/`, -and `assets/` directories are extracted from the archive. - -**Cache busting:** Static assets are served with `?v={platform_version}` -query parameter. When the platform is upgraded, caches are invalidated -automatically. - ---- - -## Full Example - -See the included `hello-dashboard.surface` archive for a complete -working example that demonstrates: - -- Manifest structure -- Mounting into `#extension-mount` -- Reading `__MANIFEST__` and `sw.auth.user` -- Using `UI.toast()` for notifications -- Using `API._get()` for authenticated requests -- Using CSS custom properties for theming -- Proper `esc()` function for XSS safety diff --git a/docs/EXTENSIONS.md b/docs/EXTENSIONS.md deleted file mode 100644 index be45ea8..0000000 --- a/docs/EXTENSIONS.md +++ /dev/null @@ -1,630 +0,0 @@ -# Chat Switchboard — Extension System Specification - -**Version:** 0.4 -**Status:** All tiers implemented (Browser v0.11.0, Starlark v0.25.0, Sidecar v0.26.0). v0.30.2 adds workflow packages and SDK. -**See also:** [ARCHITECTURE.md](ARCHITECTURE.md), [ROADMAP.md](ROADMAP.md), [PACKAGES.md](PACKAGES.md), [SDK.md](SDK.md) - ---- - -## 1. Philosophy - -Chat Switchboard is a substrate, not an application. The core provides: -authentication, provider routing, message persistence, an event bus, and a -rendering surface. Everything else — editing, writing, cluster management, -cost tracking, custom renderers — is an extension. - -The goal is that someone with a problem and some JS (or Go, or Python) can -solve it without forking the project. The "modes" — Editor, Article, Chat, -Cluster Manager — are extensions that register surfaces, tools, and event -handlers. This project doesn't have to build all of them. It has to make -them possible. - ---- - -## 2. Extension Tiers - -| | Tier 0: Browser | Tier 1: Starlark | Tier 2: Sidecar | -|---|---|---|---| -| **Runs in** | User's browser | Go server (embedded) | Separate container | -| **Language** | JavaScript | Starlark (Python subset) | Any | -| **Deployed by** | User or Admin push | Admin | Admin | -| **Latency** | Zero (client-side) | Low (in-process) | Network hop | -| **Can access** | DOM, EventBus, user context | Message data, DB reads (sandboxed) | Anything (HTTP, filesystem, network) | -| **Cannot access** | Server internals, other users' data | Network, filesystem, raw SQL | N/A (full access) | -| **Trust model** | Same-origin; user-scoped or admin-pushed | Starlark sandbox (no I/O) | Container isolation | -| **Use cases** | UI, rendering, shortcuts, client tools, modes | Routing rules, message transforms, logging | RAG, external APIs, webhooks, heavy compute | - -All three tiers communicate through the EventBus. A browser extension -publishes `tool.result.{callId}` the same way a sidecar does — the bus -doesn't care where the event originated. - -**Admin-pushed vs User-installed (Tier 0):** -Admin-pushed extensions load for all users (like a managed browser -extension policy). User-installed extensions load from user settings -and only affect that user's session. Both use the same runtime, same -manifest format. The difference is governance, not execution. - ---- - -## 3. Manifest Format - -Every extension, regardless of tier, is described by a manifest: - -```json -{ - "id": "cost-tracker", - "name": "Cost Tracker", - "version": "1.0.0", - "tier": "browser", - "author": "jeff", - "description": "Real-time token counting and cost estimation", - - "permissions": [ - "events:chat.message.*", - "events:model.selected", - "dom:input-area", - "storage:local" - ], - - "entry": "cost-tracker.js", - - "hooks": { - "chat.message.send": { "priority": 10, "async": false }, - "chat.message.received": { "priority": 50, "async": true } - }, - - "tools": [], - "surfaces": [], - - "settings": { - "showInline": { - "type": "boolean", - "label": "Show cost inline", - "default": true - } - } -} -``` - -### 3.1 Fields - -- **id**: Unique identifier. Namespaced by convention (`jeff.cost-tracker`). -- **tier**: `browser` | `starlark` | `sidecar` -- **permissions**: What the extension needs. The loader enforces these. - Undeclared access is blocked (Tier 0 via proxy, Tier 1 via sandbox, - Tier 2 via API scoping). -- **entry**: Browser: JS file path. Starlark: `.star` file. Sidecar: - endpoint URL or Docker image. -- **hooks**: EventBus events this extension subscribes to, with priority - (lower = runs first) and whether the hook is async. -- **tools**: LLM-callable tools this extension provides (see §5). -- **surfaces**: UI surfaces this extension registers (see §6). -- **settings**: User-configurable options, rendered as a form in the - extension settings UI. - ---- - -## 4. Browser Extensions (Tier 0) - -### 4.1 Lifecycle - -``` -Install → Load → Init → Active → Disable → Unload -``` - -**Install**: Admin pushes manifest + JS to server, or user adds from -settings. Stored in `extensions` table with `tier = 'browser'`. - -**Load**: On page load, after `events.js` but before `app.js`, the -extension loader injects ` - - - - - - - - - - -``` - -### 7.2 extensions.js (Core) - -The extension loader and registry. ~200 lines. Responsibilities: - -- Fetch enabled extensions from `/api/v1/extensions?tier=browser` -- Inject script tags in dependency order -- Provide `Extensions.register()` API -- Build scoped `ctx` objects per extension (permission enforcement) -- Manage surface activation/deactivation -- Collect browser tool schemas for the completion handler -- Route `tool.call.*` events to the correct handler - -### 7.3 Backend Support - -New tables (migration): - -```sql -CREATE TABLE extensions ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - ext_id VARCHAR(100) NOT NULL UNIQUE, -- manifest id - name VARCHAR(200) NOT NULL, - tier VARCHAR(20) NOT NULL DEFAULT 'browser', - manifest JSONB NOT NULL, - assets_path TEXT, -- filesystem path for browser JS - endpoint TEXT, -- URL for sidecar - script TEXT, -- inline Starlark source - installed_by UUID REFERENCES users(id), - is_system BOOLEAN DEFAULT false, - is_enabled BOOLEAN DEFAULT true, - scope VARCHAR(20) DEFAULT 'global', -- global, team, personal - team_id UUID REFERENCES teams(id), - created_at TIMESTAMPTZ DEFAULT now(), - updated_at TIMESTAMPTZ DEFAULT now() -); - -CREATE TABLE extension_user_settings ( - extension_id UUID REFERENCES extensions(id) ON DELETE CASCADE, - user_id UUID REFERENCES users(id) ON DELETE CASCADE, - settings JSONB DEFAULT '{}', - is_enabled BOOLEAN DEFAULT true, - PRIMARY KEY (extension_id, user_id) -); -``` - -New endpoints: - -``` -GET /api/v1/extensions — list enabled for current user -GET /api/v1/extensions/:id/manifest — get manifest -GET /api/v1/extensions/:id/assets/*path — serve browser JS -POST /api/v1/extensions/:id/settings — update user settings - -POST /api/v1/admin/extensions — install extension -DELETE /api/v1/admin/extensions/:id — uninstall -PUT /api/v1/admin/extensions/:id — enable/disable, config -``` - ---- - -## 8. EventBus Integration - -The routing table from `events/types.go` expands: - -```go -var Routes = map[string]Direction{ - // Core - "chat.message.*": DirBoth, - "user.presence": DirToClient, - - // Tool execution - "tool.call.*": DirToClient, // Server → specific client - "tool.result.*": DirFromClient, // Client → server - - // Surfaces - "surface.activated": DirLocal, // Client-only - "surface.deactivated": DirLocal, - - // Extension lifecycle - "extension.loaded": DirLocal, - "extension.error": DirLocal, - - // Cross-extension (cluster manager example) - "cluster.node.*": DirLocal, // Between browser extensions - "cluster.alert.*": DirBoth, // Could notify server too -} -``` - -Browser extensions use `Events.on()` and `Events.emit()` — the same API -the core app uses. `DirLocal` events stay in the browser. `DirBoth` and -`DirFromClient` events cross the WebSocket. - ---- - -## 9. Built-in Tools - -These ship with core because multiple modes and services depend on them: - -| Tool | Tier | Description | -|---|---|---| -| `web_search` | Sidecar | Search provider abstraction (SearXNG, Brave, DuckDuckGo) | -| `url_fetch` | Server | Retrieve and extract content from a URL | -| `note_create` | Server | Create a note with title, content, folder, tags | -| `note_search` | Server | Full-text search across user's notes | -| `note_update` | Server | Update note content | -| `note_list` | Server | List notes with optional folder filter | -| `kb_search` | Server | Semantic search across knowledge bases (future) | -| `task_create` | Server | Schedule a new task (future) | - -Extension-provided tools (not core, expected early extensions): -- `read_file`, `write_file`, `search_replace` (Editor mode) -- `kubectl_get`, `ceph_status`, `node_ssh` (Cluster manager) -- `generate_image` (Image gen, browser or sidecar) -- `speak_text`, `transcribe_audio` (STT/TTS, browser via Web Speech API) - ---- - -## 10. Design Principles - -1. **Extensions are first-class.** A mode or feature implemented as an - extension is indistinguishable from one built into core. No second-class - citizens. - -2. **The EventBus is the spine.** Extensions don't import each other. - They publish and subscribe to events. This is how cluster manager - extensions cooperate without knowing about each other at build time. - -3. **Tools are location-transparent.** The LLM sees a tool schema. It - doesn't know if the tool runs in the browser, in a Starlark sandbox, - or in a container. The routing is the platform's problem. - -4. **Permissions are declared, not discovered.** An extension says what it - needs upfront. The loader enforces it. No ambient authority. - -5. **The core stays small.** Auth, provider routing, message persistence, - event bus, extension loader. That's core. Everything else is an - extension — even if it ships with the project. - -6. **Progressive capability.** A browser extension with zero tools and - zero surfaces is just an EventBus subscriber. Add a tool and the LLM - can call it. Add a surface and it becomes a mode. The same manifest - format scales from "show token count" to "full IDE." - ---- - -## Appendix A: Custom Renderers - -Custom renderers are the simplest useful browser extension. They intercept -message content rendering to handle specific content types — no tools, no -surfaces, just a pattern match and a render function. - -```js -Extensions.register({ - id: 'collapsible-code', - init(ctx) { - ctx.renderers.register('collapsible-code', { - // Match fenced code blocks with >10 lines - pattern: /^```(\w+)?\n([\s\S]{10,}?)```$/gm, - render(match, container) { - const lang = match[1] || 'text'; - const code = match[2]; - const lines = code.split('\n'); - const details = document.createElement('details'); - details.innerHTML = ` - ${lang} — ${lines.length} lines - 
${escapeHtml(code)}
- `; - container.replaceWith(details); - } - }); - } -}); -``` - -Expected first-party renderer extensions (ship with core or as official): - -| Renderer | Matches | Renders | -|---|---|---| -| `collapsible-code` | Code blocks > N lines | `
` with expand/collapse | -| `html-preview` | ```html blocks | Sandboxed iframe with live preview | -| `mermaid` | ```mermaid blocks | SVG diagram via mermaid.js | -| `latex-math` | `$...$` and `$$...$$` | Rendered math via KaTeX | -| `doc-preview` | Generated HTML/PDF content | Preview panel with download link | - -These are all ~30-50 line browser extensions. They don't need tool calling -or surfaces. They just pattern-match content in rendered messages. - ---- - -## Appendix B: Model Roles - -Extensions (and core services like compaction) may need to call LLMs for -their own purposes — summarization, embedding, classification — independent -of the user's selected chat model. **Model roles** are named slots that -resolve to a specific provider+model at runtime. - -### B.1 Role Definition - -``` -┌──────────────────────────────────────────────────────────────┐ -│ Role │ Purpose │ Typical Model │ -├──────────────────────────────────────────────────────────────┤ -│ utility │ Summarization, routing │ gpt-4o-mini │ -│ │ classification, triage │ deepseek-chat │ -│ embedding │ Vector generation for │ text-embedding-3 │ -│ │ KB search, note search │ nomic-embed-text │ -└──────────────────────────────────────────────────────────────┘ -``` - -> **Note:** The `generation` role (image/media) was removed in v0.10.2. -> Image generation will be extension-managed with its own provider config -> and API surface, not a named role slot. - -### B.2 Configuration - -Admin configures roles in global settings, with optional fallback chain: - -```json -{ - "model_roles": { - "utility": { - "primary": { "provider_config_id": "abc", "model_id": "gpt-4o-mini" }, - "fallback": { "provider_config_id": "def", "model_id": "deepseek-chat" } - }, - "embedding": { - "primary": { "provider_config_id": "abc", "model_id": "text-embedding-3-small" }, - "fallback": null - } - } -} -``` - -Teams can override roles for their scope. Personal overrides TBD. - -### B.3 Extension API - -```js -// Browser extension requests a utility completion -const result = await ctx.models.complete('utility', { - messages: [{ role: 'user', content: 'Summarize this in 2 sentences: ...' }] -}); - -// Server-side: extension requests an embedding -vec, err := models.Embed(ctx, "embedding", "text to embed") -``` - -The caller doesn't know which model or provider is being used. The role -system resolves it, tries fallback if primary fails, and logs usage -against the role for cost tracking. - -### B.4 Core Consumers - -| Consumer | Role | Purpose | -|---|---|---| -| Compaction service | `utility` | Summarize long conversations | -| Smart routing | `utility` | Classify intent, pick best model | -| Summarize & Continue | `utility` | Conversation compaction (v0.10.2) | -| Knowledge bases | `embedding` | Generate vectors for similarity search | -| Note search (future) | `embedding` | Semantic search across notes | -| Image gen extension | _(extension-managed)_ | Uses own provider config, not a role | - -### B.5 Relationship to Smart Routing - -Model roles and smart routing are complementary: - -- **Model roles**: "I need *a* cheap model for summarization" → resolves - to a specific provider+model based on admin config. -- **Smart routing**: "Route *this user's chat* to the best model" → policy - engine that picks based on capabilities, cost, latency, team rules. - -Roles are for background/system tasks. Routing is for user-facing chat. -Both use the same provider infrastructure and cost tracking. diff --git a/docs/ICD/README.md b/docs/ICD/README.md deleted file mode 100644 index 6220dd5..0000000 --- a/docs/ICD/README.md +++ /dev/null @@ -1,203 +0,0 @@ -# ICD-API — Chat Switchboard Backend API Contract - -**Version:** 0.28.3 -**Updated:** 2026-03-13 -**Audience:** Frontend developers, integrators, API consumers - -> **Organization principle:** Each file is one domain — the complete story. -> Every endpoint that touches Knowledge Bases is in `knowledge.md`, every -> endpoint that touches Personas is in `personas.md`, etc. - -## Files - -| File | Domain | Endpoints | -|------|--------|-----------| -| [auth.md](auth.md) | Auth — login, register, refresh, OIDC, mTLS | 6 | -| [channels.md](channels.md) | Channels, messages, completions, participants, folders, presence | ~35 | -| [personas.md](personas.md) | Personas, persona groups, avatars, KB bindings, tool grants | ~20 | -| [knowledge.md](knowledge.md) | Knowledge bases, documents, search, channel/persona bindings | ~12 | -| [notes.md](notes.md) | Notes, wikilinks, graph, search, folders | ~10 | -| [workspaces.md](workspaces.md) | Workspaces, file ops, git, credentials | ~20 | -| [projects.md](projects.md) | Projects, channel/KB/note associations | ~14 | -| [memory.md](memory.md) | Memory extraction, review, admin | 7 | -| [providers.md](providers.md) | Provider configs, health, capabilities, routing policies | ~25 | -| [models.md](models.md) | Enabled models, user preferences | 4 | -| [notifications.md](notifications.md) | Notifications, preferences | 7 | -| [extensions.md](extensions.md) | Extensions, user settings, admin | 8 | -| [profile.md](profile.md) | User profile, avatar, password, app settings | 7 | -| [teams.md](teams.md) | Teams, members, groups, permissions, grants | ~25 | -| [admin.md](admin.md) | Platform admin — users, settings, stats, audit, usage, vault | ~25 | -| [utility.md](utility.md) | Health, export, files, storage | ~12 | -| [workflows.md](workflows.md) | Workflow definitions, stages, instances, assignments, entry | ~20 | -| [tasks.md](tasks.md) | Task definitions, runs, scheduler, team tasks | ~14 | -| [surfaces.md](surfaces.md) | Surface registry, extension surfaces | 6 | -| [websocket.md](websocket.md) | WebSocket protocol, events, rooms | — | -| [enums.md](enums.md) | All enum values, policies | — | - -## Conventions - -### Base URL - -``` -{scheme}://{host}{BASE_PATH}/api/v1 -``` - -`BASE_PATH` is empty by default. In path-routed K8s deployments (e.g. -`/staging/`), all routes shift accordingly. The frontend reads `BASE_PATH` -from a `` tag injected by the Go template engine. - -### Authentication - -JWT bearer tokens. Every request to a `protected` or `admin` route requires: - -``` -Authorization: Bearer {access_token} -``` - -**Access token:** HS256 JWT, 15-minute TTL. Claims: `user_id`, `email`, -`role`, `exp`, `iat`, `jti`. - -**Refresh token:** Opaque, DB-stored, 7-day TTL. Used to obtain new -access tokens without re-login. - -**WebSocket fallback:** `?token={access_token}` query parameter when -the `Authorization` header isn't available. - -**Cookie sync:** On every token save/refresh, the frontend writes -`sb_token` as a cookie. Go template page routes read this cookie via -`AuthOrRedirect` middleware for server-rendered surfaces. - -### Auth Modes - -| Mode | `AUTH_MODE` | How it works | -|------|------------|--------------| -| Builtin | `builtin` (default) | Username/password, bcrypt, JWT | -| mTLS | `mtls` | Reverse proxy cert headers, auto-provision | -| OIDC | `oidc` | Authorization code flow (Keycloak etc.) | - -All three modes issue the same JWT after authentication. Downstream -middleware is auth-mode-agnostic. - -### Authorization Tiers - -| Tier | Middleware | Who | -|------|-----------|-----| -| Public | none | Anyone (health, login, register, public settings) | -| Session | `AuthOrSession()` | JWT user OR anonymous session cookie (workflow visitors) | -| Authenticated | `Auth()` | Any logged-in user | -| Permission | `RequirePermission(perm)` | User whose groups grant `perm` | -| Team Admin | `RequireTeamAdmin()` | Admin of the specific team | -| Platform Admin | `RequireAdmin()` | Users with `role = "admin"` | - -### Error Envelope - -All errors return: - -```json -{ "error": "human-readable message" } -``` - -Standard HTTP status codes: 400 (bad request), 401 (not authenticated), -403 (not authorized), 404 (not found), 409 (conflict), 500 (server error), -502 (upstream provider failure). - -### Response Envelopes - -Three patterns. Every endpoint uses exactly one. - -**List (returns an array)** → always wrap in `{"data": []}`: - -```json -{ - "data": [...] -} -``` - -If paginated, pagination fields sit alongside `data`: - -```json -{ - "data": [...], - "page": 1, - "per_page": 50, - "total": 142 -} -``` - -Query params: `?page=1&per_page=50`. Not all list endpoints paginate — -unpaginated lists still use `{"data": [...]}`. - -This is a hard rule: `data` is the only array wrapper key. No -domain-specific keys (`teams`, `members`, `configs`, etc.) for arrays. -Clients parse every list response identically. - -**Single object (GET by ID, profile, health)** → return the object directly: - -```json -{ - "id": "uuid", - "name": "...", - ... -} -``` - -No wrapping. The response *is* the resource. - -**Composite (multiple named values, not an array)** → named keys: - -```json -{ - "active": 5, - "pending": 2 -} -``` - -Used for stats, counts, status endpoints — anything returning -multiple named scalars or heterogeneous data. The key names are -domain-specific and documented per endpoint. - -**Empty arrays** must serialize as `[]`, never `null`. Go handlers -must guard nil slices before serialization: - -```go -if result == nil { - result = []MyType{} -} -c.JSON(http.StatusOK, gin.H{"data": result}) -``` - -### ID Format - -All resource IDs are UUIDv4 strings, generated application-side via -`store.NewID()` (Go `uuid.New()`). This ensures compatibility across -both Postgres and SQLite backends. - -### Timestamps - -ISO 8601 with timezone: `"2025-06-15T14:30:00Z"`. Stored as -`TIMESTAMPTZ` (Postgres) or `TEXT` (SQLite). - -## Page Routes (Non-API) - -Server-rendered Go template surfaces. Not REST endpoints — return HTML. - -| Route | Surface | Description | -|-------|---------|-------------| -| `/login` | Login | Standalone login/register page | -| `/` | Chat | Main chat interface | -| `/chat/:chatID` | Chat | Chat with specific channel loaded | -| `/editor` | Editor | Workspace file editor | -| `/editor/:wsId` | Editor | Editor with specific workspace | -| `/notes` | Notes | Notes interface | -| `/notes/:noteId` | Notes | Notes with specific note loaded | -| `/admin` | Admin | Platform administration | -| `/admin/:section` | Admin | Admin with specific section | -| `/settings` | Settings | User settings | -| `/settings/:section` | Settings | Settings with specific section | -| `/w/:id` | Workflow Landing | Branded workflow entry page | -| `/w/:id/:slug` | Workflow Landing | Slug-suffixed workflow entry | -| `/s/:slug` | Extension Surface | Dynamic surface from registry | - -All page routes (except `/login`, `/w/`) require authentication via -`AuthOrRedirect` middleware. Admin routes additionally require -`RequireAdminPage()`. Workflow landing pages use `AuthOrSession`. diff --git a/docs/ICD/admin.md b/docs/ICD/admin.md deleted file mode 100644 index 0e5f7b9..0000000 --- a/docs/ICD/admin.md +++ /dev/null @@ -1,251 +0,0 @@ -# Platform Administration - -Cross-cutting admin operations that don't belong to a specific domain. - -### User Management - -``` -GET /admin/users → { "users": [...], "total": N } -POST /admin/users ← { "username", "password", "email", "role" } -PUT /admin/users/:id/role ← { "role": "admin|user" } -PUT /admin/users/:id/active ← { "is_active": false } -DELETE /admin/users/:id -POST /admin/users/:id/reset-password ← { "password": "..." } -POST /admin/users/:id/vault/reset → destroys user's UEK (BYOK keys lost) -``` - -`PUT /admin/users/:id/role` refuses to demote the last remaining admin -(returns 409). `DELETE /admin/users/:id` also refuses if the target is -the last admin, and destroys the user's vault before deleting the row. - -### Global Settings & Policies - -Settings are key-value pairs in `global_settings`. Policies are -boolean flags that gate features. - -``` -GET /admin/settings → all settings -GET /admin/settings/:key → single setting -PUT /admin/settings/:key ← { "value": ... } -``` - -The handler auto-detects: if the value is a string and the key is a -known policy name, it writes to the `policies` table. Otherwise it -writes to `global_config` as JSON. - -**Public settings** (non-admin, for FE bootstrapping): - -``` -GET /settings/public -``` - -```json -{ - "banner": { "enabled": true, "text": "DEVELOPMENT", "bg": "#007a33", "fg": "#ffffff", "position": "both" }, - "branding": { "instance_name": "Switchboard", "logo_url": "...", "tagline": "..." }, - "has_admin_prompt": true, - "storage_configured": true, - "paste_to_file_chars": 2000, - "policies": { - "allow_registration": "true", - "allow_user_byok": "true", - "allow_user_personas": "true", - "retention_ttl_days": 0 - } -} -``` - -Note: policy values are strings (`"true"` / `"false"`), not booleans. -`retention_ttl_days` is an integer (days). When > 0, channels using -global/team providers are archived on delete and purged after TTL. -Personal (BYOK) provider channels are always immediately deleted. - -### Banner Configuration - -The environment banner is stored as a single `global_config` entry -under the key `"banner"`. It's a simple object: - -```json -{ - "enabled": true, - "text": "DEVELOPMENT", - "position": "both|top|bottom", - "bg": "#007a33", - "fg": "#ffffff" -} -``` - -Set via `PUT /admin/settings/banner` with `{ "value": { ... } }`. -The admin UI provides a color picker, position selector, and preset -dropdown. The Go template base layout reads the banner from -`PageData` and renders top/bottom strips with CSS custom properties. - -### Platform Stats - -``` -GET /admin/stats -``` - -```json -{ - "users": 42, - "channels": 156, - "messages": 12847 -} -``` - -### Storage & Vault - -**Storage status:** - -``` -GET /admin/storage/status → { "backend", "healthy", "file_count", "total_bytes", ... } -GET /admin/storage/orphans → { "count": 3 } -POST /admin/storage/cleanup → removes orphaned blobs -GET /admin/storage/extraction → extraction queue status -``` - -**Vault:** - -``` -GET /admin/vault/status → { "encryption_key_set", "user_vaults_count", ... } -``` - -### Audit Log - -``` -GET /admin/audit?page=1&per_page=50&action=user.create&actor_id=uuid&resource_type=... -GET /admin/audit/actions → { "actions": ["user.create", "policy.update", ...] } -``` - -The list endpoint returns a paginated envelope: - -```json -{ - "data": [...], - "total": 128, - "page": 1, - "per_page": 50 -} -``` - -Audit entry shape: - -```json -{ - "id": "uuid", - "actor_id": "uuid", - "actor_name": "admin", - "action": "user.create", - "resource_type": "user", - "resource_id": "uuid", - "metadata": "{}", - "ip_address": "10.0.0.1", - "created_at": "..." -} -``` - -### Usage & Pricing - -``` -GET /admin/usage → { "totals", "results" } -GET /admin/usage/teams/:id → { "results": [...] } -GET /admin/usage/users/:id → { "results": [...] } -GET /admin/pricing → [ ... ] -PUT /admin/pricing ← { "provider_config_id", "model_id", "input_per_m", "output_per_m" } -DELETE /admin/pricing/:provider_config_id/:model_id -``` - -`GET /admin/pricing` returns a bare array (no envelope). - -`PUT /admin/pricing` accepts the full `PricingEntry` shape: - -```json -{ - "provider_config_id": "uuid", - "model_id": "gpt-4o", - "input_per_m": 2.50, - "output_per_m": 10.00, - "cache_create_per_m": 1.25, - "cache_read_per_m": 0.50, - "currency": "USD" -} -``` - -Pricing cannot be set for personal BYOK providers (403). - -**Personal usage** (non-admin): - -``` -GET /usage → { "totals", "results" } -``` - -Scoped to the user's BYOK provider usage only. - -### Roles (Platform) - -``` -GET /admin/roles → { "roles": [...] } -GET /admin/roles/:role -PUT /admin/roles/:role ← { "permissions": {...} } -POST /admin/roles/:role/test ← test role permissions -``` - -### Email Test - -``` -POST /admin/notifications/test-email -``` - -No request body required. Sends a test email to the authenticated -admin's own email address using the configured SMTP settings. - ---- - -### Archived Channels - -``` -GET /admin/channels/archived → { "channels": [...], "total", "page", "per_page", "retention_ttl_days" } -DELETE /admin/channels/:id/purge → permanent delete (ignores retention) -``` - -Both endpoints require platform admin auth. `GET` accepts -`?page=1&per_page=50` query parameters (max 100 per page). -`DELETE` verifies the channel is archived before purging and cleans -up associated file storage blobs. - -### Surfaces Admin - -See [surfaces.md](surfaces.md) for the full surface management API. - -``` -GET /admin/surfaces → all surfaces (including disabled) -GET /admin/surfaces/:id → surface with manifest -POST /admin/surfaces/install ← multipart .surface archive -PUT /admin/surfaces/:id/enable -PUT /admin/surfaces/:id/disable -DELETE /admin/surfaces/:id -``` - -### Tasks Admin - -See [tasks.md](tasks.md) for the full task management API. - -``` -GET /admin/tasks → all tasks across all users/teams -POST /admin/tasks/:id/run → force run -POST /admin/tasks/:id/kill → cancel active run -DELETE /admin/tasks/:id → delete task -``` - -### Notifications Admin (v0.28.6) - -``` -POST /admin/notifications/broadcast ← { "title", "message", "level?" } - → { "message": "broadcast sent", "count": N } -``` - -Sends a `system.announcement` notification to all active users. -Also emits a `system.broadcast` WebSocket event for real-time delivery. -`level` is optional (`info` | `warning` | `critical`, default `info`). -See [notifications.md](notifications.md#admin-broadcast-v0286) for full details. diff --git a/docs/ICD/auth.md b/docs/ICD/auth.md deleted file mode 100644 index 45cc96b..0000000 --- a/docs/ICD/auth.md +++ /dev/null @@ -1,180 +0,0 @@ -# Auth - -Three auth modes, configured via `AUTH_MODE` env var. All three issue the -same JWT after authentication — downstream middleware is auth-mode-agnostic. - -## Builtin Auth (default) - -Username/password authentication with bcrypt hashing. - -### Register - -``` -POST /auth/register -``` - -```json -{ - "username": "jdoe", - "password": "...", - "email": "jdoe@example.com", - "display_name": "Jane Doe" -} -``` - -Gated by `allow_registration` policy. A unique `handle` is -auto-generated from `username` (collision-safe with `-2`, `-3` suffixes). - -**Response (201):** Same token shape as Login. If the user requires admin -approval (`is_active` = false), returns `201` with: - -```json -{ "message": "Account created but requires admin approval", "user_id": "uuid" } -``` - -### Login - -``` -POST /auth/login -``` - -```json -{ "username": "jdoe", "password": "..." } -``` - -**Response (200):** - -```json -{ - "access_token": "eyJ...", - "refresh_token": "opaque-string", - "token_type": "Bearer", - "expires_in": 900, - "user": { - "id": "uuid", - "username": "jdoe", - "email": "jdoe@example.com", - "display_name": "Jane Doe", - "handle": "jdoe", - "role": "user", - "auth_source": "builtin" - } -} -``` - -### Refresh - -``` -POST /auth/refresh -``` - -```json -{ "refresh_token": "opaque-string" } -``` - -Returns new `access_token` and `refresh_token`. Old refresh token is -invalidated (rotation). - -### Logout - -``` -POST /auth/logout -``` - -```json -{ "refresh_token": "opaque-string" } -``` - -Revokes the refresh token. Returns `{ "message": "logged out" }`. - -## mTLS Auth - -Mutual TLS via reverse proxy headers. The proxy terminates TLS and -forwards cert DN fields in headers. The backend auto-provisions -users on first connection. - -**Configured by:** `AUTH_MODE=mtls` + `MTLS_HEADER_*` env vars. - -No explicit login/register endpoints. User identity is extracted from -the TLS certificate on every request. `password_hash` is NULL for -mTLS users. - -**Headers parsed:** -- `MTLS_HEADER_CN` (default: `X-SSL-Client-CN`) — common name → username -- `MTLS_HEADER_DN` (default: `X-SSL-Client-DN`) — distinguished name → metadata -- `MTLS_HEADER_VERIFY` (default: `X-SSL-Client-Verify`) — must be `SUCCESS` -- `MTLS_HEADER_FINGERPRINT` (default: `X-SSL-Client-Fingerprint`) — stable identity - -On first request with a valid cert, the system creates a user with -`auth_source=mtls` and `external_id=fingerprint`. - -## OIDC Auth - -OpenID Connect authorization code flow. Tested with Keycloak but -compatible with any OIDC-compliant IdP. - -**Configured by:** `AUTH_MODE=oidc` + `OIDC_*` env vars. - -### OIDC Login (redirect) - -``` -GET /auth/oidc/login -``` - -Redirects to the IdP authorization endpoint. Stores `state` + `nonce` -in `oidc_auth_state` table (cleaned up after callback). - -### OIDC Callback - -``` -GET /auth/oidc/callback?code=...&state=... -``` - -Exchanges authorization code for tokens, validates ID token signature -via JWKS, extracts claims. Auto-provisions user on first login with -`auth_source=oidc` and `external_id=sub`. Returns HTML fragment that -passes tokens to the frontend via URL fragment. - -**Claim mapping:** -- `sub` → `external_id` -- `preferred_username` → `username` + `handle` -- `email` → `email` -- `name` → `display_name` -- `groups` → synced to internal groups with `source=oidc` - -**Split-horizon issuer:** `OIDC_EXTERNAL_ISSUER_URL` can differ from -`OIDC_ISSUER_URL` when the IdP's internal address (Docker/K8s service) -differs from its external address. - -**OIDC env vars:** - -| Var | Required | Description | -|-----|----------|-------------| -| `OIDC_ISSUER_URL` | Yes | IdP issuer URL for discovery | -| `OIDC_EXTERNAL_ISSUER_URL` | No | External issuer (if different from internal) | -| `OIDC_CLIENT_ID` | Yes | Client ID | -| `OIDC_CLIENT_SECRET` | Yes | Client secret | -| `OIDC_REDIRECT_URL` | No | Callback URL (auto-derived if not set) | - -## Login Page Adaptation - -The `/login` page adapts by `AUTH_MODE`: - -| Mode | UI | -|------|-----| -| `builtin` | Username/password form + register link | -| `mtls` | Certificate status display | -| `oidc` | "Sign in with SSO" button | - -## User Identity Fields - -All auth modes produce users with: - -| Field | Description | -|-------|-------------| -| `auth_source` | `builtin`, `mtls`, or `oidc` | -| `external_id` | IdP subject (OIDC) or cert fingerprint (mTLS). NULL for builtin. | -| `handle` | Unique @mention handle. Auto-generated, collision-safe. | - -`handle` is the canonical identifier for @mentions (replacing username -in the resolution chain since v0.24.0). diff --git a/docs/ICD/channels.md b/docs/ICD/channels.md deleted file mode 100644 index 0501128..0000000 --- a/docs/ICD/channels.md +++ /dev/null @@ -1,765 +0,0 @@ -# Channels & Conversations - -The **channel** is the universal conversation container. Messages, -tool activity, attachments, and KB bindings all hang off a channel. -Channels support multiple participants — users, personas, and -sessions — making them the foundation for collaborative and -multi-model workflows. - -### Channel CRUD - -**List channels** — paginated, sorted by last activity. - -``` -GET /channels?page=1&per_page=50 -``` - -Optional query filters: `type`, `types` (comma-separated), `folder`, -`folder_id`, `search`, `project_id` (UUID or `"none"`), `archived`. - -Returns pagination envelope. Each channel: - -```json -{ - "id": "uuid", - "user_id": "uuid", - "title": "My Chat", - "type": "direct|dm|group|channel|workflow|service", - "ai_mode": "auto|mention_only|off", - "topic": "string|null", - "description": "", - "model": "claude-sonnet-4-20250514", - "provider_config_id": "uuid|null", - "system_prompt": "", - "is_archived": false, - "is_pinned": false, - "folder": "string|null", - "folder_id": "uuid|null", - "project_id": "uuid|null", - "workspace_id": "uuid|null", - "tags": ["tag1", "tag2"], - "settings": {}, - "message_count": 0, - "unread_count": 0, - "created_at": "...", - "updated_at": "..." -} -``` - -**Channel types:** - -| Type | Description | -|------|-------------| -| `direct` | Single-user chat (default, backward compatible) | -| `dm` | Human-to-human direct message, AI silent unless @mentioned | -| `group` | Multi-user/multi-model collaborative channel | -| `channel` | Named persistent space, configurable ai_mode | -| `workflow` | Staged channel driven by workflow definitions | -| `service` | Autonomous task execution, no human participant | - -**Create channel:** - -``` -POST /channels -``` - -```json -{ - "title": "New Chat", - "type": "direct", - "description": "", - "model": "claude-sonnet-4-20250514", - "provider_config_id": "uuid", - "system_prompt": "", - "folder": null, - "folder_id": null, - "tags": [], - "participants": ["user-uuid"] -} -``` - -On create, the authenticated user is automatically added as a -participant with `role: "owner"`. If `model` and `provider_config_id` -are provided, the channel model roster (`channel_models`) is -auto-populated with an initial entry. - -`participants` is only used for `dm` type — array of exactly one -other user UUID. The handler enforces DM dedup: if a DM already -exists between the two users, the existing channel is returned -(HTTP 200, not 201). - -**Get channel:** - -``` -GET /channels/:id -``` - -Returns single channel object. Accessible by the channel owner -**or** any user who is a participant in the channel (via -`channel_participants`). - -**Update channel:** - -``` -PUT /channels/:id -``` - -Accepts partial updates — only fields present in the body are changed. -Same field set as create, plus `is_archived`, `is_pinned`, `settings` -(JSONB merge), `workspace_id`, `ai_mode`, `topic`, `folder_id`. - -**Auth:** Owner only (keyed on `user_id` column). - -**Delete channel:** - -``` -DELETE /channels/:id -``` - -**Retention policy (v0.37.14):** When `retention_ttl_days > 0`, all -channels are archived on delete and purged after the TTL. The only -exception is channels using a Personal (BYOK) provider — those are -always hard-deleted immediately. - -| Provider scope | TTL > 0 | TTL = 0 | -|---------------|---------|---------| -| `personal` (BYOK) | Hard delete | Hard delete | -| `global` | Archive + purge after TTL | Hard delete | -| `team` | Archive + purge after TTL | Hard delete | -| NULL (no provider) | Archive + purge after TTL | Hard delete | - -When retention applies, the channel is archived (`is_archived = true`) -and stamped with `purge_after`. A background scanner purges channels -past their `purge_after` hourly. Archived channels are hidden from -the user's sidebar. - -Non-owners who call DELETE are removed as participants ("leave channel") -instead of deleting. - -**Response (immediate delete):** `{"message": "channel deleted"}` -**Response (retention):** `{"message": "channel archived for retention", "purge_after": "..."}` -**Response (leave):** `{"message": "left channel"}` - -Hard-delete cascades: messages, files, channel_models, channel_kbs, -channel_participants. Storage cleanup runs asynchronously. - -**Auth:** Owner for delete/archive; any participant for leave. - -### Message Tree - -Switchboard uses a **tree** model for messages, not a flat list. Each -message has a `parent_id` (NULL for root). Editing creates a sibling -branch; regeneration creates an alternative sibling. The **cursor** -tracks which child is "active" at each branch point. - -**Get active path** — the primary endpoint for loading chat history: - -``` -GET /channels/:id/path -``` - -Returns the messages along the active branch from root to leaf, -following the cursor at each fork. This is what the UI renders. - -```json -{ - "messages": [ - { - "id": "uuid", - "channel_id": "uuid", - "parent_id": "uuid|null", - "role": "user|assistant|system|tool", - "content": "...", - "model": "claude-sonnet-4-20250514|null", - "provider_config_id": "uuid|null", - "participant_id": "uuid|null", - "participant_type": "user|persona|session|null", - "thinking": "...|null", - "tool_calls": [...], - "tool_results": [...], - "attachments": [...], - "has_siblings": true, - "sibling_index": 0, - "sibling_count": 2, - "created_at": "..." - } - ] -} -``` - -`participant_id` and `participant_type` identify who authored the -message. For `user` messages, this is the authenticated user. For -`assistant` messages, this is the persona (if set) or null for raw -model responses. Existing messages without participants return null. - -**List all messages** — includes all branches, flat: - -``` -GET /channels/:id/messages -``` - -Legacy/debug endpoint. Returns every message in the channel regardless -of branch. Not used by the standard frontend. - -**Create message:** - -``` -POST /channels/:id/messages -``` - -```json -{ - "role": "user", - "content": "Hello", - "parent_id": "uuid|null", - "attachments": ["attachment-uuid-1"] -} -``` - -**Edit message** — creates a sibling at the same level: - -``` -POST /channels/:id/messages/:msgId/edit -``` - -```json -{ "content": "Revised question" } -``` - -Creates a new message with the same `parent_id` as the original. -Automatically updates the cursor to point to the new sibling. - -**Regenerate** — creates an alternative assistant response: - -``` -POST /channels/:id/messages/:msgId/regenerate -``` - -```json -{ - "model": "claude-sonnet-4-20250514", - "provider_config_id": "uuid" -} -``` - -Creates a new empty assistant message as a sibling of `msgId`, -then streams the completion into it. The cursor is updated. - -**Update cursor** — switch to a different branch: - -``` -PUT /channels/:id/cursor -``` - -```json -{ - "active_leaf_id": "uuid" -} -``` - -Sets which leaf message is active. The server walks the tree from -this leaf to the root and returns the full path. -Subsequent `GET /channels/:id/path` will follow the new branch. - -**List siblings** — all children of a message's parent: - -``` -GET /channels/:id/messages/:msgId/siblings -``` - -Returns `{ "siblings": [...] }` with abbreviated message objects -(id, role, content preview, created_at). - -### Completions & Streaming - -The single most complex endpoint. Sends a message to an LLM provider -and streams the response via Server-Sent Events. - -``` -POST /chat/completions -``` - -**Request:** - -```json -{ - "channel_id": "uuid", - "message": "User's message text", - "model": "claude-sonnet-4-20250514", - "provider_config_id": "uuid", - "persona_id": "uuid|null", - "parent_id": "uuid|null", - "tools_enabled": true, - "tool_ids": ["web_search", "calculator"], - "attachments": ["attachment-uuid"], - "extra_body": {} -} -``` - -`channel_id` is required. - -`extra_body` is a freeform JSON object merged into the provider request -(provider-specific parameters like `temperature`, `reasoning`, etc.). - -**Participant scoping:** The requesting user must be a participant in -the channel. Messages are attributed to the authenticated user (or -session). In `group` channels, all participants see all messages in -real time via WebSocket. - -**Persona resolution:** The primary completion path. If `persona_id` -is set (explicitly or via `@mention` resolution from message content), -the handler loads the persona's system prompt, model, provider config, -and KB bindings. Per-message `model`/`provider_config_id` override the -persona's defaults. In multi-persona channels, `@mention` in the -message content is parsed to resolve the target persona from the -channel's participant list. - -**Routing resolution:** After config resolution, the routing policy -evaluator (`evaluateRouting()`) may redirect to a different provider -based on active policies (see §10.4). The winning provider config's -credentials are loaded and used for the actual API call. - -**SSE Stream Format:** - -The response is `Content-Type: text/event-stream`. Content and -reasoning deltas use OpenAI-compatible envelope format. Tool events -use named SSE events. - -``` -data: {"choices":[{"delta":{"content":"Hello"},"finish_reason":null}],"model":"claude-sonnet-4-20250514"} - -data: {"choices":[{"delta":{"content":" world"},"finish_reason":null}],"model":"claude-sonnet-4-20250514"} - -data: {"choices":[{"delta":{"reasoning_content":"Let me think..."},"finish_reason":null}],"model":"claude-sonnet-4-20250514"} - -event: tool_use -data: [{"id":"call_1","name":"web_search","input":{"query":"..."}}] - -event: tool_result -data: [{"id":"call_1","content":"Search results..."}] - -data: {"choices":[{"delta":{"content":"Based on the search..."},"finish_reason":null}],"model":"claude-sonnet-4-20250514"} - -data: {"choices":[{"delta":{},"finish_reason":"stop"}],"model":"claude-sonnet-4-20250514"} - -data: [DONE] -``` - -| Event | `event:` field | Payload | -|-------|---------------|---------| -| Content delta | _(unnamed)_ | OpenAI envelope: `choices[0].delta.content` | -| Reasoning delta | _(unnamed)_ | OpenAI envelope: `choices[0].delta.reasoning_content` | -| Tool invocation | `tool_use` | `[{ "id", "name", "input" }]` | -| Tool result | `tool_result` | `[{ "id", "content" }]` | -| Stream complete | _(unnamed)_ | OpenAI envelope: `choices[0].finish_reason` = `"stop"` or `"budget_exceeded"` | -| End sentinel | _(unnamed)_ | `[DONE]` (literal string) | -| Error mid-stream | _(unnamed)_ | `{ "error": "message" }` | - -**Tool cycle:** Tool use and tool result events can repeat up to -`max_tool_iterations` (configured server-side). The handler executes -tools, feeds results back to the model, and continues streaming. - -**Response headers:** - -``` -X-Switchboard-Provider: providerID/configID -X-Switchboard-Route: policy-name (when routing policy is active) -X-Switchboard-Fallback: depth (when fallback chain was triggered) -``` - -**List available tools:** - -``` -GET /tools -``` - -Returns `{ "tools": [...] }` where each tool has `name`, `description`, -`parameters` (JSON Schema), and `category`. - -### Multi-Model Roster (Raw Access) - -The primary way to add AI to a channel is by adding a **persona as a -participant** (§3.7). Adding a persona automatically populates the -channel model roster with the persona's underlying model. - -For the rare case where a user needs raw model access without a -persona wrapper, the `channel_models` table provides direct model -roster management. This is the 1% escape hatch — most users should -never need it. - -``` -GET /channels/:id/models → { "models": [...] } -POST /channels/:id/models ← { "model": "...", "provider_config_id": "...", "display_name": "..." } -PATCH /channels/:id/models/:modelId ← { "display_name": "..." } -DELETE /channels/:id/models/:modelId -``` - -Each roster entry: - -```json -{ - "id": "uuid", - "channel_id": "uuid", - "model": "claude-sonnet-4-20250514", - "provider_config_id": "uuid", - "display_name": "Claude", - "is_default": true, - "persona_id": "uuid|null", - "created_at": "..." -} -``` - -`persona_id`: set when this roster entry was auto-created by adding a -persona participant. Null for manually-added raw models. - -**`@mention` resolution:** When a user types `@` in a channel, the -autocomplete shows persona participants first (by display name), then -raw model roster entries. The target of a `@mention` is always -resolved to a specific `provider_config_id` + `model` pair — never -an ambiguous bare model name. - -### Channel Utilities - -**Summarize channel** (compaction): - -``` -POST /channels/:id/summarize -``` - -Triggers conversation summarization via the utility model role. -Returns `{ "message": "summarized", "summary_id": "uuid" }`. - -**Generate title:** - -``` -POST /channels/:id/generate-title -``` - -Uses the utility model to auto-generate a title from the first -exchange. Returns `{ "title": "Generated Title" }`. - -### Files - -All binary content associated with channels — user uploads, tool- -generated artifacts, system files — lives in the unified `files` -table backed by the ObjectStore (see §X). The old `attachments` -table and routes are removed. - -**Upload (user):** - -``` -POST /channels/:id/files -Content-Type: multipart/form-data -``` - -Field: `file`. Sets `origin: "user_upload"`. Returns the file object. - -**Create (tool output):** When `workspace_write` or `workspace_patch` -tools succeed during a completion, the handler auto-records a file -reference with `origin: "tool_output"` linked to the assistant message -(v0.37.18). No public endpoint — created internally by the completion -handler via the store layer. - -**List by channel:** - -``` -GET /channels/:id/files?origin=user_upload -``` - -Returns `{ "files": [...] }`. Filterable by `origin`, `content_type`. - -**List by message:** - -``` -GET /messages/:id/files -``` - -Returns `{ "files": [...] }`. How the frontend discovers generated -artifacts to render inline. - -**List by user (file manager):** - -``` -GET /files?page=1&per_page=50 -``` - -Returns `{ "files": [...], "total": N, "page": N, "per_page": N }`. -All files owned by the authenticated user, paginated. - -**Get metadata:** - -``` -GET /files/:id -``` - -```json -{ - "id": "uuid", - "channel_id": "uuid", - "message_id": "uuid|null", - "user_id": "uuid", - "origin": "user_upload|tool_output|system", - "filename": "report.pdf", - "content_type": "application/pdf", - "size_bytes": 1048576, - "display_hint": "inline|download|thumbnail", - "extracted_text": "...|null", - "metadata": {}, - "created_at": "...", - "updated_at": "..." -} -``` - -**Download:** - -``` -GET /files/:id/download -``` - -Returns the file with appropriate `Content-Type` and -`Content-Disposition` headers. - -**Thumbnail** _(planned — v0.28.0+):_ - -``` -GET /files/:id/thumbnail -``` - -**Delete:** - -``` -DELETE /files/:id -``` - -Deletes metadata + blob. Channel deletion cascades via FK + -`DeletePrefix`. - -### Channel Participants - -The participant system is how users and AI interact in a channel. -**Personas are the primary way to add AI** — adding a persona as a -participant brings its full identity (name, avatar, system prompt, -model, provider config, KB bindings) into the channel. This is the -99% path. Raw model access without a persona is available via the -model roster (§3.4) for power users. - -Every channel has at least one participant (the owner). `direct` -channels have exactly one user participant and optionally one or more -persona participants. `group` and `workflow` channels support multiple -participants of mixed types. - -**List participants:** - -``` -GET /channels/:id/participants -``` - -Returns `{ "participants": [...] }`: - -```json -{ - "id": "uuid", - "channel_id": "uuid", - "participant_type": "user|persona|session", - "participant_id": "uuid", - "role": "owner|member|observer", - "display_name": "Jane Doe", - "avatar_url": "...|null", - "joined_at": "..." -} -``` - -**Participant types:** - -| Type | Description | -|------|-------------| -| `user` | Authenticated user. `participant_id` = `users.id` | -| `persona` | AI persona added to channel. `participant_id` = `personas.id`. Brings model, system prompt, KB bindings | -| `session` | Anonymous/session participant (workflow intake). `participant_id` = session token | - -**Participant roles:** - -| Role | Capabilities | -|------|-------------| -| `owner` | Full control: add/remove participants, delete channel, all member capabilities | -| `member` | Send messages, trigger completions, view history | -| `observer` | Read-only: view messages, no send | - -**Add participant:** - -``` -POST /channels/:id/participants -``` - -```json -{ - "participant_type": "user|persona", - "participant_id": "uuid", - "role": "member" -} -``` - -Requires `owner` role in the channel. Adding a `persona` participant: -- Makes the persona available as an `@mention` target -- Adds the persona's model to the channel model roster automatically -- Scopes the persona's KB bindings into the channel's KB resolution chain -- The persona's avatar and display name appear in the participant list - -**Update participant role:** - -``` -PATCH /channels/:id/participants/:participantId -``` - -```json -{ "role": "observer" } -``` - -**Remove participant:** - -``` -DELETE /channels/:id/participants/:participantId -``` - -Cannot remove the last owner. Removing a `persona` participant also -removes its auto-created model roster entry. - -**Completion routing:** When `@PersonaName` appears in message -content, the completion handler resolves to that persona's model, -provider config, and system prompt. When no `@mention` is present in -a multi-persona channel, the channel's default persona (first added) -handles the response. The `persona_id` field in the completion -request (§3.3) can also be set explicitly to override `@mention` -resolution. - -**Backward compatibility:** Existing `direct` channels that predate -the participant system are auto-migrated on first access — a single -`user` participant with `role: "owner"` is created from the channel's -legacy `user_id` column. - -### Presence - -Heartbeat-based online status. Not channel-scoped — tracks global user presence. - -**Heartbeat:** - -``` -POST /presence/heartbeat -``` - -Client sends every 30 seconds. Server upserts `user_presence` row. -No request body needed. Returns `{ "ok": true }`. - -**Auth:** Authenticated. - -**Bulk query:** - -``` -GET /presence?users=uuid1,uuid2,uuid3 -``` - -Returns online/offline status for listed user UUIDs. Threshold: -online if last heartbeat < 90 seconds ago. - -```json -{ - "presence": { - "uuid1": "online", - "uuid2": "offline" - } -} -``` - -Values are flat status strings. Capped at 100 user IDs per request. - -Subsequent updates delivered via WebSocket `presence.changed` events. - -**Typing indicators** are sent via `POST /channels/:id/typing` -(authenticated, broadcasts `typing.user` WebSocket event to other -user participants) and via WebSocket (see websocket.md). The typing -event payload includes `channel_id`, `user_id`, and `display_name`. - -### Chat Folders - -User-scoped grouping for chats. Folders have no semantic weight — -they are named drawers. The schema supports nesting via `parent_id` -(DnD nesting not yet implemented in the UI). - -``` -GET /folders → { "folders": [...] } -POST /folders ← { "name": "Research", "sort_order": 0 } -PUT /folders/:id ← { "name": "Renamed", "sort_order": 1 } -DELETE /folders/:id → chats become unfiled -``` - -**Folder object:** - -```json -{ - "id": "uuid", - "name": "Research", - "parent_id": "uuid|null", - "sort_order": 0, - "created_at": "...", - "updated_at": "..." -} -``` - -Moving a chat into/out of a folder is done via `PUT /channels/:id` -with `{ "folder_id": "uuid" }` or `{ "folder_id": "" }` to unbind. - -The channel list also supports `?folder_id=uuid` as a query filter. - -**Auth:** Authenticated (user-scoped). - -### Channel Configuration - -Additional channel fields (set via `PUT /channels/:id`): - -| Field | Type | Default | Description | -|-------|------|---------|-------------| -| `ai_mode` | string | `auto` | `auto`, `mention_only`, or `off` | -| `topic` | string | null | Short description shown in header | - -Fields managed internally (not settable via channel update): - -| Field | Type | Default | Description | -|-------|------|---------|-------------| -| `allow_anonymous` | boolean | false | Session participants can join. Set by workflow instance handlers on channel creation. | -| `kb_auto_inject` | boolean | false | Top-K KB chunks auto-prepend to context _(v0.28.0 — schema exists, not yet wired)_. | - -`ai_mode` behavioral matrix: - -| Channel type | Default ai_mode | Notes | -|-------------|----------------|-------| -| `direct` | `auto` | Existing behavior | -| `dm` | `mention_only` | AI silent unless @mentioned | -| `group` | `auto` | Configurable per channel | -| `channel` | `auto` | Configurable per channel | - -### Mark Read - -``` -POST /channels/:id/mark-read -``` - -Updates the user's `last_read_at` cursor. Used for unread count -calculation. - -**Auth:** Authenticated. - -### User Search - -``` -GET /users/search?q=alice -``` - -Returns `{ "users": [{ "id", "username", "display_name", "handle" }] }`. -Matches on username, display_name, and handle (case-insensitive -substring). Used for DM creation and participant picker. Excludes the -calling user. Max 20 results. - -**Auth:** Authenticated. - - ---- diff --git a/docs/ICD/enums.md b/docs/ICD/enums.md deleted file mode 100644 index f5b447c..0000000 --- a/docs/ICD/enums.md +++ /dev/null @@ -1,282 +0,0 @@ -# Appendix: Enums & Constants - -All enum values used across the API. Definitive source of truth. - -## Channel Types - -| Value | Description | -|-------|-------------| -| `direct` | Single-user chat (default, backward compatible) | -| `dm` | Human-to-human direct message, AI silent unless @mentioned | -| `group` | Multi-user/multi-model collaborative channel | -| `channel` | Named persistent space, configurable ai_mode | -| `workflow` | Staged channel driven by workflow definitions | -| `service` | Autonomous task execution, no human participant | - -## Channel AI Modes - -| Value | Description | -|-------|-------------| -| `auto` | AI responds to every message (default) | -| `mention_only` | AI responds only when @mentioned (default for `dm`) | -| `off` | AI disabled entirely | - -## Participant Types - -| Value | Description | -|-------|-------------| -| `user` | Authenticated user. `participant_id` = `users.id` | -| `persona` | AI persona. `participant_id` = `personas.id` | -| `session` | Anonymous visitor. `participant_id` = session token | - -## Participant Roles - -| Value | Description | -|-------|-------------| -| `owner` | Full control: add/remove participants, delete channel | -| `member` | Send messages, trigger completions, view history | -| `observer` | Read-only: view messages, no send | -| `visitor` | Anonymous session participant (workflow intake) | - -## Presence Statuses - -`online`, `away`, `offline` - -## Message Roles - -`user`, `assistant`, `system`, `tool` - -## Scopes - -`personal`, `team`, `global` - -## Visibility (Model Catalog) - -`enabled`, `disabled`, `team` - -## User Roles - -`admin`, `user` - -## Team Member Roles - -`admin`, `member` - -## Provider Types - -`openai`, `anthropic`, `openrouter`, `venice` (extensible via registry) - -## Model Types - -`chat`, `embedding`, `image` - -## Memory Scopes - -`user`, `persona`, `persona_user` - -## Memory Statuses - -`active`, `pending_review`, `archived` - -## Workspace Owner Types - -`user`, `project`, `channel`, `team` - -## Workspace Statuses - -`active`, `archived`, `deleting` - -## File Index Statuses - -`pending`, `indexing`, `ready`, `error`, `skipped` - -## KB Document Statuses - -`pending`, `chunking`, `embedding`, `ready`, `error` - -## Provider Health Statuses - -`healthy`, `degraded`, `down` - -## Routing Policy Types - -| Value | Description | -|-------|-------------| -| `provider_prefer` | Ordered provider fallback list | -| `team_route` | Restrict team to specific providers | -| `cost_limit` | Heuristic cost cap per request | -| `model_alias` | Alias → provider+model rewrite | -| `capability_match` | Match cheapest model with required capabilities | - -## Extension Tiers - -`browser` (implemented), `starlark` (implemented, v0.29.0), `sidecar` (future) - -## Grant Types - -| Value | Visibility | -|-------|-----------| -| `team_only` | Only the owning team | -| `global` | All authenticated users | -| `groups` | Members of specified groups | - -## Resource Grant Resource Types - -`persona`, `knowledge_base`, `project` - -## Group Sources - -| Value | Description | -|-------|-------------| -| `manual` | Admin-created | -| `oidc` | Synced from IdP groups claim | -| `system` | Platform-seeded (e.g., Everyone group) | - -## Auth Sources - -| Value | Description | -|-------|-------------| -| `builtin` | Username/password, bcrypt | -| `mtls` | Mutual TLS via reverse proxy | -| `oidc` | OpenID Connect (Keycloak etc.) | - -## Workflow Statuses - -| Value | Description | -|-------|-------------| -| `active` | Instance is running | -| `completed` | All stages finished | -| `stale` | No activity within threshold | -| `cancelled` | Manually cancelled | - -## Workflow Entry Modes - -`public_link`, `team_only` - -## Workflow History Modes - -| Value | Description | -|-------|-------------| -| `full` | Next stage sees complete chat history | -| `summary` | Utility-role summary of previous stages | -| `fresh` | Clean slate — no history carried forward | - -## Task Types - -`prompt`, `workflow` - -## Task Run Statuses - -| Value | Description | -|-------|-------------| -| `queued` | Waiting to execute | -| `running` | Currently executing | -| `completed` | Finished successfully | -| `failed` | Error during execution | -| `budget_exceeded` | Hit token/tool/wall-clock limit | -| `cancelled` | Manually killed | - -## Task Scopes - -`personal`, `team`, `global` - -## Task Output Modes - -`channel`, `note`, `webhook` - -## Assignment Statuses - -`unassigned`, `claimed`, `completed` - -## Surface Sources - -`core`, `extension` - -## Notification Types - -Convention: `domain.action`. Free-form strings — new types don't -require migration. - -| Type | Description | -|------|-------------| -| `role.fallback` | Model role fell back to secondary provider | -| `kb.ready` | Knowledge base finished indexing | -| `kb.error` | Knowledge base indexing failed | -| `grant.changed` | User added to or removed from a group | -| `memory.extracted` | New memories extracted from conversation | -| `user.mentioned` | User was @mentioned in a channel | -| `workflow.assigned` | Workflow stage assigned to team/user | -| `workflow.claimed` | Workflow assignment claimed by a user | -| `task.completed` | Scheduled task finished successfully | -| `task.failed` | Scheduled task failed | -| `task.budget_exceeded` | Task hit token/tool/wall-clock budget | - -**Planned (not yet implemented):** -`system.announcement` (v0.28.4 — admin broadcast to all users) - -## Git Auth Types - -`https_pat`, `https_basic`, `ssh_key` - -## File Origins - -`user_upload`, `tool_output`, `system` - -## File Display Hints - -`inline`, `download`, `thumbnail` - -## Proxy Modes - -`system`, `direct`, `custom` - -## Permissions - -12 permission constants, `domain.action` convention: - -| Permission | Description | -|-----------|-------------| -| `model.use` | Use AI models for completions | -| `kb.read` | Search knowledge bases | -| `kb.create` | Create knowledge bases | -| `kb.write` | Upload documents to KBs | -| `channel.create` | Create new channels | -| `channel.invite` | Add participants to channels | -| `persona.create` | Create personal personas | -| `persona.manage` | Edit/delete own personas | -| `workflow.create` | Create/edit workflow definitions | -| `tasks.create` | Create/manage tasks | -| `tasks.admin` | Manage all tasks (admin) | -| `admin.access` | Access admin panel | - -Permissions are granted via groups. The `Everyone` group -(ID `00000000-0000-0000-0000-000000000001`) provides default -permissions to all authenticated users. It is system-seeded -and editable by admins. - -## Extension Permissions (v0.29.0+) - -7 extension permission constants, `domain.action` convention. -Declared in package manifests, granted by admin review. - -| Permission | Module | Since | -|-----------|--------|-------| -| `secrets.read` | `secrets` | v0.29.0 | -| `notifications.send` | `notifications` | v0.29.0 | -| `filters.pre_completion` | — | v0.29.0 | -| `api.http` | `http` | v0.29.0 | -| `provider.complete` | `provider` | v0.29.1 | -| `db.read` | `db` | v0.29.2 | -| `db.write` | `db` | v0.29.2 | - -## Policies - -| Key | Default | Description | -|-----|---------|-------------| -| `allow_registration` | `"true"` | Allow new user self-registration | -| `allow_user_byok` | `"false"` | Allow users to add personal API keys | -| `allow_user_personas` | `"false"` | Allow users to create personal Personas | -| `allow_team_providers` | `"true"` | Allow team admins to configure team providers | -| `allow_raw_model_access` | `"true"` | Allow raw model selection without persona | -| `kb_direct_access` | `"true"` | Show KB picker in channel (false = strict enterprise) | -| `default_user_active` | `"false"` | New users active immediately (vs admin approval) | diff --git a/docs/ICD/extensions.md b/docs/ICD/extensions.md deleted file mode 100644 index 487dcda..0000000 --- a/docs/ICD/extensions.md +++ /dev/null @@ -1,581 +0,0 @@ -# Extensions - -Plugin system with three tiers: Browser JS (client-side), Starlark -sandbox (server-side, v0.29.0), Sidecar containers (server-side, future). - -### User Extensions - -**Auth:** Authenticated user - -``` -GET /extensions → {"data": [...UserExtension]} - ?tier=browser (optional filter by tier) -POST /extensions/:id/settings ← {"is_enabled": bool, "settings": {...}} - :id = extension UUID → {"ok": true} -GET /extensions/:id/manifest → {"data": } - :id = ext_id (manifest id) -GET /extensions/tools → {"data": [...tool schema objects]} -``` - -**Notes:** -- `GET /extensions` returns `UserExtension` objects: the base extension - fields plus `user_enabled` and `user_settings` overrides. -- System extensions (`is_system: true`) cannot be disabled by users. - Attempting to set `is_enabled: false` on a system extension returns 403. -- `GET /extensions/tools` returns raw tool schema JSON from all enabled - browser extensions' `manifest.tools[]` arrays. - -### Extension API Routes (v0.29.1) - -Starlark packages serve custom JSON endpoints. Mounted at -`/s/:slug/api/*path` with JWT authentication. - -**Auth:** Authenticated user (JWT — returns 401, not redirect) - -``` -ANY /s/:slug/api/*path → Starlark on_request(req) response -``` - -**Route:** `slug` is the package ID. `path` is matched against the -manifest's `api_routes` declaration. - -**Manifest `api_routes` field:** - -```json -{ - "api_routes": [ - {"method": "GET", "path": "/status"}, - {"method": "POST", "path": "/webhook"}, - {"method": "*", "path": "/proxy/*"} - ] -} -``` - -- `method`: exact match (case-insensitive), or `"*"` for any method -- `path`: exact match, or trailing `"*"` for prefix match -- Boolean `true` shorthand: all routes forwarded -- Missing or `false`: no routes served - -**Request dict passed to `on_request(req)`:** - -```python -{ - "method": "POST", - "path": "/webhook", - "headers": {"content-type": "application/json", ...}, - "query": {"page": "1", ...}, - "body": "{...}", - "user_id": "uuid" -} -``` - -**Expected return dict:** - -```python -{"status": 200, "headers": {"X-Custom": "val"}, "body": "{...}"} -``` - -- `status`: int (default 200). Return `None` for 204 No Content. -- `headers`: dict (optional). Content-Type auto-detected if not set. -- `body`: string (default ""). - -**Validation pipeline:** -1. Package exists and is enabled -2. Status is `active` (not `pending_review` or `suspended`) -3. Tier is `starlark` -4. `api.http` permission is granted -5. Method+path matches `api_routes` in manifest - -**Errors:** -- `401` — no/invalid JWT -- `403` — package suspended or missing `api.http` permission -- `404` — package not found, disabled, or route not declared -- `400` — package is not starlark tier -- `500` — Starlark execution error - -### Admin Extension Management - -**Auth:** Admin role - -``` -GET /admin/extensions → {"data": [...Extension]} -POST /admin/extensions ← {ext_id*, name*, version?, tier?, - description?, author?, manifest?, - is_system?, is_enabled?} - → {"data": Extension} (201) -PUT /admin/extensions/:id ← {name?, version?, description?, - :id = extension UUID author?, is_system?, is_enabled?, - manifest?} - → {"data": Extension} -DELETE /admin/extensions/:id → {"ok": true} - :id = extension UUID -``` - -**Install defaults:** `version` → `"0.0.0"`, `tier` → `"browser"`, -`manifest` → `{}`, `scope` → `"global"`. - -**Tier validation:** `tier` must be one of: `browser`, `starlark`, `sidecar`. - -**Duplicate rejection:** If `ext_id` is already installed, returns 409. - -### Asset Serving - -**Auth:** None (public — script tags can't send Authorization headers) - -``` -GET /extensions/:id/assets/*path → application/javascript - :id = ext_id (manifest id) -``` - -Returns the inline `_script` field from the extension's manifest. -The `*path` segment is accepted but currently ignored (all requests -return the same script). Disabled extensions return 404. - -### Extension Permissions (v0.29.0+) - -Starlark packages declare capabilities in `manifest.permissions`. -Admin must grant each before the package activates. - -| Permission | Module | Description | -|-----------|--------|-------------| -| `secrets.read` | `secrets` | Read extension secrets via GlobalConfig | -| `notifications.send` | `notifications` | Send in-app notifications | -| `filters.pre_completion` | — | Register pre-completion filter | -| `api.http` | `http` | Outbound HTTP requests (v0.29.0: module, v0.29.1: also required for API routes) | -| `provider.complete` | `provider` | LLM completion calls via BYOK chain (v0.29.1) | -| `db.read` | `db` | Read extension tables and platform views (v0.29.2) | -| `db.write` | `db` | Write extension tables (v0.29.2) | - -### Starlark Modules (v0.29.0+) - -Modules injected into the script namespace based on granted permissions: - -**`secrets`** (requires `secrets.read`): -- `secrets.get(key)` → string or None -- `secrets.list()` → list of key names - -**`notifications`** (requires `notifications.send`): -- `notifications.send(user_id, title, body?, type?)` → True - -**`http`** (requires `api.http`, v0.29.1): -- `http.get(url, headers?)` → `{"status": int, "headers": dict, "body": string}` -- `http.post(url, body?, headers?)` → response dict -- `http.put(url, body?, headers?)` → response dict -- `http.delete(url, headers?)` → response dict -- `http.request(method, url, body?, headers?)` → response dict -- SSRF protection: private/loopback/link-local IPs blocked after DNS -- Manifest `network_access`: `{"allow": [...]}` or `{"block": [...]}` - -**`provider`** (requires `provider.complete`, v0.29.1): -- `provider.complete(messages, model?, max_tokens?, temperature?)` → response dict -- Response: `{"content", "model", "finish_reason", "input_tokens", "output_tokens"}` -- Provider resolved via BYOK chain; pinnable via `requires_provider.provider_config_id` - -**`db`** (requires `db.read` or `db.write`, v0.29.2): -- `db.query(table, filters=None, order=None, limit=100)` → list of dicts - - `table`: logical name (physical: `ext_{pkg_slug}_{table}`) - - `filters`: dict of `{column: value}` equality filters (optional) - - `order`: `"col"` or `"-col"` for descending (optional) - - `limit`: max rows (default 100) -- `db.insert(table, row_dict)` → inserted row dict with auto-generated `id` (`db.write`) -- `db.update(table, id, partial_dict)` → True (`db.write`) -- `db.delete(table, id)` → True (`db.write`) -- `db.list_tables()` → list of logical table names for this package -- `db.view(view_name, filters=None, limit=100)` → list of dicts - - Allowed view names: `"users"` → `ext_view_users`, `"channels"` → `ext_view_channels` - - `ext_view_users`: `id`, `display_name`, `email` - - `ext_view_channels`: `id`, `title`, `type`, `team_id` - -### Extension Database Tables (v0.29.2) - -Starlark packages declare owned tables in `manifest.db_tables`. Tables -are created on install and dropped on uninstall. - -**Manifest `db_tables` field:** - -```json -{ - "db_tables": { - "logs": { - "columns": { - "message": "text", - "user_id": "text", - "count": "int", - "score": "real", - "active": "bool", - "created_at": "timestamp" - }, - "indexes": [ - ["user_id"], - ["user_id", "created_at"] - ] - } - } -} -``` - -- **Physical name**: `ext_{pkg_slug}_{logical_name}` (hyphens → underscores) -- **Auto-columns**: `id TEXT PRIMARY KEY` (UUID generated on insert), - `created_at` (dialect-correct timestamp default) -- **Column types**: `text`, `int`/`integer`, `real`/`float`, - `bool`/`boolean`, `timestamp` — mapped to dialect-correct SQL -- **Indexes**: each entry is a list of columns for a composite index -- **Dialect**: PG uses `TIMESTAMPTZ`/`BOOLEAN`; SQLite uses `TEXT`/`INTEGER` -- **Catalog**: tracked in `ext_data_tables` per package for lifecycle management - -### Extension Tools (v0.29.2) - -Starlark packages declare server-side tools in `manifest.tools`. These -are included in `BuildToolDefs` alongside server tools. The completion -tool loop dispatches matched calls to the `on_tool_call` entry point. - -**Manifest `tools` field (starlark tier only):** - -```json -{ - "tier": "starlark", - "permissions": ["db.read"], - "tools": [ - { - "name": "search_logs", - "description": "Search extension log entries", - "parameters": { - "type": "object", - "properties": { - "query": {"type": "string"}, - "user_id": {"type": "string"} - }, - "required": ["query"] - } - } - ] -} -``` - -**`on_tool_call(call)` entry point:** - -Called by the completion tool loop when a tool declared in `tools` is -invoked by the LLM. - -```python -def on_tool_call(call): - # call dict: - # { - # "tool_name": "search_logs", - # "tool_call_id": "call_abc123", - # "arguments": {"query": "hello", "user_id": "u1"} - # } - if call["tool_name"] == "search_logs": - rows = db.query("logs", filters={"user_id": call["arguments"]["user_id"]}) - return {"results": rows} - return {"error": "unknown tool"} -``` - -- Return value is serialized to JSON and returned as the tool result -- All sandbox modules (including `db`) are available per granted permissions -- No additional permission is required beyond package being `active` - -### Multi-File Starlark Packages (v0.38.0) - -Starlark packages support `load()` for splitting code across multiple -files. The entry point script is `script.star` (or the `entry_point` -manifest field). Submodules live in `star/`. - -**Archive structure:** - -``` -my-extension/ -├── manifest.json ← metadata only (no _starlark_script) -├── script.star ← entry point (required for starlark tier) -├── star/ ← optional submodules -│ ├── auth.star -│ ├── repos.star -│ └── helpers.star -├── js/ ← optional surface assets -└── css/ -``` - -**Manifest `entry_point` field (optional):** - -```json -{ - "tier": "starlark", - "entry_point": "script.star" -} -``` - -Default is `script.star`. The installer validates the entry point exists -in the archive for starlark-tier packages (returns 400 if missing). - -**`load()` in Starlark scripts:** - -```python -# script.star -load("star/auth.star", "auth_headers") -load("star/repos.star", "get_repos") - -def on_request(req): - headers = auth_headers(conn) - repos = get_repos(conn) - return {"status": 200, "body": json.encode(repos)} -``` - -**Constraints:** -- Package-scoped: can only load files within the package's own directory -- `.star` files only — cannot load `.js`, `.json`, etc. -- Path traversal (`..`) and absolute paths are rejected -- Circular dependencies are detected and rejected -- Loaded files get the same injected modules (`db`, `http`, etc.) as the entry point -- All loaded files share the same step limit budget (1M ops total) - -**Backward compatibility:** Existing packages with `_starlark_script` -in their manifest continue to work. The runner tries disk first, then -falls back to the manifest field. Reinstalling extracts `script.star` -to disk. - -**Install errors:** -- `400` — `starlark package missing entry point "script.star"` (or - custom `entry_point` value) - -## 5. Extension Connections (v0.38.1) - -Scoped credential management for extensions that integrate with -external services. Same scope/resolution pattern as provider configs -(personal → team → global). - -### Personal Connections - -**Auth:** Authenticated user - -``` -GET /connections → {"data": [...connection summaries]} -POST /connections ← {"type","package_id","name","config"} → {"id","type","name"} -GET /connections/:id → connection summary (owned only) -PUT /connections/:id ← {"name?","config?"} → {"message":"connection updated"} -DELETE /connections/:id → {"message":"connection deleted"} -GET /connections/resolve → resolved connection (decrypted config) - ?type=gitea&name=Work+Gitea -``` - -### Team Connections - -**Auth:** Team admin (RequireTeamAdmin middleware) - -``` -GET /teams/:teamId/connections → {"data": [...connection summaries]} -POST /teams/:teamId/connections ← {"type","package_id","name","config"} → {"id","type","name"} -PUT /teams/:teamId/connections/:id ← {"name?","config?"} → {"message":"connection updated"} -DELETE /teams/:teamId/connections/:id → {"message":"connection deleted"} -``` - -### Admin (Global) Connections - -**Auth:** Admin - -``` -GET /admin/connections → {"data": [...connection summaries]} -POST /admin/connections ← {"type","package_id","name","config"} → {"id","type","name"} -PUT /admin/connections/:id ← {"name?","config?"} → {"message":"connection updated"} -DELETE /admin/connections/:id → {"message":"connection deleted"} -``` - -**Connection summary** (list/get responses — secrets masked): -```json -{ - "id": "uuid", "type": "gitea", "package_id": "git-board", - "scope": "personal", "name": "Work Gitea", - "is_active": true, "has_config": true, - "created_at": "2026-03-25T...", "updated_at": "2026-03-25T..." -} -``` - -**Scope resolution:** `GET /connections/resolve?type=gitea` resolves -via personal → team → global chain. Returns full connection with -decrypted config. Optional `name` parameter for named resolution. - -**Config encryption:** Fields with `type: "secret"` in the package -manifest's `connections[].fields` are encrypted at rest using the -same vault pattern as provider API keys. The handler layer encrypts -on write and decrypts on read. The store is encryption-agnostic. - -**Unique constraint:** `(type, scope, owner_id, name)` — a user cannot -have two connections of the same type with the same name. - -**Starlark module:** Extensions with `connections.read` permission -get a `connections` module: -```python -conn = connections.get("gitea") # scope chain -conn = connections.get("gitea", name="Work Gitea") # named -all = connections.list("gitea") # all accessible -``` - -Each returned dict contains `id`, `type`, `name`, `scope` plus all -config fields with secrets decrypted. - -### Connection Type Discovery (v0.38.4) - -**Auth:** Authenticated user (no specific permission required) - -``` -GET /connection-types → {"data": [...connection type entries]} -``` - -Returns the merged set of connection types declared by all active packages. -When multiple packages declare the same type name, library declarations -take precedence over non-library declarations. - -**Response entry:** -```json -{ - "type": "gitea", - "label": "Gitea Instance", - "package_id": "gitea-client", - "package_title": "Gitea API Client", - "fields": { - "base_url": {"type": "url", "required": "true", "label": "Server URL"}, - "api_token": {"type": "secret", "required": "true", "label": "API Token"}, - "org": {"type": "string", "required": "false", "label": "Default Org"} - }, - "scopes": ["global", "team", "personal"] -} -``` - -Used by all three Connections management UIs (Settings, Admin, Team Admin) -to populate the connection type dropdown. Replaces client-side manifest -scanning which required admin permissions. - ---- - -### Library Composition (v0.38.4) - -Libraries declare connection types in their manifest `connections[]` field. -Consumers depend on the library and use its exported functions, passing -connection dicts obtained from `connections.get()`. - -**End-to-end pattern:** - -1. Library declares `connections: [{ "type": "gitea", ... }]` in manifest -2. Library exports functions that accept a `conn` parameter -3. Admin installs library and grants permissions (`api.http`, `connections.read`, etc.) -4. User creates a connection of the library's type via Settings > Connections -5. Consumer declares `dependencies: { "gitea-client": ">=1.0.0" }` in manifest -6. Consumer loads library: `gitea = lib.load("gitea-client")` -7. Consumer resolves connection: `conn = connections.get("gitea")` -8. Consumer calls library: `gitea.get_repos(conn)` - -The library function executes with the **library's** permission context. -The consumer does not need `api.http` or `db.*` permissions — those are -the library's concern. - -**Browser-side consumers** can also call the library's REST endpoints -directly (`GET /s/gitea-client/api/repos`) without Starlark, enabling -pure `type: "surface"` packages. - ---- - -### Library Packages (v0.38.2) - -Library packages (`type: "library"`) export Starlark functions for other -packages to consume. Libraries run with their own permission context, -isolating consumers from implementation details. - -**Manifest fields:** -- `type: "library"` — required -- `tier: "starlark"` — required (libraries are always starlark-tier) -- `exports: ["fn_a", "fn_b"]` — required, names of globals to expose -- `permissions: [...]` — the library's own permissions (not inherited by consumers) -- `dependencies: {"other-lib": ">=1.0.0"}` — optional, libraries can depend on other libraries - -**Restrictions:** Libraries cannot have `tools`, `pipes`, or a `route`. - -**Consumer usage (Starlark):** - -Consumers declare dependencies in their manifest: -```json -{ "dependencies": { "gitea-client": ">=1.0.0" } } -``` - -Then load at runtime: -```python -gitea = lib.load("gitea-client") -repos = gitea.get_repos(conn) -``` - -`lib.load()` validates the dependency record, loads the library's script -with the library's own permissions, extracts the declared exports, and -returns them as a frozen struct. Results are cached per invocation. - -**Admin endpoints:** - -| Method | Path | Description | -|--------|------|-------------| -| GET | `/api/v1/admin/packages/:id/dependencies` | Libraries this package depends on | -| GET | `/api/v1/admin/packages/:id/consumers` | Packages that depend on this library | -| GET | `/api/v1/admin/dependencies` | Full dependency graph | - -**Uninstall protection:** Libraries with active consumers return 409 on delete. - ---- - -### Config Sections (v0.38.3) - -Packages can declare a `config_section` in their manifest to inject a -Preact configuration component into the Settings, Admin, or Team Admin -surfaces. This enables headless extensions and libraries to own their -configuration UX without requiring a full surface. - -**Manifest schema:** - -```json -{ - "config_section": { - "label": "My Extension", - "icon": "M12 2L2 7...", - "component": "js/config.js", - "surfaces": ["admin", "settings", "team-admin"], - "category": "system" - } -} -``` - -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| `label` | string | Yes | Nav link text. Falls back to package title if empty. | -| `icon` | string | No | SVG path data in compact format (admin CatIcon). | -| `component` | string | No | Relative path within the package archive. Default: `js/config.js`. | -| `surfaces` | string[] | Yes | Target surfaces: `admin`, `settings`, `team-admin`. | -| `category` | string | No | Admin-only: category tab to appear under. Default: `system`. | - -**Component contract:** - -The config section component is a standard ES module exporting a -default Preact component. It is lazy-loaded via dynamic `import()` from -the existing asset-serving endpoint (`GET /surfaces/:id/:component`). - -The component uses `sw.sdk` to read/write its own package settings: -- `sw.api.admin.packages.settings(packageId)` — read -- `sw.api.admin.packages.updateSettings(packageId, data)` — write - -Settings are stored in the `package_settings` JSONB column (admin scope) -and `package_user_settings` table (user scope). - -**Discovery:** At page load, the backend queries enabled packages for -`config_section` entries targeting the current surface and passes them -as `__CONFIG_SECTIONS__` to the frontend. The surface merges them into -its nav and section module map. - -**No new tables or endpoints.** Config sections are purely manifest-driven, -using existing package storage and settings infrastructure. - ---- - -### Builtin Seeding - -On startup, `SeedBuiltinExtensions()` scans `extensions/builtin/` -for subdirectories containing `manifest.json` + `script.js`. Each -is upserted as a system extension: -- **New ext_id** → `Create` with `is_system: true` -- **Same version** → skip (idempotent) -- **Different version** → `Update` manifest, name, description, author - ---- diff --git a/docs/ICD/knowledge.md b/docs/ICD/knowledge.md deleted file mode 100644 index a0093ab..0000000 --- a/docs/ICD/knowledge.md +++ /dev/null @@ -1,271 +0,0 @@ -# Knowledge Bases - -The **Knowledge Base** (KB) is the RAG system: upload documents → chunk -→ embed (pgvector / app-level cosine for SQLite) → search via -`kb_search` tool during completions. - -### KB CRUD - -``` -GET /knowledge-bases → { "data": [KB objects], "total": N } -POST /knowledge-bases ← { "name", "description", "scope", "team_id" } -GET /knowledge-bases/:id → KB object -PUT /knowledge-bases/:id ← partial update { "name", "description" } -DELETE /knowledge-bases/:id -``` - -**Auth:** All endpoints require a valid JWT. `POST` additionally requires -the `kb.create` permission (checked via `RequirePermission` middleware). - -**Scope rules for `POST`:** - -| Scope | Requirement | -|-------|-------------| -| `personal` (default) | Any authenticated user with `kb.create` | -| `team` | `team_id` required; caller must be team admin (or system admin) | -| `global` | Caller must be system admin | - -KB object (as returned by CRUD endpoints): - -```json -{ - "id": "uuid", - "name": "Product Docs", - "description": "Internal product documentation", - "scope": "personal|team|global", - "owner_id": "uuid|null", - "team_id": "uuid|null", - "embedding_config": {}, - "document_count": 12, - "chunk_count": 48, - "total_bytes": 245760, - "status": "active|processing|error", - "created_at": "2025-01-15T10:30:00Z", - "updated_at": "2025-01-15T10:30:00Z" -} -``` - -> **Note:** The `discoverable` field is stored in the DB and present on -> the raw model, but is **not** included in the `kbResponse` struct -> returned by CRUD endpoints. It is managed via the dedicated -> discoverable endpoints below. - -### Documents - -**Upload:** - -``` -POST /knowledge-bases/:id/documents -Content-Type: multipart/form-data -``` - -**Auth:** Requires `kb.write` permission (checked via `RequirePermission` -middleware). Also requires a configured embedding model role — returns -`412 Precondition Failed` if missing. - -Field: `file` (max 10 MB). Currently supported types: **TXT, MD, CSV, -HTML**. Binary formats (PDF, DOCX, XLSX, PPTX) are planned but require -an extraction sidecar that is not yet wired in. - -Returns the document object with HTTP `202 Accepted`: - -```json -{ - "id": "uuid", - "kb_id": "uuid", - "filename": "guide.md", - "content_type": "text/markdown", - "size_bytes": 12345, - "chunk_count": 0, - "status": "pending", - "error": null, - "uploaded_by": "uuid", - "created_at": "2025-01-15T10:30:00Z", - "updated_at": "2025-01-15T10:30:00Z" -} -``` - -**List:** - -``` -GET /knowledge-bases/:id/documents -``` - -Returns `{ "data": [document objects], "total": N }`. - -**Status** (poll during processing): - -``` -GET /knowledge-bases/:id/documents/:docId/status -``` - -Returns a subset of the document object: - -```json -{ - "id": "uuid", - "status": "chunking", - "chunk_count": 0, - "error": null, - "filename": "guide.md" -} -``` - -Status progression: `pending → extracting → chunking → embedding → ready | error`. - -**Delete:** - -``` -DELETE /knowledge-bases/:id/documents/:docId -``` - -Deletes the document, its chunks, and the stored file. Returns -`{ "deleted": true }`. - -### Search - -``` -POST /knowledge-bases/:id/search -``` - -```json -{ - "query": "How do I configure SSO?", - "limit": 5, - "threshold": 0.3 -} -``` - -`limit` defaults to 5, max 20. `threshold` is the minimum cosine -similarity score (defaults to 0.3). - -Returns `{ "data": [...], "query": "...", "total": N }`: - -```json -{ - "data": [ - { - "content": "To configure SSO, navigate to...", - "source": "admin-guide.md", - "kb": "Product Docs", - "similarity": 0.87, - "metadata": {} - } - ], - "query": "How do I configure SSO?", - "total": 1 -} -``` - -### Rebuild - -Re-chunks and re-embeds all documents. Skips documents whose extracted -text is missing (would need the original file from object store). - -``` -POST /knowledge-bases/:id/rebuild -``` - -Returns `202 Accepted`: - -```json -{ - "message": "rebuild started", - "total_docs": 12, - "rebuilding": 10, - "skipped": 2 -} -``` - -Returns `503` if the ingestion pipeline is not configured, or `400` if -the KB has no documents. - -### Channel KB Bindings - -A channel can have KBs explicitly bound to it (in addition to any KBs -coming from the active Persona or parent Project). - -**Get:** - -``` -GET /channels/:id/knowledge-bases -``` - -Returns `{ "data": [ChannelKB objects] }`: - -```json -{ - "kb_id": "uuid", - "kb_name": "Product Docs", - "enabled": true, - "document_count": 12 -} -``` - -**Set** (replace all): - -``` -PUT /channels/:id/knowledge-bases -``` - -```json -{ "kb_ids": ["kb-uuid-1", "kb-uuid-2"] } -``` - -Validates that the caller has access to every referenced KB before -setting the bindings. - -### Discoverable KBs - -The `discoverable` flag controls whether a KB appears in the user's KB -picker. Non-discoverable KBs are hidden from direct access but still -searchable through Persona bindings (enterprise KB mode). - -**List discoverable** (for KB picker UI): - -``` -GET /knowledge-bases-discoverable -``` - -Returns `{ "data": [KB objects] }` — only KBs the user can see -(personal + team + discoverable global). Uses the same `kbResponse` -format as the CRUD endpoints. - -**Toggle discoverability:** - -``` -PUT /knowledge-bases/:id/discoverable -``` - -```json -{ "discoverable": false } -``` - -**Auth:** Requires `loadAndAuthorize` (KB must exist, caller must have -basic access). Additionally, only the KB **owner** or a **system admin** -may change the flag — all other callers receive `403 Forbidden`. - -### KB Resolution Chain - -When a completion fires, KBs are resolved in priority order: - -``` -Persona KBs → Project KBs → Channel KBs → Personal KBs -``` - -The `BuildKBHint` function assembles all applicable KBs into the -system prompt, and the `kb_search` tool searches across all of them. -The `kb_search` tool additionally resolves project-bound KBs and -personal KBs at search time, so all user-accessible KBs are -searchable even if not explicitly listed in the hint. - -### Persona KB Bindings - -Managed via the Persona endpoints — see [personas.md](personas.md). -Persona-KB binding is the primary enterprise mechanism for controlled -KB access. The `auto_search` flag on persona KB bindings controls -whether top-K results are prepended to context automatically (planned, -see v0.28.4) vs available only through the `kb_search` tool (current -behavior). - ---- diff --git a/docs/ICD/memory.md b/docs/ICD/memory.md deleted file mode 100644 index 21e209b..0000000 --- a/docs/ICD/memory.md +++ /dev/null @@ -1,95 +0,0 @@ -# Memory - -**Long-term memory** extracted from conversations. The system -identifies facts, preferences, and context about the user and stores -them for injection into future conversations. - -**Auth:** All user endpoints require authenticated session. -Admin endpoints require platform admin role. - ---- - -### User Memory - -``` -GET /memories → { "data": [memory objects] } -GET /memories/count → { "active": 5, "pending": 2 } -PUT /memories/:id ← { "key": "...", "value": "...", "confidence": 0.9 } -DELETE /memories/:id -POST /memories/:id/approve → approve a pending_review memory -POST /memories/:id/reject → reject (archive) a pending_review memory -``` - -Query params for `GET /memories`: - -| Param | Default | Description | -|-------|---------|-------------| -| `status` | `active` | Filter by status: `active`, `pending_review`, `archived` | -| `query` | — | Keyword filter on key+value (PG: full-text, SQLite: LIKE) | - -Memory object: - -```json -{ - "id": "uuid", - "scope": "user|persona|persona_user", - "owner_id": "uuid", - "user_id": "uuid|null", - "key": "preferred language", - "value": "Go with Gin framework for backend services", - "source_channel_id": "uuid|null", - "confidence": 0.85, - "status": "active|pending_review|archived", - "created_at": "...", - "updated_at": "..." -} -``` - -**Scopes:** - -| Scope | `owner_id` | `user_id` | Description | -|-------|-----------|-----------|-------------| -| `user` | user's ID | null | Personal facts, persist across all conversations | -| `persona` | persona's ID | null | Shared across all users of a Persona | -| `persona_user` | persona's ID | user's ID | Per-user facts within a Persona context | - -**Ownership:** `PUT` and `DELETE` enforce ownership — a user can only -modify their own user-scope memories (`owner_id == user_id`). - -### Admin Memory Review - -``` -GET /admin/memories/pending → { "data": [memory objects] } -POST /admin/memories/bulk-approve ← { "ids": ["uuid1", "uuid2"] } - → { "approved": 2 } -``` - -Platform admin can review and bulk-approve pending memories across -all users. - -### AI Tools (non-REST) - -Two tools are registered for AI use during completions: - -| Tool | Description | -|------|-------------| -| `memory_save` | Save a fact with `key`, `value`, `confidence`. Saves as `active` (bypasses review). | -| `memory_recall` | Search memories by query. Uses hybrid semantic+keyword recall when embedder is configured. | - -### Background Extraction - -A background scanner periodically finds conversations with enough -new activity and calls the utility model to extract memorable facts. -Extracted memories are saved as `pending_review`. - -Controlled by `memory_extraction_enabled` global config key and -per-persona `memory_enabled` flag. - -### Memory Injection - -Active memories are injected into the system prompt at completion -time via `BuildMemoryHint`. Uses hybrid semantic recall when an -embedder is available, falling back to keyword/confidence ranking. -Budget: ~6000 characters. - ---- diff --git a/docs/ICD/models.md b/docs/ICD/models.md deleted file mode 100644 index c81fbae..0000000 --- a/docs/ICD/models.md +++ /dev/null @@ -1,258 +0,0 @@ -# Models & Preferences - -What models are available to the current user and how they control -visibility. - ---- - -### Enabled Models - -The primary endpoint for populating model selectors: - -``` -GET /models/enabled -``` - -Returns a composite response: - -```json -{ - "data": [ UserModel, ... ], - "default_model": "claude-sonnet-4-20250514" -} -``` - -`default_model` is the admin-configured default model ID (from the -`default_model` policy). Empty string if unset. - -**UserModel object:** - -```json -{ - "id": "composite-id", - "display_name": "Claude Sonnet 4", - "model_id": "claude-sonnet-4-20250514", - "model_type": "chat", - "source": "catalog", - - "provider_config_id": "uuid", - "config_id": "uuid", - "provider_name": "Anthropic", - "provider_type": "anthropic", - - "capabilities": { - "streaming": true, - "tool_calling": true, - "vision": true, - "thinking": true, - "reasoning": false, - "code_optimized": false, - "web_search": false, - "max_context": 200000, - "max_output_tokens": 8192 - }, - - "is_persona": false, - "persona_id": "", - "persona_handle": "", - "persona_scope": "", - "persona_avatar": "", - "persona_team_name": "", - - "description": "", - "icon": "", - "avatar": "", - "system_prompt": "", - "temperature": null, - "max_tokens": null, - "tool_grants": [], - - "pricing": { - "input_per_m": 3.00, - "output_per_m": 15.00, - "currency": "USD" - }, - - "scope": "global", - "owner_id": null, - "team_name": "", - - "hidden": false, - "sort_order": 0, - - "provider_status": "healthy" -} -``` - -**Field reference:** - -| Field | Type | Description | -|-------|------|-------------| -| `id` | string | Composite key. Catalog: `{config_id}:{model_id}`. Persona: persona UUID. | -| `display_name` | string | Human-readable name (from catalog or persona). | -| `model_id` | string | Raw model identifier (e.g. `claude-sonnet-4-20250514`). | -| `model_type` | string | `"chat"`, `"embedding"`, `"image"`, etc. | -| `source` | string | `"catalog"` (DB catalog entry) or `"persona"`. | -| `provider_config_id` | string | UUID of the provider config that serves this model. | -| `config_id` | string | Alias of `provider_config_id` (frontend compat). | -| `provider_name` | string | Display name of the provider config. | -| `provider_type` | string | Provider type slug (e.g. `"anthropic"`, `"openai"`). | -| `capabilities` | object | Nested `ModelCapabilities` — see below. | -| `is_persona` | bool | `true` if this entry is a Persona, not a raw model. | -| `persona_id` | string | Persona UUID (omitted if not a persona). | -| `persona_handle` | string | `@handle` for mention routing (omitted if not a persona). | -| `persona_scope` | string | Persona scope: `"global"`, `"team"`, `"personal"` (omitted if not). | -| `persona_avatar` | string | Avatar URL (omitted if not a persona or unset). | -| `persona_team_name` | string | Owning team name for team-scoped personas (omitted if N/A). | -| `description` | string | Persona description (omitted if empty). | -| `icon` | string | Persona icon (omitted if empty). | -| `avatar` | string | Avatar URL (omitted if empty). | -| `system_prompt` | string | Persona system prompt (omitted if empty). | -| `temperature` | float\|null | Persona temperature override. | -| `max_tokens` | int\|null | Persona max-tokens override. | -| `tool_grants` | string[] | Tool IDs granted to this persona. | -| `pricing` | object\|null | Nested `ModelPricing` — see below. Omitted if unavailable. | -| `scope` | string | `"global"`, `"team"`, or `"personal"`. | -| `owner_id` | string\|null | Owner user/team ID for team/personal scoped entries. | -| `team_name` | string | Team name (persona entries only). | -| `hidden` | bool | `true` if the user has hidden this model via preferences. | -| `sort_order` | int | User's custom sort position (0 = default). | -| `provider_status` | string | Health status: `"healthy"`, `"degraded"`, `"down"`, `"unknown"`, or empty. | - -**ModelCapabilities:** - -| Field | Type | Description | -|-------|------|-------------| -| `streaming` | bool | Supports streaming responses. | -| `tool_calling` | bool | Supports function/tool calling. | -| `vision` | bool | Supports image inputs. | -| `thinking` | bool | Supports extended thinking. | -| `reasoning` | bool | Reasoning-optimized model. | -| `code_optimized` | bool | Code-optimized model. | -| `web_search` | bool | Has built-in web search. | -| `max_context` | int | Context window size in tokens. | -| `max_output_tokens` | int | Maximum output tokens. | - -Capabilities are resolved through the three-tier chain: catalog DB → -heuristic inference → admin overrides (see `providers.md` §capabilities). - -**ModelPricing:** - -| Field | Type | Description | -|-------|------|-------------| -| `input_per_m` | float | Input cost per million tokens. | -| `output_per_m` | float | Output cost per million tokens. | -| `currency` | string | Currency code (e.g. `"USD"`). | - -**Persona inheritance:** When `is_persona` is `true`, the `capabilities` -and `pricing` objects are inherited from the persona's underlying base -model. The persona adds identity (name, handle, avatar, system prompt, -tool grants) — not capability restrictions. Persona entries always have -`hidden: false`; persona visibility is controlled by the grant system -and the `is_active` flag, not by model preferences. - -**Model allowlist filtering (v0.24.2):** Non-admin users whose groups -define model restrictions will only see models in their allowlist. -Persona entries whose underlying `model_id` is not in the allowlist are -also filtered. Admins bypass this filter entirely. - -**Visibility resolution order:** - -1. Global catalog: enabled models → all authenticated users -2. Team catalog: enabled models → team members -3. Personal BYOK: enabled models → owner (requires `allow_user_byok` policy) -4. Personas: global + team + personal + shared-via-grants -5. User hidden preferences applied last (sets `hidden` flag, does not remove) -6. Model allowlist filtering (removes entries entirely for non-admins) - ---- - -### User Model Preferences - -Users can hide models they don't want to see and set per-model -defaults. Preferences are keyed on the **composite identity** -`provider_config_id:model_id` — the same model from different -providers can have independent visibility. - -``` -GET /models/preferences → { "data": [PreferenceEntry, ...] } -PUT /models/preferences ← { "model_id": "...", "provider_config_id": "uuid", ... } -POST /models/preferences/bulk ← { "entries": [...], "hidden": bool } -``` - -**GET /models/preferences** - -Returns all preference entries for the authenticated user. - -PreferenceEntry: - -```json -{ - "id": "uuid", - "user_id": "uuid", - "model_id": "grok-4.1-fast", - "provider_config_id": "uuid", - "hidden": true, - "preferred_temperature": 0.7, - "preferred_max_tokens": 4096, - "sort_order": 0, - "created_at": "2025-06-15T14:30:00Z", - "updated_at": "2025-06-15T14:30:00Z" -} -``` - -**PUT /models/preferences** - -Upserts a single preference entry. All fields except `model_id` are -optional — only provided fields are updated. If no entry exists, one -is created. - -Request body: - -```json -{ - "model_id": "claude-sonnet-4-20250514", - "provider_config_id": "uuid", - "hidden": true, - "preferred_temperature": 0.7, - "preferred_max_tokens": 4096, - "sort_order": 1 -} -``` - -`model_id` and `provider_config_id` are required. All other fields are -optional — only provided fields are updated. If no entry exists, one -is created. - -Returns `{ "message": "preference updated" }`. - -**POST /models/preferences/bulk** - -Sets the hidden state for multiple model+provider pairs at once. The -`hidden` value is applied to **all** entries in the batch — this is -not per-entry hidden control. - -Request body: - -```json -{ - "entries": [ - { "model_id": "grok-4.1-fast", "provider_config_id": "uuid" }, - { "model_id": "llama-3.1-70b", "provider_config_id": "uuid" } - ], - "hidden": true -} -``` - -Returns `{ "message": "preferences updated", "count": 2 }`. - -**Identity rule:** `provider_config_id` is required on write. The same -bare `model_id` from two different provider configs (e.g. global Venice -vs personal BYOK Venice) are independent preference entries. The -frontend composite ID format is `{provider_config_id}:{model_id}`. - -**Persona preferences:** Personas are not in this table. A persona's -visibility is controlled by the grant system and the `is_active` flag, -not by model preferences. - ---- diff --git a/docs/ICD/notes.md b/docs/ICD/notes.md deleted file mode 100644 index bbb2ef1..0000000 --- a/docs/ICD/notes.md +++ /dev/null @@ -1,105 +0,0 @@ -# Notes - -**Obsidian-style** knowledge graph with `[[wikilinks]]`, backlinks, -graph visualization, and folder organization. Notes are user-scoped -(personal notebook). Future: channel-scoped notes for workflow -artifacts. - -### CRUD - -``` -GET /notes → paginated list of noteListItem -POST /notes ← { "title", "content", "folder" } -GET /notes/:id → full note object -PUT /notes/:id ← { "title", "content", "folder" } -DELETE /notes/:id -``` - -Note object: - -```json -{ - "id": "uuid", - "user_id": "uuid", - "title": "Meeting Notes", - "content": "# Meeting Notes\n\nDiscussed [[Project Alpha]]...", - "folder": "work", - "source_message_id": "uuid|null", - "created_at": "...", - "updated_at": "..." -} -``` - -`source_message_id` links a note back to the chat message that -created it (provenance for AI-generated notes). - -`noteListItem` is the same but with `content` truncated or omitted. - -### Search - -**Full-text search:** - -``` -GET /notes/search?q=project+alpha -``` - -Uses `plainto_tsquery` (Postgres) or `LIKE` fallback (SQLite). -Returns `{ "data": [searchResult objects] }` with match highlights. - -**Title search** (lightweight, for wikilink autocomplete): - -``` -GET /notes/search-titles?q=proj -``` - -Returns `{ "data": [{ "id", "title" }] }`. - -### Graph - -**Full graph topology:** - -``` -GET /notes/graph -``` - -```json -{ - "nodes": [{ "id": "uuid", "title": "Note Title" }], - "edges": [{ "source": "uuid", "target": "uuid" }], - "unresolved": [{ "source": "uuid", "target_title": "Missing Note" }] -} -``` - -Edges are directed: source contains `[[target_title]]`. Unresolved -edges have a `target_title` but no matching note (dangling wikilink). -When a note with that title is created, the edge auto-resolves. - -**Backlinks** (who links to this note): - -``` -GET /notes/:id/backlinks -``` - -Returns `{ "data": [note objects] }` — all notes containing -`[[this note's title]]`. - -### Folders - -``` -GET /notes/folders -``` - -Returns `{ "folders": ["work", "personal", "archive"] }` — distinct -folder values across all user notes. - -### Bulk Operations - -``` -POST /notes/bulk-delete -``` - -```json -{ "ids": ["uuid-1", "uuid-2"] } -``` - ---- diff --git a/docs/ICD/notifications.md b/docs/ICD/notifications.md deleted file mode 100644 index 4b96593..0000000 --- a/docs/ICD/notifications.md +++ /dev/null @@ -1,155 +0,0 @@ -# Notifications - -Real-time notification infrastructure with WebSocket delivery and -optional email transport. - -**Auth:** All endpoints require authentication (JWT via `sb_token` -cookie or `Authorization: Bearer` header). - -### CRUD - -``` -GET /notifications → paginated list -GET /notifications/unread-count → { "count": 5 } -PATCH /notifications/:id/read → mark one as read -POST /notifications/mark-all-read → mark all as read -DELETE /notifications/:id -``` - -**`GET /notifications`** — paginated list for the authenticated user. - -Query parameters: - -| Param | Type | Default | Description | -|-------|------|---------|-------------| -| `limit` | int | 20 | 1–100, clamped | -| `offset` | int | 0 | Pagination offset | -| `unread_only` | bool | false | Filter to unread only | - -Response: - -```json -{ - "data": [ ... ], - "total": 42, - "limit": 20, - "offset": 0 -} -``` - -Notification object: - -```json -{ - "id": "uuid", - "user_id": "uuid", - "type": "role.fallback", - "title": "Role Fallback Triggered", - "body": "Primary model unavailable, using fallback", - "resource_type": "channel", - "resource_id": "uuid", - "is_read": false, - "created_at": "2025-01-15T12:00:00Z" -} -``` - -`resource_type` and `resource_id` are optional — used for -click-to-navigate. The resource may be deleted (no FK constraint). - -### Preferences - -Users control per-type delivery: - -``` -GET /notifications/preferences → { "data": [...] } -PUT /notifications/preferences/:type ← { "in_app": true, "email": false } -DELETE /notifications/preferences/:type → reset to default -``` - -**`GET /notifications/preferences`** returns all preferences set by -the user. Empty array if none are configured. - -**`PUT /notifications/preferences/:type`** creates or updates a -preference. Both `in_app` and `email` fields are optional (merges -with existing values if set). - -**`DELETE /notifications/preferences/:type`** removes the preference, -falling back to the next level in the resolution chain. Idempotent — -deleting a non-existent preference returns 200. - -Resolution: specific type pref → user wildcard `*` pref → system -default (`in_app=true`, `email=false`). - -Returns 503 if the preference store is not available (unmanaged mode). - -### Admin - -``` -POST /admin/notifications/test-email → send test email to requesting admin -``` - -Requires admin role. Loads SMTP config from platform settings, -sends a test message to the admin's registered email address. - -### Admin Broadcast (v0.28.6) - -``` -POST /admin/notifications/broadcast -``` - -**Auth:** Admin only. - -Sends a `system.announcement` notification to all active users via -`NotifyMany`. Also emits a `system.broadcast` WebSocket event for -real-time visibility. - -**Request body:** - -```json -{ - "title": "Scheduled Maintenance", - "message": "Servers will be down from 2-4 AM EST.", - "level": "warning" -} -``` - -| Field | Type | Required | Description | -|-------|------|----------|-------------| -| `title` | string | yes | Short summary shown in bell dropdown | -| `message` | string | yes | Detail text | -| `level` | string | no | `info` (default), `warning`, or `critical` | - -**Response:** - -```json -{ "message": "broadcast sent", "count": 42 } -``` - -`count` is the number of active users notified. - -### WebSocket Events - -| Event | Payload | Description | -|-------|---------|-------------| -| `notification.new` | Full notification object | New notification created | -| `notification.read` | `{"id": "uuid"}` | Single notification marked read | -| `notification.read` | `{"action": "mark_all_read"}` | All notifications marked read | -| `system.broadcast` | `{"title", "message", "level"}` | Admin announcement (v0.28.6) | - -These are targeted via `Hub.SendToUser` — no room subscription -required. Used for badge sync across tabs. - -### Notification Types - -See [enums.md](enums.md#notification-types) for the canonical list. -Types are free-form strings (`domain.action` convention) — new types -don't require migration. Core types include `system.announcement` -(admin broadcast, v0.28.6). - -### Retention - -Background cleanup prunes notifications older than 90 days (default, -configurable via `WithRetention`). Runs daily after a 5-minute -startup delay. - ---- diff --git a/docs/ICD/personas.md b/docs/ICD/personas.md deleted file mode 100644 index 79d40eb..0000000 --- a/docs/ICD/personas.md +++ /dev/null @@ -1,243 +0,0 @@ -# Personas - -A **Persona** is the unit of access control and AI identity: system -prompt + model + provider config + KB bindings + grant scoping. Users -interact with Personas, not raw provider configs. Admins curate which -Personas are available; team admins create team-scoped Personas. - -Three scopes, three route namespaces, one domain: - -| Scope | Routes | Who manages | -|-------|--------|-------------| -| Personal | `GET/POST/PUT/DELETE /personas` | The user | -| Team | `GET/POST/PUT/DELETE /teams/:teamId/personas` | Team admin | -| Global (admin) | `GET/POST/PUT/DELETE /admin/personas` | Platform admin | - -All three return the same Persona object shape: - -```json -{ - "id": "uuid", - "name": "Code Reviewer", - "handle": "code-reviewer", - "description": "Reviews code for bugs and style", - "icon": "🔍", - "avatar": "data:image/png;base64,...", - "base_model_id": "claude-sonnet-4-20250514", - "provider_config_id": "uuid|null", - "system_prompt": "You are a senior code reviewer...", - "temperature": 0.7, - "max_tokens": 4096, - "thinking_budget": null, - "top_p": null, - "scope": "personal|team|global", - "owner_id": "uuid|null", - "created_by": "uuid", - "is_active": true, - "is_shared": false, - "memory_enabled": false, - "memory_extraction_prompt": null, - "grants": [], - "kb_ids": [], - "created_at": "...", - "updated_at": "..." -} -``` - -`grants` and `kb_ids` are loaded from junction tables, not stored on the -persona row. All nullable fields (`provider_config_id`, `owner_id`, -`temperature`, `max_tokens`, `thinking_budget`, `top_p`, -`memory_extraction_prompt`) are omitted from JSON when null. - -`handle` is auto-generated from `name` on creation if not provided -(e.g. "Code Reviewer" → "code-reviewer"). Used for @mention routing. - -### Personal Personas - -Gated by `allow_user_personas` policy. - -``` -GET /personas → { "data": [...] } -POST /personas ← { "name", "base_model_id", ... } -PUT /personas/:id ← partial update -DELETE /personas/:id -``` - -**Auth:** JWT required. `POST` requires `persona:create` permission. -`PUT`/`DELETE` require `persona:manage` permission and ownership check -(`scope == "personal"` and `owner_id == caller`). - -### Team Personas - -``` -GET /teams/:teamId/personas → { "data": [...] } -POST /teams/:teamId/personas ← same shape -PUT /teams/:teamId/personas/:id ← partial update -DELETE /teams/:teamId/personas/:id -``` - -**Auth:** JWT required. Team membership for `GET`, team admin for -`POST`/`PUT`/`DELETE`. All mutating endpoints verify the persona belongs -to the team in the URL path (`scope == "team"` and `owner_id == teamId`). - -### Admin (Global) Personas - -``` -GET /admin/personas → { "data": [...] } -POST /admin/personas ← same shape -PUT /admin/personas/:id -DELETE /admin/personas/:id -``` - -**Auth:** Admin JWT required (middleware enforced). - -### Persona KB Bindings - -A Persona can have Knowledge Bases bound to it. When a channel uses -that Persona, the bound KBs are automatically scoped into `kb_search` -tool calls. Users never see or manage the underlying KBs directly — -they talk to the Persona. - -**Get bindings:** - -``` -GET /personas/:id/knowledge-bases → { "data": [KB objects] } -GET /teams/:teamId/personas/:id/knowledge-bases → { "data": [...] } -GET /admin/personas/:id/knowledge-bases → { "data": [...] } -``` - -**Set bindings** (replace all): - -``` -PUT /personas/:id/knowledge-bases -PUT /teams/:teamId/personas/:id/knowledge-bases -PUT /admin/personas/:id/knowledge-bases -``` - -```json -{ - "kb_ids": ["kb-uuid-1", "kb-uuid-2"], - "auto_search": { - "kb-uuid-1": true, - "kb-uuid-2": false - } -} -``` - -`auto_search`: when `true`, the KB is always searched (prepended to -context). When `false`, it's available via the `kb_search` tool but -not auto-injected. - -Team-scoped KB binding endpoints verify the persona belongs to the -team in the URL path before reading or writing bindings. - -### Persona Avatars - -Avatars are stored as `data:image/png;base64,...` data URIs in the -`avatar` column. Upload accepts a JSON body (not multipart): - -```json -{ "image": "data:image/png;base64,..." } -``` - -The image is decoded, resized to 128×128 PNG, and stored. - -``` -POST /personas/:id/avatar ← { "image": "base64..." } -DELETE /personas/:id/avatar -POST /admin/personas/:id/avatar ← same -DELETE /admin/personas/:id/avatar -POST /teams/:teamId/personas/:id/avatar ← same (verify team ownership) -DELETE /teams/:teamId/personas/:id/avatar -``` - -**Auth:** Personal avatar routes verify the caller owns the persona -(`scope == "personal"` and `owner_id == caller`). Admin routes are -protected by admin middleware. - -### Persona Tool Grants - -Control which tools a persona can use during completions. The completion -handler applies tool grants as a second-pass allowlist. - -``` -GET /personas/:id/tool-grants → { "data": ["web_search", "kb_search", ...] } -PUT /personas/:id/tool-grants ← { "tool_names": ["web_search", "calculator"] } -``` - -Admin equivalents: - -``` -GET /admin/personas/:id/tool-grants -PUT /admin/personas/:id/tool-grants -``` - -Team equivalents (verify persona belongs to team): - -``` -GET /teams/:teamId/personas/:id/tool-grants -PUT /teams/:teamId/personas/:id/tool-grants -``` - -When a workflow version is published, persona tool grants at that moment -are frozen into the version snapshot. - -### Persona Groups - -Saved roster templates. A persona group is a named set of personas that -can be stamped onto new conversations. - -``` -GET /persona-groups → { "data": [...] } -POST /persona-groups ← { "name", "description" } -GET /persona-groups/:id → group with members -PUT /persona-groups/:id ← partial update -DELETE /persona-groups/:id -``` - -**Members:** - -``` -POST /persona-groups/:id/members ← { "persona_id", "is_leader": false } -DELETE /persona-groups/:id/members/:memberId -``` - -`is_leader`: the leader persona responds when no @mention is present in -a group channel. Only one leader per group. Setting a new leader -automatically clears the existing one. - -**Persona Group Object:** - -```json -{ - "id": "uuid", - "name": "Support Team", - "description": "...", - "owner_id": "uuid", - "scope": "personal", - "team_id": null, - "members": [ - { - "id": "uuid", - "group_id": "uuid", - "persona_id": "uuid", - "is_leader": true, - "sort_order": 0, - "persona_name": "...", - "persona_handle": "...", - "persona_avatar": "..." - } - ], - "created_at": "...", - "updated_at": "..." -} -``` - -**Note:** Persona groups are currently personal-scope only. The `scope` -and `team_id` fields exist in the schema but `POST` hardcodes -`scope = 'personal'`. Team-scoped persona groups are a future item. - -### Resource Grants - -See [teams.md](teams.md) §15.3 for the grant system. Personas use grants -to control who can see and use them beyond their base scope. diff --git a/docs/ICD/profile.md b/docs/ICD/profile.md deleted file mode 100644 index 50d7ab1..0000000 --- a/docs/ICD/profile.md +++ /dev/null @@ -1,309 +0,0 @@ -# User Profile & Settings - -User identity, preferences, and credentials. - -**All endpoints require:** `Auth()` middleware (JWT bearer token). - ---- - -### Get Profile - -``` -GET /profile -``` - -**Auth:** Authenticated user - -Returns the current user's profile. - -```json -{ - "id": "uuid", - "username": "jdoe", - "email": "jdoe@example.com", - "display_name": "Jane Doe", - "role": "user", - "avatar": "data:image/png;base64,...", - "settings": { "theme": "dark" }, - "created_at": "2025-06-15T14:30:00Z", - "last_login_at": "2025-06-15T14:30:00Z" -} -``` - -| Field | Type | Notes | -|-------|------|-------| -| `id` | string | UUIDv4 | -| `username` | string | Unique login name | -| `email` | string | Unique, lowercased | -| `display_name` | string \| null | Optional friendly name | -| `role` | `"user"` \| `"admin"` | Platform role | -| `avatar` | string \| null | Data URI (PNG, 128×128). Omitted if unset. | -| `settings` | object | User preferences (theme, keybindings, etc.) | -| `created_at` | string | ISO 8601 timestamp | -| `last_login_at` | string \| null | ISO 8601 timestamp. Null if never logged in. | - -> **Note:** The profile endpoint returns `avatar` (not `avatar_url`) as a -> curated response shape. The `User` model uses `avatar_url` internally. - ---- - -### Update Profile - -``` -PUT /profile -``` - -**Auth:** Authenticated user - -```json -{ - "display_name": "New Name", - "email": "new@example.com" -} -``` - -Both fields are optional (partial update). Returns the full profile -(same shape as `GET /profile`). - -**Errors:** -- `409` — email already taken - ---- - -### Upload Avatar - -``` -POST /profile/avatar -``` - -**Auth:** Authenticated user - -**Content-Type:** `application/json` - -```json -{ - "image": "data:image/png;base64,..." -} -``` - -Accepts a base64 data URI or raw base64 string. The server decodes, -resizes to 128×128 PNG, and stores as a data URI. Max input: 2 MB. - -**Response:** - -```json -{ - "avatar": "data:image/png;base64,..." -} -``` - -**Errors:** -- `400` — missing `image` field, invalid base64, unsupported format, too large - ---- - -### Delete Avatar - -``` -DELETE /profile/avatar -``` - -**Auth:** Authenticated user - -**Response:** - -```json -{ - "message": "avatar removed" -} -``` - ---- - -### Change Password - -``` -POST /profile/password -``` - -**Auth:** Authenticated user (builtin auth only) - -```json -{ - "current_password": "old-pass", - "new_password": "new-pass-min-8" -} -``` - -Validates `current_password` against stored bcrypt hash. `new_password` -must be 8–128 characters. - -On success, the UEK (User Encryption Key) is re-wrapped with the new -password so BYOK-encrypted provider keys remain accessible. - -**Response:** - -```json -{ - "message": "password updated" -} -``` - -**Errors:** -- `400` — missing fields, password too short/long -- `401` — current password incorrect - ---- - -### Get Settings - -``` -GET /settings -``` - -**Auth:** Authenticated user - -Returns user-level preferences wrapped in a `settings` key. - -```json -{ - "settings": { - "theme": "dark", - "editor_keybindings": "vim", - "default_model": "claude-sonnet-4-5" - } -} -``` - -This is a **composite response** (named key wrapping a single object), -not a bare object. The `settings` key is always present; its value is -`{}` if the user has never saved preferences. - ---- - -### Update Settings - -``` -PUT /settings -``` - -**Auth:** Authenticated user - -```json -{ - "theme": "light", - "new_key": "new_value" -} -``` - -Shallow merge: incoming keys overwrite existing, unmentioned keys are -preserved. Handles edge cases: SQL NULL, JSON `null`, and corrupted -array values are all normalized to `{}` before merge. - -Returns the updated settings (same shape as `GET /settings`). - ---- - -### Get My Permissions - -``` -GET /profile/permissions -``` - -**Auth:** Authenticated user - -Returns the current user's resolved permission set, contributing group -IDs, team memberships, and UI-relevant policies. Admin users receive all -permissions by definition. - -**Response:** - -```json -{ - "permissions": ["model.use", "kb.read", "channel.create"], - "groups": ["group-id-1", "00000000-0000-0000-0000-000000000001"], - "teams": [ - { "id": "uuid", "name": "Engineering", "my_role": "member" } - ], - "policies": { - "allow_user_byok": false, - "allow_user_personas": false, - "allow_raw_model_access": true, - "kb_direct_access": true - } -} -``` - -| Field | Type | Description | -|-------|------|-------------| -| `permissions` | string[] | Resolved permission set (union of all group permissions) | -| `groups` | string[] | Contributing group IDs (always includes Everyone) | -| `teams` | object[] | User's active team memberships with `my_role` | -| `policies` | object | Boolean policy flags relevant to UI feature gating | - -`permissions` is sorted alphabetically. `teams` includes only active teams -the user is a member of. - -This endpoint is the SDK's bootstrap source for `sw.can(permission)`. -Called once at login and on each token refresh. - ---- - -### Bootstrap (v0.37.15) - -``` -GET /profile/bootstrap -``` - -**Auth:** Authenticated user - -Single-call boot payload for the SDK. Collapses what previously required -3–4 sequential requests (profile, permissions, teams/mine, settings) into -one call. The SDK calls this at startup and on token refresh. - -**Response:** - -```json -{ - "user": { - "id": "uuid", - "username": "jdoe", - "display_name": "Jane Doe", - "email": "jdoe@example.com", - "role": "user", - "avatar": "data:image/png;base64,..." - }, - "permissions": ["channel.create", "kb.read", "model.use"], - "groups": ["group-id-1", "00000000-0000-0000-0000-000000000001"], - "teams": [ - { "id": "uuid", "name": "Engineering", "my_role": "member" } - ], - "policies": { - "allow_user_byok": false, - "allow_user_personas": false, - "allow_raw_model_access": true, - "kb_direct_access": true - }, - "settings": { - "theme": "dark", - "editor_keybindings": "vim" - } -} -``` - -| Field | Type | Description | -|-------|------|-------------| -| `user` | object | Curated profile (same fields as `GET /profile` minus timestamps) | -| `permissions` | string[] | Resolved permission set (sorted). Admins get all. | -| `groups` | string[] | Contributing group IDs (always includes Everyone) | -| `teams` | object[] | Active team memberships with `my_role` | -| `policies` | object | Boolean policy flags for UI feature gating | -| `settings` | object | User preferences (`{}` if never saved) | - -`user.avatar` is omitted when unset (not `null`). - -This is a **superset** of `GET /profile/permissions` — it includes -everything that endpoint returns plus `user` and `settings`. Clients -that only need permissions can continue using the narrower endpoint. - ---- diff --git a/docs/ICD/projects.md b/docs/ICD/projects.md deleted file mode 100644 index bdb4bfc..0000000 --- a/docs/ICD/projects.md +++ /dev/null @@ -1,336 +0,0 @@ -# Projects - -**Projects** are organizational containers that group channels, -knowledge bases, notes, and files. They follow the scope model -(personal, team, global). - -All endpoints require authentication (`Authorization: Bearer `). -Admin endpoints additionally require `role=admin`. - -## Project Object - -```json -{ - "id": "uuid", - "name": "Q3 Research", - "description": "...", - "color": "#3b82f6", - "icon": "folder", - "scope": "personal|team|global", - "owner_id": "uuid", - "team_id": "uuid|null", - "workspace_id": "uuid|null", - "is_archived": false, - "settings": {}, - "channel_count": 5, - "kb_count": 2, - "note_count": 3, - "created_at": "2025-01-15T10:30:00Z", - "updated_at": "2025-01-15T10:30:00Z" -} -``` - -| Field | Type | Notes | -|-------|------|-------| -| `id` | uuid | PK, auto-generated | -| `name` | string | Required, max 200 chars | -| `description` | string | Optional | -| `color` | string? | CSS hex color, max 7 chars | -| `icon` | string? | Icon identifier, max 50 chars | -| `scope` | enum | `personal`, `team`, `global` | -| `owner_id` | uuid | Set from JWT, immutable | -| `team_id` | uuid? | Set for team-scoped projects | -| `workspace_id` | uuid? | Linked workspace | -| `is_archived` | bool | Default `false` | -| `settings` | object | JSONB, default `{}` | -| `channel_count` | int | Computed, omitted when 0 | -| `kb_count` | int | Computed, omitted when 0 | -| `note_count` | int | Computed, omitted when 0 | -| `created_at` | datetime | Auto-set | -| `updated_at` | datetime | Auto-updated on change | - -## Project CRUD - -### List Projects - -``` -GET /projects → { "data": [...] } -``` - -Query params: `?include_archived=true` to include archived projects. - -Returns projects the user can access: personal (own), team (member), -and global. Ordered by name. - -### Create Project - -``` -POST /projects → 201, project object -``` - -```json -{ - "name": "My Project", - "description": "Optional description", - "color": "#3b82f6", - "icon": "folder" -} -``` - -| Field | Required | Notes | -|-------|----------|-------| -| `name` | yes | Max 200 chars | -| `description` | no | | -| `color` | no | CSS hex | -| `icon` | no | Icon identifier | - -Scope is set to `personal` and `owner_id` is taken from the JWT. -Returns the created project object (bare, no envelope). - -### Get Project - -``` -GET /projects/:id → project object -``` - -Returns bare project object (no envelope). 404 if not found or -not accessible. - -### Update Project - -``` -PUT /projects/:id → updated project object -``` - -Partial update — only supplied fields are changed. - -```json -{ - "name": "New Name", - "description": "Updated", - "color": "#ef4444", - "icon": "star", - "is_archived": true, - "workspace_id": "uuid|null", - "settings": { "key": "value" } -} -``` - -Settings are merged (overlay, not replace). Returns the refreshed -project object after update. - -### Delete Project - -``` -DELETE /projects/:id → { "message": "project deleted" } -``` - -Only the project owner or an admin can delete. Channels get -`project_id` set to NULL; junction rows cascade. - ---- - -## Channel Association - -``` -GET /projects/:id/channels → { "data": [...] } -POST /projects/:id/channels ← { "channel_id", "position" } -DELETE /projects/:id/channels/:channelId → { "message": "channel removed" } -PUT /projects/:id/channels/reorder ← { "channel_ids": ["uuid1", "uuid2"] } -``` - -A channel can only belong to one project. `POST` performs an atomic -move if the channel is already in a different project. The user must -own the channel being added. - -**Channel association object:** - -```json -{ - "project_id": "uuid", - "channel_id": "uuid", - "position": 0, - "folder": "", - "added_at": "2025-01-15T10:30:00Z" -} -``` - -**POST request:** - -| Field | Required | Notes | -|-------|----------|-------| -| `channel_id` | yes | UUID of channel to add | -| `position` | no | Sort order (default 0) | - ---- - -## KB Association - -``` -GET /projects/:id/knowledge-bases → { "data": [...] } -POST /projects/:id/knowledge-bases ← { "kb_id", "auto_search" } -DELETE /projects/:id/knowledge-bases/:kbId → { "message": "KB removed" } -``` - -KBs bound to a project are automatically available to every channel -in that project (resolution chain injection at completion time). -Upsert on conflict — re-posting updates `auto_search`. - -**KB association object:** - -```json -{ - "project_id": "uuid", - "kb_id": "uuid", - "auto_search": true, - "added_at": "2025-01-15T10:30:00Z", - "name": "KB Name" -} -``` - -`name` is enriched via JOIN from `knowledge_bases.name`. - -**POST request:** - -| Field | Required | Notes | -|-------|----------|-------| -| `kb_id` | yes | UUID of knowledge base | -| `auto_search` | no | Default `false` | - ---- - -## Note Association - -``` -GET /projects/:id/notes → { "data": [...] } -POST /projects/:id/notes ← { "note_id" } -DELETE /projects/:id/notes/:noteId → { "message": "note removed" } -``` - -**Note association object:** - -```json -{ - "project_id": "uuid", - "note_id": "uuid", - "added_at": "2025-01-15T10:30:00Z", - "title": "Note Title" -} -``` - -`title` is enriched via JOIN from `notes.title`. -Insert uses ON CONFLICT DO NOTHING (idempotent). - ---- - -## Project Files (v0.37.17 — workspace-backed) - -Project files are stored in the project's workspace (auto-created on -first upload). The response uses `"files"` key, not `"data"`. Files -are `WorkspaceFile` objects with tree structure (directories + files). - -### List Files - -``` -GET /projects/:id/files → { "files": [...], "count": N } - ?path=/docs&recursive=true -``` - -| Param | Default | Notes | -|-------|---------|-------| -| `path` | `""` | Directory to list (empty = root) | -| `recursive` | `true` | Include subdirectories | - -### Upload File - -``` -POST /projects/:id/files ← multipart/form-data - ?path=/docs -``` - -Multipart form field: `file`. Optional `path` query param to upload -into a subdirectory. Auto-creates the project workspace on first upload. -Returns `201` with `{ path, filename, content_type, size_bytes }`. - -**Errors:** `413` if file exceeds size limit or workspace quota. - -### Download File - -``` -GET /projects/:id/files/download?path=/docs/readme.md -``` - -Streams raw file content with `Content-Disposition: attachment`. - -### Delete File - -``` -DELETE /projects/:id/files?path=/docs/readme.md - &recursive=false -``` - -`recursive=true` to delete directories with contents. - -### Create Directory - -``` -POST /projects/:id/files/mkdir ← { "path": "/docs/images" } -``` - -Path also accepted via `?path=` query param. Returns `201`. - -### Upload Archive - -``` -POST /projects/:id/archive/upload ← multipart/form-data -``` - -Multipart field: `file`. Accepts `.zip`, `.tar.gz`, `.tgz`. -Extracts contents into the workspace root. -Returns `{ ok: true, files_extracted: N }`. - -### Download Archive - -``` -GET /projects/:id/archive/download?format=zip -``` - -Bundles all project files into a zip (or `tar.gz`). -Streams with `Content-Disposition: attachment`. - ---- - -## Admin Project Management - -``` -GET /admin/projects → { "data": [...] } -DELETE /admin/projects/:id → { "message": "project deleted" } -``` - -Query params: `?include_archived=true`. - -Cross-instance visibility for platform admins. Admin list returns -an enriched object with extra fields: - -```json -{ - "id": "uuid", - "name": "...", - "description": "...", - "scope": "personal", - "owner_id": "uuid", - "team_id": null, - "is_archived": false, - "created_at": "...", - "updated_at": "...", - "channel_count": 5, - "kb_count": 2, - "note_count": 3, - "owner_name": "jdoe" -} -``` - -Note: admin list object omits `color`, `icon`, `workspace_id`, -and `settings` (uses a local struct, not the full model). - ---- diff --git a/docs/ICD/providers.md b/docs/ICD/providers.md deleted file mode 100644 index df10f42..0000000 --- a/docs/ICD/providers.md +++ /dev/null @@ -1,221 +0,0 @@ -# Providers & Routing - -The **multi-provider** system. One or more LLM providers are configured, -each with their own API keys, endpoints, and model catalogs. The routing -layer decides which provider handles each request. - -### User BYOK Provider Configs - -Gated by `allow_user_byok` policy. Personal API keys are encrypted with -the user's UEK (per-user encryption key, Argon2id-derived). Platform -admins cannot recover personal keys. - -``` -GET /api-configs → { "configs": [safeConfig objects] } -POST /api-configs ← { "name", "provider", "endpoint", "api_key", ... } -GET /api-configs/:id → safeConfig -PUT /api-configs/:id ← partial update (api_key optional) -DELETE /api-configs/:id -``` - -`safeConfig` — API keys are **never** returned: - -```json -{ - "id": "uuid", - "name": "My OpenAI", - "provider": "openai", - "endpoint": "https://api.openai.com/v1", - "model_default": "gpt-4o", - "scope": "personal", - "owner_id": "uuid", - "is_active": true, - "has_key": true, - "config": {}, - "headers": {}, - "settings": {}, - "created_at": "..." -} -``` - -**List models for a user config:** - -``` -GET /api-configs/:id/models -``` - -Returns `{ "models": [catalog entries] }`. - -**Fetch/sync models from provider API:** - -``` -POST /api-configs/:id/models/fetch -``` - -Calls the provider's model list API, upserts into the local catalog. - -### Admin Global Provider Configs - -``` -GET /admin/configs → { "configs": [configWithKey objects] } -POST /admin/configs ← { "name", "provider", "endpoint", "api_key", "config", "headers", "settings", "is_private" } -PUT /admin/configs/:id ← partial update -DELETE /admin/configs/:id -``` - -`configWithKey` is the same as `safeConfig` but comes from -`ListGlobal` — still redacts API keys, just adds the `has_key` flag. - -Admin configs use the `ENCRYPTION_KEY` env var (not per-user UEK). -`is_private`: when true, the config is available for admin-created -Personas but not directly selectable by users. - -### Model Catalog (Admin) - -The catalog is populated by fetching from provider APIs and stores -model metadata (capabilities, context window, pricing). - -``` -GET /admin/models → { "models": [catalog entries] } -PUT /admin/models/:id ← { "visibility", "display_name", ... } -PUT /admin/models/bulk ← { "provider_config_id", "visibility" } -DELETE /admin/models/:id -POST /admin/models/fetch ← { "provider_config_id": "uuid|empty" } -``` - -**Fetch** with empty `provider_config_id` syncs ALL active global -providers. Returns: - -```json -{ - "message": "models synced", - "added": 5, - "updated": 12, - "total": 47 -} -``` - -Or for multi-provider fetch: `{ "added", "updated", "total", "errors": [...] }`. - -**Bulk visibility** sets all models for a provider (or all models -globally if no `provider_config_id`) to the specified visibility -(`enabled`, `disabled`, `team`). - -### Provider Health - -Real-time health tracking per provider config. The health accumulator -records success/failure/latency on every completion, flushes to DB -every 60 seconds. - -**Get all:** - -``` -GET /admin/providers/health -``` - -Returns `{ "data": [ProviderHealth objects] }`: - -```json -{ - "provider_config_id": "uuid", - "provider_config_name": "OpenAI Production", - "status": "healthy|degraded|down", - "error_rate": 0.02, - "avg_latency_ms": 450, - "timeout_rate": 0.01, - "rate_limit_count": 3, - "last_check": "...", - "last_error": "...|null" -} -``` - -**Get single:** - -``` -GET /admin/providers/:id/health -``` - -**Auto-disable:** After `PROVIDER_AUTO_DISABLE_THRESHOLD` consecutive -"down" windows (default: 3), the provider is automatically deactivated. - -### Capability Overrides - -Admin can override any model capability detected by the catalog or -heuristic layer. - -``` -GET /admin/capability-overrides → { "data": [...] } -GET /admin/models/:id/capabilities → capabilities for one model -PUT /admin/models/:id/capabilities ← { "field": "value" } -DELETE /admin/models/:id/capabilities/:overrideId -``` - -Override fields: `supports_vision`, `supports_tools`, `supports_thinking`, -`context_window`, `max_output_tokens`, etc. - -### Routing Policies - -Policy-based request routing. Evaluated after model/config resolution, -before provider dispatch. - -**Admin CRUD:** - -``` -GET /admin/routing/policies → { "data": [...] } -GET /admin/routing/policies/:id → policy object -POST /admin/routing/policies ← { "name", "scope", "team_id", "priority", "policy_type", "config", "is_active" } -PUT /admin/routing/policies/:id -DELETE /admin/routing/policies/:id -``` - -Policy object: - -```json -{ - "id": "uuid", - "name": "Prefer Anthropic", - "scope": "global|team", - "team_id": "uuid|null", - "priority": 10, - "policy_type": "provider_prefer|team_route|cost_limit|model_alias|capability_match", - "config": {}, - "is_active": true -} -``` - -| Policy type | Config | Behavior | -|------------|--------|----------| -| `provider_prefer` | `{ "providers": ["cfg-1", "cfg-2"] }` | Ordered fallback list | -| `team_route` | `{ "providers": ["cfg-1"] }` | Restrict team to specific providers | -| `cost_limit` | `{ "max_cost_per_request": 0.50 }` | Heuristic cost cap | -| `model_alias` | `{ "alias": "fast", "target_model": "...", "target_config": "..." }` | Alias → provider+model rewrite | -| `capability_match` | `{ "require": ["tool_calling"], "prefer": "cheapest" }` | Match cheapest model with required capabilities | - -**Dry-run test:** - -``` -POST /admin/routing/test -``` - -```json -{ - "model": "claude-sonnet-4-20250514", - "user_id": "uuid", - "team_id": "uuid|null" -} -``` - -Returns the ranked candidate list with health status for each. - -### Provider Types - -Registry of supported provider types with metadata. - -``` -GET /admin/provider-types -``` - -Returns `{ "types": [...] }` with name, display name, default endpoint, -profile schema, and supported features per type. - ---- diff --git a/docs/ICD/surfaces.md b/docs/ICD/surfaces.md deleted file mode 100644 index 1f16582..0000000 --- a/docs/ICD/surfaces.md +++ /dev/null @@ -1,183 +0,0 @@ -# Surfaces - -Dynamic surface lifecycle. Core surfaces are registered at startup. -Extension surfaces are uploaded by admins and served from the -`surface_registry` table. - -## User Endpoints - -### List Enabled Surfaces - -``` -GET /surfaces -``` - -Returns enabled surfaces with minimal nav info (no `source` or `enabled` -fields — these are admin concerns). - -```json -{ - "surfaces": [ - { "id": "chat", "title": "Chat", "route": "/" }, - { "id": "editor", "title": "Editor", "route": "/editor" }, - { "id": "hello-dashboard", "title": "Hello Dashboard", "route": "/s/hello-dashboard" } - ] -} -``` - -**Auth:** Authenticated. - -## Admin Endpoints - -### List All Surfaces - -``` -GET /admin/surfaces -``` - -Returns all surfaces including disabled ones. Each entry is a full -[Surface Registry Object](#surface-registry-object). - -```json -{ - "surfaces": [ - { "id": "chat", "title": "Chat", "manifest": {...}, "enabled": true, "source": "core", "installed_at": "...", "updated_at": "..." }, - { "id": "hello-dashboard", "title": "Hello Dashboard", "manifest": {...}, "enabled": false, "source": "extension", "installed_at": "...", "updated_at": "..." } - ] -} -``` - -**Auth:** Platform admin. - -### Get Surface - -``` -GET /admin/surfaces/:id -``` - -Returns full surface details including manifest. - -**Response:** A single [Surface Registry Object](#surface-registry-object). - -**Errors:** -- `404` — surface not found (or store error). - -**Auth:** Platform admin. - -### Install Surface - -``` -POST /admin/surfaces/install -Content-Type: multipart/form-data -``` - -Field: `file` — a `.surface` or `.zip` archive (zip containing -`manifest.json` + static assets). The manifest declares `id`, `title`, -`route`, required data loaders, scripts, and CSS. - -**Size limit:** 50 MB. - -**Manifest validation:** `id` and `title` are required. - -**Errors:** -- `400` — no file, wrong extension, too large, invalid zip, missing or - invalid manifest, missing `id`/`title`. -- `409` — surface ID conflicts with an existing core surface. - -**Auth:** Platform admin. - -### Enable / Disable - -``` -PUT /admin/surfaces/:id/enable -PUT /admin/surfaces/:id/disable -``` - -Disabled surfaces redirect to `/` and hide from navigation. - -**Undisableable surfaces:** `chat` and `admin` cannot be disabled -(returns `400`). - -**Errors:** -- `400` — attempting to disable `chat` or `admin`. -- `404` — surface not found. - -**Auth:** Platform admin. - -### Delete Surface - -``` -DELETE /admin/surfaces/:id -``` - -Removes the surface, its DB row, and its extracted static assets. -Core surfaces cannot be deleted. - -**Errors:** -- `400` — attempting to delete a core surface. -- `404` — surface not found. - -**Auth:** Platform admin. - -**Defense-in-depth:** The store layer also enforces -`WHERE source = 'extension'` on delete, so even if the handler check -were bypassed, core surfaces are protected. - -## Surface Registry Object - -```json -{ - "id": "hello-dashboard", - "title": "Hello Dashboard", - "manifest": { "route": "/s/hello-dashboard", "scripts": [...], "css": [...] }, - "enabled": true, - "source": "core|extension", - "installed_at": "...", - "updated_at": "..." -} -``` - -## Internal Store Methods - -The `SurfaceRegistryStore` interface exposes methods beyond what the -HTTP API uses directly: - -- `Seed(ctx, id, title, source, manifest)` — upserts a surface at - startup. Does **not** overwrite the `enabled` flag (admin toggle - survives restarts). Called by the page engine's `SeedSurfaces()`. -- `ListEnabled(ctx) → []string` — returns IDs of all enabled - surfaces. Used by the page engine for nav rendering - (`EnabledSurfaceIDs()`), not exposed via HTTP. - -## Page Routes - -Extension surfaces are served at `/s/:slug` via `RenderExtensionSurface()`. -The page engine does a runtime DB lookup — no server restart needed after install. - -Static assets are served from `/surfaces/:id/` (nginx location block in -both unified and split deployment). In unified mode, the Go server -handles this route with path-traversal protection. In split deployment, -nginx serves directly from `/data/surfaces/`. - -## Surface Archive Format - -A `.surface` file is a **zip** archive containing: - -``` -manifest.json — required (must have "id" and "title") -js/ — optional (extracted to /data/surfaces/{id}/js/) -css/ — optional (extracted to /data/surfaces/{id}/css/) -assets/ — optional (icons, images, etc.) -script.star — optional (Starlark entry point, v0.38.0) -star/ — optional (Starlark submodules, v0.38.0) -``` - -The archive may have a single top-level directory prefix (e.g., -`my-surface/manifest.json`) — the installer strips it when locating -`manifest.json` and when extracting `js/`, `css/`, `assets/`, `star/` -directories and bare `.star` files. Files outside these directories -(e.g., `script.js` at root) are not extracted. - -See [EXTENSION-SURFACES.md](../EXTENSION-SURFACES.md) for the full -authoring guide including manifest schema, platform API, and CSS -custom properties. diff --git a/docs/ICD/tasks.md b/docs/ICD/tasks.md deleted file mode 100644 index b0371f1..0000000 --- a/docs/ICD/tasks.md +++ /dev/null @@ -1,414 +0,0 @@ -# Tasks - -Autonomous agent scheduling. A task is a cron-driven, one-shot, or -webhook-triggered execution in a headless service channel — no human -participant in the loop. - -## Configuration - -Global config keys (set via `PUT /admin/settings/:key`): - -| Key | Default | Description | -|-----|---------|-------------| -| `tasks.enabled` | `true` | Master kill switch for task scheduler | -| `tasks.allow_personal` | `true` | Allow non-admin users to create tasks | -| `tasks.max_concurrent` | `5` | Max tasks running simultaneously | -| `tasks.personal_require_byok` | `false` | Personal tasks must use BYOK provider | - -Default budget ceilings (overridable per task): - -| Key | Default | -|-----|---------| -| `tasks.default_max_tokens` | `4096` | -| `tasks.default_max_tool_calls` | `10` | -| `tasks.default_max_wall_clock` | `300` (seconds) | - -## Personal Tasks - -### List My Tasks - -``` -GET /tasks -``` - -Returns tasks owned by the current user. - -**Auth:** Authenticated. - -### Create Task - -``` -POST /tasks -``` - -```json -{ - "name": "Morning News Digest", - "description": "Summarize top tech news", - "task_type": "prompt", - "persona_id": "uuid|null", - "model_id": "claude-sonnet-4-20250514", - "system_prompt": "You are a news summarizer.", - "user_prompt": "Summarize the top 5 tech news stories from today.", - "schedule": "@daily", - "timezone": "America/New_York", - "max_tokens": 4096, - "max_tool_calls": 10, - "max_wall_clock": 300, - "output_mode": "channel", - "webhook_url": "https://...", - "provider_config_id": "uuid|null", - "notify_on_complete": false, - "notify_on_failure": true, - "tool_grants": ["web_search", "url_fetch"] -} -``` - -`task_type`: `prompt` (direct LLM execution), `workflow` (deferred — -not yet implemented; creation is rejected at the API), `action` -(no LLM — relay/webhook only; requires `task.action` permission), -or `system` (v0.28.6 — built-in Go function, admin-only). - -System tasks require `system_function` — a registered function name -from the platform's Go function registry. No LLM, no provider -resolution, no channel needed. The function receives the store set -and returns a result string. Designed for core platform ops -(`session_cleanup`, `staleness_check`, `retention_sweep`, `health_prune`) -that must not break from bad user code. Permanent track — not replaced -by Starlark in v0.29.0. - -`schedule`: cron expression, `once`, or `webhook`. Supported cron -patterns: `@hourly`, `@daily`, `@weekly`, `*/N * * * *` (every N -minutes), full 5-field cron via `robfig/cron/v3`. When `schedule` is -`webhook`, the task fires only on inbound POST to its trigger URL. - -`output_mode`: `channel` (default — output stays in service channel), -`note` (creates a note from the output), `webhook` (POST result to URL). - -**Auth:** `tasks.create` permission required. Additionally, `task.action` -permission is required for `task_type: "action"`. - -### Get Task - -``` -GET /tasks/:id -``` - -**Auth:** Owner or admin. - -### Update Task - -``` -PUT /tasks/:id -``` - -Partial update. Accepted fields: `name`, `description`, `persona_id`, -`model_id`, `system_prompt`, `user_prompt`, `workflow_id`, `tool_grants`, -`schedule`, `timezone`, `is_active`, `max_tokens`, `max_tool_calls`, -`max_wall_clock`, `output_mode`, `output_channel_id`, `webhook_url`, -`provider_config_id`, `notify_on_complete`, `notify_on_failure`. - -**Auth:** `tasks.create` permission, must be owner. - -### Delete Task - -``` -DELETE /tasks/:id -``` - -**Auth:** `tasks.create` permission, must be owner. - -### Run Now (Manual Trigger) - -``` -POST /tasks/:id/run -``` - -Immediately executes the task regardless of schedule. Creates a new -task run. Returns 409 if a run is already active or queued. - -**Auth:** `tasks.create` permission, must be owner. - -### Kill Active Run - -``` -POST /tasks/:id/kill -``` - -Cancels the active run. Sets status to `cancelled`. - -**Auth:** `tasks.create` permission, must be owner. - -### List Runs - -``` -GET /tasks/:id/runs -``` - -Returns run history for the task, most recent first. - -**Auth:** Owner or admin. - -## Webhook Trigger (Inbound) - -External systems fire a task by POSTing to its trigger URL. - -``` -POST /api/v1/hooks/t/:token -``` - -The `:token` is a per-task secret generated at creation time (returned -in the `trigger_token` field). The request body is stored as -`trigger_payload` on the run record and forwarded to the executor: - -- **Prompt tasks:** payload is prepended to the user prompt as context. -- **Action tasks:** payload is relayed to the outbound webhook as-is. -- **Workflow tasks:** (deferred) payload becomes initial stage data. - -Returns `202 Accepted`: -```json -{ - "triggered": true, - "run_id": "uuid", - "task_id": "uuid" -} -``` - -Returns `409 Conflict` if a run is already active or queued. -Returns `410 Gone` if the task is inactive. - -**Auth:** None (token-based). The trigger token *is* the credential. - -**Rate limiting:** Governed by global request rate limits. No per-task -rate limiting in v0.28 — defer to a future version if needed. - -### Task-to-Task Chaining (v0.28.6) - -Tasks can trigger other tasks by setting the `webhook_url` of Task A -to the trigger URL of Task B: - -1. Create **Task B** with `schedule: "webhook"`. Note the trigger URL - from the response (`/api/v1/hooks/t/{trigger_token}`). -2. Create **Task A** with `output_mode: "webhook"` and set `webhook_url` - to Task B's trigger URL. -3. When Task A completes, its output is POSTed to Task B's trigger URL. - Task B starts with Task A's output as its `trigger_payload`. - -This enables multi-step pipelines without external orchestration: -- Task A: "Summarize today's news" (prompt, cron: 6am) -- Task B: "Format and post to Slack" (action, webhook-triggered) - -The `webhook_secret` on Task A signs outbound payloads with HMAC-SHA256. -Task B's trigger endpoint does not currently verify signatures (the -trigger token itself is the auth mechanism). - -## Team Tasks - -Team-scoped tasks visible to all team members, manageable by team admins. - -### List Team Tasks (Member) - -``` -GET /teams/:teamId/tasks -``` - -Returns tasks scoped to the team. Read-only for members. - -**Auth:** Team member. - -### List Team Task Runs (Member) - -``` -GET /teams/:teamId/tasks/:id/runs -``` - -**Auth:** Team member. - -### Create Team Task (Admin) - -``` -POST /teams/:teamId/tasks -``` - -Same shape as personal task creation. Team scope is injected automatically. - -**Auth:** `tasks.create` permission, team admin. - -### Update/Delete/Run/Kill Team Task - -``` -PUT /teams/:teamId/tasks/:id -DELETE /teams/:teamId/tasks/:id -POST /teams/:teamId/tasks/:id/run -POST /teams/:teamId/tasks/:id/kill -``` - -**Auth:** `tasks.create` permission, team admin. - -## Admin Tasks - -Platform-wide task management. - -### List All Tasks - -``` -GET /admin/tasks -``` - -Returns all tasks across all users and teams. - -**Auth:** Platform admin. - -### Admin Run/Kill/Delete - -``` -POST /admin/tasks/:id/run -POST /admin/tasks/:id/kill -DELETE /admin/tasks/:id -``` - -**Auth:** Platform admin. - -### System Functions (v0.28.6) - -``` -GET /admin/system-functions → { "data": [SystemFuncInfo, ...] } -``` - -Returns all registered built-in system function names and descriptions. -Used by the admin UI to populate the function dropdown when creating -system tasks. - -**SystemFuncInfo:** - -```json -{ - "name": "session_cleanup", - "description": "Remove expired anonymous visitor sessions" -} -``` - -Built-in functions (v0.28.6): `session_cleanup`, `staleness_check`, -`retention_sweep`, `health_prune`. - -**Auth:** Platform admin. - -## Task Object - -```json -{ - "id": "uuid", - "owner_id": "uuid", - "team_id": "uuid|null", - "name": "Morning News Digest", - "description": "...", - "scope": "personal|team|global", - "task_type": "prompt|workflow|action|system", - "system_function": "session_cleanup", - "persona_id": "uuid|null", - "model_id": "claude-sonnet-4-20250514", - "system_prompt": "...", - "user_prompt": "...", - "workflow_id": "uuid|null", - "tool_grants": ["web_search", "url_fetch"], - "schedule": "@daily|once|webhook", - "timezone": "America/New_York", - "is_active": true, - "trigger_token": "hex-string|null", - "max_tokens": 4096, - "max_tool_calls": 10, - "max_wall_clock": 300, - "output_mode": "channel|note|webhook", - "output_channel_id": "uuid|null", - "webhook_url": "https://...", - "webhook_secret": "...", - "provider_config_id": "uuid|null", - "notify_on_complete": false, - "notify_on_failure": true, - "last_run_at": "...|null", - "next_run_at": "...|null", - "run_count": 42, - "created_at": "...", - "updated_at": "..." -} -``` - -## Task Run Object - -```json -{ - "id": "uuid", - "task_id": "uuid", - "channel_id": "uuid|null", - "status": "queued|running|completed|failed|budget_exceeded|cancelled", - "trigger_payload": "...|null", - "started_at": "...", - "completed_at": "...|null", - "tokens_used": 1234, - "tool_calls": 3, - "wall_clock": 45, - "error": "...|null" -} -``` - -## Execution - -The `TaskScheduler` is a background goroutine polling every 30 seconds. - -1. Finds tasks where `next_run_at <= now` and `is_active = true` -2. Skips if an active run exists (`GetActiveRun` returns non-nil) -3. Adopts a queued run if one exists (from webhook trigger), otherwise - creates a new run record -4. Marks `last_run_at` (records when execution started, not when it finished) -5. Creates or reuses a `service` channel (`output_channel_id`) -6. Persists the `user_prompt` as a message in the service channel - (with trigger payload prepended if present) -7. For **prompt** tasks: runs `CoreToolLoop` (headless completion) -8. For **action** tasks: skips LLM, relays trigger payload to webhook -9. Enforces budgets: `max_tokens`, `max_tool_calls`, `max_wall_clock` -10. On budget breach: status = `budget_exceeded`, owner notified -11. On completion: calculates `next_run_at` - -**Workflow tasks:** Not yet implemented. The API rejects -`task_type: "workflow"` at creation time. Reserved for a future version. - -**Provider resolution:** BYOK → team provider → global provider → routing -policy. If `tasks.personal_require_byok` is true, personal tasks that -don't have a BYOK provider fail at step 7. - -**Action tasks** skip steps 6–9. They create the service channel, fire -the outbound webhook with the trigger payload, and complete. - -## Webhooks (Outbound) - -Tasks with `webhook_url` fire a POST on completion: - -```json -{ - "task_id": "uuid", - "run_id": "uuid", - "task_name": "Morning News Digest", - "channel_id": "uuid", - "status": "completed|failed|budget_exceeded", - "completed_at": "2025-03-11T12:00:00Z", - "output": "...", - "tokens_used": 1234, - "error": "...|null" -} -``` - -Signed with HMAC-SHA256 using `webhook_secret` in the -`X-Switchboard-Signature` header. Retry: 3 attempts, exponential -backoff (1s, 5s, 25s). - -## Task Chaining - -Task A's outbound `webhook_url` can be Task B's trigger URL: - -``` -Task A (webhook_url) → POST /api/v1/hooks/t/ → Task B runs -``` - -This enables multi-step pipelines: CI failure → summarize with LLM → -post to Slack. Each link is a separate task with its own type, budget, -and notification preferences. diff --git a/docs/ICD/teams.md b/docs/ICD/teams.md deleted file mode 100644 index bf12ed4..0000000 --- a/docs/ICD/teams.md +++ /dev/null @@ -1,258 +0,0 @@ -# Teams & Access Control - -### My Teams - -``` -GET /teams/mine → { "data": [...] } -``` - -**Auth:** Authenticated user. - -Returns teams the current user is a member of (active teams only). - -Each team object: - -```json -{ - "id": "uuid", - "name": "Engineering", - "description": "...", - "is_active": true, - "settings": "{}", - "my_role": "admin|member", - "member_count": 5 -} -``` - -### Team Administration - -**Team CRUD (platform admin):** - -``` -GET /admin/teams → { "data": [...], "total": N, "page": N, "per_page": N } -POST /admin/teams ← { "name", "description" } → { "id", "name" } -GET /admin/teams/:id → team object (unwrapped) -PUT /admin/teams/:id ← { "name"?, "description"?, "is_active"?, "settings"? } -DELETE /admin/teams/:id -``` - -**Auth:** `RequireAdmin` (platform admin). - -`POST` returns `201`. `PUT`/`DELETE` return `{"ok": true}`. `PUT` with -no fields returns `400`. Duplicate name returns `409`. - -**Members (admin routes):** - -``` -GET /admin/teams/:id/members → { "data": [...] } -POST /admin/teams/:id/members ← { "user_id", "role" } → { "id" } -PUT /admin/teams/:id/members/:memberId ← { "role" } -DELETE /admin/teams/:id/members/:memberId -``` - -**Auth:** `RequireAdmin`. - -`role` must be `admin` or `member`. Duplicate membership returns `409`. - -**Team-scoped routes** (team admin self-service): - -``` -GET /teams/:teamId/members -POST /teams/:teamId/members ← { "user_id", "role" } -PUT /teams/:teamId/members/:memberId -DELETE /teams/:teamId/members/:memberId -``` - -**Auth:** `RequireTeamAdmin`. - -Same handlers as admin routes; the `getTeamID()` helper reads either -`:id` or `:teamId`. - -**Team Providers:** - -``` -GET /teams/:teamId/providers → { "data": [...], "allow_team_providers": bool } -POST /teams/:teamId/providers ← { "name", "provider", "endpoint", "api_key", ... } -PUT /teams/:teamId/providers/:id -DELETE /teams/:teamId/providers/:id -GET /teams/:teamId/providers/:id/models → { "models": [...], "provider": "name" } -``` - -**Auth:** `RequireTeamAdmin`. - -`GET` returns provider configs scoped to the team. The `models` endpoint -performs a live query to the upstream provider. - -**Team Models:** - -``` -GET /teams/:teamId/models → { "models": [...] } -``` - -**Auth:** `RequireTeamAdmin`. - -Returns all models available to this team: global admin models with -`visibility IN ('enabled', 'team')` plus live-queried team provider -models with `source` field (`"global"` or `"team"`). - -**Team Groups:** - -``` -GET /teams/:teamId/groups → { "data": [...] } -``` - -**Auth:** `RequireTeamAdmin`. - -Returns groups scoped to this team. - -**Team Personas:** See personas.md §Team Personas. - -**Team Roles:** - -``` -GET /teams/:teamId/roles → { "data": {...} } -PUT /teams/:teamId/roles/:role ← RoleConfig object -DELETE /teams/:teamId/roles/:role -``` - -**Auth:** `RequireTeamAdmin`. - -`GET` returns the team's `model_roles` settings map (composite object, -not an array). Initially empty `{}`. Each key is a role name, value is -a `RoleConfig`. `PUT` validates the role name. `DELETE` removes the -override, falling back to the global role definition. - -**Team Audit:** - -``` -GET /teams/:teamId/audit → { "data": [...], "total": N, "page": N, "per_page": N } -GET /teams/:teamId/audit/actions → { "actions": [...] } -``` - -**Auth:** `RequireTeamAdmin`. - -Audit log is scoped to actions performed by team members. Supports -query filters: `?action=`, `?actor_id=`, `?resource_type=`. - -**Team Usage:** - -``` -GET /teams/:teamId/usage → { "totals": {...}, "results": [...] } -``` - -**Auth:** `RequireTeamAdmin`. - -Returns usage against team-owned providers. Supports query params for -date range filtering. - -### Groups & Resource Grants - -Groups are ACL containers that decouple access from team membership. -A resource grant controls who can see a Persona or KB beyond its base -scope. - -**My Groups:** - -``` -GET /groups/mine → { "data": [...] } -``` - -**Auth:** Authenticated user. - -**Admin Group CRUD:** - -``` -GET /admin/groups → { "data": [...] } -POST /admin/groups ← { "name", "scope", "team_id"? } -GET /admin/groups/:id → group object (unwrapped) -PUT /admin/groups/:id ← { "name"?, "description"?, "permissions"?, ... } -DELETE /admin/groups/:id -``` - -**Auth:** `RequireAdmin`. - -`POST` returns `201`. Duplicate name returns `409`. Deleting a -system-sourced group (e.g. Everyone) returns `400`. - -**Group Members:** - -``` -GET /admin/groups/:id/members → { "data": [...] } -POST /admin/groups/:id/members ← { "user_id" } -DELETE /admin/groups/:id/members/:userId -``` - -**Auth:** `RequireAdmin`. - -**Resource Grants:** - -``` -GET /admin/grants/:type/:id → grant object (unwrapped) -PUT /admin/grants/:type/:id ← { "grant_scope", "granted_groups"?: [...] } -DELETE /admin/grants/:type/:id -``` - -**Auth:** `RequireAdmin`. - -`:type` is `persona` or `kb`. `:id` is the resource ID. - -Grant scopes: - -| `grant_scope` | Visibility | -|---------------|-----------| -| `team_only` | Only the owning team | -| `global` | All authenticated users | -| `groups` | Members of specified groups | - -Setting `grant_scope: "groups"` without `granted_groups` returns `400`. - -### Permissions - -Groups carry fine-grained permissions. Permission resolution: -union of all group permissions + Everyone group permissions. - -**List all permissions:** - -``` -GET /admin/permissions → { "permissions": ["model.use", "kb.read", ...] } -``` - -**Auth:** `RequireAdmin`. - -**Get user's effective permissions:** - -``` -GET /admin/users/:id/permissions → { "permissions": [...], "user_id": "...", "groups": [...] } -``` - -**Auth:** `RequireAdmin`. - -Returns the resolved permission set and contributing group IDs -(always includes the Everyone group). - -**Group permission fields** (set via `PUT /admin/groups/:id`): - -```json -{ - "permissions": ["model.use", "kb.read", "channel.create"], - "token_budget_daily": 100000, - "token_budget_monthly": 3000000, - "allowed_models": ["claude-sonnet-4-20250514", "gpt-4o"] -} -``` - -| Field | Type | Description | -|-------|------|-------------| -| `permissions` | string[] | Permission constants granted to members | -| `token_budget_daily` | int/null | Daily token ceiling (NULL = unlimited) | -| `token_budget_monthly` | int/null | Monthly token ceiling (NULL = unlimited) | -| `allowed_models` | string[]/null | Model allowlist (NULL = unrestricted) | - -**Everyone group:** System-seeded group (ID `00000000-0000-0000-0000-000000000001`) -whose permissions apply to all authenticated users implicitly (no membership -row needed). Editable by admins, cannot be deleted (`source=system`). - -**RequirePermission middleware:** Routes gated by permission check user's -resolved permission set. Returns 403 if the required permission is not present. - -See [enums.md](enums.md) for the full permission constant list. diff --git a/docs/ICD/utility.md b/docs/ICD/utility.md deleted file mode 100644 index 0e68d8e..0000000 --- a/docs/ICD/utility.md +++ /dev/null @@ -1,146 +0,0 @@ -# Export & Utility - -### Health Check - -``` -GET /health → { "status": "ok" } -``` - -Public, no auth. Also available at `/api/v1/health`. - -### Export - -Convert markdown to PDF or DOCX via pandoc: - -``` -POST /export -``` - -```json -{ - "content": "# My Document\n\nBody text...", - "format": "pdf|docx", - "filename": "my-document" -} -``` - -Returns the binary file with appropriate Content-Type. Requires -`pandoc` in the container (available in the unified and backend -Docker images). - ---- - -## 17b. Files & Storage - -All binary content in Chat Switchboard flows through a single -**ObjectStore** abstraction backed by PVC (filesystem) or S3. All -file metadata lives in one `files` table. The old `attachments` -table is dropped. - -### Storage Backend - -| Operation | Description | -|-----------|-------------| -| `Put(key, reader, size, contentType)` | Store a blob | -| `Get(key)` → reader, size, contentType | Retrieve a blob | -| `Delete(key)` | Remove a blob | -| `DeletePrefix(prefix)` | Bulk remove (channel deletion) | -| `Exists(key)` | Check without reading | -| `Healthy()` | Backend health check | -| `Stats()` | Aggregate metrics | - -Backends: `pvc` (`STORAGE_PATH=/data/storage`) or `s3` -(`S3_ENDPOINT`, `S3_BUCKET`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`). - -### Key Namespacing - -| Prefix | Pattern | What | -|--------|---------|------| -| `attachments/` | `attachments/{channel_id}/{file_id}_{filename}` | User uploads + AI outputs (legacy prefix retained for existing blobs) | -| `kb/` | `kb/{kb_id}/{doc_id}_{filename}` | Knowledge base documents | -| `exports/` | `exports/{user_id}/{export_id}_{filename}` | Chat/data exports | -| `workspace/` | `workspace/{workspace_id}/{path}` | Workspace files | - -### File Object - -```sql -CREATE TABLE files ( - id UUID PRIMARY KEY, - channel_id UUID REFERENCES channels(id) ON DELETE CASCADE, - message_id UUID REFERENCES messages(id) ON DELETE SET NULL, - user_id UUID REFERENCES users(id) ON DELETE SET NULL, - project_id UUID REFERENCES projects(id) ON DELETE SET NULL, - origin TEXT NOT NULL DEFAULT 'user_upload' - CHECK (origin IN ('user_upload','tool_output','system')), - filename TEXT NOT NULL, - content_type TEXT NOT NULL DEFAULT 'application/octet-stream', - size_bytes BIGINT NOT NULL DEFAULT 0, - storage_key TEXT NOT NULL, - display_hint TEXT NOT NULL DEFAULT 'download' - CHECK (display_hint IN ('inline','download','thumbnail')), - extracted_text TEXT, - metadata JSONB DEFAULT '{}', - created_at TIMESTAMPTZ DEFAULT NOW(), - updated_at TIMESTAMPTZ DEFAULT NOW() -); -``` - -**Origin:** - -| Value | Who creates | Examples | -|-------|-------------|---------| -| `user_upload` | User via upload UI | PDF, image, doc attached to message | -| `tool_output` | Tool execution during completion | DALL-E image, video gen, code artifact | -| `system` | Platform | Export, avatar stored as file | - -**Display hint:** - -| Value | Frontend behavior | -|-------|-------------------| -| `inline` | Render in message stream (images, video, audio, HTML) | -| `download` | Filename + size + download button | -| `thumbnail` | Thumbnail preview, click to expand | - -### Tool Output Flow - -1. Tool executes (image gen, video render, etc.) -2. Tool writes blob via `ObjectStore.Put` -3. Tool creates file record: `POST /files` with `origin: "tool_output"`, `message_id` = current assistant message -4. Tool result includes `file_id` reference -5. WebSocket emits `file.created` event: - -```json -{ - "event": "file.created", - "channel_id": "uuid", - "message_id": "uuid", - "file": { ...file object... } -} -``` - -6. Frontend renders inline based on `display_hint` + `content_type` - -The event fires during streaming so the user sees the artifact -before the completion finishes. - -### Content Type → Display Hint Defaults - -| Content Type | Default Hint | Rendering | -|-------------|-------------|-----------| -| `image/*` | `inline` | `` in message, lightbox on click | -| `video/*` | `inline` | `