# Changelog All notable changes to Switchboard Core are documented here. ## v0.6.3 — Dead Code Sweep + Registry Fix Pre-fork hardening: fix broken registry install, add registry settings UI, clean up dead code, narrow default bundle. ### Fixed - **Registry install**: SDK was sending `{ url }` but the Go handler expects `{ download_url }`, causing every registry Install click to return 400. ### Added - **Package Registry settings**: New "Package Registry" section in Admin > Settings with a URL input field for configuring the external registry. - **`scripts/generate-registry.sh`**: Shell script that reads `.pkg` ZIPs, extracts manifests, and emits a `registry.json` for self-hosted discovery. - **`docs/PACKAGE-REGISTRY.md`**: Documents registry JSON format, admin configuration, script usage, and self-hosting setup. ### Removed - **Dead Go code**: Orphaned channel/message type comments from `interfaces.go`, unused `roleFilterType()` from `pages.go`, stale version comment from `main.go`. - **Dead vendor JS**: `marked.min.js` (40KB) and `purify.min.js` (22KB) — unused vendor copies with zero production imports. Removed cache entries from `sw.js`. - **`dev.html`**: 676-line standalone dev gallery, not referenced by anything. - **Pre-fork version comments**: Stripped `// v0.X.X:` annotations from 72 files. Config-doc version references (compatibility requirements) preserved. ### Changed - **Default bundle narrowed** from 10 packages to 5: `notes`, `chat`, `chat-core`, `mermaid-renderer`, `schedules`. All other packages remain available via `BUNDLED_PACKAGES=*` or explicit list. --- ## v0.6.2 — Docs Polish + Dynamic OpenAPI Docs surface polish and dynamic OpenAPI spec with extension route merging. ### Added - **Dynamic OpenAPI spec**: `GET /api/docs/openapi.json` builds a merged spec at request time — kernel endpoints + auto-generated stubs for every extension `api_routes` entry. Extensions may optionally declare `api_schema` in `manifest.json` for richer path items (params, body, response examples). - **`api_schema` manifest field**: Optional array in `manifest.json`. Malformed entries are logged and skipped — never blocks extension loading. - **Tests**: 7 new handler tests (zero extensions, stubs, rich schema, multi-extension, malformed schema, disabled exclusion, required fields). ### Fixed - **Dark mode contrast**: Replaced all hardcoded light-mode fallbacks in docs CSS with theme-aware CSS variables. Tables, code blocks, nav items, and headings now readable in dark mode. - **Docs loading state**: Replaced borrowed `settings-placeholder` animation with docs-specific pulse skeleton. Added error state with retry button. - **Docs routing**: Default route (`/docs`) now redirects to `/docs/GETTING-STARTED` instead of the blank `/:section` placeholder. - **Docs scrolling**: Content area now scrolls correctly — full document visible without text overflowing viewport. - **Docs icon**: Changed from 🧩 to 📖 in the user menu. - **Docs duplicate menu entry**: Docs was appearing twice (once from surfaces API, once from hardcoded items). Added `'docs'` to `CORE_IDS` filter. - **Swagger UI**: Updated to point at dynamic `/api/docs/openapi.json` endpoint. Static YAML preserved for backward compatibility. - **OpenAPI tests on Postgres**: `openapiTestStores()` was hardcoded to `sqlite.NewStores()` — caused `pq: syntax error at or near ","` in CI. Now uses `database.IsSQLite()` check like all other handler tests. ### API | Endpoint | Method | Description | |----------|--------|-------------| | `/api/docs/openapi.json` | GET | Dynamic merged OpenAPI 3.0 spec | --- ## v0.6.1 — Backup/Restore + Documentation Operational tooling for data safety and extension authoring. ### Added - **Backup/Restore**: Full-instance backup as `.swb` ZIP archive containing core tables (JSONL), ext_data tables, and package assets. Stream to client or save server-side. Restore wipes and rebuilds from archive. Dialect-neutral (works across SQLite and Postgres). - **Admin backup section**: New "Backup" tab under `/admin` with create, download, delete, and restore operations. Destructive restore requires confirmation dialog. - **Documentation surface**: Builtin surface at `/docs` with sidebar navigation and client-side markdown rendering. Ships with 5 documents: Getting Started, Extension Guide, API Reference, Deployment, Package Format. - **Docs API**: `GET /api/v1/docs` lists available documents, `GET /api/v1/docs/:name` returns raw markdown content. Authenticated (all users). - **Tests**: 6 backup handler tests + E2E script (`ci/e2e-backup-test.sh`). ### API | Endpoint | Method | Description | |----------|--------|-------------| | `/api/v1/admin/backup` | POST | Create backup (stream or `?store=true`) | | `/api/v1/admin/backups` | GET | List server-side backups | | `/api/v1/admin/backups/:name` | GET | Download backup | | `/api/v1/admin/backups/:name` | DELETE | Delete backup | | `/api/v1/admin/restore` | POST | Restore from `.swb` upload | | `/api/v1/docs` | GET | List documentation | | `/api/v1/docs/:name` | GET | Get document content | --- ## v0.6.0 — Cluster Registry + HA MVP convergence point. PG-backed cluster registry for horizontal scaling — zero new infrastructure (no etcd/Consul/Redis). ### Added - **Cluster registry**: `node_registry` UNLOGGED table with self-registration on startup (`INSERT ... ON CONFLICT DO UPDATE`), heartbeat tick (default 10s), stale sweep (default 30s threshold), and self-eviction (`os.Exit(1)` when swept by a peer — K8s restarts the process). - **Cluster admin API**: `GET /api/v1/admin/cluster` returns all registered nodes with runtime stats (goroutines, heap, GC, uptime, ws_clients) in the standard `{data: [...]}` envelope. - **Health endpoint cluster info**: `GET /health` and `GET /api/v1/health` now include `node_id` and `cluster: {size, peers, heartbeat_age_ms}` when running on Postgres with the registry active. - **Cluster dashboard**: `cluster-dashboard` admin surface package renders one card per node with live runtime stats. Auto-refreshes every 10s. Stats are JSONB — new keys render automatically without schema migration. - **Cluster config**: `CLUSTER_NODE_ID` (default hostname-PID), `CLUSTER_HEARTBEAT_INTERVAL` (default 10s), `CLUSTER_STALE_THRESHOLD` (default 30s), `CLUSTER_ENDPOINT` (Phase 2 mesh, auto-detect). - **3-replica E2E test**: `docker-compose-e2e.yml` now runs 3 replicas. `ci/e2e-cluster-test.sh` verifies registration, stale sweep on stop, and re-registration on restart. - **5 new tests**: 3 cluster registry unit tests (stats collection, start/stop lifecycle, node ID) + 2 handler tests (list nodes, empty response). - **Curated bundle expanded**: `chat` and `cluster-dashboard` added to the default bundle (now 10 packages). Production deployments can override with `BUNDLED_PACKAGES=notes,chat,chat-core,cluster-dashboard` for a lean set. ### Changed - SQLite deployments are unaffected — cluster store is nil, all cluster code is guarded behind `database.IsPostgres() && stores.Cluster != nil`. Cluster dashboard gracefully shows "0 nodes registered" on SQLite. ## v0.5.5 — Upgrade Testing ### Fixed - **Bundled permission auto-grant on Postgres**: The `InstallBundledPackages` auto-grant SQL used SQLite-style `granted = 1` / `granted = 0` which fails on Postgres BOOLEAN columns. Now dialect-aware: `true`/`false` on Postgres, `1`/`0` on SQLite. This caused bundled extensions (notes, chat-core, etc.) to install with permissions not granted on Postgres deployments. - **E2E test API field corrections**: Fixed `ci/e2e-chat-test.sh` login field (`"username"` → `"login"`), token extraction (`"token"` → `"access_token"`), and profile endpoint (`/api/v1/me` → `/api/v1/profile`). ### Added - **Upgrade test harness**: `docker-compose-upgrade.yml` + `ci/e2e-upgrade-test.sh` orchestrate a full upgrade cycle: build old image, seed data (notes, conversations, messages, settings), stop old, start new, verify data integrity post-upgrade. Covers auth, notes, conversations, packages, and settings. - **Rolling upgrade test**: `ci/e2e-upgrade-rolling.sh` tests rolling upgrade with 2 replicas sharing Postgres. Upgrades one replica at a time, verifies cross-replica reads during the mixed-version window. - **11 new Go tests** in `upgrade_test.go`: - Schema edge cases: add index (idempotent), add column (idempotent), multi-column add, row preservation across migration. - Settings migration: global/team/user overrides preserved across package update, new keys get defaults, removed keys not deleted. - Package compatibility: bundled skip-if-present on restart, dormant status for unmet requires, enabled/type preserved across version bump. - Direct `MigrateExtTables` tests: new table creation, column add with row preservation, index idempotency. ## v0.5.4 — Package Updates ### Added - **Package update API**: `POST /api/v1/admin/packages/:id/update` accepts a `.pkg` archive, validates semver version bump (rejects same/older), applies additive schema migration, merges settings, replaces assets, and re-syncs permissions/triggers/dependencies. - **Package export API**: `GET /api/v1/admin/packages/:id/export` streams the installed package as a downloadable `.pkg` ZIP archive. Completes the manual rollback story: export before update, re-install old `.pkg` if needed. - **Semver parsing**: `ParseSemver` / `Compare` in `handlers/semver.go` with prerelease support. Used by the update handler to enforce version bumps. - **Additive schema migration**: `MigrateExtTables` diffs declared `db_tables` against existing ext_data schema. Adds new columns (`ALTER TABLE ADD COLUMN`) and new tables. No destructive changes — columns in DB but absent from manifest are preserved. `ListExtColumns` introspects via `PRAGMA table_info` (SQLite) or `information_schema` (Postgres). - **Settings merge on update**: New manifest setting keys get their declared default value; existing admin-configured values are preserved. - **Admin UI Update button**: Non-core package cards now show an Update button with a confirm dialog before upload. - **14 new Go tests**: Version bump, 5 rejection cases (same/older/core/type/id mismatch), schema add column with data preservation, schema add table, settings merge, export with valid zip, default allowlist, wildcard allowlist. ### Changed - **Curated default bundle**: Default bundled packages reduced from 23 to 8 core packages (notes, chat-core, workflow-chat, dashboard, 4 demo workflows). `BUNDLED_PACKAGES=*` installs all; explicit comma-separated list for custom. Existing deployments are unaffected (install-once, skip-if-present). - **`BUNDLED_PACKAGES` env var**: Empty value now installs curated defaults instead of all packages. Use `"*"` for previous install-all behavior. ## v0.5.3 — Chat Polish + Integration Testing ### Added - **`db.query()` search_like**: New kernel-level `search_like` kwarg for text search. Accepts `{col: pattern}` dict, generates OR-joined `LIKE` (SQLite) / `ILIKE` (Postgres) clauses. Composes with existing `filters` via AND. Reusable by any package. 5 new sandbox tests. - **Conversation search**: `GET /search?q=term` endpoint in `chat-core`. Searches conversation titles and message content, scoped to user's conversations. Returns `{conversations: [...], messages: [...]}`. - **Search UI**: Debounced search input in chat sidebar. Results view replaces conversation list showing matched conversations and message snippets with section headers. Click result to navigate, clear button to dismiss. - **`workflow-chat` library package**: `on_advance` hook that creates scoped group conversations when workflow stages require team collaboration. Adds all team members as participants, sends system message linking to the instance, enriches `stage_data` with `conversation_id`. Idempotent — skips if conversation already exists. - **Multi-user E2E test infrastructure**: `docker-compose-e2e.yml` with 2 replicas, Postgres, and nginx load balancer. `ci/e2e-chat-test.sh` shell script covering auth, conversation CRUD, message send/read, cross-replica consistency, search, and pagination. `ci/e2e-ws-listener.js` Node.js WebSocket client for realtime event testing. - **Workflow hook tests**: 6 new Go tests for `parseOnAdvanceResult` covering None, nil, error, enriched stage_data, non-dict, and nested structure cases. ### Changed - **Message pagination polish**: Scroll position now preserved when loading older messages — records `scrollHeight` before prepend, restores via `requestAnimationFrame`. Loading spinner shown at top of thread during fetch. "Load older messages" button hidden while loading. - **chat-core** bumped to v0.2.0 (new search endpoint). - **chat** bumped to v0.2.0 (search UI, pagination polish). ## v0.5.2 — Chat Surface ### Added - **`chat` surface package**: Full messaging UI built on `chat-core` library. Route `/s/chat`, icon `💬`, depends on `chat-core`. - **Conversation list sidebar**: Conversations sorted by recent activity, last message preview, relative timestamps, unread count badges. "New" button for conversation creation. - **Message thread**: Paginated message history with cursor-based load-more on scroll-to-top. Participant avatars, display names, timestamps. Scroll-to- bottom on new messages. - **Compose bar**: Auto-resizing textarea, Enter-to-send, Shift+Enter for newline. Typing indicator broadcast on keypress (3s debounce). - **Participant sidebar**: Collapsible right panel with participant list, online status via presence hub, role badges. Add participant via `sw.ui.Dialog` + `sw.ui.UserPicker`. Remove participant with confirmation. - **Create conversation dialog**: Group or Direct Message type selector. Multi- user picker with chip display for group, single picker for DM. Auto-title from participant names. - **Typing indicators**: Realtime broadcast via `POST /typing/:id` Starlark endpoint. Frontend subscribes to `conversation:{id}` channel, shows "X is typing..." with 4s auto-expire. Handles multiple typers. - **Read receipts**: Auto mark-read on conversation focus and new incoming messages. Unread counts in conversation list, cleared on selection. - **Message editing**: Inline edit mode on own messages with textarea. Save on Enter, cancel on Escape. "(edited)" label on edited messages. Realtime propagation to other participants. - **Message deletion**: Confirm dialog before delete. "This message was deleted" placeholder. Admin can delete any message. Realtime propagation. ### Fixed - **Stale token login deadlock**: REST client's 401→refresh→retry loop caused a deadlock when the refresh endpoint itself returned 401 (stale DB). Auth endpoints (`/auth/login`, `/auth/refresh`, `/auth/register`) are now excluded from the retry loop. Defensive 8s timeout on `auth.boot()` ensures login page always renders even on unexpected hangs. - **Bundled package permissions**: Bundled packages were installed with `pending_review` status and `granted=0` permissions, requiring manual admin approval. Bundled installer now auto-grants all declared permissions and sets status to `active` on install. - **Creator display name**: `chat-core.create()` now accepts `creator_display_name` so the conversation creator shows a readable name in the participant sidebar instead of a raw UUID. - **Chat dark mode theming**: Replaced non-existent CSS variable names with actual theme tokens from `variables.css`. Fixes invisible text and wrong backgrounds in dark mode. - **Dialog dropdown clipping**: UserPicker autocomplete dropdown in dialogs (New Conversation, Add Participant) no longer clipped by `overflow: auto` on `.sw-dialog__body`. - **User menu at scaled UI**: Menu primitive now divides anchor coordinates and viewport dimensions by `sw.shell.getScale()` to compensate for CSS `transform: scale()` on `#surfaceInner`. Fixes menu clipping at 110%+. ### Changed - **Named Docker volume**: `docker-compose.yml` switched from bind mount (`./data:/data`) to named volume (`sb_data:/data`). Fixes volume mount failures when Docker daemon runs on a remote host (DinD, k8s pod). Deterministic wipe via `docker compose down -v`. ## v0.5.1 — Chat Core Library ### Added - **`chat-core` library package**: Conversations, messages, participants, and read cursors as ext_data tables. Permissions: `db.write`, `realtime.publish`. Consumable by other packages via `lib.require("chat-core")`. - **4 ext_data tables**: `conversations` (title, type, created_by, updated_at), `participants` (conversation_id, participant_id, type, display_name, role, joined_at), `messages` (conversation_id, participant_id, content, content_type, edited_at), `read_cursors` (conversation_id, participant_id, last_read_message_id). - **6 exported Starlark functions**: `create()`, `send()`, `history()`, `add_participant()`, `remove_participant()`, `mark_read()`. - **15 REST endpoints**: Full CRUD for conversations, messages, participants. Cursor-based paginated message history. Per-conversation unread counts. - **Realtime events**: `message`, `message.edited`, `message.deleted`, `participant.added`, `participant.removed` published to `conversation:{id}` channels. - **`db.query()` range parameters**: New `before` and `after` kwargs accept dicts of `{col: val}` generating `col < ?` / `col > ?` WHERE clauses. Enables cursor-based pagination and efficient unread counting for all extension packages. 7 new Go tests. ### Security - **Invisible Unicode scanning gate** (GlassWorm / Trojan Source defense): `ScanSource()` detects variation selectors, bidi overrides, zero-width chars, tag characters, and other invisible Unicode across 10 ranges. `Verdict()` blocks GlassWorm payloads (>=10 variation selector / tag chars), Trojan Source (any bidi override), and excessive zero-width chars (>=6). Two enforcement points: extension install (422 rejection before files written) and Starlark execution (belt-and-suspenders). 23 new tests including GlassWorm payload simulation. ### Changed - **Admin packages type filter**: Added `Library` option alongside Surface/Extension/Full/Workflow. Filter dropdown replaced button group with themed `Dropdown` primitive (keyboard nav, CSS-var theming, consistent with rest of admin UI). - SDK version bumped from 0.5.0 to 0.5.1. ## v0.5.0 — Realtime Primitive + Dialog Audit + Permissions UI ### Added - **Realtime Starlark module**: `realtime.publish(channel, event, data)` lets extensions publish events to WebSocket channels. Gated by the new `realtime.publish` permission. Payloads limited to 7KB (pg_notify safe). Reserved event prefixes (`system.`, `workflow.`, etc.) are blocked. - **WS room protocol**: Clients send `room.subscribe` / `room.unsubscribe` messages to join/leave rooms. Intercepted in the WebSocket readPump before the bus publish gate (like ping). Per-connection cap of 100 rooms. - **SDK realtime module**: `sw.realtime.subscribe(channel, [event], callback)` manages room join/leave over WebSocket and filters incoming `realtime.*` events by room. Returns unsubscribe handle. Auto re-joins all rooms on WebSocket reconnect. `sw.realtime.channels()` debug helper. - **Admin permissions UI**: Packages page gains a "Permissions" button for packages with declared permissions. Inline drawer shows each permission with Grant/Revoke toggle, granted-by user ID, and a "Grant All" bulk action. - **Status badges**: `pending_review` (amber) and `suspended` (red) badges on package rows. Enable toggle disabled with hint when `pending_review`. - **SDK API client**: `permissions()`, `grantPerm()`, `revokePerm()`, `grantAllPerms()` methods added to `sw.api.admin.packages`. - **`starlarkToGoVal()` helper**: Converts Starlark values to Go `any` for JSON marshaling (reverse of existing `goValToStarlark`). - **8 new Go tests**: Route table cases for `realtime.*` and `room.*`, realtime module publish (happy path, empty data, missing args, reserved prefix, payload too large), `starlarkToGoVal` type conversion. ### Changed - **Dialog audit**: 5 bare `confirm()` calls migrated to `await sw.confirm()` with `{ destructive: true }` styling in tasks, schedules, notes (×2), and editor packages. Package archives rebuilt. - SDK version bumped from 0.2.3 to 0.5.0. - Event route table: `realtime.` → DirToClient, `room.subscribe` / `room.unsubscribe` → DirFromClient. - Runner gains `bus` field and `SetBus()` method; wired in main.go. ## v0.4.7 — Note Graph ### Added - **Graph API**: `GET /graph` endpoint returns all non-archived notes as nodes and resolved wikilinks as edges in a single payload. Nodes include id, title, folder_id, and tags. Edges include source, target, and link text. - **Graph view**: Canvas-based force-directed graph visualization replaces the editor pane when "Graph" button is clicked in the topbar. Force simulation uses repulsion (all pairs), attraction (edges), and center gravity with velocity damping. Supports zoom (scroll wheel) and pan (mouse drag). - **Click focus**: Single-click a node to dim unconnected nodes/edges to 15% opacity; clicked node and direct neighbors stay at full brightness. Edges between focused nodes highlighted. Click empty space to reset. - **Shift+click chain**: Hold shift and click a second node to union both neighborhoods — trace thought paths across two hops. - **Double-click open**: Double-click a graph node to navigate to that note in the editor. Graph view closes, editor opens with the selected note. - **Hover tooltips**: Hovering a node shows title, tag pills, and connection count in an HTML overlay (no additional API calls). - **Folder coloring**: Nodes colored by folder using a 10-color palette. Unfiled notes shown in gray. - **Tag filter integration**: Active tag filter dims non-matching nodes to 30% opacity, stacking with click focus. - **Orphan highlighting**: Notes with zero connections shown with dashed stroke. "Hide orphans" checkbox in toolbar to toggle visibility. ### Changed - Notes package version bumped from 0.7.0 to 0.8.0. - Topbar gains a "Graph" toggle button alongside "New Note" and "Import .md". ## v0.4.6 — Sidebar Restructure ### Added - **Sidebar tabs**: Two-tab header ("Notes" / "Outline") replaces the static sidebar header. Notes tab contains folder tree, tag filter, search, and note list. Outline tab shows the document heading tree for the active note. Outline tab disabled (grayed) when no note is selected. - **Sidebar outline**: Document outline relocated from editor-side panel into the sidebar's Outline tab. Nested heading hierarchy (H1→H2→H3) with depth indentation via `padding-left`. Click heading scrolls to it in both preview and CM6 editor modes. - **Active heading tracking**: Scroll listener (RAF-throttled) highlights the current heading in the sidebar outline based on scroll position. Uses `getBoundingClientRect()` in preview/split mode and CM6 viewport line in edit mode. ### Changed - Notes package version bumped from 0.6.0 to 0.7.0. - `EditorPane` exposes headings, scroll function, and active heading index to parent via callback props (`onHeadingsChange`, `scrollToHeadingRef`, `onActiveHeadingChange`). - Sidebar auto-resets to Notes tab when active note is deleted. ### Removed - Old `DocumentOutline` component and `.doc-outline` CSS styles (replaced by sidebar outline). ## v0.4.5 — Editor Modes + Document Outline ### Added - **Tri-state view mode**: Notes open in rendered (read-only) mode by default. Toolbar button cycles through Edit (CM6) and Split (side-by-side) modes. Replaces the old binary preview toggle. - **Split view**: Side-by-side CM6 editor + rendered preview using CSS grid. CM6 instance preserved when toggling between edit and split modes. - **Synced scroll**: In split mode, scrolling the CM6 editor proportionally scrolls the preview pane via linear ratio mapping on `scrollDOM`. - **View mode persistence**: User's preferred mode saved to localStorage (`sw.storage.local('notes')`). New `editor_mode` manifest setting with admin-level default (rendered / edit / split). - **Document outline**: Collapsible TOC panel adjacent to the editor body. `parseHeadings()` extracts heading level, text, and line number (skips fenced code blocks). Click heading to scroll — uses CM6 `EditorView.scrollIntoView()` in edit/split mode, DOM `scrollIntoView()` in rendered mode. Updates on body change with 300ms debounce. ### Changed - Notes package version bumped from 0.5.0 to 0.6.0. - Editor body wrapped in `.notes-editor__content` flex container to accommodate the outline panel. - Wikilink click handling active in both rendered and split modes. - Mobile: split view collapses to single column, outline panel hidden. --- ## v0.4.4 — Rich Editor + Import/Export ### Added - **CodeMirror 6 integration**: Notes editor now uses the vendored CM6 bundle (`CM.noteEditor()`) for rich markdown editing with syntax highlighting, wikilink autocomplete (`[[` triggers note title completion), and inline preview decorations for headings, code blocks, and blockquotes. - **CM6 dynamic loading**: Bundle loaded via `