switchboard/core
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Releases 9 Wiki Activity

V0.6.2 docs openapi (#37)
All checks were successful
CI/CD / detect-changes (push) Successful in 4s
Details
CI/CD / test-frontend (push) Successful in 6s
Details
CI/CD / test-go-pg (push) Successful in 2m46s
Details
CI/CD / test-sqlite (push) Successful in 2m49s
Details
CI/CD / build-and-deploy (push) Successful in 28s
Details

Browse Source 
Co-authored-by: Jeffrey Smith <jasafpro@gmail.com>
Co-committed-by: Jeffrey Smith <jasafpro@gmail.com>
This commit was merged in pull request #37.
This commit is contained in:
Jeffrey Smith
2026-03-31 12:01:51 +00:00
committed by xcaliber
parent 1a7f41493d
commit a887b4c78b
32 changed files with 3862 additions and 548 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
CHANGELOG.md
View File

@@ -2,6 +2,83 @@
All notable changes to Switchboard Core are documented here.
## v0.6.2 — Docs Polish + Dynamic OpenAPI
Docs surface polish and dynamic OpenAPI spec with extension route merging.
### Added
- **Dynamic OpenAPI spec**: `GET /api/docs/openapi.json` builds a merged
spec at request time — kernel endpoints + auto-generated stubs for every
extension `api_routes` entry. Extensions may optionally declare `api_schema`
in `manifest.json` for richer path items (params, body, response examples).
- **`api_schema` manifest field**: Optional array in `manifest.json`. Malformed
entries are logged and skipped — never blocks extension loading.
- **Tests**: 7 new handler tests (zero extensions, stubs, rich schema,
multi-extension, malformed schema, disabled exclusion, required fields).
### Fixed
- **Dark mode contrast**: Replaced all hardcoded light-mode fallbacks in docs
CSS with theme-aware CSS variables. Tables, code blocks, nav items, and
headings now readable in dark mode.
- **Docs loading state**: Replaced borrowed `settings-placeholder` animation
with docs-specific pulse skeleton. Added error state with retry button.
- **Docs routing**: Default route (`/docs`) now redirects to
`/docs/GETTING-STARTED` instead of the blank `/:section` placeholder.
- **Docs scrolling**: Content area now scrolls correctly — full document
visible without text overflowing viewport.
- **Docs icon**: Changed from 🧩 to 📖 in the user menu.
- **Docs duplicate menu entry**: Docs was appearing twice (once from surfaces
API, once from hardcoded items). Added `'docs'` to `CORE_IDS` filter.
- **Swagger UI**: Updated to point at dynamic `/api/docs/openapi.json`
endpoint. Static YAML preserved for backward compatibility.
- **OpenAPI tests on Postgres**: `openapiTestStores()` was hardcoded to
`sqlite.NewStores()` — caused `pq: syntax error at or near ","` in CI.
Now uses `database.IsSQLite()` check like all other handler tests.
### API
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/docs/openapi.json` | GET | Dynamic merged OpenAPI 3.0 spec |
---
## v0.6.1 — Backup/Restore + Documentation
Operational tooling for data safety and extension authoring.
### Added
- **Backup/Restore**: Full-instance backup as `.swb` ZIP archive containing
core tables (JSONL), ext_data tables, and package assets. Stream to client
or save server-side. Restore wipes and rebuilds from archive. Dialect-neutral
(works across SQLite and Postgres).
- **Admin backup section**: New "Backup" tab under `/admin` with create,
download, delete, and restore operations. Destructive restore requires
confirmation dialog.
- **Documentation surface**: Builtin surface at `/docs` with sidebar navigation
and client-side markdown rendering. Ships with 5 documents: Getting Started,
Extension Guide, API Reference, Deployment, Package Format.
- **Docs API**: `GET /api/v1/docs` lists available documents, `GET /api/v1/docs/:name`
returns raw markdown content. Authenticated (all users).
- **Tests**: 6 backup handler tests + E2E script (`ci/e2e-backup-test.sh`).
### API
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/v1/admin/backup` | POST | Create backup (stream or `?store=true`) |
| `/api/v1/admin/backups` | GET | List server-side backups |
| `/api/v1/admin/backups/:name` | GET | Download backup |
| `/api/v1/admin/backups/:name` | DELETE | Delete backup |
| `/api/v1/admin/restore` | POST | Restore from `.swb` upload |
| `/api/v1/docs` | GET | List documentation |
| `/api/v1/docs/:name` | GET | Get document content |
---
## v0.6.0 — Cluster Registry + HA
MVP convergence point. PG-backed cluster registry for horizontal scaling —

3
Dockerfile
View File

@@ -77,6 +77,9 @@ COPY --from=backend /app/database/migrations /app/database/migrations
COPY src/ /usr/share/nginx/html/
COPY VERSION /VERSION
# Documentation (v0.6.1) — served by Go backend via /api/v1/docs
COPY docs/ /app/docs/
# Inject version and build hash into index.html and sw.js at build time
RUN APP_VERSION=$(cat /VERSION | tr -d '[:space:]') && \
BUILD_HASH=$(find /usr/share/nginx/html/js -name '*.js' -exec md5sum {} + | sort | md5sum | cut -c1-8) && \

525
ROADMAP.md
View File

@@ -1,415 +1,23 @@
# Switchboard Core — Roadmap
## Current: v0.5.3 — Chat Polish + Integration Testing
## Current: v0.6.2 — Docs Polish + Dynamic OpenAPI
Fork of chat-switchboard, gutted to a pure extension platform. All AI/chat
features removed from the kernel. What remains is the minimum viable
platform that extensions build on.
Self-hosted extensible platform. Auth, identity, packages, Starlark sandbox,
storage, realtime, and ops are kernel primitives. Everything else is an extension.
### Retained kernel capabilities
**Kernel capabilities:** Auth (builtin/mTLS/OIDC) · Users/teams/groups/RBAC ·
Surfaces/extensions/libraries/workflows · Starlark sandbox (capability-gated) ·
Object storage (PVC/S3) + ext_data tables · WebSocket hub + realtime pub/sub ·
Audit log · Notifications · Scheduled tasks
- **Auth**: builtin (simple), mTLS, OIDC
- **Identity**: users, teams, groups, permissions (RBAC)
- **Packages**: surfaces, extensions, libraries, workflows
- **Starlark sandbox**: capability-gated modules
- **Storage**: object storage (PVC, S3), ext_data tables
- **Realtime**: WebSocket hub, presence, multi-replica HA
- **Ops**: audit log, notifications, maintenance goroutine
**Completed history:** v0.2.x–v0.5.x fully documented in `CHANGELOG.md`.
Highlights: RBAC + settings cascade, event bus + triggers, SDK stabilization,
workflow engine (multi-stage, team roles, signoff gate, public entry, SLA),
package distribution, Notes surface (CM6, folders, tags, backlinks, graph),
realtime primitive, Chat surface (chat-core library + surface + polish),
upgrade test harness, cluster registry + HA.
### Phase 0 (complete)
| Step | Status | Description |
|------|--------|-------------|
| 1. Module rename | ✅ | `chat-switchboard` → `switchboard-core` |
| 2. Delete packages | ✅ | 15 Go packages, 29 handler files removed |
| 3. Gut stores/models | ✅ | 40 → 20 store interfaces, kernel-only models |
| 4. Fresh migrations | ✅ | 9 files × 2 dialects, 27 tables |
| 5. Fix compilation | ✅ | `go build ./...` clean, 300+ stale route lines cut |
| 6. Fix tests | ✅ | 8 test packages pass, ~12K stale test lines pruned |
| 7. Frontend gut | ✅ | Shell + SDK only, 50+ files of chat/notes/projects code removed |
| 8. New ICD | ✅ | Full OpenAPI 3.0.3 spec — 160 operations across 22 tag groups |
| 9. CI/CD + Dockerfile | ✅ | Single unified image, FE/BE split removed, DB names updated, k8s var alignment fixes (resource quantities, image name, rollout deployment name) |
| 10. Smoke test | ✅ | K8s deploy live at switchboard.gobha.ai/test, nginx BASE_PATH fixed, login→admin flow verified, branding updated |
## v0.2.x — SDK & Triggers
The contract that extensions build against. Three trigger primitives,
SDK stabilization, and the first rebuilt extension (tasks).
### v0.2.0 — RBAC + Settings Cascade (complete)
| Step | Status | Description |
|------|--------|-------------|
| Admin → RBAC group | ✅ | `surface.admin.access` permission + Admins system group replaces `role == "admin"` checks. Admin bypass removed from permission middleware. |
| Settings cascade | ✅ | `user_overridable` flag, three-tier resolution (global → team → user), team settings API |
| ~~Settings override model~~ | ✅ | Shipped with settings cascade above |
### v0.2.1 — Default Surface + ICD
| Step | Status | Description |
|------|--------|-------------|
| Default surface routing | ✅ | `/` redirects to configurable default surface. No surfaces → admin. First install becomes default. Changeable in admin settings. |
| ICD (API contract) | ✅ | Full OpenAPI 3.0.3 spec — 160 operations, 22 tag groups, reusable component schemas. Served at `/api/docs`. |
### v0.2.2 — Event Bus + Triggers
| Step | Status | Description |
|------|--------|-------------|
| Event bus subscriptions | ✅ | Extensions register event patterns in manifest. Wired via `bus.Subscribe()` on startup. Async handler invocation. |
| Webhook triggers | ✅ | Inbound HTTP at `/api/v1/hooks/:package_id/:slug`. HMAC-SHA256 verification. Synchronous Starlark handler response. |
| Scheduled tasks | ✅ | User-created cron tasks with restricted sandbox (no raw HTTP, no DB table creation). Runs as creator identity. Templates from extensions. Dedicated schedules API. |
| Trigger admin API | ✅ | CRUD for triggers + schedules. Enable/disable, execution logs, per-package listing. |
### v0.2.3 — SDK + Task Extension
| Step | Status | Description |
|------|--------|-------------|
| SDK stabilization | ✅ | `sw.api.ext()`, `sw.storage`, `sw.theme.tokens`, `sw.ui`, `sw.slots`, `sw.actions` — six new SDK modules for extension development |
| Task extension | ✅ | Full task surface rebuilt as Starlark extension: CRUD API, kanban/list views, event triggers, webhook integration, notifications on completion |
### v0.2.4 — Shell Navigation + Schedules
| Step | Status | Description |
|------|--------|-------------|
| SDK Topbar | ✅ | `sw.shell.Topbar` — composable navigation bar (title + extension slot + bell + user menu). Surfaces get consistent nav for free. |
| Schedules surface | ✅ | New `packages/schedules/` wrapping kernel cron API. Table view, cron preview, enable/disable, manual run, execution logs. |
| Manifest icons | ✅ | `icon` field in manifest.json (emoji). Surfaces API returns icon. UserMenu renders per-surface icons. |
| UserMenu cleanup | ✅ | Removed dead Chat/Notes/Projects links. Menu driven by surfaces API. Core surfaces filtered. |
| isAdmin RBAC fix | ✅ | `can.js` isAdmin() now checks `surface.admin.access` grant instead of deprecated role column. |
### v0.2.5 — UI Polish + Dead Code Audit
| Step | Status | Description |
|------|--------|-------------|
| UI bug pass | ✅ | Reviewed admin, settings, login, welcome surfaces in light/dark themes. Fixed settings crash (models API undefined). Removed dead chat settings from both user and admin settings. |
| Dead code sweep | ✅ | Removed ~700 lines: orphaned CSS, dead Go types, stale event code, unused test helpers, dead settings UI (chat defaults, system prompt, default model, policies, web search, compaction, memory). |
| Template cleanup | ✅ | Removed stale CSS link tags and orphaned chat-pane.html. Updated doc comments throughout. |
| Package proof-of-concept status | ✅ | Created README.md for tasks + schedules with graduation criteria. Added missing manifest icons. |
| Welcome surface | ✅ | New fallback surface when no extensions installed. Topbar + welcome card with admin link. Replaces `/admin` as final redirect target. |
| Default surface routing | ✅ | Resolution chain: user preference → global config → first extension → `/welcome`. Users can set personal default in Settings > General. Admin sets global default in Admin > Settings. |
| Admin navigation | ✅ | Replaced Back button with UserMenu in admin topbar. Eliminates back-button infinite loop. |
### v0.2.6 — Admin Settings Audit
| Step | Status | Description |
|------|--------|-------------|
| Admin settings E2E | ✅ | All 7 admin settings sections verified (default surface, registration, banner, message bar, footer, vault, email). Dead code removed: `sectionCategory()` pruned of AI/routing/channel vestiges, `PublicSettings()` stripped of chat-era fields (system_prompt, retention_ttl, paste_to_file, allow_user_personas), dead `PolicyDefaults` removed (allow_raw_model_access, default_model), dead policy lookups removed (kb_direct_access), test seed data cleaned. |
| Packages surface | ✅ | Package list loads (17 total/17 enabled), type filters work, enable/disable visible, settings/export buttons present, core package (admin) protected. Removed dead `chat` from CORE_IDS. |
### v0.2.7 — User Settings Audit
| Step | Status | Description |
|------|--------|-------------|
| User settings E2E | ✅ | All 6 user settings sections verified (General, Appearance, Profile, Teams, Connections, Notifications). Dead code removed: BYOK nav section + state, personas gate filter, Message Font Size slider, `auth.permissions.changed` listener. localStorage key renamed `cs-appearance` → `sb-appearance` with one-time migration. |
| Visibility gating | ✅ | Dead BYOK/personas policy lookups removed from bootstrap and permissions handlers. `allow_user_byok` removed from `PublicSettings`. `PolicyDefaults` cleaned of `allow_user_byok` and `allow_user_personas`. Dead `msgFont` early-apply removed from base template. Stale policy-gating test assertions replaced. No empty nav sections remain. |
### v0.2.8 — Team Admin Settings Audit (Pass 1)
| Step | Status | Description |
|------|--------|-------------|
| Team admin E2E | ✅ | Audited team member management, settings cascade, role assignment. Removed dead code: `HasPrivateProviderRequirement` (BYOK vestige), `UserRole` on `TeamMember` (deprecated role column), `allow_team_providers` policy default, dead personas/providers/models ICD tests. |
| Workflow stage UI cleanup | ✅ | Removed dead personas dropdown, `history_mode` selector, stale `chat_only` mode. Updated `STAGE_MODES` to match backend CHECK constraint (`form_only`, `form_chat`, `review`, `custom`). Removed stale comments referencing deleted files. |
### v0.2.9 — Builtin Extension Retirement
| Step | Status | Description |
|------|--------|-------------|
| Retire builtin seeder | ✅ | Removed `SeedBuiltinPackages`, `seed_packages.go`, Dockerfile COPY, and `extensions/builtin/` directory. These extensions are dormant until a chat surface exists to consume them. |
| Convert to regular packages | ✅ | Repackaged 6 extensions as standard directories in `packages/` with `js/` layout and `"requires": ["chat"]` manifest metadata. Built via `build.sh` like all other packages. No auto-install — explicit install only. |
## v0.3.x — Workflow Architecture
Workflows are the core platform capability. This series implements the
full multi-step automation system with team role integration and
finalizes the extension lifecycle model.
### v0.3.0 — Schema Redesign + Stage CRUD Modernization (complete)
| Step | Status | Description |
|------|--------|-------------|
| Workflow schema redesign | ✅ | Dropped `persona_id`, `history_mode` from `workflow_stages`. Renamed `transition_rules` → `stage_config`. Updated `stage_mode` CHECK to `(form, review, delegated, automated)`. Added `audience`, `stage_type`, `starlark_hook`, `branch_rules`. Both PG + SQLite. |
| Model + store updates | ✅ | Go structs, constants, and PG/SQLite queries updated. New `ValidStageTypes`, `ValidAudiences` maps. Routing engine reads `branch_rules` directly. |
| Handler + Starlark updates | ✅ | Stage CRUD validation for new fields. Package export/import updated. Hook handler reads `stage_config`. Starlark module exposes `audience`, `stage_type`. |
| Frontend updates | ✅ | Team-admin and admin stage editors updated with new modes, audience selector, stage type selector, conditional Starlark hook input. Fixed admin `STAGE_MODES` bug. |
### v0.3.1 — Instance + Assignment Tables + Store (complete)
| Step | Status | Description |
|------|--------|-------------|
| Instance schema | ✅ | `workflow_instances` table in 007_workflows.sql (PG + SQLite). Tracks execution state: workflow_version, current_stage, stage_data, status, entry_token. |
| Assignment schema | ✅ | `workflow_assignments` table. Per-stage queue: instance_id, stage, team_id, assigned_to, status, review_data. Optimistic claim lock. |
| Models + store interface | ✅ | `WorkflowInstance`, `WorkflowAssignment` structs. 15 store methods for instance lifecycle and assignments across PG + SQLite. |
| Event types | ✅ | `workflow.started`, `workflow.cancelled`, `workflow.error` added to bus route table. |
### v0.3.2 — Workflow Engine + Handlers (complete)
| Step | Status | Description |
|------|--------|-------------|
| Store tests | ✅ | Round-trip tests for all 15 v0.3.1 store methods (instance + assignment CRUD). TruncateAll updated for new tables. |
| Stage execution engine | ✅ | `server/workflow/engine.go` — Start, Advance (with branch_rules), Cancel. Merges stage_data, creates assignments, emits events. |
| Automated stages | ✅ | `server/workflow/automated.go` — fire Starlark hook, auto-advance, cycle guard (max 10 consecutive). |
| Instance handlers | ✅ | HTTP API: Start, GetInstance, Advance, Cancel, ListInstances. Team-scoped mirrors. |
| Assignment handlers | ✅ | HTTP API: Claim, Unclaim, Complete, Cancel, ListByTeam, ListMine. Claimer identity verification. |
| Starlark module expansion | ✅ | `workflow.get_instance()`, `workflow.list_instances()` (read-only). Mutating builtins deferred pending interface extraction. |
### v0.3.3 — Public Entry + Background Jobs (complete)
| Step | Status | Description |
|------|--------|-------------|
| Public entry | ✅ | `StartPublic`, `ResumePublic`, `AdvancePublic` — token-based anonymous workflow participation. Public routes at `/api/v1/public/workflows/`. Audience-gated: only `public` stages can be advanced anonymously. |
| SLA scanner | ✅ | Background goroutine (5-min interval) checking active instances against per-stage `sla_seconds`. Fires `workflow.sla_breach` event, marks breach in instance metadata (idempotent). |
| Staleness sweep | ✅ | Per-workflow `staleness_timeout_hours` column. Scanner marks idle instances as `stale`, cancels open assignments, fires `workflow.stale` event. |
### v0.3.4 — Team Roles + Multi-party Validation (complete)
| Step | Status | Description |
|------|--------|-------------|
| Team roles | ✅ | Removed CHECK constraint on `team_members.role`. Custom roles stored in `teams.settings["roles"]`. Team roles API (`GET/PUT /teams/:teamId/roles`). Role-based stage assignment via `stage_config.required_role`. Frontend: dynamic role selects, role management panel. |
| Multi-party sign-off | ✅ | `workflow_signoffs` table (both dialects). `StageConfig.validation` with `required_approvals`, `required_role`, `reject_action`. Engine validation gate in `advanceInternal`. `SubmitSignoff` engine method. Signoff HTTP API (`POST/GET /instances/:iid/signoffs`). Frontend signoff panel in monitor tab. |
| Extension lifecycle | ✅ | Design doc: `docs/DESIGN-EXTENSION-LIFECYCLE.md`. Permanent vs PoC classification, graduation criteria, install model. |
| Trigger composition | ✅ | Design doc: `docs/DESIGN-TRIGGER-COMPOSITION.md`. Triggers/schedules can start workflows, workflows emit events, no circular invocation. |
### v0.3.5 — Settings Audit + ICD + Tests
| Step | Status | Description |
|------|--------|-------------|
| Clone endpoint | ✅ | `POST /api/v1/workflows/:id/clone` — deep copy workflow + stages. Handler-level composition of existing store methods. |
| Integration tests | ✅ | 7 engine-level tests in `workflow_engine_test.go`: full lifecycle, branch routing, public entry, signoff gate, rejection, cancel-clears-assignments, error cases. 28 total tests. |
| Settings audit pass 2 | ✅ | Added `staleness_timeout_hours` field to workflow editor. Added collapsible branch_rules JSON editor in stage form. |
| ICD update | ✅ | Fixed stale Workflow/Stage schemas. Added WorkflowInstance, WorkflowAssignment, WorkflowSignoff schemas. Added ~20 new endpoints: instances, assignments, signoffs, public entry, clone, team roles. |
### v0.3.6 — Example Workflows + Interactive Demo
Four installable `.pkg` workflow packages that prove the engine works
end-to-end, plus a demo surface for guided exploration.
See `docs/DEMO-WORKFLOWS.md` for full stage definitions and Starlark code.
| Step | Status | Description |
|------|--------|-------------|
| Bug Report Triage | ✅ | Public entry, progressive fieldsets, severity-based branch routing, team assignment, SLA timer. Pure-manifest `.pkg`. |
| Employee Onboarding | ✅ | Starlark automated stages (`db.insert`, `notifications.send`), signoff gate with `required_role`, rejection reroute. `.pkg` with `script.star`. |
| Content Approval | ✅ | Multi-party signoff (quorum of 2), rejection reroute creating review cycle. `.pkg` demonstrating signoff + reroute loop. |
| Webhook Notifier | ✅ | Starlark `http.post` + `connections.get` for outbound webhooks, delivery logging to ext_data. Proves HTTP + connections modules. |
| Demo surface | ✅ | Browser-tier walkthrough: workflow cards, stage diagrams, Starlark viewer, "Try It" buttons, API curl examples. Auto-installed, removable. |
| Engine context fix | ✅ | Add `started_by` to automated stage Starlark context dict (backward-compatible). |
| SLA package installer | ✅ | Added `sla_seconds` support to `workflowPkgStage` struct and install/export paths. |
| Snapshot format fix | ✅ | `parseSnapshotStages` helper handles both wrapped and legacy snapshot formats. |
| Workflow adoption | ✅ | `POST /teams/:teamId/workflows/:id/adopt` + `GET .../available`. Team-admin "Adopt Global" button. `TeamID` in `WorkflowPatch`. |
| Extension SDK boot | ✅ | `base.html` loads Preact globals + `boot()` for extension surfaces (was missing since Scorched Earth IV). |
| Admin teams tab fix | ✅ | Extract `.data` from paginated response in admin teams list. |
| Documentation | ✅ | `docs/DEMO-WORKFLOWS.md`: feature capability matrix, stage flows, API walkthrough, per-package READMEs. |
| Review pass | | Docker E2E walkthrough: install all packages, adopt into team, activate, publish, run each workflow end-to-end. Fix remaining UI/UX issues. |
### v0.3.7 — Package Audit (complete)
Verified all 16 packages install correctly. Packages with unmet `requires` auto-set to dormant.
| Step | Status | Description |
|------|--------|-------------|
| Manifest fixes | ✅ | Fixed 6 chat-extension manifests (`"name"` → `"title"`, was blocking install). Added explicit `"type": "surface"` to hello-dashboard, icd-test-runner, sdk-test-runner. |
| Dormant status | ✅ | Added `dormant` to `packages.status` CHECK constraint (both dialects). Installer auto-detects unmet `requires` and sets `status=dormant, enabled=false`. Enable endpoint returns 409 for dormant packages. |
| Functional audit | ✅ | Navigated to every surface in browser. 7 working: schedules, tasks, team-activity-log, git-board, hello-dashboard, icd-test-runner, sdk-test-runner. 2 broken: dashboard + editor (depend on removed `sw.*` imperative SDK) — tagged `requires: ["legacy-sdk"]` → dormant. |
| Chat-dependency tagging | ✅ | 6 chat-dependent extensions (csv-table, diff-viewer, js-sandbox, katex-renderer, mermaid-renderer, regex-tester) install as dormant. git-board and gitea-client are standalone. |
| Dependency check fix | ✅ | Relaxed library dependency validation to allow `pending_review` libraries (gitea-client declares permissions). Only `suspended`/`dormant` libraries blocked. |
| Admin UI | ✅ | Dormant badge, disabled Enable button with tooltip, Dormant stat card in admin packages view. |
| Tests | ✅ | 4 handler tests (SetStatus dormant, enable blocked, surfaces exclude dormant, admin list includes dormant). All existing tests pass. |
### v0.3.8 — Distribution
Builder image for faster builds, bundled packages for zero-config first run.
| Step | Status | Description |
|------|--------|-------------|
| Builder image | ✅ | `Dockerfile.builder` with cached Go modules + Node vendor libs for faster custom builds. |
| Bundled packages | ✅ | Dockerfile stage builds all 12 non-dormant packages into production image. `InstallBundledPackages` auto-installs on first run (skip-if-present). `SKIP_BUNDLED_PACKAGES=true` to disable. `BUNDLED_PACKAGES` allowlist for selective install (Helm/K8s). Migration 012 adds `bundled` source to CHECK constraint. |
| Distribution docs | ✅ | `docs/DISTRIBUTION.md` — quick start, bundled packages, builder image, custom builds, production deployment. |
| Tests | ✅ | 6 handler tests (fresh install, skip existing, missing dir, empty dir, dormant handling, allowlist filtering). All existing tests pass. |
## v0.4.x — Notes Surface
Obsidian-style notes rebuilt as an installable surface package.
Zero platform special-casing. Proves the full extension stack E2E.
### v0.4.0 — Core Notes CRUD + Markdown Editor
| Step | Status | Description |
|------|--------|-------------|
| Package scaffold | ✅ | `packages/notes/` with manifest, script.star, JS, CSS, README. Type: full, tier: starlark. |
| Notes CRUD backend | ✅ | Starlark `on_request()` — list, create, get, update, delete, search, stats. Lightweight list projection (no body). |
| Notes table | ✅ | `ext_notes_notes` — title, body (TEXT), folder_id, pinned, archived, creator_id, updated_at. |
| Markdown editor | ✅ | Preact+htm frontend: sidebar note list, title input, markdown textarea, auto-save (1s debounce). |
| Live preview | ✅ | Inline markdown renderer (~100 lines): headings, bold, italic, code, links, lists, blockquotes, hr. |
| Search | ✅ | Client-side LIKE search over title/body. Search bar in sidebar. |
| Pin/archive | ✅ | Pin notes to top, soft-delete via archive, hard delete with `?hard=1`. |
### v0.4.1 — Folders + Navigation Tree (complete)
| Step | Status | Description |
|------|--------|-------------|
| Folders table | ✅ | `ext_notes_folders` — name, parent_id, creator_id, sort_order. Indexed on parent_id and creator_id. |
| Folder CRUD API | ✅ | 5 Starlark handlers: list, create, update, delete folders + `POST /notes/move`. Delete cascade orphans notes and reparents child folders. |
| Folder tree UI | ✅ | `FolderTree` + `FolderNode` components. Flat-to-tree builder, expand/collapse, depth indentation, inline rename via context menu. |
| Folder filtering | ✅ | Click folder → filter notes. "All Notes" / "Unfiled" virtual views. New notes inherit active folder. |
| Editor folder select | ✅ | Dropdown with nested hierarchy (`└` prefix). Moves notes between folders via move-note API. |
| Updated stats | ✅ | Stats include `unfiled` and `folders` counts. |
### v0.4.2 — Tags + Search (complete)
| Step | Status | Description |
|------|--------|-------------|
| Tags table | ✅ | `ext_notes_tags` — note_id, tag. Indexed on both columns for bidirectional lookup. |
| Tag CRUD API | ✅ | 3 Starlark handlers: list all unique tags, get tags for note, replace tags for note (delete+reinsert). Tags normalized: lowercase, trimmed, deduped. |
| Tags in list/get/search | ✅ | `_list_notes` batch-fetches all tags in one query, attaches `tags` array to each item. `_get_note` includes tags. `_search_notes` matches against tags. |
| Tag cascade delete | ✅ | Hard-deleting a note also deletes its tag rows. Stats include `tags` count. |
| Tag input in editor | ✅ | `TagInput` component: removable pills + text field, comma/Enter to add, autocomplete dropdown from all tags. |
| Tag pills on cards | ✅ | `NoteCard` renders up to 3 tag pills with "+N" overflow. |
| Tag filter in sidebar | ✅ | `TagFilter` component: clickable pills, toggle active tag, client-side note filtering. |
| Drag-and-drop | ✅ | `NoteCard` is draggable. `FolderNode`, "All Notes", and "Unfiled" are drop targets. Uses existing move-note API. |
| Folder context menu | ✅ | Replaced `prompt()` hack with proper right-click popup menu: Add subfolder, Rename, Delete. Positioned at click coordinates, dismissed on outside click. |
### v0.4.3 — Backlinks + Wikilinks (complete)
| Step | Status | Description |
|------|--------|-------------|
| Links table | ✅ | `ext_notes_links` — source_id, target_id, link_text. Indexed on both source_id and target_id for bidirectional lookup. |
| Wikilink extraction | ✅ | `_extract_wikilinks()` parses `[[...]]` using `split("[[")` + `find("]]")` (no regex/while in Starlark). Deduplicates by lowercase. |
| Link sync on save | ✅ | `_sync_links()` called from `_create_note` and `_update_note`. Delete-all + reinsert pattern (matches tags). Resolves titles to note IDs (case-insensitive). Unresolved links stored with `target_id=""`. |
| Cascade delete | ✅ | Hard-deleting a note removes both outgoing links (source_id) and incoming backlinks (target_id). |
| Link endpoints | ✅ | `GET /links/:note_id` (outgoing) and `GET /backlinks/:note_id` (incoming, enriched with source titles). |
| Wikilink preview | ✅ | `[[Note Title]]` rendered as clickable accent-colored links in markdown preview. Unresolved links styled in danger/red. |
| Wikilink navigation | ✅ | Clicking a resolved wikilink navigates to the target note. Clicking an unresolved (red) link creates the note, navigates to it, and re-saves the source so the link resolves. |
| Backlinks panel | ✅ | Collapsible panel below editor showing all notes linking to current note. Click to navigate. Hidden when empty. |
| Stats update | ✅ | `/stats` includes `links` count. Notes package version bumped to 0.4.0. |
### v0.4.4 — Rich Editor + Import/Export (complete)
| Step | Status | Description |
|------|--------|-------------|
| CM6 integration | ✅ | Load vendored CodeMirror 6 bundle via dynamic `<script>` tag. `CM.noteEditor()` factory with markdown syntax highlighting, wikilink autocomplete, and live preview decorations. Textarea fallback if CM6 unavailable. |
| Editor wiring | ✅ | EditorPane uses CM6 with `onChange` → auto-save debounce, `onLink` → wikilink navigation, `linkCompleter` → search API autocomplete. Ctrl/Cmd+S force save. Editor fills container via CSS override. |
| Export as .md | ✅ | Export button in editor header. Downloads note as `.md` file with YAML frontmatter (title, tags, created). Pure client-side Blob download. |
| Import .md | ✅ | Import button in topbar. File picker for `.md`/`.markdown`/`.txt`. Parses YAML frontmatter for title and tags. Creates note via API with tags. |
| Notes package v0.5.0 | ✅ | Manifest version bumped from 0.4.0 → 0.5.0. |
### v0.4.5 — Editor Modes + Document Outline (complete)
| Step | Status | Description |
|------|--------|-------------|
| Default-rendered mode | ✅ | Preview is the initial state on note open. "Edit" button enters CM6. Tri-state `viewMode`: `rendered` / `edit` / `split`. |
| Split view | ✅ | Side-by-side layout: CM6 left, rendered preview right. CSS grid two-column in `.notes-editor__body--split`. |
| Synced scroll | ✅ | Split mode scroll sync. CM6 `scrollDOM` scroll listener maps position ratio to preview `scrollTop`. Linear mapping. |
| View mode setting | ✅ | `editor_mode` in manifest settings + localStorage persistence. Toolbar cycles rendered → edit → split. |
| Document outline | ✅ | `parseHeadings()` extracts headings (skips code blocks). Collapsible `DocumentOutline` panel with click-to-scroll (CM6 line dispatch or preview `scrollIntoView`). Debounced 300ms updates. |
### v0.4.6 — Sidebar Restructure (complete)
| Step | Status | Description |
|------|--------|-------------|
| Sidebar tabs | ✅ | Two-tab header at top of sidebar: "Notes" (folders → tags → search → list) and "Outline" (heading tree for active note). Outline tab enabled only when a note is selected. |
| Heading tree | ✅ | Nested heading hierarchy (H1 → H2 → H3) with depth indentation. Click → scroll to heading in editor/preview. Current heading highlighted based on scroll position. |
### v0.4.7 — Note Graph
| Step | Status | Description |
|------|--------|-------------|
| Graph API | ✅ | `GET /graph` endpoint in Starlark — `{ nodes: [{id, title, folder_id, tags}], edges: [{source, target, text}] }`. Full notes + links in one payload. |
| Graph renderer | ✅ | Canvas force-directed layout. Minimal force simulation (repulsion + attraction + damping). Nodes = circles with title labels, edges = lines. Zoom/pan via wheel + drag. Retina-aware rendering. |
| Click → focus + filter | ✅ | Single click a node: dims unconnected nodes/edges to 15% opacity. Clicked node + direct neighbors stay full brightness. Edges highlighted in accent color. Click empty space to reset. |
| Shift+click → chain | ✅ | Shift+click adds a second node to the focus set — union of both neighborhoods visible. Trace thought paths across two hops without losing context. |
| Double-click → open | ✅ | Double-click navigates to the note and exits graph view, opening the editor with the selected note. |
| Hover → tooltip | ✅ | Hover shows note title + tag pills + edge count in HTML overlay. No API call — all data in graph payload. |
| Folder/tag coloring | ✅ | Nodes colored by folder using 10-color palette. Active tag filter dims non-matching nodes to 30% opacity. Stacks with click focus. |
| Orphan highlighting | ✅ | Zero-edge nodes rendered with dashed stroke. "Hide orphans" checkbox toggle in toolbar. |
| Entry point | ✅ | "Graph" button in topbar. Replaces editor pane with full graph canvas. Single-click selects note in sidebar; double-click opens editor. |
## v0.5.x — Realtime + Chat
Communication primitive track. The kernel gains one generic realtime
module. Everything else — conversations, messages, participants — is
pure extension code. Human-to-human chat ships pre-MVP; LLM
participation is post-MVP.
### v0.5.0 — Realtime Primitive + Dialog Audit + Permissions UI
| Step | Status | Description |
|------|--------|-------------|
| `realtime.publish()` | ✅ | Starlark module: `realtime.publish(channel, event, data)`. Publishes JSON event to WebSocket hub scoped to channel. Gated by `realtime.publish` permission. 7KB payload limit. Reserved prefix validation. |
| WS room protocol | ✅ | `room.subscribe` / `room.unsubscribe` client messages intercepted in readPump. Per-connection 100-room cap. |
| `sw.realtime.subscribe()` | ✅ | SDK method: `sw.realtime.subscribe(channel, [event], callback)`. Auto room join/leave, reconnect recovery. Returns unsubscribe handle. |
| `sw.ui.Dialog` | ✅ | Pre-existing (implemented before v0.5.0). SDK modal primitive with focus trap, Promise-based API. |
| `sw.ui.Confirm` / `sw.ui.Prompt` | ✅ | Pre-existing. `sw.confirm()` and `sw.prompt()` already exposed in SDK. |
| Native dialog audit | ✅ | 5 bare `confirm()` calls migrated to `sw.confirm()` in tasks, schedules, notes (×2), editor packages. |
| Admin permissions UI | ✅ | Permissions drawer in admin Packages page. Grant/revoke per permission, grant-all bulk action. `pending_review`/`suspended` status badges. SDK API client methods added. Closes gap identified in v0.4.7. |
### v0.5.1 — Chat Core Library
| Step | Status | Description |
|------|--------|-------------|
| Library package | ✅ | `chat-core` library with `permissions: ["db.write", "realtime.publish"]`. Private ext_data tables, exported Starlark functions, REST endpoints. |
| Conversations table | ✅ | `conversations` — title, type (direct/group), created_by, updated_at. |
| Participants table | ✅ | `participants` — conversation_id, participant_id, participant_type (user/bot), display_name, role (member/admin), joined_at. |
| Messages table | ✅ | `messages` — conversation_id, participant_id, content, content_type (text/system/file), edited_at. |
| Read cursors table | ✅ | `read_cursors` — conversation_id, participant_id, last_read_message_id. |
| Exported Starlark API | ✅ | `create()`, `send()`, `history()`, `add_participant()`, `remove_participant()`, `mark_read()` — available via `lib.require("chat-core")`. |
| REST endpoints | ✅ | 15 routes: full CRUD for conversations, messages, participants. Cursor-based paginated message history. Per-conversation unread counts. |
| Realtime integration | ✅ | On `send()`: insert message, publish `realtime.publish("conversation:{id}", "message", data)`. Also emits `participant.added`, `participant.removed`, `message.edited`, `message.deleted` events. |
| db.query() range params | ✅ | Kernel enhancement: `before`/`after` kwargs for `db.query()` enabling cursor-based pagination and efficient unread counting. 7 new tests. |
| Admin Library filter | ✅ | Added `Library` type filter to admin Packages page alongside Surface/Extension/Full/Workflow. |
| Admin filter Dropdown | ✅ | Replaced button group with themed `Dropdown` primitive for type filter. Keyboard nav, consistent theming via CSS vars. |
| Unicode security gate | ✅ | `ScanSource()` detects variation selectors, bidi overrides, zero-width chars, tag characters. `Verdict()` blocks GlassWorm payloads and Trojan Source (CVE-2021-42574). Two enforcement points: install (422) + Starlark exec. 23 new tests. |
### v0.5.2 — Chat Surface
| Step | Status | Description |
|------|--------|-------------|
| Conversation list | done | Sidebar: conversation list with last message preview, unread badge, timestamp. Create conversation button. |
| Message thread | done | Main pane: message history with participant avatars, timestamps, content. Scroll-to-load-more for long threads. |
| Compose bar | done | Text input with Enter-to-send, Shift+Enter for newline. Auto-resize textarea. |
| Participant sidebar | done | Collapsible right panel: participant list with online status (via kernel presence hub). Add/remove participants using `sw.ui.Dialog`. |
| 1:1 and group | done | Direct messages (two participants) and group conversations. Type field on conversation. |
| Typing indicators | done | `realtime.publish("conversation:{id}", "typing", {participant_id})` on keypress with debounce. Surface shows "X is typing…" with auto-expire. |
| Read receipts | done | Mark-read on scroll/focus via `chat.mark_read()`. Unread count in conversation list. |
| Message editing | done | Edit own messages. `edited_at` timestamp displayed. Edit via `PUT /messages/:id`. |
| Message deletion | done | Soft-delete own messages. Replaced with "message deleted" placeholder. |
### v0.5.3 — Chat Polish + Integration Testing
| Step | Status | Description |
|------|--------|-------------|
| `db.query()` search_like | ✅ | Kernel enhancement: `search_like` kwarg on `db.query()` — generates OR-joined `LIKE` (SQLite) / `ILIKE` (Postgres) clauses. Reusable by any package. 5 new tests. |
| Conversation search | ✅ | `GET /search?q=term` endpoint in chat-core. Searches conversation titles and message content across user's conversations. Debounced search UI in sidebar with conversation + message results. |
| Message pagination polish | ✅ | Scroll position preservation on load-more (records `scrollHeight`, restores via `requestAnimationFrame`). Loading spinner at top during fetch. `has_more` button hidden while loading. |
| Multi-user E2E | ✅ | `docker-compose-e2e.yml`: 2 replicas + Postgres + nginx LB. `ci/e2e-chat-test.sh`: auth, create, send, cross-replica read, search, pagination. `ci/e2e-ws-listener.js` for realtime event testing. |
| Workflow integration | ✅ | `workflow-chat` library package: `on_advance` hook creates scoped group conversation, adds team members, sends system message, enriches `stage_data` with `conversation_id`. Idempotent. 6 new hook tests. |
### Backlog — Admin UI Polish
| Item | Description |
|------|-------------|
| "Requires Review" filter | Dropdown filter option or dedicated button for `pending_review` packages. Useful when the package list grows long. |
| Extreme UI scale polish | At 150%+ scale: user menu items overflow viewport (need scroll), topbar elements clipped. Ensure all surfaces and menus are usable at scale extremes (80%–200%). |
### v0.5.4 — Package Updates
| Step | Status | Description |
|------|--------|-------------|
| Curate bundled packages | ✅ | Default set reduced from 23 to 8 core packages. `BUNDLED_PACKAGES=*` for all; explicit list for custom. |
| Version comparison | ✅ | Semver parsing (`ParseSemver`) with prerelease support. Version bump validated on update. |
| Schema migration on update | ✅ | `MigrateExtTables` diffs `db_tables` against existing schema. Adds new columns/tables. No destructive changes. |
| Data preservation | ✅ | ext_data tables survive update. Settings merged (new keys added with defaults, existing preserved). |
| Update API | ✅ | `POST /api/v1/admin/packages/:id/update` — accepts `.pkg` archive, validates version bump, applies schema diff, replaces code. |
| Admin UI | ✅ | Update button on non-core package cards. Confirm dialog before upload. |
| Export API | ✅ | `GET /api/v1/admin/packages/:id/export` — streams installed package as .pkg ZIP. Manual rollback: export → update → re-install old .pkg if needed. |
### v0.5.5 — Upgrade Testing
| Step | Status | Description |
|------|--------|-------------|
| Test harness | ✅ | `docker-compose-upgrade.yml` + `ci/e2e-upgrade-test.sh`: build v(old) image, seed notes/conversations/settings, upgrade to v(new), verify data integrity. |
| Schema edge cases | ✅ | 11 Go unit tests: add column (idempotent), add table, add index, multi-column add, row preservation across migration. All pass on SQLite. |
| Settings migration | ✅ | Go tests verify: global/team/user overrides preserved across package update, new keys get defaults, removed keys not deleted from DB. |
| Package compatibility | ✅ | Go tests verify: bundled skip-if-present on restart, dormant status for unmet requires, enabled/type preserved across version bump. |
| Multi-replica upgrade | ✅ | `ci/e2e-upgrade-rolling.sh`: rolling upgrade with 2 replicas + shared Postgres, cross-replica reads during mixed-version window, pg_notify fan-out verified. |
---
## v0.6.0 — MVP
@@ -437,12 +45,103 @@ PG is the consensus layer. Zero new infrastructure. `UNLOGGED` table + `LISTEN/N
| Single-node regression | ✅ | One-node behavior identical to pre-cluster: one registry row, NOTIFY delivers back to same instance. No special-casing. SQLite returns nil store — all cluster code guarded. |
| Multi-node integration test | ✅ | Docker Compose: 3 instances, shared PG. `ci/e2e-cluster-test.sh`: registration, stale sweep on stop, re-registration on restart. 3 unit tests + 2 handler tests. |
### v0.6.1 — Backup/Restore + Documentation (planned)
### v0.6.1 — Backup/Restore + Documentation
| Step | Status | Description |
|------|--------|-------------|
| Backup tooling | | ext_data export/import, admin surface for backup management. |
| Documentation site | | Extension authoring guide, API reference, deployment guide. |
| Backup handler | ✅ | `POST /api/v1/admin/backup` streams `.swb` ZIP (JSONL core + ext_data tables + package assets). `POST /api/v1/admin/restore` wipes DB and restores from archive. Dialect-neutral (SQLite + Postgres). |
| Server-side backups | ✅ | `GET /api/v1/admin/backups` list, `GET /download`, `DELETE`. Store backups in `{STORAGE_PATH}/backups/`. |
| Admin backup section | ✅ | New "Backup" section under `/admin/backup`. Create (download or server-side), list, download, delete, restore with destructive confirmation. |
| Documentation API | ✅ | `GET /api/v1/docs` lists, `GET /api/v1/docs/:name` returns raw markdown. Authenticated (not admin-only). |
| Docs surface | ✅ | Builtin surface at `/docs/:section`. Sidebar navigation, client-side markdown renderer. 5 new docs: Getting Started, Extension Guide, API Reference, Deployment, Package Format. |
| Tests | ✅ | 6 handler tests (basic backup, ext_data backup, round-trip restore, schema mismatch, dump/restore table, list empty). E2E script `ci/e2e-backup-test.sh`. |
### v0.6.2 — Docs Polish + Dynamic OpenAPI
| Step | Status | Description |
|------|--------|-------------|
| Dark mode fix | ✅ | Added `--bg-code` to CSS variables (dark + light). Replaced all hardcoded light-mode fallbacks in docs CSS with theme-aware variables. Table styling, code blocks, nav items all readable in dark mode. |
| Loading & error handling | ✅ | Replaced borrowed `settings-placeholder` with docs-specific pulse animation. Added error state with retry button for failed doc list fetch. |
| Topbar navigation | ✅ | Imported `Topbar` + `UserMenu` into docs surface. Users can now navigate to other surfaces via the avatar menu. |
| Docs icon + menu entry | ✅ | Added `📖 Docs` entry to UserMenu standard items. Docs accessible from any surface's user menu. |
| `api_schema` manifest field | ✅ | Optional `api_schema` array in `manifest.json`. Parsed lazily by spec builder. Malformed entries logged and skipped — never blocks extension loading. |
| OpenAPI spec builder | ✅ | `BuildOpenAPISpec()` merges static kernel spec with extension routes. Tier 1: auto-generated stubs for all `api_routes`. Tier 2: `api_schema` replaces stubs with rich path items (params, body, response). |
| Dynamic spec endpoint | ✅ | `GET /api/docs/openapi.json` serves merged spec. Swagger UI updated to use JSON endpoint. Static YAML preserved for backward compat. |
| Tests | ✅ | 7 new handler tests: zero extensions, stubs, rich schema, multi-extension, malformed schema, disabled exclusion, required fields. |
---
## v0.6.x — Pre-Fork Hardening
Closes every audit finding before the public fork. No new features — only correctness, dead code elimination, and architectural cleanup. Sequence is fixed: each version is a gate for the next.
### v0.6.3 — Dead Code Sweep + Registry Fix
Pure cleanup. No behavior changes except fixing the broken registry install flow.
| Step | Status | Description |
|------|--------|-------------|
| Fix registry install | ☐ | SDK sends `{ url }`, handler expects `{ download_url }`. Fix `api-domains.js` to send `{ download_url: url }`. Every Install click currently returns 400. |
| Registry settings UI | ☐ | Add "Package Registry" section to admin settings with URL input field. Only way to configure registry today is a raw `PUT /api/v1/admin/settings/package_registry` — no user will find it. |
| Registry tooling + docs | ☐ | `scripts/generate-registry.sh` scans a directory of `.pkg` files and emits registry JSON. `docs/PACKAGE-REGISTRY.md` documents the format. |
| Delete dead kernel Go | ☐ | `store/interfaces.go:178–183` — orphaned ChannelListFilter comments. `pages/pages.go:922–930` — `roleFilterType()` + template registration (chat vestige, maps nonexistent roles). `main.go:67` — orphaned provider-type comment. |
| Delete dead vendor JS | ☐ | `vendor/marked.min.js` and `vendor/purify.min.js` — 62KB, zero production imports. Only referenced in test helpers. |
| Delete `dev.html` | ☐ | 676 lines, not imported by anything. Dead. |
| Remove `dashboard` from default bundle | ☐ | Requires `legacy-sdk` (doesn't exist), auto-installs as dormant on fresh installs. Confusing. Remove from `defaultBundledPackages`. |
| Remove `hello-dashboard` | ☐ | Proof-of-concept from early development. Move to `examples/` or delete. |
| Strip stale version comments | ☐ | 60+ files have `// v0.29.x:`, `// v0.33.x:` pre-fork comments. Single sed pass. |
| Narrow default bundle | ☐ | Default: `notes`, `chat`, `chat-core`, `mermaid-renderer`, `schedules`. Everything else available via registry or `BUNDLED_PACKAGES=*`. |
### v0.6.4 — Admin Health/Metrics Tab + Cluster Merge
Structural move: cluster dashboard becomes an Admin tab. Better home for health/metrics — shared context with other admin panels, no separate nav entry.
| Step | Status | Description |
|------|--------|-------------|
| "Health / Metrics" admin tab | ☐ | New tab in Admin surface. DB-agnostic metrics for all deployments. Cluster cards conditional on PG + multi-node detection. |
| Runtime metrics | ☐ | Per-node: goroutines, heap alloc/sys, stack in use, GC cycles, last GC pause, GC CPU %, uptime, WebSocket clients, extensions loaded, open FDs. |
| DB pool metrics | ☐ | All deployments: DB latency (`SELECT 1` round-trip), pool active/idle/max, wait count, wait duration. PG-only: table bloat (`n_dead_tup`), active backends (`pg_stat_activity`). |
| Cluster metrics | ☐ | PG multi-node only: cluster size, peer list with endpoint + uptime, heartbeat age per node, event bus publish/deliver rates. |
| Extension runtime metrics | ☐ | Starlark exec/min, errors/min, avg duration, HTTP outbound requests/min, trigger fires/min, schedule overruns. |
| Fatten heartbeat payload | ☐ | Heartbeat JSONB carries full metric set. `GET /api/v1/admin/metrics` for single-node SQLite fallback (same shape). |
| Retire `cluster-dashboard` | ☐ | Remove package once Admin Health tab ships. Update `defaultBundledPackages`. |
| Fix block renderer `requires` | ☐ | `mermaid-renderer`, `katex-renderer`, `csv-table`, `diff-viewer` all have `"requires": ["chat"]`. These are content renderers, not chat features. Remove constraint — they should activate without chat. |
| Health endpoint consolidation | ☐ | `/health` and `/api/v1/health` return near-identical JSON. Merge or clearly differentiate with docs. |
### v0.6.5 — Renderer Pipeline + Docs Rewrite
Most complex sub-version. Lifts block rendering to a kernel SDK primitive so all surfaces share it, then rewrites the docs for an external audience.
| Step | Status | Description |
|------|--------|-------------|
| SDK renderer primitive | ☐ | `sw.renderers.register(pattern, handler)` — kernel-level registration. Extensions call once; all surfaces consume. Decouples renderer discovery from chat surface. |
| Notes hooks SDK renderer pipeline | ☐ | Notes `renderMarkdown()` delegates fenced blocks to registered renderers instead of emitting raw `<pre>`. |
| Docs hooks SDK renderer pipeline | ☐ | Docs markdown renderer delegates fenced blocks to registered renderers. |
| Unify markdown renderer | ☐ | Three separate markdown implementations (Notes hand-rolled, Docs hand-rolled, Chat `marked`). Adopt `marked` (already vendored) across all three surfaces. Delete hand-rolled duplicates. |
| Sanitize HTML output | ☐ | DOMPurify is vendored but unused. Wire it as post-render step for user-generated markdown (Notes). Closes XSS surface. |
| Mermaid/KaTeX/CSV/Diff work everywhere | ☐ | With `requires: ["chat"]` removed (v0.6.4) and renderer pipeline lifted, these render in notes + docs + chat without surface-specific code. |
| Docs rewrite for external audience | ☐ | Remove all references to "chat-switchboard", "the fork", "the gut", "Phase 0", "scorched earth". Reframe from "we removed X" to "the kernel provides Y". Remove pre-v0.2.0 version references. |
| Add Mermaid architecture diagrams | ☐ | Diagrams in docs: system architecture overview, request flow, extension lifecycle, realtime event flow, settings cascade, cluster topology. Serves double duty: good docs + proof mermaid-renderer works in docs surface. |
| Surface alias decision | ☐ | `main.go:819–826` has six `/admin/surfaces/*` alias routes. Decision: keep permanently (document) or migrate SDK + ICD runner to `/admin/packages/` and delete. Ship one or the other, not both undocumented. |
| `CONTRIBUTING.md` + tutorial | ☐ | "Build your first extension" guide: manifest → Starlark → ext_data → API route → surface JS → install → test. Use `team-activity-log` as worked example. |
### v0.6.6 — Pre-Fork Hardening
Final pass before the hard fork. Security, correctness, and developer experience.
| Step | Status | Description |
|------|--------|-------------|
| Extension dependency auto-activation | ☐ | Installing a package with unmet `depends`/`requires` should auto-install dependencies from the bundled set, or reject with a clear error listing what's missing. Silent dormant is not acceptable for external users. |
| `ValidateManifest()` gate | ☐ | Single validation function called at install time. Catches malformed manifests early — same pattern as Unicode scan gate. |
| Package signing hook | ☐ | Optional `signature` field in manifest. `--verify` flag on install path (currently no-op). Five minutes now prevents a breaking manifest schema change post-fork. |
| OIDC state nonce validation | ☐ | `handlers/auth.go:221` — security TODO. Validate nonce on OIDC callback before accepting tokens. |
| Schema migration stub decision | ☐ | `handlers/starlark_helpers.go:93–97` — `RunSchemaMigrations` is a no-op stub called during updates. Implement real migrations or remove and document that only additive schema changes are supported. Don't ship a stub that pretends to work. |
| ICD/SDK runner update pass | ☐ | ICD runner and SDK runner are stale for cluster, backup, chat, realtime, dynamic OpenAPI. Update to cover current API surface. |
| Stale TODO resolution | ☐ | `main.go:867` (session middleware TODO, stale since v0.2.0). `auth.go:221` (OIDC nonce, covered above). `starlark_helpers.go:96` (migration stub, covered above). Resolve or delete. |
Then fork.
---
## Post-MVP
@@ -452,7 +151,8 @@ PG is the consensus layer. Zero new infrastructure. `UNLOGGED` table + `LISTEN/N
- Sidecar tier: container-based extensions
- Federation: cross-instance package sharing
- Plugin marketplace with signing and review
- Mermaid diagrams extension (nice-to-have)
---
## Design Decisions Log
@@ -473,4 +173,7 @@ PG is the consensus layer. Zero new infrastructure. `UNLOGGED` table + `LISTEN/N
| Chat as extension, not kernel | Human-to-human chat built entirely as library + surface packages. Zero kernel awareness of conversations, messages, or participants. Kernel gains one generic `realtime` module. Proves near-infinite extensibility. LLM participation layers on top via a separate bridge extension — the chat system doesn't know or care whether a participant is human or AI. |
| PG as consensus layer | Horizontal scaling uses PG as the sole coordinator (UNLOGGED node_registry + LISTEN/NOTIFY). No etcd, Consul, Redis, or Raft. Rationale: system is already tightly coupled to PG; adding a second consensus layer doubles operational complexity for zero benefit at homelab-to-small-team scale. UNLOGGED table is visible to all connections, survives session disconnect, and cleans up automatically on PG crash — correct behavior since all nodes are dead anyway. Sweep-all for health (every node deletes stale rows) is simpler than ring topology and has no edge cases. |
| Two trigger tiers | Event + webhook triggers are extension-declared (manifest contract, full sandbox). Scheduled tasks are user-created ad-hoc (restricted sandbox — no raw HTTP, no DB table creation, connections-only outbound). Separation keeps extension contracts static and user automation safe. |
| Scheduled task identity | Tasks run as their creator (RBAC-scoped). Admin-created tasks can opt into system context. Creator deactivation pauses the schedule. Ensures audit trail and permission boundaries. |
| Scheduled task identity | Tasks run as their creator (RBAC-scoped). Admin-created tasks can opt into system context. Creator deactivation pauses the schedule. Ensures audit trail and permission boundaries. |
| Builtin package rationale | A builtin must enhance kernel surfaces or be required to demonstrate the platform's own capabilities. **notes** — primary content surface, exercises ext_data/storage/SDK/settings/realtime fully. **chat + chat-core** — primary communication surface, proves the extensibility thesis (100% extension, zero kernel awareness). **mermaid-renderer** — docs surface uses Mermaid diagrams to explain the architecture; without it the platform's own documentation doesn't render (self-bootstrapping). **schedules** — UI for the kernel's scheduled task system; without it users can't manage cron jobs (kernel primitive UI). All others are domain features, examples, dormant, or LLM-only tools — available via registry or `BUNDLED_PACKAGES=*`. |
| Cluster dashboard retired | `cluster-dashboard` shipped as a standalone surface package (v0.6.0) as an expedient. Health/metrics belong inside the Admin surface as a tab — shared context, no separate nav entry, no install required. Merged in v0.6.4. |
| Block renderers decoupled from chat | `mermaid-renderer`, `katex-renderer`, `csv-table`, `diff-viewer` shipped with `"requires": ["chat"]` because renderer discovery lived inside the chat surface. These are content renderers, not chat features. v0.6.4 removes the constraint; v0.6.5 lifts renderer registration to the kernel SDK so all surfaces share it without reimplementing discovery. |

132
TURNOVER.md
View File

@@ -1,132 +0,0 @@
# Switchboard Core — Session Turnover
**Date**: 2026-03-26
**Author**: Jeffrey Smith (jasafpro@gmail.com)
**Session**: v0.2.0 PR 1 — Full RBAC migration
---
## What Was Done
### PR 1: Admin → RBAC group migration — COMPLETE
Branch `feat/admin-rbac-migration` (PR #1), 3 commits:
1. **Admin → RBAC group migration** — `surface.admin.access` permission +
Admins system group replaces hardcoded `role == "admin"` checks. Admin
bypass removed from `RequirePermission`. Bootstrap/seed/OIDC/admin handlers
sync group membership.
2. **Remove token budgets + allowed models from groups** — Provider-era cruft:
`token_budget_daily`, `token_budget_monthly`, `allowed_models` columns,
`ResolveTokenBudget()`, `ResolveModelAllowlist()`, and all store/handler/UI
code deleted. -283 lines.
3. **Drop `users.role` column** — Full RBAC. Zero magic roles. The `role` column,
`UserRoleAdmin`/`UserRoleUser` constants, `CountByRole()`, `DefaultRole` config,
`role` in JWT claims, and all role-based shortcuts removed. Everyone group
membership is now explicit (all users added on create). 28 files changed.
**Net result: -364 lines across 3 commits.**
### Architecture after this PR:
- **Everyone group** (`00000000...0001`): all users explicitly added on creation.
Carries `extension.use`, `workflow.submit`.
- **Admins group** (`00000000...0002`): carries all 7 kernel permissions including
`surface.admin.access`. Not special-cased — just a group with permissions.
- **Permission resolution**: union of all group memberships. No implicit groups,
no role shortcuts, no admin bypass.
- **Admin middleware**: checks `surface.admin.access` grant via `resolveAndCachePerms`.
- **JWT claims**: `user_id` + `email` only. No role.
- **OIDC**: `isIdPAdmin()` maps IdP role claim → Admins group membership.
No `role` column writes.
- **User creation paths**: builtin register, OIDC auto-provision, mTLS auto-provision,
admin create, bootstrap, seed — all call `EnsureEveryoneGroup()`.
---
## Known Issues
1. **Frontend test job**: FE test suite may have remaining stale references
from Phase 0 gut (pre-existing)
2. **Traefik middleware**: `rbac-traefik.yaml` needs one-time manual apply
by cluster admin (pre-existing, non-blocking)
3. **`SEED_USERS`**: Not set in K8s secrets (pre-existing)
4. **Editor modules**: `src/editor/*.mjs` still reference "Chat Switchboard"
(pre-existing, low priority)
5. **K8s deploy**: PR not yet merged to main — CI/CD will need a deploy after merge
---
## What's Next
### Merge PR #1
Review and merge `feat/admin-rbac-migration` to main.
### v0.2.0 — Remaining PRs
**PR 2: Settings cascade**
- Add `user_overridable` flag to settings schema
- RBAC controls scope auth (admin → global, team-admin → team, user → personal)
- Resolution: user → team → global (first non-null wins)
**PR 3: ICD (API contract)**
- Generate full OpenAPI spec from registered routes
- Document kernel-only endpoints
**PR 4: Trigger system**
- Time triggers (cron), webhook (inbound HTTP), event (bus subscription)
- Extensions register match expressions at install
**PR 5: SDK stabilization**
- `sb.slots()`, `sb.actions`, `sb.api.ext()`, `sb.storage`
- Theme tokens exposed to extensions
---
## Architecture Decisions (this session)
| Decision | Detail |
|----------|--------|
| Full RBAC — no magic roles | `users.role` column dropped entirely. All authorization through group membership + permission grants. |
| Explicit Everyone membership | No implicit "all users get Everyone perms". Every user added to Everyone group on create. `member_count` reflects reality. |
| Admins group not special-cased | Code never checks group identity — only checks for `surface.admin.access` permission. Any group can grant it. |
| No new migrations pre-MVP | Edit existing SQL files in place. Schema isn't in production. |
| JWT simplified | Claims carry `user_id` + `email` only. Permissions resolved server-side from groups on every request. |
| OIDC role → group sync | `isIdPAdmin()` replaces `resolveRole()`. IdP admin claim maps to Admins group membership, not a DB column. |
| Token budgets/allowed models removed | Provider-era cruft. Belongs in a future provider extension, not the kernel. |
---
## File Locations
| What | Where |
|------|-------|
| Roadmap | `ROADMAP.md` |
| Changelog | `CHANGELOG.md` |
| CI workflow | `.gitea/workflows/ci.yaml` |
| K8s manifests | `k8s/` |
| Docker compose (local dev) | `docker-compose.yml` (SQLite, port 3000) |
| Nginx template | `nginx.conf.template` |
| Migrations | `server/database/migrations/{postgres,sqlite}/` |
| Permission constants | `server/auth/permissions.go` |
| Admin middleware | `server/middleware/admin.go` |
| Group store | `server/store/{postgres,sqlite}/groups.go` |
| Bootstrap/seed | `server/handlers/auth.go` |
| Admin handlers | `server/handlers/admin.go` |
| Groups UI | `src/js/sw/surfaces/admin/groups.js` |
| Users UI | `src/js/sw/surfaces/admin/users.js` |
| SDK API | `src/js/sw/sdk/api-domains.js` |
| Memory (Claude) | `.claude/projects/-config-Projects-core/memory/` |
---
## Gitea Secrets/Vars Needed
**Secrets**: `POSTGRES_USER`, `POSTGRES_USER_PASSWORD`, `POSTGRES_ADMIN_USER`,
`POSTGRES_ADMIN_PASSWORD`, `ENCRYPTION_KEY`
**Vars**: `DOMAIN`, `NAMESPACE`, `S3_BUCKET`, `S3_ENDPOINT`, `STORAGE_BACKEND`,
`STORAGE_CLASS`, `STORAGE_SIZE`
**Not yet configured**: `S3_ACCESS_KEY`, `S3_SECRET_KEY` (using PVC for now),
`SEED_USERS` (needed for initial admin bootstrap)

2
VERSION
View File

@@ -1 +1 @@
0.6.0
0.6.2

139
ci/e2e-backup-test.sh Executable file
View File

@@ -0,0 +1,139 @@
#!/usr/bin/env bash
# e2e-backup-test.sh — Backup/restore E2E test (v0.6.1)
#
# Requires: docker compose running with a single instance.
# Usage:
# docker compose up --build -d
# ./ci/e2e-backup-test.sh
# docker compose down -v
set -euo pipefail
BASE="http://localhost:8080"
PASS=0
FAIL=0
pass() { PASS=$((PASS + 1)); echo " PASS: $1"; }
fail() { FAIL=$((FAIL + 1)); echo " FAIL: $1"; }
# ── Wait for service ─────────────────────────
echo "=== Waiting for service ==="
for i in $(seq 1 30); do
if curl -sf "$BASE/health" > /dev/null 2>&1; then
echo " service ready"
break
fi
sleep 1
if [ "$i" -eq 30 ]; then
echo "FATAL: service did not become healthy"
exit 1
fi
done
# ── Login as admin ───────────────────────────
echo "=== Authenticating ==="
TOKEN=$(curl -sf "$BASE/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"login":"admin","password":"admin"}' | jq -r '.access_token')
if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
echo "FATAL: login failed"
exit 1
fi
echo " admin token acquired"
auth() { echo "Authorization: Bearer $TOKEN"; }
# ── Seed test data ───────────────────────────
echo "=== Seeding test data ==="
# Create a test user
curl -sf "$BASE/api/v1/auth/register" \
-H "Content-Type: application/json" \
-d '{"username":"backup-test-user","email":"bktest@test.com","password":"testpass123"}' > /dev/null
USER_COUNT=$(curl -sf "$BASE/api/v1/admin/users" -H "$(auth)" | jq '.data | length')
echo " users: $USER_COUNT"
# ── Test 1: List backups (empty) ─────────────
echo "=== Test 1: List backups (empty) ==="
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
COUNT=$(echo "$LIST" | jq '.data | length')
if [ "$COUNT" -eq 0 ]; then
pass "empty backup list"
else
fail "expected 0 backups, got $COUNT"
fi
# ── Test 2: Create server-side backup ────────
echo "=== Test 2: Create server-side backup ==="
RESULT=$(curl -sf "$BASE/api/v1/admin/backup?store=true" \
-X POST \
-H "$(auth)")
FILENAME=$(echo "$RESULT" | jq -r '.data.filename')
if [ -n "$FILENAME" ] && [ "$FILENAME" != "null" ]; then
pass "created backup: $FILENAME"
else
fail "backup creation failed: $RESULT"
fi
# ── Test 3: List backups (has one) ───────────
echo "=== Test 3: List backups (has one) ==="
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
COUNT=$(echo "$LIST" | jq '.data | length')
if [ "$COUNT" -eq 1 ]; then
pass "one backup listed"
else
fail "expected 1 backup, got $COUNT"
fi
# ── Test 4: Download backup ──────────────────
echo "=== Test 4: Download backup ==="
TMPFILE=$(mktemp /tmp/backup-test-XXXXXX.swb)
HTTP_CODE=$(curl -sf -o "$TMPFILE" -w "%{http_code}" \
"$BASE/api/v1/admin/backups/$FILENAME" -H "$(auth)")
if [ "$HTTP_CODE" = "200" ] && [ -s "$TMPFILE" ]; then
pass "download OK ($(wc -c < "$TMPFILE") bytes)"
else
fail "download failed (HTTP $HTTP_CODE)"
fi
# ── Test 5: Streaming backup (direct download) ──
echo "=== Test 5: Streaming backup ==="
STREAM_FILE=$(mktemp /tmp/backup-stream-XXXXXX.swb)
HTTP_CODE=$(curl -sf -o "$STREAM_FILE" -w "%{http_code}" \
-X POST "$BASE/api/v1/admin/backup" -H "$(auth)")
if [ "$HTTP_CODE" = "200" ] && [ -s "$STREAM_FILE" ]; then
pass "streaming download OK ($(wc -c < "$STREAM_FILE") bytes)"
else
fail "streaming download failed (HTTP $HTTP_CODE)"
fi
# ── Test 6: Delete backup ────────────────────
echo "=== Test 6: Delete backup ==="
DEL=$(curl -sf -X DELETE "$BASE/api/v1/admin/backups/$FILENAME" -H "$(auth)")
DELETED=$(echo "$DEL" | jq -r '.data.deleted')
if [ "$DELETED" = "$FILENAME" ]; then
pass "deleted backup"
else
fail "delete failed: $DEL"
fi
# Verify deletion
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
COUNT=$(echo "$LIST" | jq '.data | length')
if [ "$COUNT" -eq 0 ]; then
pass "backup list empty after delete"
else
fail "expected 0 backups after delete, got $COUNT"
fi
# ── Cleanup ──────────────────────────────────
rm -f "$TMPFILE" "$STREAM_FILE"
# ── Summary ──────────────────────────────────
echo ""
echo "=== Results: $PASS passed, $FAIL failed ==="
if [ "$FAIL" -gt 0 ]; then
exit 1
fi

186
docs/API-REFERENCE.md Normal file
View File

@@ -0,0 +1,186 @@
# API Reference
Base path: `/api/v1` (all endpoints below are relative to this unless noted).
## Authentication
Most endpoints require a Bearer token in the `Authorization` header:
```
Authorization: Bearer <access_token>
```
Obtain tokens via the auth endpoints (no auth required):
| Method | Path | Description |
|--------|------|-------------|
| POST | `/auth/register` | Create account (if registration enabled) |
| POST | `/auth/login` | Login, returns access + refresh tokens |
| POST | `/auth/refresh` | Refresh access token |
| POST | `/auth/logout` | Invalidate refresh token |
| GET | `/auth/oidc/login` | OIDC login redirect (when `AUTH_MODE=oidc`) |
| GET | `/auth/oidc/callback` | OIDC callback handler |
## Error Format
All errors return JSON:
```json
{"error": "description of what went wrong"}
```
Standard HTTP status codes: 400 (bad request), 401 (unauthorized), 403 (forbidden), 404 (not found), 500 (server error).
## Pagination
List endpoints accept query parameters:
| Param | Default | Description |
|-------|---------|-------------|
| `limit` | 50 | Max items to return |
| `offset` | 0 | Number of items to skip |
## Endpoints by Category
### Profile & Settings
| Method | Path | Description |
|--------|------|-------------|
| GET | `/profile` | Current user profile |
| PUT | `/profile` | Update profile |
| POST | `/profile/password` | Change password |
| GET | `/settings` | User settings |
| PUT | `/settings` | Update user settings |
| GET | `/profile/permissions` | List granted permissions |
| GET | `/profile/bootstrap` | Bootstrap data for frontend |
### Surfaces & Extensions
| Method | Path | Description |
|--------|------|-------------|
| GET | `/surfaces` | List enabled surfaces |
| GET | `/extensions` | List user extensions |
| POST | `/extensions/:id/settings` | Update extension settings |
| GET | `/extensions/:id/manifest` | Get extension manifest |
| GET | `/settings/public` | Public platform settings |
### Notifications
| Method | Path | Description |
|--------|------|-------------|
| GET | `/notifications` | List notifications |
| GET | `/notifications/unread-count` | Unread count |
| PATCH | `/notifications/:id/read` | Mark as read |
| POST | `/notifications/mark-all-read` | Mark all read |
| DELETE | `/notifications/:id` | Delete notification |
| GET | `/notifications/preferences` | Notification preferences |
| PUT | `/notifications/preferences/:type` | Set preference |
### Workflows
| Method | Path | Description |
|--------|------|-------------|
| GET | `/workflows` | List workflows |
| POST | `/workflows` | Create workflow |
| GET | `/workflows/:id` | Get workflow |
| PATCH | `/workflows/:id` | Update workflow |
| DELETE | `/workflows/:id` | Delete workflow |
| POST | `/workflows/:id/publish` | Publish version |
| POST | `/workflows/:id/clone` | Clone workflow |
| POST | `/workflows/:id/instances` | Start instance |
| GET | `/workflows/:id/instances` | List instances |
| GET | `/assignments/mine` | My assignments |
### Teams
| Method | Path | Description |
|--------|------|-------------|
| GET | `/teams/mine` | List my teams |
| GET | `/teams/:id/members` | Team members |
| POST | `/teams/:id/members` | Add member |
| GET | `/teams/:id/roles` | Team roles |
### Connections
| Method | Path | Description |
|--------|------|-------------|
| GET | `/connections` | List connections |
| POST | `/connections` | Create connection |
| GET | `/connections/resolve` | Resolve connection by type |
| PUT | `/connections/:id` | Update connection |
| DELETE | `/connections/:id` | Delete connection |
### Packages (User)
| Method | Path | Description |
|--------|------|-------------|
| GET | `/packages` | List visible packages |
| POST | `/packages/install` | Install personal package |
| DELETE | `/packages/:id` | Delete personal package |
### Admin Endpoints
All under `/api/v1/admin/` -- require `surface.admin.access` permission.
| Method | Path | Description |
|--------|------|-------------|
| GET | `/admin/users` | List users |
| POST | `/admin/users` | Create user |
| GET | `/admin/stats` | Platform statistics |
| GET | `/admin/settings` | Global settings |
| PUT | `/admin/settings/:key` | Update setting |
| GET | `/admin/packages` | List all packages |
| POST | `/admin/packages/install` | Install package (.pkg upload) |
| POST | `/admin/packages/:id/update` | Update package |
| GET | `/admin/packages/:id/export` | Export package as .pkg |
| PUT | `/admin/packages/:id/enable` | Enable package |
| PUT | `/admin/packages/:id/disable` | Disable package |
| DELETE | `/admin/packages/:id` | Delete package |
| GET | `/admin/cluster` | Cluster node list (Postgres only) |
| POST | `/admin/backup` | Create backup |
| GET | `/admin/backups` | List backups |
| POST | `/admin/restore` | Restore from backup |
### Health & Monitoring
These are at the base path (not under `/api/v1`):
| Method | Path | Description |
|--------|------|-------------|
| GET | `/health` | Basic health check |
| GET | `/healthz/live` | Liveness probe (Kubernetes) |
| GET | `/healthz/ready` | Readiness probe (Kubernetes) |
| GET | `/metrics` | Prometheus metrics |
| GET | `/api/docs` | OpenAPI documentation UI (Swagger) |
| GET | `/api/docs/openapi.json` | Dynamic OpenAPI 3.0 spec (includes extension routes) |
| GET | `/api/docs/openapi.yaml` | Static kernel-only OpenAPI spec (YAML) |
### Webhooks (Public)
| Method | Path | Description |
|--------|------|-------------|
| GET/POST | `/api/v1/hooks/:package_id/:slug` | Inbound webhook for extensions |
| POST | `/api/v1/workflows/public/:id/start` | Public workflow entry |
## WebSocket
Connect to `/ws` with a ticket-based auth flow:
1. POST `/api/v1/ws/ticket` with Bearer token to get a one-time ticket.
2. Connect to `/ws?ticket=<ticket>`.
The WebSocket carries JSON-framed events. Subscribe to channels with:
```json
{"type": "subscribe", "channel": "notifications"}
```
Kernel event prefixes: `user.*`, `team.*`, `workflow.*`, `notification.*`, `presence.*`, `extension.*`, `admin.*`, `system.*`.
### Presence
| Method | Path | Description |
|--------|------|-------------|
| POST | `/presence/heartbeat` | Update presence status |
| GET | `/presence` | Query online users |
| GET | `/users/search` | Search users |

153
docs/DEPLOYMENT.md Normal file
View File

@@ -0,0 +1,153 @@
# Deployment Guide
## Docker Single-Instance
```bash
docker pull ghcr.io/switchboard-core/switchboard-core:latest
docker run -p 8080:80 \
-e SWITCHBOARD_ADMIN_USERNAME=admin \
-e SWITCHBOARD_ADMIN_PASSWORD=changeme \
-e JWT_SECRET="$(openssl rand -hex 32)" \
-e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
-v switchboard-data:/data \
ghcr.io/switchboard-core/switchboard-core:latest
```
This runs with SQLite and PVC storage. Suitable for evaluation and small teams.
## Docker Compose with PostgreSQL
```yaml
services:
postgres:
image: postgres:16
environment:
POSTGRES_DB: switchboard
POSTGRES_USER: switchboard
POSTGRES_PASSWORD: secretpassword
volumes:
- pg_data:/var/lib/postgresql/data
switchboard:
image: ghcr.io/switchboard-core/switchboard-core:latest
ports:
- "8080:80"
environment:
DATABASE_URL: "postgres://switchboard:secretpassword@postgres:5432/switchboard?sslmode=disable"
JWT_SECRET: "change-me-in-production"
ENCRYPTION_KEY: "change-me-in-production"
SWITCHBOARD_ADMIN_USERNAME: admin
SWITCHBOARD_ADMIN_PASSWORD: changeme
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
volumes:
- sb_storage:/data/storage
depends_on:
- postgres
volumes:
pg_data:
sb_storage:
```
## Kubernetes
See the `k8s/` directory for example manifests. Key considerations:
- Set `POSTGRES_HOST`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` env vars (assembled into DSN automatically).
- Liveness probe: `/healthz/live`. Readiness probe: `/healthz/ready`.
- Mount a PVC at `/data/storage` or configure S3.
- Store `JWT_SECRET` and `ENCRYPTION_KEY` in Kubernetes Secrets.
- Registry: `registry.gobha.me:5000/xcaliber/switchboard-core`.
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8080` | Backend API port |
| `DB_DRIVER` | auto | `postgres` or `sqlite` |
| `DATABASE_URL` | | PostgreSQL DSN or SQLite path |
| `JWT_SECRET` | `dev-secret-change-me` | Token signing key -- **must change** |
| `ENCRYPTION_KEY` | | AES-256 key for credential vault |
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, `oidc` |
| `STORAGE_BACKEND` | auto | `pvc` or `s3` |
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
| `BASE_PATH` | | URL prefix (e.g., `/switchboard`) |
| `LOG_FORMAT` | `text` | `text` or `json` |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
| `CORS_ALLOWED_ORIGINS` | | Comma-separated allowed origins |
| `BUNDLED_PACKAGES` | (empty) | `""` defaults, `"*"` all, or comma-separated |
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable bundled package install |
| `BUNDLED_PACKAGES_DIR` | `/app/bundled-packages` | Custom bundle directory |
| `SWITCHBOARD_ADMIN_USERNAME` | | Bootstrap admin username |
| `SWITCHBOARD_ADMIN_PASSWORD` | | Bootstrap admin password |
| `SEED_USERS` | | Dev seed users (ignored in production) |
### S3 Storage Variables
| Variable | Description |
|----------|-------------|
| `S3_BUCKET` | Bucket name |
| `S3_ENDPOINT` | Endpoint URL (for MinIO, Ceph, etc.) |
| `S3_ACCESS_KEY` | Access key |
| `S3_SECRET_KEY` | Secret key |
| `S3_FORCE_PATH_STYLE` | `true` for MinIO/Ceph |
### Cluster Variables (Multi-Replica)
| Variable | Default | Description |
|----------|---------|-------------|
| `CLUSTER_NODE_ID` | hostname-PID | Override for deterministic node identity |
| `CLUSTER_HEARTBEAT_INTERVAL` | `10s` | Heartbeat tick frequency |
| `CLUSTER_STALE_THRESHOLD` | `30s` | 3x heartbeat = stale node |
| `CLUSTER_ENDPOINT` | auto-detect | Advertised address for peer mesh |
## Database Configuration
**PostgreSQL** (recommended for production):
```bash
DATABASE_URL="postgres://user:pass@host:5432/switchboard?sslmode=require"
```
**SQLite** (dev, test, edge deployments):
```bash
DB_DRIVER=sqlite
DATABASE_URL=/data/switchboard.db
```
Both databases are first-class -- every query compiles and passes tests on both. The driver is auto-detected from `DATABASE_URL` if `DB_DRIVER` is not set.
## Cluster Mode
Multi-replica HA requires PostgreSQL. The kernel uses PG-backed WebSocket tickets and rate limit counters to coordinate across replicas. A cluster registry with heartbeat and stale sweep tracks active nodes.
View cluster status in Admin > Cluster or via `GET /api/v1/admin/cluster`.
## Storage Backends
**PVC**: Auto-detected if the storage path is writable. Mount a volume at `/data/storage`.
**S3-compatible**: Set `STORAGE_BACKEND=s3` and provide S3 variables. Works with AWS S3, MinIO, and Ceph.
## Backup and Restore
Available via Admin UI or API:
```
POST /api/v1/admin/backup # Create backup
GET /api/v1/admin/backups # List backups
GET /api/v1/admin/backups/:name # Download backup
POST /api/v1/admin/restore # Restore from backup
DELETE /api/v1/admin/backups/:name # Delete backup
```
## Monitoring
| Endpoint | Purpose |
|----------|---------|
| `/health` | Basic health check |
| `/healthz/live` | Kubernetes liveness probe |
| `/healthz/ready` | Kubernetes readiness probe |
| `/metrics` | Prometheus metrics |

183
docs/EXTENSION-GUIDE.md Normal file
View File

@@ -0,0 +1,183 @@
# Extension Guide
## Package Types
| Type | Description |
|------|-------------|
| `surface` | A routable UI page rendered in the shell viewport |
| `extension` | Starlark hooks, tools, API routes, DB tables |
| `full` | Both surface and extension combined |
| `library` | Shared code imported by other packages via `lib.require()` |
| `workflow` | Bundled workflow definition |
## Tiers
| Tier | Runs | Capabilities |
|------|------|-------------|
| `browser` | Client-side JS only | DOM access, SDK hooks, no server-side logic |
| `starlark` | Sandboxed server-side | DB, HTTP, notifications, secrets, API routes, realtime |
| `sidecar` | Separate container | Full runtime (future) |
## manifest.json
Every package has a `manifest.json` at its root. Example for a surface:
```json
{
"id": "my-surface",
"title": "My Surface",
"type": "full",
"tier": "starlark",
"version": "1.0.0",
"description": "A custom surface with server-side logic.",
"icon": "🔧",
"author": "you",
"route": "/s/my-surface",
"auth": "authenticated",
"permissions": ["db.write"],
"api_routes": [
{"method": "GET", "path": "/items"},
{"method": "POST", "path": "/items"}
],
"db_tables": {
"items": {
"columns": {
"title": "text",
"done": "int"
},
"indexes": [["title"]]
}
},
"settings": {
"page_size": {
"type": "string",
"label": "Items Per Page",
"description": "Number of items shown per page",
"default": "25"
}
}
}
```
### Field Reference
| Field | Required | Description |
|-------|----------|-------------|
| `id` | yes | Unique kebab-case identifier |
| `title` | yes | Display name |
| `type` | yes | `surface`, `extension`, `full`, `library`, `workflow` |
| `tier` | yes | `browser`, `starlark`, `sidecar` |
| `version` | yes | Semver string |
| `description` | no | Short description |
| `icon` | no | Emoji icon for sidebar/menu |
| `route` | surfaces | URL path (e.g., `/s/my-surface`) |
| `auth` | no | `authenticated` (default) or `public` |
| `permissions` | no | Capabilities requested: `db.write`, `http`, `notifications`, `secrets`, `realtime.publish` |
| `api_routes` | no | Array of `{method, path}` for extension HTTP endpoints |
| `api_schema` | no | OpenAPI documentation for extension API routes (see below) |
| `db_tables` | no | Table definitions (see below) |
| `settings` | no | User-configurable settings schema |
| `exports` | libraries | Functions exported for other packages |
| `hooks` | no | Event bus subscriptions |
| `schema_version` | no | Integer for additive schema migrations |
## db_tables Schema
Tables are automatically namespaced as `ext_{package_id}_{table_name}`. Column types: `text`, `int`. Every table gets an auto-generated `id` primary key and `created_at` timestamp.
```json
"db_tables": {
"notes": {
"columns": {
"title": "text",
"body": "text",
"creator_id": "text",
"pinned": "int"
},
"indexes": [
["creator_id"],
["pinned"]
]
}
}
```
## api_schema (OpenAPI Documentation)
Extensions can optionally declare an `api_schema` array in their manifest to provide rich API documentation. Declared routes appear in Swagger UI at `/api/docs` with parameters, request bodies, and response schemas. Routes without `api_schema` entries still appear as auto-generated stubs.
```json
"api_schema": [
{
"path": "/items",
"method": "GET",
"summary": "List items",
"description": "Returns paginated items for the current user",
"params": {
"limit": {"type": "integer", "default": 50, "description": "Max results"},
"offset": {"type": "integer", "default": 0}
},
"response": {
"type": "object",
"example": {"data": [{"id": "string", "title": "string"}]}
}
},
{
"path": "/items",
"method": "POST",
"summary": "Create item",
"body": {
"title": {"type": "string", "required": true},
"description": {"type": "string"}
}
}
]
```
Only `path` and `method` are required. All other fields are optional. Malformed entries are logged and skipped without blocking extension loading.
## Starlark Sandbox API
Starlark scripts run server-side with a CPU budget and memory ceiling. Available modules (granted per-permission by admin):
| Module | Permission | API |
|--------|-----------|-----|
| `db` | `db.write` | `db.query(table, filters)`, `db.insert(table, row)`, `db.update(table, id, row)`, `db.delete(table, id)` |
| `http` | `http` | `http.get(url)`, `http.post(url, body)` -- SSRF-safe, no private IPs by default |
| `notifications` | `notifications` | `notifications.send(user_id, title, body)` |
| `secrets` | `secrets` | `secrets.get(connection_type)` -- reads from the credential vault |
| `api` | (implicit) | Registers HTTP routes at `/s/:slug/api/*path` |
| `realtime` | `realtime.publish` | `realtime.publish(channel, event, data)` -- push to WebSocket clients |
The sandbox cannot spawn goroutines, access the filesystem, or import arbitrary packages.
## Permissions Model
Extensions declare required permissions in `manifest.json`. The admin must grant each permission before the extension can use the corresponding module. Permission status is visible in Admin > Packages > Permissions.
Kernel permissions for users/groups: `extension.use`, `extension.install`, `workflow.create`, `workflow.submit`, `admin.view`, `token.unlimited`.
## File Structure
```
my-package/
manifest.json # required
js/ # browser-side JavaScript
index.js
css/ # stylesheets
styles.css
script.star # Starlark entry point
star/ # additional Starlark modules
assets/ # static assets
migrations/ # schema migration files
```
## Testing Extensions
1. Build the package: `cd packages && bash build.sh my-package`
2. Upload the `.pkg` file via Admin > Packages > Install.
3. Grant permissions in Admin > Packages > Permissions.
4. Enable the package.
5. If it is a surface, navigate to its route (e.g., `/s/my-package`).
Extension API routes are accessible at `/s/{slug}/api/{path}` and require an authenticated Bearer token.

91
docs/GETTING-STARTED.md Normal file
View File

@@ -0,0 +1,91 @@
# Getting Started
## Prerequisites
- Docker and Docker Compose
## Running with Docker Compose
```bash
docker compose up --build
```
Open [http://localhost:3000](http://localhost:3000). Default credentials: `admin` / `admin`.
Data persists in the `sb_data` named volume. To reset everything:
```bash
docker compose down -v
```
## First Boot
On first start, Switchboard Core will:
1. Run database migrations (SQLite by default in compose).
2. Create the admin user from `SWITCHBOARD_ADMIN_USERNAME` / `SWITCHBOARD_ADMIN_PASSWORD` env vars.
3. Auto-install the curated default package set (notes, chat-core, dashboard, workflow demos, etc.).
No manual setup steps are required.
## Installing Additional Packages
The Docker image ships with extra packages beyond the default set. To install them:
- **Via Admin UI**: Go to `/admin` > Packages. Browse, enable, or install packages.
- **Via environment variable**: Set `BUNDLED_PACKAGES` before starting:
```bash
# Install all bundled packages
BUNDLED_PACKAGES="*" docker compose up --build
# Install specific extras
BUNDLED_PACKAGES="notes,tasks,schedules" docker compose up --build
```
You can also upload `.pkg` archives through the Admin > Packages page.
## Key URLs
| URL | Purpose |
|-----|---------|
| `/` | Redirects to your default surface |
| `/admin` | Admin panel (users, packages, settings, teams) |
| `/settings` | User settings (profile, preferences, default surface) |
| `/welcome` | Welcome page (shown when no surfaces are installed) |
| `/api/docs` | Interactive OpenAPI documentation |
## Key Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8080` | Backend API port |
| `DB_DRIVER` | auto | `postgres` or `sqlite` |
| `DATABASE_URL` | | PostgreSQL DSN or SQLite file path |
| `JWT_SECRET` | `dev-secret-change-me` | **Change in production** |
| `ENCRYPTION_KEY` | | AES-256 key for credential encryption |
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, or `oidc` |
| `STORAGE_BACKEND` | auto | `pvc` or `s3` |
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
| `SWITCHBOARD_ADMIN_USERNAME` | | Bootstrap admin username |
| `SWITCHBOARD_ADMIN_PASSWORD` | | Bootstrap admin password |
| `BUNDLED_PACKAGES` | (empty) | `""` = curated defaults, `"*"` = all, or comma-separated IDs |
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable auto-install entirely |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
| `LOG_FORMAT` | `text` | `text` or `json` |
## From Source
```bash
git clone <repo-url> && cd switchboard-core
cp server/.env.example server/.env # edit DB credentials
cd server && go run .
# Backend on http://localhost:8080
```
## Next Steps
- [Extension Guide](EXTENSION-GUIDE.md) -- Author your own packages
- [API Reference](API-REFERENCE.md) -- REST API overview
- [Deployment](DEPLOYMENT.md) -- Production deployment
- [Architecture](ARCHITECTURE.md) -- System design

130
docs/PACKAGE-FORMAT.md Normal file
View File

@@ -0,0 +1,130 @@
# Package Format
Switchboard packages are distributed as `.pkg` files -- ZIP archives with a standard internal structure.
## ZIP Structure
```
my-package.pkg
├── manifest.json # required -- package metadata
├── js/ # browser-side JavaScript
│ └── index.js
├── css/ # stylesheets
│ └── styles.css
├── script.star # Starlark entry point
├── star/ # additional Starlark modules
├── assets/ # static assets (images, etc.)
└── migrations/ # schema migration files
```
Only `manifest.json` is required. All other directories are optional and included only if present in the source.
## manifest.json Reference
```json
{
"id": "my-package",
"title": "My Package",
"type": "surface",
"tier": "browser",
"version": "1.0.0",
"description": "What this package does.",
"icon": "📦",
"author": "your-name",
"route": "/s/my-package",
"auth": "authenticated",
"permissions": [],
"api_routes": [],
"db_tables": {},
"settings": {},
"hooks": {},
"exports": [],
"schema_version": 1
}
```
### Required Fields
| Field | Description |
|-------|-------------|
| `id` | Unique kebab-case identifier |
| `title` | Human-readable display name |
| `type` | `surface`, `extension`, `full`, `library`, `workflow` |
| `tier` | `browser`, `starlark`, `sidecar` |
| `version` | Semver version string |
### Optional Fields
| Field | Description |
|-------|-------------|
| `description` | Short description |
| `icon` | Emoji for sidebar/menu display |
| `author` | Package author |
| `route` | URL path for surfaces (e.g., `/s/my-package`) |
| `auth` | `authenticated` or `public` |
| `layout` | Surface layout mode (e.g., `single`) |
| `permissions` | Array of required capabilities (`db.write`, `http`, `notifications`, `secrets`, `realtime.publish`) |
| `api_routes` | Array of `{"method": "GET", "path": "/items"}` |
| `db_tables` | Table definitions with columns and indexes |
| `settings` | User-configurable settings with type, label, description, default |
| `hooks` | Event bus subscription patterns |
| `exports` | Functions exported by library packages |
| `schema_version` | Integer for additive schema migrations |
## Package Lifecycle
1. **Install**: Upload a `.pkg` file via Admin > Packages or `POST /api/v1/admin/packages/install`. The kernel extracts the archive, creates database tables, and registers routes.
2. **Enable**: Activate the package so its routes, hooks, and UI become available. `PUT /api/v1/admin/packages/:id/enable`.
3. **Disable**: Deactivate without removing data. `PUT /api/v1/admin/packages/:id/disable`.
4. **Update**: Upload a new `.pkg` with a higher semver version. Schema changes must be additive (new columns/tables only). `POST /api/v1/admin/packages/:id/update`.
5. **Export**: Download the installed package as a `.pkg` archive. `GET /api/v1/admin/packages/:id/export`.
6. **Delete**: Remove the package and its data. `DELETE /api/v1/admin/packages/:id`.
## Bundled vs User-Installed
**Bundled packages** ship inside the Docker image at `/app/bundled-packages`. On first boot, a curated default set is auto-installed. Behavior:
- First boot: curated defaults are installed and enabled.
- Subsequent boots: already-installed packages are skipped.
- Admin uninstalls: the package stays uninstalled (never force-reinstalled).
- Control via `BUNDLED_PACKAGES` env var: empty = defaults, `"*"` = all, or comma-separated IDs.
**User-installed packages** are uploaded through the Admin UI or API. They follow the same lifecycle but are not tied to the Docker image.
## Building Packages
Packages are built by zipping the standard directories alongside `manifest.json`:
```bash
# Build all packages in the packages/ directory
cd packages && bash build.sh
# Build a single package
cd packages && bash build.sh my-package
```
Output goes to `dist/my-package.pkg`.
### Manual Build
```bash
cd packages/my-package
zip -r ../../dist/my-package.pkg manifest.json js/ css/ script.star
```
The build script automatically includes whichever standard directories exist: `js/`, `css/`, `assets/`, `script.star`, `star/`, `migrations/`.
### Custom Docker Image
To bundle custom packages into a Docker image:
1. Place your package in `packages/your-package/` with a `manifest.json`.
2. Run `cd packages && bash build.sh all`.
3. Build the image: `docker build -t my-switchboard .`
The Dockerfile builds all packages and copies them into the bundled packages directory.

1
nginx.conf.template
View File

@@ -66,6 +66,7 @@ server {
location ${BASE_PATH}/team-admin { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
location ${BASE_PATH}/settings { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
location ${BASE_PATH}/welcome { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
location ${BASE_PATH}/docs { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
location ${BASE_PATH}/w/ { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
# Extension surface page routes → backend

523
server/handlers/backup.go Normal file
View File

@@ -0,0 +1,523 @@
package handlers
import (
"archive/zip"
"bytes"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"os"
"path/filepath"
"regexp"
"strings"
"time"
"github.com/gin-gonic/gin"
"switchboard-core/database"
"switchboard-core/store"
)
// BackupHandler provides backup/restore endpoints for admin users.
type BackupHandler struct {
stores store.Stores
packagesDir string
storagePath string
}
// NewBackupHandler creates a new BackupHandler.
func NewBackupHandler(stores store.Stores, packagesDir, storagePath string) *BackupHandler {
return &BackupHandler{
stores: stores,
packagesDir: packagesDir,
storagePath: storagePath,
}
}
// backupMeta is written as meta.json inside the .swb archive.
type backupMeta struct {
Version string `json:"version"`
Timestamp time.Time `json:"timestamp"`
SchemaVersion string `json:"schema_version"`
Dialect string `json:"dialect"`
CoreTables []string `json:"core_tables"`
ExtTables []string `json:"ext_tables,omitempty"`
}
func (h *BackupHandler) backupsDir() string {
if h.storagePath == "" {
return ""
}
return filepath.Join(h.storagePath, "backups")
}
// ── CreateBackup ─────────────────────────────
// POST /api/v1/admin/backup
// Streams a .swb ZIP archive containing all core + ext_data tables as JSONL,
// plus package asset files from disk.
func (h *BackupHandler) CreateBackup(c *gin.Context) {
ctx := c.Request.Context()
db := database.DB
// Determine schema version
schemaVersion := ""
row := db.QueryRowContext(ctx, "SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 1")
row.Scan(&schemaVersion)
dialect := "sqlite"
if database.IsPostgres() {
dialect = "postgres"
}
// Enumerate ext_data tables
extEntries, err := listExtDataTables(ctx, db)
if err != nil {
log.Printf("[backup] warning: list ext_data tables: %v", err)
}
extTableNames := make([]string, len(extEntries))
for i, e := range extEntries {
extTableNames[i] = e.PhysicalName
}
meta := backupMeta{
Version: readVersionFile(),
Timestamp: time.Now().UTC(),
SchemaVersion: schemaVersion,
Dialect: dialect,
CoreTables: coreTableOrder,
ExtTables: extTableNames,
}
// Determine if we should store server-side or stream directly
storeSide := c.Query("store") == "true"
if storeSide {
h.createStoredBackup(c, ctx, db, meta, extEntries)
return
}
// Stream directly to client
filename := fmt.Sprintf("switchboard-%s-%s.swb", meta.Version, meta.Timestamp.Format("20060102-150405"))
c.Header("Content-Type", "application/zip")
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=%q", filename))
zw := zip.NewWriter(c.Writer)
defer zw.Close()
h.writeBackupContents(c, zw, meta, extEntries)
}
func (h *BackupHandler) createStoredBackup(c *gin.Context, ctx interface{ Deadline() (time.Time, bool) }, db interface{}, meta backupMeta, extEntries []extDataTableEntry) {
dir := h.backupsDir()
if dir == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "no storage path configured"})
return
}
os.MkdirAll(dir, 0755)
filename := fmt.Sprintf("switchboard-%s-%s.swb", meta.Version, meta.Timestamp.Format("20060102-150405"))
path := filepath.Join(dir, filename)
f, err := os.Create(path)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "create backup file: " + err.Error()})
return
}
defer f.Close()
zw := zip.NewWriter(f)
defer zw.Close()
h.writeBackupContents(c, zw, meta, extEntries)
c.JSON(http.StatusOK, gin.H{"data": gin.H{
"filename": filename,
"path": path,
}})
}
func (h *BackupHandler) writeBackupContents(c *gin.Context, zw *zip.Writer, meta backupMeta, extEntries []extDataTableEntry) {
ctx := c.Request.Context()
db := database.DB
// Write meta.json
mw, err := zw.Create("meta.json")
if err != nil {
log.Printf("[backup] create meta.json: %v", err)
return
}
enc := json.NewEncoder(mw)
enc.SetIndent("", " ")
enc.Encode(meta)
// Dump core tables
for _, table := range coreTableOrder {
if !tableExists(ctx, db, table) {
continue
}
fw, err := zw.Create(fmt.Sprintf("core/%s.jsonl", table))
if err != nil {
log.Printf("[backup] create core/%s.jsonl: %v", table, err)
continue
}
count, err := dumpTable(ctx, db, table, fw)
if err != nil {
log.Printf("[backup] dump %s: %v", table, err)
} else {
log.Printf("[backup] dumped %s: %d rows", table, count)
}
}
// Dump ext_data tables
for _, entry := range extEntries {
if !tableExists(ctx, db, entry.PhysicalName) {
continue
}
path := fmt.Sprintf("ext_data/%s/%s.jsonl", entry.PackageID, entry.TableName)
fw, err := zw.Create(path)
if err != nil {
log.Printf("[backup] create %s: %v", path, err)
continue
}
count, err := dumpTable(ctx, db, entry.PhysicalName, fw)
if err != nil {
log.Printf("[backup] dump %s: %v", entry.PhysicalName, err)
} else {
log.Printf("[backup] dumped %s: %d rows", entry.PhysicalName, count)
}
}
// Walk package assets
if h.packagesDir != "" {
filepath.Walk(h.packagesDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
relPath, err := filepath.Rel(h.packagesDir, path)
if err != nil {
return nil
}
relPath = filepath.ToSlash(relPath)
fw, err := zw.Create("packages/" + relPath)
if err != nil {
return nil
}
f, err := os.Open(path)
if err != nil {
return nil
}
defer f.Close()
io.Copy(fw, f)
return nil
})
}
}
// ── ListBackups ──────────────────────────────
// GET /api/v1/admin/backups
func (h *BackupHandler) ListBackups(c *gin.Context) {
dir := h.backupsDir()
if dir == "" {
c.JSON(http.StatusOK, gin.H{"data": []interface{}{}})
return
}
entries, err := os.ReadDir(dir)
if err != nil {
c.JSON(http.StatusOK, gin.H{"data": []interface{}{}})
return
}
type backupEntry struct {
Name string `json:"name"`
Size int64 `json:"size"`
CreatedAt time.Time `json:"created_at"`
}
var backups []backupEntry
for _, e := range entries {
if e.IsDir() || !strings.HasSuffix(e.Name(), ".swb") {
continue
}
info, err := e.Info()
if err != nil {
continue
}
backups = append(backups, backupEntry{
Name: e.Name(),
Size: info.Size(),
CreatedAt: info.ModTime(),
})
}
if backups == nil {
backups = []backupEntry{}
}
c.JSON(http.StatusOK, gin.H{"data": backups})
}
// ── DownloadBackup ───────────────────────────
// GET /api/v1/admin/backups/:name
func (h *BackupHandler) DownloadBackup(c *gin.Context) {
name := c.Param("name")
if !isValidBackupName(name) {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid backup name"})
return
}
dir := h.backupsDir()
if dir == "" {
c.JSON(http.StatusNotFound, gin.H{"error": "no storage configured"})
return
}
path := filepath.Join(dir, name)
if _, err := os.Stat(path); os.IsNotExist(err) {
c.JSON(http.StatusNotFound, gin.H{"error": "backup not found"})
return
}
c.Header("Content-Type", "application/zip")
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=%q", name))
c.File(path)
}
// ── DeleteBackup ─────────────────────────────
// DELETE /api/v1/admin/backups/:name
func (h *BackupHandler) DeleteBackup(c *gin.Context) {
name := c.Param("name")
if !isValidBackupName(name) {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid backup name"})
return
}
dir := h.backupsDir()
if dir == "" {
c.JSON(http.StatusNotFound, gin.H{"error": "no storage configured"})
return
}
path := filepath.Join(dir, name)
if err := os.Remove(path); err != nil {
if os.IsNotExist(err) {
c.JSON(http.StatusNotFound, gin.H{"error": "backup not found"})
} else {
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
}
return
}
c.JSON(http.StatusOK, gin.H{"data": gin.H{"deleted": name}})
}
// ── RestoreBackup ────────────────────────────
// POST /api/v1/admin/restore
// Accepts a .swb file upload, validates meta, wipes DB, restores data.
func (h *BackupHandler) RestoreBackup(c *gin.Context) {
ctx := c.Request.Context()
db := database.DB
file, header, err := c.Request.FormFile("file")
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "file required"})
return
}
defer file.Close()
// Read entire file into memory for zip.NewReader
data, err := io.ReadAll(file)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "read file: " + err.Error()})
return
}
zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid archive: " + err.Error()})
return
}
// Read meta.json
meta, err := readBackupMeta(zr)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid backup: " + err.Error()})
return
}
// Validate schema version — reject backups from future schema
currentSchema := ""
db.QueryRowContext(ctx, "SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 1").Scan(&currentSchema)
if meta.SchemaVersion > currentSchema && currentSchema != "" {
c.JSON(http.StatusBadRequest, gin.H{
"error": fmt.Sprintf("backup schema %s is newer than current %s — upgrade the application first", meta.SchemaVersion, currentSchema),
})
return
}
log.Printf("[backup] restoring from %s (schema %s, %d core tables, %d ext tables)",
header.Filename, meta.SchemaVersion, len(meta.CoreTables), len(meta.ExtTables))
// Build full table list for wipe (core + ext_data)
allTables := make([]string, 0, len(coreTableOrder)+len(meta.ExtTables))
allTables = append(allTables, coreTableOrder...)
allTables = append(allTables, meta.ExtTables...)
// Wipe all data
if err := wipeTables(ctx, db, allTables); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "wipe failed: " + err.Error()})
return
}
// Restore core tables in order
restored := 0
for _, table := range coreTableOrder {
path := fmt.Sprintf("core/%s.jsonl", table)
f := findInZip(zr, path)
if f == nil {
continue
}
rc, err := f.Open()
if err != nil {
log.Printf("[backup] open %s: %v", path, err)
continue
}
count, err := restoreTable(ctx, db, table, rc)
rc.Close()
if err != nil {
log.Printf("[backup] restore %s: %v", table, err)
c.JSON(http.StatusInternalServerError, gin.H{"error": fmt.Sprintf("restore %s: %v", table, err)})
return
}
log.Printf("[backup] restored %s: %d rows", table, count)
restored += count
}
// Restore ext_data tables
for _, extTable := range meta.ExtTables {
// Parse package_id and table_name from physical name: ext_{pkg}_{table}
pkgID, tblName := parseExtTableName(extTable)
if pkgID == "" {
continue
}
path := fmt.Sprintf("ext_data/%s/%s.jsonl", pkgID, tblName)
f := findInZip(zr, path)
if f == nil {
continue
}
// Ensure physical table exists before restoring
if !tableExists(ctx, db, extTable) {
log.Printf("[backup] skip ext table %s (not in schema)", extTable)
continue
}
rc, err := f.Open()
if err != nil {
log.Printf("[backup] open %s: %v", path, err)
continue
}
count, err := restoreTable(ctx, db, extTable, rc)
rc.Close()
if err != nil {
log.Printf("[backup] restore ext %s: %v", extTable, err)
} else {
log.Printf("[backup] restored ext %s: %d rows", extTable, count)
restored += count
}
}
// Restore package files
if h.packagesDir != "" {
for _, f := range zr.File {
if !strings.HasPrefix(f.Name, "packages/") || f.FileInfo().IsDir() {
continue
}
relPath := strings.TrimPrefix(f.Name, "packages/")
destPath := filepath.Join(h.packagesDir, filepath.FromSlash(relPath))
os.MkdirAll(filepath.Dir(destPath), 0755)
rc, err := f.Open()
if err != nil {
continue
}
out, err := os.Create(destPath)
if err != nil {
rc.Close()
continue
}
io.Copy(out, rc)
out.Close()
rc.Close()
}
}
log.Printf("[backup] restore complete: %d total rows", restored)
c.JSON(http.StatusOK, gin.H{"data": gin.H{
"rows_restored": restored,
"schema": meta.SchemaVersion,
}})
}
// ── Helpers ──────────────────────────────────
func readBackupMeta(zr *zip.Reader) (*backupMeta, error) {
f := findInZip(zr, "meta.json")
if f == nil {
return nil, fmt.Errorf("meta.json not found in archive")
}
rc, err := f.Open()
if err != nil {
return nil, err
}
defer rc.Close()
var meta backupMeta
if err := json.NewDecoder(rc).Decode(&meta); err != nil {
return nil, fmt.Errorf("decode meta.json: %w", err)
}
return &meta, nil
}
func findInZip(zr *zip.Reader, name string) *zip.File {
for _, f := range zr.File {
if f.Name == name {
return f
}
}
return nil
}
// parseExtTableName parses "ext_{pkg}_{table}" into (pkg, table).
func parseExtTableName(physical string) (string, string) {
if !strings.HasPrefix(physical, "ext_") {
return "", ""
}
rest := physical[4:]
idx := strings.Index(rest, "_")
if idx < 0 {
return "", ""
}
return rest[:idx], rest[idx+1:]
}
var validBackupNameRe = regexp.MustCompile(`^[a-zA-Z0-9._-]+\.swb$`)
func isValidBackupName(name string) bool {
return validBackupNameRe.MatchString(name) && !strings.Contains(name, "..")
}
// readVersionFile reads the VERSION file from the project root.
func readVersionFile() string {
// Try common locations
for _, path := range []string{"VERSION", "/app/VERSION", "../VERSION"} {
data, err := os.ReadFile(path)
if err == nil {
return strings.TrimSpace(string(data))
}
}
return "unknown"
}

231
server/handlers/backup_tables.go Normal file
View File

@@ -0,0 +1,231 @@
package handlers
import (
"bufio"
"context"
"database/sql"
"encoding/json"
"fmt"
"io"
"log"
"strings"
"switchboard-core/database"
)
// coreTableOrder lists tables for backup in FK-safe import order.
// Ephemeral tables (node_registry, ws_tickets, rate_limit_counters,
// user_presence, oidc_auth_state, refresh_tokens) are excluded.
var coreTableOrder = []string{
"schema_migrations",
"users",
"teams",
"team_members",
"groups",
"group_members",
"platform_policies",
"global_settings",
"packages",
"package_user_settings",
"package_team_settings",
"extension_permissions",
"ext_data_tables",
"ext_connections",
"ext_dependencies",
"resource_grants",
"notifications",
"notification_preferences",
"audit_log",
"workflows",
"workflow_stages",
"workflow_versions",
"workflow_instances",
"workflow_assignments",
"workflow_signoffs",
"triggers",
"scheduled_tasks",
"trigger_logs",
}
// dumpTable writes all rows of a table as newline-delimited JSON to w.
// Columns are discovered dynamically via rows.ColumnTypes().
func dumpTable(ctx context.Context, db *sql.DB, table string, w io.Writer) (int, error) {
rows, err := db.QueryContext(ctx, fmt.Sprintf("SELECT * FROM %s", table))
if err != nil {
return 0, fmt.Errorf("query %s: %w", table, err)
}
defer rows.Close()
cols, err := rows.Columns()
if err != nil {
return 0, fmt.Errorf("columns %s: %w", table, err)
}
count := 0
enc := json.NewEncoder(w)
enc.SetEscapeHTML(false)
for rows.Next() {
// Create scan destinations — all as interface{}
vals := make([]interface{}, len(cols))
ptrs := make([]interface{}, len(cols))
for i := range vals {
ptrs[i] = &vals[i]
}
if err := rows.Scan(ptrs...); err != nil {
return count, fmt.Errorf("scan %s row %d: %w", table, count, err)
}
row := make(map[string]interface{}, len(cols))
for i, col := range cols {
row[col] = normalizeValue(vals[i])
}
if err := enc.Encode(row); err != nil {
return count, fmt.Errorf("encode %s row %d: %w", table, count, err)
}
count++
}
return count, rows.Err()
}
// normalizeValue converts database-specific types to JSON-friendly values.
func normalizeValue(v interface{}) interface{} {
if v == nil {
return nil
}
switch val := v.(type) {
case []byte:
// Try to parse as JSON first (for JSONB columns)
var j interface{}
if err := json.Unmarshal(val, &j); err == nil {
return j
}
return string(val)
default:
return val
}
}
// restoreTable reads JSONL from r and inserts rows into the table.
// Column names come from the first JSON object. Uses dialect-adapted SQL.
func restoreTable(ctx context.Context, db *sql.DB, table string, r io.Reader) (int, error) {
scanner := bufio.NewScanner(r)
// Allow large lines (e.g. JSONB audit_log entries)
scanner.Buffer(make([]byte, 0, 64*1024), 10*1024*1024)
count := 0
for scanner.Scan() {
line := scanner.Bytes()
if len(line) == 0 {
continue
}
var row map[string]interface{}
if err := json.Unmarshal(line, &row); err != nil {
return count, fmt.Errorf("unmarshal %s line %d: %w", table, count+1, err)
}
cols := make([]string, 0, len(row))
vals := make([]interface{}, 0, len(row))
for k, v := range row {
cols = append(cols, k)
vals = append(vals, marshalIfComplex(v))
}
// Build INSERT with positional placeholders
placeholders := make([]string, len(cols))
for i := range placeholders {
placeholders[i] = fmt.Sprintf("$%d", i+1)
}
query := fmt.Sprintf("INSERT INTO %s (%s) VALUES (%s)",
table,
strings.Join(cols, ", "),
strings.Join(placeholders, ", "),
)
if _, err := database.ExecContext(ctx, query, vals...); err != nil {
return count, fmt.Errorf("insert %s row %d: %w", table, count+1, err)
}
count++
}
return count, scanner.Err()
}
// marshalIfComplex converts maps/slices back to JSON strings for DB storage.
func marshalIfComplex(v interface{}) interface{} {
if v == nil {
return nil
}
switch v.(type) {
case map[string]interface{}, []interface{}:
b, _ := json.Marshal(v)
return string(b)
default:
return v
}
}
// listExtDataTables returns all (package_id, physical_table_name) pairs
// from the ext_data_tables catalog.
func listExtDataTables(ctx context.Context, db *sql.DB) ([]extDataTableEntry, error) {
rows, err := db.QueryContext(ctx, "SELECT package_id, table_name FROM ext_data_tables ORDER BY package_id, table_name")
if err != nil {
return nil, err
}
defer rows.Close()
var entries []extDataTableEntry
for rows.Next() {
var e extDataTableEntry
if err := rows.Scan(&e.PackageID, &e.TableName); err != nil {
return nil, err
}
e.PhysicalName = fmt.Sprintf("ext_%s_%s", e.PackageID, e.TableName)
entries = append(entries, e)
}
return entries, rows.Err()
}
type extDataTableEntry struct {
PackageID string
TableName string
PhysicalName string
}
// tableExists checks if a table exists in the database.
func tableExists(ctx context.Context, db *sql.DB, table string) bool {
if database.IsPostgres() {
var exists bool
db.QueryRowContext(ctx, "SELECT EXISTS(SELECT 1 FROM information_schema.tables WHERE table_name = $1)", table).Scan(&exists)
return exists
}
// SQLite
var name string
err := db.QueryRowContext(ctx, "SELECT name FROM sqlite_master WHERE type='table' AND name=?", table).Scan(&name)
return err == nil
}
// wipeTables deletes all data from the given tables in reverse order (FK-safe).
func wipeTables(ctx context.Context, db *sql.DB, tables []string) error {
if database.IsSQLite() {
db.ExecContext(ctx, "PRAGMA foreign_keys = OFF")
defer db.ExecContext(ctx, "PRAGMA foreign_keys = ON")
}
// Delete in reverse order to respect FK constraints
for i := len(tables) - 1; i >= 0; i-- {
table := tables[i]
if !tableExists(ctx, db, table) {
continue
}
if database.IsPostgres() {
if _, err := db.ExecContext(ctx, fmt.Sprintf("TRUNCATE TABLE %s CASCADE", table)); err != nil {
log.Printf("[backup] warning: truncate %s: %v", table, err)
}
} else {
if _, err := db.ExecContext(ctx, fmt.Sprintf("DELETE FROM %s", table)); err != nil {
log.Printf("[backup] warning: delete %s: %v", table, err)
}
}
}
return nil
}

372
server/handlers/backup_test.go Normal file
View File

@@ -0,0 +1,372 @@
package handlers
import (
"archive/zip"
"bufio"
"bytes"
"context"
"encoding/json"
"mime/multipart"
"net/http"
"net/http/httptest"
"testing"
"github.com/gin-gonic/gin"
authpkg "switchboard-core/auth"
"switchboard-core/config"
"switchboard-core/database"
"switchboard-core/middleware"
"switchboard-core/store"
"switchboard-core/store/sqlite"
)
// ── Backup Test Harness ────────────────────────
type backupHarness struct {
*testHarness
adminToken string
adminID string
stores store.Stores
backupH *BackupHandler
}
func setupBackupHarness(t *testing.T) *backupHarness {
t.Helper()
database.RequireTestDB(t)
database.TruncateAll(t)
cfg := &config.Config{
JWTSecret: testJWTSecret,
BasePath: "",
}
var stores store.Stores
if database.IsSQLite() {
stores = sqlite.NewStores(database.TestDB)
} else {
t.Skip("backup tests run on SQLite only")
}
userCache := middleware.NewUserStatusCache()
storagePath := t.TempDir()
backupH := NewBackupHandler(stores, "", storagePath)
r := gin.New()
api := r.Group("/api/v1")
// Auth
auth := NewAuthHandler(cfg, stores, nil, authpkg.NewBuiltinProvider())
api.POST("/auth/register", auth.Register)
api.POST("/auth/login", auth.Login)
// Admin group
admin := api.Group("/admin")
admin.Use(middleware.Auth(cfg, stores.Users, userCache), middleware.RequireAdmin(stores))
admin.POST("/backup", backupH.CreateBackup)
admin.GET("/backups", backupH.ListBackups)
admin.GET("/backups/:name", backupH.DownloadBackup)
admin.DELETE("/backups/:name", backupH.DeleteBackup)
admin.POST("/restore", backupH.RestoreBackup)
// Seed admin
adminID := seedInsertReturningID(t,
`INSERT INTO users (username, email, password_hash, handle, auth_source) VALUES ($1, $2, $3, $4, $5) RETURNING id`,
"bk-admin", "bk-admin@test.com", "$2a$10$test", "bk-admin", "builtin",
)
database.SeedEveryoneGroupMember(t, adminID)
database.SeedAdminsGroupMember(t, adminID)
adminToken := makeToken(adminID, "bk-admin@test.com", "admin")
return &backupHarness{
testHarness: &testHarness{router: r, t: t},
adminToken: adminToken,
adminID: adminID,
stores: stores,
backupH: backupH,
}
}
// ── Tests ──────────────────────────────────────
func TestCreateBackup_Basic(t *testing.T) {
h := setupBackupHarness(t)
w := h.request("POST", "/api/v1/admin/backup", h.adminToken, nil)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
// Verify it's a valid ZIP
data := w.Body.Bytes()
zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
if err != nil {
t.Fatalf("invalid ZIP: %v", err)
}
// Must contain meta.json
var foundMeta bool
for _, f := range zr.File {
if f.Name == "meta.json" {
foundMeta = true
rc, _ := f.Open()
var meta backupMeta
json.NewDecoder(rc).Decode(&meta)
rc.Close()
if meta.Dialect != "sqlite" {
t.Errorf("expected dialect sqlite, got %s", meta.Dialect)
}
if len(meta.CoreTables) == 0 {
t.Error("expected core tables in meta")
}
}
}
if !foundMeta {
t.Fatal("meta.json not found in backup")
}
// Must contain core table JSONL files
var foundUsers bool
for _, f := range zr.File {
if f.Name == "core/users.jsonl" {
foundUsers = true
}
}
if !foundUsers {
t.Error("core/users.jsonl not found in backup")
}
}
func TestCreateBackup_WithExtData(t *testing.T) {
h := setupBackupHarness(t)
ctx := context.Background()
// Seed a package so FK constraint is satisfied
database.TestDB.ExecContext(ctx, dialectSQL(
`INSERT INTO packages (id, title, type, tier, version, source, enabled, manifest)
VALUES ($1, $2, $3, $4, $5, $6, 1, '{}')`),
"testpkg", "Test Pkg", "extension", "starlark", "0.1.0", "extension",
)
// Register an ext_data table and create it
database.TestDB.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS ext_testpkg_items (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now'))
)`)
if _, err := database.TestDB.ExecContext(ctx,
dialectSQL(`INSERT INTO ext_data_tables (package_id, table_name) VALUES ($1, $2)`),
"testpkg", "items",
); err != nil {
t.Fatalf("insert ext_data_tables: %v", err)
}
database.TestDB.ExecContext(ctx,
`INSERT INTO ext_testpkg_items (id, name) VALUES ('item1', 'Test Item')`,
)
defer database.TestDB.ExecContext(ctx, "DROP TABLE IF EXISTS ext_testpkg_items")
w := h.request("POST", "/api/v1/admin/backup", h.adminToken, nil)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d", w.Code)
}
data := w.Body.Bytes()
zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
if err != nil {
t.Fatalf("invalid ZIP: %v", err)
}
// Should contain ext_data/testpkg/items.jsonl
var foundExt bool
for _, f := range zr.File {
if f.Name == "ext_data/testpkg/items.jsonl" {
foundExt = true
rc, _ := f.Open()
scanner := bufio.NewScanner(rc)
if !scanner.Scan() {
t.Error("ext_data file is empty")
} else {
var row map[string]interface{}
json.Unmarshal(scanner.Bytes(), &row)
if row["name"] != "Test Item" {
t.Errorf("expected 'Test Item', got %v", row["name"])
}
}
rc.Close()
}
}
if !foundExt {
t.Error("ext_data/testpkg/items.jsonl not found")
}
// Check meta.json has ext table listed
metaFile := findInZip(zr, "meta.json")
rc, _ := metaFile.Open()
var meta backupMeta
json.NewDecoder(rc).Decode(&meta)
rc.Close()
found := false
for _, et := range meta.ExtTables {
if et == "ext_testpkg_items" {
found = true
}
}
if !found {
t.Errorf("ext_testpkg_items not in meta.ExtTables: %v", meta.ExtTables)
}
}
func TestRestoreBackup_RoundTrip(t *testing.T) {
h := setupBackupHarness(t)
// Create a second user so we have data to verify
seedInsertReturningID(t,
`INSERT INTO users (username, email, password_hash, handle, auth_source) VALUES ($1, $2, $3, $4, $5) RETURNING id`,
"bk-user2", "bk-user2@test.com", "$2a$10$test", "bk-user2", "builtin",
)
// Create backup
w := h.request("POST", "/api/v1/admin/backup", h.adminToken, nil)
if w.Code != http.StatusOK {
t.Fatalf("backup: expected 200, got %d: %s", w.Code, w.Body.String())
}
backupData := w.Body.Bytes()
// Verify we have 2 users
var countBefore int
database.TestDB.QueryRow("SELECT COUNT(*) FROM users").Scan(&countBefore)
if countBefore < 2 {
t.Fatalf("expected at least 2 users, got %d", countBefore)
}
// Restore directly (the handler wipes + restores, so data should round-trip)
// We use a separate router without admin middleware for restore,
// since the wipe inside RestoreBackup removes the admin group membership
// mid-request. In production, restore would use a session-based token
// that survives the wipe. For testing, we bypass the middleware.
restoreRouter := gin.New()
restoreRouter.POST("/restore", h.backupH.RestoreBackup)
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
part, _ := writer.CreateFormFile("file", "test-backup.swb")
part.Write(backupData)
writer.Close()
req := httptest.NewRequest("POST", "/restore", body)
req.Header.Set("Content-Type", writer.FormDataContentType())
rw := httptest.NewRecorder()
restoreRouter.ServeHTTP(rw, req)
if rw.Code != http.StatusOK {
t.Fatalf("restore: expected 200, got %d: %s", rw.Code, rw.Body.String())
}
// Verify data is back
var countAfterRestore int
database.TestDB.QueryRow("SELECT COUNT(*) FROM users").Scan(&countAfterRestore)
if countAfterRestore != countBefore {
t.Errorf("expected %d users after restore, got %d", countBefore, countAfterRestore)
}
// Verify specific user exists
var username string
err := database.TestDB.QueryRow("SELECT username FROM users WHERE username = 'bk-user2'").Scan(&username)
if err != nil {
t.Errorf("user bk-user2 not found after restore: %v", err)
}
}
func TestRestoreBackup_SchemaVersionMismatch(t *testing.T) {
h := setupBackupHarness(t)
// Build a fake backup with a future schema version
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
mw, _ := zw.Create("meta.json")
json.NewEncoder(mw).Encode(backupMeta{
Version: "99.0.0",
SchemaVersion: "999_future",
Dialect: "sqlite",
CoreTables: []string{"users"},
})
zw.Close()
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
part, _ := writer.CreateFormFile("file", "future.swb")
part.Write(buf.Bytes())
writer.Close()
req := httptest.NewRequest("POST", "/api/v1/admin/restore", body)
req.Header.Set("Content-Type", writer.FormDataContentType())
req.Header.Set("Authorization", "Bearer "+h.adminToken)
rw := httptest.NewRecorder()
h.router.ServeHTTP(rw, req)
if rw.Code != http.StatusBadRequest {
t.Fatalf("expected 400 for future schema, got %d: %s", rw.Code, rw.Body.String())
}
}
func TestDumpAndRestoreTable(t *testing.T) {
database.RequireTestDB(t)
ctx := context.Background()
db := database.TestDB
// Create a test table
db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS test_backup_round (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
count INTEGER DEFAULT 0
)`)
defer db.ExecContext(ctx, "DROP TABLE IF EXISTS test_backup_round")
// Insert data
db.ExecContext(ctx, `INSERT INTO test_backup_round (id, name, count) VALUES ('r1', 'alpha', 10)`)
db.ExecContext(ctx, `INSERT INTO test_backup_round (id, name, count) VALUES ('r2', 'beta', 20)`)
// Dump
var buf bytes.Buffer
count, err := dumpTable(ctx, db, "test_backup_round", &buf)
if err != nil {
t.Fatalf("dump: %v", err)
}
if count != 2 {
t.Fatalf("dump: expected 2 rows, got %d", count)
}
// Wipe
db.ExecContext(ctx, "DELETE FROM test_backup_round")
// Restore
restored, err := restoreTable(ctx, db, "test_backup_round", &buf)
if err != nil {
t.Fatalf("restore: %v", err)
}
if restored != 2 {
t.Fatalf("restore: expected 2 rows, got %d", restored)
}
// Verify
var name string
var cnt int
db.QueryRowContext(ctx, "SELECT name, count FROM test_backup_round WHERE id = 'r1'").Scan(&name, &cnt)
if name != "alpha" || cnt != 10 {
t.Errorf("expected alpha/10, got %s/%d", name, cnt)
}
}
func TestListBackups_Empty(t *testing.T) {
h := setupBackupHarness(t)
w := h.request("GET", "/api/v1/admin/backups", h.adminToken, nil)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d", w.Code)
}
var resp struct{ Data []interface{} }
json.NewDecoder(w.Body).Decode(&resp)
if len(resp.Data) != 0 {
t.Errorf("expected empty list, got %d", len(resp.Data))
}
}

134
server/handlers/docs.go Normal file
View File

@@ -0,0 +1,134 @@
package handlers
import (
"net/http"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/gin-gonic/gin"
)
// DocsHandler serves documentation files.
type DocsHandler struct {
docsDir string
}
// NewDocsHandler creates a new DocsHandler.
func NewDocsHandler(docsDir string) *DocsHandler {
return &DocsHandler{docsDir: docsDir}
}
// docEntry describes a documentation file.
type docEntry struct {
Slug string `json:"slug"`
Title string `json:"title"`
Description string `json:"description,omitempty"`
}
// docsOrder defines the display order and metadata for documentation files.
var docsOrder = []docEntry{
{Slug: "GETTING-STARTED", Title: "Getting Started", Description: "Quick start guide"},
{Slug: "ARCHITECTURE", Title: "Architecture", Description: "Kernel design and components"},
{Slug: "EXTENSION-GUIDE", Title: "Extension Guide", Description: "How to author extensions"},
{Slug: "PACKAGE-FORMAT", Title: "Package Format", Description: "The .pkg archive format"},
{Slug: "API-REFERENCE", Title: "API Reference", Description: "REST API overview"},
{Slug: "DEPLOYMENT", Title: "Deployment", Description: "Production deployment guide"},
{Slug: "DISTRIBUTION", Title: "Distribution", Description: "Docker image and bundled packages"},
}
// ListDocs returns the list of available documentation files.
// GET /api/v1/docs
func (h *DocsHandler) ListDocs(c *gin.Context) {
if h.docsDir == "" {
c.JSON(http.StatusOK, gin.H{"data": []docEntry{}})
return
}
var available []docEntry
for _, d := range docsOrder {
path := filepath.Join(h.docsDir, d.Slug+".md")
if _, err := os.Stat(path); err == nil {
available = append(available, d)
}
}
// Also add any .md files not in the ordered list
entries, err := os.ReadDir(h.docsDir)
if err == nil {
known := make(map[string]bool)
for _, d := range docsOrder {
known[d.Slug] = true
}
for _, e := range entries {
if e.IsDir() || !strings.HasSuffix(e.Name(), ".md") {
continue
}
slug := strings.TrimSuffix(e.Name(), ".md")
if known[slug] {
continue
}
// Skip design docs and other non-user-facing files
if strings.HasPrefix(slug, "DESIGN-") || strings.HasPrefix(slug, "DEMO-") {
continue
}
available = append(available, docEntry{
Slug: slug,
Title: slugToTitle(slug),
})
}
}
if available == nil {
available = []docEntry{}
}
c.JSON(http.StatusOK, gin.H{"data": available})
}
// GetDoc returns the raw markdown content of a documentation file.
// GET /api/v1/docs/:name
func (h *DocsHandler) GetDoc(c *gin.Context) {
name := c.Param("name")
if !isValidDocName(name) {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid document name"})
return
}
if h.docsDir == "" {
c.JSON(http.StatusNotFound, gin.H{"error": "docs not configured"})
return
}
path := filepath.Join(h.docsDir, name+".md")
data, err := os.ReadFile(path)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "document not found"})
return
}
c.JSON(http.StatusOK, gin.H{
"data": gin.H{
"slug": name,
"title": slugToTitle(name),
"content": string(data),
},
})
}
var validDocNameRe = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
func isValidDocName(name string) bool {
return validDocNameRe.MatchString(name) && !strings.Contains(name, "..")
}
func slugToTitle(slug string) string {
// Convert "GETTING-STARTED" → "Getting Started"
parts := strings.Split(slug, "-")
for i, p := range parts {
if len(p) > 0 {
parts[i] = strings.ToUpper(p[:1]) + strings.ToLower(p[1:])
}
}
return strings.Join(parts, " ")
}

101
server/handlers/ext_api.go
View File

@@ -332,6 +332,107 @@ func hasPermission(granted []string, perm string) bool {
return false
}
// ── api_schema support (v0.6.2) ─────────────────────────────────────
// APISchemaEntry represents one route's OpenAPI-level documentation
// declared via the optional manifest "api_schema" array.
type APISchemaEntry struct {
Path string `json:"path"`
Method string `json:"method"`
Summary string `json:"summary,omitempty"`
Description string `json:"description,omitempty"`
Params map[string]any `json:"params,omitempty"`
Body map[string]any `json:"body,omitempty"`
Response map[string]any `json:"response,omitempty"`
}
// ParseAPISchema extracts the optional api_schema array from a package manifest.
// Malformed entries are logged and skipped — never blocks extension loading.
func ParseAPISchema(manifest map[string]any) []APISchemaEntry {
raw, ok := manifest["api_schema"]
if !ok {
return nil
}
arr, ok := raw.([]any)
if !ok {
log.Printf("[ext_api] api_schema is not an array, skipping")
return nil
}
var entries []APISchemaEntry
for i, item := range arr {
m, ok := item.(map[string]any)
if !ok {
log.Printf("[ext_api] api_schema[%d]: not an object, skipping", i)
continue
}
path, _ := m["path"].(string)
method, _ := m["method"].(string)
if path == "" || method == "" {
log.Printf("[ext_api] api_schema[%d]: missing path or method, skipping", i)
continue
}
entry := APISchemaEntry{
Path: path,
Method: strings.ToUpper(method),
}
if s, ok := m["summary"].(string); ok {
entry.Summary = s
}
if s, ok := m["description"].(string); ok {
entry.Description = s
}
if p, ok := m["params"].(map[string]any); ok {
entry.Params = p
}
if b, ok := m["body"].(map[string]any); ok {
entry.Body = b
}
if r, ok := m["response"].(map[string]any); ok {
entry.Response = r
}
entries = append(entries, entry)
}
return entries
}
// ExtractAPIRoutes returns all declared path+method pairs from the manifest.
// Used by the OpenAPI builder to enumerate routes for stub generation.
func ExtractAPIRoutes(manifest map[string]any) []struct{ Method, Path string } {
raw, ok := manifest["api_routes"]
if !ok {
return nil
}
// Boolean shorthand — no specific routes to enumerate
if _, ok := raw.(bool); ok {
return nil
}
routes, ok := raw.([]any)
if !ok {
return nil
}
var result []struct{ Method, Path string }
for _, entry := range routes {
route, ok := entry.(map[string]any)
if !ok {
continue
}
method, _ := route["method"].(string)
path, _ := route["path"].(string)
if method == "" || path == "" {
continue
}
result = append(result, struct{ Method, Path string }{
Method: strings.ToUpper(method),
Path: path,
})
}
return result
}
// HasAPIRoutes checks whether a package manifest declares API routes.
// Used by startup discovery and admin UI to identify API-capable packages.
func HasAPIRoutes(manifest map[string]any) bool {

276
server/handlers/openapi.go Normal file
View File

@@ -0,0 +1,276 @@
// Package handlers — openapi.go
//
// v0.6.2: Dynamic OpenAPI spec generation. Merges the static kernel spec
// with extension-declared API routes. Extensions with api_schema get rich
// path items; others get auto-generated stubs.
//
// Two tiers:
// Tier 1 — Stubs: zero extension author effort, every route gets a stub
// Tier 2 — api_schema: optional manifest field for richer docs
package handlers
import (
"context"
"fmt"
"log"
"strings"
"switchboard-core/store"
"gopkg.in/yaml.v3"
)
// BuildOpenAPISpec produces a complete OpenAPI 3.0 spec by merging the static
// kernel spec with dynamically discovered extension routes.
//
// staticSpec is the raw YAML of the kernel's openapi.yaml.
// version is substituted into the spec's info.version field.
func BuildOpenAPISpec(stores store.Stores, staticSpec []byte, version string) (map[string]any, error) {
// 1. Parse static spec
spec := make(map[string]any)
patched := strings.Replace(string(staticSpec), "${VERSION}", version, 1)
if err := yaml.Unmarshal([]byte(patched), &spec); err != nil {
return nil, fmt.Errorf("failed to parse static OpenAPI spec: %w", err)
}
// Ensure paths map exists
paths, _ := spec["paths"].(map[string]any)
if paths == nil {
paths = make(map[string]any)
spec["paths"] = paths
}
// Ensure tags array exists
tags, _ := spec["tags"].([]any)
// 2. Enumerate enabled extensions with API routes
if stores.Packages == nil {
return spec, nil
}
pkgs, err := stores.Packages.List(context.Background())
if err != nil {
log.Printf("[openapi] Failed to list packages: %v", err)
return spec, nil // degrade gracefully — return kernel-only spec
}
for _, pkg := range pkgs {
if !pkg.Enabled || pkg.Manifest == nil {
continue
}
if !HasAPIRoutes(pkg.Manifest) {
continue
}
slug := pkg.ID
tagName := "extensions/" + slug
// Add extension tag
tags = append(tags, map[string]any{
"name": tagName,
"description": fmt.Sprintf("API routes for %s extension", pkg.Title),
})
// Build schema lookup from api_schema (if present)
schemaEntries := ParseAPISchema(pkg.Manifest)
schemaMap := make(map[string]APISchemaEntry) // key: "METHOD /path"
for _, entry := range schemaEntries {
key := entry.Method + " " + entry.Path
schemaMap[key] = entry
}
// Iterate declared routes
routes := ExtractAPIRoutes(pkg.Manifest)
if len(routes) == 0 {
// api_routes: true — no specific routes to enumerate.
// If api_schema is declared, generate entries from those.
for _, entry := range schemaEntries {
oaPath := fmt.Sprintf("/s/%s/api%s", slug, entry.Path)
method := strings.ToLower(entry.Method)
pathItem := getOrCreatePathItem(paths, oaPath)
pathItem[method] = buildSchemaOperation(entry, tagName)
}
continue
}
for _, route := range routes {
// Skip wildcard paths — can't represent meaningfully in OpenAPI
if strings.HasSuffix(route.Path, "*") {
continue
}
oaPath := fmt.Sprintf("/s/%s/api%s", slug, route.Path)
method := strings.ToLower(route.Method)
// Wildcard method: generate for common methods
methods := []string{method}
if route.Method == "*" {
methods = []string{"get", "post", "put", "delete", "patch"}
}
for _, m := range methods {
pathItem := getOrCreatePathItem(paths, oaPath)
key := strings.ToUpper(m) + " " + route.Path
if schema, ok := schemaMap[key]; ok {
pathItem[m] = buildSchemaOperation(schema, tagName)
} else {
pathItem[m] = buildStubOperation(slug, m, route.Path, tagName)
}
}
}
}
spec["tags"] = tags
return spec, nil
}
// getOrCreatePathItem returns the path item map for the given OpenAPI path,
// creating it if it doesn't exist.
func getOrCreatePathItem(paths map[string]any, path string) map[string]any {
if existing, ok := paths[path].(map[string]any); ok {
return existing
}
item := make(map[string]any)
paths[path] = item
return item
}
// buildStubOperation generates an auto-generated stub operation for routes
// without api_schema declarations.
func buildStubOperation(slug, method, path, tag string) map[string]any {
return map[string]any{
"summary": fmt.Sprintf("%s: %s %s", slug, strings.ToUpper(method), path),
"tags": []any{tag},
"responses": map[string]any{
"200": map[string]any{
"description": "Extension-defined response",
},
},
}
}
// buildSchemaOperation converts an APISchemaEntry into a full OpenAPI operation.
func buildSchemaOperation(entry APISchemaEntry, tag string) map[string]any {
op := map[string]any{
"summary": entry.Summary,
"tags": []any{tag},
}
if entry.Summary == "" {
op["summary"] = fmt.Sprintf("%s %s", entry.Method, entry.Path)
}
if entry.Description != "" {
op["description"] = entry.Description
}
// Query/path parameters
if len(entry.Params) > 0 {
var params []any
for name, def := range entry.Params {
param := map[string]any{
"name": name,
"in": "query",
}
if m, ok := def.(map[string]any); ok {
if t, ok := m["type"].(string); ok {
param["schema"] = map[string]any{"type": mapJSONType(t)}
}
if desc, ok := m["description"].(string); ok {
param["description"] = desc
}
if req, ok := m["required"].(bool); ok && req {
param["required"] = true
}
} else {
// Simple type string
if t, ok := def.(string); ok {
param["schema"] = map[string]any{"type": mapJSONType(t)}
}
}
params = append(params, param)
}
op["parameters"] = params
}
// Request body
if len(entry.Body) > 0 {
properties := make(map[string]any)
var required []any
for name, def := range entry.Body {
prop := map[string]any{}
if m, ok := def.(map[string]any); ok {
if t, ok := m["type"].(string); ok {
prop["type"] = mapJSONType(t)
}
if desc, ok := m["description"].(string); ok {
prop["description"] = desc
}
if req, ok := m["required"].(bool); ok && req {
required = append(required, name)
}
} else if t, ok := def.(string); ok {
prop["type"] = mapJSONType(t)
}
properties[name] = prop
}
schema := map[string]any{
"type": "object",
"properties": properties,
}
if len(required) > 0 {
schema["required"] = required
}
op["requestBody"] = map[string]any{
"required": true,
"content": map[string]any{
"application/json": map[string]any{
"schema": schema,
},
},
}
}
// Response
resp := map[string]any{
"200": map[string]any{
"description": "Successful response",
},
}
if entry.Response != nil {
respSchema := make(map[string]any)
if t, ok := entry.Response["type"].(string); ok {
respSchema["type"] = mapJSONType(t)
}
if ex, ok := entry.Response["example"]; ok {
respSchema["example"] = ex
}
resp["200"] = map[string]any{
"description": "Successful response",
"content": map[string]any{
"application/json": map[string]any{
"schema": respSchema,
},
},
}
}
op["responses"] = resp
return op
}
// mapJSONType converts common type names to OpenAPI types.
func mapJSONType(t string) string {
switch strings.ToLower(t) {
case "int", "integer":
return "integer"
case "float", "number", "double":
return "number"
case "bool", "boolean":
return "boolean"
case "array", "list":
return "array"
case "object", "map", "dict":
return "object"
default:
return "string"
}
}

410
server/handlers/openapi_test.go Normal file
View File

@@ -0,0 +1,410 @@
package handlers
import (
"context"
"encoding/json"
"testing"
"switchboard-core/database"
"switchboard-core/store"
postgres "switchboard-core/store/postgres"
sqlite "switchboard-core/store/sqlite"
)
// Minimal static spec for testing — valid OpenAPI 3.0 structure.
var testStaticSpec = []byte(`
openapi: "3.0.3"
info:
title: Switchboard Core
version: "${VERSION}"
paths:
/api/v1/health:
get:
summary: Health check
responses:
"200":
description: OK
tags:
- name: system
description: System endpoints
`)
func openapiTestStores(t *testing.T) store.Stores {
t.Helper()
database.RequireTestDB(t)
database.TruncateAll(t)
if database.IsSQLite() {
return sqlite.NewStores(database.TestDB)
}
return postgres.NewStores(database.TestDB)
}
// ── Test: Zero extensions → kernel spec only ──────────────────
func TestBuildOpenAPISpec_NoExtensions(t *testing.T) {
stores := openapiTestStores(t)
spec, err := BuildOpenAPISpec(stores, testStaticSpec, "0.6.2")
if err != nil {
t.Fatalf("BuildOpenAPISpec failed: %v", err)
}
// Version should be substituted
info, _ := spec["info"].(map[string]any)
if info == nil {
t.Fatal("missing info field")
}
if v, _ := info["version"].(string); v != "0.6.2" {
t.Errorf("expected version 0.6.2, got %q", v)
}
// Paths should contain the kernel endpoint
paths, _ := spec["paths"].(map[string]any)
if paths == nil {
t.Fatal("missing paths field")
}
if _, ok := paths["/api/v1/health"]; !ok {
t.Error("expected /api/v1/health in paths")
}
// Tags should contain kernel tag
tags, _ := spec["tags"].([]any)
if len(tags) < 1 {
t.Error("expected at least 1 tag")
}
}
// ── Test: Extension with api_routes but no api_schema → stubs ────
func TestBuildOpenAPISpec_StubEntries(t *testing.T) {
stores := openapiTestStores(t)
// Install a test extension with api_routes
manifest := map[string]any{
"api_routes": []any{
map[string]any{"method": "GET", "path": "/items"},
map[string]any{"method": "POST", "path": "/items"},
},
}
manifestJSON, _ := json.Marshal(manifest)
err := stores.Packages.Create(context.Background(), &store.PackageRegistration{
ID: "test-ext",
Title: "Test Extension",
Type: "extension",
Version: "1.0.0",
Tier: "starlark",
Manifest: manifest,
Enabled: true,
Status: "active",
Source: "extension",
Scope: "global",
})
_ = manifestJSON
if err != nil {
t.Fatalf("failed to create test package: %v", err)
}
spec, err := BuildOpenAPISpec(stores, testStaticSpec, "0.6.2")
if err != nil {
t.Fatalf("BuildOpenAPISpec failed: %v", err)
}
paths, _ := spec["paths"].(map[string]any)
// Check GET stub
getPath, ok := paths["/s/test-ext/api/items"]
if !ok {
t.Fatal("expected /s/test-ext/api/items in paths")
}
pathItem, _ := getPath.(map[string]any)
getOp, _ := pathItem["get"].(map[string]any)
if getOp == nil {
t.Fatal("expected GET operation")
}
summary, _ := getOp["summary"].(string)
if summary != "test-ext: GET /items" {
t.Errorf("unexpected stub summary: %q", summary)
}
// Check POST stub
postOp, _ := pathItem["post"].(map[string]any)
if postOp == nil {
t.Fatal("expected POST operation")
}
// Check tag
tags, _ := getOp["tags"].([]any)
if len(tags) != 1 || tags[0] != "extensions/test-ext" {
t.Errorf("unexpected tags: %v", tags)
}
}
// ── Test: Extension with api_schema → rich entries ────────
func TestBuildOpenAPISpec_SchemaEntries(t *testing.T) {
stores := openapiTestStores(t)
manifest := map[string]any{
"api_routes": []any{
map[string]any{"method": "GET", "path": "/items"},
map[string]any{"method": "POST", "path": "/items"},
map[string]any{"method": "DELETE", "path": "/items/:id"},
},
"api_schema": []any{
map[string]any{
"path": "/items",
"method": "GET",
"summary": "List items",
"params": map[string]any{
"limit": map[string]any{"type": "integer", "default": 50},
},
"response": map[string]any{
"type": "object",
"example": map[string]any{"data": []any{}},
},
},
map[string]any{
"path": "/items",
"method": "POST",
"summary": "Create item",
"body": map[string]any{
"name": map[string]any{"type": "string", "required": true},
"description": map[string]any{"type": "string"},
},
},
},
}
err := stores.Packages.Create(context.Background(), &store.PackageRegistration{
ID: "rich-ext",
Title: "Rich Extension",
Type: "extension",
Version: "1.0.0",
Tier: "starlark",
Manifest: manifest,
Enabled: true,
Status: "active",
Source: "extension",
Scope: "global",
})
if err != nil {
t.Fatalf("failed to create test package: %v", err)
}
spec, err := BuildOpenAPISpec(stores, testStaticSpec, "0.6.2")
if err != nil {
t.Fatalf("BuildOpenAPISpec failed: %v", err)
}
paths, _ := spec["paths"].(map[string]any)
// GET /items should have rich schema (summary = "List items")
itemsPath, ok := paths["/s/rich-ext/api/items"]
if !ok {
t.Fatal("expected /s/rich-ext/api/items in paths")
}
pathItem, _ := itemsPath.(map[string]any)
getOp, _ := pathItem["get"].(map[string]any)
if getOp == nil {
t.Fatal("expected GET operation")
}
if s, _ := getOp["summary"].(string); s != "List items" {
t.Errorf("expected rich summary, got %q", s)
}
// Should have parameters
params, _ := getOp["parameters"].([]any)
if len(params) == 0 {
t.Error("expected parameters from api_schema")
}
// POST /items should have requestBody
postOp, _ := pathItem["post"].(map[string]any)
if postOp == nil {
t.Fatal("expected POST operation")
}
if _, ok := postOp["requestBody"]; !ok {
t.Error("expected requestBody from api_schema body")
}
// DELETE /items/:id should be a stub (no api_schema for it)
deletePath, ok := paths["/s/rich-ext/api/items/:id"]
if !ok {
t.Fatal("expected /s/rich-ext/api/items/:id in paths")
}
deleteItem, _ := deletePath.(map[string]any)
deleteOp, _ := deleteItem["delete"].(map[string]any)
if deleteOp == nil {
t.Fatal("expected DELETE operation")
}
deleteSummary, _ := deleteOp["summary"].(string)
if deleteSummary != "rich-ext: DELETE /items/:id" {
t.Errorf("expected stub summary for undeclared route, got %q", deleteSummary)
}
}
// ── Test: Multiple extensions → all appear ────────
func TestBuildOpenAPISpec_MultipleExtensions(t *testing.T) {
stores := openapiTestStores(t)
for _, ext := range []struct {
id string
title string
route string
}{
{"ext-a", "Extension A", "/status"},
{"ext-b", "Extension B", "/health"},
} {
err := stores.Packages.Create(context.Background(), &store.PackageRegistration{
ID: ext.id,
Title: ext.title,
Type: "extension",
Version: "1.0.0",
Tier: "starlark",
Manifest: map[string]any{
"api_routes": []any{
map[string]any{"method": "GET", "path": ext.route},
},
},
Enabled: true,
Status: "active",
Source: "extension",
Scope: "global",
})
if err != nil {
t.Fatalf("failed to create %s: %v", ext.id, err)
}
}
spec, err := BuildOpenAPISpec(stores, testStaticSpec, "0.6.2")
if err != nil {
t.Fatalf("BuildOpenAPISpec failed: %v", err)
}
paths, _ := spec["paths"].(map[string]any)
if _, ok := paths["/s/ext-a/api/status"]; !ok {
t.Error("expected ext-a path")
}
if _, ok := paths["/s/ext-b/api/health"]; !ok {
t.Error("expected ext-b path")
}
// Verify tags
tags, _ := spec["tags"].([]any)
tagNames := make(map[string]bool)
for _, tag := range tags {
if m, ok := tag.(map[string]any); ok {
if n, ok := m["name"].(string); ok {
tagNames[n] = true
}
}
}
if !tagNames["extensions/ext-a"] {
t.Error("expected extensions/ext-a tag")
}
if !tagNames["extensions/ext-b"] {
t.Error("expected extensions/ext-b tag")
}
}
// ── Test: Malformed api_schema → stubs, no crash ────────
func TestBuildOpenAPISpec_MalformedSchema(t *testing.T) {
stores := openapiTestStores(t)
manifest := map[string]any{
"api_routes": []any{
map[string]any{"method": "GET", "path": "/data"},
},
"api_schema": []any{
"not-an-object", // malformed
map[string]any{"path": "/data"}, // missing method
map[string]any{"method": "GET"}, // missing path
map[string]any{"path": "/data", "method": "POST"}, // valid but no matching route
},
}
err := stores.Packages.Create(context.Background(), &store.PackageRegistration{
ID: "bad-schema",
Title: "Bad Schema",
Type: "extension",
Version: "1.0.0",
Tier: "starlark",
Manifest: manifest,
Enabled: true,
Status: "active",
Source: "extension",
Scope: "global",
})
if err != nil {
t.Fatalf("failed to create package: %v", err)
}
spec, err := BuildOpenAPISpec(stores, testStaticSpec, "0.6.2")
if err != nil {
t.Fatalf("BuildOpenAPISpec should not error on malformed schema: %v", err)
}
paths, _ := spec["paths"].(map[string]any)
// GET /data should still appear as a stub
if _, ok := paths["/s/bad-schema/api/data"]; !ok {
t.Error("expected stub entry for /data despite malformed api_schema")
}
}
// ── Test: Disabled extension → excluded ────────
func TestBuildOpenAPISpec_DisabledExcluded(t *testing.T) {
stores := openapiTestStores(t)
err := stores.Packages.Create(context.Background(), &store.PackageRegistration{
ID: "disabled-ext",
Title: "Disabled",
Type: "extension",
Version: "1.0.0",
Tier: "starlark",
Manifest: map[string]any{
"api_routes": []any{
map[string]any{"method": "GET", "path": "/secret"},
},
},
Enabled: false,
Status: "active",
Source: "extension",
Scope: "global",
})
if err != nil {
t.Fatalf("failed to create package: %v", err)
}
spec, err := BuildOpenAPISpec(stores, testStaticSpec, "0.6.2")
if err != nil {
t.Fatalf("BuildOpenAPISpec failed: %v", err)
}
paths, _ := spec["paths"].(map[string]any)
if _, ok := paths["/s/disabled-ext/api/secret"]; ok {
t.Error("disabled extension should not appear in spec")
}
}
// ── Test: Required OpenAPI 3.0 fields present ────────
func TestBuildOpenAPISpec_RequiredFields(t *testing.T) {
stores := openapiTestStores(t)
spec, err := BuildOpenAPISpec(stores, testStaticSpec, "1.0.0")
if err != nil {
t.Fatalf("BuildOpenAPISpec failed: %v", err)
}
if _, ok := spec["openapi"]; !ok {
t.Error("missing required 'openapi' field")
}
if _, ok := spec["info"]; !ok {
t.Error("missing required 'info' field")
}
if _, ok := spec["paths"]; !ok {
t.Error("missing required 'paths' field")
}
}

38
server/main.go
View File

@@ -311,6 +311,15 @@ func main() {
patched := bytes.Replace(openapiSpec, []byte("${VERSION}"), []byte(Version), 1)
c.Data(http.StatusOK, "application/yaml", patched)
})
// v0.6.2: Dynamic OpenAPI spec with extension routes merged in
base.GET("/api/docs/openapi.json", func(c *gin.Context) {
spec, err := handlers.BuildOpenAPISpec(stores, openapiSpec, Version)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to build spec"})
return
}
c.JSON(http.StatusOK, spec)
})
// WebSocket endpoint
base.GET("/ws", middleware.WsAuth(cfg, stores.Users, userCache, ticketAdapter), hub.HandleWebSocket)
@@ -512,6 +521,12 @@ func main() {
protected.GET("/settings", settings.GetSettings)
protected.PUT("/settings", settings.UpdateSettings)
// Documentation (v0.6.1)
docsDir := findDocsDir()
docsH := handlers.NewDocsHandler(docsDir)
protected.GET("/docs", docsH.ListDocs)
protected.GET("/docs/:name", docsH.GetDoc)
// Permission bootstrap (v0.37.1) — self-service resolved permissions
permH := handlers.NewProfilePermissionsHandler(stores)
protected.GET("/profile/permissions", permH.GetMyPermissions)
@@ -793,6 +808,14 @@ func main() {
admin.GET("/cluster", clusterH.ListNodes)
}
// ── Backup/Restore (v0.6.1) ─────────
backupH := handlers.NewBackupHandler(stores, packagesDir, cfg.StoragePath)
admin.POST("/backup", backupH.CreateBackup)
admin.GET("/backups", backupH.ListBackups)
admin.GET("/backups/:name", backupH.DownloadBackup)
admin.DELETE("/backups/:name", backupH.DeleteBackup)
admin.POST("/restore", backupH.RestoreBackup)
// Surface aliases (backward compat — same handlers)
admin.GET("/surfaces", pkgAdm.ListPackages)
admin.GET("/surfaces/:id", pkgAdm.GetPackage)
@@ -898,6 +921,21 @@ func appendClusterHealth(info gin.H, reg *cluster.Registry, stores store.Stores)
}
}
// findDocsDir locates the docs/ directory. Checks common locations.
func findDocsDir() string {
candidates := []string{
"docs",
"../docs",
"/app/docs",
}
for _, dir := range candidates {
if info, err := os.Stat(dir); err == nil && info.IsDir() {
return dir
}
}
return ""
}
// ── Vault CLI Commands ──────────────────────
func runVaultCommand(subcmd string) {

2
server/pages/loaders.go
View File

@@ -81,7 +81,7 @@ func sectionCategory(section string) string {
return "people"
case "workflows":
return "workflows"
case "settings", "storage", "packages", "broadcast":
case "settings", "storage", "packages", "broadcast", "backup":
return "system"
case "usage", "audit", "stats":
return "monitoring"

8
server/pages/pages.go
View File

@@ -249,6 +249,11 @@ func (e *Engine) registerCoreSurfaces() {
Title: "Welcome", Template: "surface-welcome", Auth: "authenticated",
Layout: "single", Source: "core",
},
{
ID: "docs", Route: "/docs/:section", AltRoutes: []string{"/docs"},
Title: "Docs", Template: "surface-docs", Auth: "authenticated",
Layout: "single", Source: "core",
},
}
}
@@ -361,6 +366,9 @@ func (e *Engine) RenderSurface(surfaceID string) gin.HandlerFunc {
if section == "" && surfaceID == "team-admin" {
section = "members"
}
if section == "" && surfaceID == "docs" {
section = "GETTING-STARTED"
}
e.Render(c, "base.html", PageData{
Surface: surfaceID,

2
server/pages/templates/base.html
View File

@@ -89,6 +89,7 @@
{{else if eq .Surface "team-admin"}}{{template "surface-team-admin" .}}
{{else if eq .Surface "settings"}}{{template "surface-settings" .}}
{{else if eq .Surface "welcome"}}{{template "surface-welcome" .}}
{{else if eq .Surface "docs"}}{{template "surface-docs" .}}
{{else if and .Manifest (eq .Manifest.Source "extension")}}{{template "surface-extension" .}}
{{else}}<div style="padding:20px">Unknown surface: {{.Surface}}</div>
{{end}}
@@ -124,6 +125,7 @@
{{if eq .Surface "team-admin"}}{{template "scripts-team-admin" .}}{{end}}
{{if eq .Surface "settings"}}{{template "scripts-settings" .}}{{end}}
{{if eq .Surface "welcome"}}{{template "scripts-welcome" .}}{{end}}
{{if eq .Surface "docs"}}{{template "scripts-docs" .}}{{end}}
{{/* v0.27.0: Extension surface JS — load Preact globals, boot SDK, then extension */}}
{{if and .Manifest (eq .Manifest.Source "extension")}}
<script type="module" nonce="{{.CSPNonce}}">

35
server/pages/templates/surfaces/docs.html Normal file
View File

@@ -0,0 +1,35 @@
{{/*
Docs surface — v0.6.1
Renders markdown documentation with sidebar navigation.
*/}}
{{define "surface-docs"}}
<div id="docs-mount" class="surface-docs">
<div style="padding:40px;text-align:center;color:var(--text-3);">Loading docs&hellip;</div>
</div>
{{end}}
{{define "css-docs"}}{{end}}
{{define "scripts-docs"}}
<script nonce="{{.CSPNonce}}">
window.__SECTION__ = '{{.Section}}';
</script>
<script type="module" nonce="{{.CSPNonce}}">
const { h, render } = await import('{{.BasePath}}/js/sw/vendor/preact.module.js');
const hooksModule = await import('{{.BasePath}}/js/sw/vendor/hooks.module.js');
const { default: htm } = await import('{{.BasePath}}/js/sw/vendor/htm.module.js');
const html = htm.bind(h);
window.preact = window.preact || { h, render };
window.hooks = window.hooks || hooksModule;
window.html = window.html || html;
// Boot SDK (idempotent)
const { boot } = await import('{{.BasePath}}/js/sw/sdk/index.js?v={{.Version}}');
await boot();
// Mount docs surface
await import('{{.BasePath}}/js/sw/surfaces/docs/index.js?v={{.Version}}');
</script>
{{end}}

2
server/static/swagger.html
View File

@@ -256,7 +256,7 @@
<script src="https://unpkg.com/swagger-ui-dist@5/swagger-ui-bundle.js"></script>
<script>
SwaggerUIBundle({
url: window.location.pathname.replace(/\/$/, '') + '/openapi.yaml',
url: window.location.pathname.replace(/\/$/, '') + '/openapi.json',
dom_id: '#swagger-ui',
deepLinking: true,
presets: [SwaggerUIBundle.presets.apis, SwaggerUIBundle.SwaggerUIStandalonePreset],

6
src/css/surfaces.css
View File

@@ -19,6 +19,12 @@
.admin-storage-value { font-size: 14px; font-weight: 500; }
/* ── Docs Surface ────────────────────────── */
.surface-docs {
display: flex; flex-direction: column; height: 100%; overflow: hidden;
}
/* ── Settings Surface ────────────────────── */
.surface-settings {

2
src/css/variables.css
View File

@@ -54,6 +54,7 @@
--input-bg: #1a1a1f;
--user-bubble: rgba(108,159,255,0.08);
--danger-bg: rgba(239,68,68,0.12);
--bg-code: #1a1a22;
--shadow-lg: 0 8px 32px rgba(0,0,0,0.5);
/* ── Layout ──────────────────────────────── */
@@ -115,6 +116,7 @@
--input-bg: #f2f3f6;
--user-bubble: rgba(74,124,219,0.06);
--danger-bg: #fef2f2;
--bg-code: #f0f0f2;
--shadow-lg: 0 8px 32px rgba(0,0,0,0.1);
color-scheme: light;

9
src/js/sw/sdk/api-domains.js
View File

@@ -277,6 +277,15 @@ export function createDomains(restClient) {
disable: (id) => rc.put(`/api/v1/admin/surfaces/${id}/disable`, {}),
del: (id) => rc.del(`/api/v1/admin/surfaces/${id}`),
},
// v0.6.1: Backup/Restore
backup: {
create: (opts) => rc.post('/api/v1/admin/backup' + _qs(opts)),
list: () => rc.get('/api/v1/admin/backups'),
download: (name) => `/api/v1/admin/backups/${name}`,
del: (name) => rc.del(`/api/v1/admin/backups/${name}`),
restore: (file) => rc.upload('/api/v1/admin/restore', file),
},
},
// ── Users ──────────────────────────────

10
src/js/sw/shell/user-menu.js
View File

@@ -36,7 +36,7 @@ export function UserMenu({ placement = 'down-right', onAction, extraItems }) {
// Fetch installed surfaces once on mount.
// Filter out core surfaces that have dedicated menu entries below.
const CORE_IDS = new Set(['admin', 'settings', 'team-admin', 'workflow', 'workflow-landing']);
const CORE_IDS = new Set(['admin', 'settings', 'team-admin', 'workflow', 'workflow-landing', 'docs']);
useEffect(() => {
if (!sw?.api?.surfaces?.list) return;
@@ -68,6 +68,11 @@ export function UserMenu({ placement = 'down-right', onAction, extraItems }) {
}
// ── Standard items ─────────────────────
// Docs — skip if current surface IS docs
if (current !== 'docs') {
list.push({ label: 'Docs', action: 'docs', icon: '\ud83d\udcd6' });
}
// Settings — skip if current surface IS settings
if (current !== 'settings') {
list.push({ label: 'Settings', action: 'settings', icon: '\u2699\ufe0f' });
@@ -107,6 +112,9 @@ export function UserMenu({ placement = 'down-right', onAction, extraItems }) {
location.href = BASE + '/login';
});
break;
case 'docs':
location.href = BASE + '/docs/GETTING-STARTED';
break;
case 'debug':
window.dispatchEvent(new CustomEvent('debug:toggle'));
break;

162
src/js/sw/surfaces/admin/backup.js Normal file
View File

@@ -0,0 +1,162 @@
/**
* Admin > System > Backup
*
* Create, download, and restore .swb backup archives.
*/
const { html } = window;
const { useState, useEffect, useCallback } = hooks;
export default function BackupSection() {
const [backups, setBackups] = useState([]);
const [loading, setLoading] = useState(true);
const [creating, setCreating] = useState(false);
const [restoring, setRestoring] = useState(false);
const load = useCallback(async () => {
try {
const resp = await sw.api.admin.backup.list();
setBackups(Array.isArray(resp) ? resp : resp.data || []);
} catch (e) { sw.toast(e.message, 'error'); }
finally { setLoading(false); }
}, []);
useEffect(() => { load(); }, []);
async function createBackup(store) {
setCreating(true);
try {
if (store) {
await sw.api.admin.backup.create({ store: 'true' });
sw.toast('Backup created', 'success');
load();
} else {
// Stream download
const url = '/api/v1/admin/backup';
const token = sw.auth.token();
const resp = await fetch(url, {
method: 'POST',
headers: { 'Authorization': 'Bearer ' + token },
});
if (!resp.ok) throw new Error('Backup failed');
const blob = await resp.blob();
const disposition = resp.headers.get('Content-Disposition') || '';
const match = disposition.match(/filename="?([^"]+)"?/);
const filename = match ? match[1] : 'backup.swb';
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = filename;
a.click();
URL.revokeObjectURL(a.href);
sw.toast('Backup downloaded', 'success');
}
} catch (e) { sw.toast(e.message, 'error'); }
finally { setCreating(false); }
}
async function downloadBackup(name) {
const token = sw.auth.token();
const resp = await fetch(sw.api.admin.backup.download(name), {
headers: { 'Authorization': 'Bearer ' + token },
});
if (!resp.ok) { sw.toast('Download failed', 'error'); return; }
const blob = await resp.blob();
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = name;
a.click();
URL.revokeObjectURL(a.href);
}
async function deleteBackup(name) {
const ok = await sw.confirm(`Delete backup "${name}"?`);
if (!ok) return;
try {
await sw.api.admin.backup.del(name);
sw.toast('Backup deleted', 'success');
load();
} catch (e) { sw.toast(e.message, 'error'); }
}
async function restoreBackup() {
const input = document.createElement('input');
input.type = 'file';
input.accept = '.swb';
input.onchange = async () => {
const file = input.files[0];
if (!file) return;
const ok = await sw.confirm(
`Restore from "${file.name}"? This will replace ALL data in the database. This action cannot be undone.`
);
if (!ok) return;
setRestoring(true);
try {
const resp = await sw.api.admin.backup.restore(file);
const d = resp.data || resp;
sw.toast(`Restored ${d.rows_restored || 0} rows`, 'success');
load();
} catch (e) { sw.toast('Restore failed: ' + e.message, 'error'); }
finally { setRestoring(false); }
};
input.click();
}
function formatBytes(b) {
if (!b) return '0 B';
if (b < 1024) return b + ' B';
if (b < 1048576) return (b / 1024).toFixed(1) + ' KB';
if (b < 1073741824) return (b / 1048576).toFixed(1) + ' MB';
return (b / 1073741824).toFixed(2) + ' GB';
}
function formatDate(d) {
if (!d) return '\u2014';
return new Date(d).toLocaleString();
}
if (loading) return html`<div class="settings-placeholder">Loading\u2026</div>`;
return html`
<div>
<div style="display:flex;gap:8px;margin-bottom:20px;flex-wrap:wrap;">
<button class="btn btn-primary" onclick=${() => createBackup(false)} disabled=${creating || restoring}>
${creating ? 'Creating\u2026' : 'Download Backup'}
</button>
<button class="btn" onclick=${() => createBackup(true)} disabled=${creating || restoring}>
${creating ? 'Creating\u2026' : 'Save to Server'}
</button>
<button class="btn btn-danger" onclick=${restoreBackup} disabled=${creating || restoring}>
${restoring ? 'Restoring\u2026' : 'Restore from File'}
</button>
</div>
<h3 style="margin:0 0 12px;">Server Backups</h3>
${backups.length === 0 ? html`
<p style="color:var(--text-secondary);">No server-side backups. Use "Save to Server" to create one.</p>
` : html`
<table class="data-table">
<thead>
<tr>
<th>Name</th>
<th>Size</th>
<th>Created</th>
<th style="width:120px;">Actions</th>
</tr>
</thead>
<tbody>
${backups.map(b => html`
<tr key=${b.name}>
<td><code>${b.name}</code></td>
<td>${formatBytes(b.size)}</td>
<td>${formatDate(b.created_at)}</td>
<td>
<button class="btn btn-sm" onclick=${() => downloadBackup(b.name)}>Download</button>
<button class="btn btn-sm btn-danger" onclick=${() => deleteBackup(b.name)} style="margin-left:4px;">Delete</button>
</td>
</tr>
`)}
</tbody>
</table>
`}
</div>
`;
}

4
src/js/sw/surfaces/admin/index.js
View File

@@ -21,7 +21,7 @@ import { UserMenu } from '../../shell/user-menu.js';
const ADMIN_SECTIONS = {
people: ['users', 'teams', 'groups'],
workflows: ['workflows'],
system: ['settings', 'storage', 'packages', 'connections', 'broadcast'],
system: ['settings', 'storage', 'packages', 'connections', 'broadcast', 'backup'],
monitoring: ['audit'],
};
@@ -30,6 +30,7 @@ const ADMIN_LABELS = {
workflows: 'Workflows',
settings: 'Settings', storage: 'Storage', packages: 'Packages',
connections: 'Connections', broadcast: 'Broadcast',
backup: 'Backup',
audit: 'Audit',
};
@@ -67,6 +68,7 @@ const sectionModules = {
packages: () => import(`./packages.js${_v}`),
connections: () => import(`./connections.js${_v}`),
broadcast: () => import(`./broadcast.js${_v}`),
backup: () => import(`./backup.js${_v}`),
audit: () => import(`./audit.js${_v}`),
};

461
src/js/sw/surfaces/docs/index.js Normal file
View File

@@ -0,0 +1,461 @@
/**
* DocsSurface — builtin documentation viewer (v0.6.2)
*
* Reads globals:
* __SECTION__ — active doc slug (e.g. "GETTING-STARTED")
* __BASE__ — base path
*
* Fetches markdown from GET /api/v1/docs/:name, renders with
* a simple markdown-to-HTML converter. Sidebar lists all docs.
*
* v0.6.2: dark mode fix, topbar navigation, error handling.
*/
const { html } = window;
const { useState, useEffect, useCallback, useMemo } = hooks;
const { render } = preact;
import { Topbar } from '../../shell/topbar.js';
function DocsSurface() {
const [docs, setDocs] = useState([]);
const [active, setActive] = useState(window.__SECTION__ || 'GETTING-STARTED');
const [content, setContent] = useState('');
const [title, setTitle] = useState('');
const [loading, setLoading] = useState(true);
const [listError, setListError] = useState(false);
// Load doc list
useEffect(() => {
if (!sw?.api?.get) return;
sw.api.get('/api/v1/docs').then(r => {
setDocs(Array.isArray(r) ? r : r.data || []);
setListError(false);
}).catch(() => { setListError(true); });
}, []);
// Load active doc
const loadDoc = useCallback(async (slug) => {
setLoading(true);
setActive(slug);
try {
const resp = await sw.api.get(`/api/v1/docs/${slug}`);
const d = resp.data || resp;
setTitle(d.title || slug);
setContent(d.content || 'Document not found.');
} catch (e) {
setContent('Failed to load document.');
}
setLoading(false);
// Update URL without reload
const base = window.__BASE__ || '';
history.replaceState(null, '', `${base}/docs/${slug}`);
}, []);
useEffect(() => { loadDoc(active); }, []);
function navigate(slug) {
if (slug === active) return;
loadDoc(slug);
// Scroll to top
const el = document.querySelector('.docs-content');
if (el) el.scrollTop = 0;
}
// Extract document outline from markdown content
const outline = useMemo(() => {
if (!content) return [];
const headings = [];
let inCode = false;
for (const line of content.split('\n')) {
if (line.startsWith('```')) { inCode = !inCode; continue; }
if (inCode) continue;
const m = line.match(/^(#{1,4})\s+(.+)/);
if (m) {
const text = m[2].replace(/[*_`\[\]]/g, '');
const id = text.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '');
headings.push({ level: m[1].length, text, id });
}
}
return headings;
}, [content]);
function scrollToHeading(id) {
const el = document.getElementById(id);
if (el) el.scrollIntoView({ behavior: 'smooth', block: 'start' });
}
function retryList() {
setListError(false);
sw.api.get('/api/v1/docs').then(r => {
setDocs(Array.isArray(r) ? r : r.data || []);
}).catch(() => { setListError(true); });
}
return html`
<${Topbar} title="Documentation" />
<div class="docs-layout">
<nav class="docs-sidebar">
<h3 class="docs-sidebar__heading">Documentation</h3>
${listError ? html`
<div class="docs-error-inline">
<p>Failed to load docs list.</p>
<button onclick=${retryList} class="docs-retry-btn">Retry</button>
</div>
` : docs.map(d => html`
<a key=${d.slug}
class="docs-nav-item ${d.slug === active ? 'active' : ''}"
onclick=${() => navigate(d.slug)}
href="javascript:void(0)">
${d.title}
</a>
`)}
</nav>
<main class="docs-content">
${loading ? html`
<div class="docs-loading">Loading\u2026</div>
` : html`
<article class="docs-article"
dangerouslySetInnerHTML=${{ __html: renderMarkdown(content) }}>
</article>
`}
</main>
${!loading && outline.length > 2 ? html`
<aside class="docs-outline">
<h4 class="docs-outline__heading">On this page</h4>
${outline.map(h => html`
<a key=${h.id}
class="docs-outline__item docs-outline__item--h${h.level}"
onclick=${(e) => { e.preventDefault(); scrollToHeading(h.id); }}
href="#${h.id}">
${h.text}
</a>
`)}
</aside>
` : null}
</div>
`;
}
// ── Simple markdown renderer ─────────────────
// Handles: headings, code blocks, inline code, bold, italic, links, lists, paragraphs, tables, hr.
// Not a full CommonMark parser — good enough for documentation.
function renderMarkdown(md) {
if (!md) return '';
let html = '';
const lines = md.split('\n');
let i = 0;
let inCode = false;
let codeLang = '';
let codeLines = [];
let inList = false;
let listType = '';
let inTable = false;
let tableRows = [];
function esc(s) {
return s.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
function inline(s) {
s = esc(s);
// Links [text](url)
s = s.replace(/\[([^\]]+)\]\(([^)]+)\)/g, '<a href="$2" target="_blank" rel="noopener">$1</a>');
// Bold **text** or __text__
s = s.replace(/\*\*(.+?)\*\*/g, '<strong>$1</strong>');
s = s.replace(/__(.+?)__/g, '<strong>$1</strong>');
// Italic *text* or _text_
s = s.replace(/(?<!\*)\*([^*]+)\*(?!\*)/g, '<em>$1</em>');
// Inline code `code`
s = s.replace(/`([^`]+)`/g, '<code>$1</code>');
return s;
}
function flushTable() {
if (!inTable || tableRows.length === 0) return;
inTable = false;
html += '<table class="data-table"><thead><tr>';
const headers = tableRows[0];
for (const h of headers) html += `<th>${inline(h.trim())}</th>`;
html += '</tr></thead><tbody>';
for (let r = 2; r < tableRows.length; r++) {
html += '<tr>';
for (const cell of tableRows[r]) html += `<td>${inline(cell.trim())}</td>`;
html += '</tr>';
}
html += '</tbody></table>';
tableRows = [];
}
function flushList() {
if (!inList) return;
inList = false;
html += listType === 'ol' ? '</ol>' : '</ul>';
}
while (i < lines.length) {
const line = lines[i];
// Fenced code blocks
if (line.startsWith('```')) {
if (inCode) {
html += `<pre><code class="language-${esc(codeLang)}">${esc(codeLines.join('\n'))}</code></pre>`;
inCode = false;
codeLines = [];
codeLang = '';
} else {
flushList();
flushTable();
inCode = true;
codeLang = line.slice(3).trim();
}
i++;
continue;
}
if (inCode) {
codeLines.push(line);
i++;
continue;
}
// Table rows
if (line.includes('|') && line.trim().startsWith('|')) {
flushList();
const cells = line.split('|').slice(1, -1);
if (!inTable) inTable = true;
// Skip separator row (---|---)
if (cells.every(c => /^[\s:-]+$/.test(c))) {
tableRows.push(cells); // keep for row counting
} else {
tableRows.push(cells);
}
i++;
continue;
} else {
flushTable();
}
// Headings
const headingMatch = line.match(/^(#{1,6})\s+(.+)/);
if (headingMatch) {
flushList();
const level = headingMatch[1].length;
const text = headingMatch[2];
const id = text.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '');
html += `<h${level} id="${id}">${inline(text)}</h${level}>`;
i++;
continue;
}
// Horizontal rule
if (/^(-{3,}|\*{3,}|_{3,})$/.test(line.trim())) {
flushList();
html += '<hr>';
i++;
continue;
}
// Unordered list
if (/^\s*[-*+]\s+/.test(line)) {
if (!inList || listType !== 'ul') {
flushList();
inList = true;
listType = 'ul';
html += '<ul>';
}
html += `<li>${inline(line.replace(/^\s*[-*+]\s+/, ''))}</li>`;
i++;
continue;
}
// Ordered list
if (/^\s*\d+\.\s+/.test(line)) {
if (!inList || listType !== 'ol') {
flushList();
inList = true;
listType = 'ol';
html += '<ol>';
}
html += `<li>${inline(line.replace(/^\s*\d+\.\s+/, ''))}</li>`;
i++;
continue;
}
flushList();
// Empty line
if (line.trim() === '') {
i++;
continue;
}
// Paragraph
html += `<p>${inline(line)}</p>`;
i++;
}
flushList();
flushTable();
return html;
}
// ── Styles ──────────────────────────────────
const style = document.createElement('style');
style.textContent = `
.docs-layout {
display: flex;
flex: 1;
min-height: 0;
overflow: hidden;
gap: 0;
}
.docs-sidebar {
width: 220px;
min-width: 220px;
height: 100%;
padding: 20px 16px;
border-right: 1px solid var(--border);
background: var(--bg-secondary);
overflow-y: auto;
box-sizing: border-box;
}
.docs-sidebar__heading {
margin: 0 0 12px;
font-size: 14px;
color: var(--text-3);
text-transform: uppercase;
letter-spacing: 0.5px;
}
.docs-nav-item {
display: block;
padding: 6px 10px;
margin: 2px 0;
border-radius: 6px;
color: var(--text);
text-decoration: none;
font-size: 13px;
cursor: pointer;
transition: background 0.15s;
}
.docs-nav-item:hover {
background: var(--bg-hover);
}
.docs-nav-item.active {
background: var(--accent-dim);
color: var(--accent);
font-weight: 600;
}
.docs-content {
flex: 1;
height: 100%;
padding: 24px 40px;
max-width: 800px;
overflow-y: auto;
box-sizing: border-box;
}
.docs-loading {
padding: 40px;
text-align: center;
color: var(--text-3);
animation: docs-pulse 1.5s ease-in-out infinite;
}
@keyframes docs-pulse {
0%, 100% { opacity: 1; }
50% { opacity: 0.4; }
}
.docs-error-inline {
padding: 12px;
color: var(--text-3);
font-size: 13px;
text-align: center;
}
.docs-retry-btn {
margin-top: 8px;
padding: 4px 12px;
background: var(--bg-hover);
border: 1px solid var(--border);
border-radius: 6px;
color: var(--text);
font-size: 12px;
cursor: pointer;
}
.docs-retry-btn:hover { background: var(--bg-elevated); }
.docs-outline {
width: 200px;
min-width: 200px;
height: 100%;
padding: 20px 12px;
border-left: 1px solid var(--border);
overflow-y: auto;
box-sizing: border-box;
}
.docs-outline__heading {
margin: 0 0 10px;
font-size: 11px;
color: var(--text-3);
text-transform: uppercase;
letter-spacing: 0.5px;
font-weight: 600;
}
.docs-outline__item {
display: block;
padding: 3px 8px;
margin: 1px 0;
border-radius: 4px;
color: var(--text-2);
text-decoration: none;
font-size: 12px;
line-height: 1.4;
cursor: pointer;
transition: color 0.15s, background 0.15s;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.docs-outline__item:hover { color: var(--text); background: var(--bg-hover); }
.docs-outline__item--h1 { padding-left: 8px; font-weight: 600; color: var(--text); }
.docs-outline__item--h2 { padding-left: 8px; }
.docs-outline__item--h3 { padding-left: 20px; }
.docs-outline__item--h4 { padding-left: 32px; font-size: 11px; }
.docs-article h1, .docs-article h2, .docs-article h3, .docs-article h4 { scroll-margin-top: 16px; }
.docs-article h1 { font-size: 28px; margin: 0 0 16px; border-bottom: 1px solid var(--border); padding-bottom: 8px; color: var(--text); }
.docs-article h2 { font-size: 22px; margin: 24px 0 12px; color: var(--text); }
.docs-article h3 { font-size: 18px; margin: 20px 0 8px; color: var(--text); }
.docs-article p { margin: 8px 0; line-height: 1.6; color: var(--text); }
.docs-article pre {
background: var(--bg-code);
border: 1px solid var(--border);
border-radius: 6px;
padding: 12px 16px;
overflow-x: auto;
font-size: 13px;
line-height: 1.5;
margin: 12px 0;
}
.docs-article code {
background: var(--bg-code);
padding: 2px 5px;
border-radius: 3px;
font-size: 0.9em;
color: var(--text);
}
.docs-article pre code { background: none; padding: 0; }
.docs-article ul, .docs-article ol { margin: 8px 0; padding-left: 24px; }
.docs-article li { margin: 4px 0; line-height: 1.5; color: var(--text); }
.docs-article hr { border: none; border-top: 1px solid var(--border); margin: 24px 0; }
.docs-article a { color: var(--accent); }
.docs-article .data-table { width: 100%; border-collapse: collapse; margin: 12px 0; font-size: 13px; }
.docs-article .data-table th { text-align: left; padding: 8px 12px; border-bottom: 2px solid var(--border); color: var(--text); font-weight: 600; }
.docs-article .data-table td { padding: 6px 12px; border-bottom: 1px solid var(--border); color: var(--text-2); }
.docs-article .data-table tr:hover td { background: var(--bg-hover); }
.docs-article strong { color: var(--text); }
`;
document.head.appendChild(style);
// ── Mount ───────────────────────────────────
const mount = document.getElementById('docs-mount');
if (mount) {
mount.innerHTML = '';
render(preact.h(DocsSurface, null), mount);
}