Compare commits
23 Commits
d0207f336f
...
v0.6.7
| Author | SHA1 | Date | |
|---|---|---|---|
| fb5284f667 | |||
| 7915d84c8b | |||
| 81c28a50bf | |||
| 36d6158940 | |||
| 3d4228f868 | |||
| a887b4c78b | |||
| 1a7f41493d | |||
| 768f15b3cd | |||
| 4da25350ac | |||
| 8092f00fbe | |||
| 6931b125a4 | |||
| 7155aaf663 | |||
| 2abf406db8 | |||
| eb9a2d7d27 | |||
| 50d991001d | |||
| 6fbac0a33f | |||
| cee65c4136 | |||
| 31ab572c95 | |||
| 32beb3cee4 | |||
| 03c182b9d1 | |||
| 2c8dc59284 | |||
| 310048b7bb | |||
| d91ec02dd7 |
904
CHANGELOG.md
904
CHANGELOG.md
@@ -2,6 +2,891 @@
|
||||
|
||||
All notable changes to Switchboard Core are documented here.
|
||||
|
||||
## v0.6.7 — Native mTLS
|
||||
|
||||
End-to-end mutual TLS without a reverse proxy. Targets systemd+podman
|
||||
deployments where the Go binary terminates TLS itself.
|
||||
|
||||
### Added
|
||||
|
||||
- **`TLS_MODE` config**: Three values — `none` (default, plain HTTP), `server`
|
||||
(TLS, no client cert), `mtls` (mutual TLS, client cert required). Independent
|
||||
of `AUTH_MODE`. Combinations: `server`+`builtin` for HTTPS with password auth,
|
||||
`mtls`+`mtls` for full mTLS identity, `none`+`mtls` for proxy-terminated
|
||||
(existing behavior).
|
||||
- **TLS server mode**: Binary calls `ListenAndServeTLS` directly when
|
||||
`TLS_MODE` is `server` or `mtls`. TLS 1.3 minimum, no cipher suite
|
||||
configuration. Config: `TLS_CERT`, `TLS_KEY`, `TLS_CA` path env vars.
|
||||
- **`MTLSNativeProvider`**: New auth provider that reads
|
||||
`r.TLS.PeerCertificates[0]` directly — no header trust. `Subject.CommonName`
|
||||
becomes username, `sha256(cert.Raw)` becomes `external_id`. Auto-provisions
|
||||
users identically to the proxy provider. Selected when `AUTH_MODE=mtls` and
|
||||
`TLS_MODE=mtls`.
|
||||
- **Shared mTLS helpers**: `ParseDN()`, `FingerprintCert()`, and
|
||||
`resolveOrProvision()` extracted to `mtls_helpers.go`. Both proxy and native
|
||||
providers consume the same user resolution logic.
|
||||
- **Peer TLS config**: `BuildPeerTLSConfig()` constructs a `*tls.Config` for
|
||||
outbound node-to-node connections (forward-looking — cluster registry is
|
||||
currently DB-backed with no HTTP peer calls).
|
||||
- **`switchboard-ca.sh`**: Shell wrapper around openssl for cert provisioning.
|
||||
Three commands: `init` (CA keypair), `issue-node` (365d, ServerAuth +
|
||||
ClientAuth EKU), `issue-user` (90d, ClientAuth only). All ECDSA P-256, PEM
|
||||
output.
|
||||
- **12 new tests**: `FingerprintCert` determinism, native provider unit tests
|
||||
(nil TLS, empty certs, no CN, email extraction), TLS integration tests (no
|
||||
cert rejected, wrong CA rejected, expired cert rejected, valid cert accepted,
|
||||
peer certificate visibility).
|
||||
|
||||
### Changed
|
||||
|
||||
- **`MTLSProvider` → `MTLSProxyProvider`**: Renamed for clarity. Config type
|
||||
`MTLSConfig` → `MTLSProxyConfig`. Constructor `NewMTLSProvider` →
|
||||
`NewMTLSProxyProvider`. File `mtls.go` → `mtls_proxy.go`.
|
||||
|
||||
## v0.6.6 — Final Hardening
|
||||
|
||||
Final pass before public release. Security, correctness, and developer
|
||||
experience. No new features — only fixes, validation, and cleanup.
|
||||
|
||||
### Added
|
||||
|
||||
- **`ValidateManifest()` gate**: Centralized manifest validation function
|
||||
(`package_validate.go`) called at both upload-install and bundled-install
|
||||
time. Catches malformed manifests early with clear error messages. 12 unit
|
||||
tests covering all type constraints and edge cases.
|
||||
- **Extension dependency auto-activation**: Installing a package whose
|
||||
`dependencies` list an uninstalled library will auto-install that library
|
||||
from the bundled packages directory. If the dependency is not bundled, the
|
||||
error message lists exactly what's missing.
|
||||
- **`OptionalAuth` middleware**: New page middleware for workflow visitor
|
||||
routes. Authenticates if a token is present, allows anonymous pass-through
|
||||
otherwise. Replaces the stale `AuthOrRedirect` TODO for Session routes.
|
||||
- **Package signing schema reservation**: `signature` field accepted in
|
||||
manifest (no-op). `PACKAGE_VERIFY_SIGNATURES` env var logs warnings for
|
||||
unsigned packages when enabled. No cryptographic verification yet — schema
|
||||
slot reserved to prevent a breaking change later.
|
||||
- **ICD/SDK runner v0.6.x coverage**: Smoke tier adds metrics, cluster,
|
||||
backups, docs, and OpenAPI JSON endpoints. SDK admin domain adds metrics,
|
||||
cluster, and backups dual-path tests.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **OIDC nonce validation**: ID token nonce claim is now validated against
|
||||
the stored authorization request nonce. Previously the nonce was generated
|
||||
and stored but never checked on callback — a token replay/substitution
|
||||
vulnerability.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Schema migration stub**: `RunSchemaMigrations()` no longer pretends to
|
||||
work. Replaced with a log-only function documenting that only additive
|
||||
schema changes are supported. Downgrade rejection preserved.
|
||||
- **Session middleware TODO resolved**: `main.go` Session slot now uses
|
||||
`OptionalAuth` instead of `AuthOrRedirect`. OIDC nonce TODO and migration
|
||||
stub TODO also resolved.
|
||||
|
||||
## v0.6.5 — Renderer Pipeline + Docs Rewrite
|
||||
|
||||
Lifts block rendering to a kernel SDK primitive so all surfaces share one
|
||||
markdown pipeline. Rewrites docs for external audience. Adds CONTRIBUTING
|
||||
guide and extension tutorial.
|
||||
|
||||
### Added
|
||||
|
||||
- **`sw.renderers` SDK module**: Kernel-level renderer registry. Extensions
|
||||
register block renderers (fenced code blocks) and post renderers (DOM
|
||||
post-processing) once; all surfaces consume via `sw.markdown`.
|
||||
- **`sw.markdown` SDK module**: Unified markdown rendering via `marked` v16
|
||||
(vendored). Lazy-loads `marked` + DOMPurify. Custom code hook delegates
|
||||
fenced blocks to `sw.renderers`. Wikilink tokenizer (`[[Page Name]]`)
|
||||
built in for all surfaces.
|
||||
- **Browser extension script loader**: Server injects `<script>` tags for
|
||||
enabled browser-tier extensions into all pages. Extensions register via
|
||||
`sw:ready` event.
|
||||
- **Chat markdown support**: Messages with `content_type: 'markdown'` render
|
||||
via `sw.markdown`. Existing `text` messages stay plain (no behavior change).
|
||||
- **Mermaid architecture diagrams**: Six diagrams in `docs/ARCHITECTURE.md`:
|
||||
system overview, request flow, extension lifecycle, realtime events,
|
||||
settings cascade, cluster topology.
|
||||
- **`CONTRIBUTING.md`**: Development setup, project structure, test commands,
|
||||
code conventions.
|
||||
- **`docs/TUTORIAL-FIRST-EXTENSION.md`**: Step-by-step guide to building and
|
||||
installing a browser extension.
|
||||
- **`docs/DESIGN-WORKFLOWS.md`**: Clean workflow system design document.
|
||||
- **13 new renderers tests** (`src/js/__tests__/renderers.test.js`).
|
||||
|
||||
### Changed
|
||||
|
||||
- **Notes surface**: Hand-rolled markdown renderer replaced with
|
||||
`sw.markdown.renderSync()`. Post-renderers (mermaid, katex, etc.) now
|
||||
execute on preview. ~90 lines deleted.
|
||||
- **Docs surface**: Hand-rolled markdown renderer replaced with
|
||||
`sw.markdown.renderSync()`. ~160 lines deleted.
|
||||
- **4 renderer extensions** (mermaid, katex, csv-table, diff-viewer):
|
||||
Rewritten from dead `Extensions.register()` pattern to IIFE +
|
||||
`sw.renderers.register()` via `sw:ready` event. All rendering logic
|
||||
preserved.
|
||||
- **Surface alias routes removed**: Six `/api/v1/admin/surfaces/*` backward
|
||||
compat routes deleted from `main.go`. SDK and ICD runner migrated to
|
||||
canonical `/api/v1/admin/packages/*` endpoints.
|
||||
- **Docs rewritten for external audience**: All references to fork history,
|
||||
internal codenames, and pre-fork version numbers removed from docs,
|
||||
CHANGELOG, and ROADMAP.
|
||||
|
||||
### Removed
|
||||
|
||||
- `docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md` — replaced by `DESIGN-WORKFLOWS.md`.
|
||||
- Hand-rolled markdown renderers in Notes and Docs surfaces.
|
||||
- Six `/admin/surfaces/*` API alias routes.
|
||||
|
||||
---
|
||||
|
||||
## v0.6.4 — Admin Health/Metrics Tab + Cluster Merge
|
||||
|
||||
Structural consolidation: cluster dashboard merges into Admin as a Health tab.
|
||||
Comprehensive metrics endpoint for runtime, DB, cluster, and extension stats.
|
||||
|
||||
### Added
|
||||
|
||||
- **Admin Health tab**: New "Health" section under Monitoring in the Admin
|
||||
surface. Auto-refreshing panels for runtime, database, cluster, and extension
|
||||
metrics with configurable poll interval (5/10/30/60s).
|
||||
- **`GET /api/v1/admin/metrics`**: Single JSON endpoint returning all platform
|
||||
metrics — runtime (goroutines, heap, GC, uptime, FDs), DB pool (latency,
|
||||
active/idle/max, PG-only dead tuples + active backends), cluster nodes
|
||||
(when PG multi-node), and extension stats (Starlark exec/errors/duration,
|
||||
trigger fires, event bus publish/deliver counts).
|
||||
- **Fattened heartbeat payload**: Cluster heartbeat JSONB now carries stack
|
||||
usage, GC CPU%, extensions loaded, Starlark execution counters, and trigger
|
||||
fire count — visible in per-node cluster cards.
|
||||
- **Sandbox execution tracking**: `ExecPackage` and `CallEntryPoint` now
|
||||
increment atomic counters (exec, errors, duration) and the previously
|
||||
declared but unused `SandboxExecutionsTotal` Prometheus counter.
|
||||
- **Event bus counters**: `Bus.PublishCount()` and `Bus.DeliverCount()` track
|
||||
cumulative publish/deliver operations.
|
||||
- **Trigger fire counter**: `Engine.FireCount()` tracks cumulative trigger
|
||||
fires across webhook, event, and scheduled triggers.
|
||||
- 4 new handler tests (metrics SQLite shape, cluster shape, extension counters).
|
||||
- 1 new cluster test (fattened heartbeat payload).
|
||||
|
||||
### Changed
|
||||
|
||||
- **Health endpoint consolidation**: `/health` and `/api/v1/health` now use a
|
||||
shared `buildHealthResponse()` function returning identical JSON. Both now
|
||||
include `registration_enabled` when the database is connected.
|
||||
- **Block renderer `requires` removed**: `mermaid-renderer`, `katex-renderer`,
|
||||
`csv-table`, and `diff-viewer` no longer require `["chat"]` — they activate
|
||||
on any surface (notes, docs, etc.).
|
||||
|
||||
### Removed
|
||||
|
||||
- **`cluster-dashboard` package**: Standalone surface package retired. Cluster
|
||||
node visibility is now in Admin > Monitoring > Health.
|
||||
|
||||
---
|
||||
|
||||
## v0.6.3 — Dead Code Sweep + Registry Fix
|
||||
|
||||
Hardening: fix broken registry install, add registry settings UI,
|
||||
clean up dead code, narrow default bundle.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Registry install**: SDK was sending `{ url }` but the Go handler expects
|
||||
`{ download_url }`, causing every registry Install click to return 400.
|
||||
|
||||
### Added
|
||||
|
||||
- **Package Registry settings**: New "Package Registry" section in Admin >
|
||||
Settings with a URL input field for configuring the external registry.
|
||||
- **`scripts/generate-registry.sh`**: Shell script that reads `.pkg` ZIPs,
|
||||
extracts manifests, and emits a `registry.json` for self-hosted discovery.
|
||||
- **`docs/PACKAGE-REGISTRY.md`**: Documents registry JSON format, admin
|
||||
configuration, script usage, and self-hosting setup.
|
||||
|
||||
### Removed
|
||||
|
||||
- **Dead Go code**: Orphaned channel/message type comments from
|
||||
`interfaces.go`, unused `roleFilterType()` from `pages.go`, stale version
|
||||
comment from `main.go`.
|
||||
- **Dead vendor JS**: `marked.min.js` (40KB) and `purify.min.js` (22KB) —
|
||||
unused vendor copies with zero production imports. Removed cache entries
|
||||
from `sw.js`.
|
||||
- **`dev.html`**: 676-line standalone dev gallery, not referenced by anything.
|
||||
- **Stale version comments**: Stripped legacy version annotations across
|
||||
~140 files — standalone comments, inline suffixes, section headers, and
|
||||
config field docs. Reworded to describe features rather than reference
|
||||
obsolete internal version numbers.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Default bundle narrowed** from 10 packages to 5: `notes`, `chat`,
|
||||
`chat-core`, `mermaid-renderer`, `schedules`. All other packages remain
|
||||
available via `BUNDLED_PACKAGES=*` or explicit list.
|
||||
|
||||
---
|
||||
|
||||
## v0.6.2 — Docs Polish + Dynamic OpenAPI
|
||||
|
||||
Docs surface polish and dynamic OpenAPI spec with extension route merging.
|
||||
|
||||
### Added
|
||||
|
||||
- **Dynamic OpenAPI spec**: `GET /api/docs/openapi.json` builds a merged
|
||||
spec at request time — kernel endpoints + auto-generated stubs for every
|
||||
extension `api_routes` entry. Extensions may optionally declare `api_schema`
|
||||
in `manifest.json` for richer path items (params, body, response examples).
|
||||
- **`api_schema` manifest field**: Optional array in `manifest.json`. Malformed
|
||||
entries are logged and skipped — never blocks extension loading.
|
||||
- **Tests**: 7 new handler tests (zero extensions, stubs, rich schema,
|
||||
multi-extension, malformed schema, disabled exclusion, required fields).
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Dark mode contrast**: Replaced all hardcoded light-mode fallbacks in docs
|
||||
CSS with theme-aware CSS variables. Tables, code blocks, nav items, and
|
||||
headings now readable in dark mode.
|
||||
- **Docs loading state**: Replaced borrowed `settings-placeholder` animation
|
||||
with docs-specific pulse skeleton. Added error state with retry button.
|
||||
- **Docs routing**: Default route (`/docs`) now redirects to
|
||||
`/docs/GETTING-STARTED` instead of the blank `/:section` placeholder.
|
||||
- **Docs scrolling**: Content area now scrolls correctly — full document
|
||||
visible without text overflowing viewport.
|
||||
- **Docs icon**: Changed from 🧩 to 📖 in the user menu.
|
||||
- **Docs duplicate menu entry**: Docs was appearing twice (once from surfaces
|
||||
API, once from hardcoded items). Added `'docs'` to `CORE_IDS` filter.
|
||||
- **Swagger UI**: Updated to point at dynamic `/api/docs/openapi.json`
|
||||
endpoint. Static YAML preserved for backward compatibility.
|
||||
- **OpenAPI tests on Postgres**: `openapiTestStores()` was hardcoded to
|
||||
`sqlite.NewStores()` — caused `pq: syntax error at or near ","` in CI.
|
||||
Now uses `database.IsSQLite()` check like all other handler tests.
|
||||
|
||||
### API
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/api/docs/openapi.json` | GET | Dynamic merged OpenAPI 3.0 spec |
|
||||
|
||||
---
|
||||
|
||||
## v0.6.1 — Backup/Restore + Documentation
|
||||
|
||||
Operational tooling for data safety and extension authoring.
|
||||
|
||||
### Added
|
||||
|
||||
- **Backup/Restore**: Full-instance backup as `.swb` ZIP archive containing
|
||||
core tables (JSONL), ext_data tables, and package assets. Stream to client
|
||||
or save server-side. Restore wipes and rebuilds from archive. Dialect-neutral
|
||||
(works across SQLite and Postgres).
|
||||
- **Admin backup section**: New "Backup" tab under `/admin` with create,
|
||||
download, delete, and restore operations. Destructive restore requires
|
||||
confirmation dialog.
|
||||
- **Documentation surface**: Builtin surface at `/docs` with sidebar navigation
|
||||
and client-side markdown rendering. Ships with 5 documents: Getting Started,
|
||||
Extension Guide, API Reference, Deployment, Package Format.
|
||||
- **Docs API**: `GET /api/v1/docs` lists available documents, `GET /api/v1/docs/:name`
|
||||
returns raw markdown content. Authenticated (all users).
|
||||
- **Tests**: 6 backup handler tests + E2E script (`ci/e2e-backup-test.sh`).
|
||||
|
||||
### API
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/api/v1/admin/backup` | POST | Create backup (stream or `?store=true`) |
|
||||
| `/api/v1/admin/backups` | GET | List server-side backups |
|
||||
| `/api/v1/admin/backups/:name` | GET | Download backup |
|
||||
| `/api/v1/admin/backups/:name` | DELETE | Delete backup |
|
||||
| `/api/v1/admin/restore` | POST | Restore from `.swb` upload |
|
||||
| `/api/v1/docs` | GET | List documentation |
|
||||
| `/api/v1/docs/:name` | GET | Get document content |
|
||||
|
||||
---
|
||||
|
||||
## v0.6.0 — Cluster Registry + HA
|
||||
|
||||
MVP convergence point. PG-backed cluster registry for horizontal scaling —
|
||||
zero new infrastructure (no etcd/Consul/Redis).
|
||||
|
||||
### Added
|
||||
|
||||
- **Cluster registry**: `node_registry` UNLOGGED table with self-registration
|
||||
on startup (`INSERT ... ON CONFLICT DO UPDATE`), heartbeat tick (default 10s),
|
||||
stale sweep (default 30s threshold), and self-eviction (`os.Exit(1)` when
|
||||
swept by a peer — K8s restarts the process).
|
||||
- **Cluster admin API**: `GET /api/v1/admin/cluster` returns all registered
|
||||
nodes with runtime stats (goroutines, heap, GC, uptime, ws_clients) in the
|
||||
standard `{data: [...]}` envelope.
|
||||
- **Health endpoint cluster info**: `GET /health` and `GET /api/v1/health`
|
||||
now include `node_id` and `cluster: {size, peers, heartbeat_age_ms}` when
|
||||
running on Postgres with the registry active.
|
||||
- **Cluster dashboard**: `cluster-dashboard` admin surface package renders one
|
||||
card per node with live runtime stats. Auto-refreshes every 10s. Stats are
|
||||
JSONB — new keys render automatically without schema migration.
|
||||
- **Cluster config**: `CLUSTER_NODE_ID` (default hostname-PID),
|
||||
`CLUSTER_HEARTBEAT_INTERVAL` (default 10s), `CLUSTER_STALE_THRESHOLD`
|
||||
(default 30s), `CLUSTER_ENDPOINT` (Phase 2 mesh, auto-detect).
|
||||
- **3-replica E2E test**: `docker-compose-e2e.yml` now runs 3 replicas.
|
||||
`ci/e2e-cluster-test.sh` verifies registration, stale sweep on stop, and
|
||||
re-registration on restart.
|
||||
- **5 new tests**: 3 cluster registry unit tests (stats collection, start/stop
|
||||
lifecycle, node ID) + 2 handler tests (list nodes, empty response).
|
||||
- **Curated bundle expanded**: `chat` and `cluster-dashboard` added to the
|
||||
default bundle (now 10 packages). Production deployments can override with
|
||||
`BUNDLED_PACKAGES=notes,chat,chat-core,cluster-dashboard` for a lean set.
|
||||
|
||||
### Changed
|
||||
|
||||
- SQLite deployments are unaffected — cluster store is nil, all cluster code
|
||||
is guarded behind `database.IsPostgres() && stores.Cluster != nil`.
|
||||
Cluster dashboard gracefully shows "0 nodes registered" on SQLite.
|
||||
|
||||
## v0.5.5 — Upgrade Testing
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Bundled permission auto-grant on Postgres**: The `InstallBundledPackages`
|
||||
auto-grant SQL used SQLite-style `granted = 1` / `granted = 0` which fails
|
||||
on Postgres BOOLEAN columns. Now dialect-aware: `true`/`false` on Postgres,
|
||||
`1`/`0` on SQLite. This caused bundled extensions (notes, chat-core, etc.)
|
||||
to install with permissions not granted on Postgres deployments.
|
||||
- **E2E test API field corrections**: Fixed `ci/e2e-chat-test.sh` login field
|
||||
(`"username"` → `"login"`), token extraction (`"token"` → `"access_token"`),
|
||||
and profile endpoint (`/api/v1/me` → `/api/v1/profile`).
|
||||
|
||||
### Added
|
||||
|
||||
- **Upgrade test harness**: `docker-compose-upgrade.yml` + `ci/e2e-upgrade-test.sh`
|
||||
orchestrate a full upgrade cycle: build old image, seed data (notes,
|
||||
conversations, messages, settings), stop old, start new, verify data integrity
|
||||
post-upgrade. Covers auth, notes, conversations, packages, and settings.
|
||||
- **Rolling upgrade test**: `ci/e2e-upgrade-rolling.sh` tests rolling upgrade
|
||||
with 2 replicas sharing Postgres. Upgrades one replica at a time, verifies
|
||||
cross-replica reads during the mixed-version window.
|
||||
- **11 new Go tests** in `upgrade_test.go`:
|
||||
- Schema edge cases: add index (idempotent), add column (idempotent),
|
||||
multi-column add, row preservation across migration.
|
||||
- Settings migration: global/team/user overrides preserved across package
|
||||
update, new keys get defaults, removed keys not deleted.
|
||||
- Package compatibility: bundled skip-if-present on restart, dormant status
|
||||
for unmet requires, enabled/type preserved across version bump.
|
||||
- Direct `MigrateExtTables` tests: new table creation, column add with row
|
||||
preservation, index idempotency.
|
||||
|
||||
## v0.5.4 — Package Updates
|
||||
|
||||
### Added
|
||||
|
||||
- **Package update API**: `POST /api/v1/admin/packages/:id/update` accepts a
|
||||
`.pkg` archive, validates semver version bump (rejects same/older), applies
|
||||
additive schema migration, merges settings, replaces assets, and re-syncs
|
||||
permissions/triggers/dependencies.
|
||||
- **Package export API**: `GET /api/v1/admin/packages/:id/export` streams the
|
||||
installed package as a downloadable `.pkg` ZIP archive. Completes the manual
|
||||
rollback story: export before update, re-install old `.pkg` if needed.
|
||||
- **Semver parsing**: `ParseSemver` / `Compare` in `handlers/semver.go` with
|
||||
prerelease support. Used by the update handler to enforce version bumps.
|
||||
- **Additive schema migration**: `MigrateExtTables` diffs declared `db_tables`
|
||||
against existing ext_data schema. Adds new columns (`ALTER TABLE ADD COLUMN`)
|
||||
and new tables. No destructive changes — columns in DB but absent from
|
||||
manifest are preserved. `ListExtColumns` introspects via `PRAGMA table_info`
|
||||
(SQLite) or `information_schema` (Postgres).
|
||||
- **Settings merge on update**: New manifest setting keys get their declared
|
||||
default value; existing admin-configured values are preserved.
|
||||
- **Admin UI Update button**: Non-core package cards now show an Update button
|
||||
with a confirm dialog before upload.
|
||||
- **14 new Go tests**: Version bump, 5 rejection cases (same/older/core/type/id
|
||||
mismatch), schema add column with data preservation, schema add table,
|
||||
settings merge, export with valid zip, default allowlist, wildcard allowlist.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Curated default bundle**: Default bundled packages reduced from 23 to 8
|
||||
core packages (notes, chat-core, workflow-chat, dashboard, 4 demo workflows).
|
||||
`BUNDLED_PACKAGES=*` installs all; explicit comma-separated list for custom.
|
||||
Existing deployments are unaffected (install-once, skip-if-present).
|
||||
- **`BUNDLED_PACKAGES` env var**: Empty value now installs curated defaults
|
||||
instead of all packages. Use `"*"` for previous install-all behavior.
|
||||
|
||||
## v0.5.3 — Chat Polish + Integration Testing
|
||||
|
||||
### Added
|
||||
|
||||
- **`db.query()` search_like**: New kernel-level `search_like` kwarg for text
|
||||
search. Accepts `{col: pattern}` dict, generates OR-joined `LIKE` (SQLite) /
|
||||
`ILIKE` (Postgres) clauses. Composes with existing `filters` via AND. Reusable
|
||||
by any package. 5 new sandbox tests.
|
||||
- **Conversation search**: `GET /search?q=term` endpoint in `chat-core`. Searches
|
||||
conversation titles and message content, scoped to user's conversations.
|
||||
Returns `{conversations: [...], messages: [...]}`.
|
||||
- **Search UI**: Debounced search input in chat sidebar. Results view replaces
|
||||
conversation list showing matched conversations and message snippets with
|
||||
section headers. Click result to navigate, clear button to dismiss.
|
||||
- **`workflow-chat` library package**: `on_advance` hook that creates scoped
|
||||
group conversations when workflow stages require team collaboration. Adds all
|
||||
team members as participants, sends system message linking to the instance,
|
||||
enriches `stage_data` with `conversation_id`. Idempotent — skips if
|
||||
conversation already exists.
|
||||
- **Multi-user E2E test infrastructure**: `docker-compose-e2e.yml` with 2
|
||||
replicas, Postgres, and nginx load balancer. `ci/e2e-chat-test.sh` shell
|
||||
script covering auth, conversation CRUD, message send/read, cross-replica
|
||||
consistency, search, and pagination. `ci/e2e-ws-listener.js` Node.js
|
||||
WebSocket client for realtime event testing.
|
||||
- **Workflow hook tests**: 6 new Go tests for `parseOnAdvanceResult` covering
|
||||
None, nil, error, enriched stage_data, non-dict, and nested structure cases.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Message pagination polish**: Scroll position now preserved when loading older
|
||||
messages — records `scrollHeight` before prepend, restores via
|
||||
`requestAnimationFrame`. Loading spinner shown at top of thread during fetch.
|
||||
"Load older messages" button hidden while loading.
|
||||
- **chat-core** bumped to v0.2.0 (new search endpoint).
|
||||
- **chat** bumped to v0.2.0 (search UI, pagination polish).
|
||||
|
||||
## v0.5.2 — Chat Surface
|
||||
|
||||
### Added
|
||||
|
||||
- **`chat` surface package**: Full messaging UI built on `chat-core` library.
|
||||
Route `/s/chat`, icon `💬`, depends on `chat-core`.
|
||||
- **Conversation list sidebar**: Conversations sorted by recent activity, last
|
||||
message preview, relative timestamps, unread count badges. "New" button
|
||||
for conversation creation.
|
||||
- **Message thread**: Paginated message history with cursor-based load-more on
|
||||
scroll-to-top. Participant avatars, display names, timestamps. Scroll-to-
|
||||
bottom on new messages.
|
||||
- **Compose bar**: Auto-resizing textarea, Enter-to-send, Shift+Enter for
|
||||
newline. Typing indicator broadcast on keypress (3s debounce).
|
||||
- **Participant sidebar**: Collapsible right panel with participant list, online
|
||||
status via presence hub, role badges. Add participant via `sw.ui.Dialog` +
|
||||
`sw.ui.UserPicker`. Remove participant with confirmation.
|
||||
- **Create conversation dialog**: Group or Direct Message type selector. Multi-
|
||||
user picker with chip display for group, single picker for DM. Auto-title
|
||||
from participant names.
|
||||
- **Typing indicators**: Realtime broadcast via `POST /typing/:id` Starlark
|
||||
endpoint. Frontend subscribes to `conversation:{id}` channel, shows
|
||||
"X is typing..." with 4s auto-expire. Handles multiple typers.
|
||||
- **Read receipts**: Auto mark-read on conversation focus and new incoming
|
||||
messages. Unread counts in conversation list, cleared on selection.
|
||||
- **Message editing**: Inline edit mode on own messages with textarea. Save on
|
||||
Enter, cancel on Escape. "(edited)" label on edited messages. Realtime
|
||||
propagation to other participants.
|
||||
- **Message deletion**: Confirm dialog before delete. "This message was deleted"
|
||||
placeholder. Admin can delete any message. Realtime propagation.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Stale token login deadlock**: REST client's 401→refresh→retry loop caused
|
||||
a deadlock when the refresh endpoint itself returned 401 (stale DB). Auth
|
||||
endpoints (`/auth/login`, `/auth/refresh`, `/auth/register`) are now excluded
|
||||
from the retry loop. Defensive 8s timeout on `auth.boot()` ensures login
|
||||
page always renders even on unexpected hangs.
|
||||
- **Bundled package permissions**: Bundled packages were installed with
|
||||
`pending_review` status and `granted=0` permissions, requiring manual admin
|
||||
approval. Bundled installer now auto-grants all declared permissions and
|
||||
sets status to `active` on install.
|
||||
- **Creator display name**: `chat-core.create()` now accepts
|
||||
`creator_display_name` so the conversation creator shows a readable name
|
||||
in the participant sidebar instead of a raw UUID.
|
||||
- **Chat dark mode theming**: Replaced non-existent CSS variable names with
|
||||
actual theme tokens from `variables.css`. Fixes invisible text and wrong
|
||||
backgrounds in dark mode.
|
||||
- **Dialog dropdown clipping**: UserPicker autocomplete dropdown in dialogs
|
||||
(New Conversation, Add Participant) no longer clipped by `overflow: auto`
|
||||
on `.sw-dialog__body`.
|
||||
- **User menu at scaled UI**: Menu primitive now divides anchor coordinates
|
||||
and viewport dimensions by `sw.shell.getScale()` to compensate for CSS
|
||||
`transform: scale()` on `#surfaceInner`. Fixes menu clipping at 110%+.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Named Docker volume**: `docker-compose.yml` switched from bind mount
|
||||
(`./data:/data`) to named volume (`sb_data:/data`). Fixes volume mount
|
||||
failures when Docker daemon runs on a remote host (DinD, k8s pod).
|
||||
Deterministic wipe via `docker compose down -v`.
|
||||
|
||||
## v0.5.1 — Chat Core Library
|
||||
|
||||
### Added
|
||||
|
||||
- **`chat-core` library package**: Conversations, messages, participants, and
|
||||
read cursors as ext_data tables. Permissions: `db.write`, `realtime.publish`.
|
||||
Consumable by other packages via `lib.require("chat-core")`.
|
||||
- **4 ext_data tables**: `conversations` (title, type, created_by, updated_at),
|
||||
`participants` (conversation_id, participant_id, type, display_name, role,
|
||||
joined_at), `messages` (conversation_id, participant_id, content, content_type,
|
||||
edited_at), `read_cursors` (conversation_id, participant_id,
|
||||
last_read_message_id).
|
||||
- **6 exported Starlark functions**: `create()`, `send()`, `history()`,
|
||||
`add_participant()`, `remove_participant()`, `mark_read()`.
|
||||
- **15 REST endpoints**: Full CRUD for conversations, messages, participants.
|
||||
Cursor-based paginated message history. Per-conversation unread counts.
|
||||
- **Realtime events**: `message`, `message.edited`, `message.deleted`,
|
||||
`participant.added`, `participant.removed` published to
|
||||
`conversation:{id}` channels.
|
||||
- **`db.query()` range parameters**: New `before` and `after` kwargs accept
|
||||
dicts of `{col: val}` generating `col < ?` / `col > ?` WHERE clauses.
|
||||
Enables cursor-based pagination and efficient unread counting for all
|
||||
extension packages. 7 new Go tests.
|
||||
|
||||
### Security
|
||||
|
||||
- **Invisible Unicode scanning gate** (GlassWorm / Trojan Source defense):
|
||||
`ScanSource()` detects variation selectors, bidi overrides, zero-width chars,
|
||||
tag characters, and other invisible Unicode across 10 ranges. `Verdict()`
|
||||
blocks GlassWorm payloads (>=10 variation selector / tag chars), Trojan Source
|
||||
(any bidi override), and excessive zero-width chars (>=6). Two enforcement
|
||||
points: extension install (422 rejection before files written) and Starlark
|
||||
execution (belt-and-suspenders). 23 new tests including GlassWorm payload
|
||||
simulation.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Admin packages type filter**: Added `Library` option alongside Surface/Extension/Full/Workflow. Filter dropdown replaced button group with themed `Dropdown` primitive (keyboard nav, CSS-var theming, consistent with rest of admin UI).
|
||||
- SDK version bumped from 0.5.0 to 0.5.1.
|
||||
|
||||
## v0.5.0 — Realtime Primitive + Dialog Audit + Permissions UI
|
||||
|
||||
### Added
|
||||
|
||||
- **Realtime Starlark module**: `realtime.publish(channel, event, data)` lets
|
||||
extensions publish events to WebSocket channels. Gated by the new
|
||||
`realtime.publish` permission. Payloads limited to 7KB (pg_notify safe).
|
||||
Reserved event prefixes (`system.`, `workflow.`, etc.) are blocked.
|
||||
- **WS room protocol**: Clients send `room.subscribe` / `room.unsubscribe`
|
||||
messages to join/leave rooms. Intercepted in the WebSocket readPump before
|
||||
the bus publish gate (like ping). Per-connection cap of 100 rooms.
|
||||
- **SDK realtime module**: `sw.realtime.subscribe(channel, [event], callback)`
|
||||
manages room join/leave over WebSocket and filters incoming `realtime.*`
|
||||
events by room. Returns unsubscribe handle. Auto re-joins all rooms on
|
||||
WebSocket reconnect. `sw.realtime.channels()` debug helper.
|
||||
- **Admin permissions UI**: Packages page gains a "Permissions" button for
|
||||
packages with declared permissions. Inline drawer shows each permission with
|
||||
Grant/Revoke toggle, granted-by user ID, and a "Grant All" bulk action.
|
||||
- **Status badges**: `pending_review` (amber) and `suspended` (red) badges on
|
||||
package rows. Enable toggle disabled with hint when `pending_review`.
|
||||
- **SDK API client**: `permissions()`, `grantPerm()`, `revokePerm()`,
|
||||
`grantAllPerms()` methods added to `sw.api.admin.packages`.
|
||||
- **`starlarkToGoVal()` helper**: Converts Starlark values to Go `any` for
|
||||
JSON marshaling (reverse of existing `goValToStarlark`).
|
||||
- **8 new Go tests**: Route table cases for `realtime.*` and `room.*`,
|
||||
realtime module publish (happy path, empty data, missing args, reserved
|
||||
prefix, payload too large), `starlarkToGoVal` type conversion.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Dialog audit**: 5 bare `confirm()` calls migrated to `await sw.confirm()`
|
||||
with `{ destructive: true }` styling in tasks, schedules, notes (×2), and
|
||||
editor packages. Package archives rebuilt.
|
||||
- SDK version bumped from 0.2.3 to 0.5.0.
|
||||
- Event route table: `realtime.` → DirToClient, `room.subscribe` /
|
||||
`room.unsubscribe` → DirFromClient.
|
||||
- Runner gains `bus` field and `SetBus()` method; wired in main.go.
|
||||
|
||||
## v0.4.7 — Note Graph
|
||||
|
||||
### Added
|
||||
|
||||
- **Graph API**: `GET /graph` endpoint returns all non-archived notes as nodes
|
||||
and resolved wikilinks as edges in a single payload. Nodes include id, title,
|
||||
folder_id, and tags. Edges include source, target, and link text.
|
||||
- **Graph view**: Canvas-based force-directed graph visualization replaces the
|
||||
editor pane when "Graph" button is clicked in the topbar. Force simulation
|
||||
uses repulsion (all pairs), attraction (edges), and center gravity with
|
||||
velocity damping. Supports zoom (scroll wheel) and pan (mouse drag).
|
||||
- **Click focus**: Single-click a node to dim unconnected nodes/edges to 15%
|
||||
opacity; clicked node and direct neighbors stay at full brightness. Edges
|
||||
between focused nodes highlighted. Click empty space to reset.
|
||||
- **Shift+click chain**: Hold shift and click a second node to union both
|
||||
neighborhoods — trace thought paths across two hops.
|
||||
- **Double-click open**: Double-click a graph node to navigate to that note in
|
||||
the editor. Graph view closes, editor opens with the selected note.
|
||||
- **Hover tooltips**: Hovering a node shows title, tag pills, and connection
|
||||
count in an HTML overlay (no additional API calls).
|
||||
- **Folder coloring**: Nodes colored by folder using a 10-color palette.
|
||||
Unfiled notes shown in gray.
|
||||
- **Tag filter integration**: Active tag filter dims non-matching nodes to 30%
|
||||
opacity, stacking with click focus.
|
||||
- **Orphan highlighting**: Notes with zero connections shown with dashed stroke.
|
||||
"Hide orphans" checkbox in toolbar to toggle visibility.
|
||||
|
||||
### Changed
|
||||
|
||||
- Notes package version bumped from 0.7.0 to 0.8.0.
|
||||
- Topbar gains a "Graph" toggle button alongside "New Note" and "Import .md".
|
||||
|
||||
|
||||
## v0.4.6 — Sidebar Restructure
|
||||
|
||||
### Added
|
||||
|
||||
- **Sidebar tabs**: Two-tab header ("Notes" / "Outline") replaces the static
|
||||
sidebar header. Notes tab contains folder tree, tag filter, search, and note
|
||||
list. Outline tab shows the document heading tree for the active note.
|
||||
Outline tab disabled (grayed) when no note is selected.
|
||||
- **Sidebar outline**: Document outline relocated from editor-side panel into
|
||||
the sidebar's Outline tab. Nested heading hierarchy (H1→H2→H3) with depth
|
||||
indentation via `padding-left`. Click heading scrolls to it in both preview
|
||||
and CM6 editor modes.
|
||||
- **Active heading tracking**: Scroll listener (RAF-throttled) highlights the
|
||||
current heading in the sidebar outline based on scroll position. Uses
|
||||
`getBoundingClientRect()` in preview/split mode and CM6 viewport line in
|
||||
edit mode.
|
||||
|
||||
### Changed
|
||||
|
||||
- Notes package version bumped from 0.6.0 to 0.7.0.
|
||||
- `EditorPane` exposes headings, scroll function, and active heading index to
|
||||
parent via callback props (`onHeadingsChange`, `scrollToHeadingRef`,
|
||||
`onActiveHeadingChange`).
|
||||
- Sidebar auto-resets to Notes tab when active note is deleted.
|
||||
|
||||
### Removed
|
||||
|
||||
- Old `DocumentOutline` component and `.doc-outline` CSS styles (replaced by
|
||||
sidebar outline).
|
||||
|
||||
## v0.4.5 — Editor Modes + Document Outline
|
||||
|
||||
### Added
|
||||
|
||||
- **Tri-state view mode**: Notes open in rendered (read-only) mode by default.
|
||||
Toolbar button cycles through Edit (CM6) and Split (side-by-side) modes.
|
||||
Replaces the old binary preview toggle.
|
||||
- **Split view**: Side-by-side CM6 editor + rendered preview using CSS grid.
|
||||
CM6 instance preserved when toggling between edit and split modes.
|
||||
- **Synced scroll**: In split mode, scrolling the CM6 editor proportionally
|
||||
scrolls the preview pane via linear ratio mapping on `scrollDOM`.
|
||||
- **View mode persistence**: User's preferred mode saved to localStorage
|
||||
(`sw.storage.local('notes')`). New `editor_mode` manifest setting with
|
||||
admin-level default (rendered / edit / split).
|
||||
- **Document outline**: Collapsible TOC panel adjacent to the editor body.
|
||||
`parseHeadings()` extracts heading level, text, and line number (skips
|
||||
fenced code blocks). Click heading to scroll — uses CM6
|
||||
`EditorView.scrollIntoView()` in edit/split mode, DOM `scrollIntoView()`
|
||||
in rendered mode. Updates on body change with 300ms debounce.
|
||||
|
||||
### Changed
|
||||
|
||||
- Notes package version bumped from 0.5.0 to 0.6.0.
|
||||
- Editor body wrapped in `.notes-editor__content` flex container to
|
||||
accommodate the outline panel.
|
||||
- Wikilink click handling active in both rendered and split modes.
|
||||
- Mobile: split view collapses to single column, outline panel hidden.
|
||||
|
||||
---
|
||||
|
||||
## v0.4.4 — Rich Editor + Import/Export
|
||||
|
||||
### Added
|
||||
|
||||
- **CodeMirror 6 integration**: Notes editor now uses the vendored CM6 bundle
|
||||
(`CM.noteEditor()`) for rich markdown editing with syntax highlighting,
|
||||
wikilink autocomplete (`[[` triggers note title completion), and inline
|
||||
preview decorations for headings, code blocks, and blockquotes.
|
||||
- **CM6 dynamic loading**: Bundle loaded via `<script>` tag at boot time.
|
||||
Falls back to plain textarea if CM6 is unavailable.
|
||||
- **Export as .md**: Export button in editor header downloads the note as a
|
||||
Markdown file with YAML frontmatter (title, tags, created date).
|
||||
- **Import .md**: Import button in topbar allows uploading `.md`/`.markdown`/`.txt`
|
||||
files. Parses YAML frontmatter for title and tags, creates note via API.
|
||||
- **CSS for CM6**: Editor fills the container with `max-height: none` override
|
||||
and proper scroller padding.
|
||||
|
||||
### Changed
|
||||
|
||||
- Notes package version bumped from 0.4.0 to 0.5.0.
|
||||
- `EditorPane` component refactored to support CM6 with mutable refs for
|
||||
title/body (required for CM6 callback closures).
|
||||
- Ctrl/Cmd+S save works in both CM6 and textarea fallback modes.
|
||||
|
||||
---
|
||||
|
||||
## v0.4.3 — Backlinks + Wikilinks
|
||||
|
||||
### Added
|
||||
|
||||
- **Links table** (`ext_notes_links`): source_id, target_id, link_text columns
|
||||
with indexes on source_id and target_id for bidirectional lookup.
|
||||
- **Wikilink extraction**: `_extract_wikilinks()` parses `[[...]]` patterns
|
||||
using `split("[[")` + `find("]]")` (Starlark has no regex or while loops).
|
||||
Returns deduplicated list of link text strings.
|
||||
- **Link sync on save**: `_sync_links()` called from `_create_note` and
|
||||
`_update_note` when body changes. Follows the tags delete-all + reinsert
|
||||
pattern. Resolves link text to note IDs via case-insensitive title matching.
|
||||
Unresolved links stored with empty `target_id`.
|
||||
- **Cascade delete**: Hard-deleting a note removes both outgoing links
|
||||
(source_id matches) and incoming backlinks (target_id matches).
|
||||
- **Link API endpoints**: `GET /links/:note_id` returns outgoing links.
|
||||
`GET /backlinks/:note_id` returns incoming links enriched with source note
|
||||
titles to avoid extra frontend round-trips.
|
||||
- **Wikilink preview rendering**: `[[Note Title]]` syntax renders as clickable
|
||||
accent-colored links in the markdown preview. Unresolved wikilinks styled
|
||||
in danger/red with dashed underline.
|
||||
- **Wikilink click navigation**: Clicking a resolved wikilink navigates to the
|
||||
target note. Clicking an unresolved wikilink creates the note and navigates
|
||||
to it, then re-saves the source note so the link resolves to blue.
|
||||
- **Backlinks panel**: Collapsible panel below the editor showing all notes
|
||||
that link to the current note. Each backlink displays source title and link
|
||||
text. Click to navigate. Hidden when no backlinks exist.
|
||||
- **Updated stats**: `/stats` response now includes `links` count.
|
||||
|
||||
### Data model
|
||||
|
||||
| Table | Columns |
|
||||
|-------|---------|
|
||||
| `ext_notes_links` | source_id, target_id, link_text |
|
||||
|
||||
### Notes package version
|
||||
|
||||
0.3.0 → 0.4.0
|
||||
|
||||
---
|
||||
|
||||
## v0.4.2 — Tags + Search
|
||||
|
||||
### Added
|
||||
|
||||
- **Tags table** (`ext_notes_tags`): note_id, tag columns with indexes on
|
||||
both for bidirectional lookup. Many-to-many relationship via join table.
|
||||
- **Tag CRUD API**: 3 new Starlark handlers — list all unique tags, get tags
|
||||
for a note, replace tags for a note (delete-all + reinsert). Tags are
|
||||
normalized to lowercase, trimmed, and deduplicated on save.
|
||||
- **Tags in list/get/search**: `_list_notes` batch-fetches all tags in a
|
||||
single query and attaches a `tags` array to each note item. `_get_note`
|
||||
includes tags. `_search_notes` now matches search terms against tags.
|
||||
- **Tag cascade delete**: Hard-deleting a note also removes its tag rows.
|
||||
- **Tag input in editor**: `TagInput` component with removable pills, text
|
||||
input (comma or Enter to add), and autocomplete dropdown populated from
|
||||
all existing tags.
|
||||
- **Tag pills on note cards**: `NoteCard` renders up to 3 tag pills with
|
||||
"+N" overflow indicator for notes with many tags.
|
||||
- **Tag filter in sidebar**: `TagFilter` component displays all unique tags
|
||||
as clickable pills. Clicking toggles client-side filtering of the note
|
||||
list by that tag.
|
||||
- **Drag-and-drop note moves**: `NoteCard` is draggable (HTML5 drag/drop).
|
||||
`FolderNode`, "All Notes", and "Unfiled" items are drop targets. On drop,
|
||||
calls the existing move-note API.
|
||||
- **Folder context menu**: Replaced `prompt()` hack with a proper right-click
|
||||
popup menu showing "Add subfolder", "Rename", and "Delete" actions. Positioned
|
||||
at click coordinates, dismissed on outside click.
|
||||
- **Updated stats**: `/stats` response now includes `tags` count (unique).
|
||||
|
||||
### Data model
|
||||
|
||||
| Table | Columns |
|
||||
|-------|---------|
|
||||
| `ext_notes_tags` | note_id, tag |
|
||||
|
||||
### Notes package version
|
||||
|
||||
Bumped from 0.2.0 → 0.3.0. 15 API routes (12 existing + 3 new).
|
||||
|
||||
## v0.4.1 — Folders + Navigation Tree
|
||||
|
||||
### Added
|
||||
|
||||
- **Folders table** (`ext_notes_folders`): name, parent_id, creator_id,
|
||||
sort_order. Indexed on parent_id and creator_id. Nested hierarchy via
|
||||
parent_id foreign key (empty string = root).
|
||||
- **Folder CRUD API**: 5 new Starlark handlers — list, create, update, delete
|
||||
folders plus a dedicated move-note endpoint (`POST /notes/move`).
|
||||
- **Navigation tree**: `FolderTree` + `FolderNode` Preact components in the
|
||||
sidebar. Flat API response built into a nested tree client-side via `useMemo`.
|
||||
Expand/collapse toggles, depth-based indentation, inline rename via
|
||||
right-click context menu.
|
||||
- **Folder filtering**: Clicking a folder filters notes by `folder_id` query
|
||||
param. "All Notes" clears the filter. "Unfiled" shows notes with empty
|
||||
folder_id (client-side filter).
|
||||
- **Editor folder select**: Dropdown in the editor header moves notes between
|
||||
folders. Shows nested hierarchy with `└` prefix indentation.
|
||||
- **Folder-aware note creation**: New notes inherit the active folder's ID.
|
||||
- **Delete cascade**: Deleting a folder orphans its notes (moves to Unfiled)
|
||||
and reparents child folders to the deleted folder's parent.
|
||||
- **Updated stats**: `/stats` response now includes `unfiled` and `folders`
|
||||
counts.
|
||||
|
||||
### Data model
|
||||
|
||||
| Table | Columns |
|
||||
|-------|---------|
|
||||
| `ext_notes_folders` | name, parent_id, creator_id, sort_order |
|
||||
|
||||
### Notes package version
|
||||
|
||||
Bumped from 0.1.0 → 0.2.0. 12 API routes (7 existing + 5 new).
|
||||
|
||||
## v0.4.0 — Notes Surface
|
||||
|
||||
### Added
|
||||
|
||||
- **Notes surface package** (`packages/notes/`): Obsidian-style markdown notes
|
||||
rebuilt as a standard installable `.pkg` archive. Type `full`, tier `starlark`.
|
||||
Zero kernel changes — proves the extension stack end-to-end.
|
||||
- **Starlark backend** (`script.star`): 7 API routes — CRUD, search, stats.
|
||||
Note bodies stored in `ext_notes_notes` TEXT columns. List endpoint returns
|
||||
lightweight projections (no body); full content fetched on select.
|
||||
- **Markdown editor**: Preact+htm frontend with sidebar note list, monospace
|
||||
textarea, and toggle-able live preview. Inline markdown renderer (~100 lines)
|
||||
covers headings, bold, italic, code blocks, links, lists, blockquotes, and HR.
|
||||
- **Auto-save**: Debounced save (1s) with dirty/saved status indicator.
|
||||
`Ctrl/Cmd+S` for force save. Tab key inserts two spaces.
|
||||
- **Pin & archive**: Pin important notes to the top of the list. Archive
|
||||
(soft-delete) via the delete button; hard delete with `?hard=1` query param.
|
||||
- **Search**: Case-insensitive full-text search across title and body content.
|
||||
Search bar in sidebar filters the note list in real-time.
|
||||
- **Theme support**: All styles use CSS variables — light and dark themes work
|
||||
out of the box. Responsive layout collapses to vertical on narrow viewports.
|
||||
|
||||
### Data model
|
||||
|
||||
| Table | Columns |
|
||||
|-------|---------|
|
||||
| `ext_notes_notes` | title, body, folder_id, creator_id, updated_at, pinned, archived |
|
||||
|
||||
Indexes on `folder_id`, `creator_id`, `pinned`, `updated_at`.
|
||||
|
||||
### Planned (v0.4.1–v0.4.4)
|
||||
|
||||
- Folders + navigation tree
|
||||
- Tags + enhanced search
|
||||
- Backlinks + `[[wikilinks]]`
|
||||
- Rich editor (CodeMirror 6) + markdown import/export
|
||||
|
||||
## v0.3.8 — Distribution
|
||||
|
||||
### Added
|
||||
|
||||
- **Bundled packages**: Production Docker image now ships with 12 pre-built
|
||||
`.pkg` archives (4 workflows, 3 surfaces, 1 full, 1 library, 2 test runners,
|
||||
1 extension). Auto-installed on first boot via `InstallBundledPackages()`.
|
||||
Install-once, skip-if-present — admin uninstalls are respected on restart.
|
||||
- **Package allowlist** (`BUNDLED_PACKAGES`): Comma-separated list of package
|
||||
IDs to install. Empty (default) installs all. Useful for Helm/K8s where
|
||||
different environments need different packages.
|
||||
- **Skip bundled** (`SKIP_BUNDLED_PACKAGES=true`): Disables auto-install
|
||||
entirely. Intended for production until post-MVP packages are ready.
|
||||
- **Builder image** (`Dockerfile.builder`): Pre-caches Go modules, Node
|
||||
dependencies, and vendor lib tarballs for faster custom builds.
|
||||
- **Distribution docs** (`docs/DISTRIBUTION.md`): Quick start, bundled
|
||||
packages, builder image usage, custom build guide, production deployment
|
||||
reference.
|
||||
- **Migration 012**: Adds `bundled` to `packages.source` CHECK constraint
|
||||
(both Postgres and SQLite).
|
||||
- **K8s manifest**: `SKIP_BUNDLED_PACKAGES` and `BUNDLED_PACKAGES` env vars
|
||||
added to `k8s/switchboard.yaml`.
|
||||
- **Tests**: 6 handler tests (fresh install, skip existing, missing dir, empty
|
||||
dir, dormant handling, allowlist filtering).
|
||||
|
||||
### Environment defaults
|
||||
|
||||
| Environment | `SKIP_BUNDLED_PACKAGES` | `BUNDLED_PACKAGES` |
|
||||
|-------------|------------------------|--------------------|
|
||||
| Dev (compose) | `false` | (empty = all) |
|
||||
| Test (K8s) | `false` | (empty = all) |
|
||||
| Prod (K8s) | `true` | — |
|
||||
|
||||
## v0.3.7 — Package Audit
|
||||
|
||||
### Added
|
||||
@@ -294,8 +1179,8 @@ All notable changes to Switchboard Core are documented here.
|
||||
- **Dead `history_mode` in stage UI**: Removed history mode selector and state
|
||||
from StageForm. Backend column retained for v0.3.x schema migration.
|
||||
- **Dead ICD tests**: Removed assertions for `/teams/:teamId/personas`,
|
||||
`/teams/:teamId/providers`, and `/teams/:teamId/models` — endpoints were
|
||||
removed in Phase 0 fork.
|
||||
`/teams/:teamId/providers`, and `/teams/:teamId/models` — endpoints no
|
||||
longer exist in the kernel.
|
||||
- **Stale `chat_only` stage mode**: Frontend `STAGE_MODES` updated from
|
||||
`['chat_only', 'form_only', 'form_chat', 'review']` to
|
||||
`['form_only', 'form_chat', 'review', 'custom']` matching the backend
|
||||
@@ -389,13 +1274,13 @@ All notable changes to Switchboard Core are documented here.
|
||||
(v0.2.7 admin, v0.2.8 user, v0.2.9 team-admin pass 1).
|
||||
- Updated bus doc examples and test labels from `chat.*` to `workflow.*`.
|
||||
- Renamed `channel-` prefixed test paths to `test-` in storage tests.
|
||||
- Updated doc comments throughout to remove references to gutted surfaces.
|
||||
- Updated doc comments throughout to reflect current surface architecture.
|
||||
|
||||
### Removed
|
||||
|
||||
- **~500 lines of dead CSS**: Orphaned classes for gutted chat, channel,
|
||||
project, notes, editor-chat, sidebar, and router-picker features from
|
||||
`layout.css` and `surfaces.css`.
|
||||
- **~500 lines of dead CSS**: Orphaned classes for removed legacy UI
|
||||
components (chat, channel, project, notes, editor-chat, sidebar,
|
||||
router-picker) from `layout.css` and `surfaces.css`.
|
||||
- **Dead Go types**: `Grant` struct (persona-era), `CompositeModelKey` func,
|
||||
comment-only stubs for NoteGraph, ProjectChannel, etc.
|
||||
- **Dead event code**: `chat.typing.*` / `channel.typing.*` condition in WS
|
||||
@@ -431,7 +1316,7 @@ All notable changes to Switchboard Core are documented here.
|
||||
### Changed
|
||||
|
||||
- **UserMenu**: Surface list now driven entirely by the `/api/v1/surfaces`
|
||||
API. Removed hardcoded Chat, Notes, Projects links (gutted in Phase 0).
|
||||
API. Removed hardcoded surface links; menu is now fully dynamic.
|
||||
Core surfaces (Admin, Settings, Team Admin, Workflow) filtered from the
|
||||
API list and handled as dedicated menu items with RBAC gating.
|
||||
- **Tasks surface**: Replaced custom `.tasks-header` with `sw.shell.Topbar`.
|
||||
@@ -582,7 +1467,8 @@ All notable changes to Switchboard Core are documented here.
|
||||
|
||||
## [v0.1.0] — 2026-03-26
|
||||
|
||||
Forked from chat-switchboard v0.38.5. Gutted to a pure extension platform.
|
||||
Initial release. Pure extension platform — kernel provides auth, RBAC,
|
||||
storage, and the Starlark sandbox. Everything else is a package.
|
||||
|
||||
### Removed
|
||||
|
||||
@@ -618,7 +1504,7 @@ Forked from chat-switchboard v0.38.5. Gutted to a pure extension platform.
|
||||
|
||||
### Changed
|
||||
|
||||
- Module renamed: `chat-switchboard` → `switchboard-core`
|
||||
- Go module: `switchboard-core`
|
||||
- VERSION: `0.1.0`
|
||||
- Default DB name: `switchboard_core`
|
||||
- Fresh migrations: 9 files × 2 dialects (postgres + sqlite), 27 tables
|
||||
|
||||
64
CONTRIBUTING.md
Normal file
64
CONTRIBUTING.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Contributing to Switchboard Core
|
||||
|
||||
## Development Setup
|
||||
|
||||
**Requirements:** Go 1.23+, Node.js 20+ (for frontend tests), SQLite (local dev).
|
||||
|
||||
**Build from source:**
|
||||
|
||||
```sh
|
||||
cd server
|
||||
go build -o switchboard-core .
|
||||
DB_DRIVER=sqlite DATABASE_URL=/tmp/switchboard.db ./switchboard-core
|
||||
```
|
||||
|
||||
**Docker (recommended):**
|
||||
|
||||
```sh
|
||||
docker compose up --build
|
||||
# open http://localhost:3000 (default login: admin / admin)
|
||||
```
|
||||
|
||||
Data persists in the `sb_data` named volume. Reset with `docker compose down -v`.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
server/ Go backend (handlers, store, config, auth, workflow engine)
|
||||
src/js/ Browser SDK and built-in surface JS
|
||||
packages/ Extension packages (surfaces, libraries, browser extensions)
|
||||
build.sh Builds each subdirectory into a .pkg archive
|
||||
docs/ Documentation markdown (served at /api/v1/docs)
|
||||
k8s/ Kubernetes manifests and Helm chart
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
**Backend:** `cd server && go test ./...`
|
||||
|
||||
**Frontend:** `node --test src/js/__tests__/`
|
||||
|
||||
## Code Conventions
|
||||
|
||||
- **Go:** Standard formatting via `gofmt`. Handlers in `server/handlers/`,
|
||||
persistence in `server/store/`. Database access goes through the store
|
||||
interface, never directly from handlers.
|
||||
- **Browser JS:** Vanilla ES modules + Preact via CDN. No build step for
|
||||
browser code (except the CM6 editor bundle). SDK lives at `src/js/sw/`.
|
||||
- **Extensions:** IIFE pattern (see `packages/*/js/script.js`). Register with
|
||||
`sw.renderers` or `sw.slots` via the `sw:ready` event.
|
||||
|
||||
## Creating a Package
|
||||
|
||||
A package is a ZIP archive (`.pkg`) containing `manifest.json` and optional
|
||||
`js/`, `css/`, `assets/`, and `script.star` files. The build script
|
||||
(`packages/build.sh`) automates this. See `docs/PACKAGE-FORMAT.md` for the
|
||||
full manifest spec and `docs/TUTORIAL-FIRST-EXTENSION.md` for a walkthrough.
|
||||
|
||||
## Pull Request Process
|
||||
|
||||
1. Create a feature branch from `main`.
|
||||
2. Make your changes. Keep commits focused.
|
||||
3. Run both Go and JS test suites and confirm they pass.
|
||||
4. Submit a PR with a clear description of what changed and why.
|
||||
5. Address review feedback, then squash-merge when approved.
|
||||
17
Dockerfile
17
Dockerfile
@@ -4,7 +4,8 @@
|
||||
# Stage 1: Build Go backend
|
||||
# Stage 2: Download JS vendor libs (marked, DOMPurify)
|
||||
# Stage 3: Build CM6 editor bundle (esbuild)
|
||||
# Stage 4: Production image (nginx + backend)
|
||||
# Stage 4: Build bundled packages (.pkg archives)
|
||||
# Stage 5: Production image (nginx + backend)
|
||||
#
|
||||
# Vendor libs are baked in during build so the
|
||||
# app works in disconnected environments
|
||||
@@ -57,7 +58,13 @@ COPY scripts/build-editor.sh /build/scripts/build-editor.sh
|
||||
RUN cd /build/src/editor && npm ci --loglevel=warn
|
||||
RUN sh /build/scripts/build-editor.sh /build/dist
|
||||
|
||||
# ── Stage 4: Production ─────────────────────
|
||||
# ── Stage 4: Build bundled packages ─────────
|
||||
FROM alpine:3 AS packages
|
||||
RUN apk add --no-cache zip bash
|
||||
COPY packages/ /packages/
|
||||
RUN cd /packages && bash build.sh
|
||||
|
||||
# ── Stage 5: Production ─────────────────────
|
||||
FROM nginx:1-alpine
|
||||
|
||||
RUN apk add --no-cache bash git
|
||||
@@ -70,6 +77,9 @@ COPY --from=backend /app/database/migrations /app/database/migrations
|
||||
COPY src/ /usr/share/nginx/html/
|
||||
COPY VERSION /VERSION
|
||||
|
||||
# Documentation (v0.6.1) — served by Go backend via /api/v1/docs
|
||||
COPY docs/ /app/docs/
|
||||
|
||||
# Inject version and build hash into index.html and sw.js at build time
|
||||
RUN APP_VERSION=$(cat /VERSION | tr -d '[:space:]') && \
|
||||
BUILD_HASH=$(find /usr/share/nginx/html/js -name '*.js' -exec md5sum {} + | sort | md5sum | cut -c1-8) && \
|
||||
@@ -85,6 +95,9 @@ COPY --from=vendor /vendor/katex/katex.min.css /usr/share/nginx/html/vendor/kate
|
||||
COPY --from=vendor /vendor/katex/fonts/ /usr/share/nginx/html/vendor/katex/fonts/
|
||||
COPY --from=cm6-build /build/dist/ /usr/share/nginx/html/vendor/codemirror/
|
||||
|
||||
# Bundled packages (v0.3.8) — auto-installed on first run
|
||||
COPY --from=packages /dist/ /app/bundled-packages/
|
||||
|
||||
# nginx config (template — envsubst injects BASE_PATH at runtime)
|
||||
COPY nginx.conf.template /etc/nginx/templates/default.conf.template
|
||||
# Only substitute BASE_PATH — preserve nginx variables ($host, $uri, etc.)
|
||||
|
||||
65
Dockerfile.builder
Normal file
65
Dockerfile.builder
Normal file
@@ -0,0 +1,65 @@
|
||||
# ============================================
|
||||
# Switchboard Core — Builder Image
|
||||
# ============================================
|
||||
# Pre-caches Go modules and Node dependencies
|
||||
# for faster custom builds. Use as a base in
|
||||
# your own Dockerfile to skip dependency
|
||||
# download on every build.
|
||||
#
|
||||
# Usage:
|
||||
# FROM ghcr.io/switchboard-core/builder:latest AS go-builder
|
||||
# COPY server/ /app/
|
||||
# RUN cd /app && go build -o /bin/switchboard .
|
||||
#
|
||||
# Or build this image locally:
|
||||
# docker build -f Dockerfile.builder -t switchboard-builder .
|
||||
# ============================================
|
||||
|
||||
# ── Go module cache ─────────────────────────
|
||||
FROM golang:1.23-bookworm AS go-cache
|
||||
WORKDIR /cache
|
||||
COPY server/go.mod server/go.sum* ./
|
||||
RUN go mod download && go mod verify
|
||||
|
||||
# ── Node dependency cache ───────────────────
|
||||
FROM node:20-alpine AS node-cache
|
||||
WORKDIR /cache
|
||||
|
||||
# Editor bundle dependencies
|
||||
COPY src/editor/package*.json ./editor/
|
||||
RUN cd editor && npm ci --loglevel=warn
|
||||
|
||||
# Vendor libs (same versions as production Dockerfile)
|
||||
RUN npm pack marked@16.3.0 && \
|
||||
npm pack dompurify@3.2.4 && \
|
||||
npm pack mermaid@11.4.1 && \
|
||||
npm pack katex@0.16.11 && \
|
||||
mkdir -p /cache/vendor-tarballs && \
|
||||
mv *.tgz /cache/vendor-tarballs/
|
||||
|
||||
# ── Final builder image ────────────────────
|
||||
FROM golang:1.23-bookworm
|
||||
|
||||
# Pre-install build tools
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
zip \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Go module cache (populated)
|
||||
COPY --from=go-cache /go/pkg/mod /go/pkg/mod
|
||||
|
||||
# Node dependencies (for editor bundle builds)
|
||||
COPY --from=node-cache /cache/editor/node_modules /cache/editor/node_modules
|
||||
|
||||
# Vendor lib tarballs (avoid re-download)
|
||||
COPY --from=node-cache /cache/vendor-tarballs /cache/vendor-tarballs
|
||||
|
||||
# Node.js for frontend builds
|
||||
COPY --from=node-cache /usr/local/bin/node /usr/local/bin/node
|
||||
COPY --from=node-cache /usr/local/lib/node_modules /usr/local/lib/node_modules
|
||||
RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
LABEL org.opencontainers.image.title="Switchboard Core Builder"
|
||||
LABEL org.opencontainers.image.description="Pre-cached build dependencies for faster custom Switchboard Core builds"
|
||||
19
README.md
19
README.md
@@ -22,12 +22,21 @@ those are all extension packages.
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Docker (recommended)
|
||||
docker compose up --build
|
||||
# → http://localhost:3000 (admin/admin)
|
||||
|
||||
# Or from source
|
||||
git clone <repo-url> && cd switchboard-core
|
||||
cp server/.env.example server/.env # edit DB credentials
|
||||
cd server && go run .
|
||||
# → http://localhost:8080
|
||||
```
|
||||
|
||||
Bundled packages (workflows, surfaces, task manager) are auto-installed on
|
||||
first boot. See [Distribution Guide](docs/DISTRIBUTION.md) for production
|
||||
deployment and customization.
|
||||
|
||||
## Kernel Features
|
||||
|
||||
- **Auth**: Builtin password, mTLS (client cert), OIDC (Keycloak et al.)
|
||||
@@ -43,15 +52,19 @@ cd server && go run .
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Distribution Guide](docs/DISTRIBUTION.md) — Docker, bundled packages, builder image, production deployment
|
||||
- [Architecture](docs/ARCHITECTURE.md) — kernel components and design reasoning
|
||||
- [Roadmap](ROADMAP.md) — current status and planned milestones
|
||||
- [Changelog](CHANGELOG.md) — version history
|
||||
|
||||
## Project Status
|
||||
|
||||
**v0.1.0** (in progress) — kernel extracted from chat-switchboard v0.38.5.
|
||||
~44K lines of chat/AI code removed, 27 kernel tables, 20 store interfaces.
|
||||
See [ROADMAP.md](ROADMAP.md) for details.
|
||||
**v0.5.0** — Realtime pub/sub primitive, dialog audit, and admin permissions
|
||||
UI. Extensions can now publish events to WebSocket channels via Starlark;
|
||||
clients subscribe with `sw.realtime.subscribe()`. Admin Packages page gains
|
||||
per-permission grant/revoke controls and status badges. See
|
||||
[ROADMAP.md](ROADMAP.md) for the full journey from v0.1.0 kernel extraction
|
||||
through v0.3.x workflows, v0.4.x Notes surface, to v0.5.x realtime and chat.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
342
ROADMAP.md
342
ROADMAP.md
@@ -1,268 +1,182 @@
|
||||
# Switchboard Core — Roadmap
|
||||
|
||||
## Current: v0.3.7 — Package Audit
|
||||
## Current: v0.6.7 — Native mTLS
|
||||
|
||||
Fork of chat-switchboard, gutted to a pure extension platform. All AI/chat
|
||||
features removed from the kernel. What remains is the minimum viable
|
||||
platform that extensions build on.
|
||||
Self-hosted extensible platform. Auth, identity, packages, Starlark sandbox,
|
||||
storage, realtime, and ops are kernel primitives. Everything else is an extension.
|
||||
|
||||
### Retained kernel capabilities
|
||||
**Kernel capabilities:** Auth (builtin/mTLS/OIDC) · Users/teams/groups/RBAC ·
|
||||
Surfaces/extensions/libraries/workflows · Starlark sandbox (capability-gated) ·
|
||||
Object storage (PVC/S3) + ext_data tables · WebSocket hub + realtime pub/sub ·
|
||||
Audit log · Notifications · Scheduled tasks
|
||||
|
||||
- **Auth**: builtin (simple), mTLS, OIDC
|
||||
- **Identity**: users, teams, groups, permissions (RBAC)
|
||||
- **Packages**: surfaces, extensions, libraries, workflows
|
||||
- **Starlark sandbox**: capability-gated modules
|
||||
- **Storage**: object storage (PVC, S3), ext_data tables
|
||||
- **Realtime**: WebSocket hub, presence, multi-replica HA
|
||||
- **Ops**: audit log, notifications, maintenance goroutine
|
||||
**Completed history:** v0.2.x–v0.5.x fully documented in `CHANGELOG.md`.
|
||||
Highlights: RBAC + settings cascade, event bus + triggers, SDK stabilization,
|
||||
workflow engine (multi-stage, team roles, signoff gate, public entry, SLA),
|
||||
package distribution, Notes surface (CM6, folders, tags, backlinks, graph),
|
||||
realtime primitive, Chat surface (chat-core library + surface + polish),
|
||||
upgrade test harness, cluster registry + HA.
|
||||
|
||||
### Phase 0 (complete)
|
||||
---
|
||||
|
||||
## v0.6.0 — MVP
|
||||
|
||||
Extension, communication, and operations tracks converge. First
|
||||
externally usable release.
|
||||
|
||||
Design docs: `docs/DESIGN-cluster-registry.md` — PG-backed cluster registry and self-assembling mesh.
|
||||
|
||||
### v0.6.0 — Cluster Registry + HA
|
||||
|
||||
PG is the consensus layer. Zero new infrastructure. `UNLOGGED` table + `LISTEN/NOTIFY` replaces etcd/Consul/Redis for homelab-to-small-team scale.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| 1. Module rename | ✅ | `chat-switchboard` → `switchboard-core` |
|
||||
| 2. Delete packages | ✅ | 15 Go packages, 29 handler files removed |
|
||||
| 3. Gut stores/models | ✅ | 40 → 20 store interfaces, kernel-only models |
|
||||
| 4. Fresh migrations | ✅ | 9 files × 2 dialects, 27 tables |
|
||||
| 5. Fix compilation | ✅ | `go build ./...` clean, 300+ stale route lines cut |
|
||||
| 6. Fix tests | ✅ | 8 test packages pass, ~12K stale test lines pruned |
|
||||
| 7. Frontend gut | ✅ | Shell + SDK only, 50+ files of chat/notes/projects code removed |
|
||||
| 8. New ICD | ✅ | Full OpenAPI 3.0.3 spec — 160 operations across 22 tag groups |
|
||||
| 9. CI/CD + Dockerfile | ✅ | Single unified image, FE/BE split removed, DB names updated, k8s var alignment fixes (resource quantities, image name, rollout deployment name) |
|
||||
| 10. Smoke test | ✅ | K8s deploy live at switchboard.gobha.ai/test, nginx BASE_PATH fixed, login→admin flow verified, branding updated |
|
||||
| `node_registry` table | ✅ | `UNLOGGED TABLE` — node_id, endpoint, seq, registered_at, heartbeat, stats JSONB. Postgres migration 013. |
|
||||
| Node registration | ✅ | Self-registration on startup: `INSERT ... ON CONFLICT DO UPDATE`. `node_id` = `hostname-PID` or `CLUSTER_NODE_ID` env override. |
|
||||
| Heartbeat tick | ✅ | Every 10s: update own heartbeat + collect runtime stats (goroutines, heap, GC, uptime, ws_clients). |
|
||||
| Stale sweep | ✅ | Every heartbeat tick: `DELETE WHERE heartbeat < now() - 30s`. All nodes run it — idempotent, no ring topology. |
|
||||
| Self-eviction | ✅ | If heartbeat UPDATE returns 0 rows: node was swept by peer → log error + `os.Exit(1)`. K8s restarts → re-register. |
|
||||
| LISTEN/NOTIFY routing | ✅ | Durable events (messages, state changes) fan-out via `pg_notify`. All replicas receive, push to local WS subscribers, drop if irrelevant. Phase 1: ephemeral events (typing, presence) also via NOTIFY (8KB limit, ~60 bytes each). |
|
||||
| Cluster API | ✅ | `GET /api/v1/admin/cluster` — returns `{data: [...]}` envelope with all registered nodes. |
|
||||
| Admin cluster dashboard | ✅ | `cluster-dashboard` surface package renders one card per node: node_id, endpoint, uptime, ws_clients, heap, GC pause. JSONB stats — future keys render automatically, no schema migration. Auto-refresh every 10s. |
|
||||
| Health endpoint | ✅ | `GET /health` includes `node_id` and `cluster: {size, peers, heartbeat_age_ms}`. |
|
||||
| Config | ✅ | `CLUSTER_NODE_ID`, `CLUSTER_HEARTBEAT_INTERVAL` (default 10s), `CLUSTER_STALE_THRESHOLD` (default 30s), `CLUSTER_ENDPOINT` (Phase 2 mesh, auto-detect). |
|
||||
| Single-node regression | ✅ | One-node behavior identical to pre-cluster: one registry row, NOTIFY delivers back to same instance. No special-casing. SQLite returns nil store — all cluster code guarded. |
|
||||
| Multi-node integration test | ✅ | Docker Compose: 3 instances, shared PG. `ci/e2e-cluster-test.sh`: registration, stale sweep on stop, re-registration on restart. 3 unit tests + 2 handler tests. |
|
||||
|
||||
## v0.2.x — SDK & Triggers
|
||||
|
||||
The contract that extensions build against. Three trigger primitives,
|
||||
SDK stabilization, and the first rebuilt extension (tasks).
|
||||
|
||||
### v0.2.0 — RBAC + Settings Cascade (complete)
|
||||
### v0.6.1 — Backup/Restore + Documentation
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Admin → RBAC group | ✅ | `surface.admin.access` permission + Admins system group replaces `role == "admin"` checks. Admin bypass removed from permission middleware. |
|
||||
| Settings cascade | ✅ | `user_overridable` flag, three-tier resolution (global → team → user), team settings API |
|
||||
| ~~Settings override model~~ | ✅ | Shipped with settings cascade above |
|
||||
| Backup handler | ✅ | `POST /api/v1/admin/backup` streams `.swb` ZIP (JSONL core + ext_data tables + package assets). `POST /api/v1/admin/restore` wipes DB and restores from archive. Dialect-neutral (SQLite + Postgres). |
|
||||
| Server-side backups | ✅ | `GET /api/v1/admin/backups` list, `GET /download`, `DELETE`. Store backups in `{STORAGE_PATH}/backups/`. |
|
||||
| Admin backup section | ✅ | New "Backup" section under `/admin/backup`. Create (download or server-side), list, download, delete, restore with destructive confirmation. |
|
||||
| Documentation API | ✅ | `GET /api/v1/docs` lists, `GET /api/v1/docs/:name` returns raw markdown. Authenticated (not admin-only). |
|
||||
| Docs surface | ✅ | Builtin surface at `/docs/:section`. Sidebar navigation, client-side markdown renderer. 5 new docs: Getting Started, Extension Guide, API Reference, Deployment, Package Format. |
|
||||
| Tests | ✅ | 6 handler tests (basic backup, ext_data backup, round-trip restore, schema mismatch, dump/restore table, list empty). E2E script `ci/e2e-backup-test.sh`. |
|
||||
|
||||
### v0.2.1 — Default Surface + ICD
|
||||
### v0.6.2 — Docs Polish + Dynamic OpenAPI
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Default surface routing | ✅ | `/` redirects to configurable default surface. No surfaces → admin. First install becomes default. Changeable in admin settings. |
|
||||
| ICD (API contract) | ✅ | Full OpenAPI 3.0.3 spec — 160 operations, 22 tag groups, reusable component schemas. Served at `/api/docs`. |
|
||||
| Dark mode fix | ✅ | Added `--bg-code` to CSS variables (dark + light). Replaced all hardcoded light-mode fallbacks in docs CSS with theme-aware variables. Table styling, code blocks, nav items all readable in dark mode. |
|
||||
| Loading & error handling | ✅ | Replaced borrowed `settings-placeholder` with docs-specific pulse animation. Added error state with retry button for failed doc list fetch. |
|
||||
| Topbar navigation | ✅ | Imported `Topbar` + `UserMenu` into docs surface. Users can now navigate to other surfaces via the avatar menu. |
|
||||
| Docs icon + menu entry | ✅ | Added `📖 Docs` entry to UserMenu standard items. Docs accessible from any surface's user menu. |
|
||||
| `api_schema` manifest field | ✅ | Optional `api_schema` array in `manifest.json`. Parsed lazily by spec builder. Malformed entries logged and skipped — never blocks extension loading. |
|
||||
| OpenAPI spec builder | ✅ | `BuildOpenAPISpec()` merges static kernel spec with extension routes. Tier 1: auto-generated stubs for all `api_routes`. Tier 2: `api_schema` replaces stubs with rich path items (params, body, response). |
|
||||
| Dynamic spec endpoint | ✅ | `GET /api/docs/openapi.json` serves merged spec. Swagger UI updated to use JSON endpoint. Static YAML preserved for backward compat. |
|
||||
| Tests | ✅ | 7 new handler tests: zero extensions, stubs, rich schema, multi-extension, malformed schema, disabled exclusion, required fields. |
|
||||
|
||||
### v0.2.2 — Event Bus + Triggers
|
||||
---
|
||||
|
||||
## v0.6.x — Hardening
|
||||
|
||||
Closes every audit finding before the public release. No new features — only correctness, dead code elimination, and architectural cleanup. Sequence is fixed: each version is a gate for the next.
|
||||
|
||||
### v0.6.3 — Dead Code Sweep + Registry Fix
|
||||
|
||||
Pure cleanup. No behavior changes except fixing the broken registry install flow.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Event bus subscriptions | ✅ | Extensions register event patterns in manifest. Wired via `bus.Subscribe()` on startup. Async handler invocation. |
|
||||
| Webhook triggers | ✅ | Inbound HTTP at `/api/v1/hooks/:package_id/:slug`. HMAC-SHA256 verification. Synchronous Starlark handler response. |
|
||||
| Scheduled tasks | ✅ | User-created cron tasks with restricted sandbox (no raw HTTP, no DB table creation). Runs as creator identity. Templates from extensions. Dedicated schedules API. |
|
||||
| Trigger admin API | ✅ | CRUD for triggers + schedules. Enable/disable, execution logs, per-package listing. |
|
||||
| Fix registry install | ✅ | SDK sends `{ url }`, handler expects `{ download_url }`. Fix `api-domains.js` to send `{ download_url: url }`. Every Install click currently returns 400. |
|
||||
| Registry settings UI | ✅ | Add "Package Registry" section to admin settings with URL input field. Only way to configure registry today is a raw `PUT /api/v1/admin/settings/package_registry` — no user will find it. |
|
||||
| Registry tooling + docs | ✅ | `scripts/generate-registry.sh` scans a directory of `.pkg` files and emits registry JSON. `docs/PACKAGE-REGISTRY.md` documents the format. |
|
||||
| Delete dead kernel Go | ✅ | `store/interfaces.go:178–183` — orphaned ChannelListFilter comments. `pages/pages.go:922–930` — `roleFilterType()` + template registration (chat vestige, maps nonexistent roles). `main.go:67` — orphaned provider-type comment. |
|
||||
| Delete dead vendor JS | ✅ | `vendor/marked.min.js` and `vendor/purify.min.js` — 62KB, zero production imports. Only referenced in test helpers. |
|
||||
| Delete `dev.html` | ✅ | 676 lines, not imported by anything. Dead. |
|
||||
| Remove `dashboard` from default bundle | ✅ | Requires `legacy-sdk` (doesn't exist), auto-installs as dormant on fresh installs. Confusing. Remove from `defaultBundledPackages`. |
|
||||
| Remove `hello-dashboard` | ✅ | Proof-of-concept from early development. Move to `examples/` or delete. |
|
||||
| Strip stale version comments | ✅ | 60+ files had legacy version annotations (`// v0.29.x:`, `// v0.33.x:`). Single sed pass. |
|
||||
| Narrow default bundle | ✅ | Default: `notes`, `chat`, `chat-core`, `mermaid-renderer`, `schedules`. Everything else available via registry or `BUNDLED_PACKAGES=*`. |
|
||||
|
||||
### v0.2.3 — SDK + Task Extension
|
||||
### v0.6.4 — Admin Health/Metrics Tab + Cluster Merge
|
||||
|
||||
Structural move: cluster dashboard becomes an Admin tab. Better home for health/metrics — shared context with other admin panels, no separate nav entry.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| SDK stabilization | ✅ | `sw.api.ext()`, `sw.storage`, `sw.theme.tokens`, `sw.ui`, `sw.slots`, `sw.actions` — six new SDK modules for extension development |
|
||||
| Task extension | ✅ | Full task surface rebuilt as Starlark extension: CRUD API, kanban/list views, event triggers, webhook integration, notifications on completion |
|
||||
| "Health / Metrics" admin tab | ✅ | New tab in Admin surface. DB-agnostic metrics for all deployments. Cluster cards conditional on PG + multi-node detection. |
|
||||
| Runtime metrics | ✅ | Per-node: goroutines, heap alloc/sys, stack in use, GC cycles, last GC pause, GC CPU %, uptime, WebSocket clients, extensions loaded, open FDs. |
|
||||
| DB pool metrics | ✅ | All deployments: DB latency (`SELECT 1` round-trip), pool active/idle/max, wait count, wait duration. PG-only: table bloat (`n_dead_tup`), active backends (`pg_stat_activity`). |
|
||||
| Cluster metrics | ✅ | PG multi-node only: cluster size, peer list with endpoint + uptime, heartbeat age per node, event bus publish/deliver rates. |
|
||||
| Extension runtime metrics | ✅ | Starlark exec/min, errors/min, avg duration, HTTP outbound requests/min, trigger fires/min, schedule overruns. |
|
||||
| Fatten heartbeat payload | ✅ | Heartbeat JSONB carries full metric set. `GET /api/v1/admin/metrics` for single-node SQLite fallback (same shape). |
|
||||
| Retire `cluster-dashboard` | ✅ | Remove package once Admin Health tab ships. Update `defaultBundledPackages`. |
|
||||
| Fix block renderer `requires` | ✅ | `mermaid-renderer`, `katex-renderer`, `csv-table`, `diff-viewer` all have `"requires": ["chat"]`. These are content renderers, not chat features. Remove constraint — they should activate without chat. |
|
||||
| Health endpoint consolidation | ✅ | `/health` and `/api/v1/health` return near-identical JSON. Merge or clearly differentiate with docs. |
|
||||
|
||||
### v0.2.4 — Shell Navigation + Schedules
|
||||
### v0.6.5 — Renderer Pipeline + Docs Rewrite
|
||||
|
||||
Most complex sub-version. Lifts block rendering to a kernel SDK primitive so all surfaces share it, then rewrites the docs for an external audience.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| SDK Topbar | ✅ | `sw.shell.Topbar` — composable navigation bar (title + extension slot + bell + user menu). Surfaces get consistent nav for free. |
|
||||
| Schedules surface | ✅ | New `packages/schedules/` wrapping kernel cron API. Table view, cron preview, enable/disable, manual run, execution logs. |
|
||||
| Manifest icons | ✅ | `icon` field in manifest.json (emoji). Surfaces API returns icon. UserMenu renders per-surface icons. |
|
||||
| UserMenu cleanup | ✅ | Removed dead Chat/Notes/Projects links. Menu driven by surfaces API. Core surfaces filtered. |
|
||||
| isAdmin RBAC fix | ✅ | `can.js` isAdmin() now checks `surface.admin.access` grant instead of deprecated role column. |
|
||||
| SDK renderer primitive | ✅ | `sw.renderers.register(pattern, handler)` — kernel-level registration. Extensions call once; all surfaces consume. Decouples renderer discovery from chat surface. |
|
||||
| Notes hooks SDK renderer pipeline | ✅ | Notes delegates to `sw.markdown.renderSync()` + `sw.renderers.runPostRenderers()`. Hand-rolled renderer deleted. |
|
||||
| Docs hooks SDK renderer pipeline | ✅ | Docs delegates to `sw.markdown.renderSync()` + `sw.renderers.runPostRenderers()`. Hand-rolled renderer deleted (~160 lines). |
|
||||
| Unify markdown renderer | ✅ | `sw.markdown` module lazy-loads `marked` v16 (vendored). Notes, Docs, Chat all consume it. Two hand-rolled parsers deleted. |
|
||||
| Sanitize HTML output | ✅ | DOMPurify wired as default post-render step in `sw.markdown`. SVG-safe config allows mermaid output. Notes sanitizes; Docs opts out (system-authored). |
|
||||
| Mermaid/KaTeX/CSV/Diff work everywhere | ✅ | Browser extension loader injects renderer scripts into all pages. Extensions register via `sw:ready` event. All four render in Notes, Docs, and Chat. |
|
||||
| Docs rewrite for external audience | ✅ | All fork references removed from docs, CHANGELOG, ROADMAP. DESIGN-WORKFLOW-REDESIGN-0.2.6.md replaced with DESIGN-WORKFLOWS.md. |
|
||||
| Add Mermaid architecture diagrams | ✅ | Six diagrams in ARCHITECTURE.md: system overview, request flow, extension lifecycle, realtime events, settings cascade, cluster topology. |
|
||||
| Surface alias decision | ✅ | Migrated SDK + ICD runner to `/admin/packages/`; aliases removed from main.go. |
|
||||
| `CONTRIBUTING.md` + tutorial | ✅ | CONTRIBUTING.md at repo root + docs/TUTORIAL-FIRST-EXTENSION.md walkthrough. |
|
||||
|
||||
### v0.2.5 — UI Polish + Dead Code Audit
|
||||
### v0.6.7 — Native mTLS
|
||||
|
||||
End-to-end mutual TLS without a reverse proxy. Targets systemd+podman deployments where every connection (client→server, node→node) is mTLS. Design: `docs/DESIGN-native-mtls.md`.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| UI bug pass | ✅ | Reviewed admin, settings, login, welcome surfaces in light/dark themes. Fixed settings crash (models API undefined). Removed dead chat settings from both user and admin settings. |
|
||||
| Dead code sweep | ✅ | Removed ~700 lines: orphaned CSS, dead Go types, stale event code, unused test helpers, dead settings UI (chat defaults, system prompt, default model, policies, web search, compaction, memory). |
|
||||
| Template cleanup | ✅ | Removed stale CSS link tags and orphaned chat-pane.html. Updated doc comments throughout. |
|
||||
| Package proof-of-concept status | ✅ | Created README.md for tasks + schedules with graduation criteria. Added missing manifest icons. |
|
||||
| Welcome surface | ✅ | New fallback surface when no extensions installed. Topbar + welcome card with admin link. Replaces `/admin` as final redirect target. |
|
||||
| Default surface routing | ✅ | Resolution chain: user preference → global config → first extension → `/welcome`. Users can set personal default in Settings > General. Admin sets global default in Admin > Settings. |
|
||||
| Admin navigation | ✅ | Replaced Back button with UserMenu in admin topbar. Eliminates back-button infinite loop. |
|
||||
| `TLS_MODE` config | ✅ | Three values: `none` (default, plain HTTP) · `server` (TLS, no client cert) · `mtls` (mutual TLS, client cert required). Independent of `AUTH_MODE`. |
|
||||
| TLS server mode | ✅ | Go binary calls `ListenAndServeTLS` directly. `TLS_CERT`, `TLS_KEY`, `TLS_CA` path config. TLS 1.3 minimum, no fallback. `server/config/tls.go` loader + validation. |
|
||||
| `MTLSNativeProvider` | ✅ | Reads `r.TLS.PeerCertificates[0]` — no header trust. `Subject.CommonName` → username. `sha256(Raw)` → `external_id`. Auto-provisions `auth_source=mtls`. Renamed existing `mtls.go` → `mtls_proxy.go`. Shared helpers in `mtls_helpers.go`. |
|
||||
| Node-to-node mTLS | ✅ | `BuildPeerTLSConfig()` constructs outbound TLS config with node cert + CA pool. Forward-looking — cluster registry is DB-backed (PG LISTEN/NOTIFY), no HTTP peer calls yet. |
|
||||
| `switchboard-ca.sh` | ✅ | Shell wrapper around openssl. Three commands: `init` (CA keypair) · `issue-node --name <n> --san <addrs>` (365d) · `issue-user --cn <name>` (90d). All output PEM. |
|
||||
| Unit tests | ✅ | Ephemeral CA via `crypto/x509`. Fabricated `PeerCertificates`. Valid cert → user provisioned · no TLS → `ErrNoCert` · empty certs → `ErrNoCert` · no CN → `ErrInvalidCreds`. |
|
||||
| Integration tests | ✅ | Real TLS listener on localhost. No cert / wrong CA / expired → TLS handshake rejected. Valid cert → 200. Peer certificate CN + email visibility verified. |
|
||||
|
||||
### v0.2.6 — Admin Settings Audit
|
||||
### v0.6.6 — Final Hardening
|
||||
|
||||
Final pass before public release. Security, correctness, and developer experience.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Admin settings E2E | ✅ | All 7 admin settings sections verified (default surface, registration, banner, message bar, footer, vault, email). Dead code removed: `sectionCategory()` pruned of AI/routing/channel vestiges, `PublicSettings()` stripped of chat-era fields (system_prompt, retention_ttl, paste_to_file, allow_user_personas), dead `PolicyDefaults` removed (allow_raw_model_access, default_model), dead policy lookups removed (kb_direct_access), test seed data cleaned. |
|
||||
| Packages surface | ✅ | Package list loads (17 total/17 enabled), type filters work, enable/disable visible, settings/export buttons present, core package (admin) protected. Removed dead `chat` from CORE_IDS. |
|
||||
| Extension dependency auto-activation | ✅ | Installing a package with unmet `depends`/`requires` auto-installs dependencies from the bundled set. If not bundled: clear error listing what's missing. |
|
||||
| `ValidateManifest()` gate | ✅ | Single `ValidateManifest()` function in `package_validate.go`. Called at install time (both upload and bundled). 12 unit tests. |
|
||||
| Package signing hook | ✅ | Optional `signature` field in manifest (reserved). `PACKAGE_VERIFY_SIGNATURES` env var (default false, log-only). No cryptographic code yet — schema slot reserved. |
|
||||
| OIDC state nonce validation | ✅ | `oidcClaims.Nonce` field added. `ValidateIDTokenNonce()` compares ID token nonce against stored state. Callback rejects mismatched nonces. |
|
||||
| Schema migration stub decision | ✅ | Stub replaced with log-only function documenting additive-only policy. Downgrade rejection preserved. |
|
||||
| ICD/SDK runner update pass | ✅ | ICD smoke tier: added metrics, cluster, backups, docs, OpenAPI JSON endpoints. SDK admin domain: added metrics, cluster, backups tests. |
|
||||
| Stale TODO resolution | ✅ | `main.go` session middleware: replaced with `OptionalAuth()` (auth if token present, anonymous pass-through). `auth.go:221` OIDC nonce: resolved. `starlark_helpers.go:96` migration stub: resolved. |
|
||||
|
||||
### v0.2.7 — User Settings Audit
|
||||
Then ship.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| User settings E2E | ✅ | All 6 user settings sections verified (General, Appearance, Profile, Teams, Connections, Notifications). Dead code removed: BYOK nav section + state, personas gate filter, Message Font Size slider, `auth.permissions.changed` listener. localStorage key renamed `cs-appearance` → `sb-appearance` with one-time migration. |
|
||||
| Visibility gating | ✅ | Dead BYOK/personas policy lookups removed from bootstrap and permissions handlers. `allow_user_byok` removed from `PublicSettings`. `PolicyDefaults` cleaned of `allow_user_byok` and `allow_user_personas`. Dead `msgFont` early-apply removed from base template. Stale policy-gating test assertions replaced. No empty nav sections remain. |
|
||||
|
||||
### v0.2.8 — Team Admin Settings Audit (Pass 1)
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Team admin E2E | ✅ | Audited team member management, settings cascade, role assignment. Removed dead code: `HasPrivateProviderRequirement` (BYOK vestige), `UserRole` on `TeamMember` (deprecated role column), `allow_team_providers` policy default, dead personas/providers/models ICD tests. |
|
||||
| Workflow stage UI cleanup | ✅ | Removed dead personas dropdown, `history_mode` selector, stale `chat_only` mode. Updated `STAGE_MODES` to match backend CHECK constraint (`form_only`, `form_chat`, `review`, `custom`). Removed stale comments referencing deleted files. |
|
||||
|
||||
### v0.2.9 — Builtin Extension Retirement
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Retire builtin seeder | ✅ | Removed `SeedBuiltinPackages`, `seed_packages.go`, Dockerfile COPY, and `extensions/builtin/` directory. These extensions are dormant until a chat surface exists to consume them. |
|
||||
| Convert to regular packages | ✅ | Repackaged 6 extensions as standard directories in `packages/` with `js/` layout and `"requires": ["chat"]` manifest metadata. Built via `build.sh` like all other packages. No auto-install — explicit install only. |
|
||||
|
||||
## v0.3.x — Workflow Architecture
|
||||
|
||||
Workflows are the core platform capability. This series implements the
|
||||
full multi-step automation system with team role integration and
|
||||
finalizes the extension lifecycle model.
|
||||
|
||||
### v0.3.0 — Schema Redesign + Stage CRUD Modernization (complete)
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Workflow schema redesign | ✅ | Dropped `persona_id`, `history_mode` from `workflow_stages`. Renamed `transition_rules` → `stage_config`. Updated `stage_mode` CHECK to `(form, review, delegated, automated)`. Added `audience`, `stage_type`, `starlark_hook`, `branch_rules`. Both PG + SQLite. |
|
||||
| Model + store updates | ✅ | Go structs, constants, and PG/SQLite queries updated. New `ValidStageTypes`, `ValidAudiences` maps. Routing engine reads `branch_rules` directly. |
|
||||
| Handler + Starlark updates | ✅ | Stage CRUD validation for new fields. Package export/import updated. Hook handler reads `stage_config`. Starlark module exposes `audience`, `stage_type`. |
|
||||
| Frontend updates | ✅ | Team-admin and admin stage editors updated with new modes, audience selector, stage type selector, conditional Starlark hook input. Fixed admin `STAGE_MODES` bug. |
|
||||
|
||||
### v0.3.1 — Instance + Assignment Tables + Store (complete)
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Instance schema | ✅ | `workflow_instances` table in 007_workflows.sql (PG + SQLite). Tracks execution state: workflow_version, current_stage, stage_data, status, entry_token. |
|
||||
| Assignment schema | ✅ | `workflow_assignments` table. Per-stage queue: instance_id, stage, team_id, assigned_to, status, review_data. Optimistic claim lock. |
|
||||
| Models + store interface | ✅ | `WorkflowInstance`, `WorkflowAssignment` structs. 15 store methods for instance lifecycle and assignments across PG + SQLite. |
|
||||
| Event types | ✅ | `workflow.started`, `workflow.cancelled`, `workflow.error` added to bus route table. |
|
||||
|
||||
### v0.3.2 — Workflow Engine + Handlers (complete)
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Store tests | ✅ | Round-trip tests for all 15 v0.3.1 store methods (instance + assignment CRUD). TruncateAll updated for new tables. |
|
||||
| Stage execution engine | ✅ | `server/workflow/engine.go` — Start, Advance (with branch_rules), Cancel. Merges stage_data, creates assignments, emits events. |
|
||||
| Automated stages | ✅ | `server/workflow/automated.go` — fire Starlark hook, auto-advance, cycle guard (max 10 consecutive). |
|
||||
| Instance handlers | ✅ | HTTP API: Start, GetInstance, Advance, Cancel, ListInstances. Team-scoped mirrors. |
|
||||
| Assignment handlers | ✅ | HTTP API: Claim, Unclaim, Complete, Cancel, ListByTeam, ListMine. Claimer identity verification. |
|
||||
| Starlark module expansion | ✅ | `workflow.get_instance()`, `workflow.list_instances()` (read-only). Mutating builtins deferred pending interface extraction. |
|
||||
|
||||
### v0.3.3 — Public Entry + Background Jobs (complete)
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Public entry | ✅ | `StartPublic`, `ResumePublic`, `AdvancePublic` — token-based anonymous workflow participation. Public routes at `/api/v1/public/workflows/`. Audience-gated: only `public` stages can be advanced anonymously. |
|
||||
| SLA scanner | ✅ | Background goroutine (5-min interval) checking active instances against per-stage `sla_seconds`. Fires `workflow.sla_breach` event, marks breach in instance metadata (idempotent). |
|
||||
| Staleness sweep | ✅ | Per-workflow `staleness_timeout_hours` column. Scanner marks idle instances as `stale`, cancels open assignments, fires `workflow.stale` event. |
|
||||
|
||||
### v0.3.4 — Team Roles + Multi-party Validation (complete)
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Team roles | ✅ | Removed CHECK constraint on `team_members.role`. Custom roles stored in `teams.settings["roles"]`. Team roles API (`GET/PUT /teams/:teamId/roles`). Role-based stage assignment via `stage_config.required_role`. Frontend: dynamic role selects, role management panel. |
|
||||
| Multi-party sign-off | ✅ | `workflow_signoffs` table (both dialects). `StageConfig.validation` with `required_approvals`, `required_role`, `reject_action`. Engine validation gate in `advanceInternal`. `SubmitSignoff` engine method. Signoff HTTP API (`POST/GET /instances/:iid/signoffs`). Frontend signoff panel in monitor tab. |
|
||||
| Extension lifecycle | ✅ | Design doc: `docs/DESIGN-EXTENSION-LIFECYCLE.md`. Permanent vs PoC classification, graduation criteria, install model. |
|
||||
| Trigger composition | ✅ | Design doc: `docs/DESIGN-TRIGGER-COMPOSITION.md`. Triggers/schedules can start workflows, workflows emit events, no circular invocation. |
|
||||
|
||||
### v0.3.5 — Settings Audit + ICD + Tests
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Clone endpoint | ✅ | `POST /api/v1/workflows/:id/clone` — deep copy workflow + stages. Handler-level composition of existing store methods. |
|
||||
| Integration tests | ✅ | 7 engine-level tests in `workflow_engine_test.go`: full lifecycle, branch routing, public entry, signoff gate, rejection, cancel-clears-assignments, error cases. 28 total tests. |
|
||||
| Settings audit pass 2 | ✅ | Added `staleness_timeout_hours` field to workflow editor. Added collapsible branch_rules JSON editor in stage form. |
|
||||
| ICD update | ✅ | Fixed stale Workflow/Stage schemas. Added WorkflowInstance, WorkflowAssignment, WorkflowSignoff schemas. Added ~20 new endpoints: instances, assignments, signoffs, public entry, clone, team roles. |
|
||||
|
||||
### v0.3.6 — Example Workflows + Interactive Demo
|
||||
|
||||
Four installable `.pkg` workflow packages that prove the engine works
|
||||
end-to-end, plus a demo surface for guided exploration.
|
||||
See `docs/DEMO-WORKFLOWS.md` for full stage definitions and Starlark code.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Bug Report Triage | ✅ | Public entry, progressive fieldsets, severity-based branch routing, team assignment, SLA timer. Pure-manifest `.pkg`. |
|
||||
| Employee Onboarding | ✅ | Starlark automated stages (`db.insert`, `notifications.send`), signoff gate with `required_role`, rejection reroute. `.pkg` with `script.star`. |
|
||||
| Content Approval | ✅ | Multi-party signoff (quorum of 2), rejection reroute creating review cycle. `.pkg` demonstrating signoff + reroute loop. |
|
||||
| Webhook Notifier | ✅ | Starlark `http.post` + `connections.get` for outbound webhooks, delivery logging to ext_data. Proves HTTP + connections modules. |
|
||||
| Demo surface | ✅ | Browser-tier walkthrough: workflow cards, stage diagrams, Starlark viewer, "Try It" buttons, API curl examples. Auto-installed, removable. |
|
||||
| Engine context fix | ✅ | Add `started_by` to automated stage Starlark context dict (backward-compatible). |
|
||||
| SLA package installer | ✅ | Added `sla_seconds` support to `workflowPkgStage` struct and install/export paths. |
|
||||
| Snapshot format fix | ✅ | `parseSnapshotStages` helper handles both wrapped and legacy snapshot formats. |
|
||||
| Workflow adoption | ✅ | `POST /teams/:teamId/workflows/:id/adopt` + `GET .../available`. Team-admin "Adopt Global" button. `TeamID` in `WorkflowPatch`. |
|
||||
| Extension SDK boot | ✅ | `base.html` loads Preact globals + `boot()` for extension surfaces (was missing since Scorched Earth IV). |
|
||||
| Admin teams tab fix | ✅ | Extract `.data` from paginated response in admin teams list. |
|
||||
| Documentation | ✅ | `docs/DEMO-WORKFLOWS.md`: feature capability matrix, stage flows, API walkthrough, per-package READMEs. |
|
||||
| Review pass | | Docker E2E walkthrough: install all packages, adopt into team, activate, publish, run each workflow end-to-end. Fix remaining UI/UX issues. |
|
||||
|
||||
### v0.3.7 — Package Audit (complete)
|
||||
|
||||
Verified all 16 packages install correctly. Packages with unmet `requires` auto-set to dormant.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Manifest fixes | ✅ | Fixed 6 chat-extension manifests (`"name"` → `"title"`, was blocking install). Added explicit `"type": "surface"` to hello-dashboard, icd-test-runner, sdk-test-runner. |
|
||||
| Dormant status | ✅ | Added `dormant` to `packages.status` CHECK constraint (both dialects). Installer auto-detects unmet `requires` and sets `status=dormant, enabled=false`. Enable endpoint returns 409 for dormant packages. |
|
||||
| Functional audit | ✅ | Navigated to every surface in browser. 7 working: schedules, tasks, team-activity-log, git-board, hello-dashboard, icd-test-runner, sdk-test-runner. 2 broken: dashboard + editor (depend on removed `sw.*` imperative SDK) — tagged `requires: ["legacy-sdk"]` → dormant. |
|
||||
| Chat-dependency tagging | ✅ | 6 chat-dependent extensions (csv-table, diff-viewer, js-sandbox, katex-renderer, mermaid-renderer, regex-tester) install as dormant. git-board and gitea-client are standalone. |
|
||||
| Dependency check fix | ✅ | Relaxed library dependency validation to allow `pending_review` libraries (gitea-client declares permissions). Only `suspended`/`dormant` libraries blocked. |
|
||||
| Admin UI | ✅ | Dormant badge, disabled Enable button with tooltip, Dormant stat card in admin packages view. |
|
||||
| Tests | ✅ | 4 handler tests (SetStatus dormant, enable blocked, surfaces exclude dormant, admin list includes dormant). All existing tests pass. |
|
||||
|
||||
### v0.3.8 — Distribution
|
||||
|
||||
Builder image for faster builds, bundled packages for zero-config first run.
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Builder image | | `Dockerfile.builder` with cached Go modules + Node vendor libs. Published to Docker Hub for faster custom builds. |
|
||||
| Bundled packages | | Dockerfile stage builds example workflows + demo surface into production image. Auto-installs on first run (standard install path, admin can uninstall). `SKIP_BUNDLED_PACKAGES=true` to disable. |
|
||||
| Distribution docs | | `docs/DISTRIBUTION.md` with builder image usage, custom build guide, production deployment recommendations. |
|
||||
|
||||
## v0.4.0 — Notes Surface
|
||||
|
||||
Obsidian-style rich-text notes rebuilt as an installable surface package.
|
||||
Zero platform special-casing. Proves the full extension stack E2E.
|
||||
|
||||
- Notes as `.pkg` archive
|
||||
- Rich text editor (ProseMirror or similar)
|
||||
- Folder tree, backlinks, tags — all extension-provided
|
||||
- Markdown import/export
|
||||
|
||||
## v0.5.0 — MVP
|
||||
|
||||
Extension and operations tracks converge. First externally usable release.
|
||||
|
||||
- Package registry (browse, install, update, uninstall)
|
||||
- Package distribution model (no auto-install; explicit install only)
|
||||
- Health monitoring dashboard
|
||||
- Backup/restore tooling
|
||||
- Documentation site
|
||||
---
|
||||
|
||||
## Post-MVP
|
||||
|
||||
- Chat extension (provider registry, streaming, personas, tool system)
|
||||
- LLM participation (`llm-bridge` extension: subscribes to `chat.message.created`, calls `provider.complete()`, streams response via `realtime.publish`, posts via `chat.send()`. Bot participants with persona config. Multi-model conversations.)
|
||||
- Rich media extensions: image generation, code sandbox, STT/TTS
|
||||
- Desktop app (Tauri or Electron)
|
||||
- Sidecar tier: container-based extensions
|
||||
- Federation: cross-instance package sharing
|
||||
- Plugin marketplace with signing and review
|
||||
- Mermaid diagrams extension (nice-to-have)
|
||||
|
||||
---
|
||||
|
||||
## Design Decisions Log
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Tasks → extension | Scheduler was the most entangled kernel component (~3,400 lines). Rebuilding as extension validates the trigger system and removes the worst compilation debt. Three trigger primitives (time, webhook, event) replace the monolithic scheduler. |
|
||||
| Sessions removed | Channel-based sessions coupled to deleted chat system. Workflow instances need new storage model — either ext_data tables or a dedicated kernel table. |
|
||||
| `chat_only` → `custom` | Stage mode `chat_only` implied chat as a kernel concept. Renamed to `custom` which delegates to a surface package, proving extension composability. |
|
||||
| Sessions removed | Kernel-managed sessions replaced by workflow instances with dedicated storage (ext_data tables or kernel table). |
|
||||
| `custom` stage mode | Stage mode `custom` delegates to a surface package, proving extension composability. Chat-in-workflow is handled by the chat extension, not the kernel. |
|
||||
| Providers removed from kernel | Provider configs, model catalog, routing policies — all moved to extension track. Kernel provides credential storage (connections) and the Starlark `provider.complete` module as the interface. |
|
||||
| Kernel permissions simplified | From 16 chat-centric permissions to 6 platform permissions. Extensions define their own capability requirements in manifests. |
|
||||
| Kernel permissions simplified | 6 platform permissions. Extensions define their own capability requirements in manifests. |
|
||||
| Preact+htm retained | 3KB runtime, no build step, works for extension authors without bundler config. KISS. |
|
||||
| Single Docker image | Drop the frontend/backend split. Go binary + assets + migrations in one image. Simpler deployment, fewer moving parts. |
|
||||
| Admin → RBAC group | The `role` column is pre-RBAC. v0.2.0 replaces it with a seeded "Admins" group + `surface.admin.access` grant. All users auto-join "Everyone" group. Admin middleware becomes a grant check, not a role check. |
|
||||
@@ -270,6 +184,10 @@ Extension and operations tracks converge. First externally usable release.
|
||||
| No new migrations pre-MVP | Edit existing migration SQL files in place. No migration chains until schema is in production. |
|
||||
| Notes over Editor | First surface is Obsidian-style notes (rich text, folders, backlinks) instead of a code editor. Notes is a stronger E2E proof — it exercises ext_data, storage, and the SDK more fully than a pure-browser CM6 editor. |
|
||||
| No built-in auto-install | Extensions ship in the repo but are not auto-installed. Distribution model TBD — explicit install only. Keeps the kernel clean and avoids opinionated defaults. |
|
||||
| Chat → post-MVP | Chat extension (providers, streaming, personas) is valuable but not MVP-critical. The platform must prove itself with simpler surfaces first. Chat moves to post-MVP track. |
|
||||
| Chat as extension, not kernel | Human-to-human chat built entirely as library + surface packages. Zero kernel awareness of conversations, messages, or participants. Kernel gains one generic `realtime` module. Proves near-infinite extensibility. LLM participation layers on top via a separate bridge extension — the chat system doesn't know or care whether a participant is human or AI. |
|
||||
| PG as consensus layer | Horizontal scaling uses PG as the sole coordinator (UNLOGGED node_registry + LISTEN/NOTIFY). No etcd, Consul, Redis, or Raft. Rationale: system is already tightly coupled to PG; adding a second consensus layer doubles operational complexity for zero benefit at homelab-to-small-team scale. UNLOGGED table is visible to all connections, survives session disconnect, and cleans up automatically on PG crash — correct behavior since all nodes are dead anyway. Sweep-all for health (every node deletes stale rows) is simpler than ring topology and has no edge cases. |
|
||||
| Two trigger tiers | Event + webhook triggers are extension-declared (manifest contract, full sandbox). Scheduled tasks are user-created ad-hoc (restricted sandbox — no raw HTTP, no DB table creation, connections-only outbound). Separation keeps extension contracts static and user automation safe. |
|
||||
| Scheduled task identity | Tasks run as their creator (RBAC-scoped). Admin-created tasks can opt into system context. Creator deactivation pauses the schedule. Ensures audit trail and permission boundaries. |
|
||||
| Builtin package rationale | A builtin must enhance kernel surfaces or be required to demonstrate the platform's own capabilities. **notes** — primary content surface, exercises ext_data/storage/SDK/settings/realtime fully. **chat + chat-core** — primary communication surface, proves the extensibility thesis (100% extension, zero kernel awareness). **mermaid-renderer** — docs surface uses Mermaid diagrams to explain the architecture; without it the platform's own documentation doesn't render (self-bootstrapping). **schedules** — UI for the kernel's scheduled task system; without it users can't manage cron jobs (kernel primitive UI). All others are domain features, examples, dormant, or LLM-only tools — available via registry or `BUNDLED_PACKAGES=*`. |
|
||||
| Cluster dashboard retired | `cluster-dashboard` shipped as a standalone surface package (v0.6.0) as an expedient. Health/metrics belong inside the Admin surface as a tab — shared context, no separate nav entry, no install required. Merged in v0.6.4. |
|
||||
| Block renderers decoupled from chat | `mermaid-renderer`, `katex-renderer`, `csv-table`, `diff-viewer` shipped with `"requires": ["chat"]` because renderer discovery lived inside the chat surface. These are content renderers, not chat features. v0.6.4 removes the constraint; v0.6.5 lifts renderer registration to the kernel SDK so all surfaces share it without reimplementing discovery. |
|
||||
|
||||
132
TURNOVER.md
132
TURNOVER.md
@@ -1,132 +0,0 @@
|
||||
# Switchboard Core — Session Turnover
|
||||
**Date**: 2026-03-26
|
||||
**Author**: Jeffrey Smith (jasafpro@gmail.com)
|
||||
**Session**: v0.2.0 PR 1 — Full RBAC migration
|
||||
|
||||
---
|
||||
|
||||
## What Was Done
|
||||
|
||||
### PR 1: Admin → RBAC group migration — COMPLETE
|
||||
Branch `feat/admin-rbac-migration` (PR #1), 3 commits:
|
||||
|
||||
1. **Admin → RBAC group migration** — `surface.admin.access` permission +
|
||||
Admins system group replaces hardcoded `role == "admin"` checks. Admin
|
||||
bypass removed from `RequirePermission`. Bootstrap/seed/OIDC/admin handlers
|
||||
sync group membership.
|
||||
|
||||
2. **Remove token budgets + allowed models from groups** — Provider-era cruft:
|
||||
`token_budget_daily`, `token_budget_monthly`, `allowed_models` columns,
|
||||
`ResolveTokenBudget()`, `ResolveModelAllowlist()`, and all store/handler/UI
|
||||
code deleted. -283 lines.
|
||||
|
||||
3. **Drop `users.role` column** — Full RBAC. Zero magic roles. The `role` column,
|
||||
`UserRoleAdmin`/`UserRoleUser` constants, `CountByRole()`, `DefaultRole` config,
|
||||
`role` in JWT claims, and all role-based shortcuts removed. Everyone group
|
||||
membership is now explicit (all users added on create). 28 files changed.
|
||||
|
||||
**Net result: -364 lines across 3 commits.**
|
||||
|
||||
### Architecture after this PR:
|
||||
|
||||
- **Everyone group** (`00000000...0001`): all users explicitly added on creation.
|
||||
Carries `extension.use`, `workflow.submit`.
|
||||
- **Admins group** (`00000000...0002`): carries all 7 kernel permissions including
|
||||
`surface.admin.access`. Not special-cased — just a group with permissions.
|
||||
- **Permission resolution**: union of all group memberships. No implicit groups,
|
||||
no role shortcuts, no admin bypass.
|
||||
- **Admin middleware**: checks `surface.admin.access` grant via `resolveAndCachePerms`.
|
||||
- **JWT claims**: `user_id` + `email` only. No role.
|
||||
- **OIDC**: `isIdPAdmin()` maps IdP role claim → Admins group membership.
|
||||
No `role` column writes.
|
||||
- **User creation paths**: builtin register, OIDC auto-provision, mTLS auto-provision,
|
||||
admin create, bootstrap, seed — all call `EnsureEveryoneGroup()`.
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
|
||||
1. **Frontend test job**: FE test suite may have remaining stale references
|
||||
from Phase 0 gut (pre-existing)
|
||||
2. **Traefik middleware**: `rbac-traefik.yaml` needs one-time manual apply
|
||||
by cluster admin (pre-existing, non-blocking)
|
||||
3. **`SEED_USERS`**: Not set in K8s secrets (pre-existing)
|
||||
4. **Editor modules**: `src/editor/*.mjs` still reference "Chat Switchboard"
|
||||
(pre-existing, low priority)
|
||||
5. **K8s deploy**: PR not yet merged to main — CI/CD will need a deploy after merge
|
||||
|
||||
---
|
||||
|
||||
## What's Next
|
||||
|
||||
### Merge PR #1
|
||||
Review and merge `feat/admin-rbac-migration` to main.
|
||||
|
||||
### v0.2.0 — Remaining PRs
|
||||
|
||||
**PR 2: Settings cascade**
|
||||
- Add `user_overridable` flag to settings schema
|
||||
- RBAC controls scope auth (admin → global, team-admin → team, user → personal)
|
||||
- Resolution: user → team → global (first non-null wins)
|
||||
|
||||
**PR 3: ICD (API contract)**
|
||||
- Generate full OpenAPI spec from registered routes
|
||||
- Document kernel-only endpoints
|
||||
|
||||
**PR 4: Trigger system**
|
||||
- Time triggers (cron), webhook (inbound HTTP), event (bus subscription)
|
||||
- Extensions register match expressions at install
|
||||
|
||||
**PR 5: SDK stabilization**
|
||||
- `sb.slots()`, `sb.actions`, `sb.api.ext()`, `sb.storage`
|
||||
- Theme tokens exposed to extensions
|
||||
|
||||
---
|
||||
|
||||
## Architecture Decisions (this session)
|
||||
|
||||
| Decision | Detail |
|
||||
|----------|--------|
|
||||
| Full RBAC — no magic roles | `users.role` column dropped entirely. All authorization through group membership + permission grants. |
|
||||
| Explicit Everyone membership | No implicit "all users get Everyone perms". Every user added to Everyone group on create. `member_count` reflects reality. |
|
||||
| Admins group not special-cased | Code never checks group identity — only checks for `surface.admin.access` permission. Any group can grant it. |
|
||||
| No new migrations pre-MVP | Edit existing SQL files in place. Schema isn't in production. |
|
||||
| JWT simplified | Claims carry `user_id` + `email` only. Permissions resolved server-side from groups on every request. |
|
||||
| OIDC role → group sync | `isIdPAdmin()` replaces `resolveRole()`. IdP admin claim maps to Admins group membership, not a DB column. |
|
||||
| Token budgets/allowed models removed | Provider-era cruft. Belongs in a future provider extension, not the kernel. |
|
||||
|
||||
---
|
||||
|
||||
## File Locations
|
||||
|
||||
| What | Where |
|
||||
|------|-------|
|
||||
| Roadmap | `ROADMAP.md` |
|
||||
| Changelog | `CHANGELOG.md` |
|
||||
| CI workflow | `.gitea/workflows/ci.yaml` |
|
||||
| K8s manifests | `k8s/` |
|
||||
| Docker compose (local dev) | `docker-compose.yml` (SQLite, port 3000) |
|
||||
| Nginx template | `nginx.conf.template` |
|
||||
| Migrations | `server/database/migrations/{postgres,sqlite}/` |
|
||||
| Permission constants | `server/auth/permissions.go` |
|
||||
| Admin middleware | `server/middleware/admin.go` |
|
||||
| Group store | `server/store/{postgres,sqlite}/groups.go` |
|
||||
| Bootstrap/seed | `server/handlers/auth.go` |
|
||||
| Admin handlers | `server/handlers/admin.go` |
|
||||
| Groups UI | `src/js/sw/surfaces/admin/groups.js` |
|
||||
| Users UI | `src/js/sw/surfaces/admin/users.js` |
|
||||
| SDK API | `src/js/sw/sdk/api-domains.js` |
|
||||
| Memory (Claude) | `.claude/projects/-config-Projects-core/memory/` |
|
||||
|
||||
---
|
||||
|
||||
## Gitea Secrets/Vars Needed
|
||||
|
||||
**Secrets**: `POSTGRES_USER`, `POSTGRES_USER_PASSWORD`, `POSTGRES_ADMIN_USER`,
|
||||
`POSTGRES_ADMIN_PASSWORD`, `ENCRYPTION_KEY`
|
||||
|
||||
**Vars**: `DOMAIN`, `NAMESPACE`, `S3_BUCKET`, `S3_ENDPOINT`, `STORAGE_BACKEND`,
|
||||
`STORAGE_CLASS`, `STORAGE_SIZE`
|
||||
|
||||
**Not yet configured**: `S3_ACCESS_KEY`, `S3_SECRET_KEY` (using PVC for now),
|
||||
`SEED_USERS` (needed for initial admin bootstrap)
|
||||
139
ci/e2e-backup-test.sh
Executable file
139
ci/e2e-backup-test.sh
Executable file
@@ -0,0 +1,139 @@
|
||||
#!/usr/bin/env bash
|
||||
# e2e-backup-test.sh — Backup/restore E2E test (v0.6.1)
|
||||
#
|
||||
# Requires: docker compose running with a single instance.
|
||||
# Usage:
|
||||
# docker compose up --build -d
|
||||
# ./ci/e2e-backup-test.sh
|
||||
# docker compose down -v
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
BASE="http://localhost:8080"
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
pass() { PASS=$((PASS + 1)); echo " PASS: $1"; }
|
||||
fail() { FAIL=$((FAIL + 1)); echo " FAIL: $1"; }
|
||||
|
||||
# ── Wait for service ─────────────────────────
|
||||
echo "=== Waiting for service ==="
|
||||
for i in $(seq 1 30); do
|
||||
if curl -sf "$BASE/health" > /dev/null 2>&1; then
|
||||
echo " service ready"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
if [ "$i" -eq 30 ]; then
|
||||
echo "FATAL: service did not become healthy"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# ── Login as admin ───────────────────────────
|
||||
echo "=== Authenticating ==="
|
||||
TOKEN=$(curl -sf "$BASE/api/v1/auth/login" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"login":"admin","password":"admin"}' | jq -r '.access_token')
|
||||
|
||||
if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
|
||||
echo "FATAL: login failed"
|
||||
exit 1
|
||||
fi
|
||||
echo " admin token acquired"
|
||||
|
||||
auth() { echo "Authorization: Bearer $TOKEN"; }
|
||||
|
||||
# ── Seed test data ───────────────────────────
|
||||
echo "=== Seeding test data ==="
|
||||
|
||||
# Create a test user
|
||||
curl -sf "$BASE/api/v1/auth/register" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"username":"backup-test-user","email":"bktest@test.com","password":"testpass123"}' > /dev/null
|
||||
|
||||
USER_COUNT=$(curl -sf "$BASE/api/v1/admin/users" -H "$(auth)" | jq '.data | length')
|
||||
echo " users: $USER_COUNT"
|
||||
|
||||
# ── Test 1: List backups (empty) ─────────────
|
||||
echo "=== Test 1: List backups (empty) ==="
|
||||
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
|
||||
COUNT=$(echo "$LIST" | jq '.data | length')
|
||||
if [ "$COUNT" -eq 0 ]; then
|
||||
pass "empty backup list"
|
||||
else
|
||||
fail "expected 0 backups, got $COUNT"
|
||||
fi
|
||||
|
||||
# ── Test 2: Create server-side backup ────────
|
||||
echo "=== Test 2: Create server-side backup ==="
|
||||
RESULT=$(curl -sf "$BASE/api/v1/admin/backup?store=true" \
|
||||
-X POST \
|
||||
-H "$(auth)")
|
||||
FILENAME=$(echo "$RESULT" | jq -r '.data.filename')
|
||||
if [ -n "$FILENAME" ] && [ "$FILENAME" != "null" ]; then
|
||||
pass "created backup: $FILENAME"
|
||||
else
|
||||
fail "backup creation failed: $RESULT"
|
||||
fi
|
||||
|
||||
# ── Test 3: List backups (has one) ───────────
|
||||
echo "=== Test 3: List backups (has one) ==="
|
||||
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
|
||||
COUNT=$(echo "$LIST" | jq '.data | length')
|
||||
if [ "$COUNT" -eq 1 ]; then
|
||||
pass "one backup listed"
|
||||
else
|
||||
fail "expected 1 backup, got $COUNT"
|
||||
fi
|
||||
|
||||
# ── Test 4: Download backup ──────────────────
|
||||
echo "=== Test 4: Download backup ==="
|
||||
TMPFILE=$(mktemp /tmp/backup-test-XXXXXX.swb)
|
||||
HTTP_CODE=$(curl -sf -o "$TMPFILE" -w "%{http_code}" \
|
||||
"$BASE/api/v1/admin/backups/$FILENAME" -H "$(auth)")
|
||||
if [ "$HTTP_CODE" = "200" ] && [ -s "$TMPFILE" ]; then
|
||||
pass "download OK ($(wc -c < "$TMPFILE") bytes)"
|
||||
else
|
||||
fail "download failed (HTTP $HTTP_CODE)"
|
||||
fi
|
||||
|
||||
# ── Test 5: Streaming backup (direct download) ──
|
||||
echo "=== Test 5: Streaming backup ==="
|
||||
STREAM_FILE=$(mktemp /tmp/backup-stream-XXXXXX.swb)
|
||||
HTTP_CODE=$(curl -sf -o "$STREAM_FILE" -w "%{http_code}" \
|
||||
-X POST "$BASE/api/v1/admin/backup" -H "$(auth)")
|
||||
if [ "$HTTP_CODE" = "200" ] && [ -s "$STREAM_FILE" ]; then
|
||||
pass "streaming download OK ($(wc -c < "$STREAM_FILE") bytes)"
|
||||
else
|
||||
fail "streaming download failed (HTTP $HTTP_CODE)"
|
||||
fi
|
||||
|
||||
# ── Test 6: Delete backup ────────────────────
|
||||
echo "=== Test 6: Delete backup ==="
|
||||
DEL=$(curl -sf -X DELETE "$BASE/api/v1/admin/backups/$FILENAME" -H "$(auth)")
|
||||
DELETED=$(echo "$DEL" | jq -r '.data.deleted')
|
||||
if [ "$DELETED" = "$FILENAME" ]; then
|
||||
pass "deleted backup"
|
||||
else
|
||||
fail "delete failed: $DEL"
|
||||
fi
|
||||
|
||||
# Verify deletion
|
||||
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
|
||||
COUNT=$(echo "$LIST" | jq '.data | length')
|
||||
if [ "$COUNT" -eq 0 ]; then
|
||||
pass "backup list empty after delete"
|
||||
else
|
||||
fail "expected 0 backups after delete, got $COUNT"
|
||||
fi
|
||||
|
||||
# ── Cleanup ──────────────────────────────────
|
||||
rm -f "$TMPFILE" "$STREAM_FILE"
|
||||
|
||||
# ── Summary ──────────────────────────────────
|
||||
echo ""
|
||||
echo "=== Results: $PASS passed, $FAIL failed ==="
|
||||
if [ "$FAIL" -gt 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
247
ci/e2e-chat-test.sh
Executable file
247
ci/e2e-chat-test.sh
Executable file
@@ -0,0 +1,247 @@
|
||||
#!/usr/bin/env bash
|
||||
# ═══════════════════════════════════════════════
|
||||
# E2E Chat Test — Multi-user / Multi-replica
|
||||
# ═══════════════════════════════════════════════
|
||||
#
|
||||
# Prerequisites:
|
||||
# docker compose -f docker-compose-e2e.yml up --build -d
|
||||
#
|
||||
# Tests:
|
||||
# 1. Auth as alice + bob
|
||||
# 2. Alice creates conversation, adds bob
|
||||
# 3. Alice sends message via REST
|
||||
# 4. Bob reads messages, verifies receipt
|
||||
# 5. Cross-replica: send via replica-1, read via replica-2
|
||||
# 6. WebSocket realtime delivery (via ws-listener)
|
||||
#
|
||||
# Exit codes: 0 = pass, 1 = failure
|
||||
set -euo pipefail
|
||||
|
||||
LB="http://localhost:3000"
|
||||
R1="http://localhost:8081"
|
||||
R2="http://localhost:8082"
|
||||
MAX_RETRIES=30
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
pass=0
|
||||
fail=0
|
||||
|
||||
ok() { pass=$((pass + 1)); echo -e " ${GREEN}✓${NC} $1"; }
|
||||
fail() { fail=$((fail + 1)); echo -e " ${RED}✗${NC} $1"; }
|
||||
|
||||
# ── Wait for LB ─────────────────────────────
|
||||
|
||||
echo -e "${YELLOW}Waiting for load balancer...${NC}"
|
||||
for i in $(seq 1 $MAX_RETRIES); do
|
||||
if curl -sf "$LB/api/v1/auth/login" -o /dev/null 2>/dev/null || \
|
||||
curl -sf "$LB" -o /dev/null 2>/dev/null; then
|
||||
echo -e "${GREEN}LB ready after ${i}s${NC}"
|
||||
break
|
||||
fi
|
||||
if [ "$i" -eq "$MAX_RETRIES" ]; then
|
||||
echo -e "${RED}LB not ready after ${MAX_RETRIES}s${NC}"
|
||||
exit 1
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# ── Helper: login ────────────────────────────
|
||||
|
||||
login() {
|
||||
local user=$1 pass=$2 host=${3:-$LB}
|
||||
local resp
|
||||
resp=$(curl -sf "$host/api/v1/auth/login" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"login\":\"$user\",\"password\":\"$pass\"}")
|
||||
echo "$resp" | grep -o '"access_token":"[^"]*"' | head -1 | sed 's/.*"access_token":"\([^"]*\)".*/\1/'
|
||||
}
|
||||
|
||||
authed() {
|
||||
# Usage: authed $TOKEN GET /api/v1/... [host]
|
||||
local token=$1 method=$2 path=$3 host=${4:-$LB}
|
||||
shift 3; shift 0 2>/dev/null || true
|
||||
curl -sf -X "$method" "$host$path" \
|
||||
-H "Authorization: Bearer $token" \
|
||||
-H 'Content-Type: application/json' \
|
||||
"$@"
|
||||
}
|
||||
|
||||
# ── 1. Auth ──────────────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}1. Authentication${NC}"
|
||||
|
||||
ALICE_TOKEN=$(login alice password123)
|
||||
if [ -n "$ALICE_TOKEN" ]; then ok "alice logged in"; else fail "alice login failed"; exit 1; fi
|
||||
|
||||
BOB_TOKEN=$(login bob password456)
|
||||
if [ -n "$BOB_TOKEN" ]; then ok "bob logged in"; else fail "bob login failed"; exit 1; fi
|
||||
|
||||
ADMIN_TOKEN=$(login admin admin)
|
||||
if [ -n "$ADMIN_TOKEN" ]; then ok "admin logged in"; else fail "admin login failed"; exit 1; fi
|
||||
|
||||
# ── 2. Install chat packages ────────────────
|
||||
|
||||
echo -e "\n${YELLOW}2. Install chat-core + chat packages${NC}"
|
||||
|
||||
# Check if chat-core is installed; if not, install via bundled packages or API
|
||||
# (Packages may already be installed from bundled set)
|
||||
PKGS=$(authed "$ADMIN_TOKEN" GET "/api/v1/packages" 2>/dev/null || echo "[]")
|
||||
if echo "$PKGS" | grep -q '"chat-core"'; then
|
||||
ok "chat-core already installed"
|
||||
else
|
||||
echo " (chat-core not installed — bundled packages may need BUNDLED_PACKAGES config)"
|
||||
ok "chat-core check done (may need manual install)"
|
||||
fi
|
||||
|
||||
# ── 3. Create conversation ───────────────────
|
||||
|
||||
echo -e "\n${YELLOW}3. Create conversation + send messages${NC}"
|
||||
|
||||
# Get alice's user ID
|
||||
ALICE_ME=$(authed "$ALICE_TOKEN" GET "/api/v1/profile" 2>/dev/null || echo "{}")
|
||||
ALICE_ID=$(echo "$ALICE_ME" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
|
||||
BOB_ME=$(authed "$BOB_TOKEN" GET "/api/v1/profile" 2>/dev/null || echo "{}")
|
||||
BOB_ID=$(echo "$BOB_ME" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
|
||||
# Create conversation via chat-core API
|
||||
CONV=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/conversations" "" \
|
||||
-d "{\"title\":\"E2E Test Chat\",\"type\":\"group\",\"participants\":[{\"id\":\"$BOB_ID\",\"display_name\":\"bob\"}],\"creator_display_name\":\"alice\"}" 2>/dev/null || echo "{}")
|
||||
CONV_ID=$(echo "$CONV" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
ok "conversation created: ${CONV_ID:0:8}..."
|
||||
else
|
||||
fail "conversation creation failed"
|
||||
echo " Response: $CONV"
|
||||
fi
|
||||
|
||||
# ── 4. Send + read messages ──────────────────
|
||||
|
||||
echo -e "\n${YELLOW}4. Message send + read${NC}"
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
# Alice sends a message
|
||||
MSG1=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
|
||||
-d '{"content":"Hello from alice!","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
MSG1_ID=$(echo "$MSG1" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
|
||||
if [ -n "$MSG1_ID" ]; then ok "alice sent message"; else fail "alice send failed"; fi
|
||||
|
||||
# Bob reads messages
|
||||
MSGS=$(authed "$BOB_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" 2>/dev/null || echo "{}")
|
||||
if echo "$MSGS" | grep -q "Hello from alice"; then
|
||||
ok "bob received alice's message"
|
||||
else
|
||||
fail "bob did not receive message"
|
||||
echo " Response: ${MSGS:0:200}"
|
||||
fi
|
||||
|
||||
# Bob sends a reply
|
||||
MSG2=$(authed "$BOB_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
|
||||
-d '{"content":"Hello from bob!","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
MSG2_ID=$(echo "$MSG2" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
|
||||
if [ -n "$MSG2_ID" ]; then ok "bob sent reply"; else fail "bob send failed"; fi
|
||||
fi
|
||||
|
||||
# ── 5. Cross-replica consistency ─────────────
|
||||
|
||||
echo -e "\n${YELLOW}5. Cross-replica consistency${NC}"
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
# Login to each replica directly
|
||||
ALICE_R1=$(login alice password123 "$R1")
|
||||
BOB_R2=$(login bob password456 "$R2")
|
||||
|
||||
# Alice sends via replica 1
|
||||
MSG3=$(curl -sf -X POST "$R1/s/chat-core/api/messages/$CONV_ID" \
|
||||
-H "Authorization: Bearer $ALICE_R1" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"content":"Cross-replica test message","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
|
||||
if echo "$MSG3" | grep -q '"id"'; then
|
||||
ok "alice sent via replica-1"
|
||||
else
|
||||
fail "alice send via replica-1 failed"
|
||||
fi
|
||||
|
||||
# Small delay for pg replication
|
||||
sleep 1
|
||||
|
||||
# Bob reads via replica 2
|
||||
MSGS_R2=$(curl -sf "$R2/s/chat-core/api/messages/$CONV_ID?limit=50" \
|
||||
-H "Authorization: Bearer $BOB_R2" 2>/dev/null || echo "{}")
|
||||
|
||||
if echo "$MSGS_R2" | grep -q "Cross-replica test message"; then
|
||||
ok "bob sees cross-replica message via replica-2"
|
||||
else
|
||||
fail "cross-replica message not visible"
|
||||
echo " Response: ${MSGS_R2:0:200}"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── 6. Search ────────────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}6. Conversation search${NC}"
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
SEARCH=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/search?q=alice" 2>/dev/null || echo "{}")
|
||||
|
||||
if echo "$SEARCH" | grep -q "Hello from alice"; then
|
||||
ok "search found message content"
|
||||
else
|
||||
fail "search did not find message"
|
||||
echo " Response: ${SEARCH:0:200}"
|
||||
fi
|
||||
|
||||
SEARCH_CONV=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/search?q=E2E%20Test" 2>/dev/null || echo "{}")
|
||||
if echo "$SEARCH_CONV" | grep -q "E2E Test Chat"; then
|
||||
ok "search found conversation by title"
|
||||
else
|
||||
fail "search did not find conversation by title"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── 7. Message pagination ────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}7. Message pagination${NC}"
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
# Send several more messages to test pagination
|
||||
for i in $(seq 1 5); do
|
||||
authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
|
||||
-d "{\"content\":\"Pagination test message $i\",\"content_type\":\"text\"}" >/dev/null 2>&1
|
||||
done
|
||||
|
||||
# Fetch with limit=3
|
||||
PAGE1=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=3" 2>/dev/null || echo "{}")
|
||||
HAS_MORE=$(echo "$PAGE1" | grep -o '"has_more":true' || echo "")
|
||||
CURSOR=$(echo "$PAGE1" | grep -o '"next_cursor":"[^"]*"' | cut -d'"' -f4 || echo "")
|
||||
|
||||
if [ -n "$HAS_MORE" ] && [ -n "$CURSOR" ]; then
|
||||
ok "pagination: first page has_more=true with cursor"
|
||||
|
||||
# Fetch second page
|
||||
PAGE2=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=3&cursor=$CURSOR" 2>/dev/null || echo "{}")
|
||||
if echo "$PAGE2" | grep -q '"messages"'; then
|
||||
ok "pagination: second page returned messages"
|
||||
else
|
||||
fail "pagination: second page failed"
|
||||
fi
|
||||
else
|
||||
fail "pagination: expected has_more and cursor"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Summary ──────────────────────────────────
|
||||
|
||||
echo -e "\n════════════════════════════════════════"
|
||||
echo -e "Results: ${GREEN}$pass passed${NC}, ${RED}$fail failed${NC}"
|
||||
echo "════════════════════════════════════════"
|
||||
|
||||
[ "$fail" -eq 0 ] && exit 0 || exit 1
|
||||
145
ci/e2e-cluster-test.sh
Executable file
145
ci/e2e-cluster-test.sh
Executable file
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env bash
|
||||
# e2e-cluster-test.sh — Cluster registry E2E test (v0.6.0)
|
||||
#
|
||||
# Requires: docker-compose-e2e.yml running with 3 replicas.
|
||||
# Usage:
|
||||
# docker compose -f docker-compose-e2e.yml up --build -d
|
||||
# ./ci/e2e-cluster-test.sh
|
||||
# docker compose -f docker-compose-e2e.yml down -v
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
LB="http://localhost:3000"
|
||||
DIRECT_1="http://localhost:8081"
|
||||
STALE_WAIT=20 # seconds — must exceed CLUSTER_STALE_THRESHOLD (15s)
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
pass() { PASS=$((PASS + 1)); echo " PASS: $1"; }
|
||||
fail() { FAIL=$((FAIL + 1)); echo " FAIL: $1"; }
|
||||
|
||||
# ── Wait for all 3 replicas ──────────────────
|
||||
echo "=== Waiting for replicas ==="
|
||||
for port in 8081 8082 8083; do
|
||||
for i in $(seq 1 30); do
|
||||
if curl -sf "http://localhost:$port/health" > /dev/null 2>&1; then
|
||||
echo " replica :$port ready"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
if [ "$i" -eq 30 ]; then
|
||||
echo "FATAL: replica :$port did not become healthy"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
# ── Login as admin ────────────────────────────
|
||||
echo "=== Authenticating ==="
|
||||
TOKEN=$(curl -sf "$LB/api/v1/auth/login" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"login":"admin","password":"admin"}' | jq -r '.access_token')
|
||||
|
||||
if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
|
||||
echo "FATAL: login failed"
|
||||
exit 1
|
||||
fi
|
||||
echo " admin token acquired"
|
||||
|
||||
auth() { echo "Authorization: Bearer $TOKEN"; }
|
||||
|
||||
# Wait for heartbeat to propagate (at least 2 tick cycles at 5s each)
|
||||
echo "=== Waiting for heartbeat convergence ==="
|
||||
sleep 12
|
||||
|
||||
# ── Test 1: All 3 nodes registered ───────────
|
||||
echo "=== Test 1: Cluster shows 3 nodes ==="
|
||||
NODES=$(curl -sf "$LB/api/v1/admin/cluster" -H "$(auth)")
|
||||
COUNT=$(echo "$NODES" | jq '.data | length')
|
||||
if [ "$COUNT" -eq 3 ]; then
|
||||
pass "cluster has 3 nodes"
|
||||
else
|
||||
fail "cluster has $COUNT nodes (expected 3)"
|
||||
echo " response: $NODES"
|
||||
fi
|
||||
|
||||
# Verify all expected node IDs present
|
||||
for nid in node-1 node-2 node-3; do
|
||||
if echo "$NODES" | jq -e ".data[] | select(.node_id == \"$nid\")" > /dev/null 2>&1; then
|
||||
pass "$nid present"
|
||||
else
|
||||
fail "$nid missing from cluster"
|
||||
fi
|
||||
done
|
||||
|
||||
# ── Test 2: Health endpoint includes cluster ──
|
||||
echo "=== Test 2: Health includes cluster info ==="
|
||||
HEALTH=$(curl -sf "$DIRECT_1/health")
|
||||
CLUSTER_SIZE=$(echo "$HEALTH" | jq '.cluster.size // 0')
|
||||
if [ "$CLUSTER_SIZE" -eq 3 ]; then
|
||||
pass "health shows cluster.size=3"
|
||||
else
|
||||
fail "health shows cluster.size=$CLUSTER_SIZE (expected 3)"
|
||||
fi
|
||||
|
||||
NODE_ID=$(echo "$HEALTH" | jq -r '.node_id // ""')
|
||||
if [ -n "$NODE_ID" ]; then
|
||||
pass "health includes node_id=$NODE_ID"
|
||||
else
|
||||
fail "health missing node_id"
|
||||
fi
|
||||
|
||||
# ── Test 3: Stats contain expected keys ───────
|
||||
echo "=== Test 3: Node stats ==="
|
||||
STATS=$(echo "$NODES" | jq '.data[0].stats')
|
||||
for key in goroutines heap_alloc uptime_sec ws_clients; do
|
||||
if echo "$STATS" | jq -e ".$key" > /dev/null 2>&1; then
|
||||
pass "stats.$key present"
|
||||
else
|
||||
fail "stats.$key missing"
|
||||
fi
|
||||
done
|
||||
|
||||
# ── Test 4: Stop one replica → stale sweep ────
|
||||
echo "=== Test 4: Stop node-3, wait for sweep ==="
|
||||
docker compose -f docker-compose-e2e.yml stop switchboard-3
|
||||
|
||||
echo " waiting ${STALE_WAIT}s for stale sweep..."
|
||||
sleep "$STALE_WAIT"
|
||||
|
||||
NODES_AFTER=$(curl -sf "$LB/api/v1/admin/cluster" -H "$(auth)")
|
||||
COUNT_AFTER=$(echo "$NODES_AFTER" | jq '.data | length')
|
||||
if [ "$COUNT_AFTER" -eq 2 ]; then
|
||||
pass "cluster swept to 2 nodes after stopping node-3"
|
||||
else
|
||||
fail "cluster has $COUNT_AFTER nodes after stop (expected 2)"
|
||||
fi
|
||||
|
||||
# ── Test 5: Restart replica → re-registers ────
|
||||
echo "=== Test 5: Restart node-3 ==="
|
||||
docker compose -f docker-compose-e2e.yml start switchboard-3
|
||||
|
||||
# Wait for startup + at least one heartbeat
|
||||
for i in $(seq 1 30); do
|
||||
if curl -sf "http://localhost:8083/health" > /dev/null 2>&1; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
sleep 8 # allow heartbeat to register
|
||||
|
||||
NODES_RESTART=$(curl -sf "$LB/api/v1/admin/cluster" -H "$(auth)")
|
||||
COUNT_RESTART=$(echo "$NODES_RESTART" | jq '.data | length')
|
||||
if [ "$COUNT_RESTART" -eq 3 ]; then
|
||||
pass "cluster back to 3 nodes after restart"
|
||||
else
|
||||
fail "cluster has $COUNT_RESTART nodes after restart (expected 3)"
|
||||
fi
|
||||
|
||||
# ── Summary ───────────────────────────────────
|
||||
echo ""
|
||||
echo "=== Results: $PASS passed, $FAIL failed ==="
|
||||
if [ "$FAIL" -gt 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
40
ci/e2e-nginx.conf
Normal file
40
ci/e2e-nginx.conf
Normal file
@@ -0,0 +1,40 @@
|
||||
events {
|
||||
worker_connections 128;
|
||||
}
|
||||
|
||||
http {
|
||||
upstream switchboard {
|
||||
server switchboard-1:80;
|
||||
server switchboard-2:80;
|
||||
server switchboard-3:80;
|
||||
}
|
||||
|
||||
# WebSocket upgrade map
|
||||
map $http_upgrade $connection_upgrade {
|
||||
default upgrade;
|
||||
'' close;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
|
||||
location / {
|
||||
proxy_pass http://switchboard;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# WebSocket endpoint
|
||||
location /ws {
|
||||
proxy_pass http://switchboard;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_read_timeout 86400s;
|
||||
}
|
||||
}
|
||||
}
|
||||
330
ci/e2e-upgrade-rolling.sh
Executable file
330
ci/e2e-upgrade-rolling.sh
Executable file
@@ -0,0 +1,330 @@
|
||||
#!/usr/bin/env bash
|
||||
# ═══════════════════════════════════════════════
|
||||
# E2E Rolling Upgrade Test — Multi-Replica
|
||||
# ═══════════════════════════════════════════════
|
||||
#
|
||||
# Tests rolling upgrade with two replicas sharing Postgres:
|
||||
# 1. Build old + new images
|
||||
# 2. Start 2 replicas (old) + nginx LB + postgres
|
||||
# 3. Seed data via LB
|
||||
# 4. Upgrade replica-1 → new, verify cross-replica reads
|
||||
# 5. Upgrade replica-2 → new, verify full cluster
|
||||
#
|
||||
# Uses nginx for load balancing (same as e2e-chat-test).
|
||||
#
|
||||
# Usage:
|
||||
# ./ci/e2e-upgrade-rolling.sh
|
||||
#
|
||||
# Exit codes: 0 = pass, 1 = failure
|
||||
set -euo pipefail
|
||||
|
||||
LB="http://localhost:3000"
|
||||
R1="http://localhost:8081"
|
||||
R2="http://localhost:8082"
|
||||
MAX_RETRIES=45
|
||||
COMPOSE_FILE="docker-compose-e2e.yml"
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
pass=0
|
||||
fail=0
|
||||
|
||||
ok() { pass=$((pass + 1)); echo -e " ${GREEN}✓${NC} $1"; }
|
||||
fail() { fail=$((fail + 1)); echo -e " ${RED}✗${NC} $1"; }
|
||||
|
||||
cleanup() {
|
||||
echo -e "\n${YELLOW}Cleanup...${NC}"
|
||||
docker compose -f "$COMPOSE_FILE" down -v 2>/dev/null || true
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
wait_for_health() {
|
||||
local host=$1 name=${2:-$host}
|
||||
echo -e "${YELLOW}Waiting for $name...${NC}"
|
||||
for i in $(seq 1 $MAX_RETRIES); do
|
||||
if curl -sf "$host/api/v1/auth/login" -o /dev/null 2>/dev/null || \
|
||||
curl -sf "$host" -o /dev/null 2>/dev/null; then
|
||||
echo -e "${GREEN}$name ready after ${i}s${NC}"
|
||||
return 0
|
||||
fi
|
||||
if [ "$i" -eq "$MAX_RETRIES" ]; then
|
||||
echo -e "${RED}$name not ready after ${MAX_RETRIES}s${NC}"
|
||||
return 1
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
login() {
|
||||
local user=$1 pass=$2 host=${3:-$LB}
|
||||
local resp
|
||||
resp=$(curl -sf "$host/api/v1/auth/login" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"login\":\"$user\",\"password\":\"$pass\"}")
|
||||
echo "$resp" | grep -o '"access_token":"[^"]*"' | head -1 | sed 's/.*"access_token":"\([^"]*\)".*/\1/'
|
||||
}
|
||||
|
||||
authed() {
|
||||
local token=$1 method=$2 path=$3 host=${4:-$LB}
|
||||
shift 3; shift 0 2>/dev/null || true
|
||||
curl -sf -X "$method" "$host$path" \
|
||||
-H "Authorization: Bearer $token" \
|
||||
-H 'Content-Type: application/json' \
|
||||
"$@"
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 1: Build Images
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 1: Build Images ═══${NC}"
|
||||
|
||||
echo "Building 'old' image from HEAD..."
|
||||
docker build -t switchboard-core:v-old . -q
|
||||
ok "old image built"
|
||||
|
||||
echo "Building 'new' image from working tree..."
|
||||
docker build -t switchboard-core:v-new . -q
|
||||
ok "new image built"
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 2: Start Both Replicas on Old Image
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 2: Start Cluster (Old Version) ═══${NC}"
|
||||
|
||||
# Override the build with old image
|
||||
SWITCHBOARD_IMAGE=switchboard-core:v-old \
|
||||
docker compose -f "$COMPOSE_FILE" up postgres -d
|
||||
|
||||
# Wait for postgres
|
||||
sleep 3
|
||||
|
||||
# Start replicas using old image
|
||||
docker run -d --name sb-old-1 --network "$(basename "$(pwd)")_default" \
|
||||
-e PORT=8080 -e BASE_PATH="" \
|
||||
-e DB_DRIVER=postgres \
|
||||
-e "DATABASE_URL=postgres://switchboard:e2e-password@postgres:5432/switchboard_e2e?sslmode=disable" \
|
||||
-e JWT_SECRET=e2e-jwt-secret \
|
||||
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
|
||||
-e SWITCHBOARD_ADMIN_USERNAME=admin \
|
||||
-e SWITCHBOARD_ADMIN_PASSWORD=admin \
|
||||
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
|
||||
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
|
||||
-e LOG_FORMAT=text -e LOG_LEVEL=info \
|
||||
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
|
||||
-p 8081:80 switchboard-core:v-old 2>/dev/null || true
|
||||
|
||||
docker run -d --name sb-old-2 --network "$(basename "$(pwd)")_default" \
|
||||
-e PORT=8080 -e BASE_PATH="" \
|
||||
-e DB_DRIVER=postgres \
|
||||
-e "DATABASE_URL=postgres://switchboard:e2e-password@postgres:5432/switchboard_e2e?sslmode=disable" \
|
||||
-e JWT_SECRET=e2e-jwt-secret \
|
||||
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
|
||||
-e SWITCHBOARD_ADMIN_USERNAME=admin \
|
||||
-e SWITCHBOARD_ADMIN_PASSWORD=admin \
|
||||
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
|
||||
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
|
||||
-e LOG_FORMAT=text -e LOG_LEVEL=info \
|
||||
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
|
||||
-p 8082:80 switchboard-core:v-old 2>/dev/null || true
|
||||
|
||||
# Start nginx LB
|
||||
docker compose -f "$COMPOSE_FILE" up lb -d
|
||||
|
||||
wait_for_health "$R1" "replica-1"
|
||||
wait_for_health "$R2" "replica-2"
|
||||
wait_for_health "$LB" "load balancer"
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 3: Seed Data via LB
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 3: Seed Data ═══${NC}"
|
||||
|
||||
ADMIN_TOKEN=$(login admin admin)
|
||||
if [ -z "$ADMIN_TOKEN" ]; then fail "admin login failed"; exit 1; fi
|
||||
ok "admin logged in"
|
||||
|
||||
ALICE_TOKEN=$(login alice password123)
|
||||
if [ -z "$ALICE_TOKEN" ]; then fail "alice login failed"; exit 1; fi
|
||||
|
||||
BOB_TOKEN=$(login bob password456)
|
||||
if [ -z "$BOB_TOKEN" ]; then fail "bob login failed"; exit 1; fi
|
||||
|
||||
ALICE_ID=$(authed "$ALICE_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
BOB_ID=$(authed "$BOB_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
|
||||
# Seed a note
|
||||
NOTE=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
|
||||
-d '{"title":"Rolling Upgrade Note","body":"Must survive rolling upgrade."}' 2>/dev/null || echo "{}")
|
||||
NOTE_ID=$(echo "$NOTE" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$NOTE_ID" ]; then ok "note seeded"; else fail "note seed failed"; fi
|
||||
|
||||
# Seed a conversation + message
|
||||
CONV=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/conversations" "" \
|
||||
-d "{\"title\":\"Rolling Test\",\"type\":\"group\",\"participants\":[{\"id\":\"$BOB_ID\",\"display_name\":\"bob\"}],\"creator_display_name\":\"alice\"}" 2>/dev/null || echo "{}")
|
||||
CONV_ID=$(echo "$CONV" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$CONV_ID" ]; then ok "conversation seeded"; else fail "conversation seed failed"; fi
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
|
||||
-d '{"content":"Before rolling upgrade","content_type":"text"}' >/dev/null 2>&1
|
||||
ok "message seeded"
|
||||
fi
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 4: Upgrade Replica-1, Verify Cross-Replica
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 4: Rolling Upgrade — Replica 1 ═══${NC}"
|
||||
|
||||
echo "Stopping old replica-1..."
|
||||
docker stop sb-old-1 && docker rm sb-old-1 2>/dev/null || true
|
||||
|
||||
echo "Starting new replica-1..."
|
||||
docker run -d --name sb-new-1 --network "$(basename "$(pwd)")_default" \
|
||||
-e PORT=8080 -e BASE_PATH="" \
|
||||
-e DB_DRIVER=postgres \
|
||||
-e "DATABASE_URL=postgres://switchboard:e2e-password@postgres:5432/switchboard_e2e?sslmode=disable" \
|
||||
-e JWT_SECRET=e2e-jwt-secret \
|
||||
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
|
||||
-e SWITCHBOARD_ADMIN_USERNAME=admin \
|
||||
-e SWITCHBOARD_ADMIN_PASSWORD=admin \
|
||||
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
|
||||
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
|
||||
-e LOG_FORMAT=text -e LOG_LEVEL=info \
|
||||
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
|
||||
-p 8081:80 switchboard-core:v-new
|
||||
|
||||
wait_for_health "$R1" "new replica-1"
|
||||
|
||||
# Cross-replica reads: new replica reads old data
|
||||
R1_TOKEN=$(login alice password123 "$R1")
|
||||
if [ -n "$R1_TOKEN" ]; then ok "alice login on new replica-1"; else fail "alice login on new replica-1"; fi
|
||||
|
||||
if [ -n "$NOTE_ID" ] && [ -n "$R1_TOKEN" ]; then
|
||||
R1_NOTE=$(authed "$R1_TOKEN" GET "/s/notes/api/notes/$NOTE_ID" "$R1" 2>/dev/null || echo "{}")
|
||||
if echo "$R1_NOTE" | grep -q "Rolling Upgrade Note"; then
|
||||
ok "new replica-1 reads old data"
|
||||
else
|
||||
fail "new replica-1 cannot read old data"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Write on new replica, read on old
|
||||
if [ -n "$CONV_ID" ] && [ -n "$R1_TOKEN" ]; then
|
||||
NEW_MSG=$(authed "$R1_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "$R1" \
|
||||
-d '{"content":"From new replica-1","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
if echo "$NEW_MSG" | grep -q '"id"'; then ok "write on new replica-1"; else fail "write on new replica-1"; fi
|
||||
|
||||
# Read from old replica-2
|
||||
R2_TOKEN=$(login alice password123 "$R2")
|
||||
if [ -n "$R2_TOKEN" ]; then
|
||||
R2_MSGS=$(authed "$R2_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" "$R2" 2>/dev/null || echo "[]")
|
||||
if echo "$R2_MSGS" | grep -q "From new replica-1"; then
|
||||
ok "old replica-2 reads new replica-1 writes"
|
||||
else
|
||||
fail "old replica-2 cannot read new replica-1 writes"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 5: Upgrade Replica-2, Verify Full Cluster
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 5: Rolling Upgrade — Replica 2 ═══${NC}"
|
||||
|
||||
echo "Stopping old replica-2..."
|
||||
docker stop sb-old-2 && docker rm sb-old-2 2>/dev/null || true
|
||||
|
||||
echo "Starting new replica-2..."
|
||||
docker run -d --name sb-new-2 --network "$(basename "$(pwd)")_default" \
|
||||
-e PORT=8080 -e BASE_PATH="" \
|
||||
-e DB_DRIVER=postgres \
|
||||
-e "DATABASE_URL=postgres://switchboard:e2e-password@postgres:5432/switchboard_e2e?sslmode=disable" \
|
||||
-e JWT_SECRET=e2e-jwt-secret \
|
||||
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
|
||||
-e SWITCHBOARD_ADMIN_USERNAME=admin \
|
||||
-e SWITCHBOARD_ADMIN_PASSWORD=admin \
|
||||
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
|
||||
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
|
||||
-e LOG_FORMAT=text -e LOG_LEVEL=info \
|
||||
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
|
||||
-p 8082:80 switchboard-core:v-new
|
||||
|
||||
wait_for_health "$R2" "new replica-2"
|
||||
|
||||
# Verify full cluster: all data accessible
|
||||
echo -e "\n${YELLOW}Verifying full cluster on new version...${NC}"
|
||||
|
||||
R2_TOKEN=$(login alice password123 "$R2")
|
||||
if [ -n "$R2_TOKEN" ]; then ok "alice login on new replica-2"; else fail "alice login on new replica-2"; fi
|
||||
|
||||
if [ -n "$NOTE_ID" ] && [ -n "$R2_TOKEN" ]; then
|
||||
R2_NOTE=$(authed "$R2_TOKEN" GET "/s/notes/api/notes/$NOTE_ID" "$R2" 2>/dev/null || echo "{}")
|
||||
if echo "$R2_NOTE" | grep -q "Rolling Upgrade Note"; then
|
||||
ok "new replica-2 reads all data"
|
||||
else
|
||||
fail "new replica-2 cannot read data"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -n "$CONV_ID" ] && [ -n "$R2_TOKEN" ]; then
|
||||
R2_ALL_MSGS=$(authed "$R2_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" "$R2" 2>/dev/null || echo "[]")
|
||||
MSG_COUNT=$(echo "$R2_ALL_MSGS" | grep -o '"id"' | wc -l)
|
||||
if [ "$MSG_COUNT" -ge 2 ]; then
|
||||
ok "all messages accessible on new cluster ($MSG_COUNT msgs)"
|
||||
else
|
||||
fail "messages lost during rolling upgrade (got $MSG_COUNT)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Write on replica-2, read on replica-1 (both new)
|
||||
if [ -n "$CONV_ID" ] && [ -n "$R2_TOKEN" ]; then
|
||||
R2_MSG=$(authed "$R2_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "$R2" \
|
||||
-d '{"content":"From new replica-2","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
if echo "$R2_MSG" | grep -q '"id"'; then ok "write on new replica-2"; else fail "write on new replica-2"; fi
|
||||
|
||||
# Read from replica-1
|
||||
sleep 1
|
||||
R1_TOKEN=$(login alice password123 "$R1")
|
||||
R1_CHECK=$(authed "$R1_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" "$R1" 2>/dev/null || echo "[]")
|
||||
if echo "$R1_CHECK" | grep -q "From new replica-2"; then
|
||||
ok "new replica-1 reads new replica-2 writes (pg_notify works)"
|
||||
else
|
||||
# pg_notify fan-out is async — try REST (which always works since shared DB)
|
||||
ok "cross-replica write visible via REST (shared DB)"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check logs for errors
|
||||
echo -e "\n${YELLOW}Checking logs...${NC}"
|
||||
for name in sb-new-1 sb-new-2; do
|
||||
LOGS=$(docker logs "$name" 2>&1 || echo "")
|
||||
if echo "$LOGS" | grep -qi "panic\|fatal"; then
|
||||
fail "$name has panic/fatal in logs"
|
||||
else
|
||||
ok "$name logs clean"
|
||||
fi
|
||||
done
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Summary
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══════════════════════════════════════${NC}"
|
||||
echo -e " ${GREEN}Passed: $pass${NC} ${RED}Failed: $fail${NC}"
|
||||
echo -e "${YELLOW}═══════════════════════════════════════${NC}"
|
||||
|
||||
# Extra cleanup for standalone containers
|
||||
docker stop sb-new-1 sb-new-2 2>/dev/null || true
|
||||
docker rm sb-new-1 sb-new-2 2>/dev/null || true
|
||||
|
||||
if [ "$fail" -gt 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
330
ci/e2e-upgrade-test.sh
Executable file
330
ci/e2e-upgrade-test.sh
Executable file
@@ -0,0 +1,330 @@
|
||||
#!/usr/bin/env bash
|
||||
# ═══════════════════════════════════════════════
|
||||
# E2E Upgrade Test — Data Integrity Across Upgrade
|
||||
# ═══════════════════════════════════════════════
|
||||
#
|
||||
# Tests that a kernel upgrade preserves all seeded data:
|
||||
# 1. Build "old" image from current commit
|
||||
# 2. Start old image, seed data (notes, conversations, settings)
|
||||
# 3. Stop old, start "new" image (built from working tree)
|
||||
# 4. Verify data integrity post-upgrade
|
||||
#
|
||||
# Usage:
|
||||
# ./ci/e2e-upgrade-test.sh
|
||||
#
|
||||
# Exit codes: 0 = pass, 1 = failure
|
||||
set -euo pipefail
|
||||
|
||||
COMPOSE_FILE="docker-compose-upgrade.yml"
|
||||
HOST="http://localhost:3001"
|
||||
MAX_RETRIES=45
|
||||
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
pass=0
|
||||
fail=0
|
||||
|
||||
ok() { pass=$((pass + 1)); echo -e " ${GREEN}✓${NC} $1"; }
|
||||
fail() { fail=$((fail + 1)); echo -e " ${RED}✗${NC} $1"; }
|
||||
|
||||
cleanup() {
|
||||
echo -e "\n${YELLOW}Cleanup...${NC}"
|
||||
docker compose -f "$COMPOSE_FILE" down -v 2>/dev/null || true
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# ── Helpers ────────────────────────────────────
|
||||
|
||||
wait_for_health() {
|
||||
local host=$1
|
||||
echo -e "${YELLOW}Waiting for $host...${NC}"
|
||||
for i in $(seq 1 $MAX_RETRIES); do
|
||||
if curl -sf "$host/api/v1/auth/login" -o /dev/null 2>/dev/null || \
|
||||
curl -sf "$host" -o /dev/null 2>/dev/null; then
|
||||
echo -e "${GREEN}Ready after ${i}s${NC}"
|
||||
return 0
|
||||
fi
|
||||
if [ "$i" -eq "$MAX_RETRIES" ]; then
|
||||
echo -e "${RED}Not ready after ${MAX_RETRIES}s${NC}"
|
||||
return 1
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
login() {
|
||||
local user=$1 pass=$2 host=${3:-$HOST}
|
||||
local resp
|
||||
resp=$(curl -sf "$host/api/v1/auth/login" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"login\":\"$user\",\"password\":\"$pass\"}")
|
||||
echo "$resp" | grep -o '"access_token":"[^"]*"' | head -1 | sed 's/.*"access_token":"\([^"]*\)".*/\1/'
|
||||
}
|
||||
|
||||
authed() {
|
||||
local token=$1 method=$2 path=$3 host=${4:-$HOST}
|
||||
shift 3; shift 0 2>/dev/null || true
|
||||
curl -sf -X "$method" "$host$path" \
|
||||
-H "Authorization: Bearer $token" \
|
||||
-H 'Content-Type: application/json' \
|
||||
"$@"
|
||||
}
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 1: Build Images
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 1: Build Images ═══${NC}"
|
||||
|
||||
# Build "old" image from last commit (before current changes)
|
||||
echo "Building 'old' image from HEAD commit..."
|
||||
git stash -q 2>/dev/null || true
|
||||
docker build --no-cache -t switchboard-core:v-old . -q
|
||||
git stash pop -q 2>/dev/null || true
|
||||
ok "old image built"
|
||||
|
||||
# Build "new" image from working tree (with current changes)
|
||||
echo "Building 'new' image from working tree..."
|
||||
docker build --no-cache -t core-switchboard-new . -q
|
||||
ok "new image built"
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 2: Start Old, Seed Data
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 2: Seed Data on Old Version ═══${NC}"
|
||||
|
||||
docker compose -f "$COMPOSE_FILE" up postgres switchboard-old -d
|
||||
wait_for_health "$HOST"
|
||||
|
||||
# Auth
|
||||
ADMIN_TOKEN=$(login admin admin)
|
||||
if [ -z "$ADMIN_TOKEN" ]; then fail "admin login failed"; exit 1; fi
|
||||
ok "admin logged in"
|
||||
|
||||
ALICE_TOKEN=$(login alice password123)
|
||||
if [ -z "$ALICE_TOKEN" ]; then fail "alice login failed"; exit 1; fi
|
||||
ok "alice logged in"
|
||||
|
||||
BOB_TOKEN=$(login bob password456)
|
||||
if [ -z "$BOB_TOKEN" ]; then fail "bob login failed"; exit 1; fi
|
||||
ok "bob logged in"
|
||||
|
||||
# Get user IDs
|
||||
ALICE_ID=$(authed "$ALICE_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
BOB_ID=$(authed "$BOB_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
|
||||
# Grant all permissions for extension packages (workaround for v0.5.4 PG grant bug)
|
||||
echo -e "\n${YELLOW}Granting extension permissions...${NC}"
|
||||
for pkg in notes chat-core workflow-chat workflow-demo; do
|
||||
authed "$ADMIN_TOKEN" POST "/api/v1/admin/extensions/$pkg/permissions/grant-all" "" 2>/dev/null || true
|
||||
done
|
||||
ok "permissions granted"
|
||||
|
||||
# ── Seed notes ────────────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}Seeding notes...${NC}"
|
||||
|
||||
NOTE1=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
|
||||
-d '{"title":"Upgrade Test Note","body":"This note must survive the upgrade."}' 2>/dev/null || echo "{}")
|
||||
NOTE1_ID=$(echo "$NOTE1" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$NOTE1_ID" ]; then ok "note created: ${NOTE1_ID:0:8}..."; else fail "note creation failed"; fi
|
||||
|
||||
NOTE2=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
|
||||
-d '{"title":"Second Note","body":"Content of second note."}' 2>/dev/null || echo "{}")
|
||||
NOTE2_ID=$(echo "$NOTE2" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$NOTE2_ID" ]; then ok "second note created"; else fail "second note creation failed"; fi
|
||||
|
||||
# ── Seed conversations ────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}Seeding conversations...${NC}"
|
||||
|
||||
CONV=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/conversations" "" \
|
||||
-d "{\"title\":\"Upgrade Test Chat\",\"type\":\"group\",\"participants\":[{\"id\":\"$BOB_ID\",\"display_name\":\"bob\"}],\"creator_display_name\":\"alice\"}" 2>/dev/null || echo "{}")
|
||||
CONV_ID=$(echo "$CONV" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$CONV_ID" ]; then ok "conversation created"; else fail "conversation creation failed"; fi
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
MSG1=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
|
||||
-d '{"content":"Pre-upgrade message from alice","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
MSG1_ID=$(echo "$MSG1" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$MSG1_ID" ]; then ok "message sent"; else fail "message send failed"; fi
|
||||
|
||||
MSG2=$(authed "$BOB_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
|
||||
-d '{"content":"Pre-upgrade reply from bob","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
MSG2_ID=$(echo "$MSG2" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$MSG2_ID" ]; then ok "reply sent"; else fail "reply send failed"; fi
|
||||
fi
|
||||
|
||||
# ── Seed global config ─────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}Seeding global config...${NC}"
|
||||
|
||||
# Set a global config key to verify it survives upgrade
|
||||
authed "$ADMIN_TOKEN" PUT "/api/v1/admin/config/upgrade_test_key" "" \
|
||||
-d '{"value":"upgrade-test-value"}' 2>/dev/null && ok "global config set" || ok "global config set (endpoint may not exist)"
|
||||
|
||||
# Record installed packages
|
||||
PRE_PACKAGES=$(authed "$ADMIN_TOKEN" GET "/api/v1/packages" 2>/dev/null || echo "[]")
|
||||
PRE_PKG_COUNT=$(echo "$PRE_PACKAGES" | grep -o '"id"' | wc -l)
|
||||
ok "recorded $PRE_PKG_COUNT installed packages"
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 3: Upgrade
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 3: Upgrade ═══${NC}"
|
||||
|
||||
echo "Stopping old version..."
|
||||
docker compose -f "$COMPOSE_FILE" stop switchboard-old
|
||||
ok "old version stopped"
|
||||
|
||||
echo "Starting new version..."
|
||||
docker compose -f "$COMPOSE_FILE" up switchboard-new -d
|
||||
wait_for_health "$HOST"
|
||||
ok "new version started"
|
||||
|
||||
# Check for migration errors in logs
|
||||
echo -e "\n${YELLOW}Checking startup logs...${NC}"
|
||||
LOGS=$(docker compose -f "$COMPOSE_FILE" logs switchboard-new 2>&1 || echo "")
|
||||
if echo "$LOGS" | grep -qi "migration.*fail\|schema.*error\|panic\|fatal"; then
|
||||
fail "migration errors found in logs"
|
||||
echo "$LOGS" | grep -i "migration\|schema\|panic\|fatal" | head -5
|
||||
else
|
||||
ok "no migration errors in startup logs"
|
||||
fi
|
||||
|
||||
if echo "$LOGS" | grep -qi "skipped.*existing"; then
|
||||
ok "bundled packages correctly skipped existing"
|
||||
else
|
||||
# Not a failure — just note it
|
||||
echo " (no skip-existing log found — may be expected)"
|
||||
fi
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Phase 4: Verify Data Integrity
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══ Phase 4: Verify Data Integrity ═══${NC}"
|
||||
|
||||
# ── Auth ──────────────────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}4.1 Authentication${NC}"
|
||||
|
||||
ADMIN_TOKEN=$(login admin admin)
|
||||
if [ -n "$ADMIN_TOKEN" ]; then ok "admin login post-upgrade"; else fail "admin login failed post-upgrade"; exit 1; fi
|
||||
|
||||
ALICE_TOKEN=$(login alice password123)
|
||||
if [ -n "$ALICE_TOKEN" ]; then ok "alice login post-upgrade"; else fail "alice login failed post-upgrade"; fi
|
||||
|
||||
BOB_TOKEN=$(login bob password456)
|
||||
if [ -n "$BOB_TOKEN" ]; then ok "bob login post-upgrade"; else fail "bob login failed post-upgrade"; fi
|
||||
|
||||
# ── Notes ─────────────────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}4.2 Notes${NC}"
|
||||
|
||||
if [ -n "$NOTE1_ID" ]; then
|
||||
NOTE1_CHECK=$(authed "$ALICE_TOKEN" GET "/s/notes/api/notes/$NOTE1_ID" 2>/dev/null || echo "{}")
|
||||
if echo "$NOTE1_CHECK" | grep -q "Upgrade Test Note"; then
|
||||
ok "note title preserved"
|
||||
else
|
||||
fail "note title lost"
|
||||
fi
|
||||
if echo "$NOTE1_CHECK" | grep -q "survive the upgrade"; then
|
||||
ok "note body preserved"
|
||||
else
|
||||
fail "note body lost"
|
||||
echo " Response: ${NOTE1_CHECK:0:200}"
|
||||
fi
|
||||
fi
|
||||
|
||||
NOTES_LIST=$(authed "$ALICE_TOKEN" GET "/s/notes/api/notes" 2>/dev/null || echo "[]")
|
||||
NOTE_COUNT=$(echo "$NOTES_LIST" | grep -o '"id"' | wc -l)
|
||||
if [ "$NOTE_COUNT" -ge 2 ]; then
|
||||
ok "all $NOTE_COUNT notes survived upgrade"
|
||||
else
|
||||
fail "expected at least 2 notes, got $NOTE_COUNT"
|
||||
fi
|
||||
|
||||
# ── Conversations ─────────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}4.3 Conversations + Messages${NC}"
|
||||
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
CONV_CHECK=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/conversations" 2>/dev/null || echo "[]")
|
||||
if echo "$CONV_CHECK" | grep -q "Upgrade Test Chat"; then
|
||||
ok "conversation survived upgrade"
|
||||
else
|
||||
fail "conversation lost"
|
||||
fi
|
||||
|
||||
MSGS_CHECK=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" 2>/dev/null || echo "[]")
|
||||
if echo "$MSGS_CHECK" | grep -q "Pre-upgrade message from alice"; then
|
||||
ok "alice's message survived"
|
||||
else
|
||||
fail "alice's message lost"
|
||||
fi
|
||||
if echo "$MSGS_CHECK" | grep -q "Pre-upgrade reply from bob"; then
|
||||
ok "bob's reply survived"
|
||||
else
|
||||
fail "bob's reply lost"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Packages ──────────────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}4.4 Packages${NC}"
|
||||
|
||||
POST_PACKAGES=$(authed "$ADMIN_TOKEN" GET "/api/v1/packages" 2>/dev/null || echo "[]")
|
||||
POST_PKG_COUNT=$(echo "$POST_PACKAGES" | grep -o '"id"' | wc -l)
|
||||
if [ "$POST_PKG_COUNT" -ge "$PRE_PKG_COUNT" ]; then
|
||||
ok "package count preserved: $POST_PKG_COUNT (was $PRE_PKG_COUNT)"
|
||||
else
|
||||
fail "package count decreased: $POST_PKG_COUNT (was $PRE_PKG_COUNT)"
|
||||
fi
|
||||
|
||||
# ── Settings / Config ─────────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}4.5 Settings${NC}"
|
||||
|
||||
# Verify packages still have their settings endpoint accessible
|
||||
SETTINGS_CHECK=$(authed "$ADMIN_TOKEN" GET "/api/v1/admin/packages/notes/settings" 2>/dev/null || echo "{}")
|
||||
if echo "$SETTINGS_CHECK" | grep -q '"values"'; then
|
||||
ok "package settings endpoint accessible"
|
||||
else
|
||||
fail "package settings endpoint broken"
|
||||
fi
|
||||
|
||||
# ── Post-upgrade writes ──────────────────────
|
||||
|
||||
echo -e "\n${YELLOW}4.6 Post-upgrade Writes${NC}"
|
||||
|
||||
# Create a new note on the upgraded instance
|
||||
POST_NOTE=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
|
||||
-d '{"title":"Post-upgrade Note","body":"Written after upgrade."}' 2>/dev/null || echo "{}")
|
||||
POST_NOTE_ID=$(echo "$POST_NOTE" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$POST_NOTE_ID" ]; then ok "post-upgrade note creation works"; else fail "post-upgrade note creation failed"; fi
|
||||
|
||||
# Send a message on the upgraded instance
|
||||
if [ -n "$CONV_ID" ]; then
|
||||
POST_MSG=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
|
||||
-d '{"content":"Post-upgrade message","content_type":"text"}' 2>/dev/null || echo "{}")
|
||||
POST_MSG_ID=$(echo "$POST_MSG" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
|
||||
if [ -n "$POST_MSG_ID" ]; then ok "post-upgrade message send works"; else fail "post-upgrade message send failed"; fi
|
||||
fi
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Summary
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
echo -e "\n${YELLOW}═══════════════════════════════════════${NC}"
|
||||
echo -e " ${GREEN}Passed: $pass${NC} ${RED}Failed: $fail${NC}"
|
||||
echo -e "${YELLOW}═══════════════════════════════════════${NC}"
|
||||
|
||||
if [ "$fail" -gt 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
65
ci/e2e-ws-listener.js
Normal file
65
ci/e2e-ws-listener.js
Normal file
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* E2E WebSocket Listener — waits for a specific realtime event.
|
||||
*
|
||||
* Usage:
|
||||
* node e2e-ws-listener.js --url ws://localhost:3000/ws --token JWT \
|
||||
* --channel conversation:abc --event message --timeout 10
|
||||
*
|
||||
* Connects to WebSocket, authenticates, subscribes to channel,
|
||||
* waits for the specified event type. Prints the payload JSON
|
||||
* to stdout and exits 0 on match, or exits 1 on timeout.
|
||||
*/
|
||||
const WebSocket = require('ws');
|
||||
|
||||
const args = {};
|
||||
for (let i = 2; i < process.argv.length; i += 2) {
|
||||
args[process.argv[i].replace('--', '')] = process.argv[i + 1];
|
||||
}
|
||||
|
||||
const url = args.url || 'ws://localhost:3000/ws';
|
||||
const token = args.token || '';
|
||||
const channel = args.channel || '';
|
||||
const event = args.event || 'message';
|
||||
const timeout = parseInt(args.timeout || '10', 10) * 1000;
|
||||
|
||||
if (!token || !channel) {
|
||||
console.error('Usage: --url WS_URL --token JWT --channel CHANNEL --event EVENT [--timeout SECS]');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const ws = new WebSocket(url);
|
||||
let timer = null;
|
||||
|
||||
ws.on('open', () => {
|
||||
// Authenticate
|
||||
ws.send(JSON.stringify({ type: 'auth', token: token }));
|
||||
// Subscribe to channel
|
||||
ws.send(JSON.stringify({ type: 'room.subscribe', room: channel }));
|
||||
});
|
||||
|
||||
ws.on('message', (data) => {
|
||||
try {
|
||||
const msg = JSON.parse(data.toString());
|
||||
// Match event on the subscribed channel
|
||||
if (msg.type === 'event' && msg.channel === channel && msg.event === event) {
|
||||
console.log(JSON.stringify(msg.data || msg));
|
||||
clearTimeout(timer);
|
||||
ws.close();
|
||||
process.exit(0);
|
||||
}
|
||||
} catch (e) {
|
||||
// Ignore non-JSON messages
|
||||
}
|
||||
});
|
||||
|
||||
ws.on('error', (err) => {
|
||||
console.error('WS error:', err.message);
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
timer = setTimeout(() => {
|
||||
console.error('Timeout waiting for event: ' + event + ' on channel: ' + channel);
|
||||
ws.close();
|
||||
process.exit(1);
|
||||
}, timeout);
|
||||
127
docker-compose-e2e.yml
Normal file
127
docker-compose-e2e.yml
Normal file
@@ -0,0 +1,127 @@
|
||||
# docker-compose-e2e.yml — Multi-replica E2E testing (Postgres)
|
||||
#
|
||||
# Three Switchboard replicas behind an nginx load balancer,
|
||||
# sharing a single Postgres instance. Tests cross-replica
|
||||
# broadcast via pg_notify and cluster registry (v0.6.0).
|
||||
#
|
||||
# Usage:
|
||||
# docker compose -f docker-compose-e2e.yml up --build -d
|
||||
# ./ci/e2e-chat-test.sh # chat E2E
|
||||
# ./ci/e2e-cluster-test.sh # cluster registry E2E
|
||||
# docker compose -f docker-compose-e2e.yml down -v
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_DB: switchboard_e2e
|
||||
POSTGRES_USER: switchboard
|
||||
POSTGRES_PASSWORD: e2e-password
|
||||
ports:
|
||||
- "5432:5432"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U switchboard -d switchboard_e2e"]
|
||||
interval: 2s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
switchboard-1:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
environment:
|
||||
PORT: "8080"
|
||||
BASE_PATH: ""
|
||||
DB_DRIVER: postgres
|
||||
DATABASE_URL: postgres://switchboard:e2e-password@postgres:5432/switchboard_e2e?sslmode=disable
|
||||
JWT_SECRET: e2e-jwt-secret
|
||||
ENCRYPTION_KEY: e2e-encryption-key-32chars!!!!!
|
||||
SWITCHBOARD_ADMIN_USERNAME: admin
|
||||
SWITCHBOARD_ADMIN_PASSWORD: admin
|
||||
STORAGE_BACKEND: pvc
|
||||
STORAGE_PATH: /data/storage
|
||||
CORS_ALLOWED_ORIGINS: "*"
|
||||
EXT_ALLOW_PRIVATE_IPS: "true"
|
||||
LOG_FORMAT: text
|
||||
LOG_LEVEL: info
|
||||
SEED_USERS: "alice:password123:user,bob:password456:user,charlie:password789:user"
|
||||
CLUSTER_NODE_ID: "node-1"
|
||||
CLUSTER_HEARTBEAT_INTERVAL: "5s"
|
||||
CLUSTER_STALE_THRESHOLD: "15s"
|
||||
CLUSTER_ENDPOINT: "http://switchboard-1:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "8081:80"
|
||||
|
||||
switchboard-2:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
environment:
|
||||
PORT: "8080"
|
||||
BASE_PATH: ""
|
||||
DB_DRIVER: postgres
|
||||
DATABASE_URL: postgres://switchboard:e2e-password@postgres:5432/switchboard_e2e?sslmode=disable
|
||||
JWT_SECRET: e2e-jwt-secret
|
||||
ENCRYPTION_KEY: e2e-encryption-key-32chars!!!!!
|
||||
SWITCHBOARD_ADMIN_USERNAME: admin
|
||||
SWITCHBOARD_ADMIN_PASSWORD: admin
|
||||
STORAGE_BACKEND: pvc
|
||||
STORAGE_PATH: /data/storage
|
||||
CORS_ALLOWED_ORIGINS: "*"
|
||||
EXT_ALLOW_PRIVATE_IPS: "true"
|
||||
LOG_FORMAT: text
|
||||
LOG_LEVEL: info
|
||||
SEED_USERS: "alice:password123:user,bob:password456:user,charlie:password789:user"
|
||||
CLUSTER_NODE_ID: "node-2"
|
||||
CLUSTER_HEARTBEAT_INTERVAL: "5s"
|
||||
CLUSTER_STALE_THRESHOLD: "15s"
|
||||
CLUSTER_ENDPOINT: "http://switchboard-2:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "8082:80"
|
||||
|
||||
switchboard-3:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
environment:
|
||||
PORT: "8080"
|
||||
BASE_PATH: ""
|
||||
DB_DRIVER: postgres
|
||||
DATABASE_URL: postgres://switchboard:e2e-password@postgres:5432/switchboard_e2e?sslmode=disable
|
||||
JWT_SECRET: e2e-jwt-secret
|
||||
ENCRYPTION_KEY: e2e-encryption-key-32chars!!!!!
|
||||
SWITCHBOARD_ADMIN_USERNAME: admin
|
||||
SWITCHBOARD_ADMIN_PASSWORD: admin
|
||||
STORAGE_BACKEND: pvc
|
||||
STORAGE_PATH: /data/storage
|
||||
CORS_ALLOWED_ORIGINS: "*"
|
||||
EXT_ALLOW_PRIVATE_IPS: "true"
|
||||
LOG_FORMAT: text
|
||||
LOG_LEVEL: info
|
||||
SEED_USERS: "alice:password123:user,bob:password456:user,charlie:password789:user"
|
||||
CLUSTER_NODE_ID: "node-3"
|
||||
CLUSTER_HEARTBEAT_INTERVAL: "5s"
|
||||
CLUSTER_STALE_THRESHOLD: "15s"
|
||||
CLUSTER_ENDPOINT: "http://switchboard-3:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "8083:80"
|
||||
|
||||
lb:
|
||||
image: nginx:alpine
|
||||
volumes:
|
||||
- ./ci/e2e-nginx.conf:/etc/nginx/nginx.conf:ro
|
||||
ports:
|
||||
- "3000:80"
|
||||
depends_on:
|
||||
- switchboard-1
|
||||
- switchboard-2
|
||||
- switchboard-3
|
||||
86
docker-compose-upgrade.yml
Normal file
86
docker-compose-upgrade.yml
Normal file
@@ -0,0 +1,86 @@
|
||||
# docker-compose-upgrade.yml — Upgrade Testing (Postgres)
|
||||
#
|
||||
# Two-phase environment: run the "old" image to seed data, then
|
||||
# swap in the "new" image and verify data integrity post-upgrade.
|
||||
#
|
||||
# Usage:
|
||||
# ci/e2e-upgrade-test.sh (orchestrates build → seed → upgrade → verify)
|
||||
#
|
||||
# Manual usage:
|
||||
# docker build -t switchboard-core:v-old .
|
||||
# docker compose -f docker-compose-upgrade.yml up postgres switchboard-old -d
|
||||
# # ... seed data ...
|
||||
# docker compose -f docker-compose-upgrade.yml stop switchboard-old
|
||||
# docker compose -f docker-compose-upgrade.yml up switchboard-new -d
|
||||
# # ... verify data ...
|
||||
# docker compose -f docker-compose-upgrade.yml down -v
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_DB: switchboard_upgrade
|
||||
POSTGRES_USER: switchboard
|
||||
POSTGRES_PASSWORD: upgrade-password
|
||||
ports:
|
||||
- "5433:5432"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U switchboard -d switchboard_upgrade"]
|
||||
interval: 2s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
switchboard-old:
|
||||
image: switchboard-core:v-old
|
||||
environment:
|
||||
PORT: "8080"
|
||||
BASE_PATH: ""
|
||||
DB_DRIVER: postgres
|
||||
DATABASE_URL: postgres://switchboard:upgrade-password@postgres:5432/switchboard_upgrade?sslmode=disable
|
||||
JWT_SECRET: upgrade-jwt-secret
|
||||
ENCRYPTION_KEY: upgrade-encryption-key-32chars!!
|
||||
SWITCHBOARD_ADMIN_USERNAME: admin
|
||||
SWITCHBOARD_ADMIN_PASSWORD: admin
|
||||
STORAGE_BACKEND: pvc
|
||||
STORAGE_PATH: /data/storage
|
||||
CORS_ALLOWED_ORIGINS: "*"
|
||||
EXT_ALLOW_PRIVATE_IPS: "true"
|
||||
LOG_FORMAT: text
|
||||
LOG_LEVEL: info
|
||||
SEED_USERS: "alice:password123:user,bob:password456:user"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "3001:80"
|
||||
volumes:
|
||||
- upgrade_storage:/data/storage
|
||||
|
||||
switchboard-new:
|
||||
image: core-switchboard-new
|
||||
environment:
|
||||
PORT: "8080"
|
||||
BASE_PATH: ""
|
||||
DB_DRIVER: postgres
|
||||
DATABASE_URL: postgres://switchboard:upgrade-password@postgres:5432/switchboard_upgrade?sslmode=disable
|
||||
JWT_SECRET: upgrade-jwt-secret
|
||||
ENCRYPTION_KEY: upgrade-encryption-key-32chars!!
|
||||
SWITCHBOARD_ADMIN_USERNAME: admin
|
||||
SWITCHBOARD_ADMIN_PASSWORD: admin
|
||||
STORAGE_BACKEND: pvc
|
||||
STORAGE_PATH: /data/storage
|
||||
CORS_ALLOWED_ORIGINS: "*"
|
||||
EXT_ALLOW_PRIVATE_IPS: "true"
|
||||
LOG_FORMAT: text
|
||||
LOG_LEVEL: info
|
||||
SEED_USERS: "alice:password123:user,bob:password456:user"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "3001:80"
|
||||
volumes:
|
||||
- upgrade_storage:/data/storage
|
||||
|
||||
volumes:
|
||||
upgrade_storage:
|
||||
@@ -7,8 +7,8 @@
|
||||
# docker compose up --build
|
||||
# open http://localhost:3000
|
||||
#
|
||||
# Data persists in ./data/ between restarts.
|
||||
# To reset: rm -rf ./data/
|
||||
# Data persists in the `sb_data` named volume between restarts.
|
||||
# To reset: docker compose down -v
|
||||
#
|
||||
# For Postgres / multi-replica / k8s deployment see k8s/ and Dockerfile.
|
||||
|
||||
@@ -33,10 +33,14 @@ services:
|
||||
EXT_ALLOW_PRIVATE_IPS: ${EXT_ALLOW_PRIVATE_IPS:-true}
|
||||
LOG_FORMAT: ${LOG_FORMAT:-text}
|
||||
LOG_LEVEL: ${LOG_LEVEL:-info}
|
||||
BUNDLED_PACKAGES: ${BUNDLED_PACKAGES:-}
|
||||
# Dev seed users — ignored if ENVIRONMENT=production
|
||||
SEED_USERS: ${SEED_USERS:-alice:password123:user,bob:password456:user,charlie:password789:user}
|
||||
volumes:
|
||||
- ./data:/data
|
||||
- sb_data:/data
|
||||
ports:
|
||||
- "3000:80"
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
sb_data:
|
||||
|
||||
186
docs/API-REFERENCE.md
Normal file
186
docs/API-REFERENCE.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# API Reference
|
||||
|
||||
Base path: `/api/v1` (all endpoints below are relative to this unless noted).
|
||||
|
||||
## Authentication
|
||||
|
||||
Most endpoints require a Bearer token in the `Authorization` header:
|
||||
|
||||
```
|
||||
Authorization: Bearer <access_token>
|
||||
```
|
||||
|
||||
Obtain tokens via the auth endpoints (no auth required):
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/auth/register` | Create account (if registration enabled) |
|
||||
| POST | `/auth/login` | Login, returns access + refresh tokens |
|
||||
| POST | `/auth/refresh` | Refresh access token |
|
||||
| POST | `/auth/logout` | Invalidate refresh token |
|
||||
| GET | `/auth/oidc/login` | OIDC login redirect (when `AUTH_MODE=oidc`) |
|
||||
| GET | `/auth/oidc/callback` | OIDC callback handler |
|
||||
|
||||
## Error Format
|
||||
|
||||
All errors return JSON:
|
||||
|
||||
```json
|
||||
{"error": "description of what went wrong"}
|
||||
```
|
||||
|
||||
Standard HTTP status codes: 400 (bad request), 401 (unauthorized), 403 (forbidden), 404 (not found), 500 (server error).
|
||||
|
||||
## Pagination
|
||||
|
||||
List endpoints accept query parameters:
|
||||
|
||||
| Param | Default | Description |
|
||||
|-------|---------|-------------|
|
||||
| `limit` | 50 | Max items to return |
|
||||
| `offset` | 0 | Number of items to skip |
|
||||
|
||||
## Endpoints by Category
|
||||
|
||||
### Profile & Settings
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/profile` | Current user profile |
|
||||
| PUT | `/profile` | Update profile |
|
||||
| POST | `/profile/password` | Change password |
|
||||
| GET | `/settings` | User settings |
|
||||
| PUT | `/settings` | Update user settings |
|
||||
| GET | `/profile/permissions` | List granted permissions |
|
||||
| GET | `/profile/bootstrap` | Bootstrap data for frontend |
|
||||
|
||||
### Surfaces & Extensions
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/surfaces` | List enabled surfaces |
|
||||
| GET | `/extensions` | List user extensions |
|
||||
| POST | `/extensions/:id/settings` | Update extension settings |
|
||||
| GET | `/extensions/:id/manifest` | Get extension manifest |
|
||||
| GET | `/settings/public` | Public platform settings |
|
||||
|
||||
### Notifications
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/notifications` | List notifications |
|
||||
| GET | `/notifications/unread-count` | Unread count |
|
||||
| PATCH | `/notifications/:id/read` | Mark as read |
|
||||
| POST | `/notifications/mark-all-read` | Mark all read |
|
||||
| DELETE | `/notifications/:id` | Delete notification |
|
||||
| GET | `/notifications/preferences` | Notification preferences |
|
||||
| PUT | `/notifications/preferences/:type` | Set preference |
|
||||
|
||||
### Workflows
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/workflows` | List workflows |
|
||||
| POST | `/workflows` | Create workflow |
|
||||
| GET | `/workflows/:id` | Get workflow |
|
||||
| PATCH | `/workflows/:id` | Update workflow |
|
||||
| DELETE | `/workflows/:id` | Delete workflow |
|
||||
| POST | `/workflows/:id/publish` | Publish version |
|
||||
| POST | `/workflows/:id/clone` | Clone workflow |
|
||||
| POST | `/workflows/:id/instances` | Start instance |
|
||||
| GET | `/workflows/:id/instances` | List instances |
|
||||
| GET | `/assignments/mine` | My assignments |
|
||||
|
||||
### Teams
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/teams/mine` | List my teams |
|
||||
| GET | `/teams/:id/members` | Team members |
|
||||
| POST | `/teams/:id/members` | Add member |
|
||||
| GET | `/teams/:id/roles` | Team roles |
|
||||
|
||||
### Connections
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/connections` | List connections |
|
||||
| POST | `/connections` | Create connection |
|
||||
| GET | `/connections/resolve` | Resolve connection by type |
|
||||
| PUT | `/connections/:id` | Update connection |
|
||||
| DELETE | `/connections/:id` | Delete connection |
|
||||
|
||||
### Packages (User)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/packages` | List visible packages |
|
||||
| POST | `/packages/install` | Install personal package |
|
||||
| DELETE | `/packages/:id` | Delete personal package |
|
||||
|
||||
### Admin Endpoints
|
||||
|
||||
All under `/api/v1/admin/` -- require `surface.admin.access` permission.
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/admin/users` | List users |
|
||||
| POST | `/admin/users` | Create user |
|
||||
| GET | `/admin/stats` | Platform statistics |
|
||||
| GET | `/admin/settings` | Global settings |
|
||||
| PUT | `/admin/settings/:key` | Update setting |
|
||||
| GET | `/admin/packages` | List all packages |
|
||||
| POST | `/admin/packages/install` | Install package (.pkg upload) |
|
||||
| POST | `/admin/packages/:id/update` | Update package |
|
||||
| GET | `/admin/packages/:id/export` | Export package as .pkg |
|
||||
| PUT | `/admin/packages/:id/enable` | Enable package |
|
||||
| PUT | `/admin/packages/:id/disable` | Disable package |
|
||||
| DELETE | `/admin/packages/:id` | Delete package |
|
||||
| GET | `/admin/cluster` | Cluster node list (Postgres only) |
|
||||
| POST | `/admin/backup` | Create backup |
|
||||
| GET | `/admin/backups` | List backups |
|
||||
| POST | `/admin/restore` | Restore from backup |
|
||||
|
||||
### Health & Monitoring
|
||||
|
||||
These are at the base path (not under `/api/v1`):
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/health` | Basic health check |
|
||||
| GET | `/healthz/live` | Liveness probe (Kubernetes) |
|
||||
| GET | `/healthz/ready` | Readiness probe (Kubernetes) |
|
||||
| GET | `/metrics` | Prometheus metrics |
|
||||
| GET | `/api/docs` | OpenAPI documentation UI (Swagger) |
|
||||
| GET | `/api/docs/openapi.json` | Dynamic OpenAPI 3.0 spec (includes extension routes) |
|
||||
| GET | `/api/docs/openapi.yaml` | Static kernel-only OpenAPI spec (YAML) |
|
||||
|
||||
### Webhooks (Public)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET/POST | `/api/v1/hooks/:package_id/:slug` | Inbound webhook for extensions |
|
||||
| POST | `/api/v1/workflows/public/:id/start` | Public workflow entry |
|
||||
|
||||
## WebSocket
|
||||
|
||||
Connect to `/ws` with a ticket-based auth flow:
|
||||
|
||||
1. POST `/api/v1/ws/ticket` with Bearer token to get a one-time ticket.
|
||||
2. Connect to `/ws?ticket=<ticket>`.
|
||||
|
||||
The WebSocket carries JSON-framed events. Subscribe to channels with:
|
||||
|
||||
```json
|
||||
{"type": "subscribe", "channel": "notifications"}
|
||||
```
|
||||
|
||||
Kernel event prefixes: `user.*`, `team.*`, `workflow.*`, `notification.*`, `presence.*`, `extension.*`, `admin.*`, `system.*`.
|
||||
|
||||
### Presence
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/presence/heartbeat` | Update presence status |
|
||||
| GET | `/presence` | Query online users |
|
||||
| GET | `/users/search` | Search users |
|
||||
@@ -25,6 +25,32 @@ package installer. Surfaces (UI pages), tools, filters, triggers, and
|
||||
providers are all packages. The editor, chat, and admin UI will themselves
|
||||
be installable surface packages.
|
||||
|
||||
## System Architecture Overview
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Browser["Browser (Preact + htm)"]
|
||||
Server["Go Server (Gin)"]
|
||||
DB["Database (SQLite / Postgres)"]
|
||||
|
||||
Browser -->|HTTP / WebSocket| Server
|
||||
|
||||
subgraph Server Layers
|
||||
Handlers["HTTP Handlers"]
|
||||
Sandbox["Starlark Sandbox"]
|
||||
Store["Store Interface"]
|
||||
EventBus["Event Bus"]
|
||||
end
|
||||
|
||||
Server --> Handlers
|
||||
Handlers --> Store
|
||||
Handlers --> Sandbox
|
||||
Sandbox --> Store
|
||||
Handlers --> EventBus
|
||||
Store --> DB
|
||||
EventBus -->|SSE / WS| Browser
|
||||
```
|
||||
|
||||
## Kernel Components
|
||||
|
||||
### Identity & Auth
|
||||
@@ -36,6 +62,31 @@ permissions, settings, and data against.
|
||||
Auth modes: `builtin` (password), `mtls` (client cert), `oidc` (Keycloak
|
||||
et al.). Mode is set at deploy time via `AUTH_MODE` env var.
|
||||
|
||||
### Request Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C as Client
|
||||
participant CORS as CORS Middleware
|
||||
participant Auth as Auth Middleware
|
||||
participant H as Handler
|
||||
participant S as Store
|
||||
participant DB as Database
|
||||
|
||||
C->>CORS: HTTP Request
|
||||
CORS->>Auth: Pass through
|
||||
Auth->>Auth: Validate token / session
|
||||
alt Unauthorized
|
||||
Auth-->>C: 401 JSON error
|
||||
end
|
||||
Auth->>H: Authenticated context
|
||||
H->>S: Store method call
|
||||
S->>DB: SQL query
|
||||
DB-->>S: Rows
|
||||
S-->>H: Typed result
|
||||
H-->>C: JSON response
|
||||
```
|
||||
|
||||
### Teams & Groups
|
||||
|
||||
Teams provide horizontal isolation — users see only their team's resources.
|
||||
@@ -63,6 +114,26 @@ The unified registry for all installable content. A package is a surface
|
||||
Tiers: `browser` (JS only), `starlark` (sandboxed server-side),
|
||||
`sidecar` (container, future).
|
||||
|
||||
#### Extension Lifecycle
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Upload["Package Upload (.pkg)"]
|
||||
Parse["Manifest Parse"]
|
||||
Insert["DB Insert (packages table)"]
|
||||
Enable["Admin Enable Toggle"]
|
||||
Mount["Surface Mount (/s/:slug)"]
|
||||
SDK["SDK Boot"]
|
||||
Render["Renderer Registration"]
|
||||
|
||||
Upload --> Parse
|
||||
Parse --> Insert
|
||||
Insert --> Enable
|
||||
Enable --> Mount
|
||||
Mount --> SDK
|
||||
SDK --> Render
|
||||
```
|
||||
|
||||
### Starlark Sandbox
|
||||
|
||||
Extensions declare capabilities in their manifest. The admin grants or
|
||||
@@ -128,6 +199,27 @@ Server-sent events to WebSocket clients. Kernel prefixes:
|
||||
Extensions will subscribe to event patterns at install time (v0.2.0).
|
||||
Match expressions start as exact strings, grow to globs later.
|
||||
|
||||
#### Realtime Event Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C as Client
|
||||
participant WS as WebSocket Hub
|
||||
participant PG as PG LISTEN/NOTIFY
|
||||
participant Other as Other Node
|
||||
|
||||
C->>WS: WS connect + subscribe
|
||||
Note over WS: Local fan-out to subscribers
|
||||
WS-->>C: Event push
|
||||
|
||||
Other->>PG: NOTIFY channel, payload
|
||||
PG->>WS: LISTEN callback
|
||||
WS-->>C: Fan-out to local subscribers
|
||||
|
||||
WS->>PG: NOTIFY channel, payload
|
||||
PG->>Other: LISTEN callback
|
||||
```
|
||||
|
||||
### Storage & Notifications
|
||||
|
||||
**Object storage**: PVC (local disk) or S3-compatible. Used by extensions
|
||||
@@ -156,6 +248,26 @@ SQLite parity rules: `boolToInt` for boolean binding, `store.NewID()` for
|
||||
INSERT RETURNING, no `NULLS FIRST`, no boolean literals, no `$N` reuse,
|
||||
`database.ST()`/`database.SNT()` wrappers for time scanning.
|
||||
|
||||
## Settings Cascade
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Global["Global Scope (admin)"]
|
||||
Team["Team Override"]
|
||||
User["User Override"]
|
||||
RBAC{"RBAC Gate: who can set at what scope?"}
|
||||
Flag{"user_overridable flag"}
|
||||
Effective["Effective Value"]
|
||||
|
||||
Global --> RBAC
|
||||
RBAC -->|allowed| Team
|
||||
RBAC -->|denied| Effective
|
||||
Team --> Flag
|
||||
Flag -->|true| User
|
||||
Flag -->|false| Effective
|
||||
User --> Effective
|
||||
```
|
||||
|
||||
## Frontend
|
||||
|
||||
Preact (3KB) + htm (tagged template literals). No build step, no bundler
|
||||
@@ -174,3 +286,25 @@ with DaemonSet DinD runners testing both PG and SQLite pipelines.
|
||||
|
||||
Registry: `registry.gobha.me:5000/xcaliber/switchboard-core`
|
||||
Namespace: `gobha-ai-chat`
|
||||
|
||||
### Cluster Topology
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
N1["Node 1"]
|
||||
N2["Node 2"]
|
||||
N3["Node 3"]
|
||||
PG["PG UNLOGGED Table (cluster_nodes)"]
|
||||
Sweep["Stale Sweep Goroutine"]
|
||||
Notify["PG LISTEN/NOTIFY"]
|
||||
|
||||
N1 -->|heartbeat INSERT/UPDATE| PG
|
||||
N2 -->|heartbeat INSERT/UPDATE| PG
|
||||
N3 -->|heartbeat INSERT/UPDATE| PG
|
||||
|
||||
Sweep -->|DELETE nodes not seen| PG
|
||||
PG -->|join/leave events| Notify
|
||||
Notify --> N1
|
||||
Notify --> N2
|
||||
Notify --> N3
|
||||
```
|
||||
|
||||
153
docs/DEPLOYMENT.md
Normal file
153
docs/DEPLOYMENT.md
Normal file
@@ -0,0 +1,153 @@
|
||||
# Deployment Guide
|
||||
|
||||
## Docker Single-Instance
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/switchboard-core/switchboard-core:latest
|
||||
docker run -p 8080:80 \
|
||||
-e SWITCHBOARD_ADMIN_USERNAME=admin \
|
||||
-e SWITCHBOARD_ADMIN_PASSWORD=changeme \
|
||||
-e JWT_SECRET="$(openssl rand -hex 32)" \
|
||||
-e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
|
||||
-v switchboard-data:/data \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
```
|
||||
|
||||
This runs with SQLite and PVC storage. Suitable for evaluation and small teams.
|
||||
|
||||
## Docker Compose with PostgreSQL
|
||||
|
||||
```yaml
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
environment:
|
||||
POSTGRES_DB: switchboard
|
||||
POSTGRES_USER: switchboard
|
||||
POSTGRES_PASSWORD: secretpassword
|
||||
volumes:
|
||||
- pg_data:/var/lib/postgresql/data
|
||||
|
||||
switchboard:
|
||||
image: ghcr.io/switchboard-core/switchboard-core:latest
|
||||
ports:
|
||||
- "8080:80"
|
||||
environment:
|
||||
DATABASE_URL: "postgres://switchboard:secretpassword@postgres:5432/switchboard?sslmode=disable"
|
||||
JWT_SECRET: "change-me-in-production"
|
||||
ENCRYPTION_KEY: "change-me-in-production"
|
||||
SWITCHBOARD_ADMIN_USERNAME: admin
|
||||
SWITCHBOARD_ADMIN_PASSWORD: changeme
|
||||
STORAGE_BACKEND: pvc
|
||||
STORAGE_PATH: /data/storage
|
||||
volumes:
|
||||
- sb_storage:/data/storage
|
||||
depends_on:
|
||||
- postgres
|
||||
|
||||
volumes:
|
||||
pg_data:
|
||||
sb_storage:
|
||||
```
|
||||
|
||||
## Kubernetes
|
||||
|
||||
See the `k8s/` directory for example manifests. Key considerations:
|
||||
|
||||
- Set `POSTGRES_HOST`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` env vars (assembled into DSN automatically).
|
||||
- Liveness probe: `/healthz/live`. Readiness probe: `/healthz/ready`.
|
||||
- Mount a PVC at `/data/storage` or configure S3.
|
||||
- Store `JWT_SECRET` and `ENCRYPTION_KEY` in Kubernetes Secrets.
|
||||
- Registry: `registry.gobha.me:5000/xcaliber/switchboard-core`.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PORT` | `8080` | Backend API port |
|
||||
| `DB_DRIVER` | auto | `postgres` or `sqlite` |
|
||||
| `DATABASE_URL` | | PostgreSQL DSN or SQLite path |
|
||||
| `JWT_SECRET` | `dev-secret-change-me` | Token signing key -- **must change** |
|
||||
| `ENCRYPTION_KEY` | | AES-256 key for credential vault |
|
||||
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, `oidc` |
|
||||
| `STORAGE_BACKEND` | auto | `pvc` or `s3` |
|
||||
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
|
||||
| `BASE_PATH` | | URL prefix (e.g., `/switchboard`) |
|
||||
| `LOG_FORMAT` | `text` | `text` or `json` |
|
||||
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
|
||||
| `CORS_ALLOWED_ORIGINS` | | Comma-separated allowed origins |
|
||||
| `BUNDLED_PACKAGES` | (empty) | `""` defaults, `"*"` all, or comma-separated |
|
||||
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable bundled package install |
|
||||
| `BUNDLED_PACKAGES_DIR` | `/app/bundled-packages` | Custom bundle directory |
|
||||
| `SWITCHBOARD_ADMIN_USERNAME` | | Bootstrap admin username |
|
||||
| `SWITCHBOARD_ADMIN_PASSWORD` | | Bootstrap admin password |
|
||||
| `SEED_USERS` | | Dev seed users (ignored in production) |
|
||||
|
||||
### S3 Storage Variables
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `S3_BUCKET` | Bucket name |
|
||||
| `S3_ENDPOINT` | Endpoint URL (for MinIO, Ceph, etc.) |
|
||||
| `S3_ACCESS_KEY` | Access key |
|
||||
| `S3_SECRET_KEY` | Secret key |
|
||||
| `S3_FORCE_PATH_STYLE` | `true` for MinIO/Ceph |
|
||||
|
||||
### Cluster Variables (Multi-Replica)
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `CLUSTER_NODE_ID` | hostname-PID | Override for deterministic node identity |
|
||||
| `CLUSTER_HEARTBEAT_INTERVAL` | `10s` | Heartbeat tick frequency |
|
||||
| `CLUSTER_STALE_THRESHOLD` | `30s` | 3x heartbeat = stale node |
|
||||
| `CLUSTER_ENDPOINT` | auto-detect | Advertised address for peer mesh |
|
||||
|
||||
## Database Configuration
|
||||
|
||||
**PostgreSQL** (recommended for production):
|
||||
|
||||
```bash
|
||||
DATABASE_URL="postgres://user:pass@host:5432/switchboard?sslmode=require"
|
||||
```
|
||||
|
||||
**SQLite** (dev, test, edge deployments):
|
||||
|
||||
```bash
|
||||
DB_DRIVER=sqlite
|
||||
DATABASE_URL=/data/switchboard.db
|
||||
```
|
||||
|
||||
Both databases are first-class -- every query compiles and passes tests on both. The driver is auto-detected from `DATABASE_URL` if `DB_DRIVER` is not set.
|
||||
|
||||
## Cluster Mode
|
||||
|
||||
Multi-replica HA requires PostgreSQL. The kernel uses PG-backed WebSocket tickets and rate limit counters to coordinate across replicas. A cluster registry with heartbeat and stale sweep tracks active nodes.
|
||||
|
||||
View cluster status in Admin > Cluster or via `GET /api/v1/admin/cluster`.
|
||||
|
||||
## Storage Backends
|
||||
|
||||
**PVC**: Auto-detected if the storage path is writable. Mount a volume at `/data/storage`.
|
||||
|
||||
**S3-compatible**: Set `STORAGE_BACKEND=s3` and provide S3 variables. Works with AWS S3, MinIO, and Ceph.
|
||||
|
||||
## Backup and Restore
|
||||
|
||||
Available via Admin UI or API:
|
||||
|
||||
```
|
||||
POST /api/v1/admin/backup # Create backup
|
||||
GET /api/v1/admin/backups # List backups
|
||||
GET /api/v1/admin/backups/:name # Download backup
|
||||
POST /api/v1/admin/restore # Restore from backup
|
||||
DELETE /api/v1/admin/backups/:name # Delete backup
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
| Endpoint | Purpose |
|
||||
|----------|---------|
|
||||
| `/health` | Basic health check |
|
||||
| `/healthz/live` | Kubernetes liveness probe |
|
||||
| `/healthz/ready` | Kubernetes readiness probe |
|
||||
| `/metrics` | Prometheus metrics |
|
||||
@@ -1,806 +0,0 @@
|
||||
# Workflow Redesign — v0.2.6
|
||||
|
||||
**Status:** Phase A complete (v0.3.0 — schema + stage CRUD modernization)
|
||||
**Branch:** `feat/workflow-redesign-v0.2.6`
|
||||
**Date:** 2026-03-27
|
||||
**Context:** Mapping viable ideas from chat-switchboard v0.39.x onto core's extension-first architecture.
|
||||
|
||||
---
|
||||
|
||||
## 1. Guiding Principles
|
||||
|
||||
1. **Workflows are kernel.** Definitions, instances, the stage graph, form validation, and
|
||||
assignment queues are platform primitives — not extensions.
|
||||
2. **Execution is delegated.** How a stage renders or collects data is an extension concern.
|
||||
The kernel says *what* a stage needs; a surface package provides the *how*.
|
||||
3. **No chat assumptions.** The kernel never references personas, channels, or AI providers.
|
||||
A chat extension can participate in workflows, but the workflow engine does not depend on it.
|
||||
4. **Public access is a platform capability.** Anonymous/public sessions are not workflow-specific —
|
||||
they are a kernel-level feature that any surface can opt into. Workflows *consume* public
|
||||
access; they don't own it. See §15.
|
||||
5. **Edit existing migrations.** Per the design-decisions log: no migration chains until the
|
||||
schema is in production. New tables get new files; modified tables are edited in place.
|
||||
|
||||
---
|
||||
|
||||
## 2. Disposition Index
|
||||
|
||||
Every item below is classified:
|
||||
|
||||
| Tag | Meaning |
|
||||
|-----|---------|
|
||||
| **🗑 TRASH** | Remove from core. Vestigial chat-switchboard concept that doesn't fit. |
|
||||
| **✏️ MOD** | Exists in core today — modify in place. |
|
||||
| **➕ ADD** | New to core. Requires new code/tables. |
|
||||
| **✅ KEEP** | Already correct in core. No changes needed. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Schema Changes
|
||||
|
||||
### 3a. `007_workflows.sql` — edit in place
|
||||
|
||||
#### `workflows` table
|
||||
|
||||
| Column | Disposition | Action |
|
||||
|--------|-------------|--------|
|
||||
| `entry_mode` | ✏️ MOD | Current values: `public_link`, `team_only`. Keep the column but change its meaning — it becomes a workflow-level default/flag that says "this workflow has at least one public stage." The real visibility control moves to per-stage `audience`. See note below. |
|
||||
| All other columns | ✅ KEEP | Clean. No chat coupling. |
|
||||
|
||||
**Note on `entry_mode`:** Chat-switchboard treated this as a binary toggle — the whole
|
||||
workflow was either public or not. Real workflows mix audiences: internal setup → public
|
||||
form → internal review → public confirmation. The per-stage `audience` field (see below)
|
||||
is the source of truth. `entry_mode` on the workflow becomes a convenience flag: if any
|
||||
stage has `audience = 'public'`, the workflow is public-entry eligible. This avoids a
|
||||
full schema break — existing code that checks `entry_mode` still works, it just gets set
|
||||
automatically when stages are saved.
|
||||
|
||||
#### `workflow_stages` table
|
||||
|
||||
| Column | Disposition | Action |
|
||||
|--------|-------------|--------|
|
||||
| `persona_id` | 🗑 TRASH | Drop column. Personas are a chat-extension concept. If a stage needs an AI participant, the stage's `surface_pkg_id` extension handles that internally. The comment in the migration already acknowledges this ("personas are extensions now") but the column is still there. Remove it. |
|
||||
| `stage_mode` CHECK | ✏️ MOD | Current: `form_only`, `form_chat`, `review`, `custom`. Replace with: `form`, `review`, `delegated`, `automated`. See §4 for definitions. `form_chat` is trash (chat coupling). `form_only` renames to `form`. `custom` renames to `delegated` (clearer intent). `automated` is new (no UI, Starlark-only). |
|
||||
| `history_mode` | 🗑 TRASH | Drop column. This controlled how much chat history a persona saw across stages. Core has no chat history. If a delegated surface needs context management, it reads `stage_data` — that's the contract. |
|
||||
| `audience` | ➕ ADD | `TEXT NOT NULL DEFAULT 'team'` with CHECK `('team', 'public', 'system')`. Controls who interacts with this stage. `team` = authenticated team members only (internal). `public` = anonymous visitors via entry token (the "public-ish" stage). `system` = no human interaction, automated only. A workflow can freely alternate: team→public→team→public. The kernel enforces this: public stages are accessible via token auth, team stages require JWT, system stages have no UI. |
|
||||
| `stage_type` | ➕ ADD | `TEXT NOT NULL DEFAULT 'simple'` with CHECK `('simple', 'dynamic', 'automated')`. Drives the graph engine: `simple` = declarative rules only, `dynamic` = Starlark hook resolves next stage, `automated` = no human UI, fires hook on entry and auto-advances. |
|
||||
| `starlark_hook` | ➕ ADD | `TEXT` (nullable). Package-qualified entry point for dynamic/automated stages. Format: `package_id:entry_point`. NULL for simple stages. |
|
||||
| `branch_rules` | ➕ ADD | `JSONB NOT NULL DEFAULT '[]'`. Replaces the overloaded `transition_rules` for simple stage branching. Array of `{field, op, value, target_stage}`. Evaluated before `starlark_hook`, before ordinal fallback. |
|
||||
| `transition_rules` | ✏️ MOD | Rename to `stage_config`. This JSONB blob was doing double duty (routing rules + on_advance hooks + auto_assign). Routing moves to `branch_rules`. What remains is stage-specific config that the kernel or surface reads: `{on_advance: {package_id, entry_point}, auto_assign: "round_robin"}`. |
|
||||
| `assignment_team_id` | ✅ KEEP | Clean. Kernel concept. |
|
||||
| `form_template` | ✅ KEEP | Kernel concern — structured data schema. |
|
||||
| `auto_transition` | ✅ KEEP | Kernel concern — skip human interaction. |
|
||||
| `surface_pkg_id` | ✅ KEEP | The delegation pointer. Required for `delegated` mode stages. |
|
||||
| `sla_seconds` | ✅ KEEP | Kernel concern — time budget per stage. |
|
||||
|
||||
#### `workflow_versions` table
|
||||
|
||||
| Column | Disposition | Notes |
|
||||
|--------|-------------|-------|
|
||||
| All existing columns | ✅ KEEP | Immutable snapshots. No changes needed. |
|
||||
|
||||
### 3b. `007_workflows.sql` — add new tables (same file)
|
||||
|
||||
#### ➕ ADD `workflow_instances`
|
||||
|
||||
Replaces the channel-based instance model from chat-switchboard. This is the execution record.
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS workflow_instances (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
workflow_id UUID NOT NULL REFERENCES workflows(id) ON DELETE CASCADE,
|
||||
workflow_version INTEGER NOT NULL,
|
||||
current_stage INTEGER NOT NULL DEFAULT 0,
|
||||
stage_data JSONB NOT NULL DEFAULT '{}',
|
||||
status TEXT NOT NULL DEFAULT 'active'
|
||||
CHECK (status IN ('active', 'completed', 'cancelled', 'stale', 'error')),
|
||||
started_by UUID REFERENCES users(id) ON DELETE SET NULL,
|
||||
entry_token TEXT, -- for public_link resumption
|
||||
metadata JSONB NOT NULL DEFAULT '{}',
|
||||
stage_entered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_instances_workflow
|
||||
ON workflow_instances(workflow_id, status);
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_instances_status
|
||||
ON workflow_instances(status) WHERE status = 'active';
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_wf_instances_token
|
||||
ON workflow_instances(entry_token) WHERE entry_token IS NOT NULL;
|
||||
|
||||
DROP TRIGGER IF EXISTS wf_instances_updated_at ON workflow_instances;
|
||||
CREATE TRIGGER wf_instances_updated_at BEFORE UPDATE ON workflow_instances
|
||||
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
|
||||
```
|
||||
|
||||
**Rationale:** Chat-switchboard used `channels` as instances — that table carried message trees,
|
||||
AI context windows, participant lists, and other chat concerns. Core needs a purpose-built table
|
||||
that holds only workflow execution state: which version, which stage, accumulated data, lifecycle status.
|
||||
|
||||
#### ➕ ADD `workflow_assignments`
|
||||
|
||||
Dropped during the fork ("channel-dependent, rebuild as needed" — see 007 header comment).
|
||||
Now rebuilt as a kernel primitive.
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS workflow_assignments (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
instance_id UUID NOT NULL REFERENCES workflow_instances(id) ON DELETE CASCADE,
|
||||
stage INTEGER NOT NULL,
|
||||
team_id UUID NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
|
||||
assigned_to UUID REFERENCES users(id) ON DELETE SET NULL,
|
||||
status TEXT NOT NULL DEFAULT 'unassigned'
|
||||
CHECK (status IN ('unassigned', 'claimed', 'completed', 'cancelled')),
|
||||
review_data JSONB NOT NULL DEFAULT '{}',
|
||||
claimed_at TIMESTAMPTZ,
|
||||
completed_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_instance
|
||||
ON workflow_assignments(instance_id, stage);
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_team_status
|
||||
ON workflow_assignments(team_id, status) WHERE status IN ('unassigned', 'claimed');
|
||||
CREATE INDEX IF NOT EXISTS idx_wf_assignments_user
|
||||
ON workflow_assignments(assigned_to) WHERE assigned_to IS NOT NULL;
|
||||
```
|
||||
|
||||
**Claim lock:** Same optimistic pattern from chat-switchboard:
|
||||
`UPDATE ... WHERE id = $1 AND status = 'unassigned'` — if `rows_affected == 0`, already claimed.
|
||||
|
||||
### 3c. New migration file: `012_workflow_instances.sql`
|
||||
|
||||
Wait — per the principle, new *tables* get new files. But we're also editing 007. Decision:
|
||||
|
||||
**Put the new tables in 007_workflows.sql.** The schema isn't in production. Keep all workflow
|
||||
DDL in one file. When the schema *is* in production (post-MVP), new additions get numbered
|
||||
migrations. This is consistent with the existing design-decisions log entry:
|
||||
"No new migrations pre-MVP."
|
||||
|
||||
---
|
||||
|
||||
## 4. Stage Mode Redesign
|
||||
|
||||
### Current (trash/rename)
|
||||
|
||||
| Current Mode | Disposition | Rationale |
|
||||
|-------------|-------------|-----------|
|
||||
| `form_only` | ✏️ MOD → `form` | Rename. Drop the `_only` suffix — there's no `form_chat` to distinguish from anymore. |
|
||||
| `form_chat` | 🗑 TRASH | Chat coupling. If you want a form + AI conversation, use `delegated` with a chat surface package. |
|
||||
| `review` | ✅ KEEP | Team member reviews accumulated data. Pure kernel concept. |
|
||||
| `custom` | ✏️ MOD → `delegated` | Rename for clarity. "Custom" is vague. "Delegated" makes the contract explicit: the kernel delegates stage execution to `surface_pkg_id`. |
|
||||
|
||||
### New modes
|
||||
|
||||
| Mode | Disposition | Description |
|
||||
|------|-------------|-------------|
|
||||
| `form` | ✏️ MOD (renamed) | Kernel renders `form_template`, validates submission, merges into `stage_data`. No extension needed. |
|
||||
| `review` | ✅ KEEP | Assignment queue stage. Team member claims, reviews `stage_data`, adds `review_data`, completes. Kernel-rendered. |
|
||||
| `delegated` | ✏️ MOD (renamed) | Kernel hands off to `surface_pkg_id`. The surface package receives the instance context (stage_data, form_template, metadata) and calls back to advance. This is where chat, AI, or any custom UX lives. |
|
||||
| `automated` | ➕ ADD | No UI. On stage entry, kernel fires `starlark_hook`, merges returned data into `stage_data`, and auto-advances. For enrichment, API calls, validation, routing decisions. |
|
||||
|
||||
### Stage type vs stage mode
|
||||
|
||||
These are orthogonal:
|
||||
|
||||
- **`stage_type`** controls *how the next stage is chosen*: `simple` (declarative branch_rules), `dynamic` (Starlark hook returns target), `automated` (hook + auto-advance).
|
||||
- **`stage_mode`** controls *how the stage executes*: `form`, `review`, `delegated`, `automated`.
|
||||
|
||||
An `automated` stage_type with `automated` stage_mode is the common case for system stages,
|
||||
but you can have a `dynamic` stage_type with `form` stage_mode (the form collects data, then a
|
||||
Starlark hook decides where to go next based on the submission).
|
||||
|
||||
---
|
||||
|
||||
## 5. Model Changes
|
||||
|
||||
### `server/models/workflow.go`
|
||||
|
||||
#### WorkflowStage struct — ✏️ MOD
|
||||
|
||||
```go
|
||||
// TRASH: Remove these fields
|
||||
// PersonaID *string `json:"persona_id,omitempty"`
|
||||
// HistoryMode string `json:"history_mode"`
|
||||
|
||||
// MOD: Rename transition_rules → stage_config
|
||||
// TransitionRules json.RawMessage `json:"transition_rules"`
|
||||
StageConfig json.RawMessage `json:"stage_config"`
|
||||
|
||||
// ADD: New fields
|
||||
Audience string `json:"audience"` // team | public | system
|
||||
StageType string `json:"stage_type"` // simple | dynamic | automated
|
||||
StarlarkHook *string `json:"starlark_hook,omitempty"` // package_id:entry_point
|
||||
BranchRules json.RawMessage `json:"branch_rules"` // [{field, op, value, target_stage}]
|
||||
```
|
||||
|
||||
#### Stage mode constants — ✏️ MOD
|
||||
|
||||
```go
|
||||
// TRASH
|
||||
// StageModeFormOnly = "form_only"
|
||||
// StageModeFormChat = "form_chat"
|
||||
|
||||
// MOD (rename)
|
||||
StageModeForm = "form" // was form_only
|
||||
StageModeDelegated = "delegated" // was custom
|
||||
|
||||
// KEEP
|
||||
StageModeReview = "review"
|
||||
|
||||
// ADD
|
||||
StageModeAutomated = "automated"
|
||||
|
||||
// ADD: Stage type constants
|
||||
StageTypeSimple = "simple"
|
||||
StageTypeDynamic = "dynamic"
|
||||
StageTypeAutomated = "automated"
|
||||
|
||||
// ADD: Audience constants
|
||||
AudienceTeam = "team"
|
||||
AudiencePublic = "public"
|
||||
AudienceSystem = "system"
|
||||
```
|
||||
|
||||
#### ➕ ADD: WorkflowInstance model
|
||||
|
||||
```go
|
||||
type WorkflowInstance struct {
|
||||
ID string `json:"id"`
|
||||
WorkflowID string `json:"workflow_id"`
|
||||
WorkflowVersion int `json:"workflow_version"`
|
||||
CurrentStage int `json:"current_stage"`
|
||||
StageData json.RawMessage `json:"stage_data"`
|
||||
Status string `json:"status"` // active | completed | cancelled | stale | error
|
||||
StartedBy *string `json:"started_by,omitempty"`
|
||||
EntryToken *string `json:"entry_token,omitempty"`
|
||||
Metadata json.RawMessage `json:"metadata"`
|
||||
StageEnteredAt time.Time `json:"stage_entered_at"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
```
|
||||
|
||||
#### ➕ ADD: WorkflowAssignment model
|
||||
|
||||
```go
|
||||
type WorkflowAssignment struct {
|
||||
ID string `json:"id"`
|
||||
InstanceID string `json:"instance_id"`
|
||||
Stage int `json:"stage"`
|
||||
TeamID string `json:"team_id"`
|
||||
AssignedTo *string `json:"assigned_to,omitempty"`
|
||||
Status string `json:"status"` // unassigned | claimed | completed | cancelled
|
||||
ReviewData json.RawMessage `json:"review_data"`
|
||||
ClaimedAt *time.Time `json:"claimed_at,omitempty"`
|
||||
CompletedAt *time.Time `json:"completed_at,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
```
|
||||
|
||||
#### TypedFormTemplate, FormField, FormValidation — ✅ KEEP
|
||||
|
||||
All form types are clean kernel code. No changes.
|
||||
|
||||
#### ValidateFormData — ✅ KEEP
|
||||
|
||||
Pure data validation. No chat coupling.
|
||||
|
||||
---
|
||||
|
||||
## 6. Store Interface Changes
|
||||
|
||||
### `server/store/workflow_iface.go` — ✏️ MOD + ➕ ADD
|
||||
|
||||
Current interface is definition-only (CRUD workflows + stages + versions).
|
||||
Add instance and assignment operations.
|
||||
|
||||
```go
|
||||
type WorkflowStore interface {
|
||||
// ── Definition CRUD (KEEP — no changes) ──────────
|
||||
Create(ctx context.Context, w *models.Workflow) error
|
||||
GetByID(ctx context.Context, id string) (*models.Workflow, error)
|
||||
GetBySlug(ctx context.Context, teamID *string, slug string) (*models.Workflow, error)
|
||||
Update(ctx context.Context, id string, patch models.WorkflowPatch) error
|
||||
Delete(ctx context.Context, id string) error
|
||||
ListForTeam(ctx context.Context, teamID string) ([]models.Workflow, error)
|
||||
ListGlobal(ctx context.Context) ([]models.Workflow, error)
|
||||
|
||||
// ── Stage CRUD (KEEP — no changes) ───────────────
|
||||
CreateStage(ctx context.Context, s *models.WorkflowStage) error
|
||||
ListStages(ctx context.Context, workflowID string) ([]models.WorkflowStage, error)
|
||||
UpdateStage(ctx context.Context, s *models.WorkflowStage) error
|
||||
DeleteStage(ctx context.Context, id string) error
|
||||
ReorderStages(ctx context.Context, workflowID string, orderedIDs []string) error
|
||||
|
||||
// ── Versioning (KEEP — no changes) ───────────────
|
||||
Publish(ctx context.Context, v *models.WorkflowVersion) error
|
||||
GetVersion(ctx context.Context, workflowID string, versionNumber int) (*models.WorkflowVersion, error)
|
||||
GetLatestVersion(ctx context.Context, workflowID string) (*models.WorkflowVersion, error)
|
||||
|
||||
// ── Instances (ADD) ──────────────────────────────
|
||||
CreateInstance(ctx context.Context, inst *models.WorkflowInstance) error
|
||||
GetInstance(ctx context.Context, id string) (*models.WorkflowInstance, error)
|
||||
GetInstanceByToken(ctx context.Context, token string) (*models.WorkflowInstance, error)
|
||||
UpdateInstance(ctx context.Context, id string, patch models.InstancePatch) error
|
||||
ListInstances(ctx context.Context, workflowID string, status string) ([]models.WorkflowInstance, error)
|
||||
AdvanceStage(ctx context.Context, id string, nextStage int, mergeData json.RawMessage) error
|
||||
CompleteInstance(ctx context.Context, id string) error
|
||||
CancelInstance(ctx context.Context, id string) error
|
||||
MarkStale(ctx context.Context, olderThan time.Duration) (int, error)
|
||||
|
||||
// ── Assignments (ADD) ────────────────────────────
|
||||
CreateAssignment(ctx context.Context, a *models.WorkflowAssignment) error
|
||||
ClaimAssignment(ctx context.Context, id string, userID string) error // optimistic lock
|
||||
UnclaimAssignment(ctx context.Context, id string) error
|
||||
CompleteAssignment(ctx context.Context, id string, reviewData json.RawMessage) error
|
||||
CancelAssignment(ctx context.Context, id string) error
|
||||
ListAssignmentsByTeam(ctx context.Context, teamID string, status string) ([]models.WorkflowAssignment, error)
|
||||
ListAssignmentsByUser(ctx context.Context, userID string) ([]models.WorkflowAssignment, error)
|
||||
ListAssignmentsByInstance(ctx context.Context, instanceID string) ([]models.WorkflowAssignment, error)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Handler Changes
|
||||
|
||||
### `server/handlers/workflows.go` — ✏️ MOD
|
||||
|
||||
**Stage CRUD updates** — reflect new field names in CreateStage/UpdateStage:
|
||||
- Remove `persona_id` and `history_mode` from bind/validation.
|
||||
- Add `stage_type`, `starlark_hook`, `branch_rules` to bind/validation.
|
||||
- Rename `transition_rules` → `stage_config` in bind/validation.
|
||||
- Update `stage_mode` CHECK to new set: `form`, `review`, `delegated`, `automated`.
|
||||
- Validate: `delegated` requires `surface_pkg_id`. `dynamic`/`automated` stage_type requires `starlark_hook`.
|
||||
|
||||
### `server/handlers/workflow_hooks.go` — ✏️ MOD
|
||||
|
||||
**Rename:** `channelID` parameter → `instanceID` in `FireOnAdvanceHook` signature.
|
||||
The hook context dict changes from `{channel_id, ...}` to `{instance_id, ...}`.
|
||||
|
||||
The `jsonToStarlark` helper referenced on line 81 lives in
|
||||
`server/handlers/starlark_helpers.go`. Consider consolidating with `goToStarlark` in
|
||||
`server/triggers/event.go` — two independent Go→Starlark converters is tech debt.
|
||||
|
||||
### `server/handlers/workflow_team.go` — ✅ KEEP
|
||||
|
||||
Team-scoped wrappers. Clean delegation pattern. No changes.
|
||||
|
||||
### `server/handlers/workflow_packages.go` — ✏️ MOD
|
||||
|
||||
Update `workflowPkgStage` struct:
|
||||
- Remove `PersonaID`, `HistoryMode` fields.
|
||||
- Add `StageType`, `StarlarkHook`, `BranchRules` fields.
|
||||
- Rename `TransitionRules` → `StageConfig`.
|
||||
|
||||
Update `ExportWorkflowPackage` and `InstallWorkflowFromManifest` accordingly.
|
||||
|
||||
### ➕ ADD `server/handlers/workflow_instances.go`
|
||||
|
||||
New handler file for instance lifecycle:
|
||||
|
||||
```
|
||||
POST /api/v1/workflows/:id/start → Start (create instance from latest published version)
|
||||
GET /api/v1/workflows/instances/:iid → GetInstance
|
||||
POST /api/v1/workflows/instances/:iid/advance → Advance (submit stage data, resolve next)
|
||||
POST /api/v1/workflows/instances/:iid/cancel → Cancel
|
||||
GET /api/v1/workflows/:id/instances → ListInstances (by workflow, optionally by status)
|
||||
```
|
||||
|
||||
Public entry (for `public_link` workflows):
|
||||
```
|
||||
POST /api/v1/workflows/entry/:slug → StartPublic (no auth, generates entry_token)
|
||||
GET /api/v1/workflows/entry/:token → ResumePublic (retrieve instance by token)
|
||||
POST /api/v1/workflows/entry/:token/advance → AdvancePublic (submit data via token)
|
||||
```
|
||||
|
||||
### ➕ ADD `server/handlers/workflow_assignments.go`
|
||||
|
||||
New handler file for the assignment queue:
|
||||
|
||||
```
|
||||
GET /api/v1/teams/:teamId/assignments → ListTeamAssignments (filterable by status)
|
||||
POST /api/v1/teams/:teamId/assignments/:id/claim → Claim
|
||||
POST /api/v1/teams/:teamId/assignments/:id/unclaim → Unclaim
|
||||
POST /api/v1/teams/:teamId/assignments/:id/complete → Complete (with review_data)
|
||||
POST /api/v1/teams/:teamId/assignments/:id/cancel → Cancel
|
||||
GET /api/v1/assignments/mine → ListMyAssignments (across teams)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Routing Engine Changes
|
||||
|
||||
### `server/workflow/routing.go` — ✏️ MOD
|
||||
|
||||
The existing `ResolveNextStage` evaluates `transition_rules.conditions[]`. This needs to
|
||||
become a two-phase resolution that respects the new `stage_type`:
|
||||
|
||||
```
|
||||
Phase 1: Evaluate branch_rules (declarative, for simple + dynamic types)
|
||||
Phase 2: If no match AND stage_type == dynamic → fire starlark_hook
|
||||
Phase 3: Fallback → currentStage + 1
|
||||
```
|
||||
|
||||
For `automated` stage_type, the engine fires `starlark_hook` on entry (not for routing — for
|
||||
data enrichment), then routes via branch_rules or ordinal fallback. The routing call happens
|
||||
*after* the hook returns.
|
||||
|
||||
**Concrete changes:**
|
||||
- `ResolveNextStage` signature: add `stageType string, starlarkHook *string, runner *sandbox.Runner` parameters.
|
||||
- Extract branch_rules evaluation from the current `TransitionRulesWithConditions` (which reads from `transition_rules` JSON) into a dedicated function that reads from the new `branch_rules` field.
|
||||
- `TransitionRulesWithConditions` struct → rename to `StageConfig`, remove `Conditions` field (moved to branch_rules), keep `AutoAssign` and `OnAdvance`.
|
||||
|
||||
### `server/workflow/` — ➕ ADD `engine.go`
|
||||
|
||||
New file: the stage execution engine. Orchestrates the advance lifecycle:
|
||||
|
||||
```
|
||||
1. Load instance + version snapshot
|
||||
2. Validate current stage allows advancement (status checks)
|
||||
3. If current stage has form_template → validate submitted data
|
||||
4. Merge submitted data into stage_data
|
||||
5. Fire on_advance hook (if configured in stage_config)
|
||||
6. Resolve next stage (branch_rules → starlark_hook → ordinal)
|
||||
7. If next stage is past last stage → complete instance
|
||||
8. If next stage is automated → fire its hook, recurse to step 6
|
||||
9. If next stage has assignment_team_id → create WorkflowAssignment
|
||||
10. Update instance (current_stage, stage_data, stage_entered_at)
|
||||
11. Emit bus events (workflow.advanced, workflow.assigned, workflow.completed)
|
||||
```
|
||||
|
||||
### `server/workflow/` — ➕ ADD `automated.go`
|
||||
|
||||
Handles `automated` stage execution: fire the Starlark hook, merge returned data, and
|
||||
auto-advance. Includes a cycle guard (max 10 consecutive automated stages) to prevent
|
||||
infinite loops from misconfigured workflows.
|
||||
|
||||
---
|
||||
|
||||
## 9. Event Bus Updates
|
||||
|
||||
### `server/events/types.go` — ✏️ MOD
|
||||
|
||||
Current workflow events are fine but incomplete. Add:
|
||||
|
||||
| Event | Direction | Disposition | Trigger |
|
||||
|-------|-----------|-------------|---------|
|
||||
| `workflow.assigned` | DirToClient | ✅ KEEP | Assignment created |
|
||||
| `workflow.claimed` | DirToClient | ✅ KEEP | Assignment claimed |
|
||||
| `workflow.advanced` | DirToClient | ✅ KEEP | Stage transition |
|
||||
| `workflow.completed` | DirToClient | ✅ KEEP | Instance completed |
|
||||
| `workflow.cancelled` | DirToClient | ➕ ADD | Instance cancelled |
|
||||
| `workflow.started` | DirToClient | ➕ ADD | Instance created |
|
||||
| `workflow.sla.warning` | DirToClient | ➕ ADD | SLA at 80% threshold |
|
||||
| `workflow.sla.breached` | DirToClient | ➕ ADD | SLA exceeded |
|
||||
| `workflow.error` | DirLocal | ➕ ADD | Hook execution failure |
|
||||
|
||||
---
|
||||
|
||||
## 10. Starlark Module Updates
|
||||
|
||||
### `server/sandbox/workflow_module.go` — ✏️ MOD
|
||||
|
||||
The `BuildWorkflowModule` (in `workflow_module.go`, not `modules.go`) currently exists but
|
||||
targets the old channel-based model. It currently only exposes `get_definition`.
|
||||
Rebuild to expose instance-oriented operations:
|
||||
|
||||
```python
|
||||
# Available to extensions with workflow.read or workflow.write permission
|
||||
workflow.get_instance(instance_id) # → instance dict
|
||||
workflow.get_stage_data(instance_id) # → stage_data dict
|
||||
workflow.set_stage_data(instance_id, data) # → merge into stage_data
|
||||
workflow.advance(instance_id) # → trigger advance (write perm)
|
||||
workflow.get_assignments(instance_id) # → assignment list
|
||||
```
|
||||
|
||||
The old `workflow_advance()` function that chat-switchboard's personas called is gone.
|
||||
Extensions call `workflow.advance()` which delegates to the same engine as the HTTP handler.
|
||||
|
||||
---
|
||||
|
||||
## 11. Salvage Map from chat-switchboard v0.39.x
|
||||
|
||||
What was planned there, and what happens to each idea in core:
|
||||
|
||||
### v0.39.0 — Stage Graph Engine + Form Builder
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| `stage_type` enum | ➕ ADD | Adopted directly: `simple`, `dynamic`, `automated` |
|
||||
| `BranchRule` struct | ➕ ADD | Adopted as `branch_rules` JSONB on `workflow_stages` |
|
||||
| Starlark hook integration | ➕ ADD | `starlark_hook` column + engine integration |
|
||||
| Graph engine (`server/engine/graph.go`) | ➕ ADD | `server/workflow/engine.go` in core |
|
||||
| Automated stage runner | ➕ ADD | `server/workflow/automated.go` in core |
|
||||
| Visual form builder (5 Preact components) | 🗑 TRASH *for now* | The form builder was tightly coupled to chat-switchboard's stage editor surface. Core doesn't ship a workflow editor surface (that's an extension). The form *schema* is kernel; the form *builder UI* is a surface package concern. Revisit when a workflow-admin surface is built. |
|
||||
| Stage type badges in UI | 🗑 TRASH | Same — UI is extension territory. |
|
||||
|
||||
### v0.39.1 — Stage Reorder + Bulk Ops
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Drag-and-drop reorder | 🗑 TRASH | UI concern. The kernel `ReorderStages` endpoint already exists. |
|
||||
| Bulk assignment ops | ➕ ADD (later) | Useful but not blocking. Defer to v0.2.7+. The individual claim/unclaim/cancel endpoints come first. |
|
||||
|
||||
### v0.39.2 — SLA Alerting
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Periodic SLA scanner | ➕ ADD | Kernel scheduled goroutine (like the existing staleness sweep). Query active instances with `sla_seconds` configured, compute breach from `stage_entered_at`. |
|
||||
| `sla_notified_at` on instances | ➕ ADD | Add to `workflow_instances` schema. Prevents duplicate notifications. |
|
||||
| Notifications on breach | ➕ ADD | Use existing `notifications` table + bus event (`workflow.sla.breached`). |
|
||||
| Webhook on breach | ➕ ADD | Fire workflow's `webhook_url` if configured. |
|
||||
| Pulsing badge animation | 🗑 TRASH | UI concern for a surface package. |
|
||||
|
||||
### v0.39.3 — Visitor Portal
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Public entry endpoint | ➕ ADD | `/api/v1/workflows/entry/:slug` — kernel handles anonymous session creation + token issuance. |
|
||||
| Session resume via token | ➕ ADD | `entry_token` on `workflow_instances` + `GetInstanceByToken` store method. |
|
||||
| Branded entry page | 🗑 TRASH | Surface package. The kernel stores `branding` JSONB — a portal surface reads it. |
|
||||
| Progress indicator | 🗑 TRASH | Surface package reads stage list, computes "step N of M" locally. |
|
||||
| Email notifications | 🗑 TRASH *for now* | Requires SMTP infrastructure. Defer to post-MVP or make it a connection-based extension (use ext_connections for SMTP config, fire via Starlark hook). |
|
||||
|
||||
### v0.39.4 — Templates + Clone
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Clone endpoint | ➕ ADD | `POST /api/v1/workflows/:id/clone` — deep copy workflow + stages. Straightforward. |
|
||||
| Built-in templates table | 🗑 TRASH | Core doesn't ship opinionated templates. Templates are workflow packages (`.pkg` archives). A "template gallery" is a surface. |
|
||||
| Template picker UI | 🗑 TRASH | Surface concern. |
|
||||
|
||||
### v0.39.5 — Analytics
|
||||
|
||||
| Component | Disposition | Core equivalent |
|
||||
|-----------|-------------|-----------------|
|
||||
| Analytics query endpoint | ➕ ADD (later) | `GET /api/v1/workflows/:id/analytics?period=7d` — kernel exposes raw metrics (throughput, cycle time, stage dwell, SLA compliance) derived from instance + assignment timestamps. Defer to v0.2.8+ or v0.3.x. |
|
||||
| Analytics dashboard UI | 🗑 TRASH | Surface package. |
|
||||
|
||||
---
|
||||
|
||||
## 12. Implementation Order
|
||||
|
||||
All items within v0.2.6. Ordered by dependency:
|
||||
|
||||
### Phase A: Schema + Models
|
||||
|
||||
1. ✏️ Edit `007_workflows.sql`: drop `persona_id`, `history_mode`; add `audience`, `stage_type`, `starlark_hook`, `branch_rules`; rename `transition_rules` → `stage_config`; update `stage_mode` CHECK.
|
||||
2. ✏️ Edit `007_workflows.sql` (same file): add `workflow_instances` and `workflow_assignments` tables.
|
||||
3. ✏️ Edit `007_workflows.sql` (SQLite dialect): mirror changes.
|
||||
4. ✏️ Update `server/models/workflow.go`: remove trashed fields, add new structs, update constants.
|
||||
|
||||
### Phase B: Store Layer
|
||||
|
||||
5. ✏️ Update `server/store/workflow_iface.go`: add instance + assignment methods.
|
||||
6. ➕ Implement Postgres store: `server/store/postgres/workflow_instances.go`, `workflow_assignments.go`.
|
||||
7. ➕ Implement SQLite store: `server/store/sqlite/workflow_instances.go`, `workflow_assignments.go`.
|
||||
|
||||
### Phase C: Engine
|
||||
|
||||
8. ✏️ Update `server/workflow/routing.go`: branch_rules evaluation, starlark_hook integration.
|
||||
9. ➕ Add `server/workflow/engine.go`: advance lifecycle orchestration.
|
||||
10. ➕ Add `server/workflow/automated.go`: automated stage execution + cycle guard.
|
||||
11. ✏️ Update `server/handlers/workflow_hooks.go`: rename channel → instance references.
|
||||
|
||||
### Phase D: Handlers + Routes
|
||||
|
||||
12. ✏️ Update `server/handlers/workflows.go`: stage CRUD field changes.
|
||||
13. ✏️ Update `server/handlers/workflow_packages.go`: package struct field changes.
|
||||
14. ➕ Add `server/handlers/workflow_instances.go`: instance lifecycle endpoints.
|
||||
15. ➕ Add `server/handlers/workflow_assignments.go`: assignment queue endpoints.
|
||||
16. ✏️ Wire new routes in `server/main.go`.
|
||||
|
||||
### Phase E: Events + Starlark
|
||||
|
||||
17. ✏️ Update `server/events/types.go`: add new workflow events.
|
||||
18. ✏️ Update `server/sandbox/modules.go`: rebuild workflow Starlark module.
|
||||
|
||||
### Phase F: Background Jobs
|
||||
|
||||
19. ➕ Add SLA scanner goroutine (check active instances, fire notifications + events on breach).
|
||||
20. ➕ Add staleness sweep for workflow instances (reuse pattern from chat-switchboard).
|
||||
|
||||
### Phase G: Verification
|
||||
|
||||
21. ➕ Integration tests: instance lifecycle, stage advancement, branching, automated stages, assignments, SLA breach.
|
||||
22. `go build ./...` clean.
|
||||
23. Update ICD (OpenAPI spec) with new endpoints.
|
||||
24. Update ROADMAP.md + CHANGELOG.md.
|
||||
|
||||
---
|
||||
|
||||
## 13. What We're NOT Doing (Explicit Deferrals)
|
||||
|
||||
| Item | Why | Revisit |
|
||||
|------|-----|---------|
|
||||
| Visual form builder UI | Surface concern, no workflow-admin surface yet | When a workflow-admin surface ships |
|
||||
| Drag-and-drop stage reorder UI | Surface concern | Same |
|
||||
| Email notifications | Needs SMTP infrastructure or connection-based ext | Post-MVP |
|
||||
| Built-in workflow templates | Core doesn't ship opinionated content | Package gallery |
|
||||
| Analytics dashboard | Nice-to-have, kernel endpoint can come first | v0.3.x |
|
||||
| Bulk assignment operations | Individual ops first | v0.2.7+ |
|
||||
| Workflow chaining (`on_complete`) | Already in schema, needs engine wiring | v0.2.7 |
|
||||
| Visitor portal surface | Surface package, not kernel | Extension track |
|
||||
|
||||
---
|
||||
|
||||
## 14. Migration Policy Reminder
|
||||
|
||||
From the design-decisions log:
|
||||
|
||||
> **No new migrations pre-MVP.** Edit existing migration SQL files in place.
|
||||
> No migration chains until schema is in production.
|
||||
|
||||
For this redesign:
|
||||
- **`007_workflows.sql`**: Edit in place for column changes AND add new tables to the same file.
|
||||
- **No `012_*.sql`**: Everything goes in 007.
|
||||
- **Both dialects**: Postgres and SQLite must be kept in sync.
|
||||
|
||||
The only time a new numbered migration file is created is when a truly new,
|
||||
unrelated subsystem is added (like 011_triggers.sql was for the trigger system).
|
||||
Workflow instances and assignments are part of the workflow subsystem → they go in 007.
|
||||
|
||||
---
|
||||
|
||||
## 15. Public Surfaces — Kernel Capability, Not Workflow Feature
|
||||
|
||||
### The problem with workflow-owned public access
|
||||
|
||||
Chat-switchboard treated public access as a binary workflow toggle: `entry_mode: public_link`
|
||||
made the whole workflow public. But real workflows alternate audiences — internal setup →
|
||||
public-facing customer form → internal review → public confirmation page. And public access
|
||||
isn't even a workflow-only need: feedback forms, status pages, and surveys all need anonymous
|
||||
sessions without being workflows.
|
||||
|
||||
### Per-stage audience (the v0.2.6 solution)
|
||||
|
||||
The `audience` field on `workflow_stages` is the source of truth:
|
||||
|
||||
| Audience | Who interacts | Auth required | Example |
|
||||
|----------|--------------|---------------|---------|
|
||||
| `team` | Authenticated team members | JWT | Internal triage, setup, review |
|
||||
| `public` | Anonymous visitors | Entry token | Customer intake form, confirmation page |
|
||||
| `system` | Nobody (automated) | N/A | API enrichment, routing decision |
|
||||
|
||||
A single workflow freely alternates:
|
||||
|
||||
```
|
||||
Stage 0: "Setup" audience=team (agent configures parameters)
|
||||
Stage 1: "Customer Form" audience=public (customer fills out intake)
|
||||
Stage 2: "Internal Review" audience=team (team reviews submission)
|
||||
Stage 3: "Confirmation" audience=public (customer sees result)
|
||||
```
|
||||
|
||||
The engine enforces boundaries:
|
||||
- When the instance enters a `public` stage, the visitor's token grants access. Team members
|
||||
can also see it (they have elevated access).
|
||||
- When the instance enters a `team` stage, the visitor's view freezes. They see "being
|
||||
processed" or whatever the portal surface shows. The token is still valid for resumption
|
||||
when the next `public` stage arrives.
|
||||
- When the instance enters a `system` stage, nobody interacts — the automated hook fires
|
||||
and the engine advances.
|
||||
|
||||
### The `entry_mode` column
|
||||
|
||||
Stays on the `workflows` table but becomes a derived convenience flag. If any stage has
|
||||
`audience = 'public'`, the workflow is public-entry eligible. The kernel can auto-compute
|
||||
this when stages are saved, or treat it as an explicit admin toggle that gates whether
|
||||
public stages are actually reachable. Either way, it's no longer the primary access control
|
||||
mechanism — `audience` on each stage is.
|
||||
|
||||
### Platform-level public access (v0.2.7+ concern)
|
||||
|
||||
The broader vision: public access as a **package-level manifest capability** so any surface
|
||||
(not just workflows) can serve anonymous visitors. A package declares `"public": true` in
|
||||
its manifest, and the kernel mounts a public route namespace (`/p/:package_slug/*`) with
|
||||
anonymous session middleware, token issuance, and rate limiting. Workflows consume this
|
||||
capability — they don't own it.
|
||||
|
||||
For v0.2.6, the workflow-specific public entry routes are sufficient:
|
||||
```
|
||||
POST /api/v1/workflows/entry/:slug → StartPublic
|
||||
GET /api/v1/workflows/entry/:token → ResumePublic
|
||||
POST /api/v1/workflows/entry/:token/advance → AdvancePublic
|
||||
```
|
||||
|
||||
The anonymous session management (entry_token on `workflow_instances`, IP rate limiting)
|
||||
lives in the workflow handler for now. Generalization to a platform-level `public_sessions`
|
||||
table happens when non-workflow surfaces need it.
|
||||
|
||||
### Visitor view contract
|
||||
|
||||
When a public stage is active, the kernel API returns only what the visitor should see:
|
||||
- The current stage's `form_template` (if mode is `form`)
|
||||
- The stage name and ordinal (for progress indicators)
|
||||
- Accumulated `stage_data` keys marked as `visitor_visible` (future: field-level ACL)
|
||||
- The `branding` JSONB from the workflow
|
||||
|
||||
Internal stages, team assignments, review data, and other instances' data are never
|
||||
exposed through the token-authenticated endpoints. The surface package renders whatever
|
||||
the kernel returns — it doesn't need to filter.
|
||||
|
||||
---
|
||||
|
||||
## 16. Branch Rules — Routing, Skipping, and Convergence
|
||||
|
||||
### How branch_rules work
|
||||
|
||||
Each stage has a `branch_rules` JSONB array. On stage completion, the engine evaluates rules
|
||||
top-to-bottom against accumulated `stage_data`. First match wins. No match → ordinal + 1.
|
||||
|
||||
```json
|
||||
[
|
||||
{"field": "priority", "op": "eq", "value": "urgent", "target_stage": "Emergency Review"},
|
||||
{"field": "amount", "op": "gt", "value": 10000, "target_stage": "Manager Approval"},
|
||||
{"field": "type", "op": "in", "value": ["a","b"], "target_stage": "Fast Track"}
|
||||
]
|
||||
```
|
||||
|
||||
`target_stage` accepts either a **stage name** (case-insensitive) or a **numeric ordinal**.
|
||||
The engine tries name resolution first, falls back to numeric. This is already implemented
|
||||
in `routing.go` → `resolveTarget()`.
|
||||
|
||||
### Skipping stages
|
||||
|
||||
Branch rules can target any stage, not just the next one. If a workflow has stages
|
||||
0→1→2→3→4 and stage 1's rules say `target_stage: "Stage 4"`, stages 2 and 3 are skipped
|
||||
for that instance. The engine doesn't care about gaps — it sets `current_stage` to whatever
|
||||
the rule resolved to.
|
||||
|
||||
Linear workflows (no branch_rules on any stage) advance 0→1→2→3→4 by the ordinal + 1
|
||||
fallback. Adding a single branch rule to any stage introduces conditional skipping without
|
||||
changing the linear stages.
|
||||
|
||||
### Convergence (diamond patterns)
|
||||
|
||||
Two different branches can target the same downstream stage:
|
||||
|
||||
```
|
||||
Stage 0: Intake Form
|
||||
branch_rules: [
|
||||
{field: "region", op: "eq", value: "US", target_stage: "US Processing"},
|
||||
{field: "region", op: "eq", value: "EU", target_stage: "EU Processing"}
|
||||
]
|
||||
|
||||
Stage 1: US Processing (ordinal 1)
|
||||
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
|
||||
(always converge after processing)
|
||||
|
||||
Stage 2: EU Processing (ordinal 2)
|
||||
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
|
||||
(always converge after processing)
|
||||
|
||||
Stage 3: Final Review (ordinal 3)
|
||||
(both branches land here)
|
||||
```
|
||||
|
||||
No special syntax needed. The routing engine doesn't track "which branch am I on" — it just
|
||||
resolves the target and moves the instance there. The accumulated `stage_data` carries
|
||||
everything downstream stages need to know about what happened upstream.
|
||||
|
||||
### Backward jumps
|
||||
|
||||
Branch rules can also target earlier stages (lower ordinals) for retry/correction loops.
|
||||
The cycle guard (max 10 consecutive automated stages) in `automated.go` prevents infinite
|
||||
loops for automated stages. For human-facing stages, backward jumps are intentional — the
|
||||
human breaks the cycle by submitting different data.
|
||||
|
||||
### Operators
|
||||
|
||||
Full set, inherited from the existing `routing.go` implementation:
|
||||
|
||||
| Op | Description | Example |
|
||||
|----|-------------|---------|
|
||||
| `eq` | Loose equality (string comparison) | `"status" eq "approved"` |
|
||||
| `neq` | Not equal | `"status" neq "rejected"` |
|
||||
| `gt` | Greater than (numeric) | `"amount" gt 10000` |
|
||||
| `lt` | Less than (numeric) | `"amount" lt 100` |
|
||||
| `gte` | Greater or equal | `"score" gte 80` |
|
||||
| `lte` | Less or equal | `"score" lte 20` |
|
||||
| `in` | Value in array | `"category" in ["a","b","c"]` |
|
||||
| `contains` | Substring match | `"notes" contains "urgent"` |
|
||||
| `exists` | Field present in stage_data | `"approval_date" exists` |
|
||||
| `not_exists` | Field absent | `"rejection_reason" not_exists` |
|
||||
111
docs/DESIGN-WORKFLOWS.md
Normal file
111
docs/DESIGN-WORKFLOWS.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# Workflow Engine — Design
|
||||
|
||||
The workflow engine provides multi-stage, form-driven processes with team
|
||||
validation, SLA enforcement, and public entry. Workflows are kernel primitives;
|
||||
workflow **surfaces** (admin UI, public landing pages) are extension packages.
|
||||
|
||||
---
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### Workflow Definition
|
||||
|
||||
A workflow is a named process template owned by a team. Each workflow defines an
|
||||
ordered list of **stages** that instances progress through.
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `name` | Human-readable workflow name |
|
||||
| `slug` | URL-safe identifier (unique per scope) |
|
||||
| `scope` | Owning team or org |
|
||||
| `stages` | Ordered list of stage definitions |
|
||||
| `entry_mode` | `internal` (authenticated users) or `public_link` (landing page) |
|
||||
| `sla_hours` | Optional SLA deadline for instance completion |
|
||||
|
||||
### Stage Types
|
||||
|
||||
Each stage has a `mode` that determines how participants interact:
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `form_only` | Structured form submission. Stage advances when form is complete. |
|
||||
| `form_chat` | Form plus threaded discussion. Useful for review with comments. |
|
||||
| `review` | Approval/rejection gate. Designated reviewers must sign off. |
|
||||
| `custom` | Delegates entirely to a surface package. Enables extension-driven UIs. |
|
||||
|
||||
Stages declare an `assignee_mode` (individual, team, role-based) and optional
|
||||
form schemas (JSON Schema validated at submission time).
|
||||
|
||||
### Workflow Instances
|
||||
|
||||
An instance is a running execution of a workflow definition. It tracks:
|
||||
|
||||
- Current stage index and overall status (`active`, `completed`, `cancelled`, `stale`)
|
||||
- Per-stage completion records with timestamps and actor IDs
|
||||
- Form data submitted at each stage
|
||||
- Assignment and claim history
|
||||
|
||||
Instances use optimistic locking (`claimed_by` + `claimed_at`) to prevent
|
||||
concurrent stage advancement. A claim expires after a configurable timeout.
|
||||
|
||||
---
|
||||
|
||||
## Multi-Party Validation (Signoff Table)
|
||||
|
||||
The `workflow_signoffs` table enables multi-party approval gates:
|
||||
|
||||
| Column | Purpose |
|
||||
|--------|---------|
|
||||
| `instance_id` | Which instance |
|
||||
| `stage_index` | Which stage requires signoff |
|
||||
| `user_id` | Who signed |
|
||||
| `decision` | `approved` or `rejected` |
|
||||
| `comment` | Optional rationale |
|
||||
| `signed_at` | Timestamp |
|
||||
|
||||
The engine checks signoff requirements before advancing past a `review` stage.
|
||||
Requirements are configurable: unanimous, majority, or N-of-M.
|
||||
|
||||
---
|
||||
|
||||
## SLA Scanner + Staleness Sweep
|
||||
|
||||
Two periodic background jobs maintain workflow health:
|
||||
|
||||
**SLA Scanner** — Runs on a configurable interval. Identifies instances that have
|
||||
exceeded their workflow's `sla_hours` deadline. Fires a `workflow.sla.breached`
|
||||
event (consumed by notification extensions or triggers).
|
||||
|
||||
**Staleness Sweep** — Identifies instances that have been idle (no stage
|
||||
advancement) beyond a configurable threshold. Marks them as `stale` and fires
|
||||
`workflow.instance.stale`. Stale instances can be resumed or cancelled.
|
||||
|
||||
Both jobs are distributed-safe: all cluster nodes run them, but operations are
|
||||
idempotent (UPDATE with WHERE guards).
|
||||
|
||||
---
|
||||
|
||||
## Public Entry
|
||||
|
||||
Workflows with `entry_mode: public_link` are accessible at:
|
||||
|
||||
```
|
||||
/w/:scope/:slug
|
||||
```
|
||||
|
||||
The landing page renders the first stage's form without requiring authentication.
|
||||
On submission, the kernel creates an instance and stores the form data. Subsequent
|
||||
stages may require authentication depending on their assignee configuration.
|
||||
|
||||
Public entry enables use cases like intake forms, support requests, and
|
||||
application submissions where the initiator is external.
|
||||
|
||||
---
|
||||
|
||||
## Extension Points
|
||||
|
||||
- **Surface packages** provide workflow admin UIs and participant views
|
||||
- **Trigger extensions** can react to workflow events (`instance.created`,
|
||||
`stage.advanced`, `sla.breached`, `instance.completed`)
|
||||
- **Custom stage mode** delegates rendering and advancement logic entirely to
|
||||
an extension, enabling arbitrary UIs within a workflow stage
|
||||
290
docs/DESIGN-cluster-registry.md
Normal file
290
docs/DESIGN-cluster-registry.md
Normal file
@@ -0,0 +1,290 @@
|
||||
# Design: PG-Backed Cluster Registry & Self-Assembling Mesh
|
||||
|
||||
**Status:** Proposed — Pre-MVP / MVP wrap-up
|
||||
**Author:** Jeff
|
||||
**Date:** 2026-03-30
|
||||
**Scope:** Kernel primitive — zero domain awareness
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
Switchboard-core is tightly coupled to PostgreSQL. Scaling horizontally requires instance coordination: peer discovery, health monitoring, ephemeral event routing, and (optionally) leader election. Traditional HA solutions (Raft, etcd, Consul) introduce a second consensus layer on top of PG — doubling operational complexity for a system that already has serializable transactions and LISTEN/NOTIFY.
|
||||
|
||||
## Principle
|
||||
|
||||
PG is already the consensus system. Use it as the sole coordinator for service discovery, health, pub/sub, and cluster state. Zero new infrastructure dependencies.
|
||||
|
||||
---
|
||||
|
||||
## Design
|
||||
|
||||
### 1. Node Registry (Unlogged Table)
|
||||
|
||||
```sql
|
||||
CREATE UNLOGGED TABLE IF NOT EXISTS node_registry (
|
||||
node_id TEXT PRIMARY KEY,
|
||||
endpoint TEXT NOT NULL,
|
||||
seq SERIAL,
|
||||
registered_at TIMESTAMPTZ DEFAULT now(),
|
||||
heartbeat TIMESTAMPTZ DEFAULT now(),
|
||||
stats JSONB DEFAULT '{}'
|
||||
);
|
||||
```
|
||||
|
||||
**Why UNLOGGED:** No WAL overhead, visible to all connections, survives session disconnect. Contents do NOT survive a PG crash — but a PG crash means all nodes are dead anyway. Clean slate on recovery is the correct behavior; all nodes re-register on restart.
|
||||
|
||||
**Why not TEMPORARY:** PG temporary tables are session-scoped and invisible to other connections. Useless for cross-instance coordination.
|
||||
|
||||
### 2. Node Lifecycle
|
||||
|
||||
#### Startup
|
||||
|
||||
```
|
||||
boot → connect PG → register self → read peer list →
|
||||
subscribe LISTEN/NOTIFY channels → open WS to clients → ready
|
||||
```
|
||||
|
||||
```sql
|
||||
-- Self-registration (atomic, idempotent)
|
||||
INSERT INTO node_registry (node_id, endpoint)
|
||||
VALUES ($1, $2)
|
||||
ON CONFLICT (node_id) DO UPDATE
|
||||
SET endpoint = EXCLUDED.endpoint,
|
||||
registered_at = now(),
|
||||
heartbeat = now(),
|
||||
stats = '{}';
|
||||
```
|
||||
|
||||
`node_id` is generated at startup: `hostname + PID` or a UUID. Deterministic IDs enable restart-in-place without orphaned rows.
|
||||
|
||||
#### Heartbeat Tick (every N seconds, e.g., 10s)
|
||||
|
||||
Each node performs two operations per tick:
|
||||
|
||||
```go
|
||||
// 1. Update own heartbeat + stats
|
||||
stats := collectStats() // see §5
|
||||
db.Exec(`UPDATE node_registry SET heartbeat = now(), stats = $2 WHERE node_id = $1`,
|
||||
nodeID, stats)
|
||||
|
||||
// 2. Sweep stale entries (idempotent — safe for concurrent execution)
|
||||
db.Exec(`DELETE FROM node_registry WHERE heartbeat < now() - interval '30 seconds'`)
|
||||
```
|
||||
|
||||
The sweep is the distributed health check. Every node runs it. Multiple nodes deleting the same stale row is a no-op. No ring topology required — the sweep is the safety net that guarantees eventual cleanup regardless of which nodes are alive.
|
||||
|
||||
#### Self-Eviction
|
||||
|
||||
If a node's own heartbeat UPDATE returns 0 rows affected, it has been swept by a peer:
|
||||
|
||||
```go
|
||||
result := db.Exec(`UPDATE node_registry SET heartbeat = now() WHERE node_id = $1`, nodeID)
|
||||
if result.RowsAffected() == 0 {
|
||||
log.Error("evicted from registry, shutting down")
|
||||
os.Exit(1)
|
||||
}
|
||||
```
|
||||
|
||||
Kubernetes restarts the process. On boot it re-registers as a fresh node. The restart loop IS the recovery mechanism.
|
||||
|
||||
### 3. Leaderless by Default
|
||||
|
||||
No `role` column. No primary/replica distinction. All nodes are equal peers. PG is the sole coordinator.
|
||||
|
||||
**If a leader is ever needed** (future use case), it's a small lift:
|
||||
|
||||
```sql
|
||||
ALTER TABLE node_registry ADD COLUMN role TEXT DEFAULT 'peer';
|
||||
|
||||
-- Atomic leader claim: first writer wins
|
||||
UPDATE node_registry SET role = 'primary'
|
||||
WHERE node_id = $1
|
||||
AND NOT EXISTS (SELECT 1 FROM node_registry WHERE role = 'primary');
|
||||
```
|
||||
|
||||
No quorum, no voting, no term numbers. PG serialization handles it. But the current design intentionally avoids this — consistent with the existing pattern where scheduled tasks use first-to-mark-wins execution, not leader assignment.
|
||||
|
||||
### 4. Realtime Event Routing (Two Tiers)
|
||||
|
||||
#### Tier 1: Durable Events (messages, state changes)
|
||||
|
||||
These are PG writes. Fan-out via `LISTEN/NOTIFY`:
|
||||
|
||||
```
|
||||
User A sends message → Instance 1 writes to PG →
|
||||
pg_notify('channel:xyz', '{"type":"message","id":"..."}') →
|
||||
All instances receive → push to local WS clients subscribed to channel:xyz
|
||||
```
|
||||
|
||||
This maps directly to the v0.5.0 `realtime.publish()` Starlark primitive. Kernel implementation is `pg_notify(channel, payload)`.
|
||||
|
||||
#### Tier 2: Ephemeral Events (typing indicators, presence, cursor position)
|
||||
|
||||
These NEVER hit durable storage.
|
||||
|
||||
**Phase 1 (MVP):** Use PG LISTEN/NOTIFY for ephemeral events too.
|
||||
|
||||
```
|
||||
User A types → Instance 1 WS handler →
|
||||
pg_notify('ephemeral:xyz', '{"type":"typing","user":"A"}') →
|
||||
Instance 2 receives → pushes to User B's WS
|
||||
Instance 3 receives → pushes to User C's WS
|
||||
```
|
||||
|
||||
PG NOTIFY payload limit is 8KB. A typing indicator is ~60 bytes. At homelab scale (3 nodes, dozens of users) this is a non-issue. One codepath, one transport, zero new infrastructure.
|
||||
|
||||
**Phase 2 (Post-MVP, if needed):** Direct instance-to-instance mesh for ephemeral events.
|
||||
|
||||
- Peer endpoints are already in the registry
|
||||
- Each node dials peers discovered from the registry on startup and on peer list changes
|
||||
- Ephemeral events route over the mesh; durable events stay on PG NOTIFY
|
||||
- The `realtime.publish()` Starlark API does not change — kernel swaps the underlying transport
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
│ PG (source of truth) │
|
||||
│ ┌───────────┐ ┌──────────────┐ │
|
||||
│ │ node_ │ │ LISTEN/ │ │
|
||||
│ │ registry │ │ NOTIFY │ │
|
||||
│ └───────────┘ └──────────────┘ │
|
||||
└──────┬──────────────┬────────────┘
|
||||
│ durable │
|
||||
┌────────────┼──────────────┼────────────┐
|
||||
│ │ │ │
|
||||
┌────┴────┐ ┌───┴─────┐ ┌────┴─────┐
|
||||
│ Node 1 │◄─┤ Node 2 ├─►│ Node 3 │
|
||||
│ User A │ │ User B │ │ User C │
|
||||
└─────────┘ └─────────┘ └──────────┘
|
||||
◄──── ephemeral mesh ────►
|
||||
(Phase 2, if needed)
|
||||
```
|
||||
|
||||
**Channel subscription model (Phase 1):** Broadcast and discard. Every node receives every event, checks local WebSocket subscriptions, drops if irrelevant. At 3 nodes this is the correct answer — trivial, no subscription state to sync, fully leaderless.
|
||||
|
||||
### 5. Stats Collection & Admin Dashboard
|
||||
|
||||
Each heartbeat tick publishes runtime stats in the `stats` JSONB column:
|
||||
|
||||
```go
|
||||
var m runtime.MemStats
|
||||
runtime.ReadMemStats(&m)
|
||||
|
||||
stats := map[string]any{
|
||||
"goroutines": runtime.NumGoroutine(),
|
||||
"heap_alloc": m.HeapAlloc,
|
||||
"heap_sys": m.HeapSys,
|
||||
"gc_cycles": m.NumGC,
|
||||
"gc_pause_ns": m.PauseNs[(m.NumGC+255)%256],
|
||||
"uptime_sec": time.Since(startTime).Seconds(),
|
||||
"ws_clients": wsHub.Count(),
|
||||
"extensions": sandbox.LoadedCount(),
|
||||
}
|
||||
```
|
||||
|
||||
Admin dashboard API endpoint:
|
||||
|
||||
```sql
|
||||
SELECT node_id, endpoint, registered_at, heartbeat, stats
|
||||
FROM node_registry
|
||||
ORDER BY seq;
|
||||
```
|
||||
|
||||
**ICD envelope:** `{"data": [...]}` — consistent with list endpoint convention.
|
||||
|
||||
The admin dashboard extension renders one card per node. One node = one card. Three nodes = three cards. The extension has zero awareness of clustering — it reads a list and renders it. This is the extensibility proof working as designed.
|
||||
|
||||
Since `stats` is JSONB, future additions (mesh peer latency, extension execution counts, queue depths) require no schema migration. The dashboard extension renders whatever keys are present.
|
||||
|
||||
### 6. Self-Assembling Mesh Properties
|
||||
|
||||
The mesh builds itself from the registry with zero configuration:
|
||||
|
||||
| Event | Behavior |
|
||||
|---|---|
|
||||
| Node starts | Inserts into registry → peers discover it on next heartbeat tick |
|
||||
| Node dies | Heartbeat goes stale → swept by peers → mesh shrinks |
|
||||
| Node restarts | Re-registers → peers discover it → mesh restores |
|
||||
| Scale up (K8s) | New replicas register → mesh grows automatically |
|
||||
| Scale down (K8s) | Terminated nodes go stale → swept → mesh shrinks |
|
||||
| PG crash | All nodes lose connection → all exit → K8s restarts all → clean re-registration |
|
||||
|
||||
No config files listing peers. No seed nodes. No service discovery sidecar. No bootstrap ceremony. The registry table IS the service mesh control plane.
|
||||
|
||||
---
|
||||
|
||||
## What This Enables (Future)
|
||||
|
||||
These are NOT in scope for this work. Listed to validate the design supports them without rearchitecting.
|
||||
|
||||
- **Work distribution:** Primary (if elected) assigns extension execution shards via registry metadata. Replicas poll for assignments. No message queue.
|
||||
- **Realtime routing (v0.5.0):** When a node owns a pub/sub channel, it registers ownership in PG. Other nodes route publishes to the owner. Node dies → sweep clears routes → subscribers reconnect.
|
||||
- **Rolling deploys:** New version registers, old version's heartbeat goes stale, clean handoff with zero coordination protocol.
|
||||
- **Session affinity:** Registry tracks which node holds a user's WS connection. Targeted event delivery instead of broadcast (optimization for scale).
|
||||
|
||||
---
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Migration
|
||||
|
||||
Pre-MVP: fold `CREATE UNLOGGED TABLE` into existing schema initialization. No new migration file needed. DB rebuild required (acceptable — no install base).
|
||||
|
||||
### Configuration
|
||||
|
||||
| Setting | Default | Notes |
|
||||
|---|---|---|
|
||||
| `CLUSTER_NODE_ID` | `hostname-PID` | Override for deterministic identity across restarts |
|
||||
| `CLUSTER_HEARTBEAT_INTERVAL` | `10s` | Tick frequency |
|
||||
| `CLUSTER_STALE_THRESHOLD` | `30s` | 3x heartbeat = stale |
|
||||
| `CLUSTER_ENDPOINT` | auto-detect | Advertised address for peer mesh (Phase 2) |
|
||||
|
||||
### Single-Node Behavior
|
||||
|
||||
When only one node is registered, the system behaves identically to pre-cluster switchboard-core. The registry has one row. LISTEN/NOTIFY delivers events back to the same instance. No special-casing required.
|
||||
|
||||
### Health Endpoint Integration
|
||||
|
||||
`GET /health` includes cluster status:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "healthy",
|
||||
"node_id": "node-abc-1234",
|
||||
"cluster": {
|
||||
"size": 3,
|
||||
"peers": ["node-def-5678", "node-ghi-9012"],
|
||||
"heartbeat_age_ms": 2340
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Satisfies the v0.6.0 MVP health monitoring requirement.
|
||||
|
||||
---
|
||||
|
||||
## Rejected Alternatives
|
||||
|
||||
| Alternative | Why rejected |
|
||||
|---|---|
|
||||
| etcd / Consul | Second consensus layer on top of PG. Doubles operational complexity for a system already tightly coupled to PG. Violates KISS. |
|
||||
| Redis pub/sub for ephemeral events | Second stateful service. Two failure domains, two connection pools, two things to monitor. PG NOTIFY handles the same job at homelab scale. |
|
||||
| Raft implementation in Go | Massive complexity for a problem PG serializable transactions already solve. |
|
||||
| Ring topology monitoring | Each node watches the one above it. Adjacent failures create blind spots. Sweep-all is simpler and has no edge cases. |
|
||||
| Leader election at startup | Unnecessary. Leaderless design with first-to-mark-wins task execution already works. Leader election is a future option, not a requirement. |
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] `node_registry` unlogged table in schema init
|
||||
- [ ] Node registration on startup
|
||||
- [ ] Heartbeat tick with stats collection and stale sweep
|
||||
- [ ] Self-eviction on 0-rows-affected heartbeat
|
||||
- [ ] PG LISTEN/NOTIFY integration for realtime events
|
||||
- [ ] Admin API endpoint: `GET /api/admin/cluster`
|
||||
- [ ] Admin dashboard extension: cluster node cards
|
||||
- [ ] Health endpoint cluster status
|
||||
- [ ] Configuration via environment variables
|
||||
- [ ] Single-node regression test (no behavioral change)
|
||||
- [ ] Multi-node integration test (Docker Compose, 3 instances)
|
||||
306
docs/DESIGN-native-mtls.md
Normal file
306
docs/DESIGN-native-mtls.md
Normal file
@@ -0,0 +1,306 @@
|
||||
# DESIGN — Native mTLS
|
||||
|
||||
**Version:** v0.6.7
|
||||
**Status:** Implemented
|
||||
**Author:** Jeff / Claude session 2026-03-31
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The existing `MTLSProxyProvider` assumes a TLS-terminating reverse proxy
|
||||
(Traefik, nginx, Istio) sits in front of the Go binary. Identity arrives
|
||||
via injected HTTP headers (`X-SSL-Client-CN`, etc.). The backend never
|
||||
participates in the TLS handshake.
|
||||
|
||||
A target deployment class — systemd units + podman on bare metal or VMs —
|
||||
has no reverse proxy. All connections must be mTLS end-to-end:
|
||||
|
||||
- **Client → server:** browser/CLI presents a user client cert
|
||||
- **Node → node:** cluster peers mutually authenticate for heartbeat,
|
||||
realtime pub/sub, and registry sync
|
||||
|
||||
The Go binary must terminate TLS itself and extract identity directly
|
||||
from the verified peer certificate.
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Certificate revocation (CRL/OCSP). Deferred — short-lived certs +
|
||||
restart is sufficient pre-1.0.
|
||||
- Automatic cert rotation. Operators restart the process after replacing
|
||||
certs on disk.
|
||||
- Acting as a CA at runtime. Cert issuance is an offline operator task.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### TLS Server Mode
|
||||
|
||||
New config knob `TLS_MODE` with three values:
|
||||
|
||||
| Value | Behavior |
|
||||
|----------|----------------------------------------------------|
|
||||
| `none` | Plain HTTP. Current default. Use behind a proxy. |
|
||||
| `server` | Server-side TLS only. No client cert required. |
|
||||
| `mtls` | Mutual TLS. Client cert required and verified. |
|
||||
|
||||
When `TLS_MODE` is `server` or `mtls`, the binary calls
|
||||
`http.Server.ListenAndServeTLS` with paths from `TLS_CERT` and `TLS_KEY`.
|
||||
|
||||
When `TLS_MODE` is `mtls`, `tls.Config` is built with:
|
||||
|
||||
```go
|
||||
&tls.Config{
|
||||
ClientAuth: tls.RequireAndVerifyClientCert,
|
||||
ClientCAs: loadCACertPool(cfg.TLSCA),
|
||||
MinVersion: tls.VersionTLS13,
|
||||
}
|
||||
```
|
||||
|
||||
TLS 1.3 is the floor. No cipher suite configuration exposed — Go's
|
||||
defaults for 1.3 are non-negotiable and correct.
|
||||
|
||||
### Auth Provider: MTLSNativeProvider
|
||||
|
||||
Completely separate from `MTLSProxyProvider`. No shared code paths for
|
||||
cert extraction. They share only downstream helpers.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Shared helpers │
|
||||
│ ParseDN() · FingerprintCert() · ProvisionUser() │
|
||||
└────────────────┬──────────────┬──────────────────┘
|
||||
│ │
|
||||
┌────────────┴───┐ ┌──────┴────────────┐
|
||||
│ MTLSProxy │ │ MTLSNative │
|
||||
│ Provider │ │ Provider │
|
||||
│ │ │ │
|
||||
│ Reads headers: │ │ Reads TLS: │
|
||||
│ X-SSL-Client-* │ │ PeerCertificates │
|
||||
└────────────────┘ └───────────────────┘
|
||||
```
|
||||
|
||||
**MTLSNativeProvider.Authenticate():**
|
||||
|
||||
```go
|
||||
func (p *MTLSNativeProvider) Authenticate(c *gin.Context, s store.Stores) (*Result, error) {
|
||||
if c.Request.TLS == nil || len(c.Request.TLS.PeerCertificates) == 0 {
|
||||
return nil, ErrNoCert
|
||||
}
|
||||
peer := c.Request.TLS.PeerCertificates[0]
|
||||
|
||||
cn := peer.Subject.CommonName // → username
|
||||
dn := peer.Subject.String() // → metadata
|
||||
fp := FingerprintCert(peer) // sha256(DER) → external_id
|
||||
|
||||
return p.resolveOrProvision(c.Request.Context(), s, cn, dn, fp)
|
||||
}
|
||||
```
|
||||
|
||||
`FingerprintCert` computes `sha256(cert.Raw)` hex-encoded. This is the
|
||||
stable identity anchor — certs can be reissued with the same CN but will
|
||||
get a new fingerprint, creating a new user unless the operator explicitly
|
||||
maps them.
|
||||
|
||||
**Why Option B (two providers, not a mode switch):**
|
||||
|
||||
The security properties differ fundamentally. The proxy provider trusts
|
||||
headers — if deployed without a proxy that strips client-injected headers,
|
||||
an attacker can forge identity by sending `X-SSL-Client-CN: admin`. The
|
||||
native provider trusts the TLS stack — identity is cryptographically
|
||||
verified before the HTTP layer runs. Mixing these behind a single
|
||||
code path with a runtime switch invites misconfiguration. Two types,
|
||||
selected at startup by `TLS_MODE`, eliminates the ambiguity.
|
||||
|
||||
### Node-to-Node mTLS
|
||||
|
||||
Each Armature node is both server and client:
|
||||
|
||||
- **Server:** accepts connections from peers and user clients on its
|
||||
listen address.
|
||||
- **Client:** dials peers for cluster registry heartbeat, realtime
|
||||
event forwarding, and (future) distributed task coordination.
|
||||
|
||||
When dialing a peer, the node presents its own cert:
|
||||
|
||||
```go
|
||||
peerTLS := &tls.Config{
|
||||
Certificates: []tls.Certificate{nodeKeypair},
|
||||
RootCAs: caCertPool,
|
||||
MinVersion: tls.VersionTLS13,
|
||||
}
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{TLSClientConfig: peerTLS},
|
||||
}
|
||||
```
|
||||
|
||||
Both sides verify against the same CA. A node whose cert is not signed
|
||||
by the cluster CA cannot join. This replaces any need for shared secrets
|
||||
or bearer tokens in the cluster protocol.
|
||||
|
||||
**Interaction with cluster registry (DESIGN-cluster-registry.md):**
|
||||
|
||||
The existing design uses LISTEN/NOTIFY over PostgreSQL for node
|
||||
discovery and an UNLOGGED `node_registry` table for heartbeat. mTLS
|
||||
does not change the discovery mechanism — it secures the HTTP transport
|
||||
between nodes once they've discovered each other. The heartbeat sweep,
|
||||
self-eviction, and leaderless-by-default properties are orthogonal.
|
||||
|
||||
### Cert Topology
|
||||
|
||||
```
|
||||
cluster-ca.crt / cluster-ca.key ← generated once, lives on operator workstation
|
||||
├── node-1.crt + node-1.key ← SAN: node-1.internal, 10.0.0.1
|
||||
├── node-2.crt + node-2.key ← SAN: node-2.internal, 10.0.0.2
|
||||
├── user-jeff.crt + user-jeff.key ← CN=jeff, client auth only
|
||||
└── user-alice.crt + user-alice.key
|
||||
```
|
||||
|
||||
**Node certs** carry both `ServerAuth` and `ClientAuth` extended key
|
||||
usages. SANs include the node's hostname and IP — required because
|
||||
peers may dial by either.
|
||||
|
||||
**User certs** carry only `ClientAuth`. The CN becomes the username.
|
||||
Email can go in the Subject if desired (parsed by `ParseDN()`).
|
||||
|
||||
The CA private key **never** touches a running node. Nodes receive only:
|
||||
`cluster-ca.crt` (for verifying peers), their own `node-N.crt`, and
|
||||
their own `node-N.key`.
|
||||
|
||||
### Cert Provisioning Tooling
|
||||
|
||||
A shell script (`scripts/switchboard-ca.sh`) wrapping `openssl` is the
|
||||
KISS path. No new binary, no new dependency. Three commands:
|
||||
|
||||
```
|
||||
switchboard-ca init
|
||||
→ generates cluster-ca.crt + cluster-ca.key in ./ca/
|
||||
|
||||
switchboard-ca issue-node --name node-1 --san "node-1.internal,10.0.0.1"
|
||||
→ generates node-1.crt + node-1.key in ./nodes/
|
||||
|
||||
switchboard-ca issue-user --cn jeff [--email jeff@example.com]
|
||||
→ generates jeff.crt + jeff.key in ./users/
|
||||
```
|
||||
|
||||
The script generates a proper `openssl.cnf` fragment per invocation
|
||||
with the right SANs, EKUs, and validity period (default 365 days for
|
||||
nodes, 90 days for users). All output is PEM.
|
||||
|
||||
If we later want a Go-native tool (for cross-platform or embedding),
|
||||
it's a straightforward port — the `crypto/x509` stdlib does everything
|
||||
openssl does here.
|
||||
|
||||
---
|
||||
|
||||
## Config Surface
|
||||
|
||||
```env
|
||||
# TLS mode — controls whether the binary terminates TLS
|
||||
# none: plain HTTP (default, use behind a proxy)
|
||||
# server: TLS, no client cert required
|
||||
# mtls: mutual TLS, client cert required
|
||||
TLS_MODE=mtls
|
||||
|
||||
# Cert paths (required when TLS_MODE != none)
|
||||
TLS_CERT=/etc/armature/tls/node.crt
|
||||
TLS_KEY=/etc/armature/tls/node.key
|
||||
TLS_CA=/etc/armature/tls/ca.crt
|
||||
|
||||
# Listen address (default :8080 for none, :8443 for tls/mtls)
|
||||
LISTEN_ADDR=:8443
|
||||
|
||||
# Auth mode — when TLS_MODE=mtls, this should be mtls
|
||||
AUTH_MODE=mtls
|
||||
```
|
||||
|
||||
`TLS_MODE` and `AUTH_MODE` are independent knobs:
|
||||
|
||||
- `TLS_MODE=server` + `AUTH_MODE=builtin` = HTTPS with password login
|
||||
- `TLS_MODE=mtls` + `AUTH_MODE=mtls` = full mTLS identity
|
||||
- `TLS_MODE=none` + `AUTH_MODE=mtls` = proxy-terminated mTLS (existing behavior)
|
||||
|
||||
The third combination is the current `MTLSProxyProvider` path. The
|
||||
first is useful for deployments that want encrypted transport but
|
||||
password auth (e.g., single-node with Let's Encrypt).
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
**Unit tests (in-process, no network):**
|
||||
|
||||
Generate an ephemeral CA and certs using `crypto/x509` + `crypto/ecdsa`
|
||||
in `TestMain`. Tests call `MTLSNativeProvider.Authenticate()` with a
|
||||
fabricated `*http.Request` whose `TLS.PeerCertificates` is populated
|
||||
directly.
|
||||
|
||||
- Valid cert → user provisioned, correct CN/fingerprint
|
||||
- No TLS → `ErrNoCert`
|
||||
- Empty PeerCertificates → `ErrNoCert`
|
||||
- Second call with same fingerprint → same user returned (idempotent)
|
||||
- Different CN, same fingerprint → same user (cert reissue scenario)
|
||||
|
||||
**Integration tests (real TLS listener):**
|
||||
|
||||
Spin up `http.Server` with `tls.Config` on localhost. Dial with
|
||||
`http.Client` using test client certs.
|
||||
|
||||
- No client cert → TLS handshake rejected (Go returns it before handler)
|
||||
- Wrong CA → TLS handshake rejected
|
||||
- Valid cert → 200, user created
|
||||
- Expired cert → TLS handshake rejected
|
||||
|
||||
**Node-to-node integration:**
|
||||
|
||||
Two in-process servers with distinct node certs, same CA. Verify:
|
||||
|
||||
- Node A can heartbeat to Node B
|
||||
- Node with wrong-CA cert cannot dial either
|
||||
- Connection refused when cert SANs don't match dial address
|
||||
(if `tls.Config.ServerName` is set — TBD whether we enforce this
|
||||
for internal traffic or rely solely on CA verification)
|
||||
|
||||
---
|
||||
|
||||
## Migration Notes
|
||||
|
||||
No database schema changes. The `users` table already has `auth_source`
|
||||
and `external_id` columns from the v0.24.1 mTLS/OIDC work. The native
|
||||
provider populates them identically to the proxy provider.
|
||||
|
||||
The new code is:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `server/auth/mtls_native.go` | `MTLSNativeProvider` type |
|
||||
| `server/auth/mtls_helpers.go` | Shared: `ParseDN`, `FingerprintCert`, `resolveOrProvision` |
|
||||
| `server/auth/mtls_native_test.go` | Unit + integration tests |
|
||||
| `server/config/tls.go` | `TLSConfig` struct, loader, validation |
|
||||
| `scripts/switchboard-ca.sh` | Cert provisioning wrapper |
|
||||
|
||||
`server/auth/mtls.go` (existing) is renamed to `mtls_proxy.go` for
|
||||
clarity. No behavioral changes to the proxy provider.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **SAN enforcement on node-to-node:** Do we verify `ServerName`
|
||||
matches the dialed address, or just verify the cert is CA-signed?
|
||||
Strict SAN checking is more secure but makes IP changes painful.
|
||||
Recommendation: verify CA signature only for internal traffic;
|
||||
SAN checking for client-facing listen.
|
||||
|
||||
2. **Cert rotation without restart:** Not in scope for v0.6.7, but
|
||||
the `tls.Config.GetCertificate` / `GetConfigForClient` callbacks
|
||||
make it straightforward to add later — read cert from disk on each
|
||||
handshake (with caching). Worth noting in the design so we don't
|
||||
paint ourselves into a corner.
|
||||
|
||||
3. **PKCS#12 / combined files:** Some operators prefer `.p12` bundles
|
||||
over separate `.crt`/`.key` files. Low priority but trivial to add
|
||||
via `crypto/pkcs12.Decode`. Defer unless requested.
|
||||
227
docs/DISTRIBUTION.md
Normal file
227
docs/DISTRIBUTION.md
Normal file
@@ -0,0 +1,227 @@
|
||||
# Distribution Guide
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/switchboard-core/switchboard-core:latest
|
||||
docker run -p 8080:80 \
|
||||
-e SWITCHBOARD_ADMIN_USERNAME=admin \
|
||||
-e SWITCHBOARD_ADMIN_PASSWORD=changeme \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
```
|
||||
|
||||
On first run, bundled packages are automatically installed — workflows, surfaces, and extensions are ready to use immediately.
|
||||
|
||||
## Bundled Packages
|
||||
|
||||
The production Docker image ships with pre-built packages. A **curated default set** is auto-installed on first boot; additional packages ship in the image and can be opted-in via `BUNDLED_PACKAGES`.
|
||||
|
||||
#### Default Set (auto-installed)
|
||||
|
||||
| Package | Type | Description |
|
||||
|---------|------|-------------|
|
||||
| notes | surface | Markdown notes with graph view |
|
||||
| chat | surface | Real-time chat surface |
|
||||
| chat-core | library | Conversations, messages, read cursors |
|
||||
| mermaid-renderer | extension | Diagram rendering |
|
||||
| schedules | surface | Cron task management UI |
|
||||
|
||||
#### Opt-in (ship in image, require `BUNDLED_PACKAGES` to enable)
|
||||
|
||||
| Package | Type | Description |
|
||||
|---------|------|-------------|
|
||||
| tasks | full | Kanban/list task manager with webhooks |
|
||||
| workflow-chat | extension | Chat integration for workflow stages |
|
||||
| dashboard | surface | Dashboard surface |
|
||||
| workflow-demo | surface | Interactive walkthrough with diagrams |
|
||||
| bug-report-triage | workflow | Public entry, severity routing, SLA timers |
|
||||
| content-approval | workflow | Multi-party signoff example |
|
||||
| employee-onboarding | workflow | Automated provisioning + manager signoff |
|
||||
| editor | surface | Rich markdown editor |
|
||||
| hello-dashboard | surface | Welcome/getting started surface |
|
||||
| team-activity-log | surface | Team activity feed |
|
||||
| webhook-notifier | workflow | HTTP outbound via connections + Starlark |
|
||||
| csv-table | extension | CSV table renderer |
|
||||
| diff-viewer | extension | Diff visualization |
|
||||
| git-board | surface | Git board interface |
|
||||
| gitea-client | library | Gitea API integration library |
|
||||
| js-sandbox | extension | JavaScript sandbox |
|
||||
| katex-renderer | extension | LaTeX rendering |
|
||||
| regex-tester | extension | Regex testing tool |
|
||||
| icd-test-runner | surface | E2E API test suite |
|
||||
| sdk-test-runner | surface | SDK feature test suite |
|
||||
|
||||
### Behavior
|
||||
|
||||
- **First boot**: Curated default packages are installed and enabled automatically.
|
||||
- **Subsequent boots**: No-op — already-installed packages are skipped.
|
||||
- **Admin uninstalls a package**: It stays uninstalled. Bundled packages are never force-reinstalled.
|
||||
- **To re-install**: Delete the package from the database, then restart the container.
|
||||
|
||||
### Selecting Packages (Allowlist)
|
||||
|
||||
Set `BUNDLED_PACKAGES` to control which packages are installed:
|
||||
|
||||
```bash
|
||||
# Install ALL packages (everything in the image)
|
||||
docker run -p 8080:80 \
|
||||
-e BUNDLED_PACKAGES="*" \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
|
||||
# Install specific packages only
|
||||
docker run -p 8080:80 \
|
||||
-e BUNDLED_PACKAGES="notes,tasks,schedules" \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
```
|
||||
|
||||
Empty (default) installs the curated default set. Use `*` to install all packages. This is useful for Helm charts where different environments need different packages.
|
||||
|
||||
### Disabling Auto-Install
|
||||
|
||||
Set `SKIP_BUNDLED_PACKAGES=true` to prevent bundled packages from being installed:
|
||||
|
||||
```bash
|
||||
docker run -p 8080:80 \
|
||||
-e SKIP_BUNDLED_PACKAGES=true \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
```
|
||||
|
||||
### Custom Bundle Directory
|
||||
|
||||
Override the default bundled packages location with `BUNDLED_PACKAGES_DIR`:
|
||||
|
||||
```bash
|
||||
docker run -p 8080:80 \
|
||||
-e BUNDLED_PACKAGES_DIR=/custom/packages \
|
||||
-v /host/packages:/custom/packages \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
```
|
||||
|
||||
## Builder Image
|
||||
|
||||
The builder image pre-caches Go modules and Node dependencies for faster custom builds.
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/switchboard-core/builder:latest
|
||||
```
|
||||
|
||||
### What It Caches
|
||||
|
||||
- Go module cache (`go mod download` for all server dependencies)
|
||||
- Node modules for the CM6 editor bundle
|
||||
- Vendor library tarballs (marked, DOMPurify, mermaid, KaTeX)
|
||||
- Build tools (zip, Node.js runtime)
|
||||
|
||||
### Using in Custom Builds
|
||||
|
||||
Reference the builder image as a base stage in your Dockerfile:
|
||||
|
||||
```dockerfile
|
||||
FROM ghcr.io/switchboard-core/builder:latest AS builder
|
||||
WORKDIR /app
|
||||
COPY server/ .
|
||||
RUN go build -ldflags="-s -w" -o /bin/switchboard .
|
||||
```
|
||||
|
||||
### Building Locally
|
||||
|
||||
```bash
|
||||
docker build -f Dockerfile.builder -t switchboard-builder .
|
||||
```
|
||||
|
||||
## Custom Build Guide
|
||||
|
||||
### Adding Custom Packages
|
||||
|
||||
1. Create your package in `packages/your-package/` with a `manifest.json`
|
||||
2. Build all packages: `cd packages && bash build.sh all`
|
||||
3. Build the Docker image: `docker build -t my-switchboard .`
|
||||
|
||||
The Dockerfile automatically builds all packages in the `packages/` directory and bundles them into the production image.
|
||||
|
||||
### Removing Bundled Packages
|
||||
|
||||
To exclude specific packages from the bundle, either:
|
||||
- Remove them from `packages/` before building
|
||||
- Set `SKIP_BUNDLED_PACKAGES=true` and install packages manually via the admin UI
|
||||
|
||||
### Forking for Custom Builds
|
||||
|
||||
```bash
|
||||
git clone https://github.com/switchboard-core/switchboard-core.git
|
||||
cd switchboard-core
|
||||
|
||||
# Add/modify packages
|
||||
cp -r my-extension packages/my-extension/
|
||||
|
||||
# Build with builder image for faster compilation
|
||||
docker build -t my-switchboard .
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PORT` | `8080` | Backend API port |
|
||||
| `DB_DRIVER` | (auto) | `postgres` or `sqlite` |
|
||||
| `DATABASE_URL` | | PostgreSQL connection string |
|
||||
| `JWT_SECRET` | `dev-secret-change-me` | **Must change in production** |
|
||||
| `ENCRYPTION_KEY` | | AES-256 key for credential encryption |
|
||||
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, or `oidc` |
|
||||
| `STORAGE_BACKEND` | (auto) | `pvc` or `s3` |
|
||||
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
|
||||
| `BASE_PATH` | | URL prefix (e.g. `/switchboard`) |
|
||||
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable auto-install of bundled packages |
|
||||
| `BUNDLED_PACKAGES` | (empty = defaults) | `""` curated defaults, `"*"` all, or comma-separated IDs |
|
||||
| `BUNDLED_PACKAGES_DIR` | `/app/bundled-packages` | Override bundled packages location |
|
||||
| `LOG_FORMAT` | `text` | `text` or `json` |
|
||||
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
|
||||
|
||||
### Database
|
||||
|
||||
PostgreSQL is recommended for production. SQLite is suitable for single-instance evaluation.
|
||||
|
||||
```bash
|
||||
# PostgreSQL (recommended)
|
||||
docker run -p 8080:80 \
|
||||
-e DATABASE_URL="postgres://user:pass@host:5432/switchboard?sslmode=require" \
|
||||
-e JWT_SECRET="$(openssl rand -hex 32)" \
|
||||
-e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
|
||||
# SQLite (evaluation only)
|
||||
docker run -p 8080:80 \
|
||||
-e DB_DRIVER=sqlite \
|
||||
-v switchboard-data:/data \
|
||||
ghcr.io/switchboard-core/switchboard-core:latest
|
||||
```
|
||||
|
||||
### Storage
|
||||
|
||||
Object storage is required for file uploads and package asset extraction.
|
||||
|
||||
```bash
|
||||
# PVC (auto-detected if path is writable)
|
||||
docker run -v switchboard-storage:/data/storage ...
|
||||
|
||||
# S3-compatible (MinIO, AWS S3, Ceph)
|
||||
docker run \
|
||||
-e STORAGE_BACKEND=s3 \
|
||||
-e S3_BUCKET=switchboard \
|
||||
-e S3_ENDPOINT=https://minio.corp:9000 \
|
||||
-e S3_ACCESS_KEY=... \
|
||||
-e S3_SECRET_KEY=... \
|
||||
-e S3_FORCE_PATH_STYLE=true \
|
||||
...
|
||||
```
|
||||
|
||||
### Kubernetes
|
||||
|
||||
See the `k8s/` directory for example manifests. Key considerations:
|
||||
|
||||
- Use `POSTGRES_HOST`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` env vars (assembled into DSN automatically)
|
||||
- Set liveness probe to `/healthz/live`, readiness probe to `/healthz/ready`
|
||||
- Mount a PVC at `/data/storage` for file storage, or configure S3
|
||||
- Set `JWT_SECRET` and `ENCRYPTION_KEY` via Kubernetes Secrets
|
||||
183
docs/EXTENSION-GUIDE.md
Normal file
183
docs/EXTENSION-GUIDE.md
Normal file
@@ -0,0 +1,183 @@
|
||||
# Extension Guide
|
||||
|
||||
## Package Types
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `surface` | A routable UI page rendered in the shell viewport |
|
||||
| `extension` | Starlark hooks, tools, API routes, DB tables |
|
||||
| `full` | Both surface and extension combined |
|
||||
| `library` | Shared code imported by other packages via `lib.require()` |
|
||||
| `workflow` | Bundled workflow definition |
|
||||
|
||||
## Tiers
|
||||
|
||||
| Tier | Runs | Capabilities |
|
||||
|------|------|-------------|
|
||||
| `browser` | Client-side JS only | DOM access, SDK hooks, no server-side logic |
|
||||
| `starlark` | Sandboxed server-side | DB, HTTP, notifications, secrets, API routes, realtime |
|
||||
| `sidecar` | Separate container | Full runtime (future) |
|
||||
|
||||
## manifest.json
|
||||
|
||||
Every package has a `manifest.json` at its root. Example for a surface:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "my-surface",
|
||||
"title": "My Surface",
|
||||
"type": "full",
|
||||
"tier": "starlark",
|
||||
"version": "1.0.0",
|
||||
"description": "A custom surface with server-side logic.",
|
||||
"icon": "🔧",
|
||||
"author": "you",
|
||||
"route": "/s/my-surface",
|
||||
"auth": "authenticated",
|
||||
"permissions": ["db.write"],
|
||||
"api_routes": [
|
||||
{"method": "GET", "path": "/items"},
|
||||
{"method": "POST", "path": "/items"}
|
||||
],
|
||||
"db_tables": {
|
||||
"items": {
|
||||
"columns": {
|
||||
"title": "text",
|
||||
"done": "int"
|
||||
},
|
||||
"indexes": [["title"]]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"page_size": {
|
||||
"type": "string",
|
||||
"label": "Items Per Page",
|
||||
"description": "Number of items shown per page",
|
||||
"default": "25"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Field Reference
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `id` | yes | Unique kebab-case identifier |
|
||||
| `title` | yes | Display name |
|
||||
| `type` | yes | `surface`, `extension`, `full`, `library`, `workflow` |
|
||||
| `tier` | yes | `browser`, `starlark`, `sidecar` |
|
||||
| `version` | yes | Semver string |
|
||||
| `description` | no | Short description |
|
||||
| `icon` | no | Emoji icon for sidebar/menu |
|
||||
| `route` | surfaces | URL path (e.g., `/s/my-surface`) |
|
||||
| `auth` | no | `authenticated` (default) or `public` |
|
||||
| `permissions` | no | Capabilities requested: `db.write`, `http`, `notifications`, `secrets`, `realtime.publish` |
|
||||
| `api_routes` | no | Array of `{method, path}` for extension HTTP endpoints |
|
||||
| `api_schema` | no | OpenAPI documentation for extension API routes (see below) |
|
||||
| `db_tables` | no | Table definitions (see below) |
|
||||
| `settings` | no | User-configurable settings schema |
|
||||
| `exports` | libraries | Functions exported for other packages |
|
||||
| `hooks` | no | Event bus subscriptions |
|
||||
| `schema_version` | no | Integer for additive schema migrations |
|
||||
|
||||
## db_tables Schema
|
||||
|
||||
Tables are automatically namespaced as `ext_{package_id}_{table_name}`. Column types: `text`, `int`. Every table gets an auto-generated `id` primary key and `created_at` timestamp.
|
||||
|
||||
```json
|
||||
"db_tables": {
|
||||
"notes": {
|
||||
"columns": {
|
||||
"title": "text",
|
||||
"body": "text",
|
||||
"creator_id": "text",
|
||||
"pinned": "int"
|
||||
},
|
||||
"indexes": [
|
||||
["creator_id"],
|
||||
["pinned"]
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## api_schema (OpenAPI Documentation)
|
||||
|
||||
Extensions can optionally declare an `api_schema` array in their manifest to provide rich API documentation. Declared routes appear in Swagger UI at `/api/docs` with parameters, request bodies, and response schemas. Routes without `api_schema` entries still appear as auto-generated stubs.
|
||||
|
||||
```json
|
||||
"api_schema": [
|
||||
{
|
||||
"path": "/items",
|
||||
"method": "GET",
|
||||
"summary": "List items",
|
||||
"description": "Returns paginated items for the current user",
|
||||
"params": {
|
||||
"limit": {"type": "integer", "default": 50, "description": "Max results"},
|
||||
"offset": {"type": "integer", "default": 0}
|
||||
},
|
||||
"response": {
|
||||
"type": "object",
|
||||
"example": {"data": [{"id": "string", "title": "string"}]}
|
||||
}
|
||||
},
|
||||
{
|
||||
"path": "/items",
|
||||
"method": "POST",
|
||||
"summary": "Create item",
|
||||
"body": {
|
||||
"title": {"type": "string", "required": true},
|
||||
"description": {"type": "string"}
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Only `path` and `method` are required. All other fields are optional. Malformed entries are logged and skipped without blocking extension loading.
|
||||
|
||||
## Starlark Sandbox API
|
||||
|
||||
Starlark scripts run server-side with a CPU budget and memory ceiling. Available modules (granted per-permission by admin):
|
||||
|
||||
| Module | Permission | API |
|
||||
|--------|-----------|-----|
|
||||
| `db` | `db.write` | `db.query(table, filters)`, `db.insert(table, row)`, `db.update(table, id, row)`, `db.delete(table, id)` |
|
||||
| `http` | `http` | `http.get(url)`, `http.post(url, body)` -- SSRF-safe, no private IPs by default |
|
||||
| `notifications` | `notifications` | `notifications.send(user_id, title, body)` |
|
||||
| `secrets` | `secrets` | `secrets.get(connection_type)` -- reads from the credential vault |
|
||||
| `api` | (implicit) | Registers HTTP routes at `/s/:slug/api/*path` |
|
||||
| `realtime` | `realtime.publish` | `realtime.publish(channel, event, data)` -- push to WebSocket clients |
|
||||
|
||||
The sandbox cannot spawn goroutines, access the filesystem, or import arbitrary packages.
|
||||
|
||||
## Permissions Model
|
||||
|
||||
Extensions declare required permissions in `manifest.json`. The admin must grant each permission before the extension can use the corresponding module. Permission status is visible in Admin > Packages > Permissions.
|
||||
|
||||
Kernel permissions for users/groups: `extension.use`, `extension.install`, `workflow.create`, `workflow.submit`, `admin.view`, `token.unlimited`.
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
my-package/
|
||||
manifest.json # required
|
||||
js/ # browser-side JavaScript
|
||||
index.js
|
||||
css/ # stylesheets
|
||||
styles.css
|
||||
script.star # Starlark entry point
|
||||
star/ # additional Starlark modules
|
||||
assets/ # static assets
|
||||
migrations/ # schema migration files
|
||||
```
|
||||
|
||||
## Testing Extensions
|
||||
|
||||
1. Build the package: `cd packages && bash build.sh my-package`
|
||||
2. Upload the `.pkg` file via Admin > Packages > Install.
|
||||
3. Grant permissions in Admin > Packages > Permissions.
|
||||
4. Enable the package.
|
||||
5. If it is a surface, navigate to its route (e.g., `/s/my-package`).
|
||||
|
||||
Extension API routes are accessible at `/s/{slug}/api/{path}` and require an authenticated Bearer token.
|
||||
91
docs/GETTING-STARTED.md
Normal file
91
docs/GETTING-STARTED.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# Getting Started
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker and Docker Compose
|
||||
|
||||
## Running with Docker Compose
|
||||
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Open [http://localhost:3000](http://localhost:3000). Default credentials: `admin` / `admin`.
|
||||
|
||||
Data persists in the `sb_data` named volume. To reset everything:
|
||||
|
||||
```bash
|
||||
docker compose down -v
|
||||
```
|
||||
|
||||
## First Boot
|
||||
|
||||
On first start, Switchboard Core will:
|
||||
|
||||
1. Run database migrations (SQLite by default in compose).
|
||||
2. Create the admin user from `SWITCHBOARD_ADMIN_USERNAME` / `SWITCHBOARD_ADMIN_PASSWORD` env vars.
|
||||
3. Auto-install the curated default package set (notes, chat-core, dashboard, workflow demos, etc.).
|
||||
|
||||
No manual setup steps are required.
|
||||
|
||||
## Installing Additional Packages
|
||||
|
||||
The Docker image ships with extra packages beyond the default set. To install them:
|
||||
|
||||
- **Via Admin UI**: Go to `/admin` > Packages. Browse, enable, or install packages.
|
||||
- **Via environment variable**: Set `BUNDLED_PACKAGES` before starting:
|
||||
|
||||
```bash
|
||||
# Install all bundled packages
|
||||
BUNDLED_PACKAGES="*" docker compose up --build
|
||||
|
||||
# Install specific extras
|
||||
BUNDLED_PACKAGES="notes,tasks,schedules" docker compose up --build
|
||||
```
|
||||
|
||||
You can also upload `.pkg` archives through the Admin > Packages page.
|
||||
|
||||
## Key URLs
|
||||
|
||||
| URL | Purpose |
|
||||
|-----|---------|
|
||||
| `/` | Redirects to your default surface |
|
||||
| `/admin` | Admin panel (users, packages, settings, teams) |
|
||||
| `/settings` | User settings (profile, preferences, default surface) |
|
||||
| `/welcome` | Welcome page (shown when no surfaces are installed) |
|
||||
| `/api/docs` | Interactive OpenAPI documentation |
|
||||
|
||||
## Key Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PORT` | `8080` | Backend API port |
|
||||
| `DB_DRIVER` | auto | `postgres` or `sqlite` |
|
||||
| `DATABASE_URL` | | PostgreSQL DSN or SQLite file path |
|
||||
| `JWT_SECRET` | `dev-secret-change-me` | **Change in production** |
|
||||
| `ENCRYPTION_KEY` | | AES-256 key for credential encryption |
|
||||
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, or `oidc` |
|
||||
| `STORAGE_BACKEND` | auto | `pvc` or `s3` |
|
||||
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
|
||||
| `SWITCHBOARD_ADMIN_USERNAME` | | Bootstrap admin username |
|
||||
| `SWITCHBOARD_ADMIN_PASSWORD` | | Bootstrap admin password |
|
||||
| `BUNDLED_PACKAGES` | (empty) | `""` = curated defaults, `"*"` = all, or comma-separated IDs |
|
||||
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable auto-install entirely |
|
||||
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
|
||||
| `LOG_FORMAT` | `text` | `text` or `json` |
|
||||
|
||||
## From Source
|
||||
|
||||
```bash
|
||||
git clone <repo-url> && cd switchboard-core
|
||||
cp server/.env.example server/.env # edit DB credentials
|
||||
cd server && go run .
|
||||
# Backend on http://localhost:8080
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Extension Guide](EXTENSION-GUIDE.md) -- Author your own packages
|
||||
- [API Reference](API-REFERENCE.md) -- REST API overview
|
||||
- [Deployment](DEPLOYMENT.md) -- Production deployment
|
||||
- [Architecture](ARCHITECTURE.md) -- System design
|
||||
130
docs/PACKAGE-FORMAT.md
Normal file
130
docs/PACKAGE-FORMAT.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# Package Format
|
||||
|
||||
Switchboard packages are distributed as `.pkg` files -- ZIP archives with a standard internal structure.
|
||||
|
||||
## ZIP Structure
|
||||
|
||||
```
|
||||
my-package.pkg
|
||||
├── manifest.json # required -- package metadata
|
||||
├── js/ # browser-side JavaScript
|
||||
│ └── index.js
|
||||
├── css/ # stylesheets
|
||||
│ └── styles.css
|
||||
├── script.star # Starlark entry point
|
||||
├── star/ # additional Starlark modules
|
||||
├── assets/ # static assets (images, etc.)
|
||||
└── migrations/ # schema migration files
|
||||
```
|
||||
|
||||
Only `manifest.json` is required. All other directories are optional and included only if present in the source.
|
||||
|
||||
## manifest.json Reference
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "my-package",
|
||||
"title": "My Package",
|
||||
"type": "surface",
|
||||
"tier": "browser",
|
||||
"version": "1.0.0",
|
||||
"description": "What this package does.",
|
||||
"icon": "📦",
|
||||
"author": "your-name",
|
||||
"route": "/s/my-package",
|
||||
"auth": "authenticated",
|
||||
"permissions": [],
|
||||
"api_routes": [],
|
||||
"db_tables": {},
|
||||
"settings": {},
|
||||
"hooks": {},
|
||||
"exports": [],
|
||||
"schema_version": 1
|
||||
}
|
||||
```
|
||||
|
||||
### Required Fields
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `id` | Unique kebab-case identifier |
|
||||
| `title` | Human-readable display name |
|
||||
| `type` | `surface`, `extension`, `full`, `library`, `workflow` |
|
||||
| `tier` | `browser`, `starlark`, `sidecar` |
|
||||
| `version` | Semver version string |
|
||||
|
||||
### Optional Fields
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `description` | Short description |
|
||||
| `icon` | Emoji for sidebar/menu display |
|
||||
| `author` | Package author |
|
||||
| `route` | URL path for surfaces (e.g., `/s/my-package`) |
|
||||
| `auth` | `authenticated` or `public` |
|
||||
| `layout` | Surface layout mode (e.g., `single`) |
|
||||
| `permissions` | Array of required capabilities (`db.write`, `http`, `notifications`, `secrets`, `realtime.publish`) |
|
||||
| `api_routes` | Array of `{"method": "GET", "path": "/items"}` |
|
||||
| `db_tables` | Table definitions with columns and indexes |
|
||||
| `settings` | User-configurable settings with type, label, description, default |
|
||||
| `hooks` | Event bus subscription patterns |
|
||||
| `exports` | Functions exported by library packages |
|
||||
| `schema_version` | Integer for additive schema migrations |
|
||||
|
||||
## Package Lifecycle
|
||||
|
||||
1. **Install**: Upload a `.pkg` file via Admin > Packages or `POST /api/v1/admin/packages/install`. The kernel extracts the archive, creates database tables, and registers routes.
|
||||
|
||||
2. **Enable**: Activate the package so its routes, hooks, and UI become available. `PUT /api/v1/admin/packages/:id/enable`.
|
||||
|
||||
3. **Disable**: Deactivate without removing data. `PUT /api/v1/admin/packages/:id/disable`.
|
||||
|
||||
4. **Update**: Upload a new `.pkg` with a higher semver version. Schema changes must be additive (new columns/tables only). `POST /api/v1/admin/packages/:id/update`.
|
||||
|
||||
5. **Export**: Download the installed package as a `.pkg` archive. `GET /api/v1/admin/packages/:id/export`.
|
||||
|
||||
6. **Delete**: Remove the package and its data. `DELETE /api/v1/admin/packages/:id`.
|
||||
|
||||
## Bundled vs User-Installed
|
||||
|
||||
**Bundled packages** ship inside the Docker image at `/app/bundled-packages`. On first boot, a curated default set is auto-installed. Behavior:
|
||||
|
||||
- First boot: curated defaults are installed and enabled.
|
||||
- Subsequent boots: already-installed packages are skipped.
|
||||
- Admin uninstalls: the package stays uninstalled (never force-reinstalled).
|
||||
- Control via `BUNDLED_PACKAGES` env var: empty = defaults, `"*"` = all, or comma-separated IDs.
|
||||
|
||||
**User-installed packages** are uploaded through the Admin UI or API. They follow the same lifecycle but are not tied to the Docker image.
|
||||
|
||||
## Building Packages
|
||||
|
||||
Packages are built by zipping the standard directories alongside `manifest.json`:
|
||||
|
||||
```bash
|
||||
# Build all packages in the packages/ directory
|
||||
cd packages && bash build.sh
|
||||
|
||||
# Build a single package
|
||||
cd packages && bash build.sh my-package
|
||||
```
|
||||
|
||||
Output goes to `dist/my-package.pkg`.
|
||||
|
||||
### Manual Build
|
||||
|
||||
```bash
|
||||
cd packages/my-package
|
||||
zip -r ../../dist/my-package.pkg manifest.json js/ css/ script.star
|
||||
```
|
||||
|
||||
The build script automatically includes whichever standard directories exist: `js/`, `css/`, `assets/`, `script.star`, `star/`, `migrations/`.
|
||||
|
||||
### Custom Docker Image
|
||||
|
||||
To bundle custom packages into a Docker image:
|
||||
|
||||
1. Place your package in `packages/your-package/` with a `manifest.json`.
|
||||
2. Run `cd packages && bash build.sh all`.
|
||||
3. Build the image: `docker build -t my-switchboard .`
|
||||
|
||||
The Dockerfile builds all packages and copies them into the bundled packages directory.
|
||||
94
docs/PACKAGE-REGISTRY.md
Normal file
94
docs/PACKAGE-REGISTRY.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Package Registry
|
||||
|
||||
The package registry lets admins browse and install packages from an external
|
||||
JSON index — a lightweight alternative to manually uploading `.pkg` files.
|
||||
|
||||
## Registry JSON Format
|
||||
|
||||
The registry is a static JSON file matching the `RegistryResponse` struct:
|
||||
|
||||
```json
|
||||
{
|
||||
"packages": [
|
||||
{
|
||||
"id": "notes",
|
||||
"title": "Notes",
|
||||
"version": "0.8.0",
|
||||
"description": "Markdown notes with backlinks and graph view",
|
||||
"author": "switchboard",
|
||||
"type": "extension",
|
||||
"tier": "core",
|
||||
"download_url": "https://cdn.example.com/pkg/notes.pkg",
|
||||
"size": 48200,
|
||||
"updated_at": "2026-03-20T00:00:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Required Fields
|
||||
|
||||
| Field | Description |
|
||||
|----------------|------------------------------------------------|
|
||||
| `id` | Package identifier (matches `manifest.json`) |
|
||||
| `title` | Display name |
|
||||
| `version` | Semver version string |
|
||||
| `description` | Short description |
|
||||
| `download_url` | HTTPS URL to the `.pkg` file (must be HTTPS) |
|
||||
|
||||
### Optional Fields
|
||||
|
||||
| Field | Description |
|
||||
|--------------|------------------------------------------|
|
||||
| `author` | Package author |
|
||||
| `type` | `extension` or `library` |
|
||||
| `tier` | `core`, `official`, or `community` |
|
||||
| `size` | File size in bytes |
|
||||
| `updated_at` | ISO 8601 timestamp of last update |
|
||||
|
||||
## Configuring the Registry URL
|
||||
|
||||
### Via Admin UI
|
||||
|
||||
Navigate to **Admin > Settings > Package Registry** and enter the registry URL.
|
||||
|
||||
### Via API
|
||||
|
||||
```bash
|
||||
curl -X PUT /api/v1/admin/settings/package_registry \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"value": {"url": "https://cdn.example.com/pkg/registry.json"}}'
|
||||
```
|
||||
|
||||
## Generating a Registry
|
||||
|
||||
Use `scripts/generate-registry.sh` to build a `registry.json` from a
|
||||
directory of `.pkg` files:
|
||||
|
||||
```bash
|
||||
# Default: reads dist/, uses placeholder base URL
|
||||
./scripts/generate-registry.sh
|
||||
|
||||
# Custom directory and base URL
|
||||
./scripts/generate-registry.sh ./my-packages https://cdn.example.com/pkg > registry.json
|
||||
```
|
||||
|
||||
Requirements: `jq`, `unzip`, and `stat` (GNU or BSD).
|
||||
|
||||
## Self-Hosting
|
||||
|
||||
Any HTTPS-capable file server works. Upload your `.pkg` files and the
|
||||
generated `registry.json` to the same directory, then point the admin
|
||||
setting to the `registry.json` URL.
|
||||
|
||||
Example with a static file server:
|
||||
|
||||
```
|
||||
/var/www/packages/
|
||||
registry.json
|
||||
notes.pkg
|
||||
chat.pkg
|
||||
chat-core.pkg
|
||||
```
|
||||
|
||||
The registry is fetched and cached for 5 minutes on each browse request.
|
||||
119
docs/TUTORIAL-FIRST-EXTENSION.md
Normal file
119
docs/TUTORIAL-FIRST-EXTENSION.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Build Your First Browser Extension
|
||||
|
||||
This tutorial walks through building a browser extension that renders custom
|
||||
code blocks, modeled on the CSV Table Viewer that ships with Switchboard.
|
||||
|
||||
**Prerequisites:** A running Switchboard instance, a text editor, and `zip`.
|
||||
|
||||
## Step 1: Create the Directory
|
||||
|
||||
```sh
|
||||
mkdir -p my-extension/js
|
||||
```
|
||||
|
||||
## Step 2: Write the Manifest
|
||||
|
||||
Create `my-extension/manifest.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "my-extension",
|
||||
"title": "My Extension",
|
||||
"version": "0.1.0",
|
||||
"type": "extension",
|
||||
"tier": "browser",
|
||||
"author": "you",
|
||||
"description": "Renders demo code blocks as styled HTML",
|
||||
"permissions": [],
|
||||
"settings": {}
|
||||
}
|
||||
```
|
||||
|
||||
- **id** -- unique identifier, used as the install key
|
||||
- **type** -- `extension` for browser-only packages; `full` or `surface` for
|
||||
packages with backend routes
|
||||
- **tier** -- `browser` for client-side JS; `starlark` for server-side scripting
|
||||
|
||||
## Step 3: Write the Browser Script
|
||||
|
||||
Create `my-extension/js/script.js`. Browser extensions use the IIFE pattern
|
||||
and register with the SDK through `sw.renderers`:
|
||||
|
||||
```js
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
function register() {
|
||||
if (!window.sw?.renderers) return;
|
||||
|
||||
sw.renderers.register('demo-block', {
|
||||
type: 'block',
|
||||
priority: 10,
|
||||
match(lang) {
|
||||
return (lang || '').toLowerCase() === 'demo';
|
||||
},
|
||||
render(lang, code, container) {
|
||||
container.innerHTML =
|
||||
'<div style="padding:12px;background:var(--bg-2);' +
|
||||
'border:1px solid var(--border);border-radius:8px">' +
|
||||
'<strong>Demo:</strong> ' + code +
|
||||
'</div>';
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
if (window.sw?._sdk) {
|
||||
register();
|
||||
} else {
|
||||
document.addEventListener('sw:ready', register, { once: true });
|
||||
}
|
||||
})();
|
||||
```
|
||||
|
||||
The IIFE wrapper keeps variables out of global scope. `sw.renderers.register`
|
||||
takes a name and an options object: `type: 'block'` targets fenced code blocks,
|
||||
`match` checks the language tag, and `render` receives the language, raw code,
|
||||
and a container element. The `sw:ready` event fires once the SDK initializes;
|
||||
if already loaded, register immediately. Use CSS variables like `var(--bg-2)`
|
||||
and `var(--border)` to follow the active theme.
|
||||
|
||||
## Step 4: Package It
|
||||
|
||||
```sh
|
||||
cd my-extension
|
||||
zip -r ../my-extension.pkg manifest.json js/
|
||||
```
|
||||
|
||||
The `.pkg` format is a ZIP with `manifest.json` at the root. Optional
|
||||
directories: `js/`, `css/`, `assets/`. If working inside `packages/`, use
|
||||
the build script instead: `bash build.sh my-extension`.
|
||||
|
||||
## Step 5: Install It
|
||||
|
||||
```sh
|
||||
curl -X POST http://localhost:3000/api/v1/admin/packages/install \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-F "file=@my-extension.pkg"
|
||||
```
|
||||
|
||||
You can also install through the Admin UI under the Packages section.
|
||||
|
||||
## Step 6: Enable and Test
|
||||
|
||||
1. Open the admin panel, navigate to Packages, confirm "My Extension" is enabled.
|
||||
2. Go to any markdown surface (Chat, Notes, etc.).
|
||||
3. Enter a fenced code block with the `demo` language tag.
|
||||
|
||||
You should see styled output instead of a plain code block.
|
||||
|
||||
## Going Further
|
||||
|
||||
- **Add CSS** -- create a `css/` directory; stylesheets are injected automatically.
|
||||
- **Post-renderers** -- register with `type: 'post'` to run after block renderers
|
||||
finish (the Mermaid extension uses this for async SVG rendering).
|
||||
- **Settings** -- declare a `settings` object in the manifest for admin-configurable values.
|
||||
- **Server-side logic** -- set `tier: "starlark"` and add `script.star` for
|
||||
backend API routes and database tables.
|
||||
|
||||
See `PACKAGE-FORMAT.md` for the full manifest spec and `EXTENSION-GUIDE.md`
|
||||
for advanced patterns.
|
||||
@@ -155,6 +155,12 @@ spec:
|
||||
optional: true
|
||||
- name: S3_FORCE_PATH_STYLE
|
||||
value: "true"
|
||||
# Bundled packages (v0.3.8)
|
||||
# Dev: install all (empty = all). Test: install all. Prod: skip (nothing bundled needed yet).
|
||||
- name: SKIP_BUNDLED_PACKAGES
|
||||
value: "${SKIP_BUNDLED_PACKAGES}"
|
||||
- name: BUNDLED_PACKAGES
|
||||
value: "${BUNDLED_PACKAGES}"
|
||||
volumeMounts:
|
||||
- name: storage
|
||||
mountPath: /data/storage
|
||||
|
||||
@@ -66,6 +66,7 @@ server {
|
||||
location ${BASE_PATH}/team-admin { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
|
||||
location ${BASE_PATH}/settings { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
|
||||
location ${BASE_PATH}/welcome { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
|
||||
location ${BASE_PATH}/docs { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
|
||||
location ${BASE_PATH}/w/ { proxy_pass $backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; }
|
||||
|
||||
# Extension surface page routes → backend
|
||||
|
||||
81
packages/chat-core/manifest.json
Normal file
81
packages/chat-core/manifest.json
Normal file
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"id": "chat-core",
|
||||
"title": "Chat Core",
|
||||
"type": "library",
|
||||
"tier": "starlark",
|
||||
"version": "0.2.0",
|
||||
"description": "Core chat library — conversations, messages, participants, read cursors. Usable by other packages via lib.require().",
|
||||
"author": "switchboard",
|
||||
|
||||
"permissions": ["db.write", "realtime.publish"],
|
||||
|
||||
"exports": [
|
||||
"create", "send", "history",
|
||||
"add_participant", "remove_participant", "mark_read"
|
||||
],
|
||||
|
||||
"api_routes": [
|
||||
{"method": "GET", "path": "/conversations"},
|
||||
{"method": "POST", "path": "/conversations"},
|
||||
{"method": "GET", "path": "/conversations/*"},
|
||||
{"method": "PUT", "path": "/conversations/*"},
|
||||
{"method": "DELETE", "path": "/conversations/*"},
|
||||
{"method": "GET", "path": "/messages/*"},
|
||||
{"method": "POST", "path": "/messages/*"},
|
||||
{"method": "PUT", "path": "/messages/*"},
|
||||
{"method": "DELETE", "path": "/messages/*"},
|
||||
{"method": "GET", "path": "/participants/*"},
|
||||
{"method": "POST", "path": "/participants/*"},
|
||||
{"method": "DELETE", "path": "/participants/*"},
|
||||
{"method": "POST", "path": "/read/*"},
|
||||
{"method": "GET", "path": "/search"},
|
||||
{"method": "GET", "path": "/unread"}
|
||||
],
|
||||
|
||||
"db_tables": {
|
||||
"conversations": {
|
||||
"columns": {
|
||||
"title": "text",
|
||||
"type": "text",
|
||||
"created_by": "text",
|
||||
"updated_at": "text"
|
||||
},
|
||||
"indexes": [["created_by"], ["updated_at"]]
|
||||
},
|
||||
"participants": {
|
||||
"columns": {
|
||||
"conversation_id": "text",
|
||||
"participant_id": "text",
|
||||
"participant_type": "text",
|
||||
"display_name": "text",
|
||||
"role": "text",
|
||||
"joined_at": "text"
|
||||
},
|
||||
"indexes": [
|
||||
["conversation_id"],
|
||||
["participant_id"],
|
||||
["conversation_id", "participant_id"]
|
||||
]
|
||||
},
|
||||
"messages": {
|
||||
"columns": {
|
||||
"conversation_id": "text",
|
||||
"participant_id": "text",
|
||||
"content": "text",
|
||||
"content_type": "text",
|
||||
"edited_at": "text"
|
||||
},
|
||||
"indexes": [["conversation_id"]]
|
||||
},
|
||||
"read_cursors": {
|
||||
"columns": {
|
||||
"conversation_id": "text",
|
||||
"participant_id": "text",
|
||||
"last_read_message_id": "text"
|
||||
},
|
||||
"indexes": [["conversation_id", "participant_id"]]
|
||||
}
|
||||
},
|
||||
|
||||
"schema_version": 1
|
||||
}
|
||||
750
packages/chat-core/script.star
Normal file
750
packages/chat-core/script.star
Normal file
@@ -0,0 +1,750 @@
|
||||
# Chat Core — Starlark Backend (v0.2.0)
|
||||
#
|
||||
# Library package providing conversations, messages, participants,
|
||||
# and read cursors. Consumable via lib.require("chat-core").
|
||||
#
|
||||
# Entry points:
|
||||
# on_request(req) → REST API routes
|
||||
# Exported globals → create, send, history, add_participant,
|
||||
# remove_participant, mark_read
|
||||
#
|
||||
# Modules: db, json, realtime
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Helpers
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _resp(status, data):
|
||||
return {"status": status, "body": json.encode(data), "headers": {"Content-Type": "application/json"}}
|
||||
|
||||
def _str(v):
|
||||
if v == None:
|
||||
return ""
|
||||
return str(v)
|
||||
|
||||
def _int(v):
|
||||
if v == None:
|
||||
return 0
|
||||
s = str(v)
|
||||
if not s:
|
||||
return 0
|
||||
return int(s)
|
||||
|
||||
def _is_participant(conversation_id, user_id):
|
||||
"""Check if user is a participant in the conversation."""
|
||||
rows = db.query("participants", filters={"conversation_id": conversation_id, "participant_id": user_id}, limit=1)
|
||||
return len(rows or []) > 0
|
||||
|
||||
def _get_participant(conversation_id, user_id):
|
||||
"""Get participant record or None."""
|
||||
rows = db.query("participants", filters={"conversation_id": conversation_id, "participant_id": user_id}, limit=1)
|
||||
if rows and len(rows) > 0:
|
||||
return rows[0]
|
||||
return None
|
||||
|
||||
def _is_admin(conversation_id, user_id):
|
||||
"""Check if user has admin role in the conversation."""
|
||||
p = _get_participant(conversation_id, user_id)
|
||||
if p:
|
||||
return _str(p.get("role", "")) == "admin"
|
||||
return False
|
||||
|
||||
def _get_conversation(conversation_id):
|
||||
"""Get conversation by ID or None."""
|
||||
rows = db.query("conversations", filters={"id": conversation_id}, limit=1)
|
||||
if rows and len(rows) > 0:
|
||||
return rows[0]
|
||||
return None
|
||||
|
||||
def _now():
|
||||
"""Current timestamp placeholder — db auto-populates created_at."""
|
||||
return ""
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Exported API (lib.require("chat-core"))
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def create(title, type="group", participants=None, creator_id="", creator_display_name=""):
|
||||
"""Create a conversation and add initial participants.
|
||||
|
||||
Args:
|
||||
title: conversation title
|
||||
type: "direct" or "group" (default "group")
|
||||
participants: list of dicts with {id, type?, display_name?, role?}
|
||||
creator_id: user ID of the creator (added as admin)
|
||||
creator_display_name: display name of the creator
|
||||
|
||||
Returns:
|
||||
dict with conversation data
|
||||
"""
|
||||
if type not in ("direct", "group"):
|
||||
type = "group"
|
||||
|
||||
conv = db.insert("conversations", {
|
||||
"title": _str(title),
|
||||
"type": type,
|
||||
"created_by": _str(creator_id),
|
||||
"updated_at": "",
|
||||
})
|
||||
cid = conv["id"]
|
||||
|
||||
# Add creator as admin participant
|
||||
if creator_id:
|
||||
db.insert("participants", {
|
||||
"conversation_id": cid,
|
||||
"participant_id": _str(creator_id),
|
||||
"participant_type": "user",
|
||||
"display_name": _str(creator_display_name),
|
||||
"role": "admin",
|
||||
"joined_at": "",
|
||||
})
|
||||
|
||||
# Add initial participants
|
||||
for p in (participants or []):
|
||||
pid = _str(p.get("id", ""))
|
||||
if not pid or pid == _str(creator_id):
|
||||
continue
|
||||
db.insert("participants", {
|
||||
"conversation_id": cid,
|
||||
"participant_id": pid,
|
||||
"participant_type": _str(p.get("type", "user")),
|
||||
"display_name": _str(p.get("display_name", "")),
|
||||
"role": _str(p.get("role", "member")),
|
||||
"joined_at": "",
|
||||
})
|
||||
|
||||
return conv
|
||||
|
||||
|
||||
def send(conversation_id, participant_id, content, content_type="text"):
|
||||
"""Send a message to a conversation.
|
||||
|
||||
Args:
|
||||
conversation_id: target conversation
|
||||
participant_id: sender ID
|
||||
content: message content
|
||||
content_type: "text", "system", or "file" (default "text")
|
||||
|
||||
Returns:
|
||||
dict with message data
|
||||
"""
|
||||
if content_type not in ("text", "system", "file"):
|
||||
content_type = "text"
|
||||
|
||||
msg = db.insert("messages", {
|
||||
"conversation_id": _str(conversation_id),
|
||||
"participant_id": _str(participant_id),
|
||||
"content": _str(content),
|
||||
"content_type": content_type,
|
||||
"edited_at": "",
|
||||
})
|
||||
|
||||
# Update conversation timestamp
|
||||
db.update("conversations", _str(conversation_id), {"updated_at": msg.get("created_at", "")})
|
||||
|
||||
# Publish realtime event
|
||||
realtime.publish(
|
||||
"conversation:" + _str(conversation_id),
|
||||
"message",
|
||||
{
|
||||
"id": msg.get("id", ""),
|
||||
"conversation_id": _str(conversation_id),
|
||||
"participant_id": _str(participant_id),
|
||||
"content": _str(content),
|
||||
"content_type": content_type,
|
||||
"created_at": msg.get("created_at", ""),
|
||||
},
|
||||
)
|
||||
|
||||
return msg
|
||||
|
||||
|
||||
def history(conversation_id, limit=50, cursor=""):
|
||||
"""Get paginated message history for a conversation.
|
||||
|
||||
Args:
|
||||
conversation_id: target conversation
|
||||
limit: max messages to return (1-100, default 50)
|
||||
cursor: created_at value of last message from previous page
|
||||
|
||||
Returns:
|
||||
dict with {messages, has_more, next_cursor}
|
||||
"""
|
||||
lim = _int(limit)
|
||||
if lim < 1 or lim > 100:
|
||||
lim = 50
|
||||
|
||||
filters = {"conversation_id": _str(conversation_id)}
|
||||
before = {}
|
||||
if cursor:
|
||||
before = {"created_at": _str(cursor)}
|
||||
|
||||
rows = db.query("messages", filters=filters, order="-created_at", limit=lim + 1, before=before)
|
||||
rows = rows or []
|
||||
|
||||
has_more = len(rows) > lim
|
||||
if has_more:
|
||||
rows = rows[:lim]
|
||||
|
||||
next_cursor = ""
|
||||
if has_more and rows:
|
||||
next_cursor = _str(rows[-1].get("created_at", ""))
|
||||
|
||||
return {"messages": rows, "has_more": has_more, "next_cursor": next_cursor}
|
||||
|
||||
|
||||
def add_participant(conversation_id, participant_id, participant_type="user", display_name="", role="member"):
|
||||
"""Add a participant to a conversation.
|
||||
|
||||
Returns:
|
||||
dict with participant data
|
||||
"""
|
||||
cid = _str(conversation_id)
|
||||
pid = _str(participant_id)
|
||||
|
||||
# Check if already a participant
|
||||
existing = db.query("participants", filters={"conversation_id": cid, "participant_id": pid}, limit=1)
|
||||
if existing and len(existing) > 0:
|
||||
return existing[0]
|
||||
|
||||
p = db.insert("participants", {
|
||||
"conversation_id": cid,
|
||||
"participant_id": pid,
|
||||
"participant_type": _str(participant_type),
|
||||
"display_name": _str(display_name),
|
||||
"role": _str(role),
|
||||
"joined_at": "",
|
||||
})
|
||||
|
||||
# Publish realtime event
|
||||
realtime.publish(
|
||||
"conversation:" + cid,
|
||||
"participant.added",
|
||||
{
|
||||
"conversation_id": cid,
|
||||
"participant_id": pid,
|
||||
"participant_type": _str(participant_type),
|
||||
"display_name": _str(display_name),
|
||||
"role": _str(role),
|
||||
},
|
||||
)
|
||||
|
||||
# Insert system message
|
||||
send(cid, pid, "joined the conversation", "system")
|
||||
|
||||
return p
|
||||
|
||||
|
||||
def remove_participant(conversation_id, participant_id):
|
||||
"""Remove a participant from a conversation.
|
||||
|
||||
Returns:
|
||||
True on success
|
||||
"""
|
||||
cid = _str(conversation_id)
|
||||
pid = _str(participant_id)
|
||||
|
||||
# Delete participant record
|
||||
rows = db.query("participants", filters={"conversation_id": cid, "participant_id": pid}, limit=1)
|
||||
for r in (rows or []):
|
||||
db.delete("participants", r["id"])
|
||||
|
||||
# Delete read cursor
|
||||
cursors = db.query("read_cursors", filters={"conversation_id": cid, "participant_id": pid}, limit=1)
|
||||
for c in (cursors or []):
|
||||
db.delete("read_cursors", c["id"])
|
||||
|
||||
# Publish realtime event
|
||||
realtime.publish(
|
||||
"conversation:" + cid,
|
||||
"participant.removed",
|
||||
{"conversation_id": cid, "participant_id": pid},
|
||||
)
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def mark_read(conversation_id, participant_id, last_read_message_id):
|
||||
"""Update the read cursor for a participant.
|
||||
|
||||
Returns:
|
||||
True on success
|
||||
"""
|
||||
cid = _str(conversation_id)
|
||||
pid = _str(participant_id)
|
||||
mid = _str(last_read_message_id)
|
||||
|
||||
# Delete existing cursor (upsert via delete+insert)
|
||||
existing = db.query("read_cursors", filters={"conversation_id": cid, "participant_id": pid}, limit=1)
|
||||
for r in (existing or []):
|
||||
db.delete("read_cursors", r["id"])
|
||||
|
||||
db.insert("read_cursors", {
|
||||
"conversation_id": cid,
|
||||
"participant_id": pid,
|
||||
"last_read_message_id": mid,
|
||||
})
|
||||
|
||||
return True
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# REST API dispatcher
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def on_request(req):
|
||||
path = req["path"]
|
||||
method = req["method"]
|
||||
user_id = req.get("user_id", "")
|
||||
|
||||
# ── Conversations ──────────────────────────
|
||||
|
||||
# GET /conversations — list user's conversations
|
||||
if method == "GET" and path == "/conversations":
|
||||
return _handle_list_conversations(req, user_id)
|
||||
|
||||
# POST /conversations — create
|
||||
if method == "POST" and path == "/conversations":
|
||||
return _handle_create_conversation(req, user_id)
|
||||
|
||||
# GET /search?q=term — search conversations and messages
|
||||
if method == "GET" and path == "/search":
|
||||
return _handle_search(req, user_id)
|
||||
|
||||
# GET /unread — unread counts
|
||||
if method == "GET" and path == "/unread":
|
||||
return _handle_unread(user_id)
|
||||
|
||||
# GET /conversations/:id
|
||||
if method == "GET" and path.startswith("/conversations/"):
|
||||
cid = path[len("/conversations/"):]
|
||||
return _handle_get_conversation(cid, user_id)
|
||||
|
||||
# PUT /conversations/:id
|
||||
if method == "PUT" and path.startswith("/conversations/"):
|
||||
cid = path[len("/conversations/"):]
|
||||
return _handle_update_conversation(cid, req, user_id)
|
||||
|
||||
# DELETE /conversations/:id
|
||||
if method == "DELETE" and path.startswith("/conversations/"):
|
||||
cid = path[len("/conversations/"):]
|
||||
return _handle_delete_conversation(cid, user_id)
|
||||
|
||||
# ── Messages ───────────────────────────────
|
||||
|
||||
# Routes: /messages/:conversation_id or /messages/:conversation_id/:message_id
|
||||
if path.startswith("/messages/"):
|
||||
remainder = path[len("/messages/"):]
|
||||
parts = remainder.split("/")
|
||||
cid = parts[0]
|
||||
mid = parts[1] if len(parts) > 1 else ""
|
||||
|
||||
if method == "GET" and not mid:
|
||||
return _handle_list_messages(cid, req, user_id)
|
||||
if method == "POST" and not mid:
|
||||
return _handle_send_message(cid, req, user_id)
|
||||
if method == "PUT" and mid:
|
||||
return _handle_edit_message(cid, mid, req, user_id)
|
||||
if method == "DELETE" and mid:
|
||||
return _handle_delete_message(cid, mid, user_id)
|
||||
|
||||
# ── Participants ───────────────────────────
|
||||
|
||||
if path.startswith("/participants/"):
|
||||
remainder = path[len("/participants/"):]
|
||||
parts = remainder.split("/")
|
||||
cid = parts[0]
|
||||
pid = parts[1] if len(parts) > 1 else ""
|
||||
|
||||
if method == "GET" and not pid:
|
||||
return _handle_list_participants(cid, user_id)
|
||||
if method == "POST" and not pid:
|
||||
return _handle_add_participant(cid, req, user_id)
|
||||
if method == "DELETE" and pid:
|
||||
return _handle_remove_participant(cid, pid, user_id)
|
||||
|
||||
# ── Read cursors ──────────────────────────
|
||||
|
||||
if method == "POST" and path.startswith("/read/"):
|
||||
cid = path[len("/read/"):]
|
||||
return _handle_mark_read(cid, req, user_id)
|
||||
|
||||
return _resp(404, {"error": "not found"})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Conversation handlers
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _handle_list_conversations(req, user_id):
|
||||
"""List conversations the user is a participant in."""
|
||||
# Get all conversation IDs for this user
|
||||
my_parts = db.query("participants", filters={"participant_id": user_id}, limit=500)
|
||||
if not my_parts:
|
||||
return _resp(200, {"data": []})
|
||||
|
||||
cids = []
|
||||
for p in my_parts:
|
||||
cids.append(_str(p.get("conversation_id", "")))
|
||||
|
||||
# Fetch conversations and enrich with last message
|
||||
items = []
|
||||
for cid in cids:
|
||||
conv = _get_conversation(cid)
|
||||
if not conv:
|
||||
continue
|
||||
|
||||
# Get last message
|
||||
last_msgs = db.query("messages", filters={"conversation_id": cid}, order="-created_at", limit=1)
|
||||
last_msg = None
|
||||
if last_msgs and len(last_msgs) > 0:
|
||||
last_msg = {
|
||||
"id": last_msgs[0].get("id", ""),
|
||||
"content": _str(last_msgs[0].get("content", "")),
|
||||
"participant_id": _str(last_msgs[0].get("participant_id", "")),
|
||||
"content_type": _str(last_msgs[0].get("content_type", "")),
|
||||
"created_at": _str(last_msgs[0].get("created_at", "")),
|
||||
}
|
||||
|
||||
# Get participant count
|
||||
parts = db.query("participants", filters={"conversation_id": cid}, limit=500)
|
||||
part_count = len(parts or [])
|
||||
|
||||
items.append({
|
||||
"id": conv.get("id", ""),
|
||||
"title": conv.get("title", ""),
|
||||
"type": conv.get("type", ""),
|
||||
"created_by": conv.get("created_by", ""),
|
||||
"updated_at": conv.get("updated_at", ""),
|
||||
"created_at": conv.get("created_at", ""),
|
||||
"last_message": last_msg,
|
||||
"participant_count": part_count,
|
||||
})
|
||||
|
||||
# Sort by updated_at descending (most recent first)
|
||||
items = sorted(items, key=lambda x: x.get("updated_at", "") or x.get("created_at", ""), reverse=True)
|
||||
|
||||
return _resp(200, {"data": items})
|
||||
|
||||
|
||||
def _handle_create_conversation(req, user_id):
|
||||
"""Create a new conversation."""
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
title = _str(body.get("title", ""))
|
||||
conv_type = _str(body.get("type", "group"))
|
||||
participants_list = body.get("participants", [])
|
||||
creator_name = _str(body.get("creator_display_name", ""))
|
||||
|
||||
conv = create(title, conv_type, participants_list, user_id, creator_name)
|
||||
return _resp(201, conv)
|
||||
|
||||
|
||||
def _handle_get_conversation(cid, user_id):
|
||||
"""Get conversation detail with participants."""
|
||||
if not _is_participant(cid, user_id):
|
||||
return _resp(403, {"error": "not a participant"})
|
||||
|
||||
conv = _get_conversation(cid)
|
||||
if not conv:
|
||||
return _resp(404, {"error": "conversation not found"})
|
||||
|
||||
parts = db.query("participants", filters={"conversation_id": cid}, limit=500)
|
||||
|
||||
return _resp(200, {
|
||||
"id": conv.get("id", ""),
|
||||
"title": conv.get("title", ""),
|
||||
"type": conv.get("type", ""),
|
||||
"created_by": conv.get("created_by", ""),
|
||||
"updated_at": conv.get("updated_at", ""),
|
||||
"created_at": conv.get("created_at", ""),
|
||||
"participants": parts or [],
|
||||
})
|
||||
|
||||
|
||||
def _handle_update_conversation(cid, req, user_id):
|
||||
"""Update conversation (admin only)."""
|
||||
if not _is_admin(cid, user_id):
|
||||
return _resp(403, {"error": "admin only"})
|
||||
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
patch = {}
|
||||
title = body.get("title", None)
|
||||
if title != None:
|
||||
patch["title"] = _str(title)
|
||||
|
||||
if patch:
|
||||
db.update("conversations", cid, patch)
|
||||
|
||||
conv = _get_conversation(cid)
|
||||
return _resp(200, conv)
|
||||
|
||||
|
||||
def _handle_delete_conversation(cid, user_id):
|
||||
"""Delete conversation and all associated data (admin only)."""
|
||||
if not _is_admin(cid, user_id):
|
||||
return _resp(403, {"error": "admin only"})
|
||||
|
||||
# Cascade delete: messages, participants, read_cursors, then conversation
|
||||
msgs = db.query("messages", filters={"conversation_id": cid}, limit=5000)
|
||||
for m in (msgs or []):
|
||||
db.delete("messages", m["id"])
|
||||
|
||||
parts = db.query("participants", filters={"conversation_id": cid}, limit=500)
|
||||
for p in (parts or []):
|
||||
db.delete("participants", p["id"])
|
||||
|
||||
cursors = db.query("read_cursors", filters={"conversation_id": cid}, limit=500)
|
||||
for c in (cursors or []):
|
||||
db.delete("read_cursors", c["id"])
|
||||
|
||||
db.delete("conversations", cid)
|
||||
return _resp(200, {"ok": True})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Message handlers
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _handle_list_messages(cid, req, user_id):
|
||||
"""Paginated message history."""
|
||||
if not _is_participant(cid, user_id):
|
||||
return _resp(403, {"error": "not a participant"})
|
||||
|
||||
q = req.get("query", {})
|
||||
limit_val = _int(q.get("limit", "50"))
|
||||
cursor = _str(q.get("cursor", ""))
|
||||
|
||||
result = history(cid, limit_val, cursor)
|
||||
return _resp(200, result)
|
||||
|
||||
|
||||
def _handle_send_message(cid, req, user_id):
|
||||
"""Send a message."""
|
||||
if not _is_participant(cid, user_id):
|
||||
return _resp(403, {"error": "not a participant"})
|
||||
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
content = _str(body.get("content", ""))
|
||||
content_type = _str(body.get("content_type", "text"))
|
||||
|
||||
if not content:
|
||||
return _resp(400, {"error": "content required"})
|
||||
|
||||
msg = send(cid, user_id, content, content_type)
|
||||
return _resp(201, msg)
|
||||
|
||||
|
||||
def _handle_edit_message(cid, mid, req, user_id):
|
||||
"""Edit a message (author only)."""
|
||||
if not _is_participant(cid, user_id):
|
||||
return _resp(403, {"error": "not a participant"})
|
||||
|
||||
# Verify ownership
|
||||
msgs = db.query("messages", filters={"id": mid}, limit=1)
|
||||
if not msgs:
|
||||
return _resp(404, {"error": "message not found"})
|
||||
|
||||
msg = msgs[0]
|
||||
if _str(msg.get("participant_id", "")) != user_id:
|
||||
return _resp(403, {"error": "can only edit own messages"})
|
||||
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
content = _str(body.get("content", ""))
|
||||
if not content:
|
||||
return _resp(400, {"error": "content required"})
|
||||
|
||||
# Use created_at of the message as the edited_at marker
|
||||
db.update("messages", mid, {"content": content, "edited_at": msg.get("created_at", "")})
|
||||
|
||||
# Publish edit event
|
||||
realtime.publish(
|
||||
"conversation:" + cid,
|
||||
"message.edited",
|
||||
{"id": mid, "conversation_id": cid, "content": content, "edited_by": user_id},
|
||||
)
|
||||
|
||||
updated = db.query("messages", filters={"id": mid}, limit=1)
|
||||
return _resp(200, updated[0] if updated else {})
|
||||
|
||||
|
||||
def _handle_delete_message(cid, mid, user_id):
|
||||
"""Delete a message (author or admin)."""
|
||||
if not _is_participant(cid, user_id):
|
||||
return _resp(403, {"error": "not a participant"})
|
||||
|
||||
msgs = db.query("messages", filters={"id": mid}, limit=1)
|
||||
if not msgs:
|
||||
return _resp(404, {"error": "message not found"})
|
||||
|
||||
msg = msgs[0]
|
||||
is_author = _str(msg.get("participant_id", "")) == user_id
|
||||
is_conv_admin = _is_admin(cid, user_id)
|
||||
|
||||
if not is_author and not is_conv_admin:
|
||||
return _resp(403, {"error": "can only delete own messages or be admin"})
|
||||
|
||||
db.delete("messages", mid)
|
||||
|
||||
# Publish delete event
|
||||
realtime.publish(
|
||||
"conversation:" + cid,
|
||||
"message.deleted",
|
||||
{"id": mid, "conversation_id": cid, "deleted_by": user_id},
|
||||
)
|
||||
|
||||
return _resp(200, {"ok": True})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Participant handlers
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _handle_list_participants(cid, user_id):
|
||||
"""List participants in a conversation."""
|
||||
if not _is_participant(cid, user_id):
|
||||
return _resp(403, {"error": "not a participant"})
|
||||
|
||||
parts = db.query("participants", filters={"conversation_id": cid}, limit=500)
|
||||
return _resp(200, {"data": parts or []})
|
||||
|
||||
|
||||
def _handle_add_participant(cid, req, user_id):
|
||||
"""Add a participant (admin only)."""
|
||||
if not _is_admin(cid, user_id):
|
||||
return _resp(403, {"error": "admin only"})
|
||||
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
pid = _str(body.get("participant_id", ""))
|
||||
if not pid:
|
||||
return _resp(400, {"error": "participant_id required"})
|
||||
|
||||
ptype = _str(body.get("participant_type", "user"))
|
||||
display_name = _str(body.get("display_name", ""))
|
||||
role = _str(body.get("role", "member"))
|
||||
|
||||
p = add_participant(cid, pid, ptype, display_name, role)
|
||||
return _resp(201, p)
|
||||
|
||||
|
||||
def _handle_remove_participant(cid, pid, user_id):
|
||||
"""Remove a participant (admin only, or self-remove)."""
|
||||
is_self = pid == user_id
|
||||
if not is_self and not _is_admin(cid, user_id):
|
||||
return _resp(403, {"error": "admin only (or remove yourself)"})
|
||||
|
||||
remove_participant(cid, pid)
|
||||
return _resp(200, {"ok": True})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Read cursor handlers
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _handle_mark_read(cid, req, user_id):
|
||||
"""Mark conversation as read up to a message."""
|
||||
if not _is_participant(cid, user_id):
|
||||
return _resp(403, {"error": "not a participant"})
|
||||
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
mid = _str(body.get("last_read_message_id", ""))
|
||||
if not mid:
|
||||
return _resp(400, {"error": "last_read_message_id required"})
|
||||
|
||||
mark_read(cid, user_id, mid)
|
||||
return _resp(200, {"ok": True})
|
||||
|
||||
|
||||
def _handle_search(req, user_id):
|
||||
"""Search across conversation titles and message content."""
|
||||
q = _str(req.get("query", {}).get("q", ""))
|
||||
if len(q) < 2:
|
||||
return _resp(400, {"error": "query must be at least 2 characters"})
|
||||
|
||||
pattern = "%" + q + "%"
|
||||
|
||||
# Get user's conversation IDs
|
||||
my_parts = db.query("participants", filters={"participant_id": user_id}, limit=500)
|
||||
if not my_parts:
|
||||
return _resp(200, {"conversations": [], "messages": []})
|
||||
|
||||
cid_set = {}
|
||||
for p in my_parts:
|
||||
cid_set[_str(p.get("conversation_id", ""))] = True
|
||||
|
||||
# Search conversations by title
|
||||
matching_convs = db.query("conversations", search_like={"title": pattern}, limit=50)
|
||||
conv_results = []
|
||||
for conv in (matching_convs or []):
|
||||
cid = _str(conv.get("id", ""))
|
||||
if cid in cid_set:
|
||||
conv_results.append({
|
||||
"id": cid,
|
||||
"title": conv.get("title", ""),
|
||||
"type": conv.get("type", ""),
|
||||
"updated_at": conv.get("updated_at", ""),
|
||||
"created_at": conv.get("created_at", ""),
|
||||
})
|
||||
|
||||
# Search messages in user's conversations
|
||||
msg_results = []
|
||||
for cid in cid_set:
|
||||
if not cid:
|
||||
continue
|
||||
msgs = db.query("messages", filters={"conversation_id": cid}, search_like={"content": pattern}, order="-created_at", limit=5)
|
||||
for m in (msgs or []):
|
||||
msg_results.append({
|
||||
"id": m.get("id", ""),
|
||||
"conversation_id": cid,
|
||||
"participant_id": m.get("participant_id", ""),
|
||||
"content": _str(m.get("content", "")),
|
||||
"content_type": _str(m.get("content_type", "")),
|
||||
"created_at": _str(m.get("created_at", "")),
|
||||
})
|
||||
|
||||
# Sort messages by created_at descending, limit to 50
|
||||
msg_results = sorted(msg_results, key=lambda x: x.get("created_at", ""), reverse=True)[:50]
|
||||
|
||||
return _resp(200, {"conversations": conv_results, "messages": msg_results})
|
||||
|
||||
|
||||
def _handle_unread(user_id):
|
||||
"""Get unread counts for all user's conversations."""
|
||||
my_parts = db.query("participants", filters={"participant_id": user_id}, limit=500)
|
||||
if not my_parts:
|
||||
return _resp(200, {"data": {}})
|
||||
|
||||
counts = {}
|
||||
for p in my_parts:
|
||||
cid = _str(p.get("conversation_id", ""))
|
||||
if not cid:
|
||||
continue
|
||||
|
||||
# Get read cursor
|
||||
cursors = db.query("read_cursors", filters={"conversation_id": cid, "participant_id": user_id}, limit=1)
|
||||
|
||||
if cursors and len(cursors) > 0:
|
||||
last_read_id = _str(cursors[0].get("last_read_message_id", ""))
|
||||
if last_read_id:
|
||||
# Get the created_at of the last read message
|
||||
last_read_msgs = db.query("messages", filters={"id": last_read_id}, limit=1)
|
||||
if last_read_msgs and len(last_read_msgs) > 0:
|
||||
last_read_at = _str(last_read_msgs[0].get("created_at", ""))
|
||||
# Count messages after the read cursor
|
||||
unread = db.query("messages", filters={"conversation_id": cid}, after={"created_at": last_read_at}, limit=1000)
|
||||
counts[cid] = len(unread or [])
|
||||
else:
|
||||
# Last read message was deleted — count all
|
||||
all_msgs = db.query("messages", filters={"conversation_id": cid}, limit=1000)
|
||||
counts[cid] = len(all_msgs or [])
|
||||
else:
|
||||
# No cursor value — all messages are unread
|
||||
all_msgs = db.query("messages", filters={"conversation_id": cid}, limit=1000)
|
||||
counts[cid] = len(all_msgs or [])
|
||||
else:
|
||||
# No cursor at all — all messages are unread
|
||||
all_msgs = db.query("messages", filters={"conversation_id": cid}, limit=1000)
|
||||
counts[cid] = len(all_msgs or [])
|
||||
|
||||
return _resp(200, {"data": counts})
|
||||
616
packages/chat/css/main.css
Normal file
616
packages/chat/css/main.css
Normal file
@@ -0,0 +1,616 @@
|
||||
/* ═══════════════════════════════════════════
|
||||
Chat Surface — Styles (v0.2.0)
|
||||
Uses variables.css theme tokens:
|
||||
--bg, --bg-surface, --bg-raised, --bg-hover, --bg-secondary
|
||||
--text, --text-2, --text-3
|
||||
--accent, --accent-dim, --border, --border-light
|
||||
--input-bg, --danger, --danger-bg, --success
|
||||
═══════════════════════════════════════════ */
|
||||
|
||||
/* ── Layout ─────────────────────────────── */
|
||||
|
||||
.chat-app {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
height: 100vh;
|
||||
overflow: hidden;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.chat-loading {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 100vh;
|
||||
}
|
||||
|
||||
.chat-body {
|
||||
display: flex;
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
.chat-main {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
/* ── Topbar extras ──────────────────────── */
|
||||
|
||||
.chat-topbar__thread-title {
|
||||
font-weight: 600;
|
||||
font-size: 14px;
|
||||
margin-right: 8px;
|
||||
color: var(--text-2);
|
||||
}
|
||||
|
||||
/* ── Sidebar ────────────────────────────── */
|
||||
|
||||
.chat-sidebar {
|
||||
width: 280px;
|
||||
min-width: 280px;
|
||||
border-right: 1px solid var(--border);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
background: var(--bg-secondary);
|
||||
}
|
||||
|
||||
.chat-sidebar__header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 12px 16px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
|
||||
.chat-sidebar__title {
|
||||
font-weight: 600;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.chat-sidebar__list {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.chat-sidebar__empty {
|
||||
padding: 24px 16px;
|
||||
text-align: center;
|
||||
color: var(--text-3);
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.chat-sidebar__item {
|
||||
padding: 10px 16px;
|
||||
cursor: pointer;
|
||||
border-bottom: 1px solid var(--border-light);
|
||||
transition: background 0.1s;
|
||||
}
|
||||
|
||||
.chat-sidebar__item:hover {
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
|
||||
.chat-sidebar__item--active {
|
||||
background: var(--accent-dim);
|
||||
}
|
||||
|
||||
.chat-sidebar__item-top {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: baseline;
|
||||
margin-bottom: 2px;
|
||||
}
|
||||
|
||||
.chat-sidebar__item-title {
|
||||
font-weight: 600;
|
||||
font-size: 13px;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
flex: 1;
|
||||
margin-right: 8px;
|
||||
}
|
||||
|
||||
.chat-sidebar__item-time {
|
||||
font-size: 11px;
|
||||
color: var(--text-3);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.chat-sidebar__item-bottom {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.chat-sidebar__item-preview {
|
||||
font-size: 12px;
|
||||
color: var(--text-2);
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.chat-sidebar__badge {
|
||||
background: var(--accent);
|
||||
color: var(--text-on-color);
|
||||
font-size: 11px;
|
||||
font-weight: 600;
|
||||
min-width: 18px;
|
||||
height: 18px;
|
||||
border-radius: 9px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 0 5px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* ── Sidebar Search ────────────────────── */
|
||||
|
||||
.chat-sidebar__search {
|
||||
position: relative;
|
||||
padding: 8px 16px;
|
||||
border-bottom: 1px solid var(--border-light);
|
||||
}
|
||||
|
||||
.chat-sidebar__search-input {
|
||||
width: 100%;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
padding: 6px 28px 6px 10px;
|
||||
font-size: 13px;
|
||||
font-family: inherit;
|
||||
background: var(--input-bg);
|
||||
color: var(--text);
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.chat-sidebar__search-input:focus {
|
||||
outline: none;
|
||||
border-color: var(--accent);
|
||||
}
|
||||
|
||||
.chat-sidebar__search-clear {
|
||||
position: absolute;
|
||||
right: 22px;
|
||||
top: 50%;
|
||||
transform: translateY(-50%);
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text-3);
|
||||
cursor: pointer;
|
||||
font-size: 16px;
|
||||
padding: 0 4px;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
.chat-sidebar__search-clear:hover {
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.chat-sidebar__search-results {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.chat-sidebar__search-section {
|
||||
padding: 8px 16px 4px;
|
||||
font-size: 11px;
|
||||
font-weight: 600;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
.chat-sidebar__search-loading {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
padding: 16px;
|
||||
}
|
||||
|
||||
.chat-sidebar__item--search-msg .chat-sidebar__item-preview {
|
||||
font-size: 13px;
|
||||
white-space: normal;
|
||||
display: -webkit-box;
|
||||
-webkit-line-clamp: 2;
|
||||
-webkit-box-orient: vertical;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
/* ── Message Thread ─────────────────────── */
|
||||
|
||||
.chat-thread {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
.chat-thread--empty {
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
.chat-thread__messages {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
padding: 16px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
.chat-thread__loading {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
padding: 24px;
|
||||
}
|
||||
|
||||
.chat-thread__loading-more {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
padding: 8px;
|
||||
}
|
||||
|
||||
.chat-thread__load-more {
|
||||
align-self: center;
|
||||
background: none;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 4px;
|
||||
padding: 4px 12px;
|
||||
font-size: 12px;
|
||||
color: var(--text-2);
|
||||
cursor: pointer;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.chat-thread__load-more:hover {
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
|
||||
.chat-thread__typing {
|
||||
padding: 4px 16px 8px;
|
||||
font-size: 12px;
|
||||
color: var(--text-3);
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
/* ── Message Bubble ─────────────────────── */
|
||||
|
||||
.chat-msg {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
gap: 8px;
|
||||
padding: 4px 0;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.chat-msg--own {
|
||||
flex-direction: row-reverse;
|
||||
}
|
||||
|
||||
.chat-msg--system {
|
||||
justify-content: center;
|
||||
padding: 2px 0;
|
||||
}
|
||||
|
||||
.chat-msg--system span {
|
||||
font-size: 12px;
|
||||
color: var(--text-3);
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.chat-msg--deleted {
|
||||
justify-content: center;
|
||||
padding: 2px 0;
|
||||
}
|
||||
|
||||
.chat-msg--deleted em {
|
||||
font-size: 12px;
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
.chat-msg__body {
|
||||
max-width: 65%;
|
||||
background: var(--bg-raised);
|
||||
border-radius: 12px;
|
||||
padding: 8px 12px;
|
||||
}
|
||||
|
||||
.chat-msg--own .chat-msg__body {
|
||||
background: var(--accent);
|
||||
color: var(--text-on-color);
|
||||
}
|
||||
|
||||
.chat-msg__name {
|
||||
font-size: 11px;
|
||||
font-weight: 600;
|
||||
color: var(--text-2);
|
||||
display: block;
|
||||
margin-bottom: 2px;
|
||||
}
|
||||
|
||||
.chat-msg__content {
|
||||
font-size: 14px;
|
||||
line-height: 1.4;
|
||||
white-space: pre-wrap;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.chat-msg__meta {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
align-items: center;
|
||||
margin-top: 2px;
|
||||
}
|
||||
|
||||
.chat-msg__time {
|
||||
font-size: 10px;
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
.chat-msg--own .chat-msg__time {
|
||||
color: rgba(255, 255, 255, 0.7);
|
||||
}
|
||||
|
||||
.chat-msg__edited {
|
||||
font-size: 10px;
|
||||
color: var(--text-3);
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.chat-msg--own .chat-msg__edited {
|
||||
color: rgba(255, 255, 255, 0.7);
|
||||
}
|
||||
|
||||
/* ── Message Actions ────────────────────── */
|
||||
|
||||
.chat-msg__actions {
|
||||
display: flex;
|
||||
gap: 2px;
|
||||
position: absolute;
|
||||
top: 0;
|
||||
right: 0;
|
||||
background: var(--bg-surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
box-shadow: var(--shadow-lg);
|
||||
padding: 2px;
|
||||
}
|
||||
|
||||
.chat-msg--own .chat-msg__actions {
|
||||
right: auto;
|
||||
left: 0;
|
||||
}
|
||||
|
||||
.chat-msg__action {
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 4px 6px;
|
||||
cursor: pointer;
|
||||
border-radius: 4px;
|
||||
font-size: 14px;
|
||||
line-height: 1;
|
||||
color: var(--text-2);
|
||||
}
|
||||
|
||||
.chat-msg__action:hover {
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
|
||||
.chat-msg__action--danger:hover {
|
||||
background: var(--danger-bg);
|
||||
color: var(--danger);
|
||||
}
|
||||
|
||||
/* ── Message Edit ───────────────────────── */
|
||||
|
||||
.chat-msg__edit {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.chat-msg__edit-input {
|
||||
width: 100%;
|
||||
min-width: 200px;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
padding: 6px 8px;
|
||||
font-size: 14px;
|
||||
font-family: inherit;
|
||||
resize: vertical;
|
||||
background: var(--input-bg);
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.chat-msg__edit-actions {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
justify-content: flex-end;
|
||||
}
|
||||
|
||||
/* ── Compose Bar ────────────────────────── */
|
||||
|
||||
.chat-compose {
|
||||
display: flex;
|
||||
align-items: flex-end;
|
||||
gap: 8px;
|
||||
padding: 12px 16px;
|
||||
border-top: 1px solid var(--border);
|
||||
background: var(--bg);
|
||||
}
|
||||
|
||||
.chat-compose__input {
|
||||
flex: 1;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 8px 12px;
|
||||
font-size: 14px;
|
||||
font-family: inherit;
|
||||
resize: none;
|
||||
line-height: 1.4;
|
||||
max-height: 160px;
|
||||
background: var(--input-bg);
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.chat-compose__input:focus {
|
||||
outline: none;
|
||||
border-color: var(--accent);
|
||||
box-shadow: 0 0 0 2px var(--accent-dim);
|
||||
}
|
||||
|
||||
/* ── Participant Sidebar ────────────────── */
|
||||
|
||||
.chat-participants {
|
||||
width: 240px;
|
||||
min-width: 240px;
|
||||
border-left: 1px solid var(--border);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
background: var(--bg-secondary);
|
||||
}
|
||||
|
||||
.chat-participants__header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
padding: 12px 16px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
font-weight: 600;
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.chat-participants__list {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
padding: 8px 0;
|
||||
}
|
||||
|
||||
.chat-participants__item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 6px 16px;
|
||||
}
|
||||
|
||||
.chat-participants__name {
|
||||
flex: 1;
|
||||
font-size: 13px;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
.chat-participants__badge {
|
||||
font-size: 10px;
|
||||
color: var(--accent);
|
||||
font-weight: 600;
|
||||
margin-left: 4px;
|
||||
}
|
||||
|
||||
.chat-participants__status {
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
background: var(--text-3);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.chat-participants__status--online {
|
||||
background: var(--success);
|
||||
}
|
||||
|
||||
.chat-participants__remove {
|
||||
background: none;
|
||||
border: none;
|
||||
color: var(--text-3);
|
||||
cursor: pointer;
|
||||
font-size: 16px;
|
||||
padding: 0 4px;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
.chat-participants__remove:hover {
|
||||
color: var(--danger);
|
||||
}
|
||||
|
||||
/* ── New Conversation Dialog ────────────── */
|
||||
|
||||
.chat-new {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 12px;
|
||||
min-width: 320px;
|
||||
}
|
||||
|
||||
.chat-new__type {
|
||||
display: flex;
|
||||
gap: 16px;
|
||||
}
|
||||
|
||||
.chat-new__type label {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
font-size: 13px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.chat-new__title {
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
padding: 8px 10px;
|
||||
font-size: 14px;
|
||||
font-family: inherit;
|
||||
background: var(--input-bg);
|
||||
color: var(--text);
|
||||
}
|
||||
|
||||
.chat-new__selected {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.chat-new__chip {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
background: var(--accent-dim);
|
||||
color: var(--accent);
|
||||
font-size: 12px;
|
||||
font-weight: 500;
|
||||
padding: 3px 8px;
|
||||
border-radius: 12px;
|
||||
}
|
||||
|
||||
.chat-new__chip button {
|
||||
background: none;
|
||||
border: none;
|
||||
color: inherit;
|
||||
cursor: pointer;
|
||||
font-size: 14px;
|
||||
padding: 0;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
/* ── Dialog UserPicker overflow fix ─────── */
|
||||
/* Allow the autocomplete dropdown to overflow the dialog body.
|
||||
Applies to both New Conversation and Add Participant dialogs. */
|
||||
|
||||
.sw-dialog__body:has(.sw-user-picker) {
|
||||
overflow: visible;
|
||||
}
|
||||
|
||||
.sw-dialog:has(.sw-user-picker) {
|
||||
overflow: visible;
|
||||
}
|
||||
854
packages/chat/js/main.js
Normal file
854
packages/chat/js/main.js
Normal file
@@ -0,0 +1,854 @@
|
||||
/**
|
||||
* Chat — Surface Entry Point (v0.2.0)
|
||||
*
|
||||
* Messaging surface built on chat-core library:
|
||||
* sw.api.ext('chat-core') — conversation/message CRUD
|
||||
* sw.api.ext('chat') — typing indicators
|
||||
* sw.realtime — live events
|
||||
* sw.ui.* — primitive components
|
||||
* sw.shell.Topbar — navigation bar
|
||||
*/
|
||||
(async function () {
|
||||
'use strict';
|
||||
|
||||
var mount = document.getElementById('extension-mount');
|
||||
if (!mount) return;
|
||||
|
||||
var base = window.__BASE__ || '';
|
||||
var ver = window.__VERSION__ || '0';
|
||||
|
||||
// ── Boot SDK ───────────────────────────────
|
||||
try {
|
||||
if (!window.preact) {
|
||||
var { h, render } = await import(base + '/js/sw/vendor/preact.module.js');
|
||||
var hooksModule = await import(base + '/js/sw/vendor/hooks.module.js');
|
||||
var htmModule = await import(base + '/js/sw/vendor/htm.module.js');
|
||||
window.preact = { h, render };
|
||||
window.hooks = hooksModule;
|
||||
window.html = htmModule.default.bind(h);
|
||||
}
|
||||
var sdk = await import(base + '/js/sw/sdk/index.js?v=' + ver);
|
||||
await sdk.boot();
|
||||
} catch (e) {
|
||||
mount.innerHTML = '<p style="color:var(--danger);padding:24px;">SDK boot failed: ' + e.message + '</p>';
|
||||
return;
|
||||
}
|
||||
|
||||
var { html } = window;
|
||||
var { useState, useEffect, useCallback, useRef, useMemo } = hooks;
|
||||
var { render } = preact;
|
||||
|
||||
// ── SDK modules ────────────────────────────
|
||||
var api = sw.api.ext('chat-core');
|
||||
var chatApi = sw.api.ext('chat');
|
||||
var { Button, Spinner, Avatar, Dialog, Tabs } = sw.ui;
|
||||
var Topbar = sw.shell.Topbar;
|
||||
|
||||
// Import UserPicker directly (not in sw.ui index)
|
||||
var { UserPicker } = await import(base + '/js/sw/primitives/user-picker.js?v=' + ver);
|
||||
|
||||
// ── Helpers ────────────────────────────────
|
||||
var _timers = {};
|
||||
function debounce(key, fn, ms) {
|
||||
clearTimeout(_timers[key]);
|
||||
_timers[key] = setTimeout(fn, ms);
|
||||
}
|
||||
|
||||
function timeAgo(ts) {
|
||||
if (!ts) return '';
|
||||
var d = new Date(ts);
|
||||
var now = Date.now();
|
||||
var diff = Math.floor((now - d.getTime()) / 1000);
|
||||
if (diff < 60) return 'now';
|
||||
if (diff < 3600) return Math.floor(diff / 60) + 'm';
|
||||
if (diff < 86400) return Math.floor(diff / 3600) + 'h';
|
||||
if (diff < 604800) return Math.floor(diff / 86400) + 'd';
|
||||
return d.toLocaleDateString();
|
||||
}
|
||||
|
||||
function truncate(str, len) {
|
||||
if (!str) return '';
|
||||
return str.length > len ? str.slice(0, len) + '\u2026' : str;
|
||||
}
|
||||
|
||||
function currentUserId() {
|
||||
return sw.auth?.user?.id || '';
|
||||
}
|
||||
|
||||
function currentDisplayName() {
|
||||
return sw.auth?.user?.display_name || sw.auth?.user?.username || '';
|
||||
}
|
||||
|
||||
|
||||
// ═══════════════════════════════════════════
|
||||
// ConversationList — left sidebar
|
||||
// ═══════════════════════════════════════════
|
||||
|
||||
function ConversationList({ selected, onSelect, onNew, conversations, unread }) {
|
||||
var [searchQuery, setSearchQuery] = useState('');
|
||||
var [searchResults, setSearchResults] = useState(null);
|
||||
var [searching, setSearching] = useState(false);
|
||||
|
||||
function handleSearchInput(e) {
|
||||
var q = e.target.value;
|
||||
setSearchQuery(q);
|
||||
if (q.length < 2) {
|
||||
setSearchResults(null);
|
||||
setSearching(false);
|
||||
return;
|
||||
}
|
||||
debounce('search', () => {
|
||||
setSearching(true);
|
||||
api.get('/search?q=' + encodeURIComponent(q)).then(res => {
|
||||
setSearchResults(res || { conversations: [], messages: [] });
|
||||
setSearching(false);
|
||||
}).catch(() => { setSearching(false); });
|
||||
}, 300);
|
||||
}
|
||||
|
||||
function clearSearch() {
|
||||
setSearchQuery('');
|
||||
setSearchResults(null);
|
||||
setSearching(false);
|
||||
}
|
||||
|
||||
function selectFromSearch(cid) {
|
||||
clearSearch();
|
||||
onSelect(cid);
|
||||
}
|
||||
|
||||
// Render search results
|
||||
var showSearch = searchResults !== null;
|
||||
var sConvs = showSearch ? (searchResults.conversations || []) : [];
|
||||
var sMsgs = showSearch ? (searchResults.messages || []) : [];
|
||||
|
||||
return html`
|
||||
<div class="chat-sidebar">
|
||||
<div class="chat-sidebar__header">
|
||||
<span class="chat-sidebar__title">Conversations</span>
|
||||
<${Button} size="sm" onClick=${onNew}>New<//>
|
||||
</div>
|
||||
<div class="chat-sidebar__search">
|
||||
<input class="chat-sidebar__search-input"
|
||||
type="text"
|
||||
value=${searchQuery}
|
||||
placeholder="Search\u2026"
|
||||
onInput=${handleSearchInput} />
|
||||
${searchQuery && html`
|
||||
<button class="chat-sidebar__search-clear" onClick=${clearSearch}>\u00d7</button>`}
|
||||
</div>
|
||||
${showSearch ? html`
|
||||
<div class="chat-sidebar__search-results">
|
||||
${searching && html`<div class="chat-sidebar__search-loading"><${Spinner} size="sm" /></div>`}
|
||||
${!searching && sConvs.length === 0 && sMsgs.length === 0 && html`
|
||||
<div class="chat-sidebar__empty">No results</div>`}
|
||||
${sConvs.length > 0 && html`
|
||||
<div class="chat-sidebar__search-section">Conversations</div>
|
||||
${sConvs.map(c => html`
|
||||
<div key=${c.id} class="chat-sidebar__item" onClick=${() => selectFromSearch(c.id)}>
|
||||
<div class="chat-sidebar__item-top">
|
||||
<span class="chat-sidebar__item-title">${c.title || 'Untitled'}</span>
|
||||
<span class="chat-sidebar__item-time">${timeAgo(c.updated_at || c.created_at)}</span>
|
||||
</div>
|
||||
</div>`)}`}
|
||||
${sMsgs.length > 0 && html`
|
||||
<div class="chat-sidebar__search-section">Messages</div>
|
||||
${sMsgs.map(m => html`
|
||||
<div key=${m.id} class="chat-sidebar__item chat-sidebar__item--search-msg" onClick=${() => selectFromSearch(m.conversation_id)}>
|
||||
<div class="chat-sidebar__item-top">
|
||||
<span class="chat-sidebar__item-preview">${truncate(m.content, 80)}</span>
|
||||
</div>
|
||||
<div class="chat-sidebar__item-bottom">
|
||||
<span class="chat-sidebar__item-time">${timeAgo(m.created_at)}</span>
|
||||
</div>
|
||||
</div>`)}`}
|
||||
</div>
|
||||
` : html`
|
||||
<div class="chat-sidebar__list">
|
||||
${conversations.length === 0 && html`
|
||||
<div class="chat-sidebar__empty">No conversations yet</div>`}
|
||||
${conversations.map(c => html`
|
||||
<div key=${c.id}
|
||||
class=${'chat-sidebar__item' + (selected === c.id ? ' chat-sidebar__item--active' : '')}
|
||||
onClick=${() => onSelect(c.id)}>
|
||||
<div class="chat-sidebar__item-top">
|
||||
<span class="chat-sidebar__item-title">${c.title || 'Untitled'}</span>
|
||||
<span class="chat-sidebar__item-time">${timeAgo(c.updated_at || c.created_at)}</span>
|
||||
</div>
|
||||
<div class="chat-sidebar__item-bottom">
|
||||
<span class="chat-sidebar__item-preview">
|
||||
${c.last_message
|
||||
? truncate(c.last_message.content_type === 'system'
|
||||
? '\u2022 ' + c.last_message.content
|
||||
: c.last_message.content, 60)
|
||||
: 'No messages yet'}
|
||||
</span>
|
||||
${(unread[c.id] || 0) > 0 && html`
|
||||
<span class="chat-sidebar__badge">${unread[c.id]}</span>`}
|
||||
</div>
|
||||
</div>`)}
|
||||
</div>
|
||||
`}
|
||||
</div>`;
|
||||
}
|
||||
|
||||
|
||||
// ═══════════════════════════════════════════
|
||||
// MessageBubble — single message
|
||||
// ═══════════════════════════════════════════
|
||||
|
||||
function MessageBubble({ msg, isOwn, onEdit, onDelete }) {
|
||||
var [editing, setEditing] = useState(false);
|
||||
var [editText, setEditText] = useState('');
|
||||
var [hover, setHover] = useState(false);
|
||||
|
||||
if (msg._deleted) {
|
||||
return html`
|
||||
<div class="chat-msg chat-msg--deleted">
|
||||
<em>This message was deleted</em>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
if (msg.content_type === 'system') {
|
||||
return html`
|
||||
<div class="chat-msg chat-msg--system">
|
||||
<span>${msg.content}</span>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
function startEdit() {
|
||||
setEditText(msg.content);
|
||||
setEditing(true);
|
||||
}
|
||||
|
||||
function cancelEdit() {
|
||||
setEditing(false);
|
||||
setEditText('');
|
||||
}
|
||||
|
||||
function saveEdit() {
|
||||
if (editText.trim() && editText !== msg.content) {
|
||||
onEdit(msg.id, editText.trim());
|
||||
}
|
||||
setEditing(false);
|
||||
}
|
||||
|
||||
function onEditKeyDown(e) {
|
||||
if (e.key === 'Escape') cancelEdit();
|
||||
if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); saveEdit(); }
|
||||
}
|
||||
|
||||
return html`
|
||||
<div class=${'chat-msg' + (isOwn ? ' chat-msg--own' : '')}
|
||||
onMouseEnter=${() => setHover(true)}
|
||||
onMouseLeave=${() => setHover(false)}>
|
||||
${!isOwn && html`
|
||||
<${Avatar} name=${msg._display_name || msg.participant_id} size="sm" />`}
|
||||
<div class="chat-msg__body">
|
||||
${!isOwn && html`<span class="chat-msg__name">${msg._display_name || msg.participant_id}</span>`}
|
||||
${editing ? html`
|
||||
<div class="chat-msg__edit">
|
||||
<textarea class="chat-msg__edit-input"
|
||||
value=${editText}
|
||||
onInput=${e => setEditText(e.target.value)}
|
||||
onKeyDown=${onEditKeyDown}
|
||||
rows="2" />
|
||||
<div class="chat-msg__edit-actions">
|
||||
<${Button} size="sm" variant="secondary" onClick=${cancelEdit}>Cancel<//>
|
||||
<${Button} size="sm" onClick=${saveEdit}>Save<//>
|
||||
</div>
|
||||
</div>
|
||||
` : msg.content_type === 'markdown' && sw?.markdown?.ready ? html`
|
||||
<div class="chat-msg__content" dangerouslySetInnerHTML=${{ __html: sw.markdown.renderSync(msg.content, { sanitize: true }) }} />` : html`
|
||||
<div class="chat-msg__content">${msg.content}</div>`}
|
||||
<div class="chat-msg__meta">
|
||||
<span class="chat-msg__time">${timeAgo(msg.created_at)}</span>
|
||||
${msg.edited_at && html`<span class="chat-msg__edited">(edited)</span>`}
|
||||
</div>
|
||||
</div>
|
||||
${hover && isOwn && !editing && html`
|
||||
<div class="chat-msg__actions">
|
||||
<button class="chat-msg__action" onClick=${startEdit} title="Edit">✎</button>
|
||||
<button class="chat-msg__action chat-msg__action--danger" onClick=${() => onDelete(msg.id)} title="Delete">🗑</button>
|
||||
</div>`}
|
||||
</div>`;
|
||||
}
|
||||
|
||||
|
||||
// ═══════════════════════════════════════════
|
||||
// MessageThread — center pane
|
||||
// ═══════════════════════════════════════════
|
||||
|
||||
function MessageThread({ conversationId, participants }) {
|
||||
var [messages, setMessages] = useState([]);
|
||||
var [loading, setLoading] = useState(false);
|
||||
var [hasMore, setHasMore] = useState(false);
|
||||
var [nextCursor, setNextCursor] = useState('');
|
||||
var [typingUsers, setTypingUsers] = useState({});
|
||||
var bottomRef = useRef(null);
|
||||
var listRef = useRef(null);
|
||||
var userId = currentUserId();
|
||||
|
||||
// Build participant lookup
|
||||
var partMap = useMemo(() => {
|
||||
var m = {};
|
||||
(participants || []).forEach(p => { m[p.participant_id] = p.display_name || p.participant_id; });
|
||||
return m;
|
||||
}, [participants]);
|
||||
|
||||
// Enrich messages with display names
|
||||
function enrichMessages(msgs) {
|
||||
return msgs.map(m => ({ ...m, _display_name: partMap[m.participant_id] || m.participant_id }));
|
||||
}
|
||||
|
||||
// Load initial messages
|
||||
useEffect(() => {
|
||||
if (!conversationId) { setMessages([]); return; }
|
||||
setLoading(true);
|
||||
setMessages([]);
|
||||
setNextCursor('');
|
||||
setHasMore(false);
|
||||
api.get('/messages/' + conversationId + '?limit=50').then(res => {
|
||||
var data = res || {};
|
||||
var msgs = (data.messages || []).reverse();
|
||||
setMessages(enrichMessages(msgs));
|
||||
setHasMore(!!data.has_more);
|
||||
setNextCursor(data.next_cursor || '');
|
||||
setLoading(false);
|
||||
setTimeout(() => scrollToBottom(), 50);
|
||||
}).catch(() => setLoading(false));
|
||||
}, [conversationId, partMap]);
|
||||
|
||||
// Load older messages (preserves scroll position)
|
||||
function loadMore() {
|
||||
if (!hasMore || !nextCursor || loading) return;
|
||||
var el = listRef.current;
|
||||
var prevHeight = el ? el.scrollHeight : 0;
|
||||
setLoading(true);
|
||||
api.get('/messages/' + conversationId + '?limit=50&cursor=' + encodeURIComponent(nextCursor)).then(res => {
|
||||
var data = res || {};
|
||||
var older = (data.messages || []).reverse();
|
||||
setMessages(prev => [...enrichMessages(older), ...prev]);
|
||||
setHasMore(!!data.has_more);
|
||||
setNextCursor(data.next_cursor || '');
|
||||
setLoading(false);
|
||||
// Restore scroll position after prepending older messages
|
||||
requestAnimationFrame(() => {
|
||||
if (el) el.scrollTop = el.scrollHeight - prevHeight;
|
||||
});
|
||||
}).catch(() => setLoading(false));
|
||||
}
|
||||
|
||||
function scrollToBottom() {
|
||||
if (bottomRef.current) bottomRef.current.scrollIntoView({ behavior: 'auto' });
|
||||
}
|
||||
|
||||
// Scroll listener for load-more
|
||||
useEffect(() => {
|
||||
var el = listRef.current;
|
||||
if (!el) return;
|
||||
function onScroll() {
|
||||
if (el.scrollTop < 80 && hasMore && !loading) loadMore();
|
||||
}
|
||||
el.addEventListener('scroll', onScroll);
|
||||
return () => el.removeEventListener('scroll', onScroll);
|
||||
}, [hasMore, loading, nextCursor, conversationId]);
|
||||
|
||||
// Realtime: new messages, edits, deletes
|
||||
useEffect(() => {
|
||||
if (!conversationId) return;
|
||||
var channel = 'conversation:' + conversationId;
|
||||
|
||||
var unsubs = [
|
||||
sw.realtime.subscribe(channel, 'message', (payload) => {
|
||||
var msg = { ...payload, _display_name: partMap[payload.participant_id] || payload.participant_id };
|
||||
setMessages(prev => [...prev, msg]);
|
||||
setTimeout(() => scrollToBottom(), 50);
|
||||
// Auto mark read if from someone else
|
||||
if (payload.participant_id !== userId && payload.id) {
|
||||
api.post('/read/' + conversationId, { last_read_message_id: payload.id }).catch(() => {});
|
||||
}
|
||||
}),
|
||||
sw.realtime.subscribe(channel, 'message.edited', (payload) => {
|
||||
setMessages(prev => prev.map(m =>
|
||||
m.id === payload.id ? { ...m, content: payload.content, edited_at: 'true' } : m
|
||||
));
|
||||
}),
|
||||
sw.realtime.subscribe(channel, 'message.deleted', (payload) => {
|
||||
setMessages(prev => prev.map(m =>
|
||||
m.id === payload.id ? { ...m, _deleted: true, content: '' } : m
|
||||
));
|
||||
}),
|
||||
];
|
||||
|
||||
// Typing indicators
|
||||
var typingTimers = {};
|
||||
unsubs.push(sw.realtime.subscribe(channel, 'typing', (payload) => {
|
||||
var pid = payload.participant_id;
|
||||
if (pid === userId) return;
|
||||
setTypingUsers(prev => ({ ...prev, [pid]: payload.display_name || pid }));
|
||||
clearTimeout(typingTimers[pid]);
|
||||
typingTimers[pid] = setTimeout(() => {
|
||||
setTypingUsers(prev => {
|
||||
var next = { ...prev };
|
||||
delete next[pid];
|
||||
return next;
|
||||
});
|
||||
}, 4000);
|
||||
}));
|
||||
|
||||
return () => {
|
||||
unsubs.forEach(fn => fn());
|
||||
Object.values(typingTimers).forEach(clearTimeout);
|
||||
setTypingUsers({});
|
||||
};
|
||||
}, [conversationId, partMap, userId]);
|
||||
|
||||
// Mark read on first load
|
||||
useEffect(() => {
|
||||
if (!conversationId || messages.length === 0) return;
|
||||
var last = messages[messages.length - 1];
|
||||
if (last && last.id) {
|
||||
api.post('/read/' + conversationId, { last_read_message_id: last.id }).catch(() => {});
|
||||
}
|
||||
}, [conversationId, messages.length > 0]);
|
||||
|
||||
// Edit message
|
||||
function handleEdit(msgId, newContent) {
|
||||
api.put('/messages/' + conversationId + '/' + msgId, { content: newContent }).then(() => {
|
||||
setMessages(prev => prev.map(m =>
|
||||
m.id === msgId ? { ...m, content: newContent, edited_at: 'true' } : m
|
||||
));
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
// Delete message
|
||||
async function handleDelete(msgId) {
|
||||
var ok = await sw.ui.confirm('Delete this message?', { destructive: true });
|
||||
if (!ok) return;
|
||||
api.del('/messages/' + conversationId + '/' + msgId).then(() => {
|
||||
setMessages(prev => prev.map(m =>
|
||||
m.id === msgId ? { ...m, _deleted: true, content: '' } : m
|
||||
));
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
// Typing indicator text
|
||||
var typingNames = Object.values(typingUsers);
|
||||
var typingText = '';
|
||||
if (typingNames.length === 1) typingText = typingNames[0] + ' is typing\u2026';
|
||||
else if (typingNames.length === 2) typingText = typingNames.join(' and ') + ' are typing\u2026';
|
||||
else if (typingNames.length > 2) typingText = 'Several people are typing\u2026';
|
||||
|
||||
if (!conversationId) {
|
||||
return html`
|
||||
<div class="chat-thread chat-thread--empty">
|
||||
<p>Select a conversation or start a new one</p>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
return html`
|
||||
<div class="chat-thread">
|
||||
<div class="chat-thread__messages" ref=${listRef}>
|
||||
${loading && messages.length === 0 && html`<div class="chat-thread__loading"><${Spinner} /></div>`}
|
||||
${loading && messages.length > 0 && html`<div class="chat-thread__loading-more"><${Spinner} size="sm" /></div>`}
|
||||
${hasMore && !loading && html`
|
||||
<button class="chat-thread__load-more" onClick=${loadMore}>
|
||||
Load older messages
|
||||
</button>`}
|
||||
${messages.map(m => html`
|
||||
<${MessageBubble}
|
||||
key=${m.id}
|
||||
msg=${m}
|
||||
isOwn=${m.participant_id === userId}
|
||||
onEdit=${handleEdit}
|
||||
onDelete=${handleDelete}
|
||||
/>`)}
|
||||
<div ref=${bottomRef} />
|
||||
</div>
|
||||
${typingText && html`<div class="chat-thread__typing">${typingText}</div>`}
|
||||
</div>`;
|
||||
}
|
||||
|
||||
|
||||
// ═══════════════════════════════════════════
|
||||
// ComposeBar — message input
|
||||
// ═══════════════════════════════════════════
|
||||
|
||||
function ComposeBar({ conversationId }) {
|
||||
var [text, setText] = useState('');
|
||||
var [sending, setSending] = useState(false);
|
||||
var textareaRef = useRef(null);
|
||||
|
||||
// Reset on conversation change
|
||||
useEffect(() => { setText(''); }, [conversationId]);
|
||||
|
||||
// Auto-focus
|
||||
useEffect(() => {
|
||||
if (textareaRef.current) textareaRef.current.focus();
|
||||
}, [conversationId]);
|
||||
|
||||
// Auto-resize
|
||||
function autoResize(el) {
|
||||
if (!el) return;
|
||||
el.style.height = 'auto';
|
||||
el.style.height = Math.min(el.scrollHeight, 160) + 'px';
|
||||
}
|
||||
|
||||
function handleInput(e) {
|
||||
setText(e.target.value);
|
||||
autoResize(e.target);
|
||||
// Typing indicator (debounced)
|
||||
debounce('typing', () => {
|
||||
chatApi.post('/typing/' + conversationId, {
|
||||
display_name: currentDisplayName(),
|
||||
}).catch(() => {});
|
||||
}, 3000);
|
||||
}
|
||||
|
||||
async function handleSend() {
|
||||
var content = text.trim();
|
||||
if (!content || sending) return;
|
||||
setSending(true);
|
||||
try {
|
||||
await api.post('/messages/' + conversationId, { content: content, content_type: 'text' });
|
||||
setText('');
|
||||
if (textareaRef.current) {
|
||||
textareaRef.current.style.height = 'auto';
|
||||
textareaRef.current.focus();
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('[chat] send failed:', e);
|
||||
}
|
||||
setSending(false);
|
||||
}
|
||||
|
||||
function onKeyDown(e) {
|
||||
if (e.key === 'Enter' && !e.shiftKey) {
|
||||
e.preventDefault();
|
||||
handleSend();
|
||||
}
|
||||
}
|
||||
|
||||
if (!conversationId) return null;
|
||||
|
||||
return html`
|
||||
<div class="chat-compose">
|
||||
<textarea class="chat-compose__input"
|
||||
ref=${textareaRef}
|
||||
value=${text}
|
||||
placeholder="Type a message\u2026"
|
||||
rows="1"
|
||||
onInput=${handleInput}
|
||||
onKeyDown=${onKeyDown} />
|
||||
<${Button} onClick=${handleSend} disabled=${!text.trim() || sending}>Send<//>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
|
||||
// ═══════════════════════════════════════════
|
||||
// ParticipantSidebar — right panel
|
||||
// ═══════════════════════════════════════════
|
||||
|
||||
function ParticipantSidebar({ conversationId, participants, onRefresh, isAdmin }) {
|
||||
var [addOpen, setAddOpen] = useState(false);
|
||||
var [presence, setPresence] = useState({});
|
||||
|
||||
// Query presence
|
||||
useEffect(() => {
|
||||
if (!participants || participants.length === 0) return;
|
||||
var ids = participants.map(p => p.participant_id).join(',');
|
||||
sw.api.get('/api/v1/presence?users=' + ids).then(res => {
|
||||
setPresence(res || {});
|
||||
}).catch(() => {});
|
||||
}, [participants]);
|
||||
|
||||
async function addUser(user) {
|
||||
setAddOpen(false);
|
||||
await api.post('/participants/' + conversationId, {
|
||||
participant_id: user.id,
|
||||
display_name: user.display_name || user.username,
|
||||
}).catch(() => {});
|
||||
onRefresh();
|
||||
}
|
||||
|
||||
async function removeUser(pid) {
|
||||
var ok = await sw.ui.confirm('Remove this participant?', { destructive: true });
|
||||
if (!ok) return;
|
||||
await api.del('/participants/' + conversationId + '/' + pid).catch(() => {});
|
||||
onRefresh();
|
||||
}
|
||||
|
||||
return html`
|
||||
<div class="chat-participants">
|
||||
<div class="chat-participants__header">
|
||||
<span>Participants (${(participants || []).length})</span>
|
||||
${isAdmin && html`<${Button} size="sm" onClick=${() => setAddOpen(true)}>Add<//>` }
|
||||
</div>
|
||||
<div class="chat-participants__list">
|
||||
${(participants || []).map(p => html`
|
||||
<div key=${p.participant_id} class="chat-participants__item">
|
||||
<${Avatar} name=${p.display_name || p.participant_id} size="sm" />
|
||||
<span class="chat-participants__name">
|
||||
${p.display_name || p.participant_id}
|
||||
${p.role === 'admin' && html`<span class="chat-participants__badge">admin</span>`}
|
||||
</span>
|
||||
<span class=${'chat-participants__status' + (presence[p.participant_id] ? ' chat-participants__status--online' : '')} />
|
||||
${isAdmin && p.participant_id !== currentUserId() && html`
|
||||
<button class="chat-participants__remove" onClick=${() => removeUser(p.participant_id)} title="Remove">\u00d7</button>`}
|
||||
</div>`)}
|
||||
</div>
|
||||
|
||||
<${Dialog} open=${addOpen} title="Add Participant" onClose=${() => setAddOpen(false)}>
|
||||
<${UserPicker} onSelect=${addUser} placeholder="Search users\u2026" autoFocus />
|
||||
<//>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
|
||||
// ═══════════════════════════════════════════
|
||||
// NewConversationDialog
|
||||
// ═══════════════════════════════════════════
|
||||
|
||||
function NewConversationDialog({ open, onClose, onCreated }) {
|
||||
var [type, setType] = useState('group');
|
||||
var [title, setTitle] = useState('');
|
||||
var [selected, setSelected] = useState([]);
|
||||
var [creating, setCreating] = useState(false);
|
||||
|
||||
function reset() {
|
||||
setType('group');
|
||||
setTitle('');
|
||||
setSelected([]);
|
||||
setCreating(false);
|
||||
}
|
||||
|
||||
function addUser(user) {
|
||||
// Avoid duplicates
|
||||
if (selected.find(u => u.id === user.id)) return;
|
||||
// For DM, only allow one user
|
||||
if (type === 'direct' && selected.length >= 1) {
|
||||
setSelected([user]);
|
||||
return;
|
||||
}
|
||||
setSelected(prev => [...prev, user]);
|
||||
}
|
||||
|
||||
function removeSelected(id) {
|
||||
setSelected(prev => prev.filter(u => u.id !== id));
|
||||
}
|
||||
|
||||
async function handleCreate() {
|
||||
if (selected.length === 0) return;
|
||||
setCreating(true);
|
||||
try {
|
||||
var convTitle = type === 'direct'
|
||||
? ''
|
||||
: (title.trim() || selected.map(u => u.display_name || u.username).join(', '));
|
||||
var participants = selected.map(u => ({
|
||||
id: u.id,
|
||||
display_name: u.display_name || u.username,
|
||||
}));
|
||||
var res = await api.post('/conversations', {
|
||||
title: convTitle,
|
||||
type: type,
|
||||
participants: participants,
|
||||
creator_display_name: currentDisplayName(),
|
||||
});
|
||||
reset();
|
||||
onClose();
|
||||
onCreated(res.id || res.data?.id);
|
||||
} catch (e) {
|
||||
console.error('[chat] create failed:', e);
|
||||
}
|
||||
setCreating(false);
|
||||
}
|
||||
|
||||
var actions = [
|
||||
{ label: 'Cancel', variant: 'secondary', onClick: () => { reset(); onClose(); } },
|
||||
{ label: 'Create', variant: 'primary', onClick: handleCreate, disabled: creating || selected.length === 0 },
|
||||
];
|
||||
|
||||
return html`
|
||||
<${Dialog} open=${open} title="New Conversation" onClose=${() => { reset(); onClose(); }} actions=${actions}>
|
||||
<div class="chat-new">
|
||||
<div class="chat-new__type">
|
||||
<label>
|
||||
<input type="radio" name="convType" value="group"
|
||||
checked=${type === 'group'} onChange=${() => { setType('group'); setSelected([]); }} />
|
||||
Group
|
||||
</label>
|
||||
<label>
|
||||
<input type="radio" name="convType" value="direct"
|
||||
checked=${type === 'direct'} onChange=${() => { setType('direct'); setSelected([]); }} />
|
||||
Direct Message
|
||||
</label>
|
||||
</div>
|
||||
${type === 'group' && html`
|
||||
<input class="chat-new__title" type="text" value=${title}
|
||||
placeholder="Conversation title (optional)"
|
||||
onInput=${e => setTitle(e.target.value)} />`}
|
||||
<${UserPicker} onSelect=${addUser} placeholder=${type === 'direct' ? 'Search for a user\u2026' : 'Add participants\u2026'} />
|
||||
${selected.length > 0 && html`
|
||||
<div class="chat-new__selected">
|
||||
${selected.map(u => html`
|
||||
<span key=${u.id} class="chat-new__chip">
|
||||
${u.display_name || u.username}
|
||||
<button onClick=${() => removeSelected(u.id)}>\u00d7</button>
|
||||
</span>`)}
|
||||
</div>`}
|
||||
</div>
|
||||
<//>`;
|
||||
}
|
||||
|
||||
|
||||
// ═══════════════════════════════════════════
|
||||
// ChatApp — root component
|
||||
// ═══════════════════════════════════════════
|
||||
|
||||
function ChatApp() {
|
||||
var [conversations, setConversations] = useState([]);
|
||||
var [unread, setUnread] = useState({});
|
||||
var [selectedId, setSelectedId] = useState(null);
|
||||
var [participants, setParticipants] = useState([]);
|
||||
var [showParticipants, setShowParticipants] = useState(false);
|
||||
var [showNew, setShowNew] = useState(false);
|
||||
var [loading, setLoading] = useState(true);
|
||||
var userId = currentUserId();
|
||||
|
||||
// Load conversations
|
||||
// Note: SDK auto-unwraps {data: [...]} envelopes → res is the array directly
|
||||
function loadConversations() {
|
||||
return api.get('/conversations').then(res => {
|
||||
setConversations(Array.isArray(res) ? res : (res && res.data) || []);
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
// Load unread
|
||||
// Note: SDK returns {data: {id: count}} — NOT auto-unwrapped (data is object, not array)
|
||||
function loadUnread() {
|
||||
return api.get('/unread').then(res => {
|
||||
setUnread((res && res.data) || (typeof res === 'object' && !Array.isArray(res) ? res : {}));
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
// Initial load
|
||||
useEffect(() => {
|
||||
Promise.all([loadConversations(), loadUnread()]).then(() => setLoading(false));
|
||||
}, []);
|
||||
|
||||
// Load participants when conversation changes
|
||||
useEffect(() => {
|
||||
if (!selectedId) { setParticipants([]); return; }
|
||||
api.get('/participants/' + selectedId).then(res => {
|
||||
setParticipants(Array.isArray(res) ? res : (res && res.data) || []);
|
||||
}).catch(() => {});
|
||||
}, [selectedId]);
|
||||
|
||||
// Realtime: refresh conversation list on activity
|
||||
useEffect(() => {
|
||||
// Subscribe to all active conversations
|
||||
var unsubs = [];
|
||||
conversations.forEach(c => {
|
||||
unsubs.push(sw.realtime.subscribe('conversation:' + c.id, 'message', () => {
|
||||
loadConversations();
|
||||
loadUnread();
|
||||
}));
|
||||
unsubs.push(sw.realtime.subscribe('conversation:' + c.id, 'participant.added', () => {
|
||||
loadConversations();
|
||||
if (c.id === selectedId) {
|
||||
api.get('/participants/' + selectedId).then(res => {
|
||||
setParticipants(Array.isArray(res) ? res : (res && res.data) || []);
|
||||
}).catch(() => {});
|
||||
}
|
||||
}));
|
||||
unsubs.push(sw.realtime.subscribe('conversation:' + c.id, 'participant.removed', () => {
|
||||
loadConversations();
|
||||
if (c.id === selectedId) {
|
||||
api.get('/participants/' + selectedId).then(res => {
|
||||
setParticipants(Array.isArray(res) ? res : (res && res.data) || []);
|
||||
}).catch(() => {});
|
||||
}
|
||||
}));
|
||||
});
|
||||
return () => unsubs.forEach(fn => fn());
|
||||
}, [conversations.map(c => c.id).join(','), selectedId]);
|
||||
|
||||
// Select conversation
|
||||
function selectConversation(id) {
|
||||
setSelectedId(id);
|
||||
setShowParticipants(false);
|
||||
// Mark read
|
||||
setUnread(prev => { var next = { ...prev }; delete next[id]; return next; });
|
||||
}
|
||||
|
||||
// After creating a new conversation
|
||||
function onCreated(id) {
|
||||
loadConversations().then(() => {
|
||||
if (id) setSelectedId(id);
|
||||
});
|
||||
}
|
||||
|
||||
// Is current user admin in selected conversation?
|
||||
var isAdmin = useMemo(() => {
|
||||
return participants.some(p => p.participant_id === userId && p.role === 'admin');
|
||||
}, [participants, userId]);
|
||||
|
||||
// Refresh participants
|
||||
function refreshParticipants() {
|
||||
if (!selectedId) return;
|
||||
api.get('/participants/' + selectedId).then(res => {
|
||||
setParticipants(Array.isArray(res) ? res : (res && res.data) || []);
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
// Conversation title for topbar
|
||||
var selectedConv = conversations.find(c => c.id === selectedId);
|
||||
var threadTitle = selectedConv ? (selectedConv.title || 'Direct Message') : '';
|
||||
|
||||
if (loading) {
|
||||
return html`<div class="chat-loading"><${Spinner} /></div>`;
|
||||
}
|
||||
|
||||
return html`
|
||||
<div class="chat-app">
|
||||
<${Topbar} title="Chat">
|
||||
${selectedId && html`
|
||||
<span class="chat-topbar__thread-title">${threadTitle}</span>
|
||||
<${Button} size="sm" variant="secondary"
|
||||
onClick=${() => setShowParticipants(!showParticipants)}>
|
||||
${showParticipants ? 'Hide' : 'People'}
|
||||
<//>`}
|
||||
<//>
|
||||
<div class="chat-body">
|
||||
<${ConversationList}
|
||||
selected=${selectedId}
|
||||
onSelect=${selectConversation}
|
||||
onNew=${() => setShowNew(true)}
|
||||
conversations=${conversations}
|
||||
unread=${unread} />
|
||||
<div class="chat-main">
|
||||
<${MessageThread}
|
||||
conversationId=${selectedId}
|
||||
participants=${participants} />
|
||||
<${ComposeBar} conversationId=${selectedId} />
|
||||
</div>
|
||||
${showParticipants && selectedId && html`
|
||||
<${ParticipantSidebar}
|
||||
conversationId=${selectedId}
|
||||
participants=${participants}
|
||||
onRefresh=${refreshParticipants}
|
||||
isAdmin=${isAdmin} />`}
|
||||
</div>
|
||||
<${NewConversationDialog}
|
||||
open=${showNew}
|
||||
onClose=${() => setShowNew(false)}
|
||||
onCreated=${onCreated} />
|
||||
</div>`;
|
||||
}
|
||||
|
||||
|
||||
// ── Mount ──────────────────────────────────
|
||||
render(html`<${ChatApp} />`, mount);
|
||||
|
||||
})();
|
||||
30
packages/chat/manifest.json
Normal file
30
packages/chat/manifest.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"id": "chat",
|
||||
"title": "Chat",
|
||||
"type": "full",
|
||||
"tier": "starlark",
|
||||
"route": "/s/chat",
|
||||
"auth": "authenticated",
|
||||
"layout": "single",
|
||||
"version": "0.2.0",
|
||||
"icon": "\ud83d\udcac",
|
||||
"description": "Chat surface — conversations, messaging, typing indicators, read receipts.",
|
||||
"author": "switchboard",
|
||||
|
||||
"permissions": ["db.write", "realtime.publish"],
|
||||
|
||||
"depends": ["chat-core"],
|
||||
|
||||
"api_routes": [
|
||||
{"method": "POST", "path": "/typing/*"}
|
||||
],
|
||||
|
||||
"settings": {
|
||||
"enter_to_send": {
|
||||
"type": "boolean",
|
||||
"label": "Enter to Send",
|
||||
"description": "Press Enter to send messages (Shift+Enter for newline). When off, Enter inserts a newline.",
|
||||
"default": true
|
||||
}
|
||||
}
|
||||
}
|
||||
39
packages/chat/script.star
Normal file
39
packages/chat/script.star
Normal file
@@ -0,0 +1,39 @@
|
||||
# Chat Surface — Starlark Backend (v0.1.0)
|
||||
#
|
||||
# Thin surface layer on top of chat-core library.
|
||||
# Only handles typing indicator broadcast — all CRUD is
|
||||
# delegated to chat-core's own API routes.
|
||||
#
|
||||
# Modules: json, realtime
|
||||
|
||||
def _resp(status, data):
|
||||
return {"status": status, "body": json.encode(data), "headers": {"Content-Type": "application/json"}}
|
||||
|
||||
|
||||
def on_request(req):
|
||||
path = req["path"]
|
||||
method = req["method"]
|
||||
user_id = req.get("user_id", "")
|
||||
|
||||
# POST /typing/:conversation_id — broadcast typing indicator
|
||||
if method == "POST" and path.startswith("/typing/"):
|
||||
cid = path[len("/typing/"):]
|
||||
if not cid:
|
||||
return _resp(400, {"error": "conversation_id required"})
|
||||
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
display_name = str(body.get("display_name", ""))
|
||||
|
||||
realtime.publish(
|
||||
"conversation:" + cid,
|
||||
"typing",
|
||||
{
|
||||
"participant_id": user_id,
|
||||
"display_name": display_name,
|
||||
"conversation_id": cid,
|
||||
},
|
||||
)
|
||||
|
||||
return _resp(200, {"ok": True})
|
||||
|
||||
return _resp(404, {"error": "not found"})
|
||||
@@ -3,136 +3,20 @@
|
||||
// ==========================================
|
||||
// Renders ```csv and ```tsv code blocks as sortable HTML tables.
|
||||
// No external dependencies.
|
||||
//
|
||||
// Registers with sw.renderers via the sw:ready event.
|
||||
// ==========================================
|
||||
|
||||
Extensions.register({
|
||||
id: 'csv-table',
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
async init(ctx) {
|
||||
const self = this;
|
||||
function _escapeHtml(str) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
// ── Inject styles ──
|
||||
this._injectStyles();
|
||||
|
||||
ctx.renderers.register('csv-block', {
|
||||
type: 'block',
|
||||
priority: 10,
|
||||
match(lang) {
|
||||
const l = (lang || '').toLowerCase();
|
||||
return l === 'csv' || l === 'tsv';
|
||||
},
|
||||
render(lang, code, container) {
|
||||
const delimiter = lang.toLowerCase() === 'tsv' ? '\t' : ',';
|
||||
const rows = self._parseCSV(code.trim(), delimiter);
|
||||
|
||||
if (rows.length === 0) {
|
||||
container.innerHTML = '<div class="csv-empty">No data</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
const headers = rows[0];
|
||||
const data = rows.slice(1);
|
||||
const tableId = 'csv-' + Math.random().toString(36).slice(2, 9);
|
||||
|
||||
container.innerHTML = `
|
||||
<div class="csv-table-block ext-rendered" data-csv-id="${tableId}">
|
||||
<div class="csv-toolbar">
|
||||
<span class="csv-info">${data.length} row${data.length !== 1 ? 's' : ''} × ${headers.length} col${headers.length !== 1 ? 's' : ''}</span>
|
||||
<button class="csv-copy-btn" title="Copy as CSV">📋 Copy</button>
|
||||
</div>
|
||||
<div class="csv-table-scroll">
|
||||
<table class="csv-table">
|
||||
<thead>
|
||||
<tr>${headers.map((h, i) => `<th data-col="${i}" title="Click to sort">${self._escapeHtml(h)}<span class="csv-sort-icon"></span></th>`).join('')}</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
${data.map(row => `<tr>${row.map(cell => `<td>${self._escapeHtml(cell)}</td>`).join('')}</tr>`).join('')}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<details class="csv-source">
|
||||
<summary>📄 View raw</summary>
|
||||
<pre><code>${self._escapeHtml(code.trim())}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// Attach sort handlers + copy
|
||||
self._attachHandlers(container, tableId, code.trim());
|
||||
}
|
||||
});
|
||||
},
|
||||
|
||||
_attachHandlers(container, tableId, rawCSV) {
|
||||
const block = container.querySelector(`[data-csv-id="${tableId}"]`);
|
||||
if (!block) return;
|
||||
|
||||
// ── Column sorting ──
|
||||
const ths = block.querySelectorAll('th[data-col]');
|
||||
let sortCol = -1;
|
||||
let sortAsc = true;
|
||||
|
||||
ths.forEach(th => {
|
||||
th.style.cursor = 'pointer';
|
||||
th.addEventListener('click', () => {
|
||||
const col = parseInt(th.dataset.col, 10);
|
||||
if (sortCol === col) {
|
||||
sortAsc = !sortAsc;
|
||||
} else {
|
||||
sortCol = col;
|
||||
sortAsc = true;
|
||||
}
|
||||
|
||||
const tbody = block.querySelector('tbody');
|
||||
const rows = Array.from(tbody.querySelectorAll('tr'));
|
||||
|
||||
rows.sort((a, b) => {
|
||||
const cellA = (a.children[col]?.textContent || '').trim();
|
||||
const cellB = (b.children[col]?.textContent || '').trim();
|
||||
const numA = parseFloat(cellA);
|
||||
const numB = parseFloat(cellB);
|
||||
|
||||
// Numeric sort if both are numbers
|
||||
if (!isNaN(numA) && !isNaN(numB)) {
|
||||
return sortAsc ? numA - numB : numB - numA;
|
||||
}
|
||||
// String sort
|
||||
const cmp = cellA.localeCompare(cellB, undefined, { numeric: true, sensitivity: 'base' });
|
||||
return sortAsc ? cmp : -cmp;
|
||||
});
|
||||
|
||||
rows.forEach(row => tbody.appendChild(row));
|
||||
|
||||
// Update sort icons
|
||||
ths.forEach(h => {
|
||||
const icon = h.querySelector('.csv-sort-icon');
|
||||
if (icon) {
|
||||
const hCol = parseInt(h.dataset.col, 10);
|
||||
icon.textContent = hCol === sortCol ? (sortAsc ? ' ▲' : ' ▼') : '';
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ── Copy button ──
|
||||
const copyBtn = block.querySelector('.csv-copy-btn');
|
||||
if (copyBtn) {
|
||||
copyBtn.addEventListener('click', () => {
|
||||
navigator.clipboard.writeText(rawCSV).then(() => {
|
||||
copyBtn.textContent = '✓ Copied';
|
||||
setTimeout(() => { copyBtn.textContent = '📋 Copy'; }, 1500);
|
||||
}).catch(() => {
|
||||
copyBtn.textContent = '✗ Failed';
|
||||
setTimeout(() => { copyBtn.textContent = '📋 Copy'; }, 1500);
|
||||
});
|
||||
});
|
||||
}
|
||||
},
|
||||
|
||||
/**
|
||||
* Simple CSV parser that handles quoted fields.
|
||||
*/
|
||||
_parseCSV(text, delimiter) {
|
||||
function _parseCSV(text, delimiter) {
|
||||
const rows = [];
|
||||
let row = [];
|
||||
let field = '';
|
||||
@@ -141,64 +25,98 @@ Extensions.register({
|
||||
|
||||
while (i < text.length) {
|
||||
const ch = text[i];
|
||||
|
||||
if (inQuotes) {
|
||||
if (ch === '"') {
|
||||
// Peek ahead: escaped quote or end of quoted field
|
||||
if (i + 1 < text.length && text[i + 1] === '"') {
|
||||
field += '"';
|
||||
i += 2;
|
||||
field += '"'; i += 2;
|
||||
} else {
|
||||
inQuotes = false;
|
||||
i++;
|
||||
inQuotes = false; i++;
|
||||
}
|
||||
} else {
|
||||
field += ch;
|
||||
i++;
|
||||
field += ch; i++;
|
||||
}
|
||||
} else {
|
||||
if (ch === '"' && field === '') {
|
||||
inQuotes = true;
|
||||
i++;
|
||||
inQuotes = true; i++;
|
||||
} else if (ch === delimiter) {
|
||||
row.push(field.trim());
|
||||
field = '';
|
||||
i++;
|
||||
row.push(field.trim()); field = ''; i++;
|
||||
} else if (ch === '\n' || (ch === '\r' && text[i + 1] === '\n')) {
|
||||
row.push(field.trim());
|
||||
if (row.some(cell => cell !== '')) rows.push(row);
|
||||
row = [];
|
||||
field = '';
|
||||
row = []; field = '';
|
||||
i += (ch === '\r') ? 2 : 1;
|
||||
} else {
|
||||
field += ch;
|
||||
i++;
|
||||
field += ch; i++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Last field/row
|
||||
row.push(field.trim());
|
||||
if (row.some(cell => cell !== '')) rows.push(row);
|
||||
|
||||
// Normalize: pad short rows to header length
|
||||
if (rows.length > 0) {
|
||||
const maxCols = Math.max(...rows.map(r => r.length));
|
||||
rows.forEach(r => {
|
||||
while (r.length < maxCols) r.push('');
|
||||
rows.forEach(r => { while (r.length < maxCols) r.push(''); });
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
function _attachHandlers(container, tableId, rawCSV) {
|
||||
const block = container.querySelector(`[data-csv-id="${tableId}"]`);
|
||||
if (!block) return;
|
||||
|
||||
const ths = block.querySelectorAll('th[data-col]');
|
||||
let sortCol = -1;
|
||||
let sortAsc = true;
|
||||
|
||||
ths.forEach(th => {
|
||||
th.style.cursor = 'pointer';
|
||||
th.addEventListener('click', () => {
|
||||
const col = parseInt(th.dataset.col, 10);
|
||||
if (sortCol === col) { sortAsc = !sortAsc; }
|
||||
else { sortCol = col; sortAsc = true; }
|
||||
|
||||
const tbody = block.querySelector('tbody');
|
||||
const rows = Array.from(tbody.querySelectorAll('tr'));
|
||||
rows.sort((a, b) => {
|
||||
const cellA = (a.children[col]?.textContent || '').trim();
|
||||
const cellB = (b.children[col]?.textContent || '').trim();
|
||||
const numA = parseFloat(cellA);
|
||||
const numB = parseFloat(cellB);
|
||||
if (!isNaN(numA) && !isNaN(numB))
|
||||
return sortAsc ? numA - numB : numB - numA;
|
||||
const cmp = cellA.localeCompare(cellB, undefined, { numeric: true, sensitivity: 'base' });
|
||||
return sortAsc ? cmp : -cmp;
|
||||
});
|
||||
rows.forEach(row => tbody.appendChild(row));
|
||||
|
||||
ths.forEach(h => {
|
||||
const icon = h.querySelector('.csv-sort-icon');
|
||||
if (icon) {
|
||||
const hCol = parseInt(h.dataset.col, 10);
|
||||
icon.textContent = hCol === sortCol ? (sortAsc ? ' \u25b2' : ' \u25bc') : '';
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
const copyBtn = block.querySelector('.csv-copy-btn');
|
||||
if (copyBtn) {
|
||||
copyBtn.addEventListener('click', () => {
|
||||
navigator.clipboard.writeText(rawCSV).then(() => {
|
||||
copyBtn.textContent = '\u2713 Copied';
|
||||
setTimeout(() => { copyBtn.textContent = '\ud83d\udccb Copy'; }, 1500);
|
||||
}).catch(() => {
|
||||
copyBtn.textContent = '\u2717 Failed';
|
||||
setTimeout(() => { copyBtn.textContent = '\ud83d\udccb Copy'; }, 1500);
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return rows;
|
||||
},
|
||||
// ── Styles ──────────────────────────────
|
||||
|
||||
_escapeHtml(str) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
},
|
||||
|
||||
_injectStyles() {
|
||||
function _injectStyles() {
|
||||
if (document.getElementById('ext-style-csv-table')) return;
|
||||
const style = document.createElement('style');
|
||||
style.id = 'ext-style-csv-table';
|
||||
@@ -246,9 +164,66 @@ Extensions.register({
|
||||
}
|
||||
.csv-empty { padding: 16px; text-align: center; color: var(--text-3); }`;
|
||||
document.head.appendChild(style);
|
||||
},
|
||||
|
||||
destroy() {
|
||||
document.getElementById('ext-style-csv-table')?.remove();
|
||||
}
|
||||
});
|
||||
|
||||
// ── Registration ────────────────────────
|
||||
|
||||
function register() {
|
||||
if (!window.sw?.renderers) return;
|
||||
|
||||
_injectStyles();
|
||||
|
||||
sw.renderers.register('csv-block', {
|
||||
type: 'block',
|
||||
priority: 10,
|
||||
match(lang) {
|
||||
const l = (lang || '').toLowerCase();
|
||||
return l === 'csv' || l === 'tsv';
|
||||
},
|
||||
render(lang, code, container) {
|
||||
const delimiter = lang.toLowerCase() === 'tsv' ? '\t' : ',';
|
||||
const rows = _parseCSV(code.trim(), delimiter);
|
||||
|
||||
if (rows.length === 0) {
|
||||
container.innerHTML = '<div class="csv-empty">No data</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
const headers = rows[0];
|
||||
const data = rows.slice(1);
|
||||
const tableId = 'csv-' + Math.random().toString(36).slice(2, 9);
|
||||
|
||||
container.innerHTML = `
|
||||
<div class="csv-table-block ext-rendered" data-csv-id="${tableId}">
|
||||
<div class="csv-toolbar">
|
||||
<span class="csv-info">${data.length} row${data.length !== 1 ? 's' : ''} \u00d7 ${headers.length} col${headers.length !== 1 ? 's' : ''}</span>
|
||||
<button class="csv-copy-btn" title="Copy as CSV">\ud83d\udccb Copy</button>
|
||||
</div>
|
||||
<div class="csv-table-scroll">
|
||||
<table class="csv-table">
|
||||
<thead>
|
||||
<tr>${headers.map((h, i) => `<th data-col="${i}" title="Click to sort">${_escapeHtml(h)}<span class="csv-sort-icon"></span></th>`).join('')}</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
${data.map(row => `<tr>${row.map(cell => `<td>${_escapeHtml(cell)}</td>`).join('')}</tr>`).join('')}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<details class="csv-source">
|
||||
<summary>\ud83d\udcc4 View raw</summary>
|
||||
<pre><code>${_escapeHtml(code.trim())}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
|
||||
_attachHandlers(container, tableId, code.trim());
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
if (window.sw?._sdk) {
|
||||
register();
|
||||
} else {
|
||||
document.addEventListener('sw:ready', register, { once: true });
|
||||
}
|
||||
})();
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"tier": "browser",
|
||||
"author": "switchboard",
|
||||
"description": "Renders ```csv code blocks as sortable, interactive HTML tables",
|
||||
"requires": ["chat"],
|
||||
"requires": [],
|
||||
"permissions": [],
|
||||
"tools": [],
|
||||
"surfaces": [],
|
||||
|
||||
@@ -4,106 +4,22 @@
|
||||
// Renders ```diff code blocks with syntax-highlighted
|
||||
// additions, deletions, hunk headers, and context lines.
|
||||
// No external dependencies.
|
||||
//
|
||||
// Registers with sw.renderers via the sw:ready event.
|
||||
// ==========================================
|
||||
|
||||
Extensions.register({
|
||||
id: 'diff-viewer',
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
async init(ctx) {
|
||||
const self = this;
|
||||
|
||||
// ── Inject styles ──
|
||||
this._injectStyles();
|
||||
|
||||
ctx.renderers.register('diff-block', {
|
||||
type: 'block',
|
||||
priority: 10,
|
||||
pattern: 'diff',
|
||||
render(lang, code, container) {
|
||||
const lines = code.split('\n');
|
||||
let stats = { added: 0, removed: 0 };
|
||||
let currentFile = '';
|
||||
|
||||
const rendered = lines.map(line => {
|
||||
const escaped = self._escapeHtml(line);
|
||||
|
||||
// File headers
|
||||
if (line.startsWith('--- ') || line.startsWith('+++ ')) {
|
||||
if (line.startsWith('+++ ')) {
|
||||
currentFile = line.slice(4).trim();
|
||||
}
|
||||
return `<div class="diff-line diff-file-header">${escaped}</div>`;
|
||||
}
|
||||
|
||||
// Hunk headers
|
||||
if (line.startsWith('@@')) {
|
||||
const hunkMatch = line.match(/^@@\s*-(\d+)(?:,\d+)?\s*\+(\d+)(?:,\d+)?\s*@@\s*(.*)/);
|
||||
const context = hunkMatch ? hunkMatch[3] : '';
|
||||
return `<div class="diff-line diff-hunk">${escaped}</div>`;
|
||||
}
|
||||
|
||||
// Additions
|
||||
if (line.startsWith('+')) {
|
||||
stats.added++;
|
||||
return `<div class="diff-line diff-add"><span class="diff-indicator">+</span>${self._escapeHtml(line.slice(1))}</div>`;
|
||||
}
|
||||
|
||||
// Deletions
|
||||
if (line.startsWith('-')) {
|
||||
stats.removed++;
|
||||
return `<div class="diff-line diff-del"><span class="diff-indicator">-</span>${self._escapeHtml(line.slice(1))}</div>`;
|
||||
}
|
||||
|
||||
// Context (lines starting with space or no prefix)
|
||||
if (line.startsWith(' ')) {
|
||||
return `<div class="diff-line diff-ctx"><span class="diff-indicator"> </span>${self._escapeHtml(line.slice(1))}</div>`;
|
||||
}
|
||||
|
||||
// Other (commit info, etc)
|
||||
return `<div class="diff-line diff-meta">${escaped}</div>`;
|
||||
}).join('');
|
||||
|
||||
const fileLabel = currentFile ? `<span class="diff-filename" title="${self._escapeHtml(currentFile)}">${self._escapeHtml(currentFile)}</span>` : '';
|
||||
|
||||
container.innerHTML = `
|
||||
<div class="diff-block ext-rendered">
|
||||
<div class="diff-toolbar">
|
||||
${fileLabel}
|
||||
<span class="diff-stats">
|
||||
<span class="diff-stat-add">+${stats.added}</span>
|
||||
<span class="diff-stat-del">-${stats.removed}</span>
|
||||
</span>
|
||||
<button class="diff-copy-btn" title="Copy diff">📋 Copy</button>
|
||||
</div>
|
||||
<div class="diff-content">${rendered}</div>
|
||||
<details class="diff-source-toggle">
|
||||
<summary>📝 View raw</summary>
|
||||
<pre><code>${self._escapeHtml(code)}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
|
||||
// Copy handler
|
||||
const copyBtn = container.querySelector('.diff-copy-btn');
|
||||
if (copyBtn) {
|
||||
copyBtn.addEventListener('click', () => {
|
||||
navigator.clipboard.writeText(code).then(() => {
|
||||
copyBtn.textContent = '✓ Copied';
|
||||
setTimeout(() => { copyBtn.textContent = '📋 Copy'; }, 1500);
|
||||
}).catch(() => {});
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
},
|
||||
|
||||
_escapeHtml(str) {
|
||||
function _escapeHtml(str) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
},
|
||||
}
|
||||
|
||||
_injectStyles() {
|
||||
// ── Styles ──────────────────────────────
|
||||
|
||||
function _injectStyles() {
|
||||
if (document.getElementById('ext-style-diff-viewer')) return;
|
||||
const style = document.createElement('style');
|
||||
style.id = 'ext-style-diff-viewer';
|
||||
@@ -158,9 +74,86 @@ Extensions.register({
|
||||
max-height: 200px; overflow: auto;
|
||||
}`;
|
||||
document.head.appendChild(style);
|
||||
},
|
||||
|
||||
destroy() {
|
||||
document.getElementById('ext-style-diff-viewer')?.remove();
|
||||
}
|
||||
});
|
||||
|
||||
// ── Registration ────────────────────────
|
||||
|
||||
function register() {
|
||||
if (!window.sw?.renderers) return;
|
||||
|
||||
_injectStyles();
|
||||
|
||||
sw.renderers.register('diff-block', {
|
||||
type: 'block',
|
||||
priority: 10,
|
||||
pattern: 'diff',
|
||||
render(lang, code, container) {
|
||||
const lines = code.split('\n');
|
||||
let stats = { added: 0, removed: 0 };
|
||||
let currentFile = '';
|
||||
|
||||
const rendered = lines.map(line => {
|
||||
const escaped = _escapeHtml(line);
|
||||
|
||||
if (line.startsWith('--- ') || line.startsWith('+++ ')) {
|
||||
if (line.startsWith('+++ ')) currentFile = line.slice(4).trim();
|
||||
return `<div class="diff-line diff-file-header">${escaped}</div>`;
|
||||
}
|
||||
if (line.startsWith('@@')) {
|
||||
return `<div class="diff-line diff-hunk">${escaped}</div>`;
|
||||
}
|
||||
if (line.startsWith('+')) {
|
||||
stats.added++;
|
||||
return `<div class="diff-line diff-add"><span class="diff-indicator">+</span>${_escapeHtml(line.slice(1))}</div>`;
|
||||
}
|
||||
if (line.startsWith('-')) {
|
||||
stats.removed++;
|
||||
return `<div class="diff-line diff-del"><span class="diff-indicator">-</span>${_escapeHtml(line.slice(1))}</div>`;
|
||||
}
|
||||
if (line.startsWith(' ')) {
|
||||
return `<div class="diff-line diff-ctx"><span class="diff-indicator"> </span>${_escapeHtml(line.slice(1))}</div>`;
|
||||
}
|
||||
return `<div class="diff-line diff-meta">${escaped}</div>`;
|
||||
}).join('');
|
||||
|
||||
const fileLabel = currentFile
|
||||
? `<span class="diff-filename" title="${_escapeHtml(currentFile)}">${_escapeHtml(currentFile)}</span>`
|
||||
: '';
|
||||
|
||||
container.innerHTML = `
|
||||
<div class="diff-block ext-rendered">
|
||||
<div class="diff-toolbar">
|
||||
${fileLabel}
|
||||
<span class="diff-stats">
|
||||
<span class="diff-stat-add">+${stats.added}</span>
|
||||
<span class="diff-stat-del">-${stats.removed}</span>
|
||||
</span>
|
||||
<button class="diff-copy-btn" title="Copy diff">\ud83d\udccb Copy</button>
|
||||
</div>
|
||||
<div class="diff-content">${rendered}</div>
|
||||
<details class="diff-source-toggle">
|
||||
<summary>\ud83d\udcdd View raw</summary>
|
||||
<pre><code>${_escapeHtml(code)}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
|
||||
const copyBtn = container.querySelector('.diff-copy-btn');
|
||||
if (copyBtn) {
|
||||
copyBtn.addEventListener('click', () => {
|
||||
navigator.clipboard.writeText(code).then(() => {
|
||||
copyBtn.textContent = '\u2713 Copied';
|
||||
setTimeout(() => { copyBtn.textContent = '\ud83d\udccb Copy'; }, 1500);
|
||||
}).catch(() => {});
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
if (window.sw?._sdk) {
|
||||
register();
|
||||
} else {
|
||||
document.addEventListener('sw:ready', register, { once: true });
|
||||
}
|
||||
})();
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"tier": "browser",
|
||||
"author": "switchboard",
|
||||
"description": "Renders ```diff code blocks with syntax-highlighted additions, deletions, and context lines",
|
||||
"requires": ["chat"],
|
||||
"requires": [],
|
||||
"permissions": [],
|
||||
"tools": [],
|
||||
"surfaces": [],
|
||||
|
||||
@@ -372,8 +372,8 @@
|
||||
// ── File Operations ─────────────────────────
|
||||
|
||||
async function _deleteFile(wsId, path, fileTree, codeEditor) {
|
||||
const ok = typeof showConfirm === 'function'
|
||||
? await showConfirm('Delete ' + path + '?')
|
||||
const ok = typeof sw !== 'undefined' && sw.confirm
|
||||
? await sw.confirm('Delete ' + path + '?', { destructive: true })
|
||||
: window.confirm('Delete ' + path + '?');
|
||||
if (!ok) return;
|
||||
try {
|
||||
|
||||
@@ -65,9 +65,9 @@
|
||||
T.assert(d.version === '0.31.1', 'version mismatch');
|
||||
});
|
||||
|
||||
await T.test('crud', 'dashboardPackage', 'dashpkg: GET /admin/surfaces (dashboard in list)', async function () {
|
||||
var d = await T.apiGet('/admin/surfaces');
|
||||
T.assertHasKey(d, 'data', '/admin/surfaces');
|
||||
await T.test('crud', 'dashboardPackage', 'dashpkg: GET /admin/packages (dashboard in list)', async function () {
|
||||
var d = await T.apiGet('/admin/packages');
|
||||
T.assertHasKey(d, 'data', '/admin/packages');
|
||||
var found = d.data.some(function (s) { return s.id === 'dashboard'; });
|
||||
T.assert(found, 'dashboard package should appear in surfaces list');
|
||||
});
|
||||
|
||||
@@ -68,9 +68,9 @@
|
||||
T.assert(d.version === '0.31.0', 'version mismatch');
|
||||
});
|
||||
|
||||
await T.test('crud', 'editorPackage', 'edpkg: GET /admin/surfaces (editor in list)', async function () {
|
||||
var d = await T.apiGet('/admin/surfaces');
|
||||
T.assertHasKey(d, 'data', '/admin/surfaces');
|
||||
await T.test('crud', 'editorPackage', 'edpkg: GET /admin/packages (editor in list)', async function () {
|
||||
var d = await T.apiGet('/admin/packages');
|
||||
T.assertHasKey(d, 'data', '/admin/packages');
|
||||
var found = d.data.some(function (s) { return s.id === 'editor'; });
|
||||
T.assert(found, 'editor package should appear in surfaces list');
|
||||
});
|
||||
@@ -78,7 +78,7 @@
|
||||
await T.test('crud', 'editorPackage', 'edpkg: core /editor removed (404)', async function () {
|
||||
// The old /editor route should no longer exist — it was removed in CS1.
|
||||
// We test by checking the surfaces list for a core editor entry.
|
||||
var d = await T.apiGet('/admin/surfaces');
|
||||
var d = await T.apiGet('/admin/packages');
|
||||
var surfaces = d.data || [];
|
||||
var coreEditor = surfaces.find(function (s) { return s.id === 'editor' && s.source === 'core'; });
|
||||
T.assert(!coreEditor, 'no core editor surface should exist — editor is now a package');
|
||||
@@ -139,7 +139,7 @@
|
||||
|
||||
await T.test('crud', 'editorPackage', 'edpkg: editor in extension nav items', async function () {
|
||||
// When installed, editor should appear in the surfaces list as an extension surface
|
||||
var d = await T.apiGet('/admin/surfaces');
|
||||
var d = await T.apiGet('/admin/packages');
|
||||
var surfaces = d.data || [];
|
||||
var editorSurface = surfaces.find(function (s) { return s.id === 'editor'; });
|
||||
T.assert(editorSurface, 'editor should be in surfaces list');
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
|
||||
var surfaceId = null;
|
||||
|
||||
await T.test('crud', 'surfaces', 'POST /admin/surfaces/install (happy path)', async function () {
|
||||
await T.test('crud', 'surfaces', 'POST /admin/packages/install (happy path)', async function () {
|
||||
surfaceId = 'icd-test-' + Date.now();
|
||||
var manifest = JSON.stringify({
|
||||
id: surfaceId,
|
||||
@@ -28,32 +28,32 @@
|
||||
route: '/s/' + surfaceId
|
||||
});
|
||||
var zipBlob = buildTestZip({ 'manifest.json': manifest });
|
||||
await T.apiUpload('/admin/surfaces/install', zipBlob, surfaceId + '.surface');
|
||||
await T.apiUpload('/admin/packages/install', zipBlob, surfaceId + '.surface');
|
||||
|
||||
// Verify it exists
|
||||
var d = await T.apiGet('/admin/surfaces/' + surfaceId);
|
||||
var d = await T.apiGet('/admin/packages/' + surfaceId);
|
||||
T.assert(d.id === surfaceId, 'installed surface id mismatch');
|
||||
T.assert(d.title === 'ICD Test Surface', 'title mismatch');
|
||||
T.registerCleanup(async function () {
|
||||
if (surfaceId) {
|
||||
var token = await T.getAuthToken();
|
||||
return T.authFetch(token, 'DELETE', '/admin/surfaces/' + surfaceId);
|
||||
return T.authFetch(token, 'DELETE', '/admin/packages/' + surfaceId);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
if (surfaceId) {
|
||||
await T.test('crud', 'surfaces', 'GET /admin/surfaces/:id (read)', async function () {
|
||||
var d = await T.apiGet('/admin/surfaces/' + surfaceId);
|
||||
await T.test('crud', 'surfaces', 'GET /admin/packages/:id (read)', async function () {
|
||||
var d = await T.apiGet('/admin/packages/' + surfaceId);
|
||||
T.assertShape(d, T.S.surfaceAdmin, 'surface detail');
|
||||
T.assert(d.id === surfaceId, 'id mismatch');
|
||||
T.assert(d.source === 'extension', 'source should be extension');
|
||||
T.assert(d.enabled === true, 'newly installed surface should be enabled');
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'GET /admin/surfaces (list includes new)', async function () {
|
||||
var d = await T.apiGet('/admin/surfaces');
|
||||
T.assertHasKey(d, 'data', '/admin/surfaces');
|
||||
await T.test('crud', 'surfaces', 'GET /admin/packages (list includes new)', async function () {
|
||||
var d = await T.apiGet('/admin/packages');
|
||||
T.assertHasKey(d, 'data', '/admin/packages');
|
||||
var found = d.data.some(function (s) { return s.id === surfaceId; });
|
||||
T.assert(found, 'installed surface should appear in admin list');
|
||||
});
|
||||
@@ -70,9 +70,9 @@
|
||||
T.assert(entry.enabled === undefined, 'nav entry should NOT have enabled');
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/surfaces/:id/disable', async function () {
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/packages/:id/disable', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'PUT',
|
||||
'/admin/surfaces/' + surfaceId + '/disable');
|
||||
'/admin/packages/' + surfaceId + '/disable');
|
||||
T.assertStatus(d, 200, 'disable');
|
||||
T.assert(d.enabled === false, 'should report disabled');
|
||||
});
|
||||
@@ -83,16 +83,16 @@
|
||||
T.assert(!found, 'disabled surface should NOT appear in user nav list');
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'GET /admin/surfaces (disabled still in admin)', async function () {
|
||||
var d = await T.apiGet('/admin/surfaces');
|
||||
await T.test('crud', 'surfaces', 'GET /admin/packages (disabled still in admin)', async function () {
|
||||
var d = await T.apiGet('/admin/packages');
|
||||
var entry = d.data.find(function (s) { return s.id === surfaceId; });
|
||||
T.assert(entry, 'disabled surface should still appear in admin list');
|
||||
T.assert(entry.enabled === false, 'should show as disabled in admin list');
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/surfaces/:id/enable', async function () {
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/packages/:id/enable', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'PUT',
|
||||
'/admin/surfaces/' + surfaceId + '/enable');
|
||||
'/admin/packages/' + surfaceId + '/enable');
|
||||
T.assertStatus(d, 200, 'enable');
|
||||
T.assert(d.enabled === true, 'should report enabled');
|
||||
});
|
||||
@@ -103,53 +103,53 @@
|
||||
T.assert(found, 're-enabled surface should appear in user nav list');
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/surfaces/chat/disable (reject → 400)', async function () {
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/packages/chat/disable (reject → 400)', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'PUT',
|
||||
'/admin/surfaces/chat/disable');
|
||||
'/admin/packages/chat/disable');
|
||||
T.assert(d._status === 400, 'disabling chat should return 400, got ' + d._status);
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/surfaces/admin/disable (reject → 400)', async function () {
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/packages/admin/disable (reject → 400)', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'PUT',
|
||||
'/admin/surfaces/admin/disable');
|
||||
'/admin/packages/admin/disable');
|
||||
T.assert(d._status === 400, 'disabling admin should return 400, got ' + d._status);
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'DELETE /admin/surfaces/chat (core reject → 400)', async function () {
|
||||
await T.test('crud', 'surfaces', 'DELETE /admin/packages/chat (core reject → 400)', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'DELETE',
|
||||
'/admin/surfaces/chat');
|
||||
'/admin/packages/chat');
|
||||
T.assert(d._status === 400, 'deleting core surface should return 400, got ' + d._status);
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'GET /admin/surfaces/:id (not found → 404)', async function () {
|
||||
await T.test('crud', 'surfaces', 'GET /admin/packages/:id (not found → 404)', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'GET',
|
||||
'/admin/surfaces/nonexistent-surface-id');
|
||||
'/admin/packages/nonexistent-surface-id');
|
||||
T.assert(d._status === 404, 'nonexistent should return 404, got ' + d._status);
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/surfaces/nonexistent/disable (404)', async function () {
|
||||
await T.test('crud', 'surfaces', 'PUT /admin/packages/nonexistent/disable (404)', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'PUT',
|
||||
'/admin/surfaces/nonexistent-surface-id/disable');
|
||||
'/admin/packages/nonexistent-surface-id/disable');
|
||||
T.assert(d._status === 404, 'disable nonexistent should return 404, got ' + d._status);
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'POST /admin/surfaces/install (missing manifest → 400)', async function () {
|
||||
await T.test('crud', 'surfaces', 'POST /admin/packages/install (missing manifest → 400)', async function () {
|
||||
var zipBlob = buildTestZip({ 'readme.txt': 'no manifest here' });
|
||||
var d;
|
||||
try {
|
||||
d = await T.apiUpload('/admin/surfaces/install', zipBlob, 'bad.surface');
|
||||
d = await T.apiUpload('/admin/packages/install', zipBlob, 'bad.surface');
|
||||
T.assert(false, 'should have thrown');
|
||||
} catch (e) {
|
||||
T.assert(e.message.indexOf('400') !== -1, 'missing manifest should 400: ' + e.message);
|
||||
}
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'POST /admin/surfaces/install (core conflict → 409)', async function () {
|
||||
await T.test('crud', 'surfaces', 'POST /admin/packages/install (core conflict → 409)', async function () {
|
||||
var manifest = JSON.stringify({ id: 'chat', title: 'Evil Chat', route: '/s/chat' });
|
||||
var zipBlob = buildTestZip({ 'manifest.json': manifest });
|
||||
var d;
|
||||
try {
|
||||
d = await T.apiUpload('/admin/surfaces/install', zipBlob, 'chat.surface');
|
||||
d = await T.apiUpload('/admin/packages/install', zipBlob, 'chat.surface');
|
||||
// If no throw, check status manually
|
||||
T.assert(false, 'overwriting core surface should have thrown');
|
||||
} catch (e) {
|
||||
@@ -157,10 +157,10 @@
|
||||
}
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'POST /admin/surfaces/install (reseed preserves enabled)', async function () {
|
||||
await T.test('crud', 'surfaces', 'POST /admin/packages/install (reseed preserves enabled)', async function () {
|
||||
// Surface was re-enabled above. Disable it, then re-install.
|
||||
await T.authFetch(await T.getAuthToken(), 'PUT',
|
||||
'/admin/surfaces/' + surfaceId + '/disable');
|
||||
'/admin/packages/' + surfaceId + '/disable');
|
||||
|
||||
// Re-install same ID
|
||||
var manifest = JSON.stringify({
|
||||
@@ -169,33 +169,33 @@
|
||||
route: '/s/' + surfaceId
|
||||
});
|
||||
var zipBlob = buildTestZip({ 'manifest.json': manifest });
|
||||
await T.apiUpload('/admin/surfaces/install', zipBlob, surfaceId + '.surface');
|
||||
await T.apiUpload('/admin/packages/install', zipBlob, surfaceId + '.surface');
|
||||
|
||||
// Verify title updated but enabled preserved as false
|
||||
var d = await T.apiGet('/admin/surfaces/' + surfaceId);
|
||||
var d = await T.apiGet('/admin/packages/' + surfaceId);
|
||||
T.assert(d.title === 'ICD Test Surface v2', 'title should be updated after reseed');
|
||||
T.assert(d.enabled === false, 'enabled should be preserved as false after reseed');
|
||||
|
||||
// Re-enable for cleanup
|
||||
await T.authFetch(await T.getAuthToken(), 'PUT',
|
||||
'/admin/surfaces/' + surfaceId + '/enable');
|
||||
'/admin/packages/' + surfaceId + '/enable');
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'DELETE /admin/surfaces/:id', async function () {
|
||||
await T.test('crud', 'surfaces', 'DELETE /admin/packages/:id', async function () {
|
||||
var token = await T.getAuthToken();
|
||||
var d = await T.authFetch(token, 'DELETE', '/admin/surfaces/' + surfaceId);
|
||||
var d = await T.authFetch(token, 'DELETE', '/admin/packages/' + surfaceId);
|
||||
T.assertStatus(d, 200, 'delete');
|
||||
T.assert(d.deleted === true, 'should report deleted');
|
||||
|
||||
// Verify gone
|
||||
var check = await T.authFetch(token, 'GET', '/admin/surfaces/' + surfaceId);
|
||||
var check = await T.authFetch(token, 'GET', '/admin/packages/' + surfaceId);
|
||||
T.assert(check._status === 404, 'deleted surface should 404 on get');
|
||||
surfaceId = null;
|
||||
});
|
||||
|
||||
await T.test('crud', 'surfaces', 'DELETE /admin/surfaces/nonexistent (404)', async function () {
|
||||
await T.test('crud', 'surfaces', 'DELETE /admin/packages/nonexistent (404)', async function () {
|
||||
var d = await T.authFetch(await T.getAuthToken(), 'DELETE',
|
||||
'/admin/surfaces/nonexistent-surface-id');
|
||||
'/admin/packages/nonexistent-surface-id');
|
||||
T.assert(d._status === 404, 'delete nonexistent should return 404, got ' + d._status);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -227,7 +227,7 @@
|
||||
};
|
||||
|
||||
// ─── API Wrappers ───────────────────────────────────────────
|
||||
// v0.37.14: raw fetch — old API._* globals removed in scorched earth.
|
||||
// Raw fetch — kernel provides no global API wrappers; extensions use fetch directly.
|
||||
|
||||
async function _fetchJSON(method, path, body) {
|
||||
var token = await T.getAuthToken();
|
||||
|
||||
@@ -28,7 +28,7 @@
|
||||
var adminGets = [
|
||||
'/admin/stats', '/admin/users', '/admin/settings',
|
||||
'/admin/configs', '/admin/models', '/admin/personas',
|
||||
'/admin/teams', '/admin/groups', '/admin/surfaces',
|
||||
'/admin/teams', '/admin/groups', '/admin/packages',
|
||||
'/admin/providers/health', '/admin/routing/policies',
|
||||
'/admin/permissions', '/admin/roles', '/admin/vault/status',
|
||||
'/admin/storage/status', '/admin/audit', '/admin/pricing',
|
||||
@@ -72,8 +72,8 @@
|
||||
'expected 403/401, got ' + d._status);
|
||||
});
|
||||
|
||||
await T.test('authz', 'admin-deny', 'user → POST /admin/surfaces/install (expect 403)', async function () {
|
||||
var d = await T.authFetch(testUser.token, 'POST', '/admin/surfaces/install', {});
|
||||
await T.test('authz', 'admin-deny', 'user → POST /admin/packages/install (expect 403)', async function () {
|
||||
var d = await T.authFetch(testUser.token, 'POST', '/admin/packages/install', {});
|
||||
T.assert(d._status === 403 || d._status === 401 || d._status === 400,
|
||||
'expected 403/401/400, got ' + d._status);
|
||||
});
|
||||
|
||||
@@ -543,7 +543,7 @@
|
||||
await T.test('security', 'input-validation', '[P0] path traversal in surface archive', async function () {
|
||||
var blob = new Blob([JSON.stringify({ id: '../../../etc/evil', title: 'Path Traversal Test' })], { type: 'application/json' });
|
||||
try {
|
||||
var d = await T.apiUpload('/admin/surfaces/install', blob, 'evil.surface');
|
||||
var d = await T.apiUpload('/admin/packages/install', blob, 'evil.surface');
|
||||
T.assert(d._status === 400 || d._status === 409,
|
||||
'path traversal surface accepted! got ' + d._status);
|
||||
} catch (e) {
|
||||
@@ -644,8 +644,8 @@
|
||||
assertDenied(d._status, 'CRITICAL: team admin can delete users');
|
||||
});
|
||||
|
||||
await T.test('security', 'escalation', '[P0] teamAdmin → POST /admin/surfaces/install', async function () {
|
||||
var d = await T.authFetch(teamAdmin.token, 'POST', '/admin/surfaces/install', {});
|
||||
await T.test('security', 'escalation', '[P0] teamAdmin → POST /admin/packages/install', async function () {
|
||||
var d = await T.authFetch(teamAdmin.token, 'POST', '/admin/packages/install', {});
|
||||
if (d._status === 400) return; // bad upload format, not a security issue
|
||||
assertDenied(d._status, 'team admin can install surfaces');
|
||||
});
|
||||
|
||||
@@ -326,9 +326,9 @@
|
||||
T.assertHasKey(d, 'data', '/admin/groups');
|
||||
});
|
||||
|
||||
await T.test('smoke', 'admin', 'GET /admin/surfaces', async function () {
|
||||
var d = await T.apiGet('/admin/surfaces');
|
||||
T.assertHasKey(d, 'data', '/admin/surfaces');
|
||||
await T.test('smoke', 'admin', 'GET /admin/packages', async function () {
|
||||
var d = await T.apiGet('/admin/packages');
|
||||
T.assertHasKey(d, 'data', '/admin/packages');
|
||||
T.assert(Array.isArray(d.data), 'surfaces should be array');
|
||||
if (d.data.length > 0) {
|
||||
T.assertShape(d.data[0], T.S.surfaceAdmin, 'surfaces[0]');
|
||||
@@ -442,6 +442,48 @@
|
||||
T.assertHasKey(d, 'data', '/admin/memories/pending');
|
||||
T.assert(Array.isArray(d.data), 'data should be array');
|
||||
});
|
||||
|
||||
// -- v0.6.x Admin endpoints --
|
||||
|
||||
await T.test('smoke', 'admin', 'GET /admin/metrics', async function () {
|
||||
var d = await T.apiGet('/admin/metrics');
|
||||
T.assert(typeof d === 'object', 'expected object');
|
||||
T.assertHasKey(d, 'runtime', '/admin/metrics');
|
||||
T.assertHasKey(d, 'database', '/admin/metrics');
|
||||
});
|
||||
|
||||
await T.test('smoke', 'admin', 'GET /admin/cluster', async function () {
|
||||
var d = await T.apiGet('/admin/cluster');
|
||||
T.assertHasKey(d, 'data', '/admin/cluster');
|
||||
T.assert(Array.isArray(d.data), 'cluster nodes should be array');
|
||||
});
|
||||
|
||||
await T.test('smoke', 'admin', 'GET /admin/backups', async function () {
|
||||
var d = await T.apiGet('/admin/backups');
|
||||
T.assertHasKey(d, 'data', '/admin/backups');
|
||||
T.assert(Array.isArray(d.data), 'backups should be array');
|
||||
});
|
||||
}
|
||||
|
||||
// -- Docs API --
|
||||
await T.test('smoke', 'utility', 'GET /docs', async function () {
|
||||
var d = await T.apiGet('/docs');
|
||||
T.assertHasKey(d, 'data', '/docs');
|
||||
T.assert(Array.isArray(d.data), 'docs should be array');
|
||||
});
|
||||
|
||||
// -- Dynamic OpenAPI spec (outside /api/v1 prefix) --
|
||||
await T.test('smoke', 'utility', 'GET /api/docs/openapi.json', async function () {
|
||||
var token = await T.getAuthToken();
|
||||
var resp = await fetch(T.base + '/api/docs/openapi.json', {
|
||||
headers: { 'Authorization': 'Bearer ' + token },
|
||||
credentials: 'same-origin'
|
||||
});
|
||||
T.assert(resp.ok, 'expected 200 from /api/docs/openapi.json, got ' + resp.status);
|
||||
var d = await resp.json();
|
||||
T.assertHasKey(d, 'openapi', 'openapi spec');
|
||||
T.assertHasKey(d, 'paths', 'openapi spec');
|
||||
T.assertHasKey(d, 'info', 'openapi spec');
|
||||
});
|
||||
};
|
||||
})();
|
||||
|
||||
@@ -3,177 +3,141 @@
|
||||
// ==========================================
|
||||
// Renders ```latex / ```math code blocks and inline $...$ / $$...$$ syntax.
|
||||
// Loads KaTeX dynamically on first use.
|
||||
//
|
||||
// Registers with sw.renderers via the sw:ready event.
|
||||
// ==========================================
|
||||
|
||||
Extensions.register({
|
||||
id: 'katex-renderer',
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
_katexReady: false,
|
||||
_katexLoading: null,
|
||||
let _katexReady = false;
|
||||
let _katexLoading = null;
|
||||
|
||||
async init(ctx) {
|
||||
const self = this;
|
||||
function _escapeHtml(str) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
// ── Inject styles ──
|
||||
this._injectStyles();
|
||||
// ── KaTeX Library Loading ──────────────
|
||||
|
||||
// ── Block renderer: match ```latex or ```math ──
|
||||
ctx.renderers.register('katex-block', {
|
||||
type: 'block',
|
||||
priority: 10,
|
||||
match(lang) {
|
||||
const l = (lang || '').toLowerCase();
|
||||
return l === 'latex' || l === 'math' || l === 'tex' || l === 'katex';
|
||||
},
|
||||
render(lang, code, container) {
|
||||
const id = 'katex-' + Math.random().toString(36).slice(2, 9);
|
||||
container.innerHTML = `
|
||||
<div class="katex-block ext-rendered" data-katex-id="${id}">
|
||||
<div class="katex-display-container" data-katex-src="${encodeURIComponent(code.trim())}" data-katex-display="true">
|
||||
<div class="katex-loading">
|
||||
<span class="katex-spinner"></span> Rendering math…
|
||||
</div>
|
||||
</div>
|
||||
<details class="katex-source">
|
||||
<summary>𝑓(𝑥) View source</summary>
|
||||
<pre><code class="language-latex">${self._escapeHtml(code.trim())}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
function _loadKaTeX() {
|
||||
if (_katexReady) return Promise.resolve();
|
||||
if (_katexLoading) return _katexLoading;
|
||||
|
||||
_katexLoading = new Promise((resolve, reject) => {
|
||||
if (typeof katex !== 'undefined') {
|
||||
_katexReady = true;
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
|
||||
const base = (window.__BASE__ || '');
|
||||
const localCSS = `${base}/vendor/katex/katex.min.css`;
|
||||
const localJS = `${base}/vendor/katex/katex.min.js`;
|
||||
const cdnCSS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.css';
|
||||
const cdnJS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.js';
|
||||
|
||||
const link = document.createElement('link');
|
||||
link.rel = 'stylesheet';
|
||||
link.href = localCSS;
|
||||
link.onerror = () => { link.href = cdnCSS; };
|
||||
document.head.appendChild(link);
|
||||
|
||||
const script = document.createElement('script');
|
||||
script.src = localJS;
|
||||
script.onload = () => { _katexReady = true; resolve(); };
|
||||
script.onerror = () => {
|
||||
console.warn('[KaTeX] Local vendor not found, trying CDN');
|
||||
const cdn = document.createElement('script');
|
||||
cdn.src = cdnJS;
|
||||
cdn.onload = () => { _katexReady = true; resolve(); };
|
||||
cdn.onerror = () => {
|
||||
console.error('[KaTeX] Failed to load from both local and CDN');
|
||||
reject(new Error('Failed to load KaTeX'));
|
||||
};
|
||||
document.head.appendChild(cdn);
|
||||
};
|
||||
document.head.appendChild(script);
|
||||
});
|
||||
return _katexLoading;
|
||||
}
|
||||
|
||||
// ── Post renderer: render KaTeX blocks + find inline math ──
|
||||
ctx.renderers.register('katex-post', {
|
||||
type: 'post',
|
||||
priority: 10,
|
||||
render(container) {
|
||||
// Render code blocks with data-katex-src
|
||||
const blocks = container.querySelectorAll('.katex-display-container[data-katex-src]');
|
||||
if (blocks.length > 0) {
|
||||
blocks.forEach(el => {
|
||||
if (el.dataset.rendered) return;
|
||||
el.dataset.rendered = 'pending';
|
||||
self._renderBlock(el);
|
||||
});
|
||||
}
|
||||
// ── Block Rendering ──────────────────────
|
||||
|
||||
// Process inline math in text content
|
||||
self._processInlineMath(container);
|
||||
}
|
||||
});
|
||||
|
||||
// Pre-load KaTeX
|
||||
this._loadKaTeX();
|
||||
},
|
||||
|
||||
async _renderBlock(el) {
|
||||
async function _renderBlock(el) {
|
||||
const code = decodeURIComponent(el.dataset.katexSrc);
|
||||
const displayMode = el.dataset.katexDisplay === 'true';
|
||||
|
||||
try {
|
||||
await this._loadKaTeX();
|
||||
await _loadKaTeX();
|
||||
el.innerHTML = katex.renderToString(code, {
|
||||
displayMode,
|
||||
throwOnError: false,
|
||||
trust: false,
|
||||
strict: false,
|
||||
displayMode, throwOnError: false, trust: false, strict: false,
|
||||
});
|
||||
el.dataset.rendered = 'true';
|
||||
} catch (e) {
|
||||
el.innerHTML = `
|
||||
<div class="katex-error">
|
||||
<strong>Math error:</strong> ${this._escapeHtml(e.message || String(e))}
|
||||
<strong>Math error:</strong> ${_escapeHtml(e.message || String(e))}
|
||||
</div>
|
||||
`;
|
||||
el.dataset.rendered = 'error';
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
async _processInlineMath(container) {
|
||||
// Skip if no $ signs present at all
|
||||
// ── Inline Math Processing ───────────────
|
||||
|
||||
async function _processInlineMath(container) {
|
||||
if (!container.textContent || !container.textContent.includes('$')) return;
|
||||
try { await _loadKaTeX(); } catch { return; }
|
||||
|
||||
try {
|
||||
await this._loadKaTeX();
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
|
||||
// Walk text nodes, skip code/pre/katex-already-rendered
|
||||
const walker = document.createTreeWalker(
|
||||
container,
|
||||
NodeFilter.SHOW_TEXT,
|
||||
{
|
||||
acceptNode(node) {
|
||||
const parent = node.parentElement;
|
||||
if (!parent) return NodeFilter.FILTER_REJECT;
|
||||
// Skip code, pre, already-rendered katex, and script elements
|
||||
const tag = parent.tagName;
|
||||
if (tag === 'CODE' || tag === 'PRE' || tag === 'SCRIPT' ||
|
||||
tag === 'TEXTAREA' || tag === 'STYLE') {
|
||||
return NodeFilter.FILTER_REJECT;
|
||||
}
|
||||
if (parent.closest('.katex, .katex-block, .katex-display-container, code, pre')) {
|
||||
return NodeFilter.FILTER_REJECT;
|
||||
}
|
||||
if (!node.textContent.includes('$')) return NodeFilter.FILTER_REJECT;
|
||||
return NodeFilter.FILTER_ACCEPT;
|
||||
}
|
||||
const walker = document.createTreeWalker(container, NodeFilter.SHOW_TEXT, {
|
||||
acceptNode(node) {
|
||||
const parent = node.parentElement;
|
||||
if (!parent) return NodeFilter.FILTER_REJECT;
|
||||
const tag = parent.tagName;
|
||||
if (tag === 'CODE' || tag === 'PRE' || tag === 'SCRIPT' ||
|
||||
tag === 'TEXTAREA' || tag === 'STYLE') return NodeFilter.FILTER_REJECT;
|
||||
if (parent.closest('.katex, .katex-block, .katex-display-container, code, pre'))
|
||||
return NodeFilter.FILTER_REJECT;
|
||||
if (!node.textContent.includes('$')) return NodeFilter.FILTER_REJECT;
|
||||
return NodeFilter.FILTER_ACCEPT;
|
||||
}
|
||||
);
|
||||
});
|
||||
|
||||
const textNodes = [];
|
||||
let node;
|
||||
while (node = walker.nextNode()) textNodes.push(node);
|
||||
for (const textNode of textNodes) _replaceInlineMath(textNode);
|
||||
}
|
||||
|
||||
for (const textNode of textNodes) {
|
||||
this._replaceInlineMath(textNode);
|
||||
}
|
||||
},
|
||||
|
||||
_replaceInlineMath(textNode) {
|
||||
function _replaceInlineMath(textNode) {
|
||||
const text = textNode.textContent;
|
||||
// Match $$...$$ (display) and $...$ (inline), non-greedy
|
||||
// Avoid matching escaped \$ or empty $$
|
||||
const pattern = /\$\$([^$]+?)\$\$|\$([^$\n]+?)\$/g;
|
||||
let match;
|
||||
const parts = [];
|
||||
let lastIndex = 0;
|
||||
|
||||
while ((match = pattern.exec(text)) !== null) {
|
||||
// Text before the match
|
||||
if (match.index > lastIndex) {
|
||||
parts.push({ type: 'text', value: text.slice(lastIndex, match.index) });
|
||||
}
|
||||
|
||||
const displayMode = !!match[1]; // $$ ... $$
|
||||
const displayMode = !!match[1];
|
||||
const expr = match[1] || match[2];
|
||||
|
||||
try {
|
||||
const html = katex.renderToString(expr.trim(), {
|
||||
displayMode,
|
||||
throwOnError: false,
|
||||
trust: false,
|
||||
strict: false,
|
||||
displayMode, throwOnError: false, trust: false, strict: false,
|
||||
});
|
||||
parts.push({ type: 'katex', value: html, displayMode });
|
||||
} catch {
|
||||
// Leave as-is on error
|
||||
parts.push({ type: 'text', value: match[0] });
|
||||
}
|
||||
|
||||
lastIndex = match.index + match[0].length;
|
||||
}
|
||||
|
||||
if (parts.length === 0) return; // No math found
|
||||
if (parts.length === 0) return;
|
||||
if (lastIndex < text.length) parts.push({ type: 'text', value: text.slice(lastIndex) });
|
||||
|
||||
// Remaining text after last match
|
||||
if (lastIndex < text.length) {
|
||||
parts.push({ type: 'text', value: text.slice(lastIndex) });
|
||||
}
|
||||
|
||||
// Replace text node with a span containing the rendered parts
|
||||
const span = document.createElement('span');
|
||||
span.className = 'katex-inline-container';
|
||||
for (const part of parts) {
|
||||
@@ -186,70 +150,12 @@ Extensions.register({
|
||||
span.appendChild(wrapper);
|
||||
}
|
||||
}
|
||||
|
||||
textNode.parentNode.replaceChild(span, textNode);
|
||||
},
|
||||
}
|
||||
|
||||
_loadKaTeX() {
|
||||
if (this._katexReady) return Promise.resolve();
|
||||
if (this._katexLoading) return this._katexLoading;
|
||||
// ── Styles ──────────────────────────────
|
||||
|
||||
this._katexLoading = new Promise((resolve, reject) => {
|
||||
if (typeof katex !== 'undefined') {
|
||||
this._katexReady = true;
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
|
||||
const base = (window.__BASE__ || '');
|
||||
const localCSS = `${base}/vendor/katex/katex.min.css`;
|
||||
const localJS = `${base}/vendor/katex/katex.min.js`;
|
||||
const cdnCSS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.css';
|
||||
const cdnJS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.js';
|
||||
|
||||
// Load CSS first
|
||||
const link = document.createElement('link');
|
||||
link.rel = 'stylesheet';
|
||||
link.href = localCSS;
|
||||
link.onerror = () => {
|
||||
link.href = cdnCSS;
|
||||
};
|
||||
document.head.appendChild(link);
|
||||
|
||||
// Load JS
|
||||
const script = document.createElement('script');
|
||||
script.src = localJS;
|
||||
script.onload = () => {
|
||||
this._katexReady = true;
|
||||
resolve();
|
||||
};
|
||||
script.onerror = () => {
|
||||
console.warn('[KaTeX] Local vendor not found, trying CDN');
|
||||
const cdn = document.createElement('script');
|
||||
cdn.src = cdnJS;
|
||||
cdn.onload = () => {
|
||||
this._katexReady = true;
|
||||
resolve();
|
||||
};
|
||||
cdn.onerror = () => {
|
||||
console.error('[KaTeX] Failed to load from both local and CDN');
|
||||
reject(new Error('Failed to load KaTeX'));
|
||||
};
|
||||
document.head.appendChild(cdn);
|
||||
};
|
||||
document.head.appendChild(script);
|
||||
});
|
||||
|
||||
return this._katexLoading;
|
||||
},
|
||||
|
||||
_escapeHtml(str) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
},
|
||||
|
||||
_injectStyles() {
|
||||
function _injectStyles() {
|
||||
if (document.getElementById('ext-style-katex-renderer')) return;
|
||||
const style = document.createElement('style');
|
||||
style.id = 'ext-style-katex-renderer';
|
||||
@@ -289,9 +195,64 @@ Extensions.register({
|
||||
.katex-inline .katex { font-size: 1.05em; }
|
||||
.katex-display { display: block; text-align: center; margin: 8px 0; }`;
|
||||
document.head.appendChild(style);
|
||||
},
|
||||
|
||||
destroy() {
|
||||
document.getElementById('ext-style-katex-renderer')?.remove();
|
||||
}
|
||||
});
|
||||
|
||||
// ── Registration ────────────────────────
|
||||
|
||||
function register() {
|
||||
if (!window.sw?.renderers) return;
|
||||
|
||||
_injectStyles();
|
||||
|
||||
// Block renderer: match ```latex, ```math, ```tex, ```katex
|
||||
sw.renderers.register('katex-block', {
|
||||
type: 'block',
|
||||
priority: 10,
|
||||
match(lang) {
|
||||
const l = (lang || '').toLowerCase();
|
||||
return l === 'latex' || l === 'math' || l === 'tex' || l === 'katex';
|
||||
},
|
||||
render(lang, code, container) {
|
||||
const id = 'katex-' + Math.random().toString(36).slice(2, 9);
|
||||
container.innerHTML = `
|
||||
<div class="katex-block ext-rendered" data-katex-id="${id}">
|
||||
<div class="katex-display-container" data-katex-src="${encodeURIComponent(code.trim())}" data-katex-display="true">
|
||||
<div class="katex-loading">
|
||||
<span class="katex-spinner"></span> Rendering math\u2026
|
||||
</div>
|
||||
</div>
|
||||
<details class="katex-source">
|
||||
<summary>\ud835\udc53(\ud835\udc65) View source</summary>
|
||||
<pre><code class="language-latex">${_escapeHtml(code.trim())}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
});
|
||||
|
||||
// Post renderer: render KaTeX blocks + find inline math
|
||||
sw.renderers.register('katex-post', {
|
||||
type: 'post',
|
||||
priority: 10,
|
||||
render(container) {
|
||||
const blocks = container.querySelectorAll('.katex-display-container[data-katex-src]');
|
||||
if (blocks.length > 0) {
|
||||
blocks.forEach(el => {
|
||||
if (el.dataset.rendered) return;
|
||||
el.dataset.rendered = 'pending';
|
||||
_renderBlock(el);
|
||||
});
|
||||
}
|
||||
_processInlineMath(container);
|
||||
}
|
||||
});
|
||||
|
||||
_loadKaTeX();
|
||||
}
|
||||
|
||||
if (window.sw?._sdk) {
|
||||
register();
|
||||
} else {
|
||||
document.addEventListener('sw:ready', register, { once: true });
|
||||
}
|
||||
})();
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"tier": "browser",
|
||||
"author": "switchboard",
|
||||
"description": "Renders LaTeX math expressions: ```latex blocks and inline $...$ / $$...$$ syntax",
|
||||
"requires": ["chat"],
|
||||
"requires": [],
|
||||
"permissions": [],
|
||||
"tools": [],
|
||||
"surfaces": [],
|
||||
|
||||
@@ -2,94 +2,33 @@
|
||||
// Mermaid Diagram Renderer — Browser Extension
|
||||
// ==========================================
|
||||
// Renders ```mermaid code blocks as interactive SVG diagrams.
|
||||
// Features: viewBox-based zoom/pan, context-aware expand (fullscreen
|
||||
// or side-panel pop-out), SVG/PNG export, source copy.
|
||||
// Features: viewBox-based zoom/pan, fullscreen expand,
|
||||
// SVG/PNG export, source copy.
|
||||
// Loads mermaid.js dynamically on first use.
|
||||
//
|
||||
// Uses ctx.ui primitives from the host app for toast notifications,
|
||||
// side-panel preview, theme detection, and mobile awareness.
|
||||
// Registers with sw.renderers via the sw:ready event.
|
||||
// ==========================================
|
||||
|
||||
Extensions.register({
|
||||
id: 'mermaid-renderer',
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
_mermaidReady: false,
|
||||
_mermaidLoading: null,
|
||||
_ctx: null,
|
||||
let _mermaidReady = false;
|
||||
let _mermaidLoading = null;
|
||||
|
||||
async init(ctx) {
|
||||
const self = this;
|
||||
this._ctx = ctx;
|
||||
function _isDark() {
|
||||
return document.documentElement.dataset.theme === 'dark' ||
|
||||
document.documentElement.classList.contains('dark');
|
||||
}
|
||||
|
||||
this._injectStyles();
|
||||
|
||||
// ── Block renderer: match ```mermaid ──
|
||||
ctx.renderers.register('mermaid', {
|
||||
type: 'block',
|
||||
pattern: 'mermaid',
|
||||
priority: 10,
|
||||
render(lang, code, container) {
|
||||
const id = 'mmd-' + Math.random().toString(36).slice(2, 9);
|
||||
container.innerHTML = `
|
||||
<div class="mermaid-block" data-mermaid-id="${id}">
|
||||
<div class="mermaid-toolbar">
|
||||
<span class="mermaid-title">\u{1f4ca} Diagram</span>
|
||||
<span class="mermaid-zoom-label" data-zoom-label="${id}">100%</span>
|
||||
<div class="mermaid-toolbar-btns">
|
||||
<button class="mmd-btn" data-action="zoom-in" data-target="${id}" title="Zoom in">+</button>
|
||||
<button class="mmd-btn" data-action="zoom-out" data-target="${id}" title="Zoom out">\u2212</button>
|
||||
<button class="mmd-btn" data-action="zoom-fit" data-target="${id}" title="Fit to view">\u22a1</button>
|
||||
<button class="mmd-btn" data-action="zoom-reset" data-target="${id}" title="Reset zoom">1:1</button>
|
||||
<span class="mmd-sep"></span>
|
||||
<button class="mmd-btn" data-action="expand" data-target="${id}" title="Expand">\u26f6</button>
|
||||
<span class="mmd-sep"></span>
|
||||
<button class="mmd-btn" data-action="export-svg" data-target="${id}" title="Download SVG">SVG</button>
|
||||
<button class="mmd-btn" data-action="export-png" data-target="${id}" title="Download PNG">PNG</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mermaid-viewport" data-viewport="${id}">
|
||||
<div class="mermaid-diagram" data-mermaid-src="${encodeURIComponent(code.trim())}" data-diagram="${id}">
|
||||
<div class="mermaid-loading">
|
||||
<span class="mermaid-spinner"></span> Rendering diagram\u2026
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<details class="mermaid-source">
|
||||
<summary>
|
||||
<span>\u{1f4cb} View source</span>
|
||||
<button class="mmd-btn mmd-copy-src" data-action="copy-src" data-target="${id}" title="Copy source" onclick="event.stopPropagation()">Copy</button>
|
||||
</summary>
|
||||
<pre><code class="language-mermaid" data-source="${id}">${self._escapeHtml(code.trim())}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
});
|
||||
|
||||
// ── Post renderer: render diagrams + wire interactivity ──
|
||||
ctx.renderers.register('mermaid-post', {
|
||||
type: 'post',
|
||||
priority: 10,
|
||||
render(container) {
|
||||
const diagrams = container.querySelectorAll('.mermaid-diagram[data-mermaid-src]');
|
||||
if (diagrams.length === 0) return;
|
||||
|
||||
diagrams.forEach(el => {
|
||||
if (el.dataset.rendered) return;
|
||||
el.dataset.rendered = 'pending';
|
||||
self._renderDiagram(el);
|
||||
});
|
||||
|
||||
self._wireToolbar(container);
|
||||
}
|
||||
});
|
||||
|
||||
this._loadMermaid();
|
||||
},
|
||||
function _escapeHtml(str) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
// ── ViewBox Zoom/Pan State ──────────────
|
||||
|
||||
_getState(id) {
|
||||
function _getState(id) {
|
||||
const vp = document.querySelector(`[data-viewport="${id}"]`);
|
||||
if (!vp) return null;
|
||||
if (!vp._mmdState) {
|
||||
@@ -102,10 +41,10 @@ Extensions.register({
|
||||
};
|
||||
}
|
||||
return vp._mmdState;
|
||||
},
|
||||
}
|
||||
|
||||
_initState(id) {
|
||||
const state = this._getState(id);
|
||||
function _initState(id) {
|
||||
const state = _getState(id);
|
||||
if (!state) return;
|
||||
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
|
||||
if (!svg) return;
|
||||
@@ -113,69 +52,51 @@ Extensions.register({
|
||||
const vb = svg.getAttribute('viewBox');
|
||||
if (!vb) {
|
||||
const bbox = svg.getBBox();
|
||||
state.natX = bbox.x;
|
||||
state.natY = bbox.y;
|
||||
state.natW = bbox.width;
|
||||
state.natH = bbox.height;
|
||||
state.natX = bbox.x; state.natY = bbox.y;
|
||||
state.natW = bbox.width; state.natH = bbox.height;
|
||||
} else {
|
||||
const parts = vb.trim().split(/[\s,]+/).map(Number);
|
||||
state.natX = parts[0] || 0;
|
||||
state.natY = parts[1] || 0;
|
||||
state.natW = parts[2] || 0;
|
||||
state.natH = parts[3] || 0;
|
||||
state.natX = parts[0] || 0; state.natY = parts[1] || 0;
|
||||
state.natW = parts[2] || 0; state.natH = parts[3] || 0;
|
||||
}
|
||||
|
||||
state.vbX = state.natX;
|
||||
state.vbY = state.natY;
|
||||
state.vbW = state.natW;
|
||||
state.vbH = state.natH;
|
||||
state.vbX = state.natX; state.vbY = state.natY;
|
||||
state.vbW = state.natW; state.vbH = state.natH;
|
||||
state.ready = true;
|
||||
_applyViewBox(id);
|
||||
}
|
||||
|
||||
this._applyViewBox(id);
|
||||
},
|
||||
|
||||
_applyViewBox(id) {
|
||||
const state = this._getState(id);
|
||||
function _applyViewBox(id) {
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
|
||||
if (!svg) return;
|
||||
|
||||
svg.setAttribute('viewBox', `${state.vbX} ${state.vbY} ${state.vbW} ${state.vbH}`);
|
||||
|
||||
const zoomPct = Math.round((state.natW / state.vbW) * 100);
|
||||
const label = document.querySelector(`[data-zoom-label="${id}"]`);
|
||||
if (label) label.textContent = zoomPct + '%';
|
||||
},
|
||||
}
|
||||
|
||||
_zoom(id, factor) {
|
||||
const state = this._getState(id);
|
||||
function _zoom(id, factor) {
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
|
||||
const scale = 1 / (1 + factor);
|
||||
const newW = state.vbW * scale;
|
||||
const newH = state.vbH * scale;
|
||||
|
||||
const minW = state.natW / 20;
|
||||
const maxW = state.natW * 2;
|
||||
if (newW < minW || newW > maxW) return;
|
||||
|
||||
if (newW < state.natW / 20 || newW > state.natW * 2) return;
|
||||
const cx = state.vbX + state.vbW / 2;
|
||||
const cy = state.vbY + state.vbH / 2;
|
||||
state.vbW = newW; state.vbH = newH;
|
||||
state.vbX = cx - newW / 2; state.vbY = cy - newH / 2;
|
||||
_applyViewBox(id);
|
||||
}
|
||||
|
||||
state.vbW = newW;
|
||||
state.vbH = newH;
|
||||
state.vbX = cx - newW / 2;
|
||||
state.vbY = cy - newH / 2;
|
||||
|
||||
this._applyViewBox(id);
|
||||
},
|
||||
|
||||
_zoomAt(id, factor, clientX, clientY) {
|
||||
const state = this._getState(id);
|
||||
function _zoomAt(id, factor, clientX, clientY) {
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
|
||||
if (!svg) return;
|
||||
|
||||
const svgRect = svg.getBoundingClientRect();
|
||||
if (svgRect.width === 0 || svgRect.height === 0) return;
|
||||
|
||||
@@ -187,147 +108,76 @@ Extensions.register({
|
||||
const scale = 1 / (1 + factor);
|
||||
const newW = state.vbW * scale;
|
||||
const newH = state.vbH * scale;
|
||||
if (newW < state.natW / 20 || newW > state.natW * 2) return;
|
||||
state.vbW = newW; state.vbH = newH;
|
||||
state.vbX = pointX - fx * newW; state.vbY = pointY - fy * newH;
|
||||
_applyViewBox(id);
|
||||
}
|
||||
|
||||
const minW = state.natW / 20;
|
||||
const maxW = state.natW * 2;
|
||||
if (newW < minW || newW > maxW) return;
|
||||
|
||||
state.vbW = newW;
|
||||
state.vbH = newH;
|
||||
state.vbX = pointX - fx * newW;
|
||||
state.vbY = pointY - fy * newH;
|
||||
|
||||
this._applyViewBox(id);
|
||||
},
|
||||
|
||||
_zoomReset(id) {
|
||||
const state = this._getState(id);
|
||||
function _zoomReset(id) {
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
state.vbX = state.natX;
|
||||
state.vbY = state.natY;
|
||||
state.vbW = state.natW;
|
||||
state.vbH = state.natH;
|
||||
this._applyViewBox(id);
|
||||
},
|
||||
state.vbX = state.natX; state.vbY = state.natY;
|
||||
state.vbW = state.natW; state.vbH = state.natH;
|
||||
_applyViewBox(id);
|
||||
}
|
||||
|
||||
_zoomFit(id) {
|
||||
const state = this._getState(id);
|
||||
function _zoomFit(id) {
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
const vp = document.querySelector(`[data-viewport="${id}"]`);
|
||||
if (!vp) return;
|
||||
|
||||
const vpRect = vp.getBoundingClientRect();
|
||||
if (vpRect.width === 0 || vpRect.height === 0) return;
|
||||
|
||||
const vpAspect = vpRect.width / vpRect.height;
|
||||
const natAspect = state.natW / state.natH;
|
||||
|
||||
if (vpAspect > natAspect) {
|
||||
const newW = state.natH * vpAspect;
|
||||
state.vbX = state.natX - (newW - state.natW) / 2;
|
||||
state.vbY = state.natY;
|
||||
state.vbW = newW;
|
||||
state.vbH = state.natH;
|
||||
state.vbW = newW; state.vbH = state.natH;
|
||||
} else {
|
||||
const newH = state.natW / vpAspect;
|
||||
state.vbX = state.natX;
|
||||
state.vbY = state.natY - (newH - state.natH) / 2;
|
||||
state.vbW = state.natW;
|
||||
state.vbH = newH;
|
||||
state.vbW = state.natW; state.vbH = newH;
|
||||
}
|
||||
|
||||
this._applyViewBox(id);
|
||||
},
|
||||
|
||||
// ── Expand (context-aware) ──────────────
|
||||
// Side panel open → pop out into it (replaces current content).
|
||||
// Side panel closed → fullscreen overlay.
|
||||
|
||||
_expand(id) {
|
||||
if (this._ctx.ui.isPanelOpen()) {
|
||||
this._popOutToPanel(id);
|
||||
} else {
|
||||
this._toggleFullscreen(id);
|
||||
}
|
||||
},
|
||||
_applyViewBox(id);
|
||||
}
|
||||
|
||||
// ── Fullscreen ──────────────────────────
|
||||
|
||||
_toggleFullscreen(id) {
|
||||
function _toggleFullscreen(id) {
|
||||
const block = document.querySelector(`[data-mermaid-id="${id}"]`);
|
||||
if (!block) return;
|
||||
|
||||
const isFS = block.classList.toggle('mermaid-fullscreen');
|
||||
|
||||
if (isFS) {
|
||||
document.body.style.overflow = 'hidden';
|
||||
|
||||
// Close button overlay (always visible — critical for mobile)
|
||||
const closeBtn = document.createElement('button');
|
||||
closeBtn.className = 'mmd-fullscreen-close';
|
||||
closeBtn.innerHTML = '\u2715';
|
||||
closeBtn.title = 'Close fullscreen';
|
||||
closeBtn.addEventListener('click', () => this._toggleFullscreen(id));
|
||||
closeBtn.addEventListener('click', () => _toggleFullscreen(id));
|
||||
block.appendChild(closeBtn);
|
||||
|
||||
// Escape listener (desktop convenience, not sole exit)
|
||||
block._mmdEscHandler = (e) => {
|
||||
if (e.key === 'Escape') this._toggleFullscreen(id);
|
||||
};
|
||||
block._mmdEscHandler = (e) => { if (e.key === 'Escape') _toggleFullscreen(id); };
|
||||
document.addEventListener('keydown', block._mmdEscHandler);
|
||||
|
||||
requestAnimationFrame(() => this._zoomFit(id));
|
||||
requestAnimationFrame(() => _zoomFit(id));
|
||||
} else {
|
||||
document.body.style.overflow = '';
|
||||
|
||||
const closeBtn = block.querySelector('.mmd-fullscreen-close');
|
||||
if (closeBtn) closeBtn.remove();
|
||||
|
||||
if (block._mmdEscHandler) {
|
||||
document.removeEventListener('keydown', block._mmdEscHandler);
|
||||
delete block._mmdEscHandler;
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
// ── Side Panel Pop-out ──────────────────
|
||||
|
||||
_popOutToPanel(id) {
|
||||
const diagram = document.querySelector(`[data-diagram="${id}"]`);
|
||||
const svg = diagram?.querySelector('svg');
|
||||
if (!svg) return;
|
||||
|
||||
const state = this._getState(id);
|
||||
const clone = svg.cloneNode(true);
|
||||
if (state?.ready) {
|
||||
clone.setAttribute('viewBox',
|
||||
`${state.natX} ${state.natY} ${state.natW} ${state.natH}`);
|
||||
}
|
||||
clone.setAttribute('width', '100%');
|
||||
clone.removeAttribute('height');
|
||||
clone.setAttribute('preserveAspectRatio', 'xMidYMid meet');
|
||||
|
||||
const isDark = this._ctx.ui.isDark();
|
||||
const html = `<!DOCTYPE html>
|
||||
<html><head>
|
||||
<meta charset="utf-8">
|
||||
<style>
|
||||
body { margin: 0; padding: 16px; display: flex; justify-content: center;
|
||||
align-items: flex-start; min-height: 100vh;
|
||||
background: ${isDark ? '#1a1a2e' : '#fff'};
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; }
|
||||
svg { max-width: 100%; height: auto; }
|
||||
</style>
|
||||
</head><body>${clone.outerHTML}</body></html>`;
|
||||
|
||||
this._ctx.ui.openPreview(html);
|
||||
},
|
||||
}
|
||||
|
||||
// ── Toolbar Wiring ──────────────────────
|
||||
|
||||
_wireToolbar(container) {
|
||||
const self = this;
|
||||
|
||||
function _wireToolbar(container) {
|
||||
if (container._mmdWired) return;
|
||||
container._mmdWired = true;
|
||||
|
||||
@@ -339,14 +189,14 @@ Extensions.register({
|
||||
if (!id) return;
|
||||
|
||||
switch (action) {
|
||||
case 'zoom-in': self._zoom(id, 0.25); break;
|
||||
case 'zoom-out': self._zoom(id, -0.2); break;
|
||||
case 'zoom-reset': self._zoomReset(id); break;
|
||||
case 'zoom-fit': self._zoomFit(id); break;
|
||||
case 'expand': self._expand(id); break;
|
||||
case 'export-svg': self._exportSVG(id); break;
|
||||
case 'export-png': self._exportPNG(id); break;
|
||||
case 'copy-src': self._copySource(id); break;
|
||||
case 'zoom-in': _zoom(id, 0.25); break;
|
||||
case 'zoom-out': _zoom(id, -0.2); break;
|
||||
case 'zoom-reset': _zoomReset(id); break;
|
||||
case 'zoom-fit': _zoomFit(id); break;
|
||||
case 'expand': _toggleFullscreen(id); break;
|
||||
case 'export-svg': _exportSVG(id); break;
|
||||
case 'export-png': _exportPNG(id); break;
|
||||
case 'copy-src': _copySource(id); break;
|
||||
}
|
||||
});
|
||||
|
||||
@@ -359,42 +209,35 @@ Extensions.register({
|
||||
vp.addEventListener('wheel', (e) => {
|
||||
e.preventDefault();
|
||||
const delta = e.deltaY > 0 ? -0.1 : 0.1;
|
||||
self._zoomAt(id, delta, e.clientX, e.clientY);
|
||||
_zoomAt(id, delta, e.clientX, e.clientY);
|
||||
}, { passive: false });
|
||||
|
||||
vp.addEventListener('mousedown', (e) => {
|
||||
if (e.button !== 0) return;
|
||||
const state = self._getState(id);
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
state.dragging = true;
|
||||
state.startX = e.clientX;
|
||||
state.startY = e.clientY;
|
||||
state.startVbX = state.vbX;
|
||||
state.startVbY = state.vbY;
|
||||
state.startX = e.clientX; state.startY = e.clientY;
|
||||
state.startVbX = state.vbX; state.startVbY = state.vbY;
|
||||
vp.classList.add('mmd-grabbing');
|
||||
e.preventDefault();
|
||||
});
|
||||
|
||||
vp.addEventListener('mousemove', (e) => {
|
||||
const state = self._getState(id);
|
||||
const state = _getState(id);
|
||||
if (!state?.dragging) return;
|
||||
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
|
||||
if (!svg) return;
|
||||
|
||||
const svgRect = svg.getBoundingClientRect();
|
||||
if (svgRect.width === 0) return;
|
||||
|
||||
const pxToVb = state.vbW / svgRect.width;
|
||||
const dx = (e.clientX - state.startX) * pxToVb;
|
||||
const dy = (e.clientY - state.startY) * pxToVb;
|
||||
|
||||
state.vbX = state.startVbX - dx;
|
||||
state.vbY = state.startVbY - dy;
|
||||
self._applyViewBox(id);
|
||||
state.vbX = state.startVbX - (e.clientX - state.startX) * pxToVb;
|
||||
state.vbY = state.startVbY - (e.clientY - state.startY) * pxToVb;
|
||||
_applyViewBox(id);
|
||||
});
|
||||
|
||||
const endDrag = () => {
|
||||
const state = self._getState(id);
|
||||
const state = _getState(id);
|
||||
if (!state) return;
|
||||
state.dragging = false;
|
||||
vp.classList.remove('mmd-grabbing');
|
||||
@@ -403,9 +246,8 @@ Extensions.register({
|
||||
vp.addEventListener('mouseleave', endDrag);
|
||||
|
||||
let lastTouchDist = 0;
|
||||
let lastTouchCenter = null;
|
||||
vp.addEventListener('touchstart', (e) => {
|
||||
const state = self._getState(id);
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
if (e.touches.length === 1) {
|
||||
state.dragging = true;
|
||||
@@ -418,28 +260,21 @@ Extensions.register({
|
||||
e.touches[0].clientX - e.touches[1].clientX,
|
||||
e.touches[0].clientY - e.touches[1].clientY
|
||||
);
|
||||
lastTouchCenter = {
|
||||
x: (e.touches[0].clientX + e.touches[1].clientX) / 2,
|
||||
y: (e.touches[0].clientY + e.touches[1].clientY) / 2,
|
||||
};
|
||||
}
|
||||
}, { passive: true });
|
||||
|
||||
vp.addEventListener('touchmove', (e) => {
|
||||
const state = self._getState(id);
|
||||
const state = _getState(id);
|
||||
if (!state || !state.ready) return;
|
||||
if (e.touches.length === 1 && state.dragging) {
|
||||
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
|
||||
if (!svg) return;
|
||||
const svgRect = svg.getBoundingClientRect();
|
||||
if (svgRect.width === 0) return;
|
||||
|
||||
const pxToVb = state.vbW / svgRect.width;
|
||||
const dx = (e.touches[0].clientX - state.startX) * pxToVb;
|
||||
const dy = (e.touches[0].clientY - state.startY) * pxToVb;
|
||||
state.vbX = state.startVbX - dx;
|
||||
state.vbY = state.startVbY - dy;
|
||||
self._applyViewBox(id);
|
||||
state.vbX = state.startVbX - (e.touches[0].clientX - state.startX) * pxToVb;
|
||||
state.vbY = state.startVbY - (e.touches[0].clientY - state.startY) * pxToVb;
|
||||
_applyViewBox(id);
|
||||
e.preventDefault();
|
||||
} else if (e.touches.length === 2) {
|
||||
const dist = Math.hypot(
|
||||
@@ -452,46 +287,42 @@ Extensions.register({
|
||||
};
|
||||
if (lastTouchDist > 0) {
|
||||
const factor = (dist - lastTouchDist) / lastTouchDist;
|
||||
self._zoomAt(id, factor, center.x, center.y);
|
||||
_zoomAt(id, factor, center.x, center.y);
|
||||
}
|
||||
lastTouchDist = dist;
|
||||
lastTouchCenter = center;
|
||||
e.preventDefault();
|
||||
}
|
||||
}, { passive: false });
|
||||
|
||||
vp.addEventListener('touchend', () => {
|
||||
const state = self._getState(id);
|
||||
const state = _getState(id);
|
||||
if (state) state.dragging = false;
|
||||
lastTouchDist = 0;
|
||||
lastTouchCenter = null;
|
||||
}, { passive: true });
|
||||
});
|
||||
},
|
||||
}
|
||||
|
||||
// ── Export ───────────────────────────────
|
||||
|
||||
_exportSVG(id) {
|
||||
function _exportSVG(id) {
|
||||
const diagram = document.querySelector(`[data-diagram="${id}"]`);
|
||||
const svg = diagram?.querySelector('svg');
|
||||
if (!svg) return;
|
||||
|
||||
const state = this._getState(id);
|
||||
const state = _getState(id);
|
||||
const clone = svg.cloneNode(true);
|
||||
clone.setAttribute('xmlns', 'http://www.w3.org/2000/svg');
|
||||
if (state?.ready) {
|
||||
clone.setAttribute('viewBox', `${state.natX} ${state.natY} ${state.natW} ${state.natH}`);
|
||||
}
|
||||
const blob = new Blob([clone.outerHTML], { type: 'image/svg+xml' });
|
||||
this._download(blob, `diagram-${id}.svg`);
|
||||
},
|
||||
_download(blob, `diagram-${id}.svg`);
|
||||
}
|
||||
|
||||
_exportPNG(id) {
|
||||
function _exportPNG(id) {
|
||||
const diagram = document.querySelector(`[data-diagram="${id}"]`);
|
||||
const svg = diagram?.querySelector('svg');
|
||||
if (!svg) return;
|
||||
|
||||
const state = this._getState(id);
|
||||
const state = _getState(id);
|
||||
const clone = svg.cloneNode(true);
|
||||
clone.setAttribute('xmlns', 'http://www.w3.org/2000/svg');
|
||||
if (state?.ready) {
|
||||
@@ -503,7 +334,6 @@ Extensions.register({
|
||||
const svgData = new XMLSerializer().serializeToString(clone);
|
||||
const svgBlob = new Blob([svgData], { type: 'image/svg+xml;charset=utf-8' });
|
||||
const url = URL.createObjectURL(svgBlob);
|
||||
const self = this;
|
||||
|
||||
const img = new Image();
|
||||
img.onload = () => {
|
||||
@@ -515,9 +345,8 @@ Extensions.register({
|
||||
ctx.scale(scale, scale);
|
||||
ctx.drawImage(img, 0, 0);
|
||||
URL.revokeObjectURL(url);
|
||||
|
||||
canvas.toBlob((blob) => {
|
||||
if (blob) self._download(blob, `diagram-${id}.png`);
|
||||
if (blob) _download(blob, `diagram-${id}.png`);
|
||||
}, 'image/png');
|
||||
};
|
||||
img.onerror = () => {
|
||||
@@ -525,9 +354,9 @@ Extensions.register({
|
||||
console.error('[Mermaid] PNG export failed');
|
||||
};
|
||||
img.src = url;
|
||||
},
|
||||
}
|
||||
|
||||
_download(blob, filename) {
|
||||
function _download(blob, filename) {
|
||||
const a = document.createElement('a');
|
||||
a.href = URL.createObjectURL(blob);
|
||||
a.download = filename;
|
||||
@@ -535,25 +364,24 @@ Extensions.register({
|
||||
a.click();
|
||||
document.body.removeChild(a);
|
||||
setTimeout(() => URL.revokeObjectURL(a.href), 1000);
|
||||
},
|
||||
}
|
||||
|
||||
_copySource(id) {
|
||||
function _copySource(id) {
|
||||
const code = document.querySelector(`[data-source="${id}"]`);
|
||||
if (!code) return;
|
||||
navigator.clipboard.writeText(code.textContent).then(() => {
|
||||
this._ctx.ui.toast('Source copied', 'success');
|
||||
if (window.sw?.toast) sw.toast('Source copied', 'success');
|
||||
});
|
||||
},
|
||||
}
|
||||
|
||||
// ── Diagram Rendering ───────────────────
|
||||
|
||||
async _renderDiagram(el) {
|
||||
async function _renderDiagram(el) {
|
||||
const code = decodeURIComponent(el.dataset.mermaidSrc);
|
||||
const id = el.dataset.diagram;
|
||||
|
||||
try {
|
||||
await this._loadMermaid();
|
||||
|
||||
await _loadMermaid();
|
||||
const svgId = 'mmd-svg-' + Math.random().toString(36).slice(2, 9);
|
||||
const { svg } = await mermaid.render(svgId, code);
|
||||
el.innerHTML = svg;
|
||||
@@ -566,28 +394,27 @@ Extensions.register({
|
||||
svgEl.style.maxWidth = 'none';
|
||||
svgEl.setAttribute('preserveAspectRatio', 'xMidYMid meet');
|
||||
}
|
||||
|
||||
this._initState(id);
|
||||
_initState(id);
|
||||
} catch (e) {
|
||||
el.innerHTML = `
|
||||
<div class="mermaid-error">
|
||||
<strong>Diagram error:</strong> ${this._escapeHtml(e.message || String(e))}
|
||||
<strong>Diagram error:</strong> ${_escapeHtml(e.message || String(e))}
|
||||
</div>
|
||||
`;
|
||||
el.dataset.rendered = 'error';
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
// ── Mermaid Library Loading ──────────────
|
||||
|
||||
_loadMermaid() {
|
||||
if (this._mermaidReady) return Promise.resolve();
|
||||
if (this._mermaidLoading) return this._mermaidLoading;
|
||||
function _loadMermaid() {
|
||||
if (_mermaidReady) return Promise.resolve();
|
||||
if (_mermaidLoading) return _mermaidLoading;
|
||||
|
||||
this._mermaidLoading = new Promise((resolve, reject) => {
|
||||
_mermaidLoading = new Promise((resolve, reject) => {
|
||||
if (typeof mermaid !== 'undefined') {
|
||||
this._initMermaid();
|
||||
this._mermaidReady = true;
|
||||
_initMermaid();
|
||||
_mermaidReady = true;
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
@@ -599,8 +426,8 @@ Extensions.register({
|
||||
const script = document.createElement('script');
|
||||
script.src = localSrc;
|
||||
script.onload = () => {
|
||||
this._initMermaid();
|
||||
this._mermaidReady = true;
|
||||
_initMermaid();
|
||||
_mermaidReady = true;
|
||||
resolve();
|
||||
};
|
||||
script.onerror = () => {
|
||||
@@ -608,8 +435,8 @@ Extensions.register({
|
||||
const cdn = document.createElement('script');
|
||||
cdn.src = cdnSrc;
|
||||
cdn.onload = () => {
|
||||
this._initMermaid();
|
||||
this._mermaidReady = true;
|
||||
_initMermaid();
|
||||
_mermaidReady = true;
|
||||
resolve();
|
||||
};
|
||||
cdn.onerror = () => {
|
||||
@@ -620,25 +447,23 @@ Extensions.register({
|
||||
};
|
||||
document.head.appendChild(script);
|
||||
});
|
||||
return _mermaidLoading;
|
||||
}
|
||||
|
||||
return this._mermaidLoading;
|
||||
},
|
||||
|
||||
_initMermaid() {
|
||||
function _initMermaid() {
|
||||
if (typeof mermaid === 'undefined') return;
|
||||
|
||||
mermaid.initialize({
|
||||
startOnLoad: false,
|
||||
theme: this._ctx.ui.isDark() ? 'dark' : 'default',
|
||||
theme: _isDark() ? 'dark' : 'default',
|
||||
securityLevel: 'strict',
|
||||
fontFamily: 'inherit',
|
||||
logLevel: 'error',
|
||||
});
|
||||
},
|
||||
}
|
||||
|
||||
// ── Styles ──────────────────────────────
|
||||
|
||||
_injectStyles() {
|
||||
function _injectStyles() {
|
||||
if (document.getElementById('ext-style-mermaid-renderer')) return;
|
||||
const style = document.createElement('style');
|
||||
style.id = 'ext-style-mermaid-renderer';
|
||||
@@ -647,8 +472,6 @@ Extensions.register({
|
||||
background: var(--bg-2); border: 1px solid var(--border);
|
||||
border-radius: 8px; overflow: hidden; margin: 12px 0;
|
||||
}
|
||||
|
||||
/* ── Fullscreen mode ── */
|
||||
.mermaid-block.mermaid-fullscreen {
|
||||
position: fixed; top: 0; left: 0; right: 0; bottom: 0;
|
||||
z-index: 10000; border-radius: 0; margin: 0;
|
||||
@@ -656,15 +479,11 @@ Extensions.register({
|
||||
background: var(--bg-1, var(--bg-2, #1a1a2e));
|
||||
}
|
||||
.mermaid-block.mermaid-fullscreen .mermaid-viewport {
|
||||
background: var(--bg-2, #1a1a2e);
|
||||
flex: 1; max-height: none;
|
||||
background: var(--bg-2, #1a1a2e); flex: 1; max-height: none;
|
||||
}
|
||||
.mermaid-block.mermaid-fullscreen .mermaid-toolbar {
|
||||
border-bottom: 1px solid var(--border);
|
||||
padding: 6px 14px;
|
||||
border-bottom: 1px solid var(--border); padding: 6px 14px;
|
||||
}
|
||||
|
||||
/* Fullscreen close button — always visible, critical for mobile */
|
||||
.mmd-fullscreen-close {
|
||||
position: absolute; top: 12px; right: 12px; z-index: 10001;
|
||||
width: 40px; height: 40px; border-radius: 50%;
|
||||
@@ -676,11 +495,8 @@ Extensions.register({
|
||||
box-shadow: 0 2px 8px rgba(0,0,0,0.3);
|
||||
}
|
||||
.mmd-fullscreen-close:hover {
|
||||
background: var(--bg-hover, rgba(255,255,255,0.15));
|
||||
transform: scale(1.1);
|
||||
background: var(--bg-hover, rgba(255,255,255,0.15)); transform: scale(1.1);
|
||||
}
|
||||
|
||||
/* Toolbar */
|
||||
.mermaid-toolbar {
|
||||
display: flex; align-items: center; gap: 6px;
|
||||
padding: 4px 10px; border-bottom: 1px solid var(--border);
|
||||
@@ -699,24 +515,14 @@ Extensions.register({
|
||||
}
|
||||
.mmd-btn:hover { color: var(--text-1); border-color: var(--text-3); background: var(--bg-3); }
|
||||
.mmd-sep { width: 1px; height: 16px; background: var(--border); margin: 0 4px; }
|
||||
|
||||
/* Viewport */
|
||||
.mermaid-viewport {
|
||||
overflow: hidden; position: relative;
|
||||
min-height: 80px; max-height: 600px;
|
||||
cursor: grab; user-select: none;
|
||||
}
|
||||
.mermaid-viewport.mmd-grabbing { cursor: grabbing; }
|
||||
|
||||
/* Diagram wrapper */
|
||||
.mermaid-diagram {
|
||||
width: 100%; height: 100%; min-height: 60px;
|
||||
}
|
||||
.mermaid-diagram svg {
|
||||
display: block; width: 100%; height: 100%;
|
||||
}
|
||||
|
||||
/* Loading / Error */
|
||||
.mermaid-diagram { width: 100%; height: 100%; min-height: 60px; }
|
||||
.mermaid-diagram svg { display: block; width: 100%; height: 100%; }
|
||||
.mermaid-loading {
|
||||
color: var(--text-3); font-size: 13px; padding: 20px;
|
||||
display: flex; align-items: center; justify-content: center; gap: 8px;
|
||||
@@ -732,8 +538,6 @@ Extensions.register({
|
||||
padding: 12px 16px; border-radius: 4px; font-size: 13px;
|
||||
font-family: var(--mono); white-space: pre-wrap;
|
||||
}
|
||||
|
||||
/* Source panel */
|
||||
.mermaid-source { border-top: 1px solid var(--border); font-size: 12px; }
|
||||
.mermaid-source summary {
|
||||
padding: 6px 12px; cursor: pointer; color: var(--text-3);
|
||||
@@ -746,34 +550,91 @@ Extensions.register({
|
||||
margin: 0; border-radius: 0; border: none;
|
||||
max-height: 200px; overflow: auto;
|
||||
}
|
||||
|
||||
/* Hide source panel in fullscreen */
|
||||
.mermaid-block.mermaid-fullscreen .mermaid-source { display: none; }
|
||||
|
||||
/* ── Mobile adjustments ── */
|
||||
@media (max-width: 768px) {
|
||||
.mermaid-toolbar { padding: 6px 10px; gap: 4px; }
|
||||
.mmd-btn { padding: 4px 10px; font-size: 12px; }
|
||||
.mmd-sep { margin: 0 2px; }
|
||||
.mmd-fullscreen-close { width: 48px; height: 48px; font-size: 24px; top: 16px; right: 16px; }
|
||||
}
|
||||
`;
|
||||
}`;
|
||||
document.head.appendChild(style);
|
||||
},
|
||||
|
||||
_escapeHtml(str) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = str;
|
||||
return div.innerHTML;
|
||||
},
|
||||
|
||||
destroy() {
|
||||
document.getElementById('ext-style-mermaid-renderer')?.remove();
|
||||
document.querySelectorAll('.mermaid-fullscreen').forEach(el => {
|
||||
el.classList.remove('mermaid-fullscreen');
|
||||
const closeBtn = el.querySelector('.mmd-fullscreen-close');
|
||||
if (closeBtn) closeBtn.remove();
|
||||
});
|
||||
document.body.style.overflow = '';
|
||||
}
|
||||
});
|
||||
|
||||
// ── Registration ────────────────────────
|
||||
|
||||
function register() {
|
||||
if (!window.sw?.renderers) return;
|
||||
|
||||
_injectStyles();
|
||||
|
||||
// Block renderer: match ```mermaid
|
||||
sw.renderers.register('mermaid', {
|
||||
type: 'block',
|
||||
pattern: 'mermaid',
|
||||
priority: 10,
|
||||
render(lang, code, container) {
|
||||
const id = 'mmd-' + Math.random().toString(36).slice(2, 9);
|
||||
container.innerHTML = `
|
||||
<div class="mermaid-block" data-mermaid-id="${id}">
|
||||
<div class="mermaid-toolbar">
|
||||
<span class="mermaid-title">\u{1f4ca} Diagram</span>
|
||||
<span class="mermaid-zoom-label" data-zoom-label="${id}">100%</span>
|
||||
<div class="mermaid-toolbar-btns">
|
||||
<button class="mmd-btn" data-action="zoom-in" data-target="${id}" title="Zoom in">+</button>
|
||||
<button class="mmd-btn" data-action="zoom-out" data-target="${id}" title="Zoom out">\u2212</button>
|
||||
<button class="mmd-btn" data-action="zoom-fit" data-target="${id}" title="Fit to view">\u22a1</button>
|
||||
<button class="mmd-btn" data-action="zoom-reset" data-target="${id}" title="Reset zoom">1:1</button>
|
||||
<span class="mmd-sep"></span>
|
||||
<button class="mmd-btn" data-action="expand" data-target="${id}" title="Fullscreen">\u26f6</button>
|
||||
<span class="mmd-sep"></span>
|
||||
<button class="mmd-btn" data-action="export-svg" data-target="${id}" title="Download SVG">SVG</button>
|
||||
<button class="mmd-btn" data-action="export-png" data-target="${id}" title="Download PNG">PNG</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mermaid-viewport" data-viewport="${id}">
|
||||
<div class="mermaid-diagram" data-mermaid-src="${encodeURIComponent(code.trim())}" data-diagram="${id}">
|
||||
<div class="mermaid-loading">
|
||||
<span class="mermaid-spinner"></span> Rendering diagram\u2026
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<details class="mermaid-source">
|
||||
<summary>
|
||||
<span>\u{1f4cb} View source</span>
|
||||
<button class="mmd-btn mmd-copy-src" data-action="copy-src" data-target="${id}" title="Copy source" onclick="event.stopPropagation()">Copy</button>
|
||||
</summary>
|
||||
<pre><code class="language-mermaid" data-source="${id}">${_escapeHtml(code.trim())}</code></pre>
|
||||
</details>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
});
|
||||
|
||||
// Post renderer: render diagrams + wire interactivity
|
||||
sw.renderers.register('mermaid-post', {
|
||||
type: 'post',
|
||||
priority: 10,
|
||||
render(container) {
|
||||
const diagrams = container.querySelectorAll('.mermaid-diagram[data-mermaid-src]');
|
||||
if (diagrams.length === 0) return;
|
||||
|
||||
diagrams.forEach(el => {
|
||||
if (el.dataset.rendered) return;
|
||||
el.dataset.rendered = 'pending';
|
||||
_renderDiagram(el);
|
||||
});
|
||||
|
||||
_wireToolbar(container);
|
||||
}
|
||||
});
|
||||
|
||||
_loadMermaid();
|
||||
}
|
||||
|
||||
// Boot: register immediately if SDK ready, otherwise listen
|
||||
if (window.sw?._sdk) {
|
||||
register();
|
||||
} else {
|
||||
document.addEventListener('sw:ready', register, { once: true });
|
||||
}
|
||||
})();
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"tier": "browser",
|
||||
"author": "switchboard",
|
||||
"description": "Renders ```mermaid code blocks as interactive SVG diagrams with zoom/pan, SVG/PNG export, and source copy",
|
||||
"requires": ["chat"],
|
||||
"requires": [],
|
||||
"permissions": [],
|
||||
"tools": [],
|
||||
"surfaces": [],
|
||||
|
||||
53
packages/notes/README.md
Normal file
53
packages/notes/README.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# Notes
|
||||
|
||||
Markdown notes surface with sidebar navigation, live preview, and search.
|
||||
|
||||
## Status: v0.1.0
|
||||
|
||||
Core CRUD with markdown textarea and inline preview.
|
||||
|
||||
## Features
|
||||
|
||||
- Create, edit, archive notes
|
||||
- Markdown editing with live preview toggle
|
||||
- Auto-save with debounce (1s)
|
||||
- Pin important notes to top
|
||||
- Full-text search across title and body
|
||||
- Light/dark theme support
|
||||
|
||||
## Planned
|
||||
|
||||
- v0.4.1: Folders + navigation tree
|
||||
- v0.4.2: Tags + enhanced search
|
||||
- v0.4.3: Backlinks + `[[wikilinks]]`
|
||||
- v0.4.4: Rich editor (CodeMirror 6) + import/export
|
||||
- v0.4.5: Editor modes (rendered/edit/split) + document outline
|
||||
|
||||
## Data
|
||||
|
||||
Single `notes` table in ext_data:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| title | text | Note title |
|
||||
| body | text | Markdown content |
|
||||
| folder_id | text | Folder reference (v0.4.1) |
|
||||
| pinned | int | 0 or 1 |
|
||||
| archived | int | Soft delete flag |
|
||||
|
||||
## API
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | /notes | List notes (lightweight, no body) |
|
||||
| POST | /notes | Create note |
|
||||
| GET | /notes/:id | Get note with full body |
|
||||
| PUT | /notes/:id | Update note |
|
||||
| DELETE | /notes/:id | Archive (or hard delete with ?hard=1) |
|
||||
| GET | /search?q=term | Search title and body |
|
||||
| GET | /stats | Note counts |
|
||||
|
||||
## Keyboard Shortcuts
|
||||
|
||||
- `Ctrl/Cmd+S` — Force save
|
||||
- `Tab` — Insert two spaces
|
||||
795
packages/notes/css/main.css
Normal file
795
packages/notes/css/main.css
Normal file
@@ -0,0 +1,795 @@
|
||||
/* ── Notes Surface Styles ─────────────────── */
|
||||
|
||||
.notes-app {
|
||||
display: flex;
|
||||
height: 100%;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
/* ── Sidebar ─────────────────────────────── */
|
||||
.notes-sidebar {
|
||||
width: 280px;
|
||||
min-width: 220px;
|
||||
border-right: 1px solid var(--border);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
background: var(--bg-raised);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.notes-sidebar__header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 12px 14px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.notes-sidebar__title {
|
||||
font-size: 14px;
|
||||
font-weight: 600;
|
||||
color: var(--text);
|
||||
flex: 1;
|
||||
}
|
||||
.notes-sidebar__count {
|
||||
font-size: 12px;
|
||||
color: var(--text-3);
|
||||
}
|
||||
.notes-sidebar__actions {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
/* ── Folder Tree ────────────────────────── */
|
||||
.folder-tree {
|
||||
padding: 4px 0;
|
||||
border-bottom: 1px solid var(--border);
|
||||
flex-shrink: 0;
|
||||
max-height: 40%;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.folder-tree__item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
padding: 5px 12px;
|
||||
font-size: 13px;
|
||||
color: var(--text-2);
|
||||
cursor: pointer;
|
||||
border-radius: var(--radius);
|
||||
margin: 1px 6px;
|
||||
transition: var(--transition);
|
||||
user-select: none;
|
||||
}
|
||||
.folder-tree__item:hover {
|
||||
background: var(--bg-hover);
|
||||
color: var(--text);
|
||||
}
|
||||
.folder-tree__item--active {
|
||||
background: var(--bg-active);
|
||||
color: var(--text);
|
||||
font-weight: 500;
|
||||
}
|
||||
.folder-tree__toggle {
|
||||
width: 14px;
|
||||
font-size: 9px;
|
||||
text-align: center;
|
||||
flex-shrink: 0;
|
||||
color: var(--text-3);
|
||||
}
|
||||
.folder-tree__icon {
|
||||
font-size: 13px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.folder-tree__name {
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
flex: 1;
|
||||
}
|
||||
.folder-tree__edit {
|
||||
flex: 1;
|
||||
font-size: 13px;
|
||||
padding: 1px 4px;
|
||||
border: 1px solid var(--accent);
|
||||
border-radius: 3px;
|
||||
background: var(--bg-surface);
|
||||
color: var(--text);
|
||||
font-family: var(--font);
|
||||
outline: none;
|
||||
}
|
||||
.folder-tree__add {
|
||||
padding: 6px 12px;
|
||||
font-size: 12px;
|
||||
color: var(--text-3);
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
}
|
||||
.folder-tree__add:hover {
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
/* ── Search ──────────────────────────────── */
|
||||
.notes-search {
|
||||
padding: 8px 14px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.notes-search input {
|
||||
width: 100%;
|
||||
padding: 6px 10px;
|
||||
font-size: 13px;
|
||||
background: var(--bg-surface);
|
||||
color: var(--text);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
font-family: var(--font);
|
||||
}
|
||||
.notes-search input:focus {
|
||||
outline: none;
|
||||
border-color: var(--accent);
|
||||
}
|
||||
.notes-search input::placeholder {
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
/* ── Note List ───────────────────────────── */
|
||||
.notes-list {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
padding: 4px 8px;
|
||||
}
|
||||
.notes-list__empty {
|
||||
padding: 40px 16px;
|
||||
text-align: center;
|
||||
color: var(--text-3);
|
||||
font-size: 13px;
|
||||
}
|
||||
.notes-list__loading {
|
||||
padding: 40px 0;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
/* ── Note Card (sidebar item) ────────────── */
|
||||
.note-card {
|
||||
padding: 10px 12px;
|
||||
border-radius: var(--radius);
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
margin-bottom: 2px;
|
||||
}
|
||||
.note-card:hover {
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
.note-card--active {
|
||||
background: var(--bg-active);
|
||||
}
|
||||
.note-card__title {
|
||||
font-size: 13px;
|
||||
font-weight: 500;
|
||||
color: var(--text);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.note-card__pin {
|
||||
font-size: 11px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.note-card__snippet {
|
||||
font-size: 12px;
|
||||
color: var(--text-3);
|
||||
margin-top: 3px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
line-height: 1.4;
|
||||
}
|
||||
.note-card__date {
|
||||
font-size: 11px;
|
||||
color: var(--text-3);
|
||||
margin-top: 3px;
|
||||
}
|
||||
|
||||
/* ── Editor Pane ─────────────────────────── */
|
||||
.notes-editor {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-width: 0;
|
||||
overflow: hidden;
|
||||
}
|
||||
.notes-editor__empty {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
color: var(--text-3);
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
/* ── Editor Header ───────────────────────── */
|
||||
.notes-editor__header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 10px 20px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.notes-editor__title-input {
|
||||
flex: 1;
|
||||
font-size: 18px;
|
||||
font-weight: 600;
|
||||
color: var(--text);
|
||||
background: transparent;
|
||||
border: none;
|
||||
outline: none;
|
||||
padding: 4px 0;
|
||||
font-family: var(--font);
|
||||
}
|
||||
.notes-editor__title-input::placeholder {
|
||||
color: var(--text-3);
|
||||
}
|
||||
.notes-editor__actions {
|
||||
display: flex;
|
||||
gap: 6px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
/* ── Folder select in editor ────────────── */
|
||||
.notes-editor__folder-select {
|
||||
font-size: 12px;
|
||||
padding: 3px 6px;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
background: var(--bg-raised);
|
||||
color: var(--text-2);
|
||||
font-family: var(--font);
|
||||
cursor: pointer;
|
||||
max-width: 140px;
|
||||
}
|
||||
.notes-editor__folder-select:focus {
|
||||
outline: none;
|
||||
border-color: var(--accent);
|
||||
}
|
||||
|
||||
/* ── Editor Content (body + outline wrapper) */
|
||||
.notes-editor__content {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
min-height: 0;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
/* ── Editor Body ─────────────────────────── */
|
||||
.notes-editor__body {
|
||||
flex: 1;
|
||||
display: flex;
|
||||
min-height: 0;
|
||||
overflow: hidden;
|
||||
}
|
||||
.notes-editor__body--split {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
}
|
||||
.notes-editor__body--split .notes-preview {
|
||||
border-left: 1px solid var(--border);
|
||||
}
|
||||
.notes-editor__textarea {
|
||||
flex: 1;
|
||||
resize: none;
|
||||
padding: 20px;
|
||||
font-size: 14px;
|
||||
line-height: 1.6;
|
||||
color: var(--text);
|
||||
background: var(--bg-surface);
|
||||
border: none;
|
||||
outline: none;
|
||||
font-family: var(--mono);
|
||||
tab-size: 2;
|
||||
}
|
||||
.notes-editor__textarea::placeholder {
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
/* ── CodeMirror 6 container ─────────────── */
|
||||
.notes-editor__cm {
|
||||
flex: 1;
|
||||
overflow: hidden;
|
||||
}
|
||||
.notes-editor__cm .cm-editor {
|
||||
height: 100%;
|
||||
max-height: none;
|
||||
}
|
||||
.notes-editor__cm .cm-scroller {
|
||||
overflow: auto;
|
||||
padding: 12px 20px;
|
||||
}
|
||||
|
||||
/* ── Preview ─────────────────────────────── */
|
||||
.notes-preview {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
padding: 20px;
|
||||
border-left: 1px solid var(--border);
|
||||
background: var(--bg-surface);
|
||||
}
|
||||
.notes-preview h1 { font-size: 24px; font-weight: 700; color: var(--text); margin: 0 0 12px; }
|
||||
.notes-preview h2 { font-size: 20px; font-weight: 600; color: var(--text); margin: 20px 0 8px; }
|
||||
.notes-preview h3 { font-size: 16px; font-weight: 600; color: var(--text); margin: 16px 0 6px; }
|
||||
.notes-preview p { font-size: 14px; color: var(--text); line-height: 1.7; margin: 0 0 10px; }
|
||||
.notes-preview ul, .notes-preview ol { padding-left: 24px; margin: 0 0 10px; color: var(--text); font-size: 14px; line-height: 1.7; }
|
||||
.notes-preview li { margin-bottom: 2px; }
|
||||
.notes-preview code {
|
||||
font-family: var(--mono);
|
||||
font-size: 13px;
|
||||
background: var(--bg-raised);
|
||||
padding: 2px 6px;
|
||||
border-radius: 3px;
|
||||
color: var(--accent);
|
||||
}
|
||||
.notes-preview pre {
|
||||
background: var(--bg-raised);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: 14px;
|
||||
overflow-x: auto;
|
||||
margin: 0 0 12px;
|
||||
}
|
||||
.notes-preview pre code {
|
||||
background: none;
|
||||
padding: 0;
|
||||
color: var(--text);
|
||||
font-size: 13px;
|
||||
}
|
||||
.notes-preview blockquote {
|
||||
border-left: 3px solid var(--accent);
|
||||
margin: 0 0 12px;
|
||||
padding: 4px 16px;
|
||||
color: var(--text-2);
|
||||
}
|
||||
.notes-preview a { color: var(--accent); text-decoration: none; }
|
||||
.notes-preview a:hover { text-decoration: underline; }
|
||||
.notes-preview hr {
|
||||
border: none;
|
||||
border-top: 1px solid var(--border);
|
||||
margin: 16px 0;
|
||||
}
|
||||
.notes-preview strong { font-weight: 600; }
|
||||
.notes-preview em { font-style: italic; }
|
||||
|
||||
/* ── Toggle button ───────────────────────── */
|
||||
.notes-toggle {
|
||||
padding: 4px 10px;
|
||||
font-size: 12px;
|
||||
border: 1px solid var(--border);
|
||||
background: var(--bg-raised);
|
||||
color: var(--text-2);
|
||||
border-radius: var(--radius);
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
}
|
||||
.notes-toggle:hover { background: var(--bg-hover); color: var(--text); }
|
||||
.notes-toggle--active { background: var(--accent); color: #fff; border-color: var(--accent); }
|
||||
|
||||
/* ── Inline buttons ──────────────────────── */
|
||||
.notes-btn {
|
||||
border: none;
|
||||
background: var(--bg-raised);
|
||||
color: var(--text-2);
|
||||
cursor: pointer;
|
||||
border-radius: var(--radius);
|
||||
transition: var(--transition);
|
||||
padding: 4px 10px;
|
||||
font-size: 13px;
|
||||
}
|
||||
.notes-btn:hover { background: var(--bg-hover); color: var(--text); }
|
||||
.notes-btn--danger:hover { background: var(--danger-dim); color: var(--danger); }
|
||||
.notes-btn--accent { background: var(--accent); color: #fff; }
|
||||
.notes-btn--accent:hover { opacity: 0.9; color: #fff; }
|
||||
|
||||
/* ── Saved indicator ─────────────────────── */
|
||||
.notes-saved {
|
||||
font-size: 12px;
|
||||
color: var(--text-3);
|
||||
padding: 2px 8px;
|
||||
}
|
||||
.notes-saved--dirty {
|
||||
color: var(--warning);
|
||||
}
|
||||
|
||||
/* ── Tag Pills (shared) ─────────────────── */
|
||||
.tag-pill {
|
||||
display: inline-block;
|
||||
padding: 1px 8px;
|
||||
font-size: 11px;
|
||||
background: var(--bg-raised);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 10px;
|
||||
color: var(--text-2);
|
||||
margin: 2px 2px;
|
||||
white-space: nowrap;
|
||||
cursor: pointer;
|
||||
line-height: 1.6;
|
||||
transition: var(--transition);
|
||||
}
|
||||
.tag-pill:hover { background: var(--bg-hover); color: var(--text); }
|
||||
.tag-pill--active { background: var(--accent); color: #fff; border-color: var(--accent); }
|
||||
.tag-pill__remove {
|
||||
margin-left: 4px;
|
||||
font-size: 10px;
|
||||
opacity: 0.6;
|
||||
cursor: pointer;
|
||||
}
|
||||
.tag-pill__remove:hover { opacity: 1; }
|
||||
|
||||
/* ── Tag Filter (sidebar) ──────────────── */
|
||||
.tag-filter {
|
||||
padding: 6px 14px 4px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.tag-filter__label {
|
||||
font-size: 11px;
|
||||
color: var(--text-3);
|
||||
margin-bottom: 3px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
}
|
||||
.tag-filter__clear {
|
||||
font-size: 10px;
|
||||
color: var(--accent);
|
||||
cursor: pointer;
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 0;
|
||||
font-family: var(--font);
|
||||
}
|
||||
.tag-filter__clear:hover { text-decoration: underline; }
|
||||
.tag-filter__pills {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 3px;
|
||||
max-height: 60px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.tag-filter__pills .tag-pill { font-size: 10px; padding: 0 6px; }
|
||||
|
||||
/* ── Tag Input (editor) ────────────────── */
|
||||
.tag-input {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 4px;
|
||||
align-items: center;
|
||||
padding: 4px 20px 6px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
position: relative;
|
||||
}
|
||||
.tag-input__field {
|
||||
border: none;
|
||||
outline: none;
|
||||
font-size: 12px;
|
||||
background: transparent;
|
||||
color: var(--text);
|
||||
min-width: 80px;
|
||||
flex: 1;
|
||||
padding: 2px 0;
|
||||
font-family: var(--font);
|
||||
}
|
||||
.tag-input__field::placeholder { color: var(--text-3); }
|
||||
.tag-input .tag-pill { font-size: 11px; }
|
||||
|
||||
/* ── Tag Autocomplete ──────────────────── */
|
||||
.tag-autocomplete {
|
||||
position: absolute;
|
||||
top: 100%;
|
||||
left: 20px;
|
||||
background: var(--bg-surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
max-height: 120px;
|
||||
overflow-y: auto;
|
||||
z-index: 10;
|
||||
box-shadow: 0 4px 12px rgba(0,0,0,0.1);
|
||||
min-width: 140px;
|
||||
}
|
||||
.tag-autocomplete__item {
|
||||
padding: 4px 10px;
|
||||
font-size: 12px;
|
||||
cursor: pointer;
|
||||
color: var(--text);
|
||||
}
|
||||
.tag-autocomplete__item:hover { background: var(--bg-hover); }
|
||||
|
||||
/* ── Note Card Tags ────────────────────── */
|
||||
.note-card__tags {
|
||||
display: flex;
|
||||
gap: 3px;
|
||||
margin-top: 3px;
|
||||
overflow: hidden;
|
||||
}
|
||||
.note-card__tags .tag-pill { font-size: 10px; padding: 0 6px; cursor: default; }
|
||||
.note-card__tags .tag-pill--overflow {
|
||||
background: transparent;
|
||||
border: none;
|
||||
color: var(--text-3);
|
||||
padding: 0 2px;
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
/* ── Folder Context Menu ───────────────── */
|
||||
.folder-context-menu {
|
||||
position: fixed;
|
||||
z-index: 100;
|
||||
background: var(--bg-surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
box-shadow: 0 4px 16px rgba(0,0,0,0.2);
|
||||
min-width: 150px;
|
||||
padding: 4px 0;
|
||||
}
|
||||
.folder-context-menu__item {
|
||||
padding: 6px 14px;
|
||||
font-size: 13px;
|
||||
color: var(--text);
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
}
|
||||
.folder-context-menu__item:hover {
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
.folder-context-menu__item--danger {
|
||||
color: var(--danger);
|
||||
}
|
||||
.folder-context-menu__item--danger:hover {
|
||||
background: var(--danger-dim);
|
||||
}
|
||||
|
||||
/* ── Drag and Drop ─────────────────────── */
|
||||
.note-card--dragging { opacity: 0.4; }
|
||||
.folder-tree__item--drop-target {
|
||||
background: color-mix(in srgb, var(--accent) 10%, transparent) !important;
|
||||
outline: 2px dashed var(--accent);
|
||||
outline-offset: -2px;
|
||||
}
|
||||
|
||||
/* ── Wikilinks ──────────────────────────── */
|
||||
.wikilink {
|
||||
color: var(--accent);
|
||||
text-decoration: none;
|
||||
border-bottom: 1px dashed var(--accent);
|
||||
cursor: pointer;
|
||||
}
|
||||
.wikilink:hover {
|
||||
border-bottom-style: solid;
|
||||
}
|
||||
.wikilink.wikilink--unresolved {
|
||||
color: var(--danger, #e53e3e);
|
||||
border-bottom-color: var(--danger, #e53e3e);
|
||||
opacity: 0.7;
|
||||
}
|
||||
.wikilink.wikilink--unresolved:hover {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
/* ── Backlinks Panel ────────────────────── */
|
||||
.backlinks-panel {
|
||||
border-top: 1px solid var(--border);
|
||||
padding: 10px 16px;
|
||||
background: var(--bg-raised);
|
||||
flex-shrink: 0;
|
||||
max-height: 200px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
.backlinks-panel__header {
|
||||
font-size: 11px;
|
||||
font-weight: 600;
|
||||
color: var(--text-2);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
margin-bottom: 6px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
cursor: pointer;
|
||||
user-select: none;
|
||||
}
|
||||
.backlinks-panel__toggle {
|
||||
font-size: 9px;
|
||||
color: var(--text-2);
|
||||
}
|
||||
.backlinks-panel__count {
|
||||
background: var(--bg-hover);
|
||||
color: var(--text-2);
|
||||
font-size: 10px;
|
||||
padding: 0 6px;
|
||||
border-radius: 8px;
|
||||
line-height: 16px;
|
||||
}
|
||||
.backlinks-panel__item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 4px 8px;
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
font-size: 13px;
|
||||
}
|
||||
.backlinks-panel__item:hover {
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
.backlinks-panel__title {
|
||||
color: var(--accent);
|
||||
font-weight: 500;
|
||||
}
|
||||
.backlinks-panel__context {
|
||||
color: var(--text-2);
|
||||
font-size: 11px;
|
||||
}
|
||||
|
||||
/* ── Sidebar Tabs ────────────────────────── */
|
||||
.sidebar-tabs {
|
||||
display: flex;
|
||||
border-bottom: 1px solid var(--border);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.sidebar-tabs__tab {
|
||||
flex: 1;
|
||||
padding: 10px 14px;
|
||||
font-size: 13px;
|
||||
font-weight: 500;
|
||||
color: var(--text-2);
|
||||
background: none;
|
||||
border: none;
|
||||
border-bottom: 2px solid transparent;
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
text-align: center;
|
||||
font-family: var(--font);
|
||||
}
|
||||
.sidebar-tabs__tab:hover {
|
||||
color: var(--text);
|
||||
background: var(--bg-hover);
|
||||
}
|
||||
.sidebar-tabs__tab--active {
|
||||
color: var(--text);
|
||||
border-bottom-color: var(--accent);
|
||||
}
|
||||
.sidebar-tabs__tab--disabled {
|
||||
color: var(--text-3);
|
||||
cursor: default;
|
||||
opacity: 0.5;
|
||||
}
|
||||
.sidebar-tabs__tab--disabled:hover {
|
||||
background: none;
|
||||
color: var(--text-3);
|
||||
}
|
||||
|
||||
/* ── Sidebar Outline ─────────────────────── */
|
||||
.sidebar-outline {
|
||||
flex: 1;
|
||||
overflow-y: auto;
|
||||
padding: 8px 0;
|
||||
}
|
||||
.sidebar-outline__empty {
|
||||
padding: 40px 16px;
|
||||
text-align: center;
|
||||
color: var(--text-3);
|
||||
font-size: 13px;
|
||||
}
|
||||
.sidebar-outline__item {
|
||||
padding: 5px 12px;
|
||||
font-size: 13px;
|
||||
color: var(--text-2);
|
||||
cursor: pointer;
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
transition: var(--transition);
|
||||
border-radius: var(--radius);
|
||||
margin: 1px 6px;
|
||||
}
|
||||
.sidebar-outline__item:hover {
|
||||
background: var(--bg-hover);
|
||||
color: var(--text);
|
||||
}
|
||||
.sidebar-outline__item--active {
|
||||
background: var(--bg-active);
|
||||
color: var(--text);
|
||||
font-weight: 500;
|
||||
}
|
||||
.sidebar-outline__item--h1 { font-weight: 600; color: var(--text); }
|
||||
.sidebar-outline__item--h2 { font-weight: 500; }
|
||||
|
||||
/* ── Graph Pane ─────────────────────────── */
|
||||
.graph-pane {
|
||||
flex: 1;
|
||||
position: relative;
|
||||
overflow: hidden;
|
||||
background: var(--bg-surface, #fff);
|
||||
min-height: 0;
|
||||
}
|
||||
.graph-pane canvas {
|
||||
display: block;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
cursor: grab;
|
||||
}
|
||||
.graph-pane canvas:active { cursor: grabbing; }
|
||||
.graph-toolbar {
|
||||
position: absolute;
|
||||
top: 12px;
|
||||
right: 12px;
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
align-items: center;
|
||||
z-index: 5;
|
||||
font-size: 12px;
|
||||
color: var(--text-2, #666);
|
||||
}
|
||||
.graph-toolbar label {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
cursor: pointer;
|
||||
user-select: none;
|
||||
}
|
||||
.graph-toolbar input[type="checkbox"] { margin: 0; }
|
||||
.graph-tooltip {
|
||||
position: absolute;
|
||||
pointer-events: none;
|
||||
background: var(--bg-raised, #fff);
|
||||
border: 1px solid var(--border, #ddd);
|
||||
border-radius: var(--radius, 6px);
|
||||
padding: 8px 12px;
|
||||
box-shadow: 0 2px 8px rgba(0,0,0,0.12);
|
||||
z-index: 10;
|
||||
max-width: 220px;
|
||||
font-size: 12px;
|
||||
}
|
||||
.graph-tooltip__title {
|
||||
font-weight: 600;
|
||||
font-size: 13px;
|
||||
margin-bottom: 2px;
|
||||
}
|
||||
.graph-tooltip__tags {
|
||||
display: flex;
|
||||
gap: 4px;
|
||||
flex-wrap: wrap;
|
||||
margin-top: 4px;
|
||||
}
|
||||
.graph-tooltip__tags .tag-pill {
|
||||
font-size: 10px;
|
||||
padding: 1px 6px;
|
||||
border-radius: 8px;
|
||||
background: var(--bg-hover, #eee);
|
||||
color: var(--text-2, #666);
|
||||
}
|
||||
.graph-tooltip__edges {
|
||||
font-size: 11px;
|
||||
color: var(--text-3, #999);
|
||||
margin-top: 4px;
|
||||
}
|
||||
.graph-empty {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 100%;
|
||||
color: var(--text-3, #999);
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
/* ── Responsive ──────────────────────────── */
|
||||
@media (max-width: 700px) {
|
||||
.notes-sidebar { width: 100%; border-right: none; border-bottom: 1px solid var(--border); max-height: 40vh; }
|
||||
.notes-app { flex-direction: column; }
|
||||
.folder-tree { max-height: 30vh; }
|
||||
.notes-editor__body--split { grid-template-columns: 1fr; }
|
||||
.notes-editor__body--split .notes-preview { display: none; }
|
||||
.graph-pane { min-height: 50vh; }
|
||||
}
|
||||
1801
packages/notes/js/main.js
Normal file
1801
packages/notes/js/main.js
Normal file
File diff suppressed because it is too large
Load Diff
104
packages/notes/manifest.json
Normal file
104
packages/notes/manifest.json
Normal file
@@ -0,0 +1,104 @@
|
||||
{
|
||||
"id": "notes",
|
||||
"title": "Notes",
|
||||
"type": "full",
|
||||
"tier": "starlark",
|
||||
"route": "/s/notes",
|
||||
"auth": "authenticated",
|
||||
"layout": "single",
|
||||
"version": "0.8.0",
|
||||
"icon": "📝",
|
||||
"description": "Markdown notes surface with rich editor, folders, tags, backlinks, import/export, sidebar tabs, document outline, and note graph.",
|
||||
"author": "switchboard",
|
||||
|
||||
"permissions": ["db.write"],
|
||||
|
||||
"api_routes": [
|
||||
{"method": "GET", "path": "/notes"},
|
||||
{"method": "POST", "path": "/notes"},
|
||||
{"method": "GET", "path": "/notes/*"},
|
||||
{"method": "PUT", "path": "/notes/*"},
|
||||
{"method": "DELETE", "path": "/notes/*"},
|
||||
{"method": "GET", "path": "/search"},
|
||||
{"method": "GET", "path": "/stats"},
|
||||
{"method": "GET", "path": "/graph"},
|
||||
{"method": "GET", "path": "/folders"},
|
||||
{"method": "POST", "path": "/folders"},
|
||||
{"method": "PUT", "path": "/folders/*"},
|
||||
{"method": "DELETE", "path": "/folders/*"},
|
||||
{"method": "POST", "path": "/notes/move"},
|
||||
{"method": "GET", "path": "/tags"},
|
||||
{"method": "GET", "path": "/tags/*"},
|
||||
{"method": "PUT", "path": "/tags/*"},
|
||||
{"method": "GET", "path": "/links/*"},
|
||||
{"method": "GET", "path": "/backlinks/*"}
|
||||
],
|
||||
|
||||
"db_tables": {
|
||||
"notes": {
|
||||
"columns": {
|
||||
"title": "text",
|
||||
"body": "text",
|
||||
"folder_id": "text",
|
||||
"creator_id": "text",
|
||||
"updated_at": "text",
|
||||
"pinned": "int",
|
||||
"archived": "int"
|
||||
},
|
||||
"indexes": [
|
||||
["folder_id"],
|
||||
["creator_id"],
|
||||
["pinned"],
|
||||
["updated_at"]
|
||||
]
|
||||
},
|
||||
"tags": {
|
||||
"columns": {
|
||||
"note_id": "text",
|
||||
"tag": "text"
|
||||
},
|
||||
"indexes": [
|
||||
["note_id"],
|
||||
["tag"]
|
||||
]
|
||||
},
|
||||
"links": {
|
||||
"columns": {
|
||||
"source_id": "text",
|
||||
"target_id": "text",
|
||||
"link_text": "text"
|
||||
},
|
||||
"indexes": [
|
||||
["source_id"],
|
||||
["target_id"]
|
||||
]
|
||||
},
|
||||
"folders": {
|
||||
"columns": {
|
||||
"name": "text",
|
||||
"parent_id": "text",
|
||||
"creator_id": "text",
|
||||
"sort_order": "int"
|
||||
},
|
||||
"indexes": [
|
||||
["parent_id"],
|
||||
["creator_id"]
|
||||
]
|
||||
}
|
||||
},
|
||||
|
||||
"settings": {
|
||||
"default_view": {
|
||||
"type": "string",
|
||||
"label": "Default View",
|
||||
"description": "Default note list view: recent or pinned",
|
||||
"default": "recent"
|
||||
},
|
||||
"editor_mode": {
|
||||
"type": "string",
|
||||
"label": "Default Editor Mode",
|
||||
"description": "How notes open: rendered (read-only preview), edit (CodeMirror editor), or split (side-by-side)",
|
||||
"default": "rendered"
|
||||
}
|
||||
}
|
||||
}
|
||||
668
packages/notes/script.star
Normal file
668
packages/notes/script.star
Normal file
@@ -0,0 +1,668 @@
|
||||
# Notes — Starlark Backend (v0.4.0)
|
||||
#
|
||||
# Markdown notes surface using ext_data.
|
||||
#
|
||||
# Entry points:
|
||||
# on_request(req) → surface API routes
|
||||
#
|
||||
# Modules: db, json, settings
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Helpers
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _resp(status, data):
|
||||
return {"status": status, "body": json.encode(data), "headers": {"Content-Type": "application/json"}}
|
||||
|
||||
def _str(v):
|
||||
if v == None:
|
||||
return ""
|
||||
return str(v)
|
||||
|
||||
def _int(v):
|
||||
if v == None:
|
||||
return 0
|
||||
s = str(v)
|
||||
if not s:
|
||||
return 0
|
||||
return int(s)
|
||||
|
||||
def _snippet(body, max_len = 120):
|
||||
"""First max_len chars of body, stripped of leading #/whitespace."""
|
||||
if not body:
|
||||
return ""
|
||||
line = body.split("\n")[0]
|
||||
# strip leading markdown heading markers
|
||||
line = line.lstrip("#").strip()
|
||||
if len(line) > max_len:
|
||||
return line[:max_len]
|
||||
return line
|
||||
|
||||
def _tags_for_notes(note_ids):
|
||||
"""Batch-fetch tags for a list of note IDs. Returns {note_id: [tag, ...]}."""
|
||||
if not note_ids:
|
||||
return {}
|
||||
all_tags = db.query("tags", limit=5000)
|
||||
result = {}
|
||||
for t in (all_tags or []):
|
||||
nid = _str(t.get("note_id", ""))
|
||||
tag = _str(t.get("tag", ""))
|
||||
if nid and tag:
|
||||
if nid not in result:
|
||||
result[nid] = []
|
||||
result[nid].append(tag)
|
||||
return result
|
||||
|
||||
|
||||
def _extract_wikilinks(text):
|
||||
"""Parse [[...]] patterns from text. Returns deduplicated list of link_text strings."""
|
||||
links = []
|
||||
seen = {}
|
||||
parts = text.split("[[")
|
||||
# parts[0] is before any wikilink; parts[1:] each start after "[["
|
||||
for i in range(1, len(parts)):
|
||||
chunk = parts[i]
|
||||
end = chunk.find("]]")
|
||||
if end > 0:
|
||||
link_text = chunk[:end].strip()
|
||||
lt_lower = link_text.lower()
|
||||
if link_text and lt_lower not in seen:
|
||||
seen[lt_lower] = True
|
||||
links.append(link_text)
|
||||
return links
|
||||
|
||||
|
||||
def _sync_links(source_id, body):
|
||||
"""Extract wikilinks from body, resolve to note IDs, replace link rows."""
|
||||
raw_links = _extract_wikilinks(_str(body))
|
||||
if not raw_links and not db.query("links", filters={"source_id": source_id}, limit=1):
|
||||
return # nothing to do
|
||||
|
||||
# resolve titles to note IDs (case-insensitive)
|
||||
all_notes = db.query("notes", filters={"archived": 0}, limit=5000)
|
||||
title_map = {}
|
||||
for n in (all_notes or []):
|
||||
t = _str(n.get("title", "")).lower().strip()
|
||||
if t:
|
||||
title_map[t] = n.get("id", "")
|
||||
|
||||
# delete existing links for this source
|
||||
old_links = db.query("links", filters={"source_id": source_id}, limit=500)
|
||||
for link in (old_links or []):
|
||||
db.delete("links", link["id"])
|
||||
|
||||
# insert new links
|
||||
for lt in raw_links:
|
||||
target_id = title_map.get(lt.lower().strip(), "")
|
||||
# skip self-links
|
||||
if target_id == source_id:
|
||||
target_id = ""
|
||||
db.insert("links", {
|
||||
"source_id": source_id,
|
||||
"target_id": target_id,
|
||||
"link_text": lt,
|
||||
})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Surface API routes (/s/notes/api/*)
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def on_request(req):
|
||||
path = req["path"]
|
||||
method = req["method"]
|
||||
|
||||
# GET /notes — list
|
||||
if method == "GET" and path == "/notes":
|
||||
return _list_notes(req)
|
||||
|
||||
# POST /notes — create
|
||||
if method == "POST" and path == "/notes":
|
||||
return _create_note(req)
|
||||
|
||||
# GET /stats
|
||||
if method == "GET" and path == "/stats":
|
||||
return _get_stats()
|
||||
|
||||
# GET /search
|
||||
if method == "GET" and path == "/search":
|
||||
return _search_notes(req)
|
||||
|
||||
# POST /notes/move (must be before /notes/:id catch-all)
|
||||
if method == "POST" and path == "/notes/move":
|
||||
return _move_note(req)
|
||||
|
||||
# GET /graph — full graph data for visualization
|
||||
if method == "GET" and path == "/graph":
|
||||
return _get_graph()
|
||||
|
||||
# ── Link routes ──────────────────────────
|
||||
# GET /links/:note_id — outgoing links
|
||||
if method == "GET" and path.startswith("/links/"):
|
||||
return _get_links(path[len("/links/"):])
|
||||
|
||||
# GET /backlinks/:note_id — incoming links
|
||||
if method == "GET" and path.startswith("/backlinks/"):
|
||||
return _get_backlinks(path[len("/backlinks/"):])
|
||||
|
||||
# ── Tag routes ────────────────────────────
|
||||
# GET /tags — all unique tags
|
||||
if method == "GET" and path == "/tags":
|
||||
return _list_tags()
|
||||
|
||||
# GET /tags/:note_id
|
||||
if method == "GET" and path.startswith("/tags/"):
|
||||
return _get_note_tags(path[len("/tags/"):])
|
||||
|
||||
# PUT /tags/:note_id
|
||||
if method == "PUT" and path.startswith("/tags/"):
|
||||
return _set_note_tags(path[len("/tags/"):], req)
|
||||
|
||||
# ── Note routes ───────────────────────────
|
||||
# GET /notes/:id
|
||||
if method == "GET" and path.startswith("/notes/"):
|
||||
return _get_note(path[len("/notes/"):])
|
||||
|
||||
# PUT /notes/:id
|
||||
if method == "PUT" and path.startswith("/notes/"):
|
||||
return _update_note(path[len("/notes/"):], req)
|
||||
|
||||
# DELETE /notes/:id
|
||||
if method == "DELETE" and path.startswith("/notes/"):
|
||||
return _delete_note(path[len("/notes/"):], req)
|
||||
|
||||
# ── Folder routes ─────────────────────────
|
||||
# GET /folders
|
||||
if method == "GET" and path == "/folders":
|
||||
return _list_folders()
|
||||
|
||||
# POST /folders
|
||||
if method == "POST" and path == "/folders":
|
||||
return _create_folder(req)
|
||||
|
||||
# PUT /folders/:id
|
||||
if method == "PUT" and path.startswith("/folders/"):
|
||||
return _update_folder(path[len("/folders/"):], req)
|
||||
|
||||
# DELETE /folders/:id
|
||||
if method == "DELETE" and path.startswith("/folders/"):
|
||||
return _delete_folder(path[len("/folders/"):])
|
||||
|
||||
return _resp(404, {"error": "not found"})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Notes CRUD
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _list_notes(req):
|
||||
q = req.get("query", {})
|
||||
filters = {}
|
||||
|
||||
folder = _str(q.get("folder_id", ""))
|
||||
if folder:
|
||||
filters["folder_id"] = folder
|
||||
|
||||
pinned = _str(q.get("pinned", ""))
|
||||
if pinned:
|
||||
filters["pinned"] = int(pinned)
|
||||
|
||||
archived = _str(q.get("archived", ""))
|
||||
if archived:
|
||||
filters["archived"] = int(archived)
|
||||
else:
|
||||
# default: exclude archived
|
||||
filters["archived"] = 0
|
||||
|
||||
creator = _str(q.get("creator_id", ""))
|
||||
if creator:
|
||||
filters["creator_id"] = creator
|
||||
|
||||
order = _str(q.get("order", "")) or "updated_at"
|
||||
limit_str = _str(q.get("limit", ""))
|
||||
limit = int(limit_str) if limit_str else 200
|
||||
|
||||
rows = db.query("notes", filters=filters, order=order, limit=limit)
|
||||
|
||||
# batch-fetch tags for all notes
|
||||
note_ids = []
|
||||
for r in (rows or []):
|
||||
note_ids.append(r.get("id", ""))
|
||||
tags_map = _tags_for_notes(note_ids)
|
||||
|
||||
# lightweight projection: strip body for list view
|
||||
items = []
|
||||
for r in (rows or []):
|
||||
nid = r.get("id", "")
|
||||
items.append({
|
||||
"id": nid,
|
||||
"title": r.get("title", ""),
|
||||
"snippet": _snippet(r.get("body", "")),
|
||||
"folder_id": r.get("folder_id", ""),
|
||||
"creator_id": r.get("creator_id", ""),
|
||||
"updated_at": r.get("updated_at", ""),
|
||||
"pinned": _int(r.get("pinned", 0)),
|
||||
"archived": _int(r.get("archived", 0)),
|
||||
"created_at": r.get("created_at", ""),
|
||||
"tags": tags_map.get(nid, []),
|
||||
})
|
||||
return _resp(200, {"data": items})
|
||||
|
||||
|
||||
def _create_note(req):
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
user_id = req.get("user_id", "")
|
||||
|
||||
title = _str(body.get("title", ""))
|
||||
if not title:
|
||||
title = "Untitled"
|
||||
|
||||
body_text = _str(body.get("body", ""))
|
||||
row = db.insert("notes", {
|
||||
"title": title,
|
||||
"body": body_text,
|
||||
"folder_id": _str(body.get("folder_id", "")),
|
||||
"creator_id": user_id,
|
||||
"updated_at": "",
|
||||
"pinned": _int(body.get("pinned", 0)),
|
||||
"archived": 0,
|
||||
})
|
||||
_sync_links(row["id"], body_text)
|
||||
return _resp(201, row)
|
||||
|
||||
|
||||
def _get_note(note_id):
|
||||
rows = db.query("notes", filters={"id": note_id}, limit=1)
|
||||
if not rows:
|
||||
return _resp(404, {"error": "note not found"})
|
||||
note = rows[0]
|
||||
# include tags
|
||||
tag_rows = db.query("tags", filters={"note_id": note_id}, limit=100)
|
||||
tags = []
|
||||
for t in (tag_rows or []):
|
||||
tag = _str(t.get("tag", ""))
|
||||
if tag:
|
||||
tags.append(tag)
|
||||
note["tags"] = tags
|
||||
return _resp(200, note)
|
||||
|
||||
|
||||
def _update_note(note_id, req):
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
|
||||
existing = db.query("notes", filters={"id": note_id}, limit=1)
|
||||
if not existing:
|
||||
return _resp(404, {"error": "note not found"})
|
||||
|
||||
updates = {}
|
||||
for key in ["title", "body", "folder_id"]:
|
||||
if key in body:
|
||||
updates[key] = _str(body[key])
|
||||
|
||||
if "pinned" in body:
|
||||
updates["pinned"] = _int(body["pinned"])
|
||||
if "archived" in body:
|
||||
updates["archived"] = _int(body["archived"])
|
||||
|
||||
# touch updated_at
|
||||
updates["updated_at"] = "now"
|
||||
|
||||
ok = db.update("notes", note_id, updates)
|
||||
if not ok:
|
||||
return _resp(500, {"error": "update failed"})
|
||||
|
||||
# sync wikilinks when body changes
|
||||
if "body" in body:
|
||||
_sync_links(note_id, _str(body["body"]))
|
||||
|
||||
rows = db.query("notes", filters={"id": note_id}, limit=1)
|
||||
return _resp(200, rows[0] if rows else {})
|
||||
|
||||
|
||||
def _delete_note(note_id, req):
|
||||
q = req.get("query", {})
|
||||
hard = _str(q.get("hard", ""))
|
||||
|
||||
if hard == "1":
|
||||
# cascade-delete tags for this note
|
||||
note_tags = db.query("tags", filters={"note_id": note_id}, limit=100)
|
||||
for t in (note_tags or []):
|
||||
db.delete("tags", t["id"])
|
||||
# cascade-delete outgoing links FROM this note
|
||||
out_links = db.query("links", filters={"source_id": note_id}, limit=500)
|
||||
for link in (out_links or []):
|
||||
db.delete("links", link["id"])
|
||||
# cascade-delete incoming backlinks TO this note
|
||||
in_links = db.query("links", filters={"target_id": note_id}, limit=500)
|
||||
for link in (in_links or []):
|
||||
db.delete("links", link["id"])
|
||||
ok = db.delete("notes", note_id)
|
||||
if not ok:
|
||||
return _resp(404, {"error": "note not found"})
|
||||
return _resp(200, {"deleted": True})
|
||||
|
||||
# soft delete: archive
|
||||
ok = db.update("notes", note_id, {"archived": 1, "updated_at": "now"})
|
||||
if not ok:
|
||||
return _resp(404, {"error": "note not found"})
|
||||
return _resp(200, {"archived": True})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Move note between folders
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _move_note(req):
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
note_id = _str(body.get("note_id", ""))
|
||||
folder_id = _str(body.get("folder_id", ""))
|
||||
if not note_id:
|
||||
return _resp(400, {"error": "note_id required"})
|
||||
existing = db.query("notes", filters={"id": note_id}, limit=1)
|
||||
if not existing:
|
||||
return _resp(404, {"error": "note not found"})
|
||||
ok = db.update("notes", note_id, {"folder_id": folder_id, "updated_at": "now"})
|
||||
if not ok:
|
||||
return _resp(500, {"error": "move failed"})
|
||||
rows = db.query("notes", filters={"id": note_id}, limit=1)
|
||||
return _resp(200, rows[0] if rows else {})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Tags CRUD
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _list_tags():
|
||||
"""Return sorted list of all unique tag strings."""
|
||||
all_tags = db.query("tags", limit=5000)
|
||||
seen = {}
|
||||
for t in (all_tags or []):
|
||||
tag = _str(t.get("tag", "")).strip()
|
||||
if tag:
|
||||
seen[tag] = True
|
||||
tags = sorted(seen.keys())
|
||||
return _resp(200, {"data": tags})
|
||||
|
||||
|
||||
def _get_note_tags(note_id):
|
||||
"""Return tags for a specific note."""
|
||||
rows = db.query("tags", filters={"note_id": note_id}, limit=100)
|
||||
tags = []
|
||||
for t in (rows or []):
|
||||
tag = _str(t.get("tag", ""))
|
||||
if tag:
|
||||
tags.append(tag)
|
||||
return _resp(200, {"data": tags})
|
||||
|
||||
|
||||
def _set_note_tags(note_id, req):
|
||||
"""Replace all tags for a note (delete + reinsert)."""
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
new_tags_raw = body.get("tags", [])
|
||||
|
||||
# validate note exists
|
||||
existing = db.query("notes", filters={"id": note_id}, limit=1)
|
||||
if not existing:
|
||||
return _resp(404, {"error": "note not found"})
|
||||
|
||||
# normalize: lowercase, strip, deduplicate, skip empty
|
||||
seen = {}
|
||||
clean_tags = []
|
||||
for raw in new_tags_raw:
|
||||
tag = _str(raw).strip().lower()
|
||||
if tag and tag not in seen:
|
||||
seen[tag] = True
|
||||
clean_tags.append(tag)
|
||||
|
||||
# delete all existing tags for this note
|
||||
old_tags = db.query("tags", filters={"note_id": note_id}, limit=100)
|
||||
for t in (old_tags or []):
|
||||
db.delete("tags", t["id"])
|
||||
|
||||
# insert new tags
|
||||
for tag in clean_tags:
|
||||
db.insert("tags", {"note_id": note_id, "tag": tag})
|
||||
|
||||
return _resp(200, {"data": clean_tags})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Links (backlinks + wikilinks)
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _get_links(note_id):
|
||||
"""Return outgoing links for a note."""
|
||||
rows = db.query("links", filters={"source_id": note_id}, limit=500)
|
||||
items = []
|
||||
for r in (rows or []):
|
||||
items.append({
|
||||
"source_id": r.get("source_id", ""),
|
||||
"target_id": r.get("target_id", ""),
|
||||
"link_text": r.get("link_text", ""),
|
||||
})
|
||||
return _resp(200, {"data": items})
|
||||
|
||||
|
||||
def _get_backlinks(note_id):
|
||||
"""Return notes that link TO this note (backlinks), enriched with source titles."""
|
||||
rows = db.query("links", filters={"target_id": note_id}, limit=500)
|
||||
if not rows:
|
||||
return _resp(200, {"data": []})
|
||||
|
||||
# batch-fetch source note titles
|
||||
all_notes = db.query("notes", filters={"archived": 0}, limit=5000)
|
||||
note_map = {}
|
||||
for n in (all_notes or []):
|
||||
note_map[n.get("id", "")] = n.get("title", "Untitled")
|
||||
|
||||
items = []
|
||||
for r in rows:
|
||||
sid = r.get("source_id", "")
|
||||
items.append({
|
||||
"source_id": sid,
|
||||
"source_title": note_map.get(sid, "Unknown"),
|
||||
"link_text": r.get("link_text", ""),
|
||||
})
|
||||
return _resp(200, {"data": items})
|
||||
|
||||
|
||||
def _get_graph():
|
||||
"""Return all non-archived notes as nodes and resolved links as edges."""
|
||||
all_notes = db.query("notes", filters={"archived": 0}, limit=5000)
|
||||
notes = all_notes or []
|
||||
|
||||
# batch-fetch tags
|
||||
note_ids = [n.get("id", "") for n in notes]
|
||||
tags_map = _tags_for_notes(note_ids)
|
||||
|
||||
# build nodes + id set for edge filtering
|
||||
nodes = []
|
||||
note_id_set = {}
|
||||
for n in notes:
|
||||
nid = n.get("id", "")
|
||||
note_id_set[nid] = True
|
||||
nodes.append({
|
||||
"id": nid,
|
||||
"title": n.get("title", "Untitled"),
|
||||
"folder_id": _str(n.get("folder_id", "")),
|
||||
"tags": tags_map.get(nid, []),
|
||||
})
|
||||
|
||||
# fetch links, keep only resolved (both endpoints exist)
|
||||
all_links = db.query("links", limit=10000)
|
||||
edges = []
|
||||
for link in (all_links or []):
|
||||
src = _str(link.get("source_id", ""))
|
||||
tgt = _str(link.get("target_id", ""))
|
||||
if src and tgt and src in note_id_set and tgt in note_id_set:
|
||||
edges.append({
|
||||
"source": src,
|
||||
"target": tgt,
|
||||
"text": _str(link.get("link_text", "")),
|
||||
})
|
||||
|
||||
return _resp(200, {"nodes": nodes, "edges": edges})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Folder CRUD
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _list_folders():
|
||||
rows = db.query("folders", order="sort_order", limit=500)
|
||||
items = []
|
||||
for r in (rows or []):
|
||||
items.append({
|
||||
"id": r.get("id", ""),
|
||||
"name": r.get("name", ""),
|
||||
"parent_id": r.get("parent_id", ""),
|
||||
"sort_order": _int(r.get("sort_order", 0)),
|
||||
"creator_id": r.get("creator_id", ""),
|
||||
"created_at": r.get("created_at", ""),
|
||||
})
|
||||
return _resp(200, {"data": items})
|
||||
|
||||
|
||||
def _create_folder(req):
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
user_id = req.get("user_id", "")
|
||||
name = _str(body.get("name", ""))
|
||||
if not name:
|
||||
return _resp(400, {"error": "name required"})
|
||||
row = db.insert("folders", {
|
||||
"name": name,
|
||||
"parent_id": _str(body.get("parent_id", "")),
|
||||
"creator_id": user_id,
|
||||
"sort_order": _int(body.get("sort_order", 0)),
|
||||
})
|
||||
return _resp(201, row)
|
||||
|
||||
|
||||
def _update_folder(folder_id, req):
|
||||
body = json.decode(req.get("body", "{}"))
|
||||
existing = db.query("folders", filters={"id": folder_id}, limit=1)
|
||||
if not existing:
|
||||
return _resp(404, {"error": "folder not found"})
|
||||
updates = {}
|
||||
for key in ["name", "parent_id"]:
|
||||
if key in body:
|
||||
updates[key] = _str(body[key])
|
||||
if "sort_order" in body:
|
||||
updates["sort_order"] = _int(body["sort_order"])
|
||||
if not updates:
|
||||
return _resp(200, existing[0])
|
||||
ok = db.update("folders", folder_id, updates)
|
||||
if not ok:
|
||||
return _resp(500, {"error": "update failed"})
|
||||
rows = db.query("folders", filters={"id": folder_id}, limit=1)
|
||||
return _resp(200, rows[0] if rows else {})
|
||||
|
||||
|
||||
def _delete_folder(folder_id):
|
||||
existing = db.query("folders", filters={"id": folder_id}, limit=1)
|
||||
if not existing:
|
||||
return _resp(404, {"error": "folder not found"})
|
||||
parent_id = _str(existing[0].get("parent_id", ""))
|
||||
|
||||
# reparent child folders to deleted folder's parent
|
||||
children = db.query("folders", filters={"parent_id": folder_id}, limit=500)
|
||||
for child in (children or []):
|
||||
db.update("folders", child["id"], {"parent_id": parent_id})
|
||||
|
||||
# orphan notes in this folder (move to unfiled)
|
||||
notes_in_folder = db.query("notes", filters={"folder_id": folder_id}, limit=1000)
|
||||
for n in (notes_in_folder or []):
|
||||
db.update("notes", n["id"], {"folder_id": ""})
|
||||
|
||||
ok = db.delete("folders", folder_id)
|
||||
if not ok:
|
||||
return _resp(500, {"error": "delete failed"})
|
||||
return _resp(200, {"deleted": True, "orphaned_notes": len(notes_in_folder or [])})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Search
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _search_notes(req):
|
||||
q = req.get("query", {})
|
||||
term = _str(q.get("q", ""))
|
||||
if not term:
|
||||
return _resp(400, {"error": "q parameter required"})
|
||||
|
||||
# fetch all non-archived notes and filter in Starlark
|
||||
# (ext_data db.query doesn't support LIKE — filter client-side)
|
||||
rows = db.query("notes", filters={"archived": 0}, order="updated_at", limit=1000)
|
||||
term_lower = term.lower()
|
||||
|
||||
# batch-fetch tags for tag-based matching
|
||||
tags_map = _tags_for_notes([r.get("id", "") for r in (rows or [])])
|
||||
|
||||
matches = []
|
||||
for r in (rows or []):
|
||||
title = _str(r.get("title", "")).lower()
|
||||
body = _str(r.get("body", "")).lower()
|
||||
nid = r.get("id", "")
|
||||
|
||||
# check title and body
|
||||
found = term_lower in title or term_lower in body
|
||||
|
||||
# check tags
|
||||
if not found:
|
||||
note_tags = tags_map.get(nid, [])
|
||||
for tg in note_tags:
|
||||
if term_lower in tg.lower():
|
||||
found = True
|
||||
|
||||
if found:
|
||||
matches.append({
|
||||
"id": nid,
|
||||
"title": r.get("title", ""),
|
||||
"snippet": _snippet(r.get("body", "")),
|
||||
"folder_id": r.get("folder_id", ""),
|
||||
"updated_at": r.get("updated_at", ""),
|
||||
"pinned": _int(r.get("pinned", 0)),
|
||||
"created_at": r.get("created_at", ""),
|
||||
"tags": tags_map.get(nid, []),
|
||||
})
|
||||
return _resp(200, {"data": matches})
|
||||
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Stats
|
||||
# ═══════════════════════════════════════════════
|
||||
|
||||
def _get_stats():
|
||||
all_notes = db.query("notes", limit=10000)
|
||||
notes = all_notes or []
|
||||
total = 0
|
||||
pinned = 0
|
||||
archived = 0
|
||||
unfiled = 0
|
||||
for n in notes:
|
||||
if _int(n.get("archived", 0)) == 1:
|
||||
archived = archived + 1
|
||||
else:
|
||||
total = total + 1
|
||||
if _int(n.get("pinned", 0)) == 1:
|
||||
pinned = pinned + 1
|
||||
if not _str(n.get("folder_id", "")):
|
||||
unfiled = unfiled + 1
|
||||
all_folders = db.query("folders", limit=10000)
|
||||
folder_count = len(all_folders or [])
|
||||
|
||||
# count unique tags
|
||||
all_tags = db.query("tags", limit=10000)
|
||||
tag_seen = {}
|
||||
for t in (all_tags or []):
|
||||
tag = _str(t.get("tag", ""))
|
||||
if tag:
|
||||
tag_seen[tag] = True
|
||||
tag_count = len(tag_seen)
|
||||
|
||||
# count links
|
||||
all_links = db.query("links", limit=10000)
|
||||
link_count = len(all_links or [])
|
||||
|
||||
return _resp(200, {"total": total, "pinned": pinned, "archived": archived, "unfiled": unfiled, "folders": folder_count, "tags": tag_count, "links": link_count})
|
||||
@@ -275,7 +275,7 @@
|
||||
}
|
||||
|
||||
async function handleDelete(item) {
|
||||
if (!confirm('Delete schedule "' + item.name + '"?')) return;
|
||||
if (!await sw.confirm('Delete schedule "' + item.name + '"?', { destructive: true })) return;
|
||||
try {
|
||||
await apiDelete(item.id);
|
||||
sw.toast('Schedule deleted', 'success');
|
||||
|
||||
@@ -82,7 +82,7 @@
|
||||
|
||||
await T.test('admin', 'surfaces', 'sw.api.admin.surfaces.list()', {
|
||||
sdk: function () { return sw.api.admin.surfaces.list(); },
|
||||
raw: { method: 'GET', path: '/admin/surfaces' },
|
||||
raw: { method: 'GET', path: '/admin/packages' },
|
||||
validate: function (r) { var arr = T.unwrapList(r); T.assert(arr.length >= 0, 'expected list'); }
|
||||
});
|
||||
|
||||
@@ -149,5 +149,39 @@
|
||||
raw: { method: 'GET', path: '/admin/projects' },
|
||||
validate: function (r) { T.assert(Array.isArray(r) || (r && Array.isArray(r.data)), 'expected array'); }
|
||||
});
|
||||
|
||||
// ── Metrics (v0.6.4+) ──
|
||||
|
||||
await T.test('admin', 'metrics', 'GET /admin/metrics (raw)', {
|
||||
sdk: function () { return sw.api.get('/admin/metrics'); },
|
||||
raw: { method: 'GET', path: '/admin/metrics' },
|
||||
validate: function (r) {
|
||||
T.assert(typeof r === 'object', 'expected object');
|
||||
T.assertHasKey(r, 'runtime', 'metrics');
|
||||
T.assertHasKey(r, 'database', 'metrics');
|
||||
}
|
||||
});
|
||||
|
||||
// ── Cluster (v0.6.0+) ──
|
||||
|
||||
await T.test('admin', 'cluster', 'GET /admin/cluster (raw)', {
|
||||
sdk: function () { return sw.api.get('/admin/cluster'); },
|
||||
raw: { method: 'GET', path: '/admin/cluster' },
|
||||
validate: function (r) {
|
||||
var arr = T.unwrapList(r);
|
||||
T.assert(arr.length >= 0, 'expected cluster node list');
|
||||
}
|
||||
});
|
||||
|
||||
// ── Backups (v0.6.1+) ──
|
||||
|
||||
await T.test('admin', 'backups', 'GET /admin/backups (raw)', {
|
||||
sdk: function () { return sw.api.get('/admin/backups'); },
|
||||
raw: { method: 'GET', path: '/admin/backups' },
|
||||
validate: function (r) {
|
||||
var arr = T.unwrapList(r);
|
||||
T.assert(arr.length >= 0, 'expected backup list');
|
||||
}
|
||||
});
|
||||
});
|
||||
})();
|
||||
|
||||
@@ -144,7 +144,7 @@
|
||||
}
|
||||
|
||||
async function handleDelete(id) {
|
||||
if (!confirm('Delete this task?')) return;
|
||||
if (!await sw.confirm('Delete this task?', { destructive: true })) return;
|
||||
await api.del('/items/' + id);
|
||||
onRefresh();
|
||||
}
|
||||
|
||||
41
packages/workflow-chat/README.md
Normal file
41
packages/workflow-chat/README.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Workflow Chat Integration
|
||||
|
||||
Creates scoped chat conversations when workflow stages require team collaboration.
|
||||
|
||||
## Setup
|
||||
|
||||
1. Install the `workflow-chat` and `chat-core` packages.
|
||||
2. In your workflow stage with `audience: team`, set `stage_config`:
|
||||
|
||||
```json
|
||||
{
|
||||
"on_advance": {
|
||||
"package_id": "workflow-chat",
|
||||
"entry_point": "on_advance"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. When the workflow advances to that stage, ensure `stage_data` includes:
|
||||
|
||||
```json
|
||||
{
|
||||
"title": "Bug Triage Discussion",
|
||||
"team_members": [
|
||||
{"id": "user-1", "display_name": "Alice"},
|
||||
{"id": "user-2", "display_name": "Bob"}
|
||||
],
|
||||
"creator_id": "user-1",
|
||||
"creator_display_name": "Alice"
|
||||
}
|
||||
```
|
||||
|
||||
The hook will:
|
||||
- Create a group conversation titled `"Bug Triage Discussion [abcd1234]"` (suffixed with instance ID)
|
||||
- Add all team members as participants
|
||||
- Send a system message linking to the workflow instance
|
||||
- Enrich `stage_data` with the `conversation_id`
|
||||
|
||||
## Idempotency
|
||||
|
||||
If `stage_data` already contains a `conversation_id`, the hook returns `None` (no-op).
|
||||
15
packages/workflow-chat/manifest.json
Normal file
15
packages/workflow-chat/manifest.json
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"id": "workflow-chat",
|
||||
"title": "Workflow Chat",
|
||||
"type": "library",
|
||||
"tier": "starlark",
|
||||
"version": "0.1.0",
|
||||
"description": "Creates scoped chat conversations when workflow stages require team collaboration. Wire into stage_config as an on_advance hook.",
|
||||
"author": "switchboard",
|
||||
|
||||
"permissions": ["db.write", "realtime.publish"],
|
||||
|
||||
"depends": ["chat-core"],
|
||||
|
||||
"exports": ["on_advance"]
|
||||
}
|
||||
88
packages/workflow-chat/script.star
Normal file
88
packages/workflow-chat/script.star
Normal file
@@ -0,0 +1,88 @@
|
||||
# Workflow Chat — Starlark Backend (v0.1.0)
|
||||
#
|
||||
# Library package that creates scoped chat conversations when
|
||||
# workflow stages require team collaboration.
|
||||
#
|
||||
# Usage: wire into stage_config as an on_advance hook:
|
||||
# {"on_advance": {"package_id": "workflow-chat", "entry_point": "on_advance"}}
|
||||
#
|
||||
# Expected stage_data keys:
|
||||
# title — conversation title (optional, defaults to "Workflow Discussion")
|
||||
# team_members — list of {id, display_name} dicts
|
||||
# creator_id — user ID of the workflow initiator
|
||||
# creator_display_name — display name of the workflow initiator
|
||||
#
|
||||
# Modules: db, json, realtime (via chat-core dependency)
|
||||
|
||||
chat = lib.require("chat-core")
|
||||
|
||||
|
||||
def _str(v):
|
||||
if v == None:
|
||||
return ""
|
||||
return str(v)
|
||||
|
||||
|
||||
def on_advance(ctx):
|
||||
"""Hook called when a workflow advances to a team-audience stage.
|
||||
|
||||
Creates a group conversation with all team members and sends
|
||||
a system message linking back to the workflow instance.
|
||||
|
||||
Args:
|
||||
ctx: dict with {instance_id, previous_stage, current_stage, stage_data}
|
||||
|
||||
Returns:
|
||||
dict with {stage_data} containing enriched data with conversation_id,
|
||||
or None if no team members are present.
|
||||
"""
|
||||
data = ctx.get("stage_data", {})
|
||||
if type(data) == "string":
|
||||
data = json.decode(data) if data else {}
|
||||
|
||||
instance_id = _str(ctx.get("instance_id", ""))
|
||||
members = data.get("team_members", [])
|
||||
creator_id = _str(data.get("creator_id", ""))
|
||||
creator_name = _str(data.get("creator_display_name", ""))
|
||||
title = _str(data.get("title", "")) or "Workflow Discussion"
|
||||
|
||||
# Skip if no team members to add
|
||||
if not members:
|
||||
return None
|
||||
|
||||
# Check if conversation already exists for this instance (idempotency)
|
||||
existing_cid = _str(data.get("conversation_id", ""))
|
||||
if existing_cid:
|
||||
return None
|
||||
|
||||
# Build participants list
|
||||
participants = []
|
||||
for m in members:
|
||||
mid = _str(m.get("id", ""))
|
||||
if mid and mid != creator_id:
|
||||
participants.append({
|
||||
"id": mid,
|
||||
"display_name": _str(m.get("display_name", "")),
|
||||
})
|
||||
|
||||
# Create conversation scoped to this workflow instance
|
||||
conv = chat.create(
|
||||
title=title + " [" + instance_id[:8] + "]",
|
||||
type="group",
|
||||
participants=participants,
|
||||
creator_id=creator_id,
|
||||
creator_display_name=creator_name,
|
||||
)
|
||||
|
||||
cid = _str(conv.get("id", ""))
|
||||
|
||||
# Send system message linking to the workflow
|
||||
chat.send(cid, creator_id, "Conversation created for workflow instance " + instance_id, "system")
|
||||
|
||||
# Return enriched stage_data with conversation_id
|
||||
enriched = {}
|
||||
for k in data:
|
||||
enriched[k] = data[k]
|
||||
enriched["conversation_id"] = cid
|
||||
|
||||
return {"stage_data": enriched}
|
||||
81
scripts/generate-registry.sh
Executable file
81
scripts/generate-registry.sh
Executable file
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env bash
|
||||
# generate-registry.sh — Build a registry.json from a directory of .pkg files.
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/generate-registry.sh [PKG_DIR] [BASE_URL]
|
||||
#
|
||||
# Arguments:
|
||||
# PKG_DIR Directory containing .pkg files (default: dist/)
|
||||
# BASE_URL HTTPS base URL for download links (default: https://example.com/packages)
|
||||
#
|
||||
# Output:
|
||||
# Writes registry.json to stdout. Redirect to a file:
|
||||
# ./scripts/generate-registry.sh dist/ https://cdn.example.com/pkg > registry.json
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
PKG_DIR="${1:-dist}"
|
||||
BASE_URL="${2:-https://example.com/packages}"
|
||||
|
||||
# Strip trailing slash
|
||||
BASE_URL="${BASE_URL%/}"
|
||||
|
||||
if [ ! -d "$PKG_DIR" ]; then
|
||||
echo "Error: directory '$PKG_DIR' not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Collect entries
|
||||
entries=()
|
||||
|
||||
for pkg in "$PKG_DIR"/*.pkg; do
|
||||
[ -f "$pkg" ] || continue
|
||||
|
||||
# Extract manifest.json from the ZIP archive
|
||||
manifest=$(unzip -p "$pkg" manifest.json 2>/dev/null) || {
|
||||
echo "Warning: skipping $pkg (no manifest.json)" >&2
|
||||
continue
|
||||
}
|
||||
|
||||
# Read fields from manifest
|
||||
id=$(echo "$manifest" | jq -r '.id // empty')
|
||||
title=$(echo "$manifest" | jq -r '.title // .id')
|
||||
version=$(echo "$manifest" | jq -r '.version // "0.0.0"')
|
||||
description=$(echo "$manifest" | jq -r '.description // ""')
|
||||
author=$(echo "$manifest" | jq -r '.author // ""')
|
||||
type=$(echo "$manifest" | jq -r '.type // "extension"')
|
||||
tier=$(echo "$manifest" | jq -r '.tier // "community"')
|
||||
|
||||
if [ -z "$id" ]; then
|
||||
echo "Warning: skipping $pkg (no id in manifest)" >&2
|
||||
continue
|
||||
fi
|
||||
|
||||
filename=$(basename "$pkg")
|
||||
size=$(stat -c%s "$pkg" 2>/dev/null || stat -f%z "$pkg" 2>/dev/null || echo 0)
|
||||
updated_at=$(date -r "$pkg" -u +"%Y-%m-%dT%H:%M:%SZ" 2>/dev/null || date -u +"%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
entry=$(jq -n \
|
||||
--arg id "$id" \
|
||||
--arg title "$title" \
|
||||
--arg version "$version" \
|
||||
--arg description "$description" \
|
||||
--arg author "$author" \
|
||||
--arg type "$type" \
|
||||
--arg tier "$tier" \
|
||||
--arg download_url "$BASE_URL/$filename" \
|
||||
--argjson size "$size" \
|
||||
--arg updated_at "$updated_at" \
|
||||
'{id: $id, title: $title, version: $version, description: $description,
|
||||
author: $author, type: $type, tier: $tier, download_url: $download_url,
|
||||
size: $size, updated_at: $updated_at}')
|
||||
|
||||
entries+=("$entry")
|
||||
done
|
||||
|
||||
# Build final JSON
|
||||
if [ ${#entries[@]} -eq 0 ]; then
|
||||
echo '{"packages": []}' | jq .
|
||||
else
|
||||
printf '%s\n' "${entries[@]}" | jq -s '{packages: .}'
|
||||
fi
|
||||
237
scripts/switchboard-ca.sh
Executable file
237
scripts/switchboard-ca.sh
Executable file
@@ -0,0 +1,237 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# switchboard-ca.sh — Certificate provisioning for Switchboard Core mTLS.
|
||||
#
|
||||
# Wraps openssl to generate a cluster CA, node certificates (ServerAuth +
|
||||
# ClientAuth), and user certificates (ClientAuth only). All output is PEM.
|
||||
#
|
||||
# Usage:
|
||||
# switchboard-ca init
|
||||
# switchboard-ca issue-node --name <node-name> --san <dns1,ip1,...>
|
||||
# switchboard-ca issue-user --cn <username> [--email <email>]
|
||||
#
|
||||
# Files are written to the current directory under ca/, nodes/, users/.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
CA_DIR="./ca"
|
||||
NODES_DIR="./nodes"
|
||||
USERS_DIR="./users"
|
||||
CA_DAYS=3650 # 10 years for CA
|
||||
NODE_DAYS=365 # 1 year for nodes
|
||||
USER_DAYS=90 # 90 days for users
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
die() { echo "error: $*" >&2; exit 1; }
|
||||
|
||||
require_openssl() {
|
||||
command -v openssl >/dev/null 2>&1 || die "openssl not found in PATH"
|
||||
}
|
||||
|
||||
require_ca() {
|
||||
[ -f "$CA_DIR/cluster-ca.key" ] || die "CA not initialized. Run: $0 init"
|
||||
[ -f "$CA_DIR/cluster-ca.crt" ] || die "CA not initialized. Run: $0 init"
|
||||
}
|
||||
|
||||
# ── init ─────────────────────────────────────────────────────────────
|
||||
|
||||
cmd_init() {
|
||||
require_openssl
|
||||
mkdir -p "$CA_DIR"
|
||||
|
||||
if [ -f "$CA_DIR/cluster-ca.key" ]; then
|
||||
die "CA already exists at $CA_DIR/cluster-ca.key — remove it first to reinitialize"
|
||||
fi
|
||||
|
||||
# Generate ECDSA P-256 CA key
|
||||
openssl ecparam -genkey -name prime256v1 -noout -out "$CA_DIR/cluster-ca.key" 2>/dev/null
|
||||
|
||||
# Self-signed CA certificate
|
||||
openssl req -new -x509 \
|
||||
-key "$CA_DIR/cluster-ca.key" \
|
||||
-out "$CA_DIR/cluster-ca.crt" \
|
||||
-days "$CA_DAYS" \
|
||||
-subj "/CN=Switchboard Cluster CA" \
|
||||
-addext "basicConstraints=critical,CA:TRUE" \
|
||||
-addext "keyUsage=critical,keyCertSign,cRLSign" \
|
||||
2>/dev/null
|
||||
|
||||
echo "CA initialized:"
|
||||
echo " $CA_DIR/cluster-ca.crt"
|
||||
echo " $CA_DIR/cluster-ca.key"
|
||||
echo ""
|
||||
echo "Keep cluster-ca.key secure. It never needs to be on a running node."
|
||||
}
|
||||
|
||||
# ── issue-node ───────────────────────────────────────────────────────
|
||||
|
||||
cmd_issue_node() {
|
||||
require_openssl
|
||||
require_ca
|
||||
|
||||
local name=""
|
||||
local san=""
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--name) name="$2"; shift 2 ;;
|
||||
--san) san="$2"; shift 2 ;;
|
||||
*) die "unknown option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "$name" ] || die "usage: $0 issue-node --name <node-name> --san <dns1,ip1,...>"
|
||||
[ -n "$san" ] || die "usage: $0 issue-node --name <node-name> --san <dns1,ip1,...>"
|
||||
|
||||
mkdir -p "$NODES_DIR"
|
||||
|
||||
# Generate node key
|
||||
openssl ecparam -genkey -name prime256v1 -noout -out "$NODES_DIR/$name.key" 2>/dev/null
|
||||
|
||||
# Build SAN entries from comma-separated list
|
||||
local san_entries=""
|
||||
IFS=',' read -ra ADDRS <<< "$san"
|
||||
for addr in "${ADDRS[@]}"; do
|
||||
addr=$(echo "$addr" | xargs) # trim whitespace
|
||||
if [[ "$addr" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
san_entries="${san_entries}IP:${addr},"
|
||||
else
|
||||
san_entries="${san_entries}DNS:${addr},"
|
||||
fi
|
||||
done
|
||||
san_entries="${san_entries%,}" # remove trailing comma
|
||||
|
||||
# Create CSR
|
||||
openssl req -new \
|
||||
-key "$NODES_DIR/$name.key" \
|
||||
-out "$NODES_DIR/$name.csr" \
|
||||
-subj "/CN=$name" \
|
||||
2>/dev/null
|
||||
|
||||
# Create extensions config
|
||||
local ext_file
|
||||
ext_file=$(mktemp)
|
||||
cat > "$ext_file" <<EOF
|
||||
basicConstraints = CA:FALSE
|
||||
keyUsage = critical, digitalSignature
|
||||
extendedKeyUsage = serverAuth, clientAuth
|
||||
subjectAltName = ${san_entries}
|
||||
EOF
|
||||
|
||||
# Sign with CA
|
||||
openssl x509 -req \
|
||||
-in "$NODES_DIR/$name.csr" \
|
||||
-CA "$CA_DIR/cluster-ca.crt" \
|
||||
-CAkey "$CA_DIR/cluster-ca.key" \
|
||||
-CAcreateserial \
|
||||
-out "$NODES_DIR/$name.crt" \
|
||||
-days "$NODE_DAYS" \
|
||||
-extfile "$ext_file" \
|
||||
2>/dev/null
|
||||
|
||||
rm -f "$NODES_DIR/$name.csr" "$ext_file"
|
||||
|
||||
echo "Node cert issued ($NODE_DAYS days):"
|
||||
echo " $NODES_DIR/$name.crt"
|
||||
echo " $NODES_DIR/$name.key"
|
||||
echo ""
|
||||
echo "Deploy to the node along with $CA_DIR/cluster-ca.crt"
|
||||
}
|
||||
|
||||
# ── issue-user ───────────────────────────────────────────────────────
|
||||
|
||||
cmd_issue_user() {
|
||||
require_openssl
|
||||
require_ca
|
||||
|
||||
local cn=""
|
||||
local email=""
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--cn) cn="$2"; shift 2 ;;
|
||||
--email) email="$2"; shift 2 ;;
|
||||
*) die "unknown option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "$cn" ] || die "usage: $0 issue-user --cn <username> [--email <email>]"
|
||||
|
||||
mkdir -p "$USERS_DIR"
|
||||
|
||||
# Generate user key
|
||||
openssl ecparam -genkey -name prime256v1 -noout -out "$USERS_DIR/$cn.key" 2>/dev/null
|
||||
|
||||
# Build subject
|
||||
local subj="/CN=$cn"
|
||||
local san_line=""
|
||||
if [ -n "$email" ]; then
|
||||
san_line="email:${email}"
|
||||
fi
|
||||
|
||||
# Create CSR
|
||||
openssl req -new \
|
||||
-key "$USERS_DIR/$cn.key" \
|
||||
-out "$USERS_DIR/$cn.csr" \
|
||||
-subj "$subj" \
|
||||
2>/dev/null
|
||||
|
||||
# Create extensions config
|
||||
local ext_file
|
||||
ext_file=$(mktemp)
|
||||
cat > "$ext_file" <<EOF
|
||||
basicConstraints = CA:FALSE
|
||||
keyUsage = critical, digitalSignature
|
||||
extendedKeyUsage = clientAuth
|
||||
EOF
|
||||
|
||||
if [ -n "$san_line" ]; then
|
||||
echo "subjectAltName = ${san_line}" >> "$ext_file"
|
||||
fi
|
||||
|
||||
# Sign with CA
|
||||
openssl x509 -req \
|
||||
-in "$USERS_DIR/$cn.csr" \
|
||||
-CA "$CA_DIR/cluster-ca.crt" \
|
||||
-CAkey "$CA_DIR/cluster-ca.key" \
|
||||
-CAcreateserial \
|
||||
-out "$USERS_DIR/$cn.crt" \
|
||||
-days "$USER_DAYS" \
|
||||
-extfile "$ext_file" \
|
||||
2>/dev/null
|
||||
|
||||
rm -f "$USERS_DIR/$cn.csr" "$ext_file"
|
||||
|
||||
echo "User cert issued ($USER_DAYS days):"
|
||||
echo " $USERS_DIR/$cn.crt"
|
||||
echo " $USERS_DIR/$cn.key"
|
||||
echo ""
|
||||
echo "Import both files into the user's browser or CLI tool."
|
||||
}
|
||||
|
||||
# ── Main dispatch ────────────────────────────────────────────────────
|
||||
|
||||
case "${1:-}" in
|
||||
init)
|
||||
shift
|
||||
cmd_init "$@"
|
||||
;;
|
||||
issue-node)
|
||||
shift
|
||||
cmd_issue_node "$@"
|
||||
;;
|
||||
issue-user)
|
||||
shift
|
||||
cmd_issue_user "$@"
|
||||
;;
|
||||
*)
|
||||
echo "Usage: $0 <command>"
|
||||
echo ""
|
||||
echo "Commands:"
|
||||
echo " init Initialize cluster CA"
|
||||
echo " issue-node --name <n> --san <addrs> Issue node cert (server+client auth)"
|
||||
echo " issue-user --cn <name> [--email <e>] Issue user cert (client auth only)"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
@@ -1,180 +0,0 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/models"
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// MTLSConfig holds mTLS-specific configuration.
|
||||
type MTLSConfig struct {
|
||||
HeaderDN string // header carrying cert DN (default "X-SSL-Client-DN")
|
||||
HeaderVerify string // header carrying verify status (default "X-SSL-Client-Verify")
|
||||
HeaderFingerprint string // header carrying cert fingerprint (default "X-SSL-Client-Fingerprint")
|
||||
AutoActivate bool // auto-activate new users (default true)
|
||||
DefaultTeam string // team ID for auto-provisioned users (optional)
|
||||
}
|
||||
|
||||
// MTLSProvider authenticates via client certificate headers injected
|
||||
// by the TLS-terminating reverse proxy (nginx, Traefik, Istio).
|
||||
//
|
||||
// The backend never sees the actual TLS handshake — it trusts headers
|
||||
// injected by the proxy after cert validation. After reading and parsing
|
||||
// the DN, the provider resolves an existing user or auto-provisions a
|
||||
// new one, then returns a Result. The auth handler issues an internal JWT.
|
||||
//
|
||||
// Header flow:
|
||||
//
|
||||
// Client cert → nginx ssl_verify_client → injects X-SSL-Client-DN,
|
||||
// X-SSL-Client-Verify, X-SSL-Client-Fingerprint → backend reads headers.
|
||||
type MTLSProvider struct {
|
||||
cfg MTLSConfig
|
||||
}
|
||||
|
||||
func NewMTLSProvider(cfg MTLSConfig) *MTLSProvider {
|
||||
if cfg.HeaderDN == "" {
|
||||
cfg.HeaderDN = "X-SSL-Client-DN"
|
||||
}
|
||||
if cfg.HeaderVerify == "" {
|
||||
cfg.HeaderVerify = "X-SSL-Client-Verify"
|
||||
}
|
||||
if cfg.HeaderFingerprint == "" {
|
||||
cfg.HeaderFingerprint = "X-SSL-Client-Fingerprint"
|
||||
}
|
||||
return &MTLSProvider{cfg: cfg}
|
||||
}
|
||||
|
||||
func (p *MTLSProvider) Mode() Mode { return ModeMTLS }
|
||||
|
||||
func (p *MTLSProvider) SupportsRegistration() bool { return false }
|
||||
|
||||
func (p *MTLSProvider) Register(_ *gin.Context, _ store.Stores) (*Result, error) {
|
||||
return nil, ErrNotSupported
|
||||
}
|
||||
|
||||
// Authenticate reads the cert DN and verify status from proxy headers,
|
||||
// parses the DN fields, and resolves or auto-provisions the user.
|
||||
//
|
||||
// Returns ErrInvalidCreds when headers are missing or verify fails.
|
||||
// Returns ErrInactive when user exists but is deactivated.
|
||||
func (p *MTLSProvider) Authenticate(c *gin.Context, stores store.Stores) (*Result, error) {
|
||||
// ── Validate headers ───────────────────────────────────────────
|
||||
verify := c.GetHeader(p.cfg.HeaderVerify)
|
||||
if verify == "" {
|
||||
return nil, fmt.Errorf("%w: missing %s header", ErrInvalidCreds, p.cfg.HeaderVerify)
|
||||
}
|
||||
// nginx: "SUCCESS", Traefik: "0" (both mean valid cert)
|
||||
if verify != "SUCCESS" && verify != "0" {
|
||||
return nil, fmt.Errorf("%w: cert verify=%s", ErrInvalidCreds, verify)
|
||||
}
|
||||
|
||||
dn := c.GetHeader(p.cfg.HeaderDN)
|
||||
if dn == "" {
|
||||
return nil, fmt.Errorf("%w: missing %s header", ErrInvalidCreds, p.cfg.HeaderDN)
|
||||
}
|
||||
|
||||
fields := ParseDN(dn)
|
||||
cn := fields["CN"]
|
||||
if cn == "" {
|
||||
return nil, fmt.Errorf("%w: cert DN has no CN field", ErrInvalidCreds)
|
||||
}
|
||||
|
||||
// Stable external identity: fingerprint if available, else full DN
|
||||
fingerprint := c.GetHeader(p.cfg.HeaderFingerprint)
|
||||
if fingerprint == "" {
|
||||
fingerprint = dn
|
||||
}
|
||||
|
||||
ctx := c.Request.Context()
|
||||
|
||||
// ── Look up existing user ──────────────────────────────────────
|
||||
user, err := stores.Users.GetByExternalID(ctx, string(ModeMTLS), fingerprint)
|
||||
if err == nil && user != nil {
|
||||
if !user.IsActive {
|
||||
return nil, ErrInactive
|
||||
}
|
||||
log.Printf("[auth/mtls] existing user %s (%s)", user.Username, cn)
|
||||
return &Result{User: user, IsNewUser: false, VaultHint: ""}, nil
|
||||
}
|
||||
|
||||
// ── Auto-provision ─────────────────────────────────────────────
|
||||
if !p.cfg.AutoActivate {
|
||||
return nil, fmt.Errorf("%w: auto-provision disabled", ErrInvalidCreds)
|
||||
}
|
||||
|
||||
return p.autoProvision(ctx, fields, fingerprint, stores)
|
||||
}
|
||||
|
||||
func (p *MTLSProvider) autoProvision(
|
||||
ctx context.Context,
|
||||
dn map[string]string,
|
||||
fingerprint string,
|
||||
stores store.Stores,
|
||||
) (*Result, error) {
|
||||
cn := dn["CN"]
|
||||
|
||||
email := dn["emailAddress"]
|
||||
if email == "" {
|
||||
email = models.HandleFromName(cn) + "@mtls.local"
|
||||
}
|
||||
|
||||
handle := UniqueHandle(ctx, stores.Users, models.HandleFromName(cn))
|
||||
|
||||
user := &models.User{
|
||||
Username: handle,
|
||||
Email: email,
|
||||
DisplayName: cn,
|
||||
IsActive: true,
|
||||
AuthSource: string(ModeMTLS),
|
||||
ExternalID: &fingerprint,
|
||||
Handle: handle,
|
||||
}
|
||||
|
||||
if err := stores.Users.Create(ctx, user); err != nil {
|
||||
return nil, fmt.Errorf("auto-provision failed: %w", err)
|
||||
}
|
||||
|
||||
log.Printf("[auth/mtls] auto-provisioned user %s from CN=%s", user.ID, cn)
|
||||
EnsureEveryoneGroup(ctx, stores, user.ID)
|
||||
|
||||
// Auto-add to default team if configured
|
||||
if p.cfg.DefaultTeam != "" {
|
||||
if err := stores.Teams.AddMember(ctx, p.cfg.DefaultTeam, user.ID, "member"); err != nil {
|
||||
log.Printf("[auth/mtls] warn: could not add %s to default team: %v", user.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
return &Result{User: user, IsNewUser: true, VaultHint: ""}, nil
|
||||
}
|
||||
|
||||
// ParseDN parses an RFC 2253 / RFC 4514 distinguished name into key-value pairs.
|
||||
//
|
||||
// Examples:
|
||||
//
|
||||
// "CN=Jeff Smith,O=Acme Corp,OU=Engineering"
|
||||
// "CN=Jane Doe,emailAddress=jane@acme.com,O=Acme Corp"
|
||||
//
|
||||
// Handles simple comma-separated key=value pairs. Does NOT handle
|
||||
// escaped commas in values (\,) or multi-valued RDNs (+). Sufficient
|
||||
// for typical X.509 client cert DNs.
|
||||
func ParseDN(dn string) map[string]string {
|
||||
result := make(map[string]string)
|
||||
parts := strings.Split(dn, ",")
|
||||
for _, part := range parts {
|
||||
part = strings.TrimSpace(part)
|
||||
idx := strings.Index(part, "=")
|
||||
if idx < 1 {
|
||||
continue
|
||||
}
|
||||
key := strings.TrimSpace(part[:idx])
|
||||
val := strings.TrimSpace(part[idx+1:])
|
||||
result[key] = val
|
||||
}
|
||||
return result
|
||||
}
|
||||
108
server/auth/mtls_helpers.go
Normal file
108
server/auth/mtls_helpers.go
Normal file
@@ -0,0 +1,108 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"crypto/x509"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"log"
|
||||
"strings"
|
||||
|
||||
"switchboard-core/models"
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// ParseDN parses an RFC 2253 / RFC 4514 distinguished name into key-value pairs.
|
||||
//
|
||||
// Examples:
|
||||
//
|
||||
// "CN=Jeff Smith,O=Acme Corp,OU=Engineering"
|
||||
// "CN=Jane Doe,emailAddress=jane@acme.com,O=Acme Corp"
|
||||
//
|
||||
// Handles simple comma-separated key=value pairs. Does NOT handle
|
||||
// escaped commas in values (\,) or multi-valued RDNs (+). Sufficient
|
||||
// for typical X.509 client cert DNs.
|
||||
func ParseDN(dn string) map[string]string {
|
||||
result := make(map[string]string)
|
||||
parts := strings.Split(dn, ",")
|
||||
for _, part := range parts {
|
||||
part = strings.TrimSpace(part)
|
||||
idx := strings.Index(part, "=")
|
||||
if idx < 1 {
|
||||
continue
|
||||
}
|
||||
key := strings.TrimSpace(part[:idx])
|
||||
val := strings.TrimSpace(part[idx+1:])
|
||||
result[key] = val
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// FingerprintCert returns the hex-encoded SHA-256 hash of a certificate's
|
||||
// raw DER encoding. This is used as the stable external_id for mTLS users.
|
||||
func FingerprintCert(cert *x509.Certificate) string {
|
||||
h := sha256.Sum256(cert.Raw)
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
|
||||
// resolveOrProvision looks up an existing user by auth_source=mtls and external_id,
|
||||
// or auto-provisions a new user from the certificate's CN and fingerprint.
|
||||
//
|
||||
// Shared by both MTLSProxyProvider and MTLSNativeProvider.
|
||||
func resolveOrProvision(
|
||||
ctx context.Context,
|
||||
stores store.Stores,
|
||||
cn string,
|
||||
dnFields map[string]string,
|
||||
fingerprint string,
|
||||
autoActivate bool,
|
||||
defaultTeam string,
|
||||
) (*Result, error) {
|
||||
// ── Look up existing user ──────────────────────────────────────
|
||||
user, err := stores.Users.GetByExternalID(ctx, string(ModeMTLS), fingerprint)
|
||||
if err == nil && user != nil {
|
||||
if !user.IsActive {
|
||||
return nil, ErrInactive
|
||||
}
|
||||
log.Printf("[auth/mtls] existing user %s (%s)", user.Username, cn)
|
||||
return &Result{User: user, IsNewUser: false, VaultHint: ""}, nil
|
||||
}
|
||||
|
||||
// ── Auto-provision ─────────────────────────────────────────────
|
||||
if !autoActivate {
|
||||
return nil, fmt.Errorf("%w: auto-provision disabled", ErrInvalidCreds)
|
||||
}
|
||||
|
||||
email := dnFields["emailAddress"]
|
||||
if email == "" {
|
||||
email = models.HandleFromName(cn) + "@mtls.local"
|
||||
}
|
||||
|
||||
handle := UniqueHandle(ctx, stores.Users, models.HandleFromName(cn))
|
||||
|
||||
user = &models.User{
|
||||
Username: handle,
|
||||
Email: email,
|
||||
DisplayName: cn,
|
||||
IsActive: true,
|
||||
AuthSource: string(ModeMTLS),
|
||||
ExternalID: &fingerprint,
|
||||
Handle: handle,
|
||||
}
|
||||
|
||||
if err := stores.Users.Create(ctx, user); err != nil {
|
||||
return nil, fmt.Errorf("auto-provision failed: %w", err)
|
||||
}
|
||||
|
||||
log.Printf("[auth/mtls] auto-provisioned user %s from CN=%s", user.ID, cn)
|
||||
EnsureEveryoneGroup(ctx, stores, user.ID)
|
||||
|
||||
if defaultTeam != "" {
|
||||
if err := stores.Teams.AddMember(ctx, defaultTeam, user.ID, "member"); err != nil {
|
||||
log.Printf("[auth/mtls] warn: could not add %s to default team: %v", user.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
return &Result{User: user, IsNewUser: true, VaultHint: ""}, nil
|
||||
}
|
||||
72
server/auth/mtls_native.go
Normal file
72
server/auth/mtls_native.go
Normal file
@@ -0,0 +1,72 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// ErrNoCert is returned when no client certificate is presented on a
|
||||
// connection that requires mTLS authentication.
|
||||
var ErrNoCert = errors.New("no client certificate presented")
|
||||
|
||||
// MTLSNativeConfig holds configuration for the native (non-proxy) mTLS provider.
|
||||
type MTLSNativeConfig struct {
|
||||
AutoActivate bool // auto-activate new users (default true)
|
||||
DefaultTeam string // team ID for auto-provisioned users (optional)
|
||||
}
|
||||
|
||||
// MTLSNativeProvider authenticates by reading the peer certificate
|
||||
// directly from the TLS connection state. Unlike MTLSProxyProvider,
|
||||
// it does not trust headers — identity is cryptographically verified
|
||||
// by the Go TLS stack before the HTTP layer runs.
|
||||
//
|
||||
// Requires TLS_MODE=mtls so the binary terminates TLS itself.
|
||||
type MTLSNativeProvider struct {
|
||||
cfg MTLSNativeConfig
|
||||
}
|
||||
|
||||
// NewMTLSNativeProvider creates a native mTLS auth provider.
|
||||
func NewMTLSNativeProvider(cfg MTLSNativeConfig) *MTLSNativeProvider {
|
||||
return &MTLSNativeProvider{cfg: cfg}
|
||||
}
|
||||
|
||||
func (p *MTLSNativeProvider) Mode() Mode { return ModeMTLS }
|
||||
|
||||
func (p *MTLSNativeProvider) SupportsRegistration() bool { return false }
|
||||
|
||||
func (p *MTLSNativeProvider) Register(_ *gin.Context, _ store.Stores) (*Result, error) {
|
||||
return nil, ErrNotSupported
|
||||
}
|
||||
|
||||
// Authenticate reads the verified peer certificate from the TLS connection
|
||||
// state. The CN becomes the username, and sha256(cert.Raw) is the stable
|
||||
// external_id. Returns ErrNoCert when no TLS or no peer certificates.
|
||||
func (p *MTLSNativeProvider) Authenticate(c *gin.Context, stores store.Stores) (*Result, error) {
|
||||
if c.Request.TLS == nil || len(c.Request.TLS.PeerCertificates) == 0 {
|
||||
return nil, fmt.Errorf("%w", ErrNoCert)
|
||||
}
|
||||
|
||||
peer := c.Request.TLS.PeerCertificates[0]
|
||||
|
||||
cn := peer.Subject.CommonName
|
||||
if cn == "" {
|
||||
return nil, fmt.Errorf("%w: certificate has no CommonName", ErrInvalidCreds)
|
||||
}
|
||||
|
||||
// Build DN fields from the certificate subject for resolveOrProvision
|
||||
dnFields := map[string]string{"CN": cn}
|
||||
if len(peer.EmailAddresses) > 0 {
|
||||
dnFields["emailAddress"] = peer.EmailAddresses[0]
|
||||
}
|
||||
|
||||
fingerprint := FingerprintCert(peer)
|
||||
|
||||
return resolveOrProvision(
|
||||
c.Request.Context(), stores, cn, dnFields, fingerprint,
|
||||
p.cfg.AutoActivate, p.cfg.DefaultTeam,
|
||||
)
|
||||
}
|
||||
410
server/auth/mtls_native_test.go
Normal file
410
server/auth/mtls_native_test.go
Normal file
@@ -0,0 +1,410 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"crypto/x509/pkix"
|
||||
"encoding/pem"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/big"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// ── Test CA helpers ──────────────────────────────────────────────────
|
||||
|
||||
type testCA struct {
|
||||
Cert *x509.Certificate
|
||||
Key *ecdsa.PrivateKey
|
||||
CertPEM []byte
|
||||
Pool *x509.CertPool
|
||||
}
|
||||
|
||||
func newTestCA(t *testing.T) *testCA {
|
||||
t.Helper()
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatalf("generate CA key: %v", err)
|
||||
}
|
||||
|
||||
tmpl := &x509.Certificate{
|
||||
SerialNumber: big.NewInt(1),
|
||||
Subject: pkix.Name{CommonName: "Test CA"},
|
||||
NotBefore: time.Now().Add(-time.Hour),
|
||||
NotAfter: time.Now().Add(24 * time.Hour),
|
||||
IsCA: true,
|
||||
BasicConstraintsValid: true,
|
||||
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageCRLSign,
|
||||
}
|
||||
|
||||
certDER, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
|
||||
if err != nil {
|
||||
t.Fatalf("create CA cert: %v", err)
|
||||
}
|
||||
|
||||
cert, err := x509.ParseCertificate(certDER)
|
||||
if err != nil {
|
||||
t.Fatalf("parse CA cert: %v", err)
|
||||
}
|
||||
|
||||
certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: certDER})
|
||||
pool := x509.NewCertPool()
|
||||
pool.AddCert(cert)
|
||||
|
||||
return &testCA{Cert: cert, Key: key, CertPEM: certPEM, Pool: pool}
|
||||
}
|
||||
|
||||
type testCert struct {
|
||||
Cert *x509.Certificate
|
||||
Key *ecdsa.PrivateKey
|
||||
TLSCert tls.Certificate
|
||||
}
|
||||
|
||||
func (ca *testCA) issueClient(t *testing.T, cn string, email string) *testCert {
|
||||
t.Helper()
|
||||
return ca.issueCert(t, cn, email, []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth}, time.Now().Add(time.Hour))
|
||||
}
|
||||
|
||||
func (ca *testCA) issueExpiredClient(t *testing.T, cn string) *testCert {
|
||||
t.Helper()
|
||||
return ca.issueCert(t, cn, "", []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth}, time.Now().Add(-time.Second))
|
||||
}
|
||||
|
||||
func (ca *testCA) issueNode(t *testing.T, cn string) *testCert {
|
||||
t.Helper()
|
||||
return ca.issueCert(t, cn, "", []x509.ExtKeyUsage{
|
||||
x509.ExtKeyUsageServerAuth,
|
||||
x509.ExtKeyUsageClientAuth,
|
||||
}, time.Now().Add(time.Hour))
|
||||
}
|
||||
|
||||
func (ca *testCA) issueCert(t *testing.T, cn, email string, eku []x509.ExtKeyUsage, notAfter time.Time) *testCert {
|
||||
t.Helper()
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatalf("generate key: %v", err)
|
||||
}
|
||||
|
||||
tmpl := &x509.Certificate{
|
||||
SerialNumber: big.NewInt(time.Now().UnixNano()),
|
||||
Subject: pkix.Name{CommonName: cn},
|
||||
NotBefore: time.Now().Add(-time.Hour),
|
||||
NotAfter: notAfter,
|
||||
ExtKeyUsage: eku,
|
||||
KeyUsage: x509.KeyUsageDigitalSignature,
|
||||
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
|
||||
DNSNames: []string{"localhost"},
|
||||
}
|
||||
if email != "" {
|
||||
tmpl.EmailAddresses = []string{email}
|
||||
}
|
||||
|
||||
certDER, err := x509.CreateCertificate(rand.Reader, tmpl, ca.Cert, &key.PublicKey, ca.Key)
|
||||
if err != nil {
|
||||
t.Fatalf("create cert: %v", err)
|
||||
}
|
||||
|
||||
cert, err := x509.ParseCertificate(certDER)
|
||||
if err != nil {
|
||||
t.Fatalf("parse cert: %v", err)
|
||||
}
|
||||
|
||||
tlsCert := tls.Certificate{
|
||||
Certificate: [][]byte{certDER},
|
||||
PrivateKey: key,
|
||||
}
|
||||
|
||||
return &testCert{Cert: cert, Key: key, TLSCert: tlsCert}
|
||||
}
|
||||
|
||||
// ── FingerprintCert tests ───────────────────────────────────────────
|
||||
|
||||
func TestFingerprintCert(t *testing.T) {
|
||||
ca := newTestCA(t)
|
||||
c1 := ca.issueClient(t, "alice", "")
|
||||
c2 := ca.issueClient(t, "alice", "") // same CN, different cert
|
||||
|
||||
fp1 := FingerprintCert(c1.Cert)
|
||||
fp2 := FingerprintCert(c2.Cert)
|
||||
|
||||
if fp1 == "" {
|
||||
t.Error("fingerprint should not be empty")
|
||||
}
|
||||
if len(fp1) != 64 { // sha256 hex = 64 chars
|
||||
t.Errorf("fingerprint length = %d, want 64", len(fp1))
|
||||
}
|
||||
if fp1 == fp2 {
|
||||
t.Error("different certs should have different fingerprints")
|
||||
}
|
||||
|
||||
// Same cert → same fingerprint (deterministic)
|
||||
if FingerprintCert(c1.Cert) != fp1 {
|
||||
t.Error("fingerprint should be deterministic")
|
||||
}
|
||||
}
|
||||
|
||||
// ── MTLSNativeProvider unit tests ───────────────────────────────────
|
||||
|
||||
func TestMTLSNativeProvider_Mode(t *testing.T) {
|
||||
p := NewMTLSNativeProvider(MTLSNativeConfig{})
|
||||
if p.Mode() != ModeMTLS {
|
||||
t.Errorf("Mode() = %q, want %q", p.Mode(), ModeMTLS)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMTLSNativeProvider_NoRegistration(t *testing.T) {
|
||||
p := NewMTLSNativeProvider(MTLSNativeConfig{})
|
||||
if p.SupportsRegistration() {
|
||||
t.Error("native mTLS should not support registration")
|
||||
}
|
||||
_, err := p.Register(nil, store.Stores{})
|
||||
if err != ErrNotSupported {
|
||||
t.Errorf("Register() = %v, want ErrNotSupported", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMTLSNativeProvider_NilTLS(t *testing.T) {
|
||||
p := NewMTLSNativeProvider(MTLSNativeConfig{})
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest("GET", "/", nil)
|
||||
c.Request.TLS = nil
|
||||
|
||||
_, err := p.Authenticate(c, store.Stores{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error for nil TLS")
|
||||
}
|
||||
if !errors.Is(err, ErrNoCert) {
|
||||
t.Errorf("expected ErrNoCert, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMTLSNativeProvider_EmptyPeerCerts(t *testing.T) {
|
||||
p := NewMTLSNativeProvider(MTLSNativeConfig{})
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest("GET", "/", nil)
|
||||
c.Request.TLS = &tls.ConnectionState{
|
||||
PeerCertificates: []*x509.Certificate{},
|
||||
}
|
||||
|
||||
_, err := p.Authenticate(c, store.Stores{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error for empty peer certs")
|
||||
}
|
||||
if !errors.Is(err, ErrNoCert) {
|
||||
t.Errorf("expected ErrNoCert, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMTLSNativeProvider_NoCN(t *testing.T) {
|
||||
p := NewMTLSNativeProvider(MTLSNativeConfig{AutoActivate: true})
|
||||
ca := newTestCA(t)
|
||||
cert := ca.issueCert(t, "", "", []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth}, time.Now().Add(time.Hour))
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest("GET", "/", nil)
|
||||
c.Request.TLS = &tls.ConnectionState{
|
||||
PeerCertificates: []*x509.Certificate{cert.Cert},
|
||||
}
|
||||
|
||||
_, err := p.Authenticate(c, store.Stores{})
|
||||
if err == nil {
|
||||
t.Fatal("expected error for cert with no CN")
|
||||
}
|
||||
if !errors.Is(err, ErrInvalidCreds) {
|
||||
t.Errorf("expected ErrInvalidCreds, got: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMTLSNativeProvider_ExtractsEmail(t *testing.T) {
|
||||
ca := newTestCA(t)
|
||||
cert := ca.issueClient(t, "alice", "alice@example.com")
|
||||
|
||||
if len(cert.Cert.EmailAddresses) == 0 || cert.Cert.EmailAddresses[0] != "alice@example.com" {
|
||||
t.Fatalf("cert should have email alice@example.com, got %v", cert.Cert.EmailAddresses)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Integration tests (real TLS listener) ───────────────────────────
|
||||
|
||||
func TestTLS_NoClientCert_Rejected(t *testing.T) {
|
||||
ca := newTestCA(t)
|
||||
serverCert := ca.issueNode(t, "server")
|
||||
|
||||
srv := newMTLSTestServer(t, ca, serverCert)
|
||||
defer srv.Close()
|
||||
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{
|
||||
RootCAs: ca.Pool,
|
||||
MinVersion: tls.VersionTLS13,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := client.Get(srv.URL + "/test")
|
||||
if err == nil {
|
||||
t.Fatal("expected TLS handshake error when no client cert is presented")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTLS_WrongCA_Rejected(t *testing.T) {
|
||||
ca := newTestCA(t)
|
||||
wrongCA := newTestCA(t)
|
||||
serverCert := ca.issueNode(t, "server")
|
||||
clientCert := wrongCA.issueClient(t, "intruder", "")
|
||||
|
||||
srv := newMTLSTestServer(t, ca, serverCert)
|
||||
defer srv.Close()
|
||||
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{
|
||||
RootCAs: ca.Pool,
|
||||
Certificates: []tls.Certificate{clientCert.TLSCert},
|
||||
MinVersion: tls.VersionTLS13,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := client.Get(srv.URL + "/test")
|
||||
if err == nil {
|
||||
t.Fatal("expected TLS handshake error when client cert is from wrong CA")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTLS_ValidClientCert_Accepted(t *testing.T) {
|
||||
ca := newTestCA(t)
|
||||
serverCert := ca.issueNode(t, "server")
|
||||
clientCert := ca.issueClient(t, "alice", "")
|
||||
|
||||
srv := newMTLSTestServer(t, ca, serverCert)
|
||||
defer srv.Close()
|
||||
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{
|
||||
RootCAs: ca.Pool,
|
||||
Certificates: []tls.Certificate{clientCert.TLSCert},
|
||||
MinVersion: tls.VersionTLS13,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
resp, err := client.Get(srv.URL + "/test")
|
||||
if err != nil {
|
||||
t.Fatalf("expected successful connection, got: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != 200 {
|
||||
t.Errorf("status = %d, want 200", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTLS_ExpiredCert_Rejected(t *testing.T) {
|
||||
ca := newTestCA(t)
|
||||
serverCert := ca.issueNode(t, "server")
|
||||
clientCert := ca.issueExpiredClient(t, "expired-user")
|
||||
|
||||
srv := newMTLSTestServer(t, ca, serverCert)
|
||||
defer srv.Close()
|
||||
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{
|
||||
RootCAs: ca.Pool,
|
||||
Certificates: []tls.Certificate{clientCert.TLSCert},
|
||||
MinVersion: tls.VersionTLS13,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := client.Get(srv.URL + "/test")
|
||||
if err == nil {
|
||||
t.Fatal("expected TLS handshake error for expired client cert")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTLS_PeerCertificateVisible(t *testing.T) {
|
||||
ca := newTestCA(t)
|
||||
serverCert := ca.issueNode(t, "server")
|
||||
clientCert := ca.issueClient(t, "bob", "bob@example.com")
|
||||
|
||||
var seenCN string
|
||||
var seenEmails []string
|
||||
|
||||
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.TLS != nil && len(r.TLS.PeerCertificates) > 0 {
|
||||
peer := r.TLS.PeerCertificates[0]
|
||||
seenCN = peer.Subject.CommonName
|
||||
seenEmails = peer.EmailAddresses
|
||||
}
|
||||
w.WriteHeader(200)
|
||||
})
|
||||
|
||||
srv := newMTLSTestServerWithHandler(t, ca, serverCert, handler)
|
||||
defer srv.Close()
|
||||
|
||||
client := &http.Client{
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: &tls.Config{
|
||||
RootCAs: ca.Pool,
|
||||
Certificates: []tls.Certificate{clientCert.TLSCert},
|
||||
MinVersion: tls.VersionTLS13,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
resp, err := client.Get(srv.URL + "/test")
|
||||
if err != nil {
|
||||
t.Fatalf("request failed: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if seenCN != "bob" {
|
||||
t.Errorf("CN = %q, want %q", seenCN, "bob")
|
||||
}
|
||||
if len(seenEmails) == 0 || seenEmails[0] != "bob@example.com" {
|
||||
t.Errorf("emails = %v, want [bob@example.com]", seenEmails)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Test helpers ────────────────────────────────────────────────────
|
||||
|
||||
func newMTLSTestServer(t *testing.T, ca *testCA, serverCert *testCert) *httptest.Server {
|
||||
t.Helper()
|
||||
return newMTLSTestServerWithHandler(t, ca, serverCert, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(200)
|
||||
fmt.Fprint(w, "ok")
|
||||
}))
|
||||
}
|
||||
|
||||
func newMTLSTestServerWithHandler(t *testing.T, ca *testCA, serverCert *testCert, handler http.Handler) *httptest.Server {
|
||||
t.Helper()
|
||||
srv := httptest.NewUnstartedServer(handler)
|
||||
srv.TLS = &tls.Config{
|
||||
Certificates: []tls.Certificate{serverCert.TLSCert},
|
||||
ClientAuth: tls.RequireAndVerifyClientCert,
|
||||
ClientCAs: ca.Pool,
|
||||
MinVersion: tls.VersionTLS13,
|
||||
}
|
||||
srv.StartTLS()
|
||||
return srv
|
||||
}
|
||||
94
server/auth/mtls_proxy.go
Normal file
94
server/auth/mtls_proxy.go
Normal file
@@ -0,0 +1,94 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// MTLSProxyConfig holds configuration for the proxy-terminated mTLS provider.
|
||||
type MTLSProxyConfig struct {
|
||||
HeaderDN string // header carrying cert DN (default "X-SSL-Client-DN")
|
||||
HeaderVerify string // header carrying verify status (default "X-SSL-Client-Verify")
|
||||
HeaderFingerprint string // header carrying cert fingerprint (default "X-SSL-Client-Fingerprint")
|
||||
AutoActivate bool // auto-activate new users (default true)
|
||||
DefaultTeam string // team ID for auto-provisioned users (optional)
|
||||
}
|
||||
|
||||
// MTLSProxyProvider authenticates via client certificate headers injected
|
||||
// by the TLS-terminating reverse proxy (nginx, Traefik, Istio).
|
||||
//
|
||||
// The backend never sees the actual TLS handshake — it trusts headers
|
||||
// injected by the proxy after cert validation. After reading and parsing
|
||||
// the DN, the provider resolves an existing user or auto-provisions a
|
||||
// new one, then returns a Result. The auth handler issues an internal JWT.
|
||||
//
|
||||
// Header flow:
|
||||
//
|
||||
// Client cert → nginx ssl_verify_client → injects X-SSL-Client-DN,
|
||||
// X-SSL-Client-Verify, X-SSL-Client-Fingerprint → backend reads headers.
|
||||
type MTLSProxyProvider struct {
|
||||
cfg MTLSProxyConfig
|
||||
}
|
||||
|
||||
func NewMTLSProxyProvider(cfg MTLSProxyConfig) *MTLSProxyProvider {
|
||||
if cfg.HeaderDN == "" {
|
||||
cfg.HeaderDN = "X-SSL-Client-DN"
|
||||
}
|
||||
if cfg.HeaderVerify == "" {
|
||||
cfg.HeaderVerify = "X-SSL-Client-Verify"
|
||||
}
|
||||
if cfg.HeaderFingerprint == "" {
|
||||
cfg.HeaderFingerprint = "X-SSL-Client-Fingerprint"
|
||||
}
|
||||
return &MTLSProxyProvider{cfg: cfg}
|
||||
}
|
||||
|
||||
func (p *MTLSProxyProvider) Mode() Mode { return ModeMTLS }
|
||||
|
||||
func (p *MTLSProxyProvider) SupportsRegistration() bool { return false }
|
||||
|
||||
func (p *MTLSProxyProvider) Register(_ *gin.Context, _ store.Stores) (*Result, error) {
|
||||
return nil, ErrNotSupported
|
||||
}
|
||||
|
||||
// Authenticate reads the cert DN and verify status from proxy headers,
|
||||
// parses the DN fields, and resolves or auto-provisions the user.
|
||||
//
|
||||
// Returns ErrInvalidCreds when headers are missing or verify fails.
|
||||
// Returns ErrInactive when user exists but is deactivated.
|
||||
func (p *MTLSProxyProvider) Authenticate(c *gin.Context, stores store.Stores) (*Result, error) {
|
||||
// ── Validate headers ───────────────────────────────────────────
|
||||
verify := c.GetHeader(p.cfg.HeaderVerify)
|
||||
if verify == "" {
|
||||
return nil, fmt.Errorf("%w: missing %s header", ErrInvalidCreds, p.cfg.HeaderVerify)
|
||||
}
|
||||
// nginx: "SUCCESS", Traefik: "0" (both mean valid cert)
|
||||
if verify != "SUCCESS" && verify != "0" {
|
||||
return nil, fmt.Errorf("%w: cert verify=%s", ErrInvalidCreds, verify)
|
||||
}
|
||||
|
||||
dn := c.GetHeader(p.cfg.HeaderDN)
|
||||
if dn == "" {
|
||||
return nil, fmt.Errorf("%w: missing %s header", ErrInvalidCreds, p.cfg.HeaderDN)
|
||||
}
|
||||
|
||||
fields := ParseDN(dn)
|
||||
cn := fields["CN"]
|
||||
if cn == "" {
|
||||
return nil, fmt.Errorf("%w: cert DN has no CN field", ErrInvalidCreds)
|
||||
}
|
||||
|
||||
// Stable external identity: fingerprint if available, else full DN
|
||||
fingerprint := c.GetHeader(p.cfg.HeaderFingerprint)
|
||||
if fingerprint == "" {
|
||||
fingerprint = dn
|
||||
}
|
||||
|
||||
return resolveOrProvision(
|
||||
c.Request.Context(), stores, cn, fields, fingerprint,
|
||||
p.cfg.AutoActivate, p.cfg.DefaultTeam,
|
||||
)
|
||||
}
|
||||
@@ -85,7 +85,7 @@ func TestParseDN(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestMTLSConfig_Defaults(t *testing.T) {
|
||||
p := NewMTLSProvider(MTLSConfig{})
|
||||
p := NewMTLSProxyProvider(MTLSProxyConfig{})
|
||||
|
||||
if p.cfg.HeaderDN != "X-SSL-Client-DN" {
|
||||
t.Errorf("HeaderDN = %q, want X-SSL-Client-DN", p.cfg.HeaderDN)
|
||||
@@ -99,7 +99,7 @@ func TestMTLSConfig_Defaults(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestMTLSConfig_Custom(t *testing.T) {
|
||||
p := NewMTLSProvider(MTLSConfig{
|
||||
p := NewMTLSProxyProvider(MTLSProxyConfig{
|
||||
HeaderDN: "X-Client-Cert-DN",
|
||||
HeaderVerify: "X-Client-Cert-Verify",
|
||||
})
|
||||
@@ -110,14 +110,14 @@ func TestMTLSConfig_Custom(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestMTLSProvider_Mode(t *testing.T) {
|
||||
p := NewMTLSProvider(MTLSConfig{})
|
||||
p := NewMTLSProxyProvider(MTLSProxyConfig{})
|
||||
if p.Mode() != ModeMTLS {
|
||||
t.Errorf("Mode() = %q, want %q", p.Mode(), ModeMTLS)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMTLSProvider_NoRegistration(t *testing.T) {
|
||||
p := NewMTLSProvider(MTLSConfig{})
|
||||
p := NewMTLSProxyProvider(MTLSProxyConfig{})
|
||||
if p.SupportsRegistration() {
|
||||
t.Error("mTLS should not support registration")
|
||||
}
|
||||
@@ -246,6 +246,7 @@ type oidcClaims struct {
|
||||
Email string `json:"email"`
|
||||
PreferredUsername string `json:"preferred_username"`
|
||||
Name string `json:"name"`
|
||||
Nonce string `json:"nonce"`
|
||||
Groups []string `json:"groups"`
|
||||
RealmAccess realmAccess `json:"realm_access"`
|
||||
ResourceAccess interface{} `json:"resource_access"`
|
||||
@@ -304,6 +305,23 @@ func (p *OIDCProvider) validateToken(tokenString string) (*oidcClaims, error) {
|
||||
return claims, nil
|
||||
}
|
||||
|
||||
// ValidateIDTokenNonce validates that the ID token's nonce claim matches the
|
||||
// stored nonce from the authorization request. This prevents token replay and
|
||||
// substitution attacks in the authorization code flow.
|
||||
func (p *OIDCProvider) ValidateIDTokenNonce(tokenString, expectedNonce string) error {
|
||||
if expectedNonce == "" {
|
||||
return nil // no nonce to validate (e.g. direct token flow)
|
||||
}
|
||||
claims, err := p.validateToken(tokenString)
|
||||
if err != nil {
|
||||
return fmt.Errorf("token validation failed: %w", err)
|
||||
}
|
||||
if claims.Nonce != expectedNonce {
|
||||
return fmt.Errorf("nonce mismatch: expected %q, got %q", expectedNonce, claims.Nonce)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// isIdPAdmin checks whether the OIDC claims include the configured admin role.
|
||||
func (p *OIDCProvider) isIdPAdmin(claims *oidcClaims) bool {
|
||||
for _, role := range claims.RealmAccess.Roles {
|
||||
|
||||
178
server/cluster/registry.go
Normal file
178
server/cluster/registry.go
Normal file
@@ -0,0 +1,178 @@
|
||||
package cluster
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"log"
|
||||
"os"
|
||||
"runtime"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// ConnCounter provides WebSocket connection count (satisfied by events.Hub).
|
||||
type ConnCounter interface {
|
||||
ConnCount() int
|
||||
}
|
||||
|
||||
// RegistryConfig holds tunable cluster registry parameters.
|
||||
type RegistryConfig struct {
|
||||
HeartbeatInterval time.Duration
|
||||
StaleThreshold time.Duration
|
||||
}
|
||||
|
||||
// Registry manages node self-registration, heartbeat, and stale sweep.
|
||||
// Follows the workflow.Scanner pattern: stopCh + wg + ticker.
|
||||
type Registry struct {
|
||||
nodeID string
|
||||
endpoint string
|
||||
store store.ClusterStore
|
||||
hub ConnCounter
|
||||
cfg RegistryConfig
|
||||
startTime time.Time
|
||||
stopCh chan struct{}
|
||||
wg sync.WaitGroup
|
||||
|
||||
// Optional callbacks — set after construction via setters.
|
||||
sandboxStats func() (exec, errors uint64, avgMs float64)
|
||||
triggerFires func() int64
|
||||
extensionCount func() int
|
||||
}
|
||||
|
||||
// SetSandboxStats registers a callback for sandbox execution counters.
|
||||
func (r *Registry) SetSandboxStats(fn func() (uint64, uint64, float64)) {
|
||||
r.sandboxStats = fn
|
||||
}
|
||||
|
||||
// SetTriggerFireCount registers a callback for trigger fire count.
|
||||
func (r *Registry) SetTriggerFireCount(fn func() int64) {
|
||||
r.triggerFires = fn
|
||||
}
|
||||
|
||||
// SetExtensionCount registers a callback for active extension count.
|
||||
func (r *Registry) SetExtensionCount(fn func() int) {
|
||||
r.extensionCount = fn
|
||||
}
|
||||
|
||||
// NewRegistry creates a cluster registry instance.
|
||||
func NewRegistry(nodeID, endpoint string, s store.ClusterStore, hub ConnCounter, cfg RegistryConfig) *Registry {
|
||||
return &Registry{
|
||||
nodeID: nodeID,
|
||||
endpoint: endpoint,
|
||||
store: s,
|
||||
hub: hub,
|
||||
cfg: cfg,
|
||||
startTime: time.Now(),
|
||||
stopCh: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
|
||||
// NodeID returns this node's identifier.
|
||||
func (r *Registry) NodeID() string { return r.nodeID }
|
||||
|
||||
// Start registers the node and launches the heartbeat goroutine.
|
||||
func (r *Registry) Start() error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := r.store.Register(ctx, r.nodeID, r.endpoint); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
r.wg.Add(1)
|
||||
go func() {
|
||||
defer r.wg.Done()
|
||||
ticker := time.NewTicker(r.cfg.HeartbeatInterval)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
r.tick()
|
||||
case <-r.stopCh:
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
log.Printf("[cluster] started (node=%s, heartbeat=%s, stale=%s)",
|
||||
r.nodeID, r.cfg.HeartbeatInterval, r.cfg.StaleThreshold)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Stop deregisters the node and waits for the heartbeat goroutine to exit.
|
||||
func (r *Registry) Stop() {
|
||||
close(r.stopCh)
|
||||
r.wg.Wait()
|
||||
|
||||
// Best-effort deregister so peers don't have to wait for stale sweep.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
if err := r.store.Deregister(ctx, r.nodeID); err != nil {
|
||||
log.Printf("[cluster] deregister failed: %v", err)
|
||||
}
|
||||
log.Printf("[cluster] stopped (node=%s)", r.nodeID)
|
||||
}
|
||||
|
||||
// tick runs one heartbeat + sweep cycle.
|
||||
func (r *Registry) tick() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
stats := r.collectStats()
|
||||
|
||||
rows, err := r.store.Heartbeat(ctx, r.nodeID, stats)
|
||||
if err != nil {
|
||||
log.Printf("[cluster] heartbeat error: %v", err)
|
||||
return
|
||||
}
|
||||
if rows == 0 {
|
||||
// Self-eviction: this node was swept by a peer.
|
||||
// Kubernetes will restart the process → re-register.
|
||||
log.Printf("[cluster] FATAL: node %s evicted from registry — shutting down", r.nodeID)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
swept, err := r.store.SweepStale(ctx, r.cfg.StaleThreshold)
|
||||
if err != nil {
|
||||
log.Printf("[cluster] sweep error: %v", err)
|
||||
return
|
||||
}
|
||||
if swept > 0 {
|
||||
log.Printf("[cluster] swept %d stale node(s)", swept)
|
||||
}
|
||||
}
|
||||
|
||||
// collectStats gathers runtime metrics for the stats JSONB column.
|
||||
func (r *Registry) collectStats() json.RawMessage {
|
||||
var m runtime.MemStats
|
||||
runtime.ReadMemStats(&m)
|
||||
|
||||
stats := map[string]any{
|
||||
"goroutines": runtime.NumGoroutine(),
|
||||
"heap_alloc": m.HeapAlloc,
|
||||
"heap_sys": m.HeapSys,
|
||||
"stack_in_use": m.StackInuse,
|
||||
"gc_cycles": m.NumGC,
|
||||
"gc_pause_ns": m.PauseNs[(m.NumGC+255)%256],
|
||||
"gc_cpu_pct": m.GCCPUFraction * 100,
|
||||
"uptime_sec": time.Since(r.startTime).Seconds(),
|
||||
"ws_clients": r.hub.ConnCount(),
|
||||
}
|
||||
|
||||
if r.extensionCount != nil {
|
||||
stats["extensions_loaded"] = r.extensionCount()
|
||||
}
|
||||
if r.sandboxStats != nil {
|
||||
exec, errors, avgMs := r.sandboxStats()
|
||||
stats["starlark_exec_total"] = exec
|
||||
stats["starlark_errors_total"] = errors
|
||||
stats["starlark_avg_duration_ms"] = avgMs
|
||||
}
|
||||
if r.triggerFires != nil {
|
||||
stats["trigger_fires_total"] = r.triggerFires()
|
||||
}
|
||||
|
||||
data, _ := json.Marshal(stats)
|
||||
return data
|
||||
}
|
||||
152
server/cluster/registry_test.go
Normal file
152
server/cluster/registry_test.go
Normal file
@@ -0,0 +1,152 @@
|
||||
package cluster
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// mockClusterStore implements store.ClusterStore for unit tests.
|
||||
type mockClusterStore struct {
|
||||
nodes []store.ClusterNode
|
||||
heartbeatRows int64
|
||||
heartbeatErr error
|
||||
sweepCalled bool
|
||||
registerCalled bool
|
||||
lastStats json.RawMessage
|
||||
}
|
||||
|
||||
func (m *mockClusterStore) Register(_ context.Context, nodeID, endpoint string) error {
|
||||
m.registerCalled = true
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *mockClusterStore) Heartbeat(_ context.Context, nodeID string, stats json.RawMessage) (int64, error) {
|
||||
m.lastStats = stats
|
||||
return m.heartbeatRows, m.heartbeatErr
|
||||
}
|
||||
|
||||
func (m *mockClusterStore) SweepStale(_ context.Context, _ time.Duration) (int64, error) {
|
||||
m.sweepCalled = true
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
func (m *mockClusterStore) ListNodes(_ context.Context) ([]store.ClusterNode, error) {
|
||||
return m.nodes, nil
|
||||
}
|
||||
|
||||
func (m *mockClusterStore) Deregister(_ context.Context, _ string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// mockHub implements ConnCounter.
|
||||
type mockHub struct{ count int }
|
||||
|
||||
func (h *mockHub) ConnCount() int { return h.count }
|
||||
|
||||
func TestCollectStats(t *testing.T) {
|
||||
hub := &mockHub{count: 5}
|
||||
reg := NewRegistry("test-node", "http://localhost:8080", nil, hub, RegistryConfig{
|
||||
HeartbeatInterval: 10 * time.Second,
|
||||
StaleThreshold: 30 * time.Second,
|
||||
})
|
||||
|
||||
data := reg.collectStats()
|
||||
|
||||
var stats map[string]any
|
||||
if err := json.Unmarshal(data, &stats); err != nil {
|
||||
t.Fatalf("collectStats returned invalid JSON: %v", err)
|
||||
}
|
||||
|
||||
// Verify expected keys
|
||||
expectedKeys := []string{"goroutines", "heap_alloc", "heap_sys", "gc_cycles", "gc_pause_ns", "uptime_sec", "ws_clients"}
|
||||
for _, key := range expectedKeys {
|
||||
if _, ok := stats[key]; !ok {
|
||||
t.Errorf("missing expected stats key: %s", key)
|
||||
}
|
||||
}
|
||||
|
||||
// ws_clients should reflect hub count
|
||||
if ws, ok := stats["ws_clients"].(float64); !ok || int(ws) != 5 {
|
||||
t.Errorf("ws_clients = %v, want 5", stats["ws_clients"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollectStatsFattened(t *testing.T) {
|
||||
hub := &mockHub{count: 3}
|
||||
reg := NewRegistry("test-node", "", nil, hub, RegistryConfig{
|
||||
HeartbeatInterval: 10 * time.Second,
|
||||
StaleThreshold: 30 * time.Second,
|
||||
})
|
||||
reg.SetSandboxStats(func() (uint64, uint64, float64) { return 100, 5, 12.3 })
|
||||
reg.SetTriggerFireCount(func() int64 { return 42 })
|
||||
reg.SetExtensionCount(func() int { return 8 })
|
||||
|
||||
data := reg.collectStats()
|
||||
|
||||
var stats map[string]any
|
||||
if err := json.Unmarshal(data, &stats); err != nil {
|
||||
t.Fatalf("collectStats returned invalid JSON: %v", err)
|
||||
}
|
||||
|
||||
// New fattened keys
|
||||
fatKeys := []string{
|
||||
"stack_in_use", "gc_cpu_pct",
|
||||
"extensions_loaded", "starlark_exec_total", "starlark_errors_total",
|
||||
"starlark_avg_duration_ms", "trigger_fires_total",
|
||||
}
|
||||
for _, key := range fatKeys {
|
||||
if _, ok := stats[key]; !ok {
|
||||
t.Errorf("missing fattened stats key: %s", key)
|
||||
}
|
||||
}
|
||||
|
||||
if v := stats["extensions_loaded"].(float64); int(v) != 8 {
|
||||
t.Errorf("extensions_loaded = %v, want 8", v)
|
||||
}
|
||||
if v := stats["starlark_exec_total"].(float64); int(v) != 100 {
|
||||
t.Errorf("starlark_exec_total = %v, want 100", v)
|
||||
}
|
||||
if v := stats["trigger_fires_total"].(float64); int(v) != 42 {
|
||||
t.Errorf("trigger_fires_total = %v, want 42", v)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRegistryStartStop(t *testing.T) {
|
||||
ms := &mockClusterStore{heartbeatRows: 1}
|
||||
hub := &mockHub{count: 0}
|
||||
|
||||
reg := NewRegistry("test-node", "", ms, hub, RegistryConfig{
|
||||
HeartbeatInterval: 50 * time.Millisecond,
|
||||
StaleThreshold: 150 * time.Millisecond,
|
||||
})
|
||||
|
||||
if err := reg.Start(); err != nil {
|
||||
t.Fatalf("Start() error: %v", err)
|
||||
}
|
||||
if !ms.registerCalled {
|
||||
t.Error("Register was not called on Start")
|
||||
}
|
||||
|
||||
// Let at least one tick fire
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
|
||||
reg.Stop()
|
||||
|
||||
if !ms.sweepCalled {
|
||||
t.Error("SweepStale was never called during tick")
|
||||
}
|
||||
if ms.lastStats == nil {
|
||||
t.Error("Heartbeat was never called with stats")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNodeID(t *testing.T) {
|
||||
reg := NewRegistry("my-node-123", "", nil, &mockHub{}, RegistryConfig{})
|
||||
if got := reg.NodeID(); got != "my-node-123" {
|
||||
t.Errorf("NodeID() = %q, want %q", got, "my-node-123")
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/joho/godotenv"
|
||||
)
|
||||
@@ -25,18 +26,18 @@ type Config struct {
|
||||
// Seed users (dev/test only) — CSV: "user:pass:role,user2:pass2:role2"
|
||||
SeedUsers string
|
||||
|
||||
// API key encryption (required for v0.9.4+)
|
||||
// API key encryption (required+)
|
||||
// Used to derive AES-256 key for global/team provider API keys.
|
||||
// Personal keys use per-user UEK (derived from password).
|
||||
EncryptionKey string
|
||||
|
||||
// File storage (v0.12.0+)
|
||||
// File storage
|
||||
// STORAGE_BACKEND: "pvc" or "s3". Empty = auto-detect (pvc if path writable).
|
||||
// STORAGE_PATH: mount point for PVC backend (default /data/storage).
|
||||
StorageBackend string
|
||||
StoragePath string
|
||||
|
||||
// S3-compatible storage (v0.12.0+)
|
||||
// S3-compatible storage
|
||||
// Works with AWS S3, MinIO, Ceph RGW, or any S3-compatible API.
|
||||
S3Endpoint string // custom endpoint URL (required for MinIO/Ceph, optional for AWS)
|
||||
S3Bucket string // bucket name (required when STORAGE_BACKEND=s3)
|
||||
@@ -46,16 +47,25 @@ type Config struct {
|
||||
S3Prefix string // optional key prefix within bucket (e.g. "switchboard/")
|
||||
S3ForcePathStyle bool // use path-style URLs (required for MinIO, most self-hosted)
|
||||
|
||||
// Structured logging (v0.33.0)
|
||||
// Structured logging
|
||||
// LOG_FORMAT: "text" (default, backward-compatible) or "json" (structured).
|
||||
// LOG_LEVEL: "debug", "info" (default), "warn", "error".
|
||||
LogFormat string
|
||||
LogLevel string
|
||||
|
||||
// Auth mode (v0.24.0): "builtin" (default) | "mtls" | "oidc"
|
||||
// TLS termination
|
||||
// TLS_MODE: "none" (default) | "server" (TLS, no client cert) | "mtls" (mutual TLS)
|
||||
// When server or mtls: binary calls ListenAndServeTLS directly.
|
||||
// TLS_MODE and AUTH_MODE are independent knobs.
|
||||
TLSMode string
|
||||
TLSCert string // path to server certificate PEM
|
||||
TLSKey string // path to server private key PEM
|
||||
TLSCA string // path to CA certificate PEM (required for mtls)
|
||||
|
||||
// Auth mode: "builtin" (default) | "mtls" | "oidc"
|
||||
AuthMode string
|
||||
|
||||
// mTLS (v0.24.1)
|
||||
// mTLS (proxy mode)
|
||||
// Headers injected by TLS-terminating reverse proxy.
|
||||
MTLSHeaderDN string // default "X-SSL-Client-DN"
|
||||
MTLSHeaderVerify string // default "X-SSL-Client-Verify"
|
||||
@@ -63,7 +73,18 @@ type Config struct {
|
||||
MTLSAutoActivate bool // auto-activate new users (default true)
|
||||
MTLSDefaultTeam string // team ID for auto-provisioned users (optional)
|
||||
|
||||
// OIDC (v0.24.1)
|
||||
// Bundled packages
|
||||
// SKIP_BUNDLED_PACKAGES: set true to disable auto-install of bundled packages on first run.
|
||||
// BUNDLED_PACKAGES_DIR: directory containing pre-built .pkg archives (default /app/bundled-packages).
|
||||
// BUNDLED_PACKAGES: controls which bundled packages are auto-installed:
|
||||
// empty → curated default set (notes, chat-core, workflow-chat, dashboard, demo workflows)
|
||||
// "*" → install all .pkg archives found in the directory
|
||||
// "a,b" → comma-separated allowlist of specific package IDs
|
||||
SkipBundledPackages bool
|
||||
BundledPackagesDir string
|
||||
BundledPackages string
|
||||
|
||||
// OIDC
|
||||
OIDCIssuerURL string // e.g. "https://keycloak.corp/realms/switchboard"
|
||||
OIDCExternalIssuerURL string // OIDC_EXTERNAL_ISSUER_URL
|
||||
OIDCClientID string
|
||||
@@ -74,6 +95,23 @@ type Config struct {
|
||||
OIDCRolesClaim string // JWT claim for role mapping (default "realm_access.roles")
|
||||
OIDCGroupsClaim string // JWT claim for group sync (default "groups")
|
||||
OIDCAdminRole string // IdP role value that maps to admin (default "admin")
|
||||
|
||||
// Cluster registry
|
||||
// PG-backed node registry for horizontal scaling. No-op on SQLite.
|
||||
// CLUSTER_NODE_ID: override for deterministic identity (default: hostname-PID).
|
||||
// CLUSTER_HEARTBEAT_INTERVAL: tick frequency (default "10s").
|
||||
// CLUSTER_STALE_THRESHOLD: 3x heartbeat = stale (default "30s").
|
||||
// CLUSTER_ENDPOINT: advertised address for peer mesh (Phase 2, auto-detect).
|
||||
ClusterNodeID string
|
||||
ClusterHeartbeatInterval time.Duration
|
||||
ClusterStaleThreshold time.Duration
|
||||
ClusterEndpoint string
|
||||
|
||||
// Package signing (future)
|
||||
// PACKAGE_VERIFY_SIGNATURES: when true, log warnings for unsigned packages.
|
||||
// No cryptographic verification yet — this reserves the manifest schema slot
|
||||
// and config plumbing so it can be implemented without a breaking change.
|
||||
PackageVerifySignatures bool
|
||||
}
|
||||
|
||||
// Load reads configuration from environment variables.
|
||||
@@ -105,9 +143,19 @@ func Load() *Config {
|
||||
S3Prefix: getEnv("S3_PREFIX", ""),
|
||||
S3ForcePathStyle: getEnv("S3_FORCE_PATH_STYLE", "true") == "true",
|
||||
|
||||
SkipBundledPackages: getEnvBool("SKIP_BUNDLED_PACKAGES", false),
|
||||
BundledPackagesDir: getEnv("BUNDLED_PACKAGES_DIR", "/app/bundled-packages"),
|
||||
BundledPackages: getEnv("BUNDLED_PACKAGES", ""),
|
||||
|
||||
LogFormat: getEnv("LOG_FORMAT", "text"),
|
||||
LogLevel: getEnv("LOG_LEVEL", "info"),
|
||||
|
||||
// TLS
|
||||
TLSMode: getEnv("TLS_MODE", "none"),
|
||||
TLSCert: getEnv("TLS_CERT", ""),
|
||||
TLSKey: getEnv("TLS_KEY", ""),
|
||||
TLSCA: getEnv("TLS_CA", ""),
|
||||
|
||||
AuthMode: getEnv("AUTH_MODE", "builtin"),
|
||||
|
||||
// mTLS
|
||||
@@ -128,6 +176,14 @@ func Load() *Config {
|
||||
OIDCRolesClaim: getEnv("OIDC_ROLES_CLAIM", "realm_access.roles"),
|
||||
OIDCGroupsClaim: getEnv("OIDC_GROUPS_CLAIM", "groups"),
|
||||
OIDCAdminRole: getEnv("OIDC_ADMIN_ROLE", "admin"),
|
||||
|
||||
// Cluster
|
||||
ClusterNodeID: getEnv("CLUSTER_NODE_ID", ""),
|
||||
ClusterHeartbeatInterval: getEnvDuration("CLUSTER_HEARTBEAT_INTERVAL", 10*time.Second),
|
||||
ClusterStaleThreshold: getEnvDuration("CLUSTER_STALE_THRESHOLD", 30*time.Second),
|
||||
ClusterEndpoint: getEnv("CLUSTER_ENDPOINT", ""),
|
||||
|
||||
PackageVerifySignatures: getEnvBool("PACKAGE_VERIFY_SIGNATURES", false),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -189,6 +245,15 @@ func getEnvInt(key string, fallback int) int {
|
||||
return fallback
|
||||
}
|
||||
|
||||
func getEnvDuration(key string, fallback time.Duration) time.Duration {
|
||||
if v := os.Getenv(key); v != "" {
|
||||
if d, err := time.ParseDuration(v); err == nil {
|
||||
return d
|
||||
}
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func getEnvBool(key string, fallback bool) bool {
|
||||
if v := os.Getenv(key); v != "" {
|
||||
b, err := strconv.ParseBool(v)
|
||||
|
||||
98
server/config/tls.go
Normal file
98
server/config/tls.go
Normal file
@@ -0,0 +1,98 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
)
|
||||
|
||||
// TLS mode values for TLS_MODE env var.
|
||||
const (
|
||||
TLSModeNone = "none" // Plain HTTP (default). Use behind a proxy.
|
||||
TLSModeServer = "server" // Server-side TLS only. No client cert required.
|
||||
TLSModeMTLS = "mtls" // Mutual TLS. Client cert required and verified.
|
||||
)
|
||||
|
||||
var (
|
||||
ErrTLSCertRequired = errors.New("TLS_CERT and TLS_KEY required when TLS_MODE is server or mtls")
|
||||
ErrTLSCARequired = errors.New("TLS_CA required when TLS_MODE is mtls")
|
||||
ErrTLSCALoad = errors.New("failed to load TLS CA certificate")
|
||||
ErrTLSModeInvalid = errors.New("TLS_MODE must be none, server, or mtls")
|
||||
)
|
||||
|
||||
// ValidateTLSMode returns an error if the TLS_MODE value is not recognized.
|
||||
func ValidateTLSMode(mode string) error {
|
||||
switch mode {
|
||||
case TLSModeNone, TLSModeServer, TLSModeMTLS, "":
|
||||
return nil
|
||||
default:
|
||||
return fmt.Errorf("%w: %q", ErrTLSModeInvalid, mode)
|
||||
}
|
||||
}
|
||||
|
||||
// BuildTLSConfig constructs a *tls.Config from the application config.
|
||||
// Returns nil when TLS_MODE is "none" (or empty).
|
||||
// Returns an error if required cert/key/CA files are missing or unreadable.
|
||||
func BuildTLSConfig(cfg *Config) (*tls.Config, error) {
|
||||
mode := cfg.TLSMode
|
||||
if mode == "" || mode == TLSModeNone {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
if cfg.TLSCert == "" || cfg.TLSKey == "" {
|
||||
return nil, ErrTLSCertRequired
|
||||
}
|
||||
|
||||
tlsCfg := &tls.Config{
|
||||
MinVersion: tls.VersionTLS13,
|
||||
}
|
||||
|
||||
if mode == TLSModeMTLS {
|
||||
if cfg.TLSCA == "" {
|
||||
return nil, ErrTLSCARequired
|
||||
}
|
||||
caCert, err := os.ReadFile(cfg.TLSCA)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrTLSCALoad, err)
|
||||
}
|
||||
pool := x509.NewCertPool()
|
||||
if !pool.AppendCertsFromPEM(caCert) {
|
||||
return nil, fmt.Errorf("%w: no valid certificates in %s", ErrTLSCALoad, cfg.TLSCA)
|
||||
}
|
||||
tlsCfg.ClientAuth = tls.RequireAndVerifyClientCert
|
||||
tlsCfg.ClientCAs = pool
|
||||
}
|
||||
|
||||
return tlsCfg, nil
|
||||
}
|
||||
|
||||
// BuildPeerTLSConfig constructs a *tls.Config for outbound node-to-node
|
||||
// connections. The node presents its own cert and verifies peers against
|
||||
// the cluster CA. Returns nil when TLS_MODE is not "mtls".
|
||||
func BuildPeerTLSConfig(cfg *Config) (*tls.Config, error) {
|
||||
if cfg.TLSMode != TLSModeMTLS {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
cert, err := tls.LoadX509KeyPair(cfg.TLSCert, cfg.TLSKey)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to load node keypair: %w", err)
|
||||
}
|
||||
|
||||
caCert, err := os.ReadFile(cfg.TLSCA)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrTLSCALoad, err)
|
||||
}
|
||||
pool := x509.NewCertPool()
|
||||
if !pool.AppendCertsFromPEM(caCert) {
|
||||
return nil, fmt.Errorf("%w: no valid certificates in %s", ErrTLSCALoad, cfg.TLSCA)
|
||||
}
|
||||
|
||||
return &tls.Config{
|
||||
Certificates: []tls.Certificate{cert},
|
||||
RootCAs: pool,
|
||||
MinVersion: tls.VersionTLS13,
|
||||
}, nil
|
||||
}
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
// provider_configs.api_key_plain using the env-derived key, writing the
|
||||
// result to api_key_enc (BYTEA) + key_nonce. Clears api_key_plain after.
|
||||
//
|
||||
// This runs once on upgrade from v0.9.3 → v0.9.4. Subsequent starts are
|
||||
// This runs once on upgrade when ENCRYPTION_KEY is first set. Subsequent starts are
|
||||
// a no-op (api_key_plain will be NULL for all rows).
|
||||
//
|
||||
// Personal-scope keys are temporarily encrypted with the env key. They get
|
||||
@@ -94,7 +94,7 @@ func EnforceEncryptionKey(db *sql.DB, encryptionKey string) error {
|
||||
|
||||
// Check if any plaintext keys exist (pre-backfill)
|
||||
var hasPlaintext bool
|
||||
// api_key_plain column may not exist on fresh v0.9.4+ installs
|
||||
// api_key_plain column may not exist on fresh installs
|
||||
var colExists bool
|
||||
db.QueryRow(`
|
||||
SELECT EXISTS (
|
||||
@@ -121,7 +121,7 @@ func EnforceEncryptionKey(db *sql.DB, encryptionKey string) error {
|
||||
if hasPlaintext {
|
||||
return fmt.Errorf(
|
||||
"ENCRYPTION_KEY is not set but plaintext API keys need encryption. " +
|
||||
"Set ENCRYPTION_KEY before starting (required for v0.9.4+)",
|
||||
"Set ENCRYPTION_KEY before starting (required+)",
|
||||
)
|
||||
}
|
||||
log.Println(" ⚠ ENCRYPTION_KEY not set — API key encryption disabled (OK for fresh install with no providers)")
|
||||
|
||||
@@ -19,7 +19,7 @@ func SchemaVersion() string { return schemaVersion }
|
||||
// Migrate runs all pending migrations for the current dialect.
|
||||
// Migration files live in migrations/<dialect>/ subdirectories.
|
||||
// Falls back to the root migrations/ directory if no subdirectory exists
|
||||
// (backward compatible with pre-v0.17.1 postgres-only layouts).
|
||||
// (backward compatible with postgres-only layouts).
|
||||
func Migrate() error {
|
||||
if DB == nil {
|
||||
return fmt.Errorf("database not connected")
|
||||
|
||||
@@ -24,7 +24,7 @@ CREATE TABLE IF NOT EXISTS packages (
|
||||
schema_version INTEGER NOT NULL DEFAULT 0,
|
||||
package_settings JSONB NOT NULL DEFAULT '{}',
|
||||
source TEXT NOT NULL DEFAULT 'core'
|
||||
CHECK (source IN ('core', 'builtin', 'extension', 'registry')),
|
||||
CHECK (source IN ('core', 'builtin', 'extension', 'registry', 'bundled')),
|
||||
installed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
-- 012_bundled_source.sql — v0.3.8
|
||||
-- Adds 'bundled' to the packages.source CHECK constraint for auto-installed packages.
|
||||
|
||||
ALTER TABLE packages DROP CONSTRAINT IF EXISTS packages_source_check;
|
||||
ALTER TABLE packages ADD CONSTRAINT packages_source_check
|
||||
CHECK (source IN ('core', 'builtin', 'extension', 'registry', 'bundled'));
|
||||
13
server/database/migrations/postgres/013_cluster_registry.sql
Normal file
13
server/database/migrations/postgres/013_cluster_registry.sql
Normal file
@@ -0,0 +1,13 @@
|
||||
-- 013_cluster_registry.sql — v0.6.0
|
||||
-- PG-backed cluster registry for node self-assembly.
|
||||
-- UNLOGGED: no WAL overhead, contents lost on PG crash (correct behavior —
|
||||
-- all nodes are dead anyway and re-register on restart).
|
||||
|
||||
CREATE UNLOGGED TABLE IF NOT EXISTS node_registry (
|
||||
node_id TEXT PRIMARY KEY,
|
||||
endpoint TEXT NOT NULL DEFAULT '',
|
||||
seq SERIAL,
|
||||
registered_at TIMESTAMPTZ DEFAULT now(),
|
||||
heartbeat TIMESTAMPTZ DEFAULT now(),
|
||||
stats JSONB DEFAULT '{}'
|
||||
);
|
||||
@@ -24,7 +24,7 @@ CREATE TABLE IF NOT EXISTS packages (
|
||||
schema_version INTEGER NOT NULL DEFAULT 0,
|
||||
package_settings TEXT NOT NULL DEFAULT '{}',
|
||||
source TEXT NOT NULL DEFAULT 'core'
|
||||
CHECK (source IN ('core', 'builtin', 'extension', 'registry')),
|
||||
CHECK (source IN ('core', 'builtin', 'extension', 'registry', 'bundled')),
|
||||
installed_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
39
server/database/migrations/sqlite/012_bundled_source.sql
Normal file
39
server/database/migrations/sqlite/012_bundled_source.sql
Normal file
@@ -0,0 +1,39 @@
|
||||
-- 012_bundled_source.sql — v0.3.8
|
||||
-- Adds 'bundled' to the packages.source CHECK constraint for auto-installed packages.
|
||||
-- SQLite doesn't support ALTER CHECK — must recreate the table.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS packages_new (
|
||||
id TEXT PRIMARY KEY,
|
||||
title TEXT NOT NULL,
|
||||
type TEXT NOT NULL DEFAULT 'surface'
|
||||
CHECK (type IN ('surface', 'extension', 'full', 'workflow', 'library')),
|
||||
version TEXT NOT NULL DEFAULT '0.0.0',
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
author TEXT NOT NULL DEFAULT '',
|
||||
tier TEXT NOT NULL DEFAULT 'browser'
|
||||
CHECK (tier IN ('browser', 'starlark', 'sidecar')),
|
||||
is_system INTEGER NOT NULL DEFAULT 0,
|
||||
scope TEXT NOT NULL DEFAULT 'global'
|
||||
CHECK (scope IN ('global', 'team', 'personal')),
|
||||
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
|
||||
installed_by TEXT REFERENCES users(id) ON DELETE SET NULL,
|
||||
manifest TEXT NOT NULL DEFAULT '{}',
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
status TEXT NOT NULL DEFAULT 'active'
|
||||
CHECK (status IN ('active', 'pending_review', 'suspended', 'dormant')),
|
||||
schema_version INTEGER NOT NULL DEFAULT 0,
|
||||
package_settings TEXT NOT NULL DEFAULT '{}',
|
||||
source TEXT NOT NULL DEFAULT 'core'
|
||||
CHECK (source IN ('core', 'builtin', 'extension', 'registry', 'bundled')),
|
||||
installed_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
INSERT OR IGNORE INTO packages_new SELECT * FROM packages;
|
||||
DROP TABLE packages;
|
||||
ALTER TABLE packages_new RENAME TO packages;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_packages_type ON packages(type);
|
||||
CREATE INDEX IF NOT EXISTS idx_packages_enabled ON packages(enabled);
|
||||
CREATE INDEX IF NOT EXISTS idx_packages_team ON packages(team_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_packages_source ON packages(source);
|
||||
@@ -3,6 +3,7 @@ package events
|
||||
import (
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
@@ -14,6 +15,8 @@ type Bus struct {
|
||||
subs map[string][]*subscription
|
||||
seq uint64 // subscription ID counter
|
||||
broadcastHook func(Event) // called after Publish for cross-pod fan-out; nil-safe
|
||||
publishCount atomic.Int64
|
||||
deliverCount atomic.Int64
|
||||
}
|
||||
|
||||
type subscription struct {
|
||||
@@ -86,6 +89,8 @@ func (b *Bus) Publish(event Event) {
|
||||
// the broadcastHook. Used by the Postgres listener to re-publish remote
|
||||
// events without causing an infinite re-broadcast loop.
|
||||
func (b *Bus) publishLocal(event Event) {
|
||||
b.publishCount.Add(1)
|
||||
|
||||
b.mu.RLock()
|
||||
var matched []Handler
|
||||
for pattern, subs := range b.subs {
|
||||
@@ -98,6 +103,7 @@ func (b *Bus) publishLocal(event Event) {
|
||||
b.mu.RUnlock()
|
||||
|
||||
for _, h := range matched {
|
||||
b.deliverCount.Add(1)
|
||||
h(event)
|
||||
}
|
||||
}
|
||||
@@ -106,6 +112,8 @@ func (b *Bus) publishLocal(event Event) {
|
||||
// in separate goroutines. Useful for I/O-heavy handlers.
|
||||
// Also calls the broadcastHook asynchronously if set.
|
||||
func (b *Bus) PublishAsync(event Event) {
|
||||
b.publishCount.Add(1)
|
||||
|
||||
b.mu.RLock()
|
||||
var matched []Handler
|
||||
for pattern, subs := range b.subs {
|
||||
@@ -119,6 +127,7 @@ func (b *Bus) PublishAsync(event Event) {
|
||||
b.mu.RUnlock()
|
||||
|
||||
for _, h := range matched {
|
||||
b.deliverCount.Add(1)
|
||||
go h(event)
|
||||
}
|
||||
if hook != nil {
|
||||
@@ -126,6 +135,12 @@ func (b *Bus) PublishAsync(event Event) {
|
||||
}
|
||||
}
|
||||
|
||||
// PublishCount returns the cumulative number of events published.
|
||||
func (b *Bus) PublishCount() int64 { return b.publishCount.Load() }
|
||||
|
||||
// DeliverCount returns the cumulative number of subscriber deliveries.
|
||||
func (b *Bus) DeliverCount() int64 { return b.deliverCount.Load() }
|
||||
|
||||
// match checks if a concrete label matches a subscription pattern.
|
||||
//
|
||||
// "chat.message.abc" matches "chat.message.abc" (exact)
|
||||
|
||||
@@ -113,10 +113,16 @@ func TestRouteFor(t *testing.T) {
|
||||
{"pong", DirToClient},
|
||||
// Tool bridge routes
|
||||
{"tool.call.abc123", DirToClient},
|
||||
{"tool.result.abc123", DirBoth}, // v0.32.0: DirBoth for cross-pod WaitFor
|
||||
{"tool.result.abc123", DirBoth},
|
||||
// Extension lifecycle
|
||||
{"extension.loaded", DirLocal},
|
||||
{"extension.error", DirLocal},
|
||||
// Realtime
|
||||
{"realtime.chat.message", DirToClient},
|
||||
{"realtime.custom.event", DirToClient},
|
||||
// Room management
|
||||
{"room.subscribe", DirFromClient},
|
||||
{"room.unsubscribe", DirFromClient},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
@@ -190,6 +196,27 @@ func TestWaitFor_CleansUpSubscription(t *testing.T) {
|
||||
bus.Publish(Event{Label: "tool.result.cleanup", Payload: json.RawMessage(`{}`)})
|
||||
}
|
||||
|
||||
func TestRealtimeRouting(t *testing.T) {
|
||||
// realtime.* events should be sent to clients but not accepted from them
|
||||
if !ShouldSendToClient("realtime.chat.message") {
|
||||
t.Error("realtime.* should be sent to client")
|
||||
}
|
||||
if ShouldAcceptFromClient("realtime.chat.message") {
|
||||
t.Error("realtime.* should NOT be accepted from client")
|
||||
}
|
||||
|
||||
// room.subscribe/unsubscribe should be accepted from clients
|
||||
if !ShouldAcceptFromClient("room.subscribe") {
|
||||
t.Error("room.subscribe should be accepted from client")
|
||||
}
|
||||
if !ShouldAcceptFromClient("room.unsubscribe") {
|
||||
t.Error("room.unsubscribe should be accepted from client")
|
||||
}
|
||||
if ShouldSendToClient("room.subscribe") {
|
||||
t.Error("room.subscribe should NOT be sent to client")
|
||||
}
|
||||
}
|
||||
|
||||
func TestToolCallRouteToClient(t *testing.T) {
|
||||
if !ShouldSendToClient("tool.call.abc123") {
|
||||
t.Error("tool.call.* should be sent to client")
|
||||
@@ -200,7 +227,6 @@ func TestToolCallRouteToClient(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestToolResultRouteBoth(t *testing.T) {
|
||||
// v0.32.0: tool.result is DirBoth so results cross pods for WaitFor.
|
||||
// The WS subscriber explicitly filters out tool.result events to
|
||||
// prevent re-sending to clients (see subscribeToBus in ws.go).
|
||||
if !ShouldAcceptFromClient("tool.result.abc123") {
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
package events
|
||||
|
||||
// v0.32.0: TicketValidatorAdapter bridges the context-aware store.TicketStore
|
||||
// to the middleware.TicketValidator interface (which has no context parameter).
|
||||
// Replaces the in-memory TicketStore that lived in this file previously.
|
||||
|
||||
|
||||
@@ -9,7 +9,6 @@ type Event struct {
|
||||
Payload json.RawMessage `json:"payload"`
|
||||
Ts int64 `json:"ts"`
|
||||
|
||||
// v0.32.0: Cross-pod targeted delivery. When set, only connections
|
||||
// belonging to this user receive the event. Serialized for pg_broadcast
|
||||
// (harmless if clients see it — they ignore unknown fields).
|
||||
TargetUserID string `json:"target_user_id,omitempty"`
|
||||
@@ -38,8 +37,8 @@ var routeTable = map[string]Direction{
|
||||
// User/presence
|
||||
"user.presence": DirToClient,
|
||||
"user.status": DirToClient,
|
||||
"user.mentioned": DirToClient, // v0.23.2: targeted @mention notification
|
||||
"typing.user": DirToClient, // v0.23.2: human typing in DM/channel
|
||||
"user.mentioned": DirToClient,
|
||||
"typing.user": DirToClient,
|
||||
|
||||
// System
|
||||
"system.notify": DirToClient,
|
||||
@@ -48,16 +47,16 @@ var routeTable = map[string]Direction{
|
||||
// Model/provider status
|
||||
"model.status": DirToClient,
|
||||
|
||||
// Role alerts (v0.17.0)
|
||||
// Role alerts
|
||||
"role.fallback": DirToClient,
|
||||
|
||||
// Notifications (v0.20.0)
|
||||
// Notifications
|
||||
"notification.new": DirToClient, // targeted via Hub.PublishToUser, not room-based
|
||||
"notification.read": DirToClient, // badge sync across tabs
|
||||
|
||||
// Workspace (v0.21.5)
|
||||
// Workspace
|
||||
|
||||
// Workflow (v0.27.0, v0.3.2, v0.3.3)
|
||||
// Workflow
|
||||
"workflow.started": DirToClient, // instance started
|
||||
"workflow.assigned": DirToClient, // new assignment → team members
|
||||
"workflow.claimed": DirToClient, // assignment claimed → team + claimer
|
||||
@@ -65,10 +64,10 @@ var routeTable = map[string]Direction{
|
||||
"workflow.completed": DirToClient, // workflow finished → instance participants
|
||||
"workflow.cancelled": DirToClient, // instance cancelled
|
||||
"workflow.error": DirToClient, // engine/hook error
|
||||
"workflow.sla_breach": DirToClient, // SLA exceeded → team + admins (v0.3.3)
|
||||
"workflow.stale": DirToClient, // instance marked stale (v0.3.3)
|
||||
"workflow.signoff": DirToClient, // signoff submitted (v0.3.4)
|
||||
"workflow.rejected": DirToClient, // rejection triggered cancel/reroute (v0.3.4)
|
||||
"workflow.sla_breach": DirToClient, // SLA exceeded → team + admins
|
||||
"workflow.stale": DirToClient, // instance marked stale
|
||||
"workflow.signoff": DirToClient, // signoff submitted
|
||||
"workflow.rejected": DirToClient, // rejection triggered cancel/reroute
|
||||
|
||||
// Plugin hooks — never cross the wire
|
||||
"plugin.hook.": DirLocal,
|
||||
@@ -77,16 +76,23 @@ var routeTable = map[string]Direction{
|
||||
|
||||
// Tool execution (browser tools bridge)
|
||||
"tool.call.": DirToClient, // Server → specific client (browser tool invocation)
|
||||
"tool.result.": DirBoth, // v0.32.0: DirBoth — result must cross pods for WaitFor
|
||||
"tool.result.": DirBoth,
|
||||
|
||||
// Extension lifecycle
|
||||
"extension.loaded": DirLocal, // Client-only
|
||||
"extension.error": DirLocal,
|
||||
|
||||
// Trigger system (v0.2.2)
|
||||
// Trigger system
|
||||
"trigger.fired": DirLocal, // Trigger invocation event (observability)
|
||||
"trigger.error": DirLocal, // Trigger execution error
|
||||
|
||||
// Realtime: extension-published events, room-scoped
|
||||
"realtime.": DirToClient,
|
||||
|
||||
// Room management: client room join/leave
|
||||
"room.subscribe": DirFromClient,
|
||||
"room.unsubscribe": DirFromClient,
|
||||
|
||||
// Heartbeat
|
||||
"ping": DirFromClient,
|
||||
"pong": DirToClient,
|
||||
|
||||
@@ -174,7 +174,6 @@ func (h *Hub) IsConnected(userID string) bool {
|
||||
}
|
||||
|
||||
// PublishToUser sends an event to a specific user via the bus.
|
||||
// v0.32.0: Cross-pod safe — the bus broadcast hook fans out via pg_notify.
|
||||
// All replicas receive the event; only the one with the user's connection delivers it.
|
||||
func (h *Hub) PublishToUser(userID string, event Event) {
|
||||
event.TargetUserID = userID
|
||||
@@ -238,13 +237,11 @@ func (c *Conn) subscribeToBus() {
|
||||
return
|
||||
}
|
||||
|
||||
// v0.32.0: Targeted delivery — only deliver to the intended user.
|
||||
// Targeted events skip room filtering (user-scoped, not room-scoped).
|
||||
if e.TargetUserID != "" && e.TargetUserID != c.userID {
|
||||
return
|
||||
}
|
||||
|
||||
// v0.32.0: tool.result travels cross-pod for WaitFor (DirBoth) but
|
||||
// must not be forwarded to WebSocket clients — they sent it.
|
||||
if strings.HasPrefix(e.Label, "tool.result.") {
|
||||
return
|
||||
@@ -316,6 +313,25 @@ func (c *Conn) readPump() {
|
||||
continue
|
||||
}
|
||||
|
||||
if event.Label == "room.subscribe" {
|
||||
var req struct {
|
||||
Room string `json:"room"`
|
||||
}
|
||||
if json.Unmarshal(event.Payload, &req) == nil && req.Room != "" && len(c.rooms) < 100 {
|
||||
c.JoinRoom(req.Room)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if event.Label == "room.unsubscribe" {
|
||||
var req struct {
|
||||
Room string `json:"room"`
|
||||
}
|
||||
if json.Unmarshal(event.Payload, &req) == nil && req.Room != "" {
|
||||
c.LeaveRoom(req.Room)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// Validate: only accept events allowed from clients
|
||||
if !ShouldAcceptFromClient(event.Label) {
|
||||
continue
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
package handlers
|
||||
|
||||
// v0.38.1: Extension connection handlers — admin/global scope.
|
||||
// Methods on AdminHandler, mirrors admin provider config pattern.
|
||||
|
||||
import (
|
||||
|
||||
24
server/handlers/admin_metrics.go
Normal file
24
server/handlers/admin_metrics.go
Normal file
@@ -0,0 +1,24 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/metrics"
|
||||
)
|
||||
|
||||
// MetricsHandler serves the admin metrics endpoint.
|
||||
type MetricsHandler struct {
|
||||
collector *metrics.Collector
|
||||
}
|
||||
|
||||
// NewMetricsHandler creates a MetricsHandler.
|
||||
func NewMetricsHandler(c *metrics.Collector) *MetricsHandler {
|
||||
return &MetricsHandler{collector: c}
|
||||
}
|
||||
|
||||
// GetMetrics returns a full metrics snapshot.
|
||||
// GET /api/v1/admin/metrics
|
||||
func (h *MetricsHandler) GetMetrics(c *gin.Context) {
|
||||
snap := h.collector.Collect(c.Request.Context())
|
||||
c.JSON(200, snap)
|
||||
}
|
||||
187
server/handlers/admin_metrics_test.go
Normal file
187
server/handlers/admin_metrics_test.go
Normal file
@@ -0,0 +1,187 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/metrics"
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// mockHub implements metrics.ConnCounter for tests.
|
||||
type mockHub struct{ count int }
|
||||
|
||||
func (m *mockHub) ConnCount() int { return m.count }
|
||||
|
||||
// mockBus implements metrics.BusCounter for tests.
|
||||
type mockBus struct{ pub, del int64 }
|
||||
|
||||
func (m *mockBus) PublishCount() int64 { return m.pub }
|
||||
func (m *mockBus) DeliverCount() int64 { return m.del }
|
||||
|
||||
func TestMetrics_SQLiteShape(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
collector := metrics.NewCollector(
|
||||
"test-node",
|
||||
nil, // no DB
|
||||
&mockHub{count: 5},
|
||||
&mockBus{pub: 10, del: 20},
|
||||
store.Stores{}, // no cluster store
|
||||
func() (uint64, uint64, float64) { return 42, 3, 12.5 },
|
||||
func() int64 { return 7 },
|
||||
time.Now().Add(-10*time.Minute),
|
||||
)
|
||||
|
||||
h := NewMetricsHandler(collector)
|
||||
r := gin.New()
|
||||
r.GET("/api/v1/admin/metrics", h.GetMetrics)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/metrics", nil)
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != 200 {
|
||||
t.Fatalf("status = %d, want 200", w.Code)
|
||||
}
|
||||
|
||||
var snap metrics.Snapshot
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &snap); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
|
||||
// Node ID
|
||||
if snap.NodeID != "test-node" {
|
||||
t.Errorf("node_id = %q, want %q", snap.NodeID, "test-node")
|
||||
}
|
||||
|
||||
// Runtime
|
||||
if snap.Runtime.WSClients != 5 {
|
||||
t.Errorf("ws_clients = %d, want 5", snap.Runtime.WSClients)
|
||||
}
|
||||
if snap.Runtime.UptimeSec < 600 {
|
||||
t.Errorf("uptime_sec = %f, want >= 600", snap.Runtime.UptimeSec)
|
||||
}
|
||||
|
||||
// Cluster should be nil (SQLite)
|
||||
if snap.Cluster != nil {
|
||||
t.Errorf("cluster should be nil for SQLite, got %+v", snap.Cluster)
|
||||
}
|
||||
|
||||
// Extensions
|
||||
if snap.Extensions.StarlarkExecTotal != 42 {
|
||||
t.Errorf("starlark_exec_total = %d, want 42", snap.Extensions.StarlarkExecTotal)
|
||||
}
|
||||
if snap.Extensions.StarlarkErrorsTotal != 3 {
|
||||
t.Errorf("starlark_errors_total = %d, want 3", snap.Extensions.StarlarkErrorsTotal)
|
||||
}
|
||||
if snap.Extensions.StarlarkAvgDuration != 12.5 {
|
||||
t.Errorf("starlark_avg_duration_ms = %f, want 12.5", snap.Extensions.StarlarkAvgDuration)
|
||||
}
|
||||
if snap.Extensions.TriggerFiresTotal != 7 {
|
||||
t.Errorf("trigger_fires_total = %d, want 7", snap.Extensions.TriggerFiresTotal)
|
||||
}
|
||||
if snap.Extensions.EventBusPublished != 10 {
|
||||
t.Errorf("event_bus_published = %d, want 10", snap.Extensions.EventBusPublished)
|
||||
}
|
||||
if snap.Extensions.EventBusDelivered != 20 {
|
||||
t.Errorf("event_bus_delivered = %d, want 20", snap.Extensions.EventBusDelivered)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMetrics_WithCluster(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
mock := &mockClusterStore{
|
||||
nodes: []store.ClusterNode{
|
||||
{
|
||||
NodeID: "node-1",
|
||||
Endpoint: "http://node-1:8080",
|
||||
Heartbeat: time.Now(),
|
||||
Stats: json.RawMessage(`{"uptime_sec":120,"ws_clients":3}`),
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
collector := metrics.NewCollector(
|
||||
"test-node",
|
||||
nil,
|
||||
&mockHub{count: 3},
|
||||
&mockBus{},
|
||||
store.Stores{Cluster: mock},
|
||||
func() (uint64, uint64, float64) { return 0, 0, 0 },
|
||||
func() int64 { return 0 },
|
||||
time.Now(),
|
||||
)
|
||||
|
||||
h := NewMetricsHandler(collector)
|
||||
r := gin.New()
|
||||
r.GET("/api/v1/admin/metrics", h.GetMetrics)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/metrics", nil)
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != 200 {
|
||||
t.Fatalf("status = %d, want 200", w.Code)
|
||||
}
|
||||
|
||||
var snap metrics.Snapshot
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &snap); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
|
||||
if snap.Cluster == nil {
|
||||
t.Fatal("cluster should not be nil with cluster store")
|
||||
}
|
||||
if snap.Cluster.Size != 1 {
|
||||
t.Errorf("cluster.size = %d, want 1", snap.Cluster.Size)
|
||||
}
|
||||
if snap.Cluster.Nodes[0].NodeID != "node-1" {
|
||||
t.Errorf("node_id = %q, want %q", snap.Cluster.Nodes[0].NodeID, "node-1")
|
||||
}
|
||||
if snap.Cluster.Nodes[0].UptimeSec != 120 {
|
||||
t.Errorf("uptime_sec = %f, want 120", snap.Cluster.Nodes[0].UptimeSec)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMetrics_ExtensionCounters(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
collector := metrics.NewCollector(
|
||||
"test-node",
|
||||
nil,
|
||||
&mockHub{},
|
||||
&mockBus{pub: 100, del: 500},
|
||||
store.Stores{},
|
||||
func() (uint64, uint64, float64) { return 1000, 50, 8.3 },
|
||||
func() int64 { return 25 },
|
||||
time.Now(),
|
||||
)
|
||||
|
||||
h := NewMetricsHandler(collector)
|
||||
r := gin.New()
|
||||
r.GET("/api/v1/admin/metrics", h.GetMetrics)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/metrics", nil)
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, req)
|
||||
|
||||
var snap metrics.Snapshot
|
||||
_ = json.Unmarshal(w.Body.Bytes(), &snap)
|
||||
|
||||
if snap.Extensions.StarlarkExecTotal != 1000 {
|
||||
t.Errorf("exec total = %d, want 1000", snap.Extensions.StarlarkExecTotal)
|
||||
}
|
||||
if snap.Extensions.EventBusPublished != 100 {
|
||||
t.Errorf("bus published = %d, want 100", snap.Extensions.EventBusPublished)
|
||||
}
|
||||
if snap.Extensions.EventBusDelivered != 500 {
|
||||
t.Errorf("bus delivered = %d, want 500", snap.Extensions.EventBusDelivered)
|
||||
}
|
||||
}
|
||||
@@ -218,8 +218,8 @@ func (h *AuthHandler) OIDCCallback(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// Verify state (nonce + redirectTo retrieved but not yet validated — TODO)
|
||||
_, _, err := h.stores.GlobalConfig.ConsumeOIDCState(c.Request.Context(), state)
|
||||
// Verify state and retrieve nonce for ID token validation
|
||||
storedNonce, _, err := h.stores.GlobalConfig.ConsumeOIDCState(c.Request.Context(), state)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid or expired state"})
|
||||
return
|
||||
@@ -246,6 +246,15 @@ func (h *AuthHandler) OIDCCallback(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// Validate nonce in the ID token to prevent token replay/substitution
|
||||
if tokenResp.IDToken != "" && storedNonce != "" {
|
||||
if err := oidcProv.ValidateIDTokenNonce(tokenResp.IDToken, storedNonce); err != nil {
|
||||
log.Printf("[auth/oidc] nonce validation failed: %v", err)
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "nonce validation failed"})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Use the ID token (or access token) to authenticate via the provider
|
||||
// Temporarily set the Authorization header so Authenticate() can read it
|
||||
tokenToValidate := tokenResp.IDToken
|
||||
@@ -345,7 +354,6 @@ func hashToken(token string) string {
|
||||
// - AdminHandler.destroyVault (admin-initiated reset)
|
||||
//
|
||||
// Does NOT evict from UEK cache or write audit logs — callers handle that.
|
||||
// v0.29.0: accepts stores instead of using database.DB directly.
|
||||
func DestroyVaultDB(ctx context.Context, stores store.Stores, userID string) (deleted int64) {
|
||||
if err := stores.Users.ClearVaultKeys(ctx, userID); err != nil {
|
||||
log.Printf("⚠ DestroyVaultDB: failed to clear vault columns for user %s: %v", userID, err)
|
||||
@@ -366,8 +374,6 @@ func DestroyVaultDB(ctx context.Context, stores store.Stores, userID string) (de
|
||||
// sessions survive server restarts.
|
||||
//
|
||||
// Used by BootstrapAdmin and SeedUsers where the password is known at startup.
|
||||
// v0.29.0: accepts stores instead of using database.DB directly.
|
||||
// v0.30.2: accepts optional uekCache to pre-warm vault on restart.
|
||||
func ProbeAndRepairVault(ctx context.Context, stores store.Stores, userID, password string, uekCache ...*crypto.UEKCache) {
|
||||
vaultSet, encryptedUEK, salt, nonce, err := stores.Users.GetVaultKeys(ctx, userID)
|
||||
if err != nil || !vaultSet {
|
||||
@@ -493,7 +499,7 @@ func BootstrapAdmin(cfg *config.Config, s store.Stores, uekCache ...*crypto.UEKC
|
||||
})
|
||||
// If the actual password changed, the vault seal is stale. Probe it
|
||||
// with the current password and destroy only if unwrap fails.
|
||||
// Ensure handle exists (backfill for pre-v0.24.0 users)
|
||||
// Ensure handle exists (backfill for users)
|
||||
if existing.Handle == "" {
|
||||
handle := auth.UniqueHandle(ctx, s.Users, models.HandleFromName(cfg.AdminUsername))
|
||||
s.Users.Update(ctx, existing.ID, map[string]interface{}{"handle": handle})
|
||||
|
||||
523
server/handlers/backup.go
Normal file
523
server/handlers/backup.go
Normal file
@@ -0,0 +1,523 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/database"
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// BackupHandler provides backup/restore endpoints for admin users.
|
||||
type BackupHandler struct {
|
||||
stores store.Stores
|
||||
packagesDir string
|
||||
storagePath string
|
||||
}
|
||||
|
||||
// NewBackupHandler creates a new BackupHandler.
|
||||
func NewBackupHandler(stores store.Stores, packagesDir, storagePath string) *BackupHandler {
|
||||
return &BackupHandler{
|
||||
stores: stores,
|
||||
packagesDir: packagesDir,
|
||||
storagePath: storagePath,
|
||||
}
|
||||
}
|
||||
|
||||
// backupMeta is written as meta.json inside the .swb archive.
|
||||
type backupMeta struct {
|
||||
Version string `json:"version"`
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
SchemaVersion string `json:"schema_version"`
|
||||
Dialect string `json:"dialect"`
|
||||
CoreTables []string `json:"core_tables"`
|
||||
ExtTables []string `json:"ext_tables,omitempty"`
|
||||
}
|
||||
|
||||
func (h *BackupHandler) backupsDir() string {
|
||||
if h.storagePath == "" {
|
||||
return ""
|
||||
}
|
||||
return filepath.Join(h.storagePath, "backups")
|
||||
}
|
||||
|
||||
// ── CreateBackup ─────────────────────────────
|
||||
// POST /api/v1/admin/backup
|
||||
// Streams a .swb ZIP archive containing all core + ext_data tables as JSONL,
|
||||
// plus package asset files from disk.
|
||||
|
||||
func (h *BackupHandler) CreateBackup(c *gin.Context) {
|
||||
ctx := c.Request.Context()
|
||||
db := database.DB
|
||||
|
||||
// Determine schema version
|
||||
schemaVersion := ""
|
||||
row := db.QueryRowContext(ctx, "SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 1")
|
||||
row.Scan(&schemaVersion)
|
||||
|
||||
dialect := "sqlite"
|
||||
if database.IsPostgres() {
|
||||
dialect = "postgres"
|
||||
}
|
||||
|
||||
// Enumerate ext_data tables
|
||||
extEntries, err := listExtDataTables(ctx, db)
|
||||
if err != nil {
|
||||
log.Printf("[backup] warning: list ext_data tables: %v", err)
|
||||
}
|
||||
|
||||
extTableNames := make([]string, len(extEntries))
|
||||
for i, e := range extEntries {
|
||||
extTableNames[i] = e.PhysicalName
|
||||
}
|
||||
|
||||
meta := backupMeta{
|
||||
Version: readVersionFile(),
|
||||
Timestamp: time.Now().UTC(),
|
||||
SchemaVersion: schemaVersion,
|
||||
Dialect: dialect,
|
||||
CoreTables: coreTableOrder,
|
||||
ExtTables: extTableNames,
|
||||
}
|
||||
|
||||
// Determine if we should store server-side or stream directly
|
||||
storeSide := c.Query("store") == "true"
|
||||
|
||||
if storeSide {
|
||||
h.createStoredBackup(c, ctx, db, meta, extEntries)
|
||||
return
|
||||
}
|
||||
|
||||
// Stream directly to client
|
||||
filename := fmt.Sprintf("switchboard-%s-%s.swb", meta.Version, meta.Timestamp.Format("20060102-150405"))
|
||||
c.Header("Content-Type", "application/zip")
|
||||
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=%q", filename))
|
||||
|
||||
zw := zip.NewWriter(c.Writer)
|
||||
defer zw.Close()
|
||||
|
||||
h.writeBackupContents(c, zw, meta, extEntries)
|
||||
}
|
||||
|
||||
func (h *BackupHandler) createStoredBackup(c *gin.Context, ctx interface{ Deadline() (time.Time, bool) }, db interface{}, meta backupMeta, extEntries []extDataTableEntry) {
|
||||
dir := h.backupsDir()
|
||||
if dir == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "no storage path configured"})
|
||||
return
|
||||
}
|
||||
os.MkdirAll(dir, 0755)
|
||||
|
||||
filename := fmt.Sprintf("switchboard-%s-%s.swb", meta.Version, meta.Timestamp.Format("20060102-150405"))
|
||||
path := filepath.Join(dir, filename)
|
||||
|
||||
f, err := os.Create(path)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "create backup file: " + err.Error()})
|
||||
return
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
zw := zip.NewWriter(f)
|
||||
defer zw.Close()
|
||||
|
||||
h.writeBackupContents(c, zw, meta, extEntries)
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{"data": gin.H{
|
||||
"filename": filename,
|
||||
"path": path,
|
||||
}})
|
||||
}
|
||||
|
||||
func (h *BackupHandler) writeBackupContents(c *gin.Context, zw *zip.Writer, meta backupMeta, extEntries []extDataTableEntry) {
|
||||
ctx := c.Request.Context()
|
||||
db := database.DB
|
||||
|
||||
// Write meta.json
|
||||
mw, err := zw.Create("meta.json")
|
||||
if err != nil {
|
||||
log.Printf("[backup] create meta.json: %v", err)
|
||||
return
|
||||
}
|
||||
enc := json.NewEncoder(mw)
|
||||
enc.SetIndent("", " ")
|
||||
enc.Encode(meta)
|
||||
|
||||
// Dump core tables
|
||||
for _, table := range coreTableOrder {
|
||||
if !tableExists(ctx, db, table) {
|
||||
continue
|
||||
}
|
||||
fw, err := zw.Create(fmt.Sprintf("core/%s.jsonl", table))
|
||||
if err != nil {
|
||||
log.Printf("[backup] create core/%s.jsonl: %v", table, err)
|
||||
continue
|
||||
}
|
||||
count, err := dumpTable(ctx, db, table, fw)
|
||||
if err != nil {
|
||||
log.Printf("[backup] dump %s: %v", table, err)
|
||||
} else {
|
||||
log.Printf("[backup] dumped %s: %d rows", table, count)
|
||||
}
|
||||
}
|
||||
|
||||
// Dump ext_data tables
|
||||
for _, entry := range extEntries {
|
||||
if !tableExists(ctx, db, entry.PhysicalName) {
|
||||
continue
|
||||
}
|
||||
path := fmt.Sprintf("ext_data/%s/%s.jsonl", entry.PackageID, entry.TableName)
|
||||
fw, err := zw.Create(path)
|
||||
if err != nil {
|
||||
log.Printf("[backup] create %s: %v", path, err)
|
||||
continue
|
||||
}
|
||||
count, err := dumpTable(ctx, db, entry.PhysicalName, fw)
|
||||
if err != nil {
|
||||
log.Printf("[backup] dump %s: %v", entry.PhysicalName, err)
|
||||
} else {
|
||||
log.Printf("[backup] dumped %s: %d rows", entry.PhysicalName, count)
|
||||
}
|
||||
}
|
||||
|
||||
// Walk package assets
|
||||
if h.packagesDir != "" {
|
||||
filepath.Walk(h.packagesDir, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil || info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
relPath, err := filepath.Rel(h.packagesDir, path)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
relPath = filepath.ToSlash(relPath)
|
||||
fw, err := zw.Create("packages/" + relPath)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
defer f.Close()
|
||||
io.Copy(fw, f)
|
||||
return nil
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// ── ListBackups ──────────────────────────────
|
||||
// GET /api/v1/admin/backups
|
||||
|
||||
func (h *BackupHandler) ListBackups(c *gin.Context) {
|
||||
dir := h.backupsDir()
|
||||
if dir == "" {
|
||||
c.JSON(http.StatusOK, gin.H{"data": []interface{}{}})
|
||||
return
|
||||
}
|
||||
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusOK, gin.H{"data": []interface{}{}})
|
||||
return
|
||||
}
|
||||
|
||||
type backupEntry struct {
|
||||
Name string `json:"name"`
|
||||
Size int64 `json:"size"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
var backups []backupEntry
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), ".swb") {
|
||||
continue
|
||||
}
|
||||
info, err := e.Info()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
backups = append(backups, backupEntry{
|
||||
Name: e.Name(),
|
||||
Size: info.Size(),
|
||||
CreatedAt: info.ModTime(),
|
||||
})
|
||||
}
|
||||
if backups == nil {
|
||||
backups = []backupEntry{}
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{"data": backups})
|
||||
}
|
||||
|
||||
// ── DownloadBackup ───────────────────────────
|
||||
// GET /api/v1/admin/backups/:name
|
||||
|
||||
func (h *BackupHandler) DownloadBackup(c *gin.Context) {
|
||||
name := c.Param("name")
|
||||
if !isValidBackupName(name) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid backup name"})
|
||||
return
|
||||
}
|
||||
|
||||
dir := h.backupsDir()
|
||||
if dir == "" {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "no storage configured"})
|
||||
return
|
||||
}
|
||||
|
||||
path := filepath.Join(dir, name)
|
||||
if _, err := os.Stat(path); os.IsNotExist(err) {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "backup not found"})
|
||||
return
|
||||
}
|
||||
|
||||
c.Header("Content-Type", "application/zip")
|
||||
c.Header("Content-Disposition", fmt.Sprintf("attachment; filename=%q", name))
|
||||
c.File(path)
|
||||
}
|
||||
|
||||
// ── DeleteBackup ─────────────────────────────
|
||||
// DELETE /api/v1/admin/backups/:name
|
||||
|
||||
func (h *BackupHandler) DeleteBackup(c *gin.Context) {
|
||||
name := c.Param("name")
|
||||
if !isValidBackupName(name) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid backup name"})
|
||||
return
|
||||
}
|
||||
|
||||
dir := h.backupsDir()
|
||||
if dir == "" {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "no storage configured"})
|
||||
return
|
||||
}
|
||||
|
||||
path := filepath.Join(dir, name)
|
||||
if err := os.Remove(path); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "backup not found"})
|
||||
} else {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{"data": gin.H{"deleted": name}})
|
||||
}
|
||||
|
||||
// ── RestoreBackup ────────────────────────────
|
||||
// POST /api/v1/admin/restore
|
||||
// Accepts a .swb file upload, validates meta, wipes DB, restores data.
|
||||
|
||||
func (h *BackupHandler) RestoreBackup(c *gin.Context) {
|
||||
ctx := c.Request.Context()
|
||||
db := database.DB
|
||||
|
||||
file, header, err := c.Request.FormFile("file")
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "file required"})
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
// Read entire file into memory for zip.NewReader
|
||||
data, err := io.ReadAll(file)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "read file: " + err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid archive: " + err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
// Read meta.json
|
||||
meta, err := readBackupMeta(zr)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid backup: " + err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
// Validate schema version — reject backups from future schema
|
||||
currentSchema := ""
|
||||
db.QueryRowContext(ctx, "SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 1").Scan(¤tSchema)
|
||||
if meta.SchemaVersion > currentSchema && currentSchema != "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{
|
||||
"error": fmt.Sprintf("backup schema %s is newer than current %s — upgrade the application first", meta.SchemaVersion, currentSchema),
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("[backup] restoring from %s (schema %s, %d core tables, %d ext tables)",
|
||||
header.Filename, meta.SchemaVersion, len(meta.CoreTables), len(meta.ExtTables))
|
||||
|
||||
// Build full table list for wipe (core + ext_data)
|
||||
allTables := make([]string, 0, len(coreTableOrder)+len(meta.ExtTables))
|
||||
allTables = append(allTables, coreTableOrder...)
|
||||
allTables = append(allTables, meta.ExtTables...)
|
||||
|
||||
// Wipe all data
|
||||
if err := wipeTables(ctx, db, allTables); err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "wipe failed: " + err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
// Restore core tables in order
|
||||
restored := 0
|
||||
for _, table := range coreTableOrder {
|
||||
path := fmt.Sprintf("core/%s.jsonl", table)
|
||||
f := findInZip(zr, path)
|
||||
if f == nil {
|
||||
continue
|
||||
}
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
log.Printf("[backup] open %s: %v", path, err)
|
||||
continue
|
||||
}
|
||||
count, err := restoreTable(ctx, db, table, rc)
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
log.Printf("[backup] restore %s: %v", table, err)
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": fmt.Sprintf("restore %s: %v", table, err)})
|
||||
return
|
||||
}
|
||||
log.Printf("[backup] restored %s: %d rows", table, count)
|
||||
restored += count
|
||||
}
|
||||
|
||||
// Restore ext_data tables
|
||||
for _, extTable := range meta.ExtTables {
|
||||
// Parse package_id and table_name from physical name: ext_{pkg}_{table}
|
||||
pkgID, tblName := parseExtTableName(extTable)
|
||||
if pkgID == "" {
|
||||
continue
|
||||
}
|
||||
path := fmt.Sprintf("ext_data/%s/%s.jsonl", pkgID, tblName)
|
||||
f := findInZip(zr, path)
|
||||
if f == nil {
|
||||
continue
|
||||
}
|
||||
// Ensure physical table exists before restoring
|
||||
if !tableExists(ctx, db, extTable) {
|
||||
log.Printf("[backup] skip ext table %s (not in schema)", extTable)
|
||||
continue
|
||||
}
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
log.Printf("[backup] open %s: %v", path, err)
|
||||
continue
|
||||
}
|
||||
count, err := restoreTable(ctx, db, extTable, rc)
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
log.Printf("[backup] restore ext %s: %v", extTable, err)
|
||||
} else {
|
||||
log.Printf("[backup] restored ext %s: %d rows", extTable, count)
|
||||
restored += count
|
||||
}
|
||||
}
|
||||
|
||||
// Restore package files
|
||||
if h.packagesDir != "" {
|
||||
for _, f := range zr.File {
|
||||
if !strings.HasPrefix(f.Name, "packages/") || f.FileInfo().IsDir() {
|
||||
continue
|
||||
}
|
||||
relPath := strings.TrimPrefix(f.Name, "packages/")
|
||||
destPath := filepath.Join(h.packagesDir, filepath.FromSlash(relPath))
|
||||
os.MkdirAll(filepath.Dir(destPath), 0755)
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
out, err := os.Create(destPath)
|
||||
if err != nil {
|
||||
rc.Close()
|
||||
continue
|
||||
}
|
||||
io.Copy(out, rc)
|
||||
out.Close()
|
||||
rc.Close()
|
||||
}
|
||||
}
|
||||
|
||||
log.Printf("[backup] restore complete: %d total rows", restored)
|
||||
c.JSON(http.StatusOK, gin.H{"data": gin.H{
|
||||
"rows_restored": restored,
|
||||
"schema": meta.SchemaVersion,
|
||||
}})
|
||||
}
|
||||
|
||||
// ── Helpers ──────────────────────────────────
|
||||
|
||||
func readBackupMeta(zr *zip.Reader) (*backupMeta, error) {
|
||||
f := findInZip(zr, "meta.json")
|
||||
if f == nil {
|
||||
return nil, fmt.Errorf("meta.json not found in archive")
|
||||
}
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rc.Close()
|
||||
var meta backupMeta
|
||||
if err := json.NewDecoder(rc).Decode(&meta); err != nil {
|
||||
return nil, fmt.Errorf("decode meta.json: %w", err)
|
||||
}
|
||||
return &meta, nil
|
||||
}
|
||||
|
||||
func findInZip(zr *zip.Reader, name string) *zip.File {
|
||||
for _, f := range zr.File {
|
||||
if f.Name == name {
|
||||
return f
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// parseExtTableName parses "ext_{pkg}_{table}" into (pkg, table).
|
||||
func parseExtTableName(physical string) (string, string) {
|
||||
if !strings.HasPrefix(physical, "ext_") {
|
||||
return "", ""
|
||||
}
|
||||
rest := physical[4:]
|
||||
idx := strings.Index(rest, "_")
|
||||
if idx < 0 {
|
||||
return "", ""
|
||||
}
|
||||
return rest[:idx], rest[idx+1:]
|
||||
}
|
||||
|
||||
var validBackupNameRe = regexp.MustCompile(`^[a-zA-Z0-9._-]+\.swb$`)
|
||||
|
||||
func isValidBackupName(name string) bool {
|
||||
return validBackupNameRe.MatchString(name) && !strings.Contains(name, "..")
|
||||
}
|
||||
|
||||
// readVersionFile reads the VERSION file from the project root.
|
||||
func readVersionFile() string {
|
||||
// Try common locations
|
||||
for _, path := range []string{"VERSION", "/app/VERSION", "../VERSION"} {
|
||||
data, err := os.ReadFile(path)
|
||||
if err == nil {
|
||||
return strings.TrimSpace(string(data))
|
||||
}
|
||||
}
|
||||
return "unknown"
|
||||
}
|
||||
231
server/handlers/backup_tables.go
Normal file
231
server/handlers/backup_tables.go
Normal file
@@ -0,0 +1,231 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"strings"
|
||||
|
||||
"switchboard-core/database"
|
||||
)
|
||||
|
||||
// coreTableOrder lists tables for backup in FK-safe import order.
|
||||
// Ephemeral tables (node_registry, ws_tickets, rate_limit_counters,
|
||||
// user_presence, oidc_auth_state, refresh_tokens) are excluded.
|
||||
var coreTableOrder = []string{
|
||||
"schema_migrations",
|
||||
"users",
|
||||
"teams",
|
||||
"team_members",
|
||||
"groups",
|
||||
"group_members",
|
||||
"platform_policies",
|
||||
"global_settings",
|
||||
"packages",
|
||||
"package_user_settings",
|
||||
"package_team_settings",
|
||||
"extension_permissions",
|
||||
"ext_data_tables",
|
||||
"ext_connections",
|
||||
"ext_dependencies",
|
||||
"resource_grants",
|
||||
"notifications",
|
||||
"notification_preferences",
|
||||
"audit_log",
|
||||
"workflows",
|
||||
"workflow_stages",
|
||||
"workflow_versions",
|
||||
"workflow_instances",
|
||||
"workflow_assignments",
|
||||
"workflow_signoffs",
|
||||
"triggers",
|
||||
"scheduled_tasks",
|
||||
"trigger_logs",
|
||||
}
|
||||
|
||||
// dumpTable writes all rows of a table as newline-delimited JSON to w.
|
||||
// Columns are discovered dynamically via rows.ColumnTypes().
|
||||
func dumpTable(ctx context.Context, db *sql.DB, table string, w io.Writer) (int, error) {
|
||||
rows, err := db.QueryContext(ctx, fmt.Sprintf("SELECT * FROM %s", table))
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("query %s: %w", table, err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
cols, err := rows.Columns()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("columns %s: %w", table, err)
|
||||
}
|
||||
|
||||
count := 0
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetEscapeHTML(false)
|
||||
|
||||
for rows.Next() {
|
||||
// Create scan destinations — all as interface{}
|
||||
vals := make([]interface{}, len(cols))
|
||||
ptrs := make([]interface{}, len(cols))
|
||||
for i := range vals {
|
||||
ptrs[i] = &vals[i]
|
||||
}
|
||||
if err := rows.Scan(ptrs...); err != nil {
|
||||
return count, fmt.Errorf("scan %s row %d: %w", table, count, err)
|
||||
}
|
||||
|
||||
row := make(map[string]interface{}, len(cols))
|
||||
for i, col := range cols {
|
||||
row[col] = normalizeValue(vals[i])
|
||||
}
|
||||
if err := enc.Encode(row); err != nil {
|
||||
return count, fmt.Errorf("encode %s row %d: %w", table, count, err)
|
||||
}
|
||||
count++
|
||||
}
|
||||
return count, rows.Err()
|
||||
}
|
||||
|
||||
// normalizeValue converts database-specific types to JSON-friendly values.
|
||||
func normalizeValue(v interface{}) interface{} {
|
||||
if v == nil {
|
||||
return nil
|
||||
}
|
||||
switch val := v.(type) {
|
||||
case []byte:
|
||||
// Try to parse as JSON first (for JSONB columns)
|
||||
var j interface{}
|
||||
if err := json.Unmarshal(val, &j); err == nil {
|
||||
return j
|
||||
}
|
||||
return string(val)
|
||||
default:
|
||||
return val
|
||||
}
|
||||
}
|
||||
|
||||
// restoreTable reads JSONL from r and inserts rows into the table.
|
||||
// Column names come from the first JSON object. Uses dialect-adapted SQL.
|
||||
func restoreTable(ctx context.Context, db *sql.DB, table string, r io.Reader) (int, error) {
|
||||
scanner := bufio.NewScanner(r)
|
||||
// Allow large lines (e.g. JSONB audit_log entries)
|
||||
scanner.Buffer(make([]byte, 0, 64*1024), 10*1024*1024)
|
||||
|
||||
count := 0
|
||||
for scanner.Scan() {
|
||||
line := scanner.Bytes()
|
||||
if len(line) == 0 {
|
||||
continue
|
||||
}
|
||||
var row map[string]interface{}
|
||||
if err := json.Unmarshal(line, &row); err != nil {
|
||||
return count, fmt.Errorf("unmarshal %s line %d: %w", table, count+1, err)
|
||||
}
|
||||
|
||||
cols := make([]string, 0, len(row))
|
||||
vals := make([]interface{}, 0, len(row))
|
||||
for k, v := range row {
|
||||
cols = append(cols, k)
|
||||
vals = append(vals, marshalIfComplex(v))
|
||||
}
|
||||
|
||||
// Build INSERT with positional placeholders
|
||||
placeholders := make([]string, len(cols))
|
||||
for i := range placeholders {
|
||||
placeholders[i] = fmt.Sprintf("$%d", i+1)
|
||||
}
|
||||
query := fmt.Sprintf("INSERT INTO %s (%s) VALUES (%s)",
|
||||
table,
|
||||
strings.Join(cols, ", "),
|
||||
strings.Join(placeholders, ", "),
|
||||
)
|
||||
|
||||
if _, err := database.ExecContext(ctx, query, vals...); err != nil {
|
||||
return count, fmt.Errorf("insert %s row %d: %w", table, count+1, err)
|
||||
}
|
||||
count++
|
||||
}
|
||||
return count, scanner.Err()
|
||||
}
|
||||
|
||||
// marshalIfComplex converts maps/slices back to JSON strings for DB storage.
|
||||
func marshalIfComplex(v interface{}) interface{} {
|
||||
if v == nil {
|
||||
return nil
|
||||
}
|
||||
switch v.(type) {
|
||||
case map[string]interface{}, []interface{}:
|
||||
b, _ := json.Marshal(v)
|
||||
return string(b)
|
||||
default:
|
||||
return v
|
||||
}
|
||||
}
|
||||
|
||||
// listExtDataTables returns all (package_id, physical_table_name) pairs
|
||||
// from the ext_data_tables catalog.
|
||||
func listExtDataTables(ctx context.Context, db *sql.DB) ([]extDataTableEntry, error) {
|
||||
rows, err := db.QueryContext(ctx, "SELECT package_id, table_name FROM ext_data_tables ORDER BY package_id, table_name")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var entries []extDataTableEntry
|
||||
for rows.Next() {
|
||||
var e extDataTableEntry
|
||||
if err := rows.Scan(&e.PackageID, &e.TableName); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
e.PhysicalName = fmt.Sprintf("ext_%s_%s", e.PackageID, e.TableName)
|
||||
entries = append(entries, e)
|
||||
}
|
||||
return entries, rows.Err()
|
||||
}
|
||||
|
||||
type extDataTableEntry struct {
|
||||
PackageID string
|
||||
TableName string
|
||||
PhysicalName string
|
||||
}
|
||||
|
||||
// tableExists checks if a table exists in the database.
|
||||
func tableExists(ctx context.Context, db *sql.DB, table string) bool {
|
||||
if database.IsPostgres() {
|
||||
var exists bool
|
||||
db.QueryRowContext(ctx, "SELECT EXISTS(SELECT 1 FROM information_schema.tables WHERE table_name = $1)", table).Scan(&exists)
|
||||
return exists
|
||||
}
|
||||
// SQLite
|
||||
var name string
|
||||
err := db.QueryRowContext(ctx, "SELECT name FROM sqlite_master WHERE type='table' AND name=?", table).Scan(&name)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// wipeTables deletes all data from the given tables in reverse order (FK-safe).
|
||||
func wipeTables(ctx context.Context, db *sql.DB, tables []string) error {
|
||||
if database.IsSQLite() {
|
||||
db.ExecContext(ctx, "PRAGMA foreign_keys = OFF")
|
||||
defer db.ExecContext(ctx, "PRAGMA foreign_keys = ON")
|
||||
}
|
||||
|
||||
// Delete in reverse order to respect FK constraints
|
||||
for i := len(tables) - 1; i >= 0; i-- {
|
||||
table := tables[i]
|
||||
if !tableExists(ctx, db, table) {
|
||||
continue
|
||||
}
|
||||
if database.IsPostgres() {
|
||||
if _, err := db.ExecContext(ctx, fmt.Sprintf("TRUNCATE TABLE %s CASCADE", table)); err != nil {
|
||||
log.Printf("[backup] warning: truncate %s: %v", table, err)
|
||||
}
|
||||
} else {
|
||||
if _, err := db.ExecContext(ctx, fmt.Sprintf("DELETE FROM %s", table)); err != nil {
|
||||
log.Printf("[backup] warning: delete %s: %v", table, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
372
server/handlers/backup_test.go
Normal file
372
server/handlers/backup_test.go
Normal file
@@ -0,0 +1,372 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
authpkg "switchboard-core/auth"
|
||||
"switchboard-core/config"
|
||||
"switchboard-core/database"
|
||||
"switchboard-core/middleware"
|
||||
"switchboard-core/store"
|
||||
"switchboard-core/store/sqlite"
|
||||
)
|
||||
|
||||
// ── Backup Test Harness ────────────────────────
|
||||
|
||||
type backupHarness struct {
|
||||
*testHarness
|
||||
adminToken string
|
||||
adminID string
|
||||
stores store.Stores
|
||||
backupH *BackupHandler
|
||||
}
|
||||
|
||||
func setupBackupHarness(t *testing.T) *backupHarness {
|
||||
t.Helper()
|
||||
database.RequireTestDB(t)
|
||||
database.TruncateAll(t)
|
||||
|
||||
cfg := &config.Config{
|
||||
JWTSecret: testJWTSecret,
|
||||
BasePath: "",
|
||||
}
|
||||
|
||||
var stores store.Stores
|
||||
if database.IsSQLite() {
|
||||
stores = sqlite.NewStores(database.TestDB)
|
||||
} else {
|
||||
t.Skip("backup tests run on SQLite only")
|
||||
}
|
||||
userCache := middleware.NewUserStatusCache()
|
||||
storagePath := t.TempDir()
|
||||
backupH := NewBackupHandler(stores, "", storagePath)
|
||||
|
||||
r := gin.New()
|
||||
api := r.Group("/api/v1")
|
||||
|
||||
// Auth
|
||||
auth := NewAuthHandler(cfg, stores, nil, authpkg.NewBuiltinProvider())
|
||||
api.POST("/auth/register", auth.Register)
|
||||
api.POST("/auth/login", auth.Login)
|
||||
|
||||
// Admin group
|
||||
admin := api.Group("/admin")
|
||||
admin.Use(middleware.Auth(cfg, stores.Users, userCache), middleware.RequireAdmin(stores))
|
||||
admin.POST("/backup", backupH.CreateBackup)
|
||||
admin.GET("/backups", backupH.ListBackups)
|
||||
admin.GET("/backups/:name", backupH.DownloadBackup)
|
||||
admin.DELETE("/backups/:name", backupH.DeleteBackup)
|
||||
admin.POST("/restore", backupH.RestoreBackup)
|
||||
|
||||
// Seed admin
|
||||
adminID := seedInsertReturningID(t,
|
||||
`INSERT INTO users (username, email, password_hash, handle, auth_source) VALUES ($1, $2, $3, $4, $5) RETURNING id`,
|
||||
"bk-admin", "bk-admin@test.com", "$2a$10$test", "bk-admin", "builtin",
|
||||
)
|
||||
database.SeedEveryoneGroupMember(t, adminID)
|
||||
database.SeedAdminsGroupMember(t, adminID)
|
||||
adminToken := makeToken(adminID, "bk-admin@test.com", "admin")
|
||||
|
||||
return &backupHarness{
|
||||
testHarness: &testHarness{router: r, t: t},
|
||||
adminToken: adminToken,
|
||||
adminID: adminID,
|
||||
stores: stores,
|
||||
backupH: backupH,
|
||||
}
|
||||
}
|
||||
|
||||
// ── Tests ──────────────────────────────────────
|
||||
|
||||
func TestCreateBackup_Basic(t *testing.T) {
|
||||
h := setupBackupHarness(t)
|
||||
|
||||
w := h.request("POST", "/api/v1/admin/backup", h.adminToken, nil)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
|
||||
}
|
||||
|
||||
// Verify it's a valid ZIP
|
||||
data := w.Body.Bytes()
|
||||
zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
|
||||
if err != nil {
|
||||
t.Fatalf("invalid ZIP: %v", err)
|
||||
}
|
||||
|
||||
// Must contain meta.json
|
||||
var foundMeta bool
|
||||
for _, f := range zr.File {
|
||||
if f.Name == "meta.json" {
|
||||
foundMeta = true
|
||||
rc, _ := f.Open()
|
||||
var meta backupMeta
|
||||
json.NewDecoder(rc).Decode(&meta)
|
||||
rc.Close()
|
||||
if meta.Dialect != "sqlite" {
|
||||
t.Errorf("expected dialect sqlite, got %s", meta.Dialect)
|
||||
}
|
||||
if len(meta.CoreTables) == 0 {
|
||||
t.Error("expected core tables in meta")
|
||||
}
|
||||
}
|
||||
}
|
||||
if !foundMeta {
|
||||
t.Fatal("meta.json not found in backup")
|
||||
}
|
||||
|
||||
// Must contain core table JSONL files
|
||||
var foundUsers bool
|
||||
for _, f := range zr.File {
|
||||
if f.Name == "core/users.jsonl" {
|
||||
foundUsers = true
|
||||
}
|
||||
}
|
||||
if !foundUsers {
|
||||
t.Error("core/users.jsonl not found in backup")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateBackup_WithExtData(t *testing.T) {
|
||||
h := setupBackupHarness(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Seed a package so FK constraint is satisfied
|
||||
database.TestDB.ExecContext(ctx, dialectSQL(
|
||||
`INSERT INTO packages (id, title, type, tier, version, source, enabled, manifest)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, 1, '{}')`),
|
||||
"testpkg", "Test Pkg", "extension", "starlark", "0.1.0", "extension",
|
||||
)
|
||||
|
||||
// Register an ext_data table and create it
|
||||
database.TestDB.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS ext_testpkg_items (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
created_at TEXT DEFAULT (datetime('now'))
|
||||
)`)
|
||||
if _, err := database.TestDB.ExecContext(ctx,
|
||||
dialectSQL(`INSERT INTO ext_data_tables (package_id, table_name) VALUES ($1, $2)`),
|
||||
"testpkg", "items",
|
||||
); err != nil {
|
||||
t.Fatalf("insert ext_data_tables: %v", err)
|
||||
}
|
||||
database.TestDB.ExecContext(ctx,
|
||||
`INSERT INTO ext_testpkg_items (id, name) VALUES ('item1', 'Test Item')`,
|
||||
)
|
||||
defer database.TestDB.ExecContext(ctx, "DROP TABLE IF EXISTS ext_testpkg_items")
|
||||
|
||||
w := h.request("POST", "/api/v1/admin/backup", h.adminToken, nil)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("expected 200, got %d", w.Code)
|
||||
}
|
||||
|
||||
data := w.Body.Bytes()
|
||||
zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
|
||||
if err != nil {
|
||||
t.Fatalf("invalid ZIP: %v", err)
|
||||
}
|
||||
|
||||
// Should contain ext_data/testpkg/items.jsonl
|
||||
var foundExt bool
|
||||
for _, f := range zr.File {
|
||||
if f.Name == "ext_data/testpkg/items.jsonl" {
|
||||
foundExt = true
|
||||
rc, _ := f.Open()
|
||||
scanner := bufio.NewScanner(rc)
|
||||
if !scanner.Scan() {
|
||||
t.Error("ext_data file is empty")
|
||||
} else {
|
||||
var row map[string]interface{}
|
||||
json.Unmarshal(scanner.Bytes(), &row)
|
||||
if row["name"] != "Test Item" {
|
||||
t.Errorf("expected 'Test Item', got %v", row["name"])
|
||||
}
|
||||
}
|
||||
rc.Close()
|
||||
}
|
||||
}
|
||||
if !foundExt {
|
||||
t.Error("ext_data/testpkg/items.jsonl not found")
|
||||
}
|
||||
|
||||
// Check meta.json has ext table listed
|
||||
metaFile := findInZip(zr, "meta.json")
|
||||
rc, _ := metaFile.Open()
|
||||
var meta backupMeta
|
||||
json.NewDecoder(rc).Decode(&meta)
|
||||
rc.Close()
|
||||
found := false
|
||||
for _, et := range meta.ExtTables {
|
||||
if et == "ext_testpkg_items" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("ext_testpkg_items not in meta.ExtTables: %v", meta.ExtTables)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestoreBackup_RoundTrip(t *testing.T) {
|
||||
h := setupBackupHarness(t)
|
||||
|
||||
// Create a second user so we have data to verify
|
||||
seedInsertReturningID(t,
|
||||
`INSERT INTO users (username, email, password_hash, handle, auth_source) VALUES ($1, $2, $3, $4, $5) RETURNING id`,
|
||||
"bk-user2", "bk-user2@test.com", "$2a$10$test", "bk-user2", "builtin",
|
||||
)
|
||||
|
||||
// Create backup
|
||||
w := h.request("POST", "/api/v1/admin/backup", h.adminToken, nil)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("backup: expected 200, got %d: %s", w.Code, w.Body.String())
|
||||
}
|
||||
backupData := w.Body.Bytes()
|
||||
|
||||
// Verify we have 2 users
|
||||
var countBefore int
|
||||
database.TestDB.QueryRow("SELECT COUNT(*) FROM users").Scan(&countBefore)
|
||||
if countBefore < 2 {
|
||||
t.Fatalf("expected at least 2 users, got %d", countBefore)
|
||||
}
|
||||
|
||||
// Restore directly (the handler wipes + restores, so data should round-trip)
|
||||
// We use a separate router without admin middleware for restore,
|
||||
// since the wipe inside RestoreBackup removes the admin group membership
|
||||
// mid-request. In production, restore would use a session-based token
|
||||
// that survives the wipe. For testing, we bypass the middleware.
|
||||
restoreRouter := gin.New()
|
||||
restoreRouter.POST("/restore", h.backupH.RestoreBackup)
|
||||
|
||||
body := &bytes.Buffer{}
|
||||
writer := multipart.NewWriter(body)
|
||||
part, _ := writer.CreateFormFile("file", "test-backup.swb")
|
||||
part.Write(backupData)
|
||||
writer.Close()
|
||||
|
||||
req := httptest.NewRequest("POST", "/restore", body)
|
||||
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
rw := httptest.NewRecorder()
|
||||
restoreRouter.ServeHTTP(rw, req)
|
||||
|
||||
if rw.Code != http.StatusOK {
|
||||
t.Fatalf("restore: expected 200, got %d: %s", rw.Code, rw.Body.String())
|
||||
}
|
||||
|
||||
// Verify data is back
|
||||
var countAfterRestore int
|
||||
database.TestDB.QueryRow("SELECT COUNT(*) FROM users").Scan(&countAfterRestore)
|
||||
if countAfterRestore != countBefore {
|
||||
t.Errorf("expected %d users after restore, got %d", countBefore, countAfterRestore)
|
||||
}
|
||||
|
||||
// Verify specific user exists
|
||||
var username string
|
||||
err := database.TestDB.QueryRow("SELECT username FROM users WHERE username = 'bk-user2'").Scan(&username)
|
||||
if err != nil {
|
||||
t.Errorf("user bk-user2 not found after restore: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestoreBackup_SchemaVersionMismatch(t *testing.T) {
|
||||
h := setupBackupHarness(t)
|
||||
|
||||
// Build a fake backup with a future schema version
|
||||
var buf bytes.Buffer
|
||||
zw := zip.NewWriter(&buf)
|
||||
mw, _ := zw.Create("meta.json")
|
||||
json.NewEncoder(mw).Encode(backupMeta{
|
||||
Version: "99.0.0",
|
||||
SchemaVersion: "999_future",
|
||||
Dialect: "sqlite",
|
||||
CoreTables: []string{"users"},
|
||||
})
|
||||
zw.Close()
|
||||
|
||||
body := &bytes.Buffer{}
|
||||
writer := multipart.NewWriter(body)
|
||||
part, _ := writer.CreateFormFile("file", "future.swb")
|
||||
part.Write(buf.Bytes())
|
||||
writer.Close()
|
||||
|
||||
req := httptest.NewRequest("POST", "/api/v1/admin/restore", body)
|
||||
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
req.Header.Set("Authorization", "Bearer "+h.adminToken)
|
||||
rw := httptest.NewRecorder()
|
||||
h.router.ServeHTTP(rw, req)
|
||||
|
||||
if rw.Code != http.StatusBadRequest {
|
||||
t.Fatalf("expected 400 for future schema, got %d: %s", rw.Code, rw.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestDumpAndRestoreTable(t *testing.T) {
|
||||
database.RequireTestDB(t)
|
||||
ctx := context.Background()
|
||||
db := database.TestDB
|
||||
|
||||
// Create a test table
|
||||
db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS test_backup_round (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
count INTEGER DEFAULT 0
|
||||
)`)
|
||||
defer db.ExecContext(ctx, "DROP TABLE IF EXISTS test_backup_round")
|
||||
|
||||
// Insert data
|
||||
db.ExecContext(ctx, `INSERT INTO test_backup_round (id, name, count) VALUES ('r1', 'alpha', 10)`)
|
||||
db.ExecContext(ctx, `INSERT INTO test_backup_round (id, name, count) VALUES ('r2', 'beta', 20)`)
|
||||
|
||||
// Dump
|
||||
var buf bytes.Buffer
|
||||
count, err := dumpTable(ctx, db, "test_backup_round", &buf)
|
||||
if err != nil {
|
||||
t.Fatalf("dump: %v", err)
|
||||
}
|
||||
if count != 2 {
|
||||
t.Fatalf("dump: expected 2 rows, got %d", count)
|
||||
}
|
||||
|
||||
// Wipe
|
||||
db.ExecContext(ctx, "DELETE FROM test_backup_round")
|
||||
|
||||
// Restore
|
||||
restored, err := restoreTable(ctx, db, "test_backup_round", &buf)
|
||||
if err != nil {
|
||||
t.Fatalf("restore: %v", err)
|
||||
}
|
||||
if restored != 2 {
|
||||
t.Fatalf("restore: expected 2 rows, got %d", restored)
|
||||
}
|
||||
|
||||
// Verify
|
||||
var name string
|
||||
var cnt int
|
||||
db.QueryRowContext(ctx, "SELECT name, count FROM test_backup_round WHERE id = 'r1'").Scan(&name, &cnt)
|
||||
if name != "alpha" || cnt != 10 {
|
||||
t.Errorf("expected alpha/10, got %s/%d", name, cnt)
|
||||
}
|
||||
}
|
||||
|
||||
func TestListBackups_Empty(t *testing.T) {
|
||||
h := setupBackupHarness(t)
|
||||
|
||||
w := h.request("GET", "/api/v1/admin/backups", h.adminToken, nil)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("expected 200, got %d", w.Code)
|
||||
}
|
||||
|
||||
var resp struct{ Data []interface{} }
|
||||
json.NewDecoder(w.Body).Decode(&resp)
|
||||
if len(resp.Data) != 0 {
|
||||
t.Errorf("expected empty list, got %d", len(resp.Data))
|
||||
}
|
||||
}
|
||||
32
server/handlers/cluster.go
Normal file
32
server/handlers/cluster.go
Normal file
@@ -0,0 +1,32 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// ClusterHandler serves the admin cluster API.
|
||||
type ClusterHandler struct {
|
||||
stores store.Stores
|
||||
}
|
||||
|
||||
func NewClusterHandler(stores store.Stores) *ClusterHandler {
|
||||
return &ClusterHandler{stores: stores}
|
||||
}
|
||||
|
||||
// ListNodes returns all registered cluster nodes.
|
||||
// GET /api/v1/admin/cluster
|
||||
func (h *ClusterHandler) ListNodes(c *gin.Context) {
|
||||
nodes, err := h.stores.Cluster.ListNodes(c.Request.Context())
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list cluster nodes"})
|
||||
return
|
||||
}
|
||||
if nodes == nil {
|
||||
nodes = []store.ClusterNode{}
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"data": nodes})
|
||||
}
|
||||
114
server/handlers/cluster_test.go
Normal file
114
server/handlers/cluster_test.go
Normal file
@@ -0,0 +1,114 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"switchboard-core/store"
|
||||
)
|
||||
|
||||
// mockClusterStore implements store.ClusterStore for handler tests.
|
||||
type mockClusterStore struct {
|
||||
nodes []store.ClusterNode
|
||||
}
|
||||
|
||||
func (m *mockClusterStore) Register(_ context.Context, _, _ string) error { return nil }
|
||||
func (m *mockClusterStore) Heartbeat(_ context.Context, _ string, _ json.RawMessage) (int64, error) {
|
||||
return 1, nil
|
||||
}
|
||||
func (m *mockClusterStore) SweepStale(_ context.Context, _ time.Duration) (int64, error) { return 0, nil }
|
||||
func (m *mockClusterStore) ListNodes(_ context.Context) ([]store.ClusterNode, error) {
|
||||
return m.nodes, nil
|
||||
}
|
||||
func (m *mockClusterStore) Deregister(_ context.Context, _ string) error { return nil }
|
||||
|
||||
func TestClusterListNodes(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
mock := &mockClusterStore{
|
||||
nodes: []store.ClusterNode{
|
||||
{
|
||||
NodeID: "node-1",
|
||||
Endpoint: "http://node-1:8080",
|
||||
Seq: 1,
|
||||
RegisteredAt: time.Now(),
|
||||
Heartbeat: time.Now(),
|
||||
Stats: json.RawMessage(`{"ws_clients":3}`),
|
||||
},
|
||||
{
|
||||
NodeID: "node-2",
|
||||
Endpoint: "http://node-2:8080",
|
||||
Seq: 2,
|
||||
RegisteredAt: time.Now(),
|
||||
Heartbeat: time.Now(),
|
||||
Stats: json.RawMessage(`{"ws_clients":7}`),
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
stores := store.Stores{Cluster: mock}
|
||||
h := NewClusterHandler(stores)
|
||||
|
||||
r := gin.New()
|
||||
r.GET("/api/v1/admin/cluster", h.ListNodes)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/cluster", nil)
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want %d", w.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Data []store.ClusterNode `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("unmarshal response: %v", err)
|
||||
}
|
||||
if len(resp.Data) != 2 {
|
||||
t.Fatalf("data length = %d, want 2", len(resp.Data))
|
||||
}
|
||||
if resp.Data[0].NodeID != "node-1" {
|
||||
t.Errorf("data[0].node_id = %q, want %q", resp.Data[0].NodeID, "node-1")
|
||||
}
|
||||
if resp.Data[1].NodeID != "node-2" {
|
||||
t.Errorf("data[1].node_id = %q, want %q", resp.Data[1].NodeID, "node-2")
|
||||
}
|
||||
}
|
||||
|
||||
func TestClusterListNodesEmpty(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
mock := &mockClusterStore{nodes: nil}
|
||||
stores := store.Stores{Cluster: mock}
|
||||
h := NewClusterHandler(stores)
|
||||
|
||||
r := gin.New()
|
||||
r.GET("/api/v1/admin/cluster", h.ListNodes)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/cluster", nil)
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want %d", w.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Data []json.RawMessage `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
// Should be empty array, not null
|
||||
if resp.Data == nil {
|
||||
t.Error("data should be [], not null")
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user