core/ROADMAP.md

Armature — Roadmap

Current: v0.6.14 — Visual Polish

Self-hosted extensible platform. Auth, identity, packages, Starlark sandbox, storage, realtime, and ops are kernel primitives. Everything else is an extension.

Kernel capabilities: Auth (builtin/mTLS/OIDC) · Users/teams/groups/RBAC · Surfaces/extensions/libraries/workflows · Starlark sandbox (capability-gated) · Object storage (PVC/S3) + ext_data tables · WebSocket hub + realtime pub/sub · Audit log · Notifications · Scheduled tasks

Completed history: v0.2.x–v0.5.x fully documented in CHANGELOG.md. Highlights: RBAC + settings cascade, event bus + triggers, SDK stabilization, workflow engine (multi-stage, team roles, signoff gate, public entry, SLA), package distribution, Notes surface (CM6, folders, tags, backlinks, graph), realtime primitive, Chat surface (chat-core library + surface + polish), upgrade test harness, cluster registry + HA.

---

v0.6.0 — MVP

Extension, communication, and operations tracks converge. First externally usable release.

Design docs: docs/DESIGN-cluster-registry.md — PG-backed cluster registry and self-assembling mesh.

v0.6.0 — Cluster Registry + HA

PG is the consensus layer. Zero new infrastructure. UNLOGGED table + LISTEN/NOTIFY replaces etcd/Consul/Redis for homelab-to-small-team scale.

Step	Status	Description
node_registry table	✅	UNLOGGED TABLE — node_id, endpoint, seq, registered_at, heartbeat, stats JSONB. Postgres migration 013.
Node registration	✅	Self-registration on startup: INSERT ... ON CONFLICT DO UPDATE. node_id = hostname-PID or CLUSTER_NODE_ID env override.
Heartbeat tick	✅	Every 10s: update own heartbeat + collect runtime stats (goroutines, heap, GC, uptime, ws_clients).
Stale sweep	✅	Every heartbeat tick: DELETE WHERE heartbeat < now() - 30s. All nodes run it — idempotent, no ring topology.
Self-eviction	✅	If heartbeat UPDATE returns 0 rows: node was swept by peer → log error + os.Exit(1). K8s restarts → re-register.
LISTEN/NOTIFY routing	✅	Durable events (messages, state changes) fan-out via pg_notify. All replicas receive, push to local WS subscribers, drop if irrelevant. Phase 1: ephemeral events (typing, presence) also via NOTIFY (8KB limit, ~60 bytes each).
Cluster API	✅	GET /api/v1/admin/cluster — returns {data: [...]} envelope with all registered nodes.
Admin cluster dashboard	✅	cluster-dashboard surface package renders one card per node: node_id, endpoint, uptime, ws_clients, heap, GC pause. JSONB stats — future keys render automatically, no schema migration. Auto-refresh every 10s.
Health endpoint	✅	GET /health includes node_id and cluster: {size, peers, heartbeat_age_ms}.
Config	✅	CLUSTER_NODE_ID, CLUSTER_HEARTBEAT_INTERVAL (default 10s), CLUSTER_STALE_THRESHOLD (default 30s), CLUSTER_ENDPOINT (Phase 2 mesh, auto-detect).
Single-node regression	✅	One-node behavior identical to pre-cluster: one registry row, NOTIFY delivers back to same instance. No special-casing. SQLite returns nil store — all cluster code guarded.
Multi-node integration test	✅	Docker Compose: 3 instances, shared PG. ci/e2e-cluster-test.sh: registration, stale sweep on stop, re-registration on restart. 3 unit tests + 2 handler tests.

v0.6.1 — Backup/Restore + Documentation

Step	Status	Description
Backup handler	✅	POST /api/v1/admin/backup streams .swb ZIP (JSONL core + ext_data tables + package assets). POST /api/v1/admin/restore wipes DB and restores from archive. Dialect-neutral (SQLite + Postgres).
Server-side backups	✅	GET /api/v1/admin/backups list, GET /download, DELETE. Store backups in {STORAGE_PATH}/backups/.
Admin backup section	✅	New "Backup" section under /admin/backup. Create (download or server-side), list, download, delete, restore with destructive confirmation.
Documentation API	✅	GET /api/v1/docs lists, GET /api/v1/docs/:name returns raw markdown. Authenticated (not admin-only).
Docs surface	✅	Builtin surface at /docs/:section. Sidebar navigation, client-side markdown renderer. 5 new docs: Getting Started, Extension Guide, API Reference, Deployment, Package Format.
Tests	✅	6 handler tests (basic backup, ext_data backup, round-trip restore, schema mismatch, dump/restore table, list empty). E2E script ci/e2e-backup-test.sh.

v0.6.2 — Docs Polish + Dynamic OpenAPI

Step	Status	Description
Dark mode fix	✅	Added --bg-code to CSS variables (dark + light). Replaced all hardcoded light-mode fallbacks in docs CSS with theme-aware variables. Table styling, code blocks, nav items all readable in dark mode.
Loading & error handling	✅	Replaced borrowed settings-placeholder with docs-specific pulse animation. Added error state with retry button for failed doc list fetch.
Topbar navigation	✅	Imported Topbar + UserMenu into docs surface. Users can now navigate to other surfaces via the avatar menu.
Docs icon + menu entry	✅	Added 📖 Docs entry to UserMenu standard items. Docs accessible from any surface's user menu.
api_schema manifest field	✅	Optional api_schema array in manifest.json. Parsed lazily by spec builder. Malformed entries logged and skipped — never blocks extension loading.
OpenAPI spec builder	✅	BuildOpenAPISpec() merges static kernel spec with extension routes. Tier 1: auto-generated stubs for all api_routes. Tier 2: api_schema replaces stubs with rich path items (params, body, response).
Dynamic spec endpoint	✅	GET /api/docs/openapi.json serves merged spec. Swagger UI updated to use JSON endpoint. Static YAML preserved for backward compat.
Tests	✅	7 new handler tests: zero extensions, stubs, rich schema, multi-extension, malformed schema, disabled exclusion, required fields.

---

v0.6.x — Hardening

Closes every audit finding before the public release. No new features — only correctness, dead code elimination, and architectural cleanup. Sequence is fixed: each version is a gate for the next.

v0.6.3 — Dead Code Sweep + Registry Fix

Pure cleanup. No behavior changes except fixing the broken registry install flow.

Step	Status	Description
Fix registry install	✅	SDK sends { url }, handler expects { download_url }. Fix api-domains.js to send { download_url: url }. Every Install click currently returns 400.
Registry settings UI	✅	Add "Package Registry" section to admin settings with URL input field. Only way to configure registry today is a raw PUT /api/v1/admin/settings/package_registry — no user will find it.
Registry tooling + docs	✅	scripts/generate-registry.sh scans a directory of .pkg files and emits registry JSON. docs/PACKAGE-REGISTRY.md documents the format.
Delete dead kernel Go	✅	store/interfaces.go:178–183 — orphaned ChannelListFilter comments. pages/pages.go:922–930 — roleFilterType() + template registration (chat vestige, maps nonexistent roles). main.go:67 — orphaned provider-type comment.
Delete dead vendor JS	✅	vendor/marked.min.js and vendor/purify.min.js — 62KB, zero production imports. Only referenced in test helpers.
Delete dev.html	✅	676 lines, not imported by anything. Dead.
Remove dashboard from default bundle	✅	Requires legacy-sdk (doesn't exist), auto-installs as dormant on fresh installs. Confusing. Remove from defaultBundledPackages.
Remove hello-dashboard	✅	Proof-of-concept from early development. Move to examples/ or delete.
Strip stale version comments	✅	60+ files had legacy version annotations (// v0.29.x:, // v0.33.x:). Single sed pass.
Narrow default bundle	✅	Default: notes, chat, chat-core, mermaid-renderer, schedules. Everything else available via registry or BUNDLED_PACKAGES=*.

v0.6.4 — Admin Health/Metrics Tab + Cluster Merge

Structural move: cluster dashboard becomes an Admin tab. Better home for health/metrics — shared context with other admin panels, no separate nav entry.

Step	Status	Description
"Health / Metrics" admin tab	✅	New tab in Admin surface. DB-agnostic metrics for all deployments. Cluster cards conditional on PG + multi-node detection.
Runtime metrics	✅	Per-node: goroutines, heap alloc/sys, stack in use, GC cycles, last GC pause, GC CPU %, uptime, WebSocket clients, extensions loaded, open FDs.
DB pool metrics	✅	All deployments: DB latency (SELECT 1 round-trip), pool active/idle/max, wait count, wait duration. PG-only: table bloat (n_dead_tup), active backends (pg_stat_activity).
Cluster metrics	✅	PG multi-node only: cluster size, peer list with endpoint + uptime, heartbeat age per node, event bus publish/deliver rates.
Extension runtime metrics	✅	Starlark exec/min, errors/min, avg duration, HTTP outbound requests/min, trigger fires/min, schedule overruns.
Fatten heartbeat payload	✅	Heartbeat JSONB carries full metric set. GET /api/v1/admin/metrics for single-node SQLite fallback (same shape).
Retire cluster-dashboard	✅	Remove package once Admin Health tab ships. Update defaultBundledPackages.
Fix block renderer requires	✅	mermaid-renderer, katex-renderer, csv-table, diff-viewer all have "requires": ["chat"]. These are content renderers, not chat features. Remove constraint — they should activate without chat.
Health endpoint consolidation	✅	/health and /api/v1/health return near-identical JSON. Merge or clearly differentiate with docs.

v0.6.5 — Renderer Pipeline + Docs Rewrite

Most complex sub-version. Lifts block rendering to a kernel SDK primitive so all surfaces share it, then rewrites the docs for an external audience.

Step	Status	Description
SDK renderer primitive	✅	sw.renderers.register(pattern, handler) — kernel-level registration. Extensions call once; all surfaces consume. Decouples renderer discovery from chat surface.
Notes hooks SDK renderer pipeline	✅	Notes delegates to sw.markdown.renderSync() + sw.renderers.runPostRenderers(). Hand-rolled renderer deleted.
Docs hooks SDK renderer pipeline	✅	Docs delegates to sw.markdown.renderSync() + sw.renderers.runPostRenderers(). Hand-rolled renderer deleted (~160 lines).
Unify markdown renderer	✅	sw.markdown module lazy-loads marked v16 (vendored). Notes, Docs, Chat all consume it. Two hand-rolled parsers deleted.
Sanitize HTML output	✅	DOMPurify wired as default post-render step in sw.markdown. SVG-safe config allows mermaid output. Notes sanitizes; Docs opts out (system-authored).
Mermaid/KaTeX/CSV/Diff work everywhere	✅	Browser extension loader injects renderer scripts into all pages. Extensions register via sw:ready event. All four render in Notes, Docs, and Chat.
Docs rewrite for external audience	✅	All fork references removed from docs, CHANGELOG, ROADMAP. DESIGN-WORKFLOW-REDESIGN-0.2.6.md replaced with DESIGN-WORKFLOWS.md.
Add Mermaid architecture diagrams	✅	Six diagrams in ARCHITECTURE.md: system overview, request flow, extension lifecycle, realtime events, settings cascade, cluster topology.
Surface alias decision	✅	Migrated SDK + ICD runner to /admin/packages/; aliases removed from main.go.
CONTRIBUTING.md + tutorial	✅	CONTRIBUTING.md at repo root + docs/TUTORIAL-FIRST-EXTENSION.md walkthrough.

v0.6.7 — Native mTLS

End-to-end mutual TLS without a reverse proxy. Targets systemd+podman deployments where every connection (client→server, node→node) is mTLS. Design: docs/DESIGN-native-mtls.md.

Step	Status	Description
TLS_MODE config	✅	Three values: none (default, plain HTTP) · server (TLS, no client cert) · mtls (mutual TLS, client cert required). Independent of AUTH_MODE.
TLS server mode	✅	Go binary calls ListenAndServeTLS directly. TLS_CERT, TLS_KEY, TLS_CA path config. TLS 1.3 minimum, no fallback. server/config/tls.go loader + validation.
MTLSNativeProvider	✅	Reads r.TLS.PeerCertificates[0] — no header trust. Subject.CommonName → username. sha256(Raw) → external_id. Auto-provisions auth_source=mtls. Renamed existing mtls.go → mtls_proxy.go. Shared helpers in mtls_helpers.go.
Node-to-node mTLS	✅	BuildPeerTLSConfig() constructs outbound TLS config with node cert + CA pool. Forward-looking — cluster registry is DB-backed (PG LISTEN/NOTIFY), no HTTP peer calls yet.
armature-ca.sh	✅	Shell wrapper around openssl. Three commands: init (CA keypair) · issue-node --name <n> --san <addrs> (365d) · issue-user --cn <name> (90d). All output PEM.
Unit tests	✅	Ephemeral CA via crypto/x509. Fabricated PeerCertificates. Valid cert → user provisioned · no TLS → ErrNoCert · empty certs → ErrNoCert · no CN → ErrInvalidCreds.
Integration tests	✅	Real TLS listener on localhost. No cert / wrong CA / expired → TLS handshake rejected. Valid cert → 200. Peer certificate CN + email visibility verified.

v0.6.6 — Final Hardening

Final pass before public release. Security, correctness, and developer experience.

Step	Status	Description
Extension dependency auto-activation	✅	Installing a package with unmet depends/requires auto-installs dependencies from the bundled set. If not bundled: clear error listing what's missing.
ValidateManifest() gate	✅	Single ValidateManifest() function in package_validate.go. Called at install time (both upload and bundled). 12 unit tests.
Package signing hook	✅	Optional signature field in manifest (reserved). PACKAGE_VERIFY_SIGNATURES env var (default false, log-only). No cryptographic code yet — schema slot reserved.
OIDC state nonce validation	✅	oidcClaims.Nonce field added. ValidateIDTokenNonce() compares ID token nonce against stored state. Callback rejects mismatched nonces.
Schema migration stub decision	✅	Stub replaced with log-only function documenting additive-only policy. Downgrade rejection preserved.
ICD/SDK runner update pass	✅	ICD smoke tier: added metrics, cluster, backups, docs, OpenAPI JSON endpoints. SDK admin domain: added metrics, cluster, backups tests.
Stale TODO resolution	✅	main.go session middleware: replaced with OptionalAuth() (auth if token present, anonymous pass-through). auth.go:221 OIDC nonce: resolved. starlark_helpers.go:96 migration stub: resolved.

Then ship.

---

Post-MVP

• LLM participation (llm-bridge extension: subscribes to chat.message.created, calls provider.complete(), streams response via realtime.publish, posts via chat.send(). Bot participants with persona config. Multi-model conversations.)
• Rich media extensions: image generation, code sandbox, STT/TTS
• Desktop app (Tauri or Electron)
• Sidecar tier: container-based extensions
• Federation: cross-instance package sharing
• Plugin marketplace with signing and review

---

Design Decisions Log

Decision	Rationale
Tasks → extension	Scheduler was the most entangled kernel component (~3,400 lines). Rebuilding as extension validates the trigger system and removes the worst compilation debt. Three trigger primitives (time, webhook, event) replace the monolithic scheduler.
Sessions removed	Kernel-managed sessions replaced by workflow instances with dedicated storage (ext_data tables or kernel table).
custom stage mode	Stage mode custom delegates to a surface package, proving extension composability. Chat-in-workflow is handled by the chat extension, not the kernel.
Providers removed from kernel	Provider configs, model catalog, routing policies — all moved to extension track. Kernel provides credential storage (connections) and the Starlark provider.complete module as the interface.
Kernel permissions simplified	6 platform permissions. Extensions define their own capability requirements in manifests.
Preact+htm retained	3KB runtime, no build step, works for extension authors without bundler config. KISS.
Single Docker image	Drop the frontend/backend split. Go binary + assets + migrations in one image. Simpler deployment, fewer moving parts.
Admin → RBAC group	The role column is pre-RBAC. v0.2.0 replaces it with a seeded "Admins" group + surface.admin.access grant. All users auto-join "Everyone" group. Admin middleware becomes a grant check, not a role check.
Settings cascade	RBAC controls scope auth (who can set at what level). user_overridable flag controls whether lower scopes can override higher. Two orthogonal axes, composes cleanly with extension manifests.
No new migrations pre-MVP	Edit existing migration SQL files in place. No migration chains until schema is in production.
Notes over Editor	First surface is Obsidian-style notes (rich text, folders, backlinks) instead of a code editor. Notes is a stronger E2E proof — it exercises ext_data, storage, and the SDK more fully than a pure-browser CM6 editor.
No built-in auto-install	Extensions ship in the repo but are not auto-installed. Distribution model TBD — explicit install only. Keeps the kernel clean and avoids opinionated defaults.
Chat as extension, not kernel	Human-to-human chat built entirely as library + surface packages. Zero kernel awareness of conversations, messages, or participants. Kernel gains one generic realtime module. Proves near-infinite extensibility. LLM participation layers on top via a separate bridge extension — the chat system doesn't know or care whether a participant is human or AI.
PG as consensus layer	Horizontal scaling uses PG as the sole coordinator (UNLOGGED node_registry + LISTEN/NOTIFY). No etcd, Consul, Redis, or Raft. Rationale: system is already tightly coupled to PG; adding a second consensus layer doubles operational complexity for zero benefit at homelab-to-small-team scale. UNLOGGED table is visible to all connections, survives session disconnect, and cleans up automatically on PG crash — correct behavior since all nodes are dead anyway. Sweep-all for health (every node deletes stale rows) is simpler than ring topology and has no edge cases.
Two trigger tiers	Event + webhook triggers are extension-declared (manifest contract, full sandbox). Scheduled tasks are user-created ad-hoc (restricted sandbox — no raw HTTP, no DB table creation, connections-only outbound). Separation keeps extension contracts static and user automation safe.
Scheduled task identity	Tasks run as their creator (RBAC-scoped). Admin-created tasks can opt into system context. Creator deactivation pauses the schedule. Ensures audit trail and permission boundaries.
Builtin package rationale	A builtin must enhance kernel surfaces or be required to demonstrate the platform's own capabilities. notes — primary content surface, exercises ext_data/storage/SDK/settings/realtime fully. chat + chat-core — primary communication surface, proves the extensibility thesis (100% extension, zero kernel awareness). mermaid-renderer — docs surface uses Mermaid diagrams to explain the architecture; without it the platform's own documentation doesn't render (self-bootstrapping). schedules — UI for the kernel's scheduled task system; without it users can't manage cron jobs (kernel primitive UI). All others are domain features, examples, dormant, or LLM-only tools — available via registry or BUNDLED_PACKAGES=*.
Cluster dashboard retired	cluster-dashboard shipped as a standalone surface package (v0.6.0) as an expedient. Health/metrics belong inside the Admin surface as a tab — shared context, no separate nav entry, no install required. Merged in v0.6.4.
Block renderers decoupled from chat	mermaid-renderer, katex-renderer, csv-table, diff-viewer shipped with "requires": ["chat"] because renderer discovery lived inside the chat surface. These are content renderers, not chat features. v0.6.4 removes the constraint; v0.6.5 lifts renderer registration to the kernel SDK so all surfaces share it without reimplementing discovery.