switchboard/core
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Releases 9 Wiki Activity

Compare commits

base: switchboard:main
Branches Tags
switchboard:main
switchboard:v0.9.9 switchboard:v0.9.0 switchboard:v0.8.5 switchboard:v0.7.12 switchboard:v0.7.4 switchboard:v0.6.18 switchboard:v0.6.16 switchboard:v0.6.8 switchboard:v0.6.7
..
compare: switchboard:df310d9aa515f40770da8158e8683d718ad014f5
Branches Tags
switchboard:main
switchboard:v0.9.9 switchboard:v0.9.0 switchboard:v0.8.5 switchboard:v0.7.12 switchboard:v0.7.4 switchboard:v0.6.18 switchboard:v0.6.16 switchboard:v0.6.8 switchboard:v0.6.7

3 Commits
main ... df310d9aa5

Author SHA1 Message Date
Jeffrey Smith
df310d9aa5 Bump VERSION to 0.2.8, finalize changelog
All checks were successful
CI/CD / detect-changes (pull_request) Successful in 21s
Details
CI/CD / test-frontend (pull_request) Successful in 5s
Details
CI/CD / test-go-pg (pull_request) Successful in 2m9s
Details
CI/CD / test-sqlite (pull_request) Successful in 2m50s
Details
CI/CD / build-and-deploy (pull_request) Successful in 1m50s
Details 
Team admin settings audit (pass 1): remove dead BYOK/provider/persona
vestiges from team store, models, policy defaults, workflow stage UI,
and ICD tests. Fix stale chat_only stage mode in frontend to match
backend CHECK constraint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 16:13:10 +00:00
Jeffrey Smith
b82640ba40 Bump VERSION to 0.2.7, finalize changelog
Some checks failed
CI/CD / detect-changes (pull_request) Successful in 4s
Details
CI/CD / test-frontend (pull_request) Successful in 5s
Details
CI/CD / test-go-pg (pull_request) Failing after 2m42s
Details
CI/CD / test-sqlite (pull_request) Successful in 2m58s
Details
CI/CD / build-and-deploy (pull_request) Has been skipped
Details 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 15:50:36 +00:00
Jeffrey Smith
66063b08ed Feat v0.2.7 user settings audit (#11) 
Audit all 6 user settings sections. Remove dead chat-era code,
rename localStorage namespace, clean up stale backend policies.

- Remove BYOK nav section (empty BYOK_ITEMS, byokEnabled state,
  "BYOK Enabled" badge) and personas gate filter from settings nav
- Remove auth.permissions.changed listener (existed solely for
  dead BYOK + personas state)
- Remove Message Font Size slider, msgFont state, and --msg-font
  CSS variable application from appearance section and base template
- Rename localStorage key cs-appearance → sb-appearance with
  one-time migration preserving existing user preferences
- Remove allow_user_byok and allow_user_personas from PolicyDefaults,
  profile bootstrap, permissions handler, PublicSettings, and test seeds
- Remove orphaned .settings-nav-footer CSS class
- Replace stale policy-gating test assertions with allow_registration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 15:38:42 +00:00
769 changed files with 18007 additions and 65655 deletions
Expand all files Collapse all files

184
.gitea/workflows/ci.yaml
View File

@@ -1,18 +1,16 @@
# .gitea/workflows/ci.yaml
# ============================================
# Armature - CI/CD Pipeline (v0.18.0)
# Switchboard Core - CI/CD Pipeline (v0.17.3)
# ============================================
# Single unified image (Go backend + nginx frontend).
# v0.1.0: Dropped FE/BE image split per ROADMAP design decision.
#
# Pipeline:
# 0. Detect changes (path-based gating for all downstream jobs)
# 1a. Frontend tests — skipped if only BE/docs/packages changed
# 1a. Frontend tests — skipped if only BE/docs changed
# 1b. Go unit tests — all non-DB packages + SQLite integration (race-enabled)
# 1c. Go test (PG) — PG store + handlers against Postgres (race-enabled)
# 1d. Test runners — re-enabled v0.7.5 (any code change triggers)
# 1e. E2E smoke — Playwright navigation smoke test against every surface
# 2. Build + Deploy — always runs (docs are served in-app)
# 2. Build + Deploy — skipped if docs-only change
#
# Test coverage mapping (no package tested by zero jobs):
# Unit packages (auto-discovered) → test-sqlite (race)
@@ -25,30 +23,28 @@
# Path gating rules:
# src/, src/editor/ → frontend tests
# server/, scripts/db-* → backend tests (PG + SQLite)
# packages/ → test-runners + e2e-smoke
# Dockerfile*, k8s/, .gitea/ → all tests (infra change)
# ci/ → infra (CI scripts)
# docs/, *.md → build-and-deploy only (docs served in-app)
# VERSION, scripts/* → build-and-deploy only (no tests)
# docs/, *.md → skip all tests + deploy
# VERSION, scripts/* → frontend + backend tests
# Tags (v*) → always full pipeline
#
# Deployment mapping (single domain, path-based):
# PR → FE + BE :dev → armature.DOMAIN/dev/ (DB wipe + fresh schema)
# Push to main → FE + BE :test → armature.DOMAIN/test/ (migrate only)
# Tag v* → FE + BE :latest → armature.DOMAIN/ (migrate only)
# PR → FE + BE :dev → switchboard.DOMAIN/dev/ (DB wipe + fresh schema)
# Push to main → FE + BE :test → switchboard.DOMAIN/test/ (migrate only)
# Tag v* → FE + BE :latest → switchboard.DOMAIN/ (migrate only)
# → Unified :latest → Docker Hub (not deployed)
#
# Database lifecycle:
# CI: bootstrap (admin creds) → create DB + role + extensions
# CI: dev wipe (app creds) → drop tables for fresh install test
# BE: auto-migrate on startup → schema_migrations tracking
# BE: bootstrap admin from ARMATURE_ADMIN_* env vars (upsert on every restart)
# BE: bootstrap admin from SWITCHBOARD_ADMIN_* env vars (upsert on every restart)
#
# Shared PG safety:
# - Bootstrap uses IF NOT EXISTS (idempotent)
# - Dev wipe REFUSES to run on databases not ending in _dev
# - Admin creds never reach the backend pods
# - Each env has its own DB name (armature_{dev,test,})
# - Each env has its own DB name (switchboard_core_{dev,test,})
#
# Required Gitea Variables:
# REGISTRY, NAMESPACE, DOMAIN, POSTGRES_HOST
@@ -62,7 +58,7 @@
# Required Gitea Secrets:
# POSTGRES_USER, POSTGRES_PASSWORD
# POSTGRES_ADMIN_USER, POSTGRES_ADMIN_PASSWORD
# ARMATURE_ADMIN_USERNAME, ARMATURE_ADMIN_PASSWORD, ARMATURE_ADMIN_EMAIL
# SWITCHBOARD_ADMIN_USERNAME, SWITCHBOARD_ADMIN_PASSWORD, SWITCHBOARD_ADMIN_EMAIL
# ENCRYPTION_KEY — AES-256 key for API key encryption (openssl rand -base64 32)
# PROVIDER_KEY — API key for live provider integration tests (optional, tests skip if missing)
# DOCKERHUB_USERNAME, DOCKERHUB_TOKEN (optional)
@@ -92,8 +88,8 @@ env:
CERT_ISSUER: ${{ vars.CERT_ISSUER_PROD || 'letsencrypt-prod' }}
POSTGRES_HOST: ${{ vars.POSTGRES_HOST || 'postgres-primary.postgres.svc.cluster.local' }}
POSTGRES_PORT: ${{ vars.POSTGRES_PORT || '5432' }}
IMAGE: ${{ vars.REGISTRY }}/armature
DOCKERHUB_IMAGE: ${{ vars.DOCKERHUB_IMAGE || 'gobha/armature' }}
IMAGE: ${{ vars.REGISTRY }}/switchboard/core
DOCKERHUB_IMAGE: ${{ vars.DOCKERHUB_IMAGE || 'gobha/switchboard-core' }}
jobs:
# ── Stage 0: Detect Changed Paths ──────────────
@@ -104,7 +100,6 @@ jobs:
outputs:
frontend: ${{ steps.filter.outputs.frontend }}
backend: ${{ steps.filter.outputs.backend }}
packages: ${{ steps.filter.outputs.packages }}
infra: ${{ steps.filter.outputs.infra }}
docs_only: ${{ steps.filter.outputs.docs_only }}
steps:
@@ -142,7 +137,7 @@ jobs:
echo "${CHANGED}" | sed 's/^/ /'
# Classify
FE=false; BE=false; PKG=false; INFRA=false; DOCS=false; OTHER=false
FE=false; BE=false; INFRA=false; DOCS=false; OTHER=false
while IFS= read -r file; do
[[ -z "$file" ]] && continue
case "$file" in
@@ -150,23 +145,19 @@ jobs:
FE=true ;;
server/*|scripts/db-*)
BE=true ;;
packages/*)
PKG=true ;;
.gitea/*|k8s/*|Dockerfile*|docker-compose*|docker-entrypoint*|nginx.conf)
INFRA=true ;;
docs/*|*.md|CHANGELOG.md|LICENSE)
DOCS=true ;;
VERSION|scripts/*)
DOCS=true ;; # deploy-only — scripts/db-* already matched as BE above
ci/*)
INFRA=true ;;
FE=true; BE=true ;;
*)
OTHER=true ;;
esac
done <<< "${CHANGED}"
# Docs-only: only docs changed, nothing else
if [[ "$DOCS" == "true" && "$FE" == "false" && "$BE" == "false" && "$PKG" == "false" && "$INFRA" == "false" && "$OTHER" == "false" ]]; then
if [[ "$DOCS" == "true" && "$FE" == "false" && "$BE" == "false" && "$INFRA" == "false" && "$OTHER" == "false" ]]; then
DOCS_ONLY=true
else
DOCS_ONLY=false
@@ -174,7 +165,6 @@ jobs:
echo "frontend=${FE}" >> "$GITHUB_OUTPUT"
echo "backend=${BE}" >> "$GITHUB_OUTPUT"
echo "packages=${PKG}" >> "$GITHUB_OUTPUT"
echo "infra=${INFRA}" >> "$GITHUB_OUTPUT"
echo "docs_only=${DOCS_ONLY}" >> "$GITHUB_OUTPUT"
@@ -182,7 +172,6 @@ jobs:
echo "━━━ Change Detection ━━━"
echo " frontend: ${FE}"
echo " backend: ${BE}"
echo " packages: ${PKG}"
echo " infra: ${INFRA}"
echo " docs_only: ${DOCS_ONLY}"
@@ -330,7 +319,7 @@ jobs:
PGPASSWORD: ${{ secrets.POSTGRES_ADMIN_PASSWORD }}
APP_USER: ${{ secrets.POSTGRES_USER }}
APP_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
DB_NAME: armature_ci
DB_NAME: switchboard_core_ci
run: |
echo "━━━ CI Test Database Setup ━━━"
# Create DB for Go integration tests (admin creds)
@@ -374,93 +363,23 @@ jobs:
PGUSER: ${{ secrets.POSTGRES_ADMIN_USER }}
PGPASSWORD: ${{ secrets.POSTGRES_ADMIN_PASSWORD }}
run: |
psql -c "DROP DATABASE IF EXISTS armature_ci;" postgres
psql -c "DROP DATABASE IF EXISTS switchboard_core_ci;" postgres
echo "✓ Dropped CI test database"
# ── Stage 1d: Surface Test Runners ──────────
# Boots the server in Docker, runs all installed test-runner packages
# via Playwright, and asserts zero failures.
# See: docker-compose.ci.yml, ci/surface-test-driver.js
#
# Runs when: backend, frontend, or packages changed (runners test all tiers).
# Skipped when: only docs changed.
test-runners:
runs-on: ubuntu-latest
needs: [detect-changes]
# DISABLED: Playwright auth bypass not working in Docker (v0.7.5).
# Re-enable once headless cookie injection is solved.
if: false
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Run surface tests (compose)
env:
BUNDLED_PACKAGES: '*'
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: admin
ARMATURE_ADMIN_EMAIL: admin@test.local
run: |
docker compose -f docker-compose.yml -f docker-compose.ci.yml up --build \
--abort-on-container-exit \
--exit-code-from test-runner
- name: Teardown
if: always()
run: docker compose -f docker-compose.yml -f docker-compose.ci.yml down -v
# ── Stage 1e: E2E Smoke Test ───────────────
# Boots the server in Docker, runs Playwright navigation smoke
# test against every surface. Asserts topbar, no JS errors.
# Screenshots saved as artifacts on failure.
# See: docker-compose.ci.yml (e2e-smoke service), ci/e2e-smoke-driver.js
e2e-smoke:
runs-on: ubuntu-latest
needs: [detect-changes]
# DISABLED: Playwright auth bypass not working in Docker (v0.7.5).
# Re-enable once headless cookie injection is solved.
if: false
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Run E2E smoke tests (compose)
env:
BUNDLED_PACKAGES: '*'
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: admin
ARMATURE_ADMIN_EMAIL: admin@test.local
run: |
docker compose -f docker-compose.yml -f docker-compose.ci.yml up --build \
--abort-on-container-exit \
--exit-code-from e2e-smoke
- name: Collect screenshots
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: /tmp/e2e-screenshots/
retention-days: 7
- name: Teardown
if: always()
run: docker compose -f docker-compose.yml -f docker-compose.ci.yml down -v
# ── Stage 2: Build, Database, Deploy ─────────
#
# Depends on all test jobs. Skipped jobs (due to path gating)
# are treated as successful — no blocking.
#
# Always runs — docs are served in-app by the Docs surface.
# Skipped entirely for docs-only changes (nothing to build).
build-and-deploy:
runs-on: ubuntu-latest
needs: [detect-changes, test-go-pg, test-frontend, test-sqlite, test-runners, e2e-smoke]
# Run unless: a needed job failed or the workflow was cancelled.
needs: [detect-changes, test-go-pg, test-frontend, test-sqlite]
# Run unless: a needed job failed, the workflow was cancelled, or it's docs-only.
# Skipped test jobs (path-gated) are fine — they don't block.
# Always deploys — docs are served in-app, VERSION needs a build.
if: |
!cancelled() && !failure()
!cancelled() && !failure() &&
needs.detect-changes.outputs.docs_only != 'true'
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -476,9 +395,9 @@ jobs:
- name: Determine environment
id: setup
run: |
# All environments share one host: armature.DOMAIN
# All environments share one host: switchboard.DOMAIN
# Environments are separated by path prefix (BASE_PATH)
DEPLOY_HOST="armature.${DOMAIN}"
DEPLOY_HOST="switchboard.${DOMAIN}"
echo "DEPLOY_HOST=${DEPLOY_HOST}" >> "$GITHUB_OUTPUT"
if [[ "${{ gitea.event_name }}" == "pull_request" ]]; then
@@ -487,14 +406,13 @@ jobs:
echo "IMAGE_TAG=dev" >> "$GITHUB_OUTPUT"
echo "BASE_PATH=/dev" >> "$GITHUB_OUTPUT"
echo "DEPLOY_SUFFIX=-dev" >> "$GITHUB_OUTPUT"
echo "DB_NAME=armature_dev" >> "$GITHUB_OUTPUT"
echo "DB_NAME=switchboard_core_dev" >> "$GITHUB_OUTPUT"
echo "DB_WIPE=true" >> "$GITHUB_OUTPUT"
echo "REPLICAS=1" >> "$GITHUB_OUTPUT"
echo "MEMORY_REQUEST=256Mi" >> "$GITHUB_OUTPUT"
echo "MEMORY_LIMIT=512Mi" >> "$GITHUB_OUTPUT"
echo "CPU_REQUEST=50m" >> "$GITHUB_OUTPUT"
echo "CPU_LIMIT=250m" >> "$GITHUB_OUTPUT"
echo "BUNDLED_PACKAGES=*" >> "$GITHUB_OUTPUT"
echo "env_label=dev (PR #${{ gitea.event.pull_request.number }})" >> "$GITHUB_OUTPUT"
elif [[ "${{ gitea.ref }}" == refs/tags/v* ]]; then
VERSION="${{ gitea.ref_name }}"
@@ -504,14 +422,13 @@ jobs:
echo "EXTRA_TAG=${VERSION}" >> "$GITHUB_OUTPUT"
echo "BASE_PATH=" >> "$GITHUB_OUTPUT"
echo "DEPLOY_SUFFIX=" >> "$GITHUB_OUTPUT"
echo "DB_NAME=armature" >> "$GITHUB_OUTPUT"
echo "DB_NAME=switchboard_core" >> "$GITHUB_OUTPUT"
echo "DB_WIPE=false" >> "$GITHUB_OUTPUT"
echo "REPLICAS=2" >> "$GITHUB_OUTPUT"
echo "MEMORY_REQUEST=256Mi" >> "$GITHUB_OUTPUT"
echo "MEMORY_LIMIT=512Mi" >> "$GITHUB_OUTPUT"
echo "CPU_REQUEST=100m" >> "$GITHUB_OUTPUT"
echo "CPU_LIMIT=500m" >> "$GITHUB_OUTPUT"
echo "BUNDLED_PACKAGES=notes,chat,chat-core,mermaid-renderer,schedules" >> "$GITHUB_OUTPUT"
echo "is_release=true" >> "$GITHUB_OUTPUT"
echo "env_label=production (${VERSION})" >> "$GITHUB_OUTPUT"
else
@@ -520,14 +437,13 @@ jobs:
echo "IMAGE_TAG=test" >> "$GITHUB_OUTPUT"
echo "BASE_PATH=/test" >> "$GITHUB_OUTPUT"
echo "DEPLOY_SUFFIX=-test" >> "$GITHUB_OUTPUT"
echo "DB_NAME=armature_test" >> "$GITHUB_OUTPUT"
echo "DB_NAME=switchboard_core_test" >> "$GITHUB_OUTPUT"
echo "DB_WIPE=false" >> "$GITHUB_OUTPUT"
echo "REPLICAS=1" >> "$GITHUB_OUTPUT"
echo "MEMORY_REQUEST=256Mi" >> "$GITHUB_OUTPUT"
echo "MEMORY_LIMIT=512Mi" >> "$GITHUB_OUTPUT"
echo "CPU_REQUEST=50m" >> "$GITHUB_OUTPUT"
echo "CPU_LIMIT=250m" >> "$GITHUB_OUTPUT"
echo "BUNDLED_PACKAGES=notes,chat,chat-core" >> "$GITHUB_OUTPUT"
echo "env_label=test (main)" >> "$GITHUB_OUTPUT"
fi
@@ -663,17 +579,14 @@ jobs:
# ── Push to Docker Hub (release only) ────────
- name: Push to Docker Hub
if: steps.setup.outputs.is_release == 'true'
env:
DH_USER: ${{ secrets.DOCKERHUB_USERNAME }}
DH_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
docker tag ${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }} ${DOCKERHUB_IMAGE}:latest
docker tag ${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }} ${DOCKERHUB_IMAGE}:${{ steps.setup.outputs.EXTRA_TAG }}
if [[ -n "${DH_TOKEN}" ]]; then
echo "${DH_TOKEN}" | docker login -u "${DH_USER}" --password-stdin
if [[ -n "${{ secrets.DOCKERHUB_TOKEN }}" ]]; then
echo "${{ secrets.DOCKERHUB_TOKEN }}" | \
docker login -u "${{ secrets.DOCKERHUB_USERNAME }}" --password-stdin
docker push ${DOCKERHUB_IMAGE}:latest
docker push ${DOCKERHUB_IMAGE}:${{ steps.setup.outputs.EXTRA_TAG }}
echo "✓ Pushed to Docker Hub"
else
echo "⚠ Docker Hub credentials not configured, skipping push"
fi
@@ -687,31 +600,31 @@ jobs:
env:
PG_USER: ${{ secrets.POSTGRES_USER }}
PG_PASS: ${{ secrets.POSTGRES_PASSWORD }}
SW_ADMIN_USER: ${{ secrets.ARMATURE_ADMIN_USERNAME }}
SW_ADMIN_PASS: ${{ secrets.ARMATURE_ADMIN_PASSWORD }}
SW_ADMIN_EMAIL: ${{ secrets.ARMATURE_ADMIN_EMAIL }}
SW_ADMIN_USER: ${{ secrets.SWITCHBOARD_ADMIN_USERNAME }}
SW_ADMIN_PASS: ${{ secrets.SWITCHBOARD_ADMIN_PASSWORD }}
SW_ADMIN_EMAIL: ${{ secrets.SWITCHBOARD_ADMIN_EMAIL }}
SEED_USERS: ${{ vars.SEED_USERS }}
ENCRYPTION_KEY: ${{ secrets.ENCRYPTION_KEY }}
run: |
kubectl create secret generic armature-db-credentials \
kubectl create secret generic switchboard-db-credentials \
--namespace=${NAMESPACE} \
--from-literal=POSTGRES_USER="${PG_USER}" \
--from-literal=POSTGRES_PASSWORD="${PG_PASS}" \
--dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic armature-admin \
kubectl create secret generic switchboard-admin \
--namespace=${NAMESPACE} \
--from-literal=username="${SW_ADMIN_USER}" \
--from-literal=password="${SW_ADMIN_PASS}" \
--from-literal=email="${SW_ADMIN_EMAIL}" \
--dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic armature-seed-users \
kubectl create secret generic switchboard-seed-users \
--namespace=${NAMESPACE} \
--from-literal=users="${SEED_USERS}" \
--dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic armature-encryption \
kubectl create secret generic switchboard-encryption \
--namespace=${NAMESPACE} \
--from-literal=ENCRYPTION_KEY="${ENCRYPTION_KEY}" \
--dry-run=client -o yaml | kubectl apply -f -
@@ -726,7 +639,7 @@ jobs:
S3_REGION: ${{ vars.S3_REGION || 'us-east-1' }}
S3_PREFIX: ${{ vars.S3_PREFIX }}
run: |
kubectl create secret generic armature-s3 \
kubectl create secret generic switchboard-s3 \
--namespace=${NAMESPACE} \
--from-literal=S3_ENDPOINT="${S3_ENDPOINT}" \
--from-literal=S3_BUCKET="${S3_BUCKET}" \
@@ -754,14 +667,13 @@ jobs:
STORAGE_CLASS: ${{ vars.STORAGE_CLASS }}
STORAGE_SIZE: ${{ vars.STORAGE_SIZE || '10Gi' }}
STORAGE_BACKEND: ${{ vars.STORAGE_BACKEND || 'pvc' }}
BUNDLED_PACKAGES: ${{ steps.setup.outputs.BUNDLED_PACKAGES }}
run: |
# Render PVC first (must exist before backend references it)
if [[ -n "${STORAGE_CLASS}" ]]; then
envsubst < k8s/storage-pvc.yaml > /tmp/storage-pvc.yaml
fi
envsubst < k8s/armature.yaml > /tmp/armature.yaml
envsubst < k8s/switchboard.yaml > /tmp/switchboard.yaml
envsubst < k8s/middleware-retry.yaml > /tmp/middleware-retry.yaml
envsubst < k8s/ingress.yaml > /tmp/ingress.yaml
@@ -775,7 +687,7 @@ jobs:
echo " ⚠ STORAGE_CLASS not set — file storage disabled"
fi
kubectl apply -f /tmp/armature.yaml
kubectl apply -f /tmp/switchboard.yaml
# Traefik retry middleware (v0.28.8) — requires RBAC for traefik.io CRDs.
# First-time setup: kubectl apply -f k8s/rbac-traefik.yaml (cluster admin)
@@ -794,8 +706,8 @@ jobs:
# Traefik invalidates the entire router if it references a missing middleware,
# which kills all routes for this path prefix.
if [[ "${MW_READY}" == "true" ]]; then
MW_REF="${NAMESPACE}-armature-retry${DEPLOY_SUFFIX}@kubernetescrd"
kubectl annotate ingress "armature${DEPLOY_SUFFIX}" \
MW_REF="${NAMESPACE}-switchboard-retry${DEPLOY_SUFFIX}@kubernetescrd"
kubectl annotate ingress "switchboard${DEPLOY_SUFFIX}" \
"traefik.ingress.kubernetes.io/router.middlewares=${MW_REF}" \
--namespace="${NAMESPACE}" --overwrite
echo " ✓ Ingress annotated with retry middleware"
@@ -806,20 +718,20 @@ jobs:
SUFFIX: ${{ steps.setup.outputs.DEPLOY_SUFFIX }}
ENV: ${{ steps.setup.outputs.ENVIRONMENT }}
run: |
kubectl rollout restart deployment/armature${SUFFIX} -n ${NAMESPACE}
kubectl rollout status deployment/armature${SUFFIX} -n ${NAMESPACE} --timeout=180s
kubectl rollout restart deployment/switchboard${SUFFIX} -n ${NAMESPACE}
kubectl rollout status deployment/switchboard${SUFFIX} -n ${NAMESPACE} --timeout=180s
echo ""
echo "━━━ Pod Status ━━━"
kubectl get pods -n ${NAMESPACE} -l app=armature,env=${ENV}
kubectl get pods -n ${NAMESPACE} -l app=switchboard,env=${ENV}
READY=$(kubectl get deployment armature${SUFFIX} -n ${NAMESPACE} -o jsonpath='{.status.readyReplicas}')
READY=$(kubectl get deployment switchboard${SUFFIX} -n ${NAMESPACE} -o jsonpath='{.status.readyReplicas}')
if [[ "${READY:-0}" -gt 0 ]]; then
echo "✅ Healthy: ${READY} pods ready"
else
echo "❌ Unhealthy: ${READY:-0} pods ready"
kubectl logs -n ${NAMESPACE} -l app=armature,env=${ENV} --tail=30
kubectl logs -n ${NAMESPACE} -l app=switchboard,env=${ENV} --tail=30
exit 1
fi

2638
CHANGELOG.md
View File

File diff suppressed because it is too large Load Diff

64
CONTRIBUTING.md
View File

@@ -1,64 +0,0 @@
# Contributing to Armature
## Development Setup
**Requirements:** Go 1.23+, Node.js 20+ (for frontend tests), SQLite (local dev).
**Build from source:**
```sh
cd server
go build -o armature .
DB_DRIVER=sqlite DATABASE_URL=/tmp/armature.db ./armature
```
**Docker (recommended):**
```sh
docker compose up --build
# open http://localhost:3000 (default login: admin / admin)
```
Data persists in the `sb_data` named volume. Reset with `docker compose down -v`.
## Project Structure
```
server/ Go backend (handlers, store, config, auth, workflow engine)
src/js/ Browser SDK and built-in surface JS
packages/ Extension packages (surfaces, libraries, browser extensions)
build.sh Builds each subdirectory into a .pkg archive
docs/ Documentation markdown (served at /api/v1/docs)
k8s/ Kubernetes manifests and Helm chart
```
## Running Tests
**Backend:** `cd server && go test ./...`
**Frontend:** `node --test src/js/__tests__/`
## Code Conventions
- **Go:** Standard formatting via `gofmt`. Handlers in `server/handlers/`,
persistence in `server/store/`. Database access goes through the store
interface, never directly from handlers.
- **Browser JS:** Vanilla ES modules + Preact via CDN. No build step for
browser code (except the CM6 editor bundle). SDK lives at `src/js/sw/`.
- **Extensions:** IIFE pattern (see `packages/*/js/script.js`). Register with
`sw.renderers` or `sw.slots` via the `sw:ready` event.
## Creating a Package
A package is a ZIP archive (`.pkg`) containing `manifest.json` and optional
`js/`, `css/`, `assets/`, and `script.star` files. The build script
(`packages/build.sh`) automates this. See `docs/PACKAGE-FORMAT.md` for the
full manifest spec and `docs/TUTORIAL-FIRST-EXTENSION.md` for a walkthrough.
## Pull Request Process
1. Create a feature branch from `main`.
2. Make your changes. Keep commits focused.
3. Run both Go and JS test suites and confirm they pass.
4. Submit a PR with a clear description of what changed and why.
5. Address review feedback, then squash-merge when approved.

24
Dockerfile
View File

@@ -1,11 +1,10 @@
# ============================================
# Armature — Unified Dockerfile
# Switchboard Core — Unified Dockerfile
# ============================================
# Stage 1: Build Go backend
# Stage 2: Download JS vendor libs (marked, DOMPurify)
# Stage 3: Build CM6 editor bundle (esbuild)
# Stage 4: Build bundled packages (.pkg archives)
# Stage 5: Production image (nginx + backend)
# Stage 4: Production image (nginx + backend)
#
# Vendor libs are baked in during build so the
# app works in disconnected environments
@@ -20,7 +19,7 @@ COPY server/go.mod server/go.sum* ./
COPY server/ .
COPY VERSION ./
RUN go mod download && go mod tidy && go mod verify
RUN CGO_ENABLED=0 go build -ldflags="-s -w -X main.Version=$(cat VERSION)" -o /bin/armature .
RUN CGO_ENABLED=0 go build -ldflags="-s -w -X main.Version=$(cat VERSION)" -o /bin/switchboard .
# ── Stage 2: Vendor JS libs ─────────────────
FROM node:20-alpine AS vendor
@@ -58,28 +57,19 @@ COPY scripts/build-editor.sh /build/scripts/build-editor.sh
RUN cd /build/src/editor && npm ci --loglevel=warn
RUN sh /build/scripts/build-editor.sh /build/dist
# ── Stage 4: Build bundled packages ─────────
FROM alpine:3 AS packages
RUN apk add --no-cache zip bash
COPY packages/ /packages/
RUN cd /packages && bash build.sh
# ── Stage 5: Production ─────────────────────
# ── Stage 4: Production ─────────────────────
FROM nginx:1-alpine
RUN apk add --no-cache bash git
# Go backend binary
COPY --from=backend /bin/armature /usr/local/bin/armature
COPY --from=backend /bin/switchboard /usr/local/bin/switchboard
COPY --from=backend /app/database/migrations /app/database/migrations
# Frontend static files
COPY src/ /usr/share/nginx/html/
COPY VERSION /VERSION
# Documentation (v0.6.1) — served by Go backend via /api/v1/docs
COPY docs/ /app/docs/
# Inject version and build hash into index.html and sw.js at build time
RUN APP_VERSION=$(cat /VERSION | tr -d '[:space:]') && \
BUILD_HASH=$(find /usr/share/nginx/html/js -name '*.js' -exec md5sum {} + | sort | md5sum | cut -c1-8) && \
@@ -95,8 +85,8 @@ COPY --from=vendor /vendor/katex/katex.min.css /usr/share/nginx/html/vendor/kate
COPY --from=vendor /vendor/katex/fonts/ /usr/share/nginx/html/vendor/katex/fonts/
COPY --from=cm6-build /build/dist/ /usr/share/nginx/html/vendor/codemirror/
# Bundled packages (v0.3.8) — auto-installed on first run
COPY --from=packages /dist/ /app/bundled-packages/
# Builtin extensions (seeded into DB on startup by backend)
COPY extensions/builtin/ /app/extensions/builtin/
# nginx config (template — envsubst injects BASE_PATH at runtime)
COPY nginx.conf.template /etc/nginx/templates/default.conf.template

65
Dockerfile.builder
View File

@@ -1,65 +0,0 @@
# ============================================
# Armature — Builder Image
# ============================================
# Pre-caches Go modules and Node dependencies
# for faster custom builds. Use as a base in
# your own Dockerfile to skip dependency
# download on every build.
#
# Usage:
# FROM ghcr.io/armature/builder:latest AS go-builder
# COPY server/ /app/
# RUN cd /app && go build -o /bin/armature .
#
# Or build this image locally:
# docker build -f Dockerfile.builder -t armature-builder .
# ============================================
# ── Go module cache ─────────────────────────
FROM golang:1.23-bookworm AS go-cache
WORKDIR /cache
COPY server/go.mod server/go.sum* ./
RUN go mod download && go mod verify
# ── Node dependency cache ───────────────────
FROM node:20-alpine AS node-cache
WORKDIR /cache
# Editor bundle dependencies
COPY src/editor/package*.json ./editor/
RUN cd editor && npm ci --loglevel=warn
# Vendor libs (same versions as production Dockerfile)
RUN npm pack marked@16.3.0 && \
npm pack dompurify@3.2.4 && \
npm pack mermaid@11.4.1 && \
npm pack katex@0.16.11 && \
mkdir -p /cache/vendor-tarballs && \
mv *.tgz /cache/vendor-tarballs/
# ── Final builder image ────────────────────
FROM golang:1.23-bookworm
# Pre-install build tools
RUN apt-get update && apt-get install -y --no-install-recommends \
zip \
&& rm -rf /var/lib/apt/lists/*
# Go module cache (populated)
COPY --from=go-cache /go/pkg/mod /go/pkg/mod
# Node dependencies (for editor bundle builds)
COPY --from=node-cache /cache/editor/node_modules /cache/editor/node_modules
# Vendor lib tarballs (avoid re-download)
COPY --from=node-cache /cache/vendor-tarballs /cache/vendor-tarballs
# Node.js for frontend builds
COPY --from=node-cache /usr/local/bin/node /usr/local/bin/node
COPY --from=node-cache /usr/local/lib/node_modules /usr/local/lib/node_modules
RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm
WORKDIR /build
LABEL org.opencontainers.image.title="Armature Builder"
LABEL org.opencontainers.image.description="Pre-cached build dependencies for faster custom Armature builds"

25
README.md
View File

@@ -1,11 +1,11 @@
# Armature
# Switchboard Core
A self-hosted extension platform. Identity, teams, permissions, workflows,
and a package system. Everything else ships as installable extensions.
## What This Is
Armature is the kernel. It provides the primitives that extensions
Switchboard Core is the kernel. It provides the primitives that extensions
build on: users, teams, groups, scoped credentials, a Starlark sandbox,
workflow orchestration, notifications, and a package installer. It does not
include AI chat, providers, personas, or any domain-specific features —
@@ -22,21 +22,12 @@ those are all extension packages.
## Quick Start
```bash
# Docker (recommended)
docker compose up --build
# → http://localhost:3000 (admin/admin)
# Or from source
git clone <repo-url> && cd armature
git clone <repo-url> && cd switchboard-core
cp server/.env.example server/.env # edit DB credentials
cd server && go run .
# → http://localhost:8080
```
Bundled packages (workflows, surfaces, task manager) are auto-installed on
first boot. See [Distribution Guide](docs/DISTRIBUTION.md) for production
deployment and customization.
## Kernel Features
- **Auth**: Builtin password, mTLS (client cert), OIDC (Keycloak et al.)
@@ -52,19 +43,15 @@ deployment and customization.
## Documentation
- [Distribution Guide](docs/DISTRIBUTION.md) — Docker, bundled packages, builder image, production deployment
- [Architecture](docs/ARCHITECTURE.md) — kernel components and design reasoning
- [Roadmap](ROADMAP.md) — current status and planned milestones
- [Changelog](CHANGELOG.md) — version history
## Project Status
**v0.5.0** — Realtime pub/sub primitive, dialog audit, and admin permissions
UI. Extensions can now publish events to WebSocket channels via Starlark;
clients subscribe with `sw.realtime.subscribe()`. Admin Packages page gains
per-permission grant/revoke controls and status badges. See
[ROADMAP.md](ROADMAP.md) for the full journey from v0.1.0 kernel extraction
through v0.3.x workflows, v0.4.x Notes surface, to v0.5.x realtime and chat.
**v0.1.0** (in progress) — kernel extracted from chat-switchboard v0.38.5.
~44K lines of chat/AI code removed, 27 kernel tables, 20 store interfaces.
See [ROADMAP.md](ROADMAP.md) for details.
## License

405
ROADMAP.md
View File

@@ -1,311 +1,184 @@
# Armature — Roadmap
# Switchboard Core — Roadmap
Self-hosted extensible platform kernel. Auth, identity, packages, Starlark
sandbox, storage, realtime, and ops are kernel primitives. Everything else
is an extension.
## Current: v0.2.0 — SDK & Triggers
**Kernel capabilities:** Auth (builtin/mTLS/OIDC) · Users/teams/groups/RBAC ·
Surfaces/extensions/libraries/workflows · Starlark sandbox (capability-gated) ·
Object storage (PVC/S3) + ext_data tables · WebSocket hub + realtime pub/sub ·
Audit log · Notifications · Scheduled tasks · Cluster registry + HA ·
Extension composability (slots/contributes/cross-package calls)
Fork of chat-switchboard, gutted to a pure extension platform. All AI/chat
features removed from the kernel. What remains is the minimum viable
platform that extensions build on.
---
### Retained kernel capabilities
## Completed — v0.6.x through v0.8.x
- **Auth**: builtin (simple), mTLS, OIDC
- **Identity**: users, teams, groups, permissions (RBAC)
- **Packages**: surfaces, extensions, libraries, workflows
- **Starlark sandbox**: capability-gated modules
- **Storage**: object storage (PVC, S3), ext_data tables
- **Realtime**: WebSocket hub, presence, multi-replica HA
- **Ops**: audit log, notifications, maintenance goroutine
All completed work is documented in `CHANGELOG.md`.
### Phase 0 (complete)
### v0.6.x — MVP + Hardening
| Step | Status | Description |
|------|--------|-------------|
| 1. Module rename | ✅ | `chat-switchboard` → `switchboard-core` |
| 2. Delete packages | ✅ | 15 Go packages, 29 handler files removed |
| 3. Gut stores/models | ✅ | 40 → 20 store interfaces, kernel-only models |
| 4. Fresh migrations | ✅ | 9 files × 2 dialects, 27 tables |
| 5. Fix compilation | ✅ | `go build ./...` clean, 300+ stale route lines cut |
| 6. Fix tests | ✅ | 8 test packages pass, ~12K stale test lines pruned |
| 7. Frontend gut | ✅ | Shell + SDK only, 50+ files of chat/notes/projects code removed |
| 8. New ICD | ✅ | Full OpenAPI 3.0.3 spec — 160 operations across 22 tag groups |
| 9. CI/CD + Dockerfile | ✅ | Single unified image, FE/BE split removed, DB names updated, k8s var alignment fixes (resource quantities, image name, rollout deployment name) |
| 10. Smoke test | ✅ | K8s deploy live at switchboard.gobha.ai/test, nginx BASE_PATH fixed, login→admin flow verified, branding updated |
| Version | Title |
|---------|-------|
| v0.6.0 | Cluster Registry + HA |
| v0.6.1 | Backup/Restore + Docs |
| v0.6.2 | Docs Polish + Dynamic OpenAPI |
| v0.6.3 | Dead Code Sweep + Registry Fix |
| v0.6.4 | Admin Health/Metrics + Cluster Merge |
| v0.6.5 | Renderer Pipeline + Docs Rewrite |
| v0.6.6 | Final Hardening |
| v0.6.7 | Native mTLS |
| v0.6.8 | Rebrand + Cookie Fix |
| v0.6.9 | Session Lifetime Config |
| v0.6.10 | Viewport Foundation |
| v0.6.11 | CSS Deduplication |
| v0.6.12 | Extension CSS Isolation |
| v0.6.13 | Responsive & Spacing |
| v0.6.14 | Visual Polish |
| v0.6.15 | User Display Audit |
| v0.6.16 | Usability Survey Gate |
| v0.6.17 | Bug Fixes & Welcome |
| v0.6.18 | CI Bundle Wiring |
## v0.2.x — SDK & Triggers
### v0.7.x — Test Infrastructure + Quality Gate
The contract that extensions build against. Three trigger primitives,
SDK stabilization, and the first rebuilt extension (tasks).
| Version | Title |
|---------|-------|
| v0.7.0 | Shell Contract + Surface Audit + Rebrand |
| v0.7.1 | Surface Runner Framework |
| v0.7.2 | Package Runners + CI Gate |
| v0.7.3 | Extension Shell Migration |
| v0.7.4 | Documentation + Surface Work |
| v0.7.5 | Headless E2E + CI Gate |
| v0.7.6 | Code Hygiene + Test Coverage |
| v0.7.7 | API Tokens + Extension Permissions |
| v0.7.8 | Bug Fixes & Admin Gaps |
| v0.7.9 | Workflow Independence Audit |
| v0.7.10 | Workflow Handoff + Assignment UI |
| v0.7.11 | Query & HTTP Ergonomics |
| v0.7.12 | Concurrent Execution Primitive |
### v0.2.0 — RBAC + Settings Cascade (complete)
### v0.8.x — Storage Primitives + Composability (Kernel Complete)
| Step | Status | Description |
|------|--------|-------------|
| Admin → RBAC group | ✅ | `surface.admin.access` permission + Admins system group replaces `role == "admin"` checks. Admin bypass removed from permission middleware. |
| Settings cascade | ✅ | `user_overridable` flag, three-tier resolution (global → team → user), team settings API |
| ~~Settings override model~~ | ✅ | Shipped with settings cascade above |
| Version | Title |
|---------|-------|
| v0.8.0 | `files` Module |
| v0.8.1 | `workspace` Module |
| v0.8.2 | Capability Negotiation |
| v0.8.3 | Vector Column Type |
| v0.8.4 | Documentation Refresh + Surface Sizing Fix |
| v0.8.5 | Extension Composability |
### v0.2.1 — Default Surface + ICD
---
| Step | Status | Description |
|------|--------|-------------|
| Default surface routing | ✅ | `/` redirects to configurable default surface. No surfaces → admin. First install becomes default. Changeable in admin settings. |
| ICD (API contract) | ✅ | Full OpenAPI 3.0.3 spec — 160 operations, 22 tag groups, reusable component schemas. Served at `/api/docs`. |
### v0.9.x — Multi-Surface Packages + Workflow Redesign
### v0.2.2 — Event Bus + Triggers
| Version | Title |
|---------|-------|
| v0.9.0 | Multi-Surface Packages |
| v0.9.1 | Server-Side Sub-Path Routing |
| v0.9.2 | Converter Consolidation |
| v0.9.3 | Team User Roles |
| v0.9.4 | Package Adoption + Roles |
| v0.9.5 | Typed Forms SDK |
| v0.9.6 | Stage Mode Collapse |
| v0.9.7 | Workflow Starlark Write Ops |
| v0.9.8 | Routing SDK Primitive |
| v0.9.9 | Surface Access via Roles |
| Step | Status | Description |
|------|--------|-------------|
| Event bus subscriptions | ✅ | Extensions register event patterns in manifest. Wired via `bus.Subscribe()` on startup. Async handler invocation. |
| Webhook triggers | ✅ | Inbound HTTP at `/api/v1/hooks/:package_id/:slug`. HMAC-SHA256 verification. Synchronous Starlark handler response. |
| Scheduled tasks | ✅ | User-created cron tasks with restricted sandbox (no raw HTTP, no DB table creation). Runs as creator identity. Templates from extensions. Dedicated schedules API. |
| Trigger admin API | ✅ | CRUD for triggers + schedules. Enable/disable, execution logs, per-package listing. |
---
### v0.2.3 — SDK + Task Extension
## Current: v0.10.x — Panels + Composable Layout
| Step | Status | Description |
|------|--------|-------------|
| SDK stabilization | ✅ | `sw.api.ext()`, `sw.storage`, `sw.theme.tokens`, `sw.ui`, `sw.slots`, `sw.actions` — six new SDK modules for extension development |
| Task extension | ✅ | Full task surface rebuilt as Starlark extension: CRUD API, kanban/list views, event triggers, webhook integration, notifications on completion |
Panels are a new kernel rendering tier between surfaces (full-page) and
block renderers (inline). They solve composable companion views — e.g.,
a notes reference panel inside chat. Design doc: `docs/DESIGN-panels.md`.
### v0.2.4 — Shell Navigation + Schedules
**v0.10.0 — Panel Manifest + Lifecycle** *(completed)*
| Step | Status | Description |
|------|--------|-------------|
| SDK Topbar | ✅ | `sw.shell.Topbar` — composable navigation bar (title + extension slot + bell + user menu). Surfaces get consistent nav for free. |
| Schedules surface | ✅ | New `packages/schedules/` wrapping kernel cron API. Table view, cron preview, enable/disable, manual run, execution logs. |
| Manifest icons | ✅ | `icon` field in manifest.json (emoji). Surfaces API returns icon. UserMenu renders per-surface icons. |
| UserMenu cleanup | ✅ | Removed dead Chat/Notes/Projects links. Menu driven by surfaces API. Core surfaces filtered. |
| isAdmin RBAC fix | ✅ | `can.js` isAdmin() now checks `surface.admin.access` grant instead of deprecated role column. |
Manifest `panels` field (provider map + consumer array, soft dependency).
`resolvePanels()` at surface load. `sw.panels` SDK module with full
lifecycle API. Lazy JS loading with module caching. 10 new tests.
### v0.2.5 — UI Polish + Dead Code Audit
| Version | Title |
|---------|-------|
| v0.10.1 | FloatingPanel Primitive |
| v0.10.2 | Docked Panels + Mode Transitions |
| v0.10.3 | Panel Communication Patterns |
| v0.10.4 | Reference Panel: Notes (basic) |
| Step | Status | Description |
|------|--------|-------------|
| UI bug pass | ✅ | Reviewed admin, settings, login, welcome surfaces in light/dark themes. Fixed settings crash (models API undefined). Removed dead chat settings from both user and admin settings. |
| Dead code sweep | ✅ | Removed ~700 lines: orphaned CSS, dead Go types, stale event code, unused test helpers, dead settings UI (chat defaults, system prompt, default model, policies, web search, compaction, memory). |
| Template cleanup | ✅ | Removed stale CSS link tags and orphaned chat-pane.html. Updated doc comments throughout. |
| Package proof-of-concept status | ✅ | Created README.md for tasks + schedules with graduation criteria. Added missing manifest icons. |
| Welcome surface | ✅ | New fallback surface when no extensions installed. Topbar + welcome card with admin link. Replaces `/admin` as final redirect target. |
| Default surface routing | ✅ | Resolution chain: user preference → global config → first extension → `/welcome`. Users can set personal default in Settings > General. Admin sets global default in Admin > Settings. |
| Admin navigation | ✅ | Replaced Back button with UserMenu in admin topbar. Eliminates back-button infinite loop. |
---
### v0.2.6 — Admin Settings Audit
### v0.11.x — Notes Reference Extension
| Step | Status | Description |
|------|--------|-------------|
| Admin settings E2E | ✅ | All 7 admin settings sections verified (default surface, registration, banner, message bar, footer, vault, email). Dead code removed: `sectionCategory()` pruned of AI/routing/channel vestiges, `PublicSettings()` stripped of chat-era fields (system_prompt, retention_ttl, paste_to_file, allow_user_personas), dead `PolicyDefaults` removed (allow_raw_model_access, default_model), dead policy lookups removed (kb_direct_access), test seed data cleaned. |
| Packages surface | ✅ | Package list loads (17 total/17 enabled), type filters work, enable/disable visible, settings/export buttons present, core package (admin) protected. Removed dead `chat` from CORE_IDS. |
Notes becomes the first reference extension — a production-quality
knowledge base that exercises every kernel primitive. The UI/UX redesign
is front-loaded as v0.11.0 so every subsequent feature version builds on
a clean visual foundation. Design doc: `docs/DESIGN-notes-v011x.md`.
### v0.2.7 — User Settings Audit
| Version | Title |
|---------|-------|
| v0.11.0 | UI/UX Foundation — visual redesign |
| v0.11.1 | Deep Folders + Navigation |
| v0.11.2 | Wikilinks + Backlinks |
| v0.11.3 | Live Preview + Rich Editing |
| v0.11.4 | Note Sharing + Permissions |
| v0.11.5 | Graph + Outline Hardening |
| v0.11.6 | Quick Switcher + Commands |
| v0.11.7 | Daily Notes + Templates |
| v0.11.8 | Transclusion + Embeds |
| v0.11.9 | Composability: Slots + Actions |
| v0.11.10 | Panel Enhancement + Quality Gate |
| Step | Status | Description |
|------|--------|-------------|
| User settings E2E | ✅ | All 6 user settings sections verified (General, Appearance, Profile, Teams, Connections, Notifications). Dead code removed: BYOK nav section + state, personas gate filter, Message Font Size slider, `auth.permissions.changed` listener. localStorage key renamed `cs-appearance` → `sb-appearance` with one-time migration. |
| Visibility gating | ✅ | Dead BYOK/personas policy lookups removed from bootstrap and permissions handlers. `allow_user_byok` removed from `PublicSettings`. `PolicyDefaults` cleaned of `allow_user_byok` and `allow_user_personas`. Dead `msgFont` early-apply removed from base template. Stale policy-gating test assertions replaced. No empty nav sections remain. |
---
### v0.2.8 — Team Admin Settings Audit (Pass 1)
### v0.12.x — Chat Reference Extension
| Step | Status | Description |
|------|--------|-------------|
| Team admin E2E | ✅ | Audited team member management, settings cascade, role assignment. Removed dead code: `HasPrivateProviderRequirement` (BYOK vestige), `UserRole` on `TeamMember` (deprecated role column), `allow_team_providers` policy default, dead personas/providers/models ICD tests. |
| Workflow stage UI cleanup | ✅ | Removed dead personas dropdown, `history_mode` selector, stale `chat_only` mode. Updated `STAGE_MODES` to match backend CHECK constraint (`form_only`, `form_chat`, `review`, `custom`). Removed stale comments referencing deleted files. |
Chat becomes the second reference extension — human-to-human messaging
built entirely on Armature's extension architecture. Where notes proved
surfaces, panels, and storage, chat proves **realtime**, **cross-package
composition**, and **extensible data models**. Chat is human-to-human
first; AI participants arrive via `llm-bridge` (v0.13.x) extending chat
through folder attributes and slot contributions.
Design doc: `docs/DESIGN-chat-v012x.md`.
### v0.2.9 — Builtin Extension Retirement
| Version | Title |
|---------|-------|
| v0.12.0 | UI/UX Foundation |
| v0.12.1 | Conversation Folders + Attributes |
| v0.12.2 | Reactions + Threads + Pins |
| v0.12.3 | Rich Compose + Attachments |
| v0.12.4 | @Mentions + Notifications |
| v0.12.5 | Link Previews + Message Formatting |
| v0.12.6 | Conversation Themes + Personality |
| v0.12.7 | Composability: Slots + Actions |
| v0.12.8 | Panels + Quality Gate |
| Step | Status | Description |
|------|--------|-------------|
| Retire builtin seeder | ⬚ | Stop auto-seeding chat-centric extensions (csv-table, diff-viewer, js-sandbox, katex-renderer, mermaid-renderer, regex-tester). Remove `SeedBuiltinPackages`, Dockerfile COPY, and `extensions/builtin/` directory. These extensions are not OBE — they're dormant until a chat surface exists to consume them. |
| Convert to regular packages | ⬚ | Repackage the 6 extensions as standard `.pkg` archives in `packages/`. Add chat dependency metadata to manifests so they can be installed when the chat extension ships post-MVP. No auto-install — explicit install only, matching the distribution model. |
---
## v0.3.x — Workflow Architecture
### v0.13.x — Reference Libraries + Extensions
Workflows are the core platform capability. This series implements the
full multi-step automation system with team role integration and
finalizes the extension lifecycle model.
Custom public root surface unblocks everything — anonymous visitors,
landing pages, and the package registry. `llm-bridge` is the key
library delivery — extends both notes and chat through composability
primitives, and introduces the **tool meta-tool** pattern:
`sw.actions.list()` becomes the LLM tool registry, so installing an
extension = granting AI a new capability. Zero configuration.
`armature.run` deployment is the dogfood gate at the end of the series.
### v0.3.0 — Workflow Design + Schema
| Version | Title |
|---------|-------|
| v0.13.0 | Custom Public Root Surface |
| v0.13.1 | `vector-store` Library |
| v0.13.2 | `llm-bridge` Core Library |
| v0.13.3 | `llm-bridge` → Chat: Multi-Persona Context + Tool Meta-Tool |
| v0.13.4 | `llm-bridge` → Notes: AI Toolbar Actions |
| v0.13.5 | `file-share` Extension |
| v0.13.6 | Package Registry Extension |
| v0.13.7 | `code-workspace` Extension |
| v0.13.8 | `image-gen` + `image-edit` Extensions |
| v0.13.9 | Tool Meta-Tool Hardening + Scoping |
| v0.13.10 | `armature.run` Deployment |
| v0.13.11 | Integration Quality Gate |
| Step | Status | Description |
|------|--------|-------------|
| Extension lifecycle | ⬚ | Define permanent vs PoC extensions. Package graduation criteria. Dependency policy. What ships with core vs what's installed separately. |
| Workflow design session | ⬚ | Define what "workflow" means in the extension-first model. Determine if the existing `workflows` table/handler survives, gets rebuilt, or gets removed. Document the Starlark contract for multi-step automation. See `docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md`. |
| Team roles | ⬚ | Different roles per team responsible for different workflow stages. Role-based stage assignment, multi-party validation (2-party sign-off at stage boundaries). |
| Trigger composition model | ⬚ | How do triggers, schedules, and workflows compose? Event chains, conditional branching, error handling. Design doc before code. |
| Settings audit pass 2 | ⬚ | Focused audit of team admin + user settings for workflow/team-role changes applied in this series. Validates new team role UI, stage assignment settings, workflow preferences. |
---
## v0.4.0 — Notes Surface
### v0.14.x — Sidecar Tier
Obsidian-style rich-text notes rebuilt as an installable surface package.
Zero platform special-casing. Proves the full extension stack E2E.
Connect-inward model: sidecars connect TO the kernel (no K8s RBAC, no
service mesh, no DNS discovery). Instance sidecars (shared infrastructure)
and user sidecars (personal local tools). Design doc:
`docs/DESIGN-sidecar-v014x.md`.
- Notes as `.pkg` archive
- Rich text editor (ProseMirror or similar)
- Folder tree, backlinks, tags — all extension-provided
- Markdown import/export
| Version | Title |
|---------|-------|
| v0.14.0 | Sidecar Registry + Auth |
| v0.14.1 | Capability Registration + Execution |
| v0.14.2 | Kernel API Access + Event Bus |
| v0.14.3 | Manifest Integration + Admin Polish |
| v0.14.4 | Reference Sidecar (`armature-embed`) + Instance Gate |
| v0.14.5 | User Sidecars + Reference (`user-bridge`) + User Gate |
## v0.5.0 — MVP
---
Extension and operations tracks converge. First externally usable release.
### v0.15.x — Polish + Stability
- Package registry (browse, install, update, uninstall)
- Package distribution model (no auto-install; explicit install only)
- Health monitoring dashboard
- Backup/restore tooling
- Documentation site
| Version | Title |
|---------|-------|
| v0.15.0 | Native Dialog Audit |
| v0.15.1 | Versioned Migrations + `armature migrate` CLI |
| v0.15.2 | Pre-1.0 Schema Freeze + Upgrade Path Validation |
## Post-MVP
---
### v1.0.0 — Stable Release
Kernel API surface frozen. Extensions are the product.
Gate criteria:
- All kernel Starlark modules documented with examples
- All `api_routes` covered by OpenAPI spec
- Upgrade path tested from v0.8.0 → v1.0.0
- Notes reference extension shipped with all v0.11.x features
- Notes UI/UX reviewed against v0.11.0 design principles — no placeholder UI
- Chat reference extension shipped with all v0.12.x features
- `llm-bridge` extends both notes and chat through composability
- Tool meta-tool demonstrated: AI uses 3+ extension actions in a
single conversation turn
- Multi-persona context archetypes demonstrated
- Admin safety rails validated
- At least 2 panels consumed cross-package
- At least 2 slot contributions per host surface demonstrated
- Note and conversation sharing functional end-to-end
- Headless E2E green on PG + SQLite
- `armature-ca.sh` + mTLS deployment guide
- Single-binary + Docker + K8s deployment paths documented
- Instance sidecar (`armature-embed`) deployed and functional
- User sidecar (`user-bridge`) functional on macOS/Linux/Windows
- Token + mTLS sidecar auth both tested
- `armature migrate` CLI functional
---
## Post-1.0 Horizon
These are candidates, not commitments. Each requires a design doc.
- **Federation** — cross-instance package sharing, identity federation
- **Package marketplace** — signing, review, discovery registry
- **Desktop app** — Tauri wrapper for local-first deployment
- **Offline/sync** — SQLite-first with PG sync for field deployments
- **Multi-tenant SaaS mode** — tenant isolation at the team boundary
---
## Design Principles
| Principle | Implication |
|-----------|-------------|
| Extensions are the product | Chat, tasks, LLM, vector search, file sharing — all extensions. Zero kernel awareness of domain logic. |
| Kernel stays thin | New kernel primitives require justification. If it can be an extension, it must be. |
| Progressive enhancement | Every feature works on SQLite. PG adds performance. pgvector adds native vectors. S3 adds scalable storage. |
| KISS-first | No unnecessary dependencies. Preact+htm (3KB), single binary, dual-DB from one codebase. |
| Changeset discipline | Each CS independently CI-green. Design docs as implementation contracts. |
| Pre-1.0 migration freedom | Schema changes fold into existing migrations. Post-1.0: proper versioned migrations. |
---
- Chat extension (provider registry, streaming, personas, tool system)
- Rich media extensions: image generation, code sandbox, STT/TTS
- Desktop app (Tauri or Electron)
- Sidecar tier: container-based extensions
- Federation: cross-instance package sharing
- Plugin marketplace with signing and review
- Mermaid diagrams extension (nice-to-have)
## Design Decisions Log
| Decision | Rationale |
|----------|-----------|
| Tasks → extension | Three trigger primitives replace the monolithic scheduler. |
| Sessions removed | Workflow instances with dedicated storage replace kernel sessions. |
| `custom` stage mode | Delegates to a surface package, proving extension composability. |
| Providers removed from kernel | Connections + Starlark `provider.complete` as the interface. |
| Kernel permissions simplified | 6 platform permissions. Extensions define their own. |
| Preact+htm retained | 3KB runtime, no build step, KISS. |
| Single Docker image | Go binary + assets + migrations. |
| Admin → RBAC group | Grant check replaces role check. |
| Settings cascade | Scope auth + `user_overridable`. Two orthogonal axes. |
| Chat as extension, not kernel | Zero kernel awareness. Proves extensibility thesis. |
| PG as consensus layer | UNLOGGED node_registry + LISTEN/NOTIFY. No etcd/Consul/Redis. |
| Two trigger tiers | Extension-declared (full sandbox) vs user ad-hoc (restricted). |
| Two-slot topbar model | Left slot (title/branding) + center slot (`flex: 1`, tabs/pickers). |
| `db` module is the structured store | No separate KV primitive. Extensions declare tables. |
| `files` module rides ObjectStore | No new kernel tables. Metadata as companion objects. |
| `workspace` for real filesystem | Flat blob store can't serve git/compilers/ffmpeg. Managed disk paths. |
| Capability negotiation at install | Fail loud with actionable message, not silently at runtime. |
| Vector column with three-tier fallback | Works everywhere, works fast with pgvector. |
| Sidecar deferred to v0.14.x | HTTP module covers external APIs. Sidecars connect inward (no k8s RBAC needed). |
| Workflow redesign before reference extensions | Clean up debt and promote primitives before building on top. |
| Panels as kernel primitive (v0.10.x) | Z-index coordination, drag/resize, layout negotiation are kernel concerns. |
| UI/UX redesign as first version in each reference series | Every feature builds on the visual foundation. Front-loading avoids double work. |
| Notes as dedicated v0.11.x series (11 versions) | Complex enough to need changeset discipline. Each version independently shippable. |
| Notes before chat | Notes is simpler (no realtime) and proves storage/rendering/composability. Chat adds realtime + llm-bridge story. |
| Chat human-to-human first (v0.12.x) | AI is an extension concern. Keeps chat testable and usable standalone. |
| Chat as dedicated v0.12.x series (9 versions) | Second reference extension. Proves realtime, extensible data models, cross-package composition. |
| Folder attributes as extension bridge | Chat stores attributes it doesn't understand. llm-bridge contributes definitions. Zero coupling. |
| Action registry as tool registry (meta-tool) | Dynamic, zero-config AI tool-use. Installed extensions = AI capabilities. Uniquely Armature. |
| Layered prompt architecture (6 layers) | Admin safety not overridable. Clear separation: platform → space → character → context → tools → user. |
| Personas in llm-bridge, not chat | Chat sees AI as just another participant_type. Persona identity is llm-bridge's concern. |
| llm-bridge after notes + chat (v0.13.x) | Proves the composability hooks work without being designed for a specific consumer. |
| Custom public root first in v0.13.x | Unblocks registry, landing page, and armature.run. Small kernel change, high leverage. |
| Package registry as reference extension | Proves public surfaces work. Needs files module for .pkg storage. Self-referential: Armature's registry runs on Armature. |
| armature.run as dogfood gate | If armature.run can't run on Armature, the platform isn't ready. Issues feed into quality gate fixes. |
| Sidecar connect-inward | No K8s RBAC, no service mesh, no DNS discovery. Works on any deployment target. |
| Sidecar HTTP/JSON, not gRPC | KISS. 4-endpoint contract. Every language has HTTP. |
| User sidecars in v0.14.5 | Prove instance contract first, extend to users with same mechanism. |
| Three-layer user sidecar RBAC | Admin controls who + what, user controls visibility. Defense in depth. |
| Separate polish (v0.15.x) | Security-critical sidecar infra and quality-of-life polish shouldn't share focus. |
| Tasks → extension | Scheduler was the most entangled kernel component (~3,400 lines). Rebuilding as extension validates the trigger system and removes the worst compilation debt. Three trigger primitives (time, webhook, event) replace the monolithic scheduler. |
| Sessions removed | Channel-based sessions coupled to deleted chat system. Workflow instances need new storage model — either ext_data tables or a dedicated kernel table. |
| `chat_only` → `custom` | Stage mode `chat_only` implied chat as a kernel concept. Renamed to `custom` which delegates to a surface package, proving extension composability. |
| Providers removed from kernel | Provider configs, model catalog, routing policies — all moved to extension track. Kernel provides credential storage (connections) and the Starlark `provider.complete` module as the interface. |
| Kernel permissions simplified | From 16 chat-centric permissions to 6 platform permissions. Extensions define their own capability requirements in manifests. |
| Preact+htm retained | 3KB runtime, no build step, works for extension authors without bundler config. KISS. |
| Single Docker image | Drop the frontend/backend split. Go binary + assets + migrations in one image. Simpler deployment, fewer moving parts. |
| Admin → RBAC group | The `role` column is pre-RBAC. v0.2.0 replaces it with a seeded "Admins" group + `surface.admin.access` grant. All users auto-join "Everyone" group. Admin middleware becomes a grant check, not a role check. |
| Settings cascade | RBAC controls scope auth (who can set at what level). `user_overridable` flag controls whether lower scopes can override higher. Two orthogonal axes, composes cleanly with extension manifests. |
| No new migrations pre-MVP | Edit existing migration SQL files in place. No migration chains until schema is in production. |
| Notes over Editor | First surface is Obsidian-style notes (rich text, folders, backlinks) instead of a code editor. Notes is a stronger E2E proof — it exercises ext_data, storage, and the SDK more fully than a pure-browser CM6 editor. |
| No built-in auto-install | Extensions ship in the repo but are not auto-installed. Distribution model TBD — explicit install only. Keeps the kernel clean and avoids opinionated defaults. |
| Chat → post-MVP | Chat extension (providers, streaming, personas) is valuable but not MVP-critical. The platform must prove itself with simpler surfaces first. Chat moves to post-MVP track. |
| Two trigger tiers | Event + webhook triggers are extension-declared (manifest contract, full sandbox). Scheduled tasks are user-created ad-hoc (restricted sandbox — no raw HTTP, no DB table creation, connections-only outbound). Separation keeps extension contracts static and user automation safe. |
| Scheduled task identity | Tasks run as their creator (RBAC-scoped). Admin-created tasks can opt into system context. Creator deactivation pauses the schedule. Ensures audit trail and permission boundaries. |

132
TURNOVER.md Normal file
View File

@@ -0,0 +1,132 @@
# Switchboard Core — Session Turnover
**Date**: 2026-03-26
**Author**: Jeffrey Smith (jasafpro@gmail.com)
**Session**: v0.2.0 PR 1 — Full RBAC migration
---
## What Was Done
### PR 1: Admin → RBAC group migration — COMPLETE
Branch `feat/admin-rbac-migration` (PR #1), 3 commits:
1. **Admin → RBAC group migration** — `surface.admin.access` permission +
Admins system group replaces hardcoded `role == "admin"` checks. Admin
bypass removed from `RequirePermission`. Bootstrap/seed/OIDC/admin handlers
sync group membership.
2. **Remove token budgets + allowed models from groups** — Provider-era cruft:
`token_budget_daily`, `token_budget_monthly`, `allowed_models` columns,
`ResolveTokenBudget()`, `ResolveModelAllowlist()`, and all store/handler/UI
code deleted. -283 lines.
3. **Drop `users.role` column** — Full RBAC. Zero magic roles. The `role` column,
`UserRoleAdmin`/`UserRoleUser` constants, `CountByRole()`, `DefaultRole` config,
`role` in JWT claims, and all role-based shortcuts removed. Everyone group
membership is now explicit (all users added on create). 28 files changed.
**Net result: -364 lines across 3 commits.**
### Architecture after this PR:
- **Everyone group** (`00000000...0001`): all users explicitly added on creation.
Carries `extension.use`, `workflow.submit`.
- **Admins group** (`00000000...0002`): carries all 7 kernel permissions including
`surface.admin.access`. Not special-cased — just a group with permissions.
- **Permission resolution**: union of all group memberships. No implicit groups,
no role shortcuts, no admin bypass.
- **Admin middleware**: checks `surface.admin.access` grant via `resolveAndCachePerms`.
- **JWT claims**: `user_id` + `email` only. No role.
- **OIDC**: `isIdPAdmin()` maps IdP role claim → Admins group membership.
No `role` column writes.
- **User creation paths**: builtin register, OIDC auto-provision, mTLS auto-provision,
admin create, bootstrap, seed — all call `EnsureEveryoneGroup()`.
---
## Known Issues
1. **Frontend test job**: FE test suite may have remaining stale references
from Phase 0 gut (pre-existing)
2. **Traefik middleware**: `rbac-traefik.yaml` needs one-time manual apply
by cluster admin (pre-existing, non-blocking)
3. **`SEED_USERS`**: Not set in K8s secrets (pre-existing)
4. **Editor modules**: `src/editor/*.mjs` still reference "Chat Switchboard"
(pre-existing, low priority)
5. **K8s deploy**: PR not yet merged to main — CI/CD will need a deploy after merge
---
## What's Next
### Merge PR #1
Review and merge `feat/admin-rbac-migration` to main.
### v0.2.0 — Remaining PRs
**PR 2: Settings cascade**
- Add `user_overridable` flag to settings schema
- RBAC controls scope auth (admin → global, team-admin → team, user → personal)
- Resolution: user → team → global (first non-null wins)
**PR 3: ICD (API contract)**
- Generate full OpenAPI spec from registered routes
- Document kernel-only endpoints
**PR 4: Trigger system**
- Time triggers (cron), webhook (inbound HTTP), event (bus subscription)
- Extensions register match expressions at install
**PR 5: SDK stabilization**
- `sb.slots()`, `sb.actions`, `sb.api.ext()`, `sb.storage`
- Theme tokens exposed to extensions
---
## Architecture Decisions (this session)
| Decision | Detail |
|----------|--------|
| Full RBAC — no magic roles | `users.role` column dropped entirely. All authorization through group membership + permission grants. |
| Explicit Everyone membership | No implicit "all users get Everyone perms". Every user added to Everyone group on create. `member_count` reflects reality. |
| Admins group not special-cased | Code never checks group identity — only checks for `surface.admin.access` permission. Any group can grant it. |
| No new migrations pre-MVP | Edit existing SQL files in place. Schema isn't in production. |
| JWT simplified | Claims carry `user_id` + `email` only. Permissions resolved server-side from groups on every request. |
| OIDC role → group sync | `isIdPAdmin()` replaces `resolveRole()`. IdP admin claim maps to Admins group membership, not a DB column. |
| Token budgets/allowed models removed | Provider-era cruft. Belongs in a future provider extension, not the kernel. |
---
## File Locations
| What | Where |
|------|-------|
| Roadmap | `ROADMAP.md` |
| Changelog | `CHANGELOG.md` |
| CI workflow | `.gitea/workflows/ci.yaml` |
| K8s manifests | `k8s/` |
| Docker compose (local dev) | `docker-compose.yml` (SQLite, port 3000) |
| Nginx template | `nginx.conf.template` |
| Migrations | `server/database/migrations/{postgres,sqlite}/` |
| Permission constants | `server/auth/permissions.go` |
| Admin middleware | `server/middleware/admin.go` |
| Group store | `server/store/{postgres,sqlite}/groups.go` |
| Bootstrap/seed | `server/handlers/auth.go` |
| Admin handlers | `server/handlers/admin.go` |
| Groups UI | `src/js/sw/surfaces/admin/groups.js` |
| Users UI | `src/js/sw/surfaces/admin/users.js` |
| SDK API | `src/js/sw/sdk/api-domains.js` |
| Memory (Claude) | `.claude/projects/-config-Projects-core/memory/` |
---
## Gitea Secrets/Vars Needed
**Secrets**: `POSTGRES_USER`, `POSTGRES_USER_PASSWORD`, `POSTGRES_ADMIN_USER`,
`POSTGRES_ADMIN_PASSWORD`, `ENCRYPTION_KEY`
**Vars**: `DOMAIN`, `NAMESPACE`, `S3_BUCKET`, `S3_ENDPOINT`, `STORAGE_BACKEND`,
`STORAGE_CLASS`, `STORAGE_SIZE`
**Not yet configured**: `S3_ACCESS_KEY`, `S3_SECRET_KEY` (using PVC for now),
`SEED_USERS` (needed for initial admin bootstrap)

2
VERSION
View File

@@ -1 +1 @@
0.10.0
0.2.8

6
chart/Chart.yaml
View File

@@ -1,11 +1,11 @@
apiVersion: v2
name: armature
description: Armature — self-hosted extension platform
name: switchboard
description: Switchboard Core — self-hosted extension platform
type: application
version: 0.1.0
appVersion: "0.0.0" # Patched at CI/release time from /VERSION
home: https://gobha.ai
sources:
- https://git.gobha.me/armature/core
- https://git.gobha.me/switchboard/core
maintainers:
- name: xcaliber

34
chart/alerting/armature-rules.yaml → chart/alerting/switchboard-rules.yaml
View File

@@ -1,27 +1,27 @@
# Armature Core — PrometheusRule alerts (v0.33.0)
# Switchboard Core — PrometheusRule alerts (v0.33.0)
# Source file for the Helm template. Deploy via:
# monitoring.prometheusRule.enabled: true
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: armature-alerts
name: switchboard-alerts
spec:
groups:
- name: armature.rules
- name: switchboard.rules
rules:
# Pod restart (possible OOM)
- alert: ArmaturePodRestart
- alert: SwitchboardPodRestart
expr: increase(kube_pod_container_status_restarts_total{container="backend"}[1h]) > 0
for: 0m
labels:
severity: warning
annotations:
summary: "Armature backend pod restarted (possible OOM)"
summary: "Switchboard backend pod restarted (possible OOM)"
description: "Container {{ $labels.container }} in pod {{ $labels.pod }} restarted."
# Provider down for 5+ minutes
- alert: ArmatureProviderDown
expr: armature_provider_status > 2
- alert: SwitchboardProviderDown
expr: switchboard_provider_status > 2
for: 5m
labels:
severity: critical
@@ -29,8 +29,8 @@ spec:
summary: "Provider {{ $labels.provider_config_id }} is down"
# DB pool >80% utilized
- alert: ArmatureDBPoolExhaustion
expr: armature_db_in_use_connections / armature_db_open_connections > 0.8
- alert: SwitchboardDBPoolExhaustion
expr: switchboard_db_in_use_connections / switchboard_db_open_connections > 0.8
for: 5m
labels:
severity: warning
@@ -38,10 +38,10 @@ spec:
summary: "DB connection pool >80% utilized"
# HTTP 5xx error rate >5%
- alert: ArmatureHighErrorRate
- alert: SwitchboardHighErrorRate
expr: >
sum(rate(armature_http_requests_total{status=~"5.."}[5m]))
/ sum(rate(armature_http_requests_total[5m])) > 0.05
sum(rate(switchboard_http_requests_total{status=~"5.."}[5m]))
/ sum(rate(switchboard_http_requests_total[5m])) > 0.05
for: 5m
labels:
severity: warning
@@ -49,10 +49,10 @@ spec:
summary: "HTTP 5xx error rate exceeds 5%"
# Task failure rate >25%
- alert: ArmatureTaskFailureRate
- alert: SwitchboardTaskFailureRate
expr: >
rate(armature_task_executions_total{status="error"}[15m])
/ rate(armature_task_executions_total[15m]) > 0.25
rate(switchboard_task_executions_total{status="error"}[15m])
/ rate(switchboard_task_executions_total[15m]) > 0.25
for: 10m
labels:
severity: warning
@@ -60,8 +60,8 @@ spec:
summary: "Task failure rate exceeds 25% over 15 minutes"
# No completions processed in 15 minutes (canary)
- alert: ArmatureNoCompletions
expr: sum(rate(armature_completions_total[10m])) == 0
- alert: SwitchboardNoCompletions
expr: sum(rate(switchboard_completions_total[10m])) == 0
for: 15m
labels:
severity: critical

38
chart/dashboards/armature-overview.json → chart/dashboards/switchboard-overview.json
View File

@@ -17,10 +17,10 @@
{ "type": "panel", "id": "gauge", "name": "Gauge", "version": "" }
],
"id": null,
"uid": "armature-overview",
"title": "Armature — Overview",
"uid": "switchboard-overview",
"title": "Switchboard Core — Overview",
"description": "System overview: request rates, latency, provider health, token usage, DB pool.",
"tags": ["armature"],
"tags": ["switchboard"],
"timezone": "browser",
"refresh": "30s",
"schemaVersion": 38,
@@ -37,7 +37,7 @@
"name": "namespace",
"type": "query",
"datasource": { "type": "prometheus", "uid": "${datasource}" },
"query": "label_values(armature_http_requests_total, namespace)",
"query": "label_values(switchboard_http_requests_total, namespace)",
"includeAll": true,
"current": { "text": "All", "value": "$__all" }
},
@@ -45,7 +45,7 @@
"name": "pod",
"type": "query",
"datasource": { "type": "prometheus", "uid": "${datasource}" },
"query": "label_values(armature_http_requests_total{namespace=~\"$namespace\"}, pod)",
"query": "label_values(switchboard_http_requests_total{namespace=~\"$namespace\"}, pod)",
"includeAll": true,
"current": { "text": "All", "value": "$__all" }
}
@@ -58,7 +58,7 @@
"gridPos": { "h": 8, "w": 6, "x": 0, "y": 0 },
"targets": [
{
"expr": "sum(rate(armature_http_requests_total{namespace=~\"$namespace\"}[5m]))",
"expr": "sum(rate(switchboard_http_requests_total{namespace=~\"$namespace\"}[5m]))",
"legendFormat": "Total req/s"
}
]
@@ -69,7 +69,7 @@
"gridPos": { "h": 8, "w": 6, "x": 6, "y": 0 },
"targets": [
{
"expr": "sum(rate(armature_http_requests_total{namespace=~\"$namespace\",status=~\"5..\"}[5m])) / sum(rate(armature_http_requests_total{namespace=~\"$namespace\"}[5m]))",
"expr": "sum(rate(switchboard_http_requests_total{namespace=~\"$namespace\",status=~\"5..\"}[5m])) / sum(rate(switchboard_http_requests_total{namespace=~\"$namespace\"}[5m]))",
"legendFormat": "5xx rate"
}
],
@@ -93,15 +93,15 @@
"gridPos": { "h": 8, "w": 6, "x": 12, "y": 0 },
"targets": [
{
"expr": "histogram_quantile(0.50, sum(rate(armature_http_request_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])) by (le))",
"expr": "histogram_quantile(0.50, sum(rate(switchboard_http_request_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])) by (le))",
"legendFormat": "p50"
},
{
"expr": "histogram_quantile(0.95, sum(rate(armature_http_request_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])) by (le))",
"expr": "histogram_quantile(0.95, sum(rate(switchboard_http_request_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])) by (le))",
"legendFormat": "p95"
},
{
"expr": "histogram_quantile(0.99, sum(rate(armature_http_request_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])) by (le))",
"expr": "histogram_quantile(0.99, sum(rate(switchboard_http_request_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])) by (le))",
"legendFormat": "p99"
}
],
@@ -113,7 +113,7 @@
"gridPos": { "h": 8, "w": 6, "x": 18, "y": 0 },
"targets": [
{
"expr": "sum(armature_websocket_connections{namespace=~\"$namespace\"})",
"expr": "sum(switchboard_websocket_connections{namespace=~\"$namespace\"})",
"legendFormat": "Active"
}
]
@@ -124,7 +124,7 @@
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
"targets": [
{
"expr": "sum by (provider_config_id) (rate(armature_completions_total{namespace=~\"$namespace\"}[5m]))",
"expr": "sum by (provider_config_id) (rate(switchboard_completions_total{namespace=~\"$namespace\"}[5m]))",
"legendFormat": "{{provider_config_id}}"
}
]
@@ -135,7 +135,7 @@
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
"targets": [
{
"expr": "histogram_quantile(0.95, sum by (provider_config_id, le) (rate(armature_completion_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])))",
"expr": "histogram_quantile(0.95, sum by (provider_config_id, le) (rate(switchboard_completion_duration_seconds_bucket{namespace=~\"$namespace\"}[5m])))",
"legendFormat": "{{provider_config_id}}"
}
],
@@ -147,7 +147,7 @@
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 16 },
"targets": [
{
"expr": "sum by (model_id) (rate(armature_completion_tokens_total{namespace=~\"$namespace\"}[5m])) * 60",
"expr": "sum by (model_id) (rate(switchboard_completion_tokens_total{namespace=~\"$namespace\"}[5m])) * 60",
"legendFormat": "{{model_id}}"
}
]
@@ -158,7 +158,7 @@
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 16 },
"targets": [
{
"expr": "armature_provider_status{namespace=~\"$namespace\"}",
"expr": "switchboard_provider_status{namespace=~\"$namespace\"}",
"legendFormat": "{{provider_config_id}}"
}
],
@@ -179,15 +179,15 @@
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 24 },
"targets": [
{
"expr": "armature_db_open_connections{namespace=~\"$namespace\"}",
"expr": "switchboard_db_open_connections{namespace=~\"$namespace\"}",
"legendFormat": "Open"
},
{
"expr": "armature_db_in_use_connections{namespace=~\"$namespace\"}",
"expr": "switchboard_db_in_use_connections{namespace=~\"$namespace\"}",
"legendFormat": "In Use"
},
{
"expr": "armature_db_idle_connections{namespace=~\"$namespace\"}",
"expr": "switchboard_db_idle_connections{namespace=~\"$namespace\"}",
"legendFormat": "Idle"
}
]
@@ -198,7 +198,7 @@
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 24 },
"targets": [
{
"expr": "sum by (status) (rate(armature_task_executions_total{namespace=~\"$namespace\"}[5m]))",
"expr": "sum by (status) (rate(switchboard_task_executions_total{namespace=~\"$namespace\"}[5m]))",
"legendFormat": "{{status}}"
}
]

2
chart/templates/NOTES.txt
View File

@@ -1,4 +1,4 @@
Armature {{ .Chart.AppVersion }} deployed.
Chat Switchboard {{ .Chart.AppVersion }} deployed.
{{- if .Values.ingress.enabled }}
Access the application at:

18
chart/templates/_helpers.tpl
View File

@@ -1,7 +1,7 @@
{{/*
Common labels
*/}}
{{- define "armature.labels" -}}
{{- define "switchboard.labels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
@@ -12,23 +12,23 @@ helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version }}
{{/*
Backend selector labels
*/}}
{{- define "armature.backend.labels" -}}
{{- define "switchboard.backend.labels" -}}
app.kubernetes.io/component: backend
{{ include "armature.labels" . }}
{{ include "switchboard.labels" . }}
{{- end }}
{{/*
Frontend selector labels
*/}}
{{- define "armature.frontend.labels" -}}
{{- define "switchboard.frontend.labels" -}}
app.kubernetes.io/component: frontend
{{ include "armature.labels" . }}
{{ include "switchboard.labels" . }}
{{- end }}
{{/*
Secret name
*/}}
{{- define "armature.secretName" -}}
{{- define "switchboard.secretName" -}}
{{- if .Values.existingSecret -}}
{{ .Values.existingSecret }}
{{- else -}}
@@ -39,21 +39,21 @@ Secret name
{{/*
Backend image
*/}}
{{- define "armature.backend.image" -}}
{{- define "switchboard.backend.image" -}}
{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag | default .Chart.AppVersion }}
{{- end }}
{{/*
Frontend image
*/}}
{{- define "armature.frontend.image" -}}
{{- define "switchboard.frontend.image" -}}
{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag | default .Chart.AppVersion }}
{{- end }}
{{/*
Database URL — assembled from postgres.* if url is empty
*/}}
{{- define "armature.databaseURL" -}}
{{- define "switchboard.databaseURL" -}}
{{- if .Values.database.url -}}
{{ .Values.database.url }}
{{- else if eq .Values.database.driver "sqlite" -}}

2
chart/templates/configmap.yaml
View File

@@ -3,7 +3,7 @@ kind: ConfigMap
metadata:
name: {{ .Release.Name }}-config
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
data:
PORT: {{ .Values.backend.port | quote }}
BASE_PATH: {{ .Values.basePath | quote }}

12
chart/templates/cronjob-backup.yaml
View File

@@ -4,7 +4,7 @@ kind: CronJob
metadata:
name: {{ .Release.Name }}-backup
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
app.kubernetes.io/component: backup
spec:
schedule: {{ .Values.backup.schedule | quote }}
@@ -35,7 +35,7 @@ spec:
- |
set -e
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
FILENAME="armature-${TIMESTAMP}.sql.gz"
FILENAME="switchboard-${TIMESTAMP}.sql.gz"
echo "Starting backup: ${FILENAME}"
pg_dump "${DATABASE_URL}" | gzip > "/backup/${FILENAME}"
@@ -58,22 +58,22 @@ spec:
# Prune old local backups beyond retention
cd /backup
ls -t armature-*.sql.gz 2>/dev/null | tail -n +{{ add1 .Values.backup.retention }} | xargs -r rm -v
ls -t switchboard-*.sql.gz 2>/dev/null | tail -n +{{ add1 .Values.backup.retention }} | xargs -r rm -v
echo "Backup complete"
env:
- name: DATABASE_URL
value: {{ include "armature.databaseURL" . | quote }}
value: {{ include "switchboard.databaseURL" . | quote }}
{{- if .Values.backup.s3.enabled }}
- name: S3_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{ include "armature.secretName" . }}
name: {{ include "switchboard.secretName" . }}
key: BACKUP_S3_ACCESS_KEY
optional: true
- name: S3_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ include "armature.secretName" . }}
name: {{ include "switchboard.secretName" . }}
key: BACKUP_S3_SECRET_KEY
optional: true
{{- end }}

8
chart/templates/deployment-backend.yaml
View File

@@ -3,7 +3,7 @@ kind: Deployment
metadata:
name: {{ .Release.Name }}-backend
labels:
{{- include "armature.backend.labels" . | nindent 4 }}
{{- include "switchboard.backend.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.backend.replicaCount }}
selector:
@@ -38,7 +38,7 @@ spec:
topologyKey: kubernetes.io/hostname
containers:
- name: backend
image: {{ include "armature.backend.image" . }}
image: {{ include "switchboard.backend.image" . }}
imagePullPolicy: {{ .Values.backend.image.pullPolicy }}
ports:
- name: http
@@ -48,10 +48,10 @@ spec:
- configMapRef:
name: {{ .Release.Name }}-config
- secretRef:
name: {{ include "armature.secretName" . }}
name: {{ include "switchboard.secretName" . }}
env:
- name: DATABASE_URL
value: {{ include "armature.databaseURL" . | quote }}
value: {{ include "switchboard.databaseURL" . | quote }}
resources:
{{- toYaml .Values.backend.resources | nindent 12 }}
{{- if .Values.persistence.enabled }}

4
chart/templates/deployment-frontend.yaml
View File

@@ -3,7 +3,7 @@ kind: Deployment
metadata:
name: {{ .Release.Name }}-frontend
labels:
{{- include "armature.frontend.labels" . | nindent 4 }}
{{- include "switchboard.frontend.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.frontend.replicaCount }}
selector:
@@ -24,7 +24,7 @@ spec:
{{- end }}
containers:
- name: frontend
image: {{ include "armature.frontend.image" . }}
image: {{ include "switchboard.frontend.image" . }}
imagePullPolicy: {{ .Values.frontend.image.pullPolicy }}
ports:
- name: http

8
chart/templates/grafana-dashboard-configmap.yaml
View File

@@ -2,13 +2,13 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "armature.fullname" . }}-grafana-dashboard
name: {{ include "switchboard.fullname" . }}-grafana-dashboard
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
{{- with .Values.monitoring.grafanaDashboard.labels }}
{{- toYaml . | nindent 4 }}
{{- end }}
data:
armature-overview.json: |-
{{- .Files.Get "dashboards/armature-overview.json" | nindent 4 }}
switchboard-overview.json: |-
{{- .Files.Get "dashboards/switchboard-overview.json" | nindent 4 }}
{{- end }}

2
chart/templates/ingress.yaml
View File

@@ -4,7 +4,7 @@ kind: Ingress
metadata:
name: {{ .Release.Name }}-ingress
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
{{- if or .Values.ingress.annotations (and (eq (default "traefik" .Values.ingress.className) "traefik") .Values.ingress.retry.annotateIngress) }}
annotations:
{{- with .Values.ingress.annotations }}

2
chart/templates/middleware-retry.yaml
View File

@@ -5,7 +5,7 @@ kind: Middleware
metadata:
name: {{ .Release.Name }}-retry
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
spec:
retry:
attempts: {{ .Values.ingress.retry.attempts | default 2 }}

34
chart/templates/prometheusrule.yaml
View File

@@ -2,57 +2,57 @@
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: {{ include "armature.fullname" . }}-alerts
name: {{ include "switchboard.fullname" . }}-alerts
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
{{- with .Values.monitoring.prometheusRule.labels }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
groups:
- name: armature.rules
- name: switchboard.rules
rules:
- alert: ArmaturePodRestart
- alert: SwitchboardPodRestart
expr: increase(kube_pod_container_status_restarts_total{container="backend"}[1h]) > 0
for: 0m
labels:
severity: warning
annotations:
summary: "Armature backend pod restarted (possible OOM)"
- alert: ArmatureProviderDown
expr: armature_provider_status > 2
summary: "Switchboard backend pod restarted (possible OOM)"
- alert: SwitchboardProviderDown
expr: switchboard_provider_status > 2
for: 5m
labels:
severity: critical
annotations:
summary: "Provider {{`{{ $labels.provider_config_id }}`}} is down"
- alert: ArmatureDBPoolExhaustion
expr: armature_db_in_use_connections / armature_db_open_connections > 0.8
- alert: SwitchboardDBPoolExhaustion
expr: switchboard_db_in_use_connections / switchboard_db_open_connections > 0.8
for: 5m
labels:
severity: warning
annotations:
summary: "DB connection pool >80% utilized"
- alert: ArmatureHighErrorRate
- alert: SwitchboardHighErrorRate
expr: |
sum(rate(armature_http_requests_total{status=~"5.."}[5m]))
/ sum(rate(armature_http_requests_total[5m])) > 0.05
sum(rate(switchboard_http_requests_total{status=~"5.."}[5m]))
/ sum(rate(switchboard_http_requests_total[5m])) > 0.05
for: 5m
labels:
severity: warning
annotations:
summary: "HTTP 5xx error rate exceeds 5%"
- alert: ArmatureTaskFailureRate
- alert: SwitchboardTaskFailureRate
expr: |
rate(armature_task_executions_total{status="error"}[15m])
/ rate(armature_task_executions_total[15m]) > 0.25
rate(switchboard_task_executions_total{status="error"}[15m])
/ rate(switchboard_task_executions_total[15m]) > 0.25
for: 10m
labels:
severity: warning
annotations:
summary: "Task failure rate exceeds 25%"
- alert: ArmatureNoCompletions
expr: sum(rate(armature_completions_total[10m])) == 0
- alert: SwitchboardNoCompletions
expr: sum(rate(switchboard_completions_total[10m])) == 0
for: 15m
labels:
severity: critical

2
chart/templates/pvc-backup.yaml
View File

@@ -4,7 +4,7 @@ kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-backup
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
app.kubernetes.io/component: backup
spec:
accessModes:

2
chart/templates/pvc.yaml
View File

@@ -4,7 +4,7 @@ kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-data
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
spec:
accessModes:
- {{ .Values.persistence.accessMode }}

8
chart/templates/secret.yaml
View File

@@ -4,7 +4,7 @@ kind: Secret
metadata:
name: {{ .Release.Name }}-secrets
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
type: Opaque
stringData:
JWT_SECRET: {{ required "jwtSecret is required" .Values.jwtSecret | quote }}
@@ -15,11 +15,11 @@ stringData:
POSTGRES_PASSWORD: {{ .Values.database.postgres.password | quote }}
{{- end }}
{{- if .Values.admin.password }}
ARMATURE_ADMIN_USERNAME: {{ .Values.admin.username | quote }}
ARMATURE_ADMIN_PASSWORD: {{ .Values.admin.password | quote }}
SWITCHBOARD_ADMIN_USERNAME: {{ .Values.admin.username | quote }}
SWITCHBOARD_ADMIN_PASSWORD: {{ .Values.admin.password | quote }}
{{- end }}
{{- if .Values.admin.email }}
ARMATURE_ADMIN_EMAIL: {{ .Values.admin.email | quote }}
SWITCHBOARD_ADMIN_EMAIL: {{ .Values.admin.email | quote }}
{{- end }}
{{- if .Values.storage.s3.accessKey }}
S3_ACCESS_KEY: {{ .Values.storage.s3.accessKey | quote }}

6
chart/templates/servicemonitor.yaml
View File

@@ -2,16 +2,16 @@
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "armature.fullname" . }}
name: {{ include "switchboard.fullname" . }}
labels:
{{- include "armature.labels" . | nindent 4 }}
{{- include "switchboard.labels" . | nindent 4 }}
{{- with .Values.monitoring.serviceMonitor.labels }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
selector:
matchLabels:
{{- include "armature.selectorLabels" . | nindent 6 }}
{{- include "switchboard.selectorLabels" . | nindent 6 }}
app.kubernetes.io/component: backend
endpoints:
- port: http

4
chart/templates/services.yaml
View File

@@ -3,7 +3,7 @@ kind: Service
metadata:
name: {{ .Release.Name }}-backend
labels:
{{- include "armature.backend.labels" . | nindent 4 }}
{{- include "switchboard.backend.labels" . | nindent 4 }}
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: {{ .Values.backend.port | quote }}
@@ -25,7 +25,7 @@ kind: Service
metadata:
name: {{ .Release.Name }}-frontend
labels:
{{- include "armature.frontend.labels" . | nindent 4 }}
{{- include "switchboard.frontend.labels" . | nindent 4 }}
spec:
type: ClusterIP
ports:

16
chart/values.yaml
View File

@@ -1,10 +1,10 @@
# Armature — Helm values
# helm install armature ./chart
# Switchboard Core — Helm values
# helm install switchboard ./chart
# ── Images ─────────────────────────────────
backend:
image:
repository: git.gobha.me/armature/core
repository: git.gobha.me/switchboard/core
tag: "" # defaults to Chart.appVersion
pullPolicy: IfNotPresent
replicaCount: 2 # v0.32.0: multi-replica HA
@@ -19,7 +19,7 @@ backend:
frontend:
image:
repository: git.gobha.me/armature/core
repository: git.gobha.me/switchboard/core
tag: "" # defaults to Chart.appVersion
pullPolicy: IfNotPresent
replicaCount: 1
@@ -41,12 +41,12 @@ database:
postgres:
host: postgresql
port: 5432
user: armature
user: switchboard
password: "" # set via secret
database: armature
database: switchboard
sslmode: disable
# For sqlite: path inside the container (requires persistence)
sqlitePath: /data/armature.db
sqlitePath: /data/switchboard.db
# ── Core ───────────────────────────────────
basePath: "" # URL prefix, e.g. "/dev"
@@ -85,7 +85,7 @@ ingress:
enabled: true
className: traefik
annotations: {}
host: armature.local
host: switchboard.local
tls:
enabled: false
secretName: ""

11
ci/Dockerfile.test-runner
View File

@@ -1,11 +0,0 @@
# ci/Dockerfile.test-runner — Pre-built Playwright test runner
#
# Build & push (one-time, repeat when Playwright version bumps):
# docker build -t registry.gobha.me:5000/ci-test-runner:latest -f ci/Dockerfile.test-runner .
# docker push registry.gobha.me:5000/ci-test-runner:latest
#
# The CI compose override uses this image directly, avoiding npm install
# on every run. Only the ci/ scripts are COPY'd at compose build time.
FROM mcr.microsoft.com/playwright:v1.52.0-noble
WORKDIR /work
RUN npm init -y && npm install playwright@1.52.0

139
ci/e2e-backup-test.sh
View File

@@ -1,139 +0,0 @@
#!/usr/bin/env bash
# e2e-backup-test.sh — Backup/restore E2E test (v0.6.1)
#
# Requires: docker compose running with a single instance.
# Usage:
# docker compose up --build -d
# ./ci/e2e-backup-test.sh
# docker compose down -v
set -euo pipefail
BASE="http://localhost:8080"
PASS=0
FAIL=0
pass() { PASS=$((PASS + 1)); echo " PASS: $1"; }
fail() { FAIL=$((FAIL + 1)); echo " FAIL: $1"; }
# ── Wait for service ─────────────────────────
echo "=== Waiting for service ==="
for i in $(seq 1 30); do
if curl -sf "$BASE/health" > /dev/null 2>&1; then
echo " service ready"
break
fi
sleep 1
if [ "$i" -eq 30 ]; then
echo "FATAL: service did not become healthy"
exit 1
fi
done
# ── Login as admin ───────────────────────────
echo "=== Authenticating ==="
TOKEN=$(curl -sf "$BASE/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"login":"admin","password":"admin"}' | jq -r '.access_token')
if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
echo "FATAL: login failed"
exit 1
fi
echo " admin token acquired"
auth() { echo "Authorization: Bearer $TOKEN"; }
# ── Seed test data ───────────────────────────
echo "=== Seeding test data ==="
# Create a test user
curl -sf "$BASE/api/v1/auth/register" \
-H "Content-Type: application/json" \
-d '{"username":"backup-test-user","email":"bktest@test.com","password":"testpass123"}' > /dev/null
USER_COUNT=$(curl -sf "$BASE/api/v1/admin/users" -H "$(auth)" | jq '.data | length')
echo " users: $USER_COUNT"
# ── Test 1: List backups (empty) ─────────────
echo "=== Test 1: List backups (empty) ==="
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
COUNT=$(echo "$LIST" | jq '.data | length')
if [ "$COUNT" -eq 0 ]; then
pass "empty backup list"
else
fail "expected 0 backups, got $COUNT"
fi
# ── Test 2: Create server-side backup ────────
echo "=== Test 2: Create server-side backup ==="
RESULT=$(curl -sf "$BASE/api/v1/admin/backup?store=true" \
-X POST \
-H "$(auth)")
FILENAME=$(echo "$RESULT" | jq -r '.data.filename')
if [ -n "$FILENAME" ] && [ "$FILENAME" != "null" ]; then
pass "created backup: $FILENAME"
else
fail "backup creation failed: $RESULT"
fi
# ── Test 3: List backups (has one) ───────────
echo "=== Test 3: List backups (has one) ==="
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
COUNT=$(echo "$LIST" | jq '.data | length')
if [ "$COUNT" -eq 1 ]; then
pass "one backup listed"
else
fail "expected 1 backup, got $COUNT"
fi
# ── Test 4: Download backup ──────────────────
echo "=== Test 4: Download backup ==="
TMPFILE=$(mktemp /tmp/backup-test-XXXXXX.swb)
HTTP_CODE=$(curl -sf -o "$TMPFILE" -w "%{http_code}" \
"$BASE/api/v1/admin/backups/$FILENAME" -H "$(auth)")
if [ "$HTTP_CODE" = "200" ] && [ -s "$TMPFILE" ]; then
pass "download OK ($(wc -c < "$TMPFILE") bytes)"
else
fail "download failed (HTTP $HTTP_CODE)"
fi
# ── Test 5: Streaming backup (direct download) ──
echo "=== Test 5: Streaming backup ==="
STREAM_FILE=$(mktemp /tmp/backup-stream-XXXXXX.swb)
HTTP_CODE=$(curl -sf -o "$STREAM_FILE" -w "%{http_code}" \
-X POST "$BASE/api/v1/admin/backup" -H "$(auth)")
if [ "$HTTP_CODE" = "200" ] && [ -s "$STREAM_FILE" ]; then
pass "streaming download OK ($(wc -c < "$STREAM_FILE") bytes)"
else
fail "streaming download failed (HTTP $HTTP_CODE)"
fi
# ── Test 6: Delete backup ────────────────────
echo "=== Test 6: Delete backup ==="
DEL=$(curl -sf -X DELETE "$BASE/api/v1/admin/backups/$FILENAME" -H "$(auth)")
DELETED=$(echo "$DEL" | jq -r '.data.deleted')
if [ "$DELETED" = "$FILENAME" ]; then
pass "deleted backup"
else
fail "delete failed: $DEL"
fi
# Verify deletion
LIST=$(curl -sf "$BASE/api/v1/admin/backups" -H "$(auth)")
COUNT=$(echo "$LIST" | jq '.data | length')
if [ "$COUNT" -eq 0 ]; then
pass "backup list empty after delete"
else
fail "expected 0 backups after delete, got $COUNT"
fi
# ── Cleanup ──────────────────────────────────
rm -f "$TMPFILE" "$STREAM_FILE"
# ── Summary ──────────────────────────────────
echo ""
echo "=== Results: $PASS passed, $FAIL failed ==="
if [ "$FAIL" -gt 0 ]; then
exit 1
fi

247
ci/e2e-chat-test.sh
View File

@@ -1,247 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# E2E Chat Test — Multi-user / Multi-replica
# ═══════════════════════════════════════════════
#
# Prerequisites:
# docker compose -f docker-compose-e2e.yml up --build -d
#
# Tests:
# 1. Auth as alice + bob
# 2. Alice creates conversation, adds bob
# 3. Alice sends message via REST
# 4. Bob reads messages, verifies receipt
# 5. Cross-replica: send via replica-1, read via replica-2
# 6. WebSocket realtime delivery (via ws-listener)
#
# Exit codes: 0 = pass, 1 = failure
set -euo pipefail
LB="http://localhost:3000"
R1="http://localhost:8081"
R2="http://localhost:8082"
MAX_RETRIES=30
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
pass=0
fail=0
ok() { pass=$((pass + 1)); echo -e " ${GREEN}✓${NC} $1"; }
fail() { fail=$((fail + 1)); echo -e " ${RED}✗${NC} $1"; }
# ── Wait for LB ─────────────────────────────
echo -e "${YELLOW}Waiting for load balancer...${NC}"
for i in $(seq 1 $MAX_RETRIES); do
if curl -sf "$LB/api/v1/auth/login" -o /dev/null 2>/dev/null || \
curl -sf "$LB" -o /dev/null 2>/dev/null; then
echo -e "${GREEN}LB ready after ${i}s${NC}"
break
fi
if [ "$i" -eq "$MAX_RETRIES" ]; then
echo -e "${RED}LB not ready after ${MAX_RETRIES}s${NC}"
exit 1
fi
sleep 1
done
# ── Helper: login ────────────────────────────
login() {
local user=$1 pass=$2 host=${3:-$LB}
local resp
resp=$(curl -sf "$host/api/v1/auth/login" \
-H 'Content-Type: application/json' \
-d "{\"login\":\"$user\",\"password\":\"$pass\"}")
echo "$resp" | grep -o '"access_token":"[^"]*"' | head -1 | sed 's/.*"access_token":"\([^"]*\)".*/\1/'
}
authed() {
# Usage: authed $TOKEN GET /api/v1/... [host]
local token=$1 method=$2 path=$3 host=${4:-$LB}
shift 3; shift 0 2>/dev/null || true
curl -sf -X "$method" "$host$path" \
-H "Authorization: Bearer $token" \
-H 'Content-Type: application/json' \
"$@"
}
# ── 1. Auth ──────────────────────────────────
echo -e "\n${YELLOW}1. Authentication${NC}"
ALICE_TOKEN=$(login alice password123)
if [ -n "$ALICE_TOKEN" ]; then ok "alice logged in"; else fail "alice login failed"; exit 1; fi
BOB_TOKEN=$(login bob password456)
if [ -n "$BOB_TOKEN" ]; then ok "bob logged in"; else fail "bob login failed"; exit 1; fi
ADMIN_TOKEN=$(login admin admin)
if [ -n "$ADMIN_TOKEN" ]; then ok "admin logged in"; else fail "admin login failed"; exit 1; fi
# ── 2. Install chat packages ────────────────
echo -e "\n${YELLOW}2. Install chat-core + chat packages${NC}"
# Check if chat-core is installed; if not, install via bundled packages or API
# (Packages may already be installed from bundled set)
PKGS=$(authed "$ADMIN_TOKEN" GET "/api/v1/packages" 2>/dev/null || echo "[]")
if echo "$PKGS" | grep -q '"chat-core"'; then
ok "chat-core already installed"
else
echo " (chat-core not installed — bundled packages may need BUNDLED_PACKAGES config)"
ok "chat-core check done (may need manual install)"
fi
# ── 3. Create conversation ───────────────────
echo -e "\n${YELLOW}3. Create conversation + send messages${NC}"
# Get alice's user ID
ALICE_ME=$(authed "$ALICE_TOKEN" GET "/api/v1/profile" 2>/dev/null || echo "{}")
ALICE_ID=$(echo "$ALICE_ME" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
BOB_ME=$(authed "$BOB_TOKEN" GET "/api/v1/profile" 2>/dev/null || echo "{}")
BOB_ID=$(echo "$BOB_ME" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
# Create conversation via chat-core API
CONV=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/conversations" "" \
-d "{\"title\":\"E2E Test Chat\",\"type\":\"group\",\"participants\":[{\"id\":\"$BOB_ID\",\"display_name\":\"bob\"}],\"creator_display_name\":\"alice\"}" 2>/dev/null || echo "{}")
CONV_ID=$(echo "$CONV" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$CONV_ID" ]; then
ok "conversation created: ${CONV_ID:0:8}..."
else
fail "conversation creation failed"
echo " Response: $CONV"
fi
# ── 4. Send + read messages ──────────────────
echo -e "\n${YELLOW}4. Message send + read${NC}"
if [ -n "$CONV_ID" ]; then
# Alice sends a message
MSG1=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
-d '{"content":"Hello from alice!","content_type":"text"}' 2>/dev/null || echo "{}")
MSG1_ID=$(echo "$MSG1" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$MSG1_ID" ]; then ok "alice sent message"; else fail "alice send failed"; fi
# Bob reads messages
MSGS=$(authed "$BOB_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" 2>/dev/null || echo "{}")
if echo "$MSGS" | grep -q "Hello from alice"; then
ok "bob received alice's message"
else
fail "bob did not receive message"
echo " Response: ${MSGS:0:200}"
fi
# Bob sends a reply
MSG2=$(authed "$BOB_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
-d '{"content":"Hello from bob!","content_type":"text"}' 2>/dev/null || echo "{}")
MSG2_ID=$(echo "$MSG2" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$MSG2_ID" ]; then ok "bob sent reply"; else fail "bob send failed"; fi
fi
# ── 5. Cross-replica consistency ─────────────
echo -e "\n${YELLOW}5. Cross-replica consistency${NC}"
if [ -n "$CONV_ID" ]; then
# Login to each replica directly
ALICE_R1=$(login alice password123 "$R1")
BOB_R2=$(login bob password456 "$R2")
# Alice sends via replica 1
MSG3=$(curl -sf -X POST "$R1/s/chat-core/api/messages/$CONV_ID" \
-H "Authorization: Bearer $ALICE_R1" \
-H 'Content-Type: application/json' \
-d '{"content":"Cross-replica test message","content_type":"text"}' 2>/dev/null || echo "{}")
if echo "$MSG3" | grep -q '"id"'; then
ok "alice sent via replica-1"
else
fail "alice send via replica-1 failed"
fi
# Small delay for pg replication
sleep 1
# Bob reads via replica 2
MSGS_R2=$(curl -sf "$R2/s/chat-core/api/messages/$CONV_ID?limit=50" \
-H "Authorization: Bearer $BOB_R2" 2>/dev/null || echo "{}")
if echo "$MSGS_R2" | grep -q "Cross-replica test message"; then
ok "bob sees cross-replica message via replica-2"
else
fail "cross-replica message not visible"
echo " Response: ${MSGS_R2:0:200}"
fi
fi
# ── 6. Search ────────────────────────────────
echo -e "\n${YELLOW}6. Conversation search${NC}"
if [ -n "$CONV_ID" ]; then
SEARCH=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/search?q=alice" 2>/dev/null || echo "{}")
if echo "$SEARCH" | grep -q "Hello from alice"; then
ok "search found message content"
else
fail "search did not find message"
echo " Response: ${SEARCH:0:200}"
fi
SEARCH_CONV=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/search?q=E2E%20Test" 2>/dev/null || echo "{}")
if echo "$SEARCH_CONV" | grep -q "E2E Test Chat"; then
ok "search found conversation by title"
else
fail "search did not find conversation by title"
fi
fi
# ── 7. Message pagination ────────────────────
echo -e "\n${YELLOW}7. Message pagination${NC}"
if [ -n "$CONV_ID" ]; then
# Send several more messages to test pagination
for i in $(seq 1 5); do
authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
-d "{\"content\":\"Pagination test message $i\",\"content_type\":\"text\"}" >/dev/null 2>&1
done
# Fetch with limit=3
PAGE1=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=3" 2>/dev/null || echo "{}")
HAS_MORE=$(echo "$PAGE1" | grep -o '"has_more":true' || echo "")
CURSOR=$(echo "$PAGE1" | grep -o '"next_cursor":"[^"]*"' | cut -d'"' -f4 || echo "")
if [ -n "$HAS_MORE" ] && [ -n "$CURSOR" ]; then
ok "pagination: first page has_more=true with cursor"
# Fetch second page
PAGE2=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=3&cursor=$CURSOR" 2>/dev/null || echo "{}")
if echo "$PAGE2" | grep -q '"messages"'; then
ok "pagination: second page returned messages"
else
fail "pagination: second page failed"
fi
else
fail "pagination: expected has_more and cursor"
fi
fi
# ── Summary ──────────────────────────────────
echo -e "\n════════════════════════════════════════"
echo -e "Results: ${GREEN}$pass passed${NC}, ${RED}$fail failed${NC}"
echo "════════════════════════════════════════"
[ "$fail" -eq 0 ] && exit 0 || exit 1

145
ci/e2e-cluster-test.sh
View File

@@ -1,145 +0,0 @@
#!/usr/bin/env bash
# e2e-cluster-test.sh — Cluster registry E2E test (v0.6.0)
#
# Requires: docker-compose-e2e.yml running with 3 replicas.
# Usage:
# docker compose -f docker-compose-e2e.yml up --build -d
# ./ci/e2e-cluster-test.sh
# docker compose -f docker-compose-e2e.yml down -v
set -euo pipefail
LB="http://localhost:3000"
DIRECT_1="http://localhost:8081"
STALE_WAIT=20 # seconds — must exceed CLUSTER_STALE_THRESHOLD (15s)
PASS=0
FAIL=0
pass() { PASS=$((PASS + 1)); echo " PASS: $1"; }
fail() { FAIL=$((FAIL + 1)); echo " FAIL: $1"; }
# ── Wait for all 3 replicas ──────────────────
echo "=== Waiting for replicas ==="
for port in 8081 8082 8083; do
for i in $(seq 1 30); do
if curl -sf "http://localhost:$port/health" > /dev/null 2>&1; then
echo " replica :$port ready"
break
fi
sleep 1
if [ "$i" -eq 30 ]; then
echo "FATAL: replica :$port did not become healthy"
exit 1
fi
done
done
# ── Login as admin ────────────────────────────
echo "=== Authenticating ==="
TOKEN=$(curl -sf "$LB/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"login":"admin","password":"admin"}' | jq -r '.access_token')
if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
echo "FATAL: login failed"
exit 1
fi
echo " admin token acquired"
auth() { echo "Authorization: Bearer $TOKEN"; }
# Wait for heartbeat to propagate (at least 2 tick cycles at 5s each)
echo "=== Waiting for heartbeat convergence ==="
sleep 12
# ── Test 1: All 3 nodes registered ───────────
echo "=== Test 1: Cluster shows 3 nodes ==="
NODES=$(curl -sf "$LB/api/v1/admin/cluster" -H "$(auth)")
COUNT=$(echo "$NODES" | jq '.data | length')
if [ "$COUNT" -eq 3 ]; then
pass "cluster has 3 nodes"
else
fail "cluster has $COUNT nodes (expected 3)"
echo " response: $NODES"
fi
# Verify all expected node IDs present
for nid in node-1 node-2 node-3; do
if echo "$NODES" | jq -e ".data[] | select(.node_id == \"$nid\")" > /dev/null 2>&1; then
pass "$nid present"
else
fail "$nid missing from cluster"
fi
done
# ── Test 2: Health endpoint includes cluster ──
echo "=== Test 2: Health includes cluster info ==="
HEALTH=$(curl -sf "$DIRECT_1/health")
CLUSTER_SIZE=$(echo "$HEALTH" | jq '.cluster.size // 0')
if [ "$CLUSTER_SIZE" -eq 3 ]; then
pass "health shows cluster.size=3"
else
fail "health shows cluster.size=$CLUSTER_SIZE (expected 3)"
fi
NODE_ID=$(echo "$HEALTH" | jq -r '.node_id // ""')
if [ -n "$NODE_ID" ]; then
pass "health includes node_id=$NODE_ID"
else
fail "health missing node_id"
fi
# ── Test 3: Stats contain expected keys ───────
echo "=== Test 3: Node stats ==="
STATS=$(echo "$NODES" | jq '.data[0].stats')
for key in goroutines heap_alloc uptime_sec ws_clients; do
if echo "$STATS" | jq -e ".$key" > /dev/null 2>&1; then
pass "stats.$key present"
else
fail "stats.$key missing"
fi
done
# ── Test 4: Stop one replica → stale sweep ────
echo "=== Test 4: Stop node-3, wait for sweep ==="
docker compose -f docker-compose-e2e.yml stop armature-3
echo " waiting ${STALE_WAIT}s for stale sweep..."
sleep "$STALE_WAIT"
NODES_AFTER=$(curl -sf "$LB/api/v1/admin/cluster" -H "$(auth)")
COUNT_AFTER=$(echo "$NODES_AFTER" | jq '.data | length')
if [ "$COUNT_AFTER" -eq 2 ]; then
pass "cluster swept to 2 nodes after stopping node-3"
else
fail "cluster has $COUNT_AFTER nodes after stop (expected 2)"
fi
# ── Test 5: Restart replica → re-registers ────
echo "=== Test 5: Restart node-3 ==="
docker compose -f docker-compose-e2e.yml start armature-3
# Wait for startup + at least one heartbeat
for i in $(seq 1 30); do
if curl -sf "http://localhost:8083/health" > /dev/null 2>&1; then
break
fi
sleep 1
done
sleep 8 # allow heartbeat to register
NODES_RESTART=$(curl -sf "$LB/api/v1/admin/cluster" -H "$(auth)")
COUNT_RESTART=$(echo "$NODES_RESTART" | jq '.data | length')
if [ "$COUNT_RESTART" -eq 3 ]; then
pass "cluster back to 3 nodes after restart"
else
fail "cluster has $COUNT_RESTART nodes after restart (expected 3)"
fi
# ── Summary ───────────────────────────────────
echo ""
echo "=== Results: $PASS passed, $FAIL failed ==="
if [ "$FAIL" -gt 0 ]; then
exit 1
fi

40
ci/e2e-nginx.conf
View File

@@ -1,40 +0,0 @@
events {
worker_connections 128;
}
http {
upstream armature {
server armature-1:80;
server armature-2:80;
server armature-3:80;
}
# WebSocket upgrade map
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 80;
location / {
proxy_pass http://armature;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# WebSocket endpoint
location /ws {
proxy_pass http://armature;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_read_timeout 86400s;
}
}
}

225
ci/e2e-smoke-driver.js
View File

@@ -1,225 +0,0 @@
#!/usr/bin/env node
/**
* E2E Smoke Driver — Playwright navigation smoke test
*
* Visits every installed surface and asserts:
* 1. The shell topbar (.sw-topbar) renders
* 2. No JS console errors
* 3. The home link exists
*
* On failure: captures full-page screenshot + console log.
*
* Usage:
* node ci/e2e-smoke-driver.js --server=http://localhost:3000 --token=TOKEN
*
* Exit codes: 0 = all passed, 1 = failures
*/
'use strict';
const { chromium } = require('playwright');
const fs = require('fs');
const path = require('path');
// Parse args
const args = {};
process.argv.slice(2).forEach(a => {
const [k, v] = a.replace(/^--/, '').split('=');
args[k] = v;
});
const SERVER = args.server || 'http://localhost:3000';
const TOKEN = args.token || '';
const SCREENSHOT_DIR = args.screenshots || '/tmp/e2e-screenshots';
if (!TOKEN) {
console.error('ERROR: --token is required');
process.exit(1);
}
// Surfaces that require URL parameters or aren't navigable directly
const SKIP_SURFACES = new Set([
'workflow', // requires /w/:id
'workflow-landing', // requires /w/:id/public
'welcome', // redirect/onboarding flow
'test-runners', // tested separately by surface-test-driver
]);
/**
* Discover all surfaces: core + extension packages with surface_route.
*/
async function discoverSurfaces(page) {
const surfaces = [];
// Core surfaces (always present)
const coreSurfaces = [
{ id: 'admin', route: '/admin' },
{ id: 'settings', route: '/settings' },
{ id: 'docs', route: '/docs' },
{ id: 'team-admin', route: '/team-admin' },
];
for (const s of coreSurfaces) {
if (!SKIP_SURFACES.has(s.id)) {
surfaces.push(s);
}
}
// Extension surfaces from API
try {
const packages = await page.evaluate(async (serverUrl) => {
const r = await fetch(serverUrl + '/api/v1/admin/packages', {
credentials: 'include',
});
if (r.status === 200) {
const body = await r.json();
return body.data || body || [];
}
return [];
}, SERVER);
for (const pkg of packages) {
if (!pkg.enabled) continue;
if (pkg.type !== 'surface' && pkg.type !== 'full') continue;
if (SKIP_SURFACES.has(pkg.id)) continue;
// Extension surfaces live at /s/{id}
const route = pkg.surface_route || ('/s/' + pkg.id);
surfaces.push({ id: pkg.id, route });
}
} catch (e) {
console.warn('WARN: Could not fetch package list:', e.message);
}
return surfaces;
}
(async () => {
const browser = await chromium.launch({ headless: true });
const context = await browser.newContext();
const page = await context.newPage();
// ── Authenticate via page context ─────────
// Navigate to health endpoint (no SPA boot, no SDK interference),
// set the arm_token cookie via document.cookie, then navigate to
// real surfaces. The Go page-auth middleware reads this cookie.
//
// We cannot use /login because the SDK boots, sees stale tokens in
// localStorage, tries /auth/refresh, fails, and clears the cookie.
// We cannot use Playwright's addCookies because SameSite=Strict
// cookies set externally aren't sent in Docker DNS environments.
console.log('Setting up auth...');
await page.goto(SERVER + '/api/v1/health', { waitUntil: 'networkidle', timeout: 30000 });
await page.evaluate((token) => {
document.cookie = `arm_token=${token}; path=/; max-age=604800`;
}, TOKEN);
// ── Discover surfaces ─────────────────────
console.log('Discovering surfaces...');
await page.goto(SERVER + '/admin', { waitUntil: 'networkidle', timeout: 30000 });
const surfaces = await discoverSurfaces(page);
console.log(`Found ${surfaces.length} surfaces: ${surfaces.map(s => s.id).join(', ')}`);
if (surfaces.length === 0) {
console.error('ERROR: No surfaces discovered');
await browser.close();
process.exit(1);
}
// ── Visit each surface ────────────────────
const results = [];
let failures = 0;
for (const surface of surfaces) {
const url = SERVER + surface.route;
const consoleErrors = [];
// Fresh console error collector per surface
const onConsole = msg => {
if (msg.type() === 'error') {
const text = msg.text();
// Ignore benign errors (favicon, service worker)
if (text.includes('favicon') || text.includes('service-worker')) return;
consoleErrors.push(text);
}
};
page.on('console', onConsole);
try {
process.stdout.write(` ${surface.id} (${surface.route}) ... `);
await page.goto(url, { waitUntil: 'networkidle', timeout: 30000 });
// Assert 1: Shell topbar renders (Preact SPA — wait for it to mount)
const topbar = await page.waitForSelector('.sw-topbar', { timeout: 15000 })
.catch(() => null);
if (!topbar) {
throw new Error('Shell topbar (.sw-topbar) not found after 15s');
}
// Assert 2: Home link exists (favicon link or any link to /)
const homeLink = await page.$('a[href="/"], a[href="./"], .sw-topbar__home');
if (!homeLink) {
throw new Error('Home link not found');
}
// Assert 3: No JS console errors
// Give a moment for any async errors to settle
await page.waitForTimeout(500);
if (consoleErrors.length > 0) {
throw new Error(`JS console errors: ${consoleErrors.join('; ')}`);
}
console.log('PASS');
results.push({ surface: surface.id, status: 'pass' });
// Capture baseline screenshot (informational, not a gate)
const slug = surface.id.replace(/[^a-z0-9-]/g, '_');
await page.screenshot({
path: path.join(SCREENSHOT_DIR, `baseline-${slug}.png`),
fullPage: true,
});
} catch (err) {
console.log('FAIL — ' + err.message);
failures++;
results.push({ surface: surface.id, status: 'fail', error: err.message });
// Screenshot + console log on failure
const slug = surface.id.replace(/[^a-z0-9-]/g, '_');
try {
await page.screenshot({
path: path.join(SCREENSHOT_DIR, `fail-${slug}.png`),
fullPage: true,
});
if (consoleErrors.length > 0) {
fs.writeFileSync(
path.join(SCREENSHOT_DIR, `fail-${slug}-console.log`),
consoleErrors.join('\n')
);
}
} catch (screenshotErr) {
console.warn(' (could not capture screenshot:', screenshotErr.message + ')');
}
} finally {
page.removeListener('console', onConsole);
}
}
// ── Summary ───────────────────────────────
console.log('\n═══ E2E Smoke Summary ═══');
console.log(` Total: ${results.length}`);
console.log(` Passed: ${results.filter(r => r.status === 'pass').length}`);
console.log(` Failed: ${failures}`);
if (failures > 0) {
console.log('\nFailed surfaces:');
for (const r of results.filter(r => r.status === 'fail')) {
console.log(` ✗ ${r.surface}: ${r.error}`);
}
}
await browser.close();
process.exit(failures > 0 ? 1 : 0);
})();

74
ci/e2e-smoke-test.sh
View File

@@ -1,74 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# E2E Smoke Test — CI Entrypoint
# ═══════════════════════════════════════════════
#
# Authenticates as admin, then runs a Playwright navigation smoke
# test against every installed surface. Asserts shell topbar
# renders, no JS console errors, and the home link works.
#
# Prerequisites:
# - Server running at $SERVER_URL (default: http://localhost:3000)
# - npx playwright install chromium
# - ADMIN_USER / ADMIN_PASS env vars (default: admin/admin)
#
# Exit codes: 0 = all surfaces passed, 1 = failures detected
set -euo pipefail
SERVER_URL="${SERVER_URL:-http://localhost:3000}"
ADMIN_USER="${ADMIN_USER:-admin}"
ADMIN_PASS="${ADMIN_PASS:-admin}"
SCREENSHOT_DIR="${SCREENSHOT_DIR:-/tmp/e2e-screenshots}"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
echo -e "${YELLOW}═══ E2E Smoke Test ═══${NC}"
echo " Server: ${SERVER_URL}"
# ── Ensure screenshot dir ────────────────────
mkdir -p "${SCREENSHOT_DIR}"
# ── Authenticate ─────────────────────────────
# Try PAT first (from bootstrap), fall back to login
PAT_FILE="/tmp/armature-admin-pat.txt"
TOKEN=""
if [ -f "$PAT_FILE" ]; then
TOKEN=$(cat "$PAT_FILE")
echo -e "${GREEN}Authenticated via bootstrap PAT${NC}"
fi
if [ -z "$TOKEN" ]; then
echo -e "${YELLOW}Authenticating via login...${NC}"
TOKEN=$(curl -sf -X POST "${SERVER_URL}/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d "{\"login\":\"${ADMIN_USER}\",\"password\":\"${ADMIN_PASS}\"}" \
| node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{console.log(JSON.parse(d).token)}catch(e){process.exit(1)}})")
fi
if [ -z "$TOKEN" ]; then
echo -e "${RED}Failed to authenticate${NC}"
exit 1
fi
echo -e "${GREEN}Authenticated${NC}"
# ── Run via Playwright ───────────────────────
echo -e "${YELLOW}Running E2E smoke tests via Playwright...${NC}"
node "$(dirname "$0")/e2e-smoke-driver.js" \
--server="${SERVER_URL}" \
--token="${TOKEN}" \
--screenshots="${SCREENSHOT_DIR}"
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
echo -e "${GREEN}═══ All E2E smoke tests passed ═══${NC}"
else
echo -e "${RED}═══ E2E smoke tests FAILED ═══${NC}"
echo -e "${YELLOW}Screenshots saved to: ${SCREENSHOT_DIR}${NC}"
fi
exit $EXIT_CODE

330
ci/e2e-upgrade-rolling.sh
View File

@@ -1,330 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# E2E Rolling Upgrade Test — Multi-Replica
# ═══════════════════════════════════════════════
#
# Tests rolling upgrade with two replicas sharing Postgres:
# 1. Build old + new images
# 2. Start 2 replicas (old) + nginx LB + postgres
# 3. Seed data via LB
# 4. Upgrade replica-1 → new, verify cross-replica reads
# 5. Upgrade replica-2 → new, verify full cluster
#
# Uses nginx for load balancing (same as e2e-chat-test).
#
# Usage:
# ./ci/e2e-upgrade-rolling.sh
#
# Exit codes: 0 = pass, 1 = failure
set -euo pipefail
LB="http://localhost:3000"
R1="http://localhost:8081"
R2="http://localhost:8082"
MAX_RETRIES=45
COMPOSE_FILE="docker-compose-e2e.yml"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
pass=0
fail=0
ok() { pass=$((pass + 1)); echo -e " ${GREEN}✓${NC} $1"; }
fail() { fail=$((fail + 1)); echo -e " ${RED}✗${NC} $1"; }
cleanup() {
echo -e "\n${YELLOW}Cleanup...${NC}"
docker compose -f "$COMPOSE_FILE" down -v 2>/dev/null || true
}
trap cleanup EXIT
wait_for_health() {
local host=$1 name=${2:-$host}
echo -e "${YELLOW}Waiting for $name...${NC}"
for i in $(seq 1 $MAX_RETRIES); do
if curl -sf "$host/api/v1/auth/login" -o /dev/null 2>/dev/null || \
curl -sf "$host" -o /dev/null 2>/dev/null; then
echo -e "${GREEN}$name ready after ${i}s${NC}"
return 0
fi
if [ "$i" -eq "$MAX_RETRIES" ]; then
echo -e "${RED}$name not ready after ${MAX_RETRIES}s${NC}"
return 1
fi
sleep 1
done
}
login() {
local user=$1 pass=$2 host=${3:-$LB}
local resp
resp=$(curl -sf "$host/api/v1/auth/login" \
-H 'Content-Type: application/json' \
-d "{\"login\":\"$user\",\"password\":\"$pass\"}")
echo "$resp" | grep -o '"access_token":"[^"]*"' | head -1 | sed 's/.*"access_token":"\([^"]*\)".*/\1/'
}
authed() {
local token=$1 method=$2 path=$3 host=${4:-$LB}
shift 3; shift 0 2>/dev/null || true
curl -sf -X "$method" "$host$path" \
-H "Authorization: Bearer $token" \
-H 'Content-Type: application/json' \
"$@"
}
# ═══════════════════════════════════════════════
# Phase 1: Build Images
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 1: Build Images ═══${NC}"
echo "Building 'old' image from HEAD..."
docker build -t armature:v-old . -q
ok "old image built"
echo "Building 'new' image from working tree..."
docker build -t armature:v-new . -q
ok "new image built"
# ═══════════════════════════════════════════════
# Phase 2: Start Both Replicas on Old Image
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 2: Start Cluster (Old Version) ═══${NC}"
# Override the build with old image
ARMATURE_IMAGE=armature:v-old \
docker compose -f "$COMPOSE_FILE" up postgres -d
# Wait for postgres
sleep 3
# Start replicas using old image
docker run -d --name sb-old-1 --network "$(basename "$(pwd)")_default" \
-e PORT=8080 -e BASE_PATH="" \
-e DB_DRIVER=postgres \
-e "DATABASE_URL=postgres://armature:e2e-password@postgres:5432/armature_e2e?sslmode=disable" \
-e JWT_SECRET=e2e-jwt-secret \
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
-e ARMATURE_ADMIN_USERNAME=admin \
-e ARMATURE_ADMIN_PASSWORD=admin \
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
-e LOG_FORMAT=text -e LOG_LEVEL=info \
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
-p 8081:80 armature:v-old 2>/dev/null || true
docker run -d --name sb-old-2 --network "$(basename "$(pwd)")_default" \
-e PORT=8080 -e BASE_PATH="" \
-e DB_DRIVER=postgres \
-e "DATABASE_URL=postgres://armature:e2e-password@postgres:5432/armature_e2e?sslmode=disable" \
-e JWT_SECRET=e2e-jwt-secret \
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
-e ARMATURE_ADMIN_USERNAME=admin \
-e ARMATURE_ADMIN_PASSWORD=admin \
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
-e LOG_FORMAT=text -e LOG_LEVEL=info \
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
-p 8082:80 armature:v-old 2>/dev/null || true
# Start nginx LB
docker compose -f "$COMPOSE_FILE" up lb -d
wait_for_health "$R1" "replica-1"
wait_for_health "$R2" "replica-2"
wait_for_health "$LB" "load balancer"
# ═══════════════════════════════════════════════
# Phase 3: Seed Data via LB
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 3: Seed Data ═══${NC}"
ADMIN_TOKEN=$(login admin admin)
if [ -z "$ADMIN_TOKEN" ]; then fail "admin login failed"; exit 1; fi
ok "admin logged in"
ALICE_TOKEN=$(login alice password123)
if [ -z "$ALICE_TOKEN" ]; then fail "alice login failed"; exit 1; fi
BOB_TOKEN=$(login bob password456)
if [ -z "$BOB_TOKEN" ]; then fail "bob login failed"; exit 1; fi
ALICE_ID=$(authed "$ALICE_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
BOB_ID=$(authed "$BOB_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
# Seed a note
NOTE=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
-d '{"title":"Rolling Upgrade Note","body":"Must survive rolling upgrade."}' 2>/dev/null || echo "{}")
NOTE_ID=$(echo "$NOTE" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$NOTE_ID" ]; then ok "note seeded"; else fail "note seed failed"; fi
# Seed a conversation + message
CONV=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/conversations" "" \
-d "{\"title\":\"Rolling Test\",\"type\":\"group\",\"participants\":[{\"id\":\"$BOB_ID\",\"display_name\":\"bob\"}],\"creator_display_name\":\"alice\"}" 2>/dev/null || echo "{}")
CONV_ID=$(echo "$CONV" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$CONV_ID" ]; then ok "conversation seeded"; else fail "conversation seed failed"; fi
if [ -n "$CONV_ID" ]; then
authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
-d '{"content":"Before rolling upgrade","content_type":"text"}' >/dev/null 2>&1
ok "message seeded"
fi
# ═══════════════════════════════════════════════
# Phase 4: Upgrade Replica-1, Verify Cross-Replica
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 4: Rolling Upgrade — Replica 1 ═══${NC}"
echo "Stopping old replica-1..."
docker stop sb-old-1 && docker rm sb-old-1 2>/dev/null || true
echo "Starting new replica-1..."
docker run -d --name sb-new-1 --network "$(basename "$(pwd)")_default" \
-e PORT=8080 -e BASE_PATH="" \
-e DB_DRIVER=postgres \
-e "DATABASE_URL=postgres://armature:e2e-password@postgres:5432/armature_e2e?sslmode=disable" \
-e JWT_SECRET=e2e-jwt-secret \
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
-e ARMATURE_ADMIN_USERNAME=admin \
-e ARMATURE_ADMIN_PASSWORD=admin \
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
-e LOG_FORMAT=text -e LOG_LEVEL=info \
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
-p 8081:80 armature:v-new
wait_for_health "$R1" "new replica-1"
# Cross-replica reads: new replica reads old data
R1_TOKEN=$(login alice password123 "$R1")
if [ -n "$R1_TOKEN" ]; then ok "alice login on new replica-1"; else fail "alice login on new replica-1"; fi
if [ -n "$NOTE_ID" ] && [ -n "$R1_TOKEN" ]; then
R1_NOTE=$(authed "$R1_TOKEN" GET "/s/notes/api/notes/$NOTE_ID" "$R1" 2>/dev/null || echo "{}")
if echo "$R1_NOTE" | grep -q "Rolling Upgrade Note"; then
ok "new replica-1 reads old data"
else
fail "new replica-1 cannot read old data"
fi
fi
# Write on new replica, read on old
if [ -n "$CONV_ID" ] && [ -n "$R1_TOKEN" ]; then
NEW_MSG=$(authed "$R1_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "$R1" \
-d '{"content":"From new replica-1","content_type":"text"}' 2>/dev/null || echo "{}")
if echo "$NEW_MSG" | grep -q '"id"'; then ok "write on new replica-1"; else fail "write on new replica-1"; fi
# Read from old replica-2
R2_TOKEN=$(login alice password123 "$R2")
if [ -n "$R2_TOKEN" ]; then
R2_MSGS=$(authed "$R2_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" "$R2" 2>/dev/null || echo "[]")
if echo "$R2_MSGS" | grep -q "From new replica-1"; then
ok "old replica-2 reads new replica-1 writes"
else
fail "old replica-2 cannot read new replica-1 writes"
fi
fi
fi
# ═══════════════════════════════════════════════
# Phase 5: Upgrade Replica-2, Verify Full Cluster
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 5: Rolling Upgrade — Replica 2 ═══${NC}"
echo "Stopping old replica-2..."
docker stop sb-old-2 && docker rm sb-old-2 2>/dev/null || true
echo "Starting new replica-2..."
docker run -d --name sb-new-2 --network "$(basename "$(pwd)")_default" \
-e PORT=8080 -e BASE_PATH="" \
-e DB_DRIVER=postgres \
-e "DATABASE_URL=postgres://armature:e2e-password@postgres:5432/armature_e2e?sslmode=disable" \
-e JWT_SECRET=e2e-jwt-secret \
-e "ENCRYPTION_KEY=e2e-encryption-key-32chars!!!!!" \
-e ARMATURE_ADMIN_USERNAME=admin \
-e ARMATURE_ADMIN_PASSWORD=admin \
-e STORAGE_BACKEND=pvc -e STORAGE_PATH=/data/storage \
-e "CORS_ALLOWED_ORIGINS=*" -e EXT_ALLOW_PRIVATE_IPS=true \
-e LOG_FORMAT=text -e LOG_LEVEL=info \
-e "SEED_USERS=alice:password123:user,bob:password456:user" \
-p 8082:80 armature:v-new
wait_for_health "$R2" "new replica-2"
# Verify full cluster: all data accessible
echo -e "\n${YELLOW}Verifying full cluster on new version...${NC}"
R2_TOKEN=$(login alice password123 "$R2")
if [ -n "$R2_TOKEN" ]; then ok "alice login on new replica-2"; else fail "alice login on new replica-2"; fi
if [ -n "$NOTE_ID" ] && [ -n "$R2_TOKEN" ]; then
R2_NOTE=$(authed "$R2_TOKEN" GET "/s/notes/api/notes/$NOTE_ID" "$R2" 2>/dev/null || echo "{}")
if echo "$R2_NOTE" | grep -q "Rolling Upgrade Note"; then
ok "new replica-2 reads all data"
else
fail "new replica-2 cannot read data"
fi
fi
if [ -n "$CONV_ID" ] && [ -n "$R2_TOKEN" ]; then
R2_ALL_MSGS=$(authed "$R2_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" "$R2" 2>/dev/null || echo "[]")
MSG_COUNT=$(echo "$R2_ALL_MSGS" | grep -o '"id"' | wc -l)
if [ "$MSG_COUNT" -ge 2 ]; then
ok "all messages accessible on new cluster ($MSG_COUNT msgs)"
else
fail "messages lost during rolling upgrade (got $MSG_COUNT)"
fi
fi
# Write on replica-2, read on replica-1 (both new)
if [ -n "$CONV_ID" ] && [ -n "$R2_TOKEN" ]; then
R2_MSG=$(authed "$R2_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "$R2" \
-d '{"content":"From new replica-2","content_type":"text"}' 2>/dev/null || echo "{}")
if echo "$R2_MSG" | grep -q '"id"'; then ok "write on new replica-2"; else fail "write on new replica-2"; fi
# Read from replica-1
sleep 1
R1_TOKEN=$(login alice password123 "$R1")
R1_CHECK=$(authed "$R1_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" "$R1" 2>/dev/null || echo "[]")
if echo "$R1_CHECK" | grep -q "From new replica-2"; then
ok "new replica-1 reads new replica-2 writes (pg_notify works)"
else
# pg_notify fan-out is async — try REST (which always works since shared DB)
ok "cross-replica write visible via REST (shared DB)"
fi
fi
# Check logs for errors
echo -e "\n${YELLOW}Checking logs...${NC}"
for name in sb-new-1 sb-new-2; do
LOGS=$(docker logs "$name" 2>&1 || echo "")
if echo "$LOGS" | grep -qi "panic\|fatal"; then
fail "$name has panic/fatal in logs"
else
ok "$name logs clean"
fi
done
# ═══════════════════════════════════════════════
# Summary
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══════════════════════════════════════${NC}"
echo -e " ${GREEN}Passed: $pass${NC} ${RED}Failed: $fail${NC}"
echo -e "${YELLOW}═══════════════════════════════════════${NC}"
# Extra cleanup for standalone containers
docker stop sb-new-1 sb-new-2 2>/dev/null || true
docker rm sb-new-1 sb-new-2 2>/dev/null || true
if [ "$fail" -gt 0 ]; then
exit 1
fi

330
ci/e2e-upgrade-test.sh
View File

@@ -1,330 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# E2E Upgrade Test — Data Integrity Across Upgrade
# ═══════════════════════════════════════════════
#
# Tests that a kernel upgrade preserves all seeded data:
# 1. Build "old" image from current commit
# 2. Start old image, seed data (notes, conversations, settings)
# 3. Stop old, start "new" image (built from working tree)
# 4. Verify data integrity post-upgrade
#
# Usage:
# ./ci/e2e-upgrade-test.sh
#
# Exit codes: 0 = pass, 1 = failure
set -euo pipefail
COMPOSE_FILE="docker-compose-upgrade.yml"
HOST="http://localhost:3001"
MAX_RETRIES=45
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
pass=0
fail=0
ok() { pass=$((pass + 1)); echo -e " ${GREEN}✓${NC} $1"; }
fail() { fail=$((fail + 1)); echo -e " ${RED}✗${NC} $1"; }
cleanup() {
echo -e "\n${YELLOW}Cleanup...${NC}"
docker compose -f "$COMPOSE_FILE" down -v 2>/dev/null || true
}
trap cleanup EXIT
# ── Helpers ────────────────────────────────────
wait_for_health() {
local host=$1
echo -e "${YELLOW}Waiting for $host...${NC}"
for i in $(seq 1 $MAX_RETRIES); do
if curl -sf "$host/api/v1/auth/login" -o /dev/null 2>/dev/null || \
curl -sf "$host" -o /dev/null 2>/dev/null; then
echo -e "${GREEN}Ready after ${i}s${NC}"
return 0
fi
if [ "$i" -eq "$MAX_RETRIES" ]; then
echo -e "${RED}Not ready after ${MAX_RETRIES}s${NC}"
return 1
fi
sleep 1
done
}
login() {
local user=$1 pass=$2 host=${3:-$HOST}
local resp
resp=$(curl -sf "$host/api/v1/auth/login" \
-H 'Content-Type: application/json' \
-d "{\"login\":\"$user\",\"password\":\"$pass\"}")
echo "$resp" | grep -o '"access_token":"[^"]*"' | head -1 | sed 's/.*"access_token":"\([^"]*\)".*/\1/'
}
authed() {
local token=$1 method=$2 path=$3 host=${4:-$HOST}
shift 3; shift 0 2>/dev/null || true
curl -sf -X "$method" "$host$path" \
-H "Authorization: Bearer $token" \
-H 'Content-Type: application/json' \
"$@"
}
# ═══════════════════════════════════════════════
# Phase 1: Build Images
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 1: Build Images ═══${NC}"
# Build "old" image from last commit (before current changes)
echo "Building 'old' image from HEAD commit..."
git stash -q 2>/dev/null || true
docker build --no-cache -t armature:v-old . -q
git stash pop -q 2>/dev/null || true
ok "old image built"
# Build "new" image from working tree (with current changes)
echo "Building 'new' image from working tree..."
docker build --no-cache -t armature:v-new . -q
ok "new image built"
# ═══════════════════════════════════════════════
# Phase 2: Start Old, Seed Data
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 2: Seed Data on Old Version ═══${NC}"
docker compose -f "$COMPOSE_FILE" up postgres armature-old -d
wait_for_health "$HOST"
# Auth
ADMIN_TOKEN=$(login admin admin)
if [ -z "$ADMIN_TOKEN" ]; then fail "admin login failed"; exit 1; fi
ok "admin logged in"
ALICE_TOKEN=$(login alice password123)
if [ -z "$ALICE_TOKEN" ]; then fail "alice login failed"; exit 1; fi
ok "alice logged in"
BOB_TOKEN=$(login bob password456)
if [ -z "$BOB_TOKEN" ]; then fail "bob login failed"; exit 1; fi
ok "bob logged in"
# Get user IDs
ALICE_ID=$(authed "$ALICE_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
BOB_ID=$(authed "$BOB_TOKEN" GET "/api/v1/profile" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
# Grant all permissions for extension packages (workaround for v0.5.4 PG grant bug)
echo -e "\n${YELLOW}Granting extension permissions...${NC}"
for pkg in notes chat-core workflow-chat workflow-demo; do
authed "$ADMIN_TOKEN" POST "/api/v1/admin/extensions/$pkg/permissions/grant-all" "" 2>/dev/null || true
done
ok "permissions granted"
# ── Seed notes ────────────────────────────────
echo -e "\n${YELLOW}Seeding notes...${NC}"
NOTE1=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
-d '{"title":"Upgrade Test Note","body":"This note must survive the upgrade."}' 2>/dev/null || echo "{}")
NOTE1_ID=$(echo "$NOTE1" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$NOTE1_ID" ]; then ok "note created: ${NOTE1_ID:0:8}..."; else fail "note creation failed"; fi
NOTE2=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
-d '{"title":"Second Note","body":"Content of second note."}' 2>/dev/null || echo "{}")
NOTE2_ID=$(echo "$NOTE2" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$NOTE2_ID" ]; then ok "second note created"; else fail "second note creation failed"; fi
# ── Seed conversations ────────────────────────
echo -e "\n${YELLOW}Seeding conversations...${NC}"
CONV=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/conversations" "" \
-d "{\"title\":\"Upgrade Test Chat\",\"type\":\"group\",\"participants\":[{\"id\":\"$BOB_ID\",\"display_name\":\"bob\"}],\"creator_display_name\":\"alice\"}" 2>/dev/null || echo "{}")
CONV_ID=$(echo "$CONV" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$CONV_ID" ]; then ok "conversation created"; else fail "conversation creation failed"; fi
if [ -n "$CONV_ID" ]; then
MSG1=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
-d '{"content":"Pre-upgrade message from alice","content_type":"text"}' 2>/dev/null || echo "{}")
MSG1_ID=$(echo "$MSG1" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$MSG1_ID" ]; then ok "message sent"; else fail "message send failed"; fi
MSG2=$(authed "$BOB_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
-d '{"content":"Pre-upgrade reply from bob","content_type":"text"}' 2>/dev/null || echo "{}")
MSG2_ID=$(echo "$MSG2" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$MSG2_ID" ]; then ok "reply sent"; else fail "reply send failed"; fi
fi
# ── Seed global config ─────────────────────────
echo -e "\n${YELLOW}Seeding global config...${NC}"
# Set a global config key to verify it survives upgrade
authed "$ADMIN_TOKEN" PUT "/api/v1/admin/config/upgrade_test_key" "" \
-d '{"value":"upgrade-test-value"}' 2>/dev/null && ok "global config set" || ok "global config set (endpoint may not exist)"
# Record installed packages
PRE_PACKAGES=$(authed "$ADMIN_TOKEN" GET "/api/v1/packages" 2>/dev/null || echo "[]")
PRE_PKG_COUNT=$(echo "$PRE_PACKAGES" | grep -o '"id"' | wc -l)
ok "recorded $PRE_PKG_COUNT installed packages"
# ═══════════════════════════════════════════════
# Phase 3: Upgrade
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 3: Upgrade ═══${NC}"
echo "Stopping old version..."
docker compose -f "$COMPOSE_FILE" stop armature-old
ok "old version stopped"
echo "Starting new version..."
docker compose -f "$COMPOSE_FILE" up armature-new -d
wait_for_health "$HOST"
ok "new version started"
# Check for migration errors in logs
echo -e "\n${YELLOW}Checking startup logs...${NC}"
LOGS=$(docker compose -f "$COMPOSE_FILE" logs armature-new 2>&1 || echo "")
if echo "$LOGS" | grep -qi "migration.*fail\|schema.*error\|panic\|fatal"; then
fail "migration errors found in logs"
echo "$LOGS" | grep -i "migration\|schema\|panic\|fatal" | head -5
else
ok "no migration errors in startup logs"
fi
if echo "$LOGS" | grep -qi "skipped.*existing"; then
ok "bundled packages correctly skipped existing"
else
# Not a failure — just note it
echo " (no skip-existing log found — may be expected)"
fi
# ═══════════════════════════════════════════════
# Phase 4: Verify Data Integrity
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══ Phase 4: Verify Data Integrity ═══${NC}"
# ── Auth ──────────────────────────────────────
echo -e "\n${YELLOW}4.1 Authentication${NC}"
ADMIN_TOKEN=$(login admin admin)
if [ -n "$ADMIN_TOKEN" ]; then ok "admin login post-upgrade"; else fail "admin login failed post-upgrade"; exit 1; fi
ALICE_TOKEN=$(login alice password123)
if [ -n "$ALICE_TOKEN" ]; then ok "alice login post-upgrade"; else fail "alice login failed post-upgrade"; fi
BOB_TOKEN=$(login bob password456)
if [ -n "$BOB_TOKEN" ]; then ok "bob login post-upgrade"; else fail "bob login failed post-upgrade"; fi
# ── Notes ─────────────────────────────────────
echo -e "\n${YELLOW}4.2 Notes${NC}"
if [ -n "$NOTE1_ID" ]; then
NOTE1_CHECK=$(authed "$ALICE_TOKEN" GET "/s/notes/api/notes/$NOTE1_ID" 2>/dev/null || echo "{}")
if echo "$NOTE1_CHECK" | grep -q "Upgrade Test Note"; then
ok "note title preserved"
else
fail "note title lost"
fi
if echo "$NOTE1_CHECK" | grep -q "survive the upgrade"; then
ok "note body preserved"
else
fail "note body lost"
echo " Response: ${NOTE1_CHECK:0:200}"
fi
fi
NOTES_LIST=$(authed "$ALICE_TOKEN" GET "/s/notes/api/notes" 2>/dev/null || echo "[]")
NOTE_COUNT=$(echo "$NOTES_LIST" | grep -o '"id"' | wc -l)
if [ "$NOTE_COUNT" -ge 2 ]; then
ok "all $NOTE_COUNT notes survived upgrade"
else
fail "expected at least 2 notes, got $NOTE_COUNT"
fi
# ── Conversations ─────────────────────────────
echo -e "\n${YELLOW}4.3 Conversations + Messages${NC}"
if [ -n "$CONV_ID" ]; then
CONV_CHECK=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/conversations" 2>/dev/null || echo "[]")
if echo "$CONV_CHECK" | grep -q "Upgrade Test Chat"; then
ok "conversation survived upgrade"
else
fail "conversation lost"
fi
MSGS_CHECK=$(authed "$ALICE_TOKEN" GET "/s/chat-core/api/messages/$CONV_ID?limit=50" 2>/dev/null || echo "[]")
if echo "$MSGS_CHECK" | grep -q "Pre-upgrade message from alice"; then
ok "alice's message survived"
else
fail "alice's message lost"
fi
if echo "$MSGS_CHECK" | grep -q "Pre-upgrade reply from bob"; then
ok "bob's reply survived"
else
fail "bob's reply lost"
fi
fi
# ── Packages ──────────────────────────────────
echo -e "\n${YELLOW}4.4 Packages${NC}"
POST_PACKAGES=$(authed "$ADMIN_TOKEN" GET "/api/v1/packages" 2>/dev/null || echo "[]")
POST_PKG_COUNT=$(echo "$POST_PACKAGES" | grep -o '"id"' | wc -l)
if [ "$POST_PKG_COUNT" -ge "$PRE_PKG_COUNT" ]; then
ok "package count preserved: $POST_PKG_COUNT (was $PRE_PKG_COUNT)"
else
fail "package count decreased: $POST_PKG_COUNT (was $PRE_PKG_COUNT)"
fi
# ── Settings / Config ─────────────────────────
echo -e "\n${YELLOW}4.5 Settings${NC}"
# Verify packages still have their settings endpoint accessible
SETTINGS_CHECK=$(authed "$ADMIN_TOKEN" GET "/api/v1/admin/packages/notes/settings" 2>/dev/null || echo "{}")
if echo "$SETTINGS_CHECK" | grep -q '"values"'; then
ok "package settings endpoint accessible"
else
fail "package settings endpoint broken"
fi
# ── Post-upgrade writes ──────────────────────
echo -e "\n${YELLOW}4.6 Post-upgrade Writes${NC}"
# Create a new note on the upgraded instance
POST_NOTE=$(authed "$ALICE_TOKEN" POST "/s/notes/api/notes" "" \
-d '{"title":"Post-upgrade Note","body":"Written after upgrade."}' 2>/dev/null || echo "{}")
POST_NOTE_ID=$(echo "$POST_NOTE" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$POST_NOTE_ID" ]; then ok "post-upgrade note creation works"; else fail "post-upgrade note creation failed"; fi
# Send a message on the upgraded instance
if [ -n "$CONV_ID" ]; then
POST_MSG=$(authed "$ALICE_TOKEN" POST "/s/chat-core/api/messages/$CONV_ID" "" \
-d '{"content":"Post-upgrade message","content_type":"text"}' 2>/dev/null || echo "{}")
POST_MSG_ID=$(echo "$POST_MSG" | grep -o '"id":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -n "$POST_MSG_ID" ]; then ok "post-upgrade message send works"; else fail "post-upgrade message send failed"; fi
fi
# ═══════════════════════════════════════════════
# Summary
# ═══════════════════════════════════════════════
echo -e "\n${YELLOW}═══════════════════════════════════════${NC}"
echo -e " ${GREEN}Passed: $pass${NC} ${RED}Failed: $fail${NC}"
echo -e "${YELLOW}═══════════════════════════════════════${NC}"
if [ "$fail" -gt 0 ]; then
exit 1
fi

232
ci/e2e-workflow-handoff.sh
View File

@@ -1,232 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# E2E Workflow Handoff Test
# ═══════════════════════════════════════════════
#
# Verifies the public→team handoff flow:
# 1. Public visitor completes a form stage
# 2. Visitor sees "submitted" screen (not the team stage)
# 3. Team user sees assignment, claims it, completes review
# 4. Instance status is "completed"
#
# Prerequisites:
# - Server running at $SERVER_URL (default: http://localhost:3000)
# - ADMIN_USER / ADMIN_PASS env vars (default: admin/admin)
#
# Exit codes: 0 = all assertions passed, 1 = failures detected
set -euo pipefail
SERVER_URL="${SERVER_URL:-http://localhost:3000}"
ADMIN_USER="${ADMIN_USER:-admin}"
ADMIN_PASS="${ADMIN_PASS:-admin}"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
PASS=0
FAIL=0
assert() {
local desc="$1" ok="$2"
if [ "$ok" = "true" ]; then
echo -e " ${GREEN}✓${NC} $desc"
PASS=$((PASS + 1))
else
echo -e " ${RED}✗${NC} $desc"
FAIL=$((FAIL + 1))
fi
}
# Parse JSON field via node (portable)
json_field() {
node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{console.log(JSON.parse(d)$1||'')}catch(e){console.log('')}})" 2>/dev/null
}
echo -e "${YELLOW}═══ E2E Workflow Handoff Test ═══${NC}"
echo " Server: ${SERVER_URL}"
# ── Authenticate ─────────────────────────────
TOKEN=$(curl -sf -X POST "${SERVER_URL}/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d "{\"login\":\"${ADMIN_USER}\",\"password\":\"${ADMIN_PASS}\"}" \
| json_field ".token" || true)
if [ -z "$TOKEN" ]; then
echo -e "${RED}Failed to authenticate${NC}"
exit 1
fi
echo -e "${GREEN}Authenticated${NC}"
AUTH="Authorization: Bearer ${TOKEN}"
# Get admin user ID
USER_ID=$(curl -sf -H "$AUTH" "${SERVER_URL}/api/v1/profile" | json_field ".id")
assert "Got admin user ID" "$([ -n "$USER_ID" ] && echo true || echo false)"
# ── Phase 1: Create team ─────────────────────
echo -e "\n${YELLOW}Phase 1: Create test team${NC}"
TEAM_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/admin/teams" \
-H "$AUTH" -H "Content-Type: application/json" \
-d '{"name": "E2E Handoff Team", "slug": "e2e-handoff-team"}' 2>/dev/null || echo '{}')
TEAM_ID=$(echo "$TEAM_RESP" | json_field ".id")
assert "Team created" "$([ -n "$TEAM_ID" ] && echo true || echo false)"
# Add admin as team member
if [ -n "$TEAM_ID" ] && [ -n "$USER_ID" ]; then
curl -sf -X POST "${SERVER_URL}/api/v1/teams/${TEAM_ID}/members" \
-H "$AUTH" -H "Content-Type: application/json" \
-d "{\"user_id\":\"${USER_ID}\",\"role\":\"admin\"}" >/dev/null 2>&1 || true
fi
# ── Phase 2: Create workflow with public + team stages ──
echo -e "\n${YELLOW}Phase 2: Create workflow (public form → team review)${NC}"
WF_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflows" \
-H "$AUTH" -H "Content-Type: application/json" \
-d '{
"name": "E2E Handoff Test",
"slug": "e2e-handoff-test",
"description": "Tests public to team handoff",
"entry_mode": "public_link",
"is_active": true
}' 2>/dev/null || echo '{"error":"failed"}')
WF_ID=$(echo "$WF_RESP" | json_field ".id")
assert "Workflow created" "$([ -n "$WF_ID" ] && echo true || echo false)"
if [ -z "$WF_ID" ]; then
echo -e "${RED}Cannot continue without workflow ID${NC}"
exit 1
fi
# Stage 1: public form
STAGE1_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflows/${WF_ID}/stages" \
-H "$AUTH" -H "Content-Type: application/json" \
-d '{
"name": "Customer Request",
"stage_mode": "form",
"audience": "public",
"ordinal": 0,
"form_template": {
"fields": [
{"key": "name", "type": "text", "label": "Your Name", "required": true},
{"key": "request", "type": "textarea", "label": "Request Details"}
]
}
}' 2>/dev/null || echo '{}')
STAGE1_ID=$(echo "$STAGE1_RESP" | json_field ".id")
assert "Public form stage created" "$([ -n "$STAGE1_ID" ] && echo true || echo false)"
# Stage 2: team review (with assignment)
STAGE2_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflows/${WF_ID}/stages" \
-H "$AUTH" -H "Content-Type: application/json" \
-d "{
\"name\": \"Team Review\",
\"stage_mode\": \"form\",
\"audience\": \"team\",
\"ordinal\": 1,
\"assignment_team_id\": \"${TEAM_ID}\"
}" 2>/dev/null || echo '{}')
STAGE2_ID=$(echo "$STAGE2_RESP" | json_field ".id")
assert "Team review stage created" "$([ -n "$STAGE2_ID" ] && echo true || echo false)"
# ── Phase 3: Public visitor starts and completes form ──
echo -e "\n${YELLOW}Phase 3: Public visitor submits form${NC}"
START_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflow-entry/global/e2e-handoff-test" \
-H "Content-Type: application/json" \
-d '{}' 2>/dev/null || echo '{"error":"failed"}')
INST_ID=$(echo "$START_RESP" | json_field ".id")
ENTRY_TOKEN=$(echo "$START_RESP" | json_field ".entry_token")
assert "Instance created" "$([ -n "$INST_ID" ] && echo true || echo false)"
assert "Entry token present" "$([ -n "$ENTRY_TOKEN" ] && echo true || echo false)"
# Submit form data (advance past stage 1)
if [ -n "$ENTRY_TOKEN" ]; then
ADV_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/public/workflows/advance/${ENTRY_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"data": {"name": "Test User", "request": "Please review this"}}' 2>/dev/null || echo '{}')
ADV_STATUS=$(echo "$ADV_RESP" | json_field ".status")
assert "Instance advanced to team stage (status=active)" "$([ "$ADV_STATUS" = "active" ] && echo true || echo false)"
fi
# ── Phase 4: Verify audience mismatch screen ──
echo -e "\n${YELLOW}Phase 4: Verify visitor sees submitted screen${NC}"
if [ -n "$INST_ID" ]; then
# Fetch the workflow page without auth (public visitor)
WF_PAGE=$(curl -sf "${SERVER_URL}/w/${INST_ID}" 2>/dev/null || echo "")
HAS_SUBMITTED=$(echo "$WF_PAGE" | grep -c "Submitted Successfully" || true)
assert "Page shows 'Submitted Successfully'" "$([ "$HAS_SUBMITTED" -gt 0 ] && echo true || echo false)"
HAS_FORM=$(echo "$WF_PAGE" | grep -c 'id="formArea"' || true)
assert "Page does NOT show form area" "$([ "$HAS_FORM" -eq 0 ] && echo true || echo false)"
fi
# ── Phase 5: Team user sees assignment ─────────
echo -e "\n${YELLOW}Phase 5: Team assignment appears${NC}"
if [ -n "$TEAM_ID" ]; then
ASSIGN_RESP=$(curl -sf -H "$AUTH" "${SERVER_URL}/api/v1/teams/${TEAM_ID}/assignments?status=unassigned" 2>/dev/null || echo '{}')
ASSIGN_COUNT=$(echo "$ASSIGN_RESP" | node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{const r=JSON.parse(d); console.log((r.data||r).length||0)}catch(e){console.log(0)}})" 2>/dev/null)
assert "Unassigned assignment exists" "$([ "$ASSIGN_COUNT" -gt 0 ] && echo true || echo false)"
# Get assignment ID
ASSIGN_ID=$(echo "$ASSIGN_RESP" | node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{const r=JSON.parse(d); console.log((r.data||r)[0].id||'')}catch(e){console.log('')}})" 2>/dev/null)
fi
# ── Phase 6: Claim and complete assignment ─────
echo -e "\n${YELLOW}Phase 6: Claim and complete${NC}"
if [ -n "$ASSIGN_ID" ]; then
# Claim
CLAIM_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/assignments/${ASSIGN_ID}/claim" \
-H "$AUTH" -H "Content-Type: application/json" -d '{}' 2>/dev/null || echo '{}')
CLAIMED=$(echo "$CLAIM_RESP" | json_field ".claimed")
assert "Assignment claimed" "$([ "$CLAIMED" = "true" ] && echo true || echo false)"
# Complete (advance the review stage)
COMPLETE_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/assignments/${ASSIGN_ID}/complete" \
-H "$AUTH" -H "Content-Type: application/json" \
-d '{"review_data": {"decision": "approved", "comment": "Looks good"}}' 2>/dev/null || echo '{}')
COMPLETED=$(echo "$COMPLETE_RESP" | json_field ".completed")
assert "Assignment completed" "$([ "$COMPLETED" = "true" ] && echo true || echo false)"
fi
# ── Phase 7: Verify instance is completed ──────
echo -e "\n${YELLOW}Phase 7: Verify final state${NC}"
if [ -n "$INST_ID" ]; then
FINAL_RESP=$(curl -sf -H "$AUTH" "${SERVER_URL}/api/v1/workflows/${WF_ID}/instances/${INST_ID}" 2>/dev/null || echo '{}')
FINAL_STATUS=$(echo "$FINAL_RESP" | json_field ".status")
assert "Instance status is completed" "$([ "$FINAL_STATUS" = "completed" ] && echo true || echo false)"
fi
# ── Cleanup ──────────────────────────────────
echo -e "\n${YELLOW}Cleanup${NC}"
curl -sf -X DELETE "${SERVER_URL}/api/v1/workflows/${WF_ID}" -H "$AUTH" >/dev/null 2>&1 || true
if [ -n "$TEAM_ID" ]; then
curl -sf -X DELETE "${SERVER_URL}/api/v1/admin/teams/${TEAM_ID}" -H "$AUTH" >/dev/null 2>&1 || true
fi
echo -e " Deleted test resources"
# ── Summary ──────────────────────────────────
echo ""
TOTAL=$((PASS + FAIL))
if [ $FAIL -eq 0 ]; then
echo -e "${GREEN}═══ All ${TOTAL} assertions passed ═══${NC}"
exit 0
else
echo -e "${RED}═══ ${FAIL}/${TOTAL} assertions failed ═══${NC}"
exit 1
fi

177
ci/e2e-workflow-nochat.sh
View File

@@ -1,177 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# E2E Workflow Test — Without Chat
# ═══════════════════════════════════════════════
#
# Verifies the complete workflow lifecycle works without chat or
# chat-core packages installed. Uses only admin API + PAT auth.
#
# Proves: workflow independence from optional packages.
#
# Prerequisites:
# - Server running at $SERVER_URL (default: http://localhost:3000)
# - ADMIN_USER / ADMIN_PASS env vars (default: admin/admin)
#
# Exit codes: 0 = all assertions passed, 1 = failures detected
set -euo pipefail
SERVER_URL="${SERVER_URL:-http://localhost:3000}"
ADMIN_USER="${ADMIN_USER:-admin}"
ADMIN_PASS="${ADMIN_PASS:-admin}"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
PASS=0
FAIL=0
assert() {
local desc="$1" ok="$2"
if [ "$ok" = "true" ]; then
echo -e " ${GREEN}✓${NC} $desc"
PASS=$((PASS + 1))
else
echo -e " ${RED}✗${NC} $desc"
FAIL=$((FAIL + 1))
fi
}
echo -e "${YELLOW}═══ E2E Workflow Independence Test ═══${NC}"
echo " Server: ${SERVER_URL}"
# ── Authenticate ─────────────────────────────
TOKEN=""
PAT_FILE="/tmp/armature-admin-pat.txt"
if [ -f "$PAT_FILE" ]; then
TOKEN=$(cat "$PAT_FILE")
fi
if [ -z "$TOKEN" ]; then
TOKEN=$(curl -sf -X POST "${SERVER_URL}/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d "{\"login\":\"${ADMIN_USER}\",\"password\":\"${ADMIN_PASS}\"}" \
| node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{console.log(JSON.parse(d).token)}catch(e){process.exit(1)}})" 2>/dev/null || true)
fi
if [ -z "$TOKEN" ]; then
echo -e "${RED}Failed to authenticate${NC}"
exit 1
fi
echo -e "${GREEN}Authenticated${NC}"
AUTH="Authorization: Bearer ${TOKEN}"
# ── Verify chat is NOT installed ─────────────
echo -e "\n${YELLOW}Phase 1: Verify chat packages not required${NC}"
CHAT_PKG=$(curl -sf -H "$AUTH" "${SERVER_URL}/api/v1/admin/packages" \
| node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{
const pkgs = JSON.parse(d).data || JSON.parse(d);
const chat = (Array.isArray(pkgs) ? pkgs : []).filter(p => p.id === 'chat' || p.id === 'chat-core');
console.log(JSON.stringify(chat));
})" 2>/dev/null || echo "[]")
echo " Chat packages found: ${CHAT_PKG}"
# ── Create a test workflow ───────────────────
echo -e "\n${YELLOW}Phase 2: Create workflow via admin API${NC}"
WF_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflows" \
-H "$AUTH" -H "Content-Type: application/json" \
-d '{
"name": "E2E No-Chat Test",
"slug": "e2e-nochat-test",
"description": "Workflow independence E2E test",
"entry_mode": "public_link",
"is_active": true
}' 2>/dev/null || echo '{"error":"failed"}')
WF_ID=$(echo "$WF_RESP" | node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{console.log(JSON.parse(d).id||'')}catch(e){console.log('')}})" 2>/dev/null)
assert "Workflow created" "$([ -n "$WF_ID" ] && echo true || echo false)"
if [ -z "$WF_ID" ]; then
echo -e "${RED}Cannot continue without workflow ID${NC}"
echo "Response: ${WF_RESP}"
exit 1
fi
# ── Add a form stage ─────────────────────────
echo -e "\n${YELLOW}Phase 3: Add form stage${NC}"
STAGE_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflows/${WF_ID}/stages" \
-H "$AUTH" -H "Content-Type: application/json" \
-d '{
"name": "Intake Form",
"stage_mode": "form",
"ordinal": 0,
"form_template": {
"fields": [
{"key": "title", "type": "text", "label": "Issue Title", "required": true},
{"key": "description", "type": "textarea", "label": "Description"}
]
}
}' 2>/dev/null || echo '{"error":"failed"}')
STAGE_ID=$(echo "$STAGE_RESP" | node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{console.log(JSON.parse(d).id||'')}catch(e){console.log('')}})" 2>/dev/null)
assert "Form stage created" "$([ -n "$STAGE_ID" ] && echo true || echo false)"
# ── Add a review stage ───────────────────────
REVIEW_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflows/${WF_ID}/stages" \
-H "$AUTH" -H "Content-Type: application/json" \
-d '{
"name": "Manager Review",
"stage_mode": "form",
"ordinal": 1
}' 2>/dev/null || echo '{"error":"failed"}')
REVIEW_ID=$(echo "$REVIEW_RESP" | node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{console.log(JSON.parse(d).id||'')}catch(e){console.log('')}})" 2>/dev/null)
assert "Review stage created" "$([ -n "$REVIEW_ID" ] && echo true || echo false)"
# ── Landing page responds ────────────────────
echo -e "\n${YELLOW}Phase 4: Landing page accessible${NC}"
LANDING_STATUS=$(curl -so /dev/null -w "%{http_code}" "${SERVER_URL}/w/global/e2e-nochat-test" 2>/dev/null || echo "000")
assert "Landing page returns 200" "$([ "$LANDING_STATUS" = "200" ] && echo true || echo false)"
# ── Start workflow via public API ────────────
echo -e "\n${YELLOW}Phase 5: Start workflow (public entry)${NC}"
START_RESP=$(curl -sf -X POST "${SERVER_URL}/api/v1/workflow-entry/global/e2e-nochat-test" \
-H "Content-Type: application/json" \
-d '{}' 2>/dev/null || echo '{"error":"failed"}')
INST_ID=$(echo "$START_RESP" | node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c; process.stdin.on('end',()=>{try{console.log(JSON.parse(d).id||'')}catch(e){console.log('')}})" 2>/dev/null)
assert "Instance created" "$([ -n "$INST_ID" ] && echo true || echo false)"
# ── Workflow page renders ────────────────────
echo -e "\n${YELLOW}Phase 6: Workflow execution page${NC}"
if [ -n "$INST_ID" ]; then
WF_PAGE_STATUS=$(curl -so /dev/null -w "%{http_code}" "${SERVER_URL}/w/${INST_ID}" 2>/dev/null || echo "000")
assert "Workflow page returns 200" "$([ "$WF_PAGE_STATUS" = "200" ] && echo true || echo false)"
fi
# ── Cleanup: delete workflow ─────────────────
echo -e "\n${YELLOW}Cleanup${NC}"
curl -sf -X DELETE "${SERVER_URL}/api/v1/workflows/${WF_ID}" \
-H "$AUTH" >/dev/null 2>&1 || true
echo -e " Deleted test workflow"
# ── Summary ──────────────────────────────────
echo ""
TOTAL=$((PASS + FAIL))
if [ $FAIL -eq 0 ]; then
echo -e "${GREEN}═══ All ${TOTAL} assertions passed ═══${NC}"
exit 0
else
echo -e "${RED}═══ ${FAIL}/${TOTAL} assertions failed ═══${NC}"
exit 1
fi

65
ci/e2e-ws-listener.js
View File

@@ -1,65 +0,0 @@
#!/usr/bin/env node
/**
* E2E WebSocket Listener — waits for a specific realtime event.
*
* Usage:
* node e2e-ws-listener.js --url ws://localhost:3000/ws --token JWT \
* --channel conversation:abc --event message --timeout 10
*
* Connects to WebSocket, authenticates, subscribes to channel,
* waits for the specified event type. Prints the payload JSON
* to stdout and exits 0 on match, or exits 1 on timeout.
*/
const WebSocket = require('ws');
const args = {};
for (let i = 2; i < process.argv.length; i += 2) {
args[process.argv[i].replace('--', '')] = process.argv[i + 1];
}
const url = args.url || 'ws://localhost:3000/ws';
const token = args.token || '';
const channel = args.channel || '';
const event = args.event || 'message';
const timeout = parseInt(args.timeout || '10', 10) * 1000;
if (!token || !channel) {
console.error('Usage: --url WS_URL --token JWT --channel CHANNEL --event EVENT [--timeout SECS]');
process.exit(1);
}
const ws = new WebSocket(url);
let timer = null;
ws.on('open', () => {
// Authenticate
ws.send(JSON.stringify({ type: 'auth', token: token }));
// Subscribe to channel
ws.send(JSON.stringify({ type: 'room.subscribe', room: channel }));
});
ws.on('message', (data) => {
try {
const msg = JSON.parse(data.toString());
// Match event on the subscribed channel
if (msg.type === 'event' && msg.channel === channel && msg.event === event) {
console.log(JSON.stringify(msg.data || msg));
clearTimeout(timer);
ws.close();
process.exit(0);
}
} catch (e) {
// Ignore non-JSON messages
}
});
ws.on('error', (err) => {
console.error('WS error:', err.message);
process.exit(1);
});
timer = setTimeout(() => {
console.error('Timeout waiting for event: ' + event + ' on channel: ' + channel);
ws.close();
process.exit(1);
}, timeout);

20
ci/keycloak-realm.json
View File

@@ -1,5 +1,5 @@
{
"realm": "armature",
"realm": "switchboard",
"enabled": true,
"registrationAllowed": false,
"loginWithEmailAllowed": true,
@@ -9,11 +9,11 @@
"realm": [
{
"name": "sb-admin",
"description": "Armature admin role"
"description": "Switchboard admin role"
},
{
"name": "sb-user",
"description": "Armature regular user role"
"description": "Switchboard regular user role"
}
]
},
@@ -29,11 +29,11 @@
],
"clients": [
{
"clientId": "armature",
"name": "Armature",
"clientId": "switchboard",
"name": "Switchboard Core",
"enabled": true,
"publicClient": false,
"secret": "armature-secret",
"secret": "switchboard-secret",
"redirectUris": [
"http://localhost:3000/*",
"http://localhost:8080/*"
@@ -69,7 +69,7 @@
"users": [
{
"username": "alice",
"email": "alice@armature.test",
"email": "alice@switchboard.test",
"firstName": "Alice",
"lastName": "Engineer",
"enabled": true,
@@ -83,7 +83,7 @@
],
"realmRoles": [
"sb-user",
"default-roles-armature"
"default-roles-switchboard"
],
"groups": [
"/engineering"
@@ -91,7 +91,7 @@
},
{
"username": "bob",
"email": "bob@armature.test",
"email": "bob@switchboard.test",
"firstName": "Bob",
"lastName": "Lead",
"enabled": true,
@@ -106,7 +106,7 @@
"realmRoles": [
"sb-admin",
"sb-user",
"default-roles-armature"
"default-roles-switchboard"
],
"groups": [
"/engineering",

56
ci/run-surface-tests.sh
View File

@@ -1,56 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# Surface Test Runner — CI Entrypoint
# ═══════════════════════════════════════════════
#
# Authenticates as admin, navigates to the test-runners surface
# via Playwright, runs all test suites, and asserts zero failures.
#
# Prerequisites:
# - Server running at $SERVER_URL (default: http://localhost:3000)
# - npx playwright install chromium
# - ADMIN_USER / ADMIN_PASS env vars (default: admin/admin)
#
# Exit codes: 0 = all tests passed, 1 = failures detected
set -euo pipefail
SERVER_URL="${SERVER_URL:-http://localhost:3000}"
ADMIN_USER="${ADMIN_USER:-admin}"
ADMIN_PASS="${ADMIN_PASS:-admin}"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
echo -e "${YELLOW}═══ Surface Test Runner ═══${NC}"
echo " Server: ${SERVER_URL}"
# ── Authenticate ─────────────────────────────
echo -e "${YELLOW}Authenticating...${NC}"
TOKEN=$(curl -sf -X POST "${SERVER_URL}/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d "{\"login\":\"${ADMIN_USER}\",\"password\":\"${ADMIN_PASS}\"}" \
| node -e "process.stdin.resume(); let d=''; process.stdin.on('data',c=>d+=c); process.stdin.on('end',()=>{try{console.log(JSON.parse(d).token)}catch(e){process.exit(1)}})")
if [ -z "$TOKEN" ]; then
echo -e "${RED}Failed to authenticate${NC}"
exit 1
fi
echo -e "${GREEN}Authenticated${NC}"
# ── Run via Playwright ───────────────────────
echo -e "${YELLOW}Running surface tests via Playwright...${NC}"
node "$(dirname "$0")/surface-test-driver.js" \
--server="${SERVER_URL}" \
--token="${TOKEN}"
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
echo -e "${GREEN}═══ All surface tests passed ═══${NC}"
else
echo -e "${RED}═══ Surface tests FAILED ═══${NC}"
fi
exit $EXIT_CODE

141
ci/surface-test-driver.js
View File

@@ -1,141 +0,0 @@
#!/usr/bin/env node
/**
* Surface Test Driver — Playwright-based CI runner
*
* Navigates to /s/test-runners as admin, clicks "Run All",
* waits for completion, then fetches results from the API.
*
* Usage:
* node ci/surface-test-driver.js --server=http://localhost:3000 --token=TOKEN
*
* Exit codes: 0 = all passed, 1 = failures
*/
'use strict';
const { chromium } = require('playwright');
// Parse args
const args = {};
process.argv.slice(2).forEach(a => {
const [k, v] = a.replace(/^--/, '').split('=');
args[k] = v;
});
const SERVER = args.server || 'http://localhost:3000';
const TOKEN = args.token || '';
if (!TOKEN) {
console.error('ERROR: --token is required');
process.exit(1);
}
(async () => {
const browser = await chromium.launch({ headless: true });
const context = await browser.newContext();
const page = await context.newPage();
// Authenticate via page context — navigate to a non-SPA endpoint,
// set the arm_token cookie via document.cookie. Cannot use /login
// (SDK boot clears stale tokens) or addCookies (SameSite=Strict
// fails in Docker DNS environments).
console.log('Setting up auth...');
await page.goto(SERVER + '/api/v1/health', { waitUntil: 'networkidle', timeout: 30000 });
await page.evaluate((token) => {
document.cookie = `arm_token=${token}; path=/; max-age=604800`;
}, TOKEN);
// Collect console errors
const consoleErrors = [];
page.on('console', msg => {
if (msg.type() === 'error') consoleErrors.push(msg.text());
});
try {
// Navigate to test runners surface
console.log('Navigating to test-runners surface...');
await page.goto(SERVER + '/s/test-runners', { waitUntil: 'networkidle', timeout: 30000 });
// Wait for the "Run All" button — suites load asynchronously so the
// button only appears once runners have registered their suites.
console.log('Waiting for Run All button...');
const runAllBtn = await page.waitForSelector('button:has-text("Run All")', { timeout: 60000 })
.catch(() => null);
if (!runAllBtn) {
// Dump page content for debugging
const text = await page.textContent('body').catch(() => '(empty)');
console.error('ERROR: "Run All" button not found. Page text:', text.substring(0, 500));
await browser.close();
process.exit(1);
}
console.log('Clicking Run All...');
await runAllBtn.click();
// Wait for results — poll until running state clears
console.log('Waiting for test completion...');
// The running indicator disappears when tests finish
// Max wait: 5 minutes
const maxWait = 300000;
const start = Date.now();
let done = false;
while (!done && (Date.now() - start) < maxWait) {
await page.waitForTimeout(2000);
// Check if results are available via API
try {
const resp = await page.evaluate(async (serverUrl) => {
const r = await fetch(serverUrl + '/api/v1/admin/test-runners/results', {
credentials: 'include'
});
if (r.status === 200) return await r.json();
return null;
}, SERVER);
if (resp && resp.summary) {
done = true;
console.log('\n═══ Results ═══');
console.log(` Total: ${resp.summary.total}`);
console.log(` Passed: ${resp.summary.passed}`);
console.log(` Failed: ${resp.summary.failed}`);
console.log(` Warned: ${resp.summary.warned}`);
console.log(` Skipped: ${resp.summary.skipped}`);
console.log(` Duration: ${resp.duration_ms}ms`);
if (resp.summary.failed > 0) {
console.log('\n═══ Failures ═══');
for (const suite of (resp.suites || [])) {
for (const test of (suite.tests || [])) {
if (test.status === 'failed') {
console.log(` ✗ ${suite.name} > ${test.name}: ${test.detail || 'failed'}`);
}
}
}
await browser.close();
process.exit(1);
}
await browser.close();
process.exit(0);
}
} catch (e) {
// Results not ready yet
}
}
if (!done) {
console.error('ERROR: Tests did not complete within timeout');
await browser.close();
process.exit(1);
}
} catch (e) {
console.error('Driver error:', e.message);
if (consoleErrors.length > 0) {
console.error('\nBrowser console errors:');
consoleErrors.forEach(e => console.error(' ' + e));
}
await browser.close();
process.exit(1);
}
})();

21
ci/wait-for-healthy.sh
View File

@@ -1,21 +0,0 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════
# Wait for Armature server to be healthy
# ═══════════════════════════════════════════════
# Usage: ./ci/wait-for-healthy.sh [URL] [MAX_RETRIES]
set -euo pipefail
HOST="${1:-http://localhost:3000}"
MAX_RETRIES="${2:-60}"
echo "Waiting for ${HOST} to be healthy..."
for i in $(seq 1 "$MAX_RETRIES"); do
if curl -sf --connect-timeout 2 "${HOST}/api/v1/health" -o /dev/null 2>/dev/null; then
echo "Server healthy after ${i}s"
exit 0
fi
sleep 1
done
echo "Server not healthy after ${MAX_RETRIES}s"
exit 1

127
docker-compose-e2e.yml
View File

@@ -1,127 +0,0 @@
# docker-compose-e2e.yml — Multi-replica E2E testing (Postgres)
#
# Three Armature replicas behind an nginx load balancer,
# sharing a single Postgres instance. Tests cross-replica
# broadcast via pg_notify and cluster registry (v0.6.0).
#
# Usage:
# docker compose -f docker-compose-e2e.yml up --build -d
# ./ci/e2e-chat-test.sh # chat E2E
# ./ci/e2e-cluster-test.sh # cluster registry E2E
# docker compose -f docker-compose-e2e.yml down -v
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: armature_e2e
POSTGRES_USER: armature
POSTGRES_PASSWORD: e2e-password
ports:
- "5432:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U armature -d armature_e2e"]
interval: 2s
timeout: 5s
retries: 10
armature-1:
build:
context: .
dockerfile: Dockerfile
environment:
PORT: "8080"
BASE_PATH: ""
DB_DRIVER: postgres
DATABASE_URL: postgres://armature:e2e-password@postgres:5432/armature_e2e?sslmode=disable
JWT_SECRET: e2e-jwt-secret
ENCRYPTION_KEY: e2e-encryption-key-32chars!!!!!
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: admin
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
CORS_ALLOWED_ORIGINS: "*"
EXT_ALLOW_PRIVATE_IPS: "true"
LOG_FORMAT: text
LOG_LEVEL: info
SEED_USERS: "alice:password123:user,bob:password456:user,charlie:password789:user"
CLUSTER_NODE_ID: "node-1"
CLUSTER_HEARTBEAT_INTERVAL: "5s"
CLUSTER_STALE_THRESHOLD: "15s"
CLUSTER_ENDPOINT: "http://armature-1:8080"
depends_on:
postgres:
condition: service_healthy
ports:
- "8081:80"
armature-2:
build:
context: .
dockerfile: Dockerfile
environment:
PORT: "8080"
BASE_PATH: ""
DB_DRIVER: postgres
DATABASE_URL: postgres://armature:e2e-password@postgres:5432/armature_e2e?sslmode=disable
JWT_SECRET: e2e-jwt-secret
ENCRYPTION_KEY: e2e-encryption-key-32chars!!!!!
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: admin
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
CORS_ALLOWED_ORIGINS: "*"
EXT_ALLOW_PRIVATE_IPS: "true"
LOG_FORMAT: text
LOG_LEVEL: info
SEED_USERS: "alice:password123:user,bob:password456:user,charlie:password789:user"
CLUSTER_NODE_ID: "node-2"
CLUSTER_HEARTBEAT_INTERVAL: "5s"
CLUSTER_STALE_THRESHOLD: "15s"
CLUSTER_ENDPOINT: "http://armature-2:8080"
depends_on:
postgres:
condition: service_healthy
ports:
- "8082:80"
armature-3:
build:
context: .
dockerfile: Dockerfile
environment:
PORT: "8080"
BASE_PATH: ""
DB_DRIVER: postgres
DATABASE_URL: postgres://armature:e2e-password@postgres:5432/armature_e2e?sslmode=disable
JWT_SECRET: e2e-jwt-secret
ENCRYPTION_KEY: e2e-encryption-key-32chars!!!!!
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: admin
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
CORS_ALLOWED_ORIGINS: "*"
EXT_ALLOW_PRIVATE_IPS: "true"
LOG_FORMAT: text
LOG_LEVEL: info
SEED_USERS: "alice:password123:user,bob:password456:user,charlie:password789:user"
CLUSTER_NODE_ID: "node-3"
CLUSTER_HEARTBEAT_INTERVAL: "5s"
CLUSTER_STALE_THRESHOLD: "15s"
CLUSTER_ENDPOINT: "http://armature-3:8080"
depends_on:
postgres:
condition: service_healthy
ports:
- "8083:80"
lb:
image: nginx:alpine
volumes:
- ./ci/e2e-nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "3000:80"
depends_on:
- armature-1
- armature-2
- armature-3

22
docker-compose-keycloak.yml
View File

@@ -6,11 +6,11 @@
# docker compose -f docker-compose.yml -f docker-compose-keycloak.yml up
#
# This extends the base docker-compose.yml to add:
# - Keycloak IdP with a pre-configured "armature" realm
# - Armature configured with AUTH_MODE=oidc pointing at Keycloak
# - Keycloak IdP with a pre-configured "switchboard" realm
# - Switchboard configured with AUTH_MODE=oidc pointing at Keycloak
#
# After startup:
# Armature: http://localhost:3000
# Switchboard: http://localhost:3000
# Keycloak: http://localhost:8180 (admin / admin)
#
# Pre-configured test users in Keycloak:
@@ -21,14 +21,14 @@
# ============================================
services:
# Override armature to use OIDC
armature:
# Override switchboard to use OIDC
switchboard:
environment:
AUTH_MODE: oidc
OIDC_ISSUER_URL: http://keycloak:8080/realms/armature
OIDC_EXTERNAL_ISSUER_URL: http://localhost:8180/realms/armature
OIDC_CLIENT_ID: armature
OIDC_CLIENT_SECRET: armature-secret
OIDC_ISSUER_URL: http://keycloak:8080/realms/switchboard
OIDC_EXTERNAL_ISSUER_URL: http://localhost:8180/realms/switchboard
OIDC_CLIENT_ID: switchboard
OIDC_CLIENT_SECRET: switchboard-secret
OIDC_REDIRECT_URL: http://localhost:3000/api/v1/auth/oidc/callback
OIDC_AUTO_ACTIVATE: "true"
OIDC_ADMIN_ROLE: sb-admin
@@ -38,7 +38,7 @@ services:
keycloak:
image: quay.io/keycloak/keycloak:26.0
container_name: armature-keycloak
container_name: switchboard-keycloak
command:
- start-dev
- --import-realm
@@ -48,7 +48,7 @@ services:
KC_HTTP_PORT: "8080"
KC_HEALTH_ENABLED: "true"
volumes:
- ./ci/keycloak-realm.json:/opt/keycloak/data/import/armature-realm.json:ro
- ./ci/keycloak-realm.json:/opt/keycloak/data/import/switchboard-realm.json:ro
ports:
- "8180:8080"
healthcheck:

86
docker-compose-upgrade.yml
View File

@@ -1,86 +0,0 @@
# docker-compose-upgrade.yml — Upgrade Testing (Postgres)
#
# Two-phase environment: run the "old" image to seed data, then
# swap in the "new" image and verify data integrity post-upgrade.
#
# Usage:
# ci/e2e-upgrade-test.sh (orchestrates build → seed → upgrade → verify)
#
# Manual usage:
# docker build -t armature:v-old .
# docker compose -f docker-compose-upgrade.yml up postgres armature-old -d
# # ... seed data ...
# docker compose -f docker-compose-upgrade.yml stop armature-old
# docker compose -f docker-compose-upgrade.yml up armature-new -d
# # ... verify data ...
# docker compose -f docker-compose-upgrade.yml down -v
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: armature_upgrade
POSTGRES_USER: armature
POSTGRES_PASSWORD: upgrade-password
ports:
- "5433:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U armature -d armature_upgrade"]
interval: 2s
timeout: 5s
retries: 10
armature-old:
image: armature:v-old
environment:
PORT: "8080"
BASE_PATH: ""
DB_DRIVER: postgres
DATABASE_URL: postgres://armature:upgrade-password@postgres:5432/armature_upgrade?sslmode=disable
JWT_SECRET: upgrade-jwt-secret
ENCRYPTION_KEY: upgrade-encryption-key-32chars!!
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: admin
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
CORS_ALLOWED_ORIGINS: "*"
EXT_ALLOW_PRIVATE_IPS: "true"
LOG_FORMAT: text
LOG_LEVEL: info
SEED_USERS: "alice:password123:user,bob:password456:user"
depends_on:
postgres:
condition: service_healthy
ports:
- "3001:80"
volumes:
- upgrade_storage:/data/storage
armature-new:
image: core-armature-new
environment:
PORT: "8080"
BASE_PATH: ""
DB_DRIVER: postgres
DATABASE_URL: postgres://armature:upgrade-password@postgres:5432/armature_upgrade?sslmode=disable
JWT_SECRET: upgrade-jwt-secret
ENCRYPTION_KEY: upgrade-encryption-key-32chars!!
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: admin
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
CORS_ALLOWED_ORIGINS: "*"
EXT_ALLOW_PRIVATE_IPS: "true"
LOG_FORMAT: text
LOG_LEVEL: info
SEED_USERS: "alice:password123:user,bob:password456:user"
depends_on:
postgres:
condition: service_healthy
ports:
- "3001:80"
volumes:
- upgrade_storage:/data/storage
volumes:
upgrade_storage:

69
docker-compose.ci.yml
View File

@@ -1,69 +0,0 @@
# docker-compose.ci.yml — CI test override
#
# Extends base docker-compose.yml. Adds a healthcheck to armature and
# Playwright-based test services. All containers share the default compose
# bridge network, so services reach armature via Docker DNS (http://armature:80).
#
# Usage (test-runner):
# docker compose -f docker-compose.yml -f docker-compose.ci.yml up --build \
# --abort-on-container-exit --exit-code-from test-runner
#
# Usage (e2e-smoke):
# docker compose -f docker-compose.yml -f docker-compose.ci.yml up --build \
# --abort-on-container-exit --exit-code-from e2e-smoke
#
# The workflow exits with the selected service's exit code (0 = pass, 1 = fail).
#
# NOTE: Do NOT use network_mode: host — the workflow container and compose
# containers are in separate network namespaces inside DinD. Use the default
# bridge network and let services talk via Docker DNS instead.
services:
armature:
healthcheck:
test: ["CMD", "curl", "-sf", "http://localhost:80/api/v1/health"]
interval: 2s
timeout: 3s
retries: 30
start_period: 5s
test-runner:
build:
context: .
dockerfile_inline: |
FROM mcr.microsoft.com/playwright:v1.52.0-noble
WORKDIR /work
RUN npm init -y && npm install playwright@1.52.0
COPY ci/ /work/ci/
RUN chmod +x /work/ci/*.sh
depends_on:
armature:
condition: service_healthy
working_dir: /work
environment:
SERVER_URL: http://armature:80
ADMIN_USER: admin
ADMIN_PASS: admin
command: ["bash", "-c", "./ci/run-surface-tests.sh"]
e2e-smoke:
build:
context: .
dockerfile_inline: |
FROM mcr.microsoft.com/playwright:v1.52.0-noble
WORKDIR /work
RUN npm init -y && npm install playwright@1.52.0
COPY ci/ /work/ci/
RUN chmod +x /work/ci/*.sh
depends_on:
armature:
condition: service_healthy
working_dir: /work
environment:
SERVER_URL: http://armature:80
ADMIN_USER: admin
ADMIN_PASS: admin
SCREENSHOT_DIR: /tmp/e2e-screenshots
volumes:
- /tmp/e2e-screenshots:/tmp/e2e-screenshots
command: ["bash", "-c", "./ci/e2e-smoke-test.sh"]

20
docker-compose.yml
View File

@@ -7,40 +7,36 @@
# docker compose up --build
# open http://localhost:3000
#
# Data persists in the `sb_data` named volume between restarts.
# To reset: docker compose down -v
# Data persists in ./data/ between restarts.
# To reset: rm -rf ./data/
#
# For Postgres / multi-replica / k8s deployment see k8s/ and Dockerfile.
services:
armature:
switchboard:
build:
context: .
dockerfile: Dockerfile
container_name: armature
container_name: switchboard-core
environment:
PORT: "8080"
BASE_PATH: ""
DB_DRIVER: sqlite
DATABASE_URL: /data/armature.db
DATABASE_URL: /data/switchboard.db
JWT_SECRET: ${JWT_SECRET:-change-me-for-production}
ENCRYPTION_KEY: ${ENCRYPTION_KEY:-change-me-for-production}
ARMATURE_ADMIN_USERNAME: ${ADMIN_USERNAME:-admin}
ARMATURE_ADMIN_PASSWORD: ${ADMIN_PASSWORD:-admin}
SWITCHBOARD_ADMIN_USERNAME: ${ADMIN_USERNAME:-admin}
SWITCHBOARD_ADMIN_PASSWORD: ${ADMIN_PASSWORD:-admin}
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
CORS_ALLOWED_ORIGINS: ${CORS_ALLOWED_ORIGINS:-http://localhost:3000}
EXT_ALLOW_PRIVATE_IPS: ${EXT_ALLOW_PRIVATE_IPS:-true}
LOG_FORMAT: ${LOG_FORMAT:-text}
LOG_LEVEL: ${LOG_LEVEL:-info}
BUNDLED_PACKAGES: ${BUNDLED_PACKAGES:-}
# Dev seed users — ignored if ENVIRONMENT=production
SEED_USERS: ${SEED_USERS:-alice:password123:user,bob:password456:user,charlie:password789:user}
volumes:
- sb_data:/data
- ./data:/data
ports:
- "3000:80"
restart: unless-stopped
volumes:
sb_data:

8
docker-entrypoint.sh
View File

@@ -1,6 +1,6 @@
#!/bin/bash
# ============================================
# Armature - Backend Launcher
# Switchboard Core - Backend Launcher
# ============================================
# Runs as an nginx entrypoint.d hook.
# Starts the Go backend in the background
@@ -20,15 +20,15 @@ else
-exec sed -i "s|%%BASE_PATH%%||g" {} +
fi
echo "🔀 Starting Armature backend..."
echo "🔀 Starting Switchboard Core backend..."
# Kill any stale backend from a previous entrypoint run
pkill -f /usr/local/bin/armature 2>/dev/null || true
pkill -f /usr/local/bin/switchboard 2>/dev/null || true
sleep 0.2
# Launch Go backend in background (from /app so migrations are found)
cd /app
/usr/local/bin/armature &
/usr/local/bin/switchboard &
BACKEND_PID=$!
# Wait for backend to be ready (max 60s)

213
docs/API-REFERENCE.md
View File

@@ -1,213 +0,0 @@
# API Reference
Base path: `/api/v1` (all endpoints below are relative to this unless noted).
## Authentication
Most endpoints require a Bearer token in the `Authorization` header:
```
Authorization: Bearer <access_token>
```
Obtain tokens via the auth endpoints (no auth required):
| Method | Path | Description |
|--------|------|-------------|
| POST | `/auth/register` | Create account (if registration enabled) |
| POST | `/auth/login` | Login, returns access + refresh tokens |
| POST | `/auth/refresh` | Refresh access token |
| POST | `/auth/logout` | Invalidate refresh token |
| GET | `/auth/oidc/login` | OIDC login redirect (when `AUTH_MODE=oidc`) |
| GET | `/auth/oidc/callback` | OIDC callback handler |
## Error Format
All errors return JSON:
```json
{"error": "description of what went wrong"}
```
Standard HTTP status codes: 400 (bad request), 401 (unauthorized), 403 (forbidden), 404 (not found), 500 (server error).
## Pagination
List endpoints accept query parameters:
| Param | Default | Description |
|-------|---------|-------------|
| `limit` | 50 | Max items to return |
| `offset` | 0 | Number of items to skip |
## Endpoints by Category
### Profile & Settings
| Method | Path | Description |
|--------|------|-------------|
| GET | `/profile` | Current user profile |
| PUT | `/profile` | Update profile |
| POST | `/profile/password` | Change password |
| GET | `/settings` | User settings |
| PUT | `/settings` | Update user settings |
| GET | `/profile/permissions` | List granted permissions |
| GET | `/profile/bootstrap` | Bootstrap data for frontend |
### Surfaces & Extensions
| Method | Path | Description |
|--------|------|-------------|
| GET | `/surfaces` | List enabled surfaces |
| GET | `/extensions` | List user extensions |
| POST | `/extensions/:id/settings` | Update extension settings |
| GET | `/extensions/:id/manifest` | Get extension manifest |
| GET | `/settings/public` | Public platform settings |
### Notifications
| Method | Path | Description |
|--------|------|-------------|
| GET | `/notifications` | List notifications |
| GET | `/notifications/unread-count` | Unread count |
| PATCH | `/notifications/:id/read` | Mark as read |
| POST | `/notifications/mark-all-read` | Mark all read |
| DELETE | `/notifications/:id` | Delete notification |
| GET | `/notifications/preferences` | Notification preferences |
| PUT | `/notifications/preferences/:type` | Set preference |
### Workflows
| Method | Path | Description |
|--------|------|-------------|
| GET | `/workflows` | List workflows |
| POST | `/workflows` | Create workflow |
| GET | `/workflows/:id` | Get workflow |
| PATCH | `/workflows/:id` | Update workflow |
| DELETE | `/workflows/:id` | Delete workflow |
| POST | `/workflows/:id/publish` | Publish version |
| POST | `/workflows/:id/clone` | Clone workflow |
| POST | `/workflows/:id/instances` | Start instance |
| GET | `/workflows/:id/instances` | List instances |
| GET | `/assignments/mine` | My assignments |
### Teams
| Method | Path | Description |
|--------|------|-------------|
| GET | `/teams/mine` | List my teams |
| GET | `/teams/:id/members` | Team members |
| POST | `/teams/:id/members` | Add member |
| GET | `/teams/:id/roles` | Team roles |
### Connections
| Method | Path | Description |
|--------|------|-------------|
| GET | `/connections` | List connections |
| POST | `/connections` | Create connection |
| GET | `/connections/resolve` | Resolve connection by type |
| PUT | `/connections/:id` | Update connection |
| DELETE | `/connections/:id` | Delete connection |
### Packages (User)
| Method | Path | Description |
|--------|------|-------------|
| GET | `/packages` | List visible packages |
| POST | `/packages/install` | Install personal package |
| DELETE | `/packages/:id` | Delete personal package |
### Admin Endpoints
All under `/api/v1/admin/` -- require `surface.admin.access` permission.
| Method | Path | Description |
|--------|------|-------------|
| GET | `/admin/users` | List users |
| POST | `/admin/users` | Create user |
| GET | `/admin/stats` | Platform statistics |
| GET | `/admin/settings` | Global settings |
| PUT | `/admin/settings/:key` | Update setting |
| GET | `/admin/packages` | List all packages |
| POST | `/admin/packages/install` | Install package (.pkg upload) |
| POST | `/admin/packages/:id/update` | Update package |
| GET | `/admin/packages/:id/export` | Export package as .pkg |
| PUT | `/admin/packages/:id/enable` | Enable package |
| PUT | `/admin/packages/:id/disable` | Disable package |
| DELETE | `/admin/packages/:id` | Delete package |
| GET | `/admin/cluster` | Cluster node list (Postgres only) |
| POST | `/admin/backup` | Create backup |
| GET | `/admin/backups` | List backups |
| POST | `/admin/restore` | Restore from backup |
### Health & Monitoring
These are at the base path (not under `/api/v1`):
| Method | Path | Description |
|--------|------|-------------|
| GET | `/health` | Basic health check |
| GET | `/healthz/live` | Liveness probe (Kubernetes) |
| GET | `/healthz/ready` | Readiness probe (Kubernetes) |
| GET | `/metrics` | Prometheus metrics |
| GET | `/api/docs` | OpenAPI documentation UI (Swagger) |
| GET | `/api/docs/openapi.json` | Dynamic OpenAPI 3.0 spec (includes extension routes) |
| GET | `/api/docs/openapi.yaml` | Static kernel-only OpenAPI spec (YAML) |
### Webhooks (Public)
| Method | Path | Description |
|--------|------|-------------|
| GET/POST | `/api/v1/hooks/:package_id/:slug` | Inbound webhook for extensions |
| POST | `/api/v1/workflows/public/:id/start` | Public workflow entry |
## WebSocket
Connect to `/ws` with a ticket-based auth flow:
1. POST `/api/v1/ws/ticket` with Bearer token to get a one-time ticket.
2. Connect to `/ws?ticket=<ticket>`.
The WebSocket carries JSON-framed events. Subscribe to channels with:
```json
{"type": "subscribe", "channel": "notifications"}
```
Kernel event prefixes: `user.*`, `team.*`, `workflow.*`, `notification.*`, `presence.*`, `extension.*`, `admin.*`, `system.*`.
### Presence
| Method | Path | Description |
|--------|------|-------------|
| POST | `/presence/heartbeat` | Update presence status |
| GET | `/presence` | Query online users |
| GET | `/users/search` | Search users |
### Forms (v0.9.5)
| Method | Path | Description |
|--------|------|-------------|
| POST | `/forms/validate` | Validate form data against a typed template |
**Request body:**
```json
{
"template": {
"fields": [
{"key": "name", "type": "text", "label": "Name", "required": true}
]
},
"data": {"name": "Alice"}
}
```
**Response:**
```json
{"valid": true, "errors": []}
```
On validation failure, `errors` contains `[{"key": "name", "message": "Name is required"}]`.

151
docs/ARCHITECTURE.md
View File

@@ -1,6 +1,6 @@
# Armature — Architecture
# Switchboard Core — Architecture
Armature is a self-hosted extension platform. It provides identity,
Switchboard Core is a self-hosted extension platform. It provides identity,
teams, permissions, storage, workflows, notifications, and a package system.
Everything else — chat, AI providers, personas, knowledge bases, notes,
tools — ships as installable extensions.
@@ -25,32 +25,6 @@ package installer. Surfaces (UI pages), tools, filters, triggers, and
providers are all packages. The editor, chat, and admin UI will themselves
be installable surface packages.
## System Architecture Overview
```mermaid
graph TD
Browser["Browser (Preact + htm)"]
Server["Go Server (Gin)"]
DB["Database (SQLite / Postgres)"]
Browser -->|HTTP / WebSocket| Server
subgraph Server Layers
Handlers["HTTP Handlers"]
Sandbox["Starlark Sandbox"]
Store["Store Interface"]
EventBus["Event Bus"]
end
Server --> Handlers
Handlers --> Store
Handlers --> Sandbox
Sandbox --> Store
Handlers --> EventBus
Store --> DB
EventBus -->|SSE / WS| Browser
```
## Kernel Components
### Identity & Auth
@@ -62,31 +36,6 @@ permissions, settings, and data against.
Auth modes: `builtin` (password), `mtls` (client cert), `oidc` (Keycloak
et al.). Mode is set at deploy time via `AUTH_MODE` env var.
### Request Flow
```mermaid
sequenceDiagram
participant C as Client
participant CORS as CORS Middleware
participant Auth as Auth Middleware
participant H as Handler
participant S as Store
participant DB as Database
C->>CORS: HTTP Request
CORS->>Auth: Pass through
Auth->>Auth: Validate token / session
alt Unauthorized
Auth-->>C: 401 JSON error
end
Auth->>H: Authenticated context
H->>S: Store method call
S->>DB: SQL query
DB-->>S: Rows
S-->>H: Typed result
H-->>C: JSON response
```
### Teams & Groups
Teams provide horizontal isolation — users see only their team's resources.
@@ -114,26 +63,6 @@ The unified registry for all installable content. A package is a surface
Tiers: `browser` (JS only), `starlark` (sandboxed server-side),
`sidecar` (container, future).
#### Extension Lifecycle
```mermaid
graph TD
Upload["Package Upload (.pkg)"]
Parse["Manifest Parse"]
Insert["DB Insert (packages table)"]
Enable["Admin Enable Toggle"]
Mount["Surface Mount (/s/:slug)"]
SDK["SDK Boot"]
Render["Renderer Registration"]
Upload --> Parse
Parse --> Insert
Insert --> Enable
Enable --> Mount
Mount --> SDK
SDK --> Render
```
### Starlark Sandbox
Extensions declare capabilities in their manifest. The admin grants or
@@ -199,27 +128,6 @@ Server-sent events to WebSocket clients. Kernel prefixes:
Extensions will subscribe to event patterns at install time (v0.2.0).
Match expressions start as exact strings, grow to globs later.
#### Realtime Event Flow
```mermaid
sequenceDiagram
participant C as Client
participant WS as WebSocket Hub
participant PG as PG LISTEN/NOTIFY
participant Other as Other Node
C->>WS: WS connect + subscribe
Note over WS: Local fan-out to subscribers
WS-->>C: Event push
Other->>PG: NOTIFY channel, payload
PG->>WS: LISTEN callback
WS-->>C: Fan-out to local subscribers
WS->>PG: NOTIFY channel, payload
PG->>Other: LISTEN callback
```
### Storage & Notifications
**Object storage**: PVC (local disk) or S3-compatible. Used by extensions
@@ -248,36 +156,15 @@ SQLite parity rules: `boolToInt` for boolean binding, `store.NewID()` for
INSERT RETURNING, no `NULLS FIRST`, no boolean literals, no `$N` reuse,
`database.ST()`/`database.SNT()` wrappers for time scanning.
## Settings Cascade
```mermaid
graph TD
Global["Global Scope (admin)"]
Team["Team Override"]
User["User Override"]
RBAC{"RBAC Gate: who can set at what scope?"}
Flag{"user_overridable flag"}
Effective["Effective Value"]
Global --> RBAC
RBAC -->|allowed| Team
RBAC -->|denied| Effective
Team --> Flag
Flag -->|true| User
Flag -->|false| Effective
User --> Effective
```
## Frontend
Preact (3KB) + htm (tagged template literals). No build step, no bundler
(except CM6 via esbuild). ES modules loaded via `<script type="module">`.
The SDK is exposed at `window.sw` — see the [Frontend JS Guide](FRONTEND-JS-GUIDE).
(except CM6 via esbuild). IIFE/global-namespace pattern with
`sb.register()`/`sb.ns()`.
The shell provides a two-slot topbar (left title + center slot) that every
surface inherits. Extensions use `window.html` and `window.preact` directly.
Hooks via `window.hooks`. Vendor libs (marked.js, DOMPurify, KaTeX,
CodeMirror 6) baked into the image.
The shell loads surfaces into a viewport. Extensions use `window.html`
and `window.preact` directly. Hooks via `window.hooks`. Vendor libs
(marked.js, DOMPurify, KaTeX, CodeMirror 6) baked into the image.
## Deployment
@@ -285,27 +172,5 @@ Single Docker image: Go binary + migrations + frontend assets + vendor
libs. Kubernetes deployment with 3-node PG cluster. CI via Gitea Actions
with DaemonSet DinD runners testing both PG and SQLite pipelines.
Registry: `registry.gobha.me:5000/xcaliber/armature`
Registry: `registry.gobha.me:5000/xcaliber/switchboard-core`
Namespace: `gobha-ai-chat`
### Cluster Topology
```mermaid
graph TD
N1["Node 1"]
N2["Node 2"]
N3["Node 3"]
PG["PG UNLOGGED Table (cluster_nodes)"]
Sweep["Stale Sweep Goroutine"]
Notify["PG LISTEN/NOTIFY"]
N1 -->|heartbeat INSERT/UPDATE| PG
N2 -->|heartbeat INSERT/UPDATE| PG
N3 -->|heartbeat INSERT/UPDATE| PG
Sweep -->|DELETE nodes not seen| PG
PG -->|join/leave events| Notify
Notify --> N1
Notify --> N2
Notify --> N3
```

330
docs/DEMO-WORKFLOWS.md
View File

@@ -1,330 +0,0 @@
# Demo Workflows — v0.3.6
Four example workflow packages that prove the engine works end-to-end.
Each is a self-contained `.pkg` archive installable via Admin > Packages.
## Feature Capability Matrix
| Feature | Bug Report | Onboarding | Content Approval | Webhook |
|---------|:----------:|:----------:|:----------------:|:-------:|
| Public entry (anonymous) | ✅ | | | |
| Progressive fieldsets | ✅ | ✅ | | |
| Branch rules (conditional routing) | ✅ | | | |
| Team assignment + claim | ✅ | | | |
| SLA timer | ✅ | | | |
| Starlark automated stage | | ✅ | | ✅ |
| db.insert (ext_data tables) | | ✅ | | ✅ |
| notifications.send | | ✅ | | |
| http.post (outbound webhook) | | | | ✅ |
| connections.get (credentials) | | | | ✅ |
| Signoff gate (required_role) | | ✅ | | |
| Multi-party signoff (quorum) | | | ✅ | |
| Rejection reroute | | ✅ | ✅ | |
| Review cycle (loop) | | | ✅ | |
---
## Package 1: Bug Report Triage
**Slug:** `bug-report-triage`
**Entry mode:** `public_link` — anyone with the link can submit
**Tier:** browser (no Starlark needed)
### Stage Flow
```
[submit] → [classify] → [fix-critical] → [verify]
public team ↘ [fix-normal] ↗ team
team
```
### Stages
| # | Name | Mode | Audience | Key Feature |
|---|------|------|----------|-------------|
| 0 | submit | form | public | 2 fieldsets: "Report Details" (email, summary, component, severity) + "Reproduction" (steps, expected, actual) |
| 1 | classify | form | team | Team reviews. Branch rules: severity=critical → fix-critical, else → fix-normal |
| 2 | fix-critical | form | team | Senior team queue. SLA: 3600s |
| 3 | fix-normal | form | team | Standard fix queue |
| 4 | verify | review | team | Verify fix, close out |
### Branch Rules (stage 1)
```json
[
{"field": "severity", "op": "eq", "value": "critical", "target_stage": "fix-critical"},
{"field": "severity", "op": "neq", "value": "critical", "target_stage": "fix-normal"}
]
```
### API Walkthrough
```bash
# 1. Start public instance (no auth required)
curl -X POST $BASE/api/v1/public/workflows/$WF_ID/start \
-H 'Content-Type: application/json' \
-d '{"data": {"reporter_email": "user@example.com", "summary": "Login broken", "component": "ui", "severity": "critical", "steps": "1. Click login", "expected": "Login page", "actual": "500 error"}}'
# Response includes entry_token for anonymous continuation
# {"id": "inst-xxx", "entry_token": "tok-yyy", ...}
# 2. Resume (check status)
curl $BASE/api/v1/public/workflows/resume/tok-yyy
# 3. Team member claims the classify assignment
curl -X POST $BASE/api/v1/teams/$TEAM/assignments/$ASSIGN_ID/claim \
-H "Authorization: Bearer $TOKEN"
# 4. Advance classify stage (triggers branch rule)
curl -X POST $BASE/api/v1/teams/$TEAM/instances/$INST/advance \
-H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' \
-d '{"data": {"severity": "critical", "notes": "Confirmed production issue"}}'
# → Routes to fix-critical (not fix-normal)
```
---
## Package 2: Employee Onboarding
**Slug:** `employee-onboarding`
**Entry mode:** `team_only`
**Tier:** starlark
**Permissions:** `db.write`, `notifications.send`
### Stage Flow
```
[new-hire-info] → [provision-accounts] → [manager-signoff] → [it-setup] → [welcome]
team automated (Starlark) review + signoff team automated
auto-advances reject → stage 0 auto-advances
```
### Stages
| # | Name | Mode | Audience | Key Feature |
|---|------|------|----------|-------------|
| 0 | new-hire-info | form | team | 2 fieldsets: "Personal Info" + "Access Requirements" |
| 1 | provision-accounts | automated | system | Starlark: db.insert provisioning records, notify HR |
| 2 | manager-signoff | review | team | 1 approval required, required_role: "manager", reject → new-hire-info |
| 3 | it-setup | form | team | IT confirms hardware + access |
| 4 | welcome | automated | system | Starlark: log completion, send welcome notification |
### Starlark Hooks
**`on_provision(ctx)`** — Automated provisioning:
```python
def on_provision(ctx):
data = ctx["stage_data"]
name = data.get("full_name", "unknown")
dept = data.get("department", "general")
# Record provision in ext_data table
db.insert("provisions", {
"employee_name": name,
"department": dept,
"needs_vpn": str(data.get("needs_vpn", False)),
"needs_admin": str(data.get("needs_admin", False)),
"equipment": data.get("equipment", "standard laptop"),
"status": "provisioned"
})
# Notify the HR user who started this workflow
notifications.send(
ctx["started_by"],
"Accounts provisioned for " + name,
"Department: " + dept + ". Ready for manager review."
)
return {"advance": True, "data": {
"provision_status": "complete",
"accounts": ["email", "vpn" if data.get("needs_vpn") else None, "admin" if data.get("needs_admin") else None]
}}
```
**`on_welcome(ctx)`** — Completion logging:
```python
def on_welcome(ctx):
data = ctx["stage_data"]
db.insert("onboarding_log", {
"employee_name": data.get("full_name", ""),
"department": data.get("department", ""),
"completed_by": ctx["started_by"],
"status": "complete"
})
notifications.send(
ctx["started_by"],
"Onboarding complete: " + data.get("full_name", ""),
"All stages finished. Welcome aboard!"
)
return {"advance": True, "data": {"onboarding_complete": True}}
```
### Signoff Gate (stage 2)
```json
{
"validation": {
"required_approvals": 1,
"required_role": "manager",
"reject_action": "new-hire-info"
}
}
```
### Prerequisites
- Team must have a custom "manager" role (via Team Admin > Members > Team Roles)
- At least one team member with the "manager" role assigned
---
## Package 3: Content Approval
**Slug:** `content-approval`
**Entry mode:** `team_only`
**Tier:** browser
### Stage Flow
```
[draft] → [review] → [publish]
team team ↕ team
[revision]
team
```
### Stages
| # | Name | Mode | Audience | Key Feature |
|---|------|------|----------|-------------|
| 0 | draft | form | team | Author submits title, body, category |
| 1 | review | review | team | 2 approvals required. Reject → revision |
| 2 | revision | form | team | Author revises. Branch rule always routes back to review |
| 3 | publish | form | team | Final confirmation |
### Review Cycle
The rejection reroute creates a loop:
1. Author submits draft → enters review
2. Reviewer A approves, Reviewer B rejects → `reject_action: "revision"` routes to stage 2
3. Author revises → branch rule routes back to review (stage 1)
4. Both reviewers approve → advances to publish (stage 3)
### Signoff Gate (stage 1)
```json
{
"validation": {
"required_approvals": 2,
"reject_action": "revision"
}
}
```
---
## Package 4: Webhook Notifier
**Slug:** `webhook-notifier`
**Entry mode:** `team_only`
**Tier:** starlark
**Permissions:** `api.http`, `connections.read`, `db.write`
### Stage Flow
```
[configure] → [fire-webhook] → [result]
team automated review
(http.post)
```
### Stages
| # | Name | Mode | Audience | Key Feature |
|---|------|------|----------|-------------|
| 0 | configure | form | team | URL + event name + payload |
| 1 | fire-webhook | automated | system | Starlark: http.post to webhook URL |
| 2 | result | review | team | Shows delivery status |
### Starlark Hook
**`on_fire(ctx)`** — Outbound HTTP:
```python
def on_fire(ctx):
data = ctx["stage_data"]
url = data.get("webhook_url", "")
# Fallback to configured connection
if not url:
conn = connections.get("webhook")
if conn:
url = conn.get("url", "")
if not url:
return {"error": "No webhook URL configured"}
payload = json.encode({
"event": data.get("event_name", "workflow.notify"),
"instance_id": ctx["instance_id"],
"workflow_id": ctx["workflow_id"],
"data": data
})
resp = http.post(url, body=payload, headers={
"Content-Type": "application/json",
"X-Armature-Event": data.get("event_name", "workflow.notify")
})
# Log the delivery
db.insert("webhook_log", {
"url": url,
"status_code": str(resp["status"]),
"event": data.get("event_name", ""),
"instance_id": ctx["instance_id"]
})
return {"advance": True, "data": {
"delivery_status": "delivered" if int(resp["status"]) < 400 else "failed",
"status_code": resp["status"],
"response_body": resp["body"][:500]
}}
```
---
## Demo Surface
A browser-tier surface (`workflow-demo`) auto-installed on first run. Provides:
- Workflow cards with feature badges and stage flow diagrams
- "Try It" buttons for interactive exploration
- Starlark code viewer showing hook source
- API curl examples for each operation
Can be uninstalled by admin like any other package.
---
## Quick Start
```bash
# Build all example packages
cd packages && ./build.sh bug-report-triage employee-onboarding content-approval webhook-notifier workflow-demo
# Install via admin API
for pkg in dist/bug-report-triage.pkg dist/employee-onboarding.pkg dist/content-approval.pkg dist/webhook-notifier.pkg dist/workflow-demo.pkg; do
curl -X POST $BASE/api/v1/admin/packages/install \
-H "Authorization: Bearer $TOKEN" \
-F "file=@$pkg"
done
```
Or with bundled packages (v0.3.8+): just `docker compose up` — all examples are auto-installed.

154
docs/DEPLOYMENT.md
View File

@@ -1,154 +0,0 @@
# Deployment Guide
## Docker Single-Instance
```bash
docker pull gobha/armature:latest
docker run -p 8080:80 \
-e ARMATURE_ADMIN_USERNAME=admin \
-e ARMATURE_ADMIN_PASSWORD=changeme \
-e JWT_SECRET="$(openssl rand -hex 32)" \
-e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
-v armature-data:/data \
gobha/armature:latest
```
This runs with SQLite and PVC storage. Suitable for evaluation and small teams.
## Docker Compose with PostgreSQL
```yaml
services:
postgres:
image: postgres:16
environment:
POSTGRES_DB: armature
POSTGRES_USER: armature
POSTGRES_PASSWORD: secretpassword
volumes:
- pg_data:/var/lib/postgresql/data
armature:
image: gobha/armature:latest
ports:
- "8080:80"
environment:
DATABASE_URL: "postgres://armature:secretpassword@postgres:5432/armature?sslmode=disable"
JWT_SECRET: "change-me-in-production"
ENCRYPTION_KEY: "change-me-in-production"
ARMATURE_ADMIN_USERNAME: admin
ARMATURE_ADMIN_PASSWORD: changeme
STORAGE_BACKEND: pvc
STORAGE_PATH: /data/storage
volumes:
- armature_storage:/data/storage
depends_on:
- postgres
volumes:
pg_data:
armature_storage:
```
## Kubernetes
See the `k8s/` directory for example manifests. Key considerations:
- Set `POSTGRES_HOST`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` env vars (assembled into DSN automatically).
- Liveness probe: `/healthz/live`. Readiness probe: `/healthz/ready`.
- Mount a PVC at `/data/storage` or configure S3.
- Store `JWT_SECRET` and `ENCRYPTION_KEY` in Kubernetes Secrets.
- Registry: `registry.gobha.me:5000/xcaliber/armature`.
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8080` | Backend API port |
| `DB_DRIVER` | auto | `postgres` or `sqlite` |
| `DATABASE_URL` | | PostgreSQL DSN or SQLite path |
| `JWT_SECRET` | `dev-secret-change-me` | Token signing key -- **must change** |
| `ENCRYPTION_KEY` | | AES-256 key for credential vault |
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, `oidc` |
| `TLS_MODE` | (empty) | `native` for node-to-node mTLS. Requires `MTLS_CERT_PATH` and `MTLS_KEY_PATH`. |
| `STORAGE_BACKEND` | auto | `pvc` or `s3` |
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
| `BASE_PATH` | | URL prefix (e.g., `/armature`) |
| `LOG_FORMAT` | `text` | `text` or `json` |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
| `CORS_ALLOWED_ORIGINS` | | Comma-separated allowed origins |
| `BUNDLED_PACKAGES` | (empty) | `""` defaults, `"*"` all, or comma-separated |
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable bundled package install |
| `BUNDLED_PACKAGES_DIR` | `/app/bundled-packages` | Custom bundle directory |
| `ARMATURE_ADMIN_USERNAME` | | Bootstrap admin username |
| `ARMATURE_ADMIN_PASSWORD` | | Bootstrap admin password |
| `SEED_USERS` | | Dev seed users (ignored in production) |
### S3 Storage Variables
| Variable | Description |
|----------|-------------|
| `S3_BUCKET` | Bucket name |
| `S3_ENDPOINT` | Endpoint URL (for MinIO, Ceph, etc.) |
| `S3_ACCESS_KEY` | Access key |
| `S3_SECRET_KEY` | Secret key |
| `S3_FORCE_PATH_STYLE` | `true` for MinIO/Ceph |
### Cluster Variables (Multi-Replica)
| Variable | Default | Description |
|----------|---------|-------------|
| `CLUSTER_NODE_ID` | hostname-PID | Override for deterministic node identity |
| `CLUSTER_HEARTBEAT_INTERVAL` | `10s` | Heartbeat tick frequency |
| `CLUSTER_STALE_THRESHOLD` | `30s` | 3x heartbeat = stale node |
| `CLUSTER_ENDPOINT` | auto-detect | Advertised address for peer mesh |
## Database Configuration
**PostgreSQL** (recommended for production):
```bash
DATABASE_URL="postgres://user:pass@host:5432/armature?sslmode=require"
```
**SQLite** (dev, test, edge deployments):
```bash
DB_DRIVER=sqlite
DATABASE_URL=/data/armature.db
```
Both databases are first-class -- every query compiles and passes tests on both. The driver is auto-detected from `DATABASE_URL` if `DB_DRIVER` is not set.
## Cluster Mode
Multi-replica HA requires PostgreSQL. The kernel uses PG-backed WebSocket tickets and rate limit counters to coordinate across replicas. A cluster registry with heartbeat and stale sweep tracks active nodes.
View cluster status in Admin > Cluster or via `GET /api/v1/admin/cluster`.
## Storage Backends
**PVC**: Auto-detected if the storage path is writable. Mount a volume at `/data/storage`.
**S3-compatible**: Set `STORAGE_BACKEND=s3` and provide S3 variables. Works with AWS S3, MinIO, and Ceph.
## Backup and Restore
Available via Admin UI or API:
```
POST /api/v1/admin/backup # Create backup
GET /api/v1/admin/backups # List backups
GET /api/v1/admin/backups/:name # Download backup
POST /api/v1/admin/restore # Restore from backup
DELETE /api/v1/admin/backups/:name # Delete backup
```
## Monitoring
| Endpoint | Purpose |
|----------|---------|
| `/health` | Basic health check |
| `/healthz/live` | Kubernetes liveness probe |
| `/healthz/ready` | Kubernetes readiness probe |
| `/metrics` | Prometheus metrics |

49
docs/DESIGN-EXTENSION-LIFECYCLE.md
View File

@@ -1,49 +0,0 @@
# Extension Lifecycle: Permanent vs PoC
## Overview
Packages (extensions) fall into two categories: **permanent** and **PoC** (proof-of-concept).
This classification drives install-by-default behavior and maintenance expectations.
## Permanent Packages
Permanent packages are part of the platform kernel or essential surfaces. They are always
available and maintained by core contributors.
| Package | Reason |
|------------------|--------------------------------|
| admin | System administration surface |
| team-admin | Team management surface |
| settings | User settings surface |
| login | Authentication surface |
| welcome | Onboarding / empty-state |
## PoC Packages
Everything else in `packages/` is PoC. These demonstrate the extension model but are not
required for platform operation. Examples: tasks, schedules, dashboard, editor, git-board,
icd-test-runner, sdk-test-runner.
Retired builtins (csv-table, diff-viewer, js-sandbox, katex-renderer, mermaid-renderer,
regex-tester) are PoC packages migrated from the kernel during the v0.2.9 retirement.
## Graduation Criteria
A PoC package may be promoted to permanent when all of the following are met:
1. **Active usage** — 3+ active users or teams depend on it.
2. **Test coverage** — At least basic smoke tests exist (Starlark or integration).
3. **No kernel special-casing** — The package operates entirely through the public extension
API. No `if pkg == "X"` branches in core.
4. **Documentation** — README with install instructions, config options, and screenshots.
5. **Maintenance owner** — A named contributor commits to reviewing issues.
## Deprecation
Packages that no longer meet graduation criteria (e.g., zero usage for 6 months) may be
archived. Archived packages remain installable but receive no updates.
## Install Model
All packages use explicit installation: `POST /api/v1/packages/:id/install`. There is no
auto-install. The kernel ships clean; teams choose what they need.

57
docs/DESIGN-TRIGGER-COMPOSITION.md
View File

@@ -1,57 +0,0 @@
# Trigger Composition: Triggers, Schedules, and Workflows
## Overview
Three automation primitives exist in the platform: **triggers** (event-driven), **schedules**
(time-driven), and **workflows** (multi-step state machines). This document defines how they
compose without creating circular dependencies.
## Composition Rules
### Triggers can start workflows
A trigger's Starlark handler may call `workflow.start(workflow_id, data)` to create a new
workflow instance. This is the primary entry point for event-driven workflow initiation.
Example: a `ticket.created` trigger fires and starts an approval workflow.
### Schedules can start workflows
A schedule's Starlark handler may also call `workflow.start(...)`. This enables time-based
batch processing workflows.
Example: a daily 9am schedule starts a standup collection workflow.
### Workflows emit events that triggers listen to
Workflow lifecycle events (`workflow.started`, `workflow.advanced`, `workflow.completed`,
`workflow.signoff`, etc.) are published on the event bus. Triggers may subscribe to these
events to perform side effects (notifications, external API calls, audit logging).
### Workflows cannot start triggers or schedules
Workflows do not directly invoke triggers or create schedules. Workflow Starlark hooks
operate within the workflow's own context and do not have access to trigger/schedule
management APIs. This prevents uncontrolled fan-out.
## Circular Invocation Guard
The automated stage cycle guard (max 10 consecutive automated stages) already prevents
infinite loops within a single workflow. Cross-workflow cycles are prevented by this rule:
**A workflow event handler (trigger) may start a new workflow instance, but the new instance
is not processed synchronously.** It enters the work queue as a separate execution context.
The bus does not re-enter the originating workflow's advance call.
This means:
- `workflow.completed` trigger → `workflow.start(B)` is allowed (async).
- Workflow A automated stage → `workflow.start(B)` where B immediately completes and
triggers A again → allowed but rate-limited by the cycle guard on each individual workflow.
## Not Supported (by design)
- Triggers cannot modify a running workflow instance (read-only access via Starlark builtins).
- Schedules cannot advance or cancel workflow instances directly.
- Workflows cannot create, modify, or delete triggers or schedules.
These restrictions keep the composition model simple and auditable.

806
docs/DESIGN-WORKFLOW-REDESIGN-0.2.6.md Normal file
View File

@@ -0,0 +1,806 @@
# Workflow Redesign — v0.2.6
**Status:** Draft
**Branch:** `feat/workflow-redesign-v0.2.6`
**Date:** 2026-03-27
**Context:** Mapping viable ideas from chat-switchboard v0.39.x onto core's extension-first architecture.
---
## 1. Guiding Principles
1. **Workflows are kernel.** Definitions, instances, the stage graph, form validation, and
assignment queues are platform primitives — not extensions.
2. **Execution is delegated.** How a stage renders or collects data is an extension concern.
The kernel says *what* a stage needs; a surface package provides the *how*.
3. **No chat assumptions.** The kernel never references personas, channels, or AI providers.
A chat extension can participate in workflows, but the workflow engine does not depend on it.
4. **Public access is a platform capability.** Anonymous/public sessions are not workflow-specific —
they are a kernel-level feature that any surface can opt into. Workflows *consume* public
access; they don't own it. See §15.
5. **Edit existing migrations.** Per the design-decisions log: no migration chains until the
schema is in production. New tables get new files; modified tables are edited in place.
---
## 2. Disposition Index
Every item below is classified:
| Tag | Meaning |
|-----|---------|
| **🗑 TRASH** | Remove from core. Vestigial chat-switchboard concept that doesn't fit. |
| **✏️ MOD** | Exists in core today — modify in place. |
| **➕ ADD** | New to core. Requires new code/tables. |
| **✅ KEEP** | Already correct in core. No changes needed. |
---
## 3. Schema Changes
### 3a. `007_workflows.sql` — edit in place
#### `workflows` table
| Column | Disposition | Action |
|--------|-------------|--------|
| `entry_mode` | ✏️ MOD | Current values: `public_link`, `team_only`. Keep the column but change its meaning — it becomes a workflow-level default/flag that says "this workflow has at least one public stage." The real visibility control moves to per-stage `audience`. See note below. |
| All other columns | ✅ KEEP | Clean. No chat coupling. |
**Note on `entry_mode`:** Chat-switchboard treated this as a binary toggle — the whole
workflow was either public or not. Real workflows mix audiences: internal setup → public
form → internal review → public confirmation. The per-stage `audience` field (see below)
is the source of truth. `entry_mode` on the workflow becomes a convenience flag: if any
stage has `audience = 'public'`, the workflow is public-entry eligible. This avoids a
full schema break — existing code that checks `entry_mode` still works, it just gets set
automatically when stages are saved.
#### `workflow_stages` table
| Column | Disposition | Action |
|--------|-------------|--------|
| `persona_id` | 🗑 TRASH | Drop column. Personas are a chat-extension concept. If a stage needs an AI participant, the stage's `surface_pkg_id` extension handles that internally. The comment in the migration already acknowledges this ("personas are extensions now") but the column is still there. Remove it. |
| `stage_mode` CHECK | ✏️ MOD | Current: `form_only`, `form_chat`, `review`, `custom`. Replace with: `form`, `review`, `delegated`, `automated`. See §4 for definitions. `form_chat` is trash (chat coupling). `form_only` renames to `form`. `custom` renames to `delegated` (clearer intent). `automated` is new (no UI, Starlark-only). |
| `history_mode` | 🗑 TRASH | Drop column. This controlled how much chat history a persona saw across stages. Core has no chat history. If a delegated surface needs context management, it reads `stage_data` — that's the contract. |
| `audience` | ➕ ADD | `TEXT NOT NULL DEFAULT 'team'` with CHECK `('team', 'public', 'system')`. Controls who interacts with this stage. `team` = authenticated team members only (internal). `public` = anonymous visitors via entry token (the "public-ish" stage). `system` = no human interaction, automated only. A workflow can freely alternate: team→public→team→public. The kernel enforces this: public stages are accessible via token auth, team stages require JWT, system stages have no UI. |
| `stage_type` | ➕ ADD | `TEXT NOT NULL DEFAULT 'simple'` with CHECK `('simple', 'dynamic', 'automated')`. Drives the graph engine: `simple` = declarative rules only, `dynamic` = Starlark hook resolves next stage, `automated` = no human UI, fires hook on entry and auto-advances. |
| `starlark_hook` | ➕ ADD | `TEXT` (nullable). Package-qualified entry point for dynamic/automated stages. Format: `package_id:entry_point`. NULL for simple stages. |
| `branch_rules` | ➕ ADD | `JSONB NOT NULL DEFAULT '[]'`. Replaces the overloaded `transition_rules` for simple stage branching. Array of `{field, op, value, target_stage}`. Evaluated before `starlark_hook`, before ordinal fallback. |
| `transition_rules` | ✏️ MOD | Rename to `stage_config`. This JSONB blob was doing double duty (routing rules + on_advance hooks + auto_assign). Routing moves to `branch_rules`. What remains is stage-specific config that the kernel or surface reads: `{on_advance: {package_id, entry_point}, auto_assign: "round_robin"}`. |
| `assignment_team_id` | ✅ KEEP | Clean. Kernel concept. |
| `form_template` | ✅ KEEP | Kernel concern — structured data schema. |
| `auto_transition` | ✅ KEEP | Kernel concern — skip human interaction. |
| `surface_pkg_id` | ✅ KEEP | The delegation pointer. Required for `delegated` mode stages. |
| `sla_seconds` | ✅ KEEP | Kernel concern — time budget per stage. |
#### `workflow_versions` table
| Column | Disposition | Notes |
|--------|-------------|-------|
| All existing columns | ✅ KEEP | Immutable snapshots. No changes needed. |
### 3b. `007_workflows.sql` — add new tables (same file)
#### ➕ ADD `workflow_instances`
Replaces the channel-based instance model from chat-switchboard. This is the execution record.
```sql
CREATE TABLE IF NOT EXISTS workflow_instances (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
workflow_id UUID NOT NULL REFERENCES workflows(id) ON DELETE CASCADE,
workflow_version INTEGER NOT NULL,
current_stage INTEGER NOT NULL DEFAULT 0,
stage_data JSONB NOT NULL DEFAULT '{}',
status TEXT NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'completed', 'cancelled', 'stale', 'error')),
started_by UUID REFERENCES users(id) ON DELETE SET NULL,
entry_token TEXT, -- for public_link resumption
metadata JSONB NOT NULL DEFAULT '{}',
stage_entered_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_wf_instances_workflow
ON workflow_instances(workflow_id, status);
CREATE INDEX IF NOT EXISTS idx_wf_instances_status
ON workflow_instances(status) WHERE status = 'active';
CREATE UNIQUE INDEX IF NOT EXISTS idx_wf_instances_token
ON workflow_instances(entry_token) WHERE entry_token IS NOT NULL;
DROP TRIGGER IF EXISTS wf_instances_updated_at ON workflow_instances;
CREATE TRIGGER wf_instances_updated_at BEFORE UPDATE ON workflow_instances
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
```
**Rationale:** Chat-switchboard used `channels` as instances — that table carried message trees,
AI context windows, participant lists, and other chat concerns. Core needs a purpose-built table
that holds only workflow execution state: which version, which stage, accumulated data, lifecycle status.
#### ➕ ADD `workflow_assignments`
Dropped during the fork ("channel-dependent, rebuild as needed" — see 007 header comment).
Now rebuilt as a kernel primitive.
```sql
CREATE TABLE IF NOT EXISTS workflow_assignments (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
instance_id UUID NOT NULL REFERENCES workflow_instances(id) ON DELETE CASCADE,
stage INTEGER NOT NULL,
team_id UUID NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
assigned_to UUID REFERENCES users(id) ON DELETE SET NULL,
status TEXT NOT NULL DEFAULT 'unassigned'
CHECK (status IN ('unassigned', 'claimed', 'completed', 'cancelled')),
review_data JSONB NOT NULL DEFAULT '{}',
claimed_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_wf_assignments_instance
ON workflow_assignments(instance_id, stage);
CREATE INDEX IF NOT EXISTS idx_wf_assignments_team_status
ON workflow_assignments(team_id, status) WHERE status IN ('unassigned', 'claimed');
CREATE INDEX IF NOT EXISTS idx_wf_assignments_user
ON workflow_assignments(assigned_to) WHERE assigned_to IS NOT NULL;
```
**Claim lock:** Same optimistic pattern from chat-switchboard:
`UPDATE ... WHERE id = $1 AND status = 'unassigned'` — if `rows_affected == 0`, already claimed.
### 3c. New migration file: `012_workflow_instances.sql`
Wait — per the principle, new *tables* get new files. But we're also editing 007. Decision:
**Put the new tables in 007_workflows.sql.** The schema isn't in production. Keep all workflow
DDL in one file. When the schema *is* in production (post-MVP), new additions get numbered
migrations. This is consistent with the existing design-decisions log entry:
"No new migrations pre-MVP."
---
## 4. Stage Mode Redesign
### Current (trash/rename)
| Current Mode | Disposition | Rationale |
|-------------|-------------|-----------|
| `form_only` | ✏️ MOD → `form` | Rename. Drop the `_only` suffix — there's no `form_chat` to distinguish from anymore. |
| `form_chat` | 🗑 TRASH | Chat coupling. If you want a form + AI conversation, use `delegated` with a chat surface package. |
| `review` | ✅ KEEP | Team member reviews accumulated data. Pure kernel concept. |
| `custom` | ✏️ MOD → `delegated` | Rename for clarity. "Custom" is vague. "Delegated" makes the contract explicit: the kernel delegates stage execution to `surface_pkg_id`. |
### New modes
| Mode | Disposition | Description |
|------|-------------|-------------|
| `form` | ✏️ MOD (renamed) | Kernel renders `form_template`, validates submission, merges into `stage_data`. No extension needed. |
| `review` | ✅ KEEP | Assignment queue stage. Team member claims, reviews `stage_data`, adds `review_data`, completes. Kernel-rendered. |
| `delegated` | ✏️ MOD (renamed) | Kernel hands off to `surface_pkg_id`. The surface package receives the instance context (stage_data, form_template, metadata) and calls back to advance. This is where chat, AI, or any custom UX lives. |
| `automated` | ➕ ADD | No UI. On stage entry, kernel fires `starlark_hook`, merges returned data into `stage_data`, and auto-advances. For enrichment, API calls, validation, routing decisions. |
### Stage type vs stage mode
These are orthogonal:
- **`stage_type`** controls *how the next stage is chosen*: `simple` (declarative branch_rules), `dynamic` (Starlark hook returns target), `automated` (hook + auto-advance).
- **`stage_mode`** controls *how the stage executes*: `form`, `review`, `delegated`, `automated`.
An `automated` stage_type with `automated` stage_mode is the common case for system stages,
but you can have a `dynamic` stage_type with `form` stage_mode (the form collects data, then a
Starlark hook decides where to go next based on the submission).
---
## 5. Model Changes
### `server/models/workflow.go`
#### WorkflowStage struct — ✏️ MOD
```go
// TRASH: Remove these fields
// PersonaID *string `json:"persona_id,omitempty"`
// HistoryMode string `json:"history_mode"`
// MOD: Rename transition_rules → stage_config
// TransitionRules json.RawMessage `json:"transition_rules"`
StageConfig json.RawMessage `json:"stage_config"`
// ADD: New fields
Audience string `json:"audience"` // team | public | system
StageType string `json:"stage_type"` // simple | dynamic | automated
StarlarkHook *string `json:"starlark_hook,omitempty"` // package_id:entry_point
BranchRules json.RawMessage `json:"branch_rules"` // [{field, op, value, target_stage}]
```
#### Stage mode constants — ✏️ MOD
```go
// TRASH
// StageModeFormOnly = "form_only"
// StageModeFormChat = "form_chat"
// MOD (rename)
StageModeForm = "form" // was form_only
StageModeDelegated = "delegated" // was custom
// KEEP
StageModeReview = "review"
// ADD
StageModeAutomated = "automated"
// ADD: Stage type constants
StageTypeSimple = "simple"
StageTypeDynamic = "dynamic"
StageTypeAutomated = "automated"
// ADD: Audience constants
AudienceTeam = "team"
AudiencePublic = "public"
AudienceSystem = "system"
```
#### ➕ ADD: WorkflowInstance model
```go
type WorkflowInstance struct {
ID string `json:"id"`
WorkflowID string `json:"workflow_id"`
WorkflowVersion int `json:"workflow_version"`
CurrentStage int `json:"current_stage"`
StageData json.RawMessage `json:"stage_data"`
Status string `json:"status"` // active | completed | cancelled | stale | error
StartedBy *string `json:"started_by,omitempty"`
EntryToken *string `json:"entry_token,omitempty"`
Metadata json.RawMessage `json:"metadata"`
StageEnteredAt time.Time `json:"stage_entered_at"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
```
#### ➕ ADD: WorkflowAssignment model
```go
type WorkflowAssignment struct {
ID string `json:"id"`
InstanceID string `json:"instance_id"`
Stage int `json:"stage"`
TeamID string `json:"team_id"`
AssignedTo *string `json:"assigned_to,omitempty"`
Status string `json:"status"` // unassigned | claimed | completed | cancelled
ReviewData json.RawMessage `json:"review_data"`
ClaimedAt *time.Time `json:"claimed_at,omitempty"`
CompletedAt *time.Time `json:"completed_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
```
#### TypedFormTemplate, FormField, FormValidation — ✅ KEEP
All form types are clean kernel code. No changes.
#### ValidateFormData — ✅ KEEP
Pure data validation. No chat coupling.
---
## 6. Store Interface Changes
### `server/store/workflow_iface.go` — ✏️ MOD + ➕ ADD
Current interface is definition-only (CRUD workflows + stages + versions).
Add instance and assignment operations.
```go
type WorkflowStore interface {
// ── Definition CRUD (KEEP — no changes) ──────────
Create(ctx context.Context, w *models.Workflow) error
GetByID(ctx context.Context, id string) (*models.Workflow, error)
GetBySlug(ctx context.Context, teamID *string, slug string) (*models.Workflow, error)
Update(ctx context.Context, id string, patch models.WorkflowPatch) error
Delete(ctx context.Context, id string) error
ListForTeam(ctx context.Context, teamID string) ([]models.Workflow, error)
ListGlobal(ctx context.Context) ([]models.Workflow, error)
// ── Stage CRUD (KEEP — no changes) ───────────────
CreateStage(ctx context.Context, s *models.WorkflowStage) error
ListStages(ctx context.Context, workflowID string) ([]models.WorkflowStage, error)
UpdateStage(ctx context.Context, s *models.WorkflowStage) error
DeleteStage(ctx context.Context, id string) error
ReorderStages(ctx context.Context, workflowID string, orderedIDs []string) error
// ── Versioning (KEEP — no changes) ───────────────
Publish(ctx context.Context, v *models.WorkflowVersion) error
GetVersion(ctx context.Context, workflowID string, versionNumber int) (*models.WorkflowVersion, error)
GetLatestVersion(ctx context.Context, workflowID string) (*models.WorkflowVersion, error)
// ── Instances (ADD) ──────────────────────────────
CreateInstance(ctx context.Context, inst *models.WorkflowInstance) error
GetInstance(ctx context.Context, id string) (*models.WorkflowInstance, error)
GetInstanceByToken(ctx context.Context, token string) (*models.WorkflowInstance, error)
UpdateInstance(ctx context.Context, id string, patch models.InstancePatch) error
ListInstances(ctx context.Context, workflowID string, status string) ([]models.WorkflowInstance, error)
AdvanceStage(ctx context.Context, id string, nextStage int, mergeData json.RawMessage) error
CompleteInstance(ctx context.Context, id string) error
CancelInstance(ctx context.Context, id string) error
MarkStale(ctx context.Context, olderThan time.Duration) (int, error)
// ── Assignments (ADD) ────────────────────────────
CreateAssignment(ctx context.Context, a *models.WorkflowAssignment) error
ClaimAssignment(ctx context.Context, id string, userID string) error // optimistic lock
UnclaimAssignment(ctx context.Context, id string) error
CompleteAssignment(ctx context.Context, id string, reviewData json.RawMessage) error
CancelAssignment(ctx context.Context, id string) error
ListAssignmentsByTeam(ctx context.Context, teamID string, status string) ([]models.WorkflowAssignment, error)
ListAssignmentsByUser(ctx context.Context, userID string) ([]models.WorkflowAssignment, error)
ListAssignmentsByInstance(ctx context.Context, instanceID string) ([]models.WorkflowAssignment, error)
}
```
---
## 7. Handler Changes
### `server/handlers/workflows.go` — ✏️ MOD
**Stage CRUD updates** — reflect new field names in CreateStage/UpdateStage:
- Remove `persona_id` and `history_mode` from bind/validation.
- Add `stage_type`, `starlark_hook`, `branch_rules` to bind/validation.
- Rename `transition_rules` → `stage_config` in bind/validation.
- Update `stage_mode` CHECK to new set: `form`, `review`, `delegated`, `automated`.
- Validate: `delegated` requires `surface_pkg_id`. `dynamic`/`automated` stage_type requires `starlark_hook`.
### `server/handlers/workflow_hooks.go` — ✏️ MOD
**Rename:** `channelID` parameter → `instanceID` in `FireOnAdvanceHook` signature.
The hook context dict changes from `{channel_id, ...}` to `{instance_id, ...}`.
The `jsonToStarlark` helper referenced on line 81 lives in
`server/handlers/starlark_helpers.go`. Consider consolidating with `goToStarlark` in
`server/triggers/event.go` — two independent Go→Starlark converters is tech debt.
### `server/handlers/workflow_team.go` — ✅ KEEP
Team-scoped wrappers. Clean delegation pattern. No changes.
### `server/handlers/workflow_packages.go` — ✏️ MOD
Update `workflowPkgStage` struct:
- Remove `PersonaID`, `HistoryMode` fields.
- Add `StageType`, `StarlarkHook`, `BranchRules` fields.
- Rename `TransitionRules` → `StageConfig`.
Update `ExportWorkflowPackage` and `InstallWorkflowFromManifest` accordingly.
### ➕ ADD `server/handlers/workflow_instances.go`
New handler file for instance lifecycle:
```
POST /api/v1/workflows/:id/start → Start (create instance from latest published version)
GET /api/v1/workflows/instances/:iid → GetInstance
POST /api/v1/workflows/instances/:iid/advance → Advance (submit stage data, resolve next)
POST /api/v1/workflows/instances/:iid/cancel → Cancel
GET /api/v1/workflows/:id/instances → ListInstances (by workflow, optionally by status)
```
Public entry (for `public_link` workflows):
```
POST /api/v1/workflows/entry/:slug → StartPublic (no auth, generates entry_token)
GET /api/v1/workflows/entry/:token → ResumePublic (retrieve instance by token)
POST /api/v1/workflows/entry/:token/advance → AdvancePublic (submit data via token)
```
### ➕ ADD `server/handlers/workflow_assignments.go`
New handler file for the assignment queue:
```
GET /api/v1/teams/:teamId/assignments → ListTeamAssignments (filterable by status)
POST /api/v1/teams/:teamId/assignments/:id/claim → Claim
POST /api/v1/teams/:teamId/assignments/:id/unclaim → Unclaim
POST /api/v1/teams/:teamId/assignments/:id/complete → Complete (with review_data)
POST /api/v1/teams/:teamId/assignments/:id/cancel → Cancel
GET /api/v1/assignments/mine → ListMyAssignments (across teams)
```
---
## 8. Routing Engine Changes
### `server/workflow/routing.go` — ✏️ MOD
The existing `ResolveNextStage` evaluates `transition_rules.conditions[]`. This needs to
become a two-phase resolution that respects the new `stage_type`:
```
Phase 1: Evaluate branch_rules (declarative, for simple + dynamic types)
Phase 2: If no match AND stage_type == dynamic → fire starlark_hook
Phase 3: Fallback → currentStage + 1
```
For `automated` stage_type, the engine fires `starlark_hook` on entry (not for routing — for
data enrichment), then routes via branch_rules or ordinal fallback. The routing call happens
*after* the hook returns.
**Concrete changes:**
- `ResolveNextStage` signature: add `stageType string, starlarkHook *string, runner *sandbox.Runner` parameters.
- Extract branch_rules evaluation from the current `TransitionRulesWithConditions` (which reads from `transition_rules` JSON) into a dedicated function that reads from the new `branch_rules` field.
- `TransitionRulesWithConditions` struct → rename to `StageConfig`, remove `Conditions` field (moved to branch_rules), keep `AutoAssign` and `OnAdvance`.
### `server/workflow/` — ➕ ADD `engine.go`
New file: the stage execution engine. Orchestrates the advance lifecycle:
```
1. Load instance + version snapshot
2. Validate current stage allows advancement (status checks)
3. If current stage has form_template → validate submitted data
4. Merge submitted data into stage_data
5. Fire on_advance hook (if configured in stage_config)
6. Resolve next stage (branch_rules → starlark_hook → ordinal)
7. If next stage is past last stage → complete instance
8. If next stage is automated → fire its hook, recurse to step 6
9. If next stage has assignment_team_id → create WorkflowAssignment
10. Update instance (current_stage, stage_data, stage_entered_at)
11. Emit bus events (workflow.advanced, workflow.assigned, workflow.completed)
```
### `server/workflow/` — ➕ ADD `automated.go`
Handles `automated` stage execution: fire the Starlark hook, merge returned data, and
auto-advance. Includes a cycle guard (max 10 consecutive automated stages) to prevent
infinite loops from misconfigured workflows.
---
## 9. Event Bus Updates
### `server/events/types.go` — ✏️ MOD
Current workflow events are fine but incomplete. Add:
| Event | Direction | Disposition | Trigger |
|-------|-----------|-------------|---------|
| `workflow.assigned` | DirToClient | ✅ KEEP | Assignment created |
| `workflow.claimed` | DirToClient | ✅ KEEP | Assignment claimed |
| `workflow.advanced` | DirToClient | ✅ KEEP | Stage transition |
| `workflow.completed` | DirToClient | ✅ KEEP | Instance completed |
| `workflow.cancelled` | DirToClient | ➕ ADD | Instance cancelled |
| `workflow.started` | DirToClient | ➕ ADD | Instance created |
| `workflow.sla.warning` | DirToClient | ➕ ADD | SLA at 80% threshold |
| `workflow.sla.breached` | DirToClient | ➕ ADD | SLA exceeded |
| `workflow.error` | DirLocal | ➕ ADD | Hook execution failure |
---
## 10. Starlark Module Updates
### `server/sandbox/workflow_module.go` — ✏️ MOD
The `BuildWorkflowModule` (in `workflow_module.go`, not `modules.go`) currently exists but
targets the old channel-based model. It currently only exposes `get_definition`.
Rebuild to expose instance-oriented operations:
```python
# Available to extensions with workflow.read or workflow.write permission
workflow.get_instance(instance_id) # → instance dict
workflow.get_stage_data(instance_id) # → stage_data dict
workflow.set_stage_data(instance_id, data) # → merge into stage_data
workflow.advance(instance_id) # → trigger advance (write perm)
workflow.get_assignments(instance_id) # → assignment list
```
The old `workflow_advance()` function that chat-switchboard's personas called is gone.
Extensions call `workflow.advance()` which delegates to the same engine as the HTTP handler.
---
## 11. Salvage Map from chat-switchboard v0.39.x
What was planned there, and what happens to each idea in core:
### v0.39.0 — Stage Graph Engine + Form Builder
| Component | Disposition | Core equivalent |
|-----------|-------------|-----------------|
| `stage_type` enum | ➕ ADD | Adopted directly: `simple`, `dynamic`, `automated` |
| `BranchRule` struct | ➕ ADD | Adopted as `branch_rules` JSONB on `workflow_stages` |
| Starlark hook integration | ➕ ADD | `starlark_hook` column + engine integration |
| Graph engine (`server/engine/graph.go`) | ➕ ADD | `server/workflow/engine.go` in core |
| Automated stage runner | ➕ ADD | `server/workflow/automated.go` in core |
| Visual form builder (5 Preact components) | 🗑 TRASH *for now* | The form builder was tightly coupled to chat-switchboard's stage editor surface. Core doesn't ship a workflow editor surface (that's an extension). The form *schema* is kernel; the form *builder UI* is a surface package concern. Revisit when a workflow-admin surface is built. |
| Stage type badges in UI | 🗑 TRASH | Same — UI is extension territory. |
### v0.39.1 — Stage Reorder + Bulk Ops
| Component | Disposition | Core equivalent |
|-----------|-------------|-----------------|
| Drag-and-drop reorder | 🗑 TRASH | UI concern. The kernel `ReorderStages` endpoint already exists. |
| Bulk assignment ops | ➕ ADD (later) | Useful but not blocking. Defer to v0.2.7+. The individual claim/unclaim/cancel endpoints come first. |
### v0.39.2 — SLA Alerting
| Component | Disposition | Core equivalent |
|-----------|-------------|-----------------|
| Periodic SLA scanner | ➕ ADD | Kernel scheduled goroutine (like the existing staleness sweep). Query active instances with `sla_seconds` configured, compute breach from `stage_entered_at`. |
| `sla_notified_at` on instances | ➕ ADD | Add to `workflow_instances` schema. Prevents duplicate notifications. |
| Notifications on breach | ➕ ADD | Use existing `notifications` table + bus event (`workflow.sla.breached`). |
| Webhook on breach | ➕ ADD | Fire workflow's `webhook_url` if configured. |
| Pulsing badge animation | 🗑 TRASH | UI concern for a surface package. |
### v0.39.3 — Visitor Portal
| Component | Disposition | Core equivalent |
|-----------|-------------|-----------------|
| Public entry endpoint | ➕ ADD | `/api/v1/workflows/entry/:slug` — kernel handles anonymous session creation + token issuance. |
| Session resume via token | ➕ ADD | `entry_token` on `workflow_instances` + `GetInstanceByToken` store method. |
| Branded entry page | 🗑 TRASH | Surface package. The kernel stores `branding` JSONB — a portal surface reads it. |
| Progress indicator | 🗑 TRASH | Surface package reads stage list, computes "step N of M" locally. |
| Email notifications | 🗑 TRASH *for now* | Requires SMTP infrastructure. Defer to post-MVP or make it a connection-based extension (use ext_connections for SMTP config, fire via Starlark hook). |
### v0.39.4 — Templates + Clone
| Component | Disposition | Core equivalent |
|-----------|-------------|-----------------|
| Clone endpoint | ➕ ADD | `POST /api/v1/workflows/:id/clone` — deep copy workflow + stages. Straightforward. |
| Built-in templates table | 🗑 TRASH | Core doesn't ship opinionated templates. Templates are workflow packages (`.pkg` archives). A "template gallery" is a surface. |
| Template picker UI | 🗑 TRASH | Surface concern. |
### v0.39.5 — Analytics
| Component | Disposition | Core equivalent |
|-----------|-------------|-----------------|
| Analytics query endpoint | ➕ ADD (later) | `GET /api/v1/workflows/:id/analytics?period=7d` — kernel exposes raw metrics (throughput, cycle time, stage dwell, SLA compliance) derived from instance + assignment timestamps. Defer to v0.2.8+ or v0.3.x. |
| Analytics dashboard UI | 🗑 TRASH | Surface package. |
---
## 12. Implementation Order
All items within v0.2.6. Ordered by dependency:
### Phase A: Schema + Models
1. ✏️ Edit `007_workflows.sql`: drop `persona_id`, `history_mode`; add `audience`, `stage_type`, `starlark_hook`, `branch_rules`; rename `transition_rules` → `stage_config`; update `stage_mode` CHECK.
2. ✏️ Edit `007_workflows.sql` (same file): add `workflow_instances` and `workflow_assignments` tables.
3. ✏️ Edit `007_workflows.sql` (SQLite dialect): mirror changes.
4. ✏️ Update `server/models/workflow.go`: remove trashed fields, add new structs, update constants.
### Phase B: Store Layer
5. ✏️ Update `server/store/workflow_iface.go`: add instance + assignment methods.
6. ➕ Implement Postgres store: `server/store/postgres/workflow_instances.go`, `workflow_assignments.go`.
7. ➕ Implement SQLite store: `server/store/sqlite/workflow_instances.go`, `workflow_assignments.go`.
### Phase C: Engine
8. ✏️ Update `server/workflow/routing.go`: branch_rules evaluation, starlark_hook integration.
9. ➕ Add `server/workflow/engine.go`: advance lifecycle orchestration.
10. ➕ Add `server/workflow/automated.go`: automated stage execution + cycle guard.
11. ✏️ Update `server/handlers/workflow_hooks.go`: rename channel → instance references.
### Phase D: Handlers + Routes
12. ✏️ Update `server/handlers/workflows.go`: stage CRUD field changes.
13. ✏️ Update `server/handlers/workflow_packages.go`: package struct field changes.
14. ➕ Add `server/handlers/workflow_instances.go`: instance lifecycle endpoints.
15. ➕ Add `server/handlers/workflow_assignments.go`: assignment queue endpoints.
16. ✏️ Wire new routes in `server/main.go`.
### Phase E: Events + Starlark
17. ✏️ Update `server/events/types.go`: add new workflow events.
18. ✏️ Update `server/sandbox/modules.go`: rebuild workflow Starlark module.
### Phase F: Background Jobs
19. ➕ Add SLA scanner goroutine (check active instances, fire notifications + events on breach).
20. ➕ Add staleness sweep for workflow instances (reuse pattern from chat-switchboard).
### Phase G: Verification
21. ➕ Integration tests: instance lifecycle, stage advancement, branching, automated stages, assignments, SLA breach.
22. `go build ./...` clean.
23. Update ICD (OpenAPI spec) with new endpoints.
24. Update ROADMAP.md + CHANGELOG.md.
---
## 13. What We're NOT Doing (Explicit Deferrals)
| Item | Why | Revisit |
|------|-----|---------|
| Visual form builder UI | Surface concern, no workflow-admin surface yet | When a workflow-admin surface ships |
| Drag-and-drop stage reorder UI | Surface concern | Same |
| Email notifications | Needs SMTP infrastructure or connection-based ext | Post-MVP |
| Built-in workflow templates | Core doesn't ship opinionated content | Package gallery |
| Analytics dashboard | Nice-to-have, kernel endpoint can come first | v0.3.x |
| Bulk assignment operations | Individual ops first | v0.2.7+ |
| Workflow chaining (`on_complete`) | Already in schema, needs engine wiring | v0.2.7 |
| Visitor portal surface | Surface package, not kernel | Extension track |
---
## 14. Migration Policy Reminder
From the design-decisions log:
> **No new migrations pre-MVP.** Edit existing migration SQL files in place.
> No migration chains until schema is in production.
For this redesign:
- **`007_workflows.sql`**: Edit in place for column changes AND add new tables to the same file.
- **No `012_*.sql`**: Everything goes in 007.
- **Both dialects**: Postgres and SQLite must be kept in sync.
The only time a new numbered migration file is created is when a truly new,
unrelated subsystem is added (like 011_triggers.sql was for the trigger system).
Workflow instances and assignments are part of the workflow subsystem → they go in 007.
---
## 15. Public Surfaces — Kernel Capability, Not Workflow Feature
### The problem with workflow-owned public access
Chat-switchboard treated public access as a binary workflow toggle: `entry_mode: public_link`
made the whole workflow public. But real workflows alternate audiences — internal setup →
public-facing customer form → internal review → public confirmation page. And public access
isn't even a workflow-only need: feedback forms, status pages, and surveys all need anonymous
sessions without being workflows.
### Per-stage audience (the v0.2.6 solution)
The `audience` field on `workflow_stages` is the source of truth:
| Audience | Who interacts | Auth required | Example |
|----------|--------------|---------------|---------|
| `team` | Authenticated team members | JWT | Internal triage, setup, review |
| `public` | Anonymous visitors | Entry token | Customer intake form, confirmation page |
| `system` | Nobody (automated) | N/A | API enrichment, routing decision |
A single workflow freely alternates:
```
Stage 0: "Setup" audience=team (agent configures parameters)
Stage 1: "Customer Form" audience=public (customer fills out intake)
Stage 2: "Internal Review" audience=team (team reviews submission)
Stage 3: "Confirmation" audience=public (customer sees result)
```
The engine enforces boundaries:
- When the instance enters a `public` stage, the visitor's token grants access. Team members
can also see it (they have elevated access).
- When the instance enters a `team` stage, the visitor's view freezes. They see "being
processed" or whatever the portal surface shows. The token is still valid for resumption
when the next `public` stage arrives.
- When the instance enters a `system` stage, nobody interacts — the automated hook fires
and the engine advances.
### The `entry_mode` column
Stays on the `workflows` table but becomes a derived convenience flag. If any stage has
`audience = 'public'`, the workflow is public-entry eligible. The kernel can auto-compute
this when stages are saved, or treat it as an explicit admin toggle that gates whether
public stages are actually reachable. Either way, it's no longer the primary access control
mechanism — `audience` on each stage is.
### Platform-level public access (v0.2.7+ concern)
The broader vision: public access as a **package-level manifest capability** so any surface
(not just workflows) can serve anonymous visitors. A package declares `"public": true` in
its manifest, and the kernel mounts a public route namespace (`/p/:package_slug/*`) with
anonymous session middleware, token issuance, and rate limiting. Workflows consume this
capability — they don't own it.
For v0.2.6, the workflow-specific public entry routes are sufficient:
```
POST /api/v1/workflows/entry/:slug → StartPublic
GET /api/v1/workflows/entry/:token → ResumePublic
POST /api/v1/workflows/entry/:token/advance → AdvancePublic
```
The anonymous session management (entry_token on `workflow_instances`, IP rate limiting)
lives in the workflow handler for now. Generalization to a platform-level `public_sessions`
table happens when non-workflow surfaces need it.
### Visitor view contract
When a public stage is active, the kernel API returns only what the visitor should see:
- The current stage's `form_template` (if mode is `form`)
- The stage name and ordinal (for progress indicators)
- Accumulated `stage_data` keys marked as `visitor_visible` (future: field-level ACL)
- The `branding` JSONB from the workflow
Internal stages, team assignments, review data, and other instances' data are never
exposed through the token-authenticated endpoints. The surface package renders whatever
the kernel returns — it doesn't need to filter.
---
## 16. Branch Rules — Routing, Skipping, and Convergence
### How branch_rules work
Each stage has a `branch_rules` JSONB array. On stage completion, the engine evaluates rules
top-to-bottom against accumulated `stage_data`. First match wins. No match → ordinal + 1.
```json
[
{"field": "priority", "op": "eq", "value": "urgent", "target_stage": "Emergency Review"},
{"field": "amount", "op": "gt", "value": 10000, "target_stage": "Manager Approval"},
{"field": "type", "op": "in", "value": ["a","b"], "target_stage": "Fast Track"}
]
```
`target_stage` accepts either a **stage name** (case-insensitive) or a **numeric ordinal**.
The engine tries name resolution first, falls back to numeric. This is already implemented
in `routing.go` → `resolveTarget()`.
### Skipping stages
Branch rules can target any stage, not just the next one. If a workflow has stages
0→1→2→3→4 and stage 1's rules say `target_stage: "Stage 4"`, stages 2 and 3 are skipped
for that instance. The engine doesn't care about gaps — it sets `current_stage` to whatever
the rule resolved to.
Linear workflows (no branch_rules on any stage) advance 0→1→2→3→4 by the ordinal + 1
fallback. Adding a single branch rule to any stage introduces conditional skipping without
changing the linear stages.
### Convergence (diamond patterns)
Two different branches can target the same downstream stage:
```
Stage 0: Intake Form
branch_rules: [
{field: "region", op: "eq", value: "US", target_stage: "US Processing"},
{field: "region", op: "eq", value: "EU", target_stage: "EU Processing"}
]
Stage 1: US Processing (ordinal 1)
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
(always converge after processing)
Stage 2: EU Processing (ordinal 2)
branch_rules: [{field: "*", op: "exists", target_stage: "Final Review"}]
(always converge after processing)
Stage 3: Final Review (ordinal 3)
(both branches land here)
```
No special syntax needed. The routing engine doesn't track "which branch am I on" — it just
resolves the target and moves the instance there. The accumulated `stage_data` carries
everything downstream stages need to know about what happened upstream.
### Backward jumps
Branch rules can also target earlier stages (lower ordinals) for retry/correction loops.
The cycle guard (max 10 consecutive automated stages) in `automated.go` prevents infinite
loops for automated stages. For human-facing stages, backward jumps are intentional — the
human breaks the cycle by submitting different data.
### Operators
Full set, inherited from the existing `routing.go` implementation:
| Op | Description | Example |
|----|-------------|---------|
| `eq` | Loose equality (string comparison) | `"status" eq "approved"` |
| `neq` | Not equal | `"status" neq "rejected"` |
| `gt` | Greater than (numeric) | `"amount" gt 10000` |
| `lt` | Less than (numeric) | `"amount" lt 100` |
| `gte` | Greater or equal | `"score" gte 80` |
| `lte` | Less or equal | `"score" lte 20` |
| `in` | Value in array | `"category" in ["a","b","c"]` |
| `contains` | Substring match | `"notes" contains "urgent"` |
| `exists` | Field present in stage_data | `"approval_date" exists` |
| `not_exists` | Field absent | `"rejection_reason" not_exists` |

111
docs/DESIGN-WORKFLOWS.md
View File

@@ -1,111 +0,0 @@
# Workflow Engine — Design
The workflow engine provides multi-stage, form-driven processes with team
validation, SLA enforcement, and public entry. Workflows are kernel primitives;
workflow **surfaces** (admin UI, public landing pages) are extension packages.
---
## Core Concepts
### Workflow Definition
A workflow is a named process template owned by a team. Each workflow defines an
ordered list of **stages** that instances progress through.
| Field | Description |
|-------|-------------|
| `name` | Human-readable workflow name |
| `slug` | URL-safe identifier (unique per scope) |
| `scope` | Owning team or org |
| `stages` | Ordered list of stage definitions |
| `entry_mode` | `internal` (authenticated users) or `public_link` (landing page) |
| `sla_hours` | Optional SLA deadline for instance completion |
### Stage Types
Each stage has a `mode` that determines how participants interact:
| Mode | Behavior |
|------|----------|
| `form_only` | Structured form submission. Stage advances when form is complete. |
| `form_chat` | Form plus threaded discussion. Useful for review with comments. |
| `review` | Approval/rejection gate. Designated reviewers must sign off. |
| `custom` | Delegates entirely to a surface package. Enables extension-driven UIs. |
Stages declare an `assignee_mode` (individual, team, role-based) and optional
form schemas (JSON Schema validated at submission time).
### Workflow Instances
An instance is a running execution of a workflow definition. It tracks:
- Current stage index and overall status (`active`, `completed`, `cancelled`, `stale`)
- Per-stage completion records with timestamps and actor IDs
- Form data submitted at each stage
- Assignment and claim history
Instances use optimistic locking (`claimed_by` + `claimed_at`) to prevent
concurrent stage advancement. A claim expires after a configurable timeout.
---
## Multi-Party Validation (Signoff Table)
The `workflow_signoffs` table enables multi-party approval gates:
| Column | Purpose |
|--------|---------|
| `instance_id` | Which instance |
| `stage_index` | Which stage requires signoff |
| `user_id` | Who signed |
| `decision` | `approved` or `rejected` |
| `comment` | Optional rationale |
| `signed_at` | Timestamp |
The engine checks signoff requirements before advancing past a `review` stage.
Requirements are configurable: unanimous, majority, or N-of-M.
---
## SLA Scanner + Staleness Sweep
Two periodic background jobs maintain workflow health:
**SLA Scanner** — Runs on a configurable interval. Identifies instances that have
exceeded their workflow's `sla_hours` deadline. Fires a `workflow.sla.breached`
event (consumed by notification extensions or triggers).
**Staleness Sweep** — Identifies instances that have been idle (no stage
advancement) beyond a configurable threshold. Marks them as `stale` and fires
`workflow.instance.stale`. Stale instances can be resumed or cancelled.
Both jobs are distributed-safe: all cluster nodes run them, but operations are
idempotent (UPDATE with WHERE guards).
---
## Public Entry
Workflows with `entry_mode: public_link` are accessible at:
```
/w/:scope/:slug
```
The landing page renders the first stage's form without requiring authentication.
On submission, the kernel creates an instance and stores the form data. Subsequent
stages may require authentication depending on their assignee configuration.
Public entry enables use cases like intake forms, support requests, and
application submissions where the initiator is external.
---
## Extension Points
- **Surface packages** provide workflow admin UIs and participant views
- **Trigger extensions** can react to workflow events (`instance.created`,
`stage.advanced`, `sla.breached`, `instance.completed`)
- **Custom stage mode** delegates rendering and advancement logic entirely to
an extension, enabling arbitrary UIs within a workflow stage

466
docs/DESIGN-batch-exec.md
View File

@@ -1,466 +0,0 @@
# DESIGN — Concurrent Execution Primitive (`batch.exec`)
**Version:** v0.7.12
**Status:** Implemented
**Author:** Jeff / Claude session 2026-04-02
---
## Problem
Starlark is single-threaded by design (`go.starlark.net` enforces one
thread per execution). Extensions that need to fan out — calling multiple
external APIs, invoking several library functions, or performing independent
I/O operations — must do so sequentially. For two `http.post()` calls
taking 200ms each, the extension blocks for 400ms regardless of whether
the calls are independent.
The v0.7.10 `http.batch()` primitive solves the narrow case of parallel
HTTP dispatch. But it doesn't help when the work is wrapped in library
functions. If a `jira-client` library exposes `create_issue()` and a
`confluence-client` library exposes `create_page()`, the extension author
must either:
1. Call them sequentially (slow), or
2. Decompose the library calls back into raw `http.post()` parameters
to use `http.batch()` (defeats the purpose of having libraries).
The platform needs a general-purpose concurrent execution primitive that
works with arbitrary Starlark callables — including library exports.
---
## Non-Goals
- **Shared mutable state between branches.** Each concurrent branch is
fully isolated. No channels, no mutexes, no shared dicts. If branches
need to coordinate, they don't belong in `batch.exec`.
- **Unlimited concurrency.** A hard cap prevents extensions from spawning
unbounded goroutines. This is a fan-out primitive, not a thread pool.
- **Automatic retry or circuit breaking.** Error handling is the caller's
responsibility. The kernel reports per-branch results and errors.
- **Nested `batch.exec()`.** A callable inside `batch.exec` cannot itself
call `batch.exec`. This prevents exponential goroutine growth and keeps
the concurrency model flat.
---
## Key Insight: Frozen Libraries Are Thread-Safe
The reason this works without exotic machinery is `lib.require()`.
When a library is loaded via `lib.require()`, its exports are wrapped in
a `starlarkstruct.FromStringDict()` — which produces a **frozen** struct.
Frozen Starlark values are immutable and safe to read from any number of
goroutines concurrently. This is a property of `go.starlark.net`, not
something we enforce.
The only mutable state in a Starlark execution is:
1. **The `starlark.Thread` itself** — step counter, print buffer, cancel
channel. Each branch gets its own thread.
2. **Module instances** — `db`, `http`, `settings`, etc. contain
configuration and hold references to shared Go objects (`*sql.DB`,
`http.Client`). Each branch gets fresh module instances, but the
underlying Go resources (`*sql.DB` connection pool, etc.) are already
designed for concurrent access.
3. **Local variables** — thread-local by definition in Starlark.
So the construction is: one new `starlark.Thread` + one new module set
per branch, with frozen library structs shared read-only across all
branches. This is exactly what `triggers/schedule.go` already does for
scheduled task execution — `buildRestrictedModules` creates a fresh
module set for each cron fire. `batch.exec` generalizes that pattern.
---
## API
```python
results, errors = batch.exec([
lambda: jira.create_issue(issue_data),
lambda: confluence.create_page(page_data),
lambda: slack.post_message(channel, msg),
])
# results[0] = return value of jira.create_issue(), or None on error
# errors[0] = None on success, or error string on failure
# All three ran concurrently.
```
### Signature
```
batch.exec(callables, timeout=10) → (results: list, errors: list)
```
**Parameters:**
| Param | Type | Description |
|-------|------|-------------|
| `callables` | `list[callable]` | Starlark callables (lambdas, named functions, bound methods). Max length: 8. |
| `timeout` | `int` (optional) | Per-branch timeout in seconds. Default 10. Max 30. Inherits parent context deadline if shorter. |
**Returns:** A 2-tuple of equal-length lists.
- `results[i]` — the return value of `callables[i]`, or `None` if it
errored.
- `errors[i]` — `None` if `callables[i]` succeeded, or a string error
message if it failed (timeout, step limit, runtime error).
**Errors (whole-call):**
- `callables` is empty → error
- `callables` length > 8 → error
- Any element is not callable → error
- Permission `batch.exec` not granted → error
### Permission
New extension permission: `batch.exec`. Declared in manifest:
```json
{
"permissions": ["batch.exec"]
}
```
This is a separate permission because concurrent execution has resource
implications (goroutines, module construction overhead). Extensions that
don't need it shouldn't pay for it. The permission doesn't imply any
other permissions — the branch inherits whatever modules the calling
package already has.
---
## Execution Model
```
batch.exec([fn_a, fn_b, fn_c])
│
├─── goroutine 1: Thread₁ + Modules₁ → fn_a() → result[0]
├─── goroutine 2: Thread₂ + Modules₂ → fn_b() → result[1]
└─── goroutine 3: Thread₃ + Modules₃ → fn_c() → result[2]
│
sync.WaitGroup.Wait()
│
return (results, errors)
```
### Per-Branch Construction
For each callable in the input list, the kernel:
1. Creates a new `sandbox.Sandbox` with the same `Config` as the parent
(same `MaxSteps` limit — each branch gets its own step budget, not a
shared one).
2. Calls `runner.buildModulesWithLibCtx()` with the **same** `packageID`,
`manifest`, and `RunContext` as the parent invocation. This produces
a fresh module set — new `DBModuleConfig`, new `HTTPModuleConfig`, etc.
— pointing at the same underlying Go resources (`*sql.DB`, etc.).
3. The `libContext` is **shared** (read path only — cached frozen exports).
Library exports are immutable. The `loading` map (cycle detection) is
not relevant because libraries are already loaded before `batch.exec`
runs. If a branch triggers a new `lib.require()`, it would need its
own `libContext` — see Open Questions.
4. Creates a new `starlark.Thread` with the branch's print handler,
step limit, and context-based cancellation.
5. Calls `starlark.Call(thread, callable, nil, nil)` — the callable
is a zero-arg lambda that closes over its arguments.
### Context & Cancellation
Each branch gets a child context derived from the parent with the
per-branch timeout applied:
```go
branchCtx, cancel := context.WithTimeout(parentCtx, branchTimeout)
defer cancel()
```
If the parent context is cancelled (e.g., HTTP request timeout), all
branches are cancelled. If one branch exceeds its timeout, only that
branch is cancelled — others continue.
### Goroutine Cap
Hard limit: **8 concurrent branches.** This is enforced at the API
boundary (list length check), not via a semaphore. Rationale:
- 8 covers the real-world fan-out patterns (2-5 API calls, small batch
operations). Nobody needs 50 concurrent Starlark branches.
- Each branch allocates a `starlark.Thread` + module instances. At 8
branches, overhead is bounded at ~8KB per thread + module construction
time (~50μs per module set).
- No semaphore means no queuing surprises. You get 8, period.
---
## Implementation
### New File: `sandbox/batch_module.go`
```go
// BuildBatchModule creates the "batch" module.
// Requires the Runner reference for per-branch module construction.
func BuildBatchModule(
ctx context.Context,
runner *Runner,
packageID string,
manifest map[string]any,
rc *RunContext,
lc *libContext,
) *starlarkstruct.Module
```
The module holds a reference to the `Runner` — same pattern as
`BuildLibModule`. It needs the runner to call `buildModulesWithLibCtx`
for each branch.
### Core Implementation Sketch
```go
func batchExec(ctx context.Context, runner *Runner, packageID string,
manifest map[string]any, rc *RunContext, parentLC *libContext,
) func(*starlark.Thread, *starlark.Builtin, starlark.Tuple, []starlark.Tuple) (starlark.Value, error) {
return func(thread *starlark.Thread, b *starlark.Builtin,
args starlark.Tuple, kwargs []starlark.Tuple,
) (starlark.Value, error) {
var callableList *starlark.List
var timeout int = 10
if err := starlark.UnpackArgs(b.Name(), args, kwargs,
"callables", &callableList,
"timeout?", &timeout,
); err != nil {
return nil, err
}
n := callableList.Len()
if n == 0 {
return nil, fmt.Errorf("batch.exec: callables list is empty")
}
if n > 8 {
return nil, fmt.Errorf("batch.exec: max 8 callables, got %d", n)
}
if timeout < 1 || timeout > 30 {
timeout = 10
}
// Validate all elements are callable.
callables := make([]starlark.Callable, n)
for i := 0; i < n; i++ {
c, ok := callableList.Index(i).(starlark.Callable)
if !ok {
return nil, fmt.Errorf("batch.exec: element %d is %s, not callable",
i, callableList.Index(i).Type())
}
callables[i] = c
}
// Execute concurrently.
type branchResult struct {
index int
value starlark.Value
err error
}
results := make([]starlark.Value, n)
errors := make([]starlark.Value, n)
var wg sync.WaitGroup
for i, callable := range callables {
wg.Add(1)
go func(idx int, fn starlark.Callable) {
defer wg.Done()
// Per-branch context with timeout.
branchCtx, cancel := context.WithTimeout(ctx,
time.Duration(timeout)*time.Second)
defer cancel()
// Fresh module set for this branch.
modules, err := runner.buildModulesWithLibCtx(
branchCtx, packageID, manifest, rc, parentLC)
if err != nil {
results[idx] = starlark.None
errors[idx] = starlark.String(err.Error())
return
}
// Fresh sandbox + thread.
sb := New(DefaultConfig())
val, _, callErr := sb.Call(branchCtx, fn, nil, nil)
if callErr != nil {
results[idx] = starlark.None
errors[idx] = starlark.String(callErr.Error())
} else {
results[idx] = val
errors[idx] = starlark.None
}
}(i, callable)
}
wg.Wait()
return starlark.Tuple{
starlark.NewList(results),
starlark.NewList(errors),
}, nil
}
}
```
### Runner Wiring
In `buildModulesWithLibCtx`, add the permission case:
```go
case models.ExtPermBatchExec:
// Deferred — wired after module map is complete (needs runner ref).
hasBatchExec = true
```
After the module map is assembled:
```go
if hasBatchExec {
modules["batch"] = BuildBatchModule(ctx, r, packageID, manifest, rc, lc)
}
```
### Permission Constant
In `models/permissions.go`:
```go
ExtPermBatchExec = "batch.exec"
```
Add to `AllExtensionPermissions` slice.
---
## Callable Closure Semantics
The callables passed to `batch.exec` are typically lambdas that close
over variables from the calling scope:
```python
issue_data = {"summary": "Review Q3 report"}
page_data = {"title": "Q3 Report", "body": content}
results, errors = batch.exec([
lambda: jira.create_issue(issue_data),
lambda: confluence.create_page(page_data),
])
```
The closed-over values (`issue_data`, `page_data`, `jira`, `confluence`)
are references to Starlark values in the calling thread's scope. Two
safety properties make this work:
1. **Library exports (`jira`, `confluence`) are frozen.** They were
returned by `lib.require()` as `starlarkstruct.FromStringDict()` —
deeply immutable. Safe to read from any goroutine.
2. **Dict/list arguments may be mutable**, but Starlark's execution
model means the calling thread is **blocked** waiting for
`batch.exec` to return. No concurrent mutation is possible because
the caller can't execute while the branches are running.
This is the same safety model as Go's `sync.WaitGroup` pattern: the
goroutine that calls `wg.Wait()` cannot proceed until all goroutines
complete, so values passed to goroutines before `wg.Add` are safe to
read without locks.
---
## Open Questions
### 1. `lib.require()` Inside Branches
If a callable triggers a `lib.require()` that hasn't been cached yet,
the shared `libContext.cache` would be written from a goroutine. Options:
**A. Prohibit: branches cannot call `lib.require()`.** The branch gets
a nil `libContext`, so `lib` module is unavailable inside `batch.exec`.
Libraries must be loaded before the batch call. Simplest, safest.
**B. Per-branch `libContext` with shared read cache.** Each branch gets
its own `libContext` whose `cache` is pre-populated from the parent's
cache (snapshot). New loads go into the branch's local cache only.
Slightly wasteful if two branches load the same library (loaded twice),
but safe.
**C. Mutex-protected shared `libContext`.** Add a `sync.RWMutex` to
`libContext`. Reads use `RLock`, writes use `Lock`. Minimal overhead,
but makes `libContext` aware of concurrency — violates its current
assumptions.
**Recommendation: Option A for v0.7.11, Option B as follow-up if needed.**
In practice, extensions call `lib.require()` at module scope (top of
script), not inside request handlers. The lambdas passed to `batch.exec`
call methods on already-loaded library structs. Option A covers all
real-world patterns.
### 2. Print Output
Each branch has its own print buffer (the `output strings.Builder` in
`Sandbox.Call`). Options:
**A. Discard.** Branch print output is lost. Simple, avoids interleaving.
**B. Collect per-branch.** Return a third list: `(results, errors, outputs)`.
Useful for debugging but clutters the API.
**C. Merge into parent.** Append all branch output to the parent thread's
print buffer, prefixed with branch index. Requires passing the parent's
`outputMu` and `output` builder — invasive.
**Recommendation: Option A for v0.7.11.** `print()` in Starlark is a
debugging tool, not a production logging facility. Branch callables that
need to report status should return structured data. If debugging demand
emerges, Option B is a backward-compatible addition.
### 3. Step Limit Scope
Each branch gets its own `MaxSteps` budget (default 1M). Should the
total across all branches be capped?
**No.** The per-branch cap is sufficient. 8 branches × 1M steps = 8M
total, which completes in under a second on any modern hardware. The
wall-clock timeout (per-branch, max 30s) is the real resource guard.
Adding a cross-branch step budget creates coupling between independent
execution paths — branch A's step count shouldn't affect branch B's
ability to complete.
---
## Testing
| Test | Description |
|------|-------------|
| Parallel ordering | 3 callables with different sleep durations. Results in input order, not completion order. |
| Partial failure | 3 callables, middle one raises error. results = [val, None, val], errors = [None, "err msg", None]. |
| Timeout per-branch | One callable sleeps beyond timeout. Others succeed. Timed-out branch returns error. |
| Parent cancellation | Cancel parent context mid-execution. All branches cancelled. |
| Cap enforcement | List of 9 callables → immediate error, nothing executed. |
| Empty list | `batch.exec([])` → error. |
| Non-callable element | `batch.exec([1, 2])` → error, nothing executed. |
| Permission gating | Package without `batch.exec` permission → module not available. |
| Frozen library sharing | Two branches call same frozen library function concurrently. No race. |
| Nested batch.exec | Callable inside batch.exec attempts batch.exec → error (module not injected in branch). |
| db module isolation | Two branches insert into same table concurrently. Both succeed, no corruption. |
| http module isolation | Two branches make HTTP calls with different headers. No cross-contamination. |
---
## Migration
No schema changes. No new tables. No new migrations.
New permission constant `batch.exec` added to `models/permissions.go`.
Extensions must declare the permission in their manifest to access the
`batch` module.

1074
docs/DESIGN-chat-v012x.md
View File

File diff suppressed because it is too large Load Diff

290
docs/DESIGN-cluster-registry.md
View File

@@ -1,290 +0,0 @@
# Design: PG-Backed Cluster Registry & Self-Assembling Mesh
**Status:** Proposed — Pre-MVP / MVP wrap-up
**Author:** Jeff
**Date:** 2026-03-30
**Scope:** Kernel primitive — zero domain awareness
---
## Problem
Armature-core is tightly coupled to PostgreSQL. Scaling horizontally requires instance coordination: peer discovery, health monitoring, ephemeral event routing, and (optionally) leader election. Traditional HA solutions (Raft, etcd, Consul) introduce a second consensus layer on top of PG — doubling operational complexity for a system that already has serializable transactions and LISTEN/NOTIFY.
## Principle
PG is already the consensus system. Use it as the sole coordinator for service discovery, health, pub/sub, and cluster state. Zero new infrastructure dependencies.
---
## Design
### 1. Node Registry (Unlogged Table)
```sql
CREATE UNLOGGED TABLE IF NOT EXISTS node_registry (
node_id TEXT PRIMARY KEY,
endpoint TEXT NOT NULL,
seq SERIAL,
registered_at TIMESTAMPTZ DEFAULT now(),
heartbeat TIMESTAMPTZ DEFAULT now(),
stats JSONB DEFAULT '{}'
);
```
**Why UNLOGGED:** No WAL overhead, visible to all connections, survives session disconnect. Contents do NOT survive a PG crash — but a PG crash means all nodes are dead anyway. Clean slate on recovery is the correct behavior; all nodes re-register on restart.
**Why not TEMPORARY:** PG temporary tables are session-scoped and invisible to other connections. Useless for cross-instance coordination.
### 2. Node Lifecycle
#### Startup
```
boot → connect PG → register self → read peer list →
subscribe LISTEN/NOTIFY channels → open WS to clients → ready
```
```sql
-- Self-registration (atomic, idempotent)
INSERT INTO node_registry (node_id, endpoint)
VALUES ($1, $2)
ON CONFLICT (node_id) DO UPDATE
SET endpoint = EXCLUDED.endpoint,
registered_at = now(),
heartbeat = now(),
stats = '{}';
```
`node_id` is generated at startup: `hostname + PID` or a UUID. Deterministic IDs enable restart-in-place without orphaned rows.
#### Heartbeat Tick (every N seconds, e.g., 10s)
Each node performs two operations per tick:
```go
// 1. Update own heartbeat + stats
stats := collectStats() // see §5
db.Exec(`UPDATE node_registry SET heartbeat = now(), stats = $2 WHERE node_id = $1`,
nodeID, stats)
// 2. Sweep stale entries (idempotent — safe for concurrent execution)
db.Exec(`DELETE FROM node_registry WHERE heartbeat < now() - interval '30 seconds'`)
```
The sweep is the distributed health check. Every node runs it. Multiple nodes deleting the same stale row is a no-op. No ring topology required — the sweep is the safety net that guarantees eventual cleanup regardless of which nodes are alive.
#### Self-Eviction
If a node's own heartbeat UPDATE returns 0 rows affected, it has been swept by a peer:
```go
result := db.Exec(`UPDATE node_registry SET heartbeat = now() WHERE node_id = $1`, nodeID)
if result.RowsAffected() == 0 {
log.Error("evicted from registry, shutting down")
os.Exit(1)
}
```
Kubernetes restarts the process. On boot it re-registers as a fresh node. The restart loop IS the recovery mechanism.
### 3. Leaderless by Default
No `role` column. No primary/replica distinction. All nodes are equal peers. PG is the sole coordinator.
**If a leader is ever needed** (future use case), it's a small lift:
```sql
ALTER TABLE node_registry ADD COLUMN role TEXT DEFAULT 'peer';
-- Atomic leader claim: first writer wins
UPDATE node_registry SET role = 'primary'
WHERE node_id = $1
AND NOT EXISTS (SELECT 1 FROM node_registry WHERE role = 'primary');
```
No quorum, no voting, no term numbers. PG serialization handles it. But the current design intentionally avoids this — consistent with the existing pattern where scheduled tasks use first-to-mark-wins execution, not leader assignment.
### 4. Realtime Event Routing (Two Tiers)
#### Tier 1: Durable Events (messages, state changes)
These are PG writes. Fan-out via `LISTEN/NOTIFY`:
```
User A sends message → Instance 1 writes to PG →
pg_notify('channel:xyz', '{"type":"message","id":"..."}') →
All instances receive → push to local WS clients subscribed to channel:xyz
```
This maps directly to the v0.5.0 `realtime.publish()` Starlark primitive. Kernel implementation is `pg_notify(channel, payload)`.
#### Tier 2: Ephemeral Events (typing indicators, presence, cursor position)
These NEVER hit durable storage.
**Phase 1 (MVP):** Use PG LISTEN/NOTIFY for ephemeral events too.
```
User A types → Instance 1 WS handler →
pg_notify('ephemeral:xyz', '{"type":"typing","user":"A"}') →
Instance 2 receives → pushes to User B's WS
Instance 3 receives → pushes to User C's WS
```
PG NOTIFY payload limit is 8KB. A typing indicator is ~60 bytes. At homelab scale (3 nodes, dozens of users) this is a non-issue. One codepath, one transport, zero new infrastructure.
**Phase 2 (Post-MVP, if needed):** Direct instance-to-instance mesh for ephemeral events.
- Peer endpoints are already in the registry
- Each node dials peers discovered from the registry on startup and on peer list changes
- Ephemeral events route over the mesh; durable events stay on PG NOTIFY
- The `realtime.publish()` Starlark API does not change — kernel swaps the underlying transport
```
┌─────────────────────────────────┐
│ PG (source of truth) │
│ ┌───────────┐ ┌──────────────┐ │
│ │ node_ │ │ LISTEN/ │ │
│ │ registry │ │ NOTIFY │ │
│ └───────────┘ └──────────────┘ │
└──────┬──────────────┬────────────┘
│ durable │
┌────────────┼──────────────┼────────────┐
│ │ │ │
┌────┴────┐ ┌───┴─────┐ ┌────┴─────┐
│ Node 1 │◄─┤ Node 2 ├─►│ Node 3 │
│ User A │ │ User B │ │ User C │
└─────────┘ └─────────┘ └──────────┘
◄──── ephemeral mesh ────►
(Phase 2, if needed)
```
**Channel subscription model (Phase 1):** Broadcast and discard. Every node receives every event, checks local WebSocket subscriptions, drops if irrelevant. At 3 nodes this is the correct answer — trivial, no subscription state to sync, fully leaderless.
### 5. Stats Collection & Admin Dashboard
Each heartbeat tick publishes runtime stats in the `stats` JSONB column:
```go
var m runtime.MemStats
runtime.ReadMemStats(&m)
stats := map[string]any{
"goroutines": runtime.NumGoroutine(),
"heap_alloc": m.HeapAlloc,
"heap_sys": m.HeapSys,
"gc_cycles": m.NumGC,
"gc_pause_ns": m.PauseNs[(m.NumGC+255)%256],
"uptime_sec": time.Since(startTime).Seconds(),
"ws_clients": wsHub.Count(),
"extensions": sandbox.LoadedCount(),
}
```
Admin dashboard API endpoint:
```sql
SELECT node_id, endpoint, registered_at, heartbeat, stats
FROM node_registry
ORDER BY seq;
```
**ICD envelope:** `{"data": [...]}` — consistent with list endpoint convention.
The admin dashboard extension renders one card per node. One node = one card. Three nodes = three cards. The extension has zero awareness of clustering — it reads a list and renders it. This is the extensibility proof working as designed.
Since `stats` is JSONB, future additions (mesh peer latency, extension execution counts, queue depths) require no schema migration. The dashboard extension renders whatever keys are present.
### 6. Self-Assembling Mesh Properties
The mesh builds itself from the registry with zero configuration:
| Event | Behavior |
|---|---|
| Node starts | Inserts into registry → peers discover it on next heartbeat tick |
| Node dies | Heartbeat goes stale → swept by peers → mesh shrinks |
| Node restarts | Re-registers → peers discover it → mesh restores |
| Scale up (K8s) | New replicas register → mesh grows automatically |
| Scale down (K8s) | Terminated nodes go stale → swept → mesh shrinks |
| PG crash | All nodes lose connection → all exit → K8s restarts all → clean re-registration |
No config files listing peers. No seed nodes. No service discovery sidecar. No bootstrap ceremony. The registry table IS the service mesh control plane.
---
## What This Enables (Future)
These are NOT in scope for this work. Listed to validate the design supports them without rearchitecting.
- **Work distribution:** Primary (if elected) assigns extension execution shards via registry metadata. Replicas poll for assignments. No message queue.
- **Realtime routing (v0.5.0):** When a node owns a pub/sub channel, it registers ownership in PG. Other nodes route publishes to the owner. Node dies → sweep clears routes → subscribers reconnect.
- **Rolling deploys:** New version registers, old version's heartbeat goes stale, clean handoff with zero coordination protocol.
- **Session affinity:** Registry tracks which node holds a user's WS connection. Targeted event delivery instead of broadcast (optimization for scale).
---
## Implementation Notes
### Migration
Pre-MVP: fold `CREATE UNLOGGED TABLE` into existing schema initialization. No new migration file needed. DB rebuild required (acceptable — no install base).
### Configuration
| Setting | Default | Notes |
|---|---|---|
| `CLUSTER_NODE_ID` | `hostname-PID` | Override for deterministic identity across restarts |
| `CLUSTER_HEARTBEAT_INTERVAL` | `10s` | Tick frequency |
| `CLUSTER_STALE_THRESHOLD` | `30s` | 3x heartbeat = stale |
| `CLUSTER_ENDPOINT` | auto-detect | Advertised address for peer mesh (Phase 2) |
### Single-Node Behavior
When only one node is registered, the system behaves identically to pre-cluster armature. The registry has one row. LISTEN/NOTIFY delivers events back to the same instance. No special-casing required.
### Health Endpoint Integration
`GET /health` includes cluster status:
```json
{
"status": "healthy",
"node_id": "node-abc-1234",
"cluster": {
"size": 3,
"peers": ["node-def-5678", "node-ghi-9012"],
"heartbeat_age_ms": 2340
}
}
```
Satisfies the v0.6.0 MVP health monitoring requirement.
---
## Rejected Alternatives
| Alternative | Why rejected |
|---|---|
| etcd / Consul | Second consensus layer on top of PG. Doubles operational complexity for a system already tightly coupled to PG. Violates KISS. |
| Redis pub/sub for ephemeral events | Second stateful service. Two failure domains, two connection pools, two things to monitor. PG NOTIFY handles the same job at homelab scale. |
| Raft implementation in Go | Massive complexity for a problem PG serializable transactions already solve. |
| Ring topology monitoring | Each node watches the one above it. Adjacent failures create blind spots. Sweep-all is simpler and has no edge cases. |
| Leader election at startup | Unnecessary. Leaderless design with first-to-mark-wins task execution already works. Leader election is a future option, not a requirement. |
---
## Checklist
- [ ] `node_registry` unlogged table in schema init
- [ ] Node registration on startup
- [ ] Heartbeat tick with stats collection and stale sweep
- [ ] Self-eviction on 0-rows-affected heartbeat
- [ ] PG LISTEN/NOTIFY integration for realtime events
- [ ] Admin API endpoint: `GET /api/admin/cluster`
- [ ] Admin dashboard extension: cluster node cards
- [ ] Health endpoint cluster status
- [ ] Configuration via environment variables
- [ ] Single-node regression test (no behavioral change)
- [ ] Multi-node integration test (Docker Compose, 3 instances)

687
docs/DESIGN-extension-composability.md
View File

@@ -1,687 +0,0 @@
# DESIGN — Extension Composability
**Version:** v0.8.5
**Status:** Implemented
**Author:** Jeff / Claude session 2026-04-02
---
## Problem
Extensions cannot meaningfully compose with each other. A Notes surface
can't accept toolbar buttons from an STT extension. An LLM bridge can't
invoke an image generator's functions. A chat surface can't display action
buttons contributed by an image-editing extension.
The runtime primitives for contribution exist — `sw.slots`, `sw.actions`,
and `sw.renderers` are live in the SDK. But there is no manifest layer
to declare the relationships, no backend mechanism for non-library packages
to call each other's exported functions, and no conventions for slot
naming or context contracts.
This blocks the entire "extensions extending extensions" pattern that
makes the platform an ecosystem rather than a collection of packages.
---
## Non-Goals
- Arbitrary inter-extension communication (message passing, shared memory).
Extensions compose through declared slots, exported functions, and events.
- Runtime dependency injection. Dependencies are declared in manifests and
resolved at load time.
- Extension sandboxing on the frontend. Browser-tier JS runs in the same
page context. Isolation is by convention and code review, not enforcement.
---
## What Already Exists
Three SDK registries are live and functional:
**`sw.slots`** — Named UI injection points. `register(name, {id, component, priority})`
adds a Preact component to a named slot. `get(name)` returns sorted entries.
Emits `slots.changed` events. Dedup by id. Priority ordering.
**`sw.actions`** — Named callable actions. `register(id, {handler, label, icon})`
exposes a function by name. `run(id, ...args)` invokes it. Cross-extension
function calls on the frontend.
**`sw.renderers`** — Block and post renderers for markdown content.
`register(name, {type, pattern, render})`. Already used by mermaid, KaTeX,
CSV, and diff extensions.
**`lib.require()`** — Backend cross-package function calls. Starlark
packages call exported functions from library packages with the library's
own permission context.
The gap: `sw.slots` has no manifest awareness — the admin can't see which
extensions contribute where, install/uninstall can't warn about orphaned
contributions. `lib.require()` is restricted to `type: "library"` packages,
so a full package like an image generator can't export callable functions.
There are no conventions for slot naming or context contracts.
---
## Architecture
Three additions: manifest declarations, backend relaxation, and SDK helpers.
### 1. Manifest Declarations
#### Host Surfaces: `slots` Field
Surfaces declare named injection points with context descriptions:
```json
{
"id": "notes",
"slots": {
"toolbar-actions": {
"description": "Toolbar action buttons",
"context": {
"noteId": "string — current note ID",
"getContent": "function — returns note body text",
"setContent": "function — replaces note body text"
}
},
"note-footer": {
"description": "Content rendered below note body",
"context": {
"noteId": "string",
"content": "string — rendered HTML content"
}
}
}
}
```
```json
{
"id": "chat",
"slots": {
"message-actions": {
"description": "Action buttons on individual messages",
"context": {
"messageId": "string",
"content": "string — message text",
"attachments": "array — [{url, type, name}]"
}
},
"image-actions": {
"description": "Action buttons overlaid on rendered images",
"context": {
"imageUrl": "string",
"messageId": "string",
"metadata": "object — generation params if available"
}
},
"composer-tools": {
"description": "Tool buttons in the message composer area",
"context": {
"conversationId": "string",
"insertText": "function — appends to composer"
}
}
}
}
```
The `context` field is documentation, not enforcement. It tells extension
authors what data the slot provides. The kernel parses and stores slot
declarations but does not validate context shapes at runtime.
#### Contributing Extensions: `contributes` Field
Extensions declare which slots they inject into:
```json
{
"id": "note-dictate",
"title": "Note Dictation",
"type": "extension",
"tier": "browser",
"contributes": {
"notes:toolbar-actions": {
"label": "Dictate",
"icon": "🎤",
"description": "Voice-to-text note dictation"
}
},
"permissions": ["api.http"]
}
```
```json
{
"id": "image-gen",
"title": "Image Generator",
"type": "full",
"tier": "starlark",
"contributes": {
"chat:composer-tools": {
"label": "Generate Image",
"icon": "🎨",
"description": "AI image generation from text prompt"
}
},
"exports": ["generate", "list_models"],
"permissions": ["api.http", "connections.read", "files.write", "db.write"]
}
```
```json
{
"id": "image-edit",
"title": "Image Editor",
"type": "extension",
"tier": "starlark",
"contributes": {
"chat:image-actions": {
"label": "Edit Image",
"icon": "✏️",
"description": "Inpaint, outpaint, upscale, style transfer"
}
},
"exports": ["inpaint", "outpaint", "upscale", "restyle"],
"permissions": ["api.http", "connections.read", "files.write"]
}
```
**Slot naming convention:** `{host-package-id}:{slot-name}`. The colon
separates namespace from slot. Extensions contributing to `notes:toolbar-actions`
are declaring a relationship with the `notes` package.
#### What the Kernel Does with Declarations
**At install time:**
- Parse `slots` field → store in manifest (no separate table needed).
- Parse `contributes` field → validate that each target slot name follows
the `{pkg}:{slot}` convention. Do NOT validate that the host package
exists — contributing extensions can be installed before or after their
host. Soft coupling.
- Parse `exports` field → already stored in manifest. No changes needed.
**At admin display time:**
- `GET /api/v1/admin/packages/:id` includes `slots` and `contributes`
from manifest. Admin UI shows "This package provides X slots" and
"This package contributes to Y slots."
- New endpoint: `GET /api/v1/admin/slots` → returns a map of all declared
slots across installed packages with their contributors. Built by
scanning all package manifests — no new table.
**At uninstall time:**
- If uninstalling a package that declares `slots`, check if any enabled
packages declare `contributes` targeting those slots. Warn (not block):
"Uninstalling 'notes' will orphan contributions from: note-dictate,
note-ai." The admin decides.
- If uninstalling a contributing package, no warning needed — the slot
just has fewer entries.
---
### 2. Backend: `lib.require()` Relaxation
Currently `lib.require()` enforces `libPkg.Type == "library"`. This
prevents full packages (which have surfaces, settings, UI) from also
exporting callable functions.
**Change:** Replace the type check with an exports check.
In `sandbox/lib_module.go`, the current guard:
```go
if libPkg.Type != "library" {
return nil, fmt.Errorf("lib.require: package %q is type %q, not library", libraryID, libPkg.Type)
}
```
Becomes:
```go
exports := extractExports(libPkg.Manifest)
if len(exports) == 0 {
return nil, fmt.Errorf("lib.require: package %q declares no exports", libraryID)
}
```
Any package that declares `"exports": [...]` is callable via `lib.require()`,
regardless of type. The dependency check (`ListByConsumer`) remains
enforced — the consumer must still declare `"depends": ["image-gen"]`
in its manifest.
The `depends` field already accepts any package ID, not just libraries.
The `DependencyStore` has no type check. This change is one line in
`lib_module.go`.
**Security implication:** A full package's exported functions run with
that package's own permissions, same as libraries today. An image-gen
package with `api.http` + `files.write` permissions exposes `generate()`
— when llm-bridge calls it, it runs with image-gen's permissions, not
llm-bridge's. This is correct and already how `lib.require()` works.
---
### 3. SDK Additions
#### `sw.slots.renderAll(name, context)` Helper
Currently host surfaces must manually iterate slot entries:
```javascript
const entries = sw.slots.get('notes:toolbar-actions');
return entries.map(e => html`<${e.component} ...${ctx} />`);
```
Add a convenience helper:
```javascript
/**
* Render all components registered in a slot.
* @param {string} name — slot name
* @param {object} context — props passed to each component
* @returns {Array<VNode>} — array of rendered vnodes
*/
renderAll(name, context = {}) {
return this.get(name).map(e => {
try {
return e.component(context);
} catch (err) {
console.error(`[sw.slots] Error rendering "${e.id}" in slot "${name}":`, err);
return null;
}
}).filter(Boolean);
}
```
Host surface usage becomes:
```javascript
// In Notes toolbar:
html`<div class="notes-toolbar">
<button onclick=${save}>Save</button>
${sw.slots.renderAll('notes:toolbar-actions', {
noteId: currentNote.id,
getContent: () => editor.getValue(),
setContent: (text) => editor.setValue(text)
})}
</div>`
```
#### `sw.slots.declare(name, description)` (Optional)
Runtime declaration for slots that don't appear in the manifest (e.g.,
dynamically created slots). Mostly for discoverability — the debug panel
can list all active slots whether manifest-declared or runtime-declared.
```javascript
// In chat surface, when rendering an image:
sw.slots.declare('chat:image-actions', 'Action buttons on images');
```
Not enforced — `sw.slots.register()` works on any name regardless.
This is a documentation/debugging aid only.
---
## Composition Patterns
### Pattern A: LLM Tool Registration
LLM bridge exposes a tool registry. Tool extensions register at load time
via `lib.require()` on the backend and `sw.actions` on the frontend.
**llm-bridge (library) — script.star:**
```python
_tools = {}
def register_tool(name, description, parameters, handler):
"""Register a callable tool for LLM function calling."""
_tools[name] = {
"name": name,
"description": description,
"parameters": parameters,
"handler": handler,
}
def complete(messages, tools=None):
"""Send messages to LLM with registered tools available."""
tool_schemas = []
for t in _tools.values():
tool_schemas.append({
"name": t["name"],
"description": t["description"],
"parameters": t["parameters"],
})
# ... call LLM API with tool_schemas, handle tool_use responses
# by dispatching to _tools[name]["handler"]
```
**image-gen (full package) — script.star:**
```python
llm = lib.require("llm-bridge")
def generate(prompt, model="dall-e-3", size="1024x1024", seed=None):
conn = connections.get("openai")
# ... call API, store result in files module ...
return {"image_url": url, "prompt": prompt, "seed": actual_seed}
# Register as LLM tool at load time
llm.register_tool(
name="generate_image",
description="Generate an image from a text description",
parameters={"prompt": "string", "size": "string"},
handler=generate,
)
```
### Pattern B: UI Contribution (Toolbar Buttons)
**note-dictate (extension) — js/index.js:**
```javascript
sw.slots.register('notes:toolbar-actions', {
id: 'note-dictate',
priority: 200,
component: ({ noteId, setContent, getContent }) => {
const [recording, setRecording] = preact.useState(false);
const toggle = async () => {
if (recording) {
// Stop recording, send audio to STT api_route
const text = await sw.api.ext('note-dictate').post('/transcribe', {
audio: audioBlob
});
setContent(getContent() + '\n' + text.transcription);
setRecording(false);
} else {
// Start recording
setRecording(true);
}
};
return html`<button class="sw-btn sw-btn--ghost"
onclick=${toggle}
title="Dictate">
${recording ? '⏹' : '🎤'}
</button>`;
}
});
```
**notes (surface) — toolbar rendering:**
```javascript
html`<div class="notes-toolbar__actions">
<button onclick=${() => saveNote()}>💾</button>
<button onclick=${() => togglePin()}>📌</button>
${sw.slots.renderAll('notes:toolbar-actions', {
noteId: note.id,
getContent: () => editor.getValue(),
setContent: (text) => editor.setValue(text),
})}
</div>`
```
Notes doesn't know dictation exists. Dictation doesn't know Notes'
internals — just the slot contract (noteId, getContent, setContent).
### Pattern C: Image Actions (Generate + Edit + Upscale)
Three independent extensions contribute to the same image display slot:
**chat surface — image rendering:**
```javascript
function ChatImage({ src, messageId, metadata }) {
return html`<div class="chat-image">
<img src=${src} />
<div class="chat-image__actions">
${sw.slots.renderAll('chat:image-actions', {
imageUrl: src,
messageId,
metadata,
})}
</div>
</div>`;
}
```
**image-gen — contributes regen button:**
```javascript
sw.slots.register('chat:image-actions', {
id: 'image-gen:regenerate',
priority: 100,
component: ({ imageUrl, metadata }) => {
const regen = async () => {
// Open prompt editor with original params
const params = await sw.prompt('Edit prompt', {
default: metadata?.prompt || '',
multiline: true,
});
if (!params) return;
const result = await sw.api.ext('image-gen').post('/generate', {
prompt: params,
seed: metadata?.seed,
negative_prompt: metadata?.negative_prompt,
});
// result triggers a new chat message with the generated image
};
return html`<button class="sw-btn sw-btn--sm" onclick=${regen} title="Regenerate">
🔄
</button>`;
}
});
```
**image-edit — contributes edit drawer:**
```javascript
sw.slots.register('chat:image-actions', {
id: 'image-edit:edit',
priority: 200,
component: ({ imageUrl, metadata }) => {
const openEditor = () => {
// Open a drawer with editing options
sw.emit('drawer.open', {
title: 'Edit Image',
component: ImageEditPanel,
props: { imageUrl, metadata },
});
};
return html`<button class="sw-btn sw-btn--sm" onclick=${openEditor} title="Edit">
✏️
</button>`;
}
});
```
**image-upscale — contributes upscale button:**
```javascript
sw.slots.register('chat:image-actions', {
id: 'image-upscale:upscale',
priority: 300,
component: ({ imageUrl }) => {
const upscale = async () => {
const result = await sw.api.ext('image-upscale').post('/upscale', {
image_url: imageUrl,
scale: 2,
});
// Display or replace with upscaled image
};
return html`<button class="sw-btn sw-btn--sm" onclick=${upscale} title="Upscale 2×">
🔍
</button>`;
}
});
```
Result: an image in chat gets three action buttons from three separate
extensions. Install one, get one button. Install all three, get all three.
Uninstall one, the others keep working. The chat surface has no knowledge
of any of them.
### Pattern D: LLM Note Restructuring
Combines backend composability (lib.require) with frontend contribution
(slots):
**note-ai (extension) — manifest.json:**
```json
{
"id": "note-ai",
"type": "extension",
"tier": "starlark",
"depends": ["llm-bridge"],
"contributes": {
"notes:toolbar-actions": {
"label": "AI Restructure",
"icon": "✨"
}
},
"permissions": ["api.http"],
"settings": {
"restructure_prompt": {
"type": "string",
"label": "Restructure Prompt",
"description": "System prompt for AI note restructuring",
"default": "Restructure the following note into clear sections with headers. Preserve all information. Use markdown formatting.",
"user_overridable": true
}
}
}
```
**note-ai — script.star (api_route handler):**
```python
llm = lib.require("llm-bridge")
def on_request(req):
if req["method"] == "POST" and req["path"] == "/restructure":
content = req["body"]["content"]
prompt = settings.get("restructure_prompt")
result = llm.complete([
{"role": "system", "content": prompt},
{"role": "user", "content": content},
])
return {"status": 200, "body": {"restructured": result["text"]}}
```
**note-ai — js/index.js:**
```javascript
sw.slots.register('notes:toolbar-actions', {
id: 'note-ai:restructure',
priority: 500,
component: ({ noteId, getContent, setContent }) => {
const [loading, setLoading] = preact.useState(false);
const restructure = async () => {
setLoading(true);
try {
const result = await sw.api.ext('note-ai').post('/restructure', {
content: getContent()
});
setContent(result.restructured);
sw.toast('Note restructured', 'success');
} catch (e) {
sw.toast('Restructure failed: ' + e.message, 'error');
} finally {
setLoading(false);
}
};
return html`<button class="sw-btn sw-btn--ghost"
onclick=${restructure}
disabled=${loading}
title="AI Restructure">
${loading ? '⏳' : '✨'}
</button>`;
}
});
```
The user configures their restructuring prompt in Settings. The button
appears in Notes toolbar. Clicking it sends the note content to the
api_route, which calls llm-bridge, which calls the configured LLM provider.
Four packages involved (notes, note-ai, llm-bridge, the LLM connection),
zero hardcoded dependencies between surfaces.
---
## Kernel Changes
### Modified Files
| File | Change |
|------|--------|
| `sandbox/lib_module.go` | Replace type check with exports check (~1 line) |
| `handlers/extensions.go` | Parse `contributes` and `slots` manifest fields (validation) |
| `handlers/packages.go` | Uninstall warning for orphaned contributions |
| `src/js/sw/sdk/slots.js` | Add `renderAll(name, context)` and `declare(name, desc)` |
| `docs/PACKAGE-FORMAT.md` | Document `slots`, `contributes`, `exports` fields |
| `docs/EXTENSION-GUIDE.md` | Composability patterns section |
### New Files
| File | Purpose |
|------|---------|
| `handlers/admin_slots.go` | `GET /admin/slots` aggregation endpoint |
### No New:
- Database tables
- Migrations
- Starlark modules
- Permission constants
- Config env vars
This is deliberately small. The runtime infrastructure exists. The
design adds manifest-level visibility, one backend guard relaxation,
and one SDK helper. The composability comes from conventions and
documentation, not kernel complexity.
---
## Slot Catalog (Initial)
These are the recommended slots for first-party packages. Extension
authors may define additional slots following the naming convention.
| Slot | Host | Context | Use Case |
|------|------|---------|----------|
| `notes:toolbar-actions` | notes | noteId, getContent, setContent | Dictation, AI tools, formatting |
| `notes:note-footer` | notes | noteId, content | Related items, AI summary, metadata |
| `chat:composer-tools` | chat | conversationId, insertText | Image gen, file attach, slash commands |
| `chat:message-actions` | chat | messageId, content, attachments | Reactions, translate, bookmark |
| `chat:image-actions` | chat | imageUrl, messageId, metadata | Regen, edit, upscale, style transfer |
| `schedules:event-actions` | schedules | eventId, event | Add to calendar, share, convert to task |
| `admin:package-actions` | admin | packageId, package | Custom admin tools per package |
Each host surface adds `sw.slots.renderAll()` calls at the appropriate
locations. Extensions contribute via `sw.slots.register()` in their JS.
The manifest `contributes` field makes the relationship visible to admins.
---
## Future Considerations
- **Slot schema validation.** Currently context contracts are documentation
only. A runtime assertion mode (dev only) could warn when a contributor
receives unexpected props. Deferred — convention is sufficient pre-1.0.
- **Slot visibility controls.** An admin might want to disable a specific
contribution without disabling the entire contributing extension. A
per-contribution enable/disable toggle in the admin UI. Deferred —
extension enable/disable is the current granularity.
- **Cross-surface slot contributions.** An extension contributing to
`notes:toolbar-actions` AND `chat:message-actions` with the same
underlying logic but different UI. The `contributes` field already
supports multiple entries. The extension's JS registers into both
slots with slot-appropriate components.
- **Backend event subscriptions between extensions.** Currently extensions
react to kernel events via `hooks`. Extension-to-extension events
(e.g., "image-gen completed" → "chat refreshes") flow through the
existing event bus — the generating extension's api_route publishes
via `realtime.publish()`, the consuming surface listens via
`sw.realtime.subscribe()`. No new primitive needed.

306
docs/DESIGN-native-mtls.md
View File

@@ -1,306 +0,0 @@
# DESIGN — Native mTLS
**Version:** v0.6.7
**Status:** Implemented
**Author:** Jeff / Claude session 2026-03-31
---
## Problem
The existing `MTLSProxyProvider` assumes a TLS-terminating reverse proxy
(Traefik, nginx, Istio) sits in front of the Go binary. Identity arrives
via injected HTTP headers (`X-SSL-Client-CN`, etc.). The backend never
participates in the TLS handshake.
A target deployment class — systemd units + podman on bare metal or VMs —
has no reverse proxy. All connections must be mTLS end-to-end:
- **Client → server:** browser/CLI presents a user client cert
- **Node → node:** cluster peers mutually authenticate for heartbeat,
realtime pub/sub, and registry sync
The Go binary must terminate TLS itself and extract identity directly
from the verified peer certificate.
---
## Non-Goals
- Certificate revocation (CRL/OCSP). Deferred — short-lived certs +
restart is sufficient pre-1.0.
- Automatic cert rotation. Operators restart the process after replacing
certs on disk.
- Acting as a CA at runtime. Cert issuance is an offline operator task.
---
## Architecture
### TLS Server Mode
New config knob `TLS_MODE` with three values:
| Value | Behavior |
|----------|----------------------------------------------------|
| `none` | Plain HTTP. Current default. Use behind a proxy. |
| `server` | Server-side TLS only. No client cert required. |
| `mtls` | Mutual TLS. Client cert required and verified. |
When `TLS_MODE` is `server` or `mtls`, the binary calls
`http.Server.ListenAndServeTLS` with paths from `TLS_CERT` and `TLS_KEY`.
When `TLS_MODE` is `mtls`, `tls.Config` is built with:
```go
&tls.Config{
ClientAuth: tls.RequireAndVerifyClientCert,
ClientCAs: loadCACertPool(cfg.TLSCA),
MinVersion: tls.VersionTLS13,
}
```
TLS 1.3 is the floor. No cipher suite configuration exposed — Go's
defaults for 1.3 are non-negotiable and correct.
### Auth Provider: MTLSNativeProvider
Completely separate from `MTLSProxyProvider`. No shared code paths for
cert extraction. They share only downstream helpers.
```
┌──────────────────────────────────────────────────┐
│ Shared helpers │
│ ParseDN() · FingerprintCert() · ProvisionUser() │
└────────────────┬──────────────┬──────────────────┘
│ │
┌────────────┴───┐ ┌──────┴────────────┐
│ MTLSProxy │ │ MTLSNative │
│ Provider │ │ Provider │
│ │ │ │
│ Reads headers: │ │ Reads TLS: │
│ X-SSL-Client-* │ │ PeerCertificates │
└────────────────┘ └───────────────────┘
```
**MTLSNativeProvider.Authenticate():**
```go
func (p *MTLSNativeProvider) Authenticate(c *gin.Context, s store.Stores) (*Result, error) {
if c.Request.TLS == nil || len(c.Request.TLS.PeerCertificates) == 0 {
return nil, ErrNoCert
}
peer := c.Request.TLS.PeerCertificates[0]
cn := peer.Subject.CommonName // → username
dn := peer.Subject.String() // → metadata
fp := FingerprintCert(peer) // sha256(DER) → external_id
return p.resolveOrProvision(c.Request.Context(), s, cn, dn, fp)
}
```
`FingerprintCert` computes `sha256(cert.Raw)` hex-encoded. This is the
stable identity anchor — certs can be reissued with the same CN but will
get a new fingerprint, creating a new user unless the operator explicitly
maps them.
**Why Option B (two providers, not a mode switch):**
The security properties differ fundamentally. The proxy provider trusts
headers — if deployed without a proxy that strips client-injected headers,
an attacker can forge identity by sending `X-SSL-Client-CN: admin`. The
native provider trusts the TLS stack — identity is cryptographically
verified before the HTTP layer runs. Mixing these behind a single
code path with a runtime switch invites misconfiguration. Two types,
selected at startup by `TLS_MODE`, eliminates the ambiguity.
### Node-to-Node mTLS
Each Armature node is both server and client:
- **Server:** accepts connections from peers and user clients on its
listen address.
- **Client:** dials peers for cluster registry heartbeat, realtime
event forwarding, and (future) distributed task coordination.
When dialing a peer, the node presents its own cert:
```go
peerTLS := &tls.Config{
Certificates: []tls.Certificate{nodeKeypair},
RootCAs: caCertPool,
MinVersion: tls.VersionTLS13,
}
client := &http.Client{
Transport: &http.Transport{TLSClientConfig: peerTLS},
}
```
Both sides verify against the same CA. A node whose cert is not signed
by the cluster CA cannot join. This replaces any need for shared secrets
or bearer tokens in the cluster protocol.
**Interaction with cluster registry (DESIGN-cluster-registry.md):**
The existing design uses LISTEN/NOTIFY over PostgreSQL for node
discovery and an UNLOGGED `node_registry` table for heartbeat. mTLS
does not change the discovery mechanism — it secures the HTTP transport
between nodes once they've discovered each other. The heartbeat sweep,
self-eviction, and leaderless-by-default properties are orthogonal.
### Cert Topology
```
cluster-ca.crt / cluster-ca.key ← generated once, lives on operator workstation
├── node-1.crt + node-1.key ← SAN: node-1.internal, 10.0.0.1
├── node-2.crt + node-2.key ← SAN: node-2.internal, 10.0.0.2
├── user-jeff.crt + user-jeff.key ← CN=jeff, client auth only
└── user-alice.crt + user-alice.key
```
**Node certs** carry both `ServerAuth` and `ClientAuth` extended key
usages. SANs include the node's hostname and IP — required because
peers may dial by either.
**User certs** carry only `ClientAuth`. The CN becomes the username.
Email can go in the Subject if desired (parsed by `ParseDN()`).
The CA private key **never** touches a running node. Nodes receive only:
`cluster-ca.crt` (for verifying peers), their own `node-N.crt`, and
their own `node-N.key`.
### Cert Provisioning Tooling
A shell script (`scripts/armature-ca.sh`) wrapping `openssl` is the
KISS path. No new binary, no new dependency. Three commands:
```
armature-ca init
→ generates cluster-ca.crt + cluster-ca.key in ./ca/
armature-ca issue-node --name node-1 --san "node-1.internal,10.0.0.1"
→ generates node-1.crt + node-1.key in ./nodes/
armature-ca issue-user --cn jeff [--email jeff@example.com]
→ generates jeff.crt + jeff.key in ./users/
```
The script generates a proper `openssl.cnf` fragment per invocation
with the right SANs, EKUs, and validity period (default 365 days for
nodes, 90 days for users). All output is PEM.
If we later want a Go-native tool (for cross-platform or embedding),
it's a straightforward port — the `crypto/x509` stdlib does everything
openssl does here.
---
## Config Surface
```env
# TLS mode — controls whether the binary terminates TLS
# none: plain HTTP (default, use behind a proxy)
# server: TLS, no client cert required
# mtls: mutual TLS, client cert required
TLS_MODE=mtls
# Cert paths (required when TLS_MODE != none)
TLS_CERT=/etc/armature/tls/node.crt
TLS_KEY=/etc/armature/tls/node.key
TLS_CA=/etc/armature/tls/ca.crt
# Listen address (default :8080 for none, :8443 for tls/mtls)
LISTEN_ADDR=:8443
# Auth mode — when TLS_MODE=mtls, this should be mtls
AUTH_MODE=mtls
```
`TLS_MODE` and `AUTH_MODE` are independent knobs:
- `TLS_MODE=server` + `AUTH_MODE=builtin` = HTTPS with password login
- `TLS_MODE=mtls` + `AUTH_MODE=mtls` = full mTLS identity
- `TLS_MODE=none` + `AUTH_MODE=mtls` = proxy-terminated mTLS (existing behavior)
The third combination is the current `MTLSProxyProvider` path. The
first is useful for deployments that want encrypted transport but
password auth (e.g., single-node with Let's Encrypt).
---
## Testing Strategy
**Unit tests (in-process, no network):**
Generate an ephemeral CA and certs using `crypto/x509` + `crypto/ecdsa`
in `TestMain`. Tests call `MTLSNativeProvider.Authenticate()` with a
fabricated `*http.Request` whose `TLS.PeerCertificates` is populated
directly.
- Valid cert → user provisioned, correct CN/fingerprint
- No TLS → `ErrNoCert`
- Empty PeerCertificates → `ErrNoCert`
- Second call with same fingerprint → same user returned (idempotent)
- Different CN, same fingerprint → same user (cert reissue scenario)
**Integration tests (real TLS listener):**
Spin up `http.Server` with `tls.Config` on localhost. Dial with
`http.Client` using test client certs.
- No client cert → TLS handshake rejected (Go returns it before handler)
- Wrong CA → TLS handshake rejected
- Valid cert → 200, user created
- Expired cert → TLS handshake rejected
**Node-to-node integration:**
Two in-process servers with distinct node certs, same CA. Verify:
- Node A can heartbeat to Node B
- Node with wrong-CA cert cannot dial either
- Connection refused when cert SANs don't match dial address
(if `tls.Config.ServerName` is set — TBD whether we enforce this
for internal traffic or rely solely on CA verification)
---
## Migration Notes
No database schema changes. The `users` table already has `auth_source`
and `external_id` columns from the v0.24.1 mTLS/OIDC work. The native
provider populates them identically to the proxy provider.
The new code is:
| File | Purpose |
|------|---------|
| `server/auth/mtls_native.go` | `MTLSNativeProvider` type |
| `server/auth/mtls_helpers.go` | Shared: `ParseDN`, `FingerprintCert`, `resolveOrProvision` |
| `server/auth/mtls_native_test.go` | Unit + integration tests |
| `server/config/tls.go` | `TLSConfig` struct, loader, validation |
| `scripts/armature-ca.sh` | Cert provisioning wrapper |
`server/auth/mtls.go` (existing) is renamed to `mtls_proxy.go` for
clarity. No behavioral changes to the proxy provider.
---
## Open Questions
1. **SAN enforcement on node-to-node:** Do we verify `ServerName`
matches the dialed address, or just verify the cert is CA-signed?
Strict SAN checking is more secure but makes IP changes painful.
Recommendation: verify CA signature only for internal traffic;
SAN checking for client-facing listen.
2. **Cert rotation without restart:** Not in scope for v0.6.7, but
the `tls.Config.GetCertificate` / `GetConfigForClient` callbacks
make it straightforward to add later — read cert from disk on each
handshake (with caching). Worth noting in the design so we don't
paint ourselves into a corner.
3. **PKCS#12 / combined files:** Some operators prefer `.p12` bundles
over separate `.crt`/`.key` files. Low priority but trivial to add
via `crypto/pkcs12.Decode`. Defer unless requested.

744
docs/DESIGN-notes-v011x.md
View File

@@ -1,744 +0,0 @@
# DESIGN: Notes Reference Extension — v0.11.x
## Status: Proposed
## Purpose
Notes becomes the first **reference extension** — a first-party package
that exercises every kernel primitive and proves the platform is capable
of delivering a production-quality application. This is not a toy demo;
it is the answer to "what can you build on Armature?"
The v0.11.x series takes the existing notes package (folders, tags,
wikilinks, graph, CM6 editor, three view modes) and builds it into a
full-featured knowledge base comparable to Obsidian, Notion, or Logseq —
but running on Armature's extension architecture.
**The UI must be clean, inviting, and genuinely enjoyable to use.** The
current notes surface is functional but visually mechanical — no
personality, no micro-interactions, no sense of craft. If the reference
extension feels like a developer prototype, the platform pitch fails
regardless of how many kernel primitives it exercises. v0.11.0 is a
full visual redesign before any feature work begins, and every subsequent
version ships with UX quality built in, not bolted on.
### What Notes Exercises
| Kernel Primitive | How Notes Uses It |
|-----------------|-------------------|
| `db.write` | Notes, folders, tags, links, shares tables |
| `sw.renderers` | Markdown rendering pipeline, custom block renderers |
| `sw.panels` | `notes.reference` panel (from v0.10.4) |
| `sw.slots` | Declares `notes:toolbar-actions`, `notes:note-footer` for extension composition |
| `sw.events` | Realtime updates, panel ↔ surface communication |
| `sw.markdown` | Unified markdown rendering with wikilink extensions |
| `sw.shell.topbar` | Topbar slots for navigation context |
| `sw.storage` | Editor state, sidebar collapse, preferences |
| `sw.actions` | Exports `notes.create`, `notes.search` for cross-package calls |
| Resource grants | Note and folder sharing via kernel permission model |
| Starlark sandbox | All backend logic in `script.star` |
| Settings cascade | Per-user editor mode, default view, daily note folder |
| Surfaces | Multi-surface: full editor + public read-only surface |
---
## What Already Exists (v0.9.0)
### Frontend (1,806 lines — `js/main.js`)
- **NotesApp** — main shell with sidebar + editor layout
- **FolderTree / FolderNode** — tree with expand/collapse, context menu,
drag-and-drop note→folder, rename, create sub-folder (schema has
`parent_id`, UI renders depth)
- **NoteCard** — list items with title, snippet, date, tags, drag handle
- **TagInput / TagFilter** — tag CRUD with autocomplete
- **EditorPane** — three view modes (rendered / edit / split), CodeMirror
6 integration with textarea fallback, auto-save, frontmatter parsing
- **BacklinksPanel** — lists notes that link to the current note
- **SidebarTabs** — Notes / Outline tabs
- **SidebarOutline** — heading tree parsed from markdown body
- **GraphPane** — canvas-based force-directed graph with folder coloring,
orphan hiding, focus mode, pan/zoom
- **Export** — download note as `.md` with frontmatter
### Backend (23K — `script.star`)
- Full CRUD: notes, folders, tags
- Wikilink extraction (`_extract_wikilinks`) and link resolution
(`_sync_links`) on every note save
- Links / backlinks queries
- Graph data endpoint (nodes + edges)
- Search (title + body substring)
- Stats endpoint
- Folder CRUD with `parent_id`
### Schema (4 tables)
- `notes` — title, body, folder_id, creator_id, updated_at, pinned, archived
- `tags` — note_id, tag
- `links` — source_id, target_id, link_text
- `folders` — name, parent_id, creator_id, sort_order
### Current CSS (795 lines — `css/main.css`)
Functional but visually flat. No animations, no keyframes, no
micro-interactions. Identical `transition: var(--transition)` on every
hover. No typographic hierarchy beyond font-size/weight. No visual
rhythm. Sidebar and editor feel like admin panels, not a writing tool.
---
## Version Plan
### v0.11.0 — UI/UX Foundation
**Goal:** Complete visual redesign of the notes surface. Every pixel
intentional. Every interaction feels crafted. This is the foundation
that all subsequent feature versions build on.
This is NOT a "make it pretty" pass on the existing CSS. It is a
ground-up rethink of the notes UI with the understanding that this is
the first thing people see when evaluating what Armature can do.
**Design principles for notes:**
- **Writing-first.** The editor area dominates. Generous whitespace.
Nothing competes for attention with the content the user is writing.
- **Quiet chrome, loud content.** Sidebar, toolbar, and navigation fade
into the background. The note body is the visual anchor — clean
typography, comfortable line height, readable measure (60–80 chars).
- **Progressive disclosure.** Folder tree, tags, backlinks, graph — all
there but not all visible simultaneously. Contextual — show what's
relevant to what the user is doing right now.
- **Micro-interactions that feel alive.** Folder expand/collapse with
rotation animation. Note cards with subtle lift on hover. Smooth
sidebar resize. Mode transitions that animate, not snap. Save
indicator that pulses, not just appears.
- **Personality without kitsch.** The UI should feel like a well-designed
indie app, not a Material Design template and not a Bootstrap theme.
**Specific deliverables:**
**Typography overhaul:**
- Rendered note body: system serif stack for body text (Georgia, serif
fallback) at 16px/1.7 line height. Headings in the system sans stack.
Comfortable reading measure — `max-width: 720px` centered in the
editor area with generous padding. Code blocks with distinct
background and a monospace stack.
- Editor (CM6): matching font size and line height so switching between
rendered/edit mode doesn't jar. CM6 theme tokens aligned with note
design tokens.
- Sidebar text: smaller (13px), tighter, utility font. Clear hierarchy
between folder names, note titles, snippet text, dates.
**Sidebar redesign:**
- **Resizable** — drag handle between sidebar and editor. Width persisted.
Smooth resize with no layout jank.
- **Collapsible** — collapse to a thin icon strip (folder + search +
graph icons) on narrow viewports or by user choice. Expand on hover
or click. Collapse state persisted.
- **Folder tree polish:** Indent guides (subtle vertical lines connecting
parent→child). Folder icons that change on expand (open folder / closed
folder, not just a triangle). Drop target highlighting with animation
(not just background color change). Smooth height animation on
expand/collapse.
- **Note cards:** Subtle left border color-coded by folder (pulls from a
soft palette, not harsh primaries). Title, snippet, relative date
("3h ago" not "2026-04-03T12:34"). Tag pills with rounded, muted
styling. Pin indicator as a subtle icon, not a text label.
- **Search:** Inline search with clear button. Results highlight matching
text. Smooth appear/disappear.
**Editor redesign:**
- **Header:** Title input styled as a large heading (not an input field
with a border). Folder breadcrumb below title in muted text. Toolbar
actions as icon buttons with tooltips — no text labels cluttering the
header. Save status indicator (saved ✓ / saving… / unsaved •) as a
small, elegant badge.
- **Mode switcher:** Segmented control (Read / Edit / Split) replacing
the cycling button. Clear visual state.
- **Rendered view:** Clean markdown rendering with proper spacing between
elements. Block quotes with a left accent border. Tables with subtle
borders and alternating row tinting. Inline code with pill-style
background. Links with underline on hover only.
- **Empty editor state:** "Select a note or create a new one" with a
softly illustrated empty state — not a plain text message.
**Graph visual refresh:**
- Dark-on-light node rendering with soft shadows (not flat circles with
outlines). Node labels that appear on hover with smooth fade-in.
Edge rendering with slight curves (not straight lines). Background
subtle dot grid.
**Transitions and animations:**
- Sidebar expand/collapse: 200ms ease-out slide.
- Folder tree expand/collapse: 150ms height animation with children
fading in.
- Note card hover: subtle translateY(-1px) + box-shadow lift.
- View mode switch: crossfade (100ms fade out → swap → 100ms fade in).
- Save indicator: pulse animation on "saving", check mark with brief
scale-up on "saved".
- Graph node hover: scale(1.2) with spring easing.
- Panel/dialog open: 150ms slide-up + fade-in (consistent with kernel
Dialog animation).
**Color and theming:**
- Notes should work beautifully in both light and dark themes.
Use CSS custom properties (already the pattern via `var(--bg-surface)`,
etc.) but add notes-specific tokens for accent colors, folder palette,
and typography:
```css
--notes-body-font: Georgia, 'Times New Roman', serif;
--notes-body-size: 16px;
--notes-body-line-height: 1.7;
--notes-body-measure: 720px;
--notes-accent: var(--accent);
--notes-folder-1: #6366f1; /* indigo */
--notes-folder-2: #8b5cf6; /* violet */
--notes-folder-3: #ec4899; /* pink */
--notes-folder-4: #f59e0b; /* amber */
--notes-folder-5: #10b981; /* emerald */
--notes-folder-6: #06b6d4; /* cyan */
```
**Responsive:**
- Below 768px: sidebar collapses to overlay (slide-in from left).
Editor goes full-width. Split view disabled. Touch-friendly tap
targets (44px minimum on all interactive elements).
- Between 768px and 1024px: sidebar narrower (220px). Editor gets
remaining space.
- Above 1024px: full layout with comfortable sidebar width.
**What this version does NOT change:**
- No new features. Same note CRUD, same folders, same tags, same
wikilinks, same graph, same three view modes.
- No backend changes. No schema changes.
- The existing JS components are restructured for the new layout but
retain their current behavior.
**Deliverable:** Complete CSS rewrite (`css/main.css`), targeted JS
changes for new layout structure (resizable sidebar, collapsible sidebar,
mode switcher component, animation hooks), and updated component
templates where the HTML structure needs to change for the new design.
---
### v0.11.1 — Deep Folders + Navigation
**Goal:** Folders become a real hierarchy with breadcrumbs, not just a
flat tree with parent_id.
**Features:**
- **Breadcrumb navigation** in editor header showing the folder path.
Click any segment to navigate to that folder's note list. Styled
consistently with the v0.11.0 muted-text breadcrumb design.
- **Drag folder→folder** to reparent. Animated drop indicator shows
nesting target with indent guide preview. Depth limit: 5 levels.
- **Folder sort** — drag to reorder within a level. Smooth reorder
animation. Persists via `sort_order` column (already in schema).
- **Collapse/expand persistence** — expanded folder set stored in
`sw.storage` keyed by user.
- **Move dialog** — select destination folder from a tree picker when
moving notes (alternative to drag for accessibility / many folders).
Uses `sw.ui.Dialog` with the new folder tree component inside.
**UX standard:** All folder interactions use the animation language
established in v0.11.0. Drop targets highlight with the folder color
from the palette. Reparent shows a brief connection-line animation.
**Backend changes:**
- Validate `parent_id` chain on create/update (no cycles, depth ≤ 5).
- Folder delete: require empty or offer cascade (move children to parent,
or move contained notes to Unfiled).
- `_list_folders()` returns full tree with `children_count` and
`note_count` for each folder.
**Schema:** No changes.
---
### v0.11.2 — Wikilinks + Backlinks
**Goal:** `[[wikilinks]]` become a first-class editing and navigation
primitive with autocomplete, previews, and unresolved link handling.
**Features:**
- **CM6 autocomplete** — typing `[[` triggers a fuzzy note title picker
with a clean dropdown styled consistently with the notes design
language (not browser-default autocomplete). Shows note title, folder
path, and snippet preview. Keyboard navigable.
- **Wikilink rendering** — rendered markdown converts `[[Title]]` to
internal links with a subtle notes-specific style (dotted underline,
small link icon). Distinct from external URLs.
- **Hover preview** — hovering a wikilink shows a floating card with
the target note's title, first ~200 chars rendered as markdown, and
folder/tag metadata. Smooth fade-in, positioned to avoid viewport
overflow. Same visual language as graph node hover cards.
- **Unresolved links** — `[[Nonexistent Note]]` renders with a dashed
styling and a muted color. Click to create the note with that title
(pre-filled). Small "+" indicator on hover.
- **Backlinks panel improvements:**
- Show context snippet (surrounding text with the `[[link]]`
highlighted).
- Group by folder with folder color indicators.
- Count badge in sidebar tab (animated increment on new backlinks).
- Navigate to the linking note and scroll to the link location.
- **Aliases** — new `aliases` column on `notes` table. Autocomplete
searches both title and aliases.
**UX standard:** Autocomplete dropdown uses the notes card styling.
Hover preview card shares visual language with graph tooltips. Unresolved
link creation is a smooth inline experience, not a navigate-away-and-back.
**Backend changes:**
- `_sync_links()` resolves aliases as well as titles.
- New endpoint: `GET /autocomplete?q=...` — fast title+alias prefix
search (limit 10).
- Backlinks response includes `context_snippet` field.
**Schema changes:**
- `notes` table: add `aliases` column (text, comma-separated).
---
### v0.11.3 — Live Preview + Rich Editing
**Goal:** The editor becomes genuinely pleasant to write in — live
preview that scrolls in sync, or an optional inline-formatted mode.
**Features:**
- **Split pane scroll sync** — scrolling the CM6 editor scrolls the
rendered preview to the corresponding position. Uses heading anchors
for coarse sync and line-height interpolation for fine sync.
- **Live preview debounce** — rendered pane updates as you type with
150ms debounce. Smooth content transitions (no jarring reflow).
- **Inline preview** (Obsidian-style) — optional mode where markdown
syntax hides and formatting appears inline while editing. CM6
decorations for bold, italic, headings, links, code blocks.
Syntax reappears when cursor enters the formatted region. This is
the "fourth mode" alongside rendered / edit / split — a true
WYSIWYG-ish editing experience without leaving markdown.
- **Image paste** — paste from clipboard, upload via `files` module,
insert `![](url)` at cursor. Progress indicator during upload.
Fallback: base64 inline if `files` module unavailable.
- **Table editing** — tab-to-next-cell, auto-expand columns, add
row/column with floating buttons on hover. Markdown tables become
usable instead of a formatting chore.
**UX standard:** Split pane has an animated gutter with a subtle drag
handle. Mode switcher expands to accommodate the fourth option (or
becomes a dropdown if space is tight). Image paste shows a smooth
inline loading skeleton.
**Backend changes:**
- Image upload endpoint: `POST /images` — receives multipart, stores
via `files` module, returns URL.
**Schema:** No changes.
---
### v0.11.4 — Note Sharing + Permissions
**Goal:** Notes become collaborative. Share a note or folder with
a team, group, individual, or the public — with read-only or read-write
permissions.
**Share Model:**
```
┌─────────────────────────────────────────────────────┐
│ Share Scope │ Granularity │ Permissions │
├──────────────────────────┼─────────────┼─────────────┤
│ Public (anonymous URL) │ note only │ read-only │
│ Team │ note/folder │ ro / rw │
│ Group │ note/folder │ ro / rw │
│ Individual (user) │ note/folder │ ro / rw │
└──────────────────────────┴─────────────┴─────────────┘
```
**Folder sharing inherits:** sharing a folder grants access to all notes
in that folder (and sub-folders). Individual note shares override folder
permissions (higher wins).
**Features:**
- **Share dialog** — clean modal (using `sw.ui.Dialog`) with scope
selector, user/team/group picker (using `sw.ui.UserPicker` or new
entity picker), permission toggle, and copy-link button. Share URL
preview with visual indicator.
- **Public notes surface** — new surface at `/s/notes/public/:share_id`.
Renders a single note in read-only mode with clean, minimal chrome.
Uses the v0.11.0 typography and rendered view styling. Armature
branding footer. No authentication required.
- **Shared-with-me view** — sidebar section (new tab) showing notes/folders
shared by others. Grouped by owner with avatar. Visual distinction
from your own notes.
- **Share indicators** — shared notes show a subtle share icon on the
note card. Folder tree shows share badge on shared folders. Hover
reveals share scope.
- **Permission enforcement** — Starlark backend checks share permissions
on every read/write.
- **Share revocation** — remove access from share dialog with
confirmation.
**UX standard:** Share dialog is the most complex new UI in the series
and must feel simple. One-click sharing for common cases (share with
team, make public). Advanced options (specific users, groups) are
available but not in the way. Copy-link button shows a brief "Copied!"
toast. Public note surface is a showcase — the best the notes rendering
can look.
**Backend changes:**
- New `note_shares` table.
- New `folder_shares` table.
- Access check function: `_can_access(user_id, note_id, permission)`.
- `_list_notes()` includes shared notes with `shared_by` metadata.
- Public note endpoint: unauthenticated GET by share_token.
**Schema changes:**
- New table: `note_shares` — note_id, share_type, target_id, permission,
share_token, created_by, created_at.
- New table: `folder_shares` — folder_id, share_type, target_id,
permission, share_token, created_by, created_at.
**Manifest changes:**
- New surface: `/notes/public/:share_id` with `auth: "public"`.
---
### v0.11.5 — Graph + Outline Hardening
**Goal:** The graph becomes a genuine navigation and discovery tool.
The outline becomes a reliable TOC.
**Graph improvements:**
- **d3-force layout** — replace hand-rolled force sim with d3-force.
Better convergence, collision avoidance, centering. ~15KB optional
vendor (same pattern as CM6).
- **Zoom controls** — explicit +/- buttons (styled as floating pills) plus
scroll-to-zoom. Reset-to-fit button.
- **Minimap** — small overview in corner with viewport rectangle.
Draggable to navigate large graphs.
- **Cluster by folder** — visual grouping with soft convex hulls in
folder colors (from the v0.11.0 palette). Toggle on/off.
- **Filter by tag** — highlight matching nodes, dim others. Smooth
opacity transition.
- **Search in graph** — highlight and smooth-pan to matching nodes.
- **Hover card** — note title, snippet, folder, tag count. Same visual
language as wikilink hover preview (v0.11.2).
**Outline improvements:**
- **Scroll sync** — active heading highlights as user scrolls. Uses
`IntersectionObserver`. Smooth highlight transition.
- **Click-to-scroll** — smooth scroll with brief heading highlight pulse.
- **Indent levels** — h1→h6 with subtle indent guides matching the
folder tree style.
- **Collapse/expand** — heading sections collapsible in outline.
Animated, consistent with folder tree animations.
**UX standard:** The graph should feel like a discovery tool, not a tech
demo. Interactions should be fluid — pan, zoom, hover, click — with no
jank. The outline should feel like a table of contents in a well-typeset
book.
---
### v0.11.6 — Quick Switcher + Commands
**Goal:** Keyboard-driven navigation. Power users never touch the mouse.
**Features:**
- **Quick switcher** (`Cmd+O` / `Ctrl+O`) — centered overlay with fuzzy
search. Clean design: large input field, results below with note
title, folder path, and snippet. Recent notes above search results.
Arrow keys + Enter. Smooth open/close animation. Matches the visual
language of command palettes in VS Code / Raycast.
- **Command palette** (`Cmd+Shift+P`) — same overlay style, listing all
available note actions as searchable commands. Extensible via
`sw.actions`.
- **Keyboard shortcuts:** Registered via a central keymap.
| Shortcut | Action |
|----------|--------|
| `Cmd+O` | Quick switcher |
| `Cmd+Shift+P` | Command palette |
| `Cmd+N` | New note |
| `Cmd+S` | Save (in edit mode) |
| `Cmd+E` | Toggle edit/rendered |
| `Cmd+Shift+E` | Toggle split view |
| `Cmd+D` | Open today's daily note |
| `Cmd+G` | Toggle graph view |
| `Cmd+B` | Toggle backlinks panel |
| `Escape` | Close active panel/dialog |
**UX standard:** The quick switcher is a high-frequency interaction —
it must open instantly (no perceptible delay), search results must
appear as-you-type, and the whole flow (Cmd+O → type → Enter) should
take under 2 seconds for a user who knows what they want.
---
### v0.11.7 — Daily Notes + Templates
**Goal:** Recurring note patterns that lower friction.
**Daily notes:**
- Auto-created daily note — navigating to "today" creates a note titled
`YYYY-MM-DD` in a configurable daily notes folder.
- Daily note template — configurable template body.
- **Calendar picker** — small, beautiful calendar widget in the sidebar.
Days with notes dot-marked. Styled as a subtle, compact component
(not a full-page calendar). Click a date to open that day's note.
- Previous/next day nav arrows in editor header for daily notes.
**Templates:**
- Notes in a "Templates" folder are templates. No special type.
- "New Note" dropdown offers template selection with preview.
- Template variables: `{{date}}`, `{{time}}`, `{{title}}`, `{{folder}}` —
simple string replacement.
- **Slash commands** in CM6 — typing `/` at line start shows a styled
command menu: `/template`, `/date`, `/time`, `/todo`, `/callout`,
`/table`, `/code`, `/divider`. Extensible via `sw.slots`.
**UX standard:** The calendar picker should feel like a subtle,
integrated part of the sidebar — not a jarring widget. Template
selection should show a live preview of the template content.
Slash commands should appear fast and be keyboard navigable.
**Settings additions:**
- `daily_note_folder` — folder name for daily notes.
- `daily_note_template` — template body.
- `template_folder` — folder name for templates.
---
### v0.11.8 — Transclusion + Embeds
**Goal:** Notes reference and embed each other's content.
**Features:**
- **Transclusion** (`![[Note Title]]`) — embeds referenced note content
inline in a clean bordered container with source title as header link.
- **Section transclusion** (`![[Note Title#Heading]]`) — embeds content
under specified heading only.
- **Block transclusion** (`![[Note Title^block-id]]`) — single paragraph
by block ID.
- **Recursion guard** — depth limited to 3. Circular references show
a clean warning card, not an error.
- **File attachments** — drag-and-drop files onto editor to upload.
Non-image files render as styled download cards (file icon, name,
size).
- **Embed preview in editor** — CM6 decoration shows read-only preview
below `![[...]]` line. Collapsible with smooth animation.
**UX standard:** Transclusions should feel like natural parts of the
document, not foreign inclusions. The border and header link should be
subtle. Embedded content uses the same typography as the host note.
Collapse animation is smooth.
---
### v0.11.9 — Composability: Slots + Actions
**Goal:** Notes becomes a host surface that other extensions enhance.
**Slot declarations:**
```json
{
"slots": {
"notes:toolbar-actions": {
"description": "Action buttons in the note editor toolbar",
"context": {
"noteId": "string",
"getContent": "function — returns markdown body",
"setContent": "function — replaces markdown body",
"getTitle": "function — returns title"
}
},
"notes:note-footer": {
"description": "Content rendered below the note body",
"context": { "noteId": "string", "content": "string — HTML" }
},
"notes:sidebar-tabs": {
"description": "Additional tabs in the notes sidebar",
"context": { "activeNoteId": "string or null" }
},
"notes:slash-commands": {
"description": "Additional slash commands in the editor",
"context": { "insertText": "function(text)" }
}
}
}
```
**Exported actions:**
```json
{
"exports": {
"actions": {
"notes.create": "Create note (params: title, body, folder_id)",
"notes.search": "Search notes (params: query, limit)",
"notes.get": "Get note by ID (params: note_id)"
}
}
}
```
**UX standard:** Slot contributions must fit visually. Toolbar action
buttons contributed by other extensions inherit the notes icon-button
styling. Sidebar tabs contributed by extensions match the built-in tab
design. The notes surface should not look different when extensions
contribute to its slots.
---
### v0.11.10 — Panel Enhancement + Quality Gate
**Goal:** The `notes.reference` panel (shipped in v0.10.4) inherits
the v0.11.x features. Final quality gate.
**Panel updates:**
- Reference panel gets the v0.11.0 visual design language.
- Search, folder filter, tag filter in panel.
- Preview pane shows rendered note content (v0.11.0 typography).
- Click-to-insert: emits `panel.notes.reference.selected` for host.
- Wikilink resolution works within panel preview.
**Quality gate criteria:**
- All v0.11.x features exercised in SDK test runner.
- Notes-runner surface tests cover: CRUD, sharing, wikilinks, daily
notes, templates, transclusion.
- Graph renders correctly with 200+ nodes at 60fps.
- Public note surface renders without authentication.
- At least one slot contribution demonstrated.
- Panel reference works in at least one other surface.
- **UX review:** every screen, every interaction, every empty state
reviewed against v0.11.0 design principles. No "placeholder" UI
surviving to this point.
- **Responsive:** full experience on mobile (sidebar overlay, touch
targets, no horizontal scroll).
- **Accessibility:** keyboard navigation through all views. ARIA labels.
Focus management on panel/dialog open/close.
- **Performance:** virtual scrolling on note list for 1000+ notes.
Graph 60fps with 500+ nodes. Lazy-load graph and CM6.
---
## Uniquely Armature
| Feature | Obsidian/Notion | Armature Notes |
|---------|-----------------|----------------|
| Extension slots | Plugin API (Obsidian) | Kernel-level composability — any package contributes to any slot without coupling |
| Sharing | Obsidian Publish (paid), Notion sharing | Built on kernel resource model — same permission patterns as every other extension |
| Panels | N/A | Notes is a panel provider — embed a notes view inside chat, dashboard, or any surface |
| Multi-surface | Single window | Full surface + public surface + reference panel, all from one package |
| Server-side logic | Local only (Obsidian) | Starlark backend — search, link resolution, access control run server-side |
| Realtime | N/A (Obsidian), built-in (Notion) | Via `sw.realtime` — same primitive available to all extensions |
| Cross-extension actions | Plugin API | `sw.actions` — any package can call `notes.create` or `notes.search` |
| Self-hosted | Obsidian Sync (paid) | Runs on your infrastructure, your data, your rules |
The thesis: **Notes isn't just a note-taking app — it's proof that
Armature's extension architecture can deliver a production-quality
knowledge base that competes with purpose-built SaaS tools, while
remaining decomposable and extensible.** And it has to *look and feel*
like it competes, not just architecturally compete.
---
## Schema Summary
### Existing tables (no changes)
- `notes` — title, body, folder_id, creator_id, updated_at, pinned, archived
- `tags` — note_id, tag
- `links` — source_id, target_id, link_text
- `folders` — name, parent_id, creator_id, sort_order
### New columns
- `notes.aliases` (text) — comma-separated aliases (v0.11.2)
### New tables
- `note_shares` — note_id, share_type, target_id, permission, share_token, created_by, created_at (v0.11.4)
- `folder_shares` — folder_id, share_type, target_id, permission, share_token, created_by, created_at (v0.11.4)
---
## Settings Summary
### Existing
- `default_view` — "recent" or "pinned"
- `editor_mode` — "rendered", "edit", or "split"
### New
| Setting | Type | Default | Version |
|---------|------|---------|---------|
| `daily_note_folder` | string | "Daily Notes" | v0.11.7 |
| `daily_note_template` | string | `"# {{date}}\n\n"` | v0.11.7 |
| `template_folder` | string | "Templates" | v0.11.7 |
| `inline_preview` | boolean | false | v0.11.3 |
| `graph_hide_orphans` | boolean | false | v0.11.5 |
| `graph_cluster_by` | string | "folder" | v0.11.5 |
---
## Design Decisions
| Decision | Rationale |
|----------|-----------|
| UI redesign as v0.11.0, before features | Every subsequent version builds on the visual foundation. Shipping features on an ugly base means reworking the UI of every feature when the redesign eventually happens. Front-loading avoids double work and ensures every version screenshot looks like a real product. |
| Notes as first reference extension | Exercises more kernel primitives than any other candidate. Complex enough to prove the platform; familiar enough to be immediately useful. |
| Dedicated v0.11.x series (11 versions) | Notes is too large for a single version. Each v0.11.x is independently shippable and CI-green. |
| UX quality threaded through every version | Each version specifies its UX standard, not just functionality. No "make it pretty later" — every feature ships with its final visual quality. |
| Share model via new tables | Notes sharing needs note/folder-specific semantics (inheritance, public URLs, share tokens). Purpose-built tables beat generic resource grants for this use case. |
| Aliases as comma-separated text | Small set (1–3 per note). Separate table adds join complexity for marginal normalization benefit. |
| d3-force for graph | Existing hand-rolled sim has poor convergence at 100+ nodes. d3-force is battle-tested, ~15KB. |
| Transclusion rendered on view, not live-synced | Live-sync requires WebSocket subs per embedded note. Disproportionate complexity pre-MVP. |
| Templates are just notes in a folder | No template schema. KISS. |
| Inline preview as fourth editing mode | Obsidian's killer UX feature. Users who want WYSIWYG-ish editing without leaving markdown get it without a ProseMirror dependency — CM6 decorations handle it. |
---
## Dependency Chain
```
v0.11.0 UI/UX Foundation ← EVERYTHING depends on this
v0.11.1 Deep Folders ← foundation for folder sharing (v0.11.4)
v0.11.2 Wikilinks ← foundation for transclusion (v0.11.8)
v0.11.3 Live Preview ← foundation for inline preview + embeds (v0.11.8)
v0.11.4 Sharing ← depends on deep folders (v0.11.1)
v0.11.5 Graph + Outline ← independent (hardening existing features)
v0.11.6 Quick Switcher ← independent (keyboard navigation)
v0.11.7 Daily Notes ← depends on deep folders (v0.11.1) for daily folder
v0.11.8 Transclusion ← depends on wikilinks (v0.11.2) + live preview (v0.11.3)
v0.11.9 Composability ← independent (slot/action declarations)
v0.11.10 Polish + Gate ← depends on all above
```
Versions v0.11.5, v0.11.6, and v0.11.9 are independent and can be
reordered if priorities shift.

743
docs/DESIGN-panels.md
View File

@@ -1,743 +0,0 @@
# DESIGN: Panels — v0.10.x
## Status: Proposed
## Problem
Packages export one rendering granularity today: a **surface** — a
full-page application that owns `#extension-mount`. There is no way
for a package to offer a lightweight, composable view of itself that
another surface can pull in.
Concrete example: the `notes` package provides a full note editor
surface at `/s/notes`. When a user is in the `chat` surface and wants
to reference their notes, the only option is to navigate away. There is
no mechanism to embed a notes panel alongside chat, whether as a
floating window, a docked sidebar, or a bottom strip.
This forces users into one-thing-at-a-time workflows and prevents the
"extensions extending extensions" composability story from reaching the
UI layer. Slots and actions allow injection of buttons and menu items,
but not entire companion views.
Without kernel coordination, package authors who need this will roll
their own floating containers, producing:
- **Z-index wars** between packages and kernel overlays (Dialog at 1000,
debug at 9999, toast in between).
- **Inconsistent drag/resize** behavior (touch support, bounds clamping,
accessibility).
- **No position persistence** — panels reset on every navigation.
- **No layout negotiation** — docked panels can't tell the host surface
to shrink.
## Non-Goals
- **Window management system.** This is not a tiling WM. The kernel
provides a small set of presentation modes (floating, docked). Complex
layouts are post-1.0 horizon.
- **Cross-surface panel sharing at runtime.** A panel runs inside one
host surface's page context. It is not an iframe or a separate
browsing context.
- **Panel-to-panel communication.** Panels communicate with their host
surface (and each other) via `sw.events`. No new IPC mechanism.
- **Server-side panel rendering.** Panels are frontend-only. The kernel
resolves dependencies and serves JS; rendering happens in the browser.
## What Already Exists
**Three rendering tiers exist today; panels fill the gap between the
first two:**
| Tier | Scope | Example |
|------|-------|---------|
| **Surface** | Full-page app, owns `#extension-mount` | Notes editor, Chat, Admin |
| *(gap)* | *Composable companion view* | *Notes reference panel in Chat* |
| **Block renderer** | Inline content unit in markdown | Mermaid diagram, KaTeX formula |
**Kernel primitives that panels build on:**
- `sw.ui.Dialog` — centered modal with focus trap, backdrop, z-index 1000.
- `sw.ui.Drawer` — slide-in side panel (left/right), backdrop, z-index 1000.
- `sw.events` — pub/sub event bus, local and realtime. Already the
communication channel between surfaces and slot contributors.
- `sw.storage` — per-user localStorage wrapper with namespaced keys.
- `sw.slots` — named UI injection points with manifest declarations.
- Manifest `depends` field — package dependency resolution at install time.
**What panels add:** A second rendering granularity between surfaces and
block renderers, with kernel-managed presentation modes and lifecycle.
---
## Architecture
### 1. Three-Tier Rendering Hierarchy
After this change, every package can export up to three things:
```
Surface — full-page app, owns the mount, has shell topbar slots
Panel — composable view, kernel-managed container, presentation-agnostic
Renderer — inline content block (sw.renderers)
```
The key architectural property: **the panel component does not know its
presentation mode.** It renders into whatever container the kernel
provides. Floating, docked-right, docked-bottom — that is a shell/layout
concern. The panel receives a mount element and a size. Nothing else.
### 2. Manifest Schema
#### Provider: `panels` field
The package that provides a panel declares it in its manifest:
```json
{
"id": "notes",
"surfaces": ["/notes"],
"panels": {
"reference": {
"entry": "js/panels/reference.js",
"title": "Notes",
"icon": "📝",
"description": "Searchable note list with quick preview",
"min_width": 280,
"min_height": 200,
"default_width": 400,
"default_height": 350
}
}
}
```
Panel IDs are namespaced: `<package_id>.<panel_key>`. The notes
reference panel above is addressed as `notes.reference` everywhere
in the SDK and in consumer manifests.
A package may declare multiple panels:
```json
{
"id": "notes",
"panels": {
"reference": { "entry": "js/panels/reference.js", ... },
"graph": { "entry": "js/panels/graph.js", ... }
}
}
```
#### Consumer: `panels` dependency
The consuming surface declares which panels it wants available:
```json
{
"id": "chat",
"surfaces": ["/chat"],
"panels": ["notes.reference"]
}
```
This is a **soft dependency**: the panel is available if the providing
package is installed and enabled. If not, `sw.panels.open()` returns
`false` and the consuming surface can degrade gracefully (hide the
button, show a tooltip explaining the missing package).
This is distinct from `depends` (hard dependency — install fails without
it). A consumer does not need the provider installed to function; the
panel is an enhancement.
### 3. Panel Entry Point Contract
The panel JS module exports a single `mount` function. The kernel calls
it with a DOM element and a context object. It returns a cleanup function.
```js
// notes/js/panels/reference.js
const { html } = window;
const { render } = preact;
export function mount(el, ctx) {
// ctx.sw — SDK instance
// ctx.params — optional params from sw.panels.open()
// ctx.panelId — 'notes.reference'
// ctx.close — function to request close
// ctx.resize — function(width, height) to request resize
render(html`<${NotesReference} sw=${ctx.sw} />`, el);
// Return cleanup
return () => render(null, el);
}
```
**Rules:**
- The panel must not assume any particular container size. Use CSS
`width: 100%; height: 100%` and let the kernel container control
dimensions.
- The panel must not create its own overlay, backdrop, or drag handle.
The kernel wraps the panel in the appropriate chrome.
- The panel must not set `z-index` on anything. The kernel manages
stacking.
- The panel may use `ctx.close()` to request its own dismissal (e.g.
user clicks an "X" inside the panel content area).
- The panel may use `ctx.resize(w, h)` to suggest a new size, but the
kernel may clamp or ignore the request.
- Communication with the host surface is via `sw.events` — the same
bus surfaces and slot contributors already use.
### 4. SDK API — `sw.panels`
New SDK module: `src/js/sw/sdk/panels.js`
```js
export function createPanels(events, storage) {
const _registry = new Map(); // panelId → manifest entry
const _active = new Map(); // panelId → { el, cleanup, mode, ... }
return {
/**
* Register a panel from manifest data. Called by the kernel
* during surface load — not by package code directly.
*/
_register(panelId, manifest) { ... },
/**
* Open a panel. Lazy-loads the JS entry if not yet loaded.
*
* @param {string} panelId — e.g. 'notes.reference'
* @param {object} opts
* @param {string} opts.mode — 'floating' | 'docked-right' | 'docked-left' | 'docked-bottom'
* @param {object} opts.params — passed to mount(el, ctx)
* @param {number} opts.width — override default width
* @param {number} opts.height — override default height
* @returns {Promise<boolean>} — false if panel not available
*/
async open(panelId, opts = {}) { ... },
/**
* Close a panel. Calls cleanup, removes container.
*/
close(panelId) { ... },
/**
* Toggle open/close.
*/
toggle(panelId, opts = {}) { ... },
/**
* Check if a panel is currently open.
*/
isOpen(panelId) { ... },
/**
* Check if a panel is available (provider installed + enabled).
*/
isAvailable(panelId) { ... },
/**
* List available panel IDs for the current surface.
*/
list() { ... },
/**
* List currently open panel IDs.
*/
active() { ... },
};
}
```
**Exposed on `sw` as:**
```js
sw.panels.open('notes.reference', { mode: 'floating' });
sw.panels.open('notes.reference', { mode: 'docked-right' });
sw.panels.close('notes.reference');
sw.panels.toggle('notes.reference');
sw.panels.isOpen('notes.reference'); // boolean
sw.panels.isAvailable('notes.reference'); // boolean
sw.panels.list(); // ['notes.reference', 'notes.graph']
sw.panels.active(); // ['notes.reference']
```
### 5. Presentation Modes
#### Floating (v0.10.1)
```
┌──────────────────────────────────────────────────────────┐
│ ← │ Chat │ 🔔 │ 👤 │
├──────────────────────────────────────────────────────────┤
│ │
│ Host surface (interactive) │
│ ┌───────────────────┐ │
│ │ ≡ Notes ─ ✕ │ │
│ │ │ │
│ │ [panel content] │ │
│ │ │ │
│ │ ◢ │ │
│ └───────────────────┘ │
│ │
└──────────────────────────────────────────────────────────┘
```
- Draggable via title bar (pointer + touch events).
- Resize via bottom-right handle.
- Viewport bounds clamping — panel cannot be dragged fully offscreen
(at least 48px of title bar must remain visible).
- No backdrop — host surface remains fully interactive.
- Z-index above surfaces, below Dialog/Drawer overlays.
- On focus or drag-start, panel gets next-z (multiple floating panels
stack correctly).
- Escape key: configurable — close panel or do nothing (default: close).
- Position and size persisted in `sw.storage` keyed by panel ID.
**Title bar chrome (kernel-provided):**
```
┌────────────────────────────────────┐
│ ≡ │ {icon} {title} │ ─ ✕ │
└────────────────────────────────────┘
│ │ │
drag handle minimize close
```
- **≡** Drag handle / grip indicator.
- **─** Minimize: collapses to a small pill at the bottom of the viewport.
Click to restore. Minimized state persisted.
- **✕** Close: calls cleanup, removes panel.
- Title and icon from manifest.
#### Docked (v0.10.2)
```
┌──────────────────────────────────────────────────────────┐
│ ← │ Chat │ 🔔 │ 👤 │
├──────────────────────────────┬──┬────────────────────────┤
│ │▐▐│ 📝 Notes ✕ │
│ Host surface │▐▐│ │
│ (shrinks to fit) │▐▐│ [panel content] │
│ │▐▐│ │
│ │▐▐│ │
└──────────────────────────────┴──┴────────────────────────┘
resize
gutter
```
- Docked panels consume space from the host surface. The
`#extension-mount` area shrinks via CSS flex or grid.
- Resize gutter between host and panel (drag to adjust split).
- Three dock positions: right (default), left, bottom.
- No z-index concerns — docked panels are in normal flow.
- Split ratio persisted in `sw.storage` keyed by panel ID + mode.
- Close button removes panel, host surface reclaims full width/height.
#### Mode Transitions (v0.10.2)
A floating panel can be dragged to a dock zone (edge highlight on
hover). A docked panel's title bar can be dragged away to float.
The panel component is never unmounted during a mode transition — only
the wrapping container changes.
The user controls presentation mode. Surfaces can suggest a default
mode in `sw.panels.open()`, but the user's last-used mode (persisted)
takes precedence.
### 6. Z-Index Strategy
Panels slot into the existing stacking context:
| Layer | Z-Index | Contents |
|-------|---------|----------|
| Base | 0 | Surface content, docked panels (in flow) |
| Floating panels | 100–199 | `sw.panels` floating mode (auto-incrementing) |
| Shell overlays | 200–299 | Dropdowns, tooltips, menus |
| Drawer | 1000 | `sw.ui.Drawer` |
| Dialog | 1000 | `sw.ui.Dialog` + confirm/prompt |
| Toast | 1100 | `sw.ui.toast` |
| Debug | 9999 | Debug panel |
Floating panels start at z-index 100. Each panel that receives focus
or drag-start gets `max(current panel z-indexes) + 1`, capped at 199.
If the cap is hit, all floating panels are renumbered starting from 100
(maintaining relative order). This prevents unbounded z-index growth.
When a Dialog or Drawer opens, floating panels remain visible beneath
the overlay — they do not auto-hide. This preserves the "see while
doing" property.
### 7. Dependency Resolution + Loading
**At install time:** The kernel validates that panel dependencies
reference valid `<package>.<panel>` identifiers. Invalid references
are warnings (soft deps), not errors.
**At surface load time:**
1. Kernel reads the surface's `panels` array from its manifest.
2. For each declared panel, check if the providing package is installed
and enabled. Build the available panel list.
3. Register available panels into `sw.panels._register()` with their
manifest metadata (entry path, title, icon, size constraints).
4. Panel JS is **not** loaded yet — lazy loading on first
`sw.panels.open()`.
**On `sw.panels.open(panelId)`:**
1. If panel JS not yet loaded: `import()` the entry module from the
package's static asset path
(`/api/v1/ext/<package>/static/js/panels/<name>.js`).
2. Call `module.mount(el, ctx)` where `el` is the content area of the
kernel-provided container (FloatingPanel or DockedPanel component).
3. Store the cleanup function returned by `mount()`.
4. Emit `panels.opened` event with `{ panelId, mode }`.
**On `sw.panels.close(panelId)`:**
1. Call stored cleanup function.
2. Remove kernel container from DOM.
3. Emit `panels.closed` event with `{ panelId }`.
### 8. Panel Communication
No new mechanism. Panels and their host surface share the same
`sw.events` bus and `sw` SDK instance. Convention-based event
namespacing:
```js
// Host surface requests the notes panel to show a specific note
sw.emit('panel.notes.reference.show', { noteId: '...' });
// Panel listens
sw.on('panel.notes.reference.show', (data) => {
navigateToNote(data.noteId);
});
// Panel notifies the host that a note was selected
sw.emit('panel.notes.reference.selected', { noteId: '...', title: '...' });
// Host listens
sw.on('panel.notes.reference.selected', (data) => {
insertNoteLink(data);
});
```
The `params` object in `sw.panels.open()` provides initial state:
```js
sw.panels.open('notes.reference', {
mode: 'floating',
params: { folderId: 'inbox', highlight: noteId }
});
```
Params are passed to `mount(el, ctx)` as `ctx.params`. This avoids
the need for an event round-trip on first open.
### 9. CSS
New file: `src/css/panels.css`
**Floating panel:**
```css
.sw-panel-floating {
position: fixed;
display: flex;
flex-direction: column;
border: 1px solid var(--border);
border-radius: var(--radius-lg);
background: var(--bg-surface);
box-shadow: var(--shadow-lg);
overflow: hidden;
/* z-index set dynamically by JS */
}
.sw-panel-floating__titlebar {
display: flex;
align-items: center;
gap: var(--sp-2);
padding: var(--sp-2) var(--sp-3);
background: var(--bg-raised);
border-bottom: 1px solid var(--border);
cursor: grab;
user-select: none;
flex-shrink: 0;
}
.sw-panel-floating__titlebar:active {
cursor: grabbing;
}
.sw-panel-floating__title {
flex: 1;
font-size: 13px;
font-weight: 600;
color: var(--text);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.sw-panel-floating__actions {
display: flex;
gap: var(--sp-1);
}
.sw-panel-floating__action {
background: none;
border: none;
color: var(--text-3);
cursor: pointer;
padding: 2px 4px;
border-radius: var(--radius-sm);
font-size: 14px;
line-height: 1;
}
.sw-panel-floating__action:hover {
color: var(--text);
background: var(--bg-hover);
}
.sw-panel-floating__body {
flex: 1;
overflow: auto;
}
.sw-panel-floating__resize {
position: absolute;
bottom: 0;
right: 0;
width: 16px;
height: 16px;
cursor: nwse-resize;
}
/* Minimized pill */
.sw-panel-pill {
position: fixed;
bottom: var(--sp-3);
display: flex;
align-items: center;
gap: var(--sp-2);
padding: var(--sp-2) var(--sp-3);
background: var(--bg-surface);
border: 1px solid var(--border);
border-radius: var(--radius);
box-shadow: var(--shadow);
cursor: pointer;
font-size: 12px;
color: var(--text-2);
z-index: 100;
}
.sw-panel-pill:hover {
color: var(--text);
background: var(--bg-raised);
}
```
**Docked panel:**
```css
.sw-panel-dock-container {
display: flex;
height: 100%;
overflow: hidden;
}
.sw-panel-dock-container--bottom {
flex-direction: column;
}
.sw-panel-dock-gutter {
flex-shrink: 0;
background: var(--bg-raised);
border: 1px solid var(--border);
cursor: col-resize;
width: 5px;
}
.sw-panel-dock-container--bottom .sw-panel-dock-gutter {
cursor: row-resize;
width: auto;
height: 5px;
}
.sw-panel-docked {
display: flex;
flex-direction: column;
overflow: hidden;
background: var(--bg-surface);
}
.sw-panel-docked__header {
display: flex;
align-items: center;
gap: var(--sp-2);
padding: var(--sp-2) var(--sp-3);
border-bottom: 1px solid var(--border);
background: var(--bg-raised);
flex-shrink: 0;
}
.sw-panel-docked__body {
flex: 1;
overflow: auto;
}
```
**Touch targets (mobile):**
```css
@media (max-width: 768px) {
.sw-panel-floating__action {
min-width: 44px;
min-height: 44px;
display: inline-flex;
align-items: center;
justify-content: center;
}
.sw-panel-floating__titlebar {
padding: var(--sp-3) var(--sp-4);
}
.sw-panel-dock-gutter {
width: 10px;
}
.sw-panel-dock-container--bottom .sw-panel-dock-gutter {
height: 10px;
}
}
```
### 10. Backend Changes
**Minimal.** The kernel backend already serves extension static assets
and reads manifests. Changes:
1. **Manifest parsing** (`server/extensions/manifest.go`): Parse the
`panels` field from provider manifests (map of panel key → entry
metadata). Parse the `panels` field from consumer manifests (array
of panel ID strings). Store in the existing `PackageRecord` struct.
2. **Surface load endpoint** (`/api/v1/surfaces/:id`): Include resolved
panel metadata in the response — for each panel declared by the
surface, include the provider's panel manifest entry (title, icon,
entry path, size constraints) if the provider is installed and
enabled. This lets the frontend `sw.panels._register()` at surface
boot without additional API calls.
3. **Static asset serving**: Already works — panels are served from
the same package static path as surface JS
(`/api/v1/ext/<package>/static/...`).
**No new tables.** No new migrations. Panel metadata lives in the
existing manifest JSON stored in the packages table.
### 11. Admin Visibility
The admin packages page already shows package manifests. With panels:
- The package detail view shows declared panels with their metadata.
- The dependency view shows which surfaces consume which panels.
- Panel availability warnings surface when a provider package is
disabled but consumers reference its panels.
This falls out naturally from the existing manifest introspection —
no new admin surfaces needed.
---
## Event Inventory
| Event | Emitted by | Payload |
|-------|-----------|---------|
| `panels.opened` | Kernel (sw.panels) | `{ panelId, mode }` |
| `panels.closed` | Kernel (sw.panels) | `{ panelId }` |
| `panels.mode_changed` | Kernel (sw.panels) | `{ panelId, from, to }` |
| `panels.focused` | Kernel (sw.panels) | `{ panelId }` |
| `panel.<id>.*` | Convention (package code) | Package-defined |
All panel events are `localOnly: true` — they do not broadcast over
the WebSocket. Panels are a frontend-only concern.
---
## Persistence
Panel state persisted in `sw.storage` (localStorage wrapper):
| Key | Value | Scope |
|-----|-------|-------|
| `panel_pos_<panelId>` | `{ x, y, w, h }` | Floating position + size |
| `panel_mode_<panelId>` | `'floating' \| 'docked-right' \| ...` | Last-used presentation mode |
| `panel_split_<panelId>` | `{ ratio: 0.3 }` | Docked split ratio |
| `panel_minimized_<panelId>` | `true` | Minimized state |
Persisted per-user (sw.storage is already user-scoped). Cleared when
the providing package is uninstalled.
---
## Version Plan
| Version | Title | Scope |
|---------|-------|-------|
| v0.10.0 | Panel Manifest + Lifecycle | Manifest `panels` field (provider + consumer). Backend parsing. `sw.panels` SDK module (register, open, close, toggle, isOpen, isAvailable, list). Lazy JS loading pipeline. No presentation UI yet — panels render into a plain unstyled container for contract validation. |
| v0.10.1 | FloatingPanel Primitive | `FloatingPanel` Preact component with drag, resize, minimize, z-index stacking, viewport bounds, position persistence. `panels.css` floating section. Touch support. |
| v0.10.2 | Docked Panels + Mode Transitions | `DockedPanel` component. Layout negotiation (flex resize of `#extension-mount`). Resize gutter. Drag-to-dock / drag-to-float transitions. `panels.css` docked section. Split ratio persistence. |
| v0.10.3 | Panel Communication Patterns | Event namespace conventions documented. `ctx.params` for initial state. SDK helper: `sw.panels.send(panelId, event, data)` as sugar over `sw.emit('panel.' + panelId + '.' + event, data)`. Panel communication section in extension developer guide. |
| v0.10.4 | Reference Panel: Notes | Notes package ships `panels.reference` — searchable note list with quick preview. Chat declares it as a consumer. End-to-end proof: open notes panel in chat, select a note, insert link. Test runner coverage. |
Each version independently CI-green. v0.10.0 is the foundation that
all subsequent versions build on.
---
## Changeset Plan — v0.10.0
| CS | Scope | Description |
|----|-------|-------------|
| CS-1 | Backend (Go) | Manifest parsing: `panels` field on provider + consumer manifests. `PackageRecord` struct update. Surface load endpoint includes resolved panel metadata. |
| CS-2 | Frontend (JS) | `sw.panels` SDK module: `createPanels()`, register, open, close, toggle, isOpen, isAvailable, list, active. Lazy `import()` loader. Unstyled container mount. |
| CS-3 | Frontend (JS) | SDK boot integration: wire `sw.panels` into SDK boot sequence. Register panels from `__MANIFEST__` surface data. |
| CS-4 | Frontend (JS) | Panel lifecycle events: `panels.opened`, `panels.closed`, `panels.focused`. |
| CS-5 | Tests | SDK test runner: panel registration, open/close lifecycle, lazy loading, availability checks, event emission. |
---
## Design Decisions
| Decision | Rationale |
|----------|-----------|
| Kernel primitive, not package-level | Z-index coordination, drag/resize consistency, layout negotiation, and position persistence are all kernel concerns. Multiple packages need this. |
| Soft dependency (not `depends`) | Chat should work without notes installed. Panel availability is a runtime check, not an install-time requirement. |
| Presentation-agnostic panel contract | `mount(el, ctx)` / cleanup is the entire API surface. Panels never know if they are floating or docked. This lets us add new modes without touching panel code. |
| Lazy loading | Panel JS is not loaded until first `sw.panels.open()`. Surfaces that declare panels but the user never opens them pay zero cost. |
| Event bus for communication | `sw.events` already exists and is the established pattern for cross-package communication. No new mechanism needed. |
| No new backend tables | Panel metadata is manifest data stored in the existing packages table JSON. No schema changes. |
| Z-index band 100–199 | Above surface content, below shell overlays and Dialog/Drawer. Renumber on cap to prevent unbounded growth. |
| User controls mode, surface suggests | The surface can pass `mode: 'docked-right'` as a default, but the user's persisted mode preference wins. Users know their own workflow. |
---
## Future Considerations (post-v0.10.x)
- **User-composable layouts.** Power users drag panels from different
packages into a persistent workspace layout. Requires a layout
persistence model beyond per-panel position.
- **Panel marketplace metadata.** Panels become a discoverability
feature: "this package provides 2 surfaces and 3 panels."
- **Multi-monitor.** Pop a floating panel into a separate browser
window via `window.open()` + `SharedWorker` for event bus. Very
post-1.0.
- **Keyboard shortcuts.** `Ctrl+Shift+N` opens the notes panel.
Requires a keybinding registry (not yet a kernel primitive).

448
docs/DESIGN-shell-contract.md
View File

@@ -1,448 +0,0 @@
# DESIGN: Shell Contract — v0.7.0
## Status: Proposed
## Problem
Extension surfaces render into `#extension-mount` with no shell chrome.
A full audit (v0.7.0 surface audit) found: 3/4 primary surfaces lack a
notification bell, 2/4 lack a user menu, 4 different topbar implementations,
user menu never updates on package/role changes, toast-and-forget error
handling, and empty states that explain nothing.
## Solution
### 1. Shell Topbar — Two-Slot Model
The kernel injects a topbar for all extension surfaces. Two named slots
(left + center) let surfaces customize without replacing the entire bar.
**Layout:**
```
┌──────────────────────────────────────────────────────────────┐
│ ← │ [left] │ [center: flex-1] │ 🔔 │ 👤 │
└──────────────────────────────────────────────────────────────┘
```
- **← (home):** Always visible. Navigates to `__BASE__/`. Simple `<a>`.
- **Left slot:** Defaults to manifest title. Surfaces override with `setLeft()`.
- **Center slot:** Empty by default. Surfaces inject tabs, search, pickers.
`flex: 1` — expands to fill available space.
- **Bell + User Menu:** Always visible. Kernel-managed.
**Template change** (`surfaces/extension.html`):
```html
{{define "surface-extension"}}
<div id="extension-surface" class="extension-surface"
data-surface-id="{{.Surface}}">
<div id="shell-topbar" class="sw-topbar sw-topbar--shell"></div>
<div id="extension-mount" class="extension-mount" data-ext="{{.Surface}}"></div>
</div>
{{end}}
```
**SDK API:**
```js
sw.shell.topbar.setLeft(vnode) // Override left slot (default: title)
sw.shell.topbar.setSlot(vnode) // Set center slot content
sw.shell.topbar.setTitle(str) // Shorthand: setLeft with plain text
sw.shell.topbar.hide() // Remove topbar entirely
sw.shell.topbar.show() // Restore after hiding
```
### 2. Three Navigation Patterns
The shell topbar provides the top bar. What happens below it is the
surface's business. Three patterns emerge naturally:
#### Pattern A — Default (simple extensions, Docs)
```
┌──────────────────────────────────────────┐
│ ← │ Surface Title │ 🔔 │ 👤 │
├──────────────────────────────────────────┤
│ │
│ Content (full width) │
│ │
└──────────────────────────────────────────┘
```
Title only, no tabs, no sidebar. Content gets everything.
Zero code required — kernel defaults handle it.
**Used by:** Docs, Notes, Chat, simple extensions.
#### Pattern B — Flat Tabs (no sidebar, full width)
```
┌──────────────────────────────────────────────────────────┐
│ ← │ Title │ Tab1 │ Tab2 │ Tab3 │ Tab4 │ │ 🔔 │ 👤 │
├──────────────────────────────────────────────────────────┤
│ │
│ Content (full width, no sidebar) │
│ │
└──────────────────────────────────────────────────────────┘
```
Tabs in the topbar center slot. No sidebar — content fills the full
viewport width. For surfaces with 3–7 sections that don't have sub-items.
More real estate for content than a sidebar layout.
**Surface code:**
```js
sw.shell.topbar.setSlot(html`
<div class="sw-topbar__tabs">
${sections.map(s => html`
<a class="sw-topbar__tab ${active === s.key ? 'active' : ''}"
href=${s.href} onClick=${navigate}>${s.label}</a>
`)}
</div>
`);
```
**Used by:** Team Admin (Members / Connections / Workflows / Settings / Activity),
Settings, Schedules.
#### Pattern C — Category Tabs + Sidebar (two-level)
```
┌─────────────────────────────────────────────────────────────┐
│ ← │ Title │ People │ Workflows │ System │ Mon │ 🔔 │ 👤 │
├──────────┬──────────────────────────────────────────────────┤
│ Users │ │
│ Teams │ Content │
│ Groups │ │
│ │ │
└──────────┴──────────────────────────────────────────────────┘
```
Major categories in the topbar (via center slot). Surface renders its own
sidebar inside the content area for sub-navigation within the active
category. The shell topbar doesn't know about the sidebar — it's a
surface-level div below `#extension-mount`.
**Surface code:**
```js
// Topbar: major categories
sw.shell.topbar.setLeft(html`
<img src="${BASE}/favicon.svg" width="18" height="18" style="vertical-align:-3px" />
<span style="margin-left:6px">Administration</span>
`);
sw.shell.topbar.setSlot(html`
<div class="sw-topbar__tabs">
${categories.map(c => html`
<a class="sw-topbar__tab ${activeCat === c.key ? 'active' : ''}"
href=${c.href} onClick=${navigate}>
<${CatIcon} paths=${c.icon} /> ${c.label}
</a>
`)}
</div>
`);
// Content area: sidebar is surface-owned
return html`
<div class="admin-body">
<div class="admin-nav">
${sidebarSections.map(s => html`...`)}
</div>
<div class="admin-content">
<${SectionComponent} />
</div>
</div>
`;
```
**Used by:** Admin.
### 3. Kernel-Provided Tab CSS
The kernel provides `.sw-topbar__tabs` and `.sw-topbar__tab` CSS so
surfaces get consistent tab styling. Not required — surfaces can style
their own slot content however they want.
```css
.sw-topbar__tabs {
display: flex;
align-items: center;
gap: var(--sp-1);
height: 100%;
}
.sw-topbar__tab {
display: flex;
align-items: center;
gap: var(--sp-2);
padding: var(--sp-2) var(--sp-3);
font-size: 13px;
font-weight: 500;
color: var(--text-2);
text-decoration: none;
border-radius: var(--radius-sm);
transition: color var(--transition), background var(--transition);
white-space: nowrap;
}
.sw-topbar__tab:hover {
color: var(--text);
background: var(--bg-hover);
}
.sw-topbar__tab.active {
color: var(--text);
background: var(--bg-2);
}
```
### 4. Primary Surface Migrations
#### Settings → Pattern B (flat tabs)
Currently: custom topbar (back + icon + "Settings"), sidebar nav.
After: shell topbar with flat tabs, no sidebar. Content full width.
Settings has 6 sections (General / Appearance / Profile / Teams /
Connections / Notifications) — perfect for flat tabs. The sidebar was
thin (~140px) and ate width from the content area for no good reason.
**Migration:**
- Delete `settings-topbar` div and CSS.
- Delete sidebar nav. Move section links into `sw.shell.topbar.setSlot()`.
- Remove `sb_settings_return` sessionStorage — shell home link handles it.
- Content area becomes full-width.
- Fix Teams section: add "Open Team Admin →" link, show role, add leave action.
Extension config sections (`__CONFIG_SECTIONS__`) render as additional
tabs after the divider. Same as today, just in the topbar instead of sidebar.
#### Admin → Pattern C (category tabs + sidebar)
Currently: custom topbar (favicon + "Administration" + category tabs + UserMenu).
After: shell topbar with category tabs in center slot, surface-owned sidebar.
**Migration:**
- Delete custom `admin-topbar` div and CSS.
- `sw.shell.topbar.setLeft()` with favicon + "Administration".
- `sw.shell.topbar.setSlot()` with category tabs (People / Workflows / System / Monitoring).
- Bell and user menu come from the shell — delete the explicit `<${UserMenu}>`.
- Admin sidebar and content area unchanged — they're below the topbar.
- Delete custom `CatIcon` renderer if category tab icons use standard SVG.
- Fix hardcoded `favicon.svg` — left slot can use theme-aware image.
**Result:** Admin looks identical to today but its topbar is kernel-managed.
Bell added for free. User menu reactive for free.
#### Team Admin → Pattern B (flat tabs)
Currently: custom topbar (back + "Team Admin: {name}"), sidebar nav.
After: shell topbar with flat tabs, no sidebar.
With Groups removed, Team Admin has 5 sections: Members / Connections /
Workflows / Settings / Activity. Perfect for flat tabs.
**Migration:**
- Delete `team-admin-topbar` div and CSS.
- `sw.shell.topbar.setTitle('Team Admin: ' + team.name)` after team fetch.
- Section tabs into `sw.shell.topbar.setSlot()`.
- Delete sidebar nav. Content full-width.
- Remove `sb_team_admin_return` sessionStorage.
- Remove Groups tab entirely (37-line dead-end).
- Fix signoff display: `user_id` → `sw.users.displayName()`.
#### Docs → Pattern A (default)
Currently: imports `shell/topbar.js` and renders it explicitly.
After: shell topbar auto-renders. No surface code needed.
**Migration:**
- Delete `import { Topbar }` and `<${Topbar}>` render.
- Shell topbar provides title + bell + user menu automatically.
- Docs sidebar (document list) is in the content area, unaffected.
### 5. User Menu Reactivity
**The single most impactful fix.**
**Backend — new WS events:**
```go
// After package install/uninstall/enable/disable:
h.hub.BroadcastToUser(userID, "package.changed", map[string]string{
"action": "installed", "id": packageID,
})
// After team role/membership change:
h.hub.BroadcastToUser(userID, "auth.changed", map[string]string{
"reason": "team_role",
})
```
**Frontend** (`user-menu.js`):
```js
useEffect(() => {
if (!sw?.api?.surfaces?.list) return;
function fetchSurfaces() {
sw.api.surfaces.list().then(data => {
const raw = Array.isArray(data) ? data : data?.data || [];
setAllSurfaces(raw);
}).catch(() => {});
}
fetchSurfaces();
const off1 = sw.on?.('package.changed', fetchSurfaces);
const off2 = sw.on?.('auth.changed', fetchSurfaces);
return () => {
if (typeof off1 === 'function') off1();
if (typeof off2 === 'function') off2();
};
}, [authenticated]);
```
**Event inventory:**
| Event | Trigger | Payload |
|-------|---------|---------|
| `package.changed` | Install, uninstall, enable, disable | `{ action, id }` |
| `auth.changed` | Team role change, group membership change | `{ reason }` |
| `notification.read` | Mark notification read | `{ id }` |
| `notification.all_read` | Mark all read | `{}` |
### 6. Notification Read Broadcast
**Backend** — after `MarkRead()` / `MarkAllRead()`:
```go
h.hub.BroadcastToUser(userID, "notification.read", map[string]string{"id": id})
h.hub.BroadcastToUser(userID, "notification.all_read", nil)
```
**Frontend** — `NotificationBell` listens for `.created`, `.read`, `.all_read`:
```js
const onRead = (e) => {
setNotifications(prev => prev.map(n =>
n.id === e.id ? { ...n, read_at: new Date().toISOString() } : n
));
};
const onAllRead = () => {
setNotifications(prev => prev.map(n => ({
...n, read_at: n.read_at || new Date().toISOString()
})));
};
```
### 7. Shell Announcement Global Dismiss
On dismiss, write: `localStorage.setItem('armature_dismissed_' + hash(text), '1')`.
On mount, check. New announcements (changed text) show again.
Implementation: inline `<script>` in `base.html` — checks localStorage on
DOMContentLoaded. Dismiss button writes the key. Works on every surface
including login.
### 8. Error Handling Pass
**Pattern** — inline error + retry replaces toast-and-forget:
```js
const [error, setError] = useState(null);
async function load() {
setError(null);
try {
const data = await sw.api.whatever.list();
setItems(data || []);
} catch (e) { setError(e.message); }
}
// In render:
${error && html`
<div class="sw-inline-error">
<span>${error}</span>
<button class="sw-btn sw-btn--secondary sw-btn--sm"
onClick=${load}>Retry</button>
</div>
`}
```
**New CSS class** (`sw-primitives.css`):
```css
.sw-inline-error {
display: flex; align-items: center; gap: var(--sp-3);
padding: var(--sp-3) var(--sp-4);
background: var(--bg-2); border: 1px solid var(--danger);
border-radius: var(--radius); font-size: 13px; color: var(--danger);
}
```
**Sections requiring this pass:**
| Surface | Section | Current | After |
|---------|---------|---------|-------|
| Admin | Workflows | Toast + empty | Inline error + retry |
| Admin | Packages | Toast + empty | Inline error + retry |
| Admin | Groups | Toast + empty | Inline error + retry |
| Team Admin | Workflows (adopt) | Toast + "No global workflows" | Inline error + retry |
| Team Admin | Members | Toast + empty | Inline error + retry |
| Settings | General | Console warn | Inline error + retry |
| Settings | Teams | Toast + empty | Inline error + retry |
| Workflow Demo (pkg) | Main | Silent swallow | Inline error + retry |
### 9. Empty State Guidance
Every "No X" empty state gets: one-line explanation, primary action or doc link.
| Surface | Section | Current | After |
|---------|---------|---------|-------|
| Admin | Groups | "No groups" | "Groups control access to surfaces and features via permissions." + Create button |
| Admin | Workflows | "No workflows" | "Workflows define multi-stage approval processes with team roles and SLA tracking." + Create button |
| Admin | Teams | "No teams" | "Teams group users for shared connections, workflows, and access control." + Create button |
| Team Admin | Workflows | "No workflows — create one or adopt" | Add: "Adopt copies a global workflow for this team to customize." |
| Settings | Notifications | "No notification preferences" | "Preferences appear when notification types are configured by an administrator." |
## Rebrand Asset Inventory
### Current → Target
| File | Current | Target |
|------|---------|--------|
| `favicon.svg` | Dark icon ✅ | Unchanged |
| `favicon-light.svg` | **MISNAMED** (520×80 wordmark) | Square icon, light mode **(NEW)** |
| `favicon-32.png` | Dark raster ✅ | Unchanged |
| `favicon-256.png` | Dark raster ✅ | Unchanged |
| `favicon-light-32.png` | Missing | Light raster **(NEW)** |
| `favicon-light-256.png` | Missing | Light raster **(NEW)** |
| `favicon.ico` | Dark ✅ | Unchanged |
| `wordmark.svg` | Missing | **RENAMED** from current `favicon-light.svg` |
| `wordmark-dark.svg` | Missing | Light text #E5E5E5 on transparent **(NEW)** |
| `manifest.json` | "Self-hosted AI chat..." | "Self-hosted extension platform..." |
## Bug Fixes (bundled)
- **evil-chat:** `finally` cleanup + tighten `409` assertion.
- **Workflow demo:** Silent `catch` → inline error + retry.
- **Hello dashboard:** Delete `packages/hello-dashboard/`.
## Changeset Plan
| CS | Scope | Description |
|----|-------|-------------|
| CS-1 | Backend (Go) | WS events: `notification.read`, `notification.all_read`, `package.changed`, `auth.changed`. Tests. |
| CS-2 | Frontend (JS + HTML) | Shell topbar component, SDK API (`setLeft`/`setSlot`/`setTitle`/`hide`), extension.html template, SDK boot auto-mount. |
| CS-3 | Frontend (JS) | User menu reactivity: listen for `package.changed` + `auth.changed`. Notification bell: listen for `.read` + `.all_read`. |
| CS-4 | Frontend (JS + HTML) | Surface migrations: Settings → Pattern B, Admin → Pattern C, Team Admin → Pattern B, Docs → Pattern A. Delete custom topbars. |
| CS-5 | Frontend (JS + CSS) | Error handling pass + empty state guidance. `sw-inline-error` CSS. All sections from inventory. |
| CS-6 | Frontend (JS) | Announcement global dismiss (localStorage). |
| CS-7 | Static + docs | Rebrand assets, manifest.json, REBRAND-SPEC.md. |
| CS-8 | Frontend (JS) | Bug fixes: evil-chat cleanup, workflow demo error, hello-dashboard deletion. |
Each changeset independently CI-green.

922
docs/DESIGN-sidecar-v014x.md
View File

@@ -1,922 +0,0 @@
# DESIGN: Sidecar Tier — v0.14.x
## Status: Proposed
## Problem
Starlark is the right sandbox for most extension logic: declarative,
safe, fast to start, no deployment pipeline. But some workloads cannot
run in Starlark:
- **ML inference** — Python + PyTorch/TF model loading, GPU access.
- **Media transcoding** — ffmpeg, ImageMagick, Whisper.
- **Language servers** — LSP processes for code-workspace.
- **Local git** — clone, diff, merge operations on workspace directories.
- **Custom runtimes** — anything needing system libraries, native code,
or long-running processes.
- **Personal local bridges** — a user's laptop exposing filesystem,
terminal, clipboard, or local compute to the Armature instance.
Today, the `http` module lets Starlark call external APIs (`api.http`
permission, domain allowlists, SSRF protection). This covers cloud
services (OpenAI, S3, external webhooks) but not **co-located processes**
that need access to Armature's data, events, and workspace filesystem,
nor **personal processes** on a user's own machine.
The sidecar tier bridges this gap: out-of-process extensions that
connect inward to the kernel, authenticate, register capabilities, and
participate in the extension ecosystem as first-class citizens — at both
the **instance level** (shared infrastructure) and the **user level**
(personal local tools).
**If done wrong, the consequences are severe:**
- Wrong auth model → arbitrary processes access kernel APIs.
- Wrong lifecycle → resource leaks, zombie processes, orphaned
registrations.
- Wrong API surface → every sidecar extension inherits the debt.
- Wrong isolation → a runaway sidecar takes down the cluster.
- Wrong discovery → fragile deployments that break on restart.
- Wrong scoping → user sidecars see other users' data.
This is invisible infrastructure. Users never see it. But every
extension that needs native code depends on the contract being right.
## Non-Goals
- **Kernel manages sidecar processes.** The kernel does not start, stop,
or restart sidecars. That's the deployment layer's job (K8s, Docker
Compose, systemd, or a human running a binary). The kernel discovers
sidecars that connect to it.
- **Sidecar marketplace.** Sidecar packages are installed like any other
`.pkg` file. The sidecar binary ships separately (container image,
standalone binary, pip package). The manifest declares the sidecar
endpoint; the operator deploys it.
- **Arbitrary bidirectional streaming.** Sidecars communicate via
HTTP/JSON request-response and WebSocket events. No gRPC, no custom
binary protocols. KISS.
- **Hot code reload.** Sidecar updates require process restart. The
kernel detects the reconnection and re-registers capabilities.
- **Multi-tenant sidecar isolation.** Instance-level sidecars run at the
instance level, not per-user. User-level isolation is handled by
user sidecars (v0.14.5), which are scoped to the owning user's
identity.
---
## Architecture
### Connect-Inward Model
The defining property: **sidecars connect TO the kernel, not the other
way around.** The kernel never reaches outward to discover or contact
a sidecar. This eliminates:
- K8s RBAC for the kernel to discover pods.
- Service mesh configuration.
- DNS-based service discovery.
- Any notion of the kernel "managing" external processes.
A sidecar is just a process that knows the kernel's address and has
credentials to authenticate. It could be a container in the same pod, a
separate deployment, a systemd service on the same machine, or a process
on a developer's laptop connected via tunnel.
```
┌─────────────────────────────────────────────────────┐
│ Deployment Layer (K8s / Docker / systemd / manual) │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ instance │ │ instance │ │ instance │ │
│ │ sidecar: │ │ sidecar: │ │ sidecar: │ │
│ │ ml-infer │ │ ffmpeg │ │ git-ops │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ │ connect inward │ │
│ ▼ ▼ ▼ │
│ ┌──────────────────────────────────────────┐ │
│ │ Armature Kernel │ │
│ │ ┌─────────────────────────────────────┐ │ │
│ │ │ Sidecar Registry (PG) │ │ │
│ │ │ ml-infer: instance, 3 caps │ │ │
│ │ │ ffmpeg: instance, 2 caps │ │ │
│ │ │ git-ops: instance, 4 caps │ │ │
│ │ │ jeff-local: user:jeff, 3 caps │ │ │
│ │ └─────────────────────────────────────┘ │ │
│ └────────────────────────────────▲─────────┘ │
│ │ │
└───────────────────────────────────┼──────────────────┘
│ connect inward
┌─────┴──────┐
│ user │
│ sidecar: │
│ jeff-local │
│ (laptop) │
└────────────┘
```
### Two Scopes
**Instance sidecars** — shared infrastructure. Deployed by the operator.
Any user's requests can invoke their capabilities. An ML inference
sidecar, a transcoding service, a git operations worker.
**User sidecars** — personal processes. Run by an individual user on
their own machine. Only the owning user's requests can invoke their
capabilities. A local file bridge, a personal Ollama instance, a
script that reacts to Armature events.
Both use the same connect-inward mechanism, same registration flow,
same capability contract. The difference is the identity binding and
access scoping.
### Two Authentication Modes
#### Mode A: Registration Token
For simple deployments (Docker Compose, single machine, dev/test).
1. Admin generates an instance sidecar token:
`armature sidecar token create --package ml-inference`
User generates a personal sidecar token (from settings UI or CLI):
`armature sidecar token create --personal`
2. Token is a signed JWT with claims: `{package_id, scope, user_id, exp}`.
Instance tokens: `scope: "instance"`, `user_id: null`.
User tokens: `scope: "user"`, `user_id: "<owner>"`.
3. Sidecar starts with the token as env: `ARMATURE_TOKEN=ey...`
4. Sidecar calls `POST /api/v1/sidecar/register`.
5. Kernel validates JWT, creates registry entry with appropriate scope.
#### Mode B: mTLS
For production deployments with certificate infrastructure.
1. Certificate CN encodes identity:
Instance: `sidecar:ml-inference`
User: `sidecar-user:jeff:local-bridge`
2. Kernel's `MTLSNativeProvider` parses the CN prefix and resolves to
the appropriate sidecar identity.
Both modes produce the same internal identity: a `SidecarIdentity`
struct with `{package_id, scope, user_id, node_id, registered_at}`.
### Sidecar Registry
```sql
CREATE TABLE IF NOT EXISTS sidecar_registry (
sidecar_id TEXT PRIMARY KEY,
package_id TEXT NOT NULL,
scope TEXT NOT NULL DEFAULT 'instance',
user_id TEXT,
endpoint TEXT NOT NULL,
heartbeat TIMESTAMPTZ DEFAULT now(),
capabilities JSONB DEFAULT '[]',
stats JSONB DEFAULT '{}',
registered_at TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX idx_sidecar_pkg ON sidecar_registry (package_id);
CREATE INDEX idx_sidecar_user ON sidecar_registry (user_id)
WHERE user_id IS NOT NULL;
```
`scope` is `'instance'` or `'user'`. `user_id` is NULL for instance
sidecars, set for user sidecars.
**Lifecycle (mirrors cluster registry):**
- **Register:** `POST /api/v1/sidecar/register` with capability
manifest. Kernel creates/updates registry row.
- **Heartbeat:** `POST /api/v1/sidecar/heartbeat` every N seconds.
- **Sweep:** Kernel's heartbeat loop sweeps stale sidecar entries
alongside node entries. 3× heartbeat interval = stale.
- **Self-eviction:** Heartbeat returns 404 → re-register or exit.
- **Deregister:** `DELETE /api/v1/sidecar/deregister` on graceful
shutdown.
### Capability Registration
On registration, the sidecar declares capabilities:
```json
POST /api/v1/sidecar/register
{
"package_id": "ml-inference",
"endpoint": "http://ml-inference:8080",
"capabilities": [
{
"name": "embed",
"description": "Generate text embeddings",
"input_schema": {
"type": "object",
"properties": {
"text": { "type": "string" },
"model": { "type": "string", "default": "all-MiniLM-L6-v2" }
},
"required": ["text"]
},
"timeout_seconds": 30
},
{
"name": "classify",
"description": "Zero-shot text classification",
"input_schema": { ... },
"timeout_seconds": 60
}
]
}
```
### Sidecar SDK (Starlark)
New `sidecar` Starlark module. Permission: `sidecar.call`.
```python
# Call an instance sidecar capability
result = sidecar.call("ml-inference", "embed", {"text": "Hello"})
# Check availability
available = sidecar.available("ml-inference")
# List capabilities
caps = sidecar.capabilities("ml-inference")
```
User sidecar calls are routed automatically. When a request originates
from user Jeff, `sidecar.call("user-bridge", "fs.search", {...})` hits
Jeff's personal sidecar. The same call from user Alice would hit Alice's
personal `user-bridge` sidecar (or fail if she doesn't have one). The
Starlark caller doesn't specify the user — the kernel resolves it from
the request context.
### Sidecar SDK (Frontend)
```js
const result = await sw.sidecar.call('ml-inference', 'embed', {
text: 'Hello, world'
});
const available = await sw.sidecar.available('user-bridge');
```
Frontend calls go through the kernel API. The kernel proxies to the
appropriate sidecar based on scope + user identity.
### Sidecar API Contract (Inbound to Kernel)
| Endpoint | Purpose | Auth |
|----------|---------|------|
| `POST /api/v1/sidecar/register` | Register + declare capabilities | Token or mTLS |
| `POST /api/v1/sidecar/heartbeat` | Heartbeat + stats | Token or mTLS |
| `DELETE /api/v1/sidecar/deregister` | Graceful shutdown | Token or mTLS |
| `GET /api/v1/ext/{pkg}/*` | Read extension data (scoped) | Token or mTLS |
| `POST /api/v1/ext/{pkg}/*` | Write extension data (scoped) | Token or mTLS |
| `POST /api/v1/sidecar/emit` | Emit events | Token or mTLS |
| `GET /api/v1/sidecar/subscribe` | WebSocket event subscription | Token or mTLS |
| `POST /api/v1/sidecar/lib/{pkg}/{fn}` | Cross-package function call | Token or mTLS |
**Scoping rules:**
- Instance sidecars: can access `/api/v1/ext/{own_package}/*` only.
- User sidecars: can access `/api/v1/ext/{own_package}/*` scoped to
the owning user's data within that package. The sidecar sees the
same data the user would see through the UI.
- Event emission scoped to the sidecar's package prefix.
### Sidecar API Contract (Outbound from Kernel)
| Endpoint | Purpose |
|----------|---------|
| `POST {sidecar_endpoint}/api/v1/exec/{capability}` | Execute a capability |
| `GET {sidecar_endpoint}/api/v1/health` | Health check |
The `/exec/{capability}` endpoint receives:
```json
{
"input": { ... },
"context": {
"user_id": "...",
"package_id": "...",
"request_id": "..."
}
}
```
Returns:
```json
{
"output": { ... },
"error": null
}
```
### Resource Limits
What the kernel enforces at the proxy layer:
- **Timeout:** Per-capability, from capability declaration.
- **Request rate:** Per-sidecar, configurable in admin.
- **Response size:** Default 10MB, configurable per-sidecar.
- **Concurrent calls:** Default 10, configurable per-sidecar.
User sidecars have separate (typically lower) defaults:
- **User sidecar rate limit:** Default 30/min (vs 100/min instance).
- **User sidecar concurrency:** Default 3 (vs 10 instance).
- **User sidecar response size:** Default 5MB (vs 10MB instance).
Admin can adjust user sidecar defaults globally. Individual users
cannot override admin limits.
### Manifest Integration
Instance sidecar manifest:
```json
{
"id": "ml-inference",
"type": "full",
"tier": "sidecar",
"version": "1.0.0",
"sidecar": {
"image": "gobha/armature-ml-inference:latest",
"required": true,
"health_endpoint": "/api/v1/health",
"env_hints": {
"ARMATURE_URL": "Kernel URL (auto-populated)",
"ARMATURE_TOKEN": "Registration token",
"MODEL_PATH": "Path to model files"
}
}
}
```
User sidecar manifest (installed as a regular package, sidecar runs
on user's machine):
```json
{
"id": "user-bridge",
"type": "full",
"tier": "sidecar",
"version": "1.0.0",
"sidecar": {
"scope": "user",
"required": false,
"download_url": "https://github.com/armature/user-bridge/releases",
"env_hints": {
"ARMATURE_URL": "Your Armature instance URL",
"ARMATURE_TOKEN": "Generate from Settings → Sidecars"
}
},
"surfaces": ["/user-bridge/settings"]
}
```
`sidecar.scope: "user"` tells the kernel this is a user sidecar
package. The package installs normally (admin or self-install depending
on permissions), but the sidecar binary runs on the user's machine.
### Tool Meta-Tool Integration
When `llm-bridge` assembles tools for a completion:
1. Instance-level actions from `sw.actions.list()` → available to all.
2. Instance sidecar capabilities → available to all.
3. **Requesting user's personal sidecar capabilities** → available to
that user only.
This means different users get different tool sets. Jeff has
`user-bridge` running with `fs.search`, `terminal.exec`, and
`clipboard.get`. Alice doesn't. When Jeff asks "Hey Max, find the auth
file on my desktop," Max has `user-bridge.fs.search` in his tool set.
When Alice asks the same, that tool doesn't exist — Max tells her he
can't access her local files.
The same AI persona, different capabilities per user, zero configuration.
### Admin UI
**Instance sidecar management:**
- Sidecar health status per registered instance sidecar.
- Capabilities list with call counts, error rates, latency.
- Token management: generate, revoke, list.
- Per-sidecar rate limit and concurrency configuration.
**User sidecar admin controls:**
- Permission: `sidecar.connect_personal` — admin grants to specific
users or groups.
- Capability allowlist for user sidecars — admin restricts which
capabilities user sidecars can declare.
- Global user sidecar resource limits (rate, concurrency, response
size).
- Monitoring: list of all connected user sidecars with owner, endpoint,
capabilities, last heartbeat.
- Kill switch: admin can revoke any user sidecar token or sweep a
specific user's sidecars.
**User settings (for users with `sidecar.connect_personal`):**
- Generate personal sidecar token.
- List personal connected sidecars with health status.
- Revoke own tokens.
- Download link / instructions for user sidecar binaries (from package
manifest `download_url`).
### Monitoring
Kernel-side metrics:
- `sidecar_call_total{package, capability, scope, status}` — counter.
- `sidecar_call_duration_seconds{package, capability, scope}` — histogram.
- `sidecar_heartbeat_age_seconds{package, scope}` — gauge.
- `sidecar_active{package, scope}` — gauge.
- `sidecar_user_count` — gauge (number of connected user sidecars).
---
## User Sidecar Use Cases
### Local Bridge (Desktop Commander Pattern)
A thin agent on the user's laptop that exposes local capabilities:
```json
{
"capabilities": [
{
"name": "fs.search",
"description": "Search local filesystem by filename or content",
"input_schema": {
"properties": {
"query": { "type": "string" },
"path": { "type": "string", "default": "~" },
"content_search": { "type": "boolean", "default": false }
},
"required": ["query"]
},
"timeout_seconds": 15
},
{
"name": "fs.read",
"description": "Read a local file",
"input_schema": {
"properties": {
"path": { "type": "string" },
"max_bytes": { "type": "integer", "default": 1048576 }
},
"required": ["path"]
},
"timeout_seconds": 10
},
{
"name": "terminal.exec",
"description": "Execute a shell command",
"input_schema": {
"properties": {
"command": { "type": "string" },
"cwd": { "type": "string", "default": "~" },
"timeout": { "type": "integer", "default": 30 }
},
"required": ["command"]
},
"timeout_seconds": 60
},
{
"name": "clipboard.get",
"description": "Read clipboard contents",
"input_schema": {},
"timeout_seconds": 5
}
]
}
```
**Tool meta-tool flow:**
```
User: "Hey Max, find the database migration script I was
working on yesterday and add it to my notes"
llm-bridge assembles (for this user):
Layer 5 tools: [...instance tools..., user-bridge.fs.search,
user-bridge.fs.read, notes.create, ...]
LLM:
1. Tool: user-bridge.fs.search({query: "migration", path: "~/code"})
2. Result: [{path: "~/code/armature/migrations/015_sidecar.sql", ...}]
3. Tool: user-bridge.fs.read({path: "~/code/armature/migrations/015_sidecar.sql"})
4. Result: {content: "CREATE TABLE IF NOT EXISTS sidecar_registry..."}
5. Tool: notes.create({title: "Migration 015 - Sidecar Registry",
body: "```sql\nCREATE TABLE..."})
6. Text: "Found your sidecar migration script and saved it to Notes."
```
Three extensions orchestrated: user's local filesystem, instance-level
notes, all through the same tool meta-tool pattern.
### Personal Compute (Local Ollama)
User runs Ollama on their workstation with a GPU:
```json
{
"capabilities": [
{
"name": "complete",
"description": "LLM completion via local Ollama",
"input_schema": {
"properties": {
"prompt": { "type": "string" },
"model": { "type": "string", "default": "llama3" }
},
"required": ["prompt"]
},
"timeout_seconds": 120
},
{
"name": "embed",
"description": "Local embedding via Ollama",
"input_schema": {
"properties": {
"text": { "type": "string" },
"model": { "type": "string", "default": "nomic-embed-text" }
},
"required": ["text"]
},
"timeout_seconds": 30
}
]
}
```
`llm-bridge` can be configured (per-user setting or persona preference)
to route completions through the user's local Ollama instead of the
instance-level provider. BYOK taken to the extreme — bring your own
hardware.
### Personal Automation
A script that reacts to Armature events:
```python
# Personal deploy watcher — connects to Armature, subscribes to
# events in the #deploys conversation, runs kubectl locally
ws = connect(f'{KERNEL_URL}/api/v1/sidecar/subscribe',
token=TOKEN)
for event in ws:
if event['type'] == 'chat.message' and 'deploy' in event['content']:
result = subprocess.run(['kubectl', 'get', 'pods', '-n', 'prod'],
capture_output=True, text=True)
# Post result back to chat
requests.post(f'{KERNEL_URL}/api/v1/sidecar/lib/chat-core/send',
json={'conversation_id': event['conversation_id'],
'content': f'```\n{result.stdout}\n```'},
headers={'Authorization': f'Bearer {TOKEN}'})
```
Personal workflow glue. No instance-level deployment required.
---
## RBAC for User Sidecars
Three-layer permission model:
### Layer 1: Admin Controls WHO
New permission: `sidecar.connect_personal`.
Admin grants to specific users, groups, or roles. Most users don't
need it. Developers, power users, AI-heavy workflows — they get the
permission.
Default: not granted. Opt-in only.
### Layer 2: Admin Controls WHAT
**Capability allowlist for user sidecars.**
Admin setting: `SIDECAR_USER_ALLOWED_CAPABILITIES`
Default: `["fs.search", "fs.read", "clipboard.get"]` — safe read-only
operations.
To enable terminal access: admin adds `"terminal.exec"` to the
allowlist. This is a deliberate escalation requiring admin action.
A user sidecar that attempts to register a capability not on the
allowlist gets a clear error: "Capability 'terminal.exec' not permitted
for user sidecars. Contact your administrator."
### Layer 3: User Controls VISIBILITY
User sidecar capabilities are **private by default** — only the owning
user's requests invoke them.
Optional: user can share a specific capability with their team via the
settings UI. Shared capabilities appear in team members' tool sets
(with attribution: "via Jeff's local bridge").
User sidecars are never instance-wide. That's what instance sidecars
are for.
---
## Security Considerations
### Instance Sidecars
1. **Compromised binary** → can read/write own package data, call
exported functions. Mitigation: package-scoped access, audit logging,
token revocation.
2. **Token theft** → impersonate sidecar. Mitigation: package-scoped
tokens, mTLS eliminates theft, revocable.
3. **SSRF via sidecar.call** → Mitigation: endpoints validated at
registration, no runtime changes.
4. **Resource exhaustion** → Mitigation: per-sidecar concurrency and
rate limits, timeout enforcement, response size limits.
5. **Privilege escalation via lib.require** → Mitigation: calls
attributed to sidecar identity, target permission checks apply.
### User Sidecars (Additional Concerns)
6. **User deploys malicious sidecar** → registers capabilities that
exfiltrate data when called. Mitigation: capability allowlist (admin
controls what capabilities can be declared), private-by-default
(only the user's own requests trigger it — they're exfiltrating
from themselves).
7. **User shares malicious capability with team** → team members
invoke it, sidecar captures their request data. Mitigation: shared
capabilities are clearly attributed ("via Jeff's local bridge").
Admin can disable capability sharing entirely. Shared capabilities
run with the CALLING user's permissions, not the sidecar owner's.
8. **User sidecar as pivot** → user's laptop compromised, attacker uses
connected sidecar to access Armature data. Mitigation: user sidecar
can only access data the user could already access through the UI.
No privilege escalation. User can revoke own tokens. Admin kill
switch.
9. **Stale user sidecars** → user's laptop goes offline, sidecar
becomes stale. Mitigation: same sweep mechanism as instance
sidecars. 3× heartbeat interval = swept. User reconnects when
laptop comes back online.
### Principle of Least Privilege
- Instance sidecar: access own package data only, middleware-enforced.
- User sidecar: access own package data scoped to owning user only.
- Event emission scoped to package prefix.
- Capability allowlist for user sidecars (admin-controlled).
- Shared capabilities run with caller's permissions.
---
## Version Plan
### v0.14.0 — Sidecar Registry + Auth
**Goal:** Sidecars can register, authenticate, and heartbeat.
- Sidecar registry table with `scope` and `user_id` columns.
- Registration endpoint with JWT token validation.
- Heartbeat + sweep integration.
- Token generation (instance tokens via admin, user tokens via settings).
- `SIDECAR_AUTH_MODE` config: `token` or `mtls`.
- mTLS auth: `sidecar:` and `sidecar-user:` CN prefixes.
- Admin UI: sidecar health on packages page.
**Instance sidecars only in this version.** User sidecar plumbing
exists in the schema but user token generation and RBAC gating land
in v0.14.5.
### v0.14.1 — Capability Registration + Execution
**Goal:** Sidecars declare capabilities. Starlark extensions call them.
- Capability manifest in registration payload.
- `sidecar` Starlark module: `call()`, `available()`, `capabilities()`.
- Kernel proxy: `sidecar.call()` → `POST {endpoint}/exec/{cap}`.
- Input validation against declared schema.
- Timeout, response size, concurrent call limits.
- Error propagation.
- `sidecar.call` permission.
- Audit logging.
### v0.14.2 — Kernel API Access + Event Bus
**Goal:** Sidecars read/write their own data and participate in events.
- Sidecar middleware: authenticate, scope to own package routes.
- `/api/v1/sidecar/lib/{pkg}/{fn}` — cross-package function calls.
- `/api/v1/sidecar/emit` — event emission (package-scoped).
- `/api/v1/sidecar/subscribe` — WebSocket event subscription.
- Frontend `sw.sidecar` SDK module.
### v0.14.3 — Manifest Integration + Admin Polish
**Goal:** Sidecar packages are first-class in install/admin flow.
- `tier: "sidecar"` manifest support.
- `sidecar` manifest field (image, required, scope, env_hints,
download_url).
- Admin UI: deployment instructions from manifest.
- Admin UI: capability list with metrics.
- Admin UI: per-sidecar rate limit configuration.
- Token management UI.
- Monitoring metrics.
### v0.14.4 — Reference Sidecar + Instance Quality Gate
**Goal:** Ship one real instance sidecar. Prove the contract.
**`armature-embed`** — minimal Python sidecar running
sentence-transformers for local embedding generation. One capability:
`embed(text) → float[]`.
`vector-store` calls `sidecar.call('armature-embed', 'embed', {...})`
for local embedding without an external API. Completes the local-first
RAG story.
**Instance sidecar quality gate:**
- Registration, heartbeat, sweep lifecycle tested.
- Token and mTLS auth both functional.
- Capability execution with limits enforced.
- Package-scoped API access enforced.
- Event emission/subscription functional.
- `sw.sidecar` frontend module functional.
- Reference sidecar runs in Docker and K8s.
- Admin UI complete.
- Monitoring metrics in Grafana.
### v0.14.5 — User Sidecars
**Goal:** Users connect personal processes to Armature.
- `sidecar.connect_personal` permission (admin-granted).
- User token generation in Settings → Sidecars.
- Capability allowlist for user sidecars (admin setting:
`SIDECAR_USER_ALLOWED_CAPABILITIES`).
- User sidecar registration with `scope: "user"`, `user_id` binding.
- Execution routing: `sidecar.call()` resolves user sidecars based
on request context (calling user gets their own sidecar).
- Capability sharing: user can share specific capabilities with team.
Shared caps attributed ("via Jeff's local bridge").
- Separate resource limits for user sidecars (lower defaults).
- User settings UI: connected sidecars, token management, sidecar
instructions from package manifests.
- Admin UI: user sidecar monitoring, kill switch.
- Tool meta-tool integration: `llm-bridge` includes user's personal
sidecar capabilities in tool set assembly.
**Reference user sidecar: `user-bridge`** — thin agent binary with
`fs.search`, `fs.read`, `clipboard.get` capabilities. Published as a
package with `sidecar.scope: "user"`. Binary as a GitHub release.
**User sidecar quality gate:**
- RBAC: user without `sidecar.connect_personal` cannot register.
- Capability allowlist enforced.
- User A cannot invoke user B's sidecar.
- Shared capabilities run with caller's permissions.
- Admin can revoke any user sidecar.
- Tool meta-tool assembles per-user tool sets correctly.
- `user-bridge` reference sidecar runs on macOS, Linux, Windows.
---
## Schema Summary
### New tables
- `sidecar_registry` — sidecar_id, package_id, scope, user_id,
endpoint, heartbeat, capabilities (JSON), stats (JSON),
registered_at. (v0.14.0)
- `sidecar_tokens` — id, package_id, scope, user_id, token_hash,
created_by, expires_at, revoked_at. (v0.14.0)
---
## Configuration
| Setting | Default | Purpose |
|---------|---------|---------|
| `SIDECAR_AUTH_MODE` | `token` | `token` or `mtls` |
| `SIDECAR_HEARTBEAT_INTERVAL` | `10s` | Expected heartbeat frequency |
| `SIDECAR_STALE_THRESHOLD` | `30s` | 3× heartbeat = stale |
| `SIDECAR_MAX_RESPONSE_SIZE` | `10485760` (10MB) | Instance default |
| `SIDECAR_DEFAULT_CONCURRENCY` | `10` | Instance default |
| `SIDECAR_DEFAULT_RATE_LIMIT` | `100/min` | Instance default |
| `SIDECAR_USER_MAX_RESPONSE_SIZE` | `5242880` (5MB) | User default |
| `SIDECAR_USER_CONCURRENCY` | `3` | User default |
| `SIDECAR_USER_RATE_LIMIT` | `30/min` | User default |
| `SIDECAR_USER_ALLOWED_CAPABILITIES` | `["fs.search","fs.read","clipboard.get"]` | Admin-controlled allowlist |
---
## Sidecar Author Contract
### Minimum Viable Sidecar (Python)
```python
from flask import Flask, request, jsonify
import requests, os, threading, time
app = Flask(__name__)
KERNEL = os.environ['ARMATURE_URL']
TOKEN = os.environ['ARMATURE_TOKEN']
def register():
requests.post(f'{KERNEL}/api/v1/sidecar/register',
json={
'package_id': 'my-sidecar',
'endpoint': f'http://localhost:8080',
'capabilities': [{
'name': 'echo',
'description': 'Echo input back',
'input_schema': {
'type': 'object',
'properties': {'text': {'type': 'string'}},
'required': ['text']
},
'timeout_seconds': 10
}]
},
headers={'Authorization': f'Bearer {TOKEN}'})
def heartbeat():
while True:
time.sleep(10)
r = requests.post(f'{KERNEL}/api/v1/sidecar/heartbeat',
json={'stats': {}},
headers={'Authorization': f'Bearer {TOKEN}'})
if r.status_code == 404:
register()
@app.route('/api/v1/exec/echo', methods=['POST'])
def exec_echo():
return jsonify({
'output': {'text': request.json['input']['text']},
'error': None
})
@app.route('/api/v1/health')
def health():
return jsonify({'status': 'ok'})
register()
threading.Thread(target=heartbeat, daemon=True).start()
app.run(port=8080)
```
Four things: register, heartbeat, `/exec/`, `/health`. That's the
contract.
---
## Design Decisions
| Decision | Rationale |
|----------|-----------|
| Connect-inward | Eliminates K8s RBAC, service mesh, DNS discovery. Works on any deployment target. |
| Two auth modes | Token for simple. mTLS for production. Same internal identity. |
| Cluster registry pattern | Proven heartbeat/sweep. No new coordination mechanism. |
| HTTP/JSON, no gRPC | KISS. 4-endpoint contract. Every language has HTTP. |
| Kernel proxies all calls | Auth, rate limits, audit, timeouts at single control point. |
| Package-scoped access | Compromised sidecar can't read other packages. |
| User sidecars as v0.14.5 | Prove instance contract first (v0.14.0–v0.14.4), then extend to users. Same mechanism, new scope. |
| Three-layer user RBAC | Admin controls who (permission), what (capability allowlist), user controls visibility (private/shared). Defense in depth. |
| Capability allowlist for user sidecars | Admin decides what user processes can expose. `terminal.exec` is a deliberate escalation. |
| Private-by-default user capabilities | User sidecar can't affect other users unless explicitly shared. |
| Shared caps run with caller's perms | Prevents privilege escalation through shared capabilities. |
| Reference instance sidecar: embedding | Simpler than LLM. Proves contract. Completes local RAG story. |
| Reference user sidecar: file bridge | Most compelling user sidecar demo. "Find a file on my laptop and save it to notes" is visceral. |
| Lower resource limits for user sidecars | User sidecars are less trusted (running on personal devices). Lower defaults, admin can't be overridden by users. |
| Separate from polish (v0.15.x) | Sidecar is security-critical infrastructure. Polish is quality-of-life. Don't mix. |
---
## Future Considerations
- **Streaming responses.** LLM sidecar needs token streaming. SSE or
chunked transfer. Deferred — batch covers embedding, classification,
transcoding.
- **SDK libraries.** Thin wrappers in Python, Go, Node, Rust for
registration + heartbeat + parsing. Convenience, not requirement.
- **Auto-scaling.** Multiple instances of same sidecar, kernel
load-balances. No schema change needed — registry supports multiple
rows per package.
- **GPU detection.** Kernel capability for manifest negotiation.
- **Sidecar-to-sidecar.** Direct calls for performance pipelines.
Currently goes through kernel.
- **User sidecar desktop app.** Electron/Tauri tray app that wraps the
user-bridge agent with a GUI for capability management and connection
status. Post-1.0.

668
docs/DESIGN-storage-primitives.md
View File

@@ -1,668 +0,0 @@
# DESIGN — Storage Primitives
**Version:** v0.8.0
**Status:** Draft
**Author:** Jeff / Claude session 2026-04-02
---
## Problem
Extensions cannot access blob storage or managed disk paths. The existing
`ObjectStore` interface (PVC + S3 backends) is fully implemented but only
exposed to the admin status endpoint — no Starlark bridge exists. The `db`
module provides structured data storage via extension-scoped tables, but
there is no equivalent for binary files, no managed filesystem for tools
like `git`, and no mechanism for extensions to declare environment
requirements (pgvector, workspace root, S3) that the kernel validates at
install time.
This blocks every future capability that depends on file handling:
RAG/vector search, LLM bridge, code workspaces, file upload/sharing,
media processing, and document indexing.
---
## Non-Goals
- Replacing the `db` module. Extension-scoped tables (`ext_{pkg}_{table}`)
are the structured data primitive and remain unchanged.
- Building a KV store. Extensions that need key-value semantics declare a
table with `key TEXT, value TEXT` columns — the infrastructure exists.
- Multi-tenant file isolation beyond package scoping. Team/user-level
file ACLs are extension-layer concerns built on top of these primitives.
- Streaming / chunked upload through Starlark. Large file ingestion goes
through HTTP routes; the `files` module handles storage after receipt.
---
## Architecture
Three new primitives plus one extension to the existing `db` module.
### Primitive 1: `files` Starlark Module
Bridges the existing `ObjectStore` into the sandbox. Follows the same
pattern as `db_module.go`: a `Build*Module` factory, permission-gated,
package-scoped key namespacing.
**Permissions:** `files.read`, `files.write`
**Starlark API:**
```python
# Write a file. content is string (UTF-8) or bytes.
# metadata is an optional dict stored alongside (JSON-serialized).
files.put(name, content, content_type="application/octet-stream", metadata={})
# Read a file. Returns dict: {"content": <bytes>, "content_type": "...", "size": N, "metadata": {...}}
# Returns None if not found.
result = files.get(name)
# Read metadata only (no content transfer). Returns dict or None.
meta = files.meta(name)
# List files by prefix. Returns list of dicts: [{"name": "...", "size": N, "content_type": "..."}]
entries = files.list(prefix="", limit=100)
# Delete a file. Idempotent — no error if missing.
files.delete(name)
# Delete all files under a prefix. Use with caution.
files.delete_prefix(prefix)
# Check existence without reading.
exists = files.exists(name)
```
**Key namespacing:**
All keys are automatically prefixed with `ext/{packageID}/`. An extension
calling `files.put("models/v1.bin", data)` writes to the ObjectStore key
`ext/my-extension/models/v1.bin`. Extensions cannot escape their namespace.
Name validation rejects `..`, absolute paths, and control characters —
same sanitization rules as `physicalTable()` in `db_module.go`.
**Implementation notes:**
- New file: `sandbox/files_module.go`
- `FilesModuleConfig` struct mirrors `DBModuleConfig`:
```go
type FilesModuleConfig struct {
PackageID string
CanWrite bool
Store storage.ObjectStore
}
```
- Metadata is stored as a companion JSON object at key
`ext/{packageID}/_meta/{name}`. This avoids schema changes — the
ObjectStore interface is unchanged. The `files` module manages the
companion transparently.
- Size limit per `files.put()` call: 50MB (configurable via
`EXT_FILES_MAX_SIZE`). Enforced in the builtin before calling
`Store.Put()`.
- `files.get()` returns content as `starlark.Bytes` for binary safety.
Starlark's `Bytes` type was added in go.starlark.net v0.0.0-20240725214946
and handles non-UTF-8 content correctly.
- If `ObjectStore` is nil (storage not configured), the module is not
injected — same pattern as `db` module when `r.db == nil`.
**Runner wiring:**
```go
// In buildModulesWithLibCtx, add cases:
case models.ExtPermFilesRead:
if filesLevel < 1 { filesLevel = 1 }
case models.ExtPermFilesWrite:
filesLevel = 2
// After permission loop:
if filesLevel > 0 && r.objectStore != nil {
modules["files"] = BuildFilesModule(ctx, FilesModuleConfig{
PackageID: packageID,
CanWrite: filesLevel == 2,
Store: r.objectStore,
})
}
```
Runner gains `SetObjectStore(s storage.ObjectStore)` setter, called from
`main.go` after `storage.Init()`.
---
### Primitive 2: `workspace` Starlark Module
Managed disk directories for extensions that need a real filesystem —
git repos, compilers, ffmpeg, pandoc, code analysis tools. These tools
cannot operate through put/get blob semantics; they need paths.
**Permission:** `workspace.manage`
**Starlark API:**
```python
# Create a named workspace directory. Returns the absolute path.
# Idempotent — returns existing path if already created.
path = workspace.create(name)
# Get the path for an existing workspace. Returns string or None.
path = workspace.path(name)
# List workspace names for this extension.
names = workspace.list()
# Delete a workspace and all its contents.
workspace.delete(name)
# Disk usage in bytes for a workspace.
size = workspace.usage(name)
```
**Directory layout:**
```
{WORKSPACE_ROOT}/
{packageID}/
{name}/
... (extension-managed contents)
```
`WORKSPACE_ROOT` defaults to `/data/workspaces` (configurable via
`WORKSPACE_ROOT` env var). The kernel creates the package subdirectory
on first `workspace.create()`. Extensions own everything below their
directory — the kernel does not inspect contents.
**Implementation notes:**
- New file: `sandbox/workspace_module.go`
- `WorkspaceModuleConfig`:
```go
type WorkspaceModuleConfig struct {
PackageID string
WorkspaceRoot string
}
```
- Name validation: same rules as table names (`^[a-z][a-z0-9_]{0,62}$`).
No path separators, no `..`, no spaces.
- `workspace.create()` calls `os.MkdirAll` for the scoped path.
- `workspace.delete()` calls `os.RemoveAll` — destructive by design.
Extensions must handle confirmation in their own UX.
- `workspace.usage()` walks the directory tree and sums file sizes.
Bounded by a 10-second context timeout to prevent hangs on huge trees.
- **Quota enforcement** (optional): `WORKSPACE_QUOTA_MB` env var. When
set, `workspace.create()` checks cumulative usage for the package
before creating. Returns error if quota exceeded. Default: unlimited.
- If `WORKSPACE_ROOT` is empty or not writable, the module is not
injected. Extensions that declare `workspace.manage` without the
root configured will have their permission granted but the module
absent — same degradation pattern as `db` when no DB is available.
**Runner wiring:**
```go
case models.ExtPermWorkspaceManage:
if r.workspaceRoot != "" {
modules["workspace"] = BuildWorkspaceModule(ctx, WorkspaceModuleConfig{
PackageID: packageID,
WorkspaceRoot: r.workspaceRoot,
})
}
```
Runner gains `SetWorkspaceRoot(path string)` setter.
**Security considerations:**
- The returned path is an absolute filesystem path. Extensions can pass
this to `http` module calls (e.g., POST a file to an API) or use it
in `db` records as a reference. They cannot execute arbitrary binaries —
the Starlark sandbox has no `os.exec`. Execution requires a sidecar
tier package or an `api_route` handler that shells out server-side.
- For sidecar-tier packages that _can_ execute binaries, the workspace
path is the designated scratch space. The kernel ensures the path is
within the scoped directory via `filepath.Clean` + prefix check.
---
### Primitive 3: Capability Negotiation
Extensions declare environment requirements in their manifest. The kernel
validates these at install time and reports failures with actionable
messages. This is what makes progressive enhancement strategies (e.g.,
three-tier vector search) work.
**Manifest field:**
```json
{
"id": "vector-store",
"capabilities": {
"required": ["files.read", "files.write"],
"optional": ["pgvector"]
},
"permissions": ["db.write", "files.write"],
"db_tables": {
"embeddings": {
"columns": {
"source_id": "text",
"chunk_text": "text",
"embedding": "vector(384)"
},
"indexes": [["source_id"]]
}
}
}
```
`capabilities.required` — install fails if any are missing. Admin gets
a clear error: "Package 'vector-store' requires capability 'pgvector'
which is not available. Run `CREATE EXTENSION vector;` in your PostgreSQL
database to enable it."
`capabilities.optional` — install succeeds regardless. The capability
state is queryable at runtime so extensions can degrade gracefully.
**Runtime query (Starlark):**
Settings module is the natural home since it's always available:
```python
# Returns True/False for a capability name.
has_pgvector = settings.has_capability("pgvector")
```
**Kernel capability registry:**
A simple function in the `handlers` package that probes the environment:
```go
func DetectCapabilities(db *sql.DB, isPostgres bool, workspaceRoot string, objStore storage.ObjectStore) map[string]bool {
caps := make(map[string]bool)
// pgvector: check pg_extension
if isPostgres && db != nil {
var exists bool
row := db.QueryRow("SELECT EXISTS(SELECT 1 FROM pg_extension WHERE extname='vector')")
if row.Scan(&exists) == nil && exists {
caps["pgvector"] = true
}
}
// workspace: check root is writable
if workspaceRoot != "" && storage.IsPathWritable(workspaceRoot) {
caps["workspace"] = true
}
// object storage: check configured and healthy
if objStore != nil {
if err := objStore.Healthy(context.Background()); err == nil {
caps["object_storage"] = true
}
}
// s3: specific backend check
if objStore != nil && objStore.Backend() == "s3" {
caps["s3"] = true
}
// postgres: dialect check
if isPostgres {
caps["postgres"] = true
}
return caps
}
```
Called once at startup, stored on the `Runner` (or a shared config
struct). Re-probed on admin request for the capabilities endpoint.
**Install-time validation:**
In the package install handler (`handlers/extensions.go`), after
`ParseDBTables` and before `SyncManifestPermissions`:
```go
if caps, ok := ParseCapabilities(manifestMap); ok {
missing := CheckRequiredCapabilities(caps.Required, detectedCaps)
if len(missing) > 0 {
// Roll back: delete the just-created package row
h.stores.Packages.Delete(c.Request.Context(), pkg.ID)
c.JSON(422, gin.H{
"error": "missing required capabilities",
"missing": missing,
"help": capabilityHelpText(missing),
})
return
}
}
```
**Admin API:**
```
GET /api/v1/admin/capabilities
→ {"pgvector": true, "workspace": true, "object_storage": true, "s3": false, "postgres": true}
```
Displayed in the Admin UI alongside storage status. Gives operators
visibility into what their deployment supports.
---
### Extension to `db` Module: Vector Column Type
The existing `mapColType()` in `ext_db_schema.go` gains a `vector` type
with progressive enhancement across backends.
**Manifest declaration:**
```json
"db_tables": {
"embeddings": {
"columns": {
"embedding": "vector(384)"
}
}
}
```
**Column type mapping:**
| Manifest type | PG + pgvector | PG without pgvector | SQLite |
|------------------|-------------------------|---------------------|--------------|
| `vector(N)` | `vector(N)` | `JSONB` | `TEXT` |
Implementation in `mapColType`:
```go
case "vector":
// vector or vector(384) — extract dimension if present
dim := extractVectorDim(typStr) // returns "384" or ""
if isPostgres && hasPgVector {
if dim != "" {
return fmt.Sprintf("vector(%s)", dim)
}
return "vector"
}
if isPostgres {
return "JSONB" // store as JSON array, brute-force search
}
return "TEXT" // SQLite: JSON array as text
```
`hasPgVector` is passed through a new field on a `SchemaConfig` struct
(or detected inline — the capability registry result is available at
table creation time).
**New `db` module function — `db.query_similar()`:**
```python
# Find rows with embeddings closest to the query vector.
# Returns list of row dicts with _distance appended.
results = db.query_similar(
table="embeddings",
column="embedding",
vector=[0.1, 0.2, ...], # query vector (list of floats)
limit=10,
filters={"source_id": "doc-123"} # optional WHERE clause
)
```
**Backend dispatch:**
- **PG + pgvector:** Uses `ORDER BY embedding <=> $1 LIMIT $2` with
native vector distance operator.
- **PG without pgvector / SQLite:** Loads candidate rows (respecting
`filters`), deserializes JSON arrays, computes cosine similarity in
Go, sorts, returns top-N. This is the "it works but slowly" fallback —
acceptable for small corpora (<10k rows), documented as such.
Implementation: new function `dbQuerySimilar()` in `db_module.go`,
gated on `db.read` permission (read-only operation). The function
checks `cfg.HasPgVector` to choose the fast or slow path.
`DBModuleConfig` gains:
```go
type DBModuleConfig struct {
PackageID string
CanWrite bool
DB *sql.DB
IsPostgres bool
HasPgVector bool // NEW: enables native vector ops
}
```
Wired from the capability registry at module build time.
---
## New Permission Constants
```go
// In models/models_extension_perm.go:
const (
ExtPermFilesRead = "files.read"
ExtPermFilesWrite = "files.write"
ExtPermWorkspaceManage = "workspace.manage"
)
```
Added to `ValidExtensionPermissions` map.
---
## Schema Changes
**Migration 015 — none required for kernel tables.**
The `files` module uses the existing `ObjectStore` interface — no new
kernel tables. Metadata companions are stored as ObjectStore objects.
The `workspace` module uses the filesystem — no tables.
Capabilities are detected at runtime — no tables.
The `vector` column type is handled by extension DDL generation in
`ext_db_schema.go` — no kernel migration.
The new permission constants are code-only changes.
---
## New Files
| File | Purpose |
|------|---------|
| `sandbox/files_module.go` | `files` Starlark module |
| `sandbox/files_module_test.go` | Tests using mock ObjectStore |
| `sandbox/workspace_module.go` | `workspace` Starlark module |
| `sandbox/workspace_module_test.go` | Tests using temp directory |
| `handlers/capabilities.go` | `DetectCapabilities()`, `ParseCapabilities()`, admin endpoint |
| `handlers/capabilities_test.go` | Tests for capability detection and validation |
## Modified Files
| File | Change |
|------|--------|
| `models/models_extension_perm.go` | Add `files.read`, `files.write`, `workspace.manage` |
| `sandbox/runner.go` | Add `SetObjectStore()`, `SetWorkspaceRoot()`, wire new modules |
| `sandbox/db_module.go` | Add `dbQuerySimilar()`, `HasPgVector` field |
| `handlers/ext_db_schema.go` | Extend `mapColType()` for `vector(N)`, pass `hasPgVector` |
| `handlers/extensions.go` | Add capability validation in install handler |
| `sandbox/settings_module.go` | Add `settings.has_capability()` |
| `server/main.go` | Call `DetectCapabilities()`, wire to runner |
| `docs/PACKAGE-FORMAT.md` | Document `capabilities` manifest field |
| `docs/EXTENSION-GUIDE.md` | Document new modules and vector column type |
---
## Changeset Plan
**CS1 — `files` module + permissions**
New files: `files_module.go`, `files_module_test.go`.
Modified: `models_extension_perm.go`, `runner.go`, `main.go`.
Scope: ObjectStore bridge, permission constants, runner wiring.
CI-green independently — no schema changes, no existing behavior affected.
**CS2 — `workspace` module**
New files: `workspace_module.go`, `workspace_module_test.go`.
Modified: `models_extension_perm.go`, `runner.go`, `main.go`.
Scope: Managed disk directories, quota enforcement.
CI-green independently.
**CS3 — Capability negotiation**
New files: `capabilities.go`, `capabilities_test.go`.
Modified: `extensions.go` (install validation), `settings_module.go`
(`has_capability`), `main.go`.
Scope: Detection, install-time validation, runtime query, admin endpoint.
CI-green independently.
**CS4 — Vector column type + `db.query_similar()`**
Modified: `ext_db_schema.go`, `db_module.go`, `db_module_test.go`.
Scope: Column type mapping, similarity query with dual-path dispatch.
Depends on CS3 (needs `HasPgVector` from capability registry).
---
## Configuration Summary
| Env Var | Default | Purpose |
|---------|---------|---------|
| `WORKSPACE_ROOT` | `/data/workspaces` | Root directory for workspace module |
| `WORKSPACE_QUOTA_MB` | `0` (unlimited) | Per-extension disk quota |
| `EXT_FILES_MAX_SIZE` | `52428800` (50MB) | Max single file size via `files.put()` |
---
## Example: Vector Store Extension
A `vector-store` library extension consuming all four primitives:
**manifest.json:**
```json
{
"id": "vector-store",
"title": "Vector Store",
"type": "library",
"tier": "starlark",
"version": "0.1.0",
"permissions": ["db.write", "files.read", "connections.read"],
"capabilities": {
"required": [],
"optional": ["pgvector"]
},
"db_tables": {
"documents": {
"columns": {
"source": "text",
"chunk_text": "text",
"page": "int",
"metadata": "text"
},
"indexes": [["source"]]
},
"embeddings": {
"columns": {
"document_id": "text",
"embedding": "vector(384)",
"chunk_index": "int"
},
"indexes": [["document_id"]]
}
},
"exports": ["ingest", "search", "search_clustered"]
}
```
**script.star:**
```python
def ingest(source_name, chunks):
"""Store document chunks and generate embeddings."""
for i, chunk in enumerate(chunks):
row = db.insert("documents", {
"source": source_name,
"chunk_text": chunk["text"],
"page": chunk.get("page", 0),
"metadata": json.encode(chunk.get("metadata", {})),
})
# Embedding generation delegated to caller (llm-bridge)
# Caller passes pre-computed vectors
if "embedding" in chunk:
db.insert("embeddings", {
"document_id": row["id"],
"embedding": json.encode(chunk["embedding"]),
"chunk_index": i,
})
def search(query_vector, limit=10, source_filter=None):
"""Semantic similarity search — dispatches to native or brute-force."""
filters = {}
if source_filter:
filters["source"] = source_filter
# query_similar handles pgvector vs fallback transparently
results = db.query_similar(
table="embeddings",
column="embedding",
vector=query_vector,
limit=limit,
)
# Hydrate with document text
enriched = []
for r in results:
docs = db.query("documents", filters={"id": r["document_id"]}, limit=1)
if docs:
enriched.append({
"text": docs[0]["chunk_text"],
"source": docs[0]["source"],
"page": docs[0]["page"],
"distance": r["_distance"],
})
return enriched
def search_clustered(embeddings_list, k):
"""K-means clustering for query-free thematic selection.
Returns k representative chunks. Clustering runs in the
extension — the kernel provides the data, not the algorithm."""
# This would be implemented by a consumer extension with
# http access to call a clustering API, or by a sidecar
# that runs sklearn. The vector-store library provides
# the data access pattern; clustering logic lives elsewhere.
pass
```
---
## Future Considerations
- **Content-addressed deduplication.** If multiple extensions store the
same PDF, the ObjectStore holds duplicate bytes. A SHA256-keyed blob
layer with refcounting would deduplicate transparently. Deferred —
adds complexity without clear need pre-1.0.
- **Extension-to-extension file sharing.** The current design isolates
file namespaces per extension. A `files.grant(name, target_pkg_id)`
primitive could enable controlled sharing. Deferred — composition
through API routes is sufficient initially.
- **Streaming upload for large files.** Starlark `files.put()` buffers
in memory. For files >50MB, extensions should use HTTP `api_routes`
with Go handlers that stream directly to ObjectStore. The `files`
module is for extension-internal storage, not user-facing upload.
- **Workspace snapshots.** `workspace.snapshot(name)` → creates a
tarball in the `files` store. Useful for backup/reproducibility.
Deferred — extensions can implement this themselves.
- **Rate limiting / quota on `db.query_similar()` fallback.** The
brute-force path loads all candidate rows into Go memory. For large
tables this is dangerous. A row-count guard (e.g., refuse if >50k
candidates) with a clear error message pointing to pgvector is the
right safety valve.

407
docs/DESIGN-surface-runners.md
View File

@@ -1,407 +0,0 @@
# DESIGN: Surface Runners — v0.7.1–v0.7.3
## Status: v0.7.1 Shipped (framework + migrations), v0.7.2–v0.7.3 Proposed
## Problem
Armature has two test tiers today:
1. **Go unit tests** — test store methods, handlers, middleware, sandbox.
Run in CI on every push. Coverage is good for kernel internals.
2. **ICD/SDK test runners** — browser-based test suites that validate API
endpoint contracts and SDK domain methods. Run manually by navigating
to `/s/icd-test-runner` or `/s/sdk-test-runner`.
Neither tier catches the class of bugs discovered during manual testing:
- **Cross-surface state:** Notification bell state not syncing, announcement
dismiss not persisting across surface navigations.
- **Package integration:** Workflow demo shows "not installed" because its
API call fails silently. The API works (unit tests pass); the surface's
integration with the API is broken.
- **Test side-effects:** ICD security tests install `evil.surface` with no
cleanup — it leaks into the menu.
- **Surface lifecycle:** Surfaces fail to load, mount into wrong containers,
miss SDK boot, or render without shell chrome.
These are integration bugs — they live at the boundary between kernel and
package, between surface and surface, between API and UI. They require a
new test tier.
## Solution Overview
Three deliverables across three versions:
| Version | Deliverable | What it catches |
|---------|-------------|-----------------|
| v0.7.1 | Runner framework (`sw.testing`) | Framework bugs, standardizes existing runners |
| v0.7.2 | Package runners + CI gate | Package integration bugs, regressions |
| v0.7.3 | Headless E2E (Playwright) | DOM rendering bugs, navigation flows, visual regressions |
## v0.7.1 — Runner Framework
### `sw.testing` SDK Module
New kernel SDK module at `src/js/sw/sdk/testing.js`. Provides structured
test authoring, lifecycle hooks, cleanup tracking, and machine-readable
results.
```js
// Extension runner registers suites during load
sw.testing.suite('notes-crud', async (s) => {
let folderId, noteId;
s.beforeAll(async () => {
// Setup: create a test folder
const r = await sw.api.post('/api/v1/ext/notes/folders', {
name: 'test-' + Date.now()
});
folderId = r.id;
s.track('folder', folderId); // auto-cleanup
});
s.test('create note', async (t) => {
const r = await sw.api.post('/api/v1/ext/notes/notes', {
title: 'Test Note', folder_id: folderId, content: '# Hello'
});
t.assert.ok(r.id, 'note has ID');
t.assert.eq(r.title, 'Test Note');
noteId = r.id;
t.track('note', noteId); // auto-cleanup
});
s.test('renderers fire', async (t) => {
// Test that mermaid block in note content triggers renderer
await sw.api.patch('/api/v1/ext/notes/notes/' + noteId, {
content: '```mermaid\ngraph LR; A-->B\n```'
});
// Renderer integration tested via DOM assertion
// (only meaningful in headless E2E — marked as browser-only)
t.browserOnly(() => {
const el = document.querySelector('.mermaid svg');
t.assert.ok(el, 'mermaid rendered to SVG');
});
});
s.afterAll(async () => {
// s.track() resources auto-cleaned here
// Manual cleanup for anything not tracked
});
});
```
### Core API
```js
sw.testing.suite(name, fn) // Register a test suite
sw.testing.run(name?) // Run one suite or all
sw.testing.results() // Get structured results (JSON)
sw.testing.on('complete', fn) // Event when run finishes
// Inside suite:
s.test(name, fn) // Register a test
s.beforeAll(fn) // Runs once before all tests
s.afterAll(fn) // Runs once after all tests (always, even on failure)
s.beforeEach(fn) // Runs before each test
s.afterEach(fn) // Runs after each test
s.track(type, id) // Register resource for auto-cleanup
s.skip(reason) // Skip entire suite
// Inside test:
t.assert.ok(val, msg) // Truthy
t.assert.eq(a, b, msg) // Deep equality
t.assert.neq(a, b, msg) // Not equal
t.assert.gt(a, b, msg) // Greater than
t.assert.match(str, re, msg) // Regex match
t.assert.throws(fn, msg) // Expects throw
t.assert.status(resp, code, msg) // HTTP status check
t.track(type, id) // Register resource for auto-cleanup
t.warn(msg) // Emit warning (non-fatal)
t.browserOnly(fn) // Only runs in headless E2E, skipped in API-only mode
t.skip(reason) // Skip this test
```
### `requires` Declarations
Runner packages declare dependencies in their manifest:
```json
{
"id": "chat-runner",
"type": "test-runner",
"title": "Chat Runner",
"requires": ["chat", "chat-core"],
"version": "0.1.0"
}
```
On load, the framework calls `GET /api/v1/surfaces` (or equivalent) to
check which packages are installed. If any `requires` entry is missing:
- Suite is marked `skipped` with reason: `"Missing required package: chat-core"`
- No tests execute — clean skip, not a failure
- The runner registry surface shows the skip reason prominently
This directly solves the "workflow demo shows not-installed" pattern:
the runner *knows* what should be installed and reports clearly when it isn't.
### Auto-Cleanup
The `track(type, id)` method registers resources for deletion in `afterAll`.
Supported resource types and their cleanup endpoints:
| Type | Cleanup Action |
|------|---------------|
| `channel` | `DELETE /api/v1/channels/:id` |
| `note` | `DELETE /api/v1/ext/notes/notes/:id` |
| `folder` | `DELETE /api/v1/ext/notes/folders/:id` |
| `workflow` | `DELETE /api/v1/workflows/:id` |
| `schedule` | `DELETE /api/v1/schedules/:id` |
| `package` | `DELETE /api/v1/admin/packages/:id` |
| `user` | `DELETE /api/v1/admin/users/:id` |
| `team` | `DELETE /api/v1/admin/teams/:id` |
Cleanup runs in reverse order (LIFO) in `afterAll`, regardless of
test pass/fail. Cleanup failures are reported as warnings, not failures.
The framework never swallows cleanup errors silently.
### Result Structure
```json
{
"runner": "notes-runner",
"timestamp": "2026-04-01T12:00:00Z",
"duration_ms": 1234,
"summary": { "total": 5, "passed": 4, "failed": 0, "warned": 1, "skipped": 0 },
"suites": [
{
"name": "notes-crud",
"status": "passed",
"duration_ms": 890,
"tests": [
{
"name": "create note",
"status": "passed",
"duration_ms": 120,
"warnings": [],
"cleanup": { "tracked": 1, "cleaned": 1, "failed": 0 }
}
]
}
],
"requires": { "met": ["notes"], "missing": [] }
}
```
### Warning Tier
Three result statuses:
- **`passed`** — assertions all passed, cleanup succeeded
- **`failed`** — at least one assertion failed
- **`warned`** — assertions passed but something non-fatal happened:
- API returned unexpected shape (extra/missing fields) but test doesn't depend on the exact field
- Cleanup failed for a tracked resource
- Timing exceeded a soft threshold
- `t.warn(msg)` called explicitly
Warnings are **never silent.** They appear in the UI and structured results.
The difference from the current `catch (e) { /* ignore */ }` pattern is
that warnings are *visible* — a human or CI system can decide whether
to investigate.
### ICD/SDK Runner Migration
The existing runners use a hand-rolled framework (`T.test()`, `T.assert()`,
`T.authFetch()`). Migration preserves all test logic:
| Current | New |
|---------|-----|
| `T.test(tier, group, name, fn)` | `s.test(name, fn)` inside `sw.testing.suite(tier + '/' + group, fn)` |
| `T.assert(cond, msg)` | `t.assert.ok(cond, msg)` |
| `T.authFetch(token, method, path, body)` | Kept as utility — not an assertion primitive |
| `T.apiPost(...)` | Kept as utility |
| Result rendering (`ui.js`) | Delegated to runner registry surface |
| No cleanup hooks | `s.track()` + `s.afterAll()` |
The ICD and SDK runners become packages with `"type": "test-runner"` in
their manifests. Their existing surfaces (`/s/icd-test-runner`,
`/s/sdk-test-runner`) are replaced by the unified runner registry at
`/s/test-runners`.
## v0.7.2 — Package Runners
### Runner Inventory
| Runner | `requires` | Key Assertions |
|--------|-----------|---------------|
| `notes-runner` | `["notes"]` | CRUD, folders, tags, backlinks, search, markdown rendering, SDK integration |
| `chat-runner` | `["chat", "chat-core"]` | Channel CRUD, messaging, participant display, renderer blocks in messages |
| `schedules-runner` | `["schedules"]` | Schedule CRUD, cron expression, toggle, Starlark exec |
| `workflow-runner` | `["content-approval"]` | Install detection, stage progression, form submission, signoff |
| `renderer-runner` | `["mermaid-renderer"]` | `sw.renderers.register` contract, post-render hooks, block rendering |
### Runner Result API
New kernel endpoints (no package required — kernel-provided):
```
POST /api/v1/test-runners/run → Run all installed runners
POST /api/v1/test-runners/run/:id → Run specific runner
GET /api/v1/test-runners/results → Last run results (JSON)
GET /api/v1/test-runners/results/:id → Last run results for specific runner
```
These endpoints enable CI to trigger and consume runner results via
`curl` without browser automation. The v0.7.3 Playwright harness is
additive — not required for CI gating.
**Auth:** Admin-only. Runners create/delete resources — they must run
with elevated permissions.
### CI Integration
New stage in `.gitea/workflows/ci.yaml`:
```yaml
test-runners:
needs: [unit-tests]
steps:
- name: Boot server
run: |
docker compose up -d
./ci/wait-for-healthy.sh
- name: Run surface runners
run: |
RESULT=$(curl -s -X POST http://localhost:8080/api/v1/test-runners/run \
-H "Authorization: Bearer $ADMIN_TOKEN")
FAILED=$(echo "$RESULT" | jq '.summary.failed')
if [ "$FAILED" != "0" ]; then
echo "$RESULT" | jq '.suites[] | select(.status == "failed")'
exit 1
fi
```
Runs in both PG and SQLite pipelines. Server boots with `BUNDLED_PACKAGES=*`
so all packages and their runners are installed.
## v0.7.3 — Headless E2E
### Playwright Harness
`ci/e2e-surface-test.sh`:
1. `docker compose up -d` (server + DB)
2. `npx playwright install chromium` (CI caches this)
3. Run `ci/e2e-surfaces.spec.ts`
4. Collect screenshots on failure
5. `docker compose down`
### Surface Navigation Smoke Test
```ts
test('all surfaces reachable', async ({ page }) => {
// Login
await page.goto('/');
await page.fill('#username', 'admin');
await page.fill('#password', 'admin');
await page.click('button[type="submit"]');
// Navigate through every installed surface
const surfaces = ['notes', 'chat', 'admin', 'settings', 'docs'];
for (const s of surfaces) {
await page.goto(`/s/${s}`);
// Assert: page loaded, no uncaught JS errors
await expect(page.locator('.sw-topbar')).toBeVisible();
// Assert: home link works
await page.click('.sw-topbar__home');
await expect(page).toHaveURL('/');
}
});
```
This is the automated version of "hello dashboard has no way out" — if
any surface fails to render a topbar or its home link doesn't work, CI
catches it.
### Screenshot on Failure
```ts
test.afterEach(async ({ page }, testInfo) => {
if (testInfo.status !== 'passed') {
await page.screenshot({
path: `ci/artifacts/failure-${testInfo.title}.png`,
fullPage: true
});
const logs = await page.evaluate(() =>
(window.__consoleErrors || []).join('\n')
);
fs.writeFileSync(
`ci/artifacts/console-${testInfo.title}.log`, logs
);
}
});
```
Artifacts saved to CI workspace. On failure, the developer gets a
screenshot + console log dump without needing to reproduce locally.
### Visual Regression (Optional)
Not a CI gate in v0.7.3 — produces a diff report for human review:
```ts
test('visual baseline - notes', async ({ page }) => {
await page.goto('/s/notes');
await expect(page.locator('.sw-topbar')).toBeVisible();
await expect(page).toHaveScreenshot('notes.png', {
maxDiffPixelRatio: 0.01
});
});
```
Playwright stores baseline screenshots in `ci/visual-baselines/`.
`toHaveScreenshot` compares against baseline and produces a diff image
on mismatch. Foundation for future visual regression gating.
## Sequencing
```
v0.7.0 Shell Contract ← prerequisite: surfaces need topbar before
│ runners can assert on it
↓
v0.7.1 Runner Framework ← standardize test authoring
│
↓
v0.7.2 Package Runners + CI ← write the actual tests, wire into CI
│
↓
v0.7.3 Headless E2E ← automate browser-based runner execution
```
Each version is independently shippable. v0.7.2's API-based CI gate
works without v0.7.3's Playwright. v0.7.3 adds coverage for DOM-level
bugs that API-only runners can't catch.
## Open Questions
1. **Runner package type.** Should `"type": "test-runner"` be a new
manifest type, or should runners be `"type": "surface"` with a
`"tags": ["test-runner"]` convention? New type is cleaner but requires
a `ValidateManifest()` update.
2. **Runner discovery.** The registry surface needs to find all installed
runners. Options: (a) scan installed packages for `type: "test-runner"`,
(b) runners register themselves via `sw.testing.register()` during SDK
boot. Option (a) is declarative and doesn't require runner JS to load
before discovery.
3. **Parallel vs sequential.** Should runners execute in parallel?
Probably not initially — shared DB state means test isolation is hard.
Sequential is safer. Parallel can be a future optimization.
4. **SQLite limitations.** Some runners (cluster, multi-node) are
PG-only. The `requires` mechanism should support
`"requires_db": "postgres"` for these cases, or runners should
self-skip when `sw.config.db_driver === 'sqlite'`.

97
docs/DESIGN-vector-column.md
View File

@@ -1,97 +0,0 @@
# Design: Vector Column Type (v0.8.3)
## Problem
Extensions building semantic search, RAG, or recommendation features need to
store and query high-dimensional vectors (embeddings). Without kernel-level
support, each extension would need to reinvent storage, serialization, and
similarity search — duplicating effort and missing the pgvector optimization
path.
## Design
### Three-tier progressive enhancement
| Backend | Column DDL | Storage | Search |
|---------|-----------|---------|--------|
| Postgres + pgvector | `vector(N)` + HNSW index | Native vector type | `<=>` operator (index-backed) |
| Postgres (no pgvector) | `JSONB` | JSON array | Go-side cosine distance |
| SQLite | `TEXT` | JSON string | Go-side cosine distance |
Extensions declare `"vector(N)"` in their manifest `db_tables` block.
The kernel maps this to the appropriate SQL type at install time based on
detected capabilities.
### Manifest example
```json
{
"db_tables": {
"documents": {
"columns": {
"title": "text",
"embedding": "vector(384)"
}
}
},
"capabilities": {
"optional": ["pgvector"]
}
}
```
### API surface
```python
# Insert (vector as list)
db.insert("documents", {"title": "hello", "embedding": [0.1, 0.2, ...]})
# Similarity search
rows = db.query_similar(
"documents", "embedding",
vector=[0.1, 0.2, ...],
limit=10,
filters={"active": True},
metric="cosine"
)
# → [{..., "_distance": 0.023}, ...]
```
### Dispatch paths
**pgvector path** — SQL-side computation with index:
```sql
SELECT *, (embedding <=> $1::vector) AS _distance
FROM ext_pkg_documents
WHERE ... ORDER BY embedding <=> $1::vector LIMIT $2
```
**Fallback path** — Go-side computation:
1. `SELECT * FROM table WHERE filters LIMIT 1000`
2. Parse each row's vector column from JSON
3. Compute cosine distance in Go
4. Sort by distance, return top N with `_distance` injected
### Dimension validation
- Manifest: 1 ≤ N ≤ 4096 (validated by `parseVectorDim`)
- Insert-time: no dimension validation (store whatever list is given)
- Query-time: dimension mismatches produce distance = 1.0 (treated as unrelated)
### Performance characteristics
| Path | 1K rows | 10K rows | 100K rows |
|------|---------|----------|-----------|
| pgvector (HNSW) | <1ms | <5ms | <10ms |
| Fallback (Go) | <10ms | ~100ms | Not recommended |
Fallback caps at 1000 rows fetched. Extensions needing large-scale similarity
search should declare `capabilities.optional: ["pgvector"]` and degrade
gracefully.
## Limitations
- Only cosine distance metric (v0.8.3). L2 / inner product can be added later.
- No dimension validation at insert time.
- Fallback path fetches at most 1000 rows — not suitable for large datasets.
- HNSW index is only created for pgvector backends.

126
docs/DESIGN-workflow-redesign.md
View File

@@ -1,126 +0,0 @@
# DESIGN — Workflow System Redesign (0.9.x)
**Version:** v0.9.0
**Status:** Draft
**Author:** Jeff / Claude session 2026-04-03
---
## Problem
The workflow system spans ~7,600 lines across 25+ files and was built
incrementally from v0.3.x through v0.7.10. It works, but several
concepts are redundant, primitives are buried, and the Starlark module
is read-only. Before shipping reference extensions (v0.10.x) that
build on workflows, the system needs cleanup and promotion of reusable
primitives.
## Audit Summary
Full audit in `/config/Downloads/AUDIT-workflow-0.9.md`. Classification:
### KEEP — Core primitives that survive
| Component | File(s) | Lines | Rationale |
|-----------|---------|-------|-----------|
| Engine core | `workflow/engine.go` | ~400 | Stateless state machine — Start, Advance, Cancel, version-pinned execution |
| Conditional routing | `workflow/routing.go` | ~500 | Pure functions, 8 operators, well-tested (497 lines of tests) |
| Automated processing | `workflow/automated.go` | ~300 | Starlark hook execution, cycle guard (max 10) |
| Scanner | `workflow/scanner.go` | ~200 | Background SLA + staleness checks, 5-minute interval |
| Signoff system | engine.go + handlers | ~400 | Multi-party approve/reject with quorum, role-gated |
| Assignment queue | handlers | ~300 | Claim/unclaim/complete/cancel lifecycle |
| Version snapshots | store + models | ~200 | Immutable snapshots, version-pinned instances |
| Typed form system | `models/workflow.go` | ~300 | 8 field types, fieldsets, conditional visibility |
| Store interface | `store/workflow_iface.go` | 61 methods | Complete lifecycle coverage |
### OBE — Superseded by new design
| Concept | Superseded by |
|---------|---------------|
| Per-stage `audience` field | Multi-page packages with mixed access declarations |
| `entry_mode` (public_link/team_only) | Package-level `scope: adoptable` + per-page access |
| `stage_mode` proliferation (4 values) | Collapse to 3: form / delegated / automated |
| `stage_type` (simple/dynamic/automated) | Redundant with `starlark_hook` presence check |
| `AdoptTeamWorkflow` clone | Package adoption model |
| `ExportWorkflowPackage` endpoint | Standard package export |
### PROMOTE — Buried features to expose as kernel primitives
1. **Roles → kernel primitive**: `team_user_roles` table, manifest `requires_roles`,
team admin UI, kernel middleware, Starlark SDK
2. **Typed forms → SDK primitive**: Move from workflow models to `forms` package,
FE SDK `sw.forms.render()` / `sw.forms.validate()`
3. **Conditional routing → SDK primitive**: Starlark `routing.evaluate(rules, data)`
4. **Workflow Starlark module → full read/write**: `workflow.start()`, `.advance()`,
`.cancel()`, `.submit_signoff()`
---
## Technical Debt
### Starlark Converter Duplication
Three files contain nearly identical Go↔Starlark conversion functions:
| File | Functions |
|------|-----------|
| `workflow/automated.go` | `goToStarlark`, `starlarkDictToMap`, `starlarkToGo` |
| `handlers/workflow_hooks.go` | `starlarkDictToMap`, `starlarkToGo`, `jsonToStarlark` |
| `sandbox/workflow_module.go` | `goValToStarlark` |
**Fix:** Consolidate into `sandbox/convert.go`.
### Snapshot Format Inconsistency
Two formats exist (wrapped `{stages, workflow}` vs legacy flat array).
Three copies of the parser across engine, instance handlers, and
assignment handlers.
**Fix:** Consolidate into one exported function. Standardize on wrapped format.
---
## Sequencing
| Version | Feature | Dependencies |
|---------|---------|-------------|
| **v0.9.0** | Starlark converter consolidation + snapshot format cleanup | None |
| **v0.9.1** | `team_user_roles` table + management API + admin UI | None |
| **v0.9.2** | `scope: adoptable` manifest field + adoption flow with role auto-populate | v0.9.1 |
| **v0.9.3** | Promote typed forms to SDK primitive (`sw.forms`) | None |
| **v0.9.4** | Deprecate `stage_type`, collapse `stage_mode` to 3 values | None |
| **v0.9.5** | Full read/write workflow Starlark module | v0.9.0 |
| **v0.9.6** | Conditional routing as SDK primitive | v0.9.5 |
| **v0.9.7** | Multi-surface manifest + kernel route resolution | None |
| **v0.9.8** | Wire roles into surface access (`access: role:X`) + kernel middleware | v0.9.1, v0.9.7 |
Each step is CI-green independently. The first four are the high-value items.
---
## Files Affected (by component)
### Converter Consolidation (v0.9.0)
- New: `sandbox/convert.go`
- Modified: `workflow/automated.go`, `handlers/workflow_hooks.go`, `sandbox/workflow_module.go`
### Team Roles (v0.9.1)
- New: migration (PG + SQLite), `store/team_roles.go`, `handlers/team_roles.go`
- Modified: manifest validation, adoption flow, team admin UI
### Typed Forms (v0.9.3)
- New: `forms/` package (extracted from `models/workflow.go`)
- New: `sw.forms` SDK module
- Modified: workflow engine to import from `forms/` instead of inline
### Workflow Starlark Module (v0.9.5)
- Modified: `sandbox/workflow_module.go` — add start/advance/cancel/signoff
- Modified: `workflow/engine.go` — extract interface to break circular import
---
## Non-Goals
- Rewriting the workflow engine. The state machine is correct and stays.
- Adding a visual workflow builder. That's an extension concern, not kernel.
- Multi-tenancy changes. Team scoping works as designed.

227
docs/DISTRIBUTION.md
View File

@@ -1,227 +0,0 @@
# Distribution Guide
## Quick Start
```bash
docker pull gobha/armature:latest
docker run -p 8080:80 \
-e ARMATURE_ADMIN_USERNAME=admin \
-e ARMATURE_ADMIN_PASSWORD=changeme \
gobha/armature:latest
```
On first run, bundled packages are automatically installed — workflows, surfaces, and extensions are ready to use immediately.
## Bundled Packages
The production Docker image ships with pre-built packages. A **curated default set** is auto-installed on first boot; additional packages ship in the image and can be opted-in via `BUNDLED_PACKAGES`.
#### Default Set (auto-installed)
| Package | Type | Description |
|---------|------|-------------|
| notes | surface | Markdown notes with graph view |
| chat | surface | Real-time chat surface |
| chat-core | library | Conversations, messages, read cursors |
| mermaid-renderer | extension | Diagram rendering |
| schedules | surface | Cron task management UI |
#### Opt-in (ship in image, require `BUNDLED_PACKAGES` to enable)
| Package | Type | Description |
|---------|------|-------------|
| tasks | full | Kanban/list task manager with webhooks |
| workflow-chat | extension | Chat integration for workflow stages |
| dashboard | surface | Dashboard surface |
| workflow-demo | surface | Interactive walkthrough with diagrams |
| bug-report-triage | workflow | Public entry, severity routing, SLA timers |
| content-approval | workflow | Multi-party signoff example |
| employee-onboarding | workflow | Automated provisioning + manager signoff |
| editor | surface | Rich markdown editor |
| hello-dashboard | surface | Welcome/getting started surface |
| team-activity-log | surface | Team activity feed |
| webhook-notifier | workflow | HTTP outbound via connections + Starlark |
| csv-table | extension | CSV table renderer |
| diff-viewer | extension | Diff visualization |
| git-board | surface | Git board interface |
| gitea-client | library | Gitea API integration library |
| js-sandbox | extension | JavaScript sandbox |
| katex-renderer | extension | LaTeX rendering |
| regex-tester | extension | Regex testing tool |
| icd-test-runner | surface | E2E API test suite |
| sdk-test-runner | surface | SDK feature test suite |
### Behavior
- **First boot**: Curated default packages are installed and enabled automatically.
- **Subsequent boots**: No-op — already-installed packages are skipped.
- **Admin uninstalls a package**: It stays uninstalled. Bundled packages are never force-reinstalled.
- **To re-install**: Delete the package from the database, then restart the container.
### Selecting Packages (Allowlist)
Set `BUNDLED_PACKAGES` to control which packages are installed:
```bash
# Install ALL packages (everything in the image)
docker run -p 8080:80 \
-e BUNDLED_PACKAGES="*" \
gobha/armature:latest
# Install specific packages only
docker run -p 8080:80 \
-e BUNDLED_PACKAGES="notes,tasks,schedules" \
gobha/armature:latest
```
Empty (default) installs the curated default set. Use `*` to install all packages. This is useful for Helm charts where different environments need different packages.
### Disabling Auto-Install
Set `SKIP_BUNDLED_PACKAGES=true` to prevent bundled packages from being installed:
```bash
docker run -p 8080:80 \
-e SKIP_BUNDLED_PACKAGES=true \
gobha/armature:latest
```
### Custom Bundle Directory
Override the default bundled packages location with `BUNDLED_PACKAGES_DIR`:
```bash
docker run -p 8080:80 \
-e BUNDLED_PACKAGES_DIR=/custom/packages \
-v /host/packages:/custom/packages \
gobha/armature:latest
```
## Builder Image
The builder image pre-caches Go modules and Node dependencies for faster custom builds.
```bash
docker pull gobha/armature-builder:latest
```
### What It Caches
- Go module cache (`go mod download` for all server dependencies)
- Node modules for the CM6 editor bundle
- Vendor library tarballs (marked, DOMPurify, mermaid, KaTeX)
- Build tools (zip, Node.js runtime)
### Using in Custom Builds
Reference the builder image as a base stage in your Dockerfile:
```dockerfile
FROM gobha/armature-builder:latest AS builder
WORKDIR /app
COPY server/ .
RUN go build -ldflags="-s -w" -o /bin/armature .
```
### Building Locally
```bash
docker build -f Dockerfile.builder -t armature-builder .
```
## Custom Build Guide
### Adding Custom Packages
1. Create your package in `packages/your-package/` with a `manifest.json`
2. Build all packages: `cd packages && bash build.sh all`
3. Build the Docker image: `docker build -t my-armature .`
The Dockerfile automatically builds all packages in the `packages/` directory and bundles them into the production image.
### Removing Bundled Packages
To exclude specific packages from the bundle, either:
- Remove them from `packages/` before building
- Set `SKIP_BUNDLED_PACKAGES=true` and install packages manually via the admin UI
### Forking for Custom Builds
```bash
git clone https://github.com/gobha/armature.git
cd armature
# Add/modify packages
cp -r my-extension packages/my-extension/
# Build with builder image for faster compilation
docker build -t my-armature .
```
## Production Deployment
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8080` | Backend API port |
| `DB_DRIVER` | (auto) | `postgres` or `sqlite` |
| `DATABASE_URL` | | PostgreSQL connection string |
| `JWT_SECRET` | `dev-secret-change-me` | **Must change in production** |
| `ENCRYPTION_KEY` | | AES-256 key for credential encryption |
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, or `oidc` |
| `STORAGE_BACKEND` | (auto) | `pvc` or `s3` |
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
| `BASE_PATH` | | URL prefix (e.g. `/armature`) |
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable auto-install of bundled packages |
| `BUNDLED_PACKAGES` | (empty = defaults) | `""` curated defaults, `"*"` all, or comma-separated IDs |
| `BUNDLED_PACKAGES_DIR` | `/app/bundled-packages` | Override bundled packages location |
| `LOG_FORMAT` | `text` | `text` or `json` |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
### Database
PostgreSQL is recommended for production. SQLite is suitable for single-instance evaluation.
```bash
# PostgreSQL (recommended)
docker run -p 8080:80 \
-e DATABASE_URL="postgres://user:pass@host:5432/armature?sslmode=require" \
-e JWT_SECRET="$(openssl rand -hex 32)" \
-e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
gobha/armature:latest
# SQLite (evaluation only)
docker run -p 8080:80 \
-e DB_DRIVER=sqlite \
-v armature-data:/data \
gobha/armature:latest
```
### Storage
Object storage is required for file uploads and package asset extraction.
```bash
# PVC (auto-detected if path is writable)
docker run -v armature-storage:/data/storage ...
# S3-compatible (MinIO, AWS S3, Ceph)
docker run \
-e STORAGE_BACKEND=s3 \
-e S3_BUCKET=armature \
-e S3_ENDPOINT=https://minio.corp:9000 \
-e S3_ACCESS_KEY=... \
-e S3_SECRET_KEY=... \
-e S3_FORCE_PATH_STYLE=true \
...
```
### Kubernetes
See the `k8s/` directory for example manifests. Key considerations:
- Use `POSTGRES_HOST`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` env vars (assembled into DSN automatically)
- Set liveness probe to `/healthz/live`, readiness probe to `/healthz/ready`
- Mount a PVC at `/data/storage` for file storage, or configure S3
- Set `JWT_SECRET` and `ENCRYPTION_KEY` via Kubernetes Secrets

196
docs/EXTENSION-CSS.md
View File

@@ -1,196 +0,0 @@
# Extension CSS Contract
> **Version**: v0.6.13 — Responsive & Spacing
This document defines the CSS contract between the Armature kernel and extension
packages. Extensions **must** follow these rules; the kernel guarantees the listed
classes and variables are stable public API.
---
## Naming Rule
All class selectors in extension CSS (`packages/{slug}/css/main.css`) must start
with `.ext-{slug}-`. The `{slug}` is the package directory name.
```css
/* Good */
.ext-my-app-sidebar { ... }
.ext-my-app-card { ... }
/* Bad — will be rejected by the linter */
.sidebar { ... }
.my-sidebar { ... }
```
**Compound selectors**: Descendant classes scoped under your `.ext-{slug}` root
are allowed to reference kernel classes or state modifiers:
```css
/* Allowed — kernel class scoped under extension namespace */
.ext-my-app .sw-btn { margin-top: 8px; }
/* Allowed — state modifier on an extension element */
.ext-my-app-item.active { ... }
```
Run `bash scripts/lint-package-css.sh` to validate. The linter checks that the
**first** class selector in every rule starts with `.ext-{slug}`.
---
## Stable Kernel Classes
Extensions may reference these classes in compound selectors. They are part of
the public API and will not change without a major version bump.
### Components (from `sw-primitives.css`)
| Class pattern | Component |
|---------------|-----------|
| `.sw-btn`, `.sw-btn--{variant}`, `.sw-btn--{size}` | Buttons |
| `.sw-input` | Text inputs |
| `.sw-field`, `.sw-field__label`, `.sw-field__hint` | Form fields |
| `.sw-dialog`, `.sw-dialog__header`, `.sw-dialog__body`, `.sw-dialog__footer` | Dialogs |
| `.sw-toast`, `.sw-toast-container` | Toast notifications |
| `.sw-menu`, `.sw-menu-item` | Context menus |
| `.sw-tabs`, `.sw-tab-btn` | Tab strips |
| `.sw-dropdown` | Custom dropdowns |
| `.sw-spinner` | Loading spinners |
| `.sw-avatar` | User avatars |
| `.sw-drawer` | Slide-out drawers |
| `.sw-banner` | Banner bars |
| `.sw-tooltip` | Tooltips |
### Extension Mount
The extension surface container has a `data-ext` attribute set to the package
slug. Use this for scoping if needed:
```css
[data-ext="my-app"] .ext-my-app-sidebar { ... }
```
---
## Stable CSS Variables
All variables from `variables.css` are public API. Extensions should use these
instead of hardcoded colors to respect the user's theme.
### Colors
| Variable | Purpose |
|----------|---------|
| `--bg` | Page background |
| `--bg-secondary` | Secondary/darker background |
| `--bg-elevated` | Elevated surface background |
| `--bg-raised` | Raised card background |
| `--bg-surface` | Surface-level background |
| `--bg-hover` | Hover state background |
| `--bg-active` | Active/pressed state background |
| `--bg-code` | Code block background |
| `--text` | Primary text color |
| `--text-2` | Secondary text color |
| `--text-3` | Tertiary/muted text color |
| `--text-on-color` | Text on colored backgrounds |
| `--accent` | Primary accent color |
| `--accent-dim` | Dimmed accent for backgrounds |
| `--accent-hover` | Accent hover state |
| `--accent-light` | Light accent variant |
| `--border` | Default border color |
| `--border-light` | Light border variant |
| `--border-elevated` | Border for elevated surfaces |
| `--danger` | Error/destructive color |
| `--danger-dim` | Dimmed danger background |
| `--danger-light` | Light danger variant |
| `--success` | Success/positive color |
| `--success-dim` | Dimmed success background |
| `--success-light` | Light success variant |
| `--warning` | Warning/caution color |
| `--warning-dim` | Dimmed warning background |
| `--warning-light` | Light warning variant |
| `--purple` | Purple accent |
| `--purple-dim` | Dimmed purple background |
### Spacing
Use spacing tokens instead of hardcoded values for padding, margin, and gap.
For sub-4px values (1px, 2px, 3px) used in borders and fine detail, hardcoded
values are acceptable.
| Variable | Value | Computed |
|----------|-------|---------|
| `--sp-1` | `0.25rem` | 4px |
| `--sp-1h` | `0.375rem` | 6px |
| `--sp-2` | `0.5rem` | 8px |
| `--sp-2h` | `0.625rem` | 10px |
| `--sp-3` | `0.75rem` | 12px |
| `--sp-4` | `1rem` | 16px |
| `--sp-5` | `1.25rem` | 20px |
| `--sp-6` | `1.5rem` | 24px |
| `--sp-8` | `2rem` | 32px |
| `--sp-10` | `2.5rem` | 40px |
| `--sp-12` | `3rem` | 48px |
Example:
```css
.ext-my-app-card {
padding: var(--sp-3) var(--sp-4); /* 12px 16px */
gap: var(--sp-2); /* 8px */
margin-bottom: var(--sp-4); /* 16px */
}
```
### Layout & Typography
| Variable | Purpose |
|----------|---------|
| `--font` | Primary font family (self-hosted, no external requests) |
| `--mono` | Monospace font family (self-hosted) |
| `--radius-sm` | Small border-radius (4px) — badges, inline controls |
| `--radius` | Default border-radius (8px) — buttons, inputs, cards |
| `--radius-lg` | Large border-radius (12px) — modals, dialogs, large cards |
| `--transition` | Default transition timing |
| `--shadow-lg` | Large elevation shadow |
| `--overlay` | Modal overlay color |
| `--glass` | Glassmorphism backdrop |
| `--input-bg` | Form input background |
| `--sidebar-w` | Sidebar width |
---
## Responsive Breakpoints
The kernel uses these standard breakpoints. Extensions should use the same
values for consistency.
| Name | Media Query | Use Case |
|------|-------------|----------|
| Mobile | `@media (max-width: 768px)` | Phone-sized, single column |
| Tablet | `@media (max-width: 1024px)` | Tablet/small laptop, narrower sidebars |
| Desktop | Default (no query) | Full layout |
CSS custom properties cannot be used in `@media` queries — use the pixel
values directly.
---
## What Is Internal
Everything not listed above is **internal kernel CSS** and may change between
minor versions. Extensions must not depend on:
- Kernel layout classes (`.admin-*`, `.surface-*`, `.sidebar`, etc.)
- Kernel CSS file load order
- Specific HTML structure of the shell or topbar
- Undocumented CSS variables
---
## Enforcement
The linter script `scripts/lint-package-css.sh` runs against all
`packages/*/css/main.css` files. It exits non-zero if any rule's first class
selector does not start with `.ext-{slug}`.

404
docs/EXTENSION-GUIDE.md
View File

@@ -1,404 +0,0 @@
# Extension Guide
## Package Types
| Type | Description |
|------|-------------|
| `surface` | A routable UI page rendered in the shell viewport |
| `extension` | Starlark hooks, tools, API routes, DB tables |
| `full` | Both surface and extension combined |
| `library` | Shared code imported by other packages via `lib.require()` |
| `workflow` | Bundled workflow definition |
## Tiers
| Tier | Runs | Capabilities |
|------|------|-------------|
| `browser` | Client-side JS only | DOM access, SDK hooks, no server-side logic |
| `starlark` | Sandboxed server-side | DB, HTTP, notifications, secrets, API routes, realtime |
| `sidecar` | Separate container | Full runtime (future) |
## manifest.json
Every package has a `manifest.json` at its root. Example for a surface:
```json
{
"id": "my-surface",
"title": "My Surface",
"type": "full",
"tier": "starlark",
"version": "1.0.0",
"description": "A custom surface with server-side logic.",
"icon": "🔧",
"author": "you",
"route": "/s/my-surface",
"auth": "authenticated",
"permissions": ["db.write"],
"api_routes": [
{"method": "GET", "path": "/items"},
{"method": "POST", "path": "/items"}
],
"db_tables": {
"items": {
"columns": {
"title": "text",
"done": "int"
},
"indexes": [["title"]]
}
},
"settings": {
"page_size": {
"type": "string",
"label": "Items Per Page",
"description": "Number of items shown per page",
"default": "25"
}
}
}
```
### Field Reference
| Field | Required | Description |
|-------|----------|-------------|
| `id` | yes | Unique kebab-case identifier |
| `title` | yes | Display name |
| `type` | yes | `surface`, `extension`, `full`, `library`, `workflow` |
| `tier` | yes | `browser`, `starlark`, `sidecar` |
| `version` | yes | Semver string |
| `description` | no | Short description |
| `icon` | no | Emoji icon for sidebar/menu |
| `route` | surfaces | URL path (e.g., `/s/my-surface`) |
| `auth` | no | `authenticated` (default) or `public` |
| `permissions` | no | Sandbox capabilities: `db.write`, `db.read`, `api.http`, `notifications`, `secrets`, `realtime.publish`, `connections.read`, `workflow.access`, `batch.exec`, `files.read`, `files.write`, `workspace.manage` |
| `api_routes` | no | Array of `{method, path}` for extension HTTP endpoints |
| `api_schema` | no | OpenAPI documentation for extension API routes (see below) |
| `db_tables` | no | Table definitions (see below) |
| `settings` | no | User-configurable settings schema |
| `exports` | no | Functions exported for cross-package calls via `lib.require()` |
| `depends` | no | Array of package IDs this package depends on |
| `slots` | no | Named UI injection points for host surfaces |
| `contributes` | no | Slot contributions into other surfaces |
| `hooks` | no | Event bus subscriptions |
| `config_section` | no | Settings/Admin panel injection (see below) |
| `capabilities` | no | Environment requirements (see below) |
| `user_permissions` | no | Permissions this extension registers for users (see below) |
| `gate_permission` | no | Permission checked before `on_request` executes |
| `schema_version` | no | Integer for additive schema migrations |
## db_tables Schema
Tables are automatically namespaced as `ext_{package_id}_{table_name}`.
Every table gets an auto-generated `id` primary key and `created_at` timestamp.
Column types: `text`, `int`, `vector(N)`.
The `vector(N)` type stores N-dimensional float vectors for similarity
search via `db.query_similar()`. Storage adapts to the backend:
Postgres + pgvector uses native `vector(N)` with HNSW indexes,
Postgres without pgvector uses `JSONB`, SQLite uses `TEXT`. N can be
1–4096. See the [Starlark Reference](STARLARK-REFERENCE) for query API.
```json
"db_tables": {
"notes": {
"columns": {
"title": "text",
"body": "text",
"creator_id": "text",
"pinned": "int"
},
"indexes": [
["creator_id"],
["pinned"]
]
}
}
```
## api_schema (OpenAPI Documentation)
Extensions can optionally declare an `api_schema` array in their manifest to provide rich API documentation. Declared routes appear in Swagger UI at `/api/docs` with parameters, request bodies, and response schemas. Routes without `api_schema` entries still appear as auto-generated stubs.
```json
"api_schema": [
{
"path": "/items",
"method": "GET",
"summary": "List items",
"description": "Returns paginated items for the current user",
"params": {
"limit": {"type": "integer", "default": 50, "description": "Max results"},
"offset": {"type": "integer", "default": 0}
},
"response": {
"type": "object",
"example": {"data": [{"id": "string", "title": "string"}]}
}
},
{
"path": "/items",
"method": "POST",
"summary": "Create item",
"body": {
"title": {"type": "string", "required": true},
"description": {"type": "string"}
}
}
]
```
Only `path` and `method` are required. All other fields are optional. Malformed entries are logged and skipped without blocking extension loading.
## Capabilities
Extensions can declare environment requirements via the `capabilities`
manifest field. The kernel validates these at install time.
```json
{
"capabilities": {
"required": ["postgres"],
"optional": ["pgvector", "workspace"]
}
}
```
- **required** — install is rejected (HTTP 422) if any capability is missing.
- **optional** — install succeeds with a logged warning. Query at runtime
with `settings.has_capability("pgvector")` to adapt behavior.
Detected capabilities: `pgvector`, `workspace`, `object_storage`, `s3`,
`postgres`. The admin can view detected capabilities at
**Admin > System > Capabilities** or via `GET /admin/capabilities`.
## User Permissions
Extensions can register custom permissions that the admin assigns to
user groups. This controls access to extension features beyond the
sandbox permission model.
```json
{
"user_permissions": ["image-gen.use", "image-gen.admin"],
"gate_permission": "image-gen.use"
}
```
- **user_permissions** — on install, these are merged into the kernel's
permission registry. On uninstall, they are removed. The admin assigns
them to groups in **Admin > Groups**.
- **gate_permission** — if set, the kernel checks this permission before
calling `on_request`. Unauthorized users get a 403 without the
extension code executing.
In Starlark, check permissions inline via `req["permissions"]` or call
`permissions.check(user_id, "image-gen.use")`.
## Extension Composability
Extensions compose with each other through three mechanisms: manifest-declared
**slots** (UI injection points), **contributions** (UI components injected into
those slots), and cross-package **function calls** via `lib.require()`.
### Slots — Host Surfaces Declare Injection Points
A surface declares named slots in its manifest where other extensions can
inject UI components:
```json
{
"id": "notes",
"type": "surface",
"slots": {
"toolbar-actions": {
"description": "Toolbar action buttons",
"context": {
"noteId": "string",
"getContent": "function — returns note body",
"setContent": "function — replaces note body"
}
}
}
}
```
The surface renders slot contents using the SDK helper:
```javascript
html`<div class="toolbar">
${sw.slots.renderAll('notes:toolbar-actions', {
noteId: note.id,
getContent: () => editor.getValue(),
setContent: (text) => editor.setValue(text),
})}
</div>`
```
### Contributions — Extensions Inject UI
An extension declares which slots it contributes to:
```json
{
"id": "note-dictate",
"type": "extension",
"contributes": {
"notes:toolbar-actions": {
"label": "Dictate",
"icon": "🎤",
"description": "Voice-to-text dictation"
}
}
}
```
In its JavaScript, it registers the component:
```javascript
sw.slots.register('notes:toolbar-actions', {
id: 'note-dictate',
priority: 200,
component: ({ noteId, setContent, getContent }) => {
// ... component implementation
return html`<button class="sw-btn sw-btn--ghost">🎤</button>`;
}
});
```
Contributions are soft-coupled — install order doesn't matter. The admin
can view all slots and contributors at **Admin > Packages** or via
`GET /api/v1/admin/slots`.
### Cross-Package Function Calls
Any package that declares `exports` in its manifest can be called by other
packages via `lib.require()`. The caller declares the dependency:
```json
{
"id": "note-ai",
"depends": ["llm-bridge"],
"permissions": ["api.http"]
}
```
In Starlark:
```python
llm = lib.require("llm-bridge")
result = llm.complete([{"role": "user", "content": prompt}])
```
The called function runs with the *target* package's permissions, not the
caller's. This is the same security model as library packages.
### Slot Naming Convention
Slot names follow `{host-package-id}:{slot-name}`. The colon separates the
namespace from the slot. Standard slots for first-party packages:
| Slot | Host | Use Case |
|------|------|----------|
| `notes:toolbar-actions` | notes | Dictation, AI tools, formatting |
| `notes:note-footer` | notes | Related items, AI summary |
| `chat:composer-tools` | chat | Image gen, file attach |
| `chat:message-actions` | chat | Reactions, translate, bookmark |
| `chat:image-actions` | chat | Regen, edit, upscale |
## Starlark Sandbox API
Starlark scripts run server-side with a 1M operation budget and no
filesystem access. See the [Starlark Reference](STARLARK-REFERENCE) for
the complete module catalog, function signatures, and permission gates.
## config_section — Settings Panel Injection
Packages can inject configuration panels into the Settings, Admin, or
Team Admin surfaces. Declare `config_section` in `manifest.json`:
```json
{
"config_section": {
"label": "My Config",
"icon": "M12 2L2 7l10 5 10-5-10-5z",
"component": "js/config.js",
"surfaces": ["settings", "admin"],
"category": "system"
}
}
```
| Field | Required | Description |
|-------|----------|-------------|
| `label` | yes | Navigation label shown in the sidebar or tab list |
| `icon` | no | SVG path data for the nav icon |
| `component` | no | JS asset path (default: `js/config.js`). Must `export default` a Preact component. |
| `surfaces` | yes | Target surfaces: `"settings"`, `"admin"`, `"team-admin"` |
| `category` | no | Admin surface category tab (default: `"system"`). Ignored for settings/team-admin. |
**How it works:**
1. At page load, the backend scans all enabled packages for `config_section`
entries targeting the current surface.
2. Matching sections are injected into the page as `__CONFIG_SECTIONS__`.
3. The frontend dynamically imports the component module and renders it
as an additional tab/section.
4. The component receives `{ packageId, teamId }` as props.
**Example component** (`js/config.js`):
```javascript
const { html } = window;
const { useState, useEffect } = hooks;
export default function MyConfig({ packageId }) {
const [val, setVal] = useState('');
useEffect(() => {
sw.api.ext(packageId).get('/settings').then(r => setVal(r.value));
}, []);
return html`<div>
<label>API Key</label>
<input value=${val} onInput=${e => setVal(e.target.value)} />
<button onClick=${() => sw.api.ext(packageId).put('/settings', { value: val })}>Save</button>
</div>`;
}
```
## Permissions Model
Extensions declare required permissions in `manifest.json`. The admin
must grant each permission before the extension can use the corresponding
sandbox module. Permission status is visible in **Admin > Packages > Permissions**.
See [Permissions & Groups](PERMISSIONS-AND-GROUPS) for the full RBAC
model, user permission slugs, and settings cascade.
## File Structure
```
my-package/
manifest.json # required
js/ # browser-side JavaScript
index.js
css/ # stylesheets
styles.css
script.star # Starlark entry point
star/ # additional Starlark modules
assets/ # static assets
migrations/ # schema migration files
```
## Testing Extensions
1. Build the package: `cd packages && bash build.sh my-package`
2. Upload the `.pkg` file via Admin > Packages > Install.
3. Grant permissions in Admin > Packages > Permissions.
4. Enable the package.
5. If it is a surface, navigate to its route (e.g., `/s/my-package`).
Extension API routes are accessible at `/s/{slug}/api/{path}` and require an authenticated Bearer token.

294
docs/FRONTEND-JS-GUIDE.md
View File

@@ -1,294 +0,0 @@
# Frontend JS Guide
Armature extensions run in the browser using **Preact + htm** — a 3 KB
runtime with no build step. The kernel provides a rich SDK at `window.sw`
that extensions use for API calls, auth, events, theming, and UI.
## Getting started
Extension surfaces are ES modules loaded via `<script type="module">`.
The SDK is available on `window.sw` after the `sw:ready` DOM event:
```javascript
document.addEventListener('sw:ready', () => {
const { html } = window;
const mount = document.getElementById('my-mount');
preact.render(html`<${App} />`, mount);
});
```
Or use the global `hooks` object for Preact hooks:
```javascript
const { useState, useEffect } = hooks;
```
## SDK modules
All modules live on the `window.sw` object. They are frozen after boot
and available to every extension.
### sw.api — REST client
Generic escape hatches for any endpoint:
```javascript
sw.api.get('/api/v1/docs')
sw.api.post('/api/v1/teams', { name: 'Eng' })
sw.api.put(path, body)
sw.api.patch(path, body)
sw.api.del(path)
sw.api.upload(path, file)
sw.api.stream(path, body, signal)
```
All methods auto-inject auth tokens and return unwrapped `data` from
`{ data: ... }` response envelopes.
**Domain namespaces** provide typed CRUD methods:
| Namespace | Key methods |
|-----------|-------------|
| `sw.api.auth` | `login`, `register`, `refresh`, `logout` |
| `sw.api.teams` | `list`, `get`, `create`, `members`, `workflows`, `assignments` |
| `sw.api.workflows` | `list`, `get`, `stages`, `instances`, `advance`, `cancel` |
| `sw.api.channels` | `list`, `get`, `create`, `update`, `del` |
| `sw.api.notifications` | `list`, `unreadCount`, `markRead`, `markAllRead` |
| `sw.api.admin` | Sub-objects for `users`, `teams`, `groups`, `packages`, `backup`, etc. |
| `sw.api.users` | `search`, `resolve` |
| `sw.api.connections` | `list`, `get`, `create`, `resolve` |
| `sw.api.ext(pkgId)` | Scoped client for extension API routes: `get`, `post`, `put`, `del` |
### sw.auth — Authentication state
```javascript
sw.auth.isAuthenticated // boolean
sw.auth.user // { id, username, display_name, email, role, avatar }
sw.auth.permissions // Set<string>
sw.auth.teams // Array<{ id, name, role }>
sw.auth.groups // Array<{ id, name, permissions }>
```
Lifecycle: `sw.auth.login(login, pw)`, `sw.auth.logout()`, `sw.auth.refresh()`.
### sw.can — RBAC gates
```javascript
sw.can('workflow.create') // true if user has permission
sw.isAdmin // true if surface.admin.access granted
sw.isTeamAdmin(teamId) // true if admin role in team
```
Use these to conditionally render UI elements.
### sw.on / sw.off / sw.emit — Event bus
```javascript
const unsub = sw.on('theme.changed', (payload) => { ... });
sw.once('auth.login', (user) => { ... });
sw.off('theme.changed'); // remove all listeners
sw.off('theme.changed', fn); // remove specific listener
sw.emit('my.event', { data });
```
The event bus bridges to the WebSocket — server-emitted events
(e.g. `notification.created`, `workflow.sla_breach`) arrive here.
### sw.theme — Theme control
```javascript
sw.theme.current // 'dark' or 'light' (resolved)
sw.theme.mode // 'dark', 'light', or 'system'
sw.theme.set('dark')
sw.theme.on('change', (theme) => { ... })
sw.theme.tokens // live CSS variables as camelCase JS object
```
### sw.storage — Namespaced localStorage
```javascript
const store = sw.storage.local('my-extension');
store.set('key', { complex: 'value' });
store.get('key') // parsed object
store.remove('key')
store.keys() // ['key', ...]
store.clear()
```
### sw.realtime — WebSocket pub/sub
```javascript
const unsub = sw.realtime.subscribe('my-channel', 'item.updated', (data) => { ... });
// Or subscribe to all events on a channel:
const unsub = sw.realtime.subscribe('my-channel', (event, data) => { ... });
```
### sw.slots — UI slot registry
Register components into named shell slots (e.g. toolbar areas):
```javascript
const unreg = sw.slots.register('topbar-actions', {
id: 'my-button',
component: MyButton,
priority: 10,
});
```
### sw.actions — Named action registry
```javascript
sw.actions.register('copy-link', {
handler: async (url) => navigator.clipboard.writeText(url),
label: 'Copy Link',
icon: 'M12 2...',
});
await sw.actions.run('copy-link', someUrl);
```
### sw.pipe — Filter pipeline
Three-stage pipeline for message processing:
```javascript
sw.pipe.pre(10, async (ctx) => { /* pre-process */ return ctx; });
sw.pipe.stream(10, async (ctx) => { /* streaming */ return ctx; });
sw.pipe.render(10, async (ctx) => { /* post-render */ return ctx; });
```
### sw.renderers — Block and post renderers
Register custom renderers for fenced code blocks or post-processing:
```javascript
sw.renderers.register('mermaid', {
type: 'block',
pattern: /^mermaid$/,
render: (code, container) => { /* render diagram */ },
});
sw.renderers.register('linkify', {
type: 'post',
render: (container) => { /* post-process rendered HTML */ },
});
```
### sw.markdown — Unified rendering
```javascript
const html = sw.markdown.renderSync(markdownString, { sanitize: false });
await sw.markdown.render(markdownString); // async variant
sw.markdown.ready // boolean — true after preload
```
Uses marked + DOMPurify + registered `sw.renderers`.
### sw.users — Identity resolution
```javascript
const user = await sw.users.resolve(userId);
const map = await sw.users.resolveMany([id1, id2]);
sw.users.displayName(userObj) // display_name || username || 'Unknown'
```
Results are cached for 60 seconds. Batch fetches use the server's
bulk resolve endpoint.
### sw.testing — Test framework
For writing package runner tests:
```javascript
sw.testing.suite('CRUD', async (s) => {
s.before(async () => { /* setup */ });
s.after(async () => { /* cleanup */ });
s.test('creates item', async (t) => {
const resp = await sw.api.post('/api/v1/items', { name: 'test' });
t.assert.status(resp, 200);
t.assert.ok(resp.id);
s.track('item', resp.id); // auto-cleanup in afterAll
});
});
await sw.testing.run();
```
### sw.shell — Topbar API
The kernel injects a two-slot topbar into every extension surface.
Extensions customize it via:
```javascript
sw.shell.topbar.setTitle('My Surface'); // text in left slot
sw.shell.topbar.setSlot(html`<${TabBar} />`); // center slot content
sw.shell.topbar.hide(); // full-bleed mode
sw.shell.topbar.show(); // restore topbar
```
### sw.toast / sw.confirm / sw.prompt — UI primitives
```javascript
sw.toast('Saved!', 'success'); // success | error | info
sw.toast('Something broke', 'error', 5000); // custom duration
const ok = await sw.confirm('Delete this item?');
const name = await sw.prompt('Enter name', 'default value');
```
## Shell topbar patterns
Every surface uses one of three patterns:
| Pattern | Description | Example |
|---------|-------------|---------|
| **A — Default** | Shell title only. No center slot content. | Docs |
| **B — Flat tabs** | `setTitle()` + tabs in center slot via `setSlot()`. Full-width content. | Settings, Team Admin |
| **C — Category tabs + sidebar** | `setTitle()` + category tabs in center slot. Surface-owned sidebar below. | Admin |
The shell provides the home link, notification bell, and user menu
on every surface for free.
## `sw.forms` — Typed Forms (v0.9.5)
Any extension can render and validate typed forms using the `sw.forms` module.
### `sw.forms.render(container, template, opts)`
Renders a typed form into a DOM container. Supports flat forms and progressive
multi-step forms (fieldsets). Returns a control handle.
```js
const handle = sw.forms.render(document.getElementById('my-form'), template, {
values: { name: 'prefilled' },
onSubmit: (data) => { console.log('submitted', data); },
});
// Programmatic access
const data = handle.getData();
handle.setErrors([{ key: 'name', message: 'Name is taken' }]);
handle.destroy();
```
### `sw.forms.validate(template, data)`
Client-side validation (no network call). Returns `{ valid, errors }`.
```js
const { valid, errors } = sw.forms.validate(template, { name: '' });
// valid === false, errors === [{ key: 'name', message: 'Name is required' }]
```
### `sw.forms.validateRemote(template, data)`
Server-side validation via `POST /api/v1/forms/validate`. Returns a Promise.
```js
const result = await sw.forms.validateRemote(template, data);
```
## Extension CSS contract
Extensions must prefix all CSS classes with `.ext-{slug}-` to avoid
conflicts with kernel styles. See the [Extension CSS](EXTENSION-CSS)
doc for the full isolation rules, available primitives from
`sw-primitives.css`, and spacing tokens.

91
docs/GETTING-STARTED.md
View File

@@ -1,91 +0,0 @@
# Getting Started
## Prerequisites
- Docker and Docker Compose
## Running with Docker Compose
```bash
docker compose up --build
```
Open [http://localhost:3000](http://localhost:3000). Default credentials: `admin` / `admin`.
Data persists in the `armature_data` named volume. To reset everything:
```bash
docker compose down -v
```
## First Boot
On first start, Armature will:
1. Run database migrations (SQLite by default in compose).
2. Create the admin user from `ARMATURE_ADMIN_USERNAME` / `ARMATURE_ADMIN_PASSWORD` env vars.
3. Auto-install the curated default package set (notes, chat-core, dashboard, workflow demos, etc.).
No manual setup steps are required.
## Installing Additional Packages
The Docker image ships with extra packages beyond the default set. To install them:
- **Via Admin UI**: Go to `/admin` > Packages. Browse, enable, or install packages.
- **Via environment variable**: Set `BUNDLED_PACKAGES` before starting:
```bash
# Install all bundled packages
BUNDLED_PACKAGES="*" docker compose up --build
# Install specific extras
BUNDLED_PACKAGES="notes,tasks,schedules" docker compose up --build
```
You can also upload `.pkg` archives through the Admin > Packages page.
## Key URLs
| URL | Purpose |
|-----|---------|
| `/` | Redirects to your default surface |
| `/admin` | Admin panel (users, packages, settings, teams) |
| `/settings` | User settings (profile, preferences, default surface) |
| `/welcome` | Welcome page (shown when no surfaces are installed) |
| `/api/docs` | Interactive OpenAPI documentation |
## Key Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8080` | Backend API port |
| `DB_DRIVER` | auto | `postgres` or `sqlite` |
| `DATABASE_URL` | | PostgreSQL DSN or SQLite file path |
| `JWT_SECRET` | `dev-secret-change-me` | **Change in production** |
| `ENCRYPTION_KEY` | | AES-256 key for credential encryption |
| `AUTH_MODE` | `builtin` | `builtin`, `mtls`, or `oidc` |
| `STORAGE_BACKEND` | auto | `pvc` or `s3` |
| `STORAGE_PATH` | `/data/storage` | PVC mount point |
| `ARMATURE_ADMIN_USERNAME` | | Bootstrap admin username |
| `ARMATURE_ADMIN_PASSWORD` | | Bootstrap admin password |
| `BUNDLED_PACKAGES` | (empty) | `""` = curated defaults, `"*"` = all, or comma-separated IDs |
| `SKIP_BUNDLED_PACKAGES` | `false` | Disable auto-install entirely |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
| `LOG_FORMAT` | `text` | `text` or `json` |
## From Source
```bash
git clone <repo-url> && cd armature
cp server/.env.example server/.env # edit DB credentials
cd server && go run .
# Backend on http://localhost:8080
```
## Next Steps
- [Extension Guide](EXTENSION-GUIDE.md) -- Author your own packages
- [API Reference](API-REFERENCE.md) -- REST API overview
- [Deployment](DEPLOYMENT.md) -- Production deployment
- [Architecture](ARCHITECTURE.md) -- System design

201
docs/MULTI-SURFACE-GUIDE.md
View File

@@ -1,201 +0,0 @@
# Multi-Surface Packages
A multi-surface package serves multiple pages from a single package, each
with its own URL path, access level, and layout. Before v0.9.0, a package
got one route (`/s/{id}`), one auth posture, and one layout. Now a single
package can serve a public submission form alongside an authenticated
dashboard and an admin settings page.
## When to Use Multi-Surface
Use multi-surface when your package has logically related pages that share
the same backend (Starlark hooks, ext API, database tables) but need:
- **Different access levels** — public intake form + authenticated dashboard
- **Multiple views** — list view, detail view, edit view, monitor view
- **Sub-pages** — settings, admin, or debug pages within the package
If your pages don't share backend state, use separate packages instead.
## Manifest Setup
Add a `surfaces` array to your manifest. Each entry declares a page:
```json
{
"id": "workflow-builder",
"title": "Workflow Builder",
"type": "full",
"version": "0.1.0",
"icon": "🔧",
"auth": "authenticated",
"layout": "single",
"surfaces": [
{ "path": "/", "title": "Workflows" },
{ "path": "/new", "title": "New Workflow", "nav": false },
{ "path": "/:id/edit", "title": "Edit Workflow", "nav": false },
{ "path": "/monitor", "title": "Monitor", "nav": true },
{ "path": "/monitor/:id", "title": "Instance Detail", "nav": false }
],
"hooks": ["surface"],
"permissions": ["workflow.access"]
}
```
This generates five server routes, all under `/s/workflow-builder/`:
| Route | Access | Nav |
|-------|--------|-----|
| `/s/workflow-builder/` | authenticated | yes |
| `/s/workflow-builder/new` | authenticated | no |
| `/s/workflow-builder/:id/edit` | authenticated | no |
| `/s/workflow-builder/monitor` | authenticated | yes |
| `/s/workflow-builder/monitor/:id` | authenticated | no |
### Surface Entry Fields
| Field | Default | Description |
|----------|----------------|-------------|
| `path` | **required** | URL path relative to `/s/{id}`. Supports `:param` segments. |
| `access` | package `auth` | `public`, `authenticated`, `admin`, or `group:{name}`. |
| `title` | package `title`| Label shown in nav and page title. |
| `layout` | package `layout`| `single` or `editor`. |
| `nav` | `true` for `/`, `false` otherwise | Whether this surface appears in the sidebar. |
### Mixed Access Levels
A package can mix public and authenticated surfaces:
```json
{
"id": "bug-tracker",
"auth": "authenticated",
"surfaces": [
{ "path": "/submit", "access": "public", "title": "Report a Bug" },
{ "path": "/", "title": "Dashboard" },
{ "path": "/:id", "title": "Bug Detail", "nav": false },
{ "path": "/admin", "access": "admin", "title": "Settings", "nav": false }
]
}
```
The kernel enforces access per-surface. Unauthenticated visitors can reach
`/submit` but are redirected to login if they try `/` or `/:id`.
## Frontend: Routing Within Your Package
### Reading the Current Surface
When your package JS loads, two globals tell you which surface was matched:
```js
const path = window.__SURFACE_PATH__ || '/';
const params = window.__SURFACE_PARAMS__ || {};
```
Use these to decide which view to render:
```js
function render() {
const path = window.__SURFACE_PATH__ || '/';
const params = window.__SURFACE_PARAMS__ || {};
const root = document.getElementById('surface-root');
switch (path) {
case '/': return renderList(root);
case '/new': return renderEditor(root, null);
case '/:id/edit': return renderEditor(root, params.id);
case '/monitor': return renderMonitor(root);
case '/monitor/:id':return renderDetail(root, params.id);
default: return render404(root);
}
}
render();
```
### SPA Navigation with `sw.navigate()`
Navigate between surfaces without a full page reload:
```js
// Navigate to a static path
sw.navigate('/new');
// Navigate with params
sw.navigate('/:id/edit', { id: 'wf-42' });
// Navigate to monitor sub-page
sw.navigate('/monitor/:id', { id: 'inst-7' });
```
`sw.navigate()` does three things:
1. Updates `window.__SURFACE_PATH__` and `window.__SURFACE_PARAMS__`
2. Calls `history.pushState()` to update the URL
3. Emits a `surface.navigate` event
### Listening for Navigation Events
Re-render when the user navigates (including browser back/forward):
```js
sw.on('surface.navigate', ({ path, params }) => {
window.__SURFACE_PATH__ = path;
window.__SURFACE_PARAMS__ = params;
render();
});
```
### Links Between Surfaces
For simple `<a>` links that do full page loads:
```html
<a href="/s/workflow-builder/monitor">Monitor</a>
```
For SPA-style navigation:
```js
button.onclick = () => sw.navigate('/monitor');
```
## API Routes
All surfaces in a package share the same ext API. API calls go through
`/s/{id}/api/*` regardless of which surface is active:
```js
// These work from any surface in the package
const items = await sw.api.get('/items');
const item = await sw.api.get(`/items/${id}`);
await sw.api.post('/items', { title: 'New item' });
```
The ext API handler checks `package.status == "active"` — if the package
is in `pending_review`, all API calls return 403.
## Backward Compatibility
Packages without a `surfaces` array continue to work. The kernel
synthesizes a single-entry array from the legacy `auth` and `layout`
fields:
```
auth: "authenticated" + layout: "single"
→ surfaces: [{ "path": "/", "access": "authenticated", "layout": "single" }]
```
No migration is required for existing packages.
## Constraints
- **No cross-package routing.** Surface paths are relative to the package
mount point. A package cannot claim arbitrary top-level routes.
- **No per-surface Starlark hooks.** All surfaces share the same backend
hooks. Use the surface path in your hook logic to differentiate.
- **No SSR.** Surface rendering is client-side. The kernel serves the
shell template; your JS renders the content.

254
docs/PACKAGE-FORMAT.md
View File

@@ -1,254 +0,0 @@
# Package Format
Armature packages are distributed as `.pkg` files -- ZIP archives with a standard internal structure.
## ZIP Structure
```
my-package.pkg
├── manifest.json # required -- package metadata
├── js/ # browser-side JavaScript
│ └── index.js
├── css/ # stylesheets
│ └── styles.css
├── script.star # Starlark entry point
├── star/ # additional Starlark modules
├── assets/ # static assets (images, etc.)
└── migrations/ # schema migration files
```
Only `manifest.json` is required. All other directories are optional and included only if present in the source.
## manifest.json Reference
```json
{
"id": "my-package",
"title": "My Package",
"type": "surface",
"tier": "browser",
"version": "1.0.0",
"description": "What this package does.",
"icon": "📦",
"author": "your-name",
"route": "/s/my-package",
"auth": "authenticated",
"permissions": [],
"api_routes": [],
"db_tables": {},
"settings": {},
"hooks": {},
"exports": [],
"schema_version": 1
}
```
### Required Fields
| Field | Description |
|-------|-------------|
| `id` | Unique kebab-case identifier |
| `title` | Human-readable display name |
| `type` | `surface`, `extension`, `full`, `library`, `workflow` |
| `tier` | `browser`, `starlark`, `sidecar` |
| `version` | Semver version string |
### Optional Fields
| Field | Description |
|-------|-------------|
| `description` | Short description |
| `icon` | Emoji for sidebar/menu display |
| `author` | Package author |
| `surfaces` | Array of surface entries with per-path access, title, layout (see below) |
| `route` | *(deprecated — use `surfaces`)* URL path for surfaces |
| `auth` | Default access level for all surfaces: `authenticated`, `public`, `admin` |
| `layout` | Default layout for all surfaces: `single`, `editor` |
| `permissions` | Array of required capabilities (`db.write`, `http`, `notifications`, `secrets`, `realtime.publish`) |
| `api_routes` | Array of `{"method": "GET", "path": "/items"}` |
| `db_tables` | Table definitions with columns and indexes |
| `settings` | User-configurable settings with type, label, description, default |
| `hooks` | Event bus subscription patterns |
| `exports` | Functions exported for cross-package calls via `lib.require()` |
| `depends` | Array of package IDs this package depends on |
| `slots` | Named UI injection points (host surfaces declare these) |
| `contributes` | Slot contributions this package injects into other surfaces |
| `capabilities` | Environment requirements: `{"required": [...], "optional": [...]}` |
| `schema_version` | Integer for additive schema migrations |
| `form_template` | Typed form template (v0.9.5). Fields array or fieldsets for progressive forms. Validated at install. |
## Multi-Surface Packages
A package can serve multiple pages, each with its own path, access level,
title, and layout. Declare a `surfaces` array in the manifest:
```json
{
"id": "bug-tracker",
"title": "Bug Tracker",
"type": "full",
"auth": "authenticated",
"layout": "single",
"surfaces": [
{ "path": "/", "title": "Dashboard" },
{ "path": "/submit", "access": "public", "title": "Report a Bug" },
{ "path": "/:id", "title": "Bug Detail", "nav": false },
{ "path": "/admin", "access": "admin", "title": "Settings", "nav": false }
]
}
```
### Surface Entry Fields
| Field | Type | Default | Description |
|----------|--------|----------------|-------------|
| `path` | string | **required** | Relative to `/s/{pkg-id}`. Supports `:param` segments. |
| `access` | string | package `auth` | `public`, `authenticated`, `admin`, `group:{name}`. |
| `title` | string | package `title`| Human label. Used in nav if `nav: true`. |
| `layout` | string | package `layout`| `single`, `editor`, or future layouts. |
| `nav` | bool | see rules | Show in sidebar navigation. |
### Nav Visibility Rules
- `path: "/"` defaults to `nav: true` (primary entry point)
- All others default to `nav: false` (sub-pages)
- Explicit `"nav": true` overrides — a package can put multiple entries in nav
- The sidebar links to the first surface with `nav: true`
### Backward Compatibility
If `surfaces` is absent, the kernel synthesizes one entry from the legacy
`auth` and `layout` fields. No existing packages break.
### Client-Side Navigation
Within a multi-surface package, use `sw.navigate()` for SPA-style routing:
```js
const path = window.__SURFACE_PATH__ || '/';
const params = window.__SURFACE_PARAMS__ || {};
if (path === '/') renderDashboard();
if (path === '/submit') renderSubmitForm();
if (path === '/:id') renderDetail(params.id);
// Navigate to another surface within the package
sw.navigate('/submit');
sw.navigate('/:id', { id: 'bug-42' });
// Listen for navigation events (including back/forward)
sw.on('surface.navigate', ({ path, params }) => {
renderView(path, params);
});
```
## Composability: Slots and Contributions
Packages compose with each other through named UI injection points (**slots**) and **contributions**.
### Declaring Slots (Host Surfaces)
Surfaces declare slots where other extensions can inject UI:
```json
{
"slots": {
"toolbar-actions": {
"description": "Toolbar action buttons",
"context": {
"noteId": "string — current note ID",
"getContent": "function — returns note body text"
}
}
}
}
```
Slot names are namespaced at runtime as `{package-id}:{slot-name}` (e.g., `notes:toolbar-actions`). The `context` field documents what data the slot provides — it is not enforced at runtime.
### Contributing to Slots
Extensions declare which slots they inject into:
```json
{
"contributes": {
"notes:toolbar-actions": {
"label": "Dictate",
"icon": "🎤",
"description": "Voice-to-text dictation"
}
}
}
```
Contributing extensions can be installed before or after their host surface — the coupling is soft. The admin can view all slots and their contributors at `GET /api/v1/admin/slots`.
### Cross-Package Function Calls
Any package that declares `exports` can be called via `lib.require()`, not just library-type packages. The caller must declare the target in its `depends` array:
```json
{
"depends": ["image-gen"],
"permissions": ["api.http"]
}
```
## Package Lifecycle
1. **Install**: Upload a `.pkg` file via Admin > Packages or `POST /api/v1/admin/packages/install`. The kernel extracts the archive, creates database tables, and registers routes.
2. **Enable**: Activate the package so its routes, hooks, and UI become available. `PUT /api/v1/admin/packages/:id/enable`.
3. **Disable**: Deactivate without removing data. `PUT /api/v1/admin/packages/:id/disable`.
4. **Update**: Upload a new `.pkg` with a higher semver version. Schema changes must be additive (new columns/tables only). `POST /api/v1/admin/packages/:id/update`.
5. **Export**: Download the installed package as a `.pkg` archive. `GET /api/v1/admin/packages/:id/export`.
6. **Delete**: Remove the package and its data. `DELETE /api/v1/admin/packages/:id`.
## Bundled vs User-Installed
**Bundled packages** ship inside the Docker image at `/app/bundled-packages`. On first boot, a curated default set is auto-installed. Behavior:
- First boot: curated defaults are installed and enabled.
- Subsequent boots: already-installed packages are skipped.
- Admin uninstalls: the package stays uninstalled (never force-reinstalled).
- Control via `BUNDLED_PACKAGES` env var: empty = defaults, `"*"` = all, or comma-separated IDs.
**User-installed packages** are uploaded through the Admin UI or API. They follow the same lifecycle but are not tied to the Docker image.
## Building Packages
Packages are built by zipping the standard directories alongside `manifest.json`:
```bash
# Build all packages in the packages/ directory
cd packages && bash build.sh
# Build a single package
cd packages && bash build.sh my-package
```
Output goes to `dist/my-package.pkg`.
### Manual Build
```bash
cd packages/my-package
zip -r ../../dist/my-package.pkg manifest.json js/ css/ script.star
```
The build script automatically includes whichever standard directories exist: `js/`, `css/`, `assets/`, `script.star`, `star/`, `migrations/`.
### Custom Docker Image
To bundle custom packages into a Docker image:
1. Place your package in `packages/your-package/` with a `manifest.json`.
2. Run `cd packages && bash build.sh all`.
3. Build the image: `docker build -t my-armature .`
The Dockerfile builds all packages and copies them into the bundled packages directory.

94
docs/PACKAGE-REGISTRY.md
View File

@@ -1,94 +0,0 @@
# Package Registry
The package registry lets admins browse and install packages from an external
JSON index — a lightweight alternative to manually uploading `.pkg` files.
## Registry JSON Format
The registry is a static JSON file matching the `RegistryResponse` struct:
```json
{
"packages": [
{
"id": "notes",
"title": "Notes",
"version": "0.8.0",
"description": "Markdown notes with backlinks and graph view",
"author": "armature",
"type": "extension",
"tier": "core",
"download_url": "https://cdn.example.com/pkg/notes.pkg",
"size": 48200,
"updated_at": "2026-03-20T00:00:00Z"
}
]
}
```
### Required Fields
| Field | Description |
|----------------|------------------------------------------------|
| `id` | Package identifier (matches `manifest.json`) |
| `title` | Display name |
| `version` | Semver version string |
| `description` | Short description |
| `download_url` | HTTPS URL to the `.pkg` file (must be HTTPS) |
### Optional Fields
| Field | Description |
|--------------|------------------------------------------|
| `author` | Package author |
| `type` | `extension` or `library` |
| `tier` | `core`, `official`, or `community` |
| `size` | File size in bytes |
| `updated_at` | ISO 8601 timestamp of last update |
## Configuring the Registry URL
### Via Admin UI
Navigate to **Admin > Settings > Package Registry** and enter the registry URL.
### Via API
```bash
curl -X PUT /api/v1/admin/settings/package_registry \
-H "Content-Type: application/json" \
-d '{"value": {"url": "https://cdn.example.com/pkg/registry.json"}}'
```
## Generating a Registry
Use `scripts/generate-registry.sh` to build a `registry.json` from a
directory of `.pkg` files:
```bash
# Default: reads dist/, uses placeholder base URL
./scripts/generate-registry.sh
# Custom directory and base URL
./scripts/generate-registry.sh ./my-packages https://cdn.example.com/pkg > registry.json
```
Requirements: `jq`, `unzip`, and `stat` (GNU or BSD).
## Self-Hosting
Any HTTPS-capable file server works. Upload your `.pkg` files and the
generated `registry.json` to the same directory, then point the admin
setting to the `registry.json` URL.
Example with a static file server:
```
/var/www/packages/
registry.json
notes.pkg
chat.pkg
chat-core.pkg
```
The registry is fetched and cached for 5 minutes on each browse request.

121
docs/PERMISSIONS-AND-GROUPS.md
View File

@@ -1,121 +0,0 @@
# Permissions & Groups
Armature uses group-based RBAC. Permissions are granted to groups, and users
inherit the union of permissions from all groups they belong to. There are no
per-user permission grants — all access flows through group membership.
## Groups
### System groups
| Group | ID | Purpose |
|-------|----|---------|
| Everyone | `00000000-...0001` | Implicit membership for every authenticated user. Default permissions: `extension.use`, `workflow.submit`. |
| Admins | `00000000-...0002` | Full platform access. Members receive all seven permission slugs. Replaces the legacy `role = admin` check. |
Every new user is automatically added to **Everyone** on registration.
Admin status is granted by adding a user to the **Admins** group in
**Admin > People > Groups**.
### Custom groups
Administrators can create additional groups under **Admin > People > Groups**.
Each custom group has:
- **Name** — display label
- **Description** — purpose (shown in admin UI)
- **Scope** — always `global` (team-scoped groups reserved for future use)
- **Permissions** — zero or more permission slugs from the table below
## Permission slugs
Seven platform permissions control access to kernel features:
| Slug | Description |
|------|-------------|
| `surface.admin.access` | Full admin panel access (tabs, settings, package management) |
| `admin.view` | Read-only admin panel access (monitoring, health, audit log) |
| `extension.use` | Use installed extension surfaces and libraries |
| `extension.install` | Install, update, enable, and disable packages |
| `workflow.create` | Create and edit workflow definitions |
| `workflow.submit` | Submit instances to public-link workflows |
| `token.unlimited` | Bypass per-user token budgets (API rate limiting) |
Permissions follow a `domain.action` naming convention.
## Permission resolution
When a request arrives, the kernel resolves the effective permission set:
1. Fetch all groups the user belongs to (including Everyone).
2. Union all permission arrays across those groups.
3. Cache the result for the duration of the request.
A user has a permission if **any** of their groups grants it.
Frontend code checks permissions via `sw.can('slug')` — see the
[Frontend JS Guide](FRONTEND-JS-GUIDE) for details.
## Extension permissions
Separate from user permissions, each **package** can request sandbox
capabilities. These are granted per-package in **Admin > Packages**:
| Permission | Grants |
|------------|--------|
| `db.read` | Query `ext_data` tables (read-only) |
| `db.write` | Insert, update, and delete rows in `ext_data` tables |
| `api.http` | Make outbound HTTP requests from Starlark |
| `notifications.send` | Send in-app notifications to users |
| `secrets.read` | Read admin-configured extension secrets |
| `realtime.publish` | Publish WebSocket events to subscribed clients |
| `connections.read` | Read external connection configs (decrypted) |
| `workflow.access` | Read workflow definitions and instances |
See the [Starlark Reference](STARLARK-REFERENCE) for how these
map to sandbox modules.
## Settings cascade
Package settings use a three-tier resolution model:
```
user override → team override → global default
```
At each tier:
- **Global** — set by admins in **Admin > Packages > Settings**
- **Team** — set by team admins in **Team Admin > Settings**
- **User** — set by users in **Settings > Extensions**
### The `user_overridable` flag
Each setting key in a package manifest can declare `user_overridable`:
```json
{
"settings": [
{ "key": "theme", "user_overridable": true },
{ "key": "api_endpoint", "user_overridable": false }
]
}
```
- `true` (default) — team and user scopes can override the global value.
- `false` — only the global (admin) value is used. Team and user values
are silently ignored during resolution.
This gives administrators a lock mechanism: set `user_overridable: false`
on security-sensitive keys to prevent lower scopes from changing them,
while allowing cosmetic preferences to flow freely.
### Resolution algorithm
1. Start with the global value for each key.
2. For each key where `user_overridable` is true (or undeclared):
- If a team-scoped value exists, it overrides global.
- If a user-scoped value exists, it overrides team.
3. For keys where `user_overridable` is false:
- Team and user values are discarded.
4. Unknown keys (not in schema) default to overridable.

171
docs/ROADMAP-v010x-shift.md
View File

@@ -1,171 +0,0 @@
# ROADMAP — v0.10.x+ Version Shift (Final)
## Summary
Four new series inserted after v0.9.x. Panels (kernel primitive), Notes
(first reference extension), Chat (second reference extension), and the
remaining reference libraries/extensions including `llm-bridge` with
the tool meta-tool pattern.
## Full Roadmap
| Series | Title | Versions | Design Doc |
|--------|-------|----------|------------|
| v0.9.x | Multi-Surface + Workflow Redesign | 10 | — |
| **v0.10.x** | **Panels + Composable Layout** | 5 | `DESIGN-panels.md` |
| **v0.11.x** | **Notes Reference Extension** | 11 | `DESIGN-notes-v011x.md` |
| **v0.12.x** | **Chat Reference Extension** | 9 | `DESIGN-chat-v012x.md` |
| **v0.13.x** | **Reference Libraries + Extensions** | 9 | — |
| v0.14.x | Sidecar Tier + Polish | 3 | — |
| v1.0.0 | Stable Release | — | — |
---
## v0.10.x — Panels + Composable Layout
| Version | Title |
|---------|-------|
| v0.10.0 | Panel Manifest + Lifecycle |
| v0.10.1 | FloatingPanel Primitive |
| v0.10.2 | Docked Panels + Mode Transitions |
| v0.10.3 | Panel Communication Patterns |
| v0.10.4 | Reference Panel: Notes (basic) |
---
## v0.11.x — Notes Reference Extension
| Version | Title |
|---------|-------|
| v0.11.0 | UI/UX Foundation |
| v0.11.1 | Deep Folders + Navigation |
| v0.11.2 | Wikilinks + Backlinks |
| v0.11.3 | Live Preview + Rich Editing |
| v0.11.4 | Note Sharing + Permissions |
| v0.11.5 | Graph + Outline Hardening |
| v0.11.6 | Quick Switcher + Commands |
| v0.11.7 | Daily Notes + Templates |
| v0.11.8 | Transclusion + Embeds |
| v0.11.9 | Composability: Slots + Actions |
| v0.11.10 | Panel Enhancement + Quality Gate |
---
## v0.12.x — Chat Reference Extension
Human-to-human first. AI via `llm-bridge` (v0.13.x).
| Version | Title |
|---------|-------|
| v0.12.0 | UI/UX Foundation |
| v0.12.1 | Conversation Folders + Attributes |
| v0.12.2 | Reactions + Threads + Pins |
| v0.12.3 | Rich Compose + Attachments |
| v0.12.4 | @Mentions + Notifications |
| v0.12.5 | Link Previews + Message Formatting |
| v0.12.6 | Conversation Themes + Personality |
| v0.12.7 | Composability: Slots + Actions |
| v0.12.8 | Panels + Quality Gate |
---
## v0.13.x — Reference Libraries + Extensions
Core libraries and remaining extensions. `llm-bridge` is the key
delivery — extends both notes and chat through composability primitives,
and introduces the tool meta-tool pattern.
| Version | Title |
|---------|-------|
| v0.13.0 | `vector-store` Library |
| v0.13.1 | `llm-bridge` Core Library |
| v0.13.2 | `llm-bridge` → Chat: Multi-Persona Context + Tool Meta-Tool |
| v0.13.3 | `llm-bridge` → Notes: AI Toolbar Actions |
| v0.13.4 | `file-share` Extension |
| v0.13.5 | `code-workspace` Extension |
| v0.13.6 | `image-gen` + `image-edit` Extensions |
| v0.13.7 | Tool Meta-Tool Hardening + Scoping |
| v0.13.8 | Integration Quality Gate |
**v0.13.1 — `llm-bridge` Core:**
Model abstraction, provider BYOK via connections, `complete()`,
`embed()`, `classify()`. Persona CRUD (name, avatar, system_prompt,
visibility). Layered prompt architecture (6 layers: admin safety →
folder context → persona identity → conversation → tools → user).
Admin safety rails as non-overridable platform setting.
**v0.13.2 — Chat Integration:**
Contributes folder attributes (system_prompt, context_policy,
model_override, allowed_tools) to chat. Contributes "Invite AI" /
"Dismiss AI" to `chat:participant-actions`. Contributes "AI Reply" /
"AI Summarize" to `chat:message-actions`. Contributes "Ask AI" to
`chat:composer-tools`. Context archetype implementation (resident /
scoped / stateless). Session tracking, gap handling, memory extraction
on dismiss. **Tool meta-tool v1:** `sw.actions.list()` → LLM tool
definitions. AI calls extension actions, results flow back into chat.
**v0.13.3 — Notes Integration:**
Contributes "AI Summarize" / "AI Translate" / "AI Fix Grammar" to
`notes:toolbar-actions`. Contributes `/ai` to `notes:slash-commands`.
Uses `notes.get` / `notes.search` actions for context.
**v0.13.7 — Tool Meta-Tool Hardening:**
Three-tier tool scoping: admin blocklist (global), folder allowed_tools
(space-level), persona allow/deny (character-level). Tool result
rendering through `sw.renderers`. Rate limiting on tool calls. Audit
logging of tool use. Error handling (tool failure → graceful message).
**v0.13.8 — Integration Quality Gate:**
AI participant lifecycle tested end-to-end. Tool meta-tool demonstrated
with 3+ extensions. All three context archetypes tested. Persona
creation/sharing across visibility levels. Admin safety rails validated
(prompt injection resistance). Performance: completion latency measured.
---
## v0.14.x — Sidecar Tier + Polish
| Version | Title |
|---------|-------|
| v0.14.0 | Sidecar Tier |
| v0.14.1 | Native Dialog Audit |
| v0.14.2 | Stability + Migration Tooling |
---
## v1.0.0 Gate Criteria
- Notes reference extension shipped (all v0.11.x)
- Chat reference extension shipped (all v0.12.x)
- `llm-bridge` extends both notes and chat through composability
- Tool meta-tool demonstrated: AI uses 3+ extension actions in a
single conversation turn
- Notes and chat UI reviewed against design principles
- At least 2 panels consumed cross-package
- At least 2 slot contributions per host surface demonstrated
- Multi-persona context archetypes demonstrated
- Admin safety rails validated
- Note and conversation sharing functional end-to-end
- Headless E2E green on PG + SQLite
- All kernel Starlark modules documented
- All API routes covered by OpenAPI spec
- Upgrade path tested from v0.8.0 → v1.0.0
- Single-binary + Docker + K8s deployment paths documented
---
## Design Decisions Log
| Decision | Rationale |
|----------|-----------|
| Panels as kernel primitive (v0.10.x) | Z-index coordination, drag/resize, layout negotiation are kernel concerns. |
| UI/UX redesign as first version in each reference series | Every feature builds on the visual foundation. |
| Notes before chat | Notes is simpler (no realtime) and proves storage/rendering/composability. Chat adds realtime + llm-bridge story. |
| Chat human-to-human first | AI is an extension concern. Keeps chat testable and usable standalone. |
| Folder attributes as extension bridge | Chat stores attributes it doesn't understand. llm-bridge contributes definitions. Zero coupling. |
| Action registry as tool registry (meta-tool) | Dynamic, zero-config AI tool-use. Installed extensions = AI capabilities. Uniquely Armature. |
| Layered prompt architecture (6 layers) | Admin safety not overridable. Clear separation: platform → space → character → context → tools → user. |
| Personas in llm-bridge, not chat | Chat sees AI as just another participant_type. Persona identity is llm-bridge's concern. |
| Background images via CSS filter | No server-side processing. `filter: blur() brightness() saturate()` + `opacity` handles any image. |
| Soft panel deps for notes↔chat | Runtime-resolved, no circular dependency. Both function independently. |
| llm-bridge after notes + chat | Proves the composability hooks work without being designed for a specific consumer. |

157
docs/ROADMAP-v010x-v100.md
View File

@@ -1,157 +0,0 @@
# ROADMAP — v0.10.x → v1.0.0 (Final)
## Full Roadmap
| Series | Title | Versions | Design Doc |
|--------|-------|----------|------------|
| v0.9.x | Multi-Surface + Workflow Redesign | 10 | — |
| **v0.10.x** | **Panels + Composable Layout** | 5 | `DESIGN-panels.md` |
| **v0.11.x** | **Notes Reference Extension** | 11 | `DESIGN-notes-v011x.md` |
| **v0.12.x** | **Chat Reference Extension** | 9 | `DESIGN-chat-v012x.md` |
| **v0.13.x** | **Reference Libraries + Extensions** | 9 | — |
| **v0.14.x** | **Sidecar Tier** | 6 | `DESIGN-sidecar-v014x.md` |
| **v0.15.x** | **Polish + Stability** | 3 | — |
| **v1.0.0** | **Stable Release** | — | — |
**Total: ~53 versions from v0.10.0 to v1.0.0**
---
## v0.10.x — Panels + Composable Layout
| Version | Title |
|---------|-------|
| v0.10.0 | Panel Manifest + Lifecycle |
| v0.10.1 | FloatingPanel Primitive |
| v0.10.2 | Docked Panels + Mode Transitions |
| v0.10.3 | Panel Communication Patterns |
| v0.10.4 | Reference Panel: Notes (basic) |
---
## v0.11.x — Notes Reference Extension
| Version | Title |
|---------|-------|
| v0.11.0 | UI/UX Foundation |
| v0.11.1 | Deep Folders + Navigation |
| v0.11.2 | Wikilinks + Backlinks |
| v0.11.3 | Live Preview + Rich Editing |
| v0.11.4 | Note Sharing + Permissions |
| v0.11.5 | Graph + Outline Hardening |
| v0.11.6 | Quick Switcher + Commands |
| v0.11.7 | Daily Notes + Templates |
| v0.11.8 | Transclusion + Embeds |
| v0.11.9 | Composability: Slots + Actions |
| v0.11.10 | Panel Enhancement + Quality Gate |
---
## v0.12.x — Chat Reference Extension
| Version | Title |
|---------|-------|
| v0.12.0 | UI/UX Foundation |
| v0.12.1 | Conversation Folders + Attributes |
| v0.12.2 | Reactions + Threads + Pins |
| v0.12.3 | Rich Compose + Attachments |
| v0.12.4 | @Mentions + Notifications |
| v0.12.5 | Link Previews + Message Formatting |
| v0.12.6 | Conversation Themes + Personality |
| v0.12.7 | Composability: Slots + Actions |
| v0.12.8 | Panels + Quality Gate |
---
## v0.13.x — Reference Libraries + Extensions
| Version | Title |
|---------|-------|
| v0.13.0 | `vector-store` Library |
| v0.13.1 | `llm-bridge` Core Library |
| v0.13.2 | `llm-bridge` → Chat: Multi-Persona + Tool Meta-Tool |
| v0.13.3 | `llm-bridge` → Notes: AI Toolbar Actions |
| v0.13.4 | `file-share` Extension |
| v0.13.5 | `code-workspace` Extension |
| v0.13.6 | `image-gen` + `image-edit` Extensions |
| v0.13.7 | Tool Meta-Tool Hardening + Scoping |
| v0.13.8 | Integration Quality Gate |
---
## v0.14.x — Sidecar Tier
Connect-inward model. Instance sidecars (shared infrastructure) and
user sidecars (personal local tools).
| Version | Title |
|---------|-------|
| v0.14.0 | Sidecar Registry + Auth |
| v0.14.1 | Capability Registration + Execution |
| v0.14.2 | Kernel API Access + Event Bus |
| v0.14.3 | Manifest Integration + Admin Polish |
| v0.14.4 | Reference Sidecar (`armature-embed`) + Instance Gate |
| v0.14.5 | User Sidecars + Reference (`user-bridge`) + User Gate |
---
## v0.15.x — Polish + Stability
| Version | Title |
|---------|-------|
| v0.15.0 | Native Dialog Audit |
| v0.15.1 | Versioned Migrations + `armature migrate` CLI |
| v0.15.2 | Pre-1.0 Schema Freeze + Upgrade Path Validation |
---
## v1.0.0 Gate Criteria
**Reference extensions:**
- Notes shipped (all v0.11.x) with UI quality review
- Chat shipped (all v0.12.x) with UI quality review
- At least 3 reference extensions beyond notes/chat
**Composability:**
- `llm-bridge` extends both notes and chat through slots/actions
- Tool meta-tool: AI uses 3+ extension actions in a single turn
- At least 2 panels consumed cross-package
- At least 2 slot contributions per host surface
**Sidecar:**
- Instance sidecar (`armature-embed`) deployed and functional
- User sidecar (`user-bridge`) functional on macOS/Linux/Windows
- Token + mTLS auth both tested
- User sidecar RBAC enforced (permission, capability allowlist, scoping)
- Tool meta-tool includes user sidecar capabilities per-user
**Infrastructure:**
- Admin safety rails validated
- Multi-persona context archetypes demonstrated
- Sharing functional (notes + conversations)
- Headless E2E green on PG + SQLite
- All kernel Starlark modules documented
- All API routes covered by OpenAPI spec
- `armature migrate` CLI functional
- Upgrade path tested v0.8.0 → v1.0.0
- Single-binary + Docker + K8s deployment paths documented
- Monitoring dashboard with kernel + sidecar metrics
---
## Design Decisions Log
| Decision | Rationale |
|----------|-----------|
| Panels as kernel primitive | Z-index, drag, layout are kernel concerns |
| UI/UX-first for reference extensions | Every feature builds on visual foundation |
| Notes → Chat → Libraries → Sidecar | Each builds on proven patterns from previous |
| Chat human-to-human first | AI is llm-bridge's job |
| Folder attributes as extension bridge | Zero coupling between chat and llm-bridge |
| Action registry as tool registry | Installed extensions = AI capabilities |
| 6-layer prompt architecture | Admin safety not overridable |
| Sidecar connect-inward | No K8s RBAC, no service mesh, no DNS discovery |
| Sidecar HTTP/JSON, not gRPC | KISS. 4-endpoint contract |
| User sidecars in v0.14.5 | Prove instance contract first, extend to users with same mechanism |
| Three-layer user sidecar RBAC | Admin controls who + what, user controls visibility |
| Separate polish (v0.15.x) | Security-critical infra and quality-of-life shouldn't share focus |

476
docs/STARLARK-REFERENCE.md
View File

@@ -1,476 +0,0 @@
# Starlark Reference
Armature extensions can include Starlark scripts for server-side logic.
Starlark is a Python-like language designed for configuration and
embedding — see the [official spec](https://github.com/google/starlark-go).
## Sandbox constraints
- **No `while` loops** — use `for` with bounded ranges.
- **No `load()`** — use `lib.require()` for library dependencies.
- **Max steps:** 1,000,000 bytecode operations per execution.
- **No filesystem or OS access** — all I/O goes through gated modules.
- **Deterministic** — same input produces same output (no `random`, no `time`).
## Always-available modules
These modules are injected into every script with no permission required.
### json
Standard Starlark JSON module.
```python
data = json.decode('{"key": "value"}')
text = json.encode({"key": "value"})
```
### settings
Read resolved package settings (global → team → user cascade).
```python
val = settings.get("theme", "light")
# Returns the resolved value, or the default if unset.
```
The cascade respects the `user_overridable` flag from the package manifest.
See [Permissions & Groups](PERMISSIONS-AND-GROUPS) for details.
```python
# Check if a runtime capability is available
if settings.has_capability("pgvector"):
# Use native vector search
...
```
`has_capability(name)` returns `True` if the named environment capability
is detected by the kernel. Detected capabilities: `pgvector`, `workspace`,
`object_storage`, `s3`, `postgres`.
### lib
Load exported functions from other packages.
```python
helpers = lib.require("my-utils")
result = helpers.format_date("2026-01-15")
```
Requirements:
- The target package must be declared in your manifest's `depends` array.
- The target package must declare `exports` in its manifest.
- The target package must be status `active`, tier `starlark`.
- Any package type (`library`, `extension`, `full`) can be called as long
as it declares exports. This enables full packages to expose callable
functions alongside their UI and API routes.
- Circular dependencies are detected and rejected.
- Results are cached per execution (calling `require` twice returns the
same object).
- The called function runs with the *target* package's permissions, not
the caller's.
### permissions
Check whether a user has a specific permission.
```python
if permissions.check(user_id, "image-gen.use"):
# User is authorized
...
```
Returns `True` if the user has the permission, `False` otherwise (including
when the user is not found). Resolves the user's groups and merges granted
permissions — works for both kernel and extension-declared permissions.
### routing
Generic rule-based decision engine. Evaluates an ordered list of conditions
against a data dict, returning the first matching rule's target string.
```python
result = routing.evaluate([
{"field": "priority", "op": "eq", "value": "critical", "target": "escalation"},
{"field": "amount", "op": "gt", "value": 10000, "target": "manager_review"},
{"field": "region", "op": "in", "value": ["EU", "UK"], "target": "gdpr_flow"},
], stage_data)
# Returns "escalation", "manager_review", "gdpr_flow", or None
```
Each rule is a dict with `field`, `op`, `value`, and `target`. Operators:
`exists`, `not_exists`, `eq`, `neq`, `gt`, `lt`, `gte`, `lte`, `in`,
`contains`. First-match-wins; returns `None` if no rule matches.
## Permission-gated modules
These modules are only available if the package has the corresponding
permission granted in **Admin > Packages**.
### secrets
**Permission:** `secrets.read`
Read admin-configured secrets for this package.
```python
api_key = secrets.get("OPENAI_KEY") # str or None
all_keys = secrets.list() # list of key names
```
Secrets are set in **Admin > Packages > Secrets** and scoped per package.
### notifications
**Permission:** `notifications.send`
Send in-app notifications to users.
```python
notifications.send(
user_id, # str — target user UUID
title, # str — notification title
body="", # str — optional body text
type="extension.notify" # str — notification type
)
```
### db
**Permission:** `db.read` (queries) or `db.write` (mutations)
Read and write extension data tables. All tables are automatically
namespaced as `ext_{package_id}_{table_name}`.
#### Read operations
```python
# Query with filters, ordering, and pagination
rows = db.query(
"tasks", # table name (without prefix)
filters={"status": "open"}, # equality WHERE clauses
order="-created_at", # column name (prefix - for DESC)
limit=50, # max 1000
before={"created_at": ts}, # range: column < value
after={"created_at": ts}, # range: column > value
search_like={"title": "%bug%"} # LIKE/ILIKE search
)
# Read from system views (read-only)
users = db.view("users", filters={"display_name": "Alice"}, limit=10)
channels = db.view("channels", limit=100)
# List all tables owned by this package
tables = db.list_tables()
```
Available views: `users`, `channels`.
#### Aggregate operations
```python
# Count rows matching filters
count = db.count("tasks", filters={"status": "open"})
# Aggregate a column (sum, avg, min, max, count)
total = db.aggregate("orders", "amount", "sum", filters={"status": "paid"})
# Returns int, float, or None (if no matching rows)
# Batch multiple queries in a single call
results = db.query_batch([
{"table": "tasks", "filters": {"status": "open"}, "limit": 10},
{"table": "logs", "order": "-created_at", "limit": 5},
])
# Returns list of result lists. Max 10 queries per batch.
# Each query spec supports: table (required), filters, order, limit, before, after, search_like
```
#### Vector similarity search
```python
# Find rows with the most similar embeddings (cosine distance)
rows = db.query_similar(
"documents", # table name
"embedding", # vector column name
vector=[0.1, 0.2, ...], # query vector (list of floats)
limit=10, # max results (default 10, max 100)
filters={"active": True}, # optional equality filters
metric="cosine", # only "cosine" supported
)
# Returns rows ordered by ascending _distance (0.0 = identical, 1.0 = orthogonal)
# Each row dict includes an injected "_distance" float key.
```
Vector columns are declared as `"vector(N)"` in the manifest `db_tables` block
(N = dimension, 1–4096). Storage varies by backend:
| Backend | Column type | Search |
|---------|-------------|--------|
| Postgres + pgvector | `vector(N)` with HNSW index | Native `<=>` operator |
| Postgres (no pgvector) | `JSONB` | Go-side cosine computation |
| SQLite | `TEXT` | Go-side cosine computation |
Insert vectors as lists: `db.insert("docs", {"embedding": [0.1, 0.2, 0.3]})`.
#### Write operations
```python
row = db.insert("tasks", {"title": "Fix bug", "status": "open"})
# Returns the inserted row dict (with generated id, created_at)
db.update("tasks", row_id, {"status": "closed"})
# Returns True on success
db.delete("tasks", row_id)
# Returns True on success
```
### http
**Permission:** `api.http`
Make outbound HTTP requests.
```python
resp = http.get("https://api.example.com/data", headers={"Authorization": "Bearer ..."})
resp = http.post(url, body='{"key": "val"}', headers={"Content-Type": "application/json"})
resp = http.put(url, body="...", headers={})
resp = http.delete(url, headers={})
resp = http.request("PATCH", url, body="...", headers={})
```
Response dict:
```python
{
"status": 200,
"headers": {"content-type": "application/json"},
"body": "..." # capped at 1 MB
}
```
#### Batch requests
```python
responses = http.batch([
{"method": "GET", "url": "https://api.example.com/a"},
{"method": "POST", "url": "https://api.example.com/b", "body": "{}", "headers": {"Content-Type": "application/json"}},
])
# Returns list of response dicts (same shape as individual calls).
# Individual failures return {"status": 0, "body": "error: ...", "headers": {}}.
# Max 10 requests per batch. Dispatched concurrently.
```
**Security:**
- Private/loopback IPs are blocked (SSRF protection).
- Packages can declare `network_access.allow` (allowlist) or
`network_access.block` (blocklist) in their manifest.
- Max 10 redirects. 10-second timeout. 1 MB response body limit.
### realtime
**Permission:** `realtime.publish`
Publish WebSocket events to subscribed clients.
```python
realtime.publish(
"my-channel", # channel name
"item.updated", # event label
{"id": "abc"} # payload dict (max 7 KB)
)
```
The payload is automatically tagged with `_pkg: package_id`.
### connections
**Permission:** `connections.read`
Read external connection configurations (secrets are decrypted).
```python
conn = connections.get("postgres", "main-db")
# Returns dict with id, type, name, scope, plus flattened config fields
# Returns None if not found
all_pg = connections.list("postgres")
# Returns list of connection dicts
```
Connections are resolved via scope chain: personal → team → global.
### workflow
**Permission:** `workflow.access`
Read workflow definitions and instances (read-only from Starlark;
mutations go through the HTTP API).
```python
defn = workflow.get_definition(workflow_id)
# Returns dict: id, name, slug, entry_mode, is_active, version, stages[]
inst = workflow.get_instance(instance_id)
# Returns dict: id, workflow_id, current_stage, status, stage_data, ...
instances = workflow.list_instances(workflow_id, status="active")
# Returns list of instance dicts
```
### batch
**Permission:** `batch.exec`
Run multiple callables concurrently. Each callable gets its own
execution thread with an independent step budget.
```python
jira = lib.require("jira-client")
confluence = lib.require("confluence-client")
results, errors = batch.exec([
lambda: jira.create_issue(issue_data),
lambda: confluence.create_page(page_data),
lambda: send_notification(user_id),
], timeout=15)
# results[i] = return value of callables[i], or None on error
# errors[i] = None on success, or error string on failure
# All three ran concurrently.
```
**Constraints:**
- Max 8 callables per call. Dispatched concurrently via goroutines.
- `timeout` (optional): 1–30 seconds per branch (default 10).
- `lib.require()` is not available inside branch callables.
Load libraries before the `batch.exec` call.
- `batch.exec()` cannot be called from within a branch (no nesting).
- `print()` output from branches is discarded.
---
### files
**Permissions:** `files.read`, `files.write`
Store and retrieve files via the kernel ObjectStore (PVC or S3).
All keys are scoped to `ext/{packageID}/` — extensions cannot access
each other's files.
```python
# Store a file with optional metadata
files.put("reports/q1.pdf", pdf_bytes,
content_type="application/pdf",
metadata={"quarter": "Q1", "year": 2026})
# Read a file
result = files.get("reports/q1.pdf")
# result = {"content": b"...", "content_type": "application/pdf",
# "size": 12345, "metadata": {"quarter": "Q1", "year": 2026}}
# Metadata only (no content transfer)
meta = files.meta("reports/q1.pdf")
# List files by prefix
entries = files.list(prefix="reports/", limit=50)
# entries = [{"name": "reports/q1.pdf", "size": 12345, "content_type": "..."}]
# Check existence
if files.exists("reports/q1.pdf"):
files.delete("reports/q1.pdf")
# Bulk delete
files.delete_prefix("temp/")
```
**Notes:**
- `content` accepts string or bytes; `get()` returns bytes.
- Maximum file size: 50 MB (configurable via `EXT_FILES_MAX_SIZE`).
- Metadata is stored as a companion JSON object, not in the file itself.
- `files.list()` automatically filters out internal metadata companions.
---
### workspace
**Permission:** `workspace.manage`
Managed disk directories for extensions that need a real filesystem
(git clones, compilers, media tools). Each workspace is scoped to
`{WORKSPACE_ROOT}/{packageID}/{name}/`.
```python
# Create a workspace (idempotent)
path = workspace.create("my-repo")
# Returns the absolute path to the directory
# Get the path (None if workspace doesn't exist)
path = workspace.path("my-repo")
# List all workspaces owned by this extension
names = workspace.list() # ["my-repo", "cache"]
# Delete a workspace and all its contents
workspace.delete("my-repo")
# Get disk usage in bytes (10-second timeout)
size = workspace.usage("my-repo") # 1048576
```
**Constraints:**
- Names must match `^[a-z][a-z0-9_]{0,62}$` (lowercase, no spaces or separators).
- Path traversal and symlink escape are blocked.
- Quota enforcement via `WORKSPACE_QUOTA_MB` env var (0 = unlimited).
- Module not available if `WORKSPACE_ROOT` is unset or not writable.
Use `settings.has_capability("workspace")` to check availability.
---
## Example: automated stage hook
A simple hook that reads a setting, queries data, and advances:
```python
def on_run(ctx):
threshold = settings.get("approval_threshold", 1000)
amount = ctx["stage_data"].get("amount", 0)
if amount > threshold:
notifications.send(
ctx["started_by"],
"High-value submission",
body="Amount %d exceeds threshold." % amount,
)
return {"advance": True, "data": {"needs_review": True}}
return {"advance": True, "data": {"needs_review": False}}
```
---
## `forms` Module (v0.9.5)
**Permission:** `forms.validate`
Validates form data against a typed form template.
### `forms.validate(template, data)`
Validates `data` (a dict) against a `template` (a dict matching the TypedFormTemplate schema).
Returns a dict: `{"valid": True/False, "errors": [{"key": "...", "message": "..."}]}`.
```python
result = forms.validate(
{"fields": [{"key": "name", "type": "text", "label": "Name", "required": True}]},
{"name": "Alice"},
)
# result["valid"] == True
# result["errors"] == []
```
Field types: `text`, `email`, `select`, `number`, `date`, `textarea`, `checkbox`, `file`.
Supports: required checks, min/max length, pattern (regex), number range, date range, select option whitelist, conditional visibility (`condition.when`/`op`/`value`).

119
docs/TUTORIAL-FIRST-EXTENSION.md
View File

@@ -1,119 +0,0 @@
# Build Your First Browser Extension
This tutorial walks through building a browser extension that renders custom
code blocks, modeled on the CSV Table Viewer that ships with Armature.
**Prerequisites:** A running Armature instance, a text editor, and `zip`.
## Step 1: Create the Directory
```sh
mkdir -p my-extension/js
```
## Step 2: Write the Manifest
Create `my-extension/manifest.json`:
```json
{
"id": "my-extension",
"title": "My Extension",
"version": "0.1.0",
"type": "extension",
"tier": "browser",
"author": "you",
"description": "Renders demo code blocks as styled HTML",
"permissions": [],
"settings": {}
}
```
- **id** -- unique identifier, used as the install key
- **type** -- `extension` for browser-only packages; `full` or `surface` for
packages with backend routes
- **tier** -- `browser` for client-side JS; `starlark` for server-side scripting
## Step 3: Write the Browser Script
Create `my-extension/js/script.js`. Browser extensions use the IIFE pattern
and register with the SDK through `sw.renderers`:
```js
(function () {
'use strict';
function register() {
if (!window.sw?.renderers) return;
sw.renderers.register('demo-block', {
type: 'block',
priority: 10,
match(lang) {
return (lang || '').toLowerCase() === 'demo';
},
render(lang, code, container) {
container.innerHTML =
'<div style="padding:12px;background:var(--bg-secondary);' +
'border:1px solid var(--border);border-radius:8px">' +
'<strong>Demo:</strong> ' + code +
'</div>';
}
});
}
if (window.sw?._sdk) {
register();
} else {
document.addEventListener('sw:ready', register, { once: true });
}
})();
```
The IIFE wrapper keeps variables out of global scope. `sw.renderers.register`
takes a name and an options object: `type: 'block'` targets fenced code blocks,
`match` checks the language tag, and `render` receives the language, raw code,
and a container element. The `sw:ready` event fires once the SDK initializes;
if already loaded, register immediately. Use CSS variables like `var(--bg-secondary)`
and `var(--border)` to follow the active theme.
## Step 4: Package It
```sh
cd my-extension
zip -r ../my-extension.pkg manifest.json js/
```
The `.pkg` format is a ZIP with `manifest.json` at the root. Optional
directories: `js/`, `css/`, `assets/`. If working inside `packages/`, use
the build script instead: `bash build.sh my-extension`.
## Step 5: Install It
```sh
curl -X POST http://localhost:3000/api/v1/admin/packages/install \
-H "Authorization: Bearer <token>" \
-F "file=@my-extension.pkg"
```
You can also install through the Admin UI under the Packages section.
## Step 6: Enable and Test
1. Open the admin panel, navigate to Packages, confirm "My Extension" is enabled.
2. Go to any markdown surface (Chat, Notes, etc.).
3. Enter a fenced code block with the `demo` language tag.
You should see styled output instead of a plain code block.
## Going Further
- **Add CSS** -- create a `css/` directory; stylesheets are injected automatically.
- **Post-renderers** -- register with `type: 'post'` to run after block renderers
finish (the Mermaid extension uses this for async SVG rendering).
- **Settings** -- declare a `settings` object in the manifest for admin-configurable values.
- **Server-side logic** -- set `tier: "starlark"` and add `script.star` for
backend API routes and database tables.
See `PACKAGE-FORMAT.md` for the full manifest spec and `EXTENSION-GUIDE.md`
for advanced patterns.

193
docs/WORKFLOWS.md
View File

@@ -1,193 +0,0 @@
# Workflows
Workflows are multi-stage processes with team assignment, validation gates,
SLA enforcement, and optional Starlark automation. They are managed in
**Team Admin > Workflows**.
## Core concepts
| Concept | Description |
|---------|-------------|
| **Definition** | A named template: stages, entry mode, staleness timeout. Created per-team or adopted from global definitions. |
| **Stage** | One step in the workflow. Has a mode, audience, optional team assignment, and optional SLA. |
| **Instance** | A running copy of a definition. Pins a published version snapshot and tracks accumulated stage data. |
| **Assignment** | A queue entry linking an instance stage to a team member. Claim → work → complete. |
| **Signoff** | An approval or rejection recorded against an instance stage (multi-party validation). |
## Entry modes
| Mode | Description |
|------|-------------|
| `team_only` | Only authenticated team members can start instances. |
| `public_link` | Anyone with the public URL can start an instance. The first stage must have `audience: public`. An `entry_token` is issued for the anonymous submitter to resume later. |
Public entry URL format:
```
{origin}/api/v1/public/workflows/{workflow_id}/start
```
## Stage modes
Each stage has a **mode** that determines how it progresses:
| Mode | Description |
|------|-------------|
| `form` | User submits structured data. Stage data is accumulated into the instance. |
| `review` | Multi-party sign-off gate. Requires configured approvals before advancing. |
| `delegated` | Assigned to a team member queue. The assignee claims, works, and completes. |
| `automated` | Starlark hook executes without user interaction. Can chain up to 10 consecutive automated stages. |
## Stage types
| Type | Description |
|------|-------------|
| `simple` | Linear — always advances to the next ordinal. |
| `dynamic` | Conditional — evaluates branch rules against stage data to pick the next stage. |
| `automated` | Combined with mode `automated` for fully scripted stages. |
## Audiences
| Audience | Description |
|----------|-------------|
| `team` | Only authenticated team members can interact. |
| `public` | Anonymous users can interact (used with `public_link` entry). |
| `system` | System-generated stages, no direct user interaction. |
## Team assignment
When a stage has `assignment_team_id` set, the engine creates an
**assignment** record:
1. Assignment enters the queue with status `unassigned`.
2. A team member **claims** the assignment (status → `claimed`).
3. The assignee works the stage and **completes** it (status → `completed`).
4. The engine auto-advances to the next stage.
A **required role** can restrict who may claim:
- Set `stage_config.required_role` to a team role name (e.g. `"reviewer"`).
- Only members with that role can claim the assignment.
Team roles are configured in **Team Admin > Settings > Roles**.
## Signoff gates (multi-party validation)
Review-mode stages can require multiple approvals before advancing.
Configure via `stage_config.validation`:
```json
{
"validation": {
"required_approvals": 2,
"required_role": "approver",
"reject_action": "cancel"
}
}
```
| Field | Description |
|-------|-------------|
| `required_approvals` | Minimum approve decisions needed to advance. |
| `required_role` | Only members with this team role can sign off. Empty = any member. |
| `reject_action` | What happens on rejection: `"cancel"` (default) cancels the instance, or a stage name to reroute. |
Each signoff records: user, decision (`approve` or `reject`), optional comment, timestamp.
## SLA enforcement
Two timeout mechanisms run in a background scanner (every 5 minutes):
### Per-stage SLA
Set `sla_seconds` on a stage. When an instance has been in that stage
longer than the threshold:
- `sla_breached` flag is set in instance metadata.
- A `workflow.sla_breach` WebSocket event is emitted.
- The instance is **not** auto-cancelled — breaches are informational.
### Per-workflow staleness
Set `staleness_timeout_hours` on the workflow definition. When an instance
has not been updated for longer than the threshold:
- Instance status is set to `stale`.
- All open assignments are cancelled.
- A `workflow.stale` WebSocket event is emitted.
## Branch rules
Dynamic stages evaluate conditions against accumulated `stage_data`
to determine the next stage. Rules are a JSON array on the stage:
```json
[
{ "field": "priority", "op": "eq", "value": "high", "target_stage": "escalation" },
{ "field": "amount", "op": "gt", "value": 10000, "target_stage": "manager-review" }
]
```
First matching rule wins. If no rules match, the next ordinal stage is used.
### Operators
| Op | Description |
|----|-------------|
| `eq` | Equal (string-normalized) |
| `neq` | Not equal |
| `gt`, `lt`, `gte`, `lte` | Numeric comparisons |
| `exists` | Field is present in stage data |
| `not_exists` | Field is absent |
| `in` | Value is in a list |
| `contains` | String contains substring |
`target_stage` can be a stage name (case-insensitive) or a numeric ordinal.
## Publishing
Workflows have a draft/publish lifecycle:
1. Edit stages and configuration in the workflow editor (draft state).
2. **Publish** creates a versioned snapshot of all stages.
3. New instances pin the latest published version.
4. Editing stages after publishing does not affect running instances.
Version numbers auto-increment. The snapshot preserves the complete
stage definition array at publish time.
## Starlark hooks
Automated stages execute a Starlark script via the `starlark_hook` field:
```
package_id:entry_point
```
For example: `my-automation:on_review` calls the `on_review` function
in the `my-automation` package. If no entry point is specified,
`on_run` is used.
The hook receives a context dict:
```python
{
"instance_id": "...",
"current_stage": "...",
"workflow_id": "...",
"started_by": "...",
"stage_data": { ... }
}
```
The hook returns a dict controlling what happens next:
| Key | Effect |
|-----|--------|
| `advance: True` | Auto-advance to the next stage |
| `data: { ... }` | Merge into stage data for the next stage |
| `error: "msg"` | Set instance status to `error` and halt |
Up to 10 consecutive automated stages can chain before the engine
stops with an error (cycle guard).
See the [Starlark Reference](STARLARK-REFERENCE) for available
sandbox modules.

6
packages/csv-table/manifest.json → extensions/builtin/csv-table/manifest.json
View File

@@ -1,12 +1,10 @@
{
"id": "csv-table",
"title": "CSV Table Viewer",
"name": "CSV Table Viewer",
"version": "1.0.0",
"type": "extension",
"tier": "browser",
"author": "armature",
"author": "switchboard",
"description": "Renders ```csv code blocks as sortable, interactive HTML tables",
"requires": [],
"permissions": [],
"tools": [],
"surfaces": [],

273
packages/csv-table/js/script.js → extensions/builtin/csv-table/script.js
View File

@@ -3,68 +3,71 @@
// ==========================================
// Renders ```csv and ```tsv code blocks as sortable HTML tables.
// No external dependencies.
//
// Registers with sw.renderers via the sw:ready event.
// ==========================================
(function () {
'use strict';
Extensions.register({
id: 'csv-table',
function _escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
async init(ctx) {
const self = this;
function _parseCSV(text, delimiter) {
const rows = [];
let row = [];
let field = '';
let inQuotes = false;
let i = 0;
// ── Inject styles ──
this._injectStyles();
while (i < text.length) {
const ch = text[i];
if (inQuotes) {
if (ch === '"') {
if (i + 1 < text.length && text[i + 1] === '"') {
field += '"'; i += 2;
} else {
inQuotes = false; i++;
}
} else {
field += ch; i++;
}
} else {
if (ch === '"' && field === '') {
inQuotes = true; i++;
} else if (ch === delimiter) {
row.push(field.trim()); field = ''; i++;
} else if (ch === '\n' || (ch === '\r' && text[i + 1] === '\n')) {
row.push(field.trim());
if (row.some(cell => cell !== '')) rows.push(row);
row = []; field = '';
i += (ch === '\r') ? 2 : 1;
} else {
field += ch; i++;
ctx.renderers.register('csv-block', {
type: 'block',
priority: 10,
match(lang) {
const l = (lang || '').toLowerCase();
return l === 'csv' || l === 'tsv';
},
render(lang, code, container) {
const delimiter = lang.toLowerCase() === 'tsv' ? '\t' : ',';
const rows = self._parseCSV(code.trim(), delimiter);
if (rows.length === 0) {
container.innerHTML = '<div class="csv-empty">No data</div>';
return;
}
const headers = rows[0];
const data = rows.slice(1);
const tableId = 'csv-' + Math.random().toString(36).slice(2, 9);
container.innerHTML = `
<div class="csv-table-block ext-rendered" data-csv-id="${tableId}">
<div class="csv-toolbar">
<span class="csv-info">${data.length} row${data.length !== 1 ? 's' : ''} × ${headers.length} col${headers.length !== 1 ? 's' : ''}</span>
<button class="csv-copy-btn" title="Copy as CSV">📋 Copy</button>
</div>
<div class="csv-table-scroll">
<table class="csv-table">
<thead>
<tr>${headers.map((h, i) => `<th data-col="${i}" title="Click to sort">${self._escapeHtml(h)}<span class="csv-sort-icon"></span></th>`).join('')}</tr>
</thead>
<tbody>
${data.map(row => `<tr>${row.map(cell => `<td>${self._escapeHtml(cell)}</td>`).join('')}</tr>`).join('')}
</tbody>
</table>
</div>
<details class="csv-source">
<summary>📄 View raw</summary>
<pre><code>${self._escapeHtml(code.trim())}</code></pre>
</details>
</div>
`;
// Attach sort handlers + copy
self._attachHandlers(container, tableId, code.trim());
}
}
});
},
row.push(field.trim());
if (row.some(cell => cell !== '')) rows.push(row);
if (rows.length > 0) {
const maxCols = Math.max(...rows.map(r => r.length));
rows.forEach(r => { while (r.length < maxCols) r.push(''); });
}
return rows;
}
function _attachHandlers(container, tableId, rawCSV) {
_attachHandlers(container, tableId, rawCSV) {
const block = container.querySelector(`[data-csv-id="${tableId}"]`);
if (!block) return;
// ── Column sorting ──
const ths = block.querySelectorAll('th[data-col]');
let sortCol = -1;
let sortAsc = true;
@@ -73,50 +76,129 @@
th.style.cursor = 'pointer';
th.addEventListener('click', () => {
const col = parseInt(th.dataset.col, 10);
if (sortCol === col) { sortAsc = !sortAsc; }
else { sortCol = col; sortAsc = true; }
if (sortCol === col) {
sortAsc = !sortAsc;
} else {
sortCol = col;
sortAsc = true;
}
const tbody = block.querySelector('tbody');
const rows = Array.from(tbody.querySelectorAll('tr'));
rows.sort((a, b) => {
const cellA = (a.children[col]?.textContent || '').trim();
const cellB = (b.children[col]?.textContent || '').trim();
const numA = parseFloat(cellA);
const numB = parseFloat(cellB);
if (!isNaN(numA) && !isNaN(numB))
// Numeric sort if both are numbers
if (!isNaN(numA) && !isNaN(numB)) {
return sortAsc ? numA - numB : numB - numA;
}
// String sort
const cmp = cellA.localeCompare(cellB, undefined, { numeric: true, sensitivity: 'base' });
return sortAsc ? cmp : -cmp;
});
rows.forEach(row => tbody.appendChild(row));
// Update sort icons
ths.forEach(h => {
const icon = h.querySelector('.csv-sort-icon');
if (icon) {
const hCol = parseInt(h.dataset.col, 10);
icon.textContent = hCol === sortCol ? (sortAsc ? ' \u25b2' : ' \u25bc') : '';
icon.textContent = hCol === sortCol ? (sortAsc ? ' ▲' : ' ▼') : '';
}
});
});
});
// ── Copy button ──
const copyBtn = block.querySelector('.csv-copy-btn');
if (copyBtn) {
copyBtn.addEventListener('click', () => {
navigator.clipboard.writeText(rawCSV).then(() => {
copyBtn.textContent = '\u2713 Copied';
setTimeout(() => { copyBtn.textContent = '\ud83d\udccb Copy'; }, 1500);
copyBtn.textContent = '✓ Copied';
setTimeout(() => { copyBtn.textContent = '📋 Copy'; }, 1500);
}).catch(() => {
copyBtn.textContent = '\u2717 Failed';
setTimeout(() => { copyBtn.textContent = '\ud83d\udccb Copy'; }, 1500);
copyBtn.textContent = '✗ Failed';
setTimeout(() => { copyBtn.textContent = '📋 Copy'; }, 1500);
});
});
}
}
},
// ── Styles ──────────────────────────────
/**
* Simple CSV parser that handles quoted fields.
*/
_parseCSV(text, delimiter) {
const rows = [];
let row = [];
let field = '';
let inQuotes = false;
let i = 0;
function _injectStyles() {
while (i < text.length) {
const ch = text[i];
if (inQuotes) {
if (ch === '"') {
// Peek ahead: escaped quote or end of quoted field
if (i + 1 < text.length && text[i + 1] === '"') {
field += '"';
i += 2;
} else {
inQuotes = false;
i++;
}
} else {
field += ch;
i++;
}
} else {
if (ch === '"' && field === '') {
inQuotes = true;
i++;
} else if (ch === delimiter) {
row.push(field.trim());
field = '';
i++;
} else if (ch === '\n' || (ch === '\r' && text[i + 1] === '\n')) {
row.push(field.trim());
if (row.some(cell => cell !== '')) rows.push(row);
row = [];
field = '';
i += (ch === '\r') ? 2 : 1;
} else {
field += ch;
i++;
}
}
}
// Last field/row
row.push(field.trim());
if (row.some(cell => cell !== '')) rows.push(row);
// Normalize: pad short rows to header length
if (rows.length > 0) {
const maxCols = Math.max(...rows.map(r => r.length));
rows.forEach(r => {
while (r.length < maxCols) r.push('');
});
}
return rows;
},
_escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
},
_injectStyles() {
if (document.getElementById('ext-style-csv-table')) return;
const style = document.createElement('style');
style.id = 'ext-style-csv-table';
@@ -164,66 +246,9 @@
}
.csv-empty { padding: 16px; text-align: center; color: var(--text-3); }`;
document.head.appendChild(style);
},
destroy() {
document.getElementById('ext-style-csv-table')?.remove();
}
// ── Registration ────────────────────────
function register() {
if (!window.sw?.renderers) return;
_injectStyles();
sw.renderers.register('csv-block', {
type: 'block',
priority: 10,
match(lang) {
const l = (lang || '').toLowerCase();
return l === 'csv' || l === 'tsv';
},
render(lang, code, container) {
const delimiter = lang.toLowerCase() === 'tsv' ? '\t' : ',';
const rows = _parseCSV(code.trim(), delimiter);
if (rows.length === 0) {
container.innerHTML = '<div class="csv-empty">No data</div>';
return;
}
const headers = rows[0];
const data = rows.slice(1);
const tableId = 'csv-' + Math.random().toString(36).slice(2, 9);
container.innerHTML = `
<div class="csv-table-block ext-rendered" data-csv-id="${tableId}">
<div class="csv-toolbar">
<span class="csv-info">${data.length} row${data.length !== 1 ? 's' : ''} \u00d7 ${headers.length} col${headers.length !== 1 ? 's' : ''}</span>
<button class="csv-copy-btn" title="Copy as CSV">\ud83d\udccb Copy</button>
</div>
<div class="csv-table-scroll">
<table class="csv-table">
<thead>
<tr>${headers.map((h, i) => `<th data-col="${i}" title="Click to sort">${_escapeHtml(h)}<span class="csv-sort-icon"></span></th>`).join('')}</tr>
</thead>
<tbody>
${data.map(row => `<tr>${row.map(cell => `<td>${_escapeHtml(cell)}</td>`).join('')}</tr>`).join('')}
</tbody>
</table>
</div>
<details class="csv-source">
<summary>\ud83d\udcc4 View raw</summary>
<pre><code>${_escapeHtml(code.trim())}</code></pre>
</details>
</div>
`;
_attachHandlers(container, tableId, code.trim());
}
});
}
if (window.sw?._sdk) {
register();
} else {
document.addEventListener('sw:ready', register, { once: true });
}
})();
});

6
packages/diff-viewer/manifest.json → extensions/builtin/diff-viewer/manifest.json
View File

@@ -1,12 +1,10 @@
{
"id": "diff-viewer",
"title": "Diff Viewer",
"name": "Diff Viewer",
"version": "1.0.0",
"type": "extension",
"tier": "browser",
"author": "armature",
"author": "switchboard",
"description": "Renders ```diff code blocks with syntax-highlighted additions, deletions, and context lines",
"requires": [],
"permissions": [],
"tools": [],
"surfaces": [],

189
packages/diff-viewer/js/script.js → extensions/builtin/diff-viewer/script.js
View File

@@ -4,22 +4,106 @@
// Renders ```diff code blocks with syntax-highlighted
// additions, deletions, hunk headers, and context lines.
// No external dependencies.
//
// Registers with sw.renderers via the sw:ready event.
// ==========================================
(function () {
'use strict';
Extensions.register({
id: 'diff-viewer',
function _escapeHtml(str) {
async init(ctx) {
const self = this;
// ── Inject styles ──
this._injectStyles();
ctx.renderers.register('diff-block', {
type: 'block',
priority: 10,
pattern: 'diff',
render(lang, code, container) {
const lines = code.split('\n');
let stats = { added: 0, removed: 0 };
let currentFile = '';
const rendered = lines.map(line => {
const escaped = self._escapeHtml(line);
// File headers
if (line.startsWith('--- ') || line.startsWith('+++ ')) {
if (line.startsWith('+++ ')) {
currentFile = line.slice(4).trim();
}
return `<div class="diff-line diff-file-header">${escaped}</div>`;
}
// Hunk headers
if (line.startsWith('@@')) {
const hunkMatch = line.match(/^@@\s*-(\d+)(?:,\d+)?\s*\+(\d+)(?:,\d+)?\s*@@\s*(.*)/);
const context = hunkMatch ? hunkMatch[3] : '';
return `<div class="diff-line diff-hunk">${escaped}</div>`;
}
// Additions
if (line.startsWith('+')) {
stats.added++;
return `<div class="diff-line diff-add"><span class="diff-indicator">+</span>${self._escapeHtml(line.slice(1))}</div>`;
}
// Deletions
if (line.startsWith('-')) {
stats.removed++;
return `<div class="diff-line diff-del"><span class="diff-indicator">-</span>${self._escapeHtml(line.slice(1))}</div>`;
}
// Context (lines starting with space or no prefix)
if (line.startsWith(' ')) {
return `<div class="diff-line diff-ctx"><span class="diff-indicator"> </span>${self._escapeHtml(line.slice(1))}</div>`;
}
// Other (commit info, etc)
return `<div class="diff-line diff-meta">${escaped}</div>`;
}).join('');
const fileLabel = currentFile ? `<span class="diff-filename" title="${self._escapeHtml(currentFile)}">${self._escapeHtml(currentFile)}</span>` : '';
container.innerHTML = `
<div class="diff-block ext-rendered">
<div class="diff-toolbar">
${fileLabel}
<span class="diff-stats">
<span class="diff-stat-add">+${stats.added}</span>
<span class="diff-stat-del">-${stats.removed}</span>
</span>
<button class="diff-copy-btn" title="Copy diff">📋 Copy</button>
</div>
<div class="diff-content">${rendered}</div>
<details class="diff-source-toggle">
<summary>📝 View raw</summary>
<pre><code>${self._escapeHtml(code)}</code></pre>
</details>
</div>
`;
// Copy handler
const copyBtn = container.querySelector('.diff-copy-btn');
if (copyBtn) {
copyBtn.addEventListener('click', () => {
navigator.clipboard.writeText(code).then(() => {
copyBtn.textContent = '✓ Copied';
setTimeout(() => { copyBtn.textContent = '📋 Copy'; }, 1500);
}).catch(() => {});
});
}
}
});
},
_escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
},
// ── Styles ──────────────────────────────
function _injectStyles() {
_injectStyles() {
if (document.getElementById('ext-style-diff-viewer')) return;
const style = document.createElement('style');
style.id = 'ext-style-diff-viewer';
@@ -74,86 +158,9 @@
max-height: 200px; overflow: auto;
}`;
document.head.appendChild(style);
},
destroy() {
document.getElementById('ext-style-diff-viewer')?.remove();
}
// ── Registration ────────────────────────
function register() {
if (!window.sw?.renderers) return;
_injectStyles();
sw.renderers.register('diff-block', {
type: 'block',
priority: 10,
pattern: 'diff',
render(lang, code, container) {
const lines = code.split('\n');
let stats = { added: 0, removed: 0 };
let currentFile = '';
const rendered = lines.map(line => {
const escaped = _escapeHtml(line);
if (line.startsWith('--- ') || line.startsWith('+++ ')) {
if (line.startsWith('+++ ')) currentFile = line.slice(4).trim();
return `<div class="diff-line diff-file-header">${escaped}</div>`;
}
if (line.startsWith('@@')) {
return `<div class="diff-line diff-hunk">${escaped}</div>`;
}
if (line.startsWith('+')) {
stats.added++;
return `<div class="diff-line diff-add"><span class="diff-indicator">+</span>${_escapeHtml(line.slice(1))}</div>`;
}
if (line.startsWith('-')) {
stats.removed++;
return `<div class="diff-line diff-del"><span class="diff-indicator">-</span>${_escapeHtml(line.slice(1))}</div>`;
}
if (line.startsWith(' ')) {
return `<div class="diff-line diff-ctx"><span class="diff-indicator"> </span>${_escapeHtml(line.slice(1))}</div>`;
}
return `<div class="diff-line diff-meta">${escaped}</div>`;
}).join('');
const fileLabel = currentFile
? `<span class="diff-filename" title="${_escapeHtml(currentFile)}">${_escapeHtml(currentFile)}</span>`
: '';
container.innerHTML = `
<div class="diff-block ext-rendered">
<div class="diff-toolbar">
${fileLabel}
<span class="diff-stats">
<span class="diff-stat-add">+${stats.added}</span>
<span class="diff-stat-del">-${stats.removed}</span>
</span>
<button class="diff-copy-btn" title="Copy diff">\ud83d\udccb Copy</button>
</div>
<div class="diff-content">${rendered}</div>
<details class="diff-source-toggle">
<summary>\ud83d\udcdd View raw</summary>
<pre><code>${_escapeHtml(code)}</code></pre>
</details>
</div>
`;
const copyBtn = container.querySelector('.diff-copy-btn');
if (copyBtn) {
copyBtn.addEventListener('click', () => {
navigator.clipboard.writeText(code).then(() => {
copyBtn.textContent = '\u2713 Copied';
setTimeout(() => { copyBtn.textContent = '\ud83d\udccb Copy'; }, 1500);
}).catch(() => {});
});
}
}
});
}
if (window.sw?._sdk) {
register();
} else {
document.addEventListener('sw:ready', register, { once: true });
}
})();
});

6
packages/js-sandbox/manifest.json → extensions/builtin/js-sandbox/manifest.json
View File

@@ -1,12 +1,10 @@
{
"id": "js-sandbox",
"title": "JavaScript Sandbox",
"name": "JavaScript Sandbox",
"version": "1.0.0",
"type": "extension",
"tier": "browser",
"author": "armature",
"author": "switchboard",
"description": "Executes JavaScript in a sandboxed iframe. Allows the LLM to run code, verify calculations, and transform data — no server compute required.",
"requires": ["chat"],
"permissions": [],
"tools": [
{

0
packages/js-sandbox/js/script.js → extensions/builtin/js-sandbox/script.js
View File

6
packages/katex-renderer/manifest.json → extensions/builtin/katex-renderer/manifest.json
View File

@@ -1,12 +1,10 @@
{
"id": "katex-renderer",
"title": "KaTeX Math",
"name": "KaTeX Math",
"version": "1.0.0",
"type": "extension",
"tier": "browser",
"author": "armature",
"author": "switchboard",
"description": "Renders LaTeX math expressions: ```latex blocks and inline $...$ / $$...$$ syntax",
"requires": [],
"permissions": [],
"tools": [],
"surfaces": [],

325
packages/katex-renderer/js/script.js → extensions/builtin/katex-renderer/script.js
View File

@@ -3,141 +3,177 @@
// ==========================================
// Renders ```latex / ```math code blocks and inline $...$ / $$...$$ syntax.
// Loads KaTeX dynamically on first use.
//
// Registers with sw.renderers via the sw:ready event.
// ==========================================
(function () {
'use strict';
Extensions.register({
id: 'katex-renderer',
let _katexReady = false;
let _katexLoading = null;
_katexReady: false,
_katexLoading: null,
function _escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
async init(ctx) {
const self = this;
// ── KaTeX Library Loading ──────────────
// ── Inject styles ──
this._injectStyles();
function _loadKaTeX() {
if (_katexReady) return Promise.resolve();
if (_katexLoading) return _katexLoading;
_katexLoading = new Promise((resolve, reject) => {
if (typeof katex !== 'undefined') {
_katexReady = true;
resolve();
return;
// ── Block renderer: match ```latex or ```math ──
ctx.renderers.register('katex-block', {
type: 'block',
priority: 10,
match(lang) {
const l = (lang || '').toLowerCase();
return l === 'latex' || l === 'math' || l === 'tex' || l === 'katex';
},
render(lang, code, container) {
const id = 'katex-' + Math.random().toString(36).slice(2, 9);
container.innerHTML = `
<div class="katex-block ext-rendered" data-katex-id="${id}">
<div class="katex-display-container" data-katex-src="${encodeURIComponent(code.trim())}" data-katex-display="true">
<div class="katex-loading">
<span class="katex-spinner"></span> Rendering math…
</div>
</div>
<details class="katex-source">
<summary>𝑓(𝑥) View source</summary>
<pre><code class="language-latex">${self._escapeHtml(code.trim())}</code></pre>
</details>
</div>
`;
}
const base = (window.__BASE__ || '');
const localCSS = `${base}/vendor/katex/katex.min.css`;
const localJS = `${base}/vendor/katex/katex.min.js`;
const cdnCSS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.css';
const cdnJS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.js';
const link = document.createElement('link');
link.rel = 'stylesheet';
link.href = localCSS;
link.onerror = () => { link.href = cdnCSS; };
document.head.appendChild(link);
const script = document.createElement('script');
script.src = localJS;
script.onload = () => { _katexReady = true; resolve(); };
script.onerror = () => {
console.warn('[KaTeX] Local vendor not found, trying CDN');
const cdn = document.createElement('script');
cdn.src = cdnJS;
cdn.onload = () => { _katexReady = true; resolve(); };
cdn.onerror = () => {
console.error('[KaTeX] Failed to load from both local and CDN');
reject(new Error('Failed to load KaTeX'));
};
document.head.appendChild(cdn);
};
document.head.appendChild(script);
});
return _katexLoading;
}
// ── Block Rendering ──────────────────────
// ── Post renderer: render KaTeX blocks + find inline math ──
ctx.renderers.register('katex-post', {
type: 'post',
priority: 10,
render(container) {
// Render code blocks with data-katex-src
const blocks = container.querySelectorAll('.katex-display-container[data-katex-src]');
if (blocks.length > 0) {
blocks.forEach(el => {
if (el.dataset.rendered) return;
el.dataset.rendered = 'pending';
self._renderBlock(el);
});
}
async function _renderBlock(el) {
// Process inline math in text content
self._processInlineMath(container);
}
});
// Pre-load KaTeX
this._loadKaTeX();
},
async _renderBlock(el) {
const code = decodeURIComponent(el.dataset.katexSrc);
const displayMode = el.dataset.katexDisplay === 'true';
try {
await _loadKaTeX();
await this._loadKaTeX();
el.innerHTML = katex.renderToString(code, {
displayMode, throwOnError: false, trust: false, strict: false,
displayMode,
throwOnError: false,
trust: false,
strict: false,
});
el.dataset.rendered = 'true';
} catch (e) {
el.innerHTML = `
<div class="katex-error">
<strong>Math error:</strong> ${_escapeHtml(e.message || String(e))}
<strong>Math error:</strong> ${this._escapeHtml(e.message || String(e))}
</div>
`;
el.dataset.rendered = 'error';
}
}
},
// ── Inline Math Processing ───────────────
async function _processInlineMath(container) {
async _processInlineMath(container) {
// Skip if no $ signs present at all
if (!container.textContent || !container.textContent.includes('$')) return;
try { await _loadKaTeX(); } catch { return; }
const walker = document.createTreeWalker(container, NodeFilter.SHOW_TEXT, {
acceptNode(node) {
const parent = node.parentElement;
if (!parent) return NodeFilter.FILTER_REJECT;
const tag = parent.tagName;
if (tag === 'CODE' || tag === 'PRE' || tag === 'SCRIPT' ||
tag === 'TEXTAREA' || tag === 'STYLE') return NodeFilter.FILTER_REJECT;
if (parent.closest('.katex, .katex-block, .katex-display-container, code, pre'))
return NodeFilter.FILTER_REJECT;
if (!node.textContent.includes('$')) return NodeFilter.FILTER_REJECT;
return NodeFilter.FILTER_ACCEPT;
try {
await this._loadKaTeX();
} catch {
return;
}
// Walk text nodes, skip code/pre/katex-already-rendered
const walker = document.createTreeWalker(
container,
NodeFilter.SHOW_TEXT,
{
acceptNode(node) {
const parent = node.parentElement;
if (!parent) return NodeFilter.FILTER_REJECT;
// Skip code, pre, already-rendered katex, and script elements
const tag = parent.tagName;
if (tag === 'CODE' || tag === 'PRE' || tag === 'SCRIPT' ||
tag === 'TEXTAREA' || tag === 'STYLE') {
return NodeFilter.FILTER_REJECT;
}
if (parent.closest('.katex, .katex-block, .katex-display-container, code, pre')) {
return NodeFilter.FILTER_REJECT;
}
if (!node.textContent.includes('$')) return NodeFilter.FILTER_REJECT;
return NodeFilter.FILTER_ACCEPT;
}
}
});
);
const textNodes = [];
let node;
while (node = walker.nextNode()) textNodes.push(node);
for (const textNode of textNodes) _replaceInlineMath(textNode);
}
function _replaceInlineMath(textNode) {
for (const textNode of textNodes) {
this._replaceInlineMath(textNode);
}
},
_replaceInlineMath(textNode) {
const text = textNode.textContent;
// Match $$...$$ (display) and $...$ (inline), non-greedy
// Avoid matching escaped \$ or empty $$
const pattern = /\$\$([^$]+?)\$\$|\$([^$\n]+?)\$/g;
let match;
const parts = [];
let lastIndex = 0;
while ((match = pattern.exec(text)) !== null) {
// Text before the match
if (match.index > lastIndex) {
parts.push({ type: 'text', value: text.slice(lastIndex, match.index) });
}
const displayMode = !!match[1];
const displayMode = !!match[1]; // $$ ... $$
const expr = match[1] || match[2];
try {
const html = katex.renderToString(expr.trim(), {
displayMode, throwOnError: false, trust: false, strict: false,
displayMode,
throwOnError: false,
trust: false,
strict: false,
});
parts.push({ type: 'katex', value: html, displayMode });
} catch {
// Leave as-is on error
parts.push({ type: 'text', value: match[0] });
}
lastIndex = match.index + match[0].length;
}
if (parts.length === 0) return;
if (lastIndex < text.length) parts.push({ type: 'text', value: text.slice(lastIndex) });
if (parts.length === 0) return; // No math found
// Remaining text after last match
if (lastIndex < text.length) {
parts.push({ type: 'text', value: text.slice(lastIndex) });
}
// Replace text node with a span containing the rendered parts
const span = document.createElement('span');
span.className = 'katex-inline-container';
for (const part of parts) {
@@ -150,12 +186,70 @@
span.appendChild(wrapper);
}
}
textNode.parentNode.replaceChild(span, textNode);
}
},
// ── Styles ──────────────────────────────
_loadKaTeX() {
if (this._katexReady) return Promise.resolve();
if (this._katexLoading) return this._katexLoading;
function _injectStyles() {
this._katexLoading = new Promise((resolve, reject) => {
if (typeof katex !== 'undefined') {
this._katexReady = true;
resolve();
return;
}
const base = (window.__BASE__ || '');
const localCSS = `${base}/vendor/katex/katex.min.css`;
const localJS = `${base}/vendor/katex/katex.min.js`;
const cdnCSS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.css';
const cdnJS = 'https://cdn.jsdelivr.net/npm/katex@0.16.11/dist/katex.min.js';
// Load CSS first
const link = document.createElement('link');
link.rel = 'stylesheet';
link.href = localCSS;
link.onerror = () => {
link.href = cdnCSS;
};
document.head.appendChild(link);
// Load JS
const script = document.createElement('script');
script.src = localJS;
script.onload = () => {
this._katexReady = true;
resolve();
};
script.onerror = () => {
console.warn('[KaTeX] Local vendor not found, trying CDN');
const cdn = document.createElement('script');
cdn.src = cdnJS;
cdn.onload = () => {
this._katexReady = true;
resolve();
};
cdn.onerror = () => {
console.error('[KaTeX] Failed to load from both local and CDN');
reject(new Error('Failed to load KaTeX'));
};
document.head.appendChild(cdn);
};
document.head.appendChild(script);
});
return this._katexLoading;
},
_escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
},
_injectStyles() {
if (document.getElementById('ext-style-katex-renderer')) return;
const style = document.createElement('style');
style.id = 'ext-style-katex-renderer';
@@ -195,64 +289,9 @@
.katex-inline .katex { font-size: 1.05em; }
.katex-display { display: block; text-align: center; margin: 8px 0; }`;
document.head.appendChild(style);
},
destroy() {
document.getElementById('ext-style-katex-renderer')?.remove();
}
// ── Registration ────────────────────────
function register() {
if (!window.sw?.renderers) return;
_injectStyles();
// Block renderer: match ```latex, ```math, ```tex, ```katex
sw.renderers.register('katex-block', {
type: 'block',
priority: 10,
match(lang) {
const l = (lang || '').toLowerCase();
return l === 'latex' || l === 'math' || l === 'tex' || l === 'katex';
},
render(lang, code, container) {
const id = 'katex-' + Math.random().toString(36).slice(2, 9);
container.innerHTML = `
<div class="katex-block ext-rendered" data-katex-id="${id}">
<div class="katex-display-container" data-katex-src="${encodeURIComponent(code.trim())}" data-katex-display="true">
<div class="katex-loading">
<span class="katex-spinner"></span> Rendering math\u2026
</div>
</div>
<details class="katex-source">
<summary>\ud835\udc53(\ud835\udc65) View source</summary>
<pre><code class="language-latex">${_escapeHtml(code.trim())}</code></pre>
</details>
</div>
`;
}
});
// Post renderer: render KaTeX blocks + find inline math
sw.renderers.register('katex-post', {
type: 'post',
priority: 10,
render(container) {
const blocks = container.querySelectorAll('.katex-display-container[data-katex-src]');
if (blocks.length > 0) {
blocks.forEach(el => {
if (el.dataset.rendered) return;
el.dataset.rendered = 'pending';
_renderBlock(el);
});
}
_processInlineMath(container);
}
});
_loadKaTeX();
}
if (window.sw?._sdk) {
register();
} else {
document.addEventListener('sw:ready', register, { once: true });
}
})();
});

6
packages/mermaid-renderer/manifest.json → extensions/builtin/mermaid-renderer/manifest.json
View File

@@ -1,12 +1,10 @@
{
"id": "mermaid-renderer",
"title": "Mermaid Diagrams",
"name": "Mermaid Diagrams",
"version": "2.0.0",
"type": "extension",
"tier": "browser",
"author": "armature",
"author": "switchboard",
"description": "Renders ```mermaid code blocks as interactive SVG diagrams with zoom/pan, SVG/PNG export, and source copy",
"requires": [],
"permissions": [],
"tools": [],
"surfaces": [],

547
packages/mermaid-renderer/js/script.js → extensions/builtin/mermaid-renderer/script.js
View File

@@ -2,33 +2,94 @@
// Mermaid Diagram Renderer — Browser Extension
// ==========================================
// Renders ```mermaid code blocks as interactive SVG diagrams.
// Features: viewBox-based zoom/pan, fullscreen expand,
// SVG/PNG export, source copy.
// Features: viewBox-based zoom/pan, context-aware expand (fullscreen
// or side-panel pop-out), SVG/PNG export, source copy.
// Loads mermaid.js dynamically on first use.
//
// Registers with sw.renderers via the sw:ready event.
// Uses ctx.ui primitives from the host app for toast notifications,
// side-panel preview, theme detection, and mobile awareness.
// ==========================================
(function () {
'use strict';
Extensions.register({
id: 'mermaid-renderer',
let _mermaidReady = false;
let _mermaidLoading = null;
_mermaidReady: false,
_mermaidLoading: null,
_ctx: null,
function _isDark() {
return document.documentElement.dataset.theme === 'dark' ||
document.documentElement.classList.contains('dark');
}
async init(ctx) {
const self = this;
this._ctx = ctx;
function _escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
this._injectStyles();
// ── Block renderer: match ```mermaid ──
ctx.renderers.register('mermaid', {
type: 'block',
pattern: 'mermaid',
priority: 10,
render(lang, code, container) {
const id = 'mmd-' + Math.random().toString(36).slice(2, 9);
container.innerHTML = `
<div class="mermaid-block" data-mermaid-id="${id}">
<div class="mermaid-toolbar">
<span class="mermaid-title">\u{1f4ca} Diagram</span>
<span class="mermaid-zoom-label" data-zoom-label="${id}">100%</span>
<div class="mermaid-toolbar-btns">
<button class="mmd-btn" data-action="zoom-in" data-target="${id}" title="Zoom in">+</button>
<button class="mmd-btn" data-action="zoom-out" data-target="${id}" title="Zoom out">\u2212</button>
<button class="mmd-btn" data-action="zoom-fit" data-target="${id}" title="Fit to view">\u22a1</button>
<button class="mmd-btn" data-action="zoom-reset" data-target="${id}" title="Reset zoom">1:1</button>
<span class="mmd-sep"></span>
<button class="mmd-btn" data-action="expand" data-target="${id}" title="Expand">\u26f6</button>
<span class="mmd-sep"></span>
<button class="mmd-btn" data-action="export-svg" data-target="${id}" title="Download SVG">SVG</button>
<button class="mmd-btn" data-action="export-png" data-target="${id}" title="Download PNG">PNG</button>
</div>
</div>
<div class="mermaid-viewport" data-viewport="${id}">
<div class="mermaid-diagram" data-mermaid-src="${encodeURIComponent(code.trim())}" data-diagram="${id}">
<div class="mermaid-loading">
<span class="mermaid-spinner"></span> Rendering diagram\u2026
</div>
</div>
</div>
<details class="mermaid-source">
<summary>
<span>\u{1f4cb} View source</span>
<button class="mmd-btn mmd-copy-src" data-action="copy-src" data-target="${id}" title="Copy source" onclick="event.stopPropagation()">Copy</button>
</summary>
<pre><code class="language-mermaid" data-source="${id}">${self._escapeHtml(code.trim())}</code></pre>
</details>
</div>
`;
}
});
// ── Post renderer: render diagrams + wire interactivity ──
ctx.renderers.register('mermaid-post', {
type: 'post',
priority: 10,
render(container) {
const diagrams = container.querySelectorAll('.mermaid-diagram[data-mermaid-src]');
if (diagrams.length === 0) return;
diagrams.forEach(el => {
if (el.dataset.rendered) return;
el.dataset.rendered = 'pending';
self._renderDiagram(el);
});
self._wireToolbar(container);
}
});
this._loadMermaid();
},
// ── ViewBox Zoom/Pan State ──────────────
function _getState(id) {
_getState(id) {
const vp = document.querySelector(`[data-viewport="${id}"]`);
if (!vp) return null;
if (!vp._mmdState) {
@@ -41,10 +102,10 @@
};
}
return vp._mmdState;
}
},
function _initState(id) {
const state = _getState(id);
_initState(id) {
const state = this._getState(id);
if (!state) return;
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
if (!svg) return;
@@ -52,51 +113,69 @@
const vb = svg.getAttribute('viewBox');
if (!vb) {
const bbox = svg.getBBox();
state.natX = bbox.x; state.natY = bbox.y;
state.natW = bbox.width; state.natH = bbox.height;
state.natX = bbox.x;
state.natY = bbox.y;
state.natW = bbox.width;
state.natH = bbox.height;
} else {
const parts = vb.trim().split(/[\s,]+/).map(Number);
state.natX = parts[0] || 0; state.natY = parts[1] || 0;
state.natW = parts[2] || 0; state.natH = parts[3] || 0;
state.natX = parts[0] || 0;
state.natY = parts[1] || 0;
state.natW = parts[2] || 0;
state.natH = parts[3] || 0;
}
state.vbX = state.natX; state.vbY = state.natY;
state.vbW = state.natW; state.vbH = state.natH;
state.vbX = state.natX;
state.vbY = state.natY;
state.vbW = state.natW;
state.vbH = state.natH;
state.ready = true;
_applyViewBox(id);
}
function _applyViewBox(id) {
const state = _getState(id);
this._applyViewBox(id);
},
_applyViewBox(id) {
const state = this._getState(id);
if (!state || !state.ready) return;
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
if (!svg) return;
svg.setAttribute('viewBox', `${state.vbX} ${state.vbY} ${state.vbW} ${state.vbH}`);
const zoomPct = Math.round((state.natW / state.vbW) * 100);
const label = document.querySelector(`[data-zoom-label="${id}"]`);
if (label) label.textContent = zoomPct + '%';
}
},
function _zoom(id, factor) {
const state = _getState(id);
_zoom(id, factor) {
const state = this._getState(id);
if (!state || !state.ready) return;
const scale = 1 / (1 + factor);
const newW = state.vbW * scale;
const newH = state.vbH * scale;
if (newW < state.natW / 20 || newW > state.natW * 2) return;
const minW = state.natW / 20;
const maxW = state.natW * 2;
if (newW < minW || newW > maxW) return;
const cx = state.vbX + state.vbW / 2;
const cy = state.vbY + state.vbH / 2;
state.vbW = newW; state.vbH = newH;
state.vbX = cx - newW / 2; state.vbY = cy - newH / 2;
_applyViewBox(id);
}
function _zoomAt(id, factor, clientX, clientY) {
const state = _getState(id);
state.vbW = newW;
state.vbH = newH;
state.vbX = cx - newW / 2;
state.vbY = cy - newH / 2;
this._applyViewBox(id);
},
_zoomAt(id, factor, clientX, clientY) {
const state = this._getState(id);
if (!state || !state.ready) return;
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
if (!svg) return;
const svgRect = svg.getBoundingClientRect();
if (svgRect.width === 0 || svgRect.height === 0) return;
@@ -108,76 +187,147 @@
const scale = 1 / (1 + factor);
const newW = state.vbW * scale;
const newH = state.vbH * scale;
if (newW < state.natW / 20 || newW > state.natW * 2) return;
state.vbW = newW; state.vbH = newH;
state.vbX = pointX - fx * newW; state.vbY = pointY - fy * newH;
_applyViewBox(id);
}
function _zoomReset(id) {
const state = _getState(id);
const minW = state.natW / 20;
const maxW = state.natW * 2;
if (newW < minW || newW > maxW) return;
state.vbW = newW;
state.vbH = newH;
state.vbX = pointX - fx * newW;
state.vbY = pointY - fy * newH;
this._applyViewBox(id);
},
_zoomReset(id) {
const state = this._getState(id);
if (!state || !state.ready) return;
state.vbX = state.natX; state.vbY = state.natY;
state.vbW = state.natW; state.vbH = state.natH;
_applyViewBox(id);
}
state.vbX = state.natX;
state.vbY = state.natY;
state.vbW = state.natW;
state.vbH = state.natH;
this._applyViewBox(id);
},
function _zoomFit(id) {
const state = _getState(id);
_zoomFit(id) {
const state = this._getState(id);
if (!state || !state.ready) return;
const vp = document.querySelector(`[data-viewport="${id}"]`);
if (!vp) return;
const vpRect = vp.getBoundingClientRect();
if (vpRect.width === 0 || vpRect.height === 0) return;
const vpAspect = vpRect.width / vpRect.height;
const natAspect = state.natW / state.natH;
if (vpAspect > natAspect) {
const newW = state.natH * vpAspect;
state.vbX = state.natX - (newW - state.natW) / 2;
state.vbY = state.natY;
state.vbW = newW; state.vbH = state.natH;
state.vbW = newW;
state.vbH = state.natH;
} else {
const newH = state.natW / vpAspect;
state.vbX = state.natX;
state.vbY = state.natY - (newH - state.natH) / 2;
state.vbW = state.natW; state.vbH = newH;
state.vbW = state.natW;
state.vbH = newH;
}
_applyViewBox(id);
}
this._applyViewBox(id);
},
// ── Expand (context-aware) ──────────────
// Side panel open → pop out into it (replaces current content).
// Side panel closed → fullscreen overlay.
_expand(id) {
if (this._ctx.ui.isPanelOpen()) {
this._popOutToPanel(id);
} else {
this._toggleFullscreen(id);
}
},
// ── Fullscreen ──────────────────────────
function _toggleFullscreen(id) {
_toggleFullscreen(id) {
const block = document.querySelector(`[data-mermaid-id="${id}"]`);
if (!block) return;
const isFS = block.classList.toggle('mermaid-fullscreen');
if (isFS) {
document.body.style.overflow = 'hidden';
// Close button overlay (always visible — critical for mobile)
const closeBtn = document.createElement('button');
closeBtn.className = 'mmd-fullscreen-close';
closeBtn.innerHTML = '\u2715';
closeBtn.title = 'Close fullscreen';
closeBtn.addEventListener('click', () => _toggleFullscreen(id));
closeBtn.addEventListener('click', () => this._toggleFullscreen(id));
block.appendChild(closeBtn);
block._mmdEscHandler = (e) => { if (e.key === 'Escape') _toggleFullscreen(id); };
// Escape listener (desktop convenience, not sole exit)
block._mmdEscHandler = (e) => {
if (e.key === 'Escape') this._toggleFullscreen(id);
};
document.addEventListener('keydown', block._mmdEscHandler);
requestAnimationFrame(() => _zoomFit(id));
requestAnimationFrame(() => this._zoomFit(id));
} else {
document.body.style.overflow = '';
const closeBtn = block.querySelector('.mmd-fullscreen-close');
if (closeBtn) closeBtn.remove();
if (block._mmdEscHandler) {
document.removeEventListener('keydown', block._mmdEscHandler);
delete block._mmdEscHandler;
}
}
}
},
// ── Side Panel Pop-out ──────────────────
_popOutToPanel(id) {
const diagram = document.querySelector(`[data-diagram="${id}"]`);
const svg = diagram?.querySelector('svg');
if (!svg) return;
const state = this._getState(id);
const clone = svg.cloneNode(true);
if (state?.ready) {
clone.setAttribute('viewBox',
`${state.natX} ${state.natY} ${state.natW} ${state.natH}`);
}
clone.setAttribute('width', '100%');
clone.removeAttribute('height');
clone.setAttribute('preserveAspectRatio', 'xMidYMid meet');
const isDark = this._ctx.ui.isDark();
const html = `<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<style>
body { margin: 0; padding: 16px; display: flex; justify-content: center;
align-items: flex-start; min-height: 100vh;
background: ${isDark ? '#1a1a2e' : '#fff'};
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; }
svg { max-width: 100%; height: auto; }
</style>
</head><body>${clone.outerHTML}</body></html>`;
this._ctx.ui.openPreview(html);
},
// ── Toolbar Wiring ──────────────────────
function _wireToolbar(container) {
_wireToolbar(container) {
const self = this;
if (container._mmdWired) return;
container._mmdWired = true;
@@ -189,14 +339,14 @@
if (!id) return;
switch (action) {
case 'zoom-in': _zoom(id, 0.25); break;
case 'zoom-out': _zoom(id, -0.2); break;
case 'zoom-reset': _zoomReset(id); break;
case 'zoom-fit': _zoomFit(id); break;
case 'expand': _toggleFullscreen(id); break;
case 'export-svg': _exportSVG(id); break;
case 'export-png': _exportPNG(id); break;
case 'copy-src': _copySource(id); break;
case 'zoom-in': self._zoom(id, 0.25); break;
case 'zoom-out': self._zoom(id, -0.2); break;
case 'zoom-reset': self._zoomReset(id); break;
case 'zoom-fit': self._zoomFit(id); break;
case 'expand': self._expand(id); break;
case 'export-svg': self._exportSVG(id); break;
case 'export-png': self._exportPNG(id); break;
case 'copy-src': self._copySource(id); break;
}
});
@@ -209,35 +359,42 @@
vp.addEventListener('wheel', (e) => {
e.preventDefault();
const delta = e.deltaY > 0 ? -0.1 : 0.1;
_zoomAt(id, delta, e.clientX, e.clientY);
self._zoomAt(id, delta, e.clientX, e.clientY);
}, { passive: false });
vp.addEventListener('mousedown', (e) => {
if (e.button !== 0) return;
const state = _getState(id);
const state = self._getState(id);
if (!state || !state.ready) return;
state.dragging = true;
state.startX = e.clientX; state.startY = e.clientY;
state.startVbX = state.vbX; state.startVbY = state.vbY;
state.startX = e.clientX;
state.startY = e.clientY;
state.startVbX = state.vbX;
state.startVbY = state.vbY;
vp.classList.add('mmd-grabbing');
e.preventDefault();
});
vp.addEventListener('mousemove', (e) => {
const state = _getState(id);
const state = self._getState(id);
if (!state?.dragging) return;
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
if (!svg) return;
const svgRect = svg.getBoundingClientRect();
if (svgRect.width === 0) return;
const pxToVb = state.vbW / svgRect.width;
state.vbX = state.startVbX - (e.clientX - state.startX) * pxToVb;
state.vbY = state.startVbY - (e.clientY - state.startY) * pxToVb;
_applyViewBox(id);
const dx = (e.clientX - state.startX) * pxToVb;
const dy = (e.clientY - state.startY) * pxToVb;
state.vbX = state.startVbX - dx;
state.vbY = state.startVbY - dy;
self._applyViewBox(id);
});
const endDrag = () => {
const state = _getState(id);
const state = self._getState(id);
if (!state) return;
state.dragging = false;
vp.classList.remove('mmd-grabbing');
@@ -246,8 +403,9 @@
vp.addEventListener('mouseleave', endDrag);
let lastTouchDist = 0;
let lastTouchCenter = null;
vp.addEventListener('touchstart', (e) => {
const state = _getState(id);
const state = self._getState(id);
if (!state || !state.ready) return;
if (e.touches.length === 1) {
state.dragging = true;
@@ -260,21 +418,28 @@
e.touches[0].clientX - e.touches[1].clientX,
e.touches[0].clientY - e.touches[1].clientY
);
lastTouchCenter = {
x: (e.touches[0].clientX + e.touches[1].clientX) / 2,
y: (e.touches[0].clientY + e.touches[1].clientY) / 2,
};
}
}, { passive: true });
vp.addEventListener('touchmove', (e) => {
const state = _getState(id);
const state = self._getState(id);
if (!state || !state.ready) return;
if (e.touches.length === 1 && state.dragging) {
const svg = document.querySelector(`[data-diagram="${id}"] svg`);
if (!svg) return;
const svgRect = svg.getBoundingClientRect();
if (svgRect.width === 0) return;
const pxToVb = state.vbW / svgRect.width;
state.vbX = state.startVbX - (e.touches[0].clientX - state.startX) * pxToVb;
state.vbY = state.startVbY - (e.touches[0].clientY - state.startY) * pxToVb;
_applyViewBox(id);
const dx = (e.touches[0].clientX - state.startX) * pxToVb;
const dy = (e.touches[0].clientY - state.startY) * pxToVb;
state.vbX = state.startVbX - dx;
state.vbY = state.startVbY - dy;
self._applyViewBox(id);
e.preventDefault();
} else if (e.touches.length === 2) {
const dist = Math.hypot(
@@ -287,42 +452,46 @@
};
if (lastTouchDist > 0) {
const factor = (dist - lastTouchDist) / lastTouchDist;
_zoomAt(id, factor, center.x, center.y);
self._zoomAt(id, factor, center.x, center.y);
}
lastTouchDist = dist;
lastTouchCenter = center;
e.preventDefault();
}
}, { passive: false });
vp.addEventListener('touchend', () => {
const state = _getState(id);
const state = self._getState(id);
if (state) state.dragging = false;
lastTouchDist = 0;
lastTouchCenter = null;
}, { passive: true });
});
}
},
// ── Export ───────────────────────────────
function _exportSVG(id) {
_exportSVG(id) {
const diagram = document.querySelector(`[data-diagram="${id}"]`);
const svg = diagram?.querySelector('svg');
if (!svg) return;
const state = _getState(id);
const state = this._getState(id);
const clone = svg.cloneNode(true);
clone.setAttribute('xmlns', 'http://www.w3.org/2000/svg');
if (state?.ready) {
clone.setAttribute('viewBox', `${state.natX} ${state.natY} ${state.natW} ${state.natH}`);
}
const blob = new Blob([clone.outerHTML], { type: 'image/svg+xml' });
_download(blob, `diagram-${id}.svg`);
}
this._download(blob, `diagram-${id}.svg`);
},
function _exportPNG(id) {
_exportPNG(id) {
const diagram = document.querySelector(`[data-diagram="${id}"]`);
const svg = diagram?.querySelector('svg');
if (!svg) return;
const state = _getState(id);
const state = this._getState(id);
const clone = svg.cloneNode(true);
clone.setAttribute('xmlns', 'http://www.w3.org/2000/svg');
if (state?.ready) {
@@ -334,6 +503,7 @@
const svgData = new XMLSerializer().serializeToString(clone);
const svgBlob = new Blob([svgData], { type: 'image/svg+xml;charset=utf-8' });
const url = URL.createObjectURL(svgBlob);
const self = this;
const img = new Image();
img.onload = () => {
@@ -345,8 +515,9 @@
ctx.scale(scale, scale);
ctx.drawImage(img, 0, 0);
URL.revokeObjectURL(url);
canvas.toBlob((blob) => {
if (blob) _download(blob, `diagram-${id}.png`);
if (blob) self._download(blob, `diagram-${id}.png`);
}, 'image/png');
};
img.onerror = () => {
@@ -354,9 +525,9 @@
console.error('[Mermaid] PNG export failed');
};
img.src = url;
}
},
function _download(blob, filename) {
_download(blob, filename) {
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = filename;
@@ -364,24 +535,25 @@
a.click();
document.body.removeChild(a);
setTimeout(() => URL.revokeObjectURL(a.href), 1000);
}
},
function _copySource(id) {
_copySource(id) {
const code = document.querySelector(`[data-source="${id}"]`);
if (!code) return;
navigator.clipboard.writeText(code.textContent).then(() => {
if (window.sw?.toast) sw.toast('Source copied', 'success');
this._ctx.ui.toast('Source copied', 'success');
});
}
},
// ── Diagram Rendering ───────────────────
async function _renderDiagram(el) {
async _renderDiagram(el) {
const code = decodeURIComponent(el.dataset.mermaidSrc);
const id = el.dataset.diagram;
try {
await _loadMermaid();
await this._loadMermaid();
const svgId = 'mmd-svg-' + Math.random().toString(36).slice(2, 9);
const { svg } = await mermaid.render(svgId, code);
el.innerHTML = svg;
@@ -394,27 +566,28 @@
svgEl.style.maxWidth = 'none';
svgEl.setAttribute('preserveAspectRatio', 'xMidYMid meet');
}
_initState(id);
this._initState(id);
} catch (e) {
el.innerHTML = `
<div class="mermaid-error">
<strong>Diagram error:</strong> ${_escapeHtml(e.message || String(e))}
<strong>Diagram error:</strong> ${this._escapeHtml(e.message || String(e))}
</div>
`;
el.dataset.rendered = 'error';
}
}
},
// ── Mermaid Library Loading ──────────────
function _loadMermaid() {
if (_mermaidReady) return Promise.resolve();
if (_mermaidLoading) return _mermaidLoading;
_loadMermaid() {
if (this._mermaidReady) return Promise.resolve();
if (this._mermaidLoading) return this._mermaidLoading;
_mermaidLoading = new Promise((resolve, reject) => {
this._mermaidLoading = new Promise((resolve, reject) => {
if (typeof mermaid !== 'undefined') {
_initMermaid();
_mermaidReady = true;
this._initMermaid();
this._mermaidReady = true;
resolve();
return;
}
@@ -426,8 +599,8 @@
const script = document.createElement('script');
script.src = localSrc;
script.onload = () => {
_initMermaid();
_mermaidReady = true;
this._initMermaid();
this._mermaidReady = true;
resolve();
};
script.onerror = () => {
@@ -435,8 +608,8 @@
const cdn = document.createElement('script');
cdn.src = cdnSrc;
cdn.onload = () => {
_initMermaid();
_mermaidReady = true;
this._initMermaid();
this._mermaidReady = true;
resolve();
};
cdn.onerror = () => {
@@ -447,23 +620,25 @@
};
document.head.appendChild(script);
});
return _mermaidLoading;
}
function _initMermaid() {
return this._mermaidLoading;
},
_initMermaid() {
if (typeof mermaid === 'undefined') return;
mermaid.initialize({
startOnLoad: false,
theme: _isDark() ? 'dark' : 'default',
theme: this._ctx.ui.isDark() ? 'dark' : 'default',
securityLevel: 'strict',
fontFamily: 'inherit',
logLevel: 'error',
});
}
},
// ── Styles ──────────────────────────────
function _injectStyles() {
_injectStyles() {
if (document.getElementById('ext-style-mermaid-renderer')) return;
const style = document.createElement('style');
style.id = 'ext-style-mermaid-renderer';
@@ -472,6 +647,8 @@
background: var(--bg-2); border: 1px solid var(--border);
border-radius: 8px; overflow: hidden; margin: 12px 0;
}
/* ── Fullscreen mode ── */
.mermaid-block.mermaid-fullscreen {
position: fixed; top: 0; left: 0; right: 0; bottom: 0;
z-index: 10000; border-radius: 0; margin: 0;
@@ -479,11 +656,15 @@
background: var(--bg-1, var(--bg-2, #1a1a2e));
}
.mermaid-block.mermaid-fullscreen .mermaid-viewport {
background: var(--bg-2, #1a1a2e); flex: 1; max-height: none;
background: var(--bg-2, #1a1a2e);
flex: 1; max-height: none;
}
.mermaid-block.mermaid-fullscreen .mermaid-toolbar {
border-bottom: 1px solid var(--border); padding: 6px 14px;
border-bottom: 1px solid var(--border);
padding: 6px 14px;
}
/* Fullscreen close button — always visible, critical for mobile */
.mmd-fullscreen-close {
position: absolute; top: 12px; right: 12px; z-index: 10001;
width: 40px; height: 40px; border-radius: 50%;
@@ -495,8 +676,11 @@
box-shadow: 0 2px 8px rgba(0,0,0,0.3);
}
.mmd-fullscreen-close:hover {
background: var(--bg-hover, rgba(255,255,255,0.15)); transform: scale(1.1);
background: var(--bg-hover, rgba(255,255,255,0.15));
transform: scale(1.1);
}
/* Toolbar */
.mermaid-toolbar {
display: flex; align-items: center; gap: 6px;
padding: 4px 10px; border-bottom: 1px solid var(--border);
@@ -515,14 +699,24 @@
}
.mmd-btn:hover { color: var(--text-1); border-color: var(--text-3); background: var(--bg-3); }
.mmd-sep { width: 1px; height: 16px; background: var(--border); margin: 0 4px; }
/* Viewport */
.mermaid-viewport {
overflow: hidden; position: relative;
min-height: 80px; max-height: 600px;
cursor: grab; user-select: none;
}
.mermaid-viewport.mmd-grabbing { cursor: grabbing; }
.mermaid-diagram { width: 100%; height: 100%; min-height: 60px; }
.mermaid-diagram svg { display: block; width: 100%; height: 100%; }
/* Diagram wrapper */
.mermaid-diagram {
width: 100%; height: 100%; min-height: 60px;
}
.mermaid-diagram svg {
display: block; width: 100%; height: 100%;
}
/* Loading / Error */
.mermaid-loading {
color: var(--text-3); font-size: 13px; padding: 20px;
display: flex; align-items: center; justify-content: center; gap: 8px;
@@ -538,6 +732,8 @@
padding: 12px 16px; border-radius: 4px; font-size: 13px;
font-family: var(--mono); white-space: pre-wrap;
}
/* Source panel */
.mermaid-source { border-top: 1px solid var(--border); font-size: 12px; }
.mermaid-source summary {
padding: 6px 12px; cursor: pointer; color: var(--text-3);
@@ -550,91 +746,34 @@
margin: 0; border-radius: 0; border: none;
max-height: 200px; overflow: auto;
}
/* Hide source panel in fullscreen */
.mermaid-block.mermaid-fullscreen .mermaid-source { display: none; }
/* ── Mobile adjustments ── */
@media (max-width: 768px) {
.mermaid-toolbar { padding: 6px 10px; gap: 4px; }
.mmd-btn { padding: 4px 10px; font-size: 12px; }
.mmd-sep { margin: 0 2px; }
.mmd-fullscreen-close { width: 48px; height: 48px; font-size: 24px; top: 16px; right: 16px; }
}`;
}
`;
document.head.appendChild(style);
}
},
// ── Registration ────────────────────────
_escapeHtml(str) {
const div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
},
function register() {
if (!window.sw?.renderers) return;
_injectStyles();
// Block renderer: match ```mermaid
sw.renderers.register('mermaid', {
type: 'block',
pattern: 'mermaid',
priority: 10,
render(lang, code, container) {
const id = 'mmd-' + Math.random().toString(36).slice(2, 9);
container.innerHTML = `
<div class="mermaid-block" data-mermaid-id="${id}">
<div class="mermaid-toolbar">
<span class="mermaid-title">\u{1f4ca} Diagram</span>
<span class="mermaid-zoom-label" data-zoom-label="${id}">100%</span>
<div class="mermaid-toolbar-btns">
<button class="mmd-btn" data-action="zoom-in" data-target="${id}" title="Zoom in">+</button>
<button class="mmd-btn" data-action="zoom-out" data-target="${id}" title="Zoom out">\u2212</button>
<button class="mmd-btn" data-action="zoom-fit" data-target="${id}" title="Fit to view">\u22a1</button>
<button class="mmd-btn" data-action="zoom-reset" data-target="${id}" title="Reset zoom">1:1</button>
<span class="mmd-sep"></span>
<button class="mmd-btn" data-action="expand" data-target="${id}" title="Fullscreen">\u26f6</button>
<span class="mmd-sep"></span>
<button class="mmd-btn" data-action="export-svg" data-target="${id}" title="Download SVG">SVG</button>
<button class="mmd-btn" data-action="export-png" data-target="${id}" title="Download PNG">PNG</button>
</div>
</div>
<div class="mermaid-viewport" data-viewport="${id}">
<div class="mermaid-diagram" data-mermaid-src="${encodeURIComponent(code.trim())}" data-diagram="${id}">
<div class="mermaid-loading">
<span class="mermaid-spinner"></span> Rendering diagram\u2026
</div>
</div>
</div>
<details class="mermaid-source">
<summary>
<span>\u{1f4cb} View source</span>
<button class="mmd-btn mmd-copy-src" data-action="copy-src" data-target="${id}" title="Copy source" onclick="event.stopPropagation()">Copy</button>
</summary>
<pre><code class="language-mermaid" data-source="${id}">${_escapeHtml(code.trim())}</code></pre>
</details>
</div>
`;
}
destroy() {
document.getElementById('ext-style-mermaid-renderer')?.remove();
document.querySelectorAll('.mermaid-fullscreen').forEach(el => {
el.classList.remove('mermaid-fullscreen');
const closeBtn = el.querySelector('.mmd-fullscreen-close');
if (closeBtn) closeBtn.remove();
});
// Post renderer: render diagrams + wire interactivity
sw.renderers.register('mermaid-post', {
type: 'post',
priority: 10,
render(container) {
const diagrams = container.querySelectorAll('.mermaid-diagram[data-mermaid-src]');
if (diagrams.length === 0) return;
diagrams.forEach(el => {
if (el.dataset.rendered) return;
el.dataset.rendered = 'pending';
_renderDiagram(el);
});
_wireToolbar(container);
}
});
_loadMermaid();
document.body.style.overflow = '';
}
// Boot: register immediately if SDK ready, otherwise listen
if (window.sw?._sdk) {
register();
} else {
document.addEventListener('sw:ready', register, { once: true });
}
})();
});

6
packages/regex-tester/manifest.json → extensions/builtin/regex-tester/manifest.json
View File

@@ -1,12 +1,10 @@
{
"id": "regex-tester",
"title": "Regex Tester",
"name": "Regex Tester",
"version": "1.0.0",
"type": "extension",
"tier": "browser",
"author": "armature",
"author": "switchboard",
"description": "Tests regular expressions against input strings. Allows the LLM to verify regex patterns and see matches, groups, and indices.",
"requires": ["chat"],
"permissions": [],
"tools": [
{

0
packages/regex-tester/js/script.js → extensions/builtin/regex-tester/script.js
View File

BIN
icons/apple-touch-icon-b-dark.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 1.6 KiB

BIN
icons/apple-touch-icon-b-light.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 1.6 KiB

BIN
icons/apple-touch-icon-e-dark.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 1.8 KiB

Some files were not shown because too many files have changed in this diff Show More