switchboard/core
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Releases 9 Wiki Activity

Changeset 0.19.0.1 (#82)

Browse Source
This commit is contained in:
xcaliber
2026-02-28 23:46:23 +00:00
parent 091ce2af6a
commit 748f49bedd
30 changed files with 3873 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files

355
README.md
View File

@@ -1,103 +1,292 @@
# Hotfix Patches — v0.18.1 → v0.18.2
# Chat Switchboard
Priority order: apply top-to-bottom. All patches are independent
except 1 and 2 which must be applied together.
A self-hosted AI chat application for enterprise and government environments. Unified interface for multiple AI providers with admin controls, team management, and security features.
---
## Quick Start
## Patch 1+2: Harden JSON scanning + SafeJSON (P0 — fixes broken chat list)
```bash
# Clone and start
git clone <repo-url> && cd chat-switchboard
docker compose up -d
**Root cause:** Corrupt `\x02` byte in a channel's `settings` column.
`scanJSON` accepts it, Gin's `c.JSON()` starts writing 200 + headers,
`json.RawMessage.MarshalJSON()` hits the bad byte, aborts, browser
gets truncated body → "Unexpected end of JSON input" → 0 chats loaded.
**Files:**
1. **NEW:** `server/handlers/safe_json.go`
Copy `patches/server/handlers/safe_json.go` into your tree.
Contains: `SafeJSON()`, hardened `scanJSON()`, hardened `scanTags()`.
2. **NEW:** `server/handlers/safe_json_test.go`
Copy `patches/server/handlers/safe_json_test.go` into your tree.
3. **EDIT:** `server/handlers/channels.go`
Apply these three changes (or diff against `patches/server/handlers/channels.go`):
a. Remove the old `scanJSON` + `jsonScanner` + `scanTags` + `tagsScanner`
definitions (~lines 101–155). Replace with a comment:
```go
// scanJSON, scanTags, SafeJSON → safe_json.go
```
b. Replace `c.JSON(http.StatusOK, paginatedResponse{` in ListChannels
with `SafeJSON(c, http.StatusOK, paginatedResponse{`
c. Replace `c.JSON(http.StatusCreated, ch)` at end of CreateChannel
with `SafeJSON(c, http.StatusCreated, ch)`
d. Replace `c.JSON(http.StatusOK, ch)` at end of GetChannel
with `SafeJSON(c, http.StatusOK, ch)`
**Verify:** `go build ./...` and `go test ./server/handlers/ -run TestScanJSON -v`
**Data fix:** Find and repair the corrupt row:
```sql
-- Find it
SELECT id, title, length(settings::text),
encode(substring(settings::bytea, 1, 20), 'hex') AS hex_preview
FROM channels
WHERE settings IS NOT NULL
AND settings::text ~ '[^\x20-\x7E\t\n\r]';
-- Fix it (reset to empty settings)
UPDATE channels SET settings = '{}'
WHERE settings IS NOT NULL
AND settings::text ~ '[^\x20-\x7E\t\n\r]';
# Access at http://localhost:3000
# Default admin: admin / admin
```
---
## Features
## Patch 3: Service Worker scheme guard (P1 — console noise)
- **Multi-Provider**: Anthropic, OpenAI, OpenRouter, Venice AI — with BYOK support
- **Projects**: Group related conversations, KBs, and notes into workspaces with scope-aware visibility
- **Team Management**: Roles (admin/user) for vertical permissions, Teams for horizontal visibility
- **Personas**: Preset configurations (model + system prompt + parameters) at global, team, or personal scope
- **Model Catalog**: Three-state visibility (enabled/disabled/team-only) with admin controls
- **Message Trees**: Edit and regenerate with full conversation forking — navigate sibling branches
- **Notes**: Markdown notes with full-text search, folders, and tags
- **Extensions**: Browser extension system with custom renderers, tool bridge, and self-contained styling
- Built-in: Mermaid diagrams, KaTeX math, CSV tables, diff viewer, JS sandbox, regex tester
- **Server Tools**: Calculator and datetime tools (auto-registered, zero config)
- **Audit Log**: All admin operations logged with actor, action, and resource details
- **Security**: JWT + refresh tokens, AES-256-GCM API key encryption, optional mTLS, optional OIDC/Keycloak, environment classification banners
- **Airgapped**: Local vendor files (marked.js, DOMPurify, KaTeX, mermaid.js), no CDN required
- **Mobile**: Responsive design, PWA manifest
- **Real-time**: WebSocket event bus for tool bridge, live updates
**Root cause:** SW fetch handler tries to cache `chrome-extension://`
URLs. `Cache.put()` rejects non-http(s) schemes.
## Architecture
**File:** `src/sw.js`
Add scheme guard after `const url = new URL(event.request.url);`:
```js
// Only handle http/https — ignore chrome-extension://, moz-extension://, etc.
if (url.protocol !== 'https:' && url.protocol !== 'http:') {
return;
}
```
┌─────────────┐ ┌──────────────────────────────┐
│ Browser │────▶│ nginx (port 80) │
│ (vanilla JS)│◀────│ ├─ /api/* → Go backend:8080 │
└─────────────┘ │ ├─ /ws → WebSocket proxy │
│ └─ /* → static files │
└──────────────────────────────┘
│
┌─────────▼─────────┐
│ Go Backend │
│ ├─ handlers/ │
│ ├─ store/postgres/ │
│ ├─ store/sqlite/ │
│ ├─ providers/ │
│ └─ capabilities/ │
└─────────┬─────────┘
│
┌─────────▼─────────┐
│ PostgreSQL 16 or │
│ SQLite (embedded) │
└───────────────────┘
```
Or copy `patches/src/sw.js`.
**Go backend** (vanilla, no framework beyond Gin) with a store layer abstracting all database access. Dual-driver architecture: `store/postgres` for production deployments, `store/sqlite` for single-user, edge, and development scenarios — selected at startup via `DB_DRIVER`. Providers package handles LLM API calls. Capabilities package resolves model features from catalog data, known model tables, and heuristic inference. Server tools (calculator, datetime) auto-register via `init()`. EventBus + WebSocket hub routes tool calls between backend and browser extensions.
---
**Frontend** is vanilla JavaScript — no build step, no bundler. 15 files organized by domain: `api.js` (HTTP client), `app.js` (state + init), `chat.js` (send, regen, edit, branch), `events.js` (event bus + WebSocket), `extensions.js` (loader, registry, renderer pipeline, tool bridge), `ui-core.js` (DOM rendering + streaming), `ui-format.js` (markdown, code blocks), `ui-primitives.js` (shared components), `ui-settings.js` / `ui-admin.js` (settings and admin panels), `notes.js`, `tokens.js`, `debug.js`, `settings-handlers.js`, `admin-handlers.js`.
## Patch 4: Settings debounce (P2 — cosmetic waste)
## Configuration
Not included as a patch file — straightforward to implement:
- In `app.js` init sequence, debounce `API.updateSettings()` calls
- A 100ms `setTimeout` wrapper that coalesces multiple rapid saves
All configuration via environment variables. See `server/.env.example` for the full list.
---
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `8080` | Backend listen port |
| `BASE_PATH` | ` ` | URL prefix (e.g. `/chat`) |
| `DB_DRIVER` | `postgres` | Database backend: `postgres` or `sqlite` |
| `DATABASE_URL` | ` ` | SQLite: path to database file (e.g. `/data/switchboard.db`) |
| `DB_HOST` | `localhost` | PostgreSQL host (ignored when `DB_DRIVER=sqlite`) |
| `DB_NAME` | `chat_switchboard` | Database name (ignored when `DB_DRIVER=sqlite`) |
| `JWT_SECRET` | (required) | Token signing key |
| `SWITCHBOARD_ADMIN_USERNAME` | ` ` | Bootstrap admin username |
| `SWITCHBOARD_ADMIN_PASSWORD` | ` ` | Bootstrap admin password |
| `ENCRYPTION_KEY` | ` ` | AES key for API key encryption (required if encrypted keys exist) |
| `SEED_USERS` | ` ` | Dev/test only: `user:pass:role,user2:pass2:role2` |
| `STORAGE_BACKEND` | (auto) | `pvc` or `s3`. Auto-detects PVC if not set. |
| `STORAGE_PATH` | `/data/storage` | PVC mount point (also scratch dir for S3 extraction) |
| `S3_ENDPOINT` | ` ` | S3 endpoint (e.g. `http://minio:9000`, required for S3) |
| `S3_BUCKET` | ` ` | S3 bucket name (must exist, required for S3) |
| `S3_ACCESS_KEY` | ` ` | S3 access key ID |
| `S3_SECRET_KEY` | ` ` | S3 secret access key |
| `S3_REGION` | `us-east-1` | S3 region |
| `S3_PREFIX` | ` ` | Optional key prefix within bucket |
| `S3_FORCE_PATH_STYLE` | `true` | Path-style URLs (required for MinIO, Ceph) |
## Future work (not in this hotfix)
## Deployment
- Apply `SafeJSON` to extension list endpoints (lines 47, 115 of
extensions.go) — same vulnerability pattern, lower risk since
extension data is seeded from known-good JSON.
Three Docker images support different scenarios:
- Harden `JSONMap.Scan` in `models/models.go` to warn-and-default
instead of returning error. Currently it fails the whole query
which gives a clean 500, but it means one corrupt row blocks all
results in a list query.
| Image | Dockerfile | Use Case |
|-------|-----------|----------|
| **Unified** | `Dockerfile` | Dev, docker-compose, single-node |
| **Backend** | `server/Dockerfile` | K8s — scale API pods independently |
| **Frontend** | `Dockerfile.frontend` | K8s — scale FE pods independently |
- Startup JSON integrity check (design doc §17.1 Layer 4) — scan
all JSONB columns on boot, log warnings for corrupt data. Not
blocking for hotfix but valuable for post-upgrade diagnostics.
### Docker Compose (development — unified image)
- JWT redaction in Gin log formatter for WebSocket URLs.
```bash
docker compose up -d # start all services
docker compose up -d --build # rebuild after code changes
docker compose --profile dev up # include Adminer DB UI
```
### Kubernetes (split images)
Build and push both images:
```bash
docker build -f server/Dockerfile -t your-registry/switchboard-api:0.11.0 server/
docker build -f Dockerfile.frontend -t your-registry/switchboard-fe:0.11.0 .
```
**Backend deployment:**
```yaml
containers:
- name: api
image: your-registry/switchboard-api:0.11.0
ports:
- containerPort: 8080
env:
- name: DB_HOST
value: "postgres-service"
- name: BASE_PATH
value: "/chat" # must match ingress path
- name: JWT_SECRET
valueFrom:
secretKeyRef:
name: switchboard-secrets
key: jwt-secret
```
**Frontend deployment:**
```yaml
containers:
- name: frontend
image: your-registry/switchboard-fe:0.11.0
ports:
- containerPort: 80
env:
- name: BASE_PATH
value: "/chat" # injected into index.html at startup
volumeMounts:
- name: branding
mountPath: /branding
readOnly: true # optional: custom logo, colors
```
**Ingress** routes `/api/*` and `/ws` to the backend Service, everything else to the frontend Service. The frontend entrypoint generates the nginx config dynamically based on `BASE_PATH`.
### Path-Based Routing
Set `BASE_PATH=/chat` to serve the application under a subpath. The frontend reads `window.__BASE__` injected at container startup, and all API calls are prefixed automatically.
### Airgapped / Disconnected
The Docker build bakes in `marked.js`, `DOMPurify`, and `KaTeX` (JS + CSS + fonts) from npm during the vendor stage. Mermaid.js loads dynamically from local vendor with CDN fallback. No CDN calls at runtime in airgapped deployments. The `src/vendor/` directory contains local copies as fallback for development without Docker.
## Database
### PostgreSQL (default)
PostgreSQL 16+ recommended. The `pgcrypto` and `vector` (pgvector) extensions are used for UUID generation and vector similarity search respectively.
### SQLite (single-user / edge / dev)
Set `DB_DRIVER=sqlite` and `DATABASE_URL=/path/to/switchboard.db` to run with an embedded SQLite database. No external dependencies — the binary is self-contained (pure Go, no CGO). The SQLite backend has full feature parity with Postgres including knowledge base vector search, which uses app-level cosine similarity computed in Go rather than pgvector.
```bash
# Minimal single-binary startup
DB_DRIVER=sqlite DATABASE_URL=./data/switchboard.db JWT_SECRET=changeme ./switchboard
```
SQLite is best suited for single-user workstations, edge deployments, air-gapped laptops, and local development. For multi-user production with concurrent writes, use PostgreSQL.
### Schema Management
The Go backend auto-migrates on startup using files in `server/database/migrations/`. For manual operations:
```bash
# Bootstrap database (superuser, creates role + DB)
scripts/db-bootstrap.sh
# Manual migration (usually not needed)
scripts/db-migrate.sh
# Validate schema
scripts/db-validate.sh
```
### Key Tables (v0.19)
| Table | Purpose |
|-------|---------|
| `users` | Accounts with role, avatar, settings, encrypted vault (UEK) |
| `provider_configs` | API provider configurations (scope: global/team/personal) |
| `model_catalog` | Synced model list with capabilities, visibility, and type |
| `personas` | Model presets (scope: global/team/personal) |
| `projects` | Workspaces grouping channels, KBs, and notes (scope: personal/team/global) |
| `project_channels` | Ordered channel-to-project membership (UNIQUE channel_id) |
| `project_knowledge_bases` | KB-to-project binding with auto_search flag |
| `project_notes` | Note-to-project association |
| `channels` | Conversations with type (direct/group/channel), optional `project_id` |
| `messages` | Message tree with parent_id for forking, tool_calls JSONB |
| `teams` / `team_members` | Organizational units |
| `notes` | Markdown notes with full-text search |
| `knowledge_bases` / `kb_documents` / `kb_chunks` | RAG pipeline with pgvector embeddings |
| `extensions` / `extension_user_settings` | Browser extension registry and per-user config |
| `usage_log` / `model_pricing` | Token usage tracking and cost calculation |
| `audit_log` | Admin action audit trail |
| `user_model_settings` | Per-user model visibility and sort preferences |
## API
All endpoints under `/api/v1/`. Authentication via `Authorization: Bearer <token>` header.
### Auth
- `POST /auth/register` — Register (if `allow_registration` policy is true)
- `POST /auth/login` — Login, returns access + refresh tokens
- `POST /auth/refresh` — Refresh access token
- `POST /auth/logout` — Revoke refresh token
### Channels & Messages
- `GET/POST /channels` — List/create conversations
- `GET/PUT/DELETE /channels/:id` — Channel CRUD
- `GET/POST /channels/:id/messages` — Message list/create
- `POST /chat/completions` — Stream AI completions (SSE)
- `POST /channels/:id/messages/:msgId/edit` — Edit and fork
- `POST /channels/:id/messages/:msgId/regenerate` — Regenerate response
### Models
- `GET /models/enabled` — Models available to the user (global + team + personal)
- `GET/PUT /models/preferences` — User model visibility settings
### Personal Providers (BYOK)
- `GET/POST /api-configs` — User's personal provider configs
- `GET/PUT/DELETE /api-configs/:id` — Provider CRUD
- `POST /api-configs/:id/models/fetch` — Refresh models from provider API
- `GET /api-configs/:id/models` — List models for a personal provider
### Teams
- `GET /teams` — Teams the user belongs to
- `GET/POST /teams/:id/providers` — Team provider management (team admins)
- `GET/POST /teams/:id/presets` — Team preset management (team admins)
### Projects
- `GET/POST /projects` — List and create projects
- `GET/PUT/DELETE /projects/:id` — Project CRUD (owner-only delete)
- `POST/DELETE /projects/:id/channels/:channelId` — Add/remove channel
- `GET /projects/:id/channels` — List project channels
- `PUT /projects/:id/channels/reorder` — Reorder channels
- `POST/DELETE /projects/:id/knowledge-bases/:kbId` — Bind/unbind KB
- `POST/DELETE /projects/:id/notes/:noteId` — Bind/unbind note
### Admin
- `GET/POST /admin/users` — User management
- `GET/PUT /admin/settings/:key` — Global settings and policies
- `GET/POST /admin/configs` — Global provider configs
- `GET/POST /admin/models/fetch` — Model catalog sync
- `GET/PUT /admin/roles/:role` — Model role configuration
- `GET /admin/usage` — Usage dashboard
- `GET /admin/audit` — Audit log
- `GET/POST/PUT/DELETE /admin/extensions` — Extension management
### Extensions
- `GET /extensions?tier=browser` — List enabled browser extensions
- `GET /extensions/:id/assets/*path` — Serve extension assets (public, no auth)
### WebSocket
- `GET /ws?token=<jwt>` — EventBus WebSocket for real-time events and tool bridge
## Development
```bash
# Backend (requires Go 1.22+)
cd server
cp .env.example .env # edit with your DB credentials
go run .
# Frontend (just serve static files)
# Use any HTTP server pointed at src/
python3 -m http.server 3000 --directory src
```
## License
Proprietary. All rights reserved.