Changeset 0.19.0 (#81)
This commit is contained in:
340
README.md
340
README.md
@@ -1,277 +1,103 @@
|
|||||||
# Chat Switchboard
|
# Hotfix Patches — v0.18.1 → v0.18.2
|
||||||
|
|
||||||
A self-hosted AI chat application for enterprise and government environments. Unified interface for multiple AI providers with admin controls, team management, and security features.
|
Priority order: apply top-to-bottom. All patches are independent
|
||||||
|
except 1 and 2 which must be applied together.
|
||||||
|
|
||||||
## Quick Start
|
---
|
||||||
|
|
||||||
```bash
|
## Patch 1+2: Harden JSON scanning + SafeJSON (P0 — fixes broken chat list)
|
||||||
# Clone and start
|
|
||||||
git clone <repo-url> && cd chat-switchboard
|
|
||||||
docker compose up -d
|
|
||||||
|
|
||||||
# Access at http://localhost:3000
|
**Root cause:** Corrupt `\x02` byte in a channel's `settings` column.
|
||||||
# Default admin: admin / admin
|
`scanJSON` accepts it, Gin's `c.JSON()` starts writing 200 + headers,
|
||||||
|
`json.RawMessage.MarshalJSON()` hits the bad byte, aborts, browser
|
||||||
|
gets truncated body → "Unexpected end of JSON input" → 0 chats loaded.
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
|
||||||
|
1. **NEW:** `server/handlers/safe_json.go`
|
||||||
|
Copy `patches/server/handlers/safe_json.go` into your tree.
|
||||||
|
Contains: `SafeJSON()`, hardened `scanJSON()`, hardened `scanTags()`.
|
||||||
|
|
||||||
|
2. **NEW:** `server/handlers/safe_json_test.go`
|
||||||
|
Copy `patches/server/handlers/safe_json_test.go` into your tree.
|
||||||
|
|
||||||
|
3. **EDIT:** `server/handlers/channels.go`
|
||||||
|
Apply these three changes (or diff against `patches/server/handlers/channels.go`):
|
||||||
|
|
||||||
|
a. Remove the old `scanJSON` + `jsonScanner` + `scanTags` + `tagsScanner`
|
||||||
|
definitions (~lines 101–155). Replace with a comment:
|
||||||
|
```go
|
||||||
|
// scanJSON, scanTags, SafeJSON → safe_json.go
|
||||||
|
```
|
||||||
|
|
||||||
|
b. Replace `c.JSON(http.StatusOK, paginatedResponse{` in ListChannels
|
||||||
|
with `SafeJSON(c, http.StatusOK, paginatedResponse{`
|
||||||
|
|
||||||
|
c. Replace `c.JSON(http.StatusCreated, ch)` at end of CreateChannel
|
||||||
|
with `SafeJSON(c, http.StatusCreated, ch)`
|
||||||
|
|
||||||
|
d. Replace `c.JSON(http.StatusOK, ch)` at end of GetChannel
|
||||||
|
with `SafeJSON(c, http.StatusOK, ch)`
|
||||||
|
|
||||||
|
**Verify:** `go build ./...` and `go test ./server/handlers/ -run TestScanJSON -v`
|
||||||
|
|
||||||
|
**Data fix:** Find and repair the corrupt row:
|
||||||
|
```sql
|
||||||
|
-- Find it
|
||||||
|
SELECT id, title, length(settings::text),
|
||||||
|
encode(substring(settings::bytea, 1, 20), 'hex') AS hex_preview
|
||||||
|
FROM channels
|
||||||
|
WHERE settings IS NOT NULL
|
||||||
|
AND settings::text ~ '[^\x20-\x7E\t\n\r]';
|
||||||
|
|
||||||
|
-- Fix it (reset to empty settings)
|
||||||
|
UPDATE channels SET settings = '{}'
|
||||||
|
WHERE settings IS NOT NULL
|
||||||
|
AND settings::text ~ '[^\x20-\x7E\t\n\r]';
|
||||||
```
|
```
|
||||||
|
|
||||||
## Features
|
---
|
||||||
|
|
||||||
- **Multi-Provider**: Anthropic, OpenAI, OpenRouter, Venice AI — with BYOK support
|
## Patch 3: Service Worker scheme guard (P1 — console noise)
|
||||||
- **Team Management**: Roles (admin/user) for vertical permissions, Teams for horizontal visibility
|
|
||||||
- **Personas**: Preset configurations (model + system prompt + parameters) at global, team, or personal scope
|
|
||||||
- **Model Catalog**: Three-state visibility (enabled/disabled/team-only) with admin controls
|
|
||||||
- **Message Trees**: Edit and regenerate with full conversation forking — navigate sibling branches
|
|
||||||
- **Notes**: Markdown notes with full-text search, folders, and tags
|
|
||||||
- **Extensions**: Browser extension system with custom renderers, tool bridge, and self-contained styling
|
|
||||||
- Built-in: Mermaid diagrams, KaTeX math, CSV tables, diff viewer, JS sandbox, regex tester
|
|
||||||
- **Server Tools**: Calculator and datetime tools (auto-registered, zero config)
|
|
||||||
- **Audit Log**: All admin operations logged with actor, action, and resource details
|
|
||||||
- **Security**: JWT + refresh tokens, AES-256-GCM API key encryption, optional mTLS, optional OIDC/Keycloak, environment classification banners
|
|
||||||
- **Airgapped**: Local vendor files (marked.js, DOMPurify, KaTeX, mermaid.js), no CDN required
|
|
||||||
- **Mobile**: Responsive design, PWA manifest
|
|
||||||
- **Real-time**: WebSocket event bus for tool bridge, live updates
|
|
||||||
|
|
||||||
## Architecture
|
**Root cause:** SW fetch handler tries to cache `chrome-extension://`
|
||||||
|
URLs. `Cache.put()` rejects non-http(s) schemes.
|
||||||
|
|
||||||
```
|
**File:** `src/sw.js`
|
||||||
┌─────────────┐ ┌──────────────────────────────┐
|
|
||||||
│ Browser │────▶│ nginx (port 80) │
|
Add scheme guard after `const url = new URL(event.request.url);`:
|
||||||
│ (vanilla JS)│◀────│ ├─ /api/* → Go backend:8080 │
|
```js
|
||||||
└─────────────┘ │ ├─ /ws → WebSocket proxy │
|
// Only handle http/https — ignore chrome-extension://, moz-extension://, etc.
|
||||||
│ └─ /* → static files │
|
if (url.protocol !== 'https:' && url.protocol !== 'http:') {
|
||||||
└──────────────────────────────┘
|
return;
|
||||||
│
|
}
|
||||||
┌─────────▼─────────┐
|
|
||||||
│ Go Backend │
|
|
||||||
│ ├─ handlers/ │
|
|
||||||
│ ├─ store/postgres/ │
|
|
||||||
│ ├─ store/sqlite/ │
|
|
||||||
│ ├─ providers/ │
|
|
||||||
│ └─ capabilities/ │
|
|
||||||
└─────────┬─────────┘
|
|
||||||
│
|
|
||||||
┌─────────▼─────────┐
|
|
||||||
│ PostgreSQL 16 or │
|
|
||||||
│ SQLite (embedded) │
|
|
||||||
└───────────────────┘
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Go backend** (vanilla, no framework beyond Gin) with a store layer abstracting all database access. Dual-driver architecture: `store/postgres` for production deployments, `store/sqlite` for single-user, edge, and development scenarios — selected at startup via `DB_DRIVER`. Providers package handles LLM API calls. Capabilities package resolves model features from catalog data, known model tables, and heuristic inference. Server tools (calculator, datetime) auto-register via `init()`. EventBus + WebSocket hub routes tool calls between backend and browser extensions.
|
Or copy `patches/src/sw.js`.
|
||||||
|
|
||||||
**Frontend** is vanilla JavaScript — no build step, no bundler. 15 files organized by domain: `api.js` (HTTP client), `app.js` (state + init), `chat.js` (send, regen, edit, branch), `events.js` (event bus + WebSocket), `extensions.js` (loader, registry, renderer pipeline, tool bridge), `ui-core.js` (DOM rendering + streaming), `ui-format.js` (markdown, code blocks), `ui-primitives.js` (shared components), `ui-settings.js` / `ui-admin.js` (settings and admin panels), `notes.js`, `tokens.js`, `debug.js`, `settings-handlers.js`, `admin-handlers.js`.
|
---
|
||||||
|
|
||||||
## Configuration
|
## Patch 4: Settings debounce (P2 — cosmetic waste)
|
||||||
|
|
||||||
All configuration via environment variables. See `server/.env.example` for the full list.
|
Not included as a patch file — straightforward to implement:
|
||||||
|
- In `app.js` init sequence, debounce `API.updateSettings()` calls
|
||||||
|
- A 100ms `setTimeout` wrapper that coalesces multiple rapid saves
|
||||||
|
|
||||||
| Variable | Default | Description |
|
---
|
||||||
|----------|---------|-------------|
|
|
||||||
| `PORT` | `8080` | Backend listen port |
|
|
||||||
| `BASE_PATH` | ` ` | URL prefix (e.g. `/chat`) |
|
|
||||||
| `DB_DRIVER` | `postgres` | Database backend: `postgres` or `sqlite` |
|
|
||||||
| `DATABASE_URL` | ` ` | SQLite: path to database file (e.g. `/data/switchboard.db`) |
|
|
||||||
| `DB_HOST` | `localhost` | PostgreSQL host (ignored when `DB_DRIVER=sqlite`) |
|
|
||||||
| `DB_NAME` | `chat_switchboard` | Database name (ignored when `DB_DRIVER=sqlite`) |
|
|
||||||
| `JWT_SECRET` | (required) | Token signing key |
|
|
||||||
| `SWITCHBOARD_ADMIN_USERNAME` | ` ` | Bootstrap admin username |
|
|
||||||
| `SWITCHBOARD_ADMIN_PASSWORD` | ` ` | Bootstrap admin password |
|
|
||||||
| `ENCRYPTION_KEY` | ` ` | AES key for API key encryption (required if encrypted keys exist) |
|
|
||||||
| `SEED_USERS` | ` ` | Dev/test only: `user:pass:role,user2:pass2:role2` |
|
|
||||||
| `STORAGE_BACKEND` | (auto) | `pvc` or `s3`. Auto-detects PVC if not set. |
|
|
||||||
| `STORAGE_PATH` | `/data/storage` | PVC mount point (also scratch dir for S3 extraction) |
|
|
||||||
| `S3_ENDPOINT` | ` ` | S3 endpoint (e.g. `http://minio:9000`, required for S3) |
|
|
||||||
| `S3_BUCKET` | ` ` | S3 bucket name (must exist, required for S3) |
|
|
||||||
| `S3_ACCESS_KEY` | ` ` | S3 access key ID |
|
|
||||||
| `S3_SECRET_KEY` | ` ` | S3 secret access key |
|
|
||||||
| `S3_REGION` | `us-east-1` | S3 region |
|
|
||||||
| `S3_PREFIX` | ` ` | Optional key prefix within bucket |
|
|
||||||
| `S3_FORCE_PATH_STYLE` | `true` | Path-style URLs (required for MinIO, Ceph) |
|
|
||||||
|
|
||||||
## Deployment
|
## Future work (not in this hotfix)
|
||||||
|
|
||||||
Three Docker images support different scenarios:
|
- Apply `SafeJSON` to extension list endpoints (lines 47, 115 of
|
||||||
|
extensions.go) — same vulnerability pattern, lower risk since
|
||||||
|
extension data is seeded from known-good JSON.
|
||||||
|
|
||||||
| Image | Dockerfile | Use Case |
|
- Harden `JSONMap.Scan` in `models/models.go` to warn-and-default
|
||||||
|-------|-----------|----------|
|
instead of returning error. Currently it fails the whole query
|
||||||
| **Unified** | `Dockerfile` | Dev, docker-compose, single-node |
|
which gives a clean 500, but it means one corrupt row blocks all
|
||||||
| **Backend** | `server/Dockerfile` | K8s — scale API pods independently |
|
results in a list query.
|
||||||
| **Frontend** | `Dockerfile.frontend` | K8s — scale FE pods independently |
|
|
||||||
|
|
||||||
### Docker Compose (development — unified image)
|
- Startup JSON integrity check (design doc §17.1 Layer 4) — scan
|
||||||
|
all JSONB columns on boot, log warnings for corrupt data. Not
|
||||||
|
blocking for hotfix but valuable for post-upgrade diagnostics.
|
||||||
|
|
||||||
```bash
|
- JWT redaction in Gin log formatter for WebSocket URLs.
|
||||||
docker compose up -d # start all services
|
|
||||||
docker compose up -d --build # rebuild after code changes
|
|
||||||
docker compose --profile dev up # include Adminer DB UI
|
|
||||||
```
|
|
||||||
|
|
||||||
### Kubernetes (split images)
|
|
||||||
|
|
||||||
Build and push both images:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker build -f server/Dockerfile -t your-registry/switchboard-api:0.11.0 server/
|
|
||||||
docker build -f Dockerfile.frontend -t your-registry/switchboard-fe:0.11.0 .
|
|
||||||
```
|
|
||||||
|
|
||||||
**Backend deployment:**
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
containers:
|
|
||||||
- name: api
|
|
||||||
image: your-registry/switchboard-api:0.11.0
|
|
||||||
ports:
|
|
||||||
- containerPort: 8080
|
|
||||||
env:
|
|
||||||
- name: DB_HOST
|
|
||||||
value: "postgres-service"
|
|
||||||
- name: BASE_PATH
|
|
||||||
value: "/chat" # must match ingress path
|
|
||||||
- name: JWT_SECRET
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: switchboard-secrets
|
|
||||||
key: jwt-secret
|
|
||||||
```
|
|
||||||
|
|
||||||
**Frontend deployment:**
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
containers:
|
|
||||||
- name: frontend
|
|
||||||
image: your-registry/switchboard-fe:0.11.0
|
|
||||||
ports:
|
|
||||||
- containerPort: 80
|
|
||||||
env:
|
|
||||||
- name: BASE_PATH
|
|
||||||
value: "/chat" # injected into index.html at startup
|
|
||||||
volumeMounts:
|
|
||||||
- name: branding
|
|
||||||
mountPath: /branding
|
|
||||||
readOnly: true # optional: custom logo, colors
|
|
||||||
```
|
|
||||||
|
|
||||||
**Ingress** routes `/api/*` and `/ws` to the backend Service, everything else to the frontend Service. The frontend entrypoint generates the nginx config dynamically based on `BASE_PATH`.
|
|
||||||
|
|
||||||
### Path-Based Routing
|
|
||||||
|
|
||||||
Set `BASE_PATH=/chat` to serve the application under a subpath. The frontend reads `window.__BASE__` injected at container startup, and all API calls are prefixed automatically.
|
|
||||||
|
|
||||||
### Airgapped / Disconnected
|
|
||||||
|
|
||||||
The Docker build bakes in `marked.js`, `DOMPurify`, and `KaTeX` (JS + CSS + fonts) from npm during the vendor stage. Mermaid.js loads dynamically from local vendor with CDN fallback. No CDN calls at runtime in airgapped deployments. The `src/vendor/` directory contains local copies as fallback for development without Docker.
|
|
||||||
|
|
||||||
## Database
|
|
||||||
|
|
||||||
### PostgreSQL (default)
|
|
||||||
|
|
||||||
PostgreSQL 16+ recommended. The `pgcrypto` and `vector` (pgvector) extensions are used for UUID generation and vector similarity search respectively.
|
|
||||||
|
|
||||||
### SQLite (single-user / edge / dev)
|
|
||||||
|
|
||||||
Set `DB_DRIVER=sqlite` and `DATABASE_URL=/path/to/switchboard.db` to run with an embedded SQLite database. No external dependencies — the binary is self-contained (pure Go, no CGO). The SQLite backend has full feature parity with Postgres including knowledge base vector search, which uses app-level cosine similarity computed in Go rather than pgvector.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Minimal single-binary startup
|
|
||||||
DB_DRIVER=sqlite DATABASE_URL=./data/switchboard.db JWT_SECRET=changeme ./switchboard
|
|
||||||
```
|
|
||||||
|
|
||||||
SQLite is best suited for single-user workstations, edge deployments, air-gapped laptops, and local development. For multi-user production with concurrent writes, use PostgreSQL.
|
|
||||||
|
|
||||||
### Schema Management
|
|
||||||
|
|
||||||
The Go backend auto-migrates on startup using files in `server/database/migrations/`. For manual operations:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Bootstrap database (superuser, creates role + DB)
|
|
||||||
scripts/db-bootstrap.sh
|
|
||||||
|
|
||||||
# Manual migration (usually not needed)
|
|
||||||
scripts/db-migrate.sh
|
|
||||||
|
|
||||||
# Validate schema
|
|
||||||
scripts/db-validate.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key Tables (v0.11)
|
|
||||||
|
|
||||||
| Table | Purpose |
|
|
||||||
|-------|---------|
|
|
||||||
| `users` | Accounts with role, avatar, settings, encrypted vault (UEK) |
|
|
||||||
| `provider_configs` | API provider configurations (scope: global/team/personal) |
|
|
||||||
| `model_catalog` | Synced model list with capabilities, visibility, and type |
|
|
||||||
| `personas` | Model presets (scope: global/team/personal) |
|
|
||||||
| `channels` | Conversations with type (direct/group/channel) |
|
|
||||||
| `messages` | Message tree with parent_id for forking, tool_calls JSONB |
|
|
||||||
| `teams` / `team_members` | Organizational units |
|
|
||||||
| `notes` | Markdown notes with full-text search |
|
|
||||||
| `extensions` / `extension_user_settings` | Browser extension registry and per-user config |
|
|
||||||
| `usage_log` / `model_pricing` | Token usage tracking and cost calculation |
|
|
||||||
| `audit_log` | Admin action audit trail |
|
|
||||||
| `user_model_settings` | Per-user model visibility and sort preferences |
|
|
||||||
|
|
||||||
## API
|
|
||||||
|
|
||||||
All endpoints under `/api/v1/`. Authentication via `Authorization: Bearer <token>` header.
|
|
||||||
|
|
||||||
### Auth
|
|
||||||
- `POST /auth/register` — Register (if `allow_registration` policy is true)
|
|
||||||
- `POST /auth/login` — Login, returns access + refresh tokens
|
|
||||||
- `POST /auth/refresh` — Refresh access token
|
|
||||||
- `POST /auth/logout` — Revoke refresh token
|
|
||||||
|
|
||||||
### Channels & Messages
|
|
||||||
- `GET/POST /channels` — List/create conversations
|
|
||||||
- `GET/PUT/DELETE /channels/:id` — Channel CRUD
|
|
||||||
- `GET/POST /channels/:id/messages` — Message list/create
|
|
||||||
- `POST /chat/completions` — Stream AI completions (SSE)
|
|
||||||
- `POST /channels/:id/messages/:msgId/edit` — Edit and fork
|
|
||||||
- `POST /channels/:id/messages/:msgId/regenerate` — Regenerate response
|
|
||||||
|
|
||||||
### Models
|
|
||||||
- `GET /models/enabled` — Models available to the user (global + team + personal)
|
|
||||||
- `GET/PUT /models/preferences` — User model visibility settings
|
|
||||||
|
|
||||||
### Personal Providers (BYOK)
|
|
||||||
- `GET/POST /api-configs` — User's personal provider configs
|
|
||||||
- `GET/PUT/DELETE /api-configs/:id` — Provider CRUD
|
|
||||||
- `POST /api-configs/:id/models/fetch` — Refresh models from provider API
|
|
||||||
- `GET /api-configs/:id/models` — List models for a personal provider
|
|
||||||
|
|
||||||
### Teams
|
|
||||||
- `GET /teams` — Teams the user belongs to
|
|
||||||
- `GET/POST /teams/:id/providers` — Team provider management (team admins)
|
|
||||||
- `GET/POST /teams/:id/presets` — Team preset management (team admins)
|
|
||||||
|
|
||||||
### Admin
|
|
||||||
- `GET/POST /admin/users` — User management
|
|
||||||
- `GET/PUT /admin/settings/:key` — Global settings and policies
|
|
||||||
- `GET/POST /admin/configs` — Global provider configs
|
|
||||||
- `GET/POST /admin/models/fetch` — Model catalog sync
|
|
||||||
- `GET/PUT /admin/roles/:role` — Model role configuration
|
|
||||||
- `GET /admin/usage` — Usage dashboard
|
|
||||||
- `GET /admin/audit` — Audit log
|
|
||||||
- `GET/POST/PUT/DELETE /admin/extensions` — Extension management
|
|
||||||
|
|
||||||
### Extensions
|
|
||||||
- `GET /extensions?tier=browser` — List enabled browser extensions
|
|
||||||
- `GET /extensions/:id/assets/*path` — Serve extension assets (public, no auth)
|
|
||||||
|
|
||||||
### WebSocket
|
|
||||||
- `GET /ws?token=<jwt>` — EventBus WebSocket for real-time events and tool bridge
|
|
||||||
|
|
||||||
## Development
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Backend (requires Go 1.22+)
|
|
||||||
cd server
|
|
||||||
cp .env.example .env # edit with your DB credentials
|
|
||||||
go run .
|
|
||||||
|
|
||||||
# Frontend (just serve static files)
|
|
||||||
# Use any HTTP server pointed at src/
|
|
||||||
python3 -m http.server 3000 --directory src
|
|
||||||
```
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
Proprietary. All rights reserved.
|
|
||||||
|
|||||||
@@ -98,60 +98,7 @@ func writeTagsArg(tags []string) interface{} {
|
|||||||
return pq.Array(tags)
|
return pq.Array(tags)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── JSON scanner (settings column) ──────────
|
// scanJSON, scanTags, SafeJSON → safe_json.go
|
||||||
// SQLite returns TEXT for JSON columns; json.RawMessage ([]byte) can't scan
|
|
||||||
// from a string with modernc.org/sqlite. This wrapper handles both dialects.
|
|
||||||
|
|
||||||
type jsonScanner struct{ dest *json.RawMessage }
|
|
||||||
|
|
||||||
func scanJSON(dest *json.RawMessage) *jsonScanner { return &jsonScanner{dest: dest} }
|
|
||||||
|
|
||||||
func (s *jsonScanner) Scan(src interface{}) error {
|
|
||||||
switch v := src.(type) {
|
|
||||||
case []byte:
|
|
||||||
*s.dest = json.RawMessage(v)
|
|
||||||
case string:
|
|
||||||
*s.dest = json.RawMessage(v)
|
|
||||||
case nil:
|
|
||||||
*s.dest = json.RawMessage("{}")
|
|
||||||
default:
|
|
||||||
*s.dest = json.RawMessage("{}")
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// tagsScanner wraps a *[]string so rows.Scan can populate it on both dialects.
|
|
||||||
type tagsScanner struct {
|
|
||||||
dest *[]string
|
|
||||||
}
|
|
||||||
|
|
||||||
func scanTags(dest *[]string) *tagsScanner {
|
|
||||||
return &tagsScanner{dest: dest}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *tagsScanner) Scan(src interface{}) error {
|
|
||||||
if database.IsSQLite() {
|
|
||||||
var raw string
|
|
||||||
switch v := src.(type) {
|
|
||||||
case string:
|
|
||||||
raw = v
|
|
||||||
case []byte:
|
|
||||||
raw = string(v)
|
|
||||||
case nil:
|
|
||||||
*s.dest = []string{}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
var arr []string
|
|
||||||
if err := json.Unmarshal([]byte(raw), &arr); err != nil {
|
|
||||||
*s.dest = []string{}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
*s.dest = arr
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
// Postgres: delegate to pq
|
|
||||||
return pq.Array(s.dest).Scan(src)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Helpers ─────────────────────────────────
|
// ── Helpers ─────────────────────────────────
|
||||||
|
|
||||||
@@ -281,7 +228,7 @@ func (h *ChannelHandler) ListChannels(c *gin.Context) {
|
|||||||
channels = append(channels, ch)
|
channels = append(channels, ch)
|
||||||
}
|
}
|
||||||
|
|
||||||
c.JSON(http.StatusOK, paginatedResponse{
|
SafeJSON(c, http.StatusOK, paginatedResponse{
|
||||||
Data: channels,
|
Data: channels,
|
||||||
Page: page,
|
Page: page,
|
||||||
PerPage: perPage,
|
PerPage: perPage,
|
||||||
@@ -399,7 +346,7 @@ func (h *ChannelHandler) CreateChannel(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
c.JSON(http.StatusCreated, ch)
|
SafeJSON(c, http.StatusCreated, ch)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Get Channel ─────────────────────────────
|
// ── Get Channel ─────────────────────────────
|
||||||
@@ -441,7 +388,7 @@ func (h *ChannelHandler) GetChannel(c *gin.Context) {
|
|||||||
}
|
}
|
||||||
ch.Tags = tags
|
ch.Tags = tags
|
||||||
|
|
||||||
c.JSON(http.StatusOK, ch)
|
SafeJSON(c, http.StatusOK, ch)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Update Channel ──────────────────────────
|
// ── Update Channel ──────────────────────────
|
||||||
|
|||||||
116
server/handlers/safe_json.go
Normal file
116
server/handlers/safe_json.go
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
"github.com/lib/pq"
|
||||||
|
|
||||||
|
"git.gobha.me/xcaliber/chat-switchboard/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ── SafeJSON ────────────────────────────────
|
||||||
|
// SafeJSON pre-marshals the response body, guaranteeing that a
|
||||||
|
// serialisation failure produces a clean 500 instead of a truncated
|
||||||
|
// 200 with half-written JSON. Use for any endpoint whose response
|
||||||
|
// contains json.RawMessage or other pass-through data from the DB.
|
||||||
|
|
||||||
|
func SafeJSON(c *gin.Context, code int, obj interface{}) {
|
||||||
|
body, err := json.Marshal(obj)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("⚠ SafeJSON: marshal failed: %v", err)
|
||||||
|
c.JSON(http.StatusInternalServerError, gin.H{
|
||||||
|
"error": "response serialization failed",
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
c.Data(code, "application/json; charset=utf-8", body)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── JSON column scanner ─────────────────────
|
||||||
|
// Handles both Postgres ([]byte) and SQLite (string) return types.
|
||||||
|
// Validates content before accepting it — corrupt data is replaced
|
||||||
|
// with "{}" and a warning is logged so admins can find the bad row.
|
||||||
|
|
||||||
|
type jsonScanner struct{ dest *json.RawMessage }
|
||||||
|
|
||||||
|
func scanJSON(dest *json.RawMessage) *jsonScanner { return &jsonScanner{dest: dest} }
|
||||||
|
|
||||||
|
func (s *jsonScanner) Scan(src interface{}) error {
|
||||||
|
switch v := src.(type) {
|
||||||
|
case []byte:
|
||||||
|
if len(v) == 0 || !json.Valid(v) {
|
||||||
|
log.Printf("⚠ scanJSON: invalid JSON in column (len=%d, preview=%.60q), defaulting to {}", len(v), v)
|
||||||
|
*s.dest = json.RawMessage("{}")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
*s.dest = json.RawMessage(v)
|
||||||
|
case string:
|
||||||
|
b := []byte(v)
|
||||||
|
if len(b) == 0 || !json.Valid(b) {
|
||||||
|
log.Printf("⚠ scanJSON: invalid JSON in column (len=%d, preview=%.60q), defaulting to {}", len(b), b)
|
||||||
|
*s.dest = json.RawMessage("{}")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
*s.dest = json.RawMessage(b)
|
||||||
|
case nil:
|
||||||
|
*s.dest = json.RawMessage("{}")
|
||||||
|
default:
|
||||||
|
*s.dest = json.RawMessage("{}")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Tags column scanner ─────────────────────
|
||||||
|
// SQLite stores tags as a JSON text array; Postgres uses text[].
|
||||||
|
// Both paths validate and fall back to an empty slice on error.
|
||||||
|
|
||||||
|
type tagsScanner struct {
|
||||||
|
dest *[]string
|
||||||
|
}
|
||||||
|
|
||||||
|
func scanTags(dest *[]string) *tagsScanner {
|
||||||
|
return &tagsScanner{dest: dest}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *tagsScanner) Scan(src interface{}) error {
|
||||||
|
if src == nil {
|
||||||
|
*s.dest = []string{}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if database.IsSQLite() {
|
||||||
|
var raw string
|
||||||
|
switch v := src.(type) {
|
||||||
|
case string:
|
||||||
|
raw = v
|
||||||
|
case []byte:
|
||||||
|
raw = string(v)
|
||||||
|
default:
|
||||||
|
*s.dest = []string{}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if raw == "" {
|
||||||
|
*s.dest = []string{}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var arr []string
|
||||||
|
if err := json.Unmarshal([]byte(raw), &arr); err != nil {
|
||||||
|
log.Printf("⚠ scanTags: invalid JSON array in column (preview=%.60q), defaulting to []", raw)
|
||||||
|
*s.dest = []string{}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
*s.dest = arr
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Postgres: delegate to pq, but catch errors
|
||||||
|
if err := pq.Array(s.dest).Scan(src); err != nil {
|
||||||
|
log.Printf("⚠ scanTags: pq.Array scan failed (%v), defaulting to []", err)
|
||||||
|
*s.dest = []string{}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
185
server/handlers/safe_json_test.go
Normal file
185
server/handlers/safe_json_test.go
Normal file
@@ -0,0 +1,185 @@
|
|||||||
|
package handlers
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/gin-gonic/gin"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestScanJSON_ValidObject(t *testing.T) {
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
if err := s.Scan([]byte(`{"key":"value"}`)); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != `{"key":"value"}` {
|
||||||
|
t.Fatalf("expected {\"key\":\"value\"}, got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScanJSON_ValidArray(t *testing.T) {
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
if err := s.Scan([]byte(`[1,2,3]`)); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != `[1,2,3]` {
|
||||||
|
t.Fatalf("expected [1,2,3], got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScanJSON_EmptyString(t *testing.T) {
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
if err := s.Scan(""); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != "{}" {
|
||||||
|
t.Fatalf("expected {}, got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScanJSON_Nil(t *testing.T) {
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
if err := s.Scan(nil); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != "{}" {
|
||||||
|
t.Fatalf("expected {}, got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScanJSON_BinaryGarbage(t *testing.T) {
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
// The exact scenario from the bug: STX control character
|
||||||
|
if err := s.Scan([]byte{0x02, 0x03, 0x04}); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != "{}" {
|
||||||
|
t.Fatalf("expected {} for binary garbage, got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScanJSON_TruncatedJSON(t *testing.T) {
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
if err := s.Scan([]byte(`{"key":"val`)); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != "{}" {
|
||||||
|
t.Fatalf("expected {} for truncated JSON, got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScanJSON_StringDialect(t *testing.T) {
|
||||||
|
// SQLite returns strings, not []byte
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
if err := s.Scan(`{"sqlite":"mode"}`); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != `{"sqlite":"mode"}` {
|
||||||
|
t.Fatalf("expected {\"sqlite\":\"mode\"}, got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestScanJSON_StringDialect_Corrupt(t *testing.T) {
|
||||||
|
var dest json.RawMessage
|
||||||
|
s := scanJSON(&dest)
|
||||||
|
if err := s.Scan("\x02\x03garbage"); err != nil {
|
||||||
|
t.Fatalf("unexpected error: %v", err)
|
||||||
|
}
|
||||||
|
if string(dest) != "{}" {
|
||||||
|
t.Fatalf("expected {} for corrupt string, got %s", dest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSafeJSON_ValidPayload(t *testing.T) {
|
||||||
|
gin.SetMode(gin.TestMode)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
c, _ := gin.CreateTestContext(w)
|
||||||
|
|
||||||
|
SafeJSON(c, http.StatusOK, gin.H{"status": "ok"})
|
||||||
|
|
||||||
|
if w.Code != http.StatusOK {
|
||||||
|
t.Fatalf("expected 200, got %d", w.Code)
|
||||||
|
}
|
||||||
|
if ct := w.Header().Get("Content-Type"); ct != "application/json; charset=utf-8" {
|
||||||
|
t.Fatalf("expected JSON content-type, got %s", ct)
|
||||||
|
}
|
||||||
|
|
||||||
|
var result map[string]string
|
||||||
|
if err := json.Unmarshal(w.Body.Bytes(), &result); err != nil {
|
||||||
|
t.Fatalf("response is not valid JSON: %v", err)
|
||||||
|
}
|
||||||
|
if result["status"] != "ok" {
|
||||||
|
t.Fatalf("expected status=ok, got %s", result["status"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSafeJSON_CorruptRawMessage(t *testing.T) {
|
||||||
|
gin.SetMode(gin.TestMode)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
c, _ := gin.CreateTestContext(w)
|
||||||
|
|
||||||
|
// Simulate the exact bug: a struct with a corrupt json.RawMessage
|
||||||
|
type response struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Settings json.RawMessage `json:"settings"`
|
||||||
|
}
|
||||||
|
resp := response{
|
||||||
|
ID: "test-123",
|
||||||
|
Settings: json.RawMessage{0x02, 0x03}, // binary garbage
|
||||||
|
}
|
||||||
|
|
||||||
|
SafeJSON(c, http.StatusOK, resp)
|
||||||
|
|
||||||
|
// Should get 500, not a truncated 200
|
||||||
|
if w.Code != http.StatusInternalServerError {
|
||||||
|
t.Fatalf("expected 500 for corrupt RawMessage, got %d", w.Code)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Body should be valid JSON with error message
|
||||||
|
var result map[string]string
|
||||||
|
if err := json.Unmarshal(w.Body.Bytes(), &result); err != nil {
|
||||||
|
t.Fatalf("error response is not valid JSON: %v", err)
|
||||||
|
}
|
||||||
|
if result["error"] != "response serialization failed" {
|
||||||
|
t.Fatalf("unexpected error message: %s", result["error"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSafeJSON_PaginatedWithCorrupt(t *testing.T) {
|
||||||
|
gin.SetMode(gin.TestMode)
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
c, _ := gin.CreateTestContext(w)
|
||||||
|
|
||||||
|
// If scanJSON is hardened, this should never happen — but SafeJSON
|
||||||
|
// is the second line of defense if something slips through.
|
||||||
|
type item struct {
|
||||||
|
Settings json.RawMessage `json:"settings"`
|
||||||
|
}
|
||||||
|
type paginated struct {
|
||||||
|
Data []item `json:"data"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
}
|
||||||
|
|
||||||
|
resp := paginated{
|
||||||
|
Data: []item{
|
||||||
|
{Settings: json.RawMessage(`{"ok":true}`)},
|
||||||
|
{Settings: json.RawMessage{0xFF, 0xFE}}, // corrupt
|
||||||
|
},
|
||||||
|
Total: 2,
|
||||||
|
}
|
||||||
|
|
||||||
|
SafeJSON(c, http.StatusOK, resp)
|
||||||
|
|
||||||
|
if w.Code != http.StatusInternalServerError {
|
||||||
|
t.Fatalf("expected 500 for paginated response with corrupt item, got %d", w.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -68,6 +68,11 @@ self.addEventListener('activate', (event) => {
|
|||||||
self.addEventListener('fetch', (event) => {
|
self.addEventListener('fetch', (event) => {
|
||||||
const url = new URL(event.request.url);
|
const url = new URL(event.request.url);
|
||||||
|
|
||||||
|
// Only handle http/https — ignore chrome-extension://, moz-extension://, etc.
|
||||||
|
if (url.protocol !== 'https:' && url.protocol !== 'http:') {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
// Never cache API calls, WebSocket upgrades, branding, extensions, or CM6 bundle
|
// Never cache API calls, WebSocket upgrades, branding, extensions, or CM6 bundle
|
||||||
if (url.pathname.includes('/api/') ||
|
if (url.pathname.includes('/api/') ||
|
||||||
url.pathname.includes('/ws') ||
|
url.pathname.includes('/ws') ||
|
||||||
|
|||||||
Reference in New Issue
Block a user