Changeset 0.9.0 (#50)

This commit is contained in:
2026-02-23 01:57:28 +00:00
parent 15be26c516
commit 8264aa6016
94 changed files with 9812 additions and 8574 deletions

View File

@@ -1,460 +1,44 @@
# 🗺️ Chat Switchboard — Roadmap
**See also:**
- [ARCHITECTURE.md](ARCHITECTURE.md) — Core services design (Notes, KBs, Tasks, Channels, Embeddings)
- [EXTENSIONS.md](EXTENSIONS.md) — Extension system spec (Browser/Starlark/Sidecar tiers, tools, surfaces)
**Versioning (pre-1.0):** `0.<major>.<minor>` — hotfixes use quad: `0.x.y.z`
No compatibility guarantees before 1.0. Post-1.0: major = long-lived (break nothing),
minor = add features (deprecate, don't remove), patch = fix something.
---
## Current State: v0.7.4
### ✅ Done
**Backend (Go)**
- [x] PostgreSQL schema + auto-migrations (go:embed, startup)
- [x] JWT auth with refresh token rotation
- [x] **Unified channel model** — chats→channels, "everything is a channel"
- [x] Channel CRUD + message persistence (`/api/v1/channels`)
- [x] Message tree (`parent_id`) with linear backfill
- [x] Participant tracking (`participant_type`/`participant_id` on messages)
- [x] `channel_members` + `channel_models` tables (schema foundation)
- [x] `channel_cursors` for branch tracking (schema foundation)
- [x] Folders + Projects tables for organization
- [x] Environment banner system (global_settings, admin presets, position control)
- [x] Streaming completion proxy (OpenAI + Anthropic + Venice + OpenRouter)
- [x] Per-user + global API config management
- [x] Admin endpoints (users, providers, models, settings, stats)
- [x] Admin bootstrap from env vars (K8s secret → upsert on every restart)
- [x] Registration with pending state (admin approval workflow)
- [x] Registration default state setting (active / pending)
- [x] User providers toggle (admin can restrict to global-only providers)
- [x] Bulk model enable/disable
- [x] Public settings endpoint (non-admin users read safe subset)
- [x] Rate limiting, error middleware, request logging
- [x] Health check endpoint (includes schema_version)
- [x] EventBus with WebSocket hub (rooms, JWT auth, heartbeat, reconnection)
- [x] Provider capability system (known models, heuristic detection, resolution chain)
- [x] Dynamic max_tokens resolution
- [x] User + admin provider model listing with capabilities
- [x] Model presets (named wrappers: global/personal scope, system prompt, temp, max_tokens)
- [x] Preset unwrap in completion handler (transparent to provider)
- [x] Tool framework: registry, type system, execution loop (tool→result→model)
- [x] Note tools: note_create, note_search, note_update, note_list
- [x] Notes API: CRUD, full-text search (tsvector), folder listing
- [x] Provider tool calling: OpenAI + Anthropic function calling in request/response/stream
- [x] Message tree: edit creates sibling, regenerate creates sibling, cursor tracking
- [x] Avatar system: user + preset avatars, server-side resize to 128×128 PNG
**Frontend (Vanilla JS)**
- [x] Professional splash page (split-panel hero + tabbed auth)
- [x] Split "New Chat" button with dropdown (Group Chat, Channel — coming soon)
- [x] Environment banner system (CSS custom props + JS init from settings)
- [x] Collapsible sidebar with time-grouped chat history
- [x] User menu flyout (Settings, Admin, Debug, Sign Out)
- [x] Model selector with capability badges (output, context, tools, vision, thinking)
- [x] Settings modal with tabs (General, Providers, Models)
- [x] Admin modal with tabs (Users, Providers, Models, Settings, Stats)
- [x] Admin settings: registration, user providers, banner config with presets
- [x] Pending user badge + approve workflow in admin Users tab
- [x] Streaming SSE display with smart scroll
- [x] Full markdown rendering (marked.js + DOMPurify, vendor + CDN fallback)
- [x] Thinking block display (`<think>`/`<thinking>` tags)
- [x] Debug modal (console intercept, network log, state inspector)
- [x] EventBus client with exponential backoff + max retries
- [x] Export (Markdown, JSON, Text)
- [x] Model selector with preset grouping (⚡ Presets optgroup)
- [x] Admin Presets tab (create, toggle, delete global presets)
- [x] Preset-aware completion flow (preset_id sent to backend)
- [x] Custom dropdown for model selector (full CSS control, dark theme)
- [x] Admin edit buttons for providers and presets (inline form reuse)
- [x] Appearance settings tab (UI scale, message font size)
- [x] Mobile responsive layout (hamburger menu, sidebar overlay, dvh)
- [x] Model/preset name in message headers (replaces generic "Assistant")
- [x] Notes modal: list/detail views, folder sidebar, create/edit/delete, search
- [x] Message editing + forking: inline edit, regen, branch navigation 1/2
- [x] Conversation path: active-path context assembly, cursor-aware
- [x] White-label branding: volume mount, org name, logo, favicon, accent color, pills
- [x] Avatar system: user profile upload, preset avatars, avatar in messages/sidebar/dropdown
- [x] Chat search / filter in sidebar (real-time title filtering)
- [x] Command palette (Ctrl+K) with fuzzy search, chat jumping, keyboard nav
- [x] PWA manifest + service worker (offline shell, install prompt)
**CI/CD (Gitea Actions)**
- [x] Three-env pipeline (dev/test/prod)
- [x] Dev: upgrade test → schema validate → wipe → fresh install test
- [x] Backend auto-migrates on startup (no CI migration step)
- [x] Admin secret sync (Gitea secrets → K8s secret → env vars)
- [x] Shared PG safety (_dev suffix guard on wipe)
- [x] Post-deploy schema verification via /api/v1/health
**Architecture Design**
- [x] ARCHITECTURE.md: core services spec (13 sections)
- [x] EXTENSIONS.md: three-tier extension system spec
- [x] "Everything is a channel" — unified model with type:'direct'
- [x] Message tree (parent_id for conversation forking)
- [x] Auth strategy design (builtin/mTLS/OIDC)
---
## ~~0.7.0 — Custom Models / Presets~~ ✅
Named wrappers around base models with bundled configuration. Admins create
org-wide presets, users create personal ones (if user providers are enabled).
- [x] `model_presets` table: name, base_model_id, system_prompt, temperature,
max_tokens, tools_enabled (jsonb), created_by, scope (global/team/personal),
team_id (nullable, for future team scoping), is_shared
- [x] Permission gating: personal presets require user_providers_enabled
- [x] Admin preset management UI (create, edit, delete org-wide presets)
- [x] User preset management UI in Settings (personal presets)
- [x] Presets appear as first-class entries in model selector
("Code Reviewer (GPT-4o)", "Research Assistant (Claude Opus)")
- [x] Completion handler unwraps preset → base model + config overrides
## ~~0.7.x — Branding + Polish~~ ✅
White-label support and UX improvements that exploit existing infrastructure.
**Branding**
- [x] Admin branding settings in global_settings: org name, tagline, accent color
- [x] `/branding/` volume mount (K8s ConfigMap, optional) for favicon, logo
- [x] Splash page reads branding config on load, falls back to Switchboard defaults
- [x] CSS accent color override from branding settings
**Profile Pictures / Avatars**
- [x] `avatar_url` column on `users` table (already existed, now wired)
- [x] `avatar` column on `model_presets` (migration 015)
- [x] Avatar upload in user Settings → Profile section (server-side resize to 128×128)
- [x] Admin/user preset avatar upload endpoint
- [x] `avatarHTML()` helper: returns `<img>` if avatar set, emoji fallback otherwise
- [x] Avatar displayed in message headers, stream, typing indicator, sidebar user area
**Message Editing + Forking**
- [x] Edit message → creates sibling (uses existing parent_id tree)
- [x] Regenerate → creates sibling model response
- [x] Branch indicator at fork points (← 1/2 →)
- [x] Context assembly follows active path, not full channel history
**UX Polish**
- [x] Chat search / filter in sidebar
- [x] UI preferences: font size + UI scale (Appearance tab, localStorage)
- [x] Mobile responsive: hamburger menu, sidebar overlay, auto-collapse
- [x] Model name in message headers (preset name or model ID, not "Assistant")
- [x] Keyboard shortcuts (Ctrl+K command palette)
- [x] PWA manifest + offline shell + install prompt
---
## 0.8.0 — Teams + Onboarding
The missing middle tier: scoped administration without system-admin access.
**Schema**
- [x] `teams` table: id, name, description, created_by
- [x] `team_members` table: team_id, user_id, role ('admin' | 'member')
- [x] Add `team_id` (nullable) to: model_presets, channels, notes
- [x] Team admin role: scoped per-team, not system-wide
(one person can be admin of Team A, member of Team B)
**Onboarding**
- [x] Registration approval → assign role + team(s) during approval
(upgrade from current binary approve → assign-and-activate)
- [x] System admin creates teams + assigns first team admin
- [x] Team admin self-manages: add/remove members
(RequireTeamAdmin middleware, scoped /teams/:teamId routes)
- [x] Team admin: create team presets (Settings → Teams tab, self-service UI)
- [ ] Team admin: manage team channels, view team usage
**Private Provider Policy**
- [x] `is_private` flag on provider configs (marks local/self-hosted endpoints)
- [x] `require_private_providers` policy per team (via teams.settings JSONB)
- [x] Completion handler enforces policy — team members restricted to private providers
- [ ] Enables HIPAA/compliance posture: provable data boundary per team
## 0.8.x — Audit + Usage Tracking
Required for enterprise and compliance. Cheap to build, expensive to retrofit.
**Audit Log**
- [x] `audit_log` table: actor_id, action, resource_type, resource_id,
metadata (jsonb), ip_address, timestamp
- [x] Every mutating handler inserts audit entry
(user CRUD, team CRUD, member management, presets, auth)
- [x] Admin audit viewer (filter by action, resource type, paginated)
- [ ] Team admin sees audit entries scoped to their team
**Model Visibility (0.8.3)**
- [x] Three-state visibility: enabled (all users) / team (preset-building only) / disabled
- [x] Migration replaces `is_enabled` boolean with `visibility` varchar
- [x] Admin model list: 3-state cycle button, bulk set all to any state
- [x] `GET /teams/:teamId/models` returns enabled+team models for preset builders
- [x] User-facing `ListEnabledModels` strictly filters `visibility = 'enabled'`
**Preset Form Unification + UX (0.8.4)**
- [x] Single `renderPresetForm(options)` function shared across admin/team contexts
— parameterized: showAvatar, showProviderConfig, onSubmit, onCancel
- [x] Remove icon field from preset forms (not rendered anywhere meaningful)
- [x] Team admin preset form includes avatar upload (reuse admin component)
- [x] Hide Providers tab in Settings when `user_providers_enabled` is false
- [x] Settings Models tab: only show `visibility='enabled'` base models
(team-only models must not leak to user model list; presets filtered out)
- [x] Model provenance labels: badge indicating global vs personal source
— same model_id from both sources shown separately (different API keys)
- [x] Admin presets list: show created_by name + team name for attribution
(e.g. "Code Reviewer · 👥 Engineering · by sarah")
**User Presets + Model Filtering (0.8.5)**
- [x] Users can create personal presets from any enabled base model
(removed user_providers_enabled gate; uses shared renderPresetForm)
- [x] Settings Models tab reworked as "My Models": toggle to hide/show
base models from selector, "My Presets" section with create/delete
- [x] `user_model_preferences` table: user_id, model_id, hidden boolean
— lightweight filter, not access control
- [x] Hidden models filtered from main model selector dropdown
- [x] User personal provider models: simple enable/disable toggle
(same visibility toggle — hide from selector)
**Team Providers (0.8.6)**
- [x] `team_id` column on `api_configs` (nullable FK, indexed)
- [x] Team admins manage team-scoped provider configs
(add/remove API keys, toggle active, same UI as personal providers)
- [x] `allow_team_providers` check: global_settings + team.settings JSONB
- [x] Team provider models available in team preset builder (ListAvailableModels)
— grouped by source: Global Models / Team Provider Models (optgroup)
- [x] Team provider models NOT exposed directly in model selector
— team members access team models ONLY through curated presets
— keeps model selector clean, avoids visibility confusion
- [x] Three-tier provider hierarchy: global → team (presets only) → personal
- [x] ListConfigs excludes team-scoped providers (team_id IS NULL filter)
**Usage / Cost Tracking**
- [ ] Capture from provider responses: prompt_tokens, completion_tokens,
cache_creation_tokens, cache_read_tokens
- [ ] `usage_log` table: channel_id, user_id, model_id, provider_id,
token counts, timestamp
- [ ] Model pricing fields on `model_configs`:
cost_input, cost_output, cost_cache_input, cost_cache_output
(per M tokens, nullable, decimal)
- [ ] `cost_source` field: 'provider' | 'admin' | null
— provider APIs populate on model fetch, admin can override,
source tracking prevents silent overwrites on re-fetch
- [ ] Cost calculated at query time (join usage × pricing)
— never store computed cost; pricing changes and historical
token counts should recalculate against current rates
- [ ] Per-user and per-team usage stats in admin panel
- [ ] Team admins see usage for their team scope
---
## ~~0.9.0 — Tool Execution + Notes~~ ✅ (pulled into 0.7.x)
Pulled forward and shipped ahead of schedule. See TOOLS_IMPL.md and FORKING_IMPL.md.
**Tool Framework**
- [x] Tool calling pipeline in completion handler (OpenAI + Anthropic)
- [x] Tool registry (built-in + future plugin tools)
- [x] Tool execution loop: model requests tool → backend executes → result fed back
- [ ] Tool permission model (which tools enabled per preset/channel)
**Notes**
- [x] `notes` table: title, content, folder, tags, source_channel_id, full-text search
- [x] Notes CRUD endpoints + search
- [x] `note_create`, `note_update`, `note_search`, `note_list` tools
- [x] Full-text search (PostgreSQL `tsvector` + `ts_rank` + `ts_headline`)
- [x] Notes UI: modal with list/detail views, folder sidebar, Markdown editor
- [ ] Team-scoped notes (awaits 0.8.0 teams)
**Conversation Forking UI**
- [x] Edit-and-resubmit creates siblings (tree structure)
- [x] Branch indicator ← 1/2 → at fork points
- [x] Context assembly follows active path
## 0.9.x — Context Management
Usability before compaction exists — long conversations shouldn't silently fail.
- [ ] Token counting on outbound requests (estimate before sending)
- [ ] Truncation strategy (sliding window or drop oldest, configurable)
- [ ] "Conversation is getting long" warning in UI
- [ ] Manual "summarize and continue" action (user-triggered pre-compaction)
- [ ] Groundwork for automated compaction in 0.14.0
---
## 0.10.0 — Web Search + URL Fetch
First external tools, using the tool framework shipped in 0.7.x.
- [ ] `web_search` tool: search provider abstraction (DuckDuckGo, SearXNG, Brave)
- [ ] `url_fetch` tool: retrieve and extract content from URLs
- [ ] Sidecar or direct HTTP from backend (configurable)
- [ ] Results injected into context
- [ ] Paired with Notes: "research X and save findings to my notes"
---
## 0.11.0 — File Handling + Vision
File input into chat — table stakes for serious use. Blobs live in object
storage, metadata and search indexes live in PostgreSQL.
**Storage Backend Abstraction**
- [ ] S3-compatible API as primary interface
(MinIO, Ceph RGW, AWS S3, GCS — anything with S3 API)
- [ ] Local PVC as zero-config fallback (single-node / dev)
- [ ] Admin config: storage backend selection, endpoint, credentials,
bucket/path, per-file and total size limits
- [ ] Reused by 0.13.0 (KB documents) and 0.14.0 (compaction snapshots)
**Metadata + Search Index (PostgreSQL)**
- [ ] `attachments` table (metadata only, never blobs):
id, message_id, channel_id, team_id, filename, mime_type,
size_bytes, storage_backend ('s3' | 'pvc'), storage_path,
checksum_sha256, uploaded_by, created_at,
search_text (tsvector, nullable)
- [ ] Text extraction on upload (PDF, DOCX, TXT, MD → tsvector)
- [ ] Full-text search across attachment content
- [ ] Access control via JOIN: attachments → channels → team_members
**Chat Integration**
- [ ] Image/file upload in chat messages
- [ ] Multimodal message assembly for vision-capable models
- [ ] Document preview in chat (images inline, files as download links)
- [ ] Paste-to-upload (clipboard image support)
*Note: media generation (image gen, video models) is a separate concern —
those are tool-use actions that depend on the tool framework and
produce attachments as output. Tracked under Future.*
---
## 0.12.0 — @mention Routing + Multi-model
The channel schema already supports multiple models. This phase adds the routing logic.
- [ ] @mention parsing in messages (users and AI models)
- [ ] Resolve mentions against `channel_models`
- [ ] Multi-model channels: fire completions per mentioned model
- [ ] "Add a model" UI per channel
- [ ] Enables: editor mode, second opinions, cross-model conversations
---
## 0.13.0 — Embeddings + Knowledge Bases
- [ ] Embedding pipeline: chunking (recursive, semantic), generation (OpenAI, local)
- [ ] `pgvector` storage and similarity search
- [ ] `knowledge_bases` table: name, description, team_id (nullable), created_by
- [ ] KB document storage via 0.11.0 storage backend (same S3/PVC abstraction)
- [ ] KB CRUD endpoints + admin UI
- [ ] `kb_search` tool (uses existing tool framework)
- [ ] Context injection in completion flow
- [ ] Notes get embedded too (once pipeline exists)
- [ ] Team admins manage team KBs (permission layer from 0.8.0)
- [ ] Per-channel KB toggle
---
## 0.14.0 — Compaction
Replaces the manual 0.9.x context management with automated background processing.
- [ ] Auto-compaction service: background job that calls an LLM to summarize
- [ ] Channel-scoped: triggers when channel exceeds context threshold
- [ ] Compaction summaries stored as system messages in the channel
- [ ] Configurable: per-channel opt-in/out, summary model selection
- [ ] Admin controls for resource limits
---
## 0.15.0 — Smart Model Routing
Rules-based routing engine — not ML, just policy.
- [ ] Admin defines routing policies:
"cheapest model with required capabilities",
"prefer private providers, fallback to cloud",
"for this team, use X; for that team, use Y"
- [ ] Model capability system already exists — routing is policy on top
- [ ] Fallback chains: primary provider down → next provider with same model
- [ ] Cost-aware: factor pricing into routing decisions
- [ ] Latency-aware: track response times per provider, prefer faster
---
## 0.16.0 — Tasks / Autonomous Agents
The capstone: everything below it combined into autonomous workflows.
- [ ] Scheduler + task runner
- [ ] `task_create` tool
- [ ] Creates `type: 'service'` channels with no human members
- [ ] Depends on: completion handler, tool execution, notes, web search
- [ ] Admin controls for resource limits, execution budgets
---
## 0.17.0 — Auth Strategy (mTLS/OIDC) + Full RBAC
Enterprise auth modes and fine-grained permissions on top of the teams foundation.
- [ ] `AUTH_MODE` env var: `builtin` | `mtls` | `oidc`
- [ ] All three resolve to the same internal user model
- [ ] `auth_source` + `external_id` columns on users table
- [ ] mTLS: header trust, auto-provision from cert DN
- [ ] OIDC: Keycloak/Okta token validation, claim extraction, role mapping
- [ ] WebSocket auth per mode
- [ ] Per-source auto-activate policy:
auto_activate (bool), default_team, default_role
- [ ] Fine-grained permissions: model access, KB write, task create,
admin delegation, token budgets per user/team
- [ ] SSO/SAML (may fold in or follow as 0.17.x)
---
## Future (post-1.0 candidates)
Items that are real but don't yet have a version assignment. Any of these
could pull left based on need.
**Desktop + Mobile**
- Desktop app (Tauri)
- Full PWA with offline capability
- Mobile-optimized layouts
**Generation + Media**
- Image generation tools (DALL-E, Stable Diffusion endpoints)
- Video model integration
- Audio/TTS tools
**Data + Portability**
- Bulk export/import (account data, conversations, settings)
- ChatGPT/other tool import
- GDPR-style "download my data"
- Backup/restore CronJob manifests + operational docs
**Platform**
- Rate limiting per user/team/tier (token budgets)
- Provider health monitoring + key rotation
- Multi-tenant SaaS mode
- Workflow builder (visual DAG for chaining models + tools)
- Plugin/extension marketplace
- Live collaboration (typing indicators, presence, co-editing)
- Virtual scroll for long conversations
---
## Extension / Plugin Architecture
**Deferred to post-1.0.** The tool execution framework (0.7.x) provides the
internal hook points. A formal plugin API, manifest format, and marketplace
are tracked in the dedicated design documents:
- [EXTENSIONS.md](EXTENSIONS.md) — Three-tier extension system spec
(Browser JS, Starlark sandbox, Sidecar containers)
- [ARCHITECTURE.md](ARCHITECTURE.md) — Core backend services that extensions
build on
# Roadmap — Chat Switchboard
## v0.9.0 (Current)
- [x] Consolidated schema (21 migrations → 1)
- [x] Store layer abstraction (all DB access via typed interfaces)
- [x] Persona-as-trust-boundary model (replaces presets)
- [x] Scope model (global/team/personal) for configs, personas, models
- [x] Capabilities resolver chain (catalog → known → heuristic)
- [x] Three-state model visibility (enabled/disabled/team-only)
- [x] User model preferences (hide, sort)
- [x] Audit log foundation
- [x] Backward-compatible API routes
## v0.9.1 (Next)
- [ ] Chat search (full-text search across messages)
- [ ] Keyboard shortcuts + command palette (Ctrl+K)
- [ ] PWA enhancements (offline indicator, install prompt)
- [ ] API key at-rest encryption
- [ ] v0.8 → v0.9 migration script (002_v08_to_v09.sql)
## v0.10
- [ ] Grant management UI (admin assigns model access per team)
- [ ] Model visibility toggle in admin panel
- [ ] Team/group notes (shared note folders)
- [ ] Conversation export (markdown, JSON)
- [ ] Plugin system (tool registration API)
## v0.11
- [ ] OIDC/Keycloak authentication mode
- [ ] mTLS client certificate authentication
- [ ] SSO integration patterns
- [ ] Rate limiting per user/team (configurable quotas)
## Future
- [ ] SQLite backend option (for single-user / dev deployments)
- [ ] Multi-model conversations (different models per message)
- [ ] Conversation templates (reusable multi-turn starters)
- [ ] Agent mode (multi-step tool use with human-in-the-loop)
- [ ] Model cost tracking and reporting