core/CHANGELOG.md

# Changelog

All notable changes to Chat Switchboard.

## [0.10.1] — 2026-02-24

### Added
- **Admin System Prompt** — Global system prompt configured in Admin › Settings.
  Injected as the first system message in every conversation, before user/preset
  system prompts. Users cannot override or disable it. Stored in
  `global_settings['system_prompt']`.
- **Personas tab** — User presets moved from the Models tab to their own
  "Personas" tab in Settings. Tab is hidden when admin disables user presets
  (`allow_user_personas` policy).
- **Team Management modal** — Team admin functionality extracted from Settings
  into its own tabbed modal (Members, Providers, Presets, Usage, Activity).
  Accessed via "Team Management" flyout menu item, or "Manage →" on team cards
  in Settings. Multi-team admins see a picker first; single-team admins go
  straight to the tabbed view. Back arrow in header returns to picker.
- **Preview pane: clear button** — Trash icon in the preview header resets the
  iframe and shows the empty hint. Also auto-clears when deleting a chat that
  had active preview content.
- **Preview/Notes pane: fullscreen** — Expand button in the header toggles
  fullscreen mode (panel fills entire viewport width).
- **Preview/Notes pane: resizable** — Drag the left edge of the side panel
  to resize (280px–70vw). Width resets on close.
- **Code block: download** — "Download" button on every code block. Infers
  file extension from language tag (e.g. `python` → `.py`, `go` → `.go`).

### Changed
- **Personal Usage scoped to BYOK** — `GET /usage` and the user Usage tab now
  only show consumption against personal (BYOK) providers. Global provider
  usage is the org's cost, not the user's. New `QueryByUserPersonal` store
  method filters on `scope = 'personal' AND owner_id = user_id`. Admin usage
  views remain unaffected. Usage tab hidden when admin disables user providers
  (`allow_user_byok` policy).
- **OpenAI streaming usage race** — Deferred the Done event until the usage
  chunk arrives. Previously, finish_reason fired Done before the usage chunk
  (which has `choices: []`), so streaming token counts were always 0 for all
  OpenAI-compatible providers.

### Fixed
- **Usage logging zero-token guard** — Removed early exit in `logUsage` that
  suppressed rows when tokens were 0. Combined with the streaming race above,
  this meant no streaming usage was ever recorded.
- **Live chat completion test** — `TestLive_VeniceChatCompletion` sent wrong
  field names (`messages`/`config_id` instead of `content`/`provider_config_id`).

## [0.10.0] — 2026-02-24

### Added
- **Model Roles** — Named model slots (`utility`, `embedding`, `generation`)
  with primary + fallback bindings. Stored in `global_settings['model_roles']`.
  Team-level overrides via `teams.settings` JSONB. New `server/roles/` package
  with `Resolver.Complete()` and `Resolver.Embed()` for automatic failover.
  Admin API: `GET/PUT /admin/roles/:role`, `POST /admin/roles/:role/test`.
  Team API: `GET/PUT/DELETE /teams/:id/roles/:role`.
- **Provider Embed() interface** — All providers implement `Embed()`. OpenAI,
  Venice, OpenRouter support `/v1/embeddings`; Anthropic returns `ErrNotSupported`.
  New types: `EmbeddingRequest`, `EmbeddingResponse`.
- **Usage Tracking** — Every completion (streaming and non-streaming) logs
  token counts and cost to `usage_log` table. Cost calculated at insert time
  from `model_pricing` (no retroactive recalculation). Provider scope
  denormalized for efficient admin filtering. Admin views exclude BYOK.
  New stores: `UsageStore`, `PricingStore`. Admin API:
  `GET /admin/usage`, `GET /admin/usage/users/:id`, `GET /admin/usage/teams/:id`.
  User API: `GET /usage` (personal summary).
- **Model Pricing** — `model_pricing` table with catalog sync and manual admin
  overrides. Sync from provider APIs (Venice, OpenRouter) auto-populates
  pricing with `source='catalog'`; manual entries never overwritten. Admin API:
  `GET/PUT/DELETE /admin/pricing`.
- **Streaming token capture** — OpenAI: `stream_options.include_usage` +
  parse usage/cache from final chunk. Anthropic: parse `message_start` for
  input/cache tokens, `message_delta` for output tokens. Cache token fields
  (`CacheCreationTokens`, `CacheReadTokens`) on `CompletionResponse`,
  `StreamEvent`, and `streamResult`.
- **Admin Roles tab** — Configure primary/fallback provider+model per role,
  test-fire from UI.
- **Admin Usage dashboard** — Period selector (7d/30d/90d), group-by
  (model/user/day/provider), totals cards, breakdown table, pricing table.
- **Team Usage dashboard** — Team admins can view usage against their
  team-owned providers via `GET /teams/:teamId/usage`. Filters to
  `provider_configs WHERE scope='team' AND owner_id=teamId`. Integrated
  into team management panel with period/group-by selectors.
- **Admin Reset Password** — Button in user list with vault destruction
  warning dialog (two-step confirmation).
- **User Usage tab** — Settings modal "Usage" tab shows personal token
  consumption and estimated costs across all conversations, including BYOK.
  Period selector (7d/30d/90d) and group-by (model/day).
- **Live Venice integration tests** — Gated behind `VENICE_API_KEY` env var.
  Tests: non-streaming completion, streaming completion, usage logging for
  both modes, pricing from catalog sync, and direct embeddings via BGE-M3.
  Uses `qwen3-4b` (Venice Small) — cheapest at $0.05/$0.15 per 1M tokens.
- **Migration 004** — `usage_log` table, `model_pricing` table, `model_roles`
  seed in `global_settings`.

### Fixed
- **UEK re-wrap on password change** — `ChangePassword` now decrypts UEK
  with old password and re-encrypts with new password + fresh salt. Previously,
  password changes silently broke all personal BYOK keys.
- **UEK destruction on admin password reset** — `ResetPassword` now nullifies
  vault columns, evicts UEK from cache, deletes personal provider configs,
  and logs an audit event. Previously, admin resets left orphaned encrypted
  UEK that silently broke personal keys.
- **Streaming completions logged zero tokens** — `StreamEvent` lacked token
  fields and providers discarded usage data from final chunks. Both OpenAI
  and Anthropic streaming parsers now capture and propagate token counts.
- **OpenAI streaming usage race condition** — The OpenAI protocol sends
  chunks in order: content → finish_reason → usage → [DONE]. The parser
  sent `Done=true` on the finish_reason chunk (step 2) before the usage
  chunk arrived (step 3, with `choices:[]`). The receiver returned
  immediately on Done, so usage was captured but never delivered.
  Fix: defer the Done event as `pendingFinish` until the usage chunk
  arrives, then flush with tokens attached. Tool-call finishes flush
  immediately (tool loop needs the event). Affects all OpenAI-compatible
  providers (OpenAI, Venice, OpenRouter).
- **Token accumulation across tool iterations** — `streamResult` in
  `stream_loop.go` now accumulates input/output/cache tokens across
  multi-tool-call iterations instead of only capturing the final iteration.
- **Admin pricing leaked BYOK entries** — `PricingStore.List()` returned
  all model_pricing rows including those from personal BYOK providers.
  Admin panel showed pricing entries for user-private providers they
  shouldn't see, with O(users × models) scale explosion. Now joins on
  `provider_configs` and filters `scope != 'personal'`. Admin `UpsertPricing`
  also validates provider scope, rejecting manual pricing on personal
  providers.
- **Team role handlers used wrong param name** — `ListTeamRoles`,
  `UpdateTeamRole`, `DeleteTeamRole` read `c.Param("id")` but routes use
  `:teamId`. Every team role operation silently got an empty team ID.
- **Usage logging suppressed for zero-token streams** — `logUsage` had an
  early exit when `inputTokens == 0 && outputTokens == 0`. Combined with the
  OpenAI streaming parser bug (below), this silently dropped all streaming
  usage rows. Removed the guard — requests are always recorded.
- **Live chat completion test used wrong field names** — `TestLive_VeniceChatCompletion`
  sent `messages` and `config_id` but the handler expects `content` and
  `provider_config_id`. Test silently skipped on the binding error.

## [0.9.4] — 2026-02-24

### Added
- **API key encryption (vault)** — Two-tier AES-256-GCM encryption for stored
  API keys. Global/team keys encrypted with env-var-derived key (HKDF-SHA256);
  personal BYOK keys encrypted with per-user encryption key (UEK) derived from
  password via Argon2id. Admin cannot recover personal keys without user's
  passphrase. New `server/crypto/` package: `vault.go`, `cache.go`,
  `resolver.go`, `backfill.go` with 11 round-trip tests.
- **UEK lifecycle** — UEK generated on registration, unwrapped on login
  (Argon2id → AES-GCM), cached in `sync.Map` for session duration, evicted
  with memory zeroing on logout. Pre-migration users auto-initialize vault
  on first login.
- **Migration 003_vault.sql** — Adds `encrypted_uek`, `uek_salt`, `uek_nonce`,
  `vault_set` to users table. Adds `api_key_enc` (BYTEA), `key_nonce`,
  `key_scope` to provider_configs. Backfills `key_scope` from existing scope.
- **Startup backfill** — `BackfillEncryptedKeys()` encrypts plaintext API keys
  on first startup with `ENCRYPTION_KEY` set. `EnforceEncryptionKey()` refuses
  startup if encrypted keys exist but env var is missing.
- **CI/CD: encryption secret** — `ENCRYPTION_KEY` Gitea secret synced to k8s
  `switchboard-encryption` secret. Backend manifest references it as optional
  `secretKeyRef`.

### Fixed
- **HTML code blocks render live in chat (XSS)** — When a model's `</think>`
  tag directly abutted a code fence (no newline), the fence wasn't recognized
  by marked.js, causing raw HTML to render as live DOM elements (canvas games,
  styles, etc.). Three-layer fix: (1) DOMPurify switched from permissive
  `ADD_TAGS` (default allows canvas, style, form, etc.) to strict
  `ALLOWED_TAGS` allowlist of only markdown-produced elements; (2) think-block
  placeholders padded with `\n\n` to ensure adjacent fences start on fresh
  lines; (3) unclosed code fences auto-closed before `marked.parse` for
  streaming protection.

## [0.9.3] — 2026-02-23

### Changed
- **Code blocks: button-driven collapse** — Replaced `<details>` wrapper with
  inline collapse/expand toggle button in the code toolbar. Auto-collapses at
  >15 lines with a fade mask; toggle available on all blocks >5 lines. User
  can always expand/collapse regardless of threshold. Smooth CSS transition
  instead of native `<details>` jump.
- **Thinking blocks: always visible** — Thinking blocks are always rendered in
  the DOM regardless of the `showThinking` setting. The setting now controls
  whether blocks start expanded (`<details open>`) or collapsed. User can
  always click to toggle. Setting label updated to "Auto-expand thinking blocks".

### Added
- **Admin default model** — New `default_model` policy in Admin → Settings.
  Dropdown populated from enabled models. When a user has no saved selection
  (fresh login, cleared browser) or their saved model is no longer available,
  the admin default is used before falling back to first visible. Resolution
  chain: `localStorage → admin default → first visible`.
- **Reasoning/thinking support for OpenAI-compatible providers** — Models that
  send `reasoning_content` in stream deltas (Grok, DeepSeek, etc.) now have
  thinking blocks streamed live and persisted as `<think>` tags. Renders as
  collapsible thinking blocks in both streaming and history views.
- **Favicon animation during generation** — Browser tab favicon pulses with
  cascading dot opacity animation while a completion is in progress, restoring
  to the static favicon when done.
- **Tool calls in message history** — Tool call metadata (name, arguments,
  result, error status) is now persisted alongside assistant messages in the
  database. History view renders tool calls as collapsed `<details>` blocks
  showing "🔧 tool_name → done" with expandable input/output JSON.
- **Notes tool: "View note" link** — When a notes tool (`note_create`,
  `note_update`, etc.) completes, a "📝 View note" button appears in both
  the live streaming tool indicator and the history tool call block. Clicking
  opens the Notes panel and navigates directly to the note.
- **Notes panel: Copy button** — "Copy" button added to note read mode,
  copies title + content as markdown to clipboard.
- **Brand hover: jitter-free crossfade** — Sidebar brand logo→collapse icon
  swap uses opacity crossfade instead of `display: none` toggle, eliminating
  layout reflow jitter on hover.

### Fixed
- **Model selector shows unavailable model** — Selector restoration searched
  all models including user-hidden ones. Saved selection (localStorage) for a
  hidden model like Claude Opus would re-select it on every page load even when
  only Grok was visible. Resolution now filters to visible models only.
- **Refresh toast shows wrong count** — "Loaded 33 models" counted all models
  including hidden; now shows only visible count ("Loaded 1 model").
- **Tool calls vanish after completion** — Live streaming tool indicators
  disappeared when `reloadActivePath()` rebuilt messages from DB. Fixed: the
  `getActivePath` query now includes `tool_calls` column, and `PathMessage`
  carries the data through to the frontend renderer.
- **Tool result "undefined results" text** — Operator precedence bug in tool
  result summary parser caused `undefined results` to display for note_create.
  Fixed summary logic to properly branch on title vs count.
- **Regenerate streams at wrong position** — Regen'd response appeared below
  the full conversation (appended at bottom) then snapped to correct position
  after completion. Fixed: display is now truncated to the parent message
  before streaming starts, so the new response streams in-place as a clean
  branch. Backend context was already correct (excludes old response).
- **Regenerate loses tools, reasoning, and tool execution** — Regen handler
  was a stripped-down copy of the completion handler missing tool definitions,
  tool execution loop, reasoning_content forwarding, and tool_calls persistence.
  Model lost access to note tools on regen and fell back to generic capabilities.
  Refactored: extracted `streamWithToolLoop()` into `stream_loop.go` as the
  single canonical streaming implementation. Both `streamCompletion` (normal
  chat) and `Regenerate` now call the shared function — only persistence
  differs. Future streaming features (new event types, tool capabilities)
  automatically apply to all code paths.
- **OpenRouter free models crash** — Free models with nil pricing caused
  `pq: invalid input syntax for type json` on catalog sync. Nil pricing now
  routes through `ToJSON()` producing `"{}"` instead of nil `[]byte`.
- **API key appears unsaved** — `ListGlobalConfigs` returned raw
  `ProviderConfig` structs where `APIKeyEnc` is `json:"-"` (never serialized),
  so `has_key` was always `undefined`. Now returns computed `has_key` field.
- **Case-insensitive usernames** — Login, registration, and all user lookups
  use `LOWER()` in SQL. All user creation paths normalize to lowercase.
  New migration `002_ci_username.sql` adds `LOWER()` unique indexes and
  normalizes existing rows.

## [0.9.2] — 2026-02-23

### Added
- **Collapsible code blocks**: Code blocks over 15 lines auto-collapse into
  `<details>` with language and line count summary. Language label shown in
  top-left corner of all code blocks.
- **HTML preview**: "Preview" button on HTML code blocks opens a sandboxed
  iframe (`allow-scripts`, no `allow-same-origin`). Auto-detected for
  untagged blocks via heuristic.
- **Token count estimate**: Live token counter below input area showing
  approximate tokens and context usage percentage when model has `max_context`.
- **Context length warning**: Dismissable banner above input at 75% (yellow)
  and 90% (red) context usage with guidance to start a new chat.
- **Proxy interception detection**: `_parseJSON()` checks Content-Type before
  `.json()` on all API paths including streaming. Typed `proxyBlocked` errors
  with proxy page title extraction and actionable splash messages.
- **Environment injection**: `window.__ENV__` wired through entrypoint, k8s,
  and index.html for dev/test/production gating.
- **Enhanced diagnostics**: `NET:PROXY` log type, Content-Type capture in
  fetch interceptor, environment info in export header and state snapshot.
- **Team admin audit scoping**: New `GET /api/v1/teams/:teamId/audit` and
  `/audit/actions` endpoints scoped to team members. Activity Log section
  in team manage panel with filter dropdown and pagination.
- **New favicon**: Switchboard panel design with provider-colored jacks.
  Animated SVG (rotating plugs), 32px PNG, 256px PNG, and ICO.
- **Seed users** (dev/test only): `SEED_USERS=user:pass:role,...` env var
  pre-creates active users on startup. Ignored in production. K8s secret
  `switchboard-seed-users` wired in backend manifest.

## [0.9.1] — 2026-02-23

### Removed
- **Static known model table**: Deleted `knownModels` map from backend and
  `KNOWN_MODELS` from frontend. The same model ID can have different capabilities
  depending on the provider (e.g. DeepSeek has tool_calling on OpenRouter but
  not on Venice). A hardcoded table can't represent this.
- **Frontend `lookupKnownCaps()`**: Removed client-side capability guessing.
  Backend is the sole source of truth via catalog → heuristic chain.

### Changed
- **Resolution chain simplified**: catalog (provider API sync) → heuristic inference.
  No intermediate known table. Providers that report capabilities via API are
  authoritative; heuristics are best-effort for unsynced models.
- **All providers updated**: OpenAI, OpenRouter, Anthropic, Venice now call
  `InferCapabilities()` directly instead of the dead known table lookup.
- **Frontend `resolveCapabilities()`**: Now passes through backend caps as-is.
  No client-side merging with a static table.
- [x] EXTENSIONS.md recovered into repo, updated with Appendix A (Custom
  Renderers) and Appendix B (Model Roles with utility/embedding/generation slots)
- [x] ROADMAP.md restructured: extension foundation pulled to v0.11.0,
  model roles to v0.10.0, dependency graph, TBD replaces post-1.0,
  removed v0.8→v0.9 migration (OBE — no public release, no test path)

### Added
- **Heuristic patterns**: Updated to detect qwen3, gpt-5, grok, kimi, minimax,
  glm-5, gemma-3 model families. Vision expanded to claude-opus/sonnet (not
  just claude-3). Reasoning expanded for thinking, grok, glm patterns.

### Fixed
- **Preset capability pills**: Presets with auto-resolve (no `provider_config_id`)
  now inherit base model capabilities via `GetByModelIDAny` catalog fallback.
- **Venice `optimizedForCode`**: Added mapping to `CodeOptimized` capability.
- **CI test stability**: BYOK journey tests use unreachable endpoints so auto-fetch
  doesn't race with simulated data injection.

## [0.9.0] — 2026-02-22

### Added
- **Schema consolidation**: 21 migrations collapsed to single `001_initial.sql`
- **Store layer**: All database access through typed interfaces (no raw SQL in handlers)
- **Persona model**: Trust-boundary model replacing old presets; scoped global/team/personal
- **Capabilities resolver**: Three-tier chain — catalog → known table → heuristic inference
- **Three-state model visibility**: enabled / disabled / team-only
- **BYOK auto-fetch**: Creating a personal provider triggers model discovery from provider API
- **User model refresh**: `POST /api-configs/:id/models/fetch` endpoint + UI button
- **Composite model IDs**: `configId:modelId` format prevents cross-provider collisions
- **Audit log foundation**: All admin operations logged with actor, action, resource
- **Journey integration tests**: API-driven test suite replacing fake-data tests
- **Frontend test suite**: 107 tests, 27 suites validating model processing pipeline
- **Live Venice API test**: Proves real BYOK → auto-fetch → models visible flow

### Fixed
- **API key storage**: `json:"-"` tag on `ProviderConfig.APIKeyEnc` silently dropped
  keys during admin create/update. Fixed with wrapper structs that bypass the tag.
- **NULL model_default scan**: `scanProviders()` crashed on NULL `model_default` column,
  silently hiding all team and BYOK models. Fixed with `sql.NullString`.
- **Nil slice serialization**: Go nil slices serialized as JSON `null` instead of `[]`,
  breaking frontend fallback chains. Fixed with `make([]T, 0)`.
- **Frontend error swallowing**: API responses with `errors` field were silently ignored.

### Changed
- Backward-compatible API routes with v0.8 field name aliases
- User model preferences table (`user_model_settings`)