switchboard/core
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Releases 9 Wiki Activity

Changeset 0.22.8 (#150)

Browse Source
This commit is contained in:
xcaliber
2026-03-04 16:06:12 +00:00
parent 389e47b0f9
commit 7e26a2a261
114 changed files with 3700 additions and 7572 deletions
Download Patch File Download Diff File Expand all files Collapse all files

584
docs/DESIGN-0.15.0.md
View File

@@ -1,584 +0,0 @@
# DESIGN-0.15.0 — Compaction
## Overview
Automatic conversation compaction. A background service monitors channels
for context pressure and triggers summarization via the utility model role,
replacing the manual "Summarize & Continue" button as the primary
compaction path. Manual summarization remains available as an explicit
user action.
Depends on: utility model role (v0.10.0), summarize handler (v0.10.2).
**Design principle: don't reinvent the wheel.** The existing
`SummarizeHandler` already has the full pipeline — path loading, summary
boundary detection, prompt construction, role resolution, tree insertion,
cursor update, usage logging. This design extracts that core logic into
a shared `compaction` package and adds a background scanner on top.
---
## 1. Extract: `compaction` Package
Move the reusable summarization pipeline out of `handlers/summarize.go`
into `server/compaction/compaction.go`. The HTTP handler becomes a thin
wrapper.
### Core type
```go
package compaction
// Service provides conversation compaction (summarization).
// Used by both the HTTP handler (manual) and the background scanner (auto).
type Service struct {
stores store.Stores
resolver *roles.Resolver
}
func NewService(stores store.Stores, resolver *roles.Resolver) *Service {
return &Service{stores: stores, resolver: resolver}
}
```
### Extracted method: `Compact`
The current `SummarizeHandler.Summarize` body becomes `Service.Compact`:
```go
type CompactRequest struct {
ChannelID string
UserID string // channel owner
TeamID *string // for role resolution
Trigger string // "manual" | "auto"
}
type CompactResult struct {
SummaryID string
SummarizedCount int
Model string
UsedFallback bool
Content string
InputTokens int
OutputTokens int
}
func (s *Service) Compact(ctx context.Context, req CompactRequest) (*CompactResult, error)
```
The method performs the same steps as today's handler:
1. Load active path via `getActivePath` (already exported within `handlers`)
2. Find summary boundary, collect messages to summarize
3. Guard: minimum 4 messages since last summary
4. Build summarization prompt
5. Call `resolver.Complete(ctx, RoleUtility, ...)`
6. Insert summary tree node with metadata
7. Update cursor
8. Log usage
The only new metadata field is `"trigger"` (`"manual"` or `"auto"`) so
the frontend can distinguish how the summary was created.
### Refactored handler
`handlers/summarize.go` shrinks to:
```go
func (h *SummarizeHandler) Summarize(c *gin.Context) {
// ownership check, rate limit check (unchanged)
// ...
result, err := h.compaction.Compact(c.Request.Context(), compaction.CompactRequest{
ChannelID: channelID,
UserID: userID,
TeamID: teamID,
Trigger: "manual",
})
// return JSON response (unchanged)
}
```
### Tree helpers
`getActivePath`, `isSummaryMessage`, `nextSiblingIndex`, `updateCursor`
are currently unexported in `handlers/tree.go`. Two options:
**Option A** — Export them from `handlers` and import in `compaction`.
Clean but creates a dependency from `compaction` → `handlers`.
**Option B** — Move tree helpers into a `treepath` (or similar) package
imported by both `handlers` and `compaction`.
Recommend **Option B** to keep the dependency graph clean:
`handlers` → `compaction` → `treepath` ← `handlers`. The `treepath`
package is pure data + SQL queries with no handler logic.
```
server/
treepath/ ← NEW: extracted from handlers/tree.go
path.go (getActivePath, getPathToLeaf, getActiveLeaf)
summary.go (isSummaryMessage, summary metadata helpers)
siblings.go (getSiblingCount, getSiblings, findLeafFromMessage)
cursor.go (updateCursor, nextSiblingIndex)
compaction/ ← NEW
compaction.go (Service, Compact)
scanner.go (Scanner, background loop)
estimator.go (token estimation)
handlers/
tree.go → thin wrappers or deleted, imports treepath
summarize.go → thin HTTP wrapper, delegates to compaction.Service
completion.go → imports treepath for path building
```
---
## 2. Server-Side Token Estimation
The background scanner needs to evaluate context pressure without making
an LLM call. The frontend already does this with a `chars / 4` heuristic
(`tokens.js`). Replicate server-side:
```go
// compaction/estimator.go
// EstimateTokens returns a rough token count using the ~4 chars/token
// heuristic. Matches the frontend Tokens.estimate() for consistency.
func EstimateTokens(text string) int {
return (len(text) + 3) / 4
}
// EstimatePath returns total estimated tokens for a message path,
// including system prompt overhead.
func EstimatePath(path []treepath.PathMessage, systemPromptLen int) int {
total := 0
if systemPromptLen > 0 {
total += EstimateTokens(string(make([]byte, systemPromptLen))) + 4
}
for _, m := range path {
total += EstimateTokens(m.Content) + 4 // +4 per-message overhead
}
return total
}
```
Not exact, but it's the same bar the user already sees in the UI. If we
need better accuracy later, swap in a tokenizer library without changing
the interface.
### Context budget resolution
The scanner needs `max_context` for whatever model the channel is using.
Resolution chain:
1. Channel has a `model` field → look up `model_catalog` for capabilities
2. Channel has a `provider_config_id` → use that provider's catalog entry
3. Fallback: use the utility role's model context window as a proxy (it's
the model that would be doing the summarization anyway)
4. Hard fallback: 128K tokens (conservative default)
```go
func (s *Scanner) getContextBudget(ctx context.Context, ch *models.Channel) int {
if ch.Model != "" {
if caps, err := s.stores.Catalog.GetCapabilities(ctx, ch.Model); err == nil {
if caps.MaxContext > 0 {
return caps.MaxContext
}
}
}
return DefaultContextBudget // 128000
}
```
---
## 3. Background Scanner
Follows the Ingester pattern from `knowledge/ingest.go`: goroutine pool
with semaphore, WaitGroup for graceful shutdown.
### Scanner type
```go
// compaction/scanner.go
type Scanner struct {
service *Service
stores store.Stores
sem chan struct{}
wg sync.WaitGroup
stopCh chan struct{}
ticker *time.Ticker
}
type ScannerConfig struct {
Enabled bool // global kill switch
Interval time.Duration // scan frequency (default: 5 minutes)
Concurrency int // max parallel compactions (default: 2)
}
func NewScanner(svc *Service, stores store.Stores, cfg ScannerConfig) *Scanner
func (sc *Scanner) Start()
func (sc *Scanner) Stop() // signals stop + drains in-flight
```
### Scan loop
```
every <interval>:
if !globalEnabled(): continue
candidates = findCandidates()
for each candidate:
sem.acquire()
go compact(candidate)
```
### Candidate query
A single SQL query finds channels that need compaction:
```sql
SELECT c.id, c.user_id, c.model, c.settings,
COUNT(m.id) as msg_count,
SUM(LENGTH(m.content)) as total_chars
FROM channels c
JOIN messages m ON m.channel_id = c.id
AND m.deleted_at IS NULL
WHERE c.deleted_at IS NULL
AND c.type = 'direct' -- only user conversations (not groups yet)
AND c.is_archived = false
-- Skip channels with very recent activity (let the user finish typing)
AND c.updated_at < NOW() - INTERVAL '2 minutes'
-- Only channels active in the last 7 days (don't compact stale channels)
AND c.updated_at > NOW() - INTERVAL '7 days'
GROUP BY c.id
HAVING COUNT(m.id) >= 10 -- minimum message count
AND SUM(LENGTH(m.content)) > 20000 -- rough: 20K chars ≈ 5K tokens minimum
ORDER BY c.updated_at DESC
LIMIT 50 -- batch size cap
```
This is a coarse filter. Each candidate then gets a precise check:
```go
func (sc *Scanner) shouldCompact(ctx context.Context, ch *models.Channel) bool {
// 1. Check channel-level opt-out
if ch.Settings != nil {
if v, ok := ch.Settings["auto_compaction"]; ok && v == false {
return false
}
}
// 2. Load active path for the channel owner
path, err := treepath.GetActivePath(ch.ID, ch.UserID)
if err != nil || len(path) < 10 {
return false
}
// 3. Find post-summary messages only
startIdx := 0
for i, m := range path {
if treepath.IsSummaryMessage(&m) {
startIdx = i + 1
}
}
postSummary := path[startIdx:]
if len(postSummary) < 8 {
return false // not enough new messages since last summary
}
// 4. Estimate tokens and check threshold
tokens := EstimatePath(postSummary, 0)
budget := sc.getContextBudget(ctx, ch)
threshold := sc.getThreshold(ch) // default 0.70
return float64(tokens) / float64(budget) >= threshold
}
```
### Threshold
Default: **0.70** (70% of context window). This is intentionally lower
than the frontend's 75% warning bar — auto-compaction should fire before
the user sees a warning, not after.
Configurable per-channel via `settings.compaction_threshold` (float) and
globally via `global_settings` key `auto_compaction_threshold`.
### Rate limiting
Auto-compaction calls use the utility role and are org-funded. They share
the existing `utility_rate_limit` budget but get a separate counter
prefix so admins can see auto vs manual usage:
```go
entry := &models.UsageEntry{
// ...
Role: &role, // "utility"
Metadata: models.JSONMap{
"trigger": "auto", // distinguishes from manual
},
}
```
If the global utility rate limit is hit, auto-compaction silently skips
the channel and retries next scan cycle. It never blocks or errors.
---
## 4. Per-Channel Configuration
Uses the existing `Channel.Settings` JSONB column. No migration needed.
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `auto_compaction` | `bool` | `true` (from global) | Enable/disable for this channel |
| `compaction_threshold` | `float` | `0.70` (from global) | Context usage ratio to trigger |
### Frontend: channel settings
Add a "Compaction" section to the channel settings panel (the gear icon
in the chat header). Two controls:
```
┌─ Compaction ──────────────────────────────────┐
│ Auto-compact [✓] │
│ Threshold [70]% of context window │
└───────────────────────────────────────────────┘
```
The manual "Summarize & Continue" button remains in the context warning
bar and works regardless of auto-compaction settings.
---
## 5. Admin Controls
New `global_settings` keys:
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `auto_compaction_enabled` | `bool` | `false` | Global kill switch. Ships **off** — admins opt in. |
| `auto_compaction_threshold` | `float` | `0.70` | Default threshold for channels without override |
| `auto_compaction_interval_minutes` | `int` | `5` | Scanner tick interval |
| `auto_compaction_concurrency` | `int` | `2` | Max parallel auto-compactions |
| `auto_compaction_cooldown_minutes` | `int` | `30` | Min time between auto-compactions of the same channel |
### Admin UI
Add an "Auto-Compaction" card to the existing admin AI settings panel
(alongside the utility rate limit and model roles):
```
┌─ Auto-Compaction ─────────────────────────────┐
│ Enabled [✓] │
│ Threshold [70]% │
│ Scan interval [5] minutes │
│ Concurrency [2] parallel │
│ Cooldown [30] minutes per channel │
└───────────────────────────────────────────────┘
```
---
## 6. Cooldown + Dedup
Prevent rapid re-compaction of the same channel:
1. **Cooldown** — after compacting channel X, don't re-evaluate it for
`cooldown_minutes`. Track in-memory:
```go
lastCompacted map[string]time.Time // channelID → timestamp
```
2. **In-flight dedup** — if channel X is currently being compacted (sem
acquired, goroutine running), skip it in the current scan. Track via:
```go
inFlight sync.Map // channelID → struct{}
```
3. **Stacking** — the existing summary boundary logic already handles
multiple summaries correctly. Auto-compaction produces the same tree
nodes as manual, so stacking "just works."
---
## 7. Lifecycle + Wiring
### main.go changes
```go
// After role resolver and stores setup:
compactionSvc := compaction.NewService(stores, roleResolver)
compactionScanner := compaction.NewScanner(compactionSvc, stores, compaction.ScannerConfig{
Enabled: getGlobalBool(stores, "auto_compaction_enabled", false),
Interval: time.Duration(getGlobalInt(stores, "auto_compaction_interval_minutes", 5)) * time.Minute,
Concurrency: getGlobalInt(stores, "auto_compaction_concurrency", 2),
})
compactionScanner.Start()
defer compactionScanner.Stop() // drain in-flight on shutdown
// Summarize handler now wraps compaction service:
summarize := handlers.NewSummarizeHandler(stores, roleResolver, compactionSvc)
```
Follows the same pattern as `kbIngester` with `defer kbIngester.Wait()`.
### Config reload
The scanner re-reads global settings at the start of each scan cycle.
No restart required to change thresholds, intervals, or the kill switch.
If `auto_compaction_enabled` flips to `false`, the scanner skips all
work on the next tick.
---
## 8. Observability
### Logging
```
🔍 compaction: scanning 50 candidates
🔍 compaction: channel abc123 qualifies (78% of 128K context, 42 messages post-summary)
📝 compaction: compacting channel abc123 for user xyz (trigger=auto)
✅ compaction: channel abc123 done (42 messages → 847 chars, model=claude-sonnet-4-5-20250929)
⏭ compaction: channel def456 skipped (cooldown, 12m remaining)
⏭ compaction: channel ghi789 skipped (rate limit)
```
### Audit log
Auto-compaction events are logged to `audit_log`:
```go
stores.Audit.Log(ctx, &models.AuditEntry{
Actor: "system:compaction",
Action: "compaction.auto",
Resource: "channel:" + channelID,
Details: models.JSONMap{
"summarized_count": result.SummarizedCount,
"model": result.Model,
"trigger": "auto",
},
})
```
### Usage tracking
All auto-compaction calls flow through the existing usage logging path
(same as manual summarization). The `metadata.trigger = "auto"` field
lets the admin usage dashboard distinguish auto from manual.
---
## 9. Frontend Changes
Minimal — the backend does the heavy lifting.
### Auto-compaction indicator
When loading a conversation that was auto-compacted, the frontend
already handles summary boundaries correctly (the collapsed history
toggle, the "X earlier messages summarized" bar). The only new UI is
a subtle indicator on the summary node:
```
▸ 42 earlier messages summarized (auto)
```
The `(auto)` label comes from `metadata.trigger` on the summary message.
### Channel settings
Add compaction controls to the channel settings panel (Section 4 above).
Saves to `PATCH /channels/:id` with `settings` field — no new endpoint.
### Admin panel
Add the auto-compaction card to the AI settings section (Section 5).
Saves via existing `PUT /admin/settings/:key` endpoints.
---
## 10. Migration
**No new migration required.**
- Channel compaction settings → existing `Channel.Settings` JSONB
- Admin settings → existing `global_settings` table
- Summary messages → existing `messages` table with metadata
- Usage tracking → existing `usage_log` table
- Audit logging → existing `audit_log` table
---
## 11. File Manifest
```
server/
treepath/ ← NEW package
path.go (extracted from handlers/tree.go)
summary.go (isSummaryMessage + helpers)
siblings.go (sibling queries)
cursor.go (cursor + sibling index)
compaction/ ← NEW package
compaction.go (Service, Compact, CompactRequest/Result)
scanner.go (Scanner, scan loop, candidate query)
estimator.go (EstimateTokens, EstimatePath)
handlers/
tree.go (updated: delegates to treepath)
summarize.go (updated: delegates to compaction.Service)
completion.go (updated: imports treepath)
main.go (updated: wire compaction scanner)
src/js/
ui-core.js (updated: show "(auto)" on auto-compacted summaries)
ui-settings.js (updated: channel compaction controls)
ui-admin.js (updated: auto-compaction admin card)
```
---
## 12. Implementation Order
**Phase 1: Extract** — Move tree helpers to `treepath`, core
summarization to `compaction.Service`. Refactor `SummarizeHandler` to
delegate. All existing tests must still pass. No new behavior.
**Phase 2: Estimator + Scanner** — Add `estimator.go`, `scanner.go`,
candidate query, `shouldCompact` logic. Wire into `main.go` with
`Start()`/`Stop()`. Global settings for admin controls. Ship with
`auto_compaction_enabled` defaulting to `false`.
**Phase 3: Frontend** — Channel compaction settings, admin panel card,
auto-compaction indicator on summary nodes.
**Phase 4: Soak + tune** — Enable on a test instance, watch logs, tune
default threshold and cooldown. Adjust candidate query filters if scan
is too expensive.
---
## 13. What This Intentionally Doesn't Do
- **Token-accurate estimation** — chars/4 is good enough. A real
tokenizer adds a dependency for marginal accuracy gain. Revisit if
users report summaries firing too early or too late.
- **Per-model prompt tuning** — the summarization prompt is one-size-fits
-all. Works fine across Claude/GPT/Llama. Revisit if quality varies
significantly across providers.
- **Group channel compaction** — auto-compaction targets `direct` channels
only (single-owner). Group/channel types need multi-cursor awareness
(which user's path to compact?). Deferred to v0.19.0 multi-participant.
- **Compaction snapshots for KB** — the roadmap mentions this. The
summary nodes we create here are already stored as messages with full
metadata. KB integration can query them later without changes to this
design.
- **Event-driven triggers** — a post-completion hook that checks
immediately after each message would reduce latency vs. the ticker.
Worth adding later but the ticker is simpler to ship and debug. The
2-minute activity gap in the candidate query achieves a similar effect
(don't compact mid-conversation).

332
docs/DESIGN-0.15.1.md
View File

@@ -1,332 +0,0 @@
# DESIGN-0.15.1 — Context Recall Tools
## Overview
After compaction (v0.15.0) or in long conversations, the model loses access
to earlier material — attachment content injected once at send time, and
message details compressed into summaries. These tools let the model
re-read source material on demand.
Depends on: file handling (v0.12.0), tool framework (v0.11.0), compaction (v0.15.0).
**Four work streams:**
1. `attachment_recall` — re-read attachment content from storage
2. `conversation_search` — keyword search over channel messages
3. Token estimator improvements — attachment-aware context counting
4. KB auto-injection — automatic knowledge base context in system prompt
---
## 1. `attachment_recall` Tool
### Purpose
Attachments are injected into context once (at the user message that
references them). On subsequent turns the model relies on its own response
as a natural summary. After compaction, that summary is gone. This tool
lets the model re-read any attachment in the current channel.
### Two-action design
Single tool, `action` parameter selects behavior:
- **`list`** — returns metadata for all attachments in the channel
(id, filename, content_type, size, created_at). The model calls this
first to discover what's available.
- **`read`** — returns content for a specific attachment_id.
Documents → `extracted_text` from DB. Images → base64 data URI.
### Security
Channel-scoped: `attachment_id` must belong to `ExecutionContext.ChannelID`.
The store query is `GetByChannel(channelID)` so cross-channel access is
impossible by construction.
### Dependencies
Late registration (same pattern as `kb_search`):
```go
func RegisterAttachmentRecall(stores store.Stores, objStore storage.ObjectStore)
```
Called from `main.go` after storage init. If `objStore` is nil (storage
not configured), the tool is **not registered** — it simply won't appear
in the tool list.
### Implementation: `server/tools/attachment_recall.go`
```go
type attachmentRecallTool struct {
stores store.Stores
objStore storage.ObjectStore
}
func (t *attachmentRecallTool) Definition() ToolDef {
return ToolDef{
Name: "attachment_recall",
DisplayName: "Attachments",
Category: "context",
Description: "Re-read file attachments from this conversation. " +
"Use action='list' to see available files, then action='read' " +
"with an attachment_id to get the content.",
Parameters: JSONSchema(map[string]interface{}{
"action": PropEnum("Action to perform", "list", "read"),
"attachment_id": Prop("string", "Attachment ID (required for read)"),
}, []string{"action"}),
}
}
```
**`list` action returns:**
```json
{
"attachments": [
{
"id": "abc-123",
"filename": "report.pdf",
"content_type": "application/pdf",
"size_bytes": 45200,
"has_text": true,
"created_at": "2026-02-26T10:00:00Z"
}
],
"count": 1
}
```
**`read` action returns:**
```json
{
"id": "abc-123",
"filename": "report.pdf",
"content_type": "application/pdf",
"content": "... extracted text ...",
"content_type_returned": "text"
}
```
For images:
```json
{
"id": "def-456",
"filename": "diagram.png",
"content_type": "image/png",
"content": "data:image/png;base64,...",
"content_type_returned": "base64_image"
}
```
Image base64 is capped at 10MB to prevent context blow-up. Larger images
return an error suggesting the user re-upload.
---
## 2. `conversation_search` Tool
### Purpose
After compaction, message details are compressed into summaries. If the
user asks about something the summary omitted, the model needs to search
the original messages. Also useful in long pre-compaction conversations
where the model's attention has drifted.
### Design
Keyword search (full-text) over messages in the current channel using
PostgreSQL's `plainto_tsquery`. No migration needed — `to_tsvector` is
computed on the fly. This is acceptable because:
- Search is scoped to a single channel (bounded message count)
- Called infrequently (tool invocation, not every request)
- Channel messages rarely exceed ~10K rows
If performance becomes an issue, a `search_vector tsvector` column with
GIN index can be added in a future migration.
### Parameters
```go
Parameters: JSONSchema(map[string]interface{}{
"query": Prop("string", "Search keywords"),
"max_results": map[string]interface{}{
"type": "integer",
"description": "Maximum results (1-20, default 5)",
},
"role_filter": PropEnum("Filter by message role", "all", "user", "assistant"),
}, []string{"query"})
```
### Returns
```json
{
"results": [
{
"message_id": "...",
"role": "user",
"excerpt": "...highlighted excerpt...",
"timestamp": "2026-02-26T10:00:00Z",
"rank": 0.95
}
],
"query": "original query",
"total": 3,
"channel_id": "..."
}
```
Uses `ts_headline` for highlighted excerpts (same pattern as `note_search`).
### Security
Channel-scoped via `ExecutionContext.ChannelID`. Only searches messages
in the current channel. Soft-deleted messages (`deleted_at IS NOT NULL`)
are excluded.
### Dependencies
Late registration:
```go
func RegisterConversationSearch(stores store.Stores)
```
No external dependencies beyond stores (uses `database.DB` directly for
the full-text query, same pattern as `note_search`).
---
## 3. Token Estimator Improvements
### Frontend (`tokens.js`)
Current `estimateConversation` counts message text only. Enhancements:
- **Staged attachments**: Before send, show estimated tokens for queued
attachments. Documents: `extracted_text.length / 4`. Images: flat
~765 tokens per image (OpenAI's base tile estimate, reasonable cross-provider).
- **Sent attachments**: After send, the augmented content is already in
the message text (documents are inlined). Images were ephemeral
ContentParts — add their estimate to the message's token count.
### Implementation
Add to `Tokens`:
```javascript
estimateAttachments(attachments) {
let total = 0;
for (const att of attachments) {
if (att.content_type?.startsWith('image/')) {
total += 765; // base tile estimate
} else if (att.extracted_text) {
total += this.estimate(att.extracted_text);
} else {
total += Math.ceil(att.size_bytes / 4); // rough fallback
}
}
return total;
}
```
Wire into `updateInputTokens()` to include staged attachment estimates.
---
## 4. KB Auto-Injection
### Purpose
Currently the model must explicitly call `kb_search` to access knowledge
base content. For common patterns (channel linked to a KB, user asks a
question), automatic injection into the system prompt is more natural.
### Design
In the completion handler, after loading conversation history and before
building the provider request:
1. Check if channel has linked KBs (or user has personal KBs)
2. Extract the user's latest message as the query
3. Run similarity search (top-K, K=3 default)
4. If results found AND context budget allows, prepend to system prompt
### Context budget check
```go
contextUsed := estimateTokens(systemPrompt + allMessages)
kbTokens := estimateTokens(kbContent)
if contextUsed + kbTokens > maxContext * 0.85 {
// Skip injection — context too full
log.Printf("⏭ KB auto-inject skipped: context %.0f%% full", pct*100)
}
```
### Channel settings
```json
{
"kb_injection": "auto" | "tool_only" | "both" | "off"
}
```
Default: `"auto"` — inject when KBs are linked. `"tool_only"` disables
auto-injection but keeps the `kb_search` tool available. `"both"` does
both. `"off"` disables all KB context.
### System prompt format
```
[Knowledge Base Context]
The following information was retrieved from the user's knowledge bases
and may be relevant to this conversation:
---
Source: Technical Docs (chunk 3 of report.pdf, similarity: 0.87)
Content: ...
---
Source: FAQ (chunk 1 of faq.md, similarity: 0.82)
Content: ...
[End Knowledge Base Context]
```
---
## File Inventory
### New files
| File | Description |
|------|-------------|
| `server/tools/attachment_recall.go` | attachment_recall tool |
| `server/tools/conversation_search.go` | conversation_search tool |
| `docs/DESIGN-0.15.1.md` | This design doc |
### Modified files
| File | Change |
|------|--------|
| `server/main.go` | Register new tools |
| `server/handlers/completion.go` | KB auto-injection in completion flow |
| `src/js/tokens.js` | Attachment-aware token estimation |
| `docs/ROADMAP.md` | Mark v0.15.1 items |
### No migration required
- `conversation_search` uses on-the-fly `to_tsvector` (no new column)
- `attachment_recall` reads existing `extracted_text` column
- KB auto-injection uses existing `kb_search` infrastructure
---
## Implementation Order
1. **Phase 1**: `attachment_recall` tool + registration ✅
2. **Phase 2**: `conversation_search` tool + registration ✅
3. **Phase 3**: Token estimator improvements (frontend) ✅
4. **Phase 4**: KB auto-injection (completion handler) — separate delivery,
requires adding embedder dependency to `CompletionHandler` and careful
latency budgeting (embedding query on every completion call)

1061
docs/DESIGN-0.17.3.md
View File

File diff suppressed because it is too large Load Diff

235
docs/DESIGN-0.18.0.md
View File

@@ -1,235 +0,0 @@
# DESIGN-0.18.0: Memory System
**Version:** 0.18.0
**Status:** Phase 1 Complete
**Depends on:** compaction (v0.15.0), knowledge bases (v0.14.0), persona-KB binding (v0.17.0), SQLite backend (v0.17.1)
---
## 1. Overview
Memory provides long-term fact persistence across conversations. Unlike
compaction (within-conversation context) or knowledge bases (static
documents), memory captures preferences, facts, and context that
accumulate over time from natural conversation.
**Key differentiator: Persona-scoped memory.** Each Persona builds
its own memory independently, preventing cross-context contamination.
A helpdesk Persona accumulates FAQ knowledge; a tutoring Persona
tracks per-student progress; a user's personal preferences stay
separate from any Persona context.
---
## 2. Scope Model
Three memory scopes with clear ownership:
```
┌─────────────────────────────────────────────────┐
│ user scope │
│ owner_id = user_id │
│ "Jeff prefers Go over Python" │
│ "Deployment uses Kubernetes + Traefik" │
│ Visible in ALL conversations for this user │
└─────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────┐
│ persona scope │
│ owner_id = persona_id │
│ "Common question: how to reset password" │
│ "Policy: refunds within 30 days only" │
│ Shared across ALL users of this Persona │
└─────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────┐
│ persona_user scope │
│ owner_id = persona_id, user_id = user_id │
│ "Student struggles with recursion" │
│ "Customer prefers email over phone" │
│ Per-user within a specific Persona │
└─────────────────────────────────────────────────┘
```
**Scope resolution at recall time:**
When a Persona is active, all three scopes are queried and merged
with priority ordering: persona_user > persona > user. When no
Persona is active, only user scope is queried.
---
## 3. Data Model
### 3.1 Table: `memories`
| Column | Type | Description |
|--------|------|-------------|
| `id` | UUID / TEXT | Primary key |
| `scope` | TEXT | `user`, `persona`, `persona_user` |
| `owner_id` | UUID / TEXT | user_id (user scope) or persona_id (persona/persona_user) |
| `user_id` | UUID / TEXT | Only for persona_user scope — identifies which user |
| `key` | TEXT | Short label (2-6 words) |
| `value` | TEXT | The fact/detail |
| `source_channel_id` | UUID / TEXT | Provenance — which conversation |
| `confidence` | REAL | 0.0-1.0, LLM-provided |
| `status` | TEXT | active, pending_review, archived |
| `embedding` | vector(3072) / TEXT | For semantic recall (Phase 2) |
| `created_at` | TIMESTAMPTZ / TEXT | |
| `updated_at` | TIMESTAMPTZ / TEXT | |
### 3.2 Indexes
- **Primary lookup:** `(scope, owner_id, status)` — filtered to `status = 'active'`
- **Persona+user:** `(owner_id, user_id)` — filtered to `scope = 'persona_user'`
- **Dedup:** unique on `(scope, owner_id, COALESCE(user_id, nil_uuid), key)`
- **FTS:** GIN on `to_tsvector(key || ' ' || value)` (Postgres) / LIKE fallback (SQLite)
- **Provenance:** `(source_channel_id)` — "where did this memory come from?"
### 3.3 Model Struct
```go
type Memory struct {
ID string `json:"id"`
Scope string `json:"scope"`
OwnerID string `json:"owner_id"`
UserID *string `json:"user_id,omitempty"`
Key string `json:"key"`
Value string `json:"value"`
SourceChannelID *string `json:"source_channel_id,omitempty"`
Confidence float64 `json:"confidence"`
Status string `json:"status"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
```
---
## 4. Store Interface
```go
type MemoryStore interface {
Upsert(ctx, *Memory) error // create or update (matched on scope+owner+user+key)
GetByID(ctx, id) (*Memory, error)
List(ctx, MemoryFilter) ([]Memory, error)
Delete(ctx, id) error // hard delete
Archive(ctx, id) error // soft delete (status → archived)
Recall(ctx, userID, *personaID, query, limit) ([]Memory, error) // scope-merged search
CountByOwner(ctx, scope, ownerID) (int, error)
}
```
**Key design decisions:**
- `Upsert` uses the unique index for conflict resolution — same key = update value
- `Recall` does a UNION across all applicable scopes with priority ordering
- Both Postgres and SQLite implementations exist (same pattern as all other stores)
- SQLite falls back to LIKE for text search (no tsvector)
---
## 5. Tools
### 5.1 `memory_save`
The LLM calls this to explicitly store a fact. Scope is automatically
determined by whether a Persona is active:
- No Persona → `user` scope, `owner_id = user_id`
- Persona active → `persona_user` scope, `owner_id = persona_id`, `user_id = user_id`
Parameters: `key` (required), `value` (required), `confidence` (optional, default 1.0)
### 5.2 `memory_recall`
The LLM calls this to search stored memories. Results merge across
all applicable scopes.
Parameters: `query` (optional), `max_results` (optional, default 20)
### 5.3 Registration
Late registration pattern (same as notes, KB search, conversation search):
```go
// main.go
tools.RegisterMemoryTools(stores)
```
---
## 6. Memory Injection
At completion time, `loadConversation()` injects relevant memories
into the system prompt — same position as KB hints and compaction
summaries:
```
[admin system prompt]
[persona/channel system prompt]
[KB hint]
[memory injection] ← NEW
[compaction summary OR message history]
```
`BuildMemoryHint()` calls `Recall()` with an empty query (returns
top memories by confidence/recency), formats as bullet points, and
caps at ~1500 tokens (~6000 chars) to preserve context budget.
---
## 7. Implementation Phases
### Phase 1: Foundation & Tools ✅ (this delivery)
- Migrations (Postgres + SQLite)
- Memory model struct
- MemoryStore interface + both implementations
- memory_save + memory_recall tools
- Memory injection in loadConversation
- 9 new files, ~5 edits to existing files
### Phase 2: Automatic Extraction & Embeddings
- Background job: post-conversation analysis extracts facts
- Configurable extraction prompt per Persona
- Extracted memories start as `pending_review`
- Embedding generation for semantic recall
- Recall upgraded to use vector similarity + keyword hybrid
### Phase 3: Review Pipeline & User Controls
- Admin review queue for pending memories
- Persona memory review for team admins
- User Settings → Memory: view, edit, delete
- Per-Persona memory toggle
- Retention policies
- Frontend memory management UI
---
## 8. Persona Memory Saves (Future: Phase 2)
Phase 1 only supports `user` and `persona_user` scopes via the tool
(the LLM saves about the user, scoped appropriately). Phase 2 adds
the ability for admins/team leads to configure Persona-scope memory
extraction — the Persona "learns" from conversations:
```
Persona system prompt addition (Phase 2):
"After each conversation, identify key facts, common questions,
and useful patterns. Save these as persona-scope memories for
future reference by all users."
```
This runs as a background job, not during the live conversation,
to avoid latency impact.
---
## 9. Privacy & Security
- User memories never cross team boundaries
- Persona memories respect group access controls
- persona_user memories are only visible to the specific user + admins
- Users can view/delete their own memories (Phase 3 UI)
- Admin can disable memory system globally or per-team
- No cross-persona memory leakage — scope isolation is enforced at the query level

244
docs/DESIGN-0.18.1.md
View File

@@ -1,244 +0,0 @@
# Design — v0.18.1 Side Panel Architecture
## Problem
The side panel is a single `<aside>` with two mutually exclusive tabs
(Preview and Notes) toggled by `display: none`. Opening Notes kills the
preview and vice versa. As more panel content arrives (KB browser,
search results, memory viewer), the single-slot model doesn't scale.
The current implementation also loses per-panel state (scroll position,
active note, preview content) on tab switches.
This is also prerequisite plumbing for v0.23.0 multi-participant
channels, where multiple panels need simultaneous visibility.
## Current State
**HTML:** Single `<aside id="sidePanel">` containing:
- Resize handle (`sidePanelResize`)
- Header with tab bar (Preview | Notes) + action buttons (clear, fullscreen, close)
- Body with two `.side-panel-page` divs toggled by display
**JS (ui-format.js lines 295–365):**
- `openSidePanel(tab)` — adds `.open` class, calls `switchSidePanelTab`
- `closeSidePanel()` — removes `.open` and `.fullscreen`, clears inline width
- `switchSidePanelTab(tab)` — toggles active class on tab buttons, toggles display on page divs
- `_initSidePanelResize()` — mousedown/move/up drag handler for width resize
**Callers (4 total):**
1. `toggleHTMLPreview()` → `openSidePanel('preview')` (ui-format.js)
2. `openNotes()` → `openSidePanel('notes')` / `closeSidePanel()` (notes.js)
3. Escape handler → `closeSidePanel()` (app.js)
4. Inline onclick handlers in index.html (tab buttons, close, fullscreen, clear)
**CSS (styles.css lines 642–737):**
- `.side-panel` base: `width: 0; min-width: 0; overflow: hidden`, transition on width
- `.side-panel.open`: `width: 480px; min-width: 480px`
- `.side-panel.fullscreen`: fixed positioning, 100% width
- Mobile (`@media max-width: 768px`): open = fixed overlay at 100vw
## Design
### Panel Registry
A lightweight registry in a new `panels.js` file. Each panel is a named
entry with its own state and DOM container. The registry owns all
open/close/focus logic — individual panels never touch the `<aside>`
directly.
```js
const PanelRegistry = {
_panels: {}, // { name: { element, onOpen, onClose, onFocus, state } }
_active: null, // currently visible panel name (single-view mode)
_secondary: null, // second visible panel (dual-view mode)
_dualMode: false,
register(name, opts) { ... },
open(name) { ... },
close(name) { ... },
toggle(name) { ... },
closeAll() { ... },
isOpen(name) { ... },
active() { ... },
};
```
**Panel registration contract:**
```js
PanelRegistry.register('notes', {
element: document.getElementById('sidePanelNotes'),
onOpen() { loadNotesList(); loadNoteFolders(); },
onClose() { /* optional cleanup */ },
onFocus() { /* optional re-activate */ },
state: {}, // panel-specific state bag (scroll pos, etc.)
});
```
Panels register themselves during initialization — Notes registers in
`notes.js`, Preview registers in `ui-format.js`. This keeps domain
logic co-located with its panel.
### DOM Structure
Replace the hardcoded tab bar with a registry-driven header. The
`<aside>` structure stays identical in shape (no layout break), but
tabs render dynamically from registered panels:
```html
<aside class="side-panel" id="sidePanel">
<div class="side-panel-resize" id="sidePanelResize"></div>
<div class="side-panel-header">
<div class="side-panel-tabs" id="sidePanelTabs">
<!-- rendered by PanelRegistry from registered panels -->
</div>
<div class="side-panel-actions" id="sidePanelActions">
<!-- per-panel action buttons injected by active panel -->
<button class="side-panel-action-btn" id="sidePanelFullscreenBtn" ...>
<button class="side-panel-close" ...>
</div>
</div>
<div class="side-panel-body" id="sidePanelBody">
<!-- all .side-panel-page divs, shown/hidden by registry -->
</div>
</aside>
```
The tab bar is re-rendered on `register()` and on `open()`/`close()`.
Each panel can declare `actions` — an array of button configs that
appear in the header when that panel is active (replacing the current
hardcoded clear button for preview).
### State Preservation
Each panel gets a `state` bag on its registry entry. Before hiding a
panel, the registry calls a `saveState()` hook (if provided) which can
snapshot scroll position, selection, etc. On re-show, `restoreState()`
re-applies. This is opt-in — panels that don't need it don't implement
the hooks.
Notes state: `{ scrollTop, editingNoteId, viewMode: 'list'|'editor'|'graph' }`
Preview state: `{ srcdoc, hasContent }`
### Dual-View Mode (Phase 2)
When enabled, the panel body splits into two panes with an internal
divider. `_active` shows on the left (or top on mobile), `_secondary`
on the right (or bottom). The outer resize handle still controls total
panel width; an inner divider controls the split ratio.
Dual mode activates via:
- `Ctrl/Cmd+Shift+\` keyboard shortcut
- Dragging a tab to the other half (stretch goal)
### Per-Panel Action Buttons
Currently Preview has a "Clear" button and both panels share Fullscreen
and Close. The new model:
- **Global actions** (always visible): Fullscreen, Close
- **Panel actions** (per-panel, in active panel's header slot):
- Preview: Clear
- Notes: (none currently, but room for "New Note" shortcut)
- Future panels declare their own
Panels declare actions at registration time:
```js
PanelRegistry.register('preview', {
element: document.getElementById('sidePanelPreview'),
actions: [
{ id: 'sidePanelClearBtn', icon: '...svg...', title: 'Clear preview',
onClick: clearPreview, visible: () => hasPreviewContent() },
],
});
```
### Preview Enhancements (Phase 3)
- **Live-update during streaming:** The chat streaming handler already
accumulates HTML code blocks. On each chunk that completes a fenced
code block tagged `html`, if the preview panel is open, update the
iframe's `srcdoc`. Debounce at 500ms to avoid thrashing.
- **Extension block preview:** Mermaid, KaTeX, etc. render inline in
chat. Add a "Pop out" button on extension-rendered blocks that opens
the rendered output in the preview panel (full-size, zoomable).
### Mobile (Phase 4)
- **Swipe navigation:** Touch event handlers on `.side-panel-body` —
swipe left/right cycles through open panels. Uses CSS `transform:
translateX()` with a 150px threshold for commit.
- **Bottom sheet mode:** On viewports < 768px, the secondary panel in
dual mode renders as a bottom sheet (40% height) instead of side-by-
side.
- **Priority collapse:** When space is tight and dual mode can't fit
(< 600px combined), collapse to single-panel with most-recently-
opened winning. The displaced panel becomes accessible via tab bar.
### Keyboard Shortcuts
| Shortcut | Action |
|----------|--------|
| `Ctrl/Cmd+\` | Cycle active panel (or open panel container if closed) |
| `Ctrl/Cmd+Shift+\` | Toggle dual-view mode |
| `Escape` | Close panel container (existing behavior, unchanged) |
## Phases
### Phase 1 — Panel Registry + State Management
Extract panel logic from `ui-format.js` into `panels.js`. Implement
`PanelRegistry` with register/open/close/toggle/closeAll. Migrate
Notes and Preview to register as panels. Dynamic tab bar rendering.
Per-panel action buttons. State preservation hooks.
**Files changed:**
- New: `src/js/panels.js`
- Modified: `src/js/ui-format.js` (remove panel functions, add preview registration)
- Modified: `src/js/notes.js` (replace `openSidePanel`/`closeSidePanel` calls with `PanelRegistry.open`/`close`)
- Modified: `src/js/app.js` (update Escape handler, add keyboard shortcuts, replace `_initSidePanelResize` call)
- Modified: `src/index.html` (remove hardcoded tab buttons, add `panels.js` script tag, make tab bar dynamic)
- No CSS changes (reuse existing `.side-panel-tab` styles)
**Backward compatibility:** All existing panel behavior works identically.
`openSidePanel()` and `closeSidePanel()` become thin wrappers around
`PanelRegistry` during transition, then get removed after all callers
migrate.
### Phase 2 — Dual-View Layout
CSS grid split inside `.side-panel-body`. Internal divider with drag
resize. Keyboard shortcut for dual toggle. Tab drag to split (stretch).
**Files changed:**
- Modified: `src/js/panels.js` (dual-view logic)
- Modified: `src/css/styles.css` (grid layout for dual panes)
### Phase 3 — Preview Enhancements
Live-update preview during streaming. Extension block "pop out" button.
**Files changed:**
- Modified: `src/js/ui-format.js` (pop-out button on extension blocks)
- Modified: `src/js/chat.js` (streaming hook for live preview update)
- Modified: `src/js/panels.js` (debounced preview update API)
### Phase 4 — Mobile
Swipe navigation, bottom sheet mode, priority collapse.
**Files changed:**
- Modified: `src/js/panels.js` (touch handlers, priority logic)
- Modified: `src/css/styles.css` (bottom sheet layout, responsive breakpoints)
## Migration Checklist
- [ ] `openSidePanel(tab)` → `PanelRegistry.open(tab)`
- [ ] `closeSidePanel()` → `PanelRegistry.closeAll()`
- [ ] `switchSidePanelTab(tab)` → `PanelRegistry.open(tab)` (open implies focus)
- [ ] `toggleSidePanelFullscreen()` → stays global (operates on container, not individual panels)
- [ ] `_initSidePanelResize()` → moves to `panels.js` (operates on container)
- [ ] Escape handler: `PanelRegistry.closeAll()`
- [ ] `openNotes()` toggle logic: `PanelRegistry.toggle('notes')`
- [ ] `toggleHTMLPreview()`: `PanelRegistry.open('preview')` + update iframe content

1151
docs/DESIGN-0.19.0.md
View File

File diff suppressed because it is too large Load Diff

194
docs/DESIGN-0.19.1.md
View File

@@ -1,194 +0,0 @@
# v0.19.1 — Active Projects, Project System Prompt, Project Detail Panel
**Status:** Implementation
**Depends on:** v0.19.0 Projects/Workspaces
---
## Overview
Three tightly-scoped improvements that make projects genuinely usable
for daily workflow rather than just organizational containers:
1. **Active Project** — one-click project activation so new chats
auto-assign without drag-drop.
2. **Project System Prompt** — per-project instructions injected into
every chat within the project at completion time.
3. **Project Detail Panel** — side panel UI for managing a project's
system prompt, KB bindings, and notes.
---
## 1. Active Project
### State
| Location | Key | Type |
|----------|-----|------|
| `App.activeProjectId` | runtime | `string \| null` |
| `localStorage` | `cs-active-project` | `string \| null` |
### Behavior
- Click project header's **pin icon** (or "Set as active" in ⋯ menu)
to activate.
- Active project gets a visual indicator: left accent border + subtle
background tint using the project's color.
- Creating a new chat (via `sendMessage` when `!App.currentChatId`)
auto-calls `addChannelToProject(activeProjectId, newChannelId)` and
sets `chat.projectId` client-side.
- Only one project can be active at a time. Clicking the pin on an
already-active project deactivates it.
- Active state persists across page reloads via `localStorage`.
- If the active project is deleted, `activeProjectId` is cleared.
### Files Changed
| File | Change |
|------|--------|
| `src/js/app.js` | Add `activeProjectId: null` to App, restore from localStorage on init |
| `src/js/projects-ui.js` | `setActiveProject(id)`, `clearActiveProject()`, menu items, pin icon |
| `src/js/chat.js` | Auto-assign in `sendMessage` after channel creation |
| `src/js/ui-core.js` | `.active` class on project group in `renderChatList` |
| `src/css/styles.css` | `.project-group.active` styling |
---
## 2. Project System Prompt
### Injection Order
```
1. Admin system prompt (global policy — no opt-out)
2. Persona OR channel prompt (role/character definition)
3. ▸ PROJECT system prompt (project-specific context) ← NEW
4. KB hint (available knowledge bases)
5. Memory hint (user facts from v0.18.0)
6. Summary / conversation (message history)
```
The project prompt sits between the role definition and the KB hint.
This lets a persona define "you are a coding assistant" while the
project adds "this project is the React dashboard for client X."
### Backend Flow (completion.go)
In `loadConversation`, after the persona/channel system prompt block:
```go
// ── Project system prompt (v0.19.1) ──
if h.stores.Projects != nil {
projID, _ := h.stores.Projects.GetProjectIDForChannel(ctx, channelID)
if projID != "" {
proj, err := h.stores.Projects.GetByID(ctx, projID)
if err == nil {
if sp, ok := proj.Settings["system_prompt"].(string); ok && sp != "" {
messages = append(messages, providers.Message{
Role: "system",
Content: sp,
})
}
}
}
}
```
### Storage
Uses existing `projects.settings` JSONB column. No migration needed.
```json
{
"system_prompt": "You are helping build the Chat Switchboard project..."
}
```
### Files Changed
| File | Change |
|------|--------|
| `server/handlers/completion.go` | Inject project system prompt in `loadConversation` |
---
## 3. Project Detail Panel
### Registration
Follows the same `PanelRegistry.register()` pattern as notes and
preview panels. Registered during app init via `_registerProjectPanel()`.
### Panel Layout
```
┌─────────────────────────────┐
│ Project: <name> [✕] │ ← header (editable name)
├─────────────────────────────┤
│ System Prompt │
│ ┌─────────────────────────┐ │
│ │ textarea (auto-resize) │ │
│ └─────────────────────────┘ │
│ [Save] │
├─────────────────────────────┤
│ Knowledge Bases [+] │
│ ┌─────────────────────────┐ │
│ │ • KB Alpha [✕] │ │
│ │ • KB Beta [✕] │ │
│ └─────────────────────────┘ │
│ (empty: "No KBs bound") │
├─────────────────────────────┤
│ Notes │
│ ┌─────────────────────────┐ │
│ │ • Meeting notes [✕] │ │
│ │ • Design spec [✕] │ │
│ └─────────────────────────┘ │
│ (empty: "No notes") │
├─────────────────────────────┤
│ Color [●] │
│ Archived [ ] │
└─────────────────────────────┘
```
### Interactions
- **System prompt:** Textarea with debounced save (1s) or explicit Save
button. Writes to `PUT /projects/:id` with
`settings: { system_prompt: "..." }`.
- **KB add:** [+] opens a dropdown of available KBs (from
`GET /knowledge-bases`), filtered to exclude already-bound ones.
Calls `POST /projects/:id/knowledge-bases`.
- **KB remove:** [✕] calls `DELETE /projects/:id/knowledge-bases/:kbId`.
- **Notes:** Read-only list with [✕] to unbind. Notes are auto-associated
from channel creation (v0.19.0).
- **Color:** Reuses the invisible color picker pattern from
`setProjectColor()`.
- **Open trigger:** Click project name in sidebar header, or
"Project details" in ⋯ menu.
### Files Changed
| File | Change |
|------|--------|
| `src/index.html` | Add `#sidePanelProject` div inside `#sidePanelBody` |
| `src/js/projects-ui.js` | `openProjectPanel(id)`, `_registerProjectPanel()`, panel rendering, KB/note management |
| `src/js/app.js` | Call `_registerProjectPanel()` in init |
| `src/css/styles.css` | `.project-panel-*` styles |
---
## Files Summary
| File | Phase | Type |
|------|-------|------|
| `server/handlers/completion.go` | 2 | Modified |
| `src/js/app.js` | 1,3 | Modified |
| `src/js/projects-ui.js` | 1,3 | Modified |
| `src/js/chat.js` | 1 | Modified |
| `src/js/ui-core.js` | 1 | Modified |
| `src/css/styles.css` | 1,3 | Modified |
| `src/index.html` | 3 | Modified |
| `VERSION` | — | Modified (0.19.0 → 0.19.1) |
| `CHANGELOG.md` | — | Modified |
No new files. No migrations. Backend change is a single injection
point in completion.go.

116
docs/DESIGN-0.19.2.md
View File

@@ -1,116 +0,0 @@
# v0.19.2 — Project Persona Default, Archive Toggle, Channel Reorder
**Status:** Implementation
**Depends on:** v0.19.1
---
## 1. Project-level Persona Default
### Concept
Bind a persona to a project so all chats inherit model, system prompt,
and parameters without selecting it each time. The persona acts as a
fallback when no explicit preset is chosen per-message.
### Resolution Chain (completion.go)
```
1. Explicit req.PresetID → highest priority
2. Project persona default → fallback when no explicit preset ← NEW
3. No persona → bare model, no persona context
```
### Storage
`projects.settings` JSONB:
```json
{ "persona_id": "uuid-here", "system_prompt": "..." }
```
### Backend Changes
In `completion.go`, after the explicit preset block, before
`resolveConfig`:
```go
// ── Project persona fallback (v0.19.2) ──
if req.PresetID == "" && h.stores.Projects != nil {
projID, _ := h.stores.Projects.GetProjectIDForChannel(ctx, channelID)
if projID != "" {
proj, _ := h.stores.Projects.GetByID(ctx, projID)
if proj != nil {
if pid, ok := proj.Settings["persona_id"].(string); ok && pid != "" {
preset := ResolvePreset(h.stores, pid, userID)
if preset != nil {
personaID = preset.ID
if req.Model == "" { req.Model = preset.BaseModelID }
// ... same defaults as explicit preset
}
}
}
}
}
```
### Frontend Changes
Project detail panel: persona picker dropdown below system prompt.
Lists available personas, saves to `settings.persona_id`.
---
## 2. Project Archive Toggle
### Behavior
- Toggle in project detail panel (checkbox)
- Archived projects hidden from sidebar by default
- "Show archived" toggle at bottom of sidebar (or in panel)
- Uses existing `is_archived` column — no migration needed
### Files Changed
- `src/js/projects-ui.js` — archive toggle in panel, sidebar filter
- `src/js/ui-core.js` — filter archived in renderChatList
- `src/css/styles.css` — dimmed style for archived
---
## 3. Channel Reorder Within Project
### Behavior
- Dragging a chat within the same project group reorders it
- Drop between chats inserts at that position
- Calls `PUT /projects/:id/channels/reorder` with new order
- Position persisted server-side, respected on next load
### Implementation
- `onProjectDrop` detects same-project reorder vs cross-project move
- Compute new channel order from DOM after drop
- Channels within project sorted by position from
`project_channels.position` (loaded alongside projects)
### Files Changed
- `src/js/projects-ui.js` — reorder logic
- `src/js/ui-core.js` — sort projChats by position
- `src/js/app.js` — store project channel positions
---
## Files Summary
| File | Change |
|------|--------|
| `server/handlers/completion.go` | Project persona fallback |
| `src/js/projects-ui.js` | Persona picker, archive toggle, reorder |
| `src/js/ui-core.js` | Archive filter, position sort |
| `src/js/app.js` | Channel positions state |
| `src/css/styles.css` | Archive styles |
| `VERSION` | 0.19.1 → 0.19.2 |
| `CHANGELOG.md` | New entry |
No migrations. No new files.

512
docs/ICD-API.md
View File

@@ -1,7 +1,7 @@
# ICD-API — Chat Switchboard Backend API Contract
**Version:** 0.22.7
**Updated:** 2026-03-03
**Version:** 0.23.0
**Updated:** 2026-03-04
**Audience:** Frontend developers, integrators, API consumers
> **Organization principle:** Each section is one domain — the complete story.
@@ -31,6 +31,7 @@
15. [Teams & Access Control](#15-teams--access-control)
16. [Platform Administration](#16-platform-administration)
17. [Export & Utility](#17-export--utility)
17b. [Files & Storage](#17b-files--storage)
18. [WebSocket Protocol](#18-websocket-protocol)
19. [Appendix: Enums](#19-appendix-enums)
@@ -118,14 +119,6 @@ both Postgres and SQLite backends.
ISO 8601 with timezone: `"2025-06-15T14:30:00Z"`. Stored as
`TIMESTAMPTZ` (Postgres) or `TEXT` (SQLite).
### Naming Note
The domain concept is **Persona** (identity + system prompt + model +
KB bindings + grant scoping). API routes currently use `/personas` for
historical reasons. A pre-1.0 rename to `/personas` is planned. This
document uses "Persona" for the concept and shows the current route
paths as-is.
---
## 2. Auth
@@ -209,8 +202,9 @@ Revokes the refresh token. Returns `{ "message": "logged out" }`.
The **channel** is the universal conversation container. Messages,
tool activity, attachments, and KB bindings all hang off a channel.
Today channels are single-user direct chats; the architecture
anticipates multi-participant channels for v0.23.0.
Channels support multiple participants — users, personas, and
sessions — making them the foundation for collaborative and
multi-model workflows.
### 3.1 Channel CRUD
@@ -225,9 +219,8 @@ Returns pagination envelope. Each channel:
```json
{
"id": "uuid",
"user_id": "uuid",
"title": "My Chat",
"type": "direct",
"type": "direct|group|workflow",
"description": "",
"model": "claude-sonnet-4-20250514",
"provider_config_id": "uuid|null",
@@ -235,11 +228,20 @@ Returns pagination envelope. Each channel:
"project_id": "uuid|null",
"folder": "string|null",
"tags": ["tag1", "tag2"],
"participant_count": 1,
"created_at": "...",
"updated_at": "..."
}
```
**Channel types:**
| Type | Description |
|------|-------------|
| `direct` | Single-user chat (default, backward compatible) |
| `group` | Multi-user/multi-model collaborative channel |
| `workflow` | Staged channel driven by workflow definitions |
**Create channel:**
```
@@ -259,9 +261,10 @@ POST /channels
}
```
On create, if `model` and `provider_config_id` are provided, the
channel model roster (`channel_models`) is auto-populated with an
initial entry.
On create, the authenticated user is automatically added as a
participant with `role: "owner"`. If `model` and `provider_config_id`
are provided, the channel model roster (`channel_models`) is
auto-populated with an initial entry.
**Get channel:**
@@ -315,6 +318,8 @@ following the cursor at each fork. This is what the UI renders.
"content": "...",
"model": "claude-sonnet-4-20250514|null",
"provider_config_id": "uuid|null",
"participant_id": "uuid|null",
"participant_type": "user|persona|session|null",
"thinking": "...|null",
"tool_calls": [...],
"tool_results": [...],
@@ -328,6 +333,11 @@ following the cursor at each fork. This is what the UI renders.
}
```
`participant_id` and `participant_type` identify who authored the
message. For `user` messages, this is the authenticated user. For
`assistant` messages, this is the persona (if set) or null for raw
model responses. Existing messages without participants return null.
**List all messages** — includes all branches, flat:
```
@@ -432,15 +442,23 @@ POST /chat/completions
}
```
`channel_id` is required. `chat_id` is accepted as a deprecated alias
(maps to `channel_id`; frontend no longer sends it — removal planned).
`channel_id` is required.
`extra_body` is a freeform JSON object merged into the provider request
(provider-specific parameters like `temperature`, `reasoning`, etc.).
**Persona resolution:** If `persona_id` is set, the handler loads the
persona's system prompt, model, provider_config_id, and KB bindings.
Per-message `model`/`provider_config_id` override the persona's defaults.
**Participant scoping:** The requesting user must be a participant in
the channel. Messages are attributed to the authenticated user (or
session). In `group` channels, all participants see all messages in
real time via WebSocket.
**Persona resolution:** The primary completion path. If `persona_id`
is set (explicitly or via `@mention` resolution from message content),
the handler loads the persona's system prompt, model, provider config,
and KB bindings. Per-message `model`/`provider_config_id` override the
persona's defaults. In multi-persona channels, `@mention` in the
message content is parsed to resolve the target persona from the
channel's participant list.
**Routing resolution:** After config resolution, the routing policy
evaluator (`evaluateRouting()`) may redirect to a different provider
@@ -502,10 +520,16 @@ GET /tools
Returns `{ "tools": [...] }` where each tool has `name`, `description`,
`parameters` (JSON Schema), and `category`.
### 3.4 Multi-Model Roster
### 3.4 Multi-Model Roster (Raw Access)
Channels can have multiple AI models. The `channel_models` table tracks
the roster. `@mention` in message content routes to specific models.
The primary way to add AI to a channel is by adding a **persona as a
participant** (§3.7). Adding a persona automatically populates the
channel model roster with the persona's underlying model.
For the rare case where a user needs raw model access without a
persona wrapper, the `channel_models` table provides direct model
roster management. This is the 1% escape hatch — most users should
never need it.
```
GET /channels/:id/models → { "models": [...] }
@@ -524,10 +548,20 @@ Each roster entry:
"provider_config_id": "uuid",
"display_name": "Claude",
"is_default": true,
"persona_id": "uuid|null",
"created_at": "..."
}
```
`persona_id`: set when this roster entry was auto-created by adding a
persona participant. Null for manually-added raw models.
**`@mention` resolution:** When a user types `@` in a channel, the
autocomplete shows persona participants first (by display name), then
raw model roster entries. The target of a `@mention` is always
resolved to a specific `provider_config_id` + `model` pair — never
an ambiguous bare model name.
### 3.5 Channel Utilities
**Summarize channel** (compaction):
@@ -548,65 +582,236 @@ POST /channels/:id/generate-title
Uses the utility model to auto-generate a title from the first
exchange. Returns `{ "title": "Generated Title" }`.
### 3.6 Attachments
### 3.6 Files
File attachments are channel-scoped. Upload stores the file in blob
storage; a background extraction pipeline processes documents (PDF,
DOCX, XLSX, etc.) into searchable text.
All binary content associated with channels — user uploads, tool-
generated artifacts, system files — lives in the unified `files`
table backed by the ObjectStore (see §X). The old `attachments`
table and routes are removed.
**Upload:**
**Upload (user):**
```
POST /channels/:id/attachments
POST /channels/:id/files
Content-Type: multipart/form-data
```
Field: `file`. Returns the attachment object.
Field: `file`. Sets `origin: "user_upload"`. Returns the file object.
**Create (tool output):**
```
POST /files
```
```json
{
"channel_id": "uuid",
"message_id": "uuid",
"filename": "generated_image.png",
"content_type": "image/png",
"display_hint": "inline",
"metadata": { "tool_name": "image_generation" }
}
```
Body: multipart `file` field or base64 `data` field.
Sets `origin: "tool_output"`. Returns file object.
**List by channel:**
```
GET /channels/:id/attachments
GET /channels/:id/files?origin=user_upload
```
Returns `{ "attachments": [...] }`.
Returns `{ "files": [...] }`. Filterable by `origin`, `content_type`.
**List by message:**
```
GET /messages/:id/files → { "files": [...] }
```
How the frontend discovers generated artifacts to render inline.
**List by user (file manager):**
```
GET /files?page=1&per_page=50 → { "files": [...], "total": N }
```
All files owned by the authenticated user, paginated.
**Get metadata:**
```
GET /attachments/:id
GET /files/:id
```
```json
{
"id": "uuid",
"channel_id": "uuid",
"message_id": "uuid|null",
"user_id": "uuid",
"origin": "user_upload|tool_output|system",
"filename": "report.pdf",
"content_type": "application/pdf",
"size": 1048576,
"storage_key": "attachments/chan-id/att-id_report.pdf",
"extracted_text_key": "...|null",
"size_bytes": 1048576,
"display_hint": "inline|download|thumbnail",
"extracted_text": "...|null",
"metadata": {},
"created_at": "..."
"created_at": "...",
"updated_at": "..."
}
```
**Download:**
```
GET /attachments/:id/download
GET /files/:id/download
```
Returns the file with appropriate `Content-Type` and
`Content-Disposition` headers.
**Thumbnail:**
```
GET /files/:id/thumbnail
```
**Delete:**
```
DELETE /attachments/:id
DELETE /files/:id
```
Deletes metadata + blob. Channel deletion cascades via FK +
`DeletePrefix`.
### 3.7 Channel Participants
The participant system is how users and AI interact in a channel.
**Personas are the primary way to add AI** — adding a persona as a
participant brings its full identity (name, avatar, system prompt,
model, provider config, KB bindings) into the channel. This is the
99% path. Raw model access without a persona is available via the
model roster (§3.4) for power users.
Every channel has at least one participant (the owner). `direct`
channels have exactly one user participant and optionally one or more
persona participants. `group` and `workflow` channels support multiple
participants of mixed types.
**List participants:**
```
GET /channels/:id/participants
```
Returns `{ "participants": [...] }`:
```json
{
"id": "uuid",
"channel_id": "uuid",
"participant_type": "user|persona|session",
"participant_id": "uuid",
"role": "owner|member|observer",
"display_name": "Jane Doe",
"avatar_url": "...|null",
"joined_at": "..."
}
```
**Participant types:**
| Type | Description |
|------|-------------|
| `user` | Authenticated user. `participant_id` = `users.id` |
| `persona` | AI persona added to channel. `participant_id` = `personas.id`. Brings model, system prompt, KB bindings |
| `session` | Anonymous/session participant (workflow intake). `participant_id` = session token |
**Participant roles:**
| Role | Capabilities |
|------|-------------|
| `owner` | Full control: add/remove participants, delete channel, all member capabilities |
| `member` | Send messages, trigger completions, view history |
| `observer` | Read-only: view messages, no send |
**Add participant:**
```
POST /channels/:id/participants
```
```json
{
"participant_type": "user|persona",
"participant_id": "uuid",
"role": "member"
}
```
Requires `owner` role in the channel. Adding a `persona` participant:
- Makes the persona available as an `@mention` target
- Adds the persona's model to the channel model roster automatically
- Scopes the persona's KB bindings into the channel's KB resolution chain
- The persona's avatar and display name appear in the participant list
**Update participant role:**
```
PATCH /channels/:id/participants/:participantId
```
```json
{ "role": "observer" }
```
**Remove participant:**
```
DELETE /channels/:id/participants/:participantId
```
Cannot remove the last owner. Removing a `persona` participant also
removes its auto-created model roster entry.
**Completion routing:** When `@PersonaName` appears in message
content, the completion handler resolves to that persona's model,
provider config, and system prompt. When no `@mention` is present in
a multi-persona channel, the channel's default persona (first added)
handles the response. The `persona_id` field in the completion
request (§3.3) can also be set explicitly to override `@mention`
resolution.
**Backward compatibility:** Existing `direct` channels that predate
the participant system are auto-migrated on first access — a single
`user` participant with `role: "owner"` is created from the channel's
legacy `user_id` column.
### 3.8 Presence
Real-time presence for channel participants. Delivered via WebSocket
to all subscribers of the channel room.
**Who's here:**
```
GET /channels/:id/presence
```
Returns `{ "participants": [{ "participant_id", "display_name", "status", "last_seen" }] }`.
`status`: `online`, `idle`, `offline`.
**Typing indicators** are sent via WebSocket (see §18.4). The typing
event payload includes `participant_id` and `participant_type` so the
UI can distinguish between human and AI typing.
---
## 4. Personas
@@ -616,6 +821,16 @@ prompt + model + provider config + KB bindings + grant scoping. Users
interact with Personas, not raw provider configs. Admins curate which
Personas are available; team admins create team-scoped Personas.
**Model inheritance:** A Persona inherits all properties of its
underlying model. This includes context window size, max output
tokens, supported capabilities (thinking, tools, vision, streaming),
input/output pricing, and provider status. When the frontend displays
a Persona, it resolves the underlying model's capabilities and shows
the same pills/badges (e.g. thinking, tools, 200k context) that the
raw model would show. The Persona adds identity (name, avatar, system
prompt, KB bindings) on top of the model's capabilities — it never
masks or reduces them.
Three scopes, three route namespaces, one domain:
| Scope | Routes | Who manages |
@@ -637,31 +852,46 @@ All three return the same Persona object shape:
"scope": "personal|team|global",
"owner_id": "uuid|null",
"is_default": false,
"is_active": true,
"avatar_url": "/api/v1/personas/uuid/avatar?v=123",
"inherited": {
"context_window": 200000,
"max_output_tokens": 8192,
"supports_vision": true,
"supports_tools": true,
"supports_thinking": true,
"supports_streaming": true,
"input_price_per_m": 3.00,
"output_price_per_m": 15.00,
"provider_status": "healthy|degraded|down|null"
},
"created_at": "...",
"updated_at": "..."
}
```
The `inherited` block is populated by the backend from the resolved
model capabilities at response time — it is not stored on the persona.
If the underlying model's capabilities change (e.g. provider upgrades
context window), the persona inherits the new values automatically.
The frontend uses `inherited` to render capability pills and context
size badges identical to the raw model.
### 4.1 Personal Personas
Gated by `allow_user_personas` policy.
```
GET /personas → { "personas": [...], "personas": [...] }
GET /personas → { "personas": [...] }
POST /personas ← { "name", "description", "system_prompt", "model", ... }
PUT /personas/:id ← partial update
DELETE /personas/:id
```
**Note:** List response currently sends both `"personas"` and `"personas"`
keys (identical arrays) for backward compatibility. The `"personas"` key
is what the frontend reads. Dual keys will be collapsed pre-1.0.
### 4.2 Team Personas
```
GET /teams/:teamId/personas → { "personas": [...], "personas": [...] }
GET /teams/:teamId/personas → { "personas": [...] }
POST /teams/:teamId/personas ← same shape
PUT /teams/:teamId/personas/:id
DELETE /teams/:teamId/personas/:id
@@ -670,7 +900,7 @@ DELETE /teams/:teamId/personas/:id
### 4.3 Admin (Global) Personas
```
GET /admin/personas → { "personas": [...], "personas": [...] }
GET /admin/personas → { "personas": [...] }
POST /admin/personas ← same shape
PUT /admin/personas/:id
DELETE /admin/personas/:id
@@ -1512,9 +1742,6 @@ The primary endpoint for populating model selectors:
GET /models/enabled
```
> **Note:** `GET /models` exists as an alias. The alias is unused by
> the frontend and will be removed pre-1.0.
Returns `{ "models": [UserModel objects] }`:
```json
@@ -1536,23 +1763,59 @@ Returns `{ "models": [UserModel objects] }`:
"output_price_per_m": 15.00,
"provider_status": "healthy|degraded|down|null",
"scope": "global|team|personal",
"source": "catalog|heuristic"
"source": "catalog|heuristic",
"is_persona": false,
"persona_id": "uuid|null",
"persona_scope": "global|team|personal|null",
"persona_avatar": "url|null",
"persona_team_name": "string|null"
}
```
The capabilities on each model are resolved through the three-tier
chain: catalog DB → heuristic inference → admin overrides (see §10.5).
When `is_persona` is `true`, the capability fields (`context_window`,
`max_output_tokens`, `supports_*`, pricing, `provider_status`) are
inherited from the persona's underlying model. The frontend renders
the same capability pills and badges for a persona as for its raw
model — the persona adds identity, not capability restrictions.
### 11.2 User Model Preferences
Users can hide models they don't want to see.
Users can hide models they don't want to see and set per-model
defaults. Preferences are keyed on the **composite identity**
`provider_config_id:model_id` — the same model from different
providers can have independent visibility.
```
GET /models/preferences → { "preferences": { "model-id": { "hidden": true } } }
PUT /models/preferences ← { "model_id": "...", "hidden": true }
POST /models/preferences/bulk ← { "preferences": { "id1": { "hidden": true }, "id2": { ... } } }
GET /models/preferences → { "preferences": [PreferenceEntry objects] }
PUT /models/preferences ← { "model_id": "...", "provider_config_id": "uuid", "hidden": true }
POST /models/preferences/bulk ← { "entries": [{ "model_id": "...", "provider_config_id": "uuid", "hidden": true }] }
```
PreferenceEntry:
```json
{
"model_id": "grok-4.1-fast",
"provider_config_id": "uuid",
"hidden": true,
"preferred_temperature": 0.7,
"preferred_max_tokens": 4096,
"sort_order": 0
}
```
**Identity rule:** `provider_config_id` is required on write. The same
bare `model_id` from two different provider configs (e.g. global Venice
vs personal BYOK Venice) are independent preference entries. The
frontend composite ID format is `{provider_config_id}:{model_id}`.
**Persona preferences:** Personas are not in this table. A persona's
visibility is controlled by the grant system (§15.3) and the
`is_active` flag, not by model preferences.
---
## 12. Notifications
@@ -2022,6 +2285,121 @@ Docker images).
---
## 17b. Files & Storage
All binary content in Chat Switchboard flows through a single
**ObjectStore** abstraction backed by PVC (filesystem) or S3. All
file metadata lives in one `files` table. The old `attachments`
table is dropped.
### 17b.1 Storage Backend
| Operation | Description |
|-----------|-------------|
| `Put(key, reader, size, contentType)` | Store a blob |
| `Get(key)` → reader, size, contentType | Retrieve a blob |
| `Delete(key)` | Remove a blob |
| `DeletePrefix(prefix)` | Bulk remove (channel deletion) |
| `Exists(key)` | Check without reading |
| `Healthy()` | Backend health check |
| `Stats()` | Aggregate metrics |
Backends: `pvc` (`STORAGE_PATH=/data/storage`) or `s3`
(`S3_ENDPOINT`, `S3_BUCKET`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`).
### 17b.2 Key Namespacing
| Prefix | Pattern | What |
|--------|---------|------|
| `attachments/` | `attachments/{channel_id}/{file_id}_{filename}` | User uploads + AI outputs (legacy prefix retained for existing blobs) |
| `kb/` | `kb/{kb_id}/{doc_id}_{filename}` | Knowledge base documents |
| `exports/` | `exports/{user_id}/{export_id}_{filename}` | Chat/data exports |
| `workspace/` | `workspace/{workspace_id}/{path}` | Workspace files |
### 17b.3 File Object
```sql
CREATE TABLE files (
id UUID PRIMARY KEY,
channel_id UUID REFERENCES channels(id) ON DELETE CASCADE,
message_id UUID REFERENCES messages(id) ON DELETE SET NULL,
user_id UUID REFERENCES users(id) ON DELETE SET NULL,
project_id UUID REFERENCES projects(id) ON DELETE SET NULL,
origin TEXT NOT NULL DEFAULT 'user_upload'
CHECK (origin IN ('user_upload','tool_output','system')),
filename TEXT NOT NULL,
content_type TEXT NOT NULL DEFAULT 'application/octet-stream',
size_bytes BIGINT NOT NULL DEFAULT 0,
storage_key TEXT NOT NULL,
display_hint TEXT NOT NULL DEFAULT 'download'
CHECK (display_hint IN ('inline','download','thumbnail')),
extracted_text TEXT,
metadata JSONB DEFAULT '{}',
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
```
**Origin:**
| Value | Who creates | Examples |
|-------|-------------|---------|
| `user_upload` | User via upload UI | PDF, image, doc attached to message |
| `tool_output` | Tool execution during completion | DALL-E image, video gen, code artifact |
| `system` | Platform | Export, avatar stored as file |
**Display hint:**
| Value | Frontend behavior |
|-------|-------------------|
| `inline` | Render in message stream (images, video, audio, HTML) |
| `download` | Filename + size + download button |
| `thumbnail` | Thumbnail preview, click to expand |
### 17b.4 Tool Output Flow
1. Tool executes (image gen, video render, etc.)
2. Tool writes blob via `ObjectStore.Put`
3. Tool creates file record: `POST /files` with `origin: "tool_output"`, `message_id` = current assistant message
4. Tool result includes `file_id` reference
5. WebSocket emits `file.created` event:
```json
{
"event": "file.created",
"channel_id": "uuid",
"message_id": "uuid",
"file": { ...file object... }
}
```
6. Frontend renders inline based on `display_hint` + `content_type`
The event fires during streaming so the user sees the artifact
before the completion finishes.
### 17b.5 Content Type → Display Hint Defaults
| Content Type | Default Hint | Rendering |
|-------------|-------------|-----------|
| `image/*` | `inline` | `<img>` in message, lightbox on click |
| `video/*` | `inline` | `<video>` player |
| `audio/*` | `inline` | `<audio>` player |
| `text/html` | `inline` | Sandboxed iframe |
| `application/pdf` | `thumbnail` | Thumb + PDF viewer on click |
| `text/*`, `application/json` | `inline` | Syntax-highlighted code block |
| Everything else | `download` | Filename + download button |
Tools can override the default hint explicitly.
### 17b.6 File Manager (Future Surface)
A user-scoped file browser backed by `GET /files?page=1&per_page=50`.
Displays all files owned by the user across all channels, grouped by
channel or date. Supports download and delete. Deleting a file that
is referenced by a message replaces the inline render with a
"file deleted" placeholder.
## 18. WebSocket Protocol
Real-time bidirectional event bus. Single connection per client.
@@ -2069,8 +2447,12 @@ Clients subscribe to rooms for scoped event delivery:
| `message.updated` | → client | Message content changed |
| `message.deleted` | → client | Message removed |
| `cursor.updated` | → client | Branch cursor moved |
| `typing.start` | ↔ both | User/AI started typing |
| `typing.start` | ↔ both | Participant started typing (payload includes `participant_id`, `participant_type`) |
| `typing.stop` | ↔ both | Typing ended |
| `participant.joined` | → client | Participant added to channel |
| `participant.left` | → client | Participant removed from channel |
| `participant.updated` | → client | Participant role changed |
| `presence.update` | → client | Participant online/idle/offline status change |
| `model.changed` | → client | Channel model roster changed |
| `persona.updated` | → client | Persona metadata changed |
| `kb.updated` | → client | Knowledge base changed |
@@ -2088,6 +2470,8 @@ Clients subscribe to rooms for scoped event delivery:
| `extension.updated` | → client | Extension config changed |
| `settings.updated` | → client | Global settings changed |
| `user.updated` | → client | User profile changed |
| `file.created` | → client | File uploaded or generated (§17b.4) |
| `file.deleted` | → client | File removed |
Direction: `→ client` = server-to-client only, `← client` = client-to-server
only, `↔ both` = bidirectional.
@@ -2098,7 +2482,19 @@ only, `↔ both` = bidirectional.
### Channel Types
`direct` (current default), `group` (v0.23.0), `workflow` (v0.25.0)
`direct` (single-user), `group` (multi-user/multi-model), `workflow` (staged)
### Participant Types
`user`, `persona`, `session`
### Participant Roles
`owner`, `member`, `observer`
### Presence Statuses
`online`, `idle`, `offline`
### Message Roles

4
docs/ICD-AUDIT.md
View File

@@ -1,8 +1,8 @@
# ICD Audit: API Contract vs ROADMAP / ARCHITECTURE
**Date:** 2026-03-03 | **Baseline:** v0.22.7
**Date:** 2026-03-04 | **Baseline:** v0.23.0
**Scope:** Cross-reference ICD-API.md against ROADMAP.md, ARCHITECTURE.md, and live codebase.
**Status:** Items §2 (naming), §3 (cleanup) resolved in v0.22.8 naming cleanup pass. See ICD-API.md (domain-organized v2) and DESIGN-SURFACES.md.
**Status:** §2 (naming) fully resolved — preset→persona rename complete across all JS, templates, CSS, Go handlers, and ICD. §3 (cleanup) resolved. §4.1 (chat_id alias) resolved. §4.2 (legacy list messages) open. ICD updated to target state including multi-participant channels.
---

19
scripts/db-validate.sh
View File

@@ -189,7 +189,7 @@ check_table "usage_log"
check_table "model_pricing"
check_table "extensions"
check_table "extension_user_settings"
check_table "attachments"
check_table "files"
check_table "knowledge_bases"
check_table "kb_documents"
check_table "kb_chunks"
@@ -296,14 +296,17 @@ check_column "extensions" "manifest"
check_column "extensions" "is_system"
check_column "extensions" "scope"
# ── Attachments ──────────────────────────
# ── Files ──────────────────────────
echo ""
echo "Attachments:"
check_column "attachments" "channel_id"
check_column "attachments" "storage_key"
check_column "attachments" "extracted_text"
check_index "idx_attachments_channel"
check_index "idx_attachments_orphan"
echo "Files:"
check_column "files" "channel_id"
check_column "files" "storage_key"
check_column "files" "extracted_text"
check_column "files" "origin"
check_column "files" "display_hint"
check_column "files" "updated_at"
check_index "idx_files_channel"
check_index "idx_files_orphan"
# ── Knowledge Bases ──────────────────────
echo ""

133
server/database/migrations/001_core.sql Normal file
View File

@@ -0,0 +1,133 @@
-- ==========================================
-- Chat Switchboard — 001 Core
-- ==========================================
-- Users, authentication, platform policies, and global settings.
-- Foundation tables with no external FK dependencies.
--
-- ICD §2 (Auth), §14 (User Profile & Settings), §16 (Platform Admin)
-- v0.22.8: Domain-grouped migration (replaces monolith 001 + incrementals)
-- ==========================================
-- ── Extensions ──────────────────────────────
CREATE EXTENSION IF NOT EXISTS "pgcrypto"; -- gen_random_uuid()
CREATE EXTENSION IF NOT EXISTS "vector"; -- pgvector (KB embeddings)
-- ── Utility: auto-update updated_at ─────────
CREATE OR REPLACE FUNCTION update_updated_at()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- =========================================
-- USERS
-- =========================================
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
username VARCHAR(50) NOT NULL,
email VARCHAR(255) NOT NULL,
password_hash TEXT NOT NULL,
display_name VARCHAR(100),
avatar_url TEXT,
role VARCHAR(20) DEFAULT 'user'
CHECK (role IN ('user', 'admin')),
is_active BOOLEAN DEFAULT true,
settings JSONB DEFAULT '{}'::jsonb,
-- Vault: per-user encryption key (UEK) for BYOK API keys
encrypted_uek BYTEA,
uek_salt BYTEA,
uek_nonce BYTEA,
vault_set BOOLEAN NOT NULL DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
last_login_at TIMESTAMPTZ
);
-- Case-insensitive unique indexes (no bare UNIQUE constraint)
CREATE UNIQUE INDEX IF NOT EXISTS users_username_ci ON users (LOWER(username));
CREATE UNIQUE INDEX IF NOT EXISTS users_email_ci ON users (LOWER(email));
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
DROP TRIGGER IF EXISTS users_updated_at ON users;
CREATE TRIGGER users_updated_at BEFORE UPDATE ON users
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN users.encrypted_uek IS 'UEK encrypted with Argon2id(password)-derived key';
COMMENT ON COLUMN users.uek_salt IS 'Argon2id salt for UEK derivation';
COMMENT ON COLUMN users.uek_nonce IS 'AES-GCM nonce for UEK wrapping';
COMMENT ON COLUMN users.vault_set IS 'true once UEK has been generated and wrapped';
-- =========================================
-- AUTH — REFRESH TOKENS
-- =========================================
CREATE TABLE IF NOT EXISTS refresh_tokens (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
expires_at TIMESTAMPTZ NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW(),
revoked_at TIMESTAMPTZ
);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_user ON refresh_tokens(user_id);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_hash ON refresh_tokens(token_hash) WHERE revoked_at IS NULL;
-- =========================================
-- PLATFORM POLICIES
-- =========================================
CREATE TABLE IF NOT EXISTS platform_policies (
key VARCHAR(50) PRIMARY KEY,
value TEXT NOT NULL,
updated_by UUID REFERENCES users(id),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Secure by default
INSERT INTO platform_policies (key, value) VALUES
('allow_user_byok', 'false'),
('allow_user_personas', 'false'),
('allow_raw_model_access', 'true'),
('allow_registration', 'true'),
('default_user_active', 'false'),
('allow_team_providers', 'true'),
('kb_direct_access', 'true')
ON CONFLICT (key) DO NOTHING;
COMMENT ON TABLE platform_policies IS 'Global admin switches controlling platform behavior';
-- =========================================
-- GLOBAL SETTINGS
-- =========================================
CREATE TABLE IF NOT EXISTS global_settings (
key VARCHAR(100) PRIMARY KEY,
value JSONB NOT NULL DEFAULT '{}'::jsonb,
updated_at TIMESTAMPTZ DEFAULT NOW(),
updated_by UUID REFERENCES users(id)
);
INSERT INTO global_settings (key, value) VALUES
('registration', '{"enabled": true}'::jsonb),
('site', '{"name": "Chat Switchboard", "tagline": "Multi-Model AI Chat"}'::jsonb),
('banner', '{"enabled": false, "text": "", "bg": "#007a33", "fg": "#ffffff"}'::jsonb),
('model_roles', '{
"utility": { "primary": null, "fallback": null },
"embedding": { "primary": null, "fallback": null },
"generation": { "primary": null, "fallback": null }
}'::jsonb)
ON CONFLICT (key) DO NOTHING;

884
server/database/migrations/001_v016_schema.sql
View File

@@ -1,884 +0,0 @@
-- ==========================================
-- Chat Switchboard — v0.16.0 Consolidated Schema
-- ==========================================
-- Clean-slate schema. Replaces all 001–009 migrations.
-- Drop DB and re-create before applying.
--
-- Changes from v0.15.1 (9-file) schema:
-- • Folded all incremental migrations into base CREATEs
-- • Dropped api_key_plain (vault backfill complete)
-- • Dropped projects, project_channels (unused, v0.19.0 redesigns)
-- • Dropped 'moderator' from users.role CHECK (dead code)
-- • CI usernames from the start (LOWER() unique indexes)
-- • Added: groups, group_members, resource_grants (v0.16.0)
-- • Added: pgvector extension declaration
--
-- Design principles:
-- • Explicit scope enums over nullable column tri-states
-- • Personas as trust boundaries with extensible grants
-- • Groups as pure access-control lists (decouple from teams)
-- • Secure by default (models hidden until admin enables)
-- • Only tables with active handlers — no placeholder tables
-- ==========================================
-- ── Extensions ──────────────────────────────
CREATE EXTENSION IF NOT EXISTS "pgcrypto"; -- gen_random_uuid()
CREATE EXTENSION IF NOT EXISTS "vector"; -- pgvector (KB embeddings)
-- ── Upgrade-path cleanup ───────────────────
-- These objects existed in the 001–009 migration chain but were removed
-- in the v0.16.0 consolidation. Safe no-ops on fresh installs.
DROP TABLE IF EXISTS project_channels CASCADE;
DROP TABLE IF EXISTS projects CASCADE;
DO $$ BEGIN
IF EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'provider_configs' AND column_name = 'api_key_plain'
) THEN
ALTER TABLE provider_configs DROP COLUMN api_key_plain;
END IF;
END $$;
-- ── Utility: auto-update updated_at ─────────
CREATE OR REPLACE FUNCTION update_updated_at()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- =========================================
-- 1. USERS
-- =========================================
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
username VARCHAR(50) NOT NULL,
email VARCHAR(255) NOT NULL,
password_hash TEXT NOT NULL,
display_name VARCHAR(100),
avatar_url TEXT,
role VARCHAR(20) DEFAULT 'user'
CHECK (role IN ('user', 'admin')),
is_active BOOLEAN DEFAULT true,
settings JSONB DEFAULT '{}'::jsonb,
-- Vault: per-user encryption key (UEK) for BYOK API keys
encrypted_uek BYTEA,
uek_salt BYTEA,
uek_nonce BYTEA,
vault_set BOOLEAN NOT NULL DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
last_login_at TIMESTAMPTZ
);
-- Case-insensitive unique indexes (no bare UNIQUE constraint)
CREATE UNIQUE INDEX IF NOT EXISTS users_username_ci ON users (LOWER(username));
CREATE UNIQUE INDEX IF NOT EXISTS users_email_ci ON users (LOWER(email));
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
DROP TRIGGER IF EXISTS users_updated_at ON users;
CREATE TRIGGER users_updated_at BEFORE UPDATE ON users
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN users.encrypted_uek IS 'UEK encrypted with Argon2id(password)-derived key';
COMMENT ON COLUMN users.uek_salt IS 'Argon2id salt for UEK derivation';
COMMENT ON COLUMN users.uek_nonce IS 'AES-GCM nonce for UEK wrapping';
COMMENT ON COLUMN users.vault_set IS 'true once UEK has been generated and wrapped';
-- =========================================
-- 2. AUTH
-- =========================================
CREATE TABLE IF NOT EXISTS refresh_tokens (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
expires_at TIMESTAMPTZ NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW(),
revoked_at TIMESTAMPTZ
);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_user ON refresh_tokens(user_id);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_hash ON refresh_tokens(token_hash) WHERE revoked_at IS NULL;
-- =========================================
-- 3. TEAMS
-- =========================================
CREATE TABLE IF NOT EXISTS teams (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(200) NOT NULL UNIQUE,
description TEXT DEFAULT '',
created_by UUID NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
is_active BOOLEAN DEFAULT true,
settings JSONB DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_teams_active ON teams(is_active) WHERE is_active = true;
DROP TRIGGER IF EXISTS teams_updated_at ON teams;
CREATE TRIGGER teams_updated_at BEFORE UPDATE ON teams
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN teams.settings IS 'Team policies: {"require_private_providers": false}';
CREATE TABLE IF NOT EXISTS team_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
team_id UUID NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role VARCHAR(20) NOT NULL DEFAULT 'member'
CHECK (role IN ('admin', 'member')),
joined_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(team_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_team_members_team ON team_members(team_id);
CREATE INDEX IF NOT EXISTS idx_team_members_user ON team_members(user_id);
-- =========================================
-- 4. GROUPS (v0.16.0)
-- =========================================
-- Pure access-control lists. Decouple resource visibility from team membership.
-- Teams = organizational structure. Roles = vertical permissions.
-- Groups = who can access what (cross-cutting).
CREATE TABLE IF NOT EXISTS groups (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(100) NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope VARCHAR(20) NOT NULL DEFAULT 'global'
CHECK (scope IN ('global', 'team')),
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
created_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
-- Global groups: team_id must be NULL
-- Team groups: team_id must be set
CONSTRAINT groups_scope_team CHECK (
(scope = 'global' AND team_id IS NULL) OR
(scope = 'team' AND team_id IS NOT NULL)
)
);
-- Unique name within scope (global names globally unique,
-- team names unique within team)
CREATE UNIQUE INDEX IF NOT EXISTS idx_groups_name_scope
ON groups(name, COALESCE(team_id, '00000000-0000-0000-0000-000000000000'));
DROP TRIGGER IF EXISTS groups_updated_at ON groups;
CREATE TRIGGER groups_updated_at BEFORE UPDATE ON groups
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE groups IS 'Access-control groups. Decouple resource visibility from team membership.';
COMMENT ON COLUMN groups.scope IS 'global: admin-managed, can span teams. team: team-admin-managed, team-internal.';
CREATE TABLE IF NOT EXISTS group_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
added_by UUID NOT NULL REFERENCES users(id),
added_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(group_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_group_members_group ON group_members(group_id);
CREATE INDEX IF NOT EXISTS idx_group_members_user ON group_members(user_id);
-- =========================================
-- 5. PROVIDER CONFIGS
-- =========================================
-- scope='global': admin-managed, visible to all users (owner_id IS NULL)
-- scope='team': team admin-managed (owner_id = teams.id)
-- scope='personal': user's own keys (owner_id = users.id)
CREATE TABLE IF NOT EXISTS provider_configs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
scope VARCHAR(10) NOT NULL
CHECK (scope IN ('global', 'team', 'personal')),
owner_id UUID,
name VARCHAR(100) NOT NULL,
provider VARCHAR(50) NOT NULL,
endpoint TEXT NOT NULL,
-- Encrypted API key (AES-256-GCM)
api_key_enc BYTEA,
key_nonce BYTEA,
key_scope TEXT NOT NULL DEFAULT 'global',
model_default VARCHAR(100),
config JSONB DEFAULT '{}'::jsonb,
headers JSONB DEFAULT '{}'::jsonb,
settings JSONB DEFAULT '{}'::jsonb,
is_active BOOLEAN DEFAULT true,
is_private BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_provider_configs_scope ON provider_configs(scope);
CREATE INDEX IF NOT EXISTS idx_provider_configs_owner ON provider_configs(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_provider_configs_active ON provider_configs(is_active) WHERE is_active = true;
DROP TRIGGER IF EXISTS provider_configs_updated_at ON provider_configs;
CREATE TRIGGER provider_configs_updated_at BEFORE UPDATE ON provider_configs
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN provider_configs.scope IS 'global=admin-managed, team=team-scoped, personal=user BYOK';
COMMENT ON COLUMN provider_configs.owner_id IS 'NULL for global; teams.id for team scope; users.id for personal scope';
COMMENT ON COLUMN provider_configs.api_key_enc IS 'AES-256-GCM encrypted API key (BYTEA)';
COMMENT ON COLUMN provider_configs.key_nonce IS 'AES-GCM nonce for api_key_enc';
COMMENT ON COLUMN provider_configs.key_scope IS 'Encryption tier: global/team use env key, personal uses UEK';
COMMENT ON COLUMN provider_configs.headers IS 'Custom HTTP headers (e.g. OpenRouter HTTP-Referer)';
COMMENT ON COLUMN provider_configs.settings IS 'Provider-specific params (e.g. Venice web_search, OpenRouter route)';
COMMENT ON COLUMN provider_configs.is_private IS 'Data stays on-prem (local/self-hosted provider)';
-- =========================================
-- 6. MODEL CATALOG
-- =========================================
-- Intrinsic capabilities for models the system knows about.
-- Disabled by default — admin must enable.
CREATE TABLE IF NOT EXISTS model_catalog (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
display_name TEXT,
model_type VARCHAR(20) DEFAULT 'chat',
capabilities JSONB NOT NULL DEFAULT '{}',
pricing JSONB,
visibility VARCHAR(10) DEFAULT 'disabled'
CHECK (visibility IN ('enabled', 'disabled', 'team')),
last_synced_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(provider_config_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_model_catalog_provider ON model_catalog(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_model_catalog_enabled ON model_catalog(visibility) WHERE visibility = 'enabled';
CREATE INDEX IF NOT EXISTS idx_model_catalog_type ON model_catalog(model_type);
DROP TRIGGER IF EXISTS model_catalog_updated_at ON model_catalog;
CREATE TRIGGER model_catalog_updated_at BEFORE UPDATE ON model_catalog
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN model_catalog.visibility IS 'disabled by default — admin must enable for global models';
COMMENT ON COLUMN model_catalog.model_type IS 'chat, embedding, image — from provider API sync';
COMMENT ON COLUMN model_catalog.capabilities IS 'Intrinsic: {"streaming","tool_calling","vision","thinking","reasoning","code_optimized","web_search","max_context","max_output_tokens"}';
-- =========================================
-- 7. PERSONAS
-- =========================================
CREATE TABLE IF NOT EXISTS personas (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(100) NOT NULL,
description TEXT DEFAULT '',
icon VARCHAR(10) DEFAULT '',
avatar TEXT DEFAULT '',
-- Base model binding
base_model_id TEXT NOT NULL,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
-- Behavioral configuration
system_prompt TEXT DEFAULT '',
temperature FLOAT,
max_tokens INT,
thinking_budget INT,
top_p FLOAT,
-- Scope & ownership
scope VARCHAR(10) NOT NULL
CHECK (scope IN ('global', 'team', 'personal')),
owner_id UUID,
created_by UUID NOT NULL REFERENCES users(id),
-- State
is_active BOOLEAN DEFAULT true,
is_shared BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_personas_scope ON personas(scope);
CREATE INDEX IF NOT EXISTS idx_personas_owner ON personas(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_personas_active ON personas(is_active) WHERE is_active = true;
DROP TRIGGER IF EXISTS personas_updated_at ON personas;
CREATE TRIGGER personas_updated_at BEFORE UPDATE ON personas
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN personas.scope IS 'global=all users, team=team members, personal=creator only';
COMMENT ON COLUMN personas.owner_id IS 'NULL for global; teams.id for team scope; users.id for personal';
COMMENT ON COLUMN personas.is_shared IS 'Personal Personas shared with others (read-only)';
-- =========================================
-- 8. PERSONA GRANTS (what a Persona can do)
-- =========================================
-- Controls tools, KBs, and API endpoints that a Persona has access TO.
-- NOT to be confused with resource_grants (who can USE a resource).
CREATE TABLE IF NOT EXISTS persona_grants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
persona_id UUID NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
grant_type VARCHAR(30) NOT NULL,
grant_ref TEXT NOT NULL,
config JSONB DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(persona_id, grant_type, grant_ref)
);
CREATE INDEX IF NOT EXISTS idx_persona_grants_persona ON persona_grants(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_grants_type ON persona_grants(grant_type);
COMMENT ON TABLE persona_grants IS 'What a Persona can do: tools it can call, KBs it can search, etc.';
COMMENT ON COLUMN persona_grants.grant_type IS 'Extensible: tool, knowledge_base, api_endpoint';
COMMENT ON COLUMN persona_grants.grant_ref IS 'tool: function name. knowledge_base: UUID. api_endpoint: identifier.';
COMMENT ON COLUMN persona_grants.config IS 'Type-specific config, e.g. {"read_only": true} for KB grants';
-- =========================================
-- 9. RESOURCE GRANTS (v0.16.0 — who can USE a resource)
-- =========================================
-- Controls which users (via groups) can access a resource.
-- NOT to be confused with persona_grants (what a Persona can do).
--
-- persona_grants: "CodeBot can use calculator" (Persona → tool)
-- resource_grants: "Engineering can use CodeBot" (group → Persona)
CREATE TABLE IF NOT EXISTS resource_grants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
resource_type VARCHAR(30) NOT NULL
CHECK (resource_type IN ('persona', 'knowledge_base')),
resource_id UUID NOT NULL,
grant_scope VARCHAR(20) NOT NULL DEFAULT 'team_only'
CHECK (grant_scope IN ('team_only', 'global', 'groups')),
granted_groups UUID[] NOT NULL DEFAULT '{}',
created_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
-- One grant row per resource
UNIQUE(resource_type, resource_id)
);
CREATE INDEX IF NOT EXISTS idx_resource_grants_resource
ON resource_grants(resource_type, resource_id);
CREATE INDEX IF NOT EXISTS idx_resource_grants_groups
ON resource_grants USING gin(granted_groups);
DROP TRIGGER IF EXISTS resource_grants_updated_at ON resource_grants;
CREATE TRIGGER resource_grants_updated_at BEFORE UPDATE ON resource_grants
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE resource_grants IS 'Who can USE a resource (personas, KBs). Cross-team access via groups.';
COMMENT ON COLUMN resource_grants.grant_scope IS 'team_only: existing team behavior. global: all users. groups: specific group list.';
COMMENT ON COLUMN resource_grants.granted_groups IS 'UUID array of group IDs. Only meaningful when grant_scope = groups.';
-- =========================================
-- 10. PLATFORM POLICIES
-- =========================================
CREATE TABLE IF NOT EXISTS platform_policies (
key VARCHAR(50) PRIMARY KEY,
value TEXT NOT NULL,
updated_by UUID REFERENCES users(id),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Secure by default
INSERT INTO platform_policies (key, value) VALUES
('allow_user_byok', 'false'),
('allow_user_personas', 'false'),
('allow_raw_model_access', 'true'),
('allow_registration', 'true'),
('default_user_active', 'false'),
('allow_team_providers', 'true')
ON CONFLICT (key) DO NOTHING;
COMMENT ON TABLE platform_policies IS 'Global admin switches controlling platform behavior';
-- =========================================
-- 11. GLOBAL SETTINGS (non-policy config)
-- =========================================
CREATE TABLE IF NOT EXISTS global_settings (
key VARCHAR(100) PRIMARY KEY,
value JSONB NOT NULL DEFAULT '{}'::jsonb,
updated_at TIMESTAMPTZ DEFAULT NOW(),
updated_by UUID REFERENCES users(id)
);
INSERT INTO global_settings (key, value) VALUES
('registration', '{"enabled": true}'::jsonb),
('site', '{"name": "Chat Switchboard", "tagline": "Multi-Model AI Chat"}'::jsonb),
('banner', '{
"enabled": false,
"text": "",
"position": "both",
"bg": "#007a33",
"fg": "#ffffff"
}'::jsonb),
('banner_presets', '{
"development": { "text": "DEVELOPMENT", "bg": "#007a33", "fg": "#ffffff" },
"testing": { "text": "TESTING", "bg": "#502b85", "fg": "#ffffff" },
"staging": { "text": "STAGING", "bg": "#0033a0", "fg": "#ffffff" },
"production": { "text": "PRODUCTION", "bg": "#c8102e", "fg": "#ffffff" },
"training": { "text": "TRAINING", "bg": "#ff8c00", "fg": "#000000" },
"demo": { "text": "DEMO", "bg": "#fce83a", "fg": "#000000" }
}'::jsonb),
('model_roles', '{
"utility": { "primary": null, "fallback": null },
"embedding": { "primary": null, "fallback": null },
"generation": { "primary": null, "fallback": null }
}'::jsonb)
ON CONFLICT (key) DO NOTHING;
-- =========================================
-- 12. USER MODEL SETTINGS
-- =========================================
CREATE TABLE IF NOT EXISTS user_model_settings (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
hidden BOOLEAN DEFAULT false,
preferred_temperature FLOAT,
preferred_max_tokens INT,
sort_order INT DEFAULT 0,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(user_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_user_model_settings_user ON user_model_settings(user_id);
DROP TRIGGER IF EXISTS user_model_settings_updated_at ON user_model_settings;
CREATE TRIGGER user_model_settings_updated_at BEFORE UPDATE ON user_model_settings
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
-- =========================================
-- 13. CHANNELS
-- =========================================
CREATE TABLE IF NOT EXISTS channels (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
title VARCHAR(500) NOT NULL,
description TEXT,
type VARCHAR(20) DEFAULT 'direct'
CHECK (type IN ('direct', 'group', 'channel')),
model VARCHAR(100),
system_prompt TEXT,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
is_archived BOOLEAN DEFAULT false,
is_pinned BOOLEAN DEFAULT false,
folder_id UUID, -- FK added after folders table created
folder TEXT, -- simple text folder name (frontend-managed)
team_id UUID REFERENCES teams(id) ON DELETE SET NULL,
settings JSONB DEFAULT '{}'::jsonb,
tags TEXT[],
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_channels_user ON channels(user_id);
CREATE INDEX IF NOT EXISTS idx_channels_updated ON channels(updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_channels_type ON channels(type);
CREATE INDEX IF NOT EXISTS idx_channels_team ON channels(team_id) WHERE team_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_channels_tags ON channels USING GIN(tags);
DROP TRIGGER IF EXISTS channels_updated_at ON channels;
CREATE TRIGGER channels_updated_at BEFORE UPDATE ON channels
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN channels.type IS 'direct=1:1 AI chat, group=multi-model, channel=named persistent';
-- =========================================
-- 14. MESSAGES (with tree/forking support)
-- =========================================
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
role VARCHAR(20) NOT NULL
CHECK (role IN ('user', 'assistant', 'system', 'tool')),
content TEXT NOT NULL,
model VARCHAR(100),
tokens_used INTEGER,
tool_calls JSONB,
metadata JSONB DEFAULT '{}'::jsonb,
parent_id UUID REFERENCES messages(id) ON DELETE SET NULL,
sibling_index INTEGER DEFAULT 0,
participant_type VARCHAR(10) DEFAULT 'user',
participant_id VARCHAR(255),
deleted_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_messages_channel ON messages(channel_id, created_at);
CREATE INDEX IF NOT EXISTS idx_messages_parent ON messages(parent_id);
CREATE INDEX IF NOT EXISTS idx_messages_alive ON messages(channel_id, created_at)
WHERE deleted_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_messages_parent_alive ON messages(parent_id, sibling_index)
WHERE deleted_at IS NULL;
COMMENT ON COLUMN messages.parent_id IS 'Tree parent for conversation forking';
COMMENT ON COLUMN messages.sibling_index IS 'Position among siblings (0-indexed)';
COMMENT ON COLUMN messages.participant_type IS 'user or model';
COMMENT ON COLUMN messages.participant_id IS 'user UUID or model identifier string';
-- =========================================
-- 15. CHANNEL MEMBERS & MODELS
-- =========================================
CREATE TABLE IF NOT EXISTS channel_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role VARCHAR(20) DEFAULT 'member',
joined_at TIMESTAMPTZ DEFAULT NOW(),
last_read_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_members_channel ON channel_members(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_members_user ON channel_members(user_id);
CREATE TABLE IF NOT EXISTS channel_models (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
model_id VARCHAR(255) NOT NULL,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
display_name VARCHAR(100),
system_prompt TEXT,
settings JSONB DEFAULT '{}'::jsonb,
is_default BOOLEAN DEFAULT false,
added_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(channel_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_models_channel ON channel_models(channel_id);
-- =========================================
-- 16. CHANNEL CURSORS (forking navigation)
-- =========================================
CREATE TABLE IF NOT EXISTS channel_cursors (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
active_leaf_id UUID REFERENCES messages(id) ON DELETE SET NULL,
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_channel ON channel_cursors(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_user ON channel_cursors(user_id);
-- =========================================
-- 17. FOLDERS
-- =========================================
-- Channel folders. Note: projects table deferred to v0.19.0.
CREATE TABLE IF NOT EXISTS folders (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name VARCHAR(200) NOT NULL,
parent_id UUID REFERENCES folders(id) ON DELETE CASCADE,
sort_order INT DEFAULT 0,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(user_id, name, parent_id)
);
CREATE INDEX IF NOT EXISTS idx_folders_user ON folders(user_id);
CREATE INDEX IF NOT EXISTS idx_folders_parent ON folders(parent_id);
DROP TRIGGER IF EXISTS folders_updated_at ON folders;
CREATE TRIGGER folders_updated_at BEFORE UPDATE ON folders
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
-- Deferred FK: channels.folder_id → folders.id
DO $$ BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint WHERE conname = 'fk_channels_folder'
) THEN
ALTER TABLE channels ADD CONSTRAINT fk_channels_folder
FOREIGN KEY (folder_id) REFERENCES folders(id) ON DELETE SET NULL;
END IF;
END $$;
CREATE INDEX IF NOT EXISTS idx_channels_folder ON channels(folder_id) WHERE folder_id IS NOT NULL;
-- =========================================
-- 18. NOTES
-- =========================================
CREATE TABLE IF NOT EXISTS notes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
title VARCHAR(500) NOT NULL,
content TEXT DEFAULT '',
folder_path TEXT DEFAULT '/',
tags TEXT[] DEFAULT '{}',
metadata JSONB DEFAULT '{}',
source_channel_id UUID REFERENCES channels(id) ON DELETE SET NULL,
team_id UUID REFERENCES teams(id) ON DELETE SET NULL,
search_vector TSVECTOR,
embedding VECTOR(3072),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_notes_user ON notes(user_id);
CREATE INDEX IF NOT EXISTS idx_notes_search ON notes USING GIN(search_vector);
CREATE INDEX IF NOT EXISTS idx_notes_folder ON notes(user_id, folder_path);
CREATE INDEX IF NOT EXISTS idx_notes_tags ON notes USING GIN(tags);
CREATE INDEX IF NOT EXISTS idx_notes_updated ON notes(user_id, updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_notes_team ON notes(team_id) WHERE team_id IS NOT NULL;
CREATE OR REPLACE FUNCTION notes_search_update_fn()
RETURNS TRIGGER AS $$
BEGIN
NEW.search_vector :=
setweight(to_tsvector('pg_catalog.english', COALESCE(NEW.title, '')), 'A') ||
setweight(to_tsvector('pg_catalog.english', COALESCE(NEW.content, '')), 'B');
NEW.updated_at := NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS notes_search_update ON notes;
CREATE TRIGGER notes_search_update
BEFORE INSERT OR UPDATE OF title, content ON notes
FOR EACH ROW EXECUTE FUNCTION notes_search_update_fn();
-- =========================================
-- 19. AUDIT LOG
-- =========================================
CREATE TABLE IF NOT EXISTS audit_log (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
actor_id UUID REFERENCES users(id) ON DELETE SET NULL,
action VARCHAR(100) NOT NULL,
resource_type VARCHAR(50) NOT NULL,
resource_id VARCHAR(255),
metadata JSONB DEFAULT '{}'::jsonb,
ip_address VARCHAR(45),
user_agent TEXT DEFAULT '',
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_audit_log_created ON audit_log(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_log_actor ON audit_log(actor_id) WHERE actor_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_audit_log_resource ON audit_log(resource_type, resource_id);
CREATE INDEX IF NOT EXISTS idx_audit_log_action ON audit_log(action);
COMMENT ON TABLE audit_log IS 'Immutable audit trail of all mutating operations';
-- =========================================
-- 20. USAGE TRACKING
-- =========================================
CREATE TABLE IF NOT EXISTS usage_log (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID REFERENCES channels(id) ON DELETE SET NULL,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
provider_scope TEXT NOT NULL DEFAULT 'global',
model_id TEXT NOT NULL,
role TEXT, -- NULL = user chat, 'utility', 'embedding', 'generation'
prompt_tokens INT NOT NULL DEFAULT 0,
completion_tokens INT NOT NULL DEFAULT 0,
cache_creation_tokens INT NOT NULL DEFAULT 0,
cache_read_tokens INT NOT NULL DEFAULT 0,
cost_input NUMERIC(12,6),
cost_output NUMERIC(12,6),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_usage_log_user ON usage_log(user_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_created ON usage_log(created_at);
CREATE INDEX IF NOT EXISTS idx_usage_log_provider ON usage_log(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_model ON usage_log(model_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_scope ON usage_log(provider_scope);
-- =========================================
-- 21. MODEL PRICING
-- =========================================
CREATE TABLE IF NOT EXISTS model_pricing (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
input_per_m NUMERIC(10,6),
output_per_m NUMERIC(10,6),
cache_create_per_m NUMERIC(10,6),
cache_read_per_m NUMERIC(10,6),
currency TEXT NOT NULL DEFAULT 'USD',
source TEXT NOT NULL DEFAULT 'manual',
updated_at TIMESTAMPTZ DEFAULT NOW(),
updated_by UUID REFERENCES users(id),
UNIQUE(provider_config_id, model_id)
);
-- =========================================
-- 22. EXTENSIONS
-- =========================================
CREATE TABLE IF NOT EXISTS extensions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
ext_id VARCHAR(100) NOT NULL UNIQUE,
name VARCHAR(200) NOT NULL,
version VARCHAR(50) NOT NULL DEFAULT '0.0.0',
tier VARCHAR(20) NOT NULL DEFAULT 'browser',
description TEXT NOT NULL DEFAULT '',
author VARCHAR(200) NOT NULL DEFAULT '',
manifest JSONB NOT NULL DEFAULT '{}',
is_system BOOLEAN NOT NULL DEFAULT false,
is_enabled BOOLEAN NOT NULL DEFAULT true,
scope VARCHAR(20) NOT NULL DEFAULT 'global',
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
installed_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_extensions_tier ON extensions(tier);
CREATE INDEX IF NOT EXISTS idx_extensions_enabled ON extensions(is_enabled) WHERE is_enabled = true;
CREATE TABLE IF NOT EXISTS extension_user_settings (
extension_id UUID NOT NULL REFERENCES extensions(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
settings JSONB NOT NULL DEFAULT '{}',
is_enabled BOOLEAN NOT NULL DEFAULT true,
PRIMARY KEY (extension_id, user_id)
);
-- =========================================
-- 23. ATTACHMENTS
-- =========================================
CREATE TABLE IF NOT EXISTS attachments (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id),
message_id UUID REFERENCES messages(id) ON DELETE SET NULL,
filename VARCHAR(255) NOT NULL,
content_type VARCHAR(127) NOT NULL,
size_bytes BIGINT NOT NULL,
storage_key TEXT NOT NULL,
extracted_text TEXT,
metadata JSONB DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_attachments_channel ON attachments(channel_id);
CREATE INDEX IF NOT EXISTS idx_attachments_user_size ON attachments(user_id);
CREATE INDEX IF NOT EXISTS idx_attachments_message ON attachments(message_id) WHERE message_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_attachments_orphan ON attachments(created_at) WHERE message_id IS NULL;
-- =========================================
-- 24. KNOWLEDGE BASES
-- =========================================
CREATE TABLE IF NOT EXISTS knowledge_bases (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope TEXT NOT NULL DEFAULT 'global',
owner_id UUID REFERENCES users(id) ON DELETE CASCADE,
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
embedding_config JSONB NOT NULL DEFAULT '{}',
document_count INT NOT NULL DEFAULT 0,
chunk_count INT NOT NULL DEFAULT 0,
total_bytes BIGINT NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'active',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT kb_scope_check CHECK (
(scope = 'global' AND owner_id IS NULL) OR
(scope = 'team' AND team_id IS NOT NULL) OR
(scope = 'personal' AND owner_id IS NOT NULL)
)
);
CREATE INDEX IF NOT EXISTS idx_kb_scope ON knowledge_bases(scope);
CREATE INDEX IF NOT EXISTS idx_kb_owner ON knowledge_bases(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_kb_team ON knowledge_bases(team_id) WHERE team_id IS NOT NULL;
CREATE TABLE IF NOT EXISTS kb_documents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
filename TEXT NOT NULL,
content_type TEXT NOT NULL,
size_bytes BIGINT NOT NULL,
storage_key TEXT NOT NULL,
extracted_text TEXT,
chunk_count INT NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'pending',
error TEXT,
uploaded_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_kbdoc_kb ON kb_documents(kb_id);
CREATE TABLE IF NOT EXISTS kb_chunks (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
document_id UUID NOT NULL REFERENCES kb_documents(id) ON DELETE CASCADE,
chunk_index INT NOT NULL,
content TEXT NOT NULL,
token_count INT NOT NULL DEFAULT 0,
embedding VECTOR(3072),
metadata JSONB NOT NULL DEFAULT '{}',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_kbchunk_kb ON kb_chunks(kb_id);
CREATE INDEX IF NOT EXISTS idx_kbchunk_doc ON kb_chunks(document_id);
-- NOTE: IVFFlat index for similarity search created after first data load
-- (needs rows to train). The ingest handler creates it.
CREATE TABLE IF NOT EXISTS channel_knowledge_bases (
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
enabled BOOLEAN NOT NULL DEFAULT true,
added_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (channel_id, kb_id)
);

86
server/database/migrations/002_teams.sql Normal file
View File

@@ -0,0 +1,86 @@
-- ==========================================
-- Chat Switchboard — 002 Teams & Access Control
-- ==========================================
-- Teams, team membership, groups (ACLs), and group membership.
-- ICD §15 (Teams & Access Control)
-- ==========================================
-- =========================================
-- TEAMS
-- =========================================
CREATE TABLE IF NOT EXISTS teams (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(200) NOT NULL UNIQUE,
description TEXT DEFAULT '',
created_by UUID NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
is_active BOOLEAN DEFAULT true,
settings JSONB DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_teams_active ON teams(is_active) WHERE is_active = true;
DROP TRIGGER IF EXISTS teams_updated_at ON teams;
CREATE TRIGGER teams_updated_at BEFORE UPDATE ON teams
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN teams.settings IS 'Team policies: {"require_private_providers": false}';
CREATE TABLE IF NOT EXISTS team_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
team_id UUID NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role VARCHAR(20) NOT NULL DEFAULT 'member'
CHECK (role IN ('admin', 'member')),
joined_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(team_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_team_members_team ON team_members(team_id);
CREATE INDEX IF NOT EXISTS idx_team_members_user ON team_members(user_id);
-- =========================================
-- GROUPS (ACLs)
-- =========================================
-- Pure access-control lists. Decouple resource visibility from team membership.
CREATE TABLE IF NOT EXISTS groups (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(100) NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope VARCHAR(20) NOT NULL DEFAULT 'global'
CHECK (scope IN ('global', 'team')),
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
created_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
CONSTRAINT groups_scope_team CHECK (
(scope = 'global' AND team_id IS NULL) OR
(scope = 'team' AND team_id IS NOT NULL)
)
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_groups_name_scope
ON groups(name, COALESCE(team_id, '00000000-0000-0000-0000-000000000000'));
DROP TRIGGER IF EXISTS groups_updated_at ON groups;
CREATE TRIGGER groups_updated_at BEFORE UPDATE ON groups
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE groups IS 'Access-control groups. Decouple resource visibility from team membership.';
COMMENT ON COLUMN groups.scope IS 'global: admin-managed, can span teams. team: team-admin-managed, team-internal.';
CREATE TABLE IF NOT EXISTS group_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
group_id UUID NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
added_by UUID NOT NULL REFERENCES users(id),
added_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(group_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_group_members_group ON group_members(group_id);
CREATE INDEX IF NOT EXISTS idx_group_members_user ON group_members(user_id);

42
server/database/migrations/002_v017_persona_kb.sql
View File

@@ -1,42 +0,0 @@
-- v0.17.0: Persona-KB Binding + Enterprise KB Mode
--
-- New: persona_knowledge_bases join table
-- New: knowledge_bases.discoverable column
-- New: kb_direct_access platform policy
-- =========================================
-- 1. Persona-KB Binding
-- =========================================
CREATE TABLE IF NOT EXISTS persona_knowledge_bases (
persona_id UUID NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
auto_search BOOLEAN NOT NULL DEFAULT false,
added_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (persona_id, kb_id)
);
CREATE INDEX IF NOT EXISTS idx_persona_kb_persona ON persona_knowledge_bases(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_kb_kb ON persona_knowledge_bases(kb_id);
COMMENT ON TABLE persona_knowledge_bases IS 'Binds KBs to Personas — the Persona becomes a gateway to its KBs';
COMMENT ON COLUMN persona_knowledge_bases.auto_search IS 'true = auto-prepend top-K results to context; false = kb_search tool only';
-- =========================================
-- 2. Enterprise KB Mode
-- =========================================
ALTER TABLE knowledge_bases
ADD COLUMN IF NOT EXISTS discoverable BOOLEAN NOT NULL DEFAULT true;
COMMENT ON COLUMN knowledge_bases.discoverable IS 'false = hidden from user KB lists, only accessible through Persona binding';
-- =========================================
-- 3. Platform Policy
-- =========================================
INSERT INTO platform_policies (key, value) VALUES
('kb_direct_access', 'true')
ON CONFLICT (key) DO NOTHING;
COMMENT ON COLUMN platform_policies.key IS 'kb_direct_access: when false, disables channel KB popup (strict enterprise mode)';

177
server/database/migrations/003_providers.sql Normal file
View File

@@ -0,0 +1,177 @@
-- ==========================================
-- Chat Switchboard — 003 Providers & Routing
-- ==========================================
-- Provider configs, model catalog, pricing, health, capability
-- overrides, and routing policies.
-- ICD §10 (Providers & Routing), §11 (Models & Preferences)
-- ==========================================
-- =========================================
-- PROVIDER CONFIGS
-- =========================================
CREATE TABLE IF NOT EXISTS provider_configs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
scope VARCHAR(10) NOT NULL
CHECK (scope IN ('global', 'team', 'personal')),
owner_id UUID,
name VARCHAR(100) NOT NULL,
provider VARCHAR(50) NOT NULL,
endpoint TEXT NOT NULL,
-- Encrypted API key (AES-256-GCM)
api_key_enc BYTEA,
key_nonce BYTEA,
key_scope TEXT NOT NULL DEFAULT 'global',
model_default VARCHAR(100),
config JSONB DEFAULT '{}'::jsonb,
headers JSONB DEFAULT '{}'::jsonb,
settings JSONB DEFAULT '{}'::jsonb,
is_active BOOLEAN DEFAULT true,
is_private BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_provider_configs_scope ON provider_configs(scope);
CREATE INDEX IF NOT EXISTS idx_provider_configs_owner ON provider_configs(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_provider_configs_active ON provider_configs(is_active) WHERE is_active = true;
DROP TRIGGER IF EXISTS provider_configs_updated_at ON provider_configs;
CREATE TRIGGER provider_configs_updated_at BEFORE UPDATE ON provider_configs
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN provider_configs.scope IS 'global=admin-managed, team=team-scoped, personal=user BYOK';
COMMENT ON COLUMN provider_configs.owner_id IS 'NULL for global; teams.id for team scope; users.id for personal scope';
COMMENT ON COLUMN provider_configs.api_key_enc IS 'AES-256-GCM encrypted API key (BYTEA)';
COMMENT ON COLUMN provider_configs.key_nonce IS 'AES-GCM nonce for api_key_enc';
COMMENT ON COLUMN provider_configs.key_scope IS 'Encryption tier: global/team use env key, personal uses UEK';
COMMENT ON COLUMN provider_configs.headers IS 'Custom HTTP headers (e.g. OpenRouter HTTP-Referer)';
COMMENT ON COLUMN provider_configs.settings IS 'Provider-specific params (e.g. Venice web_search, OpenRouter route)';
COMMENT ON COLUMN provider_configs.is_private IS 'Data stays on-prem (local/self-hosted provider)';
-- =========================================
-- MODEL CATALOG
-- =========================================
CREATE TABLE IF NOT EXISTS model_catalog (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
display_name TEXT,
model_type VARCHAR(20) DEFAULT 'chat',
capabilities JSONB NOT NULL DEFAULT '{}',
pricing JSONB,
visibility VARCHAR(10) DEFAULT 'disabled'
CHECK (visibility IN ('enabled', 'disabled', 'team')),
last_synced_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(provider_config_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_model_catalog_provider ON model_catalog(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_model_catalog_enabled ON model_catalog(visibility) WHERE visibility = 'enabled';
CREATE INDEX IF NOT EXISTS idx_model_catalog_type ON model_catalog(model_type);
DROP TRIGGER IF EXISTS model_catalog_updated_at ON model_catalog;
CREATE TRIGGER model_catalog_updated_at BEFORE UPDATE ON model_catalog
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN model_catalog.visibility IS 'disabled by default — admin must enable for global models';
COMMENT ON COLUMN model_catalog.model_type IS 'chat, embedding, image — from provider API sync';
COMMENT ON COLUMN model_catalog.capabilities IS 'Intrinsic: {"streaming","tool_calling","vision","thinking","reasoning","code_optimized","web_search","max_context","max_output_tokens"}';
-- =========================================
-- MODEL PRICING
-- =========================================
CREATE TABLE IF NOT EXISTS model_pricing (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
input_per_m NUMERIC(10,6),
output_per_m NUMERIC(10,6),
cache_create_per_m NUMERIC(10,6),
cache_read_per_m NUMERIC(10,6),
currency TEXT NOT NULL DEFAULT 'USD',
source TEXT NOT NULL DEFAULT 'manual',
updated_at TIMESTAMPTZ DEFAULT NOW(),
updated_by UUID REFERENCES users(id),
UNIQUE(provider_config_id, model_id)
);
-- =========================================
-- PROVIDER HEALTH (v0.22.0)
-- =========================================
CREATE TABLE IF NOT EXISTS provider_health (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
window_start TIMESTAMPTZ NOT NULL,
request_count INT NOT NULL DEFAULT 0,
error_count INT NOT NULL DEFAULT 0,
timeout_count INT NOT NULL DEFAULT 0,
total_latency_ms BIGINT NOT NULL DEFAULT 0,
max_latency_ms INT NOT NULL DEFAULT 0,
rate_limit_count INT NOT NULL DEFAULT 0,
last_error TEXT,
last_error_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (provider_config_id, window_start)
);
CREATE INDEX IF NOT EXISTS idx_provider_health_window
ON provider_health (provider_config_id, window_start DESC);
-- =========================================
-- CAPABILITY OVERRIDES (v0.22.0)
-- =========================================
CREATE TABLE IF NOT EXISTS capability_overrides (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
field TEXT NOT NULL,
value TEXT NOT NULL,
set_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (provider_config_id, model_id, field)
);
CREATE INDEX IF NOT EXISTS idx_capability_overrides_model
ON capability_overrides (model_id);
-- =========================================
-- ROUTING POLICIES (v0.22.2)
-- =========================================
CREATE TABLE IF NOT EXISTS routing_policies (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(200) NOT NULL,
scope VARCHAR(10) NOT NULL DEFAULT 'global'
CHECK (scope IN ('global', 'team')),
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
priority INTEGER NOT NULL DEFAULT 100,
policy_type VARCHAR(30) NOT NULL
CHECK (policy_type IN ('provider_prefer', 'team_route', 'cost_limit', 'model_alias')),
config JSONB NOT NULL DEFAULT '{}'::jsonb,
is_active BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_routing_policies_active
ON routing_policies(is_active, priority) WHERE is_active = true;
CREATE INDEX IF NOT EXISTS idx_routing_policies_team
ON routing_policies(team_id) WHERE team_id IS NOT NULL;
DROP TRIGGER IF EXISTS routing_policies_updated_at ON routing_policies;
CREATE TRIGGER routing_policies_updated_at BEFORE UPDATE ON routing_policies
FOR EACH ROW EXECUTE FUNCTION update_updated_at();

35
server/database/migrations/003_v0173_note_links.sql
View File

@@ -1,35 +0,0 @@
-- v0.17.3: Note Links + Wikilink Infrastructure
--
-- New: note_links table for [[wikilink]] graph edges
-- New: notes.source_message_id for chat-to-note provenance
-- =========================================
-- 1. Note Links
-- =========================================
CREATE TABLE IF NOT EXISTS note_links (
source_note_id UUID NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
target_note_id UUID REFERENCES notes(id) ON DELETE CASCADE,
target_title TEXT NOT NULL,
display_text TEXT,
is_transclusion BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (source_note_id, target_title)
);
CREATE INDEX IF NOT EXISTS idx_note_links_target ON note_links(target_note_id)
WHERE target_note_id IS NOT NULL;
COMMENT ON TABLE note_links IS 'Directed links between notes, extracted from [[wikilink]] syntax on save';
COMMENT ON COLUMN note_links.target_note_id IS 'NULL for unresolved links (target note does not exist yet)';
COMMENT ON COLUMN note_links.target_title IS 'Raw [[title]] text — preserved for re-resolution after renames';
COMMENT ON COLUMN note_links.is_transclusion IS 'true for ![[embed]] syntax, false for regular [[link]]';
-- =========================================
-- 2. Source Message Provenance
-- =========================================
ALTER TABLE notes ADD COLUMN IF NOT EXISTS source_message_id UUID;
CREATE INDEX IF NOT EXISTS idx_notes_source_msg ON notes(source_message_id)
WHERE source_message_id IS NOT NULL;

113
server/database/migrations/004_personas.sql Normal file
View File

@@ -0,0 +1,113 @@
-- ==========================================
-- Chat Switchboard — 004 Personas & Grants
-- ==========================================
-- Persona definitions, persona capability grants, and resource access grants.
-- ICD §4 (Personas)
-- ==========================================
-- =========================================
-- PERSONAS
-- =========================================
CREATE TABLE IF NOT EXISTS personas (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(100) NOT NULL,
description TEXT DEFAULT '',
icon VARCHAR(10) DEFAULT '',
avatar TEXT DEFAULT '',
-- Base model binding
base_model_id TEXT NOT NULL,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
-- Behavioral configuration
system_prompt TEXT DEFAULT '',
temperature FLOAT,
max_tokens INT,
thinking_budget INT,
top_p FLOAT,
-- Memory configuration (v0.18.0)
memory_enabled BOOLEAN NOT NULL DEFAULT true,
memory_extraction_prompt TEXT,
-- Scope & ownership
scope VARCHAR(10) NOT NULL
CHECK (scope IN ('global', 'team', 'personal')),
owner_id UUID,
created_by UUID NOT NULL REFERENCES users(id),
-- State
is_active BOOLEAN DEFAULT true,
is_shared BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_personas_scope ON personas(scope);
CREATE INDEX IF NOT EXISTS idx_personas_owner ON personas(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_personas_active ON personas(is_active) WHERE is_active = true;
DROP TRIGGER IF EXISTS personas_updated_at ON personas;
CREATE TRIGGER personas_updated_at BEFORE UPDATE ON personas
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON COLUMN personas.scope IS 'global=all users, team=team members, personal=creator only';
COMMENT ON COLUMN personas.owner_id IS 'NULL for global; teams.id for team scope; users.id for personal';
COMMENT ON COLUMN personas.is_shared IS 'Personal personas shared with others (read-only)';
COMMENT ON COLUMN personas.memory_enabled IS 'Whether memory tools and extraction are active for this persona';
COMMENT ON COLUMN personas.memory_extraction_prompt IS 'Custom extraction prompt; NULL = use system default';
-- =========================================
-- PERSONA GRANTS (what a persona can do)
-- =========================================
CREATE TABLE IF NOT EXISTS persona_grants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
persona_id UUID NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
grant_type VARCHAR(30) NOT NULL,
grant_ref TEXT NOT NULL,
config JSONB DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(persona_id, grant_type, grant_ref)
);
CREATE INDEX IF NOT EXISTS idx_persona_grants_persona ON persona_grants(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_grants_type ON persona_grants(grant_type);
COMMENT ON TABLE persona_grants IS 'What a persona can do: tools it can call, KBs it can search, etc.';
COMMENT ON COLUMN persona_grants.grant_type IS 'Extensible: tool, knowledge_base, api_endpoint';
COMMENT ON COLUMN persona_grants.grant_ref IS 'tool: function name. knowledge_base: UUID. api_endpoint: identifier.';
-- =========================================
-- RESOURCE GRANTS (who can USE a resource)
-- =========================================
CREATE TABLE IF NOT EXISTS resource_grants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
resource_type VARCHAR(30) NOT NULL
CHECK (resource_type IN ('persona', 'knowledge_base', 'project')),
resource_id UUID NOT NULL,
grant_scope VARCHAR(20) NOT NULL DEFAULT 'team_only'
CHECK (grant_scope IN ('team_only', 'global', 'groups')),
granted_groups UUID[] NOT NULL DEFAULT '{}',
created_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(resource_type, resource_id)
);
CREATE INDEX IF NOT EXISTS idx_resource_grants_resource
ON resource_grants(resource_type, resource_id);
CREATE INDEX IF NOT EXISTS idx_resource_grants_groups
ON resource_grants USING gin(granted_groups);
DROP TRIGGER IF EXISTS resource_grants_updated_at ON resource_grants;
CREATE TRIGGER resource_grants_updated_at BEFORE UPDATE ON resource_grants
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE resource_grants IS 'Who can USE a resource (personas, KBs, projects). Cross-team access via groups.';
COMMENT ON COLUMN resource_grants.grant_scope IS 'team_only: existing team behavior. global: all users. groups: specific group list.';

85
server/database/migrations/004_v0180_memories.sql
View File

@@ -1,85 +0,0 @@
-- v0.18.0: Memory System
--
-- Long-term memory across conversations, with scope-aware isolation.
-- Three scopes: user (personal), persona (shared), persona_user (per-user-per-persona).
--
-- Unlike KB (static documents) or compaction (within-conversation), this
-- captures facts and preferences that persist across channels.
-- =========================================
-- 1. Memories Table
-- =========================================
CREATE TABLE IF NOT EXISTS memories (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
scope TEXT NOT NULL CHECK (scope IN ('user', 'persona', 'persona_user')),
-- Owner resolution:
-- user scope: owner_id = user_id, user_id = NULL
-- persona scope: owner_id = persona_id, user_id = NULL
-- persona_user scope: owner_id = persona_id, user_id = user_id
owner_id UUID NOT NULL,
user_id UUID,
key TEXT NOT NULL, -- short label ("preferred language", "team size")
value TEXT NOT NULL, -- detail / fact content
source_channel_id UUID, -- which conversation produced this memory
confidence REAL NOT NULL DEFAULT 1.0, -- 0.0–1.0, used for ranking
status TEXT NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'pending_review', 'archived')),
-- Embedding for semantic recall (optional — same pattern as notes)
embedding vector(3072),
created_at TIMESTAMPTZ DEFAULT now(),
updated_at TIMESTAMPTZ DEFAULT now()
);
-- Primary lookup: "give me all active memories for this scope + owner"
CREATE INDEX IF NOT EXISTS idx_memories_scope_owner
ON memories(scope, owner_id, status)
WHERE status = 'active';
-- Persona+user lookup: "give me this persona's memories about this user"
CREATE INDEX IF NOT EXISTS idx_memories_persona_user
ON memories(owner_id, user_id)
WHERE scope = 'persona_user' AND status = 'active';
-- Deduplication: prevent saving the same key twice for the same scope/owner/user
CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_unique_key
ON memories(scope, owner_id, COALESCE(user_id, '00000000-0000-0000-0000-000000000000'), key);
-- Full-text search on key + value
CREATE INDEX IF NOT EXISTS idx_memories_fts
ON memories USING gin(to_tsvector('english', key || ' ' || value));
-- Source channel reference (for provenance / "where did this come from")
CREATE INDEX IF NOT EXISTS idx_memories_source_channel
ON memories(source_channel_id)
WHERE source_channel_id IS NOT NULL;
COMMENT ON TABLE memories IS 'Long-term memory facts persisted across conversations, scoped to user, persona, or user+persona';
COMMENT ON COLUMN memories.scope IS 'user = personal facts; persona = shared persona knowledge; persona_user = per-user within a persona';
COMMENT ON COLUMN memories.owner_id IS 'user_id for user scope, persona_id for persona/persona_user scopes';
COMMENT ON COLUMN memories.user_id IS 'Only set for persona_user scope — identifies which user within the persona';
COMMENT ON COLUMN memories.key IS 'Short label for the memory (e.g. "preferred language", "deployment stack")';
COMMENT ON COLUMN memories.value IS 'The actual fact/detail being remembered';
COMMENT ON COLUMN memories.confidence IS '0.0-1.0 confidence score — higher = more certain, used for ranking and pruning';
COMMENT ON COLUMN memories.status IS 'active = injected into context; pending_review = awaiting human approval; archived = soft-deleted';
-- =========================================
-- 2. Updated-at Trigger
-- =========================================
CREATE OR REPLACE FUNCTION update_memories_updated_at()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = now();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER trg_memories_updated_at
BEFORE UPDATE ON memories
FOR EACH ROW
EXECUTE FUNCTION update_memories_updated_at();

182
server/database/migrations/005_channels.sql Normal file
View File

@@ -0,0 +1,182 @@
-- ==========================================
-- Chat Switchboard — 005 Channels & Conversations
-- ==========================================
-- Channels, messages, members, models, cursors, folders, user prefs.
-- ICD §3 (Channels & Conversations)
-- Note: project_id and workspace_id FKs added by 010_projects.sql
-- ==========================================
-- =========================================
-- FOLDERS
-- =========================================
CREATE TABLE IF NOT EXISTS folders (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name VARCHAR(200) NOT NULL,
parent_id UUID REFERENCES folders(id) ON DELETE CASCADE,
sort_order INT DEFAULT 0,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(user_id, name, parent_id)
);
CREATE INDEX IF NOT EXISTS idx_folders_user ON folders(user_id);
CREATE INDEX IF NOT EXISTS idx_folders_parent ON folders(parent_id);
DROP TRIGGER IF EXISTS folders_updated_at ON folders;
CREATE TRIGGER folders_updated_at BEFORE UPDATE ON folders
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
-- =========================================
-- CHANNELS
-- =========================================
CREATE TABLE IF NOT EXISTS channels (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
title VARCHAR(500) NOT NULL,
description TEXT,
type VARCHAR(20) DEFAULT 'direct'
CHECK (type IN ('direct', 'group', 'channel')),
model VARCHAR(100),
system_prompt TEXT,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
is_archived BOOLEAN DEFAULT false,
is_pinned BOOLEAN DEFAULT false,
folder_id UUID,
folder TEXT,
team_id UUID REFERENCES teams(id) ON DELETE SET NULL,
settings JSONB DEFAULT '{}'::jsonb,
tags TEXT[],
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_channels_user ON channels(user_id);
CREATE INDEX IF NOT EXISTS idx_channels_updated ON channels(updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_channels_type ON channels(type);
CREATE INDEX IF NOT EXISTS idx_channels_team ON channels(team_id) WHERE team_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_channels_tags ON channels USING GIN(tags);
DROP TRIGGER IF EXISTS channels_updated_at ON channels;
CREATE TRIGGER channels_updated_at BEFORE UPDATE ON channels
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
-- Deferred FK: channels.folder_id → folders.id
DO $$ BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint WHERE conname = 'fk_channels_folder'
) THEN
ALTER TABLE channels ADD CONSTRAINT fk_channels_folder
FOREIGN KEY (folder_id) REFERENCES folders(id) ON DELETE SET NULL;
END IF;
END $$;
CREATE INDEX IF NOT EXISTS idx_channels_folder ON channels(folder_id) WHERE folder_id IS NOT NULL;
COMMENT ON COLUMN channels.type IS 'direct=1:1 AI chat, group=multi-model, channel=named persistent';
-- =========================================
-- MESSAGES
-- =========================================
CREATE TABLE IF NOT EXISTS messages (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
role VARCHAR(20) NOT NULL
CHECK (role IN ('user', 'assistant', 'system', 'tool')),
content TEXT NOT NULL,
model VARCHAR(100),
tokens_used INTEGER,
tool_calls JSONB,
metadata JSONB DEFAULT '{}'::jsonb,
parent_id UUID REFERENCES messages(id) ON DELETE SET NULL,
sibling_index INTEGER DEFAULT 0,
participant_type VARCHAR(10) DEFAULT 'user',
participant_id VARCHAR(255),
deleted_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_messages_channel ON messages(channel_id, created_at);
CREATE INDEX IF NOT EXISTS idx_messages_parent ON messages(parent_id);
CREATE INDEX IF NOT EXISTS idx_messages_alive ON messages(channel_id, created_at)
WHERE deleted_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_messages_parent_alive ON messages(parent_id, sibling_index)
WHERE deleted_at IS NULL;
COMMENT ON COLUMN messages.parent_id IS 'Tree parent for conversation forking';
COMMENT ON COLUMN messages.sibling_index IS 'Position among siblings (0-indexed)';
-- =========================================
-- CHANNEL MEMBERS & MODELS
-- =========================================
CREATE TABLE IF NOT EXISTS channel_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role VARCHAR(20) DEFAULT 'member',
joined_at TIMESTAMPTZ DEFAULT NOW(),
last_read_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_members_channel ON channel_members(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_members_user ON channel_members(user_id);
CREATE TABLE IF NOT EXISTS channel_models (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
model_id VARCHAR(255) NOT NULL,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
display_name VARCHAR(100),
system_prompt TEXT,
settings JSONB DEFAULT '{}'::jsonb,
is_default BOOLEAN DEFAULT false,
added_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(channel_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_models_channel ON channel_models(channel_id);
-- =========================================
-- CHANNEL CURSORS (forking navigation)
-- =========================================
CREATE TABLE IF NOT EXISTS channel_cursors (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
active_leaf_id UUID REFERENCES messages(id) ON DELETE SET NULL,
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_channel ON channel_cursors(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_user ON channel_cursors(user_id);
-- =========================================
-- USER MODEL SETTINGS
-- =========================================
CREATE TABLE IF NOT EXISTS user_model_settings (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
hidden BOOLEAN DEFAULT false,
preferred_temperature FLOAT,
preferred_max_tokens INT,
sort_order INT DEFAULT 0,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(user_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_user_model_settings_user ON user_model_settings(user_id);
DROP TRIGGER IF EXISTS user_model_settings_updated_at ON user_model_settings;
CREATE TRIGGER user_model_settings_updated_at BEFORE UPDATE ON user_model_settings
FOR EACH ROW EXECUTE FUNCTION update_updated_at();

46
server/database/migrations/005_v0180_memory_phase2.sql
View File

@@ -1,46 +0,0 @@
-- v0.18.0 Phase 2: Memory Extraction & Embeddings
--
-- Adds persona memory configuration, HNSW vector index for
-- semantic recall, and extraction tracking log.
-- =========================================
-- 1. Persona Memory Configuration
-- =========================================
ALTER TABLE personas ADD COLUMN IF NOT EXISTS memory_enabled BOOLEAN NOT NULL DEFAULT true;
ALTER TABLE personas ADD COLUMN IF NOT EXISTS memory_extraction_prompt TEXT;
COMMENT ON COLUMN personas.memory_enabled IS 'Whether memory tools and extraction are active for this persona';
COMMENT ON COLUMN personas.memory_extraction_prompt IS 'Custom extraction prompt; NULL = use system default';
-- =========================================
-- 2. Vector Index for Semantic Recall
-- =========================================
-- NOTE: HNSW indexes have a 2000-dimension limit in pgvector.
-- Our embeddings are 3072-dim (matching KB/notes schema), so we
-- skip the HNSW index. Memory tables are small enough (hundreds
-- of rows per user) that filtered sequential scans with cosine
-- distance are performant. If needed later, IVFFlat supports
-- higher dimensions, or embeddings can be reduced to 1536-dim.
-- =========================================
-- 3. Memory Extraction Log
-- =========================================
-- Tracks which messages have been processed by the extraction scanner
-- to prevent duplicate work. One row per channel+user pair.
CREATE TABLE IF NOT EXISTS memory_extraction_log (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL,
user_id UUID NOT NULL,
last_message_id UUID NOT NULL,
extracted_at TIMESTAMPTZ DEFAULT now(),
memory_count INT NOT NULL DEFAULT 0,
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_mem_extract_log_channel
ON memory_extraction_log(channel_id);
COMMENT ON TABLE memory_extraction_log IS 'Tracks extraction progress per channel+user to prevent re-processing';

114
server/database/migrations/006_knowledge.sql Normal file
View File

@@ -0,0 +1,114 @@
-- ==========================================
-- Chat Switchboard — 006 Knowledge Bases
-- ==========================================
-- Knowledge bases, documents, chunks, and bindings to channels/personas.
-- ICD §5 (Knowledge Bases)
-- ==========================================
-- =========================================
-- KNOWLEDGE BASES
-- =========================================
CREATE TABLE IF NOT EXISTS knowledge_bases (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope TEXT NOT NULL DEFAULT 'global',
owner_id UUID REFERENCES users(id) ON DELETE CASCADE,
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
embedding_config JSONB NOT NULL DEFAULT '{}',
document_count INT NOT NULL DEFAULT 0,
chunk_count INT NOT NULL DEFAULT 0,
total_bytes BIGINT NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'active',
discoverable BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT kb_scope_check CHECK (
(scope = 'global' AND owner_id IS NULL) OR
(scope = 'team' AND team_id IS NOT NULL) OR
(scope = 'personal' AND owner_id IS NOT NULL)
)
);
CREATE INDEX IF NOT EXISTS idx_kb_scope ON knowledge_bases(scope);
CREATE INDEX IF NOT EXISTS idx_kb_owner ON knowledge_bases(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_kb_team ON knowledge_bases(team_id) WHERE team_id IS NOT NULL;
COMMENT ON COLUMN knowledge_bases.discoverable IS 'false = hidden from user KB lists, only accessible through persona binding';
-- =========================================
-- KB DOCUMENTS
-- =========================================
CREATE TABLE IF NOT EXISTS kb_documents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
filename TEXT NOT NULL,
content_type TEXT NOT NULL,
size_bytes BIGINT NOT NULL,
storage_key TEXT NOT NULL,
extracted_text TEXT,
chunk_count INT NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'pending',
error TEXT,
uploaded_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_kbdoc_kb ON kb_documents(kb_id);
-- =========================================
-- KB CHUNKS
-- =========================================
CREATE TABLE IF NOT EXISTS kb_chunks (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
document_id UUID NOT NULL REFERENCES kb_documents(id) ON DELETE CASCADE,
chunk_index INT NOT NULL,
content TEXT NOT NULL,
token_count INT NOT NULL DEFAULT 0,
embedding VECTOR(3072),
metadata JSONB NOT NULL DEFAULT '{}',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_kbchunk_kb ON kb_chunks(kb_id);
CREATE INDEX IF NOT EXISTS idx_kbchunk_doc ON kb_chunks(document_id);
-- =========================================
-- CHANNEL ↔ KB BINDING
-- =========================================
CREATE TABLE IF NOT EXISTS channel_knowledge_bases (
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
enabled BOOLEAN NOT NULL DEFAULT true,
added_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (channel_id, kb_id)
);
-- =========================================
-- PERSONA ↔ KB BINDING (v0.17.0)
-- =========================================
CREATE TABLE IF NOT EXISTS persona_knowledge_bases (
persona_id UUID NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
kb_id UUID NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
auto_search BOOLEAN NOT NULL DEFAULT false,
added_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (persona_id, kb_id)
);
CREATE INDEX IF NOT EXISTS idx_persona_kb_persona ON persona_knowledge_bases(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_kb_kb ON persona_knowledge_bases(kb_id);
COMMENT ON TABLE persona_knowledge_bases IS 'Binds KBs to personas — the persona becomes a gateway to its KBs';
COMMENT ON COLUMN persona_knowledge_bases.auto_search IS 'true = auto-prepend top-K results to context; false = kb_search tool only';

75
server/database/migrations/007_notes.sql Normal file
View File

@@ -0,0 +1,75 @@
-- ==========================================
-- Chat Switchboard — 007 Notes & Links
-- ==========================================
-- Notes with full-text search, wikilink graph edges.
-- ICD §6 (Notes)
-- ==========================================
-- =========================================
-- NOTES
-- =========================================
CREATE TABLE IF NOT EXISTS notes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
title VARCHAR(500) NOT NULL,
content TEXT DEFAULT '',
folder_path TEXT DEFAULT '/',
tags TEXT[] DEFAULT '{}',
metadata JSONB DEFAULT '{}',
source_channel_id UUID REFERENCES channels(id) ON DELETE SET NULL,
source_message_id UUID,
team_id UUID REFERENCES teams(id) ON DELETE SET NULL,
search_vector TSVECTOR,
embedding VECTOR(3072),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_notes_user ON notes(user_id);
CREATE INDEX IF NOT EXISTS idx_notes_search ON notes USING GIN(search_vector);
CREATE INDEX IF NOT EXISTS idx_notes_folder ON notes(user_id, folder_path);
CREATE INDEX IF NOT EXISTS idx_notes_tags ON notes USING GIN(tags);
CREATE INDEX IF NOT EXISTS idx_notes_updated ON notes(user_id, updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_notes_team ON notes(team_id) WHERE team_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_notes_source_msg ON notes(source_message_id)
WHERE source_message_id IS NOT NULL;
CREATE OR REPLACE FUNCTION notes_search_update_fn()
RETURNS TRIGGER AS $$
BEGIN
NEW.search_vector :=
setweight(to_tsvector('pg_catalog.english', COALESCE(NEW.title, '')), 'A') ||
setweight(to_tsvector('pg_catalog.english', COALESCE(NEW.content, '')), 'B');
NEW.updated_at := NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS notes_search_update ON notes;
CREATE TRIGGER notes_search_update
BEFORE INSERT OR UPDATE OF title, content ON notes
FOR EACH ROW EXECUTE FUNCTION notes_search_update_fn();
-- =========================================
-- NOTE LINKS (v0.17.3 — wikilink graph)
-- =========================================
CREATE TABLE IF NOT EXISTS note_links (
source_note_id UUID NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
target_note_id UUID REFERENCES notes(id) ON DELETE CASCADE,
target_title TEXT NOT NULL,
display_text TEXT,
is_transclusion BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMPTZ DEFAULT now(),
PRIMARY KEY (source_note_id, target_title)
);
CREATE INDEX IF NOT EXISTS idx_note_links_target ON note_links(target_note_id)
WHERE target_note_id IS NOT NULL;
COMMENT ON TABLE note_links IS 'Directed links between notes, extracted from [[wikilink]] syntax on save';
COMMENT ON COLUMN note_links.target_note_id IS 'NULL for unresolved links (target note does not exist yet)';
COMMENT ON COLUMN note_links.target_title IS 'Raw [[title]] text — preserved for re-resolution after renames';
COMMENT ON COLUMN note_links.is_transclusion IS 'true for ![[embed]] syntax, false for regular [[link]]';

17
server/database/migrations/007_v0200_notifications.sql
View File

@@ -1,17 +0,0 @@
-- 007_v0200_notifications.sql
-- Notification infrastructure (v0.20.0 Phase 1)
CREATE TABLE IF NOT EXISTS notifications (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type VARCHAR(50) NOT NULL,
title VARCHAR(255) NOT NULL,
body TEXT DEFAULT '',
resource_type VARCHAR(50),
resource_id UUID,
is_read BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_notifications_user_unread ON notifications(user_id, is_read, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_notifications_user_created ON notifications(user_id, created_at DESC);

82
server/database/migrations/008_memory.sql Normal file
View File

@@ -0,0 +1,82 @@
-- ==========================================
-- Chat Switchboard — 008 Memory
-- ==========================================
-- Long-term memory across conversations, extraction tracking.
-- ICD §9 (Memory)
-- ==========================================
-- =========================================
-- MEMORIES
-- =========================================
CREATE TABLE IF NOT EXISTS memories (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
scope TEXT NOT NULL CHECK (scope IN ('user', 'persona', 'persona_user')),
owner_id UUID NOT NULL,
user_id UUID,
key TEXT NOT NULL,
value TEXT NOT NULL,
source_channel_id UUID,
confidence REAL NOT NULL DEFAULT 1.0,
status TEXT NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'pending_review', 'archived')),
embedding vector(3072),
created_at TIMESTAMPTZ DEFAULT now(),
updated_at TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_memories_scope_owner
ON memories(scope, owner_id, status)
WHERE status = 'active';
CREATE INDEX IF NOT EXISTS idx_memories_persona_user
ON memories(owner_id, user_id)
WHERE scope = 'persona_user' AND status = 'active';
CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_unique_key
ON memories(scope, owner_id, COALESCE(user_id, '00000000-0000-0000-0000-000000000000'), key);
CREATE INDEX IF NOT EXISTS idx_memories_fts
ON memories USING gin(to_tsvector('english', key || ' ' || value));
CREATE INDEX IF NOT EXISTS idx_memories_source_channel
ON memories(source_channel_id)
WHERE source_channel_id IS NOT NULL;
COMMENT ON TABLE memories IS 'Long-term memory facts persisted across conversations';
COMMENT ON COLUMN memories.scope IS 'user = personal facts; persona = shared; persona_user = per-user within a persona';
COMMENT ON COLUMN memories.owner_id IS 'user_id for user scope, persona_id for persona/persona_user scopes';
COMMENT ON COLUMN memories.confidence IS '0.0-1.0 confidence score — higher = more certain';
CREATE OR REPLACE FUNCTION update_memories_updated_at()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_at = now();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS trg_memories_updated_at ON memories;
CREATE TRIGGER trg_memories_updated_at
BEFORE UPDATE ON memories
FOR EACH ROW EXECUTE FUNCTION update_memories_updated_at();
-- =========================================
-- MEMORY EXTRACTION LOG (v0.18.0 Phase 2)
-- =========================================
CREATE TABLE IF NOT EXISTS memory_extraction_log (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID NOT NULL,
user_id UUID NOT NULL,
last_message_id UUID NOT NULL,
extracted_at TIMESTAMPTZ DEFAULT now(),
memory_count INT NOT NULL DEFAULT 0,
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_mem_extract_log_channel
ON memory_extraction_log(channel_id);
COMMENT ON TABLE memory_extraction_log IS 'Tracks extraction progress per channel+user to prevent re-processing';

18
server/database/migrations/008_v0200_notification_prefs.sql
View File

@@ -1,18 +0,0 @@
-- 008_v0200_notification_prefs.sql
-- Notification preferences per user per type.
-- Resolution: specific type → user '*' default → system default (in_app=true, email=false).
BEGIN;
CREATE TABLE IF NOT EXISTS notification_preferences (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type VARCHAR(50) NOT NULL, -- notification type or '*' for default
in_app BOOLEAN NOT NULL DEFAULT true,
email BOOLEAN NOT NULL DEFAULT false,
UNIQUE(user_id, type)
);
CREATE INDEX IF NOT EXISTS idx_notification_prefs_user ON notification_preferences(user_id);
COMMIT;

74
server/database/migrations/009_v0210_workspaces.sql
View File

@@ -1,74 +0,0 @@
-- v0.21.0: Workspaces (platform storage primitive)
--
-- New: workspaces table (polymorphic owner: user, project, channel, team)
-- New: workspace_files table (metadata index — filesystem is source of truth)
-- =========================================
-- 1. Workspaces Table
-- =========================================
CREATE TABLE IF NOT EXISTS workspaces (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
owner_type VARCHAR(20) NOT NULL
CHECK (owner_type IN ('user', 'project', 'channel', 'team')),
owner_id UUID NOT NULL,
name VARCHAR(200) NOT NULL,
root_path TEXT NOT NULL, -- PVC-relative: workspaces/{id}
max_bytes BIGINT, -- quota (NULL = system default)
status VARCHAR(20) NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'archived', 'deleting')),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_workspaces_owner
ON workspaces(owner_type, owner_id);
CREATE INDEX IF NOT EXISTS idx_workspaces_status
ON workspaces(status) WHERE status != 'deleting';
DROP TRIGGER IF EXISTS workspaces_updated_at ON workspaces;
CREATE TRIGGER workspaces_updated_at BEFORE UPDATE ON workspaces
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE workspaces IS 'File workspaces bound to users, projects, channels, or teams';
COMMENT ON COLUMN workspaces.owner_type IS 'Polymorphic owner: user, project, channel, team';
COMMENT ON COLUMN workspaces.root_path IS 'PVC-relative path to workspace root directory';
COMMENT ON COLUMN workspaces.max_bytes IS 'Storage quota in bytes (NULL = system default)';
-- =========================================
-- 2. Workspace Files Table
-- =========================================
-- Metadata index for workspace files. The PVC filesystem is the source
-- of truth; this table enables fast queries (list, filter by type, etc.)
-- and will hold embedding references in v0.21.2.
CREATE TABLE IF NOT EXISTS workspace_files (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
workspace_id UUID NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
path TEXT NOT NULL, -- relative path: src/main.go
is_directory BOOLEAN NOT NULL DEFAULT false,
content_type VARCHAR(100), -- MIME type
size_bytes BIGINT NOT NULL DEFAULT 0,
sha256 VARCHAR(64), -- content hash (NULL for dirs)
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_workspace_files_path
ON workspace_files(workspace_id, path);
CREATE INDEX IF NOT EXISTS idx_workspace_files_workspace
ON workspace_files(workspace_id);
CREATE INDEX IF NOT EXISTS idx_workspace_files_type
ON workspace_files(workspace_id, content_type)
WHERE content_type IS NOT NULL;
DROP TRIGGER IF EXISTS workspace_files_updated_at ON workspace_files;
CREATE TRIGGER workspace_files_updated_at BEFORE UPDATE ON workspace_files
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE workspace_files IS 'Metadata index for workspace files — filesystem is source of truth';
COMMENT ON COLUMN workspace_files.path IS 'Relative path from workspace files/ root';
COMMENT ON COLUMN workspace_files.sha256 IS 'Content hash for change detection and dedup';

123
server/database/migrations/009_workspaces.sql Normal file
View File

@@ -0,0 +1,123 @@
-- ==========================================
-- Chat Switchboard — 009 Workspaces & Git
-- ==========================================
-- File workspaces, workspace file index, semantic chunks, git credentials.
-- ICD §7 (Workspaces & Git)
-- ==========================================
-- =========================================
-- WORKSPACES
-- =========================================
CREATE TABLE IF NOT EXISTS workspaces (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
owner_type VARCHAR(20) NOT NULL
CHECK (owner_type IN ('user', 'project', 'channel', 'team')),
owner_id UUID NOT NULL,
name VARCHAR(200) NOT NULL,
root_path TEXT NOT NULL,
max_bytes BIGINT,
status VARCHAR(20) NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'archived', 'deleting')),
indexing_enabled BOOLEAN NOT NULL DEFAULT true,
-- Git integration (v0.21.4)
git_remote_url TEXT,
git_branch TEXT,
git_credential_id UUID,
git_last_sync TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_workspaces_owner ON workspaces(owner_type, owner_id);
CREATE INDEX IF NOT EXISTS idx_workspaces_status ON workspaces(status) WHERE status != 'deleting';
DROP TRIGGER IF EXISTS workspaces_updated_at ON workspaces;
CREATE TRIGGER workspaces_updated_at BEFORE UPDATE ON workspaces
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE workspaces IS 'File workspaces bound to users, projects, channels, or teams';
COMMENT ON COLUMN workspaces.root_path IS 'PVC-relative path to workspace root directory';
-- =========================================
-- WORKSPACE FILES
-- =========================================
CREATE TABLE IF NOT EXISTS workspace_files (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
workspace_id UUID NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
path TEXT NOT NULL,
is_directory BOOLEAN NOT NULL DEFAULT false,
content_type VARCHAR(100),
size_bytes BIGINT NOT NULL DEFAULT 0,
sha256 VARCHAR(64),
index_status VARCHAR(20) DEFAULT 'pending'
CHECK (index_status IN ('pending', 'indexing', 'ready', 'error', 'skipped')),
chunk_count INT NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_workspace_files_path
ON workspace_files(workspace_id, path);
CREATE INDEX IF NOT EXISTS idx_workspace_files_workspace ON workspace_files(workspace_id);
CREATE INDEX IF NOT EXISTS idx_workspace_files_type
ON workspace_files(workspace_id, content_type) WHERE content_type IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_workspace_files_index_status
ON workspace_files(workspace_id, index_status)
WHERE index_status IN ('pending', 'indexing');
DROP TRIGGER IF EXISTS workspace_files_updated_at ON workspace_files;
CREATE TRIGGER workspace_files_updated_at BEFORE UPDATE ON workspace_files
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
COMMENT ON TABLE workspace_files IS 'Metadata index for workspace files — filesystem is source of truth';
-- =========================================
-- WORKSPACE CHUNKS (v0.21.2)
-- =========================================
CREATE TABLE IF NOT EXISTS workspace_chunks (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
workspace_id UUID NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
file_id UUID NOT NULL REFERENCES workspace_files(id) ON DELETE CASCADE,
chunk_index INT NOT NULL,
content TEXT NOT NULL,
token_count INT NOT NULL DEFAULT 0,
embedding vector(3072),
metadata JSONB DEFAULT '{}',
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_file ON workspace_chunks(file_id);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_workspace ON workspace_chunks(workspace_id);
COMMENT ON TABLE workspace_chunks IS 'Text chunks with vector embeddings for workspace file semantic search';
-- =========================================
-- GIT CREDENTIALS (v0.21.4)
-- =========================================
CREATE TABLE IF NOT EXISTS git_credentials (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL DEFAULT '',
auth_type TEXT NOT NULL DEFAULT 'https_pat',
encrypted_data BYTEA NOT NULL DEFAULT '',
nonce BYTEA NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_git_credentials_user ON git_credentials(user_id);
-- Deferred FK: workspaces.git_credential_id → git_credentials.id
DO $$ BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint WHERE conname = 'fk_workspaces_git_credential'
) THEN
ALTER TABLE workspaces ADD CONSTRAINT fk_workspaces_git_credential
FOREIGN KEY (git_credential_id) REFERENCES git_credentials(id) ON DELETE SET NULL;
END IF;
END $$;

73
server/database/migrations/006_v0190_projects.sql → server/database/migrations/010_projects.sql
View File

@@ -1,27 +1,26 @@
-- v0.19.0: Projects / Workspaces
--
-- New: projects table
-- New: project_channels junction table (1:1 channel→project)
-- New: project_knowledge_bases junction table
-- New: project_notes junction table
-- New: channels.project_id denormalized FK
-- Changed: resource_grants.resource_type CHECK includes 'project'
-- Deprecated: channels.folder (column retained, no longer written)
-- ==========================================
-- Chat Switchboard — 010 Projects
-- ==========================================
-- Projects, junction tables, and cross-domain FK columns on
-- channels/projects for workspace/project bindings.
-- ICD §8 (Projects)
-- ==========================================
-- =========================================
-- 1. Projects Table
-- PROJECTS
-- =========================================
CREATE TABLE IF NOT EXISTS projects (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(200) NOT NULL,
description TEXT,
color VARCHAR(7), -- hex color e.g. '#3B82F6'
icon VARCHAR(50), -- emoji or icon name
color VARCHAR(7),
icon VARCHAR(50),
scope VARCHAR(20) NOT NULL DEFAULT 'personal'
CHECK (scope IN ('personal', 'team', 'global')),
owner_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
team_id UUID REFERENCES teams(id) ON DELETE SET NULL,
workspace_id UUID REFERENCES workspaces(id) ON DELETE SET NULL,
is_archived BOOLEAN NOT NULL DEFAULT false,
settings JSONB DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ DEFAULT NOW(),
@@ -31,6 +30,7 @@ CREATE TABLE IF NOT EXISTS projects (
CREATE INDEX IF NOT EXISTS idx_projects_owner ON projects(owner_id);
CREATE INDEX IF NOT EXISTS idx_projects_team ON projects(team_id) WHERE team_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_projects_scope ON projects(scope);
CREATE INDEX IF NOT EXISTS idx_projects_workspace ON projects(workspace_id) WHERE workspace_id IS NOT NULL;
DROP TRIGGER IF EXISTS projects_updated_at ON projects;
CREATE TRIGGER projects_updated_at BEFORE UPDATE ON projects
@@ -39,31 +39,29 @@ CREATE TRIGGER projects_updated_at BEFORE UPDATE ON projects
COMMENT ON TABLE projects IS 'Organizes channels, KBs, and notes into named workspaces';
COMMENT ON COLUMN projects.scope IS 'personal=owner only, team=team members, global=all users';
-- =========================================
-- 2. Project ↔ Channel Junction
-- PROJECT ↔ CHANNEL JUNCTION
-- =========================================
-- One project per channel (UNIQUE on channel_id).
-- Source of truth for position and folder within the project.
CREATE TABLE IF NOT EXISTS project_channels (
project_id UUID NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
channel_id UUID NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
position INTEGER NOT NULL DEFAULT 0,
folder TEXT, -- sub-folder within the project
folder TEXT,
added_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (project_id, channel_id),
UNIQUE (channel_id) -- one project per channel
UNIQUE (channel_id)
);
CREATE INDEX IF NOT EXISTS idx_project_channels_project ON project_channels(project_id);
CREATE INDEX IF NOT EXISTS idx_project_channels_channel ON project_channels(channel_id);
COMMENT ON TABLE project_channels IS 'Links channels to projects. UNIQUE(channel_id) enforces one-project-per-channel.';
COMMENT ON COLUMN project_channels.position IS 'Sort order within the project sidebar group';
COMMENT ON COLUMN project_channels.folder IS 'Optional sub-folder path within the project';
-- =========================================
-- 3. Project ↔ Knowledge Base Junction
-- PROJECT ↔ KB JUNCTION
-- =========================================
CREATE TABLE IF NOT EXISTS project_knowledge_bases (
@@ -77,11 +75,9 @@ CREATE TABLE IF NOT EXISTS project_knowledge_bases (
CREATE INDEX IF NOT EXISTS idx_project_kb_project ON project_knowledge_bases(project_id);
CREATE INDEX IF NOT EXISTS idx_project_kb_kb ON project_knowledge_bases(kb_id);
COMMENT ON TABLE project_knowledge_bases IS 'Binds KBs to Projects — injected at completion time for all project channels';
COMMENT ON COLUMN project_knowledge_bases.auto_search IS 'true = auto-prepend top-K results; false = kb_search tool only';
-- =========================================
-- 4. Project ↔ Note Junction
-- PROJECT ↔ NOTE JUNCTION
-- =========================================
CREATE TABLE IF NOT EXISTS project_notes (
@@ -94,36 +90,27 @@ CREATE TABLE IF NOT EXISTS project_notes (
CREATE INDEX IF NOT EXISTS idx_project_notes_project ON project_notes(project_id);
CREATE INDEX IF NOT EXISTS idx_project_notes_note ON project_notes(note_id);
COMMENT ON TABLE project_notes IS 'Links notes to projects for organizational grouping';
-- =========================================
-- 5. Denormalized project_id on Channels
-- CROSS-DOMAIN FK COLUMNS
-- =========================================
-- Fast sidebar query: SELECT ... FROM channels WHERE project_id = $1
-- Source of truth is project_channels; this is updated atomically.
-- Denormalized project_id + workspace_id on channels.
ALTER TABLE channels
ADD COLUMN IF NOT EXISTS project_id UUID REFERENCES projects(id) ON DELETE SET NULL;
ALTER TABLE channels
ADD COLUMN IF NOT EXISTS workspace_id UUID REFERENCES workspaces(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_channels_project ON channels(project_id) WHERE project_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_channels_workspace ON channels(workspace_id) WHERE workspace_id IS NOT NULL;
-- =========================================
-- 6. Extend resource_grants CHECK
-- FOLDER → PROJECT MIGRATION (idempotent)
-- =========================================
-- Add 'project' to allowed resource_type values.
-- Postgres ALTER TABLE ... ALTER CONSTRAINT is not supported; drop + re-add.
ALTER TABLE resource_grants DROP CONSTRAINT IF EXISTS resource_grants_resource_type_check;
ALTER TABLE resource_grants ADD CONSTRAINT resource_grants_resource_type_check
CHECK (resource_type IN ('persona', 'knowledge_base', 'project'));
-- =========================================
-- 7. Best-effort folder → project migration
-- =========================================
-- Create a personal project for each distinct folder value, then
-- associate the channels. Skip if no folder values exist.
-- This is idempotent: re-running won't duplicate projects because
-- we check for existing project_id.
-- Best-effort migration of legacy folder values into projects.
-- Only runs if there are channels with a folder value and no project_id.
DO $$
DECLARE
@@ -136,12 +123,10 @@ BEGIN
SELECT DISTINCT user_id, folder FROM channels
WHERE folder IS NOT NULL AND folder != '' AND project_id IS NULL
LOOP
-- Create project named after the folder
INSERT INTO projects (name, scope, owner_id)
VALUES (_folder, 'personal', _user_id)
RETURNING id INTO _proj_id;
-- Associate channels
FOR _ch_id IN
SELECT id FROM channels
WHERE user_id = _user_id AND folder = _folder AND project_id IS NULL

16
server/database/migrations/010_v0211_workspace_bindings.sql
View File

@@ -1,16 +0,0 @@
-- v0.21.1: Workspace bindings for channels and projects
--
-- Channels and projects can optionally bind to a workspace.
-- Resolution chain: channel workspace_id > project workspace_id.
-- Channel workspace binding
ALTER TABLE channels
ADD COLUMN IF NOT EXISTS workspace_id UUID REFERENCES workspaces(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_channels_workspace ON channels(workspace_id) WHERE workspace_id IS NOT NULL;
-- Project workspace binding
ALTER TABLE projects
ADD COLUMN IF NOT EXISTS workspace_id UUID REFERENCES workspaces(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_projects_workspace ON projects(workspace_id) WHERE workspace_id IS NOT NULL;

41
server/database/migrations/011_notifications.sql Normal file
View File

@@ -0,0 +1,41 @@
-- ==========================================
-- Chat Switchboard — 011 Notifications
-- ==========================================
-- Notification delivery and per-user preferences.
-- ICD §12 (Notifications)
-- ==========================================
-- =========================================
-- NOTIFICATIONS
-- =========================================
CREATE TABLE IF NOT EXISTS notifications (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type VARCHAR(50) NOT NULL,
title VARCHAR(255) NOT NULL,
body TEXT DEFAULT '',
resource_type VARCHAR(50),
resource_id UUID,
is_read BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_notifications_user_unread ON notifications(user_id, is_read, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_notifications_user_created ON notifications(user_id, created_at DESC);
-- =========================================
-- NOTIFICATION PREFERENCES
-- =========================================
CREATE TABLE IF NOT EXISTS notification_preferences (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type VARCHAR(50) NOT NULL,
in_app BOOLEAN NOT NULL DEFAULT true,
email BOOLEAN NOT NULL DEFAULT false,
UNIQUE(user_id, type)
);
CREATE INDEX IF NOT EXISTS idx_notification_prefs_user ON notification_preferences(user_id);

58
server/database/migrations/011_v0212_workspace_chunks.sql
View File

@@ -1,58 +0,0 @@
-- v0.21.2: Workspace indexing + semantic search
--
-- New: workspace_chunks table for text embeddings per workspace file
-- Alter: workspace_files gains index_status and chunk_count columns
-- Alter: workspaces gains indexing_enabled column
-- =========================================
-- 1. workspace_files additions
-- =========================================
ALTER TABLE workspace_files
ADD COLUMN IF NOT EXISTS index_status VARCHAR(20) DEFAULT 'pending'
CHECK (index_status IN ('pending', 'indexing', 'ready', 'error', 'skipped'));
ALTER TABLE workspace_files
ADD COLUMN IF NOT EXISTS chunk_count INT NOT NULL DEFAULT 0;
CREATE INDEX IF NOT EXISTS idx_workspace_files_index_status
ON workspace_files(workspace_id, index_status)
WHERE index_status IN ('pending', 'indexing');
-- =========================================
-- 2. workspaces addition
-- =========================================
ALTER TABLE workspaces
ADD COLUMN IF NOT EXISTS indexing_enabled BOOLEAN NOT NULL DEFAULT true;
-- =========================================
-- 3. workspace_chunks table
-- =========================================
CREATE TABLE IF NOT EXISTS workspace_chunks (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
workspace_id UUID NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
file_id UUID NOT NULL REFERENCES workspace_files(id) ON DELETE CASCADE,
chunk_index INT NOT NULL,
content TEXT NOT NULL,
token_count INT NOT NULL DEFAULT 0,
embedding vector(3072),
metadata JSONB DEFAULT '{}',
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_file
ON workspace_chunks(file_id);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_workspace
ON workspace_chunks(workspace_id);
-- Note: No vector index created here. pgvector IVFFlat/HNSW indexes are
-- limited to 2000 dimensions, but our embeddings are 3072-dim (zero-padded).
-- Sequential scan with <=> is adequate at workspace scale (thousands of
-- chunks, not millions). Add a partial/quantized index later if needed.
COMMENT ON TABLE workspace_chunks IS 'Text chunks with vector embeddings for workspace file semantic search';
COMMENT ON COLUMN workspace_chunks.embedding IS 'Vector embedding (3072-dim, zero-padded) for cosine similarity';
COMMENT ON COLUMN workspace_chunks.metadata IS 'Extensible: line_start, heading, language, etc.';

44
server/database/migrations/012_extensions.sql Normal file
View File

@@ -0,0 +1,44 @@
-- ==========================================
-- Chat Switchboard — 012 Extensions
-- ==========================================
-- Extension registry and per-user settings.
-- ICD §13 (Extensions)
-- ==========================================
-- =========================================
-- EXTENSIONS
-- =========================================
CREATE TABLE IF NOT EXISTS extensions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
ext_id VARCHAR(100) NOT NULL UNIQUE,
name VARCHAR(200) NOT NULL,
version VARCHAR(50) NOT NULL DEFAULT '0.0.0',
tier VARCHAR(20) NOT NULL DEFAULT 'browser',
description TEXT NOT NULL DEFAULT '',
author VARCHAR(200) NOT NULL DEFAULT '',
manifest JSONB NOT NULL DEFAULT '{}',
is_system BOOLEAN NOT NULL DEFAULT false,
is_enabled BOOLEAN NOT NULL DEFAULT true,
scope VARCHAR(20) NOT NULL DEFAULT 'global',
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
installed_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_extensions_tier ON extensions(tier);
CREATE INDEX IF NOT EXISTS idx_extensions_enabled ON extensions(is_enabled) WHERE is_enabled = true;
-- =========================================
-- EXTENSION USER SETTINGS
-- =========================================
CREATE TABLE IF NOT EXISTS extension_user_settings (
extension_id UUID NOT NULL REFERENCES extensions(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
settings JSONB NOT NULL DEFAULT '{}',
is_enabled BOOLEAN NOT NULL DEFAULT true,
PRIMARY KEY (extension_id, user_id)
);

21
server/database/migrations/012_v0214_git.sql
View File

@@ -1,21 +0,0 @@
-- v0.21.4: Git integration — credentials table + workspace git columns.
-- Git credentials: encrypted storage for PAT, basic auth, and SSH keys.
-- Uses the same AES-256-GCM vault pattern as BYOK provider configs.
CREATE TABLE IF NOT EXISTS git_credentials (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL DEFAULT '',
auth_type TEXT NOT NULL DEFAULT 'https_pat', -- https_pat, https_basic, ssh_key
encrypted_data BYTEA NOT NULL DEFAULT '',
nonce BYTEA NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_git_credentials_user ON git_credentials(user_id);
-- Workspace git tracking columns.
ALTER TABLE workspaces ADD COLUMN IF NOT EXISTS git_remote_url TEXT;
ALTER TABLE workspaces ADD COLUMN IF NOT EXISTS git_branch TEXT;
ALTER TABLE workspaces ADD COLUMN IF NOT EXISTS git_credential_id UUID REFERENCES git_credentials(id) ON DELETE SET NULL;
ALTER TABLE workspaces ADD COLUMN IF NOT EXISTS git_last_sync TIMESTAMPTZ;

65
server/database/migrations/013_files.sql Normal file
View File

@@ -0,0 +1,65 @@
-- ==========================================
-- Chat Switchboard — 013 Files
-- ==========================================
-- Unified files table replacing attachments.
-- Handles both upgrade (attachments exists) and fresh install.
-- ==========================================
CREATE TABLE IF NOT EXISTS files (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID REFERENCES channels(id) ON DELETE CASCADE,
message_id UUID REFERENCES messages(id) ON DELETE SET NULL,
user_id UUID REFERENCES users(id) ON DELETE SET NULL,
project_id UUID REFERENCES projects(id) ON DELETE SET NULL,
origin VARCHAR(20) NOT NULL DEFAULT 'user_upload'
CHECK (origin IN ('user_upload', 'tool_output', 'system')),
filename VARCHAR(255) NOT NULL,
content_type VARCHAR(127) NOT NULL DEFAULT 'application/octet-stream',
size_bytes BIGINT NOT NULL DEFAULT 0,
storage_key TEXT NOT NULL,
display_hint VARCHAR(20) NOT NULL DEFAULT 'download'
CHECK (display_hint IN ('inline', 'download', 'thumbnail')),
extracted_text TEXT,
metadata JSONB DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Ensure attachments table exists so the INSERT is always valid.
-- Upgrade: no-op (table already has data). Fresh install: empty table.
CREATE TABLE IF NOT EXISTS attachments (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID, user_id UUID, message_id UUID, project_id UUID,
filename VARCHAR(255), content_type VARCHAR(127), size_bytes BIGINT,
storage_key TEXT, extracted_text TEXT, metadata JSONB, created_at TIMESTAMPTZ
);
INSERT INTO files (
id, channel_id, message_id, user_id, project_id,
origin, filename, content_type, size_bytes, storage_key,
display_hint, extracted_text, metadata, created_at, updated_at
)
SELECT
id, channel_id, message_id, user_id, project_id,
'user_upload',
filename, content_type, size_bytes, storage_key,
CASE
WHEN content_type LIKE 'image/%' THEN 'inline'
WHEN content_type LIKE 'video/%' THEN 'inline'
WHEN content_type LIKE 'audio/%' THEN 'inline'
WHEN content_type = 'application/pdf' THEN 'thumbnail'
WHEN content_type LIKE 'text/%' THEN 'inline'
ELSE 'download'
END,
extracted_text, metadata, created_at, created_at
FROM attachments
ON CONFLICT (id) DO NOTHING;
DROP TABLE IF EXISTS attachments;
CREATE INDEX IF NOT EXISTS idx_files_channel ON files(channel_id);
CREATE INDEX IF NOT EXISTS idx_files_message ON files(message_id);
CREATE INDEX IF NOT EXISTS idx_files_user ON files(user_id);
CREATE INDEX IF NOT EXISTS idx_files_project ON files(project_id) WHERE project_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_files_origin ON files(origin);
CREATE INDEX IF NOT EXISTS idx_files_orphan ON files(created_at) WHERE message_id IS NULL AND origin = 'user_upload';

43
server/database/migrations/013_v0220_health.sql
View File

@@ -1,43 +0,0 @@
-- v0.22.0: Provider health tracking + capability admin overrides.
-- ── Provider Health ─────────────────────────
-- Hourly bucketed health metrics per provider config.
-- In-memory accumulator flushes to this table every 60s.
-- Background job prunes rows older than 7 days.
CREATE TABLE IF NOT EXISTS provider_health (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
window_start TIMESTAMPTZ NOT NULL, -- hourly bucket floor
request_count INT NOT NULL DEFAULT 0,
error_count INT NOT NULL DEFAULT 0,
timeout_count INT NOT NULL DEFAULT 0,
total_latency_ms BIGINT NOT NULL DEFAULT 0, -- sum for avg calculation
max_latency_ms INT NOT NULL DEFAULT 0,
last_error TEXT,
last_error_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (provider_config_id, window_start)
);
CREATE INDEX IF NOT EXISTS idx_provider_health_window
ON provider_health (provider_config_id, window_start DESC);
-- ── Capability Overrides ────────────────────
-- Admin corrections for model capabilities. Highest priority in the
-- three-tier resolution chain: catalog → heuristic → admin override.
CREATE TABLE IF NOT EXISTS capability_overrides (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
field TEXT NOT NULL, -- tool_calling, vision, thinking, reasoning, etc.
value TEXT NOT NULL, -- "true"/"false" for bools, numeric string for ints
set_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
-- Unique per (provider, model, field). NULL provider = applies to model globally.
UNIQUE (provider_config_id, model_id, field)
);
CREATE INDEX IF NOT EXISTS idx_capability_overrides_model
ON capability_overrides (model_id);

58
server/database/migrations/014_audit_usage.sql Normal file
View File

@@ -0,0 +1,58 @@
-- ==========================================
-- Chat Switchboard — 014 Audit & Usage
-- ==========================================
-- Audit trail and usage/cost tracking.
-- ICD §16 (Platform Administration), §17 (Export & Utility)
-- ==========================================
-- =========================================
-- AUDIT LOG
-- =========================================
CREATE TABLE IF NOT EXISTS audit_log (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
actor_id UUID REFERENCES users(id) ON DELETE SET NULL,
action VARCHAR(100) NOT NULL,
resource_type VARCHAR(50) NOT NULL,
resource_id VARCHAR(255),
metadata JSONB DEFAULT '{}'::jsonb,
ip_address VARCHAR(45),
user_agent TEXT DEFAULT '',
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_audit_log_created ON audit_log(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_log_actor ON audit_log(actor_id) WHERE actor_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_audit_log_resource ON audit_log(resource_type, resource_id);
CREATE INDEX IF NOT EXISTS idx_audit_log_action ON audit_log(action);
COMMENT ON TABLE audit_log IS 'Immutable audit trail of all mutating operations';
-- =========================================
-- USAGE LOG
-- =========================================
CREATE TABLE IF NOT EXISTS usage_log (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
channel_id UUID REFERENCES channels(id) ON DELETE SET NULL,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
provider_config_id UUID REFERENCES provider_configs(id) ON DELETE SET NULL,
provider_scope TEXT NOT NULL DEFAULT 'global',
model_id TEXT NOT NULL,
role TEXT,
prompt_tokens INT NOT NULL DEFAULT 0,
completion_tokens INT NOT NULL DEFAULT 0,
cache_creation_tokens INT NOT NULL DEFAULT 0,
cache_read_tokens INT NOT NULL DEFAULT 0,
cost_input NUMERIC(12,6),
cost_output NUMERIC(12,6),
routing_decision JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_usage_log_user ON usage_log(user_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_created ON usage_log(created_at);
CREATE INDEX IF NOT EXISTS idx_usage_log_provider ON usage_log(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_model ON usage_log(model_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_scope ON usage_log(provider_scope);

31
server/database/migrations/014_v0222_routing.sql
View File

@@ -1,31 +0,0 @@
-- Migration 014: v0.22.2 — Routing Policies
-- Adds routing_policies table for rules-based provider routing.
CREATE TABLE IF NOT EXISTS routing_policies (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(200) NOT NULL,
scope VARCHAR(10) NOT NULL DEFAULT 'global'
CHECK (scope IN ('global', 'team')),
team_id UUID REFERENCES teams(id) ON DELETE CASCADE,
priority INTEGER NOT NULL DEFAULT 100,
policy_type VARCHAR(30) NOT NULL
CHECK (policy_type IN ('provider_prefer', 'team_route', 'cost_limit', 'model_alias')),
config JSONB NOT NULL DEFAULT '{}'::jsonb,
is_active BOOLEAN NOT NULL DEFAULT true,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_routing_policies_active
ON routing_policies(is_active, priority) WHERE is_active = true;
CREATE INDEX IF NOT EXISTS idx_routing_policies_team
ON routing_policies(team_id) WHERE team_id IS NOT NULL;
-- Trigger for updated_at
CREATE TRIGGER routing_policies_updated_at
BEFORE UPDATE ON routing_policies
FOR EACH ROW EXECUTE FUNCTION update_updated_at();
-- Add routing_decision column to usage_log for observability.
ALTER TABLE usage_log
ADD COLUMN IF NOT EXISTS routing_decision JSONB;

23
server/database/migrations/015_health.sql Normal file
View File

@@ -0,0 +1,23 @@
-- ==========================================
-- Chat Switchboard — 015 Tool Health
-- ==========================================
-- Health tracking for built-in tools (web_search, url_fetch, etc.).
-- ==========================================
CREATE TABLE IF NOT EXISTS tool_health (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tool_name TEXT NOT NULL,
window_start TIMESTAMPTZ NOT NULL,
request_count INT NOT NULL DEFAULT 0,
error_count INT NOT NULL DEFAULT 0,
total_latency_ms BIGINT NOT NULL DEFAULT 0,
max_latency_ms INT NOT NULL DEFAULT 0,
last_error TEXT,
last_error_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (tool_name, window_start)
);
CREATE INDEX IF NOT EXISTS idx_tool_health_window
ON tool_health (tool_name, window_start DESC);

33
server/database/migrations/015_v0224_health_enhance.sql
View File

@@ -1,33 +0,0 @@
-- v0.22.4: Rate limit tracking, tool health, project files.
-- ── Rate limit column on provider_health ────
ALTER TABLE provider_health ADD COLUMN IF NOT EXISTS rate_limit_count INT NOT NULL DEFAULT 0;
-- ── Tool Health ─────────────────────────────
-- Lightweight health tracking for built-in tools (web_search, url_fetch, etc.).
-- One row per tool per hourly window, similar to provider_health.
CREATE TABLE IF NOT EXISTS tool_health (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tool_name TEXT NOT NULL, -- web_search, url_fetch, etc.
window_start TIMESTAMPTZ NOT NULL, -- hourly bucket floor
request_count INT NOT NULL DEFAULT 0,
error_count INT NOT NULL DEFAULT 0,
total_latency_ms BIGINT NOT NULL DEFAULT 0,
max_latency_ms INT NOT NULL DEFAULT 0,
last_error TEXT,
last_error_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (tool_name, window_start)
);
CREATE INDEX IF NOT EXISTS idx_tool_health_window
ON tool_health (tool_name, window_start DESC);
-- ── Project Files ───────────────────────────
-- Extends attachments to support project-scoped uploads.
-- When project_id is set, the file belongs to the project (not a message).
ALTER TABLE attachments ADD COLUMN IF NOT EXISTS project_id UUID REFERENCES projects(id) ON DELETE CASCADE;
CREATE INDEX IF NOT EXISTS idx_attachments_project
ON attachments (project_id) WHERE project_id IS NOT NULL;

98
server/database/migrations/sqlite/001_core.sql Normal file
View File

@@ -0,0 +1,98 @@
-- ==========================================
-- Chat Switchboard — 001 Core (SQLite)
-- ==========================================
-- Users, authentication, platform policies, global settings.
-- v0.22.8: Domain-grouped migration
-- ==========================================
PRAGMA journal_mode = WAL;
PRAGMA foreign_keys = ON;
-- =========================================
-- USERS
-- =========================================
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL,
email TEXT NOT NULL,
password_hash TEXT NOT NULL,
display_name TEXT,
avatar_url TEXT,
role TEXT DEFAULT 'user'
CHECK (role IN ('user', 'admin')),
is_active INTEGER DEFAULT 1,
settings TEXT DEFAULT '{}',
encrypted_uek BLOB,
uek_salt BLOB,
uek_nonce BLOB,
vault_set INTEGER NOT NULL DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
last_login_at TEXT
);
CREATE UNIQUE INDEX IF NOT EXISTS users_username_ci ON users (username COLLATE NOCASE);
CREATE UNIQUE INDEX IF NOT EXISTS users_email_ci ON users (email COLLATE NOCASE);
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
CREATE TRIGGER IF NOT EXISTS users_updated_at AFTER UPDATE ON users
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN UPDATE users SET updated_at = datetime('now') WHERE id = NEW.id; END;
-- =========================================
-- AUTH — REFRESH TOKENS
-- =========================================
CREATE TABLE IF NOT EXISTS refresh_tokens (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
expires_at TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now')),
revoked_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_user ON refresh_tokens(user_id);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_hash ON refresh_tokens(token_hash);
-- =========================================
-- PLATFORM POLICIES
-- =========================================
CREATE TABLE IF NOT EXISTS platform_policies (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_by TEXT REFERENCES users(id),
updated_at TEXT DEFAULT (datetime('now'))
);
INSERT OR IGNORE INTO platform_policies (key, value) VALUES
('allow_user_byok', 'false'),
('allow_user_personas', 'false'),
('allow_raw_model_access', 'true'),
('allow_registration', 'true'),
('default_user_active', 'false'),
('allow_team_providers', 'true'),
('kb_direct_access', 'true');
-- =========================================
-- GLOBAL SETTINGS
-- =========================================
CREATE TABLE IF NOT EXISTS global_settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL DEFAULT '{}',
updated_at TEXT DEFAULT (datetime('now')),
updated_by TEXT REFERENCES users(id)
);
INSERT OR IGNORE INTO global_settings (key, value) VALUES
('registration', '{"enabled": true}'),
('site', '{"name": "Chat Switchboard", "tagline": "Multi-Model AI Chat"}'),
('banner', '{"enabled": false, "text": "", "bg": "#007a33", "fg": "#ffffff"}'),
('model_roles', '{"utility": {"primary": null, "fallback": null}, "embedding": {"primary": null, "fallback": null}, "generation": {"primary": null, "fallback": null}}');

22
server/database/migrations/sqlite/001_embedding_patch.sql
View File

@@ -1,22 +0,0 @@
-- Schema patch for 001_v017_schema.sql
-- Change the kb_chunks table to include the embedding column:
--
-- BEFORE:
-- CREATE TABLE IF NOT EXISTS kb_chunks (
-- ...
-- token_count INTEGER NOT NULL DEFAULT 0,
-- -- embedding column omitted: vector search feature-gated for SQLite.
-- -- Embedding bytes can be stored as BLOB if sqlite-vec is available.
-- metadata TEXT NOT NULL DEFAULT '{}',
-- ...
--
-- AFTER:
-- CREATE TABLE IF NOT EXISTS kb_chunks (
-- ...
-- token_count INTEGER NOT NULL DEFAULT 0,
-- embedding TEXT, -- JSON array of float64 for app-level cosine similarity
-- metadata TEXT NOT NULL DEFAULT '{}',
-- ...
--
-- For existing SQLite databases, run:
-- ALTER TABLE kb_chunks ADD COLUMN embedding TEXT;

791
server/database/migrations/sqlite/001_v017_schema.sql
View File

@@ -1,791 +0,0 @@
-- ==========================================
-- Chat Switchboard — v0.16.0 Consolidated Schema (SQLite)
-- ==========================================
-- SQLite equivalent of the Postgres schema.
--
-- Key differences from Postgres:
-- • No extensions (pgcrypto, pgvector)
-- • UUID generated application-side (Go uuid.New())
-- • JSONB → TEXT (JSON stored as text)
-- • TEXT[] / UUID[] → TEXT (JSON arrays as text)
-- • TIMESTAMPTZ → TEXT (ISO 8601 strings)
-- • BYTEA → BLOB
-- • NUMERIC → REAL
-- • TSVECTOR/VECTOR columns omitted (feature-gated)
-- • No COMMENT ON statements
-- • Triggers use SQLite syntax (WHEN guard to prevent recursion)
-- • No partial index WHERE clauses on GIN indexes
-- ==========================================
-- ── WAL mode (also set via connection pragma) ──
PRAGMA journal_mode = WAL;
PRAGMA foreign_keys = ON;
-- =========================================
-- 1. USERS
-- =========================================
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL,
email TEXT NOT NULL,
password_hash TEXT NOT NULL,
display_name TEXT,
avatar_url TEXT,
role TEXT DEFAULT 'user'
CHECK (role IN ('user', 'admin')),
is_active INTEGER DEFAULT 1,
settings TEXT DEFAULT '{}',
-- Vault: per-user encryption key (UEK) for BYOK API keys
encrypted_uek BLOB,
uek_salt BLOB,
uek_nonce BLOB,
vault_set INTEGER NOT NULL DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
last_login_at TEXT
);
CREATE UNIQUE INDEX IF NOT EXISTS users_username_ci ON users (LOWER(username));
CREATE UNIQUE INDEX IF NOT EXISTS users_email_ci ON users (LOWER(email));
CREATE INDEX IF NOT EXISTS idx_users_username ON users(username);
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
CREATE TRIGGER IF NOT EXISTS users_updated_at AFTER UPDATE ON users
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE users SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 2. AUTH
-- =========================================
CREATE TABLE IF NOT EXISTS refresh_tokens (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
expires_at TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now')),
revoked_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_user ON refresh_tokens(user_id);
CREATE INDEX IF NOT EXISTS idx_refresh_tokens_hash ON refresh_tokens(token_hash)
WHERE revoked_at IS NULL;
-- =========================================
-- 3. TEAMS
-- =========================================
CREATE TABLE IF NOT EXISTS teams (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
description TEXT DEFAULT '',
created_by TEXT NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
is_active INTEGER DEFAULT 1,
settings TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_teams_active ON teams(is_active) WHERE is_active = 1;
CREATE TRIGGER IF NOT EXISTS teams_updated_at AFTER UPDATE ON teams
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE teams SET updated_at = datetime('now') WHERE id = NEW.id;
END;
CREATE TABLE IF NOT EXISTS team_members (
id TEXT PRIMARY KEY,
team_id TEXT NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role TEXT NOT NULL DEFAULT 'member'
CHECK (role IN ('admin', 'member')),
joined_at TEXT DEFAULT (datetime('now')),
UNIQUE(team_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_team_members_team ON team_members(team_id);
CREATE INDEX IF NOT EXISTS idx_team_members_user ON team_members(user_id);
-- =========================================
-- 4. GROUPS
-- =========================================
CREATE TABLE IF NOT EXISTS groups (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope TEXT NOT NULL DEFAULT 'global'
CHECK (scope IN ('global', 'team')),
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
created_by TEXT NOT NULL REFERENCES users(id),
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
CONSTRAINT groups_scope_team CHECK (
(scope = 'global' AND team_id IS NULL) OR
(scope = 'team' AND team_id IS NOT NULL)
)
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_groups_name_scope
ON groups(name, COALESCE(team_id, '00000000-0000-0000-0000-000000000000'));
CREATE TRIGGER IF NOT EXISTS groups_updated_at AFTER UPDATE ON groups
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE groups SET updated_at = datetime('now') WHERE id = NEW.id;
END;
CREATE TABLE IF NOT EXISTS group_members (
id TEXT PRIMARY KEY,
group_id TEXT NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
added_by TEXT NOT NULL REFERENCES users(id),
added_at TEXT DEFAULT (datetime('now')),
UNIQUE(group_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_group_members_group ON group_members(group_id);
CREATE INDEX IF NOT EXISTS idx_group_members_user ON group_members(user_id);
-- =========================================
-- 5. PROVIDER CONFIGS
-- =========================================
CREATE TABLE IF NOT EXISTS provider_configs (
id TEXT PRIMARY KEY,
scope TEXT NOT NULL
CHECK (scope IN ('global', 'team', 'personal')),
owner_id TEXT,
name TEXT NOT NULL,
provider TEXT NOT NULL,
endpoint TEXT NOT NULL,
api_key_enc BLOB,
key_nonce BLOB,
key_scope TEXT NOT NULL DEFAULT 'global',
model_default TEXT,
config TEXT DEFAULT '{}',
headers TEXT DEFAULT '{}',
settings TEXT DEFAULT '{}',
is_active INTEGER DEFAULT 1,
is_private INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_provider_configs_scope ON provider_configs(scope);
CREATE INDEX IF NOT EXISTS idx_provider_configs_owner ON provider_configs(owner_id)
WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_provider_configs_active ON provider_configs(is_active)
WHERE is_active = 1;
CREATE TRIGGER IF NOT EXISTS provider_configs_updated_at AFTER UPDATE ON provider_configs
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE provider_configs SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 6. MODEL CATALOG
-- =========================================
CREATE TABLE IF NOT EXISTS model_catalog (
id TEXT PRIMARY KEY,
provider_config_id TEXT NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
display_name TEXT,
model_type TEXT DEFAULT 'chat',
capabilities TEXT NOT NULL DEFAULT '{}',
pricing TEXT,
visibility TEXT DEFAULT 'disabled'
CHECK (visibility IN ('enabled', 'disabled', 'team')),
last_synced_at TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(provider_config_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_model_catalog_provider ON model_catalog(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_model_catalog_enabled ON model_catalog(visibility)
WHERE visibility = 'enabled';
CREATE INDEX IF NOT EXISTS idx_model_catalog_type ON model_catalog(model_type);
CREATE TRIGGER IF NOT EXISTS model_catalog_updated_at AFTER UPDATE ON model_catalog
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE model_catalog SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 7. PERSONAS
-- =========================================
CREATE TABLE IF NOT EXISTS personas (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT DEFAULT '',
icon TEXT DEFAULT '',
avatar TEXT DEFAULT '',
base_model_id TEXT NOT NULL,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
system_prompt TEXT DEFAULT '',
temperature REAL,
max_tokens INTEGER,
thinking_budget INTEGER,
top_p REAL,
scope TEXT NOT NULL
CHECK (scope IN ('global', 'team', 'personal')),
owner_id TEXT,
created_by TEXT NOT NULL REFERENCES users(id),
is_active INTEGER DEFAULT 1,
is_shared INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_personas_scope ON personas(scope);
CREATE INDEX IF NOT EXISTS idx_personas_owner ON personas(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_personas_active ON personas(is_active) WHERE is_active = 1;
CREATE TRIGGER IF NOT EXISTS personas_updated_at AFTER UPDATE ON personas
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE personas SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 8. PERSONA GRANTS
-- =========================================
CREATE TABLE IF NOT EXISTS persona_grants (
id TEXT PRIMARY KEY,
persona_id TEXT NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
grant_type TEXT NOT NULL,
grant_ref TEXT NOT NULL,
config TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')),
UNIQUE(persona_id, grant_type, grant_ref)
);
CREATE INDEX IF NOT EXISTS idx_persona_grants_persona ON persona_grants(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_grants_type ON persona_grants(grant_type);
-- =========================================
-- 9. RESOURCE GRANTS
-- =========================================
-- granted_groups stored as JSON text array: '["uuid1","uuid2"]'
CREATE TABLE IF NOT EXISTS resource_grants (
id TEXT PRIMARY KEY,
resource_type TEXT NOT NULL
CHECK (resource_type IN ('persona', 'knowledge_base')),
resource_id TEXT NOT NULL,
grant_scope TEXT NOT NULL DEFAULT 'team_only'
CHECK (grant_scope IN ('team_only', 'global', 'groups')),
granted_groups TEXT NOT NULL DEFAULT '[]',
created_by TEXT NOT NULL REFERENCES users(id),
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(resource_type, resource_id)
);
CREATE INDEX IF NOT EXISTS idx_resource_grants_resource
ON resource_grants(resource_type, resource_id);
CREATE TRIGGER IF NOT EXISTS resource_grants_updated_at AFTER UPDATE ON resource_grants
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE resource_grants SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 10. PLATFORM POLICIES
-- =========================================
CREATE TABLE IF NOT EXISTS platform_policies (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_by TEXT REFERENCES users(id),
updated_at TEXT DEFAULT (datetime('now'))
);
INSERT OR IGNORE INTO platform_policies (key, value) VALUES
('allow_user_byok', 'false'),
('allow_user_personas', 'false'),
('allow_raw_model_access', 'true'),
('allow_registration', 'true'),
('default_user_active', 'false'),
('allow_team_providers', 'true');
-- =========================================
-- 11. GLOBAL SETTINGS
-- =========================================
CREATE TABLE IF NOT EXISTS global_settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL DEFAULT '{}',
updated_at TEXT DEFAULT (datetime('now')),
updated_by TEXT REFERENCES users(id)
);
INSERT OR IGNORE INTO global_settings (key, value) VALUES
('registration', '{"enabled": true}'),
('site', '{"name": "Chat Switchboard", "tagline": "Multi-Model AI Chat"}'),
('banner', '{"enabled": false, "text": "", "position": "both", "bg": "#007a33", "fg": "#ffffff"}'),
('banner_presets', '{"development": {"text": "DEVELOPMENT", "bg": "#007a33", "fg": "#ffffff"}, "testing": {"text": "TESTING", "bg": "#502b85", "fg": "#ffffff"}, "staging": {"text": "STAGING", "bg": "#0033a0", "fg": "#ffffff"}, "production": {"text": "PRODUCTION", "bg": "#c8102e", "fg": "#ffffff"}, "training": {"text": "TRAINING", "bg": "#ff8c00", "fg": "#000000"}, "demo": {"text": "DEMO", "bg": "#fce83a", "fg": "#000000"}}'),
('model_roles', '{"utility": {"primary": null, "fallback": null}, "embedding": {"primary": null, "fallback": null}, "generation": {"primary": null, "fallback": null}}');
-- =========================================
-- 12. USER MODEL SETTINGS
-- =========================================
CREATE TABLE IF NOT EXISTS user_model_settings (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
hidden INTEGER DEFAULT 0,
preferred_temperature REAL,
preferred_max_tokens INTEGER,
sort_order INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(user_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_user_model_settings_user ON user_model_settings(user_id);
CREATE TRIGGER IF NOT EXISTS user_model_settings_updated_at AFTER UPDATE ON user_model_settings
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE user_model_settings SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 13. CHANNELS
-- =========================================
-- tags stored as JSON text array: '["tag1","tag2"]'
CREATE TABLE IF NOT EXISTS channels (
id TEXT PRIMARY KEY,
user_id TEXT REFERENCES users(id) ON DELETE CASCADE,
title TEXT NOT NULL,
description TEXT,
type TEXT DEFAULT 'direct'
CHECK (type IN ('direct', 'group', 'channel')),
model TEXT,
system_prompt TEXT,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
is_archived INTEGER DEFAULT 0,
is_pinned INTEGER DEFAULT 0,
folder_id TEXT,
folder TEXT,
team_id TEXT REFERENCES teams(id) ON DELETE SET NULL,
settings TEXT DEFAULT '{}',
tags TEXT DEFAULT '[]',
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_channels_user ON channels(user_id);
CREATE INDEX IF NOT EXISTS idx_channels_updated ON channels(updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_channels_type ON channels(type);
CREATE INDEX IF NOT EXISTS idx_channels_team ON channels(team_id) WHERE team_id IS NOT NULL;
CREATE TRIGGER IF NOT EXISTS channels_updated_at AFTER UPDATE ON channels
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE channels SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 14. MESSAGES
-- =========================================
CREATE TABLE IF NOT EXISTS messages (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
role TEXT NOT NULL
CHECK (role IN ('user', 'assistant', 'system', 'tool')),
content TEXT NOT NULL,
model TEXT,
tokens_used INTEGER,
tool_calls TEXT,
metadata TEXT DEFAULT '{}',
parent_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
sibling_index INTEGER DEFAULT 0,
participant_type TEXT DEFAULT 'user',
participant_id TEXT,
deleted_at TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_messages_channel ON messages(channel_id, created_at);
CREATE INDEX IF NOT EXISTS idx_messages_parent ON messages(parent_id);
CREATE INDEX IF NOT EXISTS idx_messages_alive ON messages(channel_id, created_at)
WHERE deleted_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_messages_parent_alive ON messages(parent_id, sibling_index)
WHERE deleted_at IS NULL;
-- =========================================
-- 15. CHANNEL MEMBERS & MODELS
-- =========================================
CREATE TABLE IF NOT EXISTS channel_members (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role TEXT DEFAULT 'member',
joined_at TEXT DEFAULT (datetime('now')),
last_read_at TEXT DEFAULT (datetime('now')),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_members_channel ON channel_members(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_members_user ON channel_members(user_id);
CREATE TABLE IF NOT EXISTS channel_models (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
display_name TEXT,
system_prompt TEXT,
settings TEXT DEFAULT '{}',
is_default INTEGER DEFAULT 0,
added_at TEXT DEFAULT (datetime('now')),
UNIQUE(channel_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_models_channel ON channel_models(channel_id);
-- =========================================
-- 16. CHANNEL CURSORS
-- =========================================
CREATE TABLE IF NOT EXISTS channel_cursors (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
active_leaf_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_channel ON channel_cursors(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_user ON channel_cursors(user_id);
-- =========================================
-- 17. FOLDERS
-- =========================================
CREATE TABLE IF NOT EXISTS folders (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL,
parent_id TEXT REFERENCES folders(id) ON DELETE CASCADE,
sort_order INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(user_id, name, parent_id)
);
CREATE INDEX IF NOT EXISTS idx_folders_user ON folders(user_id);
CREATE INDEX IF NOT EXISTS idx_folders_parent ON folders(parent_id);
CREATE INDEX IF NOT EXISTS idx_channels_folder ON channels(folder_id) WHERE folder_id IS NOT NULL;
CREATE TRIGGER IF NOT EXISTS folders_updated_at AFTER UPDATE ON folders
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE folders SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 18. NOTES
-- =========================================
-- search_vector and embedding columns omitted (Postgres-only features).
-- Full-text search uses LIKE fallback. Semantic search feature-gated.
-- tags stored as JSON text array.
CREATE TABLE IF NOT EXISTS notes (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
title TEXT NOT NULL,
content TEXT DEFAULT '',
folder_path TEXT DEFAULT '/',
tags TEXT DEFAULT '[]',
metadata TEXT DEFAULT '{}',
source_channel_id TEXT REFERENCES channels(id) ON DELETE SET NULL,
team_id TEXT REFERENCES teams(id) ON DELETE SET NULL,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_notes_user ON notes(user_id);
CREATE INDEX IF NOT EXISTS idx_notes_folder ON notes(user_id, folder_path);
CREATE INDEX IF NOT EXISTS idx_notes_updated ON notes(user_id, updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_notes_team ON notes(team_id) WHERE team_id IS NOT NULL;
CREATE TRIGGER IF NOT EXISTS notes_updated_at AFTER UPDATE ON notes
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE notes SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 19. AUDIT LOG
-- =========================================
CREATE TABLE IF NOT EXISTS audit_log (
id TEXT PRIMARY KEY,
actor_id TEXT REFERENCES users(id) ON DELETE SET NULL,
action TEXT NOT NULL,
resource_type TEXT NOT NULL,
resource_id TEXT,
metadata TEXT DEFAULT '{}',
ip_address TEXT,
user_agent TEXT DEFAULT '',
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_audit_log_created ON audit_log(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_log_actor ON audit_log(actor_id) WHERE actor_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_audit_log_resource ON audit_log(resource_type, resource_id);
CREATE INDEX IF NOT EXISTS idx_audit_log_action ON audit_log(action);
-- =========================================
-- 20. USAGE TRACKING
-- =========================================
CREATE TABLE IF NOT EXISTS usage_log (
id TEXT PRIMARY KEY,
channel_id TEXT REFERENCES channels(id) ON DELETE SET NULL,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
provider_scope TEXT NOT NULL DEFAULT 'global',
model_id TEXT NOT NULL,
role TEXT,
prompt_tokens INTEGER NOT NULL DEFAULT 0,
completion_tokens INTEGER NOT NULL DEFAULT 0,
cache_creation_tokens INTEGER NOT NULL DEFAULT 0,
cache_read_tokens INTEGER NOT NULL DEFAULT 0,
cost_input REAL,
cost_output REAL,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_usage_log_user ON usage_log(user_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_created ON usage_log(created_at);
CREATE INDEX IF NOT EXISTS idx_usage_log_provider ON usage_log(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_model ON usage_log(model_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_scope ON usage_log(provider_scope);
-- =========================================
-- 21. MODEL PRICING
-- =========================================
CREATE TABLE IF NOT EXISTS model_pricing (
id TEXT PRIMARY KEY,
provider_config_id TEXT NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
input_per_m REAL,
output_per_m REAL,
cache_create_per_m REAL,
cache_read_per_m REAL,
currency TEXT NOT NULL DEFAULT 'USD',
source TEXT NOT NULL DEFAULT 'manual',
updated_at TEXT DEFAULT (datetime('now')),
updated_by TEXT REFERENCES users(id),
UNIQUE(provider_config_id, model_id)
);
-- =========================================
-- 22. EXTENSIONS
-- =========================================
CREATE TABLE IF NOT EXISTS extensions (
id TEXT PRIMARY KEY,
ext_id TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
version TEXT NOT NULL DEFAULT '0.0.0',
tier TEXT NOT NULL DEFAULT 'browser',
description TEXT NOT NULL DEFAULT '',
author TEXT NOT NULL DEFAULT '',
manifest TEXT NOT NULL DEFAULT '{}',
is_system INTEGER NOT NULL DEFAULT 0,
is_enabled INTEGER NOT NULL DEFAULT 1,
scope TEXT NOT NULL DEFAULT 'global',
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
installed_by TEXT REFERENCES users(id) ON DELETE SET NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_extensions_tier ON extensions(tier);
CREATE INDEX IF NOT EXISTS idx_extensions_enabled ON extensions(is_enabled)
WHERE is_enabled = 1;
CREATE TABLE IF NOT EXISTS extension_user_settings (
extension_id TEXT NOT NULL REFERENCES extensions(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
settings TEXT NOT NULL DEFAULT '{}',
is_enabled INTEGER NOT NULL DEFAULT 1,
PRIMARY KEY (extension_id, user_id)
);
-- =========================================
-- 23. ATTACHMENTS
-- =========================================
CREATE TABLE IF NOT EXISTS attachments (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id),
message_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
filename TEXT NOT NULL,
content_type TEXT NOT NULL,
size_bytes INTEGER NOT NULL,
storage_key TEXT NOT NULL,
extracted_text TEXT,
metadata TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_attachments_channel ON attachments(channel_id);
CREATE INDEX IF NOT EXISTS idx_attachments_user_size ON attachments(user_id);
CREATE INDEX IF NOT EXISTS idx_attachments_message ON attachments(message_id)
WHERE message_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_attachments_orphan ON attachments(created_at)
WHERE message_id IS NULL;
-- =========================================
-- 24. KNOWLEDGE BASES
-- =========================================
-- Vector columns: kb_chunks.embedding stored as JSON TEXT for app-level cosine similarity.
-- search_vector columns omitted (Postgres-only tsvector feature).
-- KB ingestion works (stores text chunks) but similarity search is
-- feature-gated: requires sqlite-vec extension or returns graceful error.
CREATE TABLE IF NOT EXISTS knowledge_bases (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope TEXT NOT NULL DEFAULT 'global',
owner_id TEXT REFERENCES users(id) ON DELETE CASCADE,
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
embedding_config TEXT NOT NULL DEFAULT '{}',
document_count INTEGER NOT NULL DEFAULT 0,
chunk_count INTEGER NOT NULL DEFAULT 0,
total_bytes INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'active',
discoverable INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
CONSTRAINT kb_scope_check CHECK (
(scope = 'global' AND owner_id IS NULL) OR
(scope = 'team' AND team_id IS NOT NULL) OR
(scope = 'personal' AND owner_id IS NOT NULL)
)
);
CREATE INDEX IF NOT EXISTS idx_kb_scope ON knowledge_bases(scope);
CREATE INDEX IF NOT EXISTS idx_kb_owner ON knowledge_bases(owner_id) WHERE owner_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_kb_team ON knowledge_bases(team_id) WHERE team_id IS NOT NULL;
CREATE TABLE IF NOT EXISTS kb_documents (
id TEXT PRIMARY KEY,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
filename TEXT NOT NULL,
content_type TEXT NOT NULL,
size_bytes INTEGER NOT NULL,
storage_key TEXT NOT NULL,
extracted_text TEXT,
chunk_count INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'pending',
error TEXT,
uploaded_by TEXT NOT NULL REFERENCES users(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_kbdoc_kb ON kb_documents(kb_id);
CREATE TABLE IF NOT EXISTS kb_chunks (
id TEXT PRIMARY KEY,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
document_id TEXT NOT NULL REFERENCES kb_documents(id) ON DELETE CASCADE,
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
token_count INTEGER NOT NULL DEFAULT 0,
embedding TEXT, -- JSON array of float64 for app-level cosine similarity
metadata TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_kbchunk_kb ON kb_chunks(kb_id);
CREATE INDEX IF NOT EXISTS idx_kbchunk_doc ON kb_chunks(document_id);
CREATE TABLE IF NOT EXISTS channel_knowledge_bases (
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
enabled INTEGER NOT NULL DEFAULT 1,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (channel_id, kb_id)
);
-- =========================================
-- 25. PERSONA-KB BINDING (v0.17.0)
-- =========================================
CREATE TABLE IF NOT EXISTS persona_knowledge_bases (
persona_id TEXT NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
auto_search INTEGER NOT NULL DEFAULT 0,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (persona_id, kb_id)
);
CREATE INDEX IF NOT EXISTS idx_persona_kb_persona ON persona_knowledge_bases(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_kb_kb ON persona_knowledge_bases(kb_id);
-- v0.17.0 policy
INSERT OR IGNORE INTO platform_policies (key, value) VALUES
('kb_direct_access', 'true');

59
server/database/migrations/sqlite/002_teams.sql Normal file
View File

@@ -0,0 +1,59 @@
-- Chat Switchboard — 002 Teams & Access Control (SQLite)
CREATE TABLE IF NOT EXISTS teams (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
description TEXT DEFAULT '',
created_by TEXT NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
is_active INTEGER DEFAULT 1,
settings TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_teams_active ON teams(is_active);
CREATE TABLE IF NOT EXISTS team_members (
id TEXT PRIMARY KEY,
team_id TEXT NOT NULL REFERENCES teams(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role TEXT NOT NULL DEFAULT 'member'
CHECK (role IN ('admin', 'member')),
joined_at TEXT DEFAULT (datetime('now')),
UNIQUE(team_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_team_members_team ON team_members(team_id);
CREATE INDEX IF NOT EXISTS idx_team_members_user ON team_members(user_id);
CREATE TABLE IF NOT EXISTS groups (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope TEXT NOT NULL DEFAULT 'global'
CHECK (scope IN ('global', 'team')),
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
created_by TEXT NOT NULL REFERENCES users(id),
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
CHECK (
(scope = 'global' AND team_id IS NULL) OR
(scope = 'team' AND team_id IS NOT NULL)
)
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_groups_name_scope
ON groups(name, COALESCE(team_id, ''));
CREATE TABLE IF NOT EXISTS group_members (
id TEXT PRIMARY KEY,
group_id TEXT NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
added_by TEXT NOT NULL REFERENCES users(id),
added_at TEXT DEFAULT (datetime('now')),
UNIQUE(group_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_group_members_group ON group_members(group_id);
CREATE INDEX IF NOT EXISTS idx_group_members_user ON group_members(user_id);

26
server/database/migrations/sqlite/002_v0173_note_links.sql
View File

@@ -1,26 +0,0 @@
-- v0.17.3: Note Links + Wikilink Infrastructure (SQLite)
--
-- New: note_links table for [[wikilink]] graph edges
-- New: notes.source_message_id for chat-to-note provenance
-- =========================================
-- 1. Note Links
-- =========================================
CREATE TABLE IF NOT EXISTS note_links (
source_note_id TEXT NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
target_note_id TEXT REFERENCES notes(id) ON DELETE CASCADE,
target_title TEXT NOT NULL,
display_text TEXT,
is_transclusion INTEGER NOT NULL DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
PRIMARY KEY (source_note_id, target_title)
);
CREATE INDEX IF NOT EXISTS idx_note_links_target ON note_links(target_note_id);
-- =========================================
-- 2. Source Message Provenance
-- =========================================
ALTER TABLE notes ADD COLUMN source_message_id TEXT;

106
server/database/migrations/sqlite/003_providers.sql Normal file
View File

@@ -0,0 +1,106 @@
-- Chat Switchboard — 003 Providers & Routing (SQLite)
CREATE TABLE IF NOT EXISTS provider_configs (
id TEXT PRIMARY KEY,
scope TEXT NOT NULL CHECK (scope IN ('global', 'team', 'personal')),
owner_id TEXT,
name TEXT NOT NULL,
provider TEXT NOT NULL,
endpoint TEXT NOT NULL,
api_key_enc BLOB,
key_nonce BLOB,
key_scope TEXT NOT NULL DEFAULT 'global',
model_default TEXT,
config TEXT DEFAULT '{}',
headers TEXT DEFAULT '{}',
settings TEXT DEFAULT '{}',
is_active INTEGER DEFAULT 1,
is_private INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_provider_configs_scope ON provider_configs(scope);
CREATE INDEX IF NOT EXISTS idx_provider_configs_owner ON provider_configs(owner_id);
CREATE INDEX IF NOT EXISTS idx_provider_configs_active ON provider_configs(is_active);
CREATE TABLE IF NOT EXISTS model_catalog (
id TEXT PRIMARY KEY,
provider_config_id TEXT NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
display_name TEXT,
model_type TEXT DEFAULT 'chat',
capabilities TEXT NOT NULL DEFAULT '{}',
pricing TEXT,
visibility TEXT DEFAULT 'disabled' CHECK (visibility IN ('enabled', 'disabled', 'team')),
last_synced_at TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(provider_config_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_model_catalog_provider ON model_catalog(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_model_catalog_enabled ON model_catalog(visibility);
CREATE INDEX IF NOT EXISTS idx_model_catalog_type ON model_catalog(model_type);
CREATE TABLE IF NOT EXISTS model_pricing (
id TEXT PRIMARY KEY,
provider_config_id TEXT NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
input_per_m REAL,
output_per_m REAL,
cache_create_per_m REAL,
cache_read_per_m REAL,
currency TEXT NOT NULL DEFAULT 'USD',
source TEXT NOT NULL DEFAULT 'manual',
updated_at TEXT DEFAULT (datetime('now')),
updated_by TEXT REFERENCES users(id),
UNIQUE(provider_config_id, model_id)
);
CREATE TABLE IF NOT EXISTS provider_health (
id TEXT PRIMARY KEY,
provider_config_id TEXT NOT NULL REFERENCES provider_configs(id) ON DELETE CASCADE,
window_start TEXT NOT NULL,
request_count INTEGER NOT NULL DEFAULT 0,
error_count INTEGER NOT NULL DEFAULT 0,
timeout_count INTEGER NOT NULL DEFAULT 0,
total_latency_ms INTEGER NOT NULL DEFAULT 0,
max_latency_ms INTEGER NOT NULL DEFAULT 0,
rate_limit_count INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
last_error_at TEXT,
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (provider_config_id, window_start)
);
CREATE INDEX IF NOT EXISTS idx_provider_health_window ON provider_health(provider_config_id, window_start);
CREATE TABLE IF NOT EXISTS capability_overrides (
id TEXT PRIMARY KEY,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
field TEXT NOT NULL,
value TEXT NOT NULL,
set_by TEXT REFERENCES users(id) ON DELETE SET NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (provider_config_id, model_id, field)
);
CREATE INDEX IF NOT EXISTS idx_capability_overrides_model ON capability_overrides(model_id);
CREATE TABLE IF NOT EXISTS routing_policies (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
scope TEXT NOT NULL DEFAULT 'global' CHECK (scope IN ('global', 'team')),
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
priority INTEGER NOT NULL DEFAULT 100,
policy_type TEXT NOT NULL CHECK (policy_type IN ('provider_prefer', 'team_route', 'cost_limit', 'model_alias')),
config TEXT NOT NULL DEFAULT '{}',
is_active INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_routing_policies_active ON routing_policies(is_active, priority);
CREATE INDEX IF NOT EXISTS idx_routing_policies_team ON routing_policies(team_id);

61
server/database/migrations/sqlite/003_v0180_memories.sql
View File

@@ -1,61 +0,0 @@
-- v0.18.0: Memory System (SQLite)
--
-- Long-term memory across conversations, with scope-aware isolation.
-- Three scopes: user (personal), persona (shared), persona_user (per-user-per-persona).
-- =========================================
-- 1. Memories Table
-- =========================================
CREATE TABLE IF NOT EXISTS memories (
id TEXT PRIMARY KEY,
scope TEXT NOT NULL CHECK (scope IN ('user', 'persona', 'persona_user')),
-- Owner resolution:
-- user scope: owner_id = user_id, user_id = NULL
-- persona scope: owner_id = persona_id, user_id = NULL
-- persona_user scope: owner_id = persona_id, user_id = user_id
owner_id TEXT NOT NULL,
user_id TEXT,
key TEXT NOT NULL,
value TEXT NOT NULL,
source_channel_id TEXT,
confidence REAL NOT NULL DEFAULT 1.0,
status TEXT NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'pending_review', 'archived')),
-- Embedding stored as JSON array of float64 for app-level cosine similarity
embedding TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
-- Primary lookup
CREATE INDEX IF NOT EXISTS idx_memories_scope_owner
ON memories(scope, owner_id, status);
-- Persona+user lookup
CREATE INDEX IF NOT EXISTS idx_memories_persona_user
ON memories(owner_id, user_id);
-- Deduplication: prevent saving the same key twice for the same scope/owner/user
CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_unique_key
ON memories(scope, owner_id, COALESCE(user_id, '00000000-0000-0000-0000-000000000000'), key);
-- Source channel reference
CREATE INDEX IF NOT EXISTS idx_memories_source_channel
ON memories(source_channel_id);
-- =========================================
-- 2. Updated-at Trigger
-- =========================================
CREATE TRIGGER IF NOT EXISTS trg_memories_updated_at
AFTER UPDATE ON memories
FOR EACH ROW
WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE memories SET updated_at = datetime('now') WHERE id = NEW.id;
END;

57
server/database/migrations/sqlite/004_personas.sql Normal file
View File

@@ -0,0 +1,57 @@
-- Chat Switchboard — 004 Personas & Grants (SQLite)
CREATE TABLE IF NOT EXISTS personas (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT DEFAULT '',
icon TEXT DEFAULT '',
avatar TEXT DEFAULT '',
base_model_id TEXT NOT NULL,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
system_prompt TEXT DEFAULT '',
temperature REAL,
max_tokens INTEGER,
thinking_budget INTEGER,
top_p REAL,
memory_enabled INTEGER NOT NULL DEFAULT 1,
memory_extraction_prompt TEXT,
scope TEXT NOT NULL CHECK (scope IN ('global', 'team', 'personal')),
owner_id TEXT,
created_by TEXT NOT NULL REFERENCES users(id),
is_active INTEGER DEFAULT 1,
is_shared INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_personas_scope ON personas(scope);
CREATE INDEX IF NOT EXISTS idx_personas_owner ON personas(owner_id);
CREATE INDEX IF NOT EXISTS idx_personas_active ON personas(is_active);
CREATE TABLE IF NOT EXISTS persona_grants (
id TEXT PRIMARY KEY,
persona_id TEXT NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
grant_type TEXT NOT NULL,
grant_ref TEXT NOT NULL,
config TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')),
UNIQUE(persona_id, grant_type, grant_ref)
);
CREATE INDEX IF NOT EXISTS idx_persona_grants_persona ON persona_grants(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_grants_type ON persona_grants(grant_type);
CREATE TABLE IF NOT EXISTS resource_grants (
id TEXT PRIMARY KEY,
resource_type TEXT NOT NULL CHECK (resource_type IN ('persona', 'knowledge_base', 'project')),
resource_id TEXT NOT NULL,
grant_scope TEXT NOT NULL DEFAULT 'team_only'
CHECK (grant_scope IN ('team_only', 'global', 'groups')),
granted_groups TEXT NOT NULL DEFAULT '[]',
created_by TEXT NOT NULL REFERENCES users(id),
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(resource_type, resource_id)
);
CREATE INDEX IF NOT EXISTS idx_resource_grants_resource ON resource_grants(resource_type, resource_id);

22
server/database/migrations/sqlite/004_v0180_memory_phase2.sql
View File

@@ -1,22 +0,0 @@
-- v0.18.0 Phase 2: Memory Extraction & Embeddings (SQLite)
-- 1. Persona memory configuration
ALTER TABLE personas ADD COLUMN memory_enabled INTEGER NOT NULL DEFAULT 1;
ALTER TABLE personas ADD COLUMN memory_extraction_prompt TEXT;
-- 2. No HNSW index for SQLite — app-level cosine similarity used instead.
-- 3. Memory extraction log
CREATE TABLE IF NOT EXISTS memory_extraction_log (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL,
user_id TEXT NOT NULL,
last_message_id TEXT NOT NULL,
extracted_at TEXT DEFAULT (datetime('now')),
memory_count INTEGER NOT NULL DEFAULT 0,
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_mem_extract_log_channel
ON memory_extraction_log(channel_id);

120
server/database/migrations/sqlite/005_channels.sql Normal file
View File

@@ -0,0 +1,120 @@
-- Chat Switchboard — 005 Channels & Conversations (SQLite)
CREATE TABLE IF NOT EXISTS folders (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL,
parent_id TEXT REFERENCES folders(id) ON DELETE CASCADE,
sort_order INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(user_id, name, parent_id)
);
CREATE INDEX IF NOT EXISTS idx_folders_user ON folders(user_id);
CREATE INDEX IF NOT EXISTS idx_folders_parent ON folders(parent_id);
CREATE TABLE IF NOT EXISTS channels (
id TEXT PRIMARY KEY,
user_id TEXT REFERENCES users(id) ON DELETE CASCADE,
title TEXT NOT NULL,
description TEXT,
type TEXT DEFAULT 'direct' CHECK (type IN ('direct', 'group', 'channel')),
model TEXT,
system_prompt TEXT,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
is_archived INTEGER DEFAULT 0,
is_pinned INTEGER DEFAULT 0,
folder_id TEXT REFERENCES folders(id) ON DELETE SET NULL,
folder TEXT,
team_id TEXT REFERENCES teams(id) ON DELETE SET NULL,
settings TEXT DEFAULT '{}',
tags TEXT DEFAULT '[]',
project_id TEXT,
workspace_id TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_channels_user ON channels(user_id);
CREATE INDEX IF NOT EXISTS idx_channels_updated ON channels(updated_at);
CREATE INDEX IF NOT EXISTS idx_channels_type ON channels(type);
CREATE INDEX IF NOT EXISTS idx_channels_team ON channels(team_id);
CREATE INDEX IF NOT EXISTS idx_channels_folder ON channels(folder_id);
CREATE INDEX IF NOT EXISTS idx_channels_project ON channels(project_id);
CREATE INDEX IF NOT EXISTS idx_channels_workspace ON channels(workspace_id);
CREATE TABLE IF NOT EXISTS messages (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
role TEXT NOT NULL CHECK (role IN ('user', 'assistant', 'system', 'tool')),
content TEXT NOT NULL,
model TEXT,
tokens_used INTEGER,
tool_calls TEXT,
metadata TEXT DEFAULT '{}',
parent_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
sibling_index INTEGER DEFAULT 0,
participant_type TEXT DEFAULT 'user',
participant_id TEXT,
deleted_at TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_messages_channel ON messages(channel_id, created_at);
CREATE INDEX IF NOT EXISTS idx_messages_parent ON messages(parent_id);
CREATE TABLE IF NOT EXISTS channel_members (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role TEXT DEFAULT 'member',
joined_at TEXT DEFAULT (datetime('now')),
last_read_at TEXT DEFAULT (datetime('now')),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_members_channel ON channel_members(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_members_user ON channel_members(user_id);
CREATE TABLE IF NOT EXISTS channel_models (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
display_name TEXT,
system_prompt TEXT,
settings TEXT DEFAULT '{}',
is_default INTEGER DEFAULT 0,
added_at TEXT DEFAULT (datetime('now')),
UNIQUE(channel_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_models_channel ON channel_models(channel_id);
CREATE TABLE IF NOT EXISTS channel_cursors (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
active_leaf_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_channel ON channel_cursors(channel_id);
CREATE INDEX IF NOT EXISTS idx_channel_cursors_user ON channel_cursors(user_id);
CREATE TABLE IF NOT EXISTS user_model_settings (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
hidden INTEGER DEFAULT 0,
preferred_temperature REAL,
preferred_max_tokens INTEGER,
sort_order INTEGER DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(user_id, model_id)
);
CREATE INDEX IF NOT EXISTS idx_user_model_settings_user ON user_model_settings(user_id);

131
server/database/migrations/sqlite/005_v0190_projects.sql
View File

@@ -1,131 +0,0 @@
-- v0.19.0: Projects / Workspaces (SQLite)
-- =========================================
-- 1. Projects Table
-- =========================================
CREATE TABLE IF NOT EXISTS projects (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
color TEXT,
icon TEXT,
scope TEXT NOT NULL DEFAULT 'personal'
CHECK (scope IN ('personal', 'team', 'global')),
owner_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
team_id TEXT REFERENCES teams(id) ON DELETE SET NULL,
is_archived INTEGER NOT NULL DEFAULT 0,
settings TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_projects_owner ON projects(owner_id);
CREATE INDEX IF NOT EXISTS idx_projects_team ON projects(team_id);
CREATE INDEX IF NOT EXISTS idx_projects_scope ON projects(scope);
CREATE TRIGGER IF NOT EXISTS projects_updated_at AFTER UPDATE ON projects
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE projects SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 2. Project ↔ Channel Junction
-- =========================================
CREATE TABLE IF NOT EXISTS project_channels (
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
position INTEGER NOT NULL DEFAULT 0,
folder TEXT,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (project_id, channel_id),
UNIQUE (channel_id)
);
CREATE INDEX IF NOT EXISTS idx_project_channels_project ON project_channels(project_id);
CREATE INDEX IF NOT EXISTS idx_project_channels_channel ON project_channels(channel_id);
-- =========================================
-- 3. Project ↔ Knowledge Base Junction
-- =========================================
CREATE TABLE IF NOT EXISTS project_knowledge_bases (
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
auto_search INTEGER NOT NULL DEFAULT 0,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (project_id, kb_id)
);
CREATE INDEX IF NOT EXISTS idx_project_kb_project ON project_knowledge_bases(project_id);
CREATE INDEX IF NOT EXISTS idx_project_kb_kb ON project_knowledge_bases(kb_id);
-- =========================================
-- 4. Project ↔ Note Junction
-- =========================================
CREATE TABLE IF NOT EXISTS project_notes (
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
note_id TEXT NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (project_id, note_id)
);
CREATE INDEX IF NOT EXISTS idx_project_notes_project ON project_notes(project_id);
CREATE INDEX IF NOT EXISTS idx_project_notes_note ON project_notes(note_id);
-- =========================================
-- 5. Denormalized project_id on Channels
-- =========================================
ALTER TABLE channels ADD COLUMN project_id TEXT REFERENCES projects(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_channels_project ON channels(project_id);
-- =========================================
-- 6. Extend resource_grants CHECK
-- =========================================
-- SQLite cannot ALTER CHECK constraints. The original CREATE TABLE
-- defined CHECK(resource_type IN ('persona','knowledge_base')).
-- SQLite does not enforce CHECK on existing rows, and new inserts
-- with 'project' will be accepted only if we recreate the table.
-- For pragmatism: we accept 'project' values via application-level
-- validation and leave the SQLite CHECK as-is (it's advisory in
-- SQLite when PRAGMA ignore_check_constraints is implicitly off
-- for existing data, but CHECK IS enforced on INSERT).
--
-- Workaround: recreate the table with the new CHECK. This is safe
-- because resource_grants typically has very few rows.
CREATE TABLE IF NOT EXISTS resource_grants_new (
id TEXT PRIMARY KEY,
resource_type TEXT NOT NULL
CHECK (resource_type IN ('persona', 'knowledge_base', 'project')),
resource_id TEXT NOT NULL,
grant_scope TEXT NOT NULL DEFAULT 'team_only'
CHECK (grant_scope IN ('team_only', 'global', 'groups')),
granted_groups TEXT NOT NULL DEFAULT '[]',
created_by TEXT NOT NULL REFERENCES users(id),
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now')),
UNIQUE(resource_type, resource_id)
);
INSERT OR IGNORE INTO resource_grants_new
SELECT id, resource_type, resource_id, grant_scope, granted_groups,
created_by, created_at, updated_at
FROM resource_grants;
DROP TABLE IF EXISTS resource_grants;
ALTER TABLE resource_grants_new RENAME TO resource_grants;
CREATE INDEX IF NOT EXISTS idx_resource_grants_resource
ON resource_grants(resource_type, resource_id);
CREATE TRIGGER IF NOT EXISTS resource_grants_updated_at AFTER UPDATE ON resource_grants
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE resource_grants SET updated_at = datetime('now') WHERE id = NEW.id;
END;

73
server/database/migrations/sqlite/006_knowledge.sql Normal file
View File

@@ -0,0 +1,73 @@
-- Chat Switchboard — 006 Knowledge Bases (SQLite)
CREATE TABLE IF NOT EXISTS knowledge_bases (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
scope TEXT NOT NULL DEFAULT 'global',
owner_id TEXT REFERENCES users(id) ON DELETE CASCADE,
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
embedding_config TEXT NOT NULL DEFAULT '{}',
document_count INTEGER NOT NULL DEFAULT 0,
chunk_count INTEGER NOT NULL DEFAULT 0,
total_bytes INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'active',
discoverable INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_kb_scope ON knowledge_bases(scope);
CREATE INDEX IF NOT EXISTS idx_kb_owner ON knowledge_bases(owner_id);
CREATE INDEX IF NOT EXISTS idx_kb_team ON knowledge_bases(team_id);
CREATE TABLE IF NOT EXISTS kb_documents (
id TEXT PRIMARY KEY,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
filename TEXT NOT NULL,
content_type TEXT NOT NULL,
size_bytes INTEGER NOT NULL,
storage_key TEXT NOT NULL,
extracted_text TEXT,
chunk_count INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'pending',
error TEXT,
uploaded_by TEXT NOT NULL REFERENCES users(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_kbdoc_kb ON kb_documents(kb_id);
CREATE TABLE IF NOT EXISTS kb_chunks (
id TEXT PRIMARY KEY,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
document_id TEXT NOT NULL REFERENCES kb_documents(id) ON DELETE CASCADE,
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
token_count INTEGER NOT NULL DEFAULT 0,
metadata TEXT NOT NULL DEFAULT '{}',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_kbchunk_kb ON kb_chunks(kb_id);
CREATE INDEX IF NOT EXISTS idx_kbchunk_doc ON kb_chunks(document_id);
CREATE TABLE IF NOT EXISTS channel_knowledge_bases (
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
enabled INTEGER NOT NULL DEFAULT 1,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (channel_id, kb_id)
);
CREATE TABLE IF NOT EXISTS persona_knowledge_bases (
persona_id TEXT NOT NULL REFERENCES personas(id) ON DELETE CASCADE,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
auto_search INTEGER NOT NULL DEFAULT 0,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (persona_id, kb_id)
);
CREATE INDEX IF NOT EXISTS idx_persona_kb_persona ON persona_knowledge_bases(persona_id);
CREATE INDEX IF NOT EXISTS idx_persona_kb_kb ON persona_knowledge_bases(kb_id);

34
server/database/migrations/sqlite/007_notes.sql Normal file
View File

@@ -0,0 +1,34 @@
-- Chat Switchboard — 007 Notes & Links (SQLite)
CREATE TABLE IF NOT EXISTS notes (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
title TEXT NOT NULL,
content TEXT DEFAULT '',
folder_path TEXT DEFAULT '/',
tags TEXT DEFAULT '[]',
metadata TEXT DEFAULT '{}',
source_channel_id TEXT REFERENCES channels(id) ON DELETE SET NULL,
source_message_id TEXT,
team_id TEXT REFERENCES teams(id) ON DELETE SET NULL,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_notes_user ON notes(user_id);
CREATE INDEX IF NOT EXISTS idx_notes_folder ON notes(user_id, folder_path);
CREATE INDEX IF NOT EXISTS idx_notes_updated ON notes(user_id, updated_at);
CREATE INDEX IF NOT EXISTS idx_notes_team ON notes(team_id);
CREATE INDEX IF NOT EXISTS idx_notes_source_msg ON notes(source_message_id);
CREATE TABLE IF NOT EXISTS note_links (
source_note_id TEXT NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
target_note_id TEXT REFERENCES notes(id) ON DELETE CASCADE,
target_title TEXT NOT NULL,
display_text TEXT,
is_transclusion INTEGER NOT NULL DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
PRIMARY KEY (source_note_id, target_title)
);
CREATE INDEX IF NOT EXISTS idx_note_links_target ON note_links(target_note_id);

13
server/database/migrations/sqlite/007_v0200_notification_prefs.sql
View File

@@ -1,13 +0,0 @@
-- 007_v0200_notification_prefs.sql
-- Notification preferences per user per type (SQLite dialect).
CREATE TABLE IF NOT EXISTS notification_preferences (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type TEXT NOT NULL,
in_app INTEGER NOT NULL DEFAULT 1,
email INTEGER NOT NULL DEFAULT 0,
UNIQUE(user_id, type)
);
CREATE INDEX IF NOT EXISTS idx_notification_prefs_user ON notification_preferences(user_id);

34
server/database/migrations/sqlite/008_memory.sql Normal file
View File

@@ -0,0 +1,34 @@
-- Chat Switchboard — 008 Memory (SQLite)
CREATE TABLE IF NOT EXISTS memories (
id TEXT PRIMARY KEY,
scope TEXT NOT NULL CHECK (scope IN ('user', 'persona', 'persona_user')),
owner_id TEXT NOT NULL,
user_id TEXT,
key TEXT NOT NULL,
value TEXT NOT NULL,
source_channel_id TEXT,
confidence REAL NOT NULL DEFAULT 1.0,
status TEXT NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'pending_review', 'archived')),
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_memories_scope_owner ON memories(scope, owner_id, status);
CREATE INDEX IF NOT EXISTS idx_memories_persona_user ON memories(owner_id, user_id);
CREATE UNIQUE INDEX IF NOT EXISTS idx_memories_unique_key
ON memories(scope, owner_id, COALESCE(user_id, '00000000-0000-0000-0000-000000000000'), key);
CREATE INDEX IF NOT EXISTS idx_memories_source_channel ON memories(source_channel_id);
CREATE TABLE IF NOT EXISTS memory_extraction_log (
id TEXT PRIMARY KEY,
channel_id TEXT NOT NULL,
user_id TEXT NOT NULL,
last_message_id TEXT NOT NULL,
extracted_at TEXT DEFAULT (datetime('now')),
memory_count INTEGER NOT NULL DEFAULT 0,
UNIQUE(channel_id, user_id)
);
CREATE INDEX IF NOT EXISTS idx_mem_extract_log_channel ON memory_extraction_log(channel_id);

62
server/database/migrations/sqlite/008_v0210_workspaces.sql
View File

@@ -1,62 +0,0 @@
-- v0.21.0: Workspaces (platform storage primitive) — SQLite
-- =========================================
-- 1. Workspaces Table
-- =========================================
CREATE TABLE IF NOT EXISTS workspaces (
id TEXT PRIMARY KEY,
owner_type TEXT NOT NULL
CHECK (owner_type IN ('user', 'project', 'channel', 'team')),
owner_id TEXT NOT NULL,
name TEXT NOT NULL,
root_path TEXT NOT NULL,
max_bytes INTEGER,
status TEXT NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'archived', 'deleting')),
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_workspaces_owner
ON workspaces(owner_type, owner_id);
CREATE INDEX IF NOT EXISTS idx_workspaces_status
ON workspaces(status);
CREATE TRIGGER IF NOT EXISTS workspaces_updated_at AFTER UPDATE ON workspaces
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE workspaces SET updated_at = datetime('now') WHERE id = NEW.id;
END;
-- =========================================
-- 2. Workspace Files Table
-- =========================================
CREATE TABLE IF NOT EXISTS workspace_files (
id TEXT PRIMARY KEY,
workspace_id TEXT NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
path TEXT NOT NULL,
is_directory INTEGER NOT NULL DEFAULT 0,
content_type TEXT,
size_bytes INTEGER NOT NULL DEFAULT 0,
sha256 TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_workspace_files_path
ON workspace_files(workspace_id, path);
CREATE INDEX IF NOT EXISTS idx_workspace_files_workspace
ON workspace_files(workspace_id);
CREATE INDEX IF NOT EXISTS idx_workspace_files_type
ON workspace_files(workspace_id, content_type);
CREATE TRIGGER IF NOT EXISTS workspace_files_updated_at AFTER UPDATE ON workspace_files
FOR EACH ROW WHEN NEW.updated_at = OLD.updated_at
BEGIN
UPDATE workspace_files SET updated_at = datetime('now') WHERE id = NEW.id;
END;

9
server/database/migrations/sqlite/009_v0211_workspace_bindings.sql
View File

@@ -1,9 +0,0 @@
-- v0.21.1: Workspace bindings for channels and projects (SQLite)
ALTER TABLE channels ADD COLUMN workspace_id TEXT REFERENCES workspaces(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_channels_workspace ON channels(workspace_id);
ALTER TABLE projects ADD COLUMN workspace_id TEXT REFERENCES workspaces(id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_projects_workspace ON projects(workspace_id);

66
server/database/migrations/sqlite/009_workspaces.sql Normal file
View File

@@ -0,0 +1,66 @@
-- Chat Switchboard — 009 Workspaces & Git (SQLite)
CREATE TABLE IF NOT EXISTS workspaces (
id TEXT PRIMARY KEY,
owner_type TEXT NOT NULL CHECK (owner_type IN ('user', 'project', 'channel', 'team')),
owner_id TEXT NOT NULL,
name TEXT NOT NULL,
root_path TEXT NOT NULL,
max_bytes INTEGER,
status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'archived', 'deleting')),
indexing_enabled INTEGER NOT NULL DEFAULT 1,
git_remote_url TEXT,
git_branch TEXT,
git_credential_id TEXT,
git_last_sync TEXT,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_workspaces_owner ON workspaces(owner_type, owner_id);
CREATE INDEX IF NOT EXISTS idx_workspaces_status ON workspaces(status);
CREATE TABLE IF NOT EXISTS workspace_files (
id TEXT PRIMARY KEY,
workspace_id TEXT NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
path TEXT NOT NULL,
is_directory INTEGER NOT NULL DEFAULT 0,
content_type TEXT,
size_bytes INTEGER NOT NULL DEFAULT 0,
sha256 TEXT,
index_status TEXT DEFAULT 'pending' CHECK (index_status IN ('pending', 'indexing', 'ready', 'error', 'skipped')),
chunk_count INTEGER NOT NULL DEFAULT 0,
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_workspace_files_path ON workspace_files(workspace_id, path);
CREATE INDEX IF NOT EXISTS idx_workspace_files_workspace ON workspace_files(workspace_id);
CREATE INDEX IF NOT EXISTS idx_workspace_files_type ON workspace_files(workspace_id, content_type);
CREATE INDEX IF NOT EXISTS idx_workspace_files_index_status ON workspace_files(workspace_id, index_status);
CREATE TABLE IF NOT EXISTS workspace_chunks (
id TEXT PRIMARY KEY,
workspace_id TEXT NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
file_id TEXT NOT NULL REFERENCES workspace_files(id) ON DELETE CASCADE,
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
token_count INTEGER NOT NULL DEFAULT 0,
metadata TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_file ON workspace_chunks(file_id);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_workspace ON workspace_chunks(workspace_id);
CREATE TABLE IF NOT EXISTS git_credentials (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL DEFAULT '',
auth_type TEXT NOT NULL DEFAULT 'https_pat',
encrypted_data BLOB NOT NULL DEFAULT '',
nonce BLOB NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_git_credentials_user ON git_credentials(user_id);

62
server/database/migrations/sqlite/010_projects.sql Normal file
View File

@@ -0,0 +1,62 @@
-- Chat Switchboard — 010 Projects (SQLite)
CREATE TABLE IF NOT EXISTS projects (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
color TEXT,
icon TEXT,
scope TEXT NOT NULL DEFAULT 'personal' CHECK (scope IN ('personal', 'team', 'global')),
owner_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
team_id TEXT REFERENCES teams(id) ON DELETE SET NULL,
workspace_id TEXT REFERENCES workspaces(id) ON DELETE SET NULL,
is_archived INTEGER NOT NULL DEFAULT 0,
settings TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_projects_owner ON projects(owner_id);
CREATE INDEX IF NOT EXISTS idx_projects_team ON projects(team_id);
CREATE INDEX IF NOT EXISTS idx_projects_scope ON projects(scope);
CREATE INDEX IF NOT EXISTS idx_projects_workspace ON projects(workspace_id);
CREATE TABLE IF NOT EXISTS project_channels (
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
channel_id TEXT NOT NULL REFERENCES channels(id) ON DELETE CASCADE,
position INTEGER NOT NULL DEFAULT 0,
folder TEXT,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (project_id, channel_id),
UNIQUE (channel_id)
);
CREATE INDEX IF NOT EXISTS idx_project_channels_project ON project_channels(project_id);
CREATE INDEX IF NOT EXISTS idx_project_channels_channel ON project_channels(channel_id);
CREATE TABLE IF NOT EXISTS project_knowledge_bases (
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
kb_id TEXT NOT NULL REFERENCES knowledge_bases(id) ON DELETE CASCADE,
auto_search INTEGER NOT NULL DEFAULT 0,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (project_id, kb_id)
);
CREATE INDEX IF NOT EXISTS idx_project_kb_project ON project_knowledge_bases(project_id);
CREATE INDEX IF NOT EXISTS idx_project_kb_kb ON project_knowledge_bases(kb_id);
CREATE TABLE IF NOT EXISTS project_notes (
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
note_id TEXT NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
added_at TEXT NOT NULL DEFAULT (datetime('now')),
PRIMARY KEY (project_id, note_id)
);
CREATE INDEX IF NOT EXISTS idx_project_notes_project ON project_notes(project_id);
CREATE INDEX IF NOT EXISTS idx_project_notes_note ON project_notes(note_id);
-- Cross-domain FK columns on channels (SQLite cannot ADD COLUMN with FK in all cases,
-- so these may already exist from initial channel creation — safe to skip on error)
-- ALTER TABLE channels ADD COLUMN project_id TEXT REFERENCES projects(id) ON DELETE SET NULL;
-- ALTER TABLE channels ADD COLUMN workspace_id TEXT REFERENCES workspaces(id) ON DELETE SET NULL;
-- Note: SQLite ADD COLUMN IF NOT EXISTS not supported; channels table created with these columns.

31
server/database/migrations/sqlite/010_v0212_workspace_chunks.sql
View File

@@ -1,31 +0,0 @@
-- v0.21.2: Workspace indexing + semantic search (SQLite)
-- workspace_files additions
ALTER TABLE workspace_files ADD COLUMN index_status TEXT DEFAULT 'pending';
ALTER TABLE workspace_files ADD COLUMN chunk_count INTEGER NOT NULL DEFAULT 0;
CREATE INDEX IF NOT EXISTS idx_workspace_files_index_status
ON workspace_files(workspace_id, index_status);
-- workspaces addition
ALTER TABLE workspaces ADD COLUMN indexing_enabled INTEGER NOT NULL DEFAULT 1;
-- workspace_chunks table
-- Embeddings stored as JSON text for app-level cosine similarity.
CREATE TABLE IF NOT EXISTS workspace_chunks (
id TEXT PRIMARY KEY,
workspace_id TEXT NOT NULL REFERENCES workspaces(id) ON DELETE CASCADE,
file_id TEXT NOT NULL REFERENCES workspace_files(id) ON DELETE CASCADE,
chunk_index INTEGER NOT NULL,
content TEXT NOT NULL,
token_count INTEGER NOT NULL DEFAULT 0,
embedding TEXT, -- JSON array of floats
metadata TEXT DEFAULT '{}', -- JSON object
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_file
ON workspace_chunks(file_id);
CREATE INDEX IF NOT EXISTS idx_workspace_chunks_workspace
ON workspace_chunks(workspace_id);

18
server/database/migrations/sqlite/006_v0200_notifications.sql → server/database/migrations/sqlite/011_notifications.sql
View File

@@ -1,5 +1,4 @@
-- 006_v0200_notifications.sql
-- Notification infrastructure (v0.20.0 Phase 1)
-- Chat Switchboard — 011 Notifications (SQLite)
CREATE TABLE IF NOT EXISTS notifications (
id TEXT PRIMARY KEY,
@@ -13,5 +12,16 @@ CREATE TABLE IF NOT EXISTS notifications (
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_notifications_user_unread ON notifications(user_id, is_read, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_notifications_user_created ON notifications(user_id, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_notifications_user_unread ON notifications(user_id, is_read, created_at);
CREATE INDEX IF NOT EXISTS idx_notifications_user_created ON notifications(user_id, created_at);
CREATE TABLE IF NOT EXISTS notification_preferences (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type TEXT NOT NULL,
in_app INTEGER NOT NULL DEFAULT 1,
email INTEGER NOT NULL DEFAULT 0,
UNIQUE(user_id, type)
);
CREATE INDEX IF NOT EXISTS idx_notification_prefs_user ON notification_preferences(user_id);

19
server/database/migrations/sqlite/011_v0214_git.sql
View File

@@ -1,19 +0,0 @@
-- v0.21.4: Git integration — credentials table + workspace git columns.
CREATE TABLE IF NOT EXISTS git_credentials (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
auth_type TEXT NOT NULL DEFAULT 'https_pat',
encrypted_data TEXT NOT NULL DEFAULT '',
nonce TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_git_credentials_user ON git_credentials(user_id);
-- Workspace git tracking columns.
ALTER TABLE workspaces ADD COLUMN git_remote_url TEXT;
ALTER TABLE workspaces ADD COLUMN git_branch TEXT;
ALTER TABLE workspaces ADD COLUMN git_credential_id TEXT;
ALTER TABLE workspaces ADD COLUMN git_last_sync TEXT;

30
server/database/migrations/sqlite/012_extensions.sql Normal file
View File

@@ -0,0 +1,30 @@
-- Chat Switchboard — 012 Extensions (SQLite)
CREATE TABLE IF NOT EXISTS extensions (
id TEXT PRIMARY KEY,
ext_id TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
version TEXT NOT NULL DEFAULT '0.0.0',
tier TEXT NOT NULL DEFAULT 'browser',
description TEXT NOT NULL DEFAULT '',
author TEXT NOT NULL DEFAULT '',
manifest TEXT NOT NULL DEFAULT '{}',
is_system INTEGER NOT NULL DEFAULT 0,
is_enabled INTEGER NOT NULL DEFAULT 1,
scope TEXT NOT NULL DEFAULT 'global',
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
installed_by TEXT REFERENCES users(id) ON DELETE SET NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_extensions_tier ON extensions(tier);
CREATE INDEX IF NOT EXISTS idx_extensions_enabled ON extensions(is_enabled);
CREATE TABLE IF NOT EXISTS extension_user_settings (
extension_id TEXT NOT NULL REFERENCES extensions(id) ON DELETE CASCADE,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
settings TEXT NOT NULL DEFAULT '{}',
is_enabled INTEGER NOT NULL DEFAULT 1,
PRIMARY KEY (extension_id, user_id)
);

35
server/database/migrations/sqlite/012_v0220_health.sql
View File

@@ -1,35 +0,0 @@
-- v0.22.0: Provider health tracking + capability admin overrides.
CREATE TABLE IF NOT EXISTS provider_health (
id TEXT PRIMARY KEY,
provider_config_id TEXT NOT NULL,
window_start TEXT NOT NULL,
request_count INTEGER NOT NULL DEFAULT 0,
error_count INTEGER NOT NULL DEFAULT 0,
timeout_count INTEGER NOT NULL DEFAULT 0,
total_latency_ms INTEGER NOT NULL DEFAULT 0,
max_latency_ms INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
last_error_at TEXT,
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (provider_config_id, window_start)
);
CREATE INDEX IF NOT EXISTS idx_provider_health_window
ON provider_health (provider_config_id, window_start);
CREATE TABLE IF NOT EXISTS capability_overrides (
id TEXT PRIMARY KEY,
provider_config_id TEXT,
model_id TEXT NOT NULL,
field TEXT NOT NULL,
value TEXT NOT NULL,
set_by TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
UNIQUE (provider_config_id, model_id, field)
);
CREATE INDEX IF NOT EXISTS idx_capability_overrides_model
ON capability_overrides (model_id);

61
server/database/migrations/sqlite/013_files.sql Normal file
View File

@@ -0,0 +1,61 @@
-- Chat Switchboard — 013 Files (SQLite)
-- Unified files table replacing attachments.
-- Handles both upgrade (attachments exists) and fresh install.
CREATE TABLE IF NOT EXISTS files (
id TEXT PRIMARY KEY,
channel_id TEXT REFERENCES channels(id) ON DELETE CASCADE,
message_id TEXT REFERENCES messages(id) ON DELETE SET NULL,
user_id TEXT REFERENCES users(id) ON DELETE SET NULL,
project_id TEXT REFERENCES projects(id) ON DELETE SET NULL,
origin TEXT NOT NULL DEFAULT 'user_upload'
CHECK (origin IN ('user_upload', 'tool_output', 'system')),
filename TEXT NOT NULL,
content_type TEXT NOT NULL DEFAULT 'application/octet-stream',
size_bytes INTEGER NOT NULL DEFAULT 0,
storage_key TEXT NOT NULL,
display_hint TEXT NOT NULL DEFAULT 'download'
CHECK (display_hint IN ('inline', 'download', 'thumbnail')),
extracted_text TEXT,
metadata TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')),
updated_at TEXT DEFAULT (datetime('now'))
);
-- Ensure attachments table exists so the INSERT is always valid.
-- Upgrade: no-op (table already has data). Fresh install: empty table.
CREATE TABLE IF NOT EXISTS attachments (
id TEXT PRIMARY KEY, channel_id TEXT, user_id TEXT, message_id TEXT,
project_id TEXT, filename TEXT, content_type TEXT, size_bytes INTEGER,
storage_key TEXT, extracted_text TEXT, metadata TEXT, created_at TEXT
);
INSERT OR IGNORE INTO files (
id, channel_id, message_id, user_id, project_id,
origin, filename, content_type, size_bytes, storage_key,
display_hint, extracted_text, metadata, created_at, updated_at
)
SELECT
id, channel_id, message_id, user_id, project_id,
'user_upload',
filename, content_type, size_bytes, storage_key,
CASE
WHEN content_type LIKE 'image/%' THEN 'inline'
WHEN content_type LIKE 'video/%' THEN 'inline'
WHEN content_type LIKE 'audio/%' THEN 'inline'
WHEN content_type = 'application/pdf' THEN 'thumbnail'
WHEN content_type LIKE 'text/%' THEN 'inline'
ELSE 'download'
END,
extracted_text, metadata, created_at, created_at
FROM attachments;
DROP TABLE IF EXISTS attachments;
CREATE INDEX IF NOT EXISTS idx_files_channel ON files(channel_id);
CREATE INDEX IF NOT EXISTS idx_files_message ON files(message_id);
CREATE INDEX IF NOT EXISTS idx_files_user ON files(user_id);
CREATE INDEX IF NOT EXISTS idx_files_project ON files(project_id);
CREATE INDEX IF NOT EXISTS idx_files_origin ON files(origin);
CREATE INDEX IF NOT EXISTS idx_files_orphan ON files(created_at)
WHERE message_id IS NULL AND origin = 'user_upload';

24
server/database/migrations/sqlite/013_v0222_routing.sql
View File

@@ -1,24 +0,0 @@
-- Migration 013: v0.22.2 — Routing Policies (SQLite)
CREATE TABLE IF NOT EXISTS routing_policies (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
scope TEXT NOT NULL DEFAULT 'global'
CHECK (scope IN ('global', 'team')),
team_id TEXT REFERENCES teams(id) ON DELETE CASCADE,
priority INTEGER NOT NULL DEFAULT 100,
policy_type TEXT NOT NULL
CHECK (policy_type IN ('provider_prefer', 'team_route', 'cost_limit', 'model_alias')),
config TEXT NOT NULL DEFAULT '{}',
is_active INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_routing_policies_active
ON routing_policies(is_active, priority);
CREATE INDEX IF NOT EXISTS idx_routing_policies_team
ON routing_policies(team_id);
-- Add routing_decision column to usage_log
ALTER TABLE usage_log ADD COLUMN routing_decision TEXT;

42
server/database/migrations/sqlite/014_audit_usage.sql Normal file
View File

@@ -0,0 +1,42 @@
-- Chat Switchboard — 014 Audit & Usage (SQLite)
CREATE TABLE IF NOT EXISTS audit_log (
id TEXT PRIMARY KEY,
actor_id TEXT REFERENCES users(id) ON DELETE SET NULL,
action TEXT NOT NULL,
resource_type TEXT NOT NULL,
resource_id TEXT,
metadata TEXT DEFAULT '{}',
ip_address TEXT,
user_agent TEXT DEFAULT '',
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_audit_log_created ON audit_log(created_at);
CREATE INDEX IF NOT EXISTS idx_audit_log_actor ON audit_log(actor_id);
CREATE INDEX IF NOT EXISTS idx_audit_log_resource ON audit_log(resource_type, resource_id);
CREATE INDEX IF NOT EXISTS idx_audit_log_action ON audit_log(action);
CREATE TABLE IF NOT EXISTS usage_log (
id TEXT PRIMARY KEY,
channel_id TEXT REFERENCES channels(id) ON DELETE SET NULL,
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
provider_config_id TEXT REFERENCES provider_configs(id) ON DELETE SET NULL,
provider_scope TEXT NOT NULL DEFAULT 'global',
model_id TEXT NOT NULL,
role TEXT,
prompt_tokens INTEGER NOT NULL DEFAULT 0,
completion_tokens INTEGER NOT NULL DEFAULT 0,
cache_creation_tokens INTEGER NOT NULL DEFAULT 0,
cache_read_tokens INTEGER NOT NULL DEFAULT 0,
cost_input REAL,
cost_output REAL,
routing_decision TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_usage_log_user ON usage_log(user_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_created ON usage_log(created_at);
CREATE INDEX IF NOT EXISTS idx_usage_log_provider ON usage_log(provider_config_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_model ON usage_log(model_id);
CREATE INDEX IF NOT EXISTS idx_usage_log_scope ON usage_log(provider_scope);

9
server/database/migrations/sqlite/014_v0224_health_enhance.sql → server/database/migrations/sqlite/015_health.sql
View File

@@ -1,9 +1,5 @@
-- v0.22.4: Rate limit tracking, tool health, project files (SQLite).
-- Chat Switchboard — 015 Tool Health (SQLite)
-- Rate limit column (SQLite requires full rebuild or default — use default).
ALTER TABLE provider_health ADD COLUMN rate_limit_count INTEGER NOT NULL DEFAULT 0;
-- Tool Health
CREATE TABLE IF NOT EXISTS tool_health (
id TEXT PRIMARY KEY,
tool_name TEXT NOT NULL,
@@ -18,5 +14,4 @@ CREATE TABLE IF NOT EXISTS tool_health (
UNIQUE (tool_name, window_start)
);
-- Project Files
ALTER TABLE attachments ADD COLUMN project_id TEXT REFERENCES projects(id) ON DELETE CASCADE;
CREATE INDEX IF NOT EXISTS idx_tool_health_window ON tool_health(tool_name, window_start);

6
server/database/testhelper.go
View File

@@ -311,8 +311,7 @@ func TruncateAll(t *testing.T) {
INSERT INTO global_settings (key, value) VALUES
('registration', '{"enabled": true}'),
('site', '{"name": "Chat Switchboard", "tagline": "Multi-Model AI Chat"}'),
('banner', '{"enabled": false, "text": "", "position": "both", "bg": "#007a33", "fg": "#ffffff"}'),
('banner_presets', '{}'),
('banner', '{"enabled": false, "text": "", "bg": "#007a33", "fg": "#ffffff"}'),
('model_roles', '{"utility":{"primary":null,"fallback":null},"embedding":{"primary":null,"fallback":null},"generation":{"primary":null,"fallback":null}}')
ON CONFLICT (key) DO NOTHING
`)
@@ -331,8 +330,7 @@ func TruncateAll(t *testing.T) {
INSERT INTO global_settings (key, value) VALUES
('registration', '{"enabled": true}'::jsonb),
('site', '{"name": "Chat Switchboard", "tagline": "Multi-Model AI Chat"}'::jsonb),
('banner', '{"enabled": false, "text": "", "position": "both", "bg": "#007a33", "fg": "#ffffff"}'::jsonb),
('banner_presets', '{}'::jsonb),
('banner', '{"enabled": false, "text": "", "bg": "#007a33", "fg": "#ffffff"}'::jsonb),
('model_roles', '{
"utility": { "primary": null, "fallback": null },
"embedding": { "primary": null, "fallback": null },

44
server/extraction/queue.go
View File

@@ -25,7 +25,7 @@ const (
// QueueItem is the status.json written to the processing directory.
// The sidecar extractor watches this directory for pending items.
type QueueItem struct {
AttachmentID string `json:"attachment_id"`
FileID string `json:"file_id"`
StorageKey string `json:"storage_key"`
ContentType string `json:"content_type"`
Filename string `json:"filename"`
@@ -40,7 +40,7 @@ type QueueItem struct {
// ── Queue Manager ──────────────────────────
// Queue manages the filesystem-based extraction queue.
// Processing state lives in {storagePath}/processing/{attachment_id}/status.json.
// Processing state lives in {storagePath}/processing/{file_id}/status.json.
// The sidecar extractor watches for pending items and updates status.
type Queue struct {
storagePath string
@@ -69,19 +69,19 @@ func NewQueue(storagePath string, concurrency int) (*Queue, error) {
}, nil
}
// Enqueue adds an attachment to the extraction queue.
// Enqueue adds a file to the extraction queue.
// Creates {storagePath}/processing/{id}/status.json with status "pending".
func (q *Queue) Enqueue(attachmentID, storageKey, contentType, filename string) error {
func (q *Queue) Enqueue(fileID, storageKey, contentType, filename string) error {
q.mu.Lock()
defer q.mu.Unlock()
itemDir := filepath.Join(q.storagePath, "processing", attachmentID)
itemDir := filepath.Join(q.storagePath, "processing", fileID)
if err := os.MkdirAll(itemDir, 0750); err != nil {
return fmt.Errorf("extraction: create item dir: %w", err)
}
item := QueueItem{
AttachmentID: attachmentID,
FileID: fileID,
StorageKey: storageKey,
ContentType: contentType,
Filename: filename,
@@ -89,12 +89,12 @@ func (q *Queue) Enqueue(attachmentID, storageKey, contentType, filename string)
QueuedAt: time.Now().UTC().Format(time.RFC3339),
}
return q.writeStatus(attachmentID, &item)
return q.writeStatus(fileID, &item)
}
// GetStatus reads the current extraction status for an attachment.
func (q *Queue) GetStatus(attachmentID string) (*QueueItem, error) {
statusPath := filepath.Join(q.storagePath, "processing", attachmentID, "status.json")
// GetStatus reads the current extraction status for a file.
func (q *Queue) GetStatus(fileID string) (*QueueItem, error) {
statusPath := filepath.Join(q.storagePath, "processing", fileID, "status.json")
data, err := os.ReadFile(statusPath)
if err != nil {
if os.IsNotExist(err) {
@@ -111,40 +111,40 @@ func (q *Queue) GetStatus(attachmentID string) (*QueueItem, error) {
}
// MarkComplete updates status to complete and records the output key.
func (q *Queue) MarkComplete(attachmentID, outputKey string) error {
func (q *Queue) MarkComplete(fileID, outputKey string) error {
q.mu.Lock()
defer q.mu.Unlock()
item, err := q.GetStatus(attachmentID)
item, err := q.GetStatus(fileID)
if err != nil || item == nil {
return fmt.Errorf("extraction: item not found: %s", attachmentID)
return fmt.Errorf("extraction: item not found: %s", fileID)
}
item.Status = StatusComplete
item.OutputKey = outputKey
item.CompletedAt = time.Now().UTC().Format(time.RFC3339)
return q.writeStatus(attachmentID, item)
return q.writeStatus(fileID, item)
}
// MarkFailed updates status to failed with an error message.
func (q *Queue) MarkFailed(attachmentID, errMsg string) error {
func (q *Queue) MarkFailed(fileID, errMsg string) error {
q.mu.Lock()
defer q.mu.Unlock()
item, err := q.GetStatus(attachmentID)
item, err := q.GetStatus(fileID)
if err != nil || item == nil {
return fmt.Errorf("extraction: item not found: %s", attachmentID)
return fmt.Errorf("extraction: item not found: %s", fileID)
}
item.Status = StatusFailed
item.Error = errMsg
item.CompletedAt = time.Now().UTC().Format(time.RFC3339)
return q.writeStatus(attachmentID, item)
return q.writeStatus(fileID, item)
}
// Cleanup removes the processing directory for a completed/failed item.
func (q *Queue) Cleanup(attachmentID string) error {
itemDir := filepath.Join(q.storagePath, "processing", attachmentID)
func (q *Queue) Cleanup(fileID string) error {
itemDir := filepath.Join(q.storagePath, "processing", fileID)
return os.RemoveAll(itemDir)
}
@@ -227,8 +227,8 @@ func (q *Queue) ListAll() ([]QueueItem, error) {
// ── Internal Helpers ───────────────────────
func (q *Queue) writeStatus(attachmentID string, item *QueueItem) error {
statusPath := filepath.Join(q.storagePath, "processing", attachmentID, "status.json")
func (q *Queue) writeStatus(fileID string, item *QueueItem) error {
statusPath := filepath.Join(q.storagePath, "processing", fileID, "status.json")
data, err := json.MarshalIndent(item, "", " ")
if err != nil {
return err

10
server/extraction/queue_test.go
View File

@@ -20,7 +20,7 @@ func tempQueue(t *testing.T) *Queue {
func TestEnqueue_CreatesStatusFile(t *testing.T) {
q := tempQueue(t)
err := q.Enqueue("att-123", "attachments/ch/att-123_doc.pdf", "application/pdf", "doc.pdf")
err := q.Enqueue("att-123", "files/ch/att-123_doc.pdf", "application/pdf", "doc.pdf")
if err != nil {
t.Fatalf("Enqueue: %v", err)
}
@@ -35,8 +35,8 @@ func TestEnqueue_CreatesStatusFile(t *testing.T) {
if item.Status != StatusPending {
t.Errorf("status = %q, want %q", item.Status, StatusPending)
}
if item.AttachmentID != "att-123" {
t.Errorf("attachment_id = %q, want att-123", item.AttachmentID)
if item.FileID != "att-123" {
t.Errorf("file_id = %q, want att-123", item.FileID)
}
if item.ContentType != "application/pdf" {
t.Errorf("content_type = %q, want application/pdf", item.ContentType)
@@ -126,8 +126,8 @@ func TestListPending(t *testing.T) {
if len(pending) != 1 {
t.Fatalf("expected 1 pending, got %d", len(pending))
}
if pending[0].AttachmentID != "att-a" {
t.Errorf("pending[0].AttachmentID = %q, want att-a", pending[0].AttachmentID)
if pending[0].FileID != "att-a" {
t.Errorf("pending[0].FileID = %q, want att-a", pending[0].FileID)
}
}

4
server/handlers/channels.go
View File

@@ -84,7 +84,7 @@ func NewChannelHandler() *ChannelHandler {
var channelDeleteHook func(channelID string)
// SetChannelDeleteHook registers a callback invoked after channel deletion.
// Used to clean up attachment files on the storage backend.
// Used to clean up channel files on the storage backend.
func SetChannelDeleteHook(fn func(channelID string)) {
channelDeleteHook = fn
}
@@ -543,7 +543,7 @@ func (h *ChannelHandler) DeleteChannel(c *gin.Context) {
return
}
// Clean up storage files (CASCADE already removed PG attachment rows)
// Clean up storage files (CASCADE already removed PG file rows)
if channelDeleteHook != nil {
go channelDeleteHook(channelID)
}

50
server/handlers/completion.go
View File

@@ -41,7 +41,7 @@ type completionRequest struct {
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
Stream *bool `json:"stream,omitempty"`
AttachmentIDs []string `json:"attachment_ids,omitempty"` // staged attachment UUIDs to include in request
FileIDs []string `json:"file_ids,omitempty"` // staged file UUIDs to include in request
DisabledTools []string `json:"disabled_tools,omitempty"` // tool names to exclude from this request
}
@@ -50,7 +50,7 @@ type CompletionHandler struct {
vault *crypto.KeyResolver
stores store.Stores
hub *events.Hub // WebSocket hub for browser tool bridge
objStore storage.ObjectStore // file storage for attachment content (nil = disabled)
objStore storage.ObjectStore // file storage for uploaded/generated content (nil = disabled)
embedder *knowledge.Embedder // for memory semantic recall (v0.18.0)
health HealthRecorder // provider health tracking (v0.22.0, nil = disabled)
healthStore HealthStatusQuerier // health status queries for routing (v0.22.2, nil = disabled)
@@ -341,14 +341,14 @@ func (h *CompletionHandler) Complete(c *gin.Context) {
// Resolve capabilities early — needed for vision gating below
caps := h.getModelCapabilities(c, model, configID)
// Build user message — multimodal if attachments are present
// Build user message — multimodal if files are present
userMsg := providers.Message{
Role: "user",
Content: req.Content,
}
if len(req.AttachmentIDs) > 0 && h.objStore != nil {
parts, augContent, _, err := h.buildMultimodalParts(c, channelID, req.Content, req.AttachmentIDs, caps)
if len(req.FileIDs) > 0 && h.objStore != nil {
parts, augContent, _, err := h.buildMultimodalParts(c, channelID, req.Content, req.FileIDs, caps)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
@@ -370,11 +370,11 @@ func (h *CompletionHandler) Complete(c *gin.Context) {
log.Printf("Failed to persist user message: %v", err)
}
// Link attachments to the persisted message
if msgID != "" && len(req.AttachmentIDs) > 0 {
for _, attID := range req.AttachmentIDs {
if err := h.stores.Attachments.SetMessageID(c.Request.Context(), attID, msgID); err != nil {
log.Printf("Failed to link attachment %s to message %s: %v", attID, msgID, err)
// Link files to the persisted message
if msgID != "" && len(req.FileIDs) > 0 {
for _, fID := range req.FileIDs {
if err := h.stores.Files.SetMessageID(c.Request.Context(), fID, msgID); err != nil {
log.Printf("Failed to link file %s to message %s: %v", fID, msgID, err)
}
}
}
@@ -865,13 +865,13 @@ func (h *CompletionHandler) getModelCapabilities(c *gin.Context, model, apiConfi
}
// ── Multimodal Assembly ─────────────────────
// Builds content parts from attachments for the user message.
// Builds content parts from files for the user message.
//
// Returns:
// - parts: ContentParts array (non-nil only when images are present)
// - augContent: enriched text content with document context (for doc-only case)
// - validIDs: attachment IDs that were successfully processed
// - error: if any attachment is invalid or vision is needed but missing
// - validIDs: file IDs that were successfully processed
// - error: if any file is invalid or vision is needed but missing
//
// Rules:
// - Images → base64 data URI (requires vision capability)
@@ -882,7 +882,7 @@ func (h *CompletionHandler) getModelCapabilities(c *gin.Context, model, apiConfi
func (h *CompletionHandler) buildMultimodalParts(
c *gin.Context,
channelID, textContent string,
attachmentIDs []string,
fileIDs []string,
caps models.ModelCapabilities,
) ([]providers.ContentPart, string, []string, error) {
@@ -893,39 +893,39 @@ func (h *CompletionHandler) buildMultimodalParts(
var validIDs []string
hasImage := false
for _, attID := range attachmentIDs {
att, err := h.stores.Attachments.GetByID(c.Request.Context(), attID)
for _, fileID := range fileIDs {
att, err := h.stores.Files.GetByID(c.Request.Context(), fileID)
if err != nil {
return nil, "", nil, fmt.Errorf("attachment %s not found", attID)
return nil, "", nil, fmt.Errorf("file %s not found", fileID)
}
// Security: verify attachment belongs to this channel
// Security: verify file belongs to this channel
if att.ChannelID != channelID {
return nil, "", nil, fmt.Errorf("attachment %s does not belong to this channel", attID)
return nil, "", nil, fmt.Errorf("file %s does not belong to this channel", fileID)
}
if isImageContentType(att.ContentType) {
// Vision gating: reject images if model lacks vision
if !caps.Vision {
return nil, "", nil, fmt.Errorf("model does not support image input; remove image attachments or choose a vision-capable model")
return nil, "", nil, fmt.Errorf("model does not support image input; remove image files or choose a vision-capable model")
}
// Read image from storage, base64 encode, build data URI
reader, _, _, err := h.objStore.Get(c.Request.Context(), att.StorageKey)
if err != nil {
log.Printf("Failed to read attachment %s from storage: %v", attID, err)
log.Printf("Failed to read file %s from storage: %v", fileID, err)
parts = append(parts, providers.ContentPart{
Type: "text",
Text: fmt.Sprintf("[Image: %s — failed to read from storage]", att.Filename),
})
validIDs = append(validIDs, attID)
validIDs = append(validIDs, fileID)
continue
}
data, err := io.ReadAll(reader)
reader.Close()
if err != nil {
log.Printf("Failed to read attachment %s bytes: %v", attID, err)
validIDs = append(validIDs, attID)
log.Printf("Failed to read file %s bytes: %v", fileID, err)
validIDs = append(validIDs, fileID)
continue
}
@@ -964,7 +964,7 @@ func (h *CompletionHandler) buildMultimodalParts(
docTexts = append(docTexts, placeholder)
}
validIDs = append(validIDs, attID)
validIDs = append(validIDs, fileID)
}
// If images present → use ContentParts (multimodal array)

132
server/handlers/attachments.go → server/handlers/files.go
View File

@@ -24,7 +24,7 @@ import (
const (
defaultMaxFileSize = 10 * 1024 * 1024 // 10 MB
defaultMaxUploadSize = 50 * 1024 * 1024 // 50 MB total per request (future: multi-file)
defaultMaxAttachmentsPerMsg = 5 // future: multi-file per message
defaultMaxFilesPerMsg = 5 // future: multi-file per message
defaultOrphanMaxAge = 24 * time.Hour
)
@@ -51,20 +51,20 @@ var allowedMIMETypes = map[string]bool{
// ── Handler ────────────────────────────────
type AttachmentHandler struct {
type FileHandler struct {
stores store.Stores
objStore storage.ObjectStore
extQueue *extraction.Queue // nil if extraction disabled
}
func NewAttachmentHandler(stores store.Stores, objStore storage.ObjectStore, extQueue *extraction.Queue) *AttachmentHandler {
return &AttachmentHandler{stores: stores, objStore: objStore, extQueue: extQueue}
func NewFileHandler(stores store.Stores, objStore storage.ObjectStore, extQueue *extraction.Queue) *FileHandler {
return &FileHandler{stores: stores, objStore: objStore, extQueue: extQueue}
}
// ── Upload ─────────────────────────────────
// POST /api/v1/channels/:id/attachments
// Multipart form: file field "file", returns attachment metadata.
func (h *AttachmentHandler) Upload(c *gin.Context) {
// POST /api/v1/channels/:id/files
// Multipart form: file field "file", returns file metadata.
func (h *FileHandler) Upload(c *gin.Context) {
if h.objStore == nil {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": "file storage not configured"})
return
@@ -128,14 +128,16 @@ func (h *AttachmentHandler) Upload(c *gin.Context) {
return
}
// Build storage key: attachments/{channel_id}/{attachment_id}_{filename}
// Build storage key: files are stored under files/{channel_id}/{file_id}_{filename}
// We generate the ID first via a temp UUID, then use it in the key.
att := &models.Attachment{
att := &models.File{
ChannelID: channelID,
UserID: userID,
Origin: models.FileOriginUserUpload,
Filename: sanitizeFilename(header.Filename),
ContentType: contentType,
SizeBytes: header.Size,
DisplayHint: displayHintFor(contentType),
Metadata: models.JSONMap{
"extraction_status": "pending",
},
@@ -144,30 +146,30 @@ func (h *AttachmentHandler) Upload(c *gin.Context) {
// Create PG row first to get the UUID
// storage_key is set after we have the ID
att.StorageKey = "placeholder"
if err := h.stores.Attachments.Create(c.Request.Context(), att); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create attachment record"})
if err := h.stores.Files.Create(c.Request.Context(), att); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create file record"})
return
}
// Now build the real storage key and update
att.StorageKey = fmt.Sprintf("attachments/%s/%s_%s", channelID, att.ID, att.Filename)
att.StorageKey = fmt.Sprintf("files/%s/%s_%s", channelID, att.ID, att.Filename)
// Write to object store
if err := h.objStore.Put(c.Request.Context(), att.StorageKey, file, header.Size, contentType); err != nil {
// Rollback PG row on storage failure
h.stores.Attachments.Delete(c.Request.Context(), att.ID)
h.stores.Files.Delete(c.Request.Context(), att.ID)
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to store file"})
return
}
// Update storage_key in PG
database.DB.ExecContext(c.Request.Context(),
database.Q(`UPDATE attachments SET storage_key = $1 WHERE id = $2`),
database.Q(`UPDATE files SET storage_key = $1 WHERE id = $2`),
att.StorageKey, att.ID)
// For images, mark extraction as not needed (complete immediately)
if strings.HasPrefix(contentType, "image/") {
h.stores.Attachments.UpdateMetadata(c.Request.Context(), att.ID, map[string]interface{}{
h.stores.Files.UpdateMetadata(c.Request.Context(), att.ID, map[string]interface{}{
"extraction_status": "complete",
})
att.Metadata["extraction_status"] = "complete"
@@ -178,8 +180,8 @@ func (h *AttachmentHandler) Upload(c *gin.Context) {
if _, err := file.Seek(0, io.SeekStart); err == nil {
if textBytes, err := io.ReadAll(file); err == nil {
text := string(textBytes)
h.stores.Attachments.SetExtractedText(c.Request.Context(), att.ID, text)
h.stores.Attachments.UpdateMetadata(c.Request.Context(), att.ID, map[string]interface{}{
h.stores.Files.SetExtractedText(c.Request.Context(), att.ID, text)
h.stores.Files.UpdateMetadata(c.Request.Context(), att.ID, map[string]interface{}{
"extraction_status": "complete",
})
att.Metadata["extraction_status"] = "complete"
@@ -199,9 +201,9 @@ func (h *AttachmentHandler) Upload(c *gin.Context) {
}
// ── Download ───────────────────────────────
// GET /api/v1/attachments/:id/download
// GET /api/v1/files/:id/download
// Streams file content with auth check via channel membership.
func (h *AttachmentHandler) Download(c *gin.Context) {
func (h *FileHandler) Download(c *gin.Context) {
if h.objStore == nil {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": "file storage not configured"})
return
@@ -210,9 +212,9 @@ func (h *AttachmentHandler) Download(c *gin.Context) {
userID := getUserID(c)
attID := c.Param("id")
att, err := h.stores.Attachments.GetByID(c.Request.Context(), attID)
att, err := h.stores.Files.GetByID(c.Request.Context(), attID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "attachment not found"})
c.JSON(http.StatusNotFound, gin.H{"error": "file not found"})
return
}
@@ -236,14 +238,14 @@ func (h *AttachmentHandler) Download(c *gin.Context) {
}
// ── Get Metadata ───────────────────────────
// GET /api/v1/attachments/:id
func (h *AttachmentHandler) GetMetadata(c *gin.Context) {
// GET /api/v1/files/:id
func (h *FileHandler) GetMetadata(c *gin.Context) {
userID := getUserID(c)
attID := c.Param("id")
att, err := h.stores.Attachments.GetByID(c.Request.Context(), attID)
att, err := h.stores.Files.GetByID(c.Request.Context(), attID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "attachment not found"})
c.JSON(http.StatusNotFound, gin.H{"error": "file not found"})
return
}
@@ -254,9 +256,9 @@ func (h *AttachmentHandler) GetMetadata(c *gin.Context) {
c.JSON(http.StatusOK, att)
}
// ── List Channel Attachments ───────────────
// GET /api/v1/channels/:id/attachments
func (h *AttachmentHandler) ListByChannel(c *gin.Context) {
// ── List Channel Files ───────────────
// GET /api/v1/channels/:id/files
func (h *FileHandler) ListByChannel(c *gin.Context) {
userID := getUserID(c)
channelID := c.Param("id")
@@ -264,27 +266,27 @@ func (h *AttachmentHandler) ListByChannel(c *gin.Context) {
return
}
attachments, err := h.stores.Attachments.GetByChannel(c.Request.Context(), channelID)
files, err := h.stores.Files.GetByChannel(c.Request.Context(), channelID, "")
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list attachments"})
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list files"})
return
}
if attachments == nil {
attachments = []models.Attachment{}
if files == nil {
files = []models.File{}
}
c.JSON(http.StatusOK, gin.H{"attachments": attachments})
c.JSON(http.StatusOK, gin.H{"files": files})
}
// ── Delete ─────────────────────────────────
// DELETE /api/v1/attachments/:id
func (h *AttachmentHandler) DeleteAttachment(c *gin.Context) {
// DELETE /api/v1/files/:id
func (h *FileHandler) Delete(c *gin.Context) {
userID := getUserID(c)
attID := c.Param("id")
att, err := h.stores.Attachments.GetByID(c.Request.Context(), attID)
att, err := h.stores.Files.GetByID(c.Request.Context(), attID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "attachment not found"})
c.JSON(http.StatusNotFound, gin.H{"error": "file not found"})
return
}
@@ -293,9 +295,9 @@ func (h *AttachmentHandler) DeleteAttachment(c *gin.Context) {
}
// Delete from PG (returns the row for storage cleanup)
deleted, err := h.stores.Attachments.Delete(c.Request.Context(), attID)
deleted, err := h.stores.Files.Delete(c.Request.Context(), attID)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete attachment"})
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete file"})
return
}
@@ -312,13 +314,13 @@ func (h *AttachmentHandler) DeleteAttachment(c *gin.Context) {
}()
}
c.JSON(http.StatusOK, gin.H{"message": "attachment deleted"})
c.JSON(http.StatusOK, gin.H{"message": "file deleted"})
}
// ── Admin: Orphan Cleanup ──────────────────
// POST /admin/storage/cleanup
func (h *AttachmentHandler) CleanupOrphans(c *gin.Context) {
orphans, err := h.stores.Attachments.ListOrphans(c.Request.Context(), defaultOrphanMaxAge)
func (h *FileHandler) CleanupOrphans(c *gin.Context) {
orphans, err := h.stores.Files.ListOrphans(c.Request.Context(), defaultOrphanMaxAge)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list orphans"})
return
@@ -328,7 +330,7 @@ func (h *AttachmentHandler) CleanupOrphans(c *gin.Context) {
var freedBytes int64
for _, att := range orphans {
if _, err := h.stores.Attachments.Delete(c.Request.Context(), att.ID); err != nil {
if _, err := h.stores.Files.Delete(c.Request.Context(), att.ID); err != nil {
log.Printf("orphan cleanup: failed to delete %s from PG: %v", att.ID, err)
continue
}
@@ -349,8 +351,8 @@ func (h *AttachmentHandler) CleanupOrphans(c *gin.Context) {
// ── Admin: Orphan Count ────────────────────
// GET /admin/storage/orphans (for the admin panel card)
func (h *AttachmentHandler) OrphanCount(c *gin.Context) {
orphans, err := h.stores.Attachments.ListOrphans(c.Request.Context(), defaultOrphanMaxAge)
func (h *FileHandler) OrphanCount(c *gin.Context) {
orphans, err := h.stores.Files.ListOrphans(c.Request.Context(), defaultOrphanMaxAge)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to count orphans"})
return
@@ -369,7 +371,7 @@ func (h *AttachmentHandler) OrphanCount(c *gin.Context) {
// ── Admin: Extraction Queue Status ─────────
// GET /admin/storage/extraction
func (h *AttachmentHandler) ExtractionStatus(c *gin.Context) {
func (h *FileHandler) ExtractionStatus(c *gin.Context) {
if h.extQueue == nil {
c.JSON(http.StatusOK, gin.H{
"enabled": false,
@@ -403,12 +405,12 @@ func (h *AttachmentHandler) ExtractionStatus(c *gin.Context) {
// ── Channel Delete Hook ────────────────────
// Called by ChannelHandler.DeleteChannel to clean up storage.
func (h *AttachmentHandler) CleanupChannelStorage(channelID string) {
func (h *FileHandler) CleanupChannelStorage(channelID string) {
if h.objStore == nil {
return
}
// CASCADE already deleted PG rows. Clean up filesystem.
prefix := fmt.Sprintf("attachments/%s", channelID)
prefix := fmt.Sprintf("files/%s", channelID)
if err := h.objStore.DeletePrefix(context.Background(), prefix); err != nil {
log.Printf("storage cleanup for channel %s failed: %v", channelID, err)
}
@@ -418,7 +420,7 @@ func (h *AttachmentHandler) CleanupChannelStorage(channelID string) {
// verifyChannelAccess checks that the requesting user owns the channel.
// Future RBAC (v0.20.0): replace with rbac.Can(userID, channelID, permission).
func (h *AttachmentHandler) verifyChannelAccess(c *gin.Context, channelID, userID string) bool {
func (h *FileHandler) verifyChannelAccess(c *gin.Context, channelID, userID string) bool {
var ownerID string
err := database.DB.QueryRowContext(c.Request.Context(),
database.Q(`SELECT user_id FROM channels WHERE id = $1`), channelID).Scan(&ownerID)
@@ -461,7 +463,7 @@ func sanitizeFilename(name string) string {
// UploadToProject handles project-scoped file uploads.
// POST /api/v1/projects/:id/files
func (h *AttachmentHandler) UploadToProject(c *gin.Context) {
func (h *FileHandler) UploadToProject(c *gin.Context) {
if h.objStore == nil {
c.JSON(http.StatusServiceUnavailable, gin.H{"error": "file storage not configured"})
return
@@ -516,17 +518,19 @@ func (h *AttachmentHandler) UploadToProject(c *gin.Context) {
return
}
att := models.Attachment{
att := models.File{
ChannelID: "", // no channel association
UserID: userID,
ProjectID: &projectID,
Origin: models.FileOriginUserUpload,
Filename: header.Filename,
ContentType: contentType,
SizeBytes: header.Size,
StorageKey: storageKey,
DisplayHint: displayHintFor(contentType),
}
if err := h.stores.Attachments.Create(c.Request.Context(), &att); err != nil {
if err := h.stores.Files.Create(c.Request.Context(), &att); err != nil {
log.Printf("error: project file create: %v", err)
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to save file metadata"})
return
@@ -543,7 +547,7 @@ func (h *AttachmentHandler) UploadToProject(c *gin.Context) {
// ListByProject returns all files uploaded to a project.
// GET /api/v1/projects/:id/files
func (h *AttachmentHandler) ListByProject(c *gin.Context) {
func (h *FileHandler) ListByProject(c *gin.Context) {
userID := getUserID(c)
projectID := c.Param("id")
@@ -562,7 +566,7 @@ func (h *AttachmentHandler) ListByProject(c *gin.Context) {
return
}
files, err := h.stores.Attachments.GetByProject(c.Request.Context(), projectID)
files, err := h.stores.Files.GetByProject(c.Request.Context(), projectID)
if err != nil {
log.Printf("error: list project files: %v", err)
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list files"})
@@ -589,3 +593,23 @@ var extToMIME = map[string]string{
".csv": "text/csv",
".svg": "image/svg+xml",
}
// displayHintFor returns the appropriate display hint for a content type.
func displayHintFor(contentType string) string {
switch {
case strings.HasPrefix(contentType, "image/"):
return models.FileHintInline
case strings.HasPrefix(contentType, "video/"):
return models.FileHintInline
case strings.HasPrefix(contentType, "audio/"):
return models.FileHintInline
case contentType == "application/pdf":
return models.FileHintThumbnail
case strings.HasPrefix(contentType, "text/"):
return models.FileHintInline
case contentType == "application/json":
return models.FileHintInline
default:
return models.FileHintDownload
}
}

14
server/handlers/integration_test.go
View File

@@ -238,13 +238,13 @@ func setupHarness(t *testing.T) *testHarness {
protected.GET("/knowledge-bases-discoverable", kbH.ListDiscoverableKBs) // v0.17.0
protected.PUT("/knowledge-bases/:id/discoverable", kbH.SetDiscoverable) // v0.17.0
// Attachments (nil storage = upload returns 503, but metadata works)
attachH := NewAttachmentHandler(stores, nil, nil)
protected.POST("/channels/:id/attachments", attachH.Upload)
protected.GET("/channels/:id/attachments", attachH.ListByChannel)
protected.GET("/attachments/:id", attachH.GetMetadata)
protected.GET("/attachments/:id/download", attachH.Download)
protected.DELETE("/attachments/:id", attachH.DeleteAttachment)
// Files (nil storage = upload returns 503, but metadata works)
fileH := NewFileHandler(stores, nil, nil)
protected.POST("/channels/:id/files", fileH.Upload)
protected.GET("/channels/:id/files", fileH.ListByChannel)
protected.GET("/files/:id", fileH.GetMetadata)
protected.GET("/files/:id/download", fileH.Download)
protected.DELETE("/files/:id", fileH.Delete)
// Completions
completions := NewCompletionHandler(nil, stores, nil, nil, nil)

2
server/handlers/personas.go
View File

@@ -10,7 +10,7 @@ import (
"git.gobha.me/xcaliber/chat-switchboard/store"
)
// PersonaHandler handles persona (formerly preset) endpoints.
// PersonaHandler handles persona endpoints.
type PersonaHandler struct {
stores store.Stores
}

2
server/knowledge/ingest.go
View File

@@ -236,7 +236,7 @@ func (ing *Ingester) extractText(ctx context.Context, doc *models.KBDocument) (s
}
// Complex document types are not yet supported for inline extraction.
// The extraction sidecar (v0.12.0) handles attachments but isn't yet
// The extraction sidecar (v0.12.0) handles files but isn't yet
// wired into the KB pipeline. Planned for a future phase.
return "", fmt.Errorf("unsupported content type %q — text-based files only for now (txt, md, csv, html)", doc.ContentType)
}

32
server/main.go
View File

@@ -262,7 +262,7 @@ func main() {
}
// Register context recall tools (v0.15.1)
tools.RegisterAttachmentRecall(stores, objStore)
tools.RegisterFileRecall(stores, objStore)
tools.RegisterConversationSearch(stores)
// Memory tools + extraction scanner (v0.18.0)
@@ -491,13 +491,13 @@ func main() {
protected.DELETE("/projects/:id/notes/:noteId", projectH.RemoveNote)
protected.GET("/projects/:id/notes", projectH.ListNotes)
// Project files (v0.22.4) — wired via attachH which is declared later,
// Project files (v0.22.4) — wired via fileH which is declared later,
// so we use a closure that captures the variable.
protected.POST("/projects/:id/files", func(c *gin.Context) {
handlers.NewAttachmentHandler(stores, objStore, extQueue).UploadToProject(c)
handlers.NewFileHandler(stores, objStore, extQueue).UploadToProject(c)
})
protected.GET("/projects/:id/files", func(c *gin.Context) {
handlers.NewAttachmentHandler(stores, objStore, extQueue).ListByProject(c)
handlers.NewFileHandler(stores, objStore, extQueue).ListByProject(c)
})
// Notifications (v0.20.0)
@@ -553,20 +553,20 @@ func main() {
protected.GET("/git-credentials", gitCredH.List)
protected.DELETE("/git-credentials/:id", gitCredH.Delete)
// Attachments (file upload/download)
attachH := handlers.NewAttachmentHandler(stores, objStore, extQueue)
protected.POST("/channels/:id/attachments", attachH.Upload)
protected.GET("/channels/:id/attachments", attachH.ListByChannel)
protected.GET("/attachments/:id", attachH.GetMetadata)
protected.GET("/attachments/:id/download", attachH.Download)
protected.DELETE("/attachments/:id", attachH.DeleteAttachment)
// Files (upload/download)
fileH := handlers.NewFileHandler(stores, objStore, extQueue)
protected.POST("/channels/:id/files", fileH.Upload)
protected.GET("/channels/:id/files", fileH.ListByChannel)
protected.GET("/files/:id", fileH.GetMetadata)
protected.GET("/files/:id/download", fileH.Download)
protected.DELETE("/files/:id", fileH.Delete)
// Export (v0.22.4) — pandoc-based markdown → PDF/DOCX conversion
exportH := handlers.NewExportHandler()
protected.POST("/export", exportH.Convert)
// Hook: clean up storage files when channels are deleted
handlers.SetChannelDeleteHook(attachH.CleanupChannelStorage)
handlers.SetChannelDeleteHook(fileH.CleanupChannelStorage)
// Knowledge Bases (RAG — v0.14.0)
kbH := handlers.NewKnowledgeBaseHandler(stores, objStore, kbIngester, kbEmbedder)
@@ -775,10 +775,10 @@ func main() {
admin.POST("/notifications/test-email", emailAdm.TestEmail)
// Storage management (orphan cleanup)
attachAdm := handlers.NewAttachmentHandler(stores, objStore, extQueue)
admin.GET("/storage/orphans", attachAdm.OrphanCount)
admin.POST("/storage/cleanup", attachAdm.CleanupOrphans)
admin.GET("/storage/extraction", attachAdm.ExtractionStatus)
fileAdm := handlers.NewFileHandler(stores, objStore, extQueue)
admin.GET("/storage/orphans", fileAdm.OrphanCount)
admin.POST("/storage/cleanup", fileAdm.CleanupOrphans)
admin.GET("/storage/extraction", fileAdm.ExtractionStatus)
// Extensions (admin)
extAdm := handlers.NewExtensionHandler(stores)

47
server/models/models.go
View File

@@ -501,21 +501,44 @@ type NoteGraph struct {
// ATTACHMENTS
// =========================================
type Attachment struct {
ID string `json:"id" db:"id"`
ChannelID string `json:"channel_id" db:"channel_id"`
UserID string `json:"user_id" db:"user_id"`
MessageID *string `json:"message_id,omitempty" db:"message_id"`
ProjectID *string `json:"project_id,omitempty" db:"project_id"` // v0.22.4: project-scoped uploads
Filename string `json:"filename" db:"filename"`
ContentType string `json:"content_type" db:"content_type"`
SizeBytes int64 `json:"size_bytes" db:"size_bytes"`
StorageKey string `json:"-" db:"storage_key"` // never expose filesystem path
ExtractedText *string `json:"extracted_text,omitempty" db:"extracted_text"`
Metadata JSONMap `json:"metadata,omitempty" db:"metadata"`
// =========================================
// FILES
// =========================================
// FileOrigin constants
const (
FileOriginUserUpload = "user_upload"
FileOriginToolOutput = "tool_output"
FileOriginSystem = "system"
)
// FileDisplayHint constants
const (
FileHintInline = "inline"
FileHintDownload = "download"
FileHintThumbnail = "thumbnail"
)
type File struct {
ID string `json:"id" db:"id"`
ChannelID string `json:"channel_id" db:"channel_id"`
MessageID *string `json:"message_id,omitempty" db:"message_id"`
UserID string `json:"user_id" db:"user_id"`
ProjectID *string `json:"project_id,omitempty" db:"project_id"`
Origin string `json:"origin" db:"origin"` // user_upload, tool_output, system
Filename string `json:"filename" db:"filename"`
ContentType string `json:"content_type" db:"content_type"`
SizeBytes int64 `json:"size_bytes" db:"size_bytes"`
StorageKey string `json:"-" db:"storage_key"` // never expose filesystem path
DisplayHint string `json:"display_hint" db:"display_hint"` // inline, download, thumbnail
ExtractedText *string `json:"extracted_text,omitempty" db:"extracted_text"`
Metadata JSONMap `json:"metadata,omitempty" db:"metadata"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
UpdatedAt time.Time `json:"updated_at" db:"updated_at"`
}
// =========================================
// AUDIT LOG
// =========================================

6
server/pages/pages.go
View File

@@ -42,7 +42,7 @@ type Engine struct {
devMode bool
}
// BannerConfig holds classification banner settings.
// BannerConfig holds environment banner settings.
type BannerConfig struct {
Text string `json:"text"`
Color string `json:"color"`
@@ -243,10 +243,10 @@ func (e *Engine) loadBanner() BannerConfig {
if v, ok := raw["text"].(string); ok {
b.Text = v
}
if v, ok := raw["color"].(string); ok {
if v, ok := raw["fg"].(string); ok {
b.Color = v
}
if v, ok := raw["background"].(string); ok {
if v, ok := raw["bg"].(string); ok {
b.Background = v
}
return b

2
server/pages/templates/admin/settings.html
View File

@@ -54,7 +54,7 @@
<div class="form-row">
<div class="form-group" style="flex:2;min-width:200px;">
<label>Text</label>
<input type="text" id="settBannerText" placeholder="UNCLASSIFIED">
<input type="text" id="settBannerText" placeholder="DEVELOPMENT">
</div>
<div class="form-group" style="flex:1;min-width:100px;">
<label>Background</label>

38
server/pages/templates/surfaces/chat.html
View File

@@ -44,6 +44,30 @@
</div>
</div>
{{/* ── Crash Catcher (visible without devtools) ── */}}
<div id="crashBanner" style="display:none;position:fixed;top:0;left:0;right:0;z-index:99999;background:#b00020;color:#fff;padding:12px 20px;font-family:monospace;font-size:13px;max-height:40vh;overflow-y:auto;">
<strong>⚠ Init Error</strong> <button onclick="this.parentElement.style.display='none'" style="float:right;background:none;border:1px solid #fff;color:#fff;cursor:pointer;padding:2px 8px;border-radius:3px">✕</button>
<pre id="crashDetail" style="margin:8px 0 0;white-space:pre-wrap;font-size:12px;"></pre>
</div>
<script nonce="{{$.CSPNonce}}">
window.addEventListener('error', function(e) {
var b = document.getElementById('crashBanner');
var d = document.getElementById('crashDetail');
if (b && d) {
b.style.display = '';
d.textContent += (e.filename || '') + ':' + (e.lineno || '') + ' ' + (e.message || e) + '\n';
}
});
window.addEventListener('unhandledrejection', function(e) {
var b = document.getElementById('crashBanner');
var d = document.getElementById('crashDetail');
if (b && d) {
b.style.display = '';
d.textContent += 'Promise: ' + (e.reason?.message || e.reason || 'unknown') + '\n' + (e.reason?.stack || '') + '\n';
}
});
</script>
{{/* ── App Container ───────────────────────── */}}
<div id="appContainer" class="app" style="display:none;height:100%;">
@@ -220,8 +244,8 @@
<span id="contextWarningText"></span>
</div>
{{/* Attachment strip */}}
<div id="attachmentStrip" class="attachment-strip"></div>
{{/* File strip */}}
<div id="fileStrip" class="file-strip"></div>
{{/* Streaming tools display */}}
<div id="streamTools" class="stream-tools" style="display:none;"></div>
@@ -324,7 +348,7 @@
</div>
</div>
{{/* ── Hidden File Input (attachments) ─────── */}}
{{/* ── Hidden File Input ─────── */}}
<input type="file" id="fileInput" multiple style="display:none;">
{{/* ── Settings Modal ──────────────────────── */}}
@@ -450,7 +474,7 @@
<div id="settingsTeamsList"></div>
<div id="settingsTeamMembers" style="display:none;"></div>
<div id="settingsTeamPersonas" style="display:none;">
<div id="adminPersonaList"></div>
<div id="teamPersonaList"></div>
</div>
<div id="settingsTeamProviders" style="display:none;">
<button id="settingsTeamAddProviderBtn" class="btn-small" style="display:none;">+ Add Provider</button>
@@ -536,10 +560,6 @@
<div class="form-group"><label><input type="checkbox" id="adminBannerEnabled"> Banner</label></div>
<div id="bannerConfigFields" style="display:none;">
<input type="text" id="adminBannerText" placeholder="Banner text">
<div style="display:flex;gap:8px;margin-top:6px;">
<select id="adminBannerPreset"><option value="">Custom colors</option></select>
<select id="adminBannerPosition"><option value="both">Both</option><option value="top">Top</option><option value="bottom">Bottom</option></select>
</div>
<div style="display:flex;gap:8px;margin-top:6px;">
<input type="color" id="adminBannerBg" value="#007a33"><input type="text" id="adminBannerBgHex" value="#007a33" style="width:80px;">
<input type="color" id="adminBannerFg" value="#ffffff"><input type="text" id="adminBannerFgHex" value="#ffffff" style="width:80px;">
@@ -716,7 +736,7 @@
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/tokens.js?v={{$.Version}}"></script>
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/notes.js?v={{$.Version}}"></script>
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/note-graph.js?v={{$.Version}}"></script>
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/attachments.js?v={{$.Version}}"></script>
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/files.js?v={{$.Version}}"></script>
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/tools-toggle.js?v={{$.Version}}"></script>
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/knowledge-ui.js?v={{$.Version}}"></script>
<script nonce="{{$.CSPNonce}}" src="{{$.BasePath}}/js/memory-ui.js?v={{$.Version}}"></script>

2
server/pages/templates/surfaces/notes.html
View File

@@ -60,7 +60,7 @@
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/vendor/purify.min.js"></script>
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/js/ui-format.js?v={{.Version}}"></script>
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/js/chat.js?v={{.Version}}"></script>
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/js/attachments.js?v={{.Version}}"></script>
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/js/files.js?v={{.Version}}"></script>
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/js/channel-models.js?v={{.Version}}"></script>
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/js/tokens.js?v={{.Version}}"></script>
<script nonce="{{.CSPNonce}}" src="{{.BasePath}}/js/extensions.js?v={{.Version}}"></script>

8
server/storage/pvc.go
View File

@@ -36,7 +36,7 @@ func NewPVC(basePath string) (*PVCStore, error) {
}
// Create well-known subdirectories
for _, sub := range []string{"attachments", "processing"} {
for _, sub := range []string{"files", "processing"} {
if err := os.MkdirAll(filepath.Join(basePath, sub), 0o755); err != nil {
return nil, fmt.Errorf("storage/pvc: cannot create %s dir: %w", sub, err)
}
@@ -236,9 +236,9 @@ func (s *PVCStore) Stats(ctx context.Context) (*StorageStats, error) {
}
stats.Healthy = true
// Walk attachments directory for counts
attDir := filepath.Join(s.basePath, "attachments")
err := filepath.Walk(attDir, func(path string, info os.FileInfo, err error) error {
// Walk files directory for counts
fileDir := filepath.Join(s.basePath, "files")
err := filepath.Walk(fileDir, func(path string, info os.FileInfo, err error) error {
if err != nil {
return nil // skip errors (e.g. permission denied)
}

32
server/storage/pvc_test.go
View File

@@ -24,7 +24,7 @@ func TestPVC_PutGetRoundTrip(t *testing.T) {
ctx := context.Background()
data := []byte("hello, storage world")
key := "attachments/channel-1/att-1_test.txt"
key := "files/channel-1/att-1_test.txt"
// Put
err := s.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "text/plain")
@@ -56,7 +56,7 @@ func TestPVC_PutAtomicWrite(t *testing.T) {
s := tempStore(t)
ctx := context.Background()
key := "attachments/ch/file.bin"
key := "files/ch/file.bin"
data := []byte("atomic test data")
err := s.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "application/octet-stream")
@@ -65,7 +65,7 @@ func TestPVC_PutAtomicWrite(t *testing.T) {
}
// File should exist at final path, no temp files left
absPath := filepath.Join(s.basePath, "attachments", "ch")
absPath := filepath.Join(s.basePath, "files", "ch")
entries, _ := os.ReadDir(absPath)
for _, e := range entries {
if e.Name() != "file.bin" {
@@ -79,7 +79,7 @@ func TestPVC_PutSizeMismatch(t *testing.T) {
ctx := context.Background()
data := []byte("short")
err := s.Put(ctx, "attachments/ch/f.txt", bytes.NewReader(data), 999, "text/plain")
err := s.Put(ctx, "files/ch/f.txt", bytes.NewReader(data), 999, "text/plain")
if err == nil {
t.Fatal("expected size mismatch error")
}
@@ -89,7 +89,7 @@ func TestPVC_GetNotFound(t *testing.T) {
s := tempStore(t)
ctx := context.Background()
_, _, _, err := s.Get(ctx, "attachments/nonexistent/file.txt")
_, _, _, err := s.Get(ctx, "files/nonexistent/file.txt")
if err != ErrNotFound {
t.Errorf("err = %v, want ErrNotFound", err)
}
@@ -99,7 +99,7 @@ func TestPVC_Delete(t *testing.T) {
s := tempStore(t)
ctx := context.Background()
key := "attachments/ch/delete-me.txt"
key := "files/ch/delete-me.txt"
_ = s.Put(ctx, key, bytes.NewReader([]byte("bye")), 3, "text/plain")
err := s.Delete(ctx, key)
@@ -118,7 +118,7 @@ func TestPVC_DeleteIdempotent(t *testing.T) {
ctx := context.Background()
// Delete a file that doesn't exist — should not error
err := s.Delete(ctx, "attachments/no-such/file.txt")
err := s.Delete(ctx, "files/no-such/file.txt")
if err != nil {
t.Errorf("Delete nonexistent: %v", err)
}
@@ -129,18 +129,18 @@ func TestPVC_DeletePrefix(t *testing.T) {
ctx := context.Background()
// Create several files under a channel prefix
prefix := "attachments/channel-abc/"
prefix := "files/channel-abc/"
for _, name := range []string{"a.txt", "b.png", "c.pdf"} {
key := prefix + name
_ = s.Put(ctx, key, bytes.NewReader([]byte("x")), 1, "text/plain")
}
// Also create a file in a different channel
other := "attachments/channel-other/keep.txt"
other := "files/channel-other/keep.txt"
_ = s.Put(ctx, other, bytes.NewReader([]byte("y")), 1, "text/plain")
// Delete the channel prefix
err := s.DeletePrefix(ctx, "attachments/channel-abc")
err := s.DeletePrefix(ctx, "files/channel-abc")
if err != nil {
t.Fatalf("DeletePrefix: %v", err)
}
@@ -165,7 +165,7 @@ func TestPVC_DeletePrefixNonexistent(t *testing.T) {
ctx := context.Background()
// Should not error
err := s.DeletePrefix(ctx, "attachments/does-not-exist/")
err := s.DeletePrefix(ctx, "files/does-not-exist/")
if err != nil {
t.Errorf("DeletePrefix nonexistent: %v", err)
}
@@ -175,7 +175,7 @@ func TestPVC_Exists(t *testing.T) {
s := tempStore(t)
ctx := context.Background()
key := "attachments/ch/exists.txt"
key := "files/ch/exists.txt"
_ = s.Put(ctx, key, bytes.NewReader([]byte("hi")), 2, "text/plain")
exists, err := s.Exists(ctx, key)
@@ -186,7 +186,7 @@ func TestPVC_Exists(t *testing.T) {
t.Error("Exists returned false for existing file")
}
exists, err = s.Exists(ctx, "attachments/ch/nope.txt")
exists, err = s.Exists(ctx, "files/ch/nope.txt")
if err != nil {
t.Fatalf("Exists (missing): %v", err)
}
@@ -220,7 +220,7 @@ func TestPVC_Stats(t *testing.T) {
// Add some files
for i := 0; i < 3; i++ {
key := filepath.Join("attachments", "ch", string(rune('a'+i))+".txt")
key := filepath.Join("files", "ch", string(rune('a'+i))+".txt")
data := bytes.Repeat([]byte("x"), 100*(i+1))
_ = s.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "text/plain")
}
@@ -243,7 +243,7 @@ func TestPVC_PathTraversal(t *testing.T) {
bad := []string{
"../etc/passwd",
"attachments/../../etc/shadow",
"files/../../etc/shadow",
"/absolute/path",
"",
}
@@ -270,7 +270,7 @@ func TestPVC_SubdirCreation(t *testing.T) {
s := tempStore(t)
// Verify well-known subdirs were created
for _, sub := range []string{"attachments", "processing"} {
for _, sub := range []string{"files", "processing"} {
info, err := os.Stat(filepath.Join(s.basePath, sub))
if err != nil {
t.Errorf("subdir %q not created: %v", sub, err)

6
server/storage/s3.go
View File

@@ -269,10 +269,10 @@ func (s *S3Store) Stats(ctx context.Context) (*StorageStats, error) {
}
stats.Healthy = true
// Count objects under the attachments prefix
attPrefix := s.fullKey("attachments/")
// Count objects under the files prefix
filePrefix := s.fullKey("files/")
objectsCh := s.client.ListObjects(ctx, s.bucket, minio.ListObjectsOptions{
Prefix: attPrefix,
Prefix: filePrefix,
Recursive: true,
})

36
server/storage/s3_test.go
View File

@@ -44,9 +44,9 @@ func TestS3_FullKey(t *testing.T) {
key string
want string
}{
{"", "attachments/ch/f.txt", "attachments/ch/f.txt"},
{"switchboard/", "attachments/ch/f.txt", "switchboard/attachments/ch/f.txt"},
{"prefix", "attachments/ch/f.txt", "prefix/attachments/ch/f.txt"}, // trailing slash added by NewS3
{"", "files/ch/f.txt", "files/ch/f.txt"},
{"switchboard/", "files/ch/f.txt", "switchboard/files/ch/f.txt"},
{"prefix", "files/ch/f.txt", "prefix/files/ch/f.txt"}, // trailing slash added by NewS3
}
for _, tt := range tests {
@@ -68,7 +68,7 @@ func TestS3_KeyValidation(t *testing.T) {
"",
"../etc/passwd",
"/absolute/path",
"attachments/../../etc/shadow",
"files/../../etc/shadow",
}
for _, key := range bad {
if err := validateKey(key); err == nil {
@@ -77,8 +77,8 @@ func TestS3_KeyValidation(t *testing.T) {
}
good := []string{
"attachments/ch/f.txt",
"attachments/channel-1/att-abc_test.pdf",
"files/ch/f.txt",
"files/channel-1/att-abc_test.pdf",
"processing/abc123/status.json",
}
for _, key := range good {
@@ -133,7 +133,7 @@ func testS3Store(t *testing.T) *S3Store {
// Clean up prefix after test
t.Cleanup(func() {
ctx := context.Background()
_ = s.DeletePrefix(ctx, "attachments")
_ = s.DeletePrefix(ctx, "files")
_ = s.DeletePrefix(ctx, "processing")
})
@@ -145,7 +145,7 @@ func TestS3_Integration_PutGetRoundTrip(t *testing.T) {
ctx := context.Background()
data := []byte("hello, S3 storage world")
key := "attachments/channel-1/att-1_test.txt"
key := "files/channel-1/att-1_test.txt"
err := s.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "text/plain")
if err != nil {
@@ -175,7 +175,7 @@ func TestS3_Integration_GetNotFound(t *testing.T) {
s := testS3Store(t)
ctx := context.Background()
_, _, _, err := s.Get(ctx, "attachments/nonexistent/file.txt")
_, _, _, err := s.Get(ctx, "files/nonexistent/file.txt")
if err != ErrNotFound {
t.Errorf("err = %v, want ErrNotFound", err)
}
@@ -185,7 +185,7 @@ func TestS3_Integration_Delete(t *testing.T) {
s := testS3Store(t)
ctx := context.Background()
key := "attachments/ch/delete-me.txt"
key := "files/ch/delete-me.txt"
_ = s.Put(ctx, key, bytes.NewReader([]byte("bye")), 3, "text/plain")
err := s.Delete(ctx, key)
@@ -204,7 +204,7 @@ func TestS3_Integration_DeleteIdempotent(t *testing.T) {
ctx := context.Background()
// S3 delete is inherently idempotent
err := s.Delete(ctx, "attachments/no-such/file.txt")
err := s.Delete(ctx, "files/no-such/file.txt")
if err != nil {
t.Errorf("Delete nonexistent: %v", err)
}
@@ -214,16 +214,16 @@ func TestS3_Integration_DeletePrefix(t *testing.T) {
s := testS3Store(t)
ctx := context.Background()
prefix := "attachments/channel-abc/"
prefix := "files/channel-abc/"
for _, name := range []string{"a.txt", "b.png", "c.pdf"} {
key := prefix + name
_ = s.Put(ctx, key, bytes.NewReader([]byte("x")), 1, "text/plain")
}
other := "attachments/channel-other/keep.txt"
other := "files/channel-other/keep.txt"
_ = s.Put(ctx, other, bytes.NewReader([]byte("y")), 1, "text/plain")
err := s.DeletePrefix(ctx, "attachments/channel-abc")
err := s.DeletePrefix(ctx, "files/channel-abc")
if err != nil {
t.Fatalf("DeletePrefix: %v", err)
}
@@ -245,7 +245,7 @@ func TestS3_Integration_Exists(t *testing.T) {
s := testS3Store(t)
ctx := context.Background()
key := "attachments/ch/exists.txt"
key := "files/ch/exists.txt"
_ = s.Put(ctx, key, bytes.NewReader([]byte("hi")), 2, "text/plain")
exists, err := s.Exists(ctx, key)
@@ -256,7 +256,7 @@ func TestS3_Integration_Exists(t *testing.T) {
t.Error("Exists returned false for existing file")
}
exists, err = s.Exists(ctx, "attachments/ch/nope.txt")
exists, err = s.Exists(ctx, "files/ch/nope.txt")
if err != nil {
t.Fatalf("Exists (missing): %v", err)
}
@@ -278,7 +278,7 @@ func TestS3_Integration_Stats(t *testing.T) {
// Add some files
for i := 0; i < 3; i++ {
key := "attachments/ch/" + string(rune('a'+i)) + ".txt"
key := "files/ch/" + string(rune('a'+i)) + ".txt"
data := bytes.Repeat([]byte("x"), 100*(i+1))
_ = s.Put(ctx, key, bytes.NewReader(data), int64(len(data)), "text/plain")
}
@@ -306,7 +306,7 @@ func TestS3_Integration_PutUnknownSize(t *testing.T) {
ctx := context.Background()
data := []byte("unknown size upload")
key := "attachments/ch/unknown-size.txt"
key := "files/ch/unknown-size.txt"
// size=0 means unknown — S3 should handle via multipart
err := s.Put(ctx, key, bytes.NewReader(data), 0, "text/plain")

4
server/storage/storage.go
View File

@@ -24,7 +24,7 @@ var (
//
// Keys are slash-delimited paths relative to the storage root:
//
// attachments/{channel_id}/{attachment_id}_{filename}
// files/{channel_id}/{file_id}_{filename}
//
// Implementations must create intermediate directories as needed.
type ObjectStore interface {
@@ -44,7 +44,7 @@ type ObjectStore interface {
// DeletePrefix removes all objects under the given prefix.
// Used for channel deletion (bulk cleanup).
// Example: DeletePrefix(ctx, "attachments/{channel_id}/")
// Example: DeletePrefix(ctx, "files/{channel_id}/")
DeletePrefix(ctx context.Context, prefix string) error
// Exists checks if an object exists at key without reading it.

25
server/store/interfaces.go
View File

@@ -34,7 +34,7 @@ type Stores struct {
Usage UsageStore
Pricing PricingStore
Extensions ExtensionStore
Attachments AttachmentStore
Files FileStore
KnowledgeBases KnowledgeBaseStore
Groups GroupStore
ResourceGrants ResourceGrantStore
@@ -385,24 +385,27 @@ type ExtensionStore interface {
}
// =========================================
// ATTACHMENT STORE
// FILE STORE
// =========================================
type AttachmentStore interface {
Create(ctx context.Context, a *models.Attachment) error
GetByID(ctx context.Context, id string) (*models.Attachment, error)
GetByChannel(ctx context.Context, channelID string) ([]models.Attachment, error)
GetByMessage(ctx context.Context, messageID string) ([]models.Attachment, error)
GetByProject(ctx context.Context, projectID string) ([]models.Attachment, error) // v0.22.4
SetMessageID(ctx context.Context, attachmentID, messageID string) error
type FileStore interface {
Create(ctx context.Context, f *models.File) error
GetByID(ctx context.Context, id string) (*models.File, error)
GetByChannel(ctx context.Context, channelID string, origin string) ([]models.File, error)
GetByMessage(ctx context.Context, messageID string) ([]models.File, error)
GetByProject(ctx context.Context, projectID string) ([]models.File, error)
GetByUser(ctx context.Context, userID string, page, perPage int) ([]models.File, int, error) // paginated, returns total
SetMessageID(ctx context.Context, fileID, messageID string) error
UpdateMetadata(ctx context.Context, id string, metadata map[string]interface{}) error
SetExtractedText(ctx context.Context, id string, text string) error
Delete(ctx context.Context, id string) (*models.Attachment, error) // returns deleted row for storage cleanup
Delete(ctx context.Context, id string) (*models.File, error) // returns deleted row for storage cleanup
DeleteByChannel(ctx context.Context, channelID string) ([]string, error) // returns storage_keys
UserUsageBytes(ctx context.Context, userID string) (int64, error)
ListOrphans(ctx context.Context, olderThan time.Duration) ([]models.Attachment, error)
ListOrphans(ctx context.Context, olderThan time.Duration) ([]models.File, error)
}
// =========================================
// KNOWLEDGE BASE STORE
// =========================================

210
server/store/postgres/attachment.go
View File

@@ -1,210 +0,0 @@
package postgres
import (
"context"
"database/sql"
"encoding/json"
"time"
"git.gobha.me/xcaliber/chat-switchboard/models"
)
type AttachmentStore struct{}
func NewAttachmentStore() *AttachmentStore { return &AttachmentStore{} }
// ── columns shared across queries ──────────
const attachmentCols = `id, channel_id, user_id, message_id, project_id, filename, content_type,
size_bytes, storage_key, extracted_text, metadata, created_at`
// scanAttachment scans a row into an Attachment struct.
func scanAttachment(row interface{ Scan(dest ...interface{}) error }) (*models.Attachment, error) {
var a models.Attachment
var messageID sql.NullString
var projectID sql.NullString
var extractedText sql.NullString
var metadataJSON []byte
err := row.Scan(
&a.ID, &a.ChannelID, &a.UserID, &messageID, &projectID,
&a.Filename, &a.ContentType, &a.SizeBytes,
&a.StorageKey, &extractedText, &metadataJSON, &a.CreatedAt,
)
if err != nil {
return nil, err
}
a.MessageID = NullableStringPtr(messageID)
a.ProjectID = NullableStringPtr(projectID)
if extractedText.Valid {
a.ExtractedText = &extractedText.String
}
if len(metadataJSON) > 0 {
json.Unmarshal(metadataJSON, &a.Metadata)
}
return &a, nil
}
func (s *AttachmentStore) Create(ctx context.Context, a *models.Attachment) error {
return DB.QueryRowContext(ctx, `
INSERT INTO attachments (channel_id, user_id, message_id, project_id, filename, content_type,
size_bytes, storage_key, extracted_text, metadata)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
RETURNING id, created_at`,
a.ChannelID, a.UserID, models.NullString(a.MessageID),
models.NullString(a.ProjectID),
a.Filename, a.ContentType, a.SizeBytes,
a.StorageKey, models.NullString(a.ExtractedText),
ToJSON(a.Metadata),
).Scan(&a.ID, &a.CreatedAt)
}
func (s *AttachmentStore) GetByID(ctx context.Context, id string) (*models.Attachment, error) {
row := DB.QueryRowContext(ctx, `SELECT `+attachmentCols+` FROM attachments WHERE id = $1`, id)
return scanAttachment(row)
}
func (s *AttachmentStore) GetByChannel(ctx context.Context, channelID string) ([]models.Attachment, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments WHERE channel_id = $1 ORDER BY created_at`, channelID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}
func (s *AttachmentStore) GetByMessage(ctx context.Context, messageID string) ([]models.Attachment, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments WHERE message_id = $1 ORDER BY created_at`, messageID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}
// GetByProject returns all attachments for a project (v0.22.4).
func (s *AttachmentStore) GetByProject(ctx context.Context, projectID string) ([]models.Attachment, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments WHERE project_id = $1 ORDER BY created_at`, projectID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}
func (s *AttachmentStore) SetMessageID(ctx context.Context, attachmentID, messageID string) error {
_, err := DB.ExecContext(ctx,
`UPDATE attachments SET message_id = $1 WHERE id = $2`,
messageID, attachmentID)
return err
}
func (s *AttachmentStore) UpdateMetadata(ctx context.Context, id string, metadata map[string]interface{}) error {
// Merge into existing metadata using jsonb || operator
metaJSON, err := json.Marshal(metadata)
if err != nil {
return err
}
_, err = DB.ExecContext(ctx,
`UPDATE attachments SET metadata = metadata || $1::jsonb WHERE id = $2`,
metaJSON, id)
return err
}
func (s *AttachmentStore) SetExtractedText(ctx context.Context, id string, text string) error {
_, err := DB.ExecContext(ctx,
`UPDATE attachments SET extracted_text = $1 WHERE id = $2`,
text, id)
return err
}
// Delete removes an attachment and returns the deleted row (for storage cleanup).
func (s *AttachmentStore) Delete(ctx context.Context, id string) (*models.Attachment, error) {
row := DB.QueryRowContext(ctx,
`DELETE FROM attachments WHERE id = $1
RETURNING `+attachmentCols, id)
return scanAttachment(row)
}
// DeleteByChannel removes all attachments for a channel and returns storage keys.
func (s *AttachmentStore) DeleteByChannel(ctx context.Context, channelID string) ([]string, error) {
rows, err := DB.QueryContext(ctx,
`DELETE FROM attachments WHERE channel_id = $1 RETURNING storage_key`, channelID)
if err != nil {
return nil, err
}
defer rows.Close()
var keys []string
for rows.Next() {
var key string
if err := rows.Scan(&key); err != nil {
return keys, err
}
keys = append(keys, key)
}
return keys, rows.Err()
}
func (s *AttachmentStore) UserUsageBytes(ctx context.Context, userID string) (int64, error) {
var total sql.NullInt64
err := DB.QueryRowContext(ctx,
`SELECT COALESCE(SUM(size_bytes), 0) FROM attachments WHERE user_id = $1`,
userID).Scan(&total)
if err != nil {
return 0, err
}
return total.Int64, nil
}
func (s *AttachmentStore) ListOrphans(ctx context.Context, olderThan time.Duration) ([]models.Attachment, error) {
cutoff := time.Now().Add(-olderThan)
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments
WHERE message_id IS NULL AND created_at < $1
ORDER BY created_at`, cutoff)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}

251
server/store/postgres/file.go Normal file
View File

@@ -0,0 +1,251 @@
package postgres
import (
"context"
"database/sql"
"encoding/json"
"time"
"git.gobha.me/xcaliber/chat-switchboard/models"
)
// FileStore implements store.FileStore against the `files` table.
//
type FileStore struct{}
func NewFileStore() *FileStore { return &FileStore{} }
const fileCols = `id, channel_id, user_id, message_id, project_id, origin,
filename, content_type, size_bytes, storage_key, display_hint,
extracted_text, metadata, created_at, updated_at`
func scanFile(row interface{ Scan(dest ...interface{}) error }) (*models.File, error) {
var f models.File
var messageID sql.NullString
var projectID sql.NullString
var extractedText sql.NullString
var metadataJSON []byte
err := row.Scan(
&f.ID, &f.ChannelID, &f.UserID, &messageID, &projectID, &f.Origin,
&f.Filename, &f.ContentType, &f.SizeBytes,
&f.StorageKey, &f.DisplayHint,
&extractedText, &metadataJSON, &f.CreatedAt, &f.UpdatedAt,
)
if err != nil {
return nil, err
}
f.MessageID = NullableStringPtr(messageID)
f.ProjectID = NullableStringPtr(projectID)
if extractedText.Valid {
f.ExtractedText = &extractedText.String
}
if len(metadataJSON) > 0 {
json.Unmarshal(metadataJSON, &f.Metadata)
}
return &f, nil
}
func (s *FileStore) Create(ctx context.Context, f *models.File) error {
if f.Origin == "" {
f.Origin = models.FileOriginUserUpload
}
if f.DisplayHint == "" {
f.DisplayHint = models.FileHintDownload
}
return DB.QueryRowContext(ctx, `
INSERT INTO files (channel_id, user_id, message_id, project_id, origin,
filename, content_type, size_bytes, storage_key, display_hint,
extracted_text, metadata)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12)
RETURNING id, created_at, updated_at`,
f.ChannelID, f.UserID, models.NullString(f.MessageID),
models.NullString(f.ProjectID), f.Origin,
f.Filename, f.ContentType, f.SizeBytes,
f.StorageKey, f.DisplayHint, models.NullString(f.ExtractedText),
ToJSON(f.Metadata),
).Scan(&f.ID, &f.CreatedAt, &f.UpdatedAt)
}
func (s *FileStore) GetByID(ctx context.Context, id string) (*models.File, error) {
row := DB.QueryRowContext(ctx, `SELECT `+fileCols+` FROM files WHERE id = $1`, id)
return scanFile(row)
}
func (s *FileStore) GetByChannel(ctx context.Context, channelID string, origin string) ([]models.File, error) {
var rows *sql.Rows
var err error
if origin != "" {
rows, err = DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE channel_id = $1 AND origin = $2 ORDER BY created_at`,
channelID, origin)
} else {
rows, err = DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE channel_id = $1 ORDER BY created_at`,
channelID)
}
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}
func (s *FileStore) GetByMessage(ctx context.Context, messageID string) ([]models.File, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE message_id = $1 ORDER BY created_at`, messageID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}
func (s *FileStore) GetByProject(ctx context.Context, projectID string) ([]models.File, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE project_id = $1 ORDER BY created_at`, projectID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}
func (s *FileStore) GetByUser(ctx context.Context, userID string, page, perPage int) ([]models.File, int, error) {
var total int
err := DB.QueryRowContext(ctx,
`SELECT COUNT(*) FROM files WHERE user_id = $1`, userID).Scan(&total)
if err != nil {
return nil, 0, err
}
offset := (page - 1) * perPage
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE user_id = $1
ORDER BY created_at DESC LIMIT $2 OFFSET $3`, userID, perPage, offset)
if err != nil {
return nil, 0, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, total, err
}
out = append(out, *f)
}
return out, total, rows.Err()
}
func (s *FileStore) SetMessageID(ctx context.Context, fileID, messageID string) error {
_, err := DB.ExecContext(ctx,
`UPDATE files SET message_id = $1, updated_at = NOW() WHERE id = $2`,
messageID, fileID)
return err
}
func (s *FileStore) UpdateMetadata(ctx context.Context, id string, metadata map[string]interface{}) error {
metaJSON, err := json.Marshal(metadata)
if err != nil {
return err
}
_, err = DB.ExecContext(ctx,
`UPDATE files SET metadata = metadata || $1::jsonb, updated_at = NOW() WHERE id = $2`,
metaJSON, id)
return err
}
func (s *FileStore) SetExtractedText(ctx context.Context, id string, text string) error {
_, err := DB.ExecContext(ctx,
`UPDATE files SET extracted_text = $1, updated_at = NOW() WHERE id = $2`,
text, id)
return err
}
func (s *FileStore) Delete(ctx context.Context, id string) (*models.File, error) {
row := DB.QueryRowContext(ctx,
`DELETE FROM files WHERE id = $1 RETURNING `+fileCols, id)
return scanFile(row)
}
func (s *FileStore) DeleteByChannel(ctx context.Context, channelID string) ([]string, error) {
rows, err := DB.QueryContext(ctx,
`DELETE FROM files WHERE channel_id = $1 RETURNING storage_key`, channelID)
if err != nil {
return nil, err
}
defer rows.Close()
var keys []string
for rows.Next() {
var key string
if err := rows.Scan(&key); err != nil {
return keys, err
}
keys = append(keys, key)
}
return keys, rows.Err()
}
func (s *FileStore) UserUsageBytes(ctx context.Context, userID string) (int64, error) {
var total sql.NullInt64
err := DB.QueryRowContext(ctx,
`SELECT COALESCE(SUM(size_bytes), 0) FROM files WHERE user_id = $1`,
userID).Scan(&total)
if err != nil {
return 0, err
}
return total.Int64, nil
}
func (s *FileStore) ListOrphans(ctx context.Context, olderThan time.Duration) ([]models.File, error) {
cutoff := time.Now().Add(-olderThan)
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files
WHERE message_id IS NULL AND origin = 'user_upload' AND created_at < $1
ORDER BY created_at`, cutoff)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}

2
server/store/postgres/stores.go
View File

@@ -27,7 +27,7 @@ func NewStores(db *sql.DB) store.Stores {
Usage: NewUsageStore(),
Pricing: NewPricingStore(),
Extensions: NewExtensionStore(),
Attachments: NewAttachmentStore(),
Files: NewFileStore(),
KnowledgeBases: NewKnowledgeBaseStore(),
Groups: NewGroupStore(),
ResourceGrants: NewResourceGrantStore(),

213
server/store/sqlite/attachment.go
View File

@@ -1,213 +0,0 @@
package sqlite
import (
"context"
"database/sql"
"encoding/json"
"time"
"git.gobha.me/xcaliber/chat-switchboard/models"
"git.gobha.me/xcaliber/chat-switchboard/store"
)
type AttachmentStore struct{}
func NewAttachmentStore() *AttachmentStore { return &AttachmentStore{} }
// ── columns shared across queries ──────────
const attachmentCols = `id, channel_id, user_id, message_id, project_id, filename, content_type,
size_bytes, storage_key, extracted_text, metadata, created_at`
// scanAttachment scans a row into an Attachment struct.
func scanAttachment(row interface{ Scan(dest ...interface{}) error }) (*models.Attachment, error) {
var a models.Attachment
var messageID sql.NullString
var projectID sql.NullString
var extractedText sql.NullString
var metadataJSON []byte
err := row.Scan(
&a.ID, &a.ChannelID, &a.UserID, &messageID, &projectID,
&a.Filename, &a.ContentType, &a.SizeBytes,
&a.StorageKey, &extractedText, &metadataJSON, st(&a.CreatedAt),
)
if err != nil {
return nil, err
}
a.MessageID = NullableStringPtr(messageID)
a.ProjectID = NullableStringPtr(projectID)
if extractedText.Valid {
a.ExtractedText = &extractedText.String
}
if len(metadataJSON) > 0 {
json.Unmarshal(metadataJSON, &a.Metadata)
}
return &a, nil
}
func (s *AttachmentStore) Create(ctx context.Context, a *models.Attachment) error {
a.ID = store.NewID()
a.CreatedAt = time.Now().UTC()
_, err := DB.ExecContext(ctx, `
INSERT INTO attachments (id, channel_id, user_id, message_id, project_id, filename, content_type,
size_bytes, storage_key, extracted_text, metadata, created_at)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?)`,
a.ID, a.ChannelID, a.UserID, models.NullString(a.MessageID),
models.NullString(a.ProjectID),
a.Filename, a.ContentType, a.SizeBytes,
a.StorageKey, models.NullString(a.ExtractedText),
ToJSON(a.Metadata), a.CreatedAt.Format(timeFmt),
)
return err
}
func (s *AttachmentStore) GetByID(ctx context.Context, id string) (*models.Attachment, error) {
row := DB.QueryRowContext(ctx, `SELECT `+attachmentCols+` FROM attachments WHERE id = ?`, id)
return scanAttachment(row)
}
func (s *AttachmentStore) GetByChannel(ctx context.Context, channelID string) ([]models.Attachment, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments WHERE channel_id = ? ORDER BY created_at`, channelID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}
func (s *AttachmentStore) GetByMessage(ctx context.Context, messageID string) ([]models.Attachment, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments WHERE message_id = ? ORDER BY created_at`, messageID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}
// GetByProject returns all attachments for a project (v0.22.4).
func (s *AttachmentStore) GetByProject(ctx context.Context, projectID string) ([]models.Attachment, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments WHERE project_id = ? ORDER BY created_at`, projectID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}
func (s *AttachmentStore) SetMessageID(ctx context.Context, attachmentID, messageID string) error {
_, err := DB.ExecContext(ctx,
`UPDATE attachments SET message_id = ? WHERE id = ?`,
messageID, attachmentID)
return err
}
func (s *AttachmentStore) UpdateMetadata(ctx context.Context, id string, metadata map[string]interface{}) error {
// Merge into existing metadata using jsonb || operator
metaJSON, err := json.Marshal(metadata)
if err != nil {
return err
}
_, err = DB.ExecContext(ctx,
`UPDATE attachments SET metadata = metadata || ? WHERE id = ?`,
metaJSON, id)
return err
}
func (s *AttachmentStore) SetExtractedText(ctx context.Context, id string, text string) error {
_, err := DB.ExecContext(ctx,
`UPDATE attachments SET extracted_text = ? WHERE id = ?`,
text, id)
return err
}
// Delete removes an attachment and returns the deleted row (for storage cleanup).
func (s *AttachmentStore) Delete(ctx context.Context, id string) (*models.Attachment, error) {
row := DB.QueryRowContext(ctx,
`DELETE FROM attachments WHERE id = ?
RETURNING `+attachmentCols, id)
return scanAttachment(row)
}
// DeleteByChannel removes all attachments for a channel and returns storage keys.
func (s *AttachmentStore) DeleteByChannel(ctx context.Context, channelID string) ([]string, error) {
rows, err := DB.QueryContext(ctx,
`DELETE FROM attachments WHERE channel_id = ? RETURNING storage_key`, channelID)
if err != nil {
return nil, err
}
defer rows.Close()
var keys []string
for rows.Next() {
var key string
if err := rows.Scan(&key); err != nil {
return keys, err
}
keys = append(keys, key)
}
return keys, rows.Err()
}
func (s *AttachmentStore) UserUsageBytes(ctx context.Context, userID string) (int64, error) {
var total sql.NullInt64
err := DB.QueryRowContext(ctx,
`SELECT COALESCE(SUM(size_bytes), 0) FROM attachments WHERE user_id = ?`,
userID).Scan(&total)
if err != nil {
return 0, err
}
return total.Int64, nil
}
func (s *AttachmentStore) ListOrphans(ctx context.Context, olderThan time.Duration) ([]models.Attachment, error) {
cutoff := time.Now().Add(-olderThan)
rows, err := DB.QueryContext(ctx,
`SELECT `+attachmentCols+` FROM attachments
WHERE message_id IS NULL AND created_at < ?
ORDER BY created_at`, cutoff)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.Attachment
for rows.Next() {
a, err := scanAttachment(rows)
if err != nil {
return nil, err
}
out = append(out, *a)
}
return out, rows.Err()
}

256
server/store/sqlite/file.go Normal file
View File

@@ -0,0 +1,256 @@
package sqlite
import (
"context"
"database/sql"
"encoding/json"
"time"
"git.gobha.me/xcaliber/chat-switchboard/models"
"git.gobha.me/xcaliber/chat-switchboard/store"
)
// FileStore implements store.FileStore against the `files` table.
//
type FileStore struct{}
func NewFileStore() *FileStore { return &FileStore{} }
const fileCols = `id, channel_id, user_id, message_id, project_id, origin,
filename, content_type, size_bytes, storage_key, display_hint,
extracted_text, metadata, created_at, updated_at`
func scanFile(row interface{ Scan(dest ...interface{}) error }) (*models.File, error) {
var f models.File
var messageID sql.NullString
var projectID sql.NullString
var extractedText sql.NullString
var metadataJSON []byte
err := row.Scan(
&f.ID, &f.ChannelID, &f.UserID, &messageID, &projectID, &f.Origin,
&f.Filename, &f.ContentType, &f.SizeBytes,
&f.StorageKey, &f.DisplayHint,
&extractedText, &metadataJSON, st(&f.CreatedAt), st(&f.UpdatedAt),
)
if err != nil {
return nil, err
}
f.MessageID = NullableStringPtr(messageID)
f.ProjectID = NullableStringPtr(projectID)
if extractedText.Valid {
f.ExtractedText = &extractedText.String
}
if len(metadataJSON) > 0 {
json.Unmarshal(metadataJSON, &f.Metadata)
}
return &f, nil
}
func (s *FileStore) Create(ctx context.Context, f *models.File) error {
f.ID = store.NewID()
now := time.Now().UTC()
f.CreatedAt = now
f.UpdatedAt = now
if f.Origin == "" {
f.Origin = models.FileOriginUserUpload
}
if f.DisplayHint == "" {
f.DisplayHint = models.FileHintDownload
}
_, err := DB.ExecContext(ctx, `
INSERT INTO files (id, channel_id, user_id, message_id, project_id, origin,
filename, content_type, size_bytes, storage_key, display_hint,
extracted_text, metadata, created_at, updated_at)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`,
f.ID, f.ChannelID, f.UserID, models.NullString(f.MessageID),
models.NullString(f.ProjectID), f.Origin,
f.Filename, f.ContentType, f.SizeBytes,
f.StorageKey, f.DisplayHint, models.NullString(f.ExtractedText),
ToJSON(f.Metadata), now.Format(timeFmt), now.Format(timeFmt),
)
return err
}
func (s *FileStore) GetByID(ctx context.Context, id string) (*models.File, error) {
row := DB.QueryRowContext(ctx, `SELECT `+fileCols+` FROM files WHERE id = ?`, id)
return scanFile(row)
}
func (s *FileStore) GetByChannel(ctx context.Context, channelID string, origin string) ([]models.File, error) {
var rows *sql.Rows
var err error
if origin != "" {
rows, err = DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE channel_id = ? AND origin = ? ORDER BY created_at`,
channelID, origin)
} else {
rows, err = DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE channel_id = ? ORDER BY created_at`,
channelID)
}
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}
func (s *FileStore) GetByMessage(ctx context.Context, messageID string) ([]models.File, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE message_id = ? ORDER BY created_at`, messageID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}
func (s *FileStore) GetByProject(ctx context.Context, projectID string) ([]models.File, error) {
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE project_id = ? ORDER BY created_at`, projectID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}
func (s *FileStore) GetByUser(ctx context.Context, userID string, page, perPage int) ([]models.File, int, error) {
var total int
err := DB.QueryRowContext(ctx,
`SELECT COUNT(*) FROM files WHERE user_id = ?`, userID).Scan(&total)
if err != nil {
return nil, 0, err
}
offset := (page - 1) * perPage
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files WHERE user_id = ?
ORDER BY created_at DESC LIMIT ? OFFSET ?`, userID, perPage, offset)
if err != nil {
return nil, 0, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, total, err
}
out = append(out, *f)
}
return out, total, rows.Err()
}
func (s *FileStore) SetMessageID(ctx context.Context, fileID, messageID string) error {
_, err := DB.ExecContext(ctx,
`UPDATE files SET message_id = ?, updated_at = ? WHERE id = ?`,
messageID, time.Now().UTC().Format(timeFmt), fileID)
return err
}
func (s *FileStore) UpdateMetadata(ctx context.Context, id string, metadata map[string]interface{}) error {
metaJSON, err := json.Marshal(metadata)
if err != nil {
return err
}
_, err = DB.ExecContext(ctx,
`UPDATE files SET metadata = json_patch(metadata, ?), updated_at = ? WHERE id = ?`,
string(metaJSON), time.Now().UTC().Format(timeFmt), id)
return err
}
func (s *FileStore) SetExtractedText(ctx context.Context, id string, text string) error {
_, err := DB.ExecContext(ctx,
`UPDATE files SET extracted_text = ?, updated_at = ? WHERE id = ?`,
text, time.Now().UTC().Format(timeFmt), id)
return err
}
func (s *FileStore) Delete(ctx context.Context, id string) (*models.File, error) {
row := DB.QueryRowContext(ctx,
`DELETE FROM files WHERE id = ? RETURNING `+fileCols, id)
return scanFile(row)
}
func (s *FileStore) DeleteByChannel(ctx context.Context, channelID string) ([]string, error) {
rows, err := DB.QueryContext(ctx,
`DELETE FROM files WHERE channel_id = ? RETURNING storage_key`, channelID)
if err != nil {
return nil, err
}
defer rows.Close()
var keys []string
for rows.Next() {
var key string
if err := rows.Scan(&key); err != nil {
return keys, err
}
keys = append(keys, key)
}
return keys, rows.Err()
}
func (s *FileStore) UserUsageBytes(ctx context.Context, userID string) (int64, error) {
var total sql.NullInt64
err := DB.QueryRowContext(ctx,
`SELECT COALESCE(SUM(size_bytes), 0) FROM files WHERE user_id = ?`,
userID).Scan(&total)
if err != nil {
return 0, err
}
return total.Int64, nil
}
func (s *FileStore) ListOrphans(ctx context.Context, olderThan time.Duration) ([]models.File, error) {
cutoff := time.Now().Add(-olderThan)
rows, err := DB.QueryContext(ctx,
`SELECT `+fileCols+` FROM files
WHERE message_id IS NULL AND origin = 'user_upload' AND created_at < ?
ORDER BY created_at`, cutoff)
if err != nil {
return nil, err
}
defer rows.Close()
var out []models.File
for rows.Next() {
f, err := scanFile(rows)
if err != nil {
return nil, err
}
out = append(out, *f)
}
return out, rows.Err()
}

2
server/store/sqlite/stores.go
View File

@@ -27,7 +27,7 @@ func NewStores(db *sql.DB) store.Stores {
Usage: NewUsageStore(),
Pricing: NewPricingStore(),
Extensions: NewExtensionStore(),
Attachments: NewAttachmentStore(),
Files: NewFileStore(),
KnowledgeBases: NewKnowledgeBaseStore(),
Groups: NewGroupStore(),
ResourceGrants: NewResourceGrantStore(),

68
server/tools/attachment_recall.go → server/tools/file_recall.go
View File

@@ -14,52 +14,52 @@ import (
)
// ── Late Registration ────────────────────────
// attachment_recall needs stores + objStore, which aren't available at
// file_recall needs stores + objStore, which aren't available at
// init time. Called from main.go after storage init.
// If objStore is nil (storage not configured), the tool is not registered.
func RegisterAttachmentRecall(stores store.Stores, objStore storage.ObjectStore) {
func RegisterFileRecall(stores store.Stores, objStore storage.ObjectStore) {
if objStore == nil {
log.Printf("ℹ attachment_recall: storage not configured, tool not registered")
log.Printf("ℹ file_recall: storage not configured, tool not registered")
return
}
Register(&attachmentRecallTool{
Register(&fileRecallTool{
stores: stores,
objStore: objStore,
})
}
// ═══════════════════════════════════════════
// attachment_recall
// file_recall
// ═══════════════════════════════════════════
const maxImageBytes = 10 * 1024 * 1024 // 10 MB cap for base64 images
type attachmentRecallTool struct {
type fileRecallTool struct {
stores store.Stores
objStore storage.ObjectStore
}
func (t *attachmentRecallTool) Definition() ToolDef {
func (t *fileRecallTool) Definition() ToolDef {
return ToolDef{
Name: "attachment_recall",
DisplayName: "Attachments",
Name: "file_recall",
DisplayName: "Files",
Category: "context",
Description: "Re-read file attachments from this conversation. " +
Description: "Re-read files from this conversation. " +
"Use action='list' to see available files, then action='read' " +
"with an attachment_id to retrieve the file content. " +
"with a file_id to retrieve the file content. " +
"Documents return extracted text; images return base64 data.",
Parameters: JSONSchema(map[string]interface{}{
"action": PropEnum("Action: 'list' to see files, 'read' to get content", "list", "read"),
"attachment_id": Prop("string", "Attachment ID to read (required for action='read')"),
"file_id": Prop("string", "File ID to read (required for action='read')"),
}, []string{"action"}),
}
}
func (t *attachmentRecallTool) Execute(ctx context.Context, execCtx ExecutionContext, argsJSON string) (string, error) {
func (t *fileRecallTool) Execute(ctx context.Context, execCtx ExecutionContext, argsJSON string) (string, error) {
var args struct {
Action string `json:"action"`
AttachmentID string `json:"attachment_id"`
FileID string `json:"file_id"`
}
if err := json.Unmarshal([]byte(argsJSON), &args); err != nil {
return "", fmt.Errorf("invalid arguments: %w", err)
@@ -67,22 +67,22 @@ func (t *attachmentRecallTool) Execute(ctx context.Context, execCtx ExecutionCon
switch args.Action {
case "list":
return t.listAttachments(ctx, execCtx)
return t.listFiles(ctx, execCtx)
case "read":
if args.AttachmentID == "" {
return "", fmt.Errorf("attachment_id is required for action='read'")
if args.FileID == "" {
return "", fmt.Errorf("file_id is required for action='read'")
}
return t.readAttachment(ctx, execCtx, args.AttachmentID)
return t.readFile(ctx, execCtx, args.FileID)
default:
return "", fmt.Errorf("invalid action: %q (use 'list' or 'read')", args.Action)
}
}
// listAttachments returns metadata for all attachments in the channel.
func (t *attachmentRecallTool) listAttachments(ctx context.Context, execCtx ExecutionContext) (string, error) {
atts, err := t.stores.Attachments.GetByChannel(ctx, execCtx.ChannelID)
// listFiles returns metadata for all files in the channel.
func (t *fileRecallTool) listFiles(ctx context.Context, execCtx ExecutionContext) (string, error) {
atts, err := t.stores.Files.GetByChannel(ctx, execCtx.ChannelID, "")
if err != nil {
return "", fmt.Errorf("failed to list attachments: %w", err)
return "", fmt.Errorf("failed to list files: %w", err)
}
type attInfo struct {
@@ -106,32 +106,32 @@ func (t *attachmentRecallTool) listAttachments(ctx context.Context, execCtx Exec
})
}
log.Printf("📎 attachment_recall: list → %d files in channel %s", len(items), execCtx.ChannelID)
log.Printf("📎 file_recall: list → %d files in channel %s", len(items), execCtx.ChannelID)
out, _ := json.Marshal(map[string]interface{}{
"attachments": items,
"files": items,
"count": len(items),
"channel_id": execCtx.ChannelID,
})
return string(out), nil
}
// readAttachment returns the content of a specific attachment.
func (t *attachmentRecallTool) readAttachment(ctx context.Context, execCtx ExecutionContext, attachmentID string) (string, error) {
att, err := t.stores.Attachments.GetByID(ctx, attachmentID)
// readFile returns the content of a specific file.
func (t *fileRecallTool) readFile(ctx context.Context, execCtx ExecutionContext, fileID string) (string, error) {
att, err := t.stores.Files.GetByID(ctx, fileID)
if err != nil {
return "", fmt.Errorf("attachment not found: %s", attachmentID)
return "", fmt.Errorf("file not found: %s", fileID)
}
// Security: verify attachment belongs to this channel
// Security: verify file belongs to this channel
if att.ChannelID != execCtx.ChannelID {
return "", fmt.Errorf("attachment %s does not belong to this conversation", attachmentID)
return "", fmt.Errorf("file %s does not belong to this conversation", fileID)
}
// Documents: return extracted text
if !isImageType(att.ContentType) {
if att.ExtractedText != nil && *att.ExtractedText != "" {
log.Printf("📎 attachment_recall: read text → %s (%s, %d chars)",
log.Printf("📎 file_recall: read text → %s (%s, %d chars)",
att.Filename, att.ID, len(*att.ExtractedText))
out, _ := json.Marshal(map[string]interface{}{
"id": att.ID,
@@ -153,20 +153,20 @@ func (t *attachmentRecallTool) readAttachment(ctx context.Context, execCtx Execu
reader, size, _, err := t.objStore.Get(ctx, att.StorageKey)
if err != nil {
return "", fmt.Errorf("failed to read attachment from storage: %w", err)
return "", fmt.Errorf("failed to read file from storage: %w", err)
}
defer reader.Close()
// Read and base64-encode
data := make([]byte, size)
if _, err := io.ReadFull(reader, data); err != nil {
return "", fmt.Errorf("failed to read attachment data: %w", err)
return "", fmt.Errorf("failed to read file data: %w", err)
}
b64 := base64.StdEncoding.EncodeToString(data)
dataURI := fmt.Sprintf("data:%s;base64,%s", att.ContentType, b64)
log.Printf("📎 attachment_recall: read image → %s (%s, %d bytes)",
log.Printf("📎 file_recall: read image → %s (%s, %d bytes)",
att.Filename, att.ID, size)
out, _ := json.Marshal(map[string]interface{}{

2
src/css/persona-kb.css
View File

@@ -70,7 +70,7 @@
opacity: 0.8;
}
/* ── Persona-KB Section (in preset form) ─── */
/* ── Persona-KB Section (in persona form) ─── */
.persona-kb-section {
margin: 0.75rem 0;

8
src/css/styles.css
View File

@@ -1125,12 +1125,12 @@ a:hover { text-decoration: underline; }
.stop-btn { display: none; color: var(--warning); }
.stop-btn.visible { display: flex; }
/* ── Attachments ─────────────────────────── */
/* ── Files ─────────────────────────── */
.attach-btn { color: var(--text-3); }
.attach-btn:hover { color: var(--text); }
.attachment-strip {
.file-strip {
max-width: 768px; margin: 0 auto; padding: 6px 8px 2px;
display: flex; gap: 6px; flex-wrap: wrap;
}
@@ -1169,9 +1169,9 @@ a:hover { text-decoration: underline; }
background: color-mix(in srgb, var(--accent) 4%, var(--bg));
}
/* ── Message Attachments ────────────────────── */
/* ── Message Files ────────────────────── */
.msg-attachments {
.msg-files {
display: flex; gap: 8px; flex-wrap: wrap;
margin-top: 8px;
}

10
src/js/admin-handlers.js
View File

@@ -291,11 +291,11 @@ async function createAdminPersona(vals) {
try {
let personaId = _editingPersonaId;
if (_editingPersonaId) {
await API.adminUpdatePersona(_editingPersonaId, preset);
await API.adminUpdatePersona(_editingPersonaId, persona);
UI.toast('Persona updated', 'success');
} else {
const result = await API.adminCreatePersona(preset);
personaId = result?.id || result?.preset?.id;
const result = await API.adminCreatePersona(persona);
personaId = result?.id || result?.persona?.id;
UI.toast('Persona created', 'success');
}
@@ -307,7 +307,7 @@ async function createAdminPersona(vals) {
// Save KB bindings (v0.17.0)
if (personaId && vals._kbValues && _adminPersonaForm?.kbPicker) {
try { await API.adminSetPresetKBs(personaId, vals._kbValues); }
try { await API.adminSetPersonaKBs(personaId, vals._kbValues); }
catch (e) { console.warn('Persona KB binding failed:', e.message); }
}
@@ -403,7 +403,7 @@ async function settingsRemoveTeamMember(teamId, memberId, email) {
async function settingsDeleteTeamPersona(teamId, personaId, name) {
if (!await showConfirm(`Delete team persona "${name}"?`)) return;
try {
await API.teamDeletePreset(teamId, personaId);
await API.teamDeletePersona(teamId, personaId);
UI.toast('Persona deleted');
await UI.loadTeamManagePersonas(teamId);
await fetchModels();

4
src/js/admin-scaffold.js
View File

@@ -146,11 +146,7 @@
'<div class="settings-section"><h4>Environment Banner</h4>' +
'<label class="checkbox-label"><input type="checkbox" id="adminBannerEnabled"> Show environment banner</label>' +
'<div id="bannerConfigFields" style="display:none;margin-top:8px">' +
'<div class="form-group"><label>Preset</label><select id="adminBannerPreset"><option value="">Custom</option></select></div>' +
'<div class="form-group"><label>Text</label><input type="text" id="adminBannerText" placeholder="DEVELOPMENT"></div>' +
'<div class="form-group"><label>Position</label>' +
'<select id="adminBannerPosition"><option value="both">Top &amp; Bottom</option><option value="top">Top</option><option value="bottom">Bottom</option></select>' +
'</div>' +
'<div class="form-row">' +
'<div class="form-group"><label>Background</label><input type="color" id="adminBannerBg" value="#007a33"><input type="text" id="adminBannerBgHex" value="#007a33" style="width:80px;margin-left:4px"></div>' +
'<div class="form-group"><label>Foreground</label><input type="color" id="adminBannerFg" value="#ffffff"><input type="text" id="adminBannerFgHex" value="#ffffff" style="width:80px;margin-left:4px"></div>' +

Some files were not shown because too many files have changed in this diff Show More