# DESIGN-0.15.1 — Context Recall Tools ## Overview After compaction (v0.15.0) or in long conversations, the model loses access to earlier material — attachment content injected once at send time, and message details compressed into summaries. These tools let the model re-read source material on demand. Depends on: file handling (v0.12.0), tool framework (v0.11.0), compaction (v0.15.0). **Four work streams:** 1. `attachment_recall` — re-read attachment content from storage 2. `conversation_search` — keyword search over channel messages 3. Token estimator improvements — attachment-aware context counting 4. KB auto-injection — automatic knowledge base context in system prompt --- ## 1. `attachment_recall` Tool ### Purpose Attachments are injected into context once (at the user message that references them). On subsequent turns the model relies on its own response as a natural summary. After compaction, that summary is gone. This tool lets the model re-read any attachment in the current channel. ### Two-action design Single tool, `action` parameter selects behavior: - **`list`** — returns metadata for all attachments in the channel (id, filename, content_type, size, created_at). The model calls this first to discover what's available. - **`read`** — returns content for a specific attachment_id. Documents → `extracted_text` from DB. Images → base64 data URI. ### Security Channel-scoped: `attachment_id` must belong to `ExecutionContext.ChannelID`. The store query is `GetByChannel(channelID)` so cross-channel access is impossible by construction. ### Dependencies Late registration (same pattern as `kb_search`): ```go func RegisterAttachmentRecall(stores store.Stores, objStore storage.ObjectStore) ``` Called from `main.go` after storage init. If `objStore` is nil (storage not configured), the tool is **not registered** — it simply won't appear in the tool list. ### Implementation: `server/tools/attachment_recall.go` ```go type attachmentRecallTool struct { stores store.Stores objStore storage.ObjectStore } func (t *attachmentRecallTool) Definition() ToolDef { return ToolDef{ Name: "attachment_recall", DisplayName: "Attachments", Category: "context", Description: "Re-read file attachments from this conversation. " + "Use action='list' to see available files, then action='read' " + "with an attachment_id to get the content.", Parameters: JSONSchema(map[string]interface{}{ "action": PropEnum("Action to perform", "list", "read"), "attachment_id": Prop("string", "Attachment ID (required for read)"), }, []string{"action"}), } } ``` **`list` action returns:** ```json { "attachments": [ { "id": "abc-123", "filename": "report.pdf", "content_type": "application/pdf", "size_bytes": 45200, "has_text": true, "created_at": "2026-02-26T10:00:00Z" } ], "count": 1 } ``` **`read` action returns:** ```json { "id": "abc-123", "filename": "report.pdf", "content_type": "application/pdf", "content": "... extracted text ...", "content_type_returned": "text" } ``` For images: ```json { "id": "def-456", "filename": "diagram.png", "content_type": "image/png", "content": "data:image/png;base64,...", "content_type_returned": "base64_image" } ``` Image base64 is capped at 10MB to prevent context blow-up. Larger images return an error suggesting the user re-upload. --- ## 2. `conversation_search` Tool ### Purpose After compaction, message details are compressed into summaries. If the user asks about something the summary omitted, the model needs to search the original messages. Also useful in long pre-compaction conversations where the model's attention has drifted. ### Design Keyword search (full-text) over messages in the current channel using PostgreSQL's `plainto_tsquery`. No migration needed — `to_tsvector` is computed on the fly. This is acceptable because: - Search is scoped to a single channel (bounded message count) - Called infrequently (tool invocation, not every request) - Channel messages rarely exceed ~10K rows If performance becomes an issue, a `search_vector tsvector` column with GIN index can be added in a future migration. ### Parameters ```go Parameters: JSONSchema(map[string]interface{}{ "query": Prop("string", "Search keywords"), "max_results": map[string]interface{}{ "type": "integer", "description": "Maximum results (1-20, default 5)", }, "role_filter": PropEnum("Filter by message role", "all", "user", "assistant"), }, []string{"query"}) ``` ### Returns ```json { "results": [ { "message_id": "...", "role": "user", "excerpt": "...highlighted excerpt...", "timestamp": "2026-02-26T10:00:00Z", "rank": 0.95 } ], "query": "original query", "total": 3, "channel_id": "..." } ``` Uses `ts_headline` for highlighted excerpts (same pattern as `note_search`). ### Security Channel-scoped via `ExecutionContext.ChannelID`. Only searches messages in the current channel. Soft-deleted messages (`deleted_at IS NOT NULL`) are excluded. ### Dependencies Late registration: ```go func RegisterConversationSearch(stores store.Stores) ``` No external dependencies beyond stores (uses `database.DB` directly for the full-text query, same pattern as `note_search`). --- ## 3. Token Estimator Improvements ### Frontend (`tokens.js`) Current `estimateConversation` counts message text only. Enhancements: - **Staged attachments**: Before send, show estimated tokens for queued attachments. Documents: `extracted_text.length / 4`. Images: flat ~765 tokens per image (OpenAI's base tile estimate, reasonable cross-provider). - **Sent attachments**: After send, the augmented content is already in the message text (documents are inlined). Images were ephemeral ContentParts — add their estimate to the message's token count. ### Implementation Add to `Tokens`: ```javascript estimateAttachments(attachments) { let total = 0; for (const att of attachments) { if (att.content_type?.startsWith('image/')) { total += 765; // base tile estimate } else if (att.extracted_text) { total += this.estimate(att.extracted_text); } else { total += Math.ceil(att.size_bytes / 4); // rough fallback } } return total; } ``` Wire into `updateInputTokens()` to include staged attachment estimates. --- ## 4. KB Auto-Injection ### Purpose Currently the model must explicitly call `kb_search` to access knowledge base content. For common patterns (channel linked to a KB, user asks a question), automatic injection into the system prompt is more natural. ### Design In the completion handler, after loading conversation history and before building the provider request: 1. Check if channel has linked KBs (or user has personal KBs) 2. Extract the user's latest message as the query 3. Run similarity search (top-K, K=3 default) 4. If results found AND context budget allows, prepend to system prompt ### Context budget check ```go contextUsed := estimateTokens(systemPrompt + allMessages) kbTokens := estimateTokens(kbContent) if contextUsed + kbTokens > maxContext * 0.85 { // Skip injection — context too full log.Printf("⏭ KB auto-inject skipped: context %.0f%% full", pct*100) } ``` ### Channel settings ```json { "kb_injection": "auto" | "tool_only" | "both" | "off" } ``` Default: `"auto"` — inject when KBs are linked. `"tool_only"` disables auto-injection but keeps the `kb_search` tool available. `"both"` does both. `"off"` disables all KB context. ### System prompt format ``` [Knowledge Base Context] The following information was retrieved from the user's knowledge bases and may be relevant to this conversation: --- Source: Technical Docs (chunk 3 of report.pdf, similarity: 0.87) Content: ... --- Source: FAQ (chunk 1 of faq.md, similarity: 0.82) Content: ... [End Knowledge Base Context] ``` --- ## File Inventory ### New files | File | Description | |------|-------------| | `server/tools/attachment_recall.go` | attachment_recall tool | | `server/tools/conversation_search.go` | conversation_search tool | | `docs/DESIGN-0.15.1.md` | This design doc | ### Modified files | File | Change | |------|--------| | `server/main.go` | Register new tools | | `server/handlers/completion.go` | KB auto-injection in completion flow | | `src/js/tokens.js` | Attachment-aware token estimation | | `docs/ROADMAP.md` | Mark v0.15.1 items | ### No migration required - `conversation_search` uses on-the-fly `to_tsvector` (no new column) - `attachment_recall` reads existing `extracted_text` column - KB auto-injection uses existing `kb_search` infrastructure --- ## Implementation Order 1. **Phase 1**: `attachment_recall` tool + registration ✅ 2. **Phase 2**: `conversation_search` tool + registration ✅ 3. **Phase 3**: Token estimator improvements (frontend) ✅ 4. **Phase 4**: KB auto-injection (completion handler) — separate delivery, requires adding embedder dependency to `CompletionHandler` and careful latency budgeting (embedding query on every completion call)