Changeset 0.15.0 (#71)
This commit is contained in:
231
docs/DESIGN-0.13.1.md
Normal file
231
docs/DESIGN-0.13.1.md
Normal file
@@ -0,0 +1,231 @@
|
||||
# DESIGN-0.13.1 — Web Search + URL Fetch + Tool Toggle
|
||||
|
||||
## Overview
|
||||
|
||||
Built-in `web_search` and `url_fetch` tools using the existing tool framework (v0.11.0),
|
||||
plus a chat-bar tools toggle menu so users can enable/disable tool categories per-session.
|
||||
|
||||
Depends on: tool framework (v0.11.0), admin panel (v0.13.0).
|
||||
|
||||
---
|
||||
|
||||
## 1. Tool Categories
|
||||
|
||||
Add a `Category` field to `ToolDef` so tools self-declare their group.
|
||||
The frontend uses categories for the toggle menu; the backend ignores them.
|
||||
|
||||
| Category | Tools | Default |
|
||||
|-----------|---------------------------------------------|---------|
|
||||
| search | `web_search`, `url_fetch` | ON |
|
||||
| notes | `note_create`, `note_search`, `note_update`, `note_list` | ON |
|
||||
| utilities | `datetime`, `calculator` | ON |
|
||||
| browser | (extension-defined: `js_eval`, `regex_test`)| ON |
|
||||
|
||||
```go
|
||||
type ToolDef struct {
|
||||
Name string `json:"name"`
|
||||
DisplayName string `json:"display_name,omitempty"` // human-readable label for UI
|
||||
Description string `json:"description"`
|
||||
Parameters json.RawMessage `json:"parameters"`
|
||||
Category string `json:"category,omitempty"` // NEW
|
||||
}
|
||||
```
|
||||
|
||||
## 2. Per-Request Tool Filtering
|
||||
|
||||
### Request field
|
||||
```json
|
||||
{ "disabled_tools": ["web_search", "url_fetch"] }
|
||||
```
|
||||
|
||||
### Backend filter
|
||||
`buildToolDefs()` skips tools whose name appears in `disabled_tools`.
|
||||
Zero API changes — the field is optional, empty = all tools enabled.
|
||||
|
||||
### Frontend state
|
||||
`localStorage` key: `cs-disabled-tools` → JSON array of tool names.
|
||||
The tools toggle menu flips categories on/off which maps to individual tool names.
|
||||
|
||||
## 3. web_search Tool
|
||||
|
||||
### Search Provider Interface
|
||||
```go
|
||||
type SearchProvider interface {
|
||||
Search(ctx context.Context, query string, maxResults int) ([]SearchResult, error)
|
||||
Name() string
|
||||
}
|
||||
|
||||
type SearchResult struct {
|
||||
Title string `json:"title"`
|
||||
URL string `json:"url"`
|
||||
Snippet string `json:"snippet"`
|
||||
}
|
||||
```
|
||||
|
||||
### DuckDuckGo Provider (default — no API key)
|
||||
Uses the DuckDuckGo HTML search endpoint. No rate-limit key required.
|
||||
Falls back gracefully if blocked (returns error result to LLM, doesn't crash).
|
||||
|
||||
### SearXNG Provider (self-hosted)
|
||||
Configurable endpoint + optional API key. JSON API (`/search?format=json`).
|
||||
Good fit for airgapped deployments.
|
||||
|
||||
### Admin Config
|
||||
Stored in `global_config` table (existing):
|
||||
- `search_provider`: `"duckduckgo"` | `"searxng"` (default: `"duckduckgo"`)
|
||||
- `search_endpoint`: SearXNG URL (only used when provider=searxng)
|
||||
- `search_max_results`: `5` (default)
|
||||
|
||||
Admin UI: System → Settings section (new "Search" subsection).
|
||||
|
||||
### Tool Schema
|
||||
```json
|
||||
{
|
||||
"name": "web_search",
|
||||
"description": "Search the web for current information. Returns titles, URLs, and snippets.",
|
||||
"category": "search",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": { "type": "string", "description": "Search query" },
|
||||
"max_results": { "type": "integer", "description": "Max results (1-10, default 5)" }
|
||||
},
|
||||
"required": ["query"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 4. url_fetch Tool
|
||||
|
||||
HTTP GET with content extraction. Reuses the text extraction pattern
|
||||
from file handling (v0.12.0) where possible.
|
||||
|
||||
### Behavior
|
||||
1. HTTP GET with reasonable timeout (15s) and User-Agent
|
||||
2. Follow redirects (max 3)
|
||||
3. Extract readable text content (strip HTML tags, scripts, styles)
|
||||
4. Truncate to ~8000 chars to avoid flooding the context
|
||||
5. Return title + extracted text + metadata
|
||||
|
||||
### Safety
|
||||
- Respect robots.txt? No — the LLM is fetching on behalf of the user, not crawling.
|
||||
- Block private/internal IPs (10.x, 192.168.x, 127.x, ::1) to prevent SSRF.
|
||||
- Configurable domain allowlist/blocklist in admin settings (future).
|
||||
|
||||
### Tool Schema
|
||||
```json
|
||||
{
|
||||
"name": "url_fetch",
|
||||
"description": "Fetch and extract readable text content from a URL.",
|
||||
"category": "search",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"url": { "type": "string", "description": "URL to fetch" },
|
||||
"max_length": { "type": "integer", "description": "Max content length in chars (default 8000)" }
|
||||
},
|
||||
"required": ["url"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 5. Frontend: Tools Toggle Menu
|
||||
|
||||
### UI Position
|
||||
Chat input bar, left side, next to the attach button. Wrench/tool icon.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────┐
|
||||
│ 📎 🔧 [message input...] [Send]│
|
||||
└──────────────────────────────────────┘
|
||||
▲
|
||||
┌────────────────┐
|
||||
│ ☑ Web Search │
|
||||
│ ☑ Notes │
|
||||
│ ☑ Utilities │
|
||||
│ ☑ Extensions │
|
||||
└────────────────┘
|
||||
```
|
||||
|
||||
### Behavior
|
||||
- Click tool icon → popup above (using existing popup-menu pattern)
|
||||
- Each category row has a toggle checkbox + expand chevron
|
||||
- **Category toggle**: master on/off for all tools in the category
|
||||
- **Expand chevron**: shows individual tool rows indented below
|
||||
- **Tri-state checkbox**: if some tools in a category are disabled, category shows indeterminate (─)
|
||||
- Toggling a category adds/removes all its tool names from `cs-disabled-tools`
|
||||
- Toggling an individual tool adds/removes just that name
|
||||
- State persists across page reloads via localStorage
|
||||
- Disabled tools sent as `disabled_tools` array in completion request
|
||||
- Tool icon shows visual indicator when any category is disabled
|
||||
- If model doesn't support tool_calling, icon is hidden/grayed
|
||||
|
||||
```
|
||||
┌────────────────────────┐
|
||||
│ ☑ 🔍 Web Search › │ ← click › to expand
|
||||
│ ☑ Search │
|
||||
│ ☑ Fetch URL │
|
||||
│ ☑ 📝 Notes › │
|
||||
│ ☐ 🧮 Utilities › │ ← category off = all children off
|
||||
│ ☑ 🧩 Extensions › │
|
||||
└────────────────────────┘
|
||||
```
|
||||
|
||||
### Tool Manifest
|
||||
Frontend needs to know the category mapping and display names.
|
||||
|
||||
**API endpoint** — `GET /api/v1/tools` returns `[{name, display_name, category, description}]`
|
||||
The frontend caches this on login. `display_name` provides human-readable labels
|
||||
(e.g. "Search" instead of "web_search", "Create" instead of "note_create").
|
||||
|
||||
### New API Endpoint
|
||||
```
|
||||
GET /api/v1/tools → [{name, description, category}]
|
||||
```
|
||||
Returns server-side tools + browser extension tool schemas.
|
||||
Used by frontend to build the toggle menu dynamically.
|
||||
|
||||
## 6. File Structure
|
||||
|
||||
### New backend files
|
||||
- `server/tools/websearch.go` — web_search tool + search provider interface
|
||||
- `server/tools/urlfetch.go` — url_fetch tool
|
||||
- `server/tools/search/duckduckgo.go` — DuckDuckGo provider
|
||||
- `server/tools/search/searxng.go` — SearXNG provider
|
||||
- `server/tools/search/provider.go` — interface + registry
|
||||
|
||||
### Modified backend files
|
||||
- `server/tools/types.go` — add Category to ToolDef
|
||||
- `server/tools/registry.go` — add AllDefinitionsFiltered(disabled)
|
||||
- `server/handlers/completion.go` — accept disabled_tools, filter in buildToolDefs
|
||||
- `server/handlers/messages.go` — same filtering
|
||||
- `server/handlers/routes.go` — add GET /api/v1/tools endpoint
|
||||
- `server/tools/datetime.go` — add Category: "utilities"
|
||||
- `server/tools/calculator.go` — add Category: "utilities"
|
||||
- `server/tools/notes.go` — add Category: "notes"
|
||||
|
||||
### New/modified frontend files
|
||||
- `src/index.html` — tools toggle button in input-wrap
|
||||
- `src/css/styles.css` — tools popup styling
|
||||
- `src/js/chat.js` — send disabled_tools in completion, wire toggle
|
||||
- `src/js/api.js` — add disabled_tools param, add getTools() method
|
||||
|
||||
## 7. Phased Delivery
|
||||
|
||||
**Phase 1: Backend tools + filtering**
|
||||
- ToolDef.Category, disabled_tools filtering
|
||||
- web_search + url_fetch tools
|
||||
- DuckDuckGo provider
|
||||
- GET /api/v1/tools endpoint
|
||||
- Tests
|
||||
|
||||
**Phase 2: Frontend toggle**
|
||||
- Tools toggle popup menu
|
||||
- localStorage persistence
|
||||
- Completion request wiring
|
||||
- CSS
|
||||
|
||||
**Phase 3: Admin config + SearXNG**
|
||||
- Search provider settings in admin panel
|
||||
- SearXNG provider implementation
|
||||
- Domain filtering
|
||||
Reference in New Issue
Block a user