core/DESIGN-0.13.1.md

# DESIGN-0.13.1 — Web Search + URL Fetch + Tool Toggle

## Overview

Built-in `web_search` and `url_fetch` tools using the existing tool framework (v0.11.0),
plus a chat-bar tools toggle menu so users can enable/disable tool categories per-session.

Depends on: tool framework (v0.11.0), admin panel (v0.13.0).

---

## 1. Tool Categories

Add a `Category` field to `ToolDef` so tools self-declare their group.
The frontend uses categories for the toggle menu; the backend ignores them.

| Category  | Tools                                       | Default |
|-----------|---------------------------------------------|---------|
| search    | `web_search`, `url_fetch`                   | ON      |
| notes     | `note_create`, `note_search`, `note_update`, `note_list` | ON |
| utilities | `datetime`, `calculator`                    | ON      |
| browser   | (extension-defined: `js_eval`, `regex_test`)| ON      |

```go
type ToolDef struct {
    Name        string          `json:"name"`
    DisplayName string          `json:"display_name,omitempty"` // human-readable label for UI
    Description string          `json:"description"`
    Parameters  json.RawMessage `json:"parameters"`
    Category    string          `json:"category,omitempty"` // NEW
}
```

## 2. Per-Request Tool Filtering

### Request field
```json
{ "disabled_tools": ["web_search", "url_fetch"] }
```

### Backend filter
`buildToolDefs()` skips tools whose name appears in `disabled_tools`.
Zero API changes — the field is optional, empty = all tools enabled.

### Frontend state
`localStorage` key: `cs-disabled-tools` → JSON array of tool names.
The tools toggle menu flips categories on/off which maps to individual tool names.

## 3. web_search Tool

### Search Provider Interface
```go
type SearchProvider interface {
    Search(ctx context.Context, query string, maxResults int) ([]SearchResult, error)
    Name() string
}

type SearchResult struct {
    Title   string `json:"title"`
    URL     string `json:"url"`
    Snippet string `json:"snippet"`
}
```

### DuckDuckGo Provider (default — no API key)
Uses the DuckDuckGo HTML search endpoint. No rate-limit key required.
Falls back gracefully if blocked (returns error result to LLM, doesn't crash).

### SearXNG Provider (self-hosted)
Configurable endpoint + optional API key. JSON API (`/search?format=json`).
Good fit for airgapped deployments.

### Admin Config
Stored in `global_config` table (existing):
- `search_provider`: `"duckduckgo"` | `"searxng"` (default: `"duckduckgo"`)
- `search_endpoint`: SearXNG URL (only used when provider=searxng)
- `search_max_results`: `5` (default)

Admin UI: System → Settings section (new "Search" subsection).

### Tool Schema
```json
{
  "name": "web_search",
  "description": "Search the web for current information. Returns titles, URLs, and snippets.",
  "category": "search",
  "parameters": {
    "type": "object",
    "properties": {
      "query": { "type": "string", "description": "Search query" },
      "max_results": { "type": "integer", "description": "Max results (1-10, default 5)" }
    },
    "required": ["query"]
  }
}
```

## 4. url_fetch Tool

HTTP GET with content extraction. Reuses the text extraction pattern
from file handling (v0.12.0) where possible.

### Behavior
1. HTTP GET with reasonable timeout (15s) and User-Agent
2. Follow redirects (max 3)
3. Extract readable text content (strip HTML tags, scripts, styles)
4. Truncate to ~8000 chars to avoid flooding the context
5. Return title + extracted text + metadata

### Safety
- Respect robots.txt? No — the LLM is fetching on behalf of the user, not crawling.
- Block private/internal IPs (10.x, 192.168.x, 127.x, ::1) to prevent SSRF.
- Configurable domain allowlist/blocklist in admin settings (future).

### Tool Schema
```json
{
  "name": "url_fetch",
  "description": "Fetch and extract readable text content from a URL.",
  "category": "search",
  "parameters": {
    "type": "object",
    "properties": {
      "url": { "type": "string", "description": "URL to fetch" },
      "max_length": { "type": "integer", "description": "Max content length in chars (default 8000)" }
    },
    "required": ["url"]
  }
}
```

## 5. Frontend: Tools Toggle Menu

### UI Position
Chat input bar, left side, next to the attach button. Wrench/tool icon.

```
┌──────────────────────────────────────┐
│ 📎  🔧  [message input...]    [Send]│
└──────────────────────────────────────┘
         ▲
    ┌────────────────┐
    │ ☑ Web Search   │
    │ ☑ Notes        │
    │ ☑ Utilities    │
    │ ☑ Extensions   │
    └────────────────┘
```

### Behavior
- Click tool icon → popup above (using existing popup-menu pattern)
- Each category row has a toggle checkbox + expand chevron
- **Category toggle**: master on/off for all tools in the category
- **Expand chevron**: shows individual tool rows indented below
- **Tri-state checkbox**: if some tools in a category are disabled, category shows indeterminate (─)
- Toggling a category adds/removes all its tool names from `cs-disabled-tools`
- Toggling an individual tool adds/removes just that name
- State persists across page reloads via localStorage
- Disabled tools sent as `disabled_tools` array in completion request
- Tool icon shows visual indicator when any category is disabled
- If model doesn't support tool_calling, icon is hidden/grayed

```
┌────────────────────────┐
│ ☑ 🔍 Web Search     › │  ← click › to expand
│   ☑ Search             │
│   ☑ Fetch URL          │
│ ☑ 📝 Notes           › │
│ ☐ 🧮 Utilities      › │  ← category off = all children off
│ ☑ 🧩 Extensions     › │
└────────────────────────┘
```

### Tool Manifest
Frontend needs to know the category mapping and display names.

**API endpoint** — `GET /api/v1/tools` returns `[{name, display_name, category, description}]`
The frontend caches this on login. `display_name` provides human-readable labels
(e.g. "Search" instead of "web_search", "Create" instead of "note_create").

### New API Endpoint
```
GET /api/v1/tools → [{name, description, category}]
```
Returns server-side tools + browser extension tool schemas.
Used by frontend to build the toggle menu dynamically.

## 6. File Structure

### New backend files
- `server/tools/websearch.go` — web_search tool + search provider interface
- `server/tools/urlfetch.go` — url_fetch tool
- `server/tools/search/duckduckgo.go` — DuckDuckGo provider
- `server/tools/search/searxng.go` — SearXNG provider
- `server/tools/search/provider.go` — interface + registry

### Modified backend files
- `server/tools/types.go` — add Category to ToolDef
- `server/tools/registry.go` — add AllDefinitionsFiltered(disabled)
- `server/handlers/completion.go` — accept disabled_tools, filter in buildToolDefs
- `server/handlers/messages.go` — same filtering
- `server/handlers/routes.go` — add GET /api/v1/tools endpoint
- `server/tools/datetime.go` — add Category: "utilities"
- `server/tools/calculator.go` — add Category: "utilities"
- `server/tools/notes.go` — add Category: "notes"

### New/modified frontend files
- `src/index.html` — tools toggle button in input-wrap
- `src/css/styles.css` — tools popup styling
- `src/js/chat.js` — send disabled_tools in completion, wire toggle
- `src/js/api.js` — add disabled_tools param, add getTools() method

## 7. Phased Delivery

**Phase 1: Backend tools + filtering**
- ToolDef.Category, disabled_tools filtering
- web_search + url_fetch tools
- DuckDuckGo provider
- GET /api/v1/tools endpoint
- Tests

**Phase 2: Frontend toggle**
- Tools toggle popup menu
- localStorage persistence
- Completion request wiring
- CSS

**Phase 3: Admin config + SearXNG**
- Search provider settings in admin panel
- SearXNG provider implementation
- Domain filtering