Changeset 0.17.2 (#77)
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
# Architecture — Chat Switchboard v0.11
|
||||
# Architecture — Chat Switchboard v0.17
|
||||
|
||||
## Deployment Modes
|
||||
|
||||
@@ -8,9 +8,9 @@ Three Docker images support different deployment scenarios:
|
||||
|-------|-----------|----------|----------|
|
||||
| **Unified** | `Dockerfile` | nginx + Go backend | Dev, docker-compose, single-node |
|
||||
| **Backend** | `server/Dockerfile` | Go binary only | K8s — scale API independently |
|
||||
| **Frontend** | `Dockerfile.frontend` | nginx + static files | K8s — scale FE independently |
|
||||
| **Frontend** | `Dockerfile.frontend` | nginx + static files + CM6 bundle | K8s — scale FE independently |
|
||||
|
||||
**Unified** bundles everything in one container. Nginx serves static files and proxies `/api/*` and `/ws` to the Go backend running on `:8080`. Good for development and small deployments.
|
||||
**Unified** bundles everything in one container (4-stage Docker build: vendor libs → CM6 bundle → Go backend → nginx runtime). Nginx serves static files and proxies `/api/*` and `/ws` to the Go backend running on `:8080`. Good for development and small deployments.
|
||||
|
||||
**Split (Backend + Frontend)** for production K8s: Ingress routes `/api/*` and `/ws` to the backend Service, everything else to the frontend Service. The frontend entrypoint (`docker-entrypoint-fe.sh`) handles `BASE_PATH` injection into `index.html` and dynamic nginx config generation at startup. Supports branding volume mounts at `/branding/`.
|
||||
|
||||
@@ -29,8 +29,8 @@ Three Docker images support different deployment scenarios:
|
||||
└──────┬──────┘ └────────────────┘
|
||||
│
|
||||
┌──────▼──────┐
|
||||
│ PostgreSQL │
|
||||
└─────────────┘
|
||||
│ PostgreSQL │ (or SQLite for
|
||||
└─────────────┘ single-node)
|
||||
```
|
||||
|
||||
## Design Principles
|
||||
@@ -39,7 +39,7 @@ Three Docker images support different deployment scenarios:
|
||||
|
||||
2. **Roles vs Teams**: Clean separation between vertical permissions (Roles: admin, user) and horizontal visibility (Teams: organizational units). A user's role determines what they can *do*; their team membership determines what they can *see*.
|
||||
|
||||
3. **Store Layer**: All database access goes through typed Go interfaces (`store.Stores`). Handlers never write raw SQL. This enables future portability (SQLite for dev, Postgres for prod) and testability (mock stores).
|
||||
3. **Store Layer**: All database access goes through typed Go interfaces (`store.Stores`). Handlers never write raw SQL. The store layer has two implementations — `store/postgres/` and `store/sqlite/` — selected at startup via `DB_DRIVER`. This enables Postgres for production and SQLite for single-node or air-gapped deployments.
|
||||
|
||||
4. **Scope Model**: Provider configs, personas, and model settings all use a three-value `scope` column: `global` (admin-managed, visible to all), `team` (team-admin-managed, visible to team), `personal` (user-managed, visible to owner). The `owner_id` column points to the owning user or team depending on scope.
|
||||
|
||||
@@ -112,37 +112,51 @@ server/
|
||||
├── main.go # Wiring: stores → handlers → routes
|
||||
├── config/config.go # Env-based configuration
|
||||
├── database/
|
||||
│ ├── database.go # Connection management
|
||||
│ ├── database.go # Connection management (PG + SQLite)
|
||||
│ ├── migrate.go # Auto-migration on startup
|
||||
│ └── migrations/
|
||||
│ └── 001_v09_schema.sql # Consolidated schema
|
||||
│ ├── 001_v016_schema.sql # Consolidated schema (PG)
|
||||
│ └── 002_v017_persona_kb.sql
|
||||
├── store/
|
||||
│ ├── interfaces.go # Store interfaces + shared types
|
||||
│ └── postgres/ # Postgres implementations
|
||||
│ ├── postgres/ # Postgres implementations (19 stores)
|
||||
│ │ ├── stores.go # NewStores() constructor
|
||||
│ │ ├── provider.go # ProviderStore
|
||||
│ │ ├── catalog.go # CatalogStore
|
||||
│ │ ├── persona.go # PersonaStore
|
||||
│ │ ├── channel.go # ChannelStore
|
||||
│ │ ├── message.go # MessageStore
|
||||
│ │ ├── user.go # UserStore
|
||||
│ │ ├── team.go # TeamStore
|
||||
│ │ ├── note.go # NoteStore
|
||||
│ │ ├── knowledge.go # KnowledgeStore
|
||||
│ │ └── ...
|
||||
│ └── sqlite/ # SQLite implementations (19 stores)
|
||||
│ ├── stores.go # NewStores() constructor
|
||||
│ ├── provider.go # ProviderStore
|
||||
│ ├── catalog.go # CatalogStore
|
||||
│ ├── persona.go # PersonaStore
|
||||
│ ├── user.go # UserStore
|
||||
│ ├── team.go # TeamStore
|
||||
│ ├── policy.go # PolicyStore
|
||||
│ ├── audit.go # AuditStore
|
||||
│ └── ...
|
||||
│ └── ... # Mirror of postgres/ with dialect adaptations
|
||||
├── models/models.go # Shared domain types
|
||||
├── capabilities/
|
||||
│ ├── intrinsic.go # Heuristic detection + resolution
|
||||
│ └── resolver.go # ModelsForUser() unified resolver
|
||||
├── compaction/ # Conversation summarization engine
|
||||
├── crypto/ # AES-256-GCM API key encryption
|
||||
├── events/ # EventBus + WebSocket hub
|
||||
├── extraction/ # Document text extraction pipeline
|
||||
├── handlers/ # HTTP handlers (Gin)
|
||||
│ ├── auth.go # Login, register, refresh, logout
|
||||
│ ├── admin.go # User/config/model management
|
||||
│ ├── channels.go # Channel CRUD
|
||||
│ ├── messages.go # Message CRUD + forking
|
||||
│ ├── completion.go # Chat completions (SSE streaming)
|
||||
│ ├── stream_loop.go # Shared streaming + tool execution loop
|
||||
│ ├── capabilities.go # Model list + ResolveModelCaps
|
||||
│ ├── presets.go # Persona CRUD (all scopes)
|
||||
│ ├── apiconfigs.go # User provider config CRUD (BYOK)
|
||||
│ ├── teams.go # Team management
|
||||
│ ├── notes.go # Notes CRUD + search
|
||||
│ ├── knowledge.go # Knowledge base management
|
||||
│ └── ...
|
||||
├── knowledge/ # KB chunking, embedding, search
|
||||
├── providers/ # LLM provider adapters
|
||||
│ ├── provider.go # Provider interface
|
||||
│ ├── anthropic.go
|
||||
@@ -150,8 +164,8 @@ server/
|
||||
│ ├── openrouter.go
|
||||
│ └── venice.go
|
||||
├── middleware/ # Auth, admin, CORS, rate limiting
|
||||
├── events/ # EventBus + WebSocket hub
|
||||
└── tools/ # Built-in tool definitions (notes)
|
||||
├── storage/ # Blob storage (PVC + S3)
|
||||
└── tools/ # Built-in tool definitions
|
||||
```
|
||||
|
||||
## Store Layer Pattern
|
||||
@@ -210,25 +224,80 @@ Used by: `provider_configs`, `personas`, `model_catalog` (visibility column adds
|
||||
|
||||
## Schema Migration
|
||||
|
||||
Single consolidated migration (`001_v09_schema.sql`) replaces the previous 21 incremental migrations. The Go backend auto-migrates on startup:
|
||||
Migrations are handled by the backend on startup (no separate migration job):
|
||||
|
||||
1. Creates `schema_migrations` table if absent
|
||||
2. Checks which migration files have been applied
|
||||
3. Applies any new `.sql` files in order
|
||||
|
||||
Postgres uses consolidated schemas (`001_v016_schema.sql` + incremental). SQLite uses application-generated UUIDs and `datetime()` instead of `gen_random_uuid()` and `now()`. Both drivers share the same migration tracking table.
|
||||
|
||||
## Frontend Architecture
|
||||
|
||||
Vanilla JavaScript, no build step. Five files with clear responsibilities:
|
||||
Vanilla JavaScript, no framework. The frontend ships as static files served by nginx. The only build step is the CM6 editor bundle, compiled at Docker build time via esbuild (IIFE output, no runtime bundler).
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `api.js` | HTTP client with token refresh. All backend calls. |
|
||||
| `app.js` | Application state machine. Business logic. |
|
||||
| `ui.js` | DOM rendering. All `document.createElement` calls. |
|
||||
| `events.js` | Labeled event bus with WebSocket bridge. |
|
||||
| `debug.js` | Admin debug panel (model list, stats, config). |
|
||||
### File Structure
|
||||
|
||||
Communication: `app.js` calls `API.*` methods, updates state, then calls `UI.*` methods to render. `events.js` handles real-time updates via WebSocket. No framework, no virtual DOM, no reactive bindings.
|
||||
```
|
||||
src/
|
||||
├── index.html # Single-page app shell
|
||||
├── sw.js # Service worker (offline cache)
|
||||
├── manifest.json # PWA manifest
|
||||
├── css/styles.css # All styles (CSS variables, light/dark themes)
|
||||
├── js/
|
||||
│ ├── api.js # HTTP client with token refresh
|
||||
│ ├── app.js # Application state machine, startup, routing
|
||||
│ ├── chat.js # Chat input abstraction, message send/receive
|
||||
│ ├── events.js # Labeled event bus + WebSocket bridge
|
||||
│ ├── debug.js # Debug panel, diagnostics, state export
|
||||
│ ├── ui-core.js # DOM rendering, sidebar, modals, panels
|
||||
│ ├── ui-settings.js # Settings tabs, theme, appearance, teams
|
||||
│ ├── ui-admin.js # Admin panel rendering
|
||||
│ ├── admin-handlers.js # Admin CRUD operations + extension editors
|
||||
│ ├── settings-handlers.js # User settings CRUD, command palette
|
||||
│ ├── tokens.js # Input token counting + context budget
|
||||
│ ├── attachments.js # File upload, paste-to-file, lightbox
|
||||
│ ├── notes.js # Notes panel CRUD
|
||||
│ ├── knowledge.js # Knowledge base UI
|
||||
│ └── __tests__/ # Node.js test suite (node --test)
|
||||
├── editor/ # CM6 source (compiled at build time)
|
||||
│ ├── package.json # CM6 + language mode dependencies
|
||||
│ ├── package-lock.json # Lockfile for npm ci
|
||||
│ ├── build.mjs # esbuild script → IIFE bundle
|
||||
│ ├── index.mjs # Bundle entrypoint (window.CM)
|
||||
│ ├── chat-input.mjs # Chat input factory (markdown, code blocks)
|
||||
│ ├── code-editor.mjs # Code editor factory (syntax, keybindings)
|
||||
│ └── theme.mjs # Switchboard + chat input themes
|
||||
└── vendor/ # Vendored libraries (local + CDN fallback)
|
||||
├── marked/ # Markdown renderer
|
||||
├── purify/ # DOMPurify (XSS protection)
|
||||
├── mermaid/ # Diagram renderer
|
||||
├── katex/ # Math renderer
|
||||
└── codemirror/ # CM6 bundle (built by Docker)
|
||||
└── codemirror.bundle.js # ~295KB min, ~90KB gzip
|
||||
```
|
||||
|
||||
### CM6 Integration
|
||||
|
||||
The CodeMirror 6 bundle provides two factory functions exposed on `window.CM`:
|
||||
|
||||
**`CM.chatInput(target, opts)`** — Markdown-mode editor for the chat input area. Features: auto-growing height, Enter=send / Shift+Enter=newline, WYSIWYG fenced code block decorations (visual container with monospace font and accent border), inline code shortcut (Ctrl/Cmd+E), spell check. Always uses standard keybindings.
|
||||
|
||||
**`CM.codeEditor(target, opts)`** — Full-featured code editor for the admin extension panel (JSON manifests, JavaScript scripts). Features: line numbers, bracket matching, search/replace, fold gutter, 10 bundled language modes. Supports Vim and Emacs keybindings via user preference (reconfigurable at runtime via compartments).
|
||||
|
||||
Both factories return a clean API: `getValue()`, `setValue()`, `focus()`, `destroy()`. The `ChatInput` abstraction in `chat.js` wraps the CM6 instance with a textarea fallback — all callsites use `ChatInput.getValue()` etc., never raw DOM access.
|
||||
|
||||
**Graceful degradation**: Every integration point checks `window.CM` before using CM6. If the bundle fails to load (air-gapped without the build step, CDN issues), the app falls back to native `<textarea>` with zero breakage.
|
||||
|
||||
### Theme System
|
||||
|
||||
CSS variables define the color palette in `:root` (dark, default) and `[data-theme="light"]` (light override). All UI components — including CM6 editors — reference these variables, so theme switches propagate instantly without editor reconfiguration.
|
||||
|
||||
The appearance settings offer three modes: Light, Dark, System. System mode listens to `prefers-color-scheme` and re-evaluates on OS theme change. Theme changes emit `theme.changed` on the EventBus; CM6 code editors toggle the `oneDark` syntax theme via compartment reconfiguration.
|
||||
|
||||
### Communication Pattern
|
||||
|
||||
`app.js` calls `API.*` methods, updates state, then calls `UI.*` methods to render. `events.js` handles real-time updates via WebSocket. No framework, no virtual DOM, no reactive bindings.
|
||||
|
||||
## Security Model
|
||||
|
||||
|
||||
451
docs/DESIGN-0.17.3.md
Normal file
451
docs/DESIGN-0.17.3.md
Normal file
@@ -0,0 +1,451 @@
|
||||
# DESIGN: Notes Rich Text Editor + Obsidian-Style Linking
|
||||
|
||||
**Version:** v0.17.3
|
||||
**Status:** Draft
|
||||
**Scope:** Upgrade the notes editor from a plain `<textarea>` to a CM6-powered
|
||||
markdown editor with live preview, `[[wikilink]]` resolution, and a backlinks
|
||||
system.
|
||||
|
||||
**Depends on:** v0.17.2 (CodeMirror 6 bundle provides `CM.codeEditor()` and
|
||||
the markdown language mode).
|
||||
|
||||
---
|
||||
|
||||
## Motivation
|
||||
|
||||
The notes system (v0.9.3 side panel, CRUD, folders, search) stores markdown
|
||||
but edits it as raw text in a `<textarea>`. Now that CM6 is bundled, we can
|
||||
give notes the same live-preview markdown experience that the chat input gets
|
||||
— plus something the chat input doesn't need: **inter-note linking**.
|
||||
|
||||
The Obsidian model is the right reference: markdown files with `[[wikilinks]]`
|
||||
resolved at render time, bidirectional backlinks, and create-on-reference.
|
||||
Our advantage over Obsidian: we already have **semantic search via embeddings**
|
||||
(v0.14.0 KB infrastructure), so link autocomplete can be smarter than
|
||||
filename matching.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### New CM6 Factory: `CM.noteEditor()`
|
||||
|
||||
A third factory alongside `chatInput()` and `codeEditor()`, tailored for
|
||||
long-form markdown editing in the notes panel.
|
||||
|
||||
```javascript
|
||||
CM.noteEditor(target, {
|
||||
value: '', // initial markdown content
|
||||
darkMode: true,
|
||||
onChange: (text) => {}, // live content callback
|
||||
onLink: (title) => {}, // [[link]] activated callback
|
||||
linkCompleter: async (query) => [], // fuzzy note search for [[ autocomplete
|
||||
});
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
|
||||
- Full markdown syntax highlighting (headings, bold, italic, code, lists, links)
|
||||
- Live inline preview: headings render at size, bold/italic styled, code blocks
|
||||
highlighted — the document is still markdown but *looks* formatted
|
||||
- `[[` triggers autocomplete dropdown of existing notes (fuzzy search)
|
||||
- `[[Note Title]]` tokens rendered as clickable chips (CM6 `Decoration.widget`)
|
||||
- Line numbers off (it's a note, not code)
|
||||
- Spell check enabled (`EditorView.contentAttributes: { spellcheck: 'true' }`)
|
||||
- Vim/Emacs keybindings respected (user preference from v0.17.2)
|
||||
|
||||
### Wikilink Syntax
|
||||
|
||||
Standard double-bracket syntax with optional display text:
|
||||
|
||||
```
|
||||
[[Note Title]] → links to note titled "Note Title"
|
||||
[[Note Title|display text]] → shows "display text", links to "Note Title"
|
||||
```
|
||||
|
||||
Resolution order:
|
||||
1. Exact title match (case-insensitive)
|
||||
2. Fuzzy title match (for autocomplete, not for rendering)
|
||||
3. Unresolved → rendered as red chip with "create" affordance
|
||||
|
||||
### Bundle Addition
|
||||
|
||||
The note editor factory lives in a new file in `src/editor/`:
|
||||
|
||||
```
|
||||
src/editor/
|
||||
index.mjs # existing — adds noteEditor to window.CM
|
||||
chat-input.mjs # existing
|
||||
code-editor.mjs # existing
|
||||
note-editor.mjs # NEW — noteEditor() factory
|
||||
wikilink.mjs # NEW — CM6 extension: parse, decorate, autocomplete [[links]]
|
||||
theme.mjs # existing
|
||||
```
|
||||
|
||||
The wikilink extension is CM6-native: a `ViewPlugin` that scans for `[[...]]`
|
||||
patterns and replaces them with `Decoration.widget` nodes (clickable chips).
|
||||
The autocomplete uses CM6's `autocompletion` with a custom `completeFromList`
|
||||
source triggered by `[[`.
|
||||
|
||||
No bundle size concern — the wikilink plugin is tiny custom code, not an
|
||||
external dependency.
|
||||
|
||||
---
|
||||
|
||||
## Data Model
|
||||
|
||||
### `note_links` Table
|
||||
|
||||
Tracks directed links between notes, extracted on save.
|
||||
|
||||
```sql
|
||||
-- Postgres
|
||||
CREATE TABLE IF NOT EXISTS note_links (
|
||||
source_note_id UUID NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
|
||||
target_note_id UUID REFERENCES notes(id) ON DELETE CASCADE,
|
||||
target_title TEXT NOT NULL, -- raw [[title]] text, for unresolved links
|
||||
display_text TEXT, -- optional |alias
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
PRIMARY KEY (source_note_id, target_title)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_note_links_target ON note_links(target_note_id)
|
||||
WHERE target_note_id IS NOT NULL;
|
||||
|
||||
-- SQLite
|
||||
CREATE TABLE IF NOT EXISTS note_links (
|
||||
source_note_id TEXT NOT NULL REFERENCES notes(id) ON DELETE CASCADE,
|
||||
target_note_id TEXT REFERENCES notes(id) ON DELETE CASCADE,
|
||||
target_title TEXT NOT NULL,
|
||||
display_text TEXT,
|
||||
created_at TEXT DEFAULT (datetime('now')),
|
||||
PRIMARY KEY (source_note_id, target_title)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_note_links_target ON note_links(target_note_id)
|
||||
WHERE target_note_id IS NOT NULL;
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
|
||||
- `target_note_id` is **nullable** — an unresolved link (target note doesn't
|
||||
exist yet) stores the title but has `NULL` for the FK. When a note with that
|
||||
title is created, a background pass resolves dangling links.
|
||||
- Primary key is `(source_note_id, target_title)` — a note can only link to
|
||||
the same title once (last one wins if duplicated).
|
||||
- `target_title` is always stored for human readability and re-resolution
|
||||
after renames.
|
||||
|
||||
### Link Extraction (on save)
|
||||
|
||||
When a note is saved, the backend:
|
||||
|
||||
1. Parses content for `[[...]]` patterns (simple regex: `\[\[([^\]|]+)(?:\|([^\]]+))?\]\]`)
|
||||
2. Resolves each title to a note ID: `SELECT id FROM notes WHERE LOWER(title) = LOWER($1) AND user_id = $2`
|
||||
3. Deletes existing links for this source note: `DELETE FROM note_links WHERE source_note_id = $1`
|
||||
4. Inserts fresh links with resolved (or NULL) target IDs
|
||||
|
||||
This full-replace approach is simple and correct — notes don't have thousands
|
||||
of links, so the cost is negligible.
|
||||
|
||||
### Backlinks Query
|
||||
|
||||
```sql
|
||||
SELECT n.id, n.title, n.folder_path, n.updated_at, nl.display_text
|
||||
FROM note_links nl
|
||||
JOIN notes n ON n.id = nl.source_note_id
|
||||
WHERE nl.target_note_id = $1
|
||||
ORDER BY n.updated_at DESC;
|
||||
```
|
||||
|
||||
### Dangling Link Resolution
|
||||
|
||||
When a new note is created, resolve any dangling links pointing at its title:
|
||||
|
||||
```sql
|
||||
UPDATE note_links
|
||||
SET target_note_id = $1
|
||||
WHERE target_note_id IS NULL
|
||||
AND LOWER(target_title) = LOWER($2)
|
||||
AND source_note_id IN (SELECT id FROM notes WHERE user_id = $3);
|
||||
```
|
||||
|
||||
This runs once on note creation — cheap and ensures links "light up"
|
||||
retroactively.
|
||||
|
||||
---
|
||||
|
||||
## API Changes
|
||||
|
||||
### Existing Endpoints (modified)
|
||||
|
||||
**`PUT /api/v1/notes/:id`** — after updating note content, re-extract links.
|
||||
No API signature change; link extraction is a server-side side effect.
|
||||
|
||||
**`POST /api/v1/notes`** — after creating a note, extract links AND resolve
|
||||
dangling links from other notes that reference this title.
|
||||
|
||||
### New Endpoints
|
||||
|
||||
**`GET /api/v1/notes/:id/backlinks`**
|
||||
|
||||
Returns notes that link to this note.
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [
|
||||
{
|
||||
"id": "uuid",
|
||||
"title": "Meeting Notes 2026-02-28",
|
||||
"folder_path": "/work",
|
||||
"updated_at": "2026-02-28T10:00:00Z",
|
||||
"display_text": null
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**`GET /api/v1/notes/search-titles?q=<query>&limit=10`**
|
||||
|
||||
Lightweight fuzzy title search for the `[[` autocomplete. Returns titles
|
||||
and IDs only — no content, no embeddings. Fast path.
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [
|
||||
{ "id": "uuid", "title": "Project Kickoff Notes" },
|
||||
{ "id": "uuid", "title": "Project Architecture" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Uses `ILIKE` on Postgres, `LIKE` on SQLite (case-insensitive via COLLATE).
|
||||
The semantic search endpoint (`GET /api/v1/notes/search`) remains available
|
||||
for deeper searches but is overkill for autocomplete.
|
||||
|
||||
---
|
||||
|
||||
## Frontend Integration
|
||||
|
||||
### Note Editor Swap
|
||||
|
||||
In `notes.js`, `openNoteEditor()` currently sets `.value` on the
|
||||
`#noteEditorContent` textarea. After v0.17.3:
|
||||
|
||||
```javascript
|
||||
// Replace textarea with CM6 note editor
|
||||
const container = document.getElementById('noteEditorContentContainer');
|
||||
const editor = CM.noteEditor(container, {
|
||||
value: note.content || '',
|
||||
darkMode: document.body.classList.contains('dark-theme'),
|
||||
onChange: (text) => {
|
||||
// Could auto-save or mark dirty
|
||||
},
|
||||
onLink: (title) => {
|
||||
// Navigate to linked note
|
||||
const linked = _findNoteByTitle(title);
|
||||
if (linked) {
|
||||
openNoteEditor(linked.id);
|
||||
} else {
|
||||
// Offer to create
|
||||
_createNoteFromLink(title);
|
||||
}
|
||||
},
|
||||
linkCompleter: async (query) => {
|
||||
const resp = await API.searchNoteTitles(query, 10);
|
||||
return (resp.data || []).map(n => ({ label: n.title, id: n.id }));
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
The HTML changes from:
|
||||
```html
|
||||
<textarea id="noteEditorContent" rows="12" ...></textarea>
|
||||
```
|
||||
To:
|
||||
```html
|
||||
<div id="noteEditorContentContainer" class="notes-content-input"></div>
|
||||
```
|
||||
|
||||
With graceful fallback: if `window.CM?.noteEditor` is undefined, create a
|
||||
`<textarea>` inside the container (same behavior as today).
|
||||
|
||||
### Backlinks Panel
|
||||
|
||||
Below the note editor, a collapsible "Backlinks" section:
|
||||
|
||||
```html
|
||||
<div id="noteBacklinks" class="note-backlinks" style="display:none">
|
||||
<div class="note-backlinks-header" onclick="toggleBacklinks()">
|
||||
<span>Linked mentions</span>
|
||||
<span id="noteBacklinksCount" class="badge">0</span>
|
||||
</div>
|
||||
<div id="noteBacklinksList" class="note-backlinks-list"></div>
|
||||
</div>
|
||||
```
|
||||
|
||||
Loaded on `openNoteEditor()` for existing notes:
|
||||
```javascript
|
||||
async function loadBacklinks(noteId) {
|
||||
const resp = await API.getNoteBacklinks(noteId);
|
||||
const links = resp.data || [];
|
||||
const el = document.getElementById('noteBacklinksList');
|
||||
const count = document.getElementById('noteBacklinksCount');
|
||||
count.textContent = links.length;
|
||||
document.getElementById('noteBacklinks').style.display =
|
||||
links.length > 0 ? '' : 'none';
|
||||
el.innerHTML = links.map(n => `
|
||||
<div class="note-backlink-item" onclick="openNoteEditor('${n.id}')">
|
||||
<span class="note-backlink-title">${esc(n.title)}</span>
|
||||
<span class="note-backlink-folder text-muted">${esc(n.folder_path)}</span>
|
||||
</div>
|
||||
`).join('');
|
||||
}
|
||||
```
|
||||
|
||||
### Note Preview (read mode)
|
||||
|
||||
The existing note list view shows raw markdown. After this release,
|
||||
`[[Note Title]]` patterns in rendered markdown (via marked.js) become
|
||||
clickable links. A small marked.js extension or post-render pass
|
||||
converts `[[...]]` to `<a>` tags with `onclick` handlers.
|
||||
|
||||
---
|
||||
|
||||
## Access Control
|
||||
|
||||
**Links respect existing note ownership.** A user can only:
|
||||
- Create links to notes they own (their own notes)
|
||||
- See backlinks from notes they own
|
||||
- Autocomplete searches their own notes
|
||||
|
||||
Team-scoped notes (via `team_id` on the notes table) follow the same
|
||||
grant model as other team resources. When team notes ship (potentially
|
||||
v0.23.0 channel-scoped notes), the link resolution query adds
|
||||
`AND (user_id = $1 OR team_id IN (...))`.
|
||||
|
||||
For v0.17.3, the scope is **personal notes only** — which is the
|
||||
current notes model.
|
||||
|
||||
---
|
||||
|
||||
## Store Layer
|
||||
|
||||
### `NoteLinkStore` Interface
|
||||
|
||||
```go
|
||||
type NoteLinkStore interface {
|
||||
// ReplaceLinks deletes existing links for sourceNoteID and inserts new ones.
|
||||
// Each link has TargetTitle (always set) and TargetNoteID (nullable, resolved).
|
||||
ReplaceLinks(ctx context.Context, sourceNoteID string, links []NoteLink) error
|
||||
|
||||
// ResolveByTitle sets target_note_id on dangling links matching the title.
|
||||
ResolveByTitle(ctx context.Context, userID, targetNoteID, title string) error
|
||||
|
||||
// Backlinks returns notes that link to the given note.
|
||||
Backlinks(ctx context.Context, noteID string) ([]NoteLinkResult, error)
|
||||
}
|
||||
|
||||
type NoteLink struct {
|
||||
TargetNoteID *string // nil for unresolved
|
||||
TargetTitle string
|
||||
DisplayText string
|
||||
}
|
||||
|
||||
type NoteLinkResult struct {
|
||||
SourceNoteID string
|
||||
Title string
|
||||
FolderPath string
|
||||
UpdatedAt time.Time
|
||||
DisplayText string
|
||||
}
|
||||
```
|
||||
|
||||
Implementations in both `store/postgres/note_link.go` and
|
||||
`store/sqlite/note_link.go`.
|
||||
|
||||
### Link Extraction Helper
|
||||
|
||||
```go
|
||||
// ExtractWikilinks parses [[Title]] and [[Title|Display]] from markdown content.
|
||||
func ExtractWikilinks(content string) []NoteLink {
|
||||
re := regexp.MustCompile(`\[\[([^\]|]+?)(?:\|([^\]]+?))?\]\]`)
|
||||
matches := re.FindAllStringSubmatch(content, -1)
|
||||
seen := make(map[string]bool)
|
||||
var links []NoteLink
|
||||
for _, m := range matches {
|
||||
title := strings.TrimSpace(m[1])
|
||||
if seen[strings.ToLower(title)] { continue }
|
||||
seen[strings.ToLower(title)] = true
|
||||
link := NoteLink{TargetTitle: title}
|
||||
if len(m) > 2 && m[2] != "" {
|
||||
link.DisplayText = strings.TrimSpace(m[2])
|
||||
}
|
||||
links = append(links, link)
|
||||
}
|
||||
return links
|
||||
}
|
||||
```
|
||||
|
||||
This lives in a shared `notes/` package (or in the handler) — it's pure
|
||||
string parsing with no DB dependency.
|
||||
|
||||
---
|
||||
|
||||
## Migration
|
||||
|
||||
**Migration 002** (both Postgres and SQLite):
|
||||
|
||||
```sql
|
||||
-- 002_note_links.sql
|
||||
CREATE TABLE IF NOT EXISTS note_links ( ... );
|
||||
CREATE INDEX ...;
|
||||
```
|
||||
|
||||
No backfill needed — existing notes have no `[[links]]` in their content,
|
||||
so the table starts empty. Links populate organically as users edit notes
|
||||
with the new editor.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Checklist
|
||||
|
||||
### Phase 1: Backend (links infrastructure)
|
||||
- [ ] Migration 002: `note_links` table (Postgres + SQLite)
|
||||
- [ ] `NoteLinkStore` interface + Postgres/SQLite implementations
|
||||
- [ ] `ExtractWikilinks()` helper with tests
|
||||
- [ ] Link extraction on note create/update in handlers
|
||||
- [ ] Dangling link resolution on note create
|
||||
- [ ] `GET /api/v1/notes/:id/backlinks` endpoint
|
||||
- [ ] `GET /api/v1/notes/search-titles?q=` endpoint
|
||||
- [ ] API client additions in `api.js`
|
||||
|
||||
### Phase 2: Frontend (CM6 note editor)
|
||||
- [ ] `src/editor/note-editor.mjs` — `CM.noteEditor()` factory
|
||||
- [ ] `src/editor/wikilink.mjs` — CM6 extension (parse, decorate, autocomplete)
|
||||
- [ ] Rebuild CM6 bundle (add new modules to `index.mjs`)
|
||||
- [ ] Replace `<textarea>` with CM6 note editor in `notes.js`
|
||||
- [ ] `saveNote()` reads from CM6 `.getValue()`
|
||||
- [ ] Graceful fallback if CM6 unavailable
|
||||
- [ ] Backlinks panel UI below editor
|
||||
- [ ] `[[link]]` rendering in note list preview (marked.js extension)
|
||||
|
||||
### Phase 3: Polish
|
||||
- [ ] Wikilink chips styled to match app theme (accent color, hover state)
|
||||
- [ ] Unresolved links styled distinctly (red/dashed, "click to create")
|
||||
- [ ] Autocomplete dropdown styled (note title + folder path hint)
|
||||
- [ ] Keyboard navigation in autocomplete (↑/↓, Enter to select, Esc to dismiss)
|
||||
- [ ] Mobile/touch testing for link taps
|
||||
- [ ] Update ARCHITECTURE.md
|
||||
|
||||
---
|
||||
|
||||
## Future (not in v0.17.3)
|
||||
|
||||
- **Graph view**: visual node graph of note connections (d3 force layout)
|
||||
- **Orphan detection**: notes with no inbound or outbound links
|
||||
- **Link suggestions**: "notes that might be related" via embedding similarity
|
||||
(reuse KB infrastructure)
|
||||
- **Transclusion**: `![[Note Title]]` embeds the target note's content inline
|
||||
- **Tag linking**: `#tag` syntax as an alternative navigation path
|
||||
- **Team note links**: cross-user linking when team/channel-scoped notes ship
|
||||
@@ -1,7 +1,7 @@
|
||||
# DESIGN: CodeMirror 6 Integration
|
||||
|
||||
**Version:** v0.18.0 (or fits into current cycle)
|
||||
**Status:** Draft
|
||||
**Version:** v0.17.2
|
||||
**Status:** Complete
|
||||
**Scope:** Add CM6 as a vendored dependency, bundled at Docker build time via esbuild. Three integration surfaces.
|
||||
|
||||
---
|
||||
|
||||
@@ -60,7 +60,7 @@ v0.16.0 User Groups v0.17.0 Persona-KB Binding
|
||||
│
|
||||
┌───────┴──────────────┐
|
||||
│ │
|
||||
v0.17.1 SQLite Backend ✅ v0.17.2 CodeMirror 6
|
||||
v0.17.1 SQLite Backend ✅ v0.17.2 CodeMirror 6 ✅
|
||||
(dual DB) (editor bundle, chat
|
||||
│ input, ext editor)
|
||||
└───────┬──────────────┘
|
||||
@@ -286,7 +286,7 @@ Depends on: v0.17.0 DB debt cleanup (store layer is sole DB interface).
|
||||
|
||||
---
|
||||
|
||||
## v0.17.2 — CodeMirror 6 Integration
|
||||
## v0.17.2 — CodeMirror 6 Integration ✅
|
||||
|
||||
Rich editor infrastructure for the frontend. CM6 is ESM-native; the build
|
||||
step runs in Docker (esbuild → single IIFE bundle). No change to dev
|
||||
@@ -296,40 +296,44 @@ Depends on: nothing (frontend-only). Prerequisite for: extension surfaces
|
||||
(v0.21.0 editor mode). See [DESIGN-CM6.md](DESIGN-CM6.md) for full spec.
|
||||
|
||||
**Build Pipeline**
|
||||
- [ ] `src/editor/` directory: `package.json`, `build.mjs`, `index.mjs`, language/theme modules
|
||||
- [ ] `scripts/build-editor.sh`: shared build script for both Dockerfiles and local dev
|
||||
- [ ] New Docker stage (`cm6-build`): `npm ci` + `node build.mjs` → `vendor/codemirror/`
|
||||
- [ ] Both `Dockerfile.frontend` and `Dockerfile` (unified) use the shared build script
|
||||
- [ ] Bundle output: `codemirror.bundle.js` (~295KB min, ~90KB gzip) + `codemirror.bundle.css`
|
||||
- [ ] Graceful degradation: all integration points check `window.CM` — falls back to `<textarea>` if bundle unavailable
|
||||
- [x] `src/editor/` directory: `package.json`, `build.mjs`, `index.mjs`, language/theme modules
|
||||
- [x] `scripts/build-editor.sh`: shared build script for both Dockerfiles and local dev
|
||||
- [x] New Docker stage (`cm6-build`): `npm ci` + `node build.mjs` → `vendor/codemirror/`
|
||||
- [x] Both `Dockerfile.frontend` and `Dockerfile` (unified) use the shared build script
|
||||
- [x] Bundle output: `codemirror.bundle.js` (~295KB min, ~90KB gzip)
|
||||
- [x] Graceful degradation: all integration points check `window.CM` — falls back to `<textarea>` if bundle unavailable
|
||||
|
||||
**Languages (bundled)**
|
||||
- [ ] Markdown, JavaScript, JSON, SQL, HTML, CSS, YAML, Go, Python, Rust
|
||||
- [ ] Additional modes: one-line import + rebuild
|
||||
- [x] Markdown, JavaScript, JSON, SQL, HTML, CSS, YAML, Go, Python, Rust
|
||||
- [x] Additional modes: one-line import + rebuild
|
||||
|
||||
**Phase 1: Extension Editor** (admin panel)
|
||||
- [ ] `CM.codeEditor()` factory: line numbers, bracket matching, auto-indent, search/replace
|
||||
- [ ] Replace manifest `<textarea>` → CM6 JSON mode
|
||||
- [ ] Replace script `<textarea>` → CM6 JavaScript mode
|
||||
- [ ] Remove manual Tab-key handler in `admin-handlers.js` (CM6 handles it)
|
||||
- [ ] Update `saveAdminExtension()` to use `.getValue()`
|
||||
- [x] `CM.codeEditor()` factory: line numbers, bracket matching, auto-indent, search/replace
|
||||
- [x] Replace manifest `<textarea>` → CM6 JSON mode
|
||||
- [x] Replace script `<textarea>` → CM6 JavaScript mode
|
||||
- [x] Remove manual Tab-key handler in `admin-handlers.js` (CM6 handles it)
|
||||
- [x] Update `saveAdminExtension()` to use `.getValue()`
|
||||
|
||||
**Phase 2: Chat Input** (markdown mode)
|
||||
- [ ] `CM.chatInput()` factory: markdown highlighting, minimal chrome (no line numbers, no gutter)
|
||||
- [ ] Enter=send, Shift+Enter=newline keybindings
|
||||
- [ ] Auto-growing height, placeholder text
|
||||
- [ ] Wire `onChange` → `updateInputTokens()`
|
||||
- [ ] Update `chat.js`, `attachments.js`, `tokens.js` to use CM API
|
||||
- [ ] Test paste handling (plain text, code, attachments) and mobile/touch input
|
||||
- [x] `CM.chatInput()` factory: markdown highlighting, minimal chrome (no line numbers, no gutter)
|
||||
- [x] Enter=send, Shift+Enter=newline keybindings
|
||||
- [x] Auto-growing height, placeholder text
|
||||
- [x] Wire `onChange` → `updateInputTokens()`
|
||||
- [x] Update `chat.js`, `attachments.js`, `tokens.js` to use CM API
|
||||
- [x] WYSIWYG fenced code block decorations (visual container matching claude.ai UX)
|
||||
- [x] Inline code shortcut (Ctrl/Cmd+E)
|
||||
|
||||
**Phase 3: Polish**
|
||||
- [ ] Theme integration: CSS variable overrides (`--bg`, `--border`, `--accent`, etc.)
|
||||
- [ ] Dark/light mode switching (listen for theme toggle event)
|
||||
- [ ] Vim/Emacs keybinding preference in appearance settings (`@replit/codemirror-vim`, `@replit/codemirror-emacs`)
|
||||
- [ ] Vim/Emacs applies to code editor + extension editor only (never chat input)
|
||||
- [ ] SW cache: exclude `vendor/codemirror/` or add to `SHELL_FILES` with version bust
|
||||
- [ ] Update `debug.js` snapshot to include CM6 version
|
||||
- [ ] Documentation in ARCHITECTURE.md
|
||||
- [x] Theme integration: CSS variable overrides (`--bg`, `--border`, `--accent`, etc.)
|
||||
- [x] Dark/light/system mode toggle in appearance settings
|
||||
- [x] Vim/Emacs keybinding preference in appearance settings (`@replit/codemirror-vim`, `@replit/codemirror-emacs`)
|
||||
- [x] Vim/Emacs applies to code editor + extension editor only (never chat input)
|
||||
- [x] SW cache: exclude `vendor/codemirror/` with version bust
|
||||
- [x] Update `debug.js` snapshot to include CM6 version
|
||||
- [x] Documentation in ARCHITECTURE.md
|
||||
|
||||
**CI/CD**
|
||||
- [x] Path-based change detection gating (FE-only → skip BE tests, docs-only → skip all)
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user