Changeset 0.28.0.1 (#173)
This commit is contained in:
498
docs/archive/DESIGN-CM6.md
Normal file
498
docs/archive/DESIGN-CM6.md
Normal file
@@ -0,0 +1,498 @@
|
||||
# DESIGN: CodeMirror 6 Integration
|
||||
|
||||
**Version:** v0.17.2
|
||||
**Status:** Complete
|
||||
**Scope:** Add CM6 as a vendored dependency, bundled at Docker build time via esbuild. Three integration surfaces.
|
||||
|
||||
---
|
||||
|
||||
## Motivation
|
||||
|
||||
Three features converge on the same dependency:
|
||||
|
||||
1. **Chat input** — live markdown rendering (backtick → code block, bold, etc.)
|
||||
2. **Extension editor** — syntax-highlighted JavaScript/JSON editing in admin panel (replaces bare `<textarea>`)
|
||||
3. **Code editing surface** (roadmap) — future extension surface type for code review, snippets, etc.
|
||||
|
||||
CM6 is ESM-native and expects a bundler. The build step is contained in the Docker image build — the frontend continues to ship as static assets served by nginx. No runtime build tooling, no change to the developer experience for non-CM6 code.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Build Pipeline
|
||||
|
||||
New Docker stage between the existing `vendor` stage and the final `nginx` stage:
|
||||
|
||||
```
|
||||
Stage 1: vendor (existing — marked, purify, mermaid, katex)
|
||||
Stage 2: cm6-build (NEW — npm install + esbuild → single IIFE bundle)
|
||||
Stage 3: nginx (existing — copies src/ + vendor/ + cm6 bundle)
|
||||
```
|
||||
|
||||
The CM6 build stage:
|
||||
|
||||
```dockerfile
|
||||
FROM node:20-alpine AS cm6-build
|
||||
WORKDIR /build
|
||||
COPY src/editor/package.json src/editor/build.mjs ./
|
||||
RUN npm install --production=false
|
||||
RUN node build.mjs
|
||||
# Output: /build/dist/codemirror.bundle.js (~180-250KB minified)
|
||||
# /build/dist/codemirror.bundle.css (~5-10KB)
|
||||
```
|
||||
|
||||
Final stage addition:
|
||||
|
||||
```dockerfile
|
||||
COPY --from=cm6-build /build/dist/ /usr/share/nginx/html/vendor/codemirror/
|
||||
```
|
||||
|
||||
### Bundle Structure
|
||||
|
||||
A single esbuild entrypoint (`src/editor/index.mjs`) that imports CM6 packages and exposes factory functions on `window.CM`:
|
||||
|
||||
```javascript
|
||||
// src/editor/index.mjs — esbuild entrypoint
|
||||
import { EditorView, keymap, placeholder, ViewPlugin, ... } from '@codemirror/view';
|
||||
import { EditorState, ... } from '@codemirror/state';
|
||||
import { markdown, markdownLanguage } from '@codemirror/lang-markdown';
|
||||
import { javascript } from '@codemirror/lang-javascript';
|
||||
import { json } from '@codemirror/lang-json';
|
||||
import { sql } from '@codemirror/lang-sql';
|
||||
import { html } from '@codemirror/lang-html';
|
||||
import { css } from '@codemirror/lang-css';
|
||||
import { yaml } from '@codemirror/lang-yaml';
|
||||
import { go } from '@codemirror/lang-go';
|
||||
import { python } from '@codemirror/lang-python';
|
||||
import { rust } from '@codemirror/lang-rust';
|
||||
import { oneDark } from '@codemirror/theme-one-dark';
|
||||
import { defaultKeymap, indentWithTab } from '@codemirror/commands';
|
||||
import { bracketMatching, ... } from '@codemirror/language';
|
||||
import { highlightSelectionMatches, searchKeymap } from '@codemirror/search';
|
||||
import { autocompletion } from '@codemirror/autocomplete';
|
||||
import { vim } from '@replit/codemirror-vim';
|
||||
import { emacs } from '@replit/codemirror-emacs';
|
||||
|
||||
// Expose on window for script-tag consumption
|
||||
window.CM = {
|
||||
EditorView,
|
||||
EditorState,
|
||||
|
||||
// Factory: chat input (markdown mode, minimal chrome)
|
||||
chatInput(target, opts = {}) { ... },
|
||||
|
||||
// Factory: code editor (full features, language auto-detect)
|
||||
codeEditor(target, opts = {}) { ... },
|
||||
|
||||
// Supported languages for code editor
|
||||
languages: { markdown, javascript, json, sql, html, css, yaml, go, python, rust },
|
||||
|
||||
// Keybinding modes (applied via user preference)
|
||||
keybindings: { vim, emacs },
|
||||
};
|
||||
```
|
||||
|
||||
### esbuild Config
|
||||
|
||||
```javascript
|
||||
// src/editor/build.mjs
|
||||
import { build } from 'esbuild';
|
||||
|
||||
await build({
|
||||
entryPoints: ['index.mjs'],
|
||||
bundle: true,
|
||||
minify: true,
|
||||
format: 'iife',
|
||||
target: ['es2020'],
|
||||
outfile: 'dist/codemirror.bundle.js',
|
||||
// CSS extracted automatically by esbuild
|
||||
});
|
||||
```
|
||||
|
||||
### package.json (editor only)
|
||||
|
||||
```json
|
||||
{
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"devDependencies": {
|
||||
"esbuild": "^0.25.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@codemirror/autocomplete": "^6",
|
||||
"@codemirror/commands": "^6",
|
||||
"@codemirror/lang-css": "^6",
|
||||
"@codemirror/lang-go": "^6",
|
||||
"@codemirror/lang-html": "^6",
|
||||
"@codemirror/lang-javascript": "^6",
|
||||
"@codemirror/lang-json": "^6",
|
||||
"@codemirror/lang-markdown": "^6",
|
||||
"@codemirror/lang-sql": "^6",
|
||||
"@codemirror/lang-python": "^6",
|
||||
"@codemirror/lang-rust": "^6",
|
||||
"@codemirror/lang-yaml": "^6",
|
||||
"@codemirror/language": "^6",
|
||||
"@codemirror/search": "^6",
|
||||
"@codemirror/state": "^6",
|
||||
"@codemirror/theme-one-dark": "^6",
|
||||
"@codemirror/view": "^6",
|
||||
"@replit/codemirror-vim": "^6",
|
||||
"@replit/codemirror-emacs": "^6"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Surfaces
|
||||
|
||||
### 1. Chat Input — Markdown Mode
|
||||
|
||||
Replace `<textarea id="messageInput">` with a CM6 instance configured for chat composition.
|
||||
|
||||
**Behavior:**
|
||||
|
||||
- Markdown syntax highlighting as you type (fenced code blocks, bold, italic, links, etc.)
|
||||
- Shift+Enter inserts newline (existing behavior preserved)
|
||||
- Enter sends message (existing behavior preserved)
|
||||
- Auto-growing height (CM6 handles this natively via `EditorView.contentAttributes`)
|
||||
- Placeholder text: "Send a message..."
|
||||
- No line numbers, no gutter — minimal chrome
|
||||
- Tab inserts real tab (indentWithTab) inside code blocks; normal behavior outside
|
||||
|
||||
**Factory:**
|
||||
|
||||
```javascript
|
||||
CM.chatInput(document.getElementById('messageInputContainer'), {
|
||||
placeholder: 'Send a message...',
|
||||
onSubmit: (text) => sendMessage(text), // Enter key
|
||||
onChange: (text) => updateInputTokens(text), // live token count
|
||||
darkMode: true,
|
||||
});
|
||||
```
|
||||
|
||||
**Migration path:**
|
||||
|
||||
The existing code reads `document.getElementById('messageInput').value` in many places. The factory returns an object with a `.getValue()` / `.setValue()` / `.focus()` API that matches the textarea interface. A thin shim keeps all existing callsites working:
|
||||
|
||||
```javascript
|
||||
// After CM.chatInput() creates the editor:
|
||||
const editor = CM.chatInput(...);
|
||||
|
||||
// Shim: existing code that reads .value still works
|
||||
Object.defineProperty(document.getElementById('messageInput'), 'value', {
|
||||
get: () => editor.getValue(),
|
||||
set: (v) => editor.setValue(v),
|
||||
});
|
||||
```
|
||||
|
||||
Alternatively (cleaner): introduce `ChatInput.getValue()` / `ChatInput.setValue()` / `ChatInput.focus()` and update callsites. There aren't many — `chat.js`, `attachments.js`, `tokens.js`, and a few event handlers.
|
||||
|
||||
### 2. Extension Editor — JavaScript/JSON Mode
|
||||
|
||||
Replace the two `<textarea>` elements in `editAdminExtension()` (`admin-handlers.js` line 706-710) with CM6 instances.
|
||||
|
||||
**Behavior:**
|
||||
|
||||
- Manifest textarea → JSON mode with bracket matching, auto-indent
|
||||
- Script textarea → JavaScript mode with bracket matching, auto-indent
|
||||
- Line numbers enabled
|
||||
- Search/replace (Ctrl+F) — currently impossible in a textarea
|
||||
- Tab key inserts proper indent (replaces the manual keydown handler at line 726-736)
|
||||
- Themed to match the app's dark/light mode
|
||||
|
||||
**Factory:**
|
||||
|
||||
```javascript
|
||||
CM.codeEditor(document.getElementById('extEdit-manifest-container'), {
|
||||
language: 'json',
|
||||
value: manifestJSON,
|
||||
lineNumbers: true,
|
||||
darkMode: isDark,
|
||||
});
|
||||
|
||||
CM.codeEditor(document.getElementById('extEdit-script-container'), {
|
||||
language: 'javascript',
|
||||
value: script,
|
||||
lineNumbers: true,
|
||||
darkMode: isDark,
|
||||
keymap: prefs.editorKeymap || 'standard', // 'standard' | 'vim' | 'emacs'
|
||||
});
|
||||
```
|
||||
|
||||
**Save integration:** `saveAdminExtension()` currently reads `.value` from the textareas. The CM6 instances expose `.getValue()` — update the save function to call that instead.
|
||||
|
||||
### 3. Future Code Surface (Roadmap)
|
||||
|
||||
The `CM.codeEditor()` factory with language auto-detection covers this. No additional work needed now — just document that it's available for the extensions surface type when that ships.
|
||||
|
||||
---
|
||||
|
||||
## Loading Strategy
|
||||
|
||||
The CM6 bundle is **not** in `SHELL_FILES` (service worker precache list). It loads on demand:
|
||||
|
||||
```html
|
||||
<!-- index.html: load after core app scripts, before app.js init -->
|
||||
<script src="vendor/codemirror/codemirror.bundle.js?v=%%APP_VERSION%%"
|
||||
onerror="console.warn('CM6 not available — falling back to textarea')"></script>
|
||||
<link rel="stylesheet" href="vendor/codemirror/codemirror.bundle.css?v=%%APP_VERSION%%"
|
||||
onerror="this.remove()">
|
||||
```
|
||||
|
||||
**Graceful degradation:** If the bundle fails to load (disconnected environment without vendored copy, build issue, etc.), `window.CM` is undefined and all integration points fall back to plain `<textarea>`. Each factory callsite checks:
|
||||
|
||||
```javascript
|
||||
if (window.CM) {
|
||||
CM.chatInput(container, opts);
|
||||
} else {
|
||||
// existing textarea behavior — unchanged
|
||||
}
|
||||
```
|
||||
|
||||
This means CM6 is a progressive enhancement, not a hard dependency.
|
||||
|
||||
---
|
||||
|
||||
## SW Cache Considerations
|
||||
|
||||
Add `vendor/codemirror/` to the SW exclusion list (same fix as the extensions path discussed earlier):
|
||||
|
||||
```javascript
|
||||
if (url.pathname.includes('/api/') ||
|
||||
url.pathname.includes('/ws') ||
|
||||
url.pathname.includes('/branding/') ||
|
||||
url.pathname.includes('/extensions/') ||
|
||||
url.pathname.includes('/vendor/codemirror/') ||
|
||||
event.request.method !== 'GET') {
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
Or, more broadly, exclude all of `/vendor/` if you want vendor updates to always bypass the SW cache. The core vendors (marked, purify) are small enough that network-first is fine.
|
||||
|
||||
Alternatively, since the bundle is versioned via `?v=%%APP_VERSION%%` and only changes on new builds, it could safely be in `SHELL_FILES` for precaching — unlike extensions which are user-editable at runtime. Either approach works.
|
||||
|
||||
---
|
||||
|
||||
## Theme Integration
|
||||
|
||||
CM6's `oneDark` theme for dark mode. For light mode, the CM6 default is clean and neutral.
|
||||
|
||||
Both need CSS variable overrides to match the app's existing `--bg`, `--bg-2`, `--border`, `--text`, `--accent` palette:
|
||||
|
||||
```javascript
|
||||
const switchboardTheme = EditorView.theme({
|
||||
'&': {
|
||||
backgroundColor: 'var(--bg)',
|
||||
color: 'var(--text)',
|
||||
fontSize: '14px',
|
||||
},
|
||||
'.cm-content': {
|
||||
fontFamily: 'var(--mono)',
|
||||
caretColor: 'var(--accent)',
|
||||
},
|
||||
'.cm-cursor': {
|
||||
borderLeftColor: 'var(--accent)',
|
||||
},
|
||||
'&.cm-focused .cm-selectionBackground, .cm-selectionBackground': {
|
||||
backgroundColor: 'var(--accent-muted, rgba(99,102,241,0.2))',
|
||||
},
|
||||
'.cm-gutters': {
|
||||
backgroundColor: 'var(--bg-2)',
|
||||
color: 'var(--text-3)',
|
||||
borderRight: '1px solid var(--border)',
|
||||
},
|
||||
}, { dark: document.body.classList.contains('dark-theme') });
|
||||
```
|
||||
|
||||
This goes into the bundle so it's applied automatically.
|
||||
|
||||
---
|
||||
|
||||
## File Layout
|
||||
|
||||
```
|
||||
src/
|
||||
editor/
|
||||
package.json # CM6 deps + esbuild
|
||||
build.mjs # esbuild script
|
||||
index.mjs # bundle entrypoint, exports window.CM
|
||||
theme.mjs # switchboard theme overrides
|
||||
chat-input.mjs # chatInput() factory
|
||||
code-editor.mjs # codeEditor() factory
|
||||
js/
|
||||
... # existing app code (unchanged)
|
||||
vendor/
|
||||
marked.min.js # existing
|
||||
purify.min.js # existing
|
||||
codemirror/ # gitignored — built by Docker
|
||||
codemirror.bundle.js
|
||||
codemirror.bundle.css
|
||||
```
|
||||
|
||||
The `src/editor/` directory is self-contained. It has its own `package.json` and `node_modules` (inside Docker only). No npm dependencies bleed into the main `src/js/` code. The build output lands in `vendor/codemirror/` which is gitignored like the other Docker-built vendors.
|
||||
|
||||
---
|
||||
|
||||
## Dockerfile Changes
|
||||
|
||||
```dockerfile
|
||||
# Stage 1: vendor libs (existing)
|
||||
FROM node:20-alpine AS vendor
|
||||
# ... existing marked, purify, mermaid, katex extraction ...
|
||||
|
||||
# Stage 2: CM6 bundle (NEW)
|
||||
FROM node:20-alpine AS cm6-build
|
||||
WORKDIR /build
|
||||
COPY src/editor/ ./
|
||||
RUN npm ci
|
||||
RUN node build.mjs
|
||||
|
||||
# Stage 3: nginx (existing, with additions)
|
||||
FROM nginx:1-alpine
|
||||
# ... existing setup ...
|
||||
COPY src/ /usr/share/nginx/html/
|
||||
COPY --from=vendor /vendor/ /usr/share/nginx/html/vendor/
|
||||
COPY --from=cm6-build /build/dist/ /usr/share/nginx/html/vendor/codemirror/
|
||||
# ... rest unchanged ...
|
||||
```
|
||||
|
||||
The `vendor` and `cm6-build` stages run in parallel (Docker BuildKit), so build time impact is minimal.
|
||||
|
||||
---
|
||||
|
||||
## Migration Checklist
|
||||
|
||||
### Phase 1: Bundle + Extension Editor
|
||||
- [ ] Create `src/editor/` with package.json, build.mjs, index.mjs
|
||||
- [ ] Implement `CM.codeEditor()` factory
|
||||
- [ ] Update Dockerfile.frontend with cm6-build stage
|
||||
- [ ] Replace extension editor textareas in `admin-handlers.js`
|
||||
- [ ] Remove manual Tab key handler (CM6 handles it)
|
||||
- [ ] Verify save/load round-trip with CM6 `.getValue()`
|
||||
- [ ] Add graceful fallback if `window.CM` undefined
|
||||
|
||||
### Phase 2: Chat Input
|
||||
- [ ] Implement `CM.chatInput()` factory with markdown mode
|
||||
- [ ] Wire Enter=send, Shift+Enter=newline keybindings
|
||||
- [ ] Integrate with `updateInputTokens()` via onChange callback
|
||||
- [ ] Update `chat.js`, `attachments.js`, `tokens.js` to use CM API
|
||||
- [ ] Auto-grow height behavior matching current textarea
|
||||
- [ ] Test paste handling (plain text, code, attachments)
|
||||
- [ ] Test mobile/touch input
|
||||
|
||||
### Phase 3: Polish
|
||||
- [ ] Theme integration with CSS variables
|
||||
- [ ] Dark/light mode switching (listen for theme toggle)
|
||||
- [ ] Editor keymap preference UI (Standard / Vim / Emacs) in appearance settings
|
||||
- [ ] Vim/Emacs modes on code editor + extension editor only (not chat input)
|
||||
- [ ] Add to `SHELL_FILES` in sw.js (or exclude from SW cache)
|
||||
- [ ] Update `debug.js` snapshot to include CM6 version
|
||||
- [ ] Documentation in ARCHITECTURE.md
|
||||
|
||||
---
|
||||
|
||||
## Bundle Size Estimate
|
||||
|
||||
Based on CM6 package sizes (minified + tree-shaken by esbuild):
|
||||
|
||||
| Package | Approx Size |
|
||||
|---------|-------------|
|
||||
| @codemirror/view + state + commands | ~90KB |
|
||||
| @codemirror/language + highlight | ~30KB |
|
||||
| lang-markdown + lang-javascript + lang-json | ~40KB |
|
||||
| lang-go + lang-sql + lang-html + lang-css + lang-yaml | ~50KB |
|
||||
| lang-python + lang-rust | ~20KB |
|
||||
| theme-one-dark | ~5KB |
|
||||
| search + autocomplete | ~20KB |
|
||||
| @replit/codemirror-vim + emacs | ~40KB |
|
||||
| **Total (minified)** | **~295KB** |
|
||||
| **Gzipped** | **~90KB** |
|
||||
|
||||
For comparison: mermaid.min.js is ~1.2MB. This is modest.
|
||||
|
||||
---
|
||||
|
||||
## Resolved Decisions
|
||||
|
||||
1. **Language modes** — Ship Python, Rust, and YAML upfront alongside Go, JS, JSON, SQL, HTML, CSS, and Markdown. Covers the team's stack. Additional modes are a one-line import + rebuild.
|
||||
|
||||
2. **Local dev build script** — Provide a `scripts/build-editor.sh` that runs the esbuild step standalone, shared by both Dockerfile.frontend and the unified Dockerfile. Developers can also run it locally to get the CM6 bundle without a full Docker build.
|
||||
|
||||
3. **Vim/Emacs keybindings** — Ship both. Exposed as a user preference toggle (default: standard). Applied via `CM.keybindings.vim` / `CM.keybindings.emacs` as CM6 extensions at editor creation time.
|
||||
|
||||
---
|
||||
|
||||
## Build Script
|
||||
|
||||
Shared script used by both Dockerfiles and local dev:
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
# scripts/build-editor.sh — Build CM6 bundle
|
||||
# Usage: ./scripts/build-editor.sh [output-dir]
|
||||
#
|
||||
# Defaults to src/vendor/codemirror/ for local dev.
|
||||
# Dockerfiles pass /build/dist/ as the output dir.
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
EDITOR_DIR="${SCRIPT_DIR}/../src/editor"
|
||||
OUTPUT_DIR="${1:-${SCRIPT_DIR}/../src/vendor/codemirror}"
|
||||
|
||||
cd "${EDITOR_DIR}"
|
||||
|
||||
# Install if needed (CI/Docker will already have node_modules)
|
||||
[ -d node_modules ] || npm ci
|
||||
|
||||
mkdir -p "${OUTPUT_DIR}"
|
||||
node build.mjs --outdir="${OUTPUT_DIR}"
|
||||
|
||||
echo "✅ CM6 bundle → ${OUTPUT_DIR}"
|
||||
```
|
||||
|
||||
Dockerfile usage:
|
||||
|
||||
```dockerfile
|
||||
FROM node:20-alpine AS cm6-build
|
||||
WORKDIR /build
|
||||
COPY src/editor/ /build/src/editor/
|
||||
COPY scripts/build-editor.sh /build/scripts/build-editor.sh
|
||||
RUN cd /build/src/editor && npm ci
|
||||
RUN sh /build/scripts/build-editor.sh /build/dist
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Keybinding Preference
|
||||
|
||||
Add to user appearance preferences (alongside dark/light theme toggle):
|
||||
|
||||
```
|
||||
Editor keybindings: [Standard ▾] [Vim] [Emacs]
|
||||
```
|
||||
|
||||
Stored in `localStorage` under `cs-appearance`:
|
||||
|
||||
```javascript
|
||||
const prefs = JSON.parse(localStorage.getItem('cs-appearance') || '{}');
|
||||
// prefs.editorKeymap = 'standard' | 'vim' | 'emacs'
|
||||
```
|
||||
|
||||
Applied at editor creation time:
|
||||
|
||||
```javascript
|
||||
const keymapExt = [];
|
||||
if (prefs.editorKeymap === 'vim') keymapExt.push(CM.keybindings.vim());
|
||||
else if (prefs.editorKeymap === 'emacs') keymapExt.push(CM.keybindings.emacs());
|
||||
|
||||
CM.codeEditor(target, {
|
||||
language: 'javascript',
|
||||
extensions: keymapExt,
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
**Note:** Vim/Emacs modes apply only to the code editor and extension editor surfaces. The chat input always uses standard keybindings to avoid confusing Enter-to-send behavior with modal editing.
|
||||
Reference in New Issue
Block a user