core/docs/archive/DESIGN-CM6.md

DESIGN: CodeMirror 6 Integration

Version: v0.17.2 Status: Complete Scope: Add CM6 as a vendored dependency, bundled at Docker build time via esbuild. Three integration surfaces.

---

Motivation

Three features converge on the same dependency:

1. Chat input — live markdown rendering (backtick → code block, bold, etc.)
2. Extension editor — syntax-highlighted JavaScript/JSON editing in admin panel (replaces bare <textarea>)
3. Code editing surface (roadmap) — future extension surface type for code review, snippets, etc.

CM6 is ESM-native and expects a bundler. The build step is contained in the Docker image build — the frontend continues to ship as static assets served by nginx. No runtime build tooling, no change to the developer experience for non-CM6 code.

---

Architecture

Build Pipeline

New Docker stage between the existing vendor stage and the final nginx stage:

Stage 1: vendor       (existing — marked, purify, mermaid, katex)
Stage 2: cm6-build    (NEW — npm install + esbuild → single IIFE bundle)
Stage 3: nginx        (existing — copies src/ + vendor/ + cm6 bundle)

The CM6 build stage:

FROM node:20-alpine AS cm6-build
WORKDIR /build
COPY src/editor/package.json src/editor/build.mjs ./
RUN npm install --production=false
RUN node build.mjs
# Output: /build/dist/codemirror.bundle.js (~180-250KB minified)
#         /build/dist/codemirror.bundle.css (~5-10KB)

Final stage addition:

COPY --from=cm6-build /build/dist/ /usr/share/nginx/html/vendor/codemirror/

Bundle Structure

A single esbuild entrypoint (src/editor/index.mjs) that imports CM6 packages and exposes factory functions on window.CM:

// src/editor/index.mjs — esbuild entrypoint
import { EditorView, keymap, placeholder, ViewPlugin, ... } from '@codemirror/view';
import { EditorState, ... } from '@codemirror/state';
import { markdown, markdownLanguage } from '@codemirror/lang-markdown';
import { javascript } from '@codemirror/lang-javascript';
import { json } from '@codemirror/lang-json';
import { sql } from '@codemirror/lang-sql';
import { html } from '@codemirror/lang-html';
import { css } from '@codemirror/lang-css';
import { yaml } from '@codemirror/lang-yaml';
import { go } from '@codemirror/lang-go';
import { python } from '@codemirror/lang-python';
import { rust } from '@codemirror/lang-rust';
import { oneDark } from '@codemirror/theme-one-dark';
import { defaultKeymap, indentWithTab } from '@codemirror/commands';
import { bracketMatching, ... } from '@codemirror/language';
import { highlightSelectionMatches, searchKeymap } from '@codemirror/search';
import { autocompletion } from '@codemirror/autocomplete';
import { vim } from '@replit/codemirror-vim';
import { emacs } from '@replit/codemirror-emacs';

// Expose on window for script-tag consumption
window.CM = {
    EditorView,
    EditorState,

    // Factory: chat input (markdown mode, minimal chrome)
    chatInput(target, opts = {}) { ... },

    // Factory: code editor (full features, language auto-detect)
    codeEditor(target, opts = {}) { ... },

    // Supported languages for code editor
    languages: { markdown, javascript, json, sql, html, css, yaml, go, python, rust },

    // Keybinding modes (applied via user preference)
    keybindings: { vim, emacs },
};

esbuild Config

// src/editor/build.mjs
import { build } from 'esbuild';

await build({
    entryPoints: ['index.mjs'],
    bundle: true,
    minify: true,
    format: 'iife',
    target: ['es2020'],
    outfile: 'dist/codemirror.bundle.js',
    // CSS extracted automatically by esbuild
});

package.json (editor only)

{
  "private": true,
  "type": "module",
  "devDependencies": {
    "esbuild": "^0.25.0"
  },
  "dependencies": {
    "@codemirror/autocomplete": "^6",
    "@codemirror/commands": "^6",
    "@codemirror/lang-css": "^6",
    "@codemirror/lang-go": "^6",
    "@codemirror/lang-html": "^6",
    "@codemirror/lang-javascript": "^6",
    "@codemirror/lang-json": "^6",
    "@codemirror/lang-markdown": "^6",
    "@codemirror/lang-sql": "^6",
    "@codemirror/lang-python": "^6",
    "@codemirror/lang-rust": "^6",
    "@codemirror/lang-yaml": "^6",
    "@codemirror/language": "^6",
    "@codemirror/search": "^6",
    "@codemirror/state": "^6",
    "@codemirror/theme-one-dark": "^6",
    "@codemirror/view": "^6",
    "@replit/codemirror-vim": "^6",
    "@replit/codemirror-emacs": "^6"
  }
}

---

Integration Surfaces

1. Chat Input — Markdown Mode

Replace <textarea id="messageInput"> with a CM6 instance configured for chat composition.

Behavior:

• Markdown syntax highlighting as you type (fenced code blocks, bold, italic, links, etc.)
• Shift+Enter inserts newline (existing behavior preserved)
• Enter sends message (existing behavior preserved)
• Auto-growing height (CM6 handles this natively via EditorView.contentAttributes)
• Placeholder text: "Send a message..."
• No line numbers, no gutter — minimal chrome
• Tab inserts real tab (indentWithTab) inside code blocks; normal behavior outside

Factory:

CM.chatInput(document.getElementById('messageInputContainer'), {
    placeholder: 'Send a message...',
    onSubmit: (text) => sendMessage(text),        // Enter key
    onChange: (text) => updateInputTokens(text),   // live token count
    darkMode: true,
});

Migration path:

The existing code reads document.getElementById('messageInput').value in many places. The factory returns an object with a .getValue() / .setValue() / .focus() API that matches the textarea interface. A thin shim keeps all existing callsites working:

// After CM.chatInput() creates the editor:
const editor = CM.chatInput(...);

// Shim: existing code that reads .value still works
Object.defineProperty(document.getElementById('messageInput'), 'value', {
    get: () => editor.getValue(),
    set: (v) => editor.setValue(v),
});

Alternatively (cleaner): introduce ChatInput.getValue() / ChatInput.setValue() / ChatInput.focus() and update callsites. There aren't many — chat.js, attachments.js, tokens.js, and a few event handlers.

2. Extension Editor — JavaScript/JSON Mode

Replace the two <textarea> elements in editAdminExtension() (admin-handlers.js line 706-710) with CM6 instances.

Behavior:

• Manifest textarea → JSON mode with bracket matching, auto-indent
• Script textarea → JavaScript mode with bracket matching, auto-indent
• Line numbers enabled
• Search/replace (Ctrl+F) — currently impossible in a textarea
• Tab key inserts proper indent (replaces the manual keydown handler at line 726-736)
• Themed to match the app's dark/light mode

Factory:

CM.codeEditor(document.getElementById('extEdit-manifest-container'), {
    language: 'json',
    value: manifestJSON,
    lineNumbers: true,
    darkMode: isDark,
});

CM.codeEditor(document.getElementById('extEdit-script-container'), {
    language: 'javascript',
    value: script,
    lineNumbers: true,
    darkMode: isDark,
    keymap: prefs.editorKeymap || 'standard',  // 'standard' | 'vim' | 'emacs'
});

Save integration: saveAdminExtension() currently reads .value from the textareas. The CM6 instances expose .getValue() — update the save function to call that instead.

3. Future Code Surface (Roadmap)

The CM.codeEditor() factory with language auto-detection covers this. No additional work needed now — just document that it's available for the extensions surface type when that ships.

---

Loading Strategy

The CM6 bundle is not in SHELL_FILES (service worker precache list). It loads on demand:

<!-- index.html: load after core app scripts, before app.js init -->
<script src="vendor/codemirror/codemirror.bundle.js?v=%%APP_VERSION%%"
        onerror="console.warn('CM6 not available — falling back to textarea')"></script>
<link rel="stylesheet" href="vendor/codemirror/codemirror.bundle.css?v=%%APP_VERSION%%"
      onerror="this.remove()">

Graceful degradation: If the bundle fails to load (disconnected environment without vendored copy, build issue, etc.), window.CM is undefined and all integration points fall back to plain <textarea>. Each factory callsite checks:

if (window.CM) {
    CM.chatInput(container, opts);
} else {
    // existing textarea behavior — unchanged
}

This means CM6 is a progressive enhancement, not a hard dependency.

---

SW Cache Considerations

Add vendor/codemirror/ to the SW exclusion list (same fix as the extensions path discussed earlier):

if (url.pathname.includes('/api/') ||
    url.pathname.includes('/ws') ||
    url.pathname.includes('/branding/') ||
    url.pathname.includes('/extensions/') ||
    url.pathname.includes('/vendor/codemirror/') ||
    event.request.method !== 'GET') {
    return;
}

Or, more broadly, exclude all of /vendor/ if you want vendor updates to always bypass the SW cache. The core vendors (marked, purify) are small enough that network-first is fine.

Alternatively, since the bundle is versioned via ?v=%%APP_VERSION%% and only changes on new builds, it could safely be in SHELL_FILES for precaching — unlike extensions which are user-editable at runtime. Either approach works.

---

Theme Integration

CM6's oneDark theme for dark mode. For light mode, the CM6 default is clean and neutral.

Both need CSS variable overrides to match the app's existing --bg, --bg-2, --border, --text, --accent palette:

const switchboardTheme = EditorView.theme({
    '&': {
        backgroundColor: 'var(--bg)',
        color: 'var(--text)',
        fontSize: '14px',
    },
    '.cm-content': {
        fontFamily: 'var(--mono)',
        caretColor: 'var(--accent)',
    },
    '.cm-cursor': {
        borderLeftColor: 'var(--accent)',
    },
    '&.cm-focused .cm-selectionBackground, .cm-selectionBackground': {
        backgroundColor: 'var(--accent-muted, rgba(99,102,241,0.2))',
    },
    '.cm-gutters': {
        backgroundColor: 'var(--bg-2)',
        color: 'var(--text-3)',
        borderRight: '1px solid var(--border)',
    },
}, { dark: document.body.classList.contains('dark-theme') });

This goes into the bundle so it's applied automatically.

---

File Layout

src/
  editor/
    package.json          # CM6 deps + esbuild
    build.mjs             # esbuild script
    index.mjs             # bundle entrypoint, exports window.CM
    theme.mjs             # switchboard theme overrides
    chat-input.mjs        # chatInput() factory
    code-editor.mjs       # codeEditor() factory
  js/
    ...                   # existing app code (unchanged)
  vendor/
    marked.min.js         # existing
    purify.min.js         # existing
    codemirror/           # gitignored — built by Docker
      codemirror.bundle.js
      codemirror.bundle.css

The src/editor/ directory is self-contained. It has its own package.json and node_modules (inside Docker only). No npm dependencies bleed into the main src/js/ code. The build output lands in vendor/codemirror/ which is gitignored like the other Docker-built vendors.

---

Dockerfile Changes

# Stage 1: vendor libs (existing)
FROM node:20-alpine AS vendor
# ... existing marked, purify, mermaid, katex extraction ...

# Stage 2: CM6 bundle (NEW)
FROM node:20-alpine AS cm6-build
WORKDIR /build
COPY src/editor/ ./
RUN npm ci
RUN node build.mjs

# Stage 3: nginx (existing, with additions)
FROM nginx:1-alpine
# ... existing setup ...
COPY src/ /usr/share/nginx/html/
COPY --from=vendor /vendor/ /usr/share/nginx/html/vendor/
COPY --from=cm6-build /build/dist/ /usr/share/nginx/html/vendor/codemirror/
# ... rest unchanged ...

The vendor and cm6-build stages run in parallel (Docker BuildKit), so build time impact is minimal.

---

Migration Checklist

Phase 1: Bundle + Extension Editor

• Create src/editor/ with package.json, build.mjs, index.mjs
• Implement CM.codeEditor() factory
• Update Dockerfile.frontend with cm6-build stage
• Replace extension editor textareas in admin-handlers.js
• Remove manual Tab key handler (CM6 handles it)
• Verify save/load round-trip with CM6 .getValue()
• Add graceful fallback if window.CM undefined

Phase 2: Chat Input

• Implement CM.chatInput() factory with markdown mode
• Wire Enter=send, Shift+Enter=newline keybindings
• Integrate with updateInputTokens() via onChange callback
• Update chat.js, attachments.js, tokens.js to use CM API
• Auto-grow height behavior matching current textarea
• Test paste handling (plain text, code, attachments)
• Test mobile/touch input

Phase 3: Polish

• Theme integration with CSS variables
• Dark/light mode switching (listen for theme toggle)
• Editor keymap preference UI (Standard / Vim / Emacs) in appearance settings
• Vim/Emacs modes on code editor + extension editor only (not chat input)
• Add to SHELL_FILES in sw.js (or exclude from SW cache)
• Update debug.js snapshot to include CM6 version
• Documentation in ARCHITECTURE.md

---

Bundle Size Estimate

Based on CM6 package sizes (minified + tree-shaken by esbuild):

Package	Approx Size
@codemirror/view + state + commands	~90KB
@codemirror/language + highlight	~30KB
lang-markdown + lang-javascript + lang-json	~40KB
lang-go + lang-sql + lang-html + lang-css + lang-yaml	~50KB
lang-python + lang-rust	~20KB
theme-one-dark	~5KB
search + autocomplete	~20KB
@replit/codemirror-vim + emacs	~40KB
Total (minified)	~295KB
Gzipped	~90KB

For comparison: mermaid.min.js is ~1.2MB. This is modest.

---

Resolved Decisions

1. Language modes — Ship Python, Rust, and YAML upfront alongside Go, JS, JSON, SQL, HTML, CSS, and Markdown. Covers the team's stack. Additional modes are a one-line import + rebuild.

2. Local dev build script — Provide a scripts/build-editor.sh that runs the esbuild step standalone, shared by both Dockerfile.frontend and the unified Dockerfile. Developers can also run it locally to get the CM6 bundle without a full Docker build.

3. Vim/Emacs keybindings — Ship both. Exposed as a user preference toggle (default: standard). Applied via CM.keybindings.vim / CM.keybindings.emacs as CM6 extensions at editor creation time.

---

Build Script

Shared script used by both Dockerfiles and local dev:

#!/bin/sh
# scripts/build-editor.sh — Build CM6 bundle
# Usage: ./scripts/build-editor.sh [output-dir]
#
# Defaults to src/vendor/codemirror/ for local dev.
# Dockerfiles pass /build/dist/ as the output dir.
set -e

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
EDITOR_DIR="${SCRIPT_DIR}/../src/editor"
OUTPUT_DIR="${1:-${SCRIPT_DIR}/../src/vendor/codemirror}"

cd "${EDITOR_DIR}"

# Install if needed (CI/Docker will already have node_modules)
[ -d node_modules ] || npm ci

mkdir -p "${OUTPUT_DIR}"
node build.mjs --outdir="${OUTPUT_DIR}"

echo "✅ CM6 bundle → ${OUTPUT_DIR}"

Dockerfile usage:

FROM node:20-alpine AS cm6-build
WORKDIR /build
COPY src/editor/ /build/src/editor/
COPY scripts/build-editor.sh /build/scripts/build-editor.sh
RUN cd /build/src/editor && npm ci
RUN sh /build/scripts/build-editor.sh /build/dist

---

Keybinding Preference

Add to user appearance preferences (alongside dark/light theme toggle):

Editor keybindings: [Standard ▾] [Vim] [Emacs]

Stored in localStorage under cs-appearance:

const prefs = JSON.parse(localStorage.getItem('cs-appearance') || '{}');
// prefs.editorKeymap = 'standard' | 'vim' | 'emacs'

Applied at editor creation time:

const keymapExt = [];
if (prefs.editorKeymap === 'vim') keymapExt.push(CM.keybindings.vim());
else if (prefs.editorKeymap === 'emacs') keymapExt.push(CM.keybindings.emacs());

CM.codeEditor(target, {
    language: 'javascript',
    extensions: keymapExt,
    // ...
});

Note: Vim/Emacs modes apply only to the code editor and extension editor surfaces. The chat input always uses standard keybindings to avoid confusing Enter-to-send behavior with modal editing.