977 lines
37 KiB
Markdown
977 lines
37 KiB
Markdown
# Design — v0.21.x: Workspaces + Extension Surfaces
|
||
|
||
**Status:** Planning
|
||
**Depends on:** v0.20.0 (notifications), v0.17.2 (CM6), v0.11.0 (extension foundation)
|
||
**Feeds into:** v0.22.0 (smart routing), v0.23.0 (multi-participant channels)
|
||
|
||
---
|
||
|
||
## Guiding Principle
|
||
|
||
The **workspace** is a platform primitive — not an editor-mode concept. Even in
|
||
pure chat mode, uploading a zip, having the AI operate on real files at real
|
||
paths, and downloading the result as an archive is a first-class workflow. Every
|
||
surface (chat, editor, article, future modes) benefits from workspace access.
|
||
|
||
This changes the layering: workspace infrastructure lands first as foundational
|
||
storage, then surfaces consume it.
|
||
|
||
---
|
||
|
||
## Release Decomposition
|
||
|
||
```
|
||
v0.21.0 Workspace Storage (platform primitive)
|
||
│
|
||
v0.21.1 Workspace Tools + Channel/Project Binding
|
||
│
|
||
v0.21.2 Workspace Indexing + Semantic Search
|
||
│
|
||
v0.21.3 Surface Infrastructure + REPL
|
||
│
|
||
v0.21.4 Git Integration
|
||
│
|
||
v0.21.5 Editor Surface (Development Mode)
|
||
│
|
||
v0.21.6 Article Surface + Document Output Pipeline
|
||
```
|
||
|
||
---
|
||
|
||
## v0.21.0 — Workspace Storage
|
||
|
||
Pure backend. No tools, no UI, no surfaces. Just the storage primitive with
|
||
full CRUD API and archive support.
|
||
|
||
### Data Model
|
||
|
||
**`workspaces` table**
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| `id` | UUID | PK |
|
||
| `owner_type` | TEXT | `user`, `project`, `channel`, `team` |
|
||
| `owner_id` | UUID | FK polymorphic |
|
||
| `name` | TEXT | display name |
|
||
| `root_path` | TEXT | PVC-relative path: `workspaces/{id}` |
|
||
| `max_bytes` | BIGINT | quota (NULL = system default) |
|
||
| `status` | TEXT | `active`, `archived`, `deleting` |
|
||
| `created_at` | TIMESTAMPTZ | |
|
||
| `updated_at` | TIMESTAMPTZ | |
|
||
|
||
Index: `(owner_type, owner_id)` — lookup workspace(s) for a given owner.
|
||
|
||
**`workspace_files` table** (metadata index — filesystem is source of truth)
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| `id` | UUID | PK |
|
||
| `workspace_id` | UUID | FK → workspaces |
|
||
| `path` | TEXT | relative path from workspace root: `src/main.go` |
|
||
| `is_directory` | BOOLEAN | |
|
||
| `content_type` | TEXT | MIME type (detected or inferred) |
|
||
| `size_bytes` | BIGINT | 0 for dirs |
|
||
| `sha256` | TEXT | content hash (NULL for dirs) |
|
||
| `created_at` | TIMESTAMPTZ | |
|
||
| `updated_at` | TIMESTAMPTZ | |
|
||
|
||
Index: `UNIQUE (workspace_id, path)` — one entry per path.
|
||
Index: `(workspace_id, content_type)` — filter by type.
|
||
|
||
Design note: the DB index is a metadata cache. The PVC filesystem is the source
|
||
of truth. Syncing happens on write operations and optionally via a reconcile
|
||
endpoint. This avoids the complexity of fsnotify while keeping queries fast.
|
||
|
||
### Filesystem Layout (PVC)
|
||
|
||
```
|
||
/data/storage/
|
||
attachments/ # existing (v0.12.0)
|
||
processing/ # existing extraction queue
|
||
workspaces/ # NEW
|
||
{workspace_id}/
|
||
.workspace.json # metadata: owner, quotas, created_at
|
||
files/ # actual file tree root
|
||
src/
|
||
main.go
|
||
README.md
|
||
```
|
||
|
||
Why a separate `files/` subdirectory: keeps workspace metadata (`.workspace.json`,
|
||
future `.git/` in v0.21.4) out of the user's file namespace.
|
||
|
||
### Store Interface
|
||
|
||
```go
|
||
type WorkspaceStore interface {
|
||
// CRUD
|
||
Create(ctx context.Context, w *models.Workspace) error
|
||
GetByID(ctx context.Context, id string) (*models.Workspace, error)
|
||
Update(ctx context.Context, id string, patch models.WorkspacePatch) error
|
||
Delete(ctx context.Context, id string) error
|
||
|
||
// Ownership lookup
|
||
GetByOwner(ctx context.Context, ownerType, ownerID string) (*models.Workspace, error)
|
||
ListByOwner(ctx context.Context, ownerType, ownerID string) ([]models.Workspace, error)
|
||
|
||
// File index
|
||
UpsertFile(ctx context.Context, f *models.WorkspaceFile) error
|
||
DeleteFile(ctx context.Context, workspaceID, path string) error
|
||
DeleteFilesByPrefix(ctx context.Context, workspaceID, prefix string) error
|
||
GetFile(ctx context.Context, workspaceID, path string) (*models.WorkspaceFile, error)
|
||
ListFiles(ctx context.Context, workspaceID, prefix string, recursive bool) ([]models.WorkspaceFile, error)
|
||
|
||
// Stats
|
||
GetDiskUsage(ctx context.Context, workspaceID string) (int64, error)
|
||
}
|
||
```
|
||
|
||
### Workspace FS (Filesystem Operations)
|
||
|
||
Separate from the store — this operates on PVC, not DB.
|
||
|
||
```go
|
||
type WorkspaceFS struct {
|
||
basePath string // e.g. /data/storage/workspaces
|
||
store WorkspaceStore // for metadata sync
|
||
}
|
||
|
||
// File CRUD — all paths relative to workspace root
|
||
func (fs *WorkspaceFS) ReadFile(ctx context.Context, w *models.Workspace, path string) (io.ReadCloser, int64, error)
|
||
func (fs *WorkspaceFS) WriteFile(ctx context.Context, w *models.Workspace, path string, r io.Reader, size int64) error
|
||
func (fs *WorkspaceFS) DeleteFile(ctx context.Context, w *models.Workspace, path string) error
|
||
func (fs *WorkspaceFS) Mkdir(ctx context.Context, w *models.Workspace, path string) error
|
||
func (fs *WorkspaceFS) Stat(ctx context.Context, w *models.Workspace, path string) (*models.WorkspaceFile, error)
|
||
func (fs *WorkspaceFS) ListDir(ctx context.Context, w *models.Workspace, prefix string, recursive bool) ([]models.WorkspaceFile, error)
|
||
|
||
// Tree: returns full file listing with sizes (for file tree UI)
|
||
func (fs *WorkspaceFS) Tree(ctx context.Context, w *models.Workspace) ([]models.WorkspaceFile, error)
|
||
|
||
// Archive operations
|
||
func (fs *WorkspaceFS) ExtractArchive(ctx context.Context, w *models.Workspace, r io.Reader, format string) (int, error)
|
||
func (fs *WorkspaceFS) CreateArchive(ctx context.Context, w *models.Workspace, format string) (io.ReadCloser, error)
|
||
|
||
// Maintenance
|
||
func (fs *WorkspaceFS) Reconcile(ctx context.Context, w *models.Workspace) error // sync FS → DB index
|
||
func (fs *WorkspaceFS) Destroy(ctx context.Context, w *models.Workspace) error // rm -rf + DB cleanup
|
||
```
|
||
|
||
WriteFile updates the DB index (upsert workspace_files) after successful write.
|
||
DeleteFile removes the DB row. Reconcile walks the FS and patches any drift.
|
||
|
||
### API Endpoints
|
||
|
||
```
|
||
POST /api/v1/workspaces # create workspace
|
||
GET /api/v1/workspaces/:id # get metadata
|
||
PATCH /api/v1/workspaces/:id # update name/quota
|
||
DELETE /api/v1/workspaces/:id # delete (async cleanup)
|
||
|
||
GET /api/v1/workspaces/:id/files?path=&recursive= # list files
|
||
GET /api/v1/workspaces/:id/files/read?path= # read file content
|
||
PUT /api/v1/workspaces/:id/files/write?path= # write file (body = content)
|
||
DELETE /api/v1/workspaces/:id/files?path= # delete file/dir
|
||
POST /api/v1/workspaces/:id/files/mkdir?path= # create directory
|
||
|
||
POST /api/v1/workspaces/:id/archive/upload # upload + extract zip/tar.gz
|
||
GET /api/v1/workspaces/:id/archive/download?format= # download as zip/tar.gz
|
||
|
||
POST /api/v1/workspaces/:id/reconcile # force FS → DB sync
|
||
GET /api/v1/workspaces/:id/stats # disk usage, file count
|
||
```
|
||
|
||
### Security
|
||
|
||
- Workspace access inherits from owner: if you can access the project/channel,
|
||
you can access its workspace.
|
||
- Path traversal guard: all paths are cleaned and must resolve within the
|
||
workspace root. Symlinks are rejected.
|
||
- File size limit: configurable per-workspace `max_bytes` with system default
|
||
(`WORKSPACE_MAX_BYTES`, default 500MB).
|
||
- Archive extraction bomb protection: max files (10,000), max total extracted
|
||
size (workspace quota), max single file (100MB).
|
||
|
||
### Migrations
|
||
|
||
- `xxx_workspace_tables.sql` (Postgres + SQLite): `workspaces`, `workspace_files`
|
||
|
||
### Checklist
|
||
|
||
- [ ] `models.Workspace`, `models.WorkspaceFile`, `models.WorkspacePatch` structs
|
||
- [ ] `WorkspaceStore` interface in `store/interfaces.go`
|
||
- [ ] Postgres implementation: `store/postgres/workspace.go`
|
||
- [ ] SQLite implementation: `store/sqlite/workspace.go`
|
||
- [ ] `WorkspaceFS` in new `server/workspace/` package
|
||
- [ ] Path traversal validation (must not escape workspace root)
|
||
- [ ] Archive extract (zip, tar.gz) with bomb protection
|
||
- [ ] Archive create (zip, tar.gz)
|
||
- [ ] MIME type detection on write (`net/http.DetectContentType` + extension fallback)
|
||
- [ ] Workspace CRUD handlers (6 endpoints)
|
||
- [ ] File CRUD handlers (5 endpoints)
|
||
- [ ] Archive handlers (2 endpoints)
|
||
- [ ] Reconcile + stats handlers (2 endpoints)
|
||
- [ ] `Stores.Workspaces` wired in `store/postgres/stores.go` + `store/sqlite/stores.go`
|
||
- [ ] Integration tests: workspace CRUD, file CRUD, archive round-trip, path traversal rejection
|
||
- [ ] PVC directory creation on startup (`/data/storage/workspaces/`)
|
||
|
||
---
|
||
|
||
## v0.21.1 — Workspace Tools + Channel/Project Binding
|
||
|
||
Make workspaces useful in chat mode. AI can operate on files through tool calls.
|
||
Channels and projects get workspace bindings.
|
||
|
||
### Tools (category: `workspace`)
|
||
|
||
**`workspace_ls`** — list files in workspace
|
||
|
||
```json
|
||
{
|
||
"path": "src/",
|
||
"recursive": false
|
||
}
|
||
```
|
||
|
||
Returns: array of `{ path, is_directory, content_type, size_bytes }`.
|
||
|
||
**`workspace_read`** — read a file
|
||
|
||
```json
|
||
{
|
||
"path": "src/main.go"
|
||
}
|
||
```
|
||
|
||
Returns: file content as text (with truncation guard for large files, e.g. 50KB
|
||
soft limit with "file truncated" indicator). Binary files return a base64
|
||
summary or "binary file, N bytes" message.
|
||
|
||
**`workspace_write`** — create or overwrite a file
|
||
|
||
```json
|
||
{
|
||
"path": "src/main.go",
|
||
"content": "package main\n\nfunc main() {\n}\n"
|
||
}
|
||
```
|
||
|
||
Returns: confirmation with path and size. Creates parent directories
|
||
automatically.
|
||
|
||
**`workspace_rm`** — delete a file or directory
|
||
|
||
```json
|
||
{
|
||
"path": "src/old_module/",
|
||
"recursive": false
|
||
}
|
||
```
|
||
|
||
Returns: confirmation. Recursive required for non-empty directories.
|
||
|
||
**`workspace_mv`** — rename or move a file
|
||
|
||
```json
|
||
{
|
||
"source": "src/old.go",
|
||
"destination": "src/new.go"
|
||
}
|
||
```
|
||
|
||
**`workspace_patch`** — apply a text diff/patch to a file
|
||
|
||
```json
|
||
{
|
||
"path": "src/main.go",
|
||
"operations": [
|
||
{ "find": "oldFunction()", "replace": "newFunction()" },
|
||
{ "find": "// TODO: remove", "replace": "" }
|
||
]
|
||
}
|
||
```
|
||
|
||
The `find`/`replace` approach (same pattern as `str_replace` in many AI
|
||
coding tools) is more reliable for LLMs than unified diffs. Each operation's
|
||
`find` string must match exactly once in the file.
|
||
|
||
### Channel Workspace Binding
|
||
|
||
**Schema change:** `channels` table gains optional `workspace_id` FK.
|
||
|
||
**Behavior:**
|
||
- When a workspace is bound to a channel, workspace tools are injected into
|
||
the tool set for that channel's completions (same pattern as KB tools via
|
||
persona binding).
|
||
- New channel setting: "Workspace" toggle in channel settings.
|
||
- Creating a workspace from channel settings auto-binds it.
|
||
|
||
**Archive upload flow (chat mode):**
|
||
1. User uploads a zip/tar.gz to a channel with a workspace.
|
||
2. Upload handler detects archive MIME type + workspace binding.
|
||
3. Extracts into workspace (not attachment store).
|
||
4. Workspace tools become available for AI to operate on files.
|
||
5. User or AI can call archive download to get results.
|
||
|
||
Non-archive files still go to attachment store as before (backward compatible).
|
||
|
||
### Project Workspace Binding
|
||
|
||
**Schema change:** `projects` table gains optional `workspace_id` FK.
|
||
|
||
**Behavior:**
|
||
- Project detail panel gets a "Files" tab showing workspace file tree.
|
||
- All channels in a project inherit the project workspace (unless the channel
|
||
has its own).
|
||
- Workspace resolution: channel workspace > project workspace (same override
|
||
pattern as persona/KB resolution).
|
||
|
||
### Frontend
|
||
|
||
- Channel settings: workspace toggle, create/bind workspace.
|
||
- Project detail panel: Files tab with tree view (reusable component for
|
||
editor surface later).
|
||
- Chat bar: workspace indicator when active (similar to KB indicator).
|
||
|
||
### Checklist
|
||
|
||
- [ ] `workspace_ls` tool implementation
|
||
- [ ] `workspace_read` tool with text truncation guard + binary detection
|
||
- [ ] `workspace_write` tool with auto-mkdir
|
||
- [ ] `workspace_rm` tool with recursive guard
|
||
- [ ] `workspace_mv` tool
|
||
- [ ] `workspace_patch` tool with find/replace operations
|
||
- [ ] Tool registration in `workspace` category
|
||
- [ ] Tool injection in completion handler when workspace bound
|
||
- [ ] `channels.workspace_id` column (migration, both dialects)
|
||
- [ ] `projects.workspace_id` column (migration, both dialects)
|
||
- [ ] Archive-to-workspace upload handler (detect archive + workspace → extract)
|
||
- [ ] Workspace resolution: channel → project fallback
|
||
- [ ] Channel settings UI: workspace toggle
|
||
- [ ] Project detail panel: Files tab
|
||
- [ ] Chat bar: workspace indicator
|
||
- [ ] Integration tests: tool execution, workspace resolution, archive upload flow
|
||
- [ ] Update `ExecutionContext` with workspace ID
|
||
|
||
---
|
||
|
||
## v0.21.2 — Workspace Indexing + Semantic Search
|
||
|
||
Embed text files in workspace for semantic search. Reuses the existing
|
||
`knowledge/` chunker and embedder — they're already decoupled from KB-specific
|
||
concerns.
|
||
|
||
### Data Model
|
||
|
||
**`workspace_chunks` table**
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| `id` | UUID | PK |
|
||
| `workspace_id` | UUID | FK → workspaces |
|
||
| `file_id` | UUID | FK → workspace_files |
|
||
| `chunk_index` | INT | ordinal within file |
|
||
| `content` | TEXT | chunk text |
|
||
| `token_count` | INT | estimated |
|
||
| `embedding` | vector(3072) / JSON | pgvector or JSON text (SQLite) |
|
||
| `metadata` | JSONB / JSON | extensible: line_start, heading, etc. |
|
||
|
||
Index: pgvector IVFFlat on `embedding` (Postgres), app-level cosine (SQLite).
|
||
Index: `(file_id)` — re-index on file change.
|
||
|
||
### Indexing Pipeline
|
||
|
||
```
|
||
file write/upload
|
||
→ is text file? (same textMIMETypes + code extensions)
|
||
→ extract content (already in memory from write, or read from FS)
|
||
→ chunk (reuse knowledge.SplitText with configurable ChunkConfig)
|
||
→ embed (reuse knowledge.Embedder)
|
||
→ store in workspace_chunks (DELETE old chunks for file_id, INSERT new)
|
||
→ update workspace_files.sha256 (content-addressed skip on no-change)
|
||
```
|
||
|
||
**Code-aware MIME types** (extend `textMIMETypes`):
|
||
|
||
The existing KB ingester recognizes `text/plain`, `text/markdown`, `text/csv`,
|
||
`text/html`. For workspace indexing, we also recognize source code by extension:
|
||
|
||
```go
|
||
var indexableExtensions = map[string]bool{
|
||
".go": true, ".py": true, ".js": true, ".ts": true, ".jsx": true,
|
||
".tsx": true, ".rs": true, ".c": true, ".cpp": true, ".h": true,
|
||
".hpp": true, ".java": true, ".rb": true, ".php": true, ".swift": true,
|
||
".kt": true, ".scala": true, ".sh": true, ".bash": true, ".zsh": true,
|
||
".sql": true, ".yaml": true, ".yml": true, ".toml": true, ".json": true,
|
||
".xml": true, ".css": true, ".scss": true, ".less": true,
|
||
".md": true, ".txt": true, ".csv": true, ".html": true, ".htm": true,
|
||
".dockerfile": true, ".tf": true, ".hcl": true, ".proto": true,
|
||
".graphql": true, ".vue": true, ".svelte": true,
|
||
".pl": true, ".pm": true, // Perl
|
||
".lua": true, ".zig": true, ".nim": true, ".ex": true, ".exs": true,
|
||
}
|
||
```
|
||
|
||
**Content-addressed skip:** if `workspace_files.sha256` matches the new write,
|
||
skip re-chunking and re-embedding. This makes bulk operations (archive extract)
|
||
efficient — only changed files get re-indexed.
|
||
|
||
**Background indexing:** file writes trigger async indexing (same goroutine
|
||
pattern as KB ingestion with semaphore). The write API returns immediately; the
|
||
`workspace_files` row gets an `index_status` column: `pending`, `indexing`,
|
||
`ready`, `error`, `skipped` (binary/non-text).
|
||
|
||
### Code-Aware Chunking
|
||
|
||
For source code, the default chunker works reasonably well but we add a
|
||
code-specific separator hierarchy:
|
||
|
||
```go
|
||
func CodeChunkConfig() ChunkConfig {
|
||
return ChunkConfig{
|
||
ChunkSize: 1500, // larger chunks for code (functions can be long)
|
||
ChunkOverlap: 200,
|
||
Separators: []string{"\n\nfunc ", "\n\nclass ", "\n\ndef ", "\n\n", "\n", " "},
|
||
}
|
||
}
|
||
```
|
||
|
||
This is a simple heuristic — it splits on function/class boundaries when
|
||
possible, falling back to paragraph and line boundaries. It doesn't require
|
||
an AST parser and handles most languages well enough for search.
|
||
|
||
### Search Tool
|
||
|
||
**`workspace_search`** — semantic search across workspace files
|
||
|
||
```json
|
||
{
|
||
"query": "database connection pooling",
|
||
"top_k": 10,
|
||
"file_pattern": "*.go"
|
||
}
|
||
```
|
||
|
||
Returns: array of `{ file_path, chunk_content, score, line_hint }`.
|
||
|
||
The `file_pattern` is optional glob filtering (applied post-search on the
|
||
`workspace_files.path` join). `line_hint` is approximate line number from
|
||
chunk metadata.
|
||
|
||
### Workspace Store Extensions
|
||
|
||
```go
|
||
// Added to WorkspaceStore interface
|
||
InsertChunks(ctx context.Context, chunks []models.WorkspaceChunk) error
|
||
DeleteChunksByFile(ctx context.Context, fileID string) error
|
||
SimilaritySearch(ctx context.Context, workspaceID string, embedding []float64, topK int) ([]models.WorkspaceChunkResult, error)
|
||
UpdateFileIndexStatus(ctx context.Context, fileID, status string) error
|
||
```
|
||
|
||
### `workspace_files` Schema Addition
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| `index_status` | TEXT | `pending`, `indexing`, `ready`, `error`, `skipped` |
|
||
| `chunk_count` | INT | number of chunks after indexing |
|
||
|
||
### Indexing Configuration
|
||
|
||
- **Per-workspace toggle:** `workspaces.indexing_enabled` (default: `true`
|
||
when embedding role is configured, `false` otherwise).
|
||
- **System setting:** `WORKSPACE_INDEXING_ENABLED` env var (global kill switch).
|
||
- **Concurrency:** shared semaphore with KB ingestion (don't overwhelm the
|
||
embedding provider).
|
||
|
||
### Checklist
|
||
|
||
- [ ] `workspace_chunks` table (Postgres + SQLite migrations)
|
||
- [ ] `workspace_files.index_status` and `chunk_count` columns
|
||
- [ ] `WorkspaceChunk`, `WorkspaceChunkResult` model structs
|
||
- [ ] Store methods: `InsertChunks`, `DeleteChunksByFile`, `SimilaritySearch`, `UpdateFileIndexStatus`
|
||
- [ ] Postgres implementation with pgvector
|
||
- [ ] SQLite implementation with app-level cosine
|
||
- [ ] `workspace.Indexer` — reuses `knowledge.SplitText` + `knowledge.Embedder`
|
||
- [ ] Indexable type detection: MIME type + extension map
|
||
- [ ] Code-aware chunk config
|
||
- [ ] Content-addressed skip (sha256 check)
|
||
- [ ] Background indexing goroutine with semaphore
|
||
- [ ] `workspace_search` tool
|
||
- [ ] `workspace_search` glob filtering on file path
|
||
- [ ] Hook indexer into `WorkspaceFS.WriteFile` and `ExtractArchive`
|
||
- [ ] Workspace index status endpoint: `GET /api/v1/workspaces/:id/index-status`
|
||
- [ ] Integration tests: index pipeline, semantic search, content-addressed skip
|
||
- [ ] Notification on index complete/error (reuse notification service)
|
||
|
||
---
|
||
|
||
## v0.21.3 — Surface Infrastructure + REPL
|
||
|
||
Pure UI architecture. No workspace dependency. Can develop in parallel with
|
||
v0.21.0–v0.21.2 on the frontend side.
|
||
|
||
### Surface Registration API
|
||
|
||
Extends the existing `ctx.*` extension API from v0.11.0.
|
||
|
||
```js
|
||
Extensions.register({
|
||
id: 'my-mode',
|
||
init(ctx) {
|
||
ctx.surfaces.register('my-surface', {
|
||
label: 'My Mode',
|
||
icon: 'code', // lucide icon name
|
||
regions: ['surface-main', 'sidebar-content'],
|
||
activate() { /* take over regions */ },
|
||
deactivate() { /* restore regions */ }
|
||
});
|
||
}
|
||
});
|
||
```
|
||
|
||
### Region Management
|
||
|
||
```js
|
||
ctx.ui.replace(regionId, element) // saves current content, swaps in element
|
||
ctx.ui.restore(regionId) // restores saved content
|
||
```
|
||
|
||
Regions are DOM containers identified by `data-surface-region` attributes.
|
||
`replace()` detaches the current children (preserved in memory, not destroyed),
|
||
inserts the new element. `restore()` re-attaches the saved children.
|
||
|
||
This is critical for CM6 — editor instances preserve state across mode switches
|
||
because the DOM nodes aren't destroyed, just detached.
|
||
|
||
### Surface Regions
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────┐
|
||
│ [surface-header] │
|
||
├──────────────┬───────────────────────────────────┤
|
||
│ │ │
|
||
│ [sidebar- │ [surface-main] │
|
||
│ content] │ (chat messages / editor / │
|
||
│ │ article / custom) │
|
||
│ │ │
|
||
│ ├───────────────────────────────────┤
|
||
│ │ [surface-footer] │
|
||
│ │ (chat input / editor toolbar) │
|
||
└──────────────┴───────────────────────────────────┘
|
||
```
|
||
|
||
The sidebar-nav region (mode selector) is managed by the surface system itself,
|
||
not by individual extensions.
|
||
|
||
### Mode Selector
|
||
|
||
Appears in the sidebar below the brand logo, above chat history, when ≥1
|
||
extension surface is registered. Shows icon buttons for each mode. Chat is
|
||
always the first (default) mode.
|
||
|
||
```
|
||
[💬] [</>] [📄] ← mode selector (chat, editor, article)
|
||
──────────────
|
||
Chat History... ← sidebar-content (replaced per mode)
|
||
```
|
||
|
||
### Events
|
||
|
||
```js
|
||
Events.on('surface.activated', ({ surface, previous }) => { ... });
|
||
Events.on('surface.deactivated', ({ surface }) => { ... });
|
||
```
|
||
|
||
### REPL / Debug Console (#70)
|
||
|
||
Fourth tab in debug modal (Ctrl+Shift+L). See roadmap for full spec:
|
||
|
||
- `AsyncFunction` wrapper for top-level `await`
|
||
- Injected globals: `API`, `Events`, `Extensions`, `DebugLog`, `Surfaces`
|
||
- Pretty-print results (collapsible JSON, red errors with stack)
|
||
- Command history: ↑/↓ with `sessionStorage`
|
||
- Tab-completion on live object graphs
|
||
- Event label hints on `Events.publish('`/`Events.on('`
|
||
- Multi-line: Shift+Enter for newline, Enter to run
|
||
- Admin-gated OR `?debug=1` URL param
|
||
|
||
### Checklist
|
||
|
||
- [ ] `data-surface-region` attributes on index.html containers
|
||
- [ ] `surfaces.js` — SurfaceRegistry: register, activate, deactivate, getCurrent
|
||
- [ ] `ctx.ui.replace()` / `ctx.ui.restore()` with DOM preservation
|
||
- [ ] `ctx.surfaces.register()` API for extensions
|
||
- [ ] Mode selector component in sidebar
|
||
- [ ] Chat as default surface (implicit, always registered)
|
||
- [ ] `surface.activated` / `surface.deactivated` EventBus events
|
||
- [ ] REPL tab: AsyncFunction wrapper, global injection
|
||
- [ ] REPL tab: pretty-print results
|
||
- [ ] REPL tab: command history (sessionStorage)
|
||
- [ ] REPL tab: tab-completion on object graphs
|
||
- [ ] REPL tab: event label hints
|
||
- [ ] REPL tab: admin gate
|
||
- [ ] REPL tab: toolbar (clear, copy, help)
|
||
- [ ] Mobile: mode selector collapses to hamburger/bottom nav
|
||
- [ ] Integration with extension loader (surfaces from manifest)
|
||
- [ ] Documentation in EXTENSIONS.md §6 update
|
||
|
||
---
|
||
|
||
## v0.21.4 — Git Integration
|
||
|
||
Layer git operations onto workspace. A workspace can optionally track a git
|
||
remote. Credentials go through the vault.
|
||
|
||
### Schema Changes
|
||
|
||
**`workspaces` table additions:**
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| `git_remote_url` | TEXT | nullable — HTTPS or SSH URL |
|
||
| `git_branch` | TEXT | nullable — default branch after clone |
|
||
| `git_credential_id` | UUID | nullable — FK to git_credentials |
|
||
| `git_last_sync` | TIMESTAMPTZ | nullable — last pull/push |
|
||
|
||
**`git_credentials` table:**
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| `id` | UUID | PK |
|
||
| `user_id` | UUID | FK → users (owner) |
|
||
| `name` | TEXT | display name: "GitHub PAT", "GitLab token" |
|
||
| `auth_type` | TEXT | `https_pat`, `https_basic`, `ssh_key` |
|
||
| `encrypted_data` | BYTEA | AES-256-GCM (same vault pattern as BYOK) |
|
||
| `created_at` | TIMESTAMPTZ | |
|
||
|
||
Encrypted data contains: `{ "token": "..." }` for PAT, `{ "username": "...",
|
||
"password": "..." }` for basic, `{ "private_key": "...", "passphrase": "..." }`
|
||
for SSH.
|
||
|
||
### Git Operations (server-side)
|
||
|
||
Uses `os/exec` calling `git` binary (not a Go git library — keeps binary size
|
||
down and avoids CGO). The git binary is already in the Docker image for other
|
||
purposes.
|
||
|
||
```go
|
||
type GitOps struct {
|
||
fs *WorkspaceFS
|
||
credentials CredentialResolver // vault decryption
|
||
}
|
||
|
||
func (g *GitOps) Clone(ctx context.Context, w *models.Workspace, url, branch string, credID string) error
|
||
func (g *GitOps) Pull(ctx context.Context, w *models.Workspace) error
|
||
func (g *GitOps) Push(ctx context.Context, w *models.Workspace, message string) error
|
||
func (g *GitOps) Status(ctx context.Context, w *models.Workspace) (*GitStatus, error)
|
||
func (g *GitOps) Diff(ctx context.Context, w *models.Workspace, path string) (string, error)
|
||
func (g *GitOps) Commit(ctx context.Context, w *models.Workspace, message string, paths []string) error
|
||
func (g *GitOps) Log(ctx context.Context, w *models.Workspace, n int) ([]GitLogEntry, error)
|
||
func (g *GitOps) BranchList(ctx context.Context, w *models.Workspace) ([]string, string, error)
|
||
func (g *GitOps) Checkout(ctx context.Context, w *models.Workspace, branch string) error
|
||
```
|
||
|
||
Credential injection: temporary `.git-credentials` file or `GIT_ASKPASS` script
|
||
created for the duration of the operation, then securely wiped. SSH keys use
|
||
`GIT_SSH_COMMAND` with a temp key file.
|
||
|
||
### Git Tools (category: `workspace`)
|
||
|
||
**`git_status`** — show working tree status
|
||
**`git_diff`** — show diff for file or all changes
|
||
**`git_commit`** — stage and commit files
|
||
**`git_log`** — recent commit history
|
||
**`git_branch`** — list/switch branches
|
||
|
||
These are only injected when the workspace has `git_remote_url` set.
|
||
|
||
### API Endpoints
|
||
|
||
```
|
||
POST /api/v1/workspaces/:id/git/clone # { url, branch, credential_id }
|
||
POST /api/v1/workspaces/:id/git/pull
|
||
POST /api/v1/workspaces/:id/git/push
|
||
GET /api/v1/workspaces/:id/git/status
|
||
GET /api/v1/workspaces/:id/git/diff?path=
|
||
POST /api/v1/workspaces/:id/git/commit # { message, paths }
|
||
GET /api/v1/workspaces/:id/git/log?n=
|
||
GET /api/v1/workspaces/:id/git/branches
|
||
POST /api/v1/workspaces/:id/git/checkout # { branch }
|
||
|
||
# Credential management
|
||
POST /api/v1/git-credentials # create
|
||
GET /api/v1/git-credentials # list (user-scoped)
|
||
DELETE /api/v1/git-credentials/:id # delete
|
||
```
|
||
|
||
### Post-Operation Hooks
|
||
|
||
After `clone`, `pull`, `checkout`: trigger workspace reconcile (sync FS → DB
|
||
index) and re-index changed files (v0.21.2). This ensures the file tree and
|
||
search index stay current after git operations.
|
||
|
||
### Frontend
|
||
|
||
- Workspace settings: git remote URL, credential picker, clone button.
|
||
- Git credential management in user Settings.
|
||
- Future (editor surface): git status indicators in file tree, commit UI.
|
||
|
||
### Checklist
|
||
|
||
- [ ] `git_credentials` table (Postgres + SQLite migrations)
|
||
- [ ] `workspaces` git columns (migration)
|
||
- [ ] `GitCredentialStore` interface + both implementations
|
||
- [ ] Vault integration for credential encryption/decryption
|
||
- [ ] `workspace.GitOps` with exec-based git operations
|
||
- [ ] Credential injection: temp `.git-credentials` / `GIT_SSH_COMMAND`
|
||
- [ ] Clone with depth option (shallow clone for large repos)
|
||
- [ ] Pull, push, status, diff, commit, log, branch, checkout
|
||
- [ ] Post-operation reconcile + re-index hook
|
||
- [ ] Git tool implementations (5 tools)
|
||
- [ ] Git API endpoints (9 endpoints)
|
||
- [ ] Credential CRUD endpoints (3 endpoints)
|
||
- [ ] Workspace settings UI: git config section
|
||
- [ ] User Settings: git credentials management
|
||
- [ ] Integration tests: clone, commit, push/pull cycle
|
||
- [ ] `.gitignore` respect in workspace indexing (skip ignored files)
|
||
- [ ] Security: reject `file://` and local path URLs in clone
|
||
|
||
---
|
||
|
||
## v0.21.5 — Editor Surface (Development Mode)
|
||
|
||
The IDE-like experience. Built entirely as a browser-tier extension consuming
|
||
workspace primitives from v0.21.0–v0.21.4 and surface infrastructure from
|
||
v0.21.3.
|
||
|
||
### Layout
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────┐
|
||
│ [surface-header] workspace name | branch | status │
|
||
├──────────────┬───────────────────┬───────────────────┤
|
||
│ File Tree │ Code Editor │ AI Chat Panel │
|
||
│ (sidebar) │ (CM6, main) │ (resizable) │
|
||
│ │ │ │
|
||
│ 📁 src/ │ 1│ package main │ You: refactor │
|
||
│ 📄 main.go│ 2│ │ the handler... │
|
||
│ 📄 util.go│ 3│ func main() { │ │
|
||
│ 📁 tests/ │ 4│ ... │ AI: I'll update │
|
||
│ 📄 go.mod │ 5│ } │ main.go... │
|
||
│ │ │ │
|
||
├──────────────┴───────────────────┴───────────────────┤
|
||
│ [surface-footer] file: src/main.go | Ln 4, Col 12 │
|
||
└──────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### Components
|
||
|
||
**File Tree** (sidebar-content region)
|
||
- Tree view backed by `workspace_ls` API (or cached from `Tree()`)
|
||
- File icons by extension/MIME type
|
||
- Click to open in editor
|
||
- Right-click context menu: rename, delete, new file, new folder
|
||
- Git status indicators (if git-enabled): modified, staged, untracked
|
||
- Drag-drop file reorder (within folders)
|
||
|
||
**Code Editor** (surface-main region)
|
||
- CM6 `codeEditor()` factory (already exists from v0.17.2)
|
||
- Language auto-detection from file extension
|
||
- Tab bar for multiple open files
|
||
- Modified indicator (dot on tab)
|
||
- Save: Ctrl/Cmd+S → `workspace_write` API
|
||
- Auto-save on tab switch / mode switch (configurable)
|
||
|
||
**AI Chat Panel** (split pane, resizable)
|
||
- Reuses the existing chat rendering pipeline
|
||
- Same channel context — conversation history carries over between modes
|
||
- Workspace tools auto-injected
|
||
- Tool call results update the editor live (if the modified file is open)
|
||
|
||
**Status Bar** (surface-footer region)
|
||
- Current file path
|
||
- Cursor position (line, column)
|
||
- Language mode
|
||
- Encoding
|
||
- Git branch (if applicable)
|
||
|
||
### Extension Manifest
|
||
|
||
```json
|
||
{
|
||
"id": "editor-mode",
|
||
"name": "Development",
|
||
"version": "1.0.0",
|
||
"type": "browser",
|
||
"surfaces": [{
|
||
"id": "editor",
|
||
"label": "Editor",
|
||
"icon": "code-2",
|
||
"regions": ["surface-main", "surface-footer", "sidebar-content", "surface-header"]
|
||
}],
|
||
"permissions": ["workspace"]
|
||
}
|
||
```
|
||
|
||
### Live Update on Tool Calls
|
||
|
||
When the AI calls `workspace_write` or `workspace_patch` and the target file
|
||
is open in the editor, the editor content updates live. Implementation:
|
||
|
||
1. Tool call completes → workspace write handler fires EventBus event
|
||
`workspace.file.changed` with `{ workspaceId, path }`.
|
||
2. Editor surface subscribes to this event.
|
||
3. On match (file is open), fetch fresh content from API and update CM6
|
||
state via `view.dispatch({ changes })`.
|
||
|
||
### Checklist
|
||
|
||
- [ ] `editor-mode` extension directory structure
|
||
- [ ] File tree component (tree view, icons, context menu)
|
||
- [ ] Tab bar component (open files, modified indicator)
|
||
- [ ] Split pane layout (editor + chat, resizable divider)
|
||
- [ ] CM6 editor integration with language detection
|
||
- [ ] Save flow: Ctrl/Cmd+S → API → DB index update
|
||
- [ ] Auto-save on tab/mode switch
|
||
- [ ] Live update on `workspace.file.changed` event
|
||
- [ ] Status bar: file, cursor, language, branch
|
||
- [ ] Surface registration via manifest
|
||
- [ ] Git status indicators in file tree (if v0.21.4 landed)
|
||
- [ ] Mobile: single-pane mode (editor or chat, toggle)
|
||
- [ ] Keyboard shortcuts: Ctrl+P (quick open), Ctrl+Shift+F (workspace search)
|
||
- [ ] Quick open: fuzzy file finder using `workspace_files` index
|
||
|
||
---
|
||
|
||
## v0.21.6 — Article Surface + Document Output Pipeline
|
||
|
||
Writing-focused surface. The conversation is secondary — the document is the
|
||
artifact.
|
||
|
||
### Document Output Pipeline
|
||
|
||
Before article mode, we need a way for extensions to produce downloadable
|
||
output. This is the pipeline:
|
||
|
||
```
|
||
extension generates content
|
||
→ POST /api/v1/workspaces/:id/files/write (save to workspace)
|
||
→ or POST /api/v1/workspaces/:id/export (convert + save)
|
||
→ user downloads from workspace or gets direct download link
|
||
```
|
||
|
||
Export endpoint supports format conversion (markdown → HTML, markdown → PDF via
|
||
pandoc if available, markdown → DOCX). This is a stretch goal — initial version
|
||
just saves to workspace and lets users download the raw file.
|
||
|
||
### Layout
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────┐
|
||
│ [surface-header] Document Title [Export ▾] │
|
||
├──────────────┬───────────────────────────────────────┤
|
||
│ Outline │ Rich Text Editor (CM6 markdown) │
|
||
│ │ │
|
||
│ § Intro │ # My Article │
|
||
│ § Methods │ │
|
||
│ § Data │ Compelling introduction paragraph │
|
||
│ § Analysis│ about the topic at hand... │
|
||
│ § Results │ │
|
||
│ § Discussion│ ## Methods │
|
||
│ │ │
|
||
│──────────────│ ### Data Collection │
|
||
│ AI Assistant │ │
|
||
│ │ We gathered data from... │
|
||
│ "Expand the │ │
|
||
│ methods │ │
|
||
│ section..." │ │
|
||
├──────────────┴───────────────────────────────────────┤
|
||
│ [surface-footer] Words: 1,234 | Reading: ~5 min │
|
||
└──────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### Components
|
||
|
||
**Outline** (sidebar-content, top section)
|
||
- Auto-generated from heading structure in document
|
||
- Click to scroll to section
|
||
- Drag to reorder sections
|
||
|
||
**AI Assistant** (sidebar-content, bottom section)
|
||
- Lightweight chat interface for document-focused prompts
|
||
- Tools: `workspace_read`, `workspace_write`, `workspace_search`
|
||
- Additional article tools:
|
||
- `suggest_outline` — propose structure for a topic
|
||
- `expand_section` — flesh out a heading with content
|
||
- `check_citations` — verify URLs are reachable (via url_fetch)
|
||
|
||
**Rich Text Editor** (surface-main)
|
||
- CM6 `noteEditor()` factory (from v0.17.3) with enhanced live preview
|
||
- Heading decorations, blockquotes, code blocks, links
|
||
- Focus mode: dim everything except current section
|
||
|
||
**Export** (surface-header dropdown)
|
||
- Download as Markdown (.md)
|
||
- Download as HTML (rendered)
|
||
- Download as PDF (if pandoc available)
|
||
- Copy to clipboard (markdown)
|
||
|
||
### Checklist
|
||
|
||
- [ ] `article-mode` extension directory structure
|
||
- [ ] Outline component (auto-generated from headings, click-to-scroll)
|
||
- [ ] Sidebar split: outline (top) + AI assistant (bottom)
|
||
- [ ] Rich text editor with enhanced markdown live preview
|
||
- [ ] Export dropdown: markdown, HTML, PDF
|
||
- [ ] Article-specific tools: `suggest_outline`, `expand_section`
|
||
- [ ] Focus mode (dim non-active sections)
|
||
- [ ] Word count + reading time in status bar
|
||
- [ ] Surface registration via manifest
|
||
- [ ] Auto-save document to workspace
|
||
- [ ] Mobile: outline collapses to top dropdown
|
||
|
||
---
|
||
|
||
## Cross-Cutting Concerns
|
||
|
||
### Migration Strategy
|
||
|
||
Each sub-release gets its own migration file(s), both Postgres and SQLite.
|
||
Migrations run on backend startup (existing pattern). Numbering continues
|
||
from v0.20.0's last migration.
|
||
|
||
### Test Strategy
|
||
|
||
- Backend integration tests against both Postgres and SQLite for every store
|
||
method and handler (existing CI matrix pattern).
|
||
- Workspace FS tests: use a temp directory, not the real PVC.
|
||
- Tool execution tests: mock workspace with pre-populated files.
|
||
- Surface tests: manual testing + REPL verification (surface infrastructure
|
||
is frontend-only, no backend tests).
|
||
|
||
### Backward Compatibility
|
||
|
||
- Channels without workspaces work exactly as before.
|
||
- Projects without workspaces work exactly as before.
|
||
- Archive uploads to channels without workspaces go to attachment store.
|
||
- Workspace tools only appear when a workspace is bound.
|
||
|
||
### Documentation Updates
|
||
|
||
Each sub-release updates:
|
||
- `VERSION` → `0.21.N`
|
||
- `CHANGELOG.md` — detailed entry
|
||
- `ARCHITECTURE.md` — workspace section, surface system section
|
||
- `EXTENSIONS.md` — §6 update for surface implementation
|
||
- `ROADMAP.md` — mark completed items
|
||
|
||
### Deployment
|
||
|
||
- PVC: ensure `workspaces/` directory is created at startup
|
||
(same pattern as `attachments/` directory creation).
|
||
- Git binary: verify `git` is in the Docker image (add to Dockerfile
|
||
if not present).
|
||
- Embedding role: workspace indexing gracefully degrades when no embedding
|
||
model is configured (files still work, search doesn't).
|
||
- Quota defaults: `WORKSPACE_MAX_BYTES=524288000` (500MB),
|
||
`WORKSPACE_MAX_FILES=10000`.
|