# Design — v0.19.0 Projects / Workspaces **Version:** 0.19.0 **Status:** Draft **Depends on:** user groups + resource grants (v0.16.0), knowledge bases (v0.14.0), side panel (v0.18.1) --- ## 1. Overview Projects are organizational containers that group related conversations, knowledge bases, and notes into a shared workspace. They sit between "individual channel" and "team" in the hierarchy — a team might have many projects, a user might have personal projects across teams. Projects solve two concrete problems: 1. **Sidebar sprawl:** Power users accumulate dozens of channels. The flat (or single-folder) chat list doesn't scale. Projects give channels meaningful grouping with shared context. 2. **Shared context:** Today, attaching a KB or Persona to a channel is per-channel. Projects let you say "every conversation in this project uses these KBs and this Persona" — one config, many channels. **What this release does NOT include:** Notifications are deferred to v0.19.1 (see §12 for the split rationale and forward pointers). --- ## 2. Scope Model Projects reuse the existing three-value scope model: ``` ┌─────────────────────────────────────────────────┐ │ personal scope │ │ owner_id = user_id, team_id = NULL │ │ Only visible to the owning user │ │ Use case: personal workspace, side projects │ └─────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────┐ │ team scope │ │ owner_id = creator, team_id = team_id │ │ Visible to team members (respects grants) │ │ Use case: department projects, client work │ └─────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────┐ │ global scope │ │ owner_id = admin, team_id = NULL │ │ Visible to all users (admin-created) │ │ Use case: company-wide initiatives, templates │ └─────────────────────────────────────────────────┘ ``` **Access control:** Projects use the same grant model as Personas and KBs (v0.16.0): `team_only`, `global`, or `groups`. The `resource_grants` table gets `'project'` added to its `resource_type` CHECK constraint. --- ## 3. Data Model ### 3.1 Table: `projects` | Column | Type | Description | |--------|------|-------------| | `id` | UUID / TEXT | Primary key | | `name` | TEXT NOT NULL | Display name | | `description` | TEXT DEFAULT '' | Optional description | | `scope` | TEXT NOT NULL | `personal`, `team`, `global` | | `owner_id` | UUID / TEXT | Creating user | | `team_id` | UUID / TEXT | NULL for personal/global | | `persona_id` | UUID / TEXT | Default Persona for new channels (nullable) | | `settings` | JSONB / JSON | Project-level config (see §3.7) | | `is_archived` | BOOLEAN DEFAULT false | Soft archive | | `created_at` | TIMESTAMPTZ / TEXT | | | `updated_at` | TIMESTAMPTZ / TEXT | | **Constraints:** ```sql CONSTRAINT project_scope_check CHECK ( (scope = 'personal' AND team_id IS NULL) OR (scope = 'team' AND team_id IS NOT NULL) OR (scope = 'global' AND team_id IS NULL) ) ``` **Indexes:** ```sql CREATE INDEX idx_projects_owner ON projects(owner_id); CREATE INDEX idx_projects_team ON projects(team_id) WHERE team_id IS NOT NULL; CREATE INDEX idx_projects_scope ON projects(scope); ``` ### 3.2 Junction Tables Separate junction tables per resource type — not a polymorphic `project_resources` table. Rationale in §3.3. **`project_channels`** | Column | Type | Description | |--------|------|-------------| | `project_id` | UUID / TEXT | FK → projects ON DELETE CASCADE | | `channel_id` | UUID / TEXT | UNIQUE, FK → channels ON DELETE CASCADE | | `position` | INT DEFAULT 0 | Sort order within project | | `folder_path` | TEXT DEFAULT '/' | Sub-folder within project | | `added_at` | TIMESTAMPTZ / TEXT | | ```sql PRIMARY KEY (project_id, channel_id) UNIQUE (channel_id) -- one project per channel (see §16 Q2) CREATE INDEX idx_project_channels_channel ON project_channels(channel_id); ``` **`project_knowledge_bases`** | Column | Type | Description | |--------|------|-------------| | `project_id` | UUID / TEXT | FK → projects ON DELETE CASCADE | | `kb_id` | UUID / TEXT | FK → knowledge_bases ON DELETE CASCADE | | `auto_search` | BOOLEAN DEFAULT true | Include in all project channels | | `added_at` | TIMESTAMPTZ / TEXT | | ```sql PRIMARY KEY (project_id, kb_id) ``` **`project_notes`** | Column | Type | Description | |--------|------|-------------| | `project_id` | UUID / TEXT | FK → projects ON DELETE CASCADE | | `note_id` | UUID / TEXT | FK → notes ON DELETE CASCADE | | `added_at` | TIMESTAMPTZ / TEXT | | ```sql PRIMARY KEY (project_id, note_id) ``` ### 3.3 Why Three Tables, Not One The roadmap spec proposed a single `project_resources` table with `(resource_type, resource_id)`. Three tables are better here: - **Referential integrity.** FK constraints enforce that resources exist. `ON DELETE CASCADE` handles cleanup when a channel, KB, or note is deleted — no application-level orphan scanning. - **Existing pattern.** `channel_knowledge_bases`, `persona_knowledge_bases`, and `group_members` all use dedicated junction tables. - **Type-specific columns.** `project_channels` needs `position` and `folder_path`. `project_knowledge_bases` needs `auto_search`. A polymorphic table would need nullable type-specific columns or a JSONB bag, which is worse. - **SQLite compatibility.** Simpler constraints, no CHECK on polymorphic type strings, no mixed-resource indexes. ### 3.4 Channel Integration Channels get a denormalized `project_id` for fast sidebar queries. The junction table `project_channels` is the source of truth for position and folder_path. Both are kept in sync by the store layer. ```sql ALTER TABLE channels ADD COLUMN project_id UUID REFERENCES projects(id) ON DELETE SET NULL; CREATE INDEX idx_channels_project ON channels(project_id) WHERE project_id IS NOT NULL; ``` **On project delete:** Channels detach (`project_id → NULL`). They are never cascade-deleted — only the junction row is removed. KBs and notes similarly survive; only the association is dropped. ### 3.5 Folder/Column Cleanup The channels table currently has two unused grouping columns: ```sql folder_id UUID -- FK placeholder, never wired to a folders table folder TEXT -- frontend-managed text label, barely used ``` With projects providing real hierarchy: - `folder_id` — **Dropped** in this migration (confirmed unused in store layer and all handlers). - `folder` — **Deprecated.** Column remains for backward compat but is no longer read or written by any handler after v0.19.0. Migration best-effort copies non-empty `folder` values into personal projects (one project per distinct folder name, channels attached). Can be dropped in a future schema consolidation. ### 3.6 Model Structs ```go type Project struct { BaseModel Name string `json:"name" db:"name"` Description string `json:"description" db:"description"` Scope string `json:"scope" db:"scope"` OwnerID string `json:"owner_id" db:"owner_id"` TeamID *string `json:"team_id,omitempty" db:"team_id"` PersonaID *string `json:"persona_id,omitempty" db:"persona_id"` Settings JSONMap `json:"settings,omitempty" db:"settings"` IsArchived bool `json:"is_archived" db:"is_archived"` // Computed (from COUNT joins, not stored) ChannelCount int `json:"channel_count,omitempty"` KBCount int `json:"kb_count,omitempty"` NoteCount int `json:"note_count,omitempty"` } type ProjectPatch struct { Name *string `json:"name,omitempty"` Description *string `json:"description,omitempty"` PersonaID *string `json:"persona_id,omitempty"` Settings JSONMap `json:"settings,omitempty"` IsArchived *bool `json:"is_archived,omitempty"` } type ProjectChannel struct { ProjectID string `json:"project_id" db:"project_id"` ChannelID string `json:"channel_id" db:"channel_id"` Position int `json:"position" db:"position"` FolderPath string `json:"folder_path" db:"folder_path"` } ``` ### 3.7 Project Settings (JSONB) The `settings` column stores project-level configuration as JSON. Initial keys: ```json { "auto_persona": true, "auto_kbs": true, "default_model": null, "description_visible": true } ``` | Key | Type | Description | |-----|------|-------------| | `auto_persona` | bool | Apply project Persona to new channels automatically | | `auto_kbs` | bool | Inject project KBs into channels at completion time | | `default_model` | string | Override model for new channels (nullable) | | `description_visible` | bool | Show description in sidebar project header | Settings are intentionally sparse. Resist the urge to add per-project copies of global settings — use project Persona config for model behavior and project KBs for context. --- ## 4. Store Interface ```go type ProjectStore interface { // CRUD Create(ctx context.Context, p *models.Project) error GetByID(ctx context.Context, id string) (*models.Project, error) Update(ctx context.Context, id string, patch models.ProjectPatch) error Delete(ctx context.Context, id string) error // Listing (scope-aware) ListForUser(ctx context.Context, userID string, opts store.ListOptions) ([]models.Project, int, error) ListForTeam(ctx context.Context, teamID string, opts store.ListOptions) ([]models.Project, int, error) ListAccessible(ctx context.Context, userID string) ([]models.Project, error) // Resource association AddChannel(ctx context.Context, projectID, channelID string) error RemoveChannel(ctx context.Context, projectID, channelID string) error ListChannels(ctx context.Context, projectID string) ([]models.ProjectChannel, error) MoveChannel(ctx context.Context, projectID, channelID string, position int, folderPath string) error AddKB(ctx context.Context, projectID, kbID string, autoSearch bool) error RemoveKB(ctx context.Context, projectID, kbID string) error ListKBs(ctx context.Context, projectID string) ([]models.KnowledgeBase, error) AddNote(ctx context.Context, projectID, noteID string) error RemoveNote(ctx context.Context, projectID, noteID string) error ListNotes(ctx context.Context, projectID string) ([]models.Note, error) // KB resolution (for completion-time injection) GetProjectKBsForChannel(ctx context.Context, channelID string) ([]models.KnowledgeBase, error) // Access check UserCanAccess(ctx context.Context, userID, projectID string) (bool, error) } ``` **Key design decisions:** - `AddChannel` writes both `project_channels` junction row and `channels.project_id` denorm column in a single transaction. If the channel already belongs to a different project, AddChannel removes the old association first (atomic move). This enables drag-between-projects and "Move to project" context menu. - `RemoveChannel` clears `channels.project_id` and deletes the junction row. - `GetProjectKBsForChannel` joins `channels → project_channels → project_knowledge_bases → knowledge_bases` — used at completion time for virtual KB injection (see §6). - `ListAccessible` uses the same grant resolution pattern as `PersonaStore.ListAccessible`: union of personal + team + granted. - Both Postgres and SQLite implementations required (same pattern as all other stores). --- ## 5. API Endpoints All endpoints require authentication. Team/global project mutations require team_admin or admin role respectively. ### 5.1 Projects CRUD ``` POST /api/v1/projects Create project GET /api/v1/projects List accessible projects GET /api/v1/projects/:id Get project (with resource counts) PUT /api/v1/projects/:id Update project DELETE /api/v1/projects/:id Delete project (detaches resources) ``` **Create request:** ```json { "name": "Q3 Launch", "description": "Product launch planning", "scope": "team", "team_id": "uuid", "persona_id": "uuid-or-null", "settings": {} } ``` **List response:** ```json { "data": [ { "id": "uuid", "name": "Q3 Launch", "scope": "team", "channel_count": 5, "kb_count": 2, "note_count": 12, "persona_id": "uuid", "updated_at": "2026-02-28T..." } ], "total": 3, "page": 1, "per_page": 50 } ``` ### 5.2 Resource Association ``` POST /api/v1/projects/:id/channels Add channel to project DELETE /api/v1/projects/:id/channels/:cid Remove channel from project GET /api/v1/projects/:id/channels List project channels PUT /api/v1/projects/:id/channels/:cid Move/reorder channel POST /api/v1/projects/:id/knowledge-bases Add KB DELETE /api/v1/projects/:id/knowledge-bases/:kid Remove KB GET /api/v1/projects/:id/knowledge-bases List project KBs POST /api/v1/projects/:id/notes Add note DELETE /api/v1/projects/:id/notes/:nid Remove note GET /api/v1/projects/:id/notes List project notes ``` **Add channel request:** ```json { "channel_id": "uuid", "folder_path": "/design", "position": 0 } ``` **Move channel request:** ```json { "position": 2, "folder_path": "/design/iterations" } ``` ### 5.3 Project Grants Reuses the existing resource grants API pattern: ``` GET /api/v1/projects/:id/grants Get project grant PUT /api/v1/projects/:id/grants Set project grant ``` **Request/response uses existing grant model:** ```json { "grant_scope": "groups", "granted_groups": ["uuid-group-1", "uuid-group-2"] } ``` **Migration:** Add `'project'` to `resource_grants.resource_type` CHECK: ```sql ALTER TABLE resource_grants DROP CONSTRAINT resource_grants_resource_type_check; ALTER TABLE resource_grants ADD CONSTRAINT resource_grants_resource_type_check CHECK (resource_type IN ('persona', 'knowledge_base', 'project')); ``` ### 5.4 Channel List Enhancement The existing `GET /api/v1/channels` endpoint gains project filtering: ``` GET /api/v1/channels?project_id=uuid Channels in a project GET /api/v1/channels?project_id=none Unassigned channels only ``` **Technical debt addressed here:** The `ListChannels` handler currently does raw SQL against `database.DB` instead of using `ChannelStore`. This gets migrated to the store layer as part of this work. The handler is the only one still bypassing the store — fixing it improves both maintainability and SQLite compat testing. --- ## 6. KB Injection at Completion Time Project KBs use **virtual injection** at completion time — the same pattern as Persona-KB binding (v0.17.0). No physical rows are created in `channel_knowledge_bases`. **Priority chain for KB resolution:** ``` channel KBs (explicit per-channel) → highest priority ↓ project KBs (auto_search = true) → inherited from project ↓ persona KBs (persona_knowledge_bases) → inherited from Persona ``` At completion time, `loadConversation()` already calls `buildKBContext()` which resolves channel + persona KBs. This extends to include project KBs: ```go // In completion handler, after resolving channel func resolveKBs(ctx context.Context, stores store.Stores, channelID string, personaID *string) []string { var kbIDs []string // 1. Channel-level (explicit) channelKBs, _ := stores.KnowledgeBases.ListForChannel(ctx, channelID) kbIDs = append(kbIDs, extractIDs(channelKBs)...) // 2. Project-level (virtual injection) projectKBs, _ := stores.Projects.GetProjectKBsForChannel(ctx, channelID) kbIDs = append(kbIDs, extractIDs(projectKBs)...) // 3. Persona-level if personaID != nil { personaKBs, _ := stores.KnowledgeBases.ListForPersona(ctx, *personaID) kbIDs = append(kbIDs, extractIDs(personaKBs)...) } return deduplicate(kbIDs) } ``` **Why virtual, not physical:** Adding a KB to a project should immediately apply to all project channels without backfilling junction rows. Removing a KB should immediately stop applying. The Persona-KB pattern proved this works — query-time resolution is simple and the JOIN cost is negligible for the cardinality involved. --- ## 7. Sidebar UI ### 7.1 Layout The sidebar transforms from a flat chat list to a grouped view: ``` ┌──────────────────────────────┐ │ 🔀 Chat Switchboard │ │ │ │ 🔍 Search... │ │ │ │ [+ New Chat] │ │ │ │ 📌 Pinned │ │ Chat about deployment │ │ Weekly standup notes │ │ │ │ ▼ Q3 Launch (team) ⚙ │ │ ├── /design │ │ │ Chat: mockup review │ │ │ Chat: color palette │ │ ├── /copy │ │ │ Chat: landing page │ │ └── Chat: kickoff │ │ │ │ ▼ Side Projects (personal) ⚙ │ │ Chat: rust experiments │ │ Chat: garden planner │ │ │ │ ▶ Archived Project ⚙ │ │ │ │ ── Recent ── │ │ Chat: quick question │ │ Chat: random thought │ │ │ │ [+ New Project] │ └──────────────────────────────┘ ``` **Sections (top to bottom):** 1. **Pinned** — pinned channels, regardless of project assignment (existing behavior, unchanged) 2. **Projects** — collapsible groups, sorted by `updated_at` desc. Each project shows its channels organized by `folder_path`. 3. **Recent** — unassigned channels (`project_id IS NULL`, `is_pinned = false`), sorted by `updated_at` desc. This is the existing chat list behavior for channels not yet organized. ### 7.2 Interactions | Action | Behavior | |--------|----------| | Click project header | Toggle expand/collapse (state persisted) | | Click ⚙ gear icon | Open project settings in side panel | | Drag channel → project | `POST /projects/:id/channels` | | Drag channel → "Recent" | `DELETE /projects/:id/channels/:cid` | | Drag channel within project | `PUT /projects/:id/channels/:cid` (reorder) | | Right-click project | Context menu: rename, archive, delete, manage KBs | | `[+ New Project]` | Inline creation: name input → scope picker | | Double-click project name | Inline rename (same UX as chat rename from v0.17.0) | ### 7.3 State Persistence ```js // sessionStorage key: 'cs-project-collapse' // Value: JSON object { [projectId]: boolean } // true = collapsed, absent/false = expanded ``` ### 7.4 Data Loading The sidebar needs all projects + their channels in one round trip to avoid N+1 rendering. Two options: **Option A — Two calls (preferred):** ``` GET /api/v1/projects → project list with counts GET /api/v1/channels?per_page=200 → all channels (project_id included) ``` Client-side grouping by `project_id`. This reuses existing channel loading and adds one extra call. Channel data is already loaded at startup in `loadChats()`. **Option B — Single enriched call:** ``` GET /api/v1/projects?include=channels ``` Returns projects with embedded channel arrays. More efficient but requires a new response shape and duplicates channel data if the flat list is also needed. Option A wins on KISS. The project list is small (rarely > 20), the channel list is already loaded, and grouping is trivial in JS. ### 7.5 Mobile On viewports ≤ 768px: - Projects render as collapsible accordions (same as desktop) - Drag-and-drop disabled — use context menu → "Move to project" instead - Project gear icon opens a bottom sheet instead of side panel - Folder paths shown as flat prefixes (no tree indentation) --- ## 8. Project Settings Panel Project settings open in the side panel (v0.18.1 PanelRegistry). Registers as `'project-settings'` panel. **Sections:** 1. **General** — name, description, scope badge (read-only for non-admin), archive toggle 2. **Default Persona** — persona picker dropdown (same component as channel persona picker from `persona-kb.js`) 3. **Knowledge Bases** — checkbox list of accessible KBs with `auto_search` toggle per KB 4. **Access** — grant picker (team_only / global / groups), reuses the same grant picker UI from Persona settings 5. **Resources** — summary counts (N channels, N KBs, N notes) 6. **Danger Zone** — delete project (confirmation dialog: "Resources will be detached, not deleted") --- ## 9. Channel Creation Within Projects When creating a new channel from within a project context (clicking "New Chat" while a project is expanded, or via project context menu): 1. Channel created via existing `POST /api/v1/channels` 2. Immediately associated via `POST /projects/:id/channels` 3. If project has `auto_persona: true` and `persona_id` set, the channel inherits the project's default Persona 4. Project KBs automatically apply at completion time via virtual injection (§6) — no explicit channel-KB linking needed The frontend tracks an "active project context" so new channels created while browsing a project are auto-associated. Context clears when the user clicks outside any project or into "Recent." ```js // In app state App.activeProjectId = null; // set when user interacts with a project ``` --- ## 10. Enterprise Use Cases ### 10.1 Project Templates (Phase 3 stretch) Team admins create projects pre-configured with Personas, KBs, and settings. Templates are projects with `settings.is_template: true`. Creating "from template" copies configuration into a new project. Not in initial delivery — data model supports it without changes. ### 10.2 Cross-Team Projects Global-scope projects with group grants enable cross-team work. Members of granted groups see the project and its channels. Uses existing grant resolution from v0.16.0 — no new access control code. ### 10.3 Admin Oversight Global admins can list all projects via admin panel. New "Projects" section under the "People" category (alongside Teams and Groups). Table view with scope, owner, team, resource counts, last updated. --- ## 11. Implementation Phases ### Phase 1 — Data Model + Store + API Backend foundation. All store operations, migrations, and API endpoints. **New files:** - `server/database/migrations/006_v0190_projects.sql` (Postgres) - `server/database/migrations/sqlite/006_v0190_projects.sql` (SQLite) - `server/store/postgres/project.go` (ProjectStore implementation) - `server/store/sqlite/project.go` (ProjectStore implementation) - `server/handlers/projects.go` (CRUD + resource association + grants) **Modified files:** - `server/models/models.go` — add Project, ProjectPatch, ProjectChannel structs - `server/store/interfaces.go` — add ProjectStore interface, Projects field to Stores - `server/store/postgres/stores.go` — wire ProjectStore in NewStores() - `server/store/sqlite/stores.go` — wire ProjectStore in NewStores() - `server/main.go` — wire project routes, inject store - `server/handlers/completion.go` — extend KB resolution chain (§6) - `server/handlers/channels.go` — migrate ListChannels to store layer, add `project_id` filter parameter **Technical debt resolved:** - `ListChannels` handler: raw SQL → ChannelStore (only remaining handler bypassing the store layer) - `folder_id` column dropped (unused FK placeholder) ### Phase 2 — Sidebar UI Frontend project grouping, collapse/expand, drag-and-drop. **New files:** - `src/js/projects.js` — ProjectManager: CRUD calls, sidebar rendering, drag handlers, active project context, collapse state **Modified files:** - `src/js/ui-core.js` — `renderChatList()` → delegates to ProjectManager for grouped rendering, falls back to flat list when no projects exist - `src/js/chat.js` — `loadChats()` adds project fetch, channel objects include `project_id` - `src/js/api.js` — project API methods (CRUD, resource association) - `src/js/app.js` — `App.activeProjectId`, keyboard shortcuts, project context for new channel creation - `src/css/styles.css` — project group styles, folder indent, drag target highlights, collapse animations - `src/index.html` — sidebar structure updates, "New Project" button ### Phase 3 — Settings Panel + Admin Side panel project settings, admin panel integration. **New files:** - `src/js/project-settings.js` — PanelRegistry registration, settings form, KB picker, grant picker, delete flow **Modified files:** - `src/js/panels.js` — project-settings panel registration - `src/js/ui-admin.js` — admin "Projects" section (table + CRUD) - `src/js/persona-kb.js` — extract KB picker as reusable component for both Persona and Project settings - `src/css/styles.css` — settings panel styles --- ## 12. Deferred: Notifications (→ v0.19.1, v0.19.2) Notifications are split out because: 1. **Independent value.** Projects are useful without notifications. Notifications are useful without projects. Neither blocks the other. 2. **Scope control.** Notifications span persistence, real-time WebSocket delivery, email transport with SMTP, HTML templates, digest batching, and per-user preferences. That's a standalone feature set with its own store, handler, and UI. 3. **Email is ops-heavy.** SMTP configuration, template rendering, and digest scheduling affect deployment requirements. Air-gapped envs won't have SMTP. Separating lets us ship in-app notifications first. **v0.19.1 — In-App Notifications:** - `notifications` table + NotificationStore (both dialects) - In-app notification bell (header bar, unread count badge) - Notification dropdown: grouped by type, mark read, click-to-navigate - WebSocket push via EventBus (`notification.new` → `DirToClient` in routeTable) - Sources: grant changes, KB processing complete, project invites, memory review queue (completes v0.18.0 admin workflow), team invites **v0.19.2 — Email Transport (tentative):** - SMTP config in admin settings (host, port, from, TLS) - HTML + plaintext templates, branded with instance name - Per-user preferences: per-type toggles (in-app / email / off) - Digest mode: batch low-priority notifications (hourly/daily) - Admin enable/disable email globally or per-team --- ## 13. Forward Compatibility ### 13.1 Multi-Participant Channels (v0.23.0) Projects don't assume single-owner channels. A project channel that later becomes a group channel (with `channel_participants`) works unchanged — `project_channels` joins on `channel_id`, not `user_id`. ### 13.2 Workflow Engine (v0.25.0) Workflow channels (`type='workflow'`) can belong to projects. A project could group all workflow instances for a process. The `project_id` on channels is type-agnostic. ### 13.3 Project-Scoped Memory (future) Not implemented, but the data model supports it. If a project has a default Persona, memories from project channels naturally feed into persona-scoped memory via existing v0.18.0 mechanics. No schema changes. ### 13.4 Extension Surfaces (v0.21.0) The project settings panel uses PanelRegistry (v0.18.1). When extension surfaces land, a project could define which surfaces/modes are available to its channels via `settings` JSON extension — no schema change. --- ## 14. Migration Safety ### 14.1 Postgres Migration `006_v0190_projects.sql` is additive: - New tables: `projects`, `project_channels`, `project_knowledge_bases`, `project_notes` - New column: `channels.project_id` (nullable FK, ON DELETE SET NULL) - Altered CHECK: `resource_grants.resource_type` adds `'project'` - Dropped column: `channels.folder_id` (confirmed unused) All operations are backward-compatible. Existing channels work without project assignment. The migration is safe to apply to live databases (no table locks beyond the brief ALTER TABLE). ### 14.2 SQLite `sqlite/006_v0190_projects.sql` with standard dialect adaptations: - `UUID` → `TEXT` - `TIMESTAMPTZ` → `TEXT` - `JSONB` → `TEXT` (JSON) - `gen_random_uuid()` → app-side `store.NewID()` - `ALTER TABLE channels ADD COLUMN project_id TEXT` (no FK enforcement in SQLite, but the store layer handles referential integrity) Note: `ALTER TABLE DROP COLUMN` requires SQLite ≥ 3.35.0. Ubuntu 24 ships 3.45, so `folder_id` drop is safe. Verify minimum SQLite version in CI. ### 14.3 Rollback All new tables can be dropped without affecting existing data. The `channels.project_id` column can be dropped (nullable, no data loss). The `resource_grants` CHECK can be reverted. No destructive operations on existing tables. ### 14.4 Folder Migration (best-effort) For users with existing `channels.folder` values, the migration creates personal projects from distinct folder names: ```sql -- Pseudo-SQL (actual implementation in Go for SQLite compat) -- For each distinct (user_id, folder) pair where folder is non-empty: -- 1. Create a personal project with name = folder -- 2. Insert project_channels row -- 3. Set channels.project_id ``` Implementation is in the Go migration runner (not raw SQL) because SQLite needs `store.NewID()` for UUID generation. If no `folder` values exist, nothing happens. This is a convenience migration, not a correctness requirement — users can reorganize manually. --- ## 15. Test Plan ### 15.1 Store Integration Tests (both dialects) - Project CRUD: create, read, update, delete, list - Scope enforcement: personal projects invisible to other users, team projects visible to team members, global visible to all - Resource association: add/remove channels, KBs, notes - Cascade: project delete detaches channels (project_id → NULL) - Cascade: channel delete removes junction row, project unaffected - Cascade: KB delete removes junction row, project unaffected - `GetProjectKBsForChannel`: correct KBs through the full join chain (channel → project_channel → project_kb → kb) - `GetProjectKBsForChannel`: returns empty for unassigned channels - Grant resolution: `UserCanAccess` for team_only, global, group grants - `ListAccessible`: union of personal + team + granted - One-project-per-channel: UNIQUE constraint on `project_channels.channel_id` rejects duplicate assignment - `MoveChannel`: position and folder_path update correctly ### 15.2 Handler Integration Tests - CRUD with auth: admin creates global, team_admin creates team, user creates personal, unauthorized users rejected - Grant management: set/get grants, verify access changes - Channel filter: `?project_id=uuid` returns only project channels, `?project_id=none` returns only unassigned - KB injection: completion with project KBs includes them in search context alongside channel and persona KBs ### 15.3 Frontend Tests (manual + smoke) - Sidebar renders project groups with correct channels - Collapse/expand persists across page refresh - Drag channel into project updates sidebar and API - Drag channel between projects moves cleanly - Drag channel to "Recent" removes from project - New channel within project context auto-associates - Project settings panel opens with correct data - Project delete detaches channels (they appear in "Recent") ### 15.4 Unhappy Path / Defensive Tests - Corrupt JSON in `channels.settings` → `scanJSON` defaults to `{}`, warning logged, channel list still returns valid response - Corrupt JSON in `channels.tags` → `scanTags` defaults to `[]`, warning logged - `SafeJSON` catches marshal failure → returns 500, not truncated 200 - Startup integrity check detects corrupt rows and logs warnings - Channel moved between projects: old junction row removed, new one created, `project_id` denorm updated — all atomic - Project delete with channels: channels detach cleanly, appear in "Recent", no orphaned junction rows - KB removed from project: immediately stops appearing in completion context for project channels (virtual injection, no stale rows) - Non-existent project_id filter on channels → empty result, not error - Adding channel to project when channel already in another project → atomic move, not duplicate --- ## 16. Resolved Questions 1. **Should "New Chat" always prompt for project?** **No.** New chats go to "Recent" by default. Users drag them into projects when worth keeping. Channels created from within a project context (expanded project group) auto-associate. Low-friction quick-chat UX preserved. 2. **Channel in multiple projects?** **One project per channel.** Enforced by UNIQUE on `project_channels.channel_id`. Moving a channel between projects must be easy — the UI should support drag-to-different-project and a "Move to project" context menu action. The `AddChannel` store method handles the move atomically: remove from old project (if any), add to new, update denorm column. 3. **Project-level search?** **Deferred to v0.19.1 or v0.19.2.** The junction tables make this straightforward — `WHERE channel_id IN (SELECT channel_id FROM project_channels WHERE project_id = $1)`. Not MVP but real utility for power users with many project channels. 4. **Note auto-association?** **Yes.** When a note is created from a message in a project channel, auto-associate it with the project via `project_notes`. The provenance chain (channel → project → note) is natural and expected. Implemented in the note creation handler by checking `channel.project_id` and inserting the junction row. --- ## 17. Bugfixes Bundled with v0.19.0 ### 17.0 Defensive Coding: Unhappy Path Philosophy Chat Switchboard needs to be resilient to dirty data, especially post-upgrade. Users (and developers) don't always back up before upgrading. Migrations can leave stale or corrupt data. External inputs (API keys, provider responses, pasted content) can inject unexpected bytes. The codebase should **warn and continue** rather than crash or produce truncated responses. **Principles:** - **Validate on read, not just write.** Even if the write path ensures valid JSON, the read path must handle the case where it isn't. Migrations, manual DB edits, encoding bugs, and version skew can all produce invalid data. - **Degrade gracefully.** A corrupt `settings` column on one channel should not prevent loading all channels. Default to `{}` or `[]`, log a warning with the row ID, and continue. - **Never send 200 with a broken body.** Pre-validate serialization for endpoints that touch user-controlled JSON columns. If marshaling fails, return 500 with a useful error, not a truncated stream. - **Log actionable context.** Warnings should include: table, column, row ID, and the nature of the corruption. This lets admins find and fix the specific row without guessing. - **Apply everywhere JSON columns are scanned.** The `scanJSON` and `scanTags` helpers are used across channels, settings, personas, and knowledge bases. The fix must be in the shared helpers, not patched per-handler. ### 17.1 Channels endpoint returns truncated JSON (P0) **Root cause found:** A channel row has a corrupted `settings` column containing a `\x02` (STX control byte) instead of valid JSON. The `scanJSON` helper copies raw bytes without validation — the scan succeeds. Then `c.JSON(200, ...)` starts writing the response (status + headers flushed), the JSON encoder hits the corrupt `RawMessage`, `MarshalJSON()` validates the content, finds `\x02`, aborts mid-stream. Truncated body. Browser gets "Unexpected end of JSON input". Server log confirmation (fires on every channels request): ``` Error #01: json: error calling MarshalJSON for type json.RawMessage: invalid character '\x02' looking for beginning of value ``` **Fix (three layers):** **Layer 1 — Harden `scanJSON` (shared helper):** ```go func (s *jsonScanner) Scan(src interface{}) error { switch v := src.(type) { case []byte: if !json.Valid(v) { log.Printf("⚠ scanJSON: invalid JSON in column (len=%d, preview=%.40q), defaulting to {}", len(v), v) *s.dest = json.RawMessage("{}") return nil } *s.dest = json.RawMessage(v) case string: b := []byte(v) if !json.Valid(b) { log.Printf("⚠ scanJSON: invalid JSON in column (len=%d, preview=%.40q), defaulting to {}", len(b), b) *s.dest = json.RawMessage("{}") return nil } *s.dest = json.RawMessage(b) case nil: *s.dest = json.RawMessage("{}") default: *s.dest = json.RawMessage("{}") } return nil } ``` This ensures no corrupt data reaches the JSON encoder. The warning log includes a truncated preview to help identify the source without dumping the full corrupt blob. **Layer 2 — Same treatment for `scanTags`:** Apply identical validation to the tags scanner. If the stored value isn't valid JSON array, default to `[]` and warn. **Layer 3 — Pre-marshal on list endpoints:** For endpoints that return arrays of objects with `json.RawMessage` fields, pre-marshal the response to catch errors before headers are sent: ```go // Helper: SafeJSON writes status + body only after successful marshal func SafeJSON(c *gin.Context, code int, obj interface{}) { body, err := json.Marshal(obj) if err != nil { log.Printf("⚠ SafeJSON: marshal failed: %v", err) c.JSON(http.StatusInternalServerError, gin.H{ "error": "response serialization failed", }) return } c.Data(code, "application/json; charset=utf-8", body) } ``` Apply to: `ListChannels`, `GetChannel`, `ListPersonas`, and any other handler returning models with JSONB fields. **Layer 4 — Startup integrity check (warn-only):** On application boot, after migrations complete, run a lightweight scan for corrupt JSON columns: ```go func checkJSONIntegrity(db *sql.DB) { tables := []struct{ table, column string }{ {"channels", "settings"}, {"channels", "tags"}, {"personas", "settings"}, {"knowledge_bases", "metadata"}, {"users", "settings"}, } for _, t := range tables { // Postgres: check for non-JSON values query := fmt.Sprintf( `SELECT id FROM %s WHERE %s IS NOT NULL AND %s::text != '' AND %s::text NOT SIMILAR TO '[{"[\s]%%'`, t.table, t.column, t.column, t.column) rows, err := db.Query(query) if err != nil { continue } var ids []string for rows.Next() { var id string rows.Scan(&id) ids = append(ids, id) } rows.Close() if len(ids) > 0 { log.Printf("⚠ Corrupt JSON detected: %s.%s in %d rows: %v", t.table, t.column, len(ids), ids) } } } ``` This doesn't repair anything — just surfaces problems in the startup log so admins know. Repair is manual or via a future `--repair` flag. ### 17.2 Service Worker caches chrome-extension:// URLs **Symptom:** `TypeError: Failed to execute 'put' on 'Cache': Request scheme 'chrome-extension' is unsupported` on every page load. **Root cause:** The SW fetch handler's exclusion checks are all pathname-based (`/api/`, `/ws`, etc.). URLs from browser extensions pass through and hit `Cache.put()` which rejects non-http(s) schemes. **Fix:** Scheme guard at the top of the fetch handler: ```js self.addEventListener('fetch', (event) => { const url = new URL(event.request.url); if (url.protocol !== 'https:' && url.protocol !== 'http:') { return; // Ignore chrome-extension://, moz-extension://, etc. } // ... existing handler ... }); ``` ### 17.3 Branding 404 on custom.css **Symptom:** `GET /branding/custom.css 404` on every page load when no custom branding is configured. **Fix:** Serve an empty CSS file as the default when no custom branding is mounted. Either: - (a) Backend: static handler returns empty `text/css` body with `Cache-Control: no-cache` if file doesn't exist, or - (b) Frontend: conditionally emit the `` tag based on a `has_custom_branding` flag in the health/settings response. Option (a) is simpler — one handler change, no frontend coordination. ### 17.4 Duplicate PUT /settings during init **Symptom:** Two `PUT /api/v1/settings` calls within ~50ms during startup. Confirmed in server logs — two PUTs at 21:21:23 and again at 21:21:36 (second page load). **Root cause:** Two independent code paths saving settings on init. **Fix:** Consolidate or debounce. A `settingsDirty` flag + 100ms `setTimeout` debounce in the API layer ensures only one PUT fires per init cycle. ### 17.5 WebSocket token in URL (minor security hygiene) **Observation from server logs:** The WS endpoint logs include the full JWT in the URL query string: ``` GET "/test/ws?token=eyJhbG..." ``` This means the JWT appears in: Gin access logs, nginx access logs, and any log aggregator. Access tokens are short-lived (15 min) so the risk is low, but it's worth: - Redacting the token in Gin's log formatter (show `ws?token=[REDACTED]`) - Or switching to WS subprotocol auth (send token as first message after connect) in a future release Not blocking for v0.19.0 but worth noting.