diff --git a/CHANGELOG.md b/CHANGELOG.md index 4575407..429e3b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,43 @@ # Changelog +## [0.35.1] — 2026-03-20 + +### Summary + +Data Portability FE Wiring: connects the v0.34.0 backend (export, +import, ChatGPT import, GDPR delete, backup CronJob) to the frontend. +New Settings → Data & Privacy section. Admin team export button. + +### New + +- **Settings → Data & Privacy section** — new nav link and full-page + section with four features: + - **Download My Data** — streams `GET /export/me` as `.switchboard` + zip blob download (conversations, notes, memories, projects, files, + settings, usage). + - **Import Data** — file picker for `.switchboard` / `.zip` archives, + uploads to `POST /import/me` with import/skip/error counts in toast. + - **Import from ChatGPT** — file picker for ChatGPT export zips, + uploads to `POST /import/chatgpt`, maps conversations to direct + chats. + - **Delete My Account** — danger zone with password confirmation, + calls `DELETE /me`, shows anonymized username, clears tokens, and + redirects to `/login`. +- **Admin team export button** — 📦 icon button on each team row in + Admin → Teams. Calls `GET /admin/teams/:id/export` and downloads + `.switchboard` archive. + +### New files + +- `src/js/data-portability.js` — Settings section renderer + admin + team export helper. + +### Changed + +- `settings.html` — added nav link, section title, content area, + script tag, and dynamic section dispatch for `data`. +- `ui-admin.js` — added export button to team list action cells. + ## [0.35.0] — 2026-03-19 ### Summary @@ -64,6 +102,105 @@ Observability: structured logging, Prometheus metrics, OpenAPI docs, Grafana dashboards, alerting rules, and a built-in admin monitoring dashboard. Operate the platform without reading Go source code. +## [0.34.0] — 2026-03-19 + +### Summary + +Data Portability: bulk export/import for user and team data, GDPR +"download my data" and "delete my data" with cascade + audit trail, +ChatGPT conversation import, and Kubernetes backup/restore CronJob +manifests. + +### New + +- **User data export** (`GET /api/v1/export/me`) — streams a + `.switchboard` zip archive containing all user-scoped entities: + channels, messages, notes, note links, memories, projects, project + associations, workspaces, workspace files, file attachments (blobs), + folders, user model settings, notification preferences, usage entries, + persona groups, and persona group members. Manifest includes entity + counts and scope metadata. Sensitive fields (password hashes, vault + keys, provider config IDs) stripped via `export.Sanitize*` functions. + File blobs capped at 100 MB per file, 500 MB total archive, 10K files. +- **User data import** (`POST /api/v1/import/me`) — accepts + `.switchboard` zip upload, validates manifest and format version, + imports entities in FK-dependency order with UUID dedup (skip if ID + exists). User ID remapping for cross-instance portability (source + user → current user). Returns imported/skipped counts and error list. +- **ChatGPT import** (`POST /api/v1/import/chatgpt`) — accepts ChatGPT + export zip containing `conversations.json`. Maps ChatGPT conversation + format (alternating author roles, message parts) into Switchboard + channels + messages. Handles `system`, `user`, `assistant`, and + `tool` roles. Title becomes channel name. +- **GDPR account deletion** (`DELETE /api/v1/me`) — requires explicit + `"confirm": "DELETE"` + password verification. Prevents deleting the + last admin. Cascade: soft-deletes user data (messages anonymized, + channels/notes/memories/projects deleted), revokes tokens, deletes + personal provider configs, anonymizes user record to + `deleted-user-{hash}`. Full audit trail. +- **Team data export** (`GET /api/v1/admin/teams/:id/export`) — admin + endpoint streaming a `.switchboard` archive of all team-scoped + entities: channels, messages, members, personas, persona KBs, + knowledge bases, KB documents, workflows, workflow versions/stages, + groups, group members, resource grants, and projects. Sanitizes + personas (strips provider binding), workflows (strips webhook + secrets). +- **Team data import** (`POST /api/v1/admin/teams/:id/import`) — admin + endpoint accepting `.switchboard` archive for team data restoration. +- **Backup/restore CronJob** — Helm chart templates for scheduled + `pg_dump` backups. `cronjob-backup.yaml` runs on configurable + schedule (default: daily at 02:00), writes gzip-compressed dump to + PVC with configurable retention (default: 7 backups). Optional S3 + offload. `pvc-backup.yaml` for backup storage. Gated by + `backup.enabled` in `values.yaml`. + +### New files + +- `server/export/format.go` — `.switchboard` archive format: zip + with `manifest.json`, `data/*.json` entity files, `files/` blob + directory. `ArchiveWriter` with size tracking, `ArchiveReader` + with manifest validation. +- `server/export/entities.go` — per-entity sanitization: `SanitizeUser` + (strips password hash, vault fields), `SanitizeChannels` (strips + provider config), `SanitizeMessages`, `SanitizeChannelModels`, + `SanitizeUserModelSettings`, `SanitizeUsageEntries`, + `SanitizeWorkspaces` (strips git creds), `SanitizePersonas`, + `SanitizeWorkflows` (strips webhook secrets). +- `server/export/chatgpt.go` — ChatGPT `conversations.json` parser. + `ParseChatGPTExport` reads zip, finds `conversations.json`, + maps to `ChatGPTConversation` structs with `ChatGPTMessage` + (author role, content parts, create_time). +- `server/handlers/export_data.go` — `DataExportHandler` with + `ExportMyData` and `ExportTeam` endpoints. +- `server/handlers/import_data.go` — `DataImportHandler` with + `ImportMyData`, `ImportTeam`, and `ImportChatGPT` endpoints. + FK-dependency-ordered import with dedup. +- `server/handlers/gdpr.go` — `GDPRHandler` with `DeleteMyAccount`. +- `server/store/postgres/export.go` — `ExportStore` Postgres + implementation: 25+ batch-read queries for user/team export, + import methods with `ON CONFLICT DO NOTHING`, `SoftDeleteUserData`, + `AnonymizeUser`, `DeleteUserTokens`, `CountActiveAdmins`. +- `server/store/sqlite/export.go` — `ExportStore` SQLite + implementation with identical interface. +- `chart/templates/cronjob-backup.yaml` — Kubernetes CronJob for + scheduled pg_dump with gzip compression and retention cleanup. +- `chart/templates/pvc-backup.yaml` — PersistentVolumeClaim for + backup storage (gated by `backup.persistence.enabled`). + +### Schema + +- No new migrations — uses existing tables only. +- `ExportStore` added to `store.Stores` struct. + +### Changed + +- `store/interfaces.go` — added `ExportStore` interface with 25+ + methods for batch export reads, import writes, and GDPR operations. +- `chart/values.yaml` — added `backup` section with `enabled`, + `schedule`, `retention`, `persistence`, and `s3Offload` config. +- `main.go` — wired `DataExportHandler`, `DataImportHandler`, + `GDPRHandler` routes under authenticated and admin groups. + ### New - **Structured logging (`slog`)** — `LOG_FORMAT=json|text`, diff --git a/VERSION b/VERSION index 7b52f5e..731b95d 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.35.0 +0.35.1 diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 447a145..9500a06 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -32,7 +32,7 @@ v0.9.x–v0.28.7 Foundation through Platform Polish ✅ │ │ v0.29.0 Starlark Sandbox ✅ v0.32.0 Multi-Replica HA ✅ v0.29.1 API Extensions ✅ v0.33.0 Observability ✅ - v0.29.2 DB Extensions ✅ v0.34.0 Data Portability + v0.29.2 DB Extensions ✅ v0.34.0 Data Portability ✅ v0.29.3 Workflow Forms ✅ │ v0.30.0 Package Lifecycle ✅ │ v0.30.1 SDK Adoption ✅ │ @@ -414,17 +414,17 @@ Depends on: v0.32.0 (multi-replica metrics aggregation). - Full OpenAPI spec coverage (all 20 ICD domains) - Bearer token / mTLS auth documentation in spec -### v0.34.0 — Data Portability +### v0.34.0 — Data Portability ✅ Export, import, backup, compliance. Depends on: v0.29.2 (DB extensions — extension data in exports). -- [ ] Bulk export/import: account data, conversations, settings, files -- [ ] GDPR "download my data" + "delete my data" (cascade + audit trail) -- [ ] ChatGPT/other tool import (conversation format mapping) -- [ ] Backup/restore CronJob manifests for K8s -- [ ] Admin export: team/user config (excludes vault-encrypted keys) +- [x] Bulk export/import: account data, conversations, settings, files +- [x] GDPR "download my data" + "delete my data" (cascade + audit trail) +- [x] ChatGPT/other tool import (conversation format mapping) +- [x] Backup/restore CronJob manifests for K8s +- [x] Admin export: team/user config (excludes vault-encrypted keys) ### v0.35.0 — Workflow Product @@ -538,7 +538,7 @@ reading Go source code. Team admins build workflows visually. - Extension track through v0.31.2 (full package ecosystem, visual workflow builder, SDK-based surfaces, team workflow self-service) - Operations track through v0.34.0 (multi-replica HA, observability, - data portability) + data portability) ✅ - Workflow product v0.35.0 (form rendering, data pipeline, conditional routing, structured review, monitoring dashboard) - Full OpenAPI spec v0.36.0 (complete API documentation for all domains) diff --git a/server/pages/templates/surfaces/settings.html b/server/pages/templates/surfaces/settings.html index 75776d2..8245f31 100644 --- a/server/pages/templates/surfaces/settings.html +++ b/server/pages/templates/surfaces/settings.html @@ -33,6 +33,7 @@ Workflows Tasks Git Keys + Data & Privacy {{/* BYOK-gated nav items */}}