# DESIGN-0.34.0 — Data Portability

Export your data, import it elsewhere, delete your account. Six
changesets covering user export, user import, GDPR delete, admin
team export/import, ChatGPT import, and K8s scheduled backups.

Depends on: v0.33.0 (Observability, current HEAD).

**Design decision:** One new migration (021) — indexes only, no new
tables. The export archive is a ZIP file (`.switchboard` extension)
containing a JSON manifest and per-entity JSON files. Sensitive fields
(password hashes, encrypted keys, storage keys) are stripped at export
time via entity sanitization.

**Archive format:** `.switchboard` ZIP with `format_version=1`:
```
manifest.json
data/users.json
data/channels.json
data/messages.json
data/channel_participants.json
data/channel_models.json
data/channel_cursors.json
data/notes.json
data/note_links.json
data/memories.json
data/projects.json
data/project_channels.json
data/project_knowledge_bases.json
data/project_notes.json
data/workspaces.json
data/workspace_files.json
data/folders.json
data/user_settings.json
data/notification_preferences.json
data/usage_entries.json
data/persona_groups.json
data/persona_group_members.json
data/files.json
files/{file_id}/{filename}
```

---

## What Already Existed

- **Package export** — `.pkg` ZIP format, `package_export.go`
- **Markdown export** — pandoc-based, `export.go`
- **Workspace archive** — ZIP/tar.gz with size limits, `workspace/archive.go`
- **Soft deletes** — `deleted_at` on messages
- **Audit trail** — `audit_logs` table
- **39 store interfaces** covering all entity types

---

## CS0 — Export Format + User Export

Foundation changeset. Defines the archive format and ExportStore interface.

### `server/export/format.go` (new)

Archive types: `Manifest`, `ManifestScope`, `ArchiveWriter`, `ArchiveReader`.
Size limits: 500 MB archive, 100 MB per file, 10K files max.
Key methods: `WriteManifest`, `WriteEntityJSON`, `WriteFile`,
`ReadManifest`, `ReadEntityJSON`, `FileEntries`.

### `server/export/entities.go` (new)

Per-entity sanitization functions. Export types strip sensitive fields:
- `ExportChannel` — removes `provider_config_id`
- `ExportMessage` — no changes needed
- `ExportWorkspace` — removes `root_path`, git credentials
- `ExportUsageEntry` — removes `provider_config_id`
- Users — removes `password_hash`, `encrypted_uek`, `uek_salt`

### `server/store/interfaces.go` (modified)

Added `Export ExportStore` to `Stores` struct. The `ExportStore`
interface provides 21 user-scoped read methods, 20 import methods
(returning `(imported, skipped, error)` counts), 14 team-scoped
read methods, and 4 GDPR methods.

### `server/store/{postgres,sqlite}/export.go` (new)

Full implementations for both dialects (~1400-1500 lines each).
Import uses `INSERT ON CONFLICT DO NOTHING` (PG) / `INSERT OR IGNORE`
(SQLite) for UUID-based dedup.

### `server/handlers/export_data.go` (new)

`DataExportHandler.ExportMyData` — `GET /api/v1/export/me`.
Streams ZIP directly to response using `zip.NewWriter(c.Writer)`.
Includes file blobs from object store.

### Migration 021

Indexes only — `idx_messages_channel_active`, `idx_channels_user_id`,
`idx_files_user_id`. PG version uses partial index
(`WHERE deleted_at IS NULL`).

---

## CS1 — User Import

### `server/handlers/import_data.go` (new)

`DataImportHandler.ImportMyData` — `POST /api/v1/import/me`.

Opens archive, validates manifest `format_version`, reads entities
in FK-dependency order, remaps `user_id` for cross-instance import.
Returns `{imported: {channels: N, ...}, skipped: {...}, errors: [...]}`.

Import order: Folders → Projects → Workspaces → Channels →
Participants/Models/Cursors → Messages → Notes → Note Links →
Memories → Project links → Workspace files → Files + blobs →
Settings → Persona groups.

---

## CS2 — GDPR Account Delete

### `server/handlers/gdpr.go` (new)

`GDPRHandler.DeleteMyAccount` — `DELETE /api/v1/me`.

Requires `{"confirm": "DELETE", "password": "..."}`. Verifies
password, prevents last-admin deletion, then:

1. Soft-deletes messages, archives channels
2. Hard-deletes: notes, memories, workspaces, files, projects, settings
3. Revokes tokens (refresh + WS tickets)
4. Anonymizes user: `deleted-user-{sha256(id)[:12]}`
5. Writes audit log entry

---

## CS3 — Admin Team Export/Import

`DataExportHandler.ExportTeam` — `GET /api/v1/admin/teams/:id/export`
`DataImportHandler.ImportTeam` — `POST /api/v1/admin/teams/:id/import`

Same archive format, scoped to team entities (channels, personas,
knowledge bases, workflows, groups, projects, resource grants).
Team import skips members whose `user_id` doesn't exist on target.

---

## CS4 — ChatGPT Import

### `server/export/chatgpt.go` (new)

Parses ChatGPT `conversations.json` format. Each conversation maps
to one channel (type=direct). The `mapping` DAG is walked depth-first
to produce ordered messages with `parent_id` and `sibling_index`.

Role mapping: `user`→user, `assistant`→assistant, `system`→system,
`tool`→tool. Content: `parts[]` joined with `\n`, non-string parts
skipped. Timestamps: `float64` epoch → `time.Time`.

`DataImportHandler.ImportChatGPT` — `POST /api/v1/import/chatgpt`.
Accepts multipart upload of `conversations.json` or ZIP containing it.

---

## CS5 — K8s Backup + ICD Tests

### `chart/templates/cronjob-backup.yaml` (new)

CronJob running `pg_dump | gzip` on schedule (default daily 02:00 UTC).
Optional S3 upload via AWS CLI. Local retention pruning. Gated on
`backup.enabled=true` and `database.driver=postgres`.

### `chart/templates/pvc-backup.yaml` (new)

Conditional PVC for local backup storage (when `backup.persistence.enabled`).

### `chart/values.yaml` (modified)

New `backup:` section with schedule, retention, S3 config, persistence,
and resource limits.

### ICD Tests

~30 new tests in `crud/portability.js` covering:
- Export download + archive validation (ZIP structure, manifest, entities)
- Sensitive field exclusion verification
- Import re-import (dedup validation)
- Invalid archive rejection
- ChatGPT import (valid, invalid JSON, empty array)
- ChatGPT channel/message verification
- GDPR delete flow (register → create data → delete → verify lockout)
- GDPR validation (missing confirm, wrong password)
- Admin team export/import

---

## New Routes

| Method | Path | Auth | Handler |
|--------|------|------|---------|
| GET | `/api/v1/export/me` | JWT | `DataExportHandler.ExportMyData` |
| POST | `/api/v1/import/me` | JWT | `DataImportHandler.ImportMyData` |
| POST | `/api/v1/import/chatgpt` | JWT | `DataImportHandler.ImportChatGPT` |
| DELETE | `/api/v1/me` | JWT | `GDPRHandler.DeleteMyAccount` |
| GET | `/api/v1/admin/teams/:id/export` | Admin | `DataExportHandler.ExportTeam` |
| POST | `/api/v1/admin/teams/:id/import` | Admin | `DataImportHandler.ImportTeam` |

---

## Verification

1. `cd server && go build ./...` — all changesets compile clean
2. `helm template ./chart --set backup.enabled=true` — CronJob renders
3. ICD test suite: ~615 tests (585 existing + ~30 portability)
4. Manual smoke: export → inspect ZIP → re-import → verify dedup
5. GDPR: delete → verify login fails → inspect anonymized DB record
6. Sensitive fields: no `password_hash`, `encrypted_uek`, `storage_key` in export