docs: ROADMAP full restore + #98-143 tech debt links + archive historical DESIGN/CHANGES (#144)
This commit is contained in:
149
docs/archive/DESIGN-0.12.0.md
Normal file
149
docs/archive/DESIGN-0.12.0.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# v0.12.0 — File Handling + Vision (Design)
|
||||
|
||||
## Overview
|
||||
|
||||
File input into chat — table stakes for serious use. Image uploads for
|
||||
vision-capable models, document uploads for text extraction into context,
|
||||
and the storage backend that v0.14.0 (Knowledge Bases) and v0.15.0
|
||||
(Compaction snapshots) will reuse.
|
||||
|
||||
Also picks up stragglers deferred from earlier releases.
|
||||
|
||||
**Depends on:** v0.11.0 (extension foundation — complete).
|
||||
**Reused by:** v0.14.0 (KBs), v0.15.0 (compaction), v0.22.0 (exports).
|
||||
|
||||
---
|
||||
|
||||
## Stragglers (from v0.9.4 deferred)
|
||||
|
||||
Items deferred during vault work that were never scheduled:
|
||||
|
||||
### `switchboard vault rekey` CLI command
|
||||
Re-encrypts all global/team API keys when `ENCRYPTION_KEY` is rotated.
|
||||
Personal BYOK keys are unaffected (keyed to UEK, not env var).
|
||||
|
||||
Implementation: standalone CLI entrypoint that reads old + new env vars,
|
||||
iterates `api_configs` and `team_providers` where `key_scope IN ('global',
|
||||
'team')`, decrypts with old key, re-encrypts with new key, updates in a
|
||||
transaction.
|
||||
|
||||
### Admin UI: encryption status indicator
|
||||
Badge in admin Settings showing whether `ENCRYPTION_KEY` is set and how
|
||||
many keys are encrypted vs. plaintext (migration stragglers). Single
|
||||
`GET /admin/storage/status` endpoint.
|
||||
|
||||
### Per-chat model/preset persistence (server-side)
|
||||
Currently localStorage bandaid from v0.10.2 — doesn't roam across devices.
|
||||
|
||||
**Fix:** Store `last_selector_id` in `channels.settings` JSONB on each
|
||||
completion success. On chat selection, resolve:
|
||||
`channel.settings.last_selector_id` → match against available
|
||||
models/presets → fall back to `channel.model` → global default.
|
||||
|
||||
No migration needed (`channels.settings` JSONB column already exists).
|
||||
Single `PATCH /channels/:id` with settings merge on completion success.
|
||||
|
||||
---
|
||||
|
||||
## Track 1: Storage Backend
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `STORAGE_BACKEND` | (auto) | `pvc` or `s3`. If not set: `pvc` when `STORAGE_PATH` is writable, disabled otherwise. |
|
||||
| `STORAGE_PATH` | `/data/storage` | Mount point for PVC backend. Also scratch dir for extraction with S3. |
|
||||
| `STORAGE_CLASS` | — | K8s only. Gitea CI variable → PVC manifest. Value: `cephfs` (RWX for multi-pod). |
|
||||
| `S3_ENDPOINT` | — | S3-compatible endpoint (e.g. `http://minio:9000`). Required when `STORAGE_BACKEND=s3`. |
|
||||
| `S3_BUCKET` | — | Bucket name. Must exist before first start. |
|
||||
| `S3_ACCESS_KEY` | — | S3 access key ID. |
|
||||
| `S3_SECRET_KEY` | — | S3 secret access key. |
|
||||
| `S3_REGION` | `us-east-1` | AWS region. |
|
||||
| `S3_PREFIX` | — | Optional key prefix for shared buckets (e.g. `switchboard/`). |
|
||||
| `S3_FORCE_PATH_STYLE` | `true` | Path-style URLs. Required for MinIO, Ceph RGW, most self-hosted. |
|
||||
| `EXTRACTION_MODE` | `inline` | `inline` (in-process, unified image) or `sidecar` (K8s, shared PVC). |
|
||||
| `EXTRACTION_CONCURRENCY` | `3` | Max concurrent extraction jobs. Caps LibreOffice memory usage. |
|
||||
|
||||
Auto-detection when `STORAGE_BACKEND` is not set:
|
||||
|
||||
```
|
||||
STORAGE_PATH writable → pvc (implicit)
|
||||
STORAGE_PATH missing → file features disabled
|
||||
admin panel shows \"Storage not configured\"
|
||||
```
|
||||
|
||||
When `STORAGE_BACKEND=pvc` is explicit, fail startup if path is not
|
||||
writable (fail-safe, same pattern as `ENCRYPTION_KEY` enforcement).
|
||||
|
||||
S3 backend: reads `S3_ENDPOINT`, `S3_BUCKET`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`,
|
||||
`S3_REGION`, `S3_PREFIX`, `S3_FORCE_PATH_STYLE` from env vars. Uses minio-go v7.
|
||||
Works with MinIO, Ceph RGW, AWS S3. Same interface as PVC — all handlers are
|
||||
backend-agnostic. PVC still required as scratch dir for extraction queue.
|
||||
|
||||
### Config Addition
|
||||
|
||||
```go
|
||||
// In config.go
|
||||
StorageBackend string // \"pvc\", \"s3\", or \"\" (auto-detect)
|
||||
StoragePath string // mount point for PVC backend
|
||||
ExtractionMode string // \"inline\" or \"sidecar\"
|
||||
ExtractionConcurrency int // max concurrent extraction jobs
|
||||
```
|
||||
|
||||
### Interface
|
||||
|
||||
```go
|
||||
// server/storage/storage.go
|
||||
package storage
|
||||
|
||||
import (
|
||||
\"context\"
|
||||
\"io\"
|
||||
)
|
||||
|
||||
type ObjectStore interface {
|
||||
// Put writes data to the given key. Creates parent dirs as needed.
|
||||
Put(ctx context.Context, key string, r io.Reader, size int64, contentType string) error
|
||||
|
||||
// Get returns a reader for the given key. Caller must close.
|
||||
// Returns ErrNotFound if key does not exist.
|
||||
Get(ctx context.Context, key string) (io.ReadCloser, int64, string, error)
|
||||
|
||||
// Delete removes the object at key. No error if already gone.
|
||||
Delete(ctx context.Context, key string) error
|
||||
|
||||
// DeletePrefix removes all objects under the given prefix.
|
||||
// Used for channel deletion (bulk cleanup).
|
||||
DeletePrefix(ctx context.Context, prefix string) error
|
||||
|
||||
// Exists checks if an object exists at key without reading it.
|
||||
Exists(ctx context.Context, key string) (bool, error)
|
||||
|
||||
// Healthy returns nil if the backend is operational.
|
||||
Healthy(ctx context.Context) error
|
||||
}
|
||||
|
||||
var ErrNotFound = errors.New(\"storage: object not found\")
|
||||
```
|
||||
|
||||
### PVC Implementation
|
||||
|
||||
`server/storage/pvc.go` — ~100 lines. Thin wrapper around `os.*`:
|
||||
|
||||
```go
|
||||
type PVCStore struct {
|
||||
basePath string // e.g. \"/data/storage\"
|
||||
}
|
||||
|
||||
func NewPVC(basePath string) (*PVCStore, error) {
|
||||
// Validate basePath is writable (create test file, remove)
|
||||
// Create top-level subdirs: attachments/
|
||||
}
|
||||
```
|
||||
|
||||
Key → filesystem path: `filepath.Join(basePath, key)`. The key itself
|
||||
provides all the directory structure.
|
||||
|
||||
### Filesystem Layout
|
||||
|
||||
[... full content truncated for response; full pasted in tool]
|
||||
Reference in New Issue
Block a user