149 lines
5.4 KiB
Markdown
149 lines
5.4 KiB
Markdown
# v0.12.0 — File Handling + Vision (Design)
|
|
|
|
## Overview
|
|
|
|
File input into chat — table stakes for serious use. Image uploads for
|
|
vision-capable models, document uploads for text extraction into context,
|
|
and the storage backend that v0.14.0 (Knowledge Bases) and v0.15.0
|
|
(Compaction snapshots) will reuse.
|
|
|
|
Also picks up stragglers deferred from earlier releases.
|
|
|
|
**Depends on:** v0.11.0 (extension foundation — complete).
|
|
**Reused by:** v0.14.0 (KBs), v0.15.0 (compaction), v0.22.0 (exports).
|
|
|
|
---
|
|
|
|
## Stragglers (from v0.9.4 deferred)
|
|
|
|
Items deferred during vault work that were never scheduled:
|
|
|
|
### `switchboard vault rekey` CLI command
|
|
Re-encrypts all global/team API keys when `ENCRYPTION_KEY` is rotated.
|
|
Personal BYOK keys are unaffected (keyed to UEK, not env var).
|
|
|
|
Implementation: standalone CLI entrypoint that reads old + new env vars,
|
|
iterates `api_configs` and `team_providers` where `key_scope IN ('global',
|
|
'team')`, decrypts with old key, re-encrypts with new key, updates in a
|
|
transaction.
|
|
|
|
### Admin UI: encryption status indicator
|
|
Badge in admin Settings showing whether `ENCRYPTION_KEY` is set and how
|
|
many keys are encrypted vs. plaintext (migration stragglers). Single
|
|
`GET /admin/storage/status` endpoint.
|
|
|
|
### Per-chat model/preset persistence (server-side)
|
|
Currently localStorage bandaid from v0.10.2 — doesn't roam across devices.
|
|
|
|
**Fix:** Store `last_selector_id` in `channels.settings` JSONB on each
|
|
completion success. On chat selection, resolve:
|
|
`channel.settings.last_selector_id` → match against available
|
|
models/presets → fall back to `channel.model` → global default.
|
|
|
|
No migration needed (`channels.settings` JSONB column already exists).
|
|
Single `PATCH /channels/:id` with settings merge on completion success.
|
|
|
|
---
|
|
|
|
## Track 1: Storage Backend
|
|
|
|
### Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `STORAGE_BACKEND` | (auto) | `pvc` or `s3`. If not set: `pvc` when `STORAGE_PATH` is writable, disabled otherwise. |
|
|
| `STORAGE_PATH` | `/data/storage` | Mount point for PVC backend. Also scratch dir for extraction with S3. |
|
|
| `STORAGE_CLASS` | — | K8s only. Gitea CI variable → PVC manifest. Value: `cephfs` (RWX for multi-pod). |
|
|
| `S3_ENDPOINT` | — | S3-compatible endpoint (e.g. `http://minio:9000`). Required when `STORAGE_BACKEND=s3`. |
|
|
| `S3_BUCKET` | — | Bucket name. Must exist before first start. |
|
|
| `S3_ACCESS_KEY` | — | S3 access key ID. |
|
|
| `S3_SECRET_KEY` | — | S3 secret access key. |
|
|
| `S3_REGION` | `us-east-1` | AWS region. |
|
|
| `S3_PREFIX` | — | Optional key prefix for shared buckets (e.g. `switchboard/`). |
|
|
| `S3_FORCE_PATH_STYLE` | `true` | Path-style URLs. Required for MinIO, Ceph RGW, most self-hosted. |
|
|
| `EXTRACTION_MODE` | `inline` | `inline` (in-process, unified image) or `sidecar` (K8s, shared PVC). |
|
|
| `EXTRACTION_CONCURRENCY` | `3` | Max concurrent extraction jobs. Caps LibreOffice memory usage. |
|
|
|
|
Auto-detection when `STORAGE_BACKEND` is not set:
|
|
|
|
```
|
|
STORAGE_PATH writable → pvc (implicit)
|
|
STORAGE_PATH missing → file features disabled
|
|
admin panel shows \"Storage not configured\"
|
|
```
|
|
|
|
When `STORAGE_BACKEND=pvc` is explicit, fail startup if path is not
|
|
writable (fail-safe, same pattern as `ENCRYPTION_KEY` enforcement).
|
|
|
|
S3 backend: reads `S3_ENDPOINT`, `S3_BUCKET`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`,
|
|
`S3_REGION`, `S3_PREFIX`, `S3_FORCE_PATH_STYLE` from env vars. Uses minio-go v7.
|
|
Works with MinIO, Ceph RGW, AWS S3. Same interface as PVC — all handlers are
|
|
backend-agnostic. PVC still required as scratch dir for extraction queue.
|
|
|
|
### Config Addition
|
|
|
|
```go
|
|
// In config.go
|
|
StorageBackend string // \"pvc\", \"s3\", or \"\" (auto-detect)
|
|
StoragePath string // mount point for PVC backend
|
|
ExtractionMode string // \"inline\" or \"sidecar\"
|
|
ExtractionConcurrency int // max concurrent extraction jobs
|
|
```
|
|
|
|
### Interface
|
|
|
|
```go
|
|
// server/storage/storage.go
|
|
package storage
|
|
|
|
import (
|
|
\"context\"
|
|
\"io\"
|
|
)
|
|
|
|
type ObjectStore interface {
|
|
// Put writes data to the given key. Creates parent dirs as needed.
|
|
Put(ctx context.Context, key string, r io.Reader, size int64, contentType string) error
|
|
|
|
// Get returns a reader for the given key. Caller must close.
|
|
// Returns ErrNotFound if key does not exist.
|
|
Get(ctx context.Context, key string) (io.ReadCloser, int64, string, error)
|
|
|
|
// Delete removes the object at key. No error if already gone.
|
|
Delete(ctx context.Context, key string) error
|
|
|
|
// DeletePrefix removes all objects under the given prefix.
|
|
// Used for channel deletion (bulk cleanup).
|
|
DeletePrefix(ctx context.Context, prefix string) error
|
|
|
|
// Exists checks if an object exists at key without reading it.
|
|
Exists(ctx context.Context, key string) (bool, error)
|
|
|
|
// Healthy returns nil if the backend is operational.
|
|
Healthy(ctx context.Context) error
|
|
}
|
|
|
|
var ErrNotFound = errors.New(\"storage: object not found\")
|
|
```
|
|
|
|
### PVC Implementation
|
|
|
|
`server/storage/pvc.go` — ~100 lines. Thin wrapper around `os.*`:
|
|
|
|
```go
|
|
type PVCStore struct {
|
|
basePath string // e.g. \"/data/storage\"
|
|
}
|
|
|
|
func NewPVC(basePath string) (*PVCStore, error) {
|
|
// Validate basePath is writable (create test file, remove)
|
|
// Create top-level subdirs: attachments/
|
|
}
|
|
```
|
|
|
|
Key → filesystem path: `filepath.Join(basePath, key)`. The key itself
|
|
provides all the directory structure.
|
|
|
|
### Filesystem Layout
|
|
|
|
[... full content truncated for response; full pasted in tool] |