107 lines
4.2 KiB
Markdown
107 lines
4.2 KiB
Markdown
# v0.15.0 Phase 4 — Guard Rail + Integration Tests
|
||
|
||
## Files
|
||
|
||
```
|
||
server/compaction/compaction.go UPDATED — context budget guard rail
|
||
server/compaction/compaction_test.go NEW — scanner integration tests (DB required)
|
||
server/compaction/testmain_test.go NEW — TestMain for compaction package
|
||
server/database/seed_helpers.go NEW — SeedTestMessage, SeedTestMessages, SeedTestCursor
|
||
server/handlers/live_compaction_test.go NEW — e2e tests with Venice/Qwen3-4B
|
||
```
|
||
|
||
## Changes
|
||
|
||
### 1. Context Budget Guard Rail (`compaction.go`)
|
||
|
||
Added between prompt construction and LLM call. Prevents sending
|
||
truncated input to small utility models.
|
||
|
||
**How it works:**
|
||
1. Estimates total prompt tokens (system + conversation content)
|
||
2. Resolves the utility model's `max_context` from catalog
|
||
3. Reserves 20% for output (the summary itself)
|
||
4. If `estimated_prompt > max_context × 0.80`, returns `ErrContextBudget`
|
||
|
||
**Resolution chain** for utility model context:
|
||
- `Resolver.GetConfig()` → primary binding → `Catalog.GetByModelID()`
|
||
- Fallback: `Catalog.GetByModelIDAny()` (different provider)
|
||
- Hard fallback: `DefaultBudget` (128K)
|
||
|
||
The scanner's `doCompact()` already handles errors gracefully — a budget
|
||
error just logs a warning and retries next cycle (by which time the
|
||
conversation may have been manually compacted or the admin may have
|
||
upgraded the utility model).
|
||
|
||
**New export:** `ErrContextBudget` — callers can use `errors.Is()` to
|
||
distinguish budget failures from LLM errors.
|
||
|
||
### 2. Test Seed Helpers (`database/seed_helpers.go`)
|
||
|
||
New file in the `database` package alongside `testhelper.go`:
|
||
|
||
- `SeedTestMessage(t, channelID, parentID, role, content) → msgID`
|
||
- `SeedTestMessages(t, channelID, count, contentSize) → []msgIDs`
|
||
Creates a linear alternating user/assistant chain
|
||
- `SeedTestCursor(t, channelID, userID, leafID)`
|
||
Sets the active leaf for tree path resolution
|
||
|
||
### 3. Scanner Integration Tests (`compaction/compaction_test.go`)
|
||
|
||
All DB-dependent tests use `database.RequireTestDB(t)` and skip
|
||
gracefully when no DB is configured.
|
||
|
||
| Test | What it verifies |
|
||
|------|-----------------|
|
||
| `FindCandidates_ReturnsQualifying` | Channel with ≥10 msgs, ≥20K chars, proper age range appears |
|
||
| `FindCandidates_ExcludesTooFewMessages` | <10 messages → excluded |
|
||
| `FindCandidates_ExcludesTooRecent` | Updated just now → excluded (activity gap) |
|
||
| `FindCandidates_ExcludesArchived` | `is_archived=true` → excluded |
|
||
| `Cooldown` | Timestamps recorded and checked correctly |
|
||
| `InFlightDedup` | sync.Map prevents concurrent compaction of same channel |
|
||
| `ShouldCompact_ChannelOptOut` | `settings.auto_compaction=false` → rejected |
|
||
| `IsEnabled` | Reads `auto_compaction_enabled` from global_settings |
|
||
| `GetThreshold_Default` | Returns 0.70 when no override |
|
||
| `GetThreshold_GlobalOverride` | Reads from global_settings |
|
||
| `GetThreshold_ChannelOverride` | Channel setting takes precedence |
|
||
| `GetCooldownDuration_Default` | Returns 30min default |
|
||
| `GetCooldownDuration_Override` | Reads from global_settings |
|
||
| `GuardRailMath` | Validates token math for 32K vs 128K models |
|
||
|
||
### 4. Live E2E Tests (`handlers/live_compaction_test.go`)
|
||
|
||
Requires: `TEST_DATABASE_URL` + `VENICE_API_KEY`
|
||
|
||
| Test | What it verifies |
|
||
|------|-----------------|
|
||
| `CompactionFullPipeline` | Seeds 10 realistic messages → Compact() → verifies summary message in DB, metadata fields, cursor update, usage log entry |
|
||
| `CompactionContextBudgetGuardRail` | Seeds 160K chars of content → Compact() returns ErrContextBudget with 32K model |
|
||
|
||
## Running Tests
|
||
|
||
```bash
|
||
# Unit only (no DB)
|
||
cd server && go test ./compaction/ -run TestEstimate -v
|
||
|
||
# Integration (requires DB)
|
||
cd server && go test ./compaction/ -v
|
||
|
||
# Live e2e (requires DB + Venice key)
|
||
cd server && VENICE_API_KEY=... go test ./handlers/ -run TestLive_Compaction -v
|
||
```
|
||
|
||
## Surgical Edit
|
||
|
||
### `database/testhelper.go`
|
||
|
||
Add this import if not already present (the new `seed_helpers.go` file
|
||
is a separate file in the same package, so no edit needed to
|
||
`testhelper.go` itself):
|
||
|
||
```go
|
||
// No changes needed — seed_helpers.go is a new file in package database
|
||
```
|
||
|
||
The `strings` import in `seed_helpers.go` is used by `SeedTestMessages`
|
||
for `strings.Repeat`.
|