Changeset 0.22.2 (#96)
This commit is contained in:
@@ -152,6 +152,12 @@ server/
|
||||
├── events/ # EventBus + WebSocket hub
|
||||
├── extraction/ # Document text extraction pipeline
|
||||
├── health/ # Provider health tracking (v0.22.0)
|
||||
├── routing/ # Policy-based request routing (v0.22.2)
|
||||
│ ├── types.go # Policy, Candidate, Context, Decision types
|
||||
│ ├── evaluator.go # Policy evaluation engine (4 policy types)
|
||||
│ ├── evaluator_test.go # 12 tests
|
||||
│ ├── fallback.go # RunWithFallback candidate dispatcher
|
||||
│ └── convert.go # DB model → routing type conversion
|
||||
│ ├── accumulator.go # In-memory counters, 60s flush to DB, status derivation
|
||||
│ └── accumulator_test.go # Concurrent recording, flush isolation, threshold tests
|
||||
├── memory/ # Long-term memory extraction + scanning
|
||||
|
||||
@@ -99,7 +99,7 @@ v0.22.0 Provider Health + Capability Overrides ✅
|
||||
│
|
||||
v0.22.1 Provider Extensions (declarative config) ✅
|
||||
│
|
||||
v0.22.2 Routing Policies + Fallback Chains
|
||||
v0.22.2 Routing Policies + Fallback Chains ✅
|
||||
│
|
||||
v0.22.3 Provider Admin UI + Deferred Polish
|
||||
│
|
||||
@@ -940,30 +940,33 @@ Depends on: provider health (v0.22.0).
|
||||
|
||||
---
|
||||
|
||||
## v0.22.2 — Routing Policies + Fallback Chains
|
||||
## v0.22.2 — Routing Policies + Fallback Chains ✅
|
||||
|
||||
Rules-based routing engine — policy, not ML. Evaluated in `resolveConfig()`
|
||||
Rules-based routing engine — policy, not ML. Evaluated after `resolveConfig()`
|
||||
between "what model was requested" and "which provider serves it."
|
||||
|
||||
Depends on: provider health (v0.22.0), provider extensions (v0.22.1).
|
||||
|
||||
**Routing Policies**
|
||||
- [ ] `routing_policies` table: `id`, `name`, `scope` (global/team), `team_id`, `priority` (lower wins), `policy_type`, `config` (JSONB), `is_active`
|
||||
- [ ] Policy types: `capability_match` ("cheapest model with tool_calling"), `provider_prefer` ("prefer provider X, fallback Y"), `team_route` ("team X uses provider Y"), `cost_limit` ("max $X/request")
|
||||
- [ ] Policy evaluator: takes requested model + user context → returns ranked list of `(providerConfigID, modelID)` candidates
|
||||
- [ ] Integration point: `resolveConfig()` calls evaluator after preset unwrap, before provider dispatch
|
||||
- [x] `routing_policies` table: `id`, `name`, `scope` (global/team), `team_id`, `priority` (lower wins), `policy_type`, `config` (JSONB), `is_active`
|
||||
- [x] Policy types: `provider_prefer` (ordered fallback list), `team_route` (restrict team to providers), `cost_limit` (heuristic cost cap), `model_alias` (alias → provider+model rewrite)
|
||||
- [x] Policy evaluator: takes requested model + user context → returns ranked list of `(providerConfigID, modelID)` candidates
|
||||
- [x] Integration point: `evaluateRouting()` called after resolveConfig, before provider dispatch — reloads winning config credentials on switch
|
||||
- [ ] _(deferred → v0.22.3)_ `capability_match` policy type ("cheapest model with tool_calling")
|
||||
|
||||
**Fallback Chains**
|
||||
- [ ] Fallback chain configuration: ordered list of providers for a given model or model family
|
||||
- [ ] Health-aware: skip providers with status `down`, prefer `healthy` over `degraded`
|
||||
- [ ] Automatic retry on provider error: next provider in chain, transparent to client (configurable max retries)
|
||||
- [ ] Latency-aware preference: track response time percentiles, prefer faster providers (weight configurable)
|
||||
- [ ] Cost-aware preference: factor pricing into routing decisions (cheapest-first, budget ceiling per request)
|
||||
- [x] Fallback runner: `RunWithFallback()` tries candidates in order with configurable max retries
|
||||
- [x] Health-aware: skip providers with status `down`, prefer `healthy` over `degraded`
|
||||
- [x] All-down graceful degradation: keeps candidates rather than failing
|
||||
- [ ] _(deferred → v0.22.3)_ Latency-aware preference: track response time percentiles, prefer faster providers
|
||||
- [ ] _(deferred → v0.22.3)_ Cost-aware preference with budget ceiling per request
|
||||
|
||||
**Routing Metadata**
|
||||
- [ ] `X-Switchboard-Provider` response header: which provider actually served the request (debug mode)
|
||||
- [ ] `routing_decision` field in usage_log: selected provider, policy that matched, fallback depth
|
||||
- [ ] `GET /api/v1/admin/routing/test` endpoint: dry-run policy evaluation for a given model + user
|
||||
- [x] `X-Switchboard-Provider` response header: `providerID/configID` on every completion
|
||||
- [x] `X-Switchboard-Route` header: policy name when routing is active
|
||||
- [x] `routing_decision` JSONB column on `usage_log`
|
||||
- [x] `POST /api/v1/admin/routing/test` endpoint: dry-run policy evaluation with live health status
|
||||
- [x] Admin CRUD: GET/POST/PUT/DELETE `/api/v1/admin/routing/policies`
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user