Archived

This repository has been archived on 2026-04-03. You can view files and clone it. You cannot open issues or pull requests or push a commit.

Files

xcaliber 58313f7e31 Changeset 0.28.0.1 (#173 )

2026-03-11 14:45:37 +00:00

5.8 KiB

Raw Blame History

Providers & Routing

The multi-provider system. One or more LLM providers are configured, each with their own API keys, endpoints, and model catalogs. The routing layer decides which provider handles each request.

User BYOK Provider Configs

Gated by allow_user_byok policy. Personal API keys are encrypted with the user's UEK (per-user encryption key, Argon2id-derived). Platform admins cannot recover personal keys.

GET    /api-configs                 → { "configs": [safeConfig objects] }
POST   /api-configs                 ← { "name", "provider", "endpoint", "api_key", ... }
GET    /api-configs/:id             → safeConfig
PUT    /api-configs/:id             ← partial update (api_key optional)
DELETE /api-configs/:id

safeConfig — API keys are never returned:

{
  "id": "uuid",
  "name": "My OpenAI",
  "provider": "openai",
  "endpoint": "https://api.openai.com/v1",
  "model_default": "gpt-4o",
  "scope": "personal",
  "owner_id": "uuid",
  "is_active": true,
  "has_key": true,
  "config": {},
  "headers": {},
  "settings": {},
  "created_at": "..."
}

List models for a user config:

GET /api-configs/:id/models

Returns { "models": [catalog entries] }.

Fetch/sync models from provider API:

POST /api-configs/:id/models/fetch

Calls the provider's model list API, upserts into the local catalog.

Admin Global Provider Configs

GET    /admin/configs               → { "configs": [configWithKey objects] }
POST   /admin/configs               ← { "name", "provider", "endpoint", "api_key", "config", "headers", "settings", "is_private" }
PUT    /admin/configs/:id           ← partial update
DELETE /admin/configs/:id

configWithKey is the same as safeConfig but comes from ListGlobal — still redacts API keys, just adds the has_key flag.

Admin configs use the ENCRYPTION_KEY env var (not per-user UEK). is_private: when true, the config is available for admin-created Personas but not directly selectable by users.

Model Catalog (Admin)

The catalog is populated by fetching from provider APIs and stores model metadata (capabilities, context window, pricing).

GET    /admin/models                → { "models": [catalog entries] }
PUT    /admin/models/:id            ← { "visibility", "display_name", ... }
PUT    /admin/models/bulk           ← { "provider_config_id", "visibility" }
DELETE /admin/models/:id
POST   /admin/models/fetch          ← { "provider_config_id": "uuid|empty" }

Fetch with empty provider_config_id syncs ALL active global providers. Returns:

{
  "message": "models synced",
  "added": 5,
  "updated": 12,
  "total": 47
}

Or for multi-provider fetch: { "added", "updated", "total", "errors": [...] }.

Bulk visibility sets all models for a provider (or all models globally if no provider_config_id) to the specified visibility (enabled, disabled, team).

Provider Health

Real-time health tracking per provider config. The health accumulator records success/failure/latency on every completion, flushes to DB every 60 seconds.

Get all:

GET /admin/providers/health

Returns { "data": [ProviderHealth objects] }:

{
  "provider_config_id": "uuid",
  "provider_config_name": "OpenAI Production",
  "status": "healthy|degraded|down",
  "error_rate": 0.02,
  "avg_latency_ms": 450,
  "timeout_rate": 0.01,
  "rate_limit_count": 3,
  "last_check": "...",
  "last_error": "...|null"
}

Get single:

GET /admin/providers/:id/health

Auto-disable: After PROVIDER_AUTO_DISABLE_THRESHOLD consecutive "down" windows (default: 3), the provider is automatically deactivated.

Capability Overrides

Admin can override any model capability detected by the catalog or heuristic layer.

GET    /admin/capability-overrides           → { "data": [...] }
GET    /admin/models/:id/capabilities        → capabilities for one model
PUT    /admin/models/:id/capabilities        ← { "field": "value" }
DELETE /admin/models/:id/capabilities/:overrideId

Override fields: supports_vision, supports_tools, supports_thinking, context_window, max_output_tokens, etc.

Routing Policies

Policy-based request routing. Evaluated after model/config resolution, before provider dispatch.

Admin CRUD:

GET    /admin/routing/policies       → { "data": [...] }
GET    /admin/routing/policies/:id   → policy object
POST   /admin/routing/policies       ← { "name", "scope", "team_id", "priority", "policy_type", "config", "is_active" }
PUT    /admin/routing/policies/:id
DELETE /admin/routing/policies/:id

Policy object:

{
  "id": "uuid",
  "name": "Prefer Anthropic",
  "scope": "global|team",
  "team_id": "uuid|null",
  "priority": 10,
  "policy_type": "provider_prefer|team_route|cost_limit|model_alias|capability_match",
  "config": {},
  "is_active": true
}

Policy type	Config	Behavior
`provider_prefer`	`{ "providers": ["cfg-1", "cfg-2"] }`	Ordered fallback list
`team_route`	`{ "providers": ["cfg-1"] }`	Restrict team to specific providers
`cost_limit`	`{ "max_cost_per_request": 0.50 }`	Heuristic cost cap
`model_alias`	`{ "alias": "fast", "target_model": "...", "target_config": "..." }`	Alias → provider+model rewrite
`capability_match`	`{ "require": ["tool_calling"], "prefer": "cheapest" }`	Match cheapest model with required capabilities

Dry-run test:

POST /admin/routing/test

{
  "model": "claude-sonnet-4-20250514",
  "user_id": "uuid",
  "team_id": "uuid|null"
}

Returns the ranked candidate list with health status for each.

Provider Types

Registry of supported provider types with metadata.

GET /admin/provider-types

Returns { "types": [...] } with name, display name, default endpoint, profile schema, and supported features per type.

5.8 KiB Raw Blame History