Feat v0.6.6 final hardening #41
42
CHANGELOG.md
42
CHANGELOG.md
@@ -2,6 +2,48 @@
|
||||
|
||||
All notable changes to Switchboard Core are documented here.
|
||||
|
||||
## v0.6.6 — Final Hardening
|
||||
|
||||
Final pass before public release. Security, correctness, and developer
|
||||
experience. No new features — only fixes, validation, and cleanup.
|
||||
|
||||
### Added
|
||||
|
||||
- **`ValidateManifest()` gate**: Centralized manifest validation function
|
||||
(`package_validate.go`) called at both upload-install and bundled-install
|
||||
time. Catches malformed manifests early with clear error messages. 12 unit
|
||||
tests covering all type constraints and edge cases.
|
||||
- **Extension dependency auto-activation**: Installing a package whose
|
||||
`dependencies` list an uninstalled library will auto-install that library
|
||||
from the bundled packages directory. If the dependency is not bundled, the
|
||||
error message lists exactly what's missing.
|
||||
- **`OptionalAuth` middleware**: New page middleware for workflow visitor
|
||||
routes. Authenticates if a token is present, allows anonymous pass-through
|
||||
otherwise. Replaces the stale `AuthOrRedirect` TODO for Session routes.
|
||||
- **Package signing schema reservation**: `signature` field accepted in
|
||||
manifest (no-op). `PACKAGE_VERIFY_SIGNATURES` env var logs warnings for
|
||||
unsigned packages when enabled. No cryptographic verification yet — schema
|
||||
slot reserved to prevent a breaking change later.
|
||||
- **ICD/SDK runner v0.6.x coverage**: Smoke tier adds metrics, cluster,
|
||||
backups, docs, and OpenAPI JSON endpoints. SDK admin domain adds metrics,
|
||||
cluster, and backups dual-path tests.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **OIDC nonce validation**: ID token nonce claim is now validated against
|
||||
the stored authorization request nonce. Previously the nonce was generated
|
||||
and stored but never checked on callback — a token replay/substitution
|
||||
vulnerability.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Schema migration stub**: `RunSchemaMigrations()` no longer pretends to
|
||||
work. Replaced with a log-only function documenting that only additive
|
||||
schema changes are supported. Downgrade rejection preserved.
|
||||
- **Session middleware TODO resolved**: `main.go` Session slot now uses
|
||||
`OptionalAuth` instead of `AuthOrRedirect`. OIDC nonce TODO and migration
|
||||
stub TODO also resolved.
|
||||
|
||||
## v0.6.5 — Renderer Pipeline + Docs Rewrite
|
||||
|
||||
Lifts block rendering to a kernel SDK primitive so all surfaces share one
|
||||
|
||||
16
ROADMAP.md
16
ROADMAP.md
@@ -1,6 +1,6 @@
|
||||
# Switchboard Core — Roadmap
|
||||
|
||||
## Current: v0.6.5 — Renderer Pipeline + Docs Rewrite
|
||||
## Current: v0.6.6 — Final Hardening
|
||||
|
||||
Self-hosted extensible platform. Auth, identity, packages, Starlark sandbox,
|
||||
storage, realtime, and ops are kernel primitives. Everything else is an extension.
|
||||
@@ -145,13 +145,13 @@ Final pass before public release. Security, correctness, and developer experienc
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Extension dependency auto-activation | ☐ | Installing a package with unmet `depends`/`requires` should auto-install dependencies from the bundled set, or reject with a clear error listing what's missing. Silent dormant is not acceptable for external users. |
|
||||
| `ValidateManifest()` gate | ☐ | Single validation function called at install time. Catches malformed manifests early — same pattern as Unicode scan gate. |
|
||||
| Package signing hook | ☐ | Optional `signature` field in manifest. `--verify` flag on install path (currently no-op). Five minutes now prevents a breaking manifest schema change post-fork. |
|
||||
| OIDC state nonce validation | ☐ | `handlers/auth.go:221` — security TODO. Validate nonce on OIDC callback before accepting tokens. |
|
||||
| Schema migration stub decision | ☐ | `handlers/starlark_helpers.go:93–97` — `RunSchemaMigrations` is a no-op stub called during updates. Implement real migrations or remove and document that only additive schema changes are supported. Don't ship a stub that pretends to work. |
|
||||
| ICD/SDK runner update pass | ☐ | ICD runner and SDK runner are stale for cluster, backup, chat, realtime, dynamic OpenAPI. Update to cover current API surface. |
|
||||
| Stale TODO resolution | ☐ | `main.go:867` (session middleware TODO, stale since v0.2.0). `auth.go:221` (OIDC nonce, covered above). `starlark_helpers.go:96` (migration stub, covered above). Resolve or delete. |
|
||||
| Extension dependency auto-activation | ✅ | Installing a package with unmet `depends`/`requires` auto-installs dependencies from the bundled set. If not bundled: clear error listing what's missing. |
|
||||
| `ValidateManifest()` gate | ✅ | Single `ValidateManifest()` function in `package_validate.go`. Called at install time (both upload and bundled). 12 unit tests. |
|
||||
| Package signing hook | ✅ | Optional `signature` field in manifest (reserved). `PACKAGE_VERIFY_SIGNATURES` env var (default false, log-only). No cryptographic code yet — schema slot reserved. |
|
||||
| OIDC state nonce validation | ✅ | `oidcClaims.Nonce` field added. `ValidateIDTokenNonce()` compares ID token nonce against stored state. Callback rejects mismatched nonces. |
|
||||
| Schema migration stub decision | ✅ | Stub replaced with log-only function documenting additive-only policy. Downgrade rejection preserved. |
|
||||
| ICD/SDK runner update pass | ✅ | ICD smoke tier: added metrics, cluster, backups, docs, OpenAPI JSON endpoints. SDK admin domain: added metrics, cluster, backups tests. |
|
||||
| Stale TODO resolution | ✅ | `main.go` session middleware: replaced with `OptionalAuth()` (auth if token present, anonymous pass-through). `auth.go:221` OIDC nonce: resolved. `starlark_helpers.go:96` migration stub: resolved. |
|
||||
|
||||
Then ship.
|
||||
|
||||
|
||||
@@ -442,6 +442,48 @@
|
||||
T.assertHasKey(d, 'data', '/admin/memories/pending');
|
||||
T.assert(Array.isArray(d.data), 'data should be array');
|
||||
});
|
||||
|
||||
// -- v0.6.x Admin endpoints --
|
||||
|
||||
await T.test('smoke', 'admin', 'GET /admin/metrics', async function () {
|
||||
var d = await T.apiGet('/admin/metrics');
|
||||
T.assert(typeof d === 'object', 'expected object');
|
||||
T.assertHasKey(d, 'runtime', '/admin/metrics');
|
||||
T.assertHasKey(d, 'database', '/admin/metrics');
|
||||
});
|
||||
|
||||
await T.test('smoke', 'admin', 'GET /admin/cluster', async function () {
|
||||
var d = await T.apiGet('/admin/cluster');
|
||||
T.assertHasKey(d, 'data', '/admin/cluster');
|
||||
T.assert(Array.isArray(d.data), 'cluster nodes should be array');
|
||||
});
|
||||
|
||||
await T.test('smoke', 'admin', 'GET /admin/backups', async function () {
|
||||
var d = await T.apiGet('/admin/backups');
|
||||
T.assertHasKey(d, 'data', '/admin/backups');
|
||||
T.assert(Array.isArray(d.data), 'backups should be array');
|
||||
});
|
||||
}
|
||||
|
||||
// -- Docs API --
|
||||
await T.test('smoke', 'utility', 'GET /docs', async function () {
|
||||
var d = await T.apiGet('/docs');
|
||||
T.assertHasKey(d, 'data', '/docs');
|
||||
T.assert(Array.isArray(d.data), 'docs should be array');
|
||||
});
|
||||
|
||||
// -- Dynamic OpenAPI spec (outside /api/v1 prefix) --
|
||||
await T.test('smoke', 'utility', 'GET /api/docs/openapi.json', async function () {
|
||||
var token = await T.getAuthToken();
|
||||
var resp = await fetch(T.base + '/api/docs/openapi.json', {
|
||||
headers: { 'Authorization': 'Bearer ' + token },
|
||||
credentials: 'same-origin'
|
||||
});
|
||||
T.assert(resp.ok, 'expected 200 from /api/docs/openapi.json, got ' + resp.status);
|
||||
var d = await resp.json();
|
||||
T.assertHasKey(d, 'openapi', 'openapi spec');
|
||||
T.assertHasKey(d, 'paths', 'openapi spec');
|
||||
T.assertHasKey(d, 'info', 'openapi spec');
|
||||
});
|
||||
};
|
||||
})();
|
||||
|
||||
@@ -149,5 +149,39 @@
|
||||
raw: { method: 'GET', path: '/admin/projects' },
|
||||
validate: function (r) { T.assert(Array.isArray(r) || (r && Array.isArray(r.data)), 'expected array'); }
|
||||
});
|
||||
|
||||
// ── Metrics (v0.6.4+) ──
|
||||
|
||||
await T.test('admin', 'metrics', 'GET /admin/metrics (raw)', {
|
||||
sdk: function () { return sw.api.get('/admin/metrics'); },
|
||||
raw: { method: 'GET', path: '/admin/metrics' },
|
||||
validate: function (r) {
|
||||
T.assert(typeof r === 'object', 'expected object');
|
||||
T.assertHasKey(r, 'runtime', 'metrics');
|
||||
T.assertHasKey(r, 'database', 'metrics');
|
||||
}
|
||||
});
|
||||
|
||||
// ── Cluster (v0.6.0+) ──
|
||||
|
||||
await T.test('admin', 'cluster', 'GET /admin/cluster (raw)', {
|
||||
sdk: function () { return sw.api.get('/admin/cluster'); },
|
||||
raw: { method: 'GET', path: '/admin/cluster' },
|
||||
validate: function (r) {
|
||||
var arr = T.unwrapList(r);
|
||||
T.assert(arr.length >= 0, 'expected cluster node list');
|
||||
}
|
||||
});
|
||||
|
||||
// ── Backups (v0.6.1+) ──
|
||||
|
||||
await T.test('admin', 'backups', 'GET /admin/backups (raw)', {
|
||||
sdk: function () { return sw.api.get('/admin/backups'); },
|
||||
raw: { method: 'GET', path: '/admin/backups' },
|
||||
validate: function (r) {
|
||||
var arr = T.unwrapList(r);
|
||||
T.assert(arr.length >= 0, 'expected backup list');
|
||||
}
|
||||
});
|
||||
});
|
||||
})();
|
||||
|
||||
@@ -246,6 +246,7 @@ type oidcClaims struct {
|
||||
Email string `json:"email"`
|
||||
PreferredUsername string `json:"preferred_username"`
|
||||
Name string `json:"name"`
|
||||
Nonce string `json:"nonce"`
|
||||
Groups []string `json:"groups"`
|
||||
RealmAccess realmAccess `json:"realm_access"`
|
||||
ResourceAccess interface{} `json:"resource_access"`
|
||||
@@ -304,6 +305,23 @@ func (p *OIDCProvider) validateToken(tokenString string) (*oidcClaims, error) {
|
||||
return claims, nil
|
||||
}
|
||||
|
||||
// ValidateIDTokenNonce validates that the ID token's nonce claim matches the
|
||||
// stored nonce from the authorization request. This prevents token replay and
|
||||
// substitution attacks in the authorization code flow.
|
||||
func (p *OIDCProvider) ValidateIDTokenNonce(tokenString, expectedNonce string) error {
|
||||
if expectedNonce == "" {
|
||||
return nil // no nonce to validate (e.g. direct token flow)
|
||||
}
|
||||
claims, err := p.validateToken(tokenString)
|
||||
if err != nil {
|
||||
return fmt.Errorf("token validation failed: %w", err)
|
||||
}
|
||||
if claims.Nonce != expectedNonce {
|
||||
return fmt.Errorf("nonce mismatch: expected %q, got %q", expectedNonce, claims.Nonce)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// isIdPAdmin checks whether the OIDC claims include the configured admin role.
|
||||
func (p *OIDCProvider) isIdPAdmin(claims *oidcClaims) bool {
|
||||
for _, role := range claims.RealmAccess.Roles {
|
||||
|
||||
@@ -97,6 +97,12 @@ type Config struct {
|
||||
ClusterHeartbeatInterval time.Duration
|
||||
ClusterStaleThreshold time.Duration
|
||||
ClusterEndpoint string
|
||||
|
||||
// Package signing (future)
|
||||
// PACKAGE_VERIFY_SIGNATURES: when true, log warnings for unsigned packages.
|
||||
// No cryptographic verification yet — this reserves the manifest schema slot
|
||||
// and config plumbing so it can be implemented without a breaking change.
|
||||
PackageVerifySignatures bool
|
||||
}
|
||||
|
||||
// Load reads configuration from environment variables.
|
||||
@@ -161,6 +167,8 @@ func Load() *Config {
|
||||
ClusterHeartbeatInterval: getEnvDuration("CLUSTER_HEARTBEAT_INTERVAL", 10*time.Second),
|
||||
ClusterStaleThreshold: getEnvDuration("CLUSTER_STALE_THRESHOLD", 30*time.Second),
|
||||
ClusterEndpoint: getEnv("CLUSTER_ENDPOINT", ""),
|
||||
|
||||
PackageVerifySignatures: getEnvBool("PACKAGE_VERIFY_SIGNATURES", false),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -112,6 +112,8 @@ func (b *Bus) publishLocal(event Event) {
|
||||
// in separate goroutines. Useful for I/O-heavy handlers.
|
||||
// Also calls the broadcastHook asynchronously if set.
|
||||
func (b *Bus) PublishAsync(event Event) {
|
||||
b.publishCount.Add(1)
|
||||
|
||||
b.mu.RLock()
|
||||
var matched []Handler
|
||||
for pattern, subs := range b.subs {
|
||||
@@ -125,6 +127,7 @@ func (b *Bus) PublishAsync(event Event) {
|
||||
b.mu.RUnlock()
|
||||
|
||||
for _, h := range matched {
|
||||
b.deliverCount.Add(1)
|
||||
go h(event)
|
||||
}
|
||||
if hook != nil {
|
||||
|
||||
@@ -218,8 +218,8 @@ func (h *AuthHandler) OIDCCallback(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// Verify state (nonce + redirectTo retrieved but not yet validated — TODO)
|
||||
_, _, err := h.stores.GlobalConfig.ConsumeOIDCState(c.Request.Context(), state)
|
||||
// Verify state and retrieve nonce for ID token validation
|
||||
storedNonce, _, err := h.stores.GlobalConfig.ConsumeOIDCState(c.Request.Context(), state)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid or expired state"})
|
||||
return
|
||||
@@ -246,6 +246,15 @@ func (h *AuthHandler) OIDCCallback(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// Validate nonce in the ID token to prevent token replay/substitution
|
||||
if tokenResp.IDToken != "" && storedNonce != "" {
|
||||
if err := oidcProv.ValidateIDTokenNonce(tokenResp.IDToken, storedNonce); err != nil {
|
||||
log.Printf("[auth/oidc] nonce validation failed: %v", err)
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "nonce validation failed"})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Use the ID token (or access token) to authenticate via the provider
|
||||
// Temporarily set the Authorization header so Authenticate() can read it
|
||||
tokenToValidate := tokenResp.IDToken
|
||||
|
||||
154
server/handlers/package_validate.go
Normal file
154
server/handlers/package_validate.go
Normal file
@@ -0,0 +1,154 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
// validManifestID matches lowercase alphanumeric slugs with optional hyphens.
|
||||
// Duplicated from packages.go for use in standalone validation.
|
||||
var validManifestID = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}[a-z0-9]$`)
|
||||
|
||||
// ManifestInfo holds the parsed and validated fields from a manifest.json.
|
||||
type ManifestInfo struct {
|
||||
ID string
|
||||
Title string
|
||||
Type string // surface, extension, full, workflow, library
|
||||
Version string
|
||||
Description string
|
||||
Author string
|
||||
Tier string
|
||||
SchemaVersion int
|
||||
HasRoute bool
|
||||
HasTools bool
|
||||
HasPipes bool
|
||||
HasHooks bool
|
||||
HasSettings bool
|
||||
HasExports bool
|
||||
Dependencies map[string]any
|
||||
Requires []string
|
||||
Signature string // reserved for future package signing
|
||||
}
|
||||
|
||||
// ValidateManifest parses a manifest map and validates all required fields,
|
||||
// type constraints, and structural rules. Returns a ManifestInfo or an error
|
||||
// describing the first validation failure.
|
||||
func ValidateManifest(manifest map[string]any) (*ManifestInfo, error) {
|
||||
if manifest == nil {
|
||||
return nil, fmt.Errorf("manifest is nil")
|
||||
}
|
||||
|
||||
info := &ManifestInfo{}
|
||||
|
||||
// ── Required fields ──────────────────────────────────────────
|
||||
info.ID, _ = manifest["id"].(string)
|
||||
info.Title, _ = manifest["title"].(string)
|
||||
if info.ID == "" || info.Title == "" {
|
||||
return nil, fmt.Errorf("manifest must have 'id' and 'title'")
|
||||
}
|
||||
|
||||
// ── Package ID slug validation ───────────────────────────────
|
||||
if !validManifestID.MatchString(info.ID) {
|
||||
return nil, fmt.Errorf("package id must be a lowercase slug (a-z, 0-9, hyphens, 2-64 chars)")
|
||||
}
|
||||
|
||||
// ── Type validation ──────────────────────────────────────────
|
||||
info.Type, _ = manifest["type"].(string)
|
||||
if info.Type == "" {
|
||||
info.Type = "surface" // backward compat
|
||||
}
|
||||
validTypes := map[string]bool{
|
||||
"surface": true, "extension": true, "full": true,
|
||||
"workflow": true, "library": true,
|
||||
}
|
||||
if !validTypes[info.Type] {
|
||||
return nil, fmt.Errorf("manifest type must be 'surface', 'extension', 'full', 'workflow', or 'library'")
|
||||
}
|
||||
|
||||
// ── Extract optional fields ──────────────────────────────────
|
||||
info.Version, _ = manifest["version"].(string)
|
||||
if info.Version == "" {
|
||||
info.Version = "0.0.0"
|
||||
}
|
||||
info.Description, _ = manifest["description"].(string)
|
||||
info.Author, _ = manifest["author"].(string)
|
||||
info.Tier, _ = manifest["tier"].(string)
|
||||
if info.Tier == "" {
|
||||
info.Tier = "browser"
|
||||
}
|
||||
info.Signature, _ = manifest["signature"].(string)
|
||||
|
||||
info.HasRoute = manifest["route"] != nil || manifest["routes"] != nil
|
||||
info.HasTools = manifest["tools"] != nil
|
||||
info.HasPipes = manifest["pipes"] != nil
|
||||
info.HasHooks = manifest["hooks"] != nil
|
||||
info.HasSettings = manifest["settings"] != nil
|
||||
info.HasExports = false
|
||||
|
||||
if deps, ok := manifest["dependencies"].(map[string]any); ok {
|
||||
info.Dependencies = deps
|
||||
}
|
||||
if reqs, ok := manifest["requires"].([]any); ok {
|
||||
for _, r := range reqs {
|
||||
if s, ok := r.(string); ok && s != "" {
|
||||
info.Requires = append(info.Requires, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
info.SchemaVersion = ParseSchemaVersion(manifest)
|
||||
|
||||
// ── Type-specific constraints ────────────────────────────────
|
||||
hasExtBehavior := info.HasTools || info.HasPipes || info.HasHooks
|
||||
|
||||
switch info.Type {
|
||||
case "surface":
|
||||
// route is optional for surfaces
|
||||
case "extension":
|
||||
if !hasExtBehavior {
|
||||
return nil, fmt.Errorf("extension packages require at least one of: tools, pipes, hooks")
|
||||
}
|
||||
if info.HasRoute {
|
||||
return nil, fmt.Errorf("extension packages cannot have a route — use type 'full' for both")
|
||||
}
|
||||
case "full":
|
||||
if !info.HasRoute {
|
||||
return nil, fmt.Errorf("full packages require a route")
|
||||
}
|
||||
if !hasExtBehavior && !info.HasSettings {
|
||||
return nil, fmt.Errorf("full packages require at least one of: tools, pipes, hooks, settings")
|
||||
}
|
||||
case "workflow":
|
||||
if manifest["workflow_definition"] == nil {
|
||||
return nil, fmt.Errorf("workflow packages require a 'workflow_definition' in the manifest")
|
||||
}
|
||||
case "library":
|
||||
exports, _ := manifest["exports"].([]any)
|
||||
if len(exports) == 0 {
|
||||
return nil, fmt.Errorf("library packages require an 'exports' array")
|
||||
}
|
||||
info.HasExports = true
|
||||
if info.HasTools || info.HasPipes {
|
||||
return nil, fmt.Errorf("library packages cannot have tools or pipes")
|
||||
}
|
||||
if info.HasRoute {
|
||||
return nil, fmt.Errorf("library packages cannot have a route")
|
||||
}
|
||||
}
|
||||
|
||||
return info, nil
|
||||
}
|
||||
|
||||
// ValidateManifestJSON parses raw JSON bytes into a manifest map and validates.
|
||||
func ValidateManifestJSON(data []byte) (map[string]any, *ManifestInfo, error) {
|
||||
var manifest map[string]any
|
||||
if err := json.Unmarshal(data, &manifest); err != nil {
|
||||
return nil, nil, fmt.Errorf("invalid manifest.json: %w", err)
|
||||
}
|
||||
info, err := ValidateManifest(manifest)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
return manifest, info, nil
|
||||
}
|
||||
166
server/handlers/package_validate_test.go
Normal file
166
server/handlers/package_validate_test.go
Normal file
@@ -0,0 +1,166 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestValidateManifest_ValidSurface(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "my-surface",
|
||||
"title": "My Surface",
|
||||
"type": "surface",
|
||||
}
|
||||
info, err := ValidateManifest(m)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if info.ID != "my-surface" {
|
||||
t.Errorf("expected id 'my-surface', got %q", info.ID)
|
||||
}
|
||||
if info.Type != "surface" {
|
||||
t.Errorf("expected type 'surface', got %q", info.Type)
|
||||
}
|
||||
if info.Version != "0.0.0" {
|
||||
t.Errorf("expected default version '0.0.0', got %q", info.Version)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_MissingID(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"title": "No ID",
|
||||
}
|
||||
_, err := ValidateManifest(m)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for missing id")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_MissingTitle(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "no-title",
|
||||
}
|
||||
_, err := ValidateManifest(m)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for missing title")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_InvalidID(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "UPPERCASE-BAD",
|
||||
"title": "Bad ID",
|
||||
}
|
||||
_, err := ValidateManifest(m)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for invalid package id")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_InvalidType(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "my-pkg",
|
||||
"title": "My Package",
|
||||
"type": "invalid",
|
||||
}
|
||||
_, err := ValidateManifest(m)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for invalid type")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_ExtensionNoTools(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "my-ext",
|
||||
"title": "My Extension",
|
||||
"type": "extension",
|
||||
}
|
||||
_, err := ValidateManifest(m)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for extension without tools/pipes/hooks")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_LibraryNoExports(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "my-lib",
|
||||
"title": "My Library",
|
||||
"type": "library",
|
||||
}
|
||||
_, err := ValidateManifest(m)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for library without exports")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_ValidLibrary(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "my-lib",
|
||||
"title": "My Library",
|
||||
"type": "library",
|
||||
"version": "1.2.0",
|
||||
"exports": []any{"module_a", "module_b"},
|
||||
}
|
||||
info, err := ValidateManifest(m)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if info.Version != "1.2.0" {
|
||||
t.Errorf("expected version '1.2.0', got %q", info.Version)
|
||||
}
|
||||
if !info.HasExports {
|
||||
t.Error("expected HasExports to be true")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_NilManifest(t *testing.T) {
|
||||
_, err := ValidateManifest(nil)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for nil manifest")
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_SignatureField(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "my-pkg",
|
||||
"title": "Signed Package",
|
||||
"type": "surface",
|
||||
"signature": "sha256:abc123",
|
||||
}
|
||||
info, err := ValidateManifest(m)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if info.Signature != "sha256:abc123" {
|
||||
t.Errorf("expected signature 'sha256:abc123', got %q", info.Signature)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_Dependencies(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "consumer",
|
||||
"title": "Consumer Package",
|
||||
"type": "surface",
|
||||
"dependencies": map[string]any{
|
||||
"chat-core": ">=1.0.0",
|
||||
},
|
||||
}
|
||||
info, err := ValidateManifest(m)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(info.Dependencies) != 1 {
|
||||
t.Errorf("expected 1 dependency, got %d", len(info.Dependencies))
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateManifest_WorkflowNoDef(t *testing.T) {
|
||||
m := map[string]any{
|
||||
"id": "my-wf",
|
||||
"title": "My Workflow",
|
||||
"type": "workflow",
|
||||
}
|
||||
_, err := ValidateManifest(m)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for workflow without workflow_definition")
|
||||
}
|
||||
}
|
||||
@@ -29,10 +29,11 @@ var validPackageID = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}[a-z0-9]$`)
|
||||
// PackageHandler manages unified package lifecycle (admin-only).
|
||||
// Replaces SurfaceHandler.
|
||||
type PackageHandler struct {
|
||||
stores store.Stores
|
||||
packagesDir string // e.g. /data/packages — where archives are extracted
|
||||
sandbox *sandbox.Sandbox
|
||||
runner *sandbox.Runner
|
||||
stores store.Stores
|
||||
packagesDir string // e.g. /data/packages — where archives are extracted
|
||||
bundledDir string // e.g. /app/bundled-packages — .pkg archive source
|
||||
sandbox *sandbox.Sandbox
|
||||
runner *sandbox.Runner
|
||||
}
|
||||
|
||||
func NewPackageHandler(s store.Stores, packagesDir ...string) *PackageHandler {
|
||||
@@ -43,6 +44,12 @@ func NewPackageHandler(s store.Stores, packagesDir ...string) *PackageHandler {
|
||||
return &PackageHandler{stores: s, packagesDir: dir}
|
||||
}
|
||||
|
||||
// SetBundledDir sets the path to the bundled packages directory.
|
||||
// Used by dependency auto-activation to resolve missing dependencies.
|
||||
func (h *PackageHandler) SetBundledDir(dir string) {
|
||||
h.bundledDir = dir
|
||||
}
|
||||
|
||||
// SetSandbox attaches a Starlark sandbox for schema migrations.
|
||||
func (h *PackageHandler) SetSandbox(sb *sandbox.Sandbox) {
|
||||
h.sandbox = sb
|
||||
@@ -319,79 +326,22 @@ func (h *PackageHandler) InstallPackage(c *gin.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
// Validate required fields
|
||||
pkgID, _ := manifest["id"].(string)
|
||||
title, _ := manifest["title"].(string)
|
||||
if pkgID == "" || title == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "manifest must have 'id' and 'title'"})
|
||||
// Validate manifest structure (required fields, type, constraints)
|
||||
mInfo, err := ValidateManifest(manifest)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
pkgID := mInfo.ID
|
||||
title := mInfo.Title
|
||||
pkgType := mInfo.Type
|
||||
|
||||
// Validate package ID is a safe slug
|
||||
if !validPackageID.MatchString(pkgID) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "package id must be a lowercase slug (a-z, 0-9, hyphens, 2-64 chars)"})
|
||||
return
|
||||
}
|
||||
|
||||
// Determine type (default: surface for backward compat)
|
||||
pkgType, _ := manifest["type"].(string)
|
||||
if pkgType == "" {
|
||||
pkgType = "surface"
|
||||
}
|
||||
if pkgType != "surface" && pkgType != "extension" && pkgType != "full" && pkgType != "workflow" && pkgType != "library" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "manifest type must be 'surface', 'extension', 'full', 'workflow', or 'library'"})
|
||||
return
|
||||
}
|
||||
|
||||
// Type-specific validation
|
||||
hasRoute := manifest["route"] != nil || manifest["routes"] != nil
|
||||
hasTools := manifest["tools"] != nil
|
||||
hasPipes := manifest["pipes"] != nil
|
||||
hasHooks := manifest["hooks"] != nil
|
||||
hasSettings := manifest["settings"] != nil
|
||||
hasExtBehavior := hasTools || hasPipes || hasHooks
|
||||
|
||||
switch pkgType {
|
||||
case "surface":
|
||||
// route is optional for surfaces (core surfaces don't always have it in manifest)
|
||||
case "extension":
|
||||
if !hasExtBehavior {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "extension packages require at least one of: tools, pipes, hooks"})
|
||||
return
|
||||
}
|
||||
if hasRoute {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "extension packages cannot have a route — use type 'full' for both"})
|
||||
return
|
||||
}
|
||||
case "full":
|
||||
if !hasRoute {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "full packages require a route"})
|
||||
return
|
||||
}
|
||||
// or a settings schema. A surface-with-settings is a valid "full" package
|
||||
// (e.g. editor: browser-tier surface + admin-configurable settings).
|
||||
if !hasExtBehavior && !hasSettings {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "full packages require at least one of: tools, pipes, hooks, settings"})
|
||||
return
|
||||
}
|
||||
case "workflow":
|
||||
if manifest["workflow_definition"] == nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "workflow packages require a 'workflow_definition' in the manifest"})
|
||||
return
|
||||
}
|
||||
case "library":
|
||||
exports, _ := manifest["exports"].([]any)
|
||||
if len(exports) == 0 {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "library packages require an 'exports' array"})
|
||||
return
|
||||
}
|
||||
if hasTools || hasPipes {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "library packages cannot have tools or pipes"})
|
||||
return
|
||||
}
|
||||
if hasRoute {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "library packages cannot have a route"})
|
||||
return
|
||||
// Package signing hook (schema reserved — no crypto verification yet)
|
||||
if os.Getenv("PACKAGE_VERIFY_SIGNATURES") == "true" {
|
||||
if mInfo.Signature != "" {
|
||||
log.Printf("[packages] %s: signature field present (%s) — verification not yet implemented", pkgID, mInfo.Signature)
|
||||
} else {
|
||||
log.Printf("[packages] %s: unsigned package (PACKAGE_VERIFY_SIGNATURES is enabled)", pkgID)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -443,17 +393,11 @@ func (h *PackageHandler) InstallPackage(c *gin.Context) {
|
||||
log.Printf("[packages] Extracted %s to %s", pkgID, destDir)
|
||||
}
|
||||
|
||||
// Extract manifest fields for DB columns
|
||||
version, _ := manifest["version"].(string)
|
||||
if version == "" {
|
||||
version = "0.0.0"
|
||||
}
|
||||
description, _ := manifest["description"].(string)
|
||||
author, _ := manifest["author"].(string)
|
||||
tier, _ := manifest["tier"].(string)
|
||||
if tier == "" {
|
||||
tier = "browser"
|
||||
}
|
||||
// Use validated manifest fields for DB columns
|
||||
version := mInfo.Version
|
||||
description := mInfo.Description
|
||||
author := mInfo.Author
|
||||
tier := mInfo.Tier
|
||||
|
||||
userID := c.GetString("user_id")
|
||||
|
||||
@@ -543,6 +487,7 @@ func (h *PackageHandler) InstallPackage(c *gin.Context) {
|
||||
}
|
||||
|
||||
// Libraries must be installed first; consumers declare them.
|
||||
// If a dependency is missing but available as a bundled package, auto-install it.
|
||||
if deps, ok := manifest["dependencies"].(map[string]any); ok && len(deps) > 0 {
|
||||
// Clear stale dependency records from a previous install of the same consumer.
|
||||
if h.stores.Dependencies != nil {
|
||||
@@ -551,8 +496,25 @@ func (h *PackageHandler) InstallPackage(c *gin.Context) {
|
||||
for libID, vSpec := range deps {
|
||||
lib, err := h.stores.Packages.Get(c.Request.Context(), libID)
|
||||
if err != nil || lib == nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "dependency not found: " + libID})
|
||||
return
|
||||
// Attempt auto-install from bundled packages
|
||||
if h.bundledDir != "" {
|
||||
pkgPath := filepath.Join(h.bundledDir, libID+".pkg")
|
||||
if _, statErr := os.Stat(pkgPath); statErr == nil {
|
||||
log.Printf("[packages] auto-installing bundled dependency %s for %s", libID, pkgID)
|
||||
if _, installErr := installBundledPackage(pkgPath, h.packagesDir, h.stores, h.runner); installErr != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{
|
||||
"error": fmt.Sprintf("dependency %s auto-install failed: %v", libID, installErr),
|
||||
})
|
||||
return
|
||||
}
|
||||
// Re-fetch after install
|
||||
lib, err = h.stores.Packages.Get(c.Request.Context(), libID)
|
||||
}
|
||||
}
|
||||
if lib == nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "dependency not found: " + libID + " (not installed and not available as a bundled package)"})
|
||||
return
|
||||
}
|
||||
}
|
||||
if lib.Type != "library" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "dependency is not a library: " + libID})
|
||||
|
||||
@@ -146,10 +146,14 @@ func installBundledPackage(pkgPath, packagesDir string, stores store.Stores, run
|
||||
return "", err
|
||||
}
|
||||
|
||||
pkgID, _ := manifest["id"].(string)
|
||||
if pkgID == "" {
|
||||
return "", fmt.Errorf("manifest missing 'id'")
|
||||
// Validate manifest structure
|
||||
mInfo, err := ValidateManifest(manifest)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("invalid manifest: %w", err)
|
||||
}
|
||||
pkgID := mInfo.ID
|
||||
title := mInfo.Title
|
||||
pkgType := mInfo.Type
|
||||
|
||||
// Skip if already exists (admin uninstalled → stays uninstalled)
|
||||
existing, _ := stores.Packages.Get(ctx, pkgID)
|
||||
@@ -157,16 +161,6 @@ func installBundledPackage(pkgPath, packagesDir string, stores store.Stores, run
|
||||
return "skipped", nil
|
||||
}
|
||||
|
||||
title, _ := manifest["title"].(string)
|
||||
if title == "" {
|
||||
return "", fmt.Errorf("manifest missing 'title'")
|
||||
}
|
||||
|
||||
pkgType, _ := manifest["type"].(string)
|
||||
if pkgType == "" {
|
||||
pkgType = "surface"
|
||||
}
|
||||
|
||||
// Extract static assets to packagesDir/{id}/
|
||||
if packagesDir != "" {
|
||||
if err := extractPackageAssets(zr, packagesDir, pkgID); err != nil {
|
||||
@@ -174,17 +168,11 @@ func installBundledPackage(pkgPath, packagesDir string, stores store.Stores, run
|
||||
}
|
||||
}
|
||||
|
||||
// Extract manifest fields for DB columns
|
||||
version, _ := manifest["version"].(string)
|
||||
if version == "" {
|
||||
version = "0.0.0"
|
||||
}
|
||||
description, _ := manifest["description"].(string)
|
||||
author, _ := manifest["author"].(string)
|
||||
tier, _ := manifest["tier"].(string)
|
||||
if tier == "" {
|
||||
tier = "browser"
|
||||
}
|
||||
// Use validated manifest fields for DB columns
|
||||
version := mInfo.Version
|
||||
description := mInfo.Description
|
||||
author := mInfo.Author
|
||||
tier := mInfo.Tier
|
||||
|
||||
// Register in database via Seed (upsert)
|
||||
if err := stores.Packages.Seed(ctx, pkgID, title, "bundled", manifest); err != nil {
|
||||
|
||||
@@ -171,6 +171,7 @@ func TestBundledInstall_DormantPackage(t *testing.T) {
|
||||
"id": "dormant-pkg",
|
||||
"title": "Dormant Package",
|
||||
"type": "extension",
|
||||
"hooks": []string{"on_install"},
|
||||
"requires": []string{"chat"},
|
||||
})
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ package handlers
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
"go.starlark.net/starlark"
|
||||
)
|
||||
@@ -90,9 +91,13 @@ func ParseSchemaVersion(manifest any) int {
|
||||
return 0
|
||||
}
|
||||
|
||||
// RunSchemaMigrations runs extension schema migrations between versions.
|
||||
// Stub — full implementation will be added when extension DB schemas are stabilized.
|
||||
// RunSchemaMigrations logs a schema version change during package update.
|
||||
//
|
||||
// Only additive schema changes are supported: extensions can ADD columns and
|
||||
// tables via db_tables in the manifest, but cannot modify or delete existing
|
||||
// schema. If a migration between versions requires destructive changes,
|
||||
// uninstall and reinstall the package manually.
|
||||
func RunSchemaMigrations(ctx any, sandbox any, stores any, db any, isPostgres bool, packageID string, manifest any, fromVersion, toVersion int) error {
|
||||
// TODO: implement extension schema migrations
|
||||
log.Printf("[pkg-migrate] %s: schema version %d → %d (only additive changes are applied automatically; destructive migrations require manual uninstall/reinstall)", packageID, fromVersion, toVersion)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -451,6 +451,7 @@ func TestUpgrade_PackageDormantOnUnmetRequires(t *testing.T) {
|
||||
"title": "Future",
|
||||
"type": "extension",
|
||||
"version": "1.0.0",
|
||||
"hooks": []string{"on_install"},
|
||||
"requires": []string{"kernel>=99.0.0"},
|
||||
})
|
||||
|
||||
|
||||
@@ -764,6 +764,7 @@ func main() {
|
||||
}
|
||||
pkgAdm := handlers.NewPackageHandler(stores, packagesDir)
|
||||
pkgAdm.SetSandbox(sandbox.New(sandbox.DefaultConfig()))
|
||||
pkgAdm.SetBundledDir(cfg.BundledPackagesDir)
|
||||
|
||||
// Package registry — must be registered before /packages/:id
|
||||
registryH := handlers.NewRegistryHandler(stores, packagesDir, pkgAdm)
|
||||
@@ -867,7 +868,7 @@ func main() {
|
||||
middleware.AuthOrRedirect(cfg, stores.Users, userCache),
|
||||
middleware.RequireAdminPage(stores),
|
||||
},
|
||||
Session: middleware.AuthOrRedirect(cfg, stores.Users, userCache), // TODO: session middleware for workflow visitors
|
||||
Session: middleware.OptionalAuth(cfg, stores.Users, userCache),
|
||||
})
|
||||
|
||||
// Mounted at /s/:slug/api/* with JWT auth (returns 401, not redirect).
|
||||
|
||||
@@ -97,6 +97,64 @@ func RequireAdminPage(stores store.Stores) gin.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// OptionalAuth authenticates the user if a valid token is present (cookie,
|
||||
// header, or query param) but allows anonymous pass-through otherwise.
|
||||
// Use for page routes that should be accessible by both authenticated users
|
||||
// and anonymous visitors (e.g. public workflow entry pages).
|
||||
func OptionalAuth(cfg *config.Config, users store.UserStore, cache *UserStatusCache) gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
if !database.IsConnected() {
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
|
||||
tokenString := ""
|
||||
if cookie, err := c.Cookie("sb_token"); err == nil && cookie != "" {
|
||||
tokenString = cookie
|
||||
}
|
||||
if tokenString == "" {
|
||||
header := c.GetHeader("Authorization")
|
||||
if strings.HasPrefix(header, "Bearer ") {
|
||||
tokenString = strings.TrimPrefix(header, "Bearer ")
|
||||
}
|
||||
}
|
||||
if tokenString == "" {
|
||||
tokenString = c.Query("token")
|
||||
}
|
||||
|
||||
// No token → anonymous pass-through
|
||||
if tokenString == "" {
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
|
||||
claims, ok := parseAndValidateJWT(tokenString, cfg.JWTSecret)
|
||||
if !ok {
|
||||
// Invalid token → treat as anonymous
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
|
||||
if claims.UserID != "" {
|
||||
if entry, hit := cache.get(claims.UserID); hit {
|
||||
if entry.isActive {
|
||||
c.Set("user_id", claims.UserID)
|
||||
c.Set("email", claims.Email)
|
||||
}
|
||||
} else {
|
||||
user, err := users.GetByID(c.Request.Context(), claims.UserID)
|
||||
if err == nil && user.IsActive {
|
||||
cache.set(claims.UserID, true)
|
||||
c.Set("user_id", claims.UserID)
|
||||
c.Set("email", claims.Email)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
c.Next()
|
||||
}
|
||||
}
|
||||
|
||||
func redirectToLogin(c *gin.Context, loginPath string) {
|
||||
// Save intended destination for post-login redirect
|
||||
intended := c.Request.URL.Path
|
||||
|
||||
@@ -48,10 +48,12 @@ function formatNum(v) {
|
||||
|
||||
// ── Stat card ───────────────────────────────────
|
||||
|
||||
function Stat({ label, value }) {
|
||||
function Stat({ label, value, hint }) {
|
||||
return html`
|
||||
<div style="padding:8px 12px;background:var(--bg-secondary, #f5f5f5);border-radius:6px;min-width:120px;">
|
||||
<div style="font-size:11px;color:var(--text-muted, #888);margin-bottom:2px;">${label}</div>
|
||||
<div style="padding:8px 12px;background:var(--bg-secondary, #f5f5f5);border-radius:6px;min-width:120px;" title=${hint || ''}>
|
||||
<div style="font-size:11px;color:var(--text-muted, #888);margin-bottom:2px;">
|
||||
${label}${hint ? html`<span style="margin-left:3px;cursor:help;opacity:0.5;" title=${hint}>\u24D8</span>` : ''}
|
||||
</div>
|
||||
<div style="font-size:16px;font-weight:600;font-variant-numeric:tabular-nums;">${value}</div>
|
||||
</div>
|
||||
`;
|
||||
@@ -148,8 +150,8 @@ function ExtensionPanel({ m }) {
|
||||
<${Stat} label="Starlark Errors" value=${formatNum(m.starlark_errors_total)} />
|
||||
<${Stat} label="Avg Duration" value=${formatMs(m.starlark_avg_duration_ms)} />
|
||||
<${Stat} label="Trigger Fires" value=${formatNum(m.trigger_fires_total)} />
|
||||
<${Stat} label="Events Published" value=${formatNum(m.event_bus_published)} />
|
||||
<${Stat} label="Events Delivered" value=${formatNum(m.event_bus_delivered)} />
|
||||
<${Stat} label="Events Published" value=${formatNum(m.event_bus_published)} hint="Total events emitted by the event bus. Not all events have active subscribers \u2014 published > delivered is normal (e.g. startup events fire before any clients connect)." />
|
||||
<${Stat} label="Events Delivered" value=${formatNum(m.event_bus_delivered)} hint="Events that matched at least one subscriber and were dispatched. Lower than published when events fire with no listeners (e.g. during startup)." />
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
Reference in New Issue
Block a user