Feat v0.6.4 health metrics (#39)
All checks were successful
CI/CD / detect-changes (push) Successful in 3s
CI/CD / test-frontend (push) Successful in 6s
CI/CD / test-go-pg (push) Successful in 2m42s
CI/CD / test-sqlite (push) Successful in 2m48s
CI/CD / build-and-deploy (push) Successful in 1m5s

Co-authored-by: Jeffrey Smith <jasafpro@gmail.com>
Co-committed-by: Jeffrey Smith <jasafpro@gmail.com>
This commit was merged in pull request #39.
This commit is contained in:
2026-03-31 14:05:49 +00:00
committed by xcaliber
parent 3d4228f868
commit 36d6158940
22 changed files with 937 additions and 286 deletions

View File

@@ -2,6 +2,50 @@
All notable changes to Switchboard Core are documented here. All notable changes to Switchboard Core are documented here.
## v0.6.4 — Admin Health/Metrics Tab + Cluster Merge
Structural consolidation: cluster dashboard merges into Admin as a Health tab.
Comprehensive metrics endpoint for runtime, DB, cluster, and extension stats.
### Added
- **Admin Health tab**: New "Health" section under Monitoring in the Admin
surface. Auto-refreshing panels for runtime, database, cluster, and extension
metrics with configurable poll interval (5/10/30/60s).
- **`GET /api/v1/admin/metrics`**: Single JSON endpoint returning all platform
metrics — runtime (goroutines, heap, GC, uptime, FDs), DB pool (latency,
active/idle/max, PG-only dead tuples + active backends), cluster nodes
(when PG multi-node), and extension stats (Starlark exec/errors/duration,
trigger fires, event bus publish/deliver counts).
- **Fattened heartbeat payload**: Cluster heartbeat JSONB now carries stack
usage, GC CPU%, extensions loaded, Starlark execution counters, and trigger
fire count — visible in per-node cluster cards.
- **Sandbox execution tracking**: `ExecPackage` and `CallEntryPoint` now
increment atomic counters (exec, errors, duration) and the previously
declared but unused `SandboxExecutionsTotal` Prometheus counter.
- **Event bus counters**: `Bus.PublishCount()` and `Bus.DeliverCount()` track
cumulative publish/deliver operations.
- **Trigger fire counter**: `Engine.FireCount()` tracks cumulative trigger
fires across webhook, event, and scheduled triggers.
- 4 new handler tests (metrics SQLite shape, cluster shape, extension counters).
- 1 new cluster test (fattened heartbeat payload).
### Changed
- **Health endpoint consolidation**: `/health` and `/api/v1/health` now use a
shared `buildHealthResponse()` function returning identical JSON. Both now
include `registration_enabled` when the database is connected.
- **Block renderer `requires` removed**: `mermaid-renderer`, `katex-renderer`,
`csv-table`, and `diff-viewer` no longer require `["chat"]` — they activate
on any surface (notes, docs, etc.).
### Removed
- **`cluster-dashboard` package**: Standalone surface package retired. Cluster
node visibility is now in Admin > Monitoring > Health.
---
## v0.6.3 — Dead Code Sweep + Registry Fix ## v0.6.3 — Dead Code Sweep + Registry Fix
Pre-fork hardening: fix broken registry install, add registry settings UI, Pre-fork hardening: fix broken registry install, add registry settings UI,

View File

@@ -1,6 +1,6 @@
# Switchboard Core — Roadmap # Switchboard Core — Roadmap
## Current: v0.6.3Dead Code Sweep + Registry Fix ## Current: v0.6.4Admin Health/Metrics Tab + Cluster Merge
Self-hosted extensible platform. Auth, identity, packages, Starlark sandbox, Self-hosted extensible platform. Auth, identity, packages, Starlark sandbox,
storage, realtime, and ops are kernel primitives. Everything else is an extension. storage, realtime, and ops are kernel primitives. Everything else is an extension.
@@ -98,15 +98,15 @@ Structural move: cluster dashboard becomes an Admin tab. Better home for health/
| Step | Status | Description | | Step | Status | Description |
|------|--------|-------------| |------|--------|-------------|
| "Health / Metrics" admin tab | | New tab in Admin surface. DB-agnostic metrics for all deployments. Cluster cards conditional on PG + multi-node detection. | | "Health / Metrics" admin tab | | New tab in Admin surface. DB-agnostic metrics for all deployments. Cluster cards conditional on PG + multi-node detection. |
| Runtime metrics | | Per-node: goroutines, heap alloc/sys, stack in use, GC cycles, last GC pause, GC CPU %, uptime, WebSocket clients, extensions loaded, open FDs. | | Runtime metrics | | Per-node: goroutines, heap alloc/sys, stack in use, GC cycles, last GC pause, GC CPU %, uptime, WebSocket clients, extensions loaded, open FDs. |
| DB pool metrics | | All deployments: DB latency (`SELECT 1` round-trip), pool active/idle/max, wait count, wait duration. PG-only: table bloat (`n_dead_tup`), active backends (`pg_stat_activity`). | | DB pool metrics | | All deployments: DB latency (`SELECT 1` round-trip), pool active/idle/max, wait count, wait duration. PG-only: table bloat (`n_dead_tup`), active backends (`pg_stat_activity`). |
| Cluster metrics | | PG multi-node only: cluster size, peer list with endpoint + uptime, heartbeat age per node, event bus publish/deliver rates. | | Cluster metrics | | PG multi-node only: cluster size, peer list with endpoint + uptime, heartbeat age per node, event bus publish/deliver rates. |
| Extension runtime metrics | | Starlark exec/min, errors/min, avg duration, HTTP outbound requests/min, trigger fires/min, schedule overruns. | | Extension runtime metrics | | Starlark exec/min, errors/min, avg duration, HTTP outbound requests/min, trigger fires/min, schedule overruns. |
| Fatten heartbeat payload | | Heartbeat JSONB carries full metric set. `GET /api/v1/admin/metrics` for single-node SQLite fallback (same shape). | | Fatten heartbeat payload | | Heartbeat JSONB carries full metric set. `GET /api/v1/admin/metrics` for single-node SQLite fallback (same shape). |
| Retire `cluster-dashboard` | | Remove package once Admin Health tab ships. Update `defaultBundledPackages`. | | Retire `cluster-dashboard` | | Remove package once Admin Health tab ships. Update `defaultBundledPackages`. |
| Fix block renderer `requires` | | `mermaid-renderer`, `katex-renderer`, `csv-table`, `diff-viewer` all have `"requires": ["chat"]`. These are content renderers, not chat features. Remove constraint — they should activate without chat. | | Fix block renderer `requires` | | `mermaid-renderer`, `katex-renderer`, `csv-table`, `diff-viewer` all have `"requires": ["chat"]`. These are content renderers, not chat features. Remove constraint — they should activate without chat. |
| Health endpoint consolidation | | `/health` and `/api/v1/health` return near-identical JSON. Merge or clearly differentiate with docs. | | Health endpoint consolidation | | `/health` and `/api/v1/health` return near-identical JSON. Merge or clearly differentiate with docs. |
### v0.6.5 — Renderer Pipeline + Docs Rewrite ### v0.6.5 — Renderer Pipeline + Docs Rewrite

View File

@@ -1 +1 @@
0.6.3 0.6.4

View File

@@ -1,100 +0,0 @@
.cluster-dashboard {
max-width: 1200px;
margin: 0 auto;
padding: 2rem 1.5rem;
}
.cluster-header {
display: flex;
align-items: center;
gap: 1rem;
margin-bottom: 1.5rem;
}
.cluster-header h1 {
margin: 0;
font-size: 1.5rem;
color: var(--text-primary, #111);
}
.cluster-status {
font-size: 0.85rem;
color: var(--text-secondary, #666);
background: var(--bg-secondary, #f3f4f6);
padding: 0.25rem 0.75rem;
border-radius: 999px;
}
.cluster-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(320px, 1fr));
gap: 1rem;
}
.cluster-card {
background: var(--bg-primary, #fff);
border: 1px solid var(--border, #e5e7eb);
border-radius: 8px;
padding: 1.25rem;
}
.card-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 0.5rem;
}
.node-id {
font-weight: 600;
font-size: 0.9rem;
font-family: var(--font-mono, monospace);
color: var(--text-primary, #111);
}
.heartbeat-badge {
font-size: 0.75rem;
color: var(--text-secondary, #666);
background: var(--bg-secondary, #f3f4f6);
padding: 0.15rem 0.5rem;
border-radius: 4px;
}
.node-endpoint {
font-size: 0.8rem;
color: var(--text-secondary, #666);
margin-bottom: 0.75rem;
font-family: var(--font-mono, monospace);
}
.card-stats {
display: grid;
grid-template-columns: 1fr 1fr 1fr;
gap: 0.5rem;
}
.stat {
display: flex;
flex-direction: column;
gap: 0.15rem;
}
.stat-label {
font-size: 0.7rem;
text-transform: uppercase;
letter-spacing: 0.03em;
color: var(--text-secondary, #999);
}
.stat-value {
font-size: 0.9rem;
font-weight: 500;
font-family: var(--font-mono, monospace);
color: var(--text-primary, #111);
}
.cluster-empty {
color: var(--text-secondary, #666);
text-align: center;
padding: 3rem 1rem;
}

View File

@@ -1,124 +0,0 @@
/**
* Cluster Dashboard — admin surface showing registered cluster nodes.
*
* Polls GET /api/v1/admin/cluster every 10s. Renders one card per node
* with runtime stats from the JSONB stats column. New keys added to
* stats are rendered automatically — no code change required.
*/
(function () {
'use strict';
var mount = document.getElementById('extension-mount');
if (!mount) return;
var REFRESH_MS = 10000;
var timer = null;
mount.innerHTML =
'<div class="cluster-dashboard">' +
'<div class="cluster-header">' +
'<h1>Cluster</h1>' +
'<span class="cluster-status" id="clusterStatus">loading...</span>' +
'</div>' +
'<div class="cluster-grid" id="clusterGrid"></div>' +
'</div>';
load();
timer = setInterval(load, REFRESH_MS);
// Cleanup on navigation (SPA)
window.addEventListener('beforeunload', function () {
if (timer) clearInterval(timer);
});
function load() {
var api = (window.sw && window.sw.api) || (typeof API !== 'undefined' && API);
if (!api) return;
var get = api._get || api.get;
if (!get) return;
get.call(api, '/api/v1/admin/cluster').then(render).catch(function () {
// 404 = SQLite (no cluster route registered), or other fetch error
render({ data: [] });
});
}
function render(resp) {
var nodes = (resp && resp.data) || [];
var status = document.getElementById('clusterStatus');
var grid = document.getElementById('clusterGrid');
status.textContent = nodes.length + ' node' + (nodes.length !== 1 ? 's' : '') + ' registered';
if (nodes.length === 0) {
grid.innerHTML = '<p class="cluster-empty">No nodes registered. The cluster registry requires PostgreSQL — SQLite runs single-node only.</p>';
return;
}
grid.innerHTML = nodes.map(nodeCard).join('');
}
function nodeCard(node) {
var stats = {};
try { stats = typeof node.stats === 'string' ? JSON.parse(node.stats) : (node.stats || {}); } catch (e) { /* ignore */ }
var uptimeSec = stats.uptime_sec || 0;
var uptime = formatDuration(uptimeSec);
var heap = formatBytes(stats.heap_alloc || 0);
var gcPause = ((stats.gc_pause_ns || 0) / 1e6).toFixed(1) + ' ms';
var wsClients = stats.ws_clients != null ? stats.ws_clients : '-';
var goroutines = stats.goroutines != null ? stats.goroutines : '-';
var gcCycles = stats.gc_cycles != null ? stats.gc_cycles : '-';
var heartbeatAge = timeSince(node.heartbeat);
return (
'<div class="cluster-card">' +
'<div class="card-header">' +
'<span class="node-id">' + esc(node.node_id) + '</span>' +
'<span class="heartbeat-badge" title="Last heartbeat">' + heartbeatAge + '</span>' +
'</div>' +
(node.endpoint ? '<div class="node-endpoint">' + esc(node.endpoint) + '</div>' : '') +
'<div class="card-stats">' +
stat('Uptime', uptime) +
stat('WS Clients', wsClients) +
stat('Goroutines', goroutines) +
stat('Heap', heap) +
stat('GC Pause', gcPause) +
stat('GC Cycles', gcCycles) +
'</div>' +
'</div>'
);
}
function stat(label, value) {
return '<div class="stat"><span class="stat-label">' + label + '</span><span class="stat-value">' + value + '</span></div>';
}
function formatBytes(b) {
if (b < 1024) return b + ' B';
if (b < 1024 * 1024) return (b / 1024).toFixed(1) + ' KB';
return (b / (1024 * 1024)).toFixed(1) + ' MB';
}
function formatDuration(sec) {
if (sec < 60) return Math.floor(sec) + 's';
if (sec < 3600) return Math.floor(sec / 60) + 'm ' + Math.floor(sec % 60) + 's';
var h = Math.floor(sec / 3600);
var m = Math.floor((sec % 3600) / 60);
return h + 'h ' + m + 'm';
}
function timeSince(iso) {
if (!iso) return '-';
var ms = Date.now() - new Date(iso).getTime();
if (ms < 0) ms = 0;
var sec = Math.floor(ms / 1000);
if (sec < 60) return sec + 's ago';
return Math.floor(sec / 60) + 'm ago';
}
function esc(s) {
var d = document.createElement('div');
d.appendChild(document.createTextNode(s || ''));
return d.innerHTML;
}
})();

View File

@@ -1,13 +0,0 @@
{
"id": "cluster-dashboard",
"icon": "🔗",
"type": "surface",
"title": "Cluster",
"route": "/s/cluster-dashboard",
"auth": "admin",
"layout": "single",
"components": [],
"hooks": ["surface"],
"version": "0.6.0",
"description": "Cluster node registry — health, runtime stats, and peer visibility."
}

View File

@@ -6,7 +6,7 @@
"tier": "browser", "tier": "browser",
"author": "switchboard", "author": "switchboard",
"description": "Renders ```csv code blocks as sortable, interactive HTML tables", "description": "Renders ```csv code blocks as sortable, interactive HTML tables",
"requires": ["chat"], "requires": [],
"permissions": [], "permissions": [],
"tools": [], "tools": [],
"surfaces": [], "surfaces": [],

View File

@@ -6,7 +6,7 @@
"tier": "browser", "tier": "browser",
"author": "switchboard", "author": "switchboard",
"description": "Renders ```diff code blocks with syntax-highlighted additions, deletions, and context lines", "description": "Renders ```diff code blocks with syntax-highlighted additions, deletions, and context lines",
"requires": ["chat"], "requires": [],
"permissions": [], "permissions": [],
"tools": [], "tools": [],
"surfaces": [], "surfaces": [],

View File

@@ -6,7 +6,7 @@
"tier": "browser", "tier": "browser",
"author": "switchboard", "author": "switchboard",
"description": "Renders LaTeX math expressions: ```latex blocks and inline $...$ / $$...$$ syntax", "description": "Renders LaTeX math expressions: ```latex blocks and inline $...$ / $$...$$ syntax",
"requires": ["chat"], "requires": [],
"permissions": [], "permissions": [],
"tools": [], "tools": [],
"surfaces": [], "surfaces": [],

View File

@@ -6,7 +6,7 @@
"tier": "browser", "tier": "browser",
"author": "switchboard", "author": "switchboard",
"description": "Renders ```mermaid code blocks as interactive SVG diagrams with zoom/pan, SVG/PNG export, and source copy", "description": "Renders ```mermaid code blocks as interactive SVG diagrams with zoom/pan, SVG/PNG export, and source copy",
"requires": ["chat"], "requires": [],
"permissions": [], "permissions": [],
"tools": [], "tools": [],
"surfaces": [], "surfaces": [],

View File

@@ -34,6 +34,26 @@ type Registry struct {
startTime time.Time startTime time.Time
stopCh chan struct{} stopCh chan struct{}
wg sync.WaitGroup wg sync.WaitGroup
// Optional callbacks — set after construction via setters.
sandboxStats func() (exec, errors uint64, avgMs float64)
triggerFires func() int64
extensionCount func() int
}
// SetSandboxStats registers a callback for sandbox execution counters.
func (r *Registry) SetSandboxStats(fn func() (uint64, uint64, float64)) {
r.sandboxStats = fn
}
// SetTriggerFireCount registers a callback for trigger fire count.
func (r *Registry) SetTriggerFireCount(fn func() int64) {
r.triggerFires = fn
}
// SetExtensionCount registers a callback for active extension count.
func (r *Registry) SetExtensionCount(fn func() int) {
r.extensionCount = fn
} }
// NewRegistry creates a cluster registry instance. // NewRegistry creates a cluster registry instance.
@@ -132,12 +152,27 @@ func (r *Registry) collectStats() json.RawMessage {
"goroutines": runtime.NumGoroutine(), "goroutines": runtime.NumGoroutine(),
"heap_alloc": m.HeapAlloc, "heap_alloc": m.HeapAlloc,
"heap_sys": m.HeapSys, "heap_sys": m.HeapSys,
"stack_in_use": m.StackInuse,
"gc_cycles": m.NumGC, "gc_cycles": m.NumGC,
"gc_pause_ns": m.PauseNs[(m.NumGC+255)%256], "gc_pause_ns": m.PauseNs[(m.NumGC+255)%256],
"gc_cpu_pct": m.GCCPUFraction * 100,
"uptime_sec": time.Since(r.startTime).Seconds(), "uptime_sec": time.Since(r.startTime).Seconds(),
"ws_clients": r.hub.ConnCount(), "ws_clients": r.hub.ConnCount(),
} }
if r.extensionCount != nil {
stats["extensions_loaded"] = r.extensionCount()
}
if r.sandboxStats != nil {
exec, errors, avgMs := r.sandboxStats()
stats["starlark_exec_total"] = exec
stats["starlark_errors_total"] = errors
stats["starlark_avg_duration_ms"] = avgMs
}
if r.triggerFires != nil {
stats["trigger_fires_total"] = r.triggerFires()
}
data, _ := json.Marshal(stats) data, _ := json.Marshal(stats)
return data return data
} }

View File

@@ -75,6 +75,46 @@ func TestCollectStats(t *testing.T) {
} }
} }
func TestCollectStatsFattened(t *testing.T) {
hub := &mockHub{count: 3}
reg := NewRegistry("test-node", "", nil, hub, RegistryConfig{
HeartbeatInterval: 10 * time.Second,
StaleThreshold: 30 * time.Second,
})
reg.SetSandboxStats(func() (uint64, uint64, float64) { return 100, 5, 12.3 })
reg.SetTriggerFireCount(func() int64 { return 42 })
reg.SetExtensionCount(func() int { return 8 })
data := reg.collectStats()
var stats map[string]any
if err := json.Unmarshal(data, &stats); err != nil {
t.Fatalf("collectStats returned invalid JSON: %v", err)
}
// New fattened keys
fatKeys := []string{
"stack_in_use", "gc_cpu_pct",
"extensions_loaded", "starlark_exec_total", "starlark_errors_total",
"starlark_avg_duration_ms", "trigger_fires_total",
}
for _, key := range fatKeys {
if _, ok := stats[key]; !ok {
t.Errorf("missing fattened stats key: %s", key)
}
}
if v := stats["extensions_loaded"].(float64); int(v) != 8 {
t.Errorf("extensions_loaded = %v, want 8", v)
}
if v := stats["starlark_exec_total"].(float64); int(v) != 100 {
t.Errorf("starlark_exec_total = %v, want 100", v)
}
if v := stats["trigger_fires_total"].(float64); int(v) != 42 {
t.Errorf("trigger_fires_total = %v, want 42", v)
}
}
func TestRegistryStartStop(t *testing.T) { func TestRegistryStartStop(t *testing.T) {
ms := &mockClusterStore{heartbeatRows: 1} ms := &mockClusterStore{heartbeatRows: 1}
hub := &mockHub{count: 0} hub := &mockHub{count: 0}

View File

@@ -3,6 +3,7 @@ package events
import ( import (
"strings" "strings"
"sync" "sync"
"sync/atomic"
"time" "time"
) )
@@ -14,6 +15,8 @@ type Bus struct {
subs map[string][]*subscription subs map[string][]*subscription
seq uint64 // subscription ID counter seq uint64 // subscription ID counter
broadcastHook func(Event) // called after Publish for cross-pod fan-out; nil-safe broadcastHook func(Event) // called after Publish for cross-pod fan-out; nil-safe
publishCount atomic.Int64
deliverCount atomic.Int64
} }
type subscription struct { type subscription struct {
@@ -86,6 +89,8 @@ func (b *Bus) Publish(event Event) {
// the broadcastHook. Used by the Postgres listener to re-publish remote // the broadcastHook. Used by the Postgres listener to re-publish remote
// events without causing an infinite re-broadcast loop. // events without causing an infinite re-broadcast loop.
func (b *Bus) publishLocal(event Event) { func (b *Bus) publishLocal(event Event) {
b.publishCount.Add(1)
b.mu.RLock() b.mu.RLock()
var matched []Handler var matched []Handler
for pattern, subs := range b.subs { for pattern, subs := range b.subs {
@@ -98,6 +103,7 @@ func (b *Bus) publishLocal(event Event) {
b.mu.RUnlock() b.mu.RUnlock()
for _, h := range matched { for _, h := range matched {
b.deliverCount.Add(1)
h(event) h(event)
} }
} }
@@ -126,6 +132,12 @@ func (b *Bus) PublishAsync(event Event) {
} }
} }
// PublishCount returns the cumulative number of events published.
func (b *Bus) PublishCount() int64 { return b.publishCount.Load() }
// DeliverCount returns the cumulative number of subscriber deliveries.
func (b *Bus) DeliverCount() int64 { return b.deliverCount.Load() }
// match checks if a concrete label matches a subscription pattern. // match checks if a concrete label matches a subscription pattern.
// //
// "chat.message.abc" matches "chat.message.abc" (exact) // "chat.message.abc" matches "chat.message.abc" (exact)

View File

@@ -0,0 +1,24 @@
package handlers
import (
"github.com/gin-gonic/gin"
"switchboard-core/metrics"
)
// MetricsHandler serves the admin metrics endpoint.
type MetricsHandler struct {
collector *metrics.Collector
}
// NewMetricsHandler creates a MetricsHandler.
func NewMetricsHandler(c *metrics.Collector) *MetricsHandler {
return &MetricsHandler{collector: c}
}
// GetMetrics returns a full metrics snapshot.
// GET /api/v1/admin/metrics
func (h *MetricsHandler) GetMetrics(c *gin.Context) {
snap := h.collector.Collect(c.Request.Context())
c.JSON(200, snap)
}

View File

@@ -0,0 +1,187 @@
package handlers
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/gin-gonic/gin"
"switchboard-core/metrics"
"switchboard-core/store"
)
// mockHub implements metrics.ConnCounter for tests.
type mockHub struct{ count int }
func (m *mockHub) ConnCount() int { return m.count }
// mockBus implements metrics.BusCounter for tests.
type mockBus struct{ pub, del int64 }
func (m *mockBus) PublishCount() int64 { return m.pub }
func (m *mockBus) DeliverCount() int64 { return m.del }
func TestMetrics_SQLiteShape(t *testing.T) {
gin.SetMode(gin.TestMode)
collector := metrics.NewCollector(
"test-node",
nil, // no DB
&mockHub{count: 5},
&mockBus{pub: 10, del: 20},
store.Stores{}, // no cluster store
func() (uint64, uint64, float64) { return 42, 3, 12.5 },
func() int64 { return 7 },
time.Now().Add(-10*time.Minute),
)
h := NewMetricsHandler(collector)
r := gin.New()
r.GET("/api/v1/admin/metrics", h.GetMetrics)
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/metrics", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("status = %d, want 200", w.Code)
}
var snap metrics.Snapshot
if err := json.Unmarshal(w.Body.Bytes(), &snap); err != nil {
t.Fatalf("unmarshal: %v", err)
}
// Node ID
if snap.NodeID != "test-node" {
t.Errorf("node_id = %q, want %q", snap.NodeID, "test-node")
}
// Runtime
if snap.Runtime.WSClients != 5 {
t.Errorf("ws_clients = %d, want 5", snap.Runtime.WSClients)
}
if snap.Runtime.UptimeSec < 600 {
t.Errorf("uptime_sec = %f, want >= 600", snap.Runtime.UptimeSec)
}
// Cluster should be nil (SQLite)
if snap.Cluster != nil {
t.Errorf("cluster should be nil for SQLite, got %+v", snap.Cluster)
}
// Extensions
if snap.Extensions.StarlarkExecTotal != 42 {
t.Errorf("starlark_exec_total = %d, want 42", snap.Extensions.StarlarkExecTotal)
}
if snap.Extensions.StarlarkErrorsTotal != 3 {
t.Errorf("starlark_errors_total = %d, want 3", snap.Extensions.StarlarkErrorsTotal)
}
if snap.Extensions.StarlarkAvgDuration != 12.5 {
t.Errorf("starlark_avg_duration_ms = %f, want 12.5", snap.Extensions.StarlarkAvgDuration)
}
if snap.Extensions.TriggerFiresTotal != 7 {
t.Errorf("trigger_fires_total = %d, want 7", snap.Extensions.TriggerFiresTotal)
}
if snap.Extensions.EventBusPublished != 10 {
t.Errorf("event_bus_published = %d, want 10", snap.Extensions.EventBusPublished)
}
if snap.Extensions.EventBusDelivered != 20 {
t.Errorf("event_bus_delivered = %d, want 20", snap.Extensions.EventBusDelivered)
}
}
func TestMetrics_WithCluster(t *testing.T) {
gin.SetMode(gin.TestMode)
mock := &mockClusterStore{
nodes: []store.ClusterNode{
{
NodeID: "node-1",
Endpoint: "http://node-1:8080",
Heartbeat: time.Now(),
Stats: json.RawMessage(`{"uptime_sec":120,"ws_clients":3}`),
},
},
}
collector := metrics.NewCollector(
"test-node",
nil,
&mockHub{count: 3},
&mockBus{},
store.Stores{Cluster: mock},
func() (uint64, uint64, float64) { return 0, 0, 0 },
func() int64 { return 0 },
time.Now(),
)
h := NewMetricsHandler(collector)
r := gin.New()
r.GET("/api/v1/admin/metrics", h.GetMetrics)
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/metrics", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("status = %d, want 200", w.Code)
}
var snap metrics.Snapshot
if err := json.Unmarshal(w.Body.Bytes(), &snap); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if snap.Cluster == nil {
t.Fatal("cluster should not be nil with cluster store")
}
if snap.Cluster.Size != 1 {
t.Errorf("cluster.size = %d, want 1", snap.Cluster.Size)
}
if snap.Cluster.Nodes[0].NodeID != "node-1" {
t.Errorf("node_id = %q, want %q", snap.Cluster.Nodes[0].NodeID, "node-1")
}
if snap.Cluster.Nodes[0].UptimeSec != 120 {
t.Errorf("uptime_sec = %f, want 120", snap.Cluster.Nodes[0].UptimeSec)
}
}
func TestMetrics_ExtensionCounters(t *testing.T) {
gin.SetMode(gin.TestMode)
collector := metrics.NewCollector(
"test-node",
nil,
&mockHub{},
&mockBus{pub: 100, del: 500},
store.Stores{},
func() (uint64, uint64, float64) { return 1000, 50, 8.3 },
func() int64 { return 25 },
time.Now(),
)
h := NewMetricsHandler(collector)
r := gin.New()
r.GET("/api/v1/admin/metrics", h.GetMetrics)
req := httptest.NewRequest(http.MethodGet, "/api/v1/admin/metrics", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
var snap metrics.Snapshot
_ = json.Unmarshal(w.Body.Bytes(), &snap)
if snap.Extensions.StarlarkExecTotal != 1000 {
t.Errorf("exec total = %d, want 1000", snap.Extensions.StarlarkExecTotal)
}
if snap.Extensions.EventBusPublished != 100 {
t.Errorf("bus published = %d, want 100", snap.Extensions.EventBusPublished)
}
if snap.Extensions.EventBusDelivered != 500 {
t.Errorf("bus delivered = %d, want 500", snap.Extensions.EventBusDelivered)
}
}

View File

@@ -56,6 +56,7 @@ func main() {
} }
// ── Server startup ────────────────────── // ── Server startup ──────────────────────
startTime := time.Now()
cfg := config.Load() cfg := config.Load()
// Structured logging — must be first so all subsequent // Structured logging — must be first so all subsequent
@@ -200,19 +201,40 @@ func main() {
// ── WebSocket Hub ───────────────────────── // ── WebSocket Hub ─────────────────────────
hub := events.NewHub(bus, middleware.GetAllowedOrigins(cfg)) hub := events.NewHub(bus, middleware.GetAllowedOrigins(cfg))
// ── Cluster Registry ──────────── // ── Node Identity ────────────────
// PG-backed node self-registration + heartbeat. No-op on SQLite. // Used by cluster registry (PG) and metrics endpoint (all deployments).
var clusterReg *cluster.Registry
if database.IsPostgres() && stores.Cluster != nil {
nodeID := cfg.ClusterNodeID nodeID := cfg.ClusterNodeID
if nodeID == "" { if nodeID == "" {
hostname, _ := os.Hostname() hostname, _ := os.Hostname()
nodeID = fmt.Sprintf("%s-%d", hostname, os.Getpid()) nodeID = fmt.Sprintf("%s-%d", hostname, os.Getpid())
} }
// ── Cluster Registry ────────────
// PG-backed node self-registration + heartbeat. No-op on SQLite.
var clusterReg *cluster.Registry
if database.IsPostgres() && stores.Cluster != nil {
clusterReg = cluster.NewRegistry(nodeID, cfg.ClusterEndpoint, stores.Cluster, hub, cluster.RegistryConfig{ clusterReg = cluster.NewRegistry(nodeID, cfg.ClusterEndpoint, stores.Cluster, hub, cluster.RegistryConfig{
HeartbeatInterval: cfg.ClusterHeartbeatInterval, HeartbeatInterval: cfg.ClusterHeartbeatInterval,
StaleThreshold: cfg.ClusterStaleThreshold, StaleThreshold: cfg.ClusterStaleThreshold,
}) })
clusterReg.SetSandboxStats(sandbox.SandboxStats)
clusterReg.SetTriggerFireCount(triggerEngine.FireCount)
clusterReg.SetExtensionCount(func() int {
if stores.Packages == nil {
return 0
}
pkgs, err := stores.Packages.List(context.Background())
if err != nil {
return 0
}
count := 0
for _, p := range pkgs {
if p.Status == "active" {
count++
}
}
return count
})
if err := clusterReg.Start(); err != nil { if err := clusterReg.Start(); err != nil {
log.Printf("⚠ Cluster registry failed to start: %v", err) log.Printf("⚠ Cluster registry failed to start: %v", err)
clusterReg = nil clusterReg = nil
@@ -255,7 +277,7 @@ func main() {
} }
// Health check (k8s probes hit this directly) // Health check (k8s probes hit this directly)
base.GET("/health", func(c *gin.Context) { buildHealthResponse := func() gin.H {
info := gin.H{ info := gin.H{
"status": "ok", "status": "ok",
"version": Version, "version": Version,
@@ -263,8 +285,15 @@ func main() {
"database_name": database.Name(), "database_name": database.Name(),
"schema_version": database.SchemaVersion(), "schema_version": database.SchemaVersion(),
} }
if database.IsConnected() {
info["registration_enabled"] = handlers.IsRegistrationEnabled(stores)
}
appendClusterHealth(info, clusterReg, stores) appendClusterHealth(info, clusterReg, stores)
c.JSON(200, info) return info
}
base.GET("/health", func(c *gin.Context) {
c.JSON(200, buildHealthResponse())
}) })
// Liveness: process is alive and serving (no dependency checks). // Liveness: process is alive and serving (no dependency checks).
@@ -352,20 +381,9 @@ func main() {
api := base.Group("/api/v1") api := base.Group("/api/v1")
{ {
// Health (routable through ingress) // Health (routable through ingress — same shape as /health)
api.GET("/health", func(c *gin.Context) { api.GET("/health", func(c *gin.Context) {
info := gin.H{ c.JSON(200, buildHealthResponse())
"status": "ok",
"version": Version,
"schema_version": database.SchemaVersion(),
"database": database.IsConnected(),
"database_name": database.Name(),
}
if database.IsConnected() {
info["registration_enabled"] = handlers.IsRegistrationEnabled(stores)
}
appendClusterHealth(info, clusterReg, stores)
c.JSON(200, info)
}) })
authGroup := api.Group("/auth") authGroup := api.Group("/auth")
@@ -794,6 +812,16 @@ func main() {
admin.GET("/cluster", clusterH.ListNodes) admin.GET("/cluster", clusterH.ListNodes)
} }
// ── Metrics ─────────────────
metricsCollector := metrics.NewCollector(
nodeID, database.DB, hub, bus, stores,
sandbox.SandboxStats,
triggerEngine.FireCount,
startTime,
)
metricsH := handlers.NewMetricsHandler(metricsCollector)
admin.GET("/metrics", metricsH.GetMetrics)
// ── Backup/Restore ───────── // ── Backup/Restore ─────────
backupH := handlers.NewBackupHandler(stores, packagesDir, cfg.StoragePath) backupH := handlers.NewBackupHandler(stores, packagesDir, cfg.StoragePath)
admin.POST("/backup", backupH.CreateBackup) admin.POST("/backup", backupH.CreateBackup)

256
server/metrics/collector.go Normal file
View File

@@ -0,0 +1,256 @@
// Package metrics — collector.go
//
// On-demand metrics collector for the admin /api/v1/admin/metrics endpoint.
// Gathers runtime, database, cluster, and extension stats into a single JSON snapshot.
package metrics
import (
"context"
"database/sql"
"encoding/json"
"os"
"runtime"
"time"
"switchboard-core/database"
"switchboard-core/store"
)
// Snapshot is the top-level JSON response from GET /api/v1/admin/metrics.
type Snapshot struct {
NodeID string `json:"node_id"`
Runtime RuntimeMetrics `json:"runtime"`
DB DBMetrics `json:"db"`
Cluster *ClusterMetrics `json:"cluster,omitempty"`
Extensions ExtensionMetrics `json:"extensions"`
}
type RuntimeMetrics struct {
Goroutines int `json:"goroutines"`
HeapAlloc uint64 `json:"heap_alloc"`
HeapSys uint64 `json:"heap_sys"`
StackInUse uint64 `json:"stack_in_use"`
GCCycles uint32 `json:"gc_cycles"`
GCPauseNs uint64 `json:"gc_pause_ns"`
GCCPUPercent float64 `json:"gc_cpu_pct"`
UptimeSec float64 `json:"uptime_sec"`
WSClients int `json:"ws_clients"`
ExtensionsLoaded int `json:"extensions_loaded"`
OpenFDs int `json:"open_fds"`
}
type DBMetrics struct {
LatencyMs float64 `json:"latency_ms"`
PoolActive int `json:"pool_active"`
PoolIdle int `json:"pool_idle"`
PoolMax int `json:"pool_max"`
WaitCount int64 `json:"wait_count"`
WaitDuration float64 `json:"wait_duration_ms"`
// PG-only fields (zero/omitted on SQLite)
DeadTuples *int64 `json:"dead_tuples,omitempty"`
ActiveBackends *int `json:"active_backends,omitempty"`
}
type ClusterMetrics struct {
Size int `json:"size"`
Nodes []ClusterNode `json:"nodes"`
}
type ClusterNode struct {
NodeID string `json:"node_id"`
Endpoint string `json:"endpoint"`
UptimeSec float64 `json:"uptime_sec"`
HeartbeatAge int64 `json:"heartbeat_age_ms"`
Stats json.RawMessage `json:"stats"`
}
type ExtensionMetrics struct {
StarlarkExecTotal uint64 `json:"starlark_exec_total"`
StarlarkErrorsTotal uint64 `json:"starlark_errors_total"`
StarlarkAvgDuration float64 `json:"starlark_avg_duration_ms"`
TriggerFiresTotal int64 `json:"trigger_fires_total"`
EventBusPublished int64 `json:"event_bus_published"`
EventBusDelivered int64 `json:"event_bus_delivered"`
}
// ConnCounter provides WebSocket connection count (satisfied by events.Hub).
type ConnCounter interface {
ConnCount() int
}
// BusCounter provides publish/deliver counts (satisfied by events.Bus).
type BusCounter interface {
PublishCount() int64
DeliverCount() int64
}
// SandboxStatsFunc returns cumulative sandbox execution counters.
type SandboxStatsFunc func() (execCount, errorCount uint64, avgDurationMs float64)
// TriggerFireCountFunc returns cumulative trigger fire count.
type TriggerFireCountFunc func() int64
// Collector gathers metrics on demand for the admin endpoint.
type Collector struct {
nodeID string
db *sql.DB
hub ConnCounter
bus BusCounter
stores store.Stores
sandboxStats SandboxStatsFunc
triggerFireCount TriggerFireCountFunc
startTime time.Time
}
// NewCollector creates a metrics collector with all required dependencies.
func NewCollector(nodeID string, db *sql.DB, hub ConnCounter, bus BusCounter, stores store.Stores, sandboxFn SandboxStatsFunc, triggerFn TriggerFireCountFunc, startTime time.Time) *Collector {
return &Collector{
nodeID: nodeID,
db: db,
hub: hub,
bus: bus,
stores: stores,
sandboxStats: sandboxFn,
triggerFireCount: triggerFn,
startTime: startTime,
}
}
// Collect gathers all metrics synchronously and returns a snapshot.
func (c *Collector) Collect(ctx context.Context) *Snapshot {
snap := &Snapshot{NodeID: c.nodeID}
snap.Runtime = c.collectRuntime()
snap.DB = c.collectDB(ctx)
snap.Cluster = c.collectCluster(ctx)
snap.Extensions = c.collectExtensions()
return snap
}
func (c *Collector) collectRuntime() RuntimeMetrics {
var m runtime.MemStats
runtime.ReadMemStats(&m)
extCount := 0
if c.stores.Packages != nil {
if pkgs, err := c.stores.Packages.List(context.Background()); err == nil {
for _, p := range pkgs {
if p.Status == "active" {
extCount++
}
}
}
}
return RuntimeMetrics{
Goroutines: runtime.NumGoroutine(),
HeapAlloc: m.HeapAlloc,
HeapSys: m.HeapSys,
StackInUse: m.StackInuse,
GCCycles: m.NumGC,
GCPauseNs: m.PauseNs[(m.NumGC+255)%256],
GCCPUPercent: m.GCCPUFraction * 100,
UptimeSec: time.Since(c.startTime).Seconds(),
WSClients: c.hub.ConnCount(),
ExtensionsLoaded: extCount,
OpenFDs: countOpenFDs(),
}
}
func (c *Collector) collectDB(ctx context.Context) DBMetrics {
dm := DBMetrics{}
if c.db == nil {
return dm
}
// Pool stats
stats := c.db.Stats()
dm.PoolActive = stats.InUse
dm.PoolIdle = stats.Idle
dm.PoolMax = stats.MaxOpenConnections
dm.WaitCount = stats.WaitCount
dm.WaitDuration = float64(stats.WaitDuration.Milliseconds())
// Latency probe
probeCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
start := time.Now()
if err := c.db.PingContext(probeCtx); err == nil {
dm.LatencyMs = float64(time.Since(start).Microseconds()) / 1000.0
}
// PG-only stats
if database.IsPostgres() {
var deadTuples int64
if err := c.db.QueryRowContext(ctx, `SELECT COALESCE(SUM(n_dead_tup), 0) FROM pg_stat_user_tables`).Scan(&deadTuples); err == nil {
dm.DeadTuples = &deadTuples
}
var activeBackends int
if err := c.db.QueryRowContext(ctx, `SELECT count(*) FROM pg_stat_activity WHERE datname = current_database()`).Scan(&activeBackends); err == nil {
dm.ActiveBackends = &activeBackends
}
}
return dm
}
func (c *Collector) collectCluster(ctx context.Context) *ClusterMetrics {
if c.stores.Cluster == nil {
return nil
}
nodes, err := c.stores.Cluster.ListNodes(ctx)
if err != nil {
return nil
}
if len(nodes) == 0 {
return nil
}
cm := &ClusterMetrics{
Size: len(nodes),
Nodes: make([]ClusterNode, len(nodes)),
}
now := time.Now()
for i, n := range nodes {
// Extract uptime from stats JSONB
var uptimeSec float64
var statsMap map[string]any
if json.Unmarshal(n.Stats, &statsMap) == nil {
if u, ok := statsMap["uptime_sec"].(float64); ok {
uptimeSec = u
}
}
cm.Nodes[i] = ClusterNode{
NodeID: n.NodeID,
Endpoint: n.Endpoint,
UptimeSec: uptimeSec,
HeartbeatAge: now.Sub(n.Heartbeat).Milliseconds(),
Stats: n.Stats,
}
}
return cm
}
func (c *Collector) collectExtensions() ExtensionMetrics {
em := ExtensionMetrics{}
if c.sandboxStats != nil {
em.StarlarkExecTotal, em.StarlarkErrorsTotal, em.StarlarkAvgDuration = c.sandboxStats()
}
if c.triggerFireCount != nil {
em.TriggerFiresTotal = c.triggerFireCount()
}
if c.bus != nil {
em.EventBusPublished = c.bus.PublishCount()
em.EventBusDelivered = c.bus.DeliverCount()
}
return em
}
// countOpenFDs counts open file descriptors via /proc/self/fd (Linux only).
func countOpenFDs() int {
entries, err := os.ReadDir("/proc/self/fd")
if err != nil {
return -1
}
return len(entries)
}

View File

@@ -23,16 +23,37 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"sync/atomic"
"time"
"go.starlark.net/starlark" "go.starlark.net/starlark"
starlarkjson "go.starlark.net/lib/json" starlarkjson "go.starlark.net/lib/json"
"switchboard-core/events" "switchboard-core/events"
"switchboard-core/metrics"
"switchboard-core/models" "switchboard-core/models"
"switchboard-core/store" "switchboard-core/store"
) )
// sandboxStats tracks cumulative execution counters for the admin metrics endpoint.
var sandboxStats struct {
execCount atomic.Int64
errorCount atomic.Int64
totalDurationNs atomic.Int64
}
// SandboxStats returns cumulative execution counters.
func SandboxStats() (execCount, errorCount uint64, avgDurationMs float64) {
exec := sandboxStats.execCount.Load()
errs := sandboxStats.errorCount.Load()
totalNs := sandboxStats.totalDurationNs.Load()
if exec > 0 {
avgDurationMs = float64(totalNs) / float64(exec) / 1e6
}
return uint64(exec), uint64(errs), avgDurationMs
}
// RunContext carries per-invocation state that modules need but which // RunContext carries per-invocation state that modules need but which
// varies per caller (API route vs filter vs task). Nil is safe — modules // varies per caller (API route vs filter vs task). Nil is safe — modules
// that need RunContext fields gracefully degrade. // that need RunContext fields gracefully degrade.
@@ -142,7 +163,20 @@ func (r *Runner) ExecPackage(ctx context.Context, pkg *store.PackageRegistration
log.Printf(" 🔧 runner: exec %s (%d modules granted)", pkg.ID, len(modules)) log.Printf(" 🔧 runner: exec %s (%d modules granted)", pkg.ID, len(modules))
return r.sandbox.ExecWithLoader(ctx, pkg.ID+"/script.star", script, modules, loader) start := time.Now()
result, err := r.sandbox.ExecWithLoader(ctx, pkg.ID+"/script.star", script, modules, loader)
duration := time.Since(start)
sandboxStats.execCount.Add(1)
sandboxStats.totalDurationNs.Add(int64(duration))
status := "success"
if err != nil {
sandboxStats.errorCount.Add(1)
status = "error"
}
metrics.SandboxExecutionsTotal.WithLabelValues("exec", status).Inc()
return result, err
} }
// loadScript reads the entry point script from disk (primary path) // loadScript reads the entry point script from disk (primary path)
@@ -266,8 +300,20 @@ func (r *Runner) CallEntryPoint(ctx context.Context, pkg *store.PackageRegistrat
return nil, result.Output, fmt.Errorf("package %q: %s is not callable", pkg.ID, entryPoint) return nil, result.Output, fmt.Errorf("package %q: %s is not callable", pkg.ID, entryPoint)
} }
val, callOutput, err := r.sandbox.Call(ctx, callable, args, kwargs) start := time.Now()
return val, result.Output + callOutput, err val, callOutput, callErr := r.sandbox.Call(ctx, callable, args, kwargs)
duration := time.Since(start)
sandboxStats.execCount.Add(1)
sandboxStats.totalDurationNs.Add(int64(duration))
status := "success"
if callErr != nil {
sandboxStats.errorCount.Add(1)
status = "error"
}
metrics.SandboxExecutionsTotal.WithLabelValues(entryPoint, status).Inc()
return val, result.Output + callOutput, callErr
} }
// buildModules assembles the module map based on granted permissions. // buildModules assembles the module map based on granted permissions.

View File

@@ -9,6 +9,7 @@ import (
"encoding/json" "encoding/json"
"log" "log"
"sync" "sync"
"sync/atomic"
"time" "time"
"github.com/robfig/cron/v3" "github.com/robfig/cron/v3"
@@ -30,10 +31,15 @@ type Engine struct {
unsubs map[string]func() // trigger_id → bus unsubscribe unsubs map[string]func() // trigger_id → bus unsubscribe
cronIDs map[string]cron.EntryID // scheduled_task_id → cron entry cronIDs map[string]cron.EntryID // scheduled_task_id → cron entry
fireCount atomic.Int64 // cumulative trigger fires
ctx context.Context ctx context.Context
cancel context.CancelFunc cancel context.CancelFunc
} }
// FireCount returns the cumulative number of trigger fires.
func (e *Engine) FireCount() int64 { return e.fireCount.Load() }
// New creates a trigger engine. Call Start() to begin processing. // New creates a trigger engine. Call Start() to begin processing.
func New(stores store.Stores, runner *sandbox.Runner, bus *events.Bus) *Engine { func New(stores store.Stores, runner *sandbox.Runner, bus *events.Bus) *Engine {
return &Engine{ return &Engine{
@@ -201,8 +207,9 @@ func (e *Engine) logExecution(triggerID, scheduledTaskID string, firedAt time.Ti
} }
} }
// publishEvent emits a trigger lifecycle event on the bus. // publishEvent emits a trigger lifecycle event on the bus and increments the fire counter.
func (e *Engine) publishEvent(label string, triggerID string) { func (e *Engine) publishEvent(label string, triggerID string) {
e.fireCount.Add(1)
if e.bus != nil { if e.bus != nil {
payload, _ := json.Marshal(map[string]any{"trigger_id": triggerID}) payload, _ := json.Marshal(map[string]any{"trigger_id": triggerID})
e.bus.Publish(events.Event{ e.bus.Publish(events.Event{

View File

@@ -163,6 +163,7 @@ export function createDomains(restClient) {
// ── 18. Admin ────────────────────────── // ── 18. Admin ──────────────────────────
admin: { admin: {
stats: () => rc.get('/api/v1/admin/stats'), stats: () => rc.get('/api/v1/admin/stats'),
metrics: () => rc.get('/api/v1/admin/metrics'),
users: { users: {
list: (opts) => rc.get('/api/v1/admin/users' + _qs(opts)), list: (opts) => rc.get('/api/v1/admin/users' + _qs(opts)),

View File

@@ -0,0 +1,207 @@
/**
* Admin > Monitoring > Health
*
* Polls GET /api/v1/admin/metrics and displays runtime, DB, cluster,
* and extension metrics. Cluster panel only shows on PG multi-node.
*/
const { html } = window;
const { useState, useEffect, useRef, useCallback } = hooks;
// ── Formatting helpers ──────────────────────────
function formatBytes(b) {
if (b == null) return '\u2014';
if (b < 1024) return b + ' B';
if (b < 1024 * 1024) return (b / 1024).toFixed(1) + ' KB';
if (b < 1024 * 1024 * 1024) return (b / (1024 * 1024)).toFixed(1) + ' MB';
return (b / (1024 * 1024 * 1024)).toFixed(2) + ' GB';
}
function formatDuration(sec) {
if (sec == null) return '\u2014';
if (sec < 60) return Math.floor(sec) + 's';
if (sec < 3600) return Math.floor(sec / 60) + 'm ' + Math.floor(sec % 60) + 's';
const h = Math.floor(sec / 3600);
const m = Math.floor((sec % 3600) / 60);
if (h < 24) return h + 'h ' + m + 'm';
const d = Math.floor(h / 24);
return d + 'd ' + (h % 24) + 'h';
}
function formatMs(ms) {
if (ms == null) return '\u2014';
if (ms < 1) return '<1 ms';
if (ms < 1000) return ms.toFixed(1) + ' ms';
return (ms / 1000).toFixed(2) + ' s';
}
function formatPct(v) {
if (v == null) return '\u2014';
return v.toFixed(2) + '%';
}
function formatNum(v) {
if (v == null) return '\u2014';
if (typeof v === 'number') return v.toLocaleString();
return String(v);
}
// ── Stat card ───────────────────────────────────
function Stat({ label, value }) {
return html`
<div style="padding:8px 12px;background:var(--bg-secondary, #f5f5f5);border-radius:6px;min-width:120px;">
<div style="font-size:11px;color:var(--text-muted, #888);margin-bottom:2px;">${label}</div>
<div style="font-size:16px;font-weight:600;font-variant-numeric:tabular-nums;">${value}</div>
</div>
`;
}
// ── Panels ──────────────────────────────────────
function PanelHeader({ title }) {
return html`<h3 style="margin:0 0 12px;font-size:14px;font-weight:600;text-transform:uppercase;letter-spacing:0.5px;color:var(--text-muted, #888);">${title}</h3>`;
}
function RuntimePanel({ m }) {
return html`
<div style="margin-bottom:24px;">
<${PanelHeader} title="Runtime" />
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:8px;">
<${Stat} label="Uptime" value=${formatDuration(m.uptime_sec)} />
<${Stat} label="Goroutines" value=${formatNum(m.goroutines)} />
<${Stat} label="Heap Alloc" value=${formatBytes(m.heap_alloc)} />
<${Stat} label="Heap Sys" value=${formatBytes(m.heap_sys)} />
<${Stat} label="Stack In Use" value=${formatBytes(m.stack_in_use)} />
<${Stat} label="GC Cycles" value=${formatNum(m.gc_cycles)} />
<${Stat} label="Last GC Pause" value=${formatMs((m.gc_pause_ns || 0) / 1e6)} />
<${Stat} label="GC CPU" value=${formatPct(m.gc_cpu_pct)} />
<${Stat} label="WS Clients" value=${formatNum(m.ws_clients)} />
<${Stat} label="Extensions" value=${formatNum(m.extensions_loaded)} />
<${Stat} label="Open FDs" value=${m.open_fds >= 0 ? formatNum(m.open_fds) : '\u2014'} />
</div>
</div>
`;
}
function DBPanel({ m }) {
return html`
<div style="margin-bottom:24px;">
<${PanelHeader} title="Database" />
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:8px;">
<${Stat} label="Latency" value=${formatMs(m.latency_ms)} />
<${Stat} label="Pool Active" value=${formatNum(m.pool_active)} />
<${Stat} label="Pool Idle" value=${formatNum(m.pool_idle)} />
<${Stat} label="Pool Max" value=${formatNum(m.pool_max)} />
<${Stat} label="Wait Count" value=${formatNum(m.wait_count)} />
<${Stat} label="Wait Duration" value=${formatMs(m.wait_duration_ms)} />
${m.dead_tuples != null && html`<${Stat} label="Dead Tuples" value=${formatNum(m.dead_tuples)} />`}
${m.active_backends != null && html`<${Stat} label="Active Backends" value=${formatNum(m.active_backends)} />`}
</div>
</div>
`;
}
function ClusterPanel({ cluster }) {
return html`
<div style="margin-bottom:24px;">
<${PanelHeader} title=${'Cluster \u2014 ' + cluster.size + ' node' + (cluster.size !== 1 ? 's' : '')} />
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(300px,1fr));gap:12px;">
${cluster.nodes.map(n => html`<${NodeCard} key=${n.node_id} node=${n} />`)}
</div>
</div>
`;
}
function NodeCard({ node }) {
const stats = node.stats || {};
const ageLabel = node.heartbeat_age_ms < 15000
? Math.floor(node.heartbeat_age_ms / 1000) + 's ago'
: Math.floor(node.heartbeat_age_ms / 60000) + 'm ago';
const ageColor = node.heartbeat_age_ms < 15000 ? 'var(--accent, #2a6)' : 'var(--warning, #c80)';
return html`
<div style="border:1px solid var(--border, #ddd);border-radius:8px;padding:12px;background:var(--bg-primary, #fff);overflow:hidden;">
<div style="display:flex;justify-content:space-between;align-items:center;margin-bottom:8px;gap:8px;">
<strong style="font-size:13px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;min-width:0;" title=${node.node_id}>${node.node_id}</strong>
<span style="font-size:11px;color:${ageColor};font-weight:600;flex-shrink:0;">${ageLabel}</span>
</div>
${node.endpoint && html`<div style="font-size:11px;color:var(--text-muted, #888);margin-bottom:8px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;">${node.endpoint}</div>`}
<div style="display:grid;grid-template-columns:repeat(3,1fr);gap:6px;overflow:hidden;">
<${Stat} label="Uptime" value=${formatDuration(stats.uptime_sec)} />
<${Stat} label="WS Clients" value=${formatNum(stats.ws_clients)} />
<${Stat} label="Goroutines" value=${formatNum(stats.goroutines)} />
<${Stat} label="Heap" value=${formatBytes(stats.heap_alloc)} />
<${Stat} label="GC Pause" value=${formatMs((stats.gc_pause_ns || 0) / 1e6)} />
<${Stat} label="GC Cycles" value=${formatNum(stats.gc_cycles)} />
</div>
</div>
`;
}
function ExtensionPanel({ m }) {
return html`
<div style="margin-bottom:24px;">
<${PanelHeader} title="Extensions" />
<div style="display:grid;grid-template-columns:repeat(auto-fill,minmax(140px,1fr));gap:8px;">
<${Stat} label="Starlark Execs" value=${formatNum(m.starlark_exec_total)} />
<${Stat} label="Starlark Errors" value=${formatNum(m.starlark_errors_total)} />
<${Stat} label="Avg Duration" value=${formatMs(m.starlark_avg_duration_ms)} />
<${Stat} label="Trigger Fires" value=${formatNum(m.trigger_fires_total)} />
<${Stat} label="Events Published" value=${formatNum(m.event_bus_published)} />
<${Stat} label="Events Delivered" value=${formatNum(m.event_bus_delivered)} />
</div>
</div>
`;
}
// ── Main component ──────────────────────────────
export default function HealthSection() {
const [data, setData] = useState(null);
const [error, setError] = useState(null);
const [interval, setInterval_] = useState(10);
const timerRef = useRef(null);
const load = useCallback(async () => {
try {
const resp = await sw.api.admin.metrics();
setData(resp);
setError(null);
} catch (e) {
setError(e.message);
}
}, []);
useEffect(() => {
load();
timerRef.current = setInterval(load, interval * 1000);
return () => clearInterval(timerRef.current);
}, [load, interval]);
if (error && !data) {
return html`<div class="settings-placeholder" style="color:var(--error,red);">Failed to load metrics: ${error}</div>`;
}
if (!data) {
return html`<div class="settings-placeholder">Loading metrics\u2026</div>`;
}
return html`
<div>
<div style="display:flex;justify-content:space-between;align-items:center;margin-bottom:16px;gap:8px;">
${data.node_id && html`<span style="font-size:11px;color:var(--text-muted, #888);overflow:hidden;text-overflow:ellipsis;white-space:nowrap;min-width:0;" title=${data.node_id}>Viewing from <strong style="color:var(--text-primary, #ccc);">${data.node_id}</strong></span>`}
<span style="font-size:11px;color:var(--text-muted, #888);flex-shrink:0;">Refresh</span>
<select style="font-size:12px;padding:2px 6px;" value=${interval} onChange=${e => setInterval_(+e.target.value)}>
<option value="5">5s</option>
<option value="10">10s</option>
<option value="30">30s</option>
<option value="60">60s</option>
</select>
</div>
<${RuntimePanel} m=${data.runtime} />
<${DBPanel} m=${data.db} />
${data.cluster && html`<${ClusterPanel} cluster=${data.cluster} />`}
<${ExtensionPanel} m=${data.extensions} />
</div>
`;
}

View File

@@ -22,7 +22,7 @@ const ADMIN_SECTIONS = {
people: ['users', 'teams', 'groups'], people: ['users', 'teams', 'groups'],
workflows: ['workflows'], workflows: ['workflows'],
system: ['settings', 'storage', 'packages', 'connections', 'broadcast', 'backup'], system: ['settings', 'storage', 'packages', 'connections', 'broadcast', 'backup'],
monitoring: ['audit'], monitoring: ['health', 'audit'],
}; };
const ADMIN_LABELS = { const ADMIN_LABELS = {
@@ -31,7 +31,7 @@ const ADMIN_LABELS = {
settings: 'Settings', storage: 'Storage', packages: 'Packages', settings: 'Settings', storage: 'Storage', packages: 'Packages',
connections: 'Connections', broadcast: 'Broadcast', connections: 'Connections', broadcast: 'Broadcast',
backup: 'Backup', backup: 'Backup',
audit: 'Audit', health: 'Health', audit: 'Audit',
}; };
// ── Extension config sections ────── // ── Extension config sections ──────
@@ -52,7 +52,7 @@ const CATEGORY_META = {
people: { label: 'People', first: 'users', icon: 'M17 21v-2a4 4 0 0 0-4-4H5a4 4 0 0 0-4 4v2|C9 7 4' }, people: { label: 'People', first: 'users', icon: 'M17 21v-2a4 4 0 0 0-4-4H5a4 4 0 0 0-4 4v2|C9 7 4' },
workflows: { label: 'Workflows', first: 'workflows', icon: 'P16 3 21 3 21 8|L4 20 21 3|P21 16 21 21 16 21|L15 15 21 21|L4 4 9 9' }, workflows: { label: 'Workflows', first: 'workflows', icon: 'P16 3 21 3 21 8|L4 20 21 3|P21 16 21 21 16 21|L15 15 21 21|L4 4 9 9' },
system: { label: 'System', first: 'settings', icon: 'C12 12 3|M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 0 1-4 0v-.09' }, system: { label: 'System', first: 'settings', icon: 'C12 12 3|M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 0 1-4 0v-.09' },
monitoring: { label: 'Monitoring', first: 'audit', icon: 'L18 20 18 10|L12 20 12 4|L6 20 6 14' }, monitoring: { label: 'Monitoring', first: 'health', icon: 'L18 20 18 10|L12 20 12 4|L6 20 6 14' },
}; };
// ── Lazy section imports ──────────────────── // ── Lazy section imports ────────────────────
@@ -69,6 +69,7 @@ const sectionModules = {
connections: () => import(`./connections.js${_v}`), connections: () => import(`./connections.js${_v}`),
broadcast: () => import(`./broadcast.js${_v}`), broadcast: () => import(`./broadcast.js${_v}`),
backup: () => import(`./backup.js${_v}`), backup: () => import(`./backup.js${_v}`),
health: () => import(`./health.js${_v}`),
audit: () => import(`./audit.js${_v}`), audit: () => import(`./audit.js${_v}`),
}; };