Feat v0.7.12 batch exec #66
20
CHANGELOG.md
20
CHANGELOG.md
@@ -2,6 +2,26 @@
|
||||
|
||||
All notable changes to Armature are documented here.
|
||||
|
||||
## v0.7.12 — Concurrent Execution Primitive
|
||||
|
||||
New `batch` sandbox module. Enables extensions to parallelize arbitrary
|
||||
Starlark callables — including frozen library exports from `lib.require()`.
|
||||
|
||||
**batch module (permission: `batch.exec`)**
|
||||
|
||||
- `batch.exec(callables, timeout=10)` — runs up to 8 zero-arg callables concurrently, each in its own `starlark.Thread` with independent step budget. Returns `(results, errors)` tuple with ordered results. Per-branch timeout (1–30s, default 10).
|
||||
- Nested `batch.exec()` calls are prohibited (prevents exponential goroutine growth).
|
||||
- `lib.require()` not available inside branches — load libraries before the batch call.
|
||||
- `print()` output from branches is discarded.
|
||||
|
||||
**Internal**
|
||||
|
||||
- New file: `sandbox/batch_module.go`.
|
||||
- New permission constant: `ExtPermBatchExec`.
|
||||
- Runner wiring: `buildModulesWithLibCtx` creates batch module when permission granted.
|
||||
- Design doc: `docs/DESIGN-batch-exec.md`.
|
||||
- 12 new tests (parallel ordering, partial failure, timeout, cancellation, cap enforcement, nesting prevention, frozen sharing, permission gating).
|
||||
|
||||
## v0.7.11 — Query & HTTP Ergonomics
|
||||
|
||||
Four new Starlark sandbox builtins. No new permissions, no schema changes.
|
||||
|
||||
10
ROADMAP.md
10
ROADMAP.md
@@ -1,6 +1,6 @@
|
||||
# Armature — Roadmap
|
||||
|
||||
## Current: v0.7.11 — Query & HTTP Ergonomics
|
||||
## Current: v0.7.12 — Concurrent Execution Primitive
|
||||
|
||||
Self-hosted extensible platform kernel. Auth, identity, packages, Starlark
|
||||
sandbox, storage, realtime, and ops are kernel primitives. Everything else
|
||||
@@ -371,10 +371,10 @@ fresh module instances (`db`, `http`, etc.) — same pattern as
|
||||
|
||||
| Step | Status | Description |
|
||||
|------|--------|-------------|
|
||||
| Design doc | | `docs/DESIGN-batch-exec.md`. Thread isolation model, module construction, error semantics (partial results on failure), permission model, concurrency cap. |
|
||||
| `batch.exec()` | | `batch.exec(callables)` → `(results, errors)` tuple. List of Starlark callables (lambdas, function refs). Each runs in its own `starlark.Thread` with fresh modules. Concurrent goroutines capped at 8. Ordered results. Errors are per-callable (None on success, string on failure). Permission: `batch.exec` (new). New file: `sandbox/batch_module.go`. |
|
||||
| Runner wiring | | `buildModulesWithLibCtx` creates `batch` module when `batch.exec` permission granted. Module receives runner reference for per-branch module construction. |
|
||||
| Tests | | Unit tests: parallel execution ordering, partial failure, cap enforcement, frozen library sharing, permission gating. |
|
||||
| Design doc | done | `docs/DESIGN-batch-exec.md`. Thread isolation model, module construction, error semantics (partial results on failure), permission model, concurrency cap. |
|
||||
| `batch.exec()` | done | `batch.exec(callables)` → `(results, errors)` tuple. List of Starlark callables (lambdas, function refs). Each runs in its own `starlark.Thread` with fresh modules. Concurrent goroutines capped at 8. Ordered results. Errors are per-callable (None on success, string on failure). Permission: `batch.exec` (new). New file: `sandbox/batch_module.go`. |
|
||||
| Runner wiring | done | `buildModulesWithLibCtx` creates `batch` module when `batch.exec` permission granted. Module receives runner reference for per-branch module construction. |
|
||||
| Tests | done | 12 unit tests: parallel execution ordering, partial failure, timeout per-branch, parent cancellation, cap enforcement, empty list, non-callable, permission gating, frozen struct sharing, nested batch.exec, default timeout, timeout clamping. All pass with `-race`. |
|
||||
|
||||
---
|
||||
|
||||
|
||||
466
docs/DESIGN-batch-exec.md
Normal file
466
docs/DESIGN-batch-exec.md
Normal file
@@ -0,0 +1,466 @@
|
||||
# DESIGN — Concurrent Execution Primitive (`batch.exec`)
|
||||
|
||||
**Version:** v0.7.12
|
||||
**Status:** Implemented
|
||||
**Author:** Jeff / Claude session 2026-04-02
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
Starlark is single-threaded by design (`go.starlark.net` enforces one
|
||||
thread per execution). Extensions that need to fan out — calling multiple
|
||||
external APIs, invoking several library functions, or performing independent
|
||||
I/O operations — must do so sequentially. For two `http.post()` calls
|
||||
taking 200ms each, the extension blocks for 400ms regardless of whether
|
||||
the calls are independent.
|
||||
|
||||
The v0.7.10 `http.batch()` primitive solves the narrow case of parallel
|
||||
HTTP dispatch. But it doesn't help when the work is wrapped in library
|
||||
functions. If a `jira-client` library exposes `create_issue()` and a
|
||||
`confluence-client` library exposes `create_page()`, the extension author
|
||||
must either:
|
||||
|
||||
1. Call them sequentially (slow), or
|
||||
2. Decompose the library calls back into raw `http.post()` parameters
|
||||
to use `http.batch()` (defeats the purpose of having libraries).
|
||||
|
||||
The platform needs a general-purpose concurrent execution primitive that
|
||||
works with arbitrary Starlark callables — including library exports.
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- **Shared mutable state between branches.** Each concurrent branch is
|
||||
fully isolated. No channels, no mutexes, no shared dicts. If branches
|
||||
need to coordinate, they don't belong in `batch.exec`.
|
||||
- **Unlimited concurrency.** A hard cap prevents extensions from spawning
|
||||
unbounded goroutines. This is a fan-out primitive, not a thread pool.
|
||||
- **Automatic retry or circuit breaking.** Error handling is the caller's
|
||||
responsibility. The kernel reports per-branch results and errors.
|
||||
- **Nested `batch.exec()`.** A callable inside `batch.exec` cannot itself
|
||||
call `batch.exec`. This prevents exponential goroutine growth and keeps
|
||||
the concurrency model flat.
|
||||
|
||||
---
|
||||
|
||||
## Key Insight: Frozen Libraries Are Thread-Safe
|
||||
|
||||
The reason this works without exotic machinery is `lib.require()`.
|
||||
|
||||
When a library is loaded via `lib.require()`, its exports are wrapped in
|
||||
a `starlarkstruct.FromStringDict()` — which produces a **frozen** struct.
|
||||
Frozen Starlark values are immutable and safe to read from any number of
|
||||
goroutines concurrently. This is a property of `go.starlark.net`, not
|
||||
something we enforce.
|
||||
|
||||
The only mutable state in a Starlark execution is:
|
||||
|
||||
1. **The `starlark.Thread` itself** — step counter, print buffer, cancel
|
||||
channel. Each branch gets its own thread.
|
||||
2. **Module instances** — `db`, `http`, `settings`, etc. contain
|
||||
configuration and hold references to shared Go objects (`*sql.DB`,
|
||||
`http.Client`). Each branch gets fresh module instances, but the
|
||||
underlying Go resources (`*sql.DB` connection pool, etc.) are already
|
||||
designed for concurrent access.
|
||||
3. **Local variables** — thread-local by definition in Starlark.
|
||||
|
||||
So the construction is: one new `starlark.Thread` + one new module set
|
||||
per branch, with frozen library structs shared read-only across all
|
||||
branches. This is exactly what `triggers/schedule.go` already does for
|
||||
scheduled task execution — `buildRestrictedModules` creates a fresh
|
||||
module set for each cron fire. `batch.exec` generalizes that pattern.
|
||||
|
||||
---
|
||||
|
||||
## API
|
||||
|
||||
```python
|
||||
results, errors = batch.exec([
|
||||
lambda: jira.create_issue(issue_data),
|
||||
lambda: confluence.create_page(page_data),
|
||||
lambda: slack.post_message(channel, msg),
|
||||
])
|
||||
|
||||
# results[0] = return value of jira.create_issue(), or None on error
|
||||
# errors[0] = None on success, or error string on failure
|
||||
# All three ran concurrently.
|
||||
```
|
||||
|
||||
### Signature
|
||||
|
||||
```
|
||||
batch.exec(callables, timeout=10) → (results: list, errors: list)
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Param | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `callables` | `list[callable]` | Starlark callables (lambdas, named functions, bound methods). Max length: 8. |
|
||||
| `timeout` | `int` (optional) | Per-branch timeout in seconds. Default 10. Max 30. Inherits parent context deadline if shorter. |
|
||||
|
||||
**Returns:** A 2-tuple of equal-length lists.
|
||||
|
||||
- `results[i]` — the return value of `callables[i]`, or `None` if it
|
||||
errored.
|
||||
- `errors[i]` — `None` if `callables[i]` succeeded, or a string error
|
||||
message if it failed (timeout, step limit, runtime error).
|
||||
|
||||
**Errors (whole-call):**
|
||||
|
||||
- `callables` is empty → error
|
||||
- `callables` length > 8 → error
|
||||
- Any element is not callable → error
|
||||
- Permission `batch.exec` not granted → error
|
||||
|
||||
### Permission
|
||||
|
||||
New extension permission: `batch.exec`. Declared in manifest:
|
||||
|
||||
```json
|
||||
{
|
||||
"permissions": ["batch.exec"]
|
||||
}
|
||||
```
|
||||
|
||||
This is a separate permission because concurrent execution has resource
|
||||
implications (goroutines, module construction overhead). Extensions that
|
||||
don't need it shouldn't pay for it. The permission doesn't imply any
|
||||
other permissions — the branch inherits whatever modules the calling
|
||||
package already has.
|
||||
|
||||
---
|
||||
|
||||
## Execution Model
|
||||
|
||||
```
|
||||
batch.exec([fn_a, fn_b, fn_c])
|
||||
│
|
||||
├─── goroutine 1: Thread₁ + Modules₁ → fn_a() → result[0]
|
||||
├─── goroutine 2: Thread₂ + Modules₂ → fn_b() → result[1]
|
||||
└─── goroutine 3: Thread₃ + Modules₃ → fn_c() → result[2]
|
||||
│
|
||||
sync.WaitGroup.Wait()
|
||||
│
|
||||
return (results, errors)
|
||||
```
|
||||
|
||||
### Per-Branch Construction
|
||||
|
||||
For each callable in the input list, the kernel:
|
||||
|
||||
1. Creates a new `sandbox.Sandbox` with the same `Config` as the parent
|
||||
(same `MaxSteps` limit — each branch gets its own step budget, not a
|
||||
shared one).
|
||||
2. Calls `runner.buildModulesWithLibCtx()` with the **same** `packageID`,
|
||||
`manifest`, and `RunContext` as the parent invocation. This produces
|
||||
a fresh module set — new `DBModuleConfig`, new `HTTPModuleConfig`, etc.
|
||||
— pointing at the same underlying Go resources (`*sql.DB`, etc.).
|
||||
3. The `libContext` is **shared** (read path only — cached frozen exports).
|
||||
Library exports are immutable. The `loading` map (cycle detection) is
|
||||
not relevant because libraries are already loaded before `batch.exec`
|
||||
runs. If a branch triggers a new `lib.require()`, it would need its
|
||||
own `libContext` — see Open Questions.
|
||||
4. Creates a new `starlark.Thread` with the branch's print handler,
|
||||
step limit, and context-based cancellation.
|
||||
5. Calls `starlark.Call(thread, callable, nil, nil)` — the callable
|
||||
is a zero-arg lambda that closes over its arguments.
|
||||
|
||||
### Context & Cancellation
|
||||
|
||||
Each branch gets a child context derived from the parent with the
|
||||
per-branch timeout applied:
|
||||
|
||||
```go
|
||||
branchCtx, cancel := context.WithTimeout(parentCtx, branchTimeout)
|
||||
defer cancel()
|
||||
```
|
||||
|
||||
If the parent context is cancelled (e.g., HTTP request timeout), all
|
||||
branches are cancelled. If one branch exceeds its timeout, only that
|
||||
branch is cancelled — others continue.
|
||||
|
||||
### Goroutine Cap
|
||||
|
||||
Hard limit: **8 concurrent branches.** This is enforced at the API
|
||||
boundary (list length check), not via a semaphore. Rationale:
|
||||
|
||||
- 8 covers the real-world fan-out patterns (2-5 API calls, small batch
|
||||
operations). Nobody needs 50 concurrent Starlark branches.
|
||||
- Each branch allocates a `starlark.Thread` + module instances. At 8
|
||||
branches, overhead is bounded at ~8KB per thread + module construction
|
||||
time (~50μs per module set).
|
||||
- No semaphore means no queuing surprises. You get 8, period.
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
### New File: `sandbox/batch_module.go`
|
||||
|
||||
```go
|
||||
// BuildBatchModule creates the "batch" module.
|
||||
// Requires the Runner reference for per-branch module construction.
|
||||
func BuildBatchModule(
|
||||
ctx context.Context,
|
||||
runner *Runner,
|
||||
packageID string,
|
||||
manifest map[string]any,
|
||||
rc *RunContext,
|
||||
lc *libContext,
|
||||
) *starlarkstruct.Module
|
||||
```
|
||||
|
||||
The module holds a reference to the `Runner` — same pattern as
|
||||
`BuildLibModule`. It needs the runner to call `buildModulesWithLibCtx`
|
||||
for each branch.
|
||||
|
||||
### Core Implementation Sketch
|
||||
|
||||
```go
|
||||
func batchExec(ctx context.Context, runner *Runner, packageID string,
|
||||
manifest map[string]any, rc *RunContext, parentLC *libContext,
|
||||
) func(*starlark.Thread, *starlark.Builtin, starlark.Tuple, []starlark.Tuple) (starlark.Value, error) {
|
||||
return func(thread *starlark.Thread, b *starlark.Builtin,
|
||||
args starlark.Tuple, kwargs []starlark.Tuple,
|
||||
) (starlark.Value, error) {
|
||||
var callableList *starlark.List
|
||||
var timeout int = 10
|
||||
|
||||
if err := starlark.UnpackArgs(b.Name(), args, kwargs,
|
||||
"callables", &callableList,
|
||||
"timeout?", &timeout,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
n := callableList.Len()
|
||||
if n == 0 {
|
||||
return nil, fmt.Errorf("batch.exec: callables list is empty")
|
||||
}
|
||||
if n > 8 {
|
||||
return nil, fmt.Errorf("batch.exec: max 8 callables, got %d", n)
|
||||
}
|
||||
if timeout < 1 || timeout > 30 {
|
||||
timeout = 10
|
||||
}
|
||||
|
||||
// Validate all elements are callable.
|
||||
callables := make([]starlark.Callable, n)
|
||||
for i := 0; i < n; i++ {
|
||||
c, ok := callableList.Index(i).(starlark.Callable)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("batch.exec: element %d is %s, not callable",
|
||||
i, callableList.Index(i).Type())
|
||||
}
|
||||
callables[i] = c
|
||||
}
|
||||
|
||||
// Execute concurrently.
|
||||
type branchResult struct {
|
||||
index int
|
||||
value starlark.Value
|
||||
err error
|
||||
}
|
||||
|
||||
results := make([]starlark.Value, n)
|
||||
errors := make([]starlark.Value, n)
|
||||
var wg sync.WaitGroup
|
||||
|
||||
for i, callable := range callables {
|
||||
wg.Add(1)
|
||||
go func(idx int, fn starlark.Callable) {
|
||||
defer wg.Done()
|
||||
|
||||
// Per-branch context with timeout.
|
||||
branchCtx, cancel := context.WithTimeout(ctx,
|
||||
time.Duration(timeout)*time.Second)
|
||||
defer cancel()
|
||||
|
||||
// Fresh module set for this branch.
|
||||
modules, err := runner.buildModulesWithLibCtx(
|
||||
branchCtx, packageID, manifest, rc, parentLC)
|
||||
if err != nil {
|
||||
results[idx] = starlark.None
|
||||
errors[idx] = starlark.String(err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
// Fresh sandbox + thread.
|
||||
sb := New(DefaultConfig())
|
||||
val, _, callErr := sb.Call(branchCtx, fn, nil, nil)
|
||||
|
||||
if callErr != nil {
|
||||
results[idx] = starlark.None
|
||||
errors[idx] = starlark.String(callErr.Error())
|
||||
} else {
|
||||
results[idx] = val
|
||||
errors[idx] = starlark.None
|
||||
}
|
||||
}(i, callable)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
return starlark.Tuple{
|
||||
starlark.NewList(results),
|
||||
starlark.NewList(errors),
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Runner Wiring
|
||||
|
||||
In `buildModulesWithLibCtx`, add the permission case:
|
||||
|
||||
```go
|
||||
case models.ExtPermBatchExec:
|
||||
// Deferred — wired after module map is complete (needs runner ref).
|
||||
hasBatchExec = true
|
||||
```
|
||||
|
||||
After the module map is assembled:
|
||||
|
||||
```go
|
||||
if hasBatchExec {
|
||||
modules["batch"] = BuildBatchModule(ctx, r, packageID, manifest, rc, lc)
|
||||
}
|
||||
```
|
||||
|
||||
### Permission Constant
|
||||
|
||||
In `models/permissions.go`:
|
||||
|
||||
```go
|
||||
ExtPermBatchExec = "batch.exec"
|
||||
```
|
||||
|
||||
Add to `AllExtensionPermissions` slice.
|
||||
|
||||
---
|
||||
|
||||
## Callable Closure Semantics
|
||||
|
||||
The callables passed to `batch.exec` are typically lambdas that close
|
||||
over variables from the calling scope:
|
||||
|
||||
```python
|
||||
issue_data = {"summary": "Review Q3 report"}
|
||||
page_data = {"title": "Q3 Report", "body": content}
|
||||
|
||||
results, errors = batch.exec([
|
||||
lambda: jira.create_issue(issue_data),
|
||||
lambda: confluence.create_page(page_data),
|
||||
])
|
||||
```
|
||||
|
||||
The closed-over values (`issue_data`, `page_data`, `jira`, `confluence`)
|
||||
are references to Starlark values in the calling thread's scope. Two
|
||||
safety properties make this work:
|
||||
|
||||
1. **Library exports (`jira`, `confluence`) are frozen.** They were
|
||||
returned by `lib.require()` as `starlarkstruct.FromStringDict()` —
|
||||
deeply immutable. Safe to read from any goroutine.
|
||||
|
||||
2. **Dict/list arguments may be mutable**, but Starlark's execution
|
||||
model means the calling thread is **blocked** waiting for
|
||||
`batch.exec` to return. No concurrent mutation is possible because
|
||||
the caller can't execute while the branches are running.
|
||||
|
||||
This is the same safety model as Go's `sync.WaitGroup` pattern: the
|
||||
goroutine that calls `wg.Wait()` cannot proceed until all goroutines
|
||||
complete, so values passed to goroutines before `wg.Add` are safe to
|
||||
read without locks.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
### 1. `lib.require()` Inside Branches
|
||||
|
||||
If a callable triggers a `lib.require()` that hasn't been cached yet,
|
||||
the shared `libContext.cache` would be written from a goroutine. Options:
|
||||
|
||||
**A. Prohibit: branches cannot call `lib.require()`.** The branch gets
|
||||
a nil `libContext`, so `lib` module is unavailable inside `batch.exec`.
|
||||
Libraries must be loaded before the batch call. Simplest, safest.
|
||||
|
||||
**B. Per-branch `libContext` with shared read cache.** Each branch gets
|
||||
its own `libContext` whose `cache` is pre-populated from the parent's
|
||||
cache (snapshot). New loads go into the branch's local cache only.
|
||||
Slightly wasteful if two branches load the same library (loaded twice),
|
||||
but safe.
|
||||
|
||||
**C. Mutex-protected shared `libContext`.** Add a `sync.RWMutex` to
|
||||
`libContext`. Reads use `RLock`, writes use `Lock`. Minimal overhead,
|
||||
but makes `libContext` aware of concurrency — violates its current
|
||||
assumptions.
|
||||
|
||||
**Recommendation: Option A for v0.7.11, Option B as follow-up if needed.**
|
||||
In practice, extensions call `lib.require()` at module scope (top of
|
||||
script), not inside request handlers. The lambdas passed to `batch.exec`
|
||||
call methods on already-loaded library structs. Option A covers all
|
||||
real-world patterns.
|
||||
|
||||
### 2. Print Output
|
||||
|
||||
Each branch has its own print buffer (the `output strings.Builder` in
|
||||
`Sandbox.Call`). Options:
|
||||
|
||||
**A. Discard.** Branch print output is lost. Simple, avoids interleaving.
|
||||
|
||||
**B. Collect per-branch.** Return a third list: `(results, errors, outputs)`.
|
||||
Useful for debugging but clutters the API.
|
||||
|
||||
**C. Merge into parent.** Append all branch output to the parent thread's
|
||||
print buffer, prefixed with branch index. Requires passing the parent's
|
||||
`outputMu` and `output` builder — invasive.
|
||||
|
||||
**Recommendation: Option A for v0.7.11.** `print()` in Starlark is a
|
||||
debugging tool, not a production logging facility. Branch callables that
|
||||
need to report status should return structured data. If debugging demand
|
||||
emerges, Option B is a backward-compatible addition.
|
||||
|
||||
### 3. Step Limit Scope
|
||||
|
||||
Each branch gets its own `MaxSteps` budget (default 1M). Should the
|
||||
total across all branches be capped?
|
||||
|
||||
**No.** The per-branch cap is sufficient. 8 branches × 1M steps = 8M
|
||||
total, which completes in under a second on any modern hardware. The
|
||||
wall-clock timeout (per-branch, max 30s) is the real resource guard.
|
||||
Adding a cross-branch step budget creates coupling between independent
|
||||
execution paths — branch A's step count shouldn't affect branch B's
|
||||
ability to complete.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
| Test | Description |
|
||||
|------|-------------|
|
||||
| Parallel ordering | 3 callables with different sleep durations. Results in input order, not completion order. |
|
||||
| Partial failure | 3 callables, middle one raises error. results = [val, None, val], errors = [None, "err msg", None]. |
|
||||
| Timeout per-branch | One callable sleeps beyond timeout. Others succeed. Timed-out branch returns error. |
|
||||
| Parent cancellation | Cancel parent context mid-execution. All branches cancelled. |
|
||||
| Cap enforcement | List of 9 callables → immediate error, nothing executed. |
|
||||
| Empty list | `batch.exec([])` → error. |
|
||||
| Non-callable element | `batch.exec([1, 2])` → error, nothing executed. |
|
||||
| Permission gating | Package without `batch.exec` permission → module not available. |
|
||||
| Frozen library sharing | Two branches call same frozen library function concurrently. No race. |
|
||||
| Nested batch.exec | Callable inside batch.exec attempts batch.exec → error (module not injected in branch). |
|
||||
| db module isolation | Two branches insert into same table concurrently. Both succeed, no corruption. |
|
||||
| http module isolation | Two branches make HTTP calls with different headers. No cross-contamination. |
|
||||
|
||||
---
|
||||
|
||||
## Migration
|
||||
|
||||
No schema changes. No new tables. No new migrations.
|
||||
|
||||
New permission constant `batch.exec` added to `models/permissions.go`.
|
||||
Extensions must declare the permission in their manifest to access the
|
||||
`batch` module.
|
||||
@@ -242,6 +242,39 @@ instances = workflow.list_instances(workflow_id, status="active")
|
||||
# Returns list of instance dicts
|
||||
```
|
||||
|
||||
### batch
|
||||
|
||||
**Permission:** `batch.exec`
|
||||
|
||||
Run multiple callables concurrently. Each callable gets its own
|
||||
execution thread with an independent step budget.
|
||||
|
||||
```python
|
||||
jira = lib.require("jira-client")
|
||||
confluence = lib.require("confluence-client")
|
||||
|
||||
results, errors = batch.exec([
|
||||
lambda: jira.create_issue(issue_data),
|
||||
lambda: confluence.create_page(page_data),
|
||||
lambda: send_notification(user_id),
|
||||
], timeout=15)
|
||||
|
||||
# results[i] = return value of callables[i], or None on error
|
||||
# errors[i] = None on success, or error string on failure
|
||||
# All three ran concurrently.
|
||||
```
|
||||
|
||||
**Constraints:**
|
||||
|
||||
- Max 8 callables per call. Dispatched concurrently via goroutines.
|
||||
- `timeout` (optional): 1–30 seconds per branch (default 10).
|
||||
- `lib.require()` is not available inside branch callables.
|
||||
Load libraries before the `batch.exec` call.
|
||||
- `batch.exec()` cannot be called from within a branch (no nesting).
|
||||
- `print()` output from branches is discarded.
|
||||
|
||||
---
|
||||
|
||||
## Example: automated stage hook
|
||||
|
||||
A simple hook that reads a setting, queries data, and advances:
|
||||
|
||||
@@ -36,6 +36,7 @@ const (
|
||||
ExtPermConnectionsRead = "connections.read"
|
||||
ExtPermTriggersRegister = "triggers.register"
|
||||
ExtPermRealtimePublish = "realtime.publish"
|
||||
ExtPermBatchExec = "batch.exec"
|
||||
)
|
||||
|
||||
// ValidExtensionPermissions is the set of recognized permission keys.
|
||||
@@ -50,6 +51,7 @@ var ValidExtensionPermissions = map[string]bool{
|
||||
ExtPermConnectionsRead: true,
|
||||
ExtPermTriggersRegister: true,
|
||||
ExtPermRealtimePublish: true,
|
||||
ExtPermBatchExec: true,
|
||||
}
|
||||
|
||||
// ── Extension Permission Model ───────────────
|
||||
|
||||
140
server/sandbox/batch_module.go
Normal file
140
server/sandbox/batch_module.go
Normal file
@@ -0,0 +1,140 @@
|
||||
// Package sandbox — batch_module.go
|
||||
//
|
||||
// Permission: batch.exec
|
||||
//
|
||||
// Starlark API:
|
||||
// results, errors = batch.exec([fn_a, fn_b, fn_c], timeout=10)
|
||||
//
|
||||
// Runs a list of zero-arg callables concurrently, each in its own
|
||||
// starlark.Thread with an independent step budget. Results and errors
|
||||
// are returned in input order. Max 8 callables per call.
|
||||
//
|
||||
// Each branch gets a fresh Sandbox (thread + step counter) and a
|
||||
// child context with the per-branch timeout. The callable's closed-over
|
||||
// bindings (frozen library structs, module builtins) are used as-is —
|
||||
// the Go resources underneath (*sql.DB, http.Client) are concurrent-safe.
|
||||
//
|
||||
// lib.require() is not available inside branches (Option A).
|
||||
// print() output from branches is discarded.
|
||||
// Nested batch.exec() is prohibited via an atomic flag.
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"go.starlark.net/starlark"
|
||||
"go.starlark.net/starlarkstruct"
|
||||
)
|
||||
|
||||
const (
|
||||
batchMaxCallables = 8
|
||||
batchDefaultTimeout = 10
|
||||
batchMaxTimeout = 30
|
||||
)
|
||||
|
||||
// BuildBatchModule creates the "batch" module.
|
||||
// The runner/packageID/manifest/rc/lc parameters are stored for future
|
||||
// expansion (Option B: per-branch lib.require) but are not used in v1.
|
||||
func BuildBatchModule(
|
||||
ctx context.Context,
|
||||
runner *Runner,
|
||||
packageID string,
|
||||
manifest map[string]any,
|
||||
rc *RunContext,
|
||||
lc *libContext,
|
||||
) *starlarkstruct.Module {
|
||||
var running atomic.Bool
|
||||
|
||||
return MakeModule("batch", starlark.StringDict{
|
||||
"exec": starlark.NewBuiltin("batch.exec", batchExec(ctx, &running)),
|
||||
})
|
||||
}
|
||||
|
||||
// batchExec returns the batch.exec builtin implementation.
|
||||
func batchExec(
|
||||
ctx context.Context,
|
||||
running *atomic.Bool,
|
||||
) func(*starlark.Thread, *starlark.Builtin, starlark.Tuple, []starlark.Tuple) (starlark.Value, error) {
|
||||
return func(
|
||||
thread *starlark.Thread, b *starlark.Builtin,
|
||||
args starlark.Tuple, kwargs []starlark.Tuple,
|
||||
) (starlark.Value, error) {
|
||||
// Nesting guard: parent thread is blocked in wg.Wait(), so only
|
||||
// a branch goroutine could re-enter. The captured builtin reference
|
||||
// makes this reachable even though branches don't get fresh modules.
|
||||
if !running.CompareAndSwap(false, true) {
|
||||
return nil, fmt.Errorf("batch.exec: nested calls are not allowed")
|
||||
}
|
||||
defer running.Store(false)
|
||||
|
||||
var callableList *starlark.List
|
||||
var timeout int = batchDefaultTimeout
|
||||
|
||||
if err := starlark.UnpackArgs(b.Name(), args, kwargs,
|
||||
"callables", &callableList,
|
||||
"timeout?", &timeout,
|
||||
); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
n := callableList.Len()
|
||||
if n == 0 {
|
||||
return nil, fmt.Errorf("batch.exec: callables list is empty")
|
||||
}
|
||||
if n > batchMaxCallables {
|
||||
return nil, fmt.Errorf("batch.exec: max %d callables, got %d", batchMaxCallables, n)
|
||||
}
|
||||
if timeout < 1 || timeout > batchMaxTimeout {
|
||||
timeout = batchDefaultTimeout
|
||||
}
|
||||
|
||||
// Validate all elements are callable before spawning goroutines.
|
||||
callables := make([]starlark.Callable, n)
|
||||
for i := 0; i < n; i++ {
|
||||
c, ok := callableList.Index(i).(starlark.Callable)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("batch.exec: element %d is %s, not callable",
|
||||
i, callableList.Index(i).Type())
|
||||
}
|
||||
callables[i] = c
|
||||
}
|
||||
|
||||
// Execute concurrently.
|
||||
results := make([]starlark.Value, n)
|
||||
errors := make([]starlark.Value, n)
|
||||
var wg sync.WaitGroup
|
||||
|
||||
for i, callable := range callables {
|
||||
wg.Add(1)
|
||||
go func(idx int, fn starlark.Callable) {
|
||||
defer wg.Done()
|
||||
|
||||
branchCtx, cancel := context.WithTimeout(ctx,
|
||||
time.Duration(timeout)*time.Second)
|
||||
defer cancel()
|
||||
|
||||
sb := New(DefaultConfig())
|
||||
val, _, callErr := sb.Call(branchCtx, fn, nil, nil)
|
||||
|
||||
if callErr != nil {
|
||||
results[idx] = starlark.None
|
||||
errors[idx] = starlark.String(callErr.Error())
|
||||
} else {
|
||||
results[idx] = val
|
||||
errors[idx] = starlark.None
|
||||
}
|
||||
}(i, callable)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
return starlark.Tuple{
|
||||
starlark.NewList(results),
|
||||
starlark.NewList(errors),
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
324
server/sandbox/batch_module_test.go
Normal file
324
server/sandbox/batch_module_test.go
Normal file
@@ -0,0 +1,324 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go.starlark.net/starlark"
|
||||
)
|
||||
|
||||
// execWithBatch runs a Starlark script with the batch module available.
|
||||
func execWithBatch(t *testing.T, script string) (*Result, error) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
batchMod := BuildBatchModule(ctx, nil, "test-pkg", nil, nil, nil)
|
||||
sb := New(DefaultConfig())
|
||||
return sb.Exec(ctx, "test.star", script, map[string]starlark.Value{
|
||||
"batch": batchMod,
|
||||
})
|
||||
}
|
||||
|
||||
// ─── Parallel Ordering ──────────────────────
|
||||
|
||||
func TestBatchExec_ParallelOrdering(t *testing.T) {
|
||||
result, err := execWithBatch(t, `
|
||||
results, errors = batch.exec([
|
||||
lambda: 10,
|
||||
lambda: 20,
|
||||
lambda: 30,
|
||||
])
|
||||
`)
|
||||
if err != nil {
|
||||
t.Fatalf("exec error: %v", err)
|
||||
}
|
||||
|
||||
// Verify results are in input order.
|
||||
globals := result.Globals
|
||||
results := globals["results"].(*starlark.List)
|
||||
errors := globals["errors"].(*starlark.List)
|
||||
|
||||
if results.Len() != 3 {
|
||||
t.Fatalf("expected 3 results, got %d", results.Len())
|
||||
}
|
||||
|
||||
for i, want := range []int{10, 20, 30} {
|
||||
v, _ := starlark.AsInt32(results.Index(i))
|
||||
got := int(v)
|
||||
if got != want {
|
||||
t.Errorf("results[%d] = %d, want %d", i, got, want)
|
||||
}
|
||||
if errors.Index(i) != starlark.None {
|
||||
t.Errorf("errors[%d] = %v, want None", i, errors.Index(i))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Partial Failure ────────────────────────
|
||||
|
||||
func TestBatchExec_PartialFailure(t *testing.T) {
|
||||
result, err := execWithBatch(t, `
|
||||
def ok_a():
|
||||
return "a"
|
||||
|
||||
def bad():
|
||||
return 1 // 0
|
||||
|
||||
def ok_c():
|
||||
return "c"
|
||||
|
||||
results, errors = batch.exec([ok_a, bad, ok_c])
|
||||
`)
|
||||
if err != nil {
|
||||
t.Fatalf("exec error: %v", err)
|
||||
}
|
||||
|
||||
results := result.Globals["results"].(*starlark.List)
|
||||
errors := result.Globals["errors"].(*starlark.List)
|
||||
|
||||
// results[0] = "a", results[2] = "c"
|
||||
if s, _ := starlark.AsString(results.Index(0)); s != "a" {
|
||||
t.Errorf("results[0] = %v, want 'a'", results.Index(0))
|
||||
}
|
||||
if s, _ := starlark.AsString(results.Index(2)); s != "c" {
|
||||
t.Errorf("results[2] = %v, want 'c'", results.Index(2))
|
||||
}
|
||||
|
||||
// results[1] = None (failed)
|
||||
if results.Index(1) != starlark.None {
|
||||
t.Errorf("results[1] = %v, want None", results.Index(1))
|
||||
}
|
||||
|
||||
// errors[0] and errors[2] = None (succeeded)
|
||||
if errors.Index(0) != starlark.None {
|
||||
t.Errorf("errors[0] = %v, want None", errors.Index(0))
|
||||
}
|
||||
if errors.Index(2) != starlark.None {
|
||||
t.Errorf("errors[2] = %v, want None", errors.Index(2))
|
||||
}
|
||||
|
||||
// errors[1] should be a string containing the error
|
||||
errStr, ok := starlark.AsString(errors.Index(1))
|
||||
if !ok || errStr == "" {
|
||||
t.Errorf("errors[1] = %v, want non-empty error string", errors.Index(1))
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Timeout Per-Branch ─────────────────────
|
||||
|
||||
func TestBatchExec_TimeoutPerBranch(t *testing.T) {
|
||||
// Use a tight timeout. The slow lambda burns steps in an infinite loop,
|
||||
// which will hit either the step limit or the context timeout.
|
||||
result, err := execWithBatch(t, `
|
||||
def fast():
|
||||
return "done"
|
||||
|
||||
def slow():
|
||||
x = 0
|
||||
for i in range(999999999):
|
||||
x = x + 1
|
||||
return x
|
||||
|
||||
results, errors = batch.exec([fast, slow], timeout=1)
|
||||
`)
|
||||
if err != nil {
|
||||
t.Fatalf("exec error: %v", err)
|
||||
}
|
||||
|
||||
results := result.Globals["results"].(*starlark.List)
|
||||
errors := result.Globals["errors"].(*starlark.List)
|
||||
|
||||
// Fast branch should succeed.
|
||||
if s, _ := starlark.AsString(results.Index(0)); s != "done" {
|
||||
t.Errorf("results[0] = %v, want 'done'", results.Index(0))
|
||||
}
|
||||
if errors.Index(0) != starlark.None {
|
||||
t.Errorf("errors[0] = %v, want None", errors.Index(0))
|
||||
}
|
||||
|
||||
// Slow branch should fail (step limit or timeout).
|
||||
if results.Index(1) != starlark.None {
|
||||
t.Errorf("results[1] = %v, want None", results.Index(1))
|
||||
}
|
||||
errStr, _ := starlark.AsString(errors.Index(1))
|
||||
if errStr == "" {
|
||||
t.Errorf("errors[1] should contain error string")
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Parent Cancellation ────────────────────
|
||||
|
||||
func TestBatchExec_ParentCancellation(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel() // cancel immediately
|
||||
|
||||
batchMod := BuildBatchModule(ctx, nil, "test-pkg", nil, nil, nil)
|
||||
sb := New(DefaultConfig())
|
||||
result, err := sb.Exec(ctx, "test.star", `
|
||||
results, errors = batch.exec([lambda: 1, lambda: 2])
|
||||
`, map[string]starlark.Value{"batch": batchMod})
|
||||
|
||||
// Either the Exec itself fails (context cancelled before script runs)
|
||||
// or the batch.exec branches all fail.
|
||||
if err != nil {
|
||||
// Script-level cancellation — acceptable.
|
||||
return
|
||||
}
|
||||
|
||||
errors := result.Globals["errors"].(*starlark.List)
|
||||
allErrored := true
|
||||
for i := 0; i < errors.Len(); i++ {
|
||||
if errors.Index(i) == starlark.None {
|
||||
allErrored = false
|
||||
}
|
||||
}
|
||||
if !allErrored {
|
||||
t.Errorf("expected all branches to error on cancelled context, errors=%v", errors)
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Cap Enforcement ────────────────────────
|
||||
|
||||
func TestBatchExec_CapEnforcement(t *testing.T) {
|
||||
_, err := execWithBatch(t, `
|
||||
fns = [lambda: i for i in range(9)]
|
||||
batch.exec(fns)
|
||||
`)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for 9 callables, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "max 8") {
|
||||
t.Errorf("error = %q, want to contain 'max 8'", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Empty List ─────────────────────────────
|
||||
|
||||
func TestBatchExec_EmptyList(t *testing.T) {
|
||||
_, err := execWithBatch(t, `batch.exec([])`)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for empty list, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "empty") {
|
||||
t.Errorf("error = %q, want to contain 'empty'", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Non-Callable Element ───────────────────
|
||||
|
||||
func TestBatchExec_NonCallableElement(t *testing.T) {
|
||||
_, err := execWithBatch(t, `batch.exec([1, 2])`)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for non-callable, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "not callable") {
|
||||
t.Errorf("error = %q, want to contain 'not callable'", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Permission Gating ──────────────────────
|
||||
|
||||
func TestBatchExec_PermissionGating(t *testing.T) {
|
||||
sb := New(DefaultConfig())
|
||||
_, err := sb.Exec(context.Background(), "test.star",
|
||||
`batch.exec([lambda: 1])`,
|
||||
map[string]starlark.Value{}, // no batch module
|
||||
)
|
||||
if err == nil {
|
||||
t.Fatal("expected error when batch module not available")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "batch") {
|
||||
t.Errorf("error = %q, want to mention 'batch'", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Frozen Struct Sharing ──────────────────
|
||||
|
||||
func TestBatchExec_FrozenStructSharing(t *testing.T) {
|
||||
// Two branches read the same frozen tuple. Should not race.
|
||||
result, err := execWithBatch(t, `
|
||||
data = (1, 2, 3) # tuples are frozen/immutable
|
||||
|
||||
results, errors = batch.exec([
|
||||
lambda: data[0] + data[1],
|
||||
lambda: data[1] + data[2],
|
||||
])
|
||||
`)
|
||||
if err != nil {
|
||||
t.Fatalf("exec error: %v", err)
|
||||
}
|
||||
|
||||
results := result.Globals["results"].(*starlark.List)
|
||||
|
||||
got0, _ := starlark.AsInt32(results.Index(0))
|
||||
got1, _ := starlark.AsInt32(results.Index(1))
|
||||
if got0 != 3 {
|
||||
t.Errorf("results[0] = %d, want 3", got0)
|
||||
}
|
||||
if got1 != 5 {
|
||||
t.Errorf("results[1] = %d, want 5", got1)
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Nested batch.exec ──────────────────────
|
||||
|
||||
func TestBatchExec_NestedBatchExec(t *testing.T) {
|
||||
result, err := execWithBatch(t, `
|
||||
results, errors = batch.exec([
|
||||
lambda: batch.exec([lambda: 1]),
|
||||
])
|
||||
`)
|
||||
if err != nil {
|
||||
t.Fatalf("exec error: %v", err)
|
||||
}
|
||||
|
||||
// The inner batch.exec should fail with nesting error.
|
||||
errors := result.Globals["errors"].(*starlark.List)
|
||||
errStr, _ := starlark.AsString(errors.Index(0))
|
||||
if !strings.Contains(errStr, "nested") {
|
||||
t.Errorf("errors[0] = %q, want to contain 'nested'", errStr)
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Default Timeout ────────────────────────
|
||||
|
||||
func TestBatchExec_DefaultTimeout(t *testing.T) {
|
||||
// Call without explicit timeout — should succeed.
|
||||
result, err := execWithBatch(t, `
|
||||
results, errors = batch.exec([lambda: 42])
|
||||
`)
|
||||
if err != nil {
|
||||
t.Fatalf("exec error: %v", err)
|
||||
}
|
||||
|
||||
results := result.Globals["results"].(*starlark.List)
|
||||
got, _ := starlark.AsInt32(results.Index(0))
|
||||
if got != 42 {
|
||||
t.Errorf("results[0] = %d, want 42", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Timeout Clamping ───────────────────────
|
||||
|
||||
func TestBatchExec_TimeoutClamp(t *testing.T) {
|
||||
// timeout=0 should be clamped to default, not crash.
|
||||
start := time.Now()
|
||||
result, err := execWithBatch(t, `
|
||||
results, errors = batch.exec([lambda: "ok"], timeout=0)
|
||||
`)
|
||||
elapsed := time.Since(start)
|
||||
if err != nil {
|
||||
t.Fatalf("exec error: %v", err)
|
||||
}
|
||||
|
||||
results := result.Globals["results"].(*starlark.List)
|
||||
if s, _ := starlark.AsString(results.Index(0)); s != "ok" {
|
||||
t.Errorf("results[0] = %v, want 'ok'", results.Index(0))
|
||||
}
|
||||
|
||||
// Should complete quickly (clamped to default 10s, but lambda is instant).
|
||||
if elapsed > 2*time.Second {
|
||||
t.Errorf("took %v, expected fast completion", elapsed)
|
||||
}
|
||||
}
|
||||
@@ -338,6 +338,7 @@ func (r *Runner) buildModulesWithLibCtx(ctx context.Context, packageID string, m
|
||||
|
||||
// Track db permission level: 0=none, 1=read, 2=write
|
||||
dbLevel := 0
|
||||
hasBatchExec := false
|
||||
|
||||
for _, perm := range granted {
|
||||
switch perm {
|
||||
@@ -374,6 +375,9 @@ func (r *Runner) buildModulesWithLibCtx(ctx context.Context, packageID string, m
|
||||
if r.bus != nil {
|
||||
modules["realtime"] = BuildRealtimeModule(ctx, r.bus, packageID)
|
||||
}
|
||||
|
||||
case models.ExtPermBatchExec:
|
||||
hasBatchExec = true
|
||||
}
|
||||
}
|
||||
|
||||
@@ -404,5 +408,12 @@ func (r *Runner) buildModulesWithLibCtx(ctx context.Context, packageID string, m
|
||||
modules["lib"] = BuildLibModule(ctx, r, packageID, rc, lc)
|
||||
}
|
||||
|
||||
// Concurrent execution primitive — wired after all other modules
|
||||
// so the runner reference can construct per-branch module sets if
|
||||
// needed in future (Option B).
|
||||
if hasBatchExec {
|
||||
modules["batch"] = BuildBatchModule(ctx, r, packageID, manifest, rc, lc)
|
||||
}
|
||||
|
||||
return modules, nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user