# DESIGN — Concurrent Execution Primitive (`batch.exec`) **Version:** v0.7.12 **Status:** Implemented **Author:** Jeff / Claude session 2026-04-02 --- ## Problem Starlark is single-threaded by design (`go.starlark.net` enforces one thread per execution). Extensions that need to fan out — calling multiple external APIs, invoking several library functions, or performing independent I/O operations — must do so sequentially. For two `http.post()` calls taking 200ms each, the extension blocks for 400ms regardless of whether the calls are independent. The v0.7.10 `http.batch()` primitive solves the narrow case of parallel HTTP dispatch. But it doesn't help when the work is wrapped in library functions. If a `jira-client` library exposes `create_issue()` and a `confluence-client` library exposes `create_page()`, the extension author must either: 1. Call them sequentially (slow), or 2. Decompose the library calls back into raw `http.post()` parameters to use `http.batch()` (defeats the purpose of having libraries). The platform needs a general-purpose concurrent execution primitive that works with arbitrary Starlark callables — including library exports. --- ## Non-Goals - **Shared mutable state between branches.** Each concurrent branch is fully isolated. No channels, no mutexes, no shared dicts. If branches need to coordinate, they don't belong in `batch.exec`. - **Unlimited concurrency.** A hard cap prevents extensions from spawning unbounded goroutines. This is a fan-out primitive, not a thread pool. - **Automatic retry or circuit breaking.** Error handling is the caller's responsibility. The kernel reports per-branch results and errors. - **Nested `batch.exec()`.** A callable inside `batch.exec` cannot itself call `batch.exec`. This prevents exponential goroutine growth and keeps the concurrency model flat. --- ## Key Insight: Frozen Libraries Are Thread-Safe The reason this works without exotic machinery is `lib.require()`. When a library is loaded via `lib.require()`, its exports are wrapped in a `starlarkstruct.FromStringDict()` — which produces a **frozen** struct. Frozen Starlark values are immutable and safe to read from any number of goroutines concurrently. This is a property of `go.starlark.net`, not something we enforce. The only mutable state in a Starlark execution is: 1. **The `starlark.Thread` itself** — step counter, print buffer, cancel channel. Each branch gets its own thread. 2. **Module instances** — `db`, `http`, `settings`, etc. contain configuration and hold references to shared Go objects (`*sql.DB`, `http.Client`). Each branch gets fresh module instances, but the underlying Go resources (`*sql.DB` connection pool, etc.) are already designed for concurrent access. 3. **Local variables** — thread-local by definition in Starlark. So the construction is: one new `starlark.Thread` + one new module set per branch, with frozen library structs shared read-only across all branches. This is exactly what `triggers/schedule.go` already does for scheduled task execution — `buildRestrictedModules` creates a fresh module set for each cron fire. `batch.exec` generalizes that pattern. --- ## API ```python results, errors = batch.exec([ lambda: jira.create_issue(issue_data), lambda: confluence.create_page(page_data), lambda: slack.post_message(channel, msg), ]) # results[0] = return value of jira.create_issue(), or None on error # errors[0] = None on success, or error string on failure # All three ran concurrently. ``` ### Signature ``` batch.exec(callables, timeout=10) → (results: list, errors: list) ``` **Parameters:** | Param | Type | Description | |-------|------|-------------| | `callables` | `list[callable]` | Starlark callables (lambdas, named functions, bound methods). Max length: 8. | | `timeout` | `int` (optional) | Per-branch timeout in seconds. Default 10. Max 30. Inherits parent context deadline if shorter. | **Returns:** A 2-tuple of equal-length lists. - `results[i]` — the return value of `callables[i]`, or `None` if it errored. - `errors[i]` — `None` if `callables[i]` succeeded, or a string error message if it failed (timeout, step limit, runtime error). **Errors (whole-call):** - `callables` is empty → error - `callables` length > 8 → error - Any element is not callable → error - Permission `batch.exec` not granted → error ### Permission New extension permission: `batch.exec`. Declared in manifest: ```json { "permissions": ["batch.exec"] } ``` This is a separate permission because concurrent execution has resource implications (goroutines, module construction overhead). Extensions that don't need it shouldn't pay for it. The permission doesn't imply any other permissions — the branch inherits whatever modules the calling package already has. --- ## Execution Model ``` batch.exec([fn_a, fn_b, fn_c]) │ ├─── goroutine 1: Thread₁ + Modules₁ → fn_a() → result[0] ├─── goroutine 2: Thread₂ + Modules₂ → fn_b() → result[1] └─── goroutine 3: Thread₃ + Modules₃ → fn_c() → result[2] │ sync.WaitGroup.Wait() │ return (results, errors) ``` ### Per-Branch Construction For each callable in the input list, the kernel: 1. Creates a new `sandbox.Sandbox` with the same `Config` as the parent (same `MaxSteps` limit — each branch gets its own step budget, not a shared one). 2. Calls `runner.buildModulesWithLibCtx()` with the **same** `packageID`, `manifest`, and `RunContext` as the parent invocation. This produces a fresh module set — new `DBModuleConfig`, new `HTTPModuleConfig`, etc. — pointing at the same underlying Go resources (`*sql.DB`, etc.). 3. The `libContext` is **shared** (read path only — cached frozen exports). Library exports are immutable. The `loading` map (cycle detection) is not relevant because libraries are already loaded before `batch.exec` runs. If a branch triggers a new `lib.require()`, it would need its own `libContext` — see Open Questions. 4. Creates a new `starlark.Thread` with the branch's print handler, step limit, and context-based cancellation. 5. Calls `starlark.Call(thread, callable, nil, nil)` — the callable is a zero-arg lambda that closes over its arguments. ### Context & Cancellation Each branch gets a child context derived from the parent with the per-branch timeout applied: ```go branchCtx, cancel := context.WithTimeout(parentCtx, branchTimeout) defer cancel() ``` If the parent context is cancelled (e.g., HTTP request timeout), all branches are cancelled. If one branch exceeds its timeout, only that branch is cancelled — others continue. ### Goroutine Cap Hard limit: **8 concurrent branches.** This is enforced at the API boundary (list length check), not via a semaphore. Rationale: - 8 covers the real-world fan-out patterns (2-5 API calls, small batch operations). Nobody needs 50 concurrent Starlark branches. - Each branch allocates a `starlark.Thread` + module instances. At 8 branches, overhead is bounded at ~8KB per thread + module construction time (~50μs per module set). - No semaphore means no queuing surprises. You get 8, period. --- ## Implementation ### New File: `sandbox/batch_module.go` ```go // BuildBatchModule creates the "batch" module. // Requires the Runner reference for per-branch module construction. func BuildBatchModule( ctx context.Context, runner *Runner, packageID string, manifest map[string]any, rc *RunContext, lc *libContext, ) *starlarkstruct.Module ``` The module holds a reference to the `Runner` — same pattern as `BuildLibModule`. It needs the runner to call `buildModulesWithLibCtx` for each branch. ### Core Implementation Sketch ```go func batchExec(ctx context.Context, runner *Runner, packageID string, manifest map[string]any, rc *RunContext, parentLC *libContext, ) func(*starlark.Thread, *starlark.Builtin, starlark.Tuple, []starlark.Tuple) (starlark.Value, error) { return func(thread *starlark.Thread, b *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple, ) (starlark.Value, error) { var callableList *starlark.List var timeout int = 10 if err := starlark.UnpackArgs(b.Name(), args, kwargs, "callables", &callableList, "timeout?", &timeout, ); err != nil { return nil, err } n := callableList.Len() if n == 0 { return nil, fmt.Errorf("batch.exec: callables list is empty") } if n > 8 { return nil, fmt.Errorf("batch.exec: max 8 callables, got %d", n) } if timeout < 1 || timeout > 30 { timeout = 10 } // Validate all elements are callable. callables := make([]starlark.Callable, n) for i := 0; i < n; i++ { c, ok := callableList.Index(i).(starlark.Callable) if !ok { return nil, fmt.Errorf("batch.exec: element %d is %s, not callable", i, callableList.Index(i).Type()) } callables[i] = c } // Execute concurrently. type branchResult struct { index int value starlark.Value err error } results := make([]starlark.Value, n) errors := make([]starlark.Value, n) var wg sync.WaitGroup for i, callable := range callables { wg.Add(1) go func(idx int, fn starlark.Callable) { defer wg.Done() // Per-branch context with timeout. branchCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second) defer cancel() // Fresh module set for this branch. modules, err := runner.buildModulesWithLibCtx( branchCtx, packageID, manifest, rc, parentLC) if err != nil { results[idx] = starlark.None errors[idx] = starlark.String(err.Error()) return } // Fresh sandbox + thread. sb := New(DefaultConfig()) val, _, callErr := sb.Call(branchCtx, fn, nil, nil) if callErr != nil { results[idx] = starlark.None errors[idx] = starlark.String(callErr.Error()) } else { results[idx] = val errors[idx] = starlark.None } }(i, callable) } wg.Wait() return starlark.Tuple{ starlark.NewList(results), starlark.NewList(errors), }, nil } } ``` ### Runner Wiring In `buildModulesWithLibCtx`, add the permission case: ```go case models.ExtPermBatchExec: // Deferred — wired after module map is complete (needs runner ref). hasBatchExec = true ``` After the module map is assembled: ```go if hasBatchExec { modules["batch"] = BuildBatchModule(ctx, r, packageID, manifest, rc, lc) } ``` ### Permission Constant In `models/permissions.go`: ```go ExtPermBatchExec = "batch.exec" ``` Add to `AllExtensionPermissions` slice. --- ## Callable Closure Semantics The callables passed to `batch.exec` are typically lambdas that close over variables from the calling scope: ```python issue_data = {"summary": "Review Q3 report"} page_data = {"title": "Q3 Report", "body": content} results, errors = batch.exec([ lambda: jira.create_issue(issue_data), lambda: confluence.create_page(page_data), ]) ``` The closed-over values (`issue_data`, `page_data`, `jira`, `confluence`) are references to Starlark values in the calling thread's scope. Two safety properties make this work: 1. **Library exports (`jira`, `confluence`) are frozen.** They were returned by `lib.require()` as `starlarkstruct.FromStringDict()` — deeply immutable. Safe to read from any goroutine. 2. **Dict/list arguments may be mutable**, but Starlark's execution model means the calling thread is **blocked** waiting for `batch.exec` to return. No concurrent mutation is possible because the caller can't execute while the branches are running. This is the same safety model as Go's `sync.WaitGroup` pattern: the goroutine that calls `wg.Wait()` cannot proceed until all goroutines complete, so values passed to goroutines before `wg.Add` are safe to read without locks. --- ## Open Questions ### 1. `lib.require()` Inside Branches If a callable triggers a `lib.require()` that hasn't been cached yet, the shared `libContext.cache` would be written from a goroutine. Options: **A. Prohibit: branches cannot call `lib.require()`.** The branch gets a nil `libContext`, so `lib` module is unavailable inside `batch.exec`. Libraries must be loaded before the batch call. Simplest, safest. **B. Per-branch `libContext` with shared read cache.** Each branch gets its own `libContext` whose `cache` is pre-populated from the parent's cache (snapshot). New loads go into the branch's local cache only. Slightly wasteful if two branches load the same library (loaded twice), but safe. **C. Mutex-protected shared `libContext`.** Add a `sync.RWMutex` to `libContext`. Reads use `RLock`, writes use `Lock`. Minimal overhead, but makes `libContext` aware of concurrency — violates its current assumptions. **Recommendation: Option A for v0.7.11, Option B as follow-up if needed.** In practice, extensions call `lib.require()` at module scope (top of script), not inside request handlers. The lambdas passed to `batch.exec` call methods on already-loaded library structs. Option A covers all real-world patterns. ### 2. Print Output Each branch has its own print buffer (the `output strings.Builder` in `Sandbox.Call`). Options: **A. Discard.** Branch print output is lost. Simple, avoids interleaving. **B. Collect per-branch.** Return a third list: `(results, errors, outputs)`. Useful for debugging but clutters the API. **C. Merge into parent.** Append all branch output to the parent thread's print buffer, prefixed with branch index. Requires passing the parent's `outputMu` and `output` builder — invasive. **Recommendation: Option A for v0.7.11.** `print()` in Starlark is a debugging tool, not a production logging facility. Branch callables that need to report status should return structured data. If debugging demand emerges, Option B is a backward-compatible addition. ### 3. Step Limit Scope Each branch gets its own `MaxSteps` budget (default 1M). Should the total across all branches be capped? **No.** The per-branch cap is sufficient. 8 branches × 1M steps = 8M total, which completes in under a second on any modern hardware. The wall-clock timeout (per-branch, max 30s) is the real resource guard. Adding a cross-branch step budget creates coupling between independent execution paths — branch A's step count shouldn't affect branch B's ability to complete. --- ## Testing | Test | Description | |------|-------------| | Parallel ordering | 3 callables with different sleep durations. Results in input order, not completion order. | | Partial failure | 3 callables, middle one raises error. results = [val, None, val], errors = [None, "err msg", None]. | | Timeout per-branch | One callable sleeps beyond timeout. Others succeed. Timed-out branch returns error. | | Parent cancellation | Cancel parent context mid-execution. All branches cancelled. | | Cap enforcement | List of 9 callables → immediate error, nothing executed. | | Empty list | `batch.exec([])` → error. | | Non-callable element | `batch.exec([1, 2])` → error, nothing executed. | | Permission gating | Package without `batch.exec` permission → module not available. | | Frozen library sharing | Two branches call same frozen library function concurrently. No race. | | Nested batch.exec | Callable inside batch.exec attempts batch.exec → error (module not injected in branch). | | db module isolation | Two branches insert into same table concurrently. Both succeed, no corruption. | | http module isolation | Two branches make HTTP calls with different headers. No cross-contamination. | --- ## Migration No schema changes. No new tables. No new migrations. New permission constant `batch.exec` added to `models/permissions.go`. Extensions must declare the permission in their manifest to access the `batch` module.