core/docs/DESIGN-batch-exec.md

DESIGN — Concurrent Execution Primitive (batch.exec)

Version: v0.7.12 Status: Implemented Author: Jeff / Claude session 2026-04-02

---

Problem

Starlark is single-threaded by design (go.starlark.net enforces one thread per execution). Extensions that need to fan out — calling multiple external APIs, invoking several library functions, or performing independent I/O operations — must do so sequentially. For two http.post() calls taking 200ms each, the extension blocks for 400ms regardless of whether the calls are independent.

The v0.7.10 http.batch() primitive solves the narrow case of parallel HTTP dispatch. But it doesn't help when the work is wrapped in library functions. If a jira-client library exposes create_issue() and a confluence-client library exposes create_page(), the extension author must either:

1. Call them sequentially (slow), or
2. Decompose the library calls back into raw http.post() parameters to use http.batch() (defeats the purpose of having libraries).

The platform needs a general-purpose concurrent execution primitive that works with arbitrary Starlark callables — including library exports.

---

Non-Goals

• Shared mutable state between branches. Each concurrent branch is fully isolated. No channels, no mutexes, no shared dicts. If branches need to coordinate, they don't belong in batch.exec.
• Unlimited concurrency. A hard cap prevents extensions from spawning unbounded goroutines. This is a fan-out primitive, not a thread pool.
• Automatic retry or circuit breaking. Error handling is the caller's responsibility. The kernel reports per-branch results and errors.
• Nested batch.exec(). A callable inside batch.exec cannot itself call batch.exec. This prevents exponential goroutine growth and keeps the concurrency model flat.

---

Key Insight: Frozen Libraries Are Thread-Safe

The reason this works without exotic machinery is lib.require().

When a library is loaded via lib.require(), its exports are wrapped in a starlarkstruct.FromStringDict() — which produces a frozen struct. Frozen Starlark values are immutable and safe to read from any number of goroutines concurrently. This is a property of go.starlark.net, not something we enforce.

The only mutable state in a Starlark execution is:

1. The starlark.Thread itself — step counter, print buffer, cancel channel. Each branch gets its own thread.
2. Module instances — db, http, settings, etc. contain configuration and hold references to shared Go objects (*sql.DB, http.Client). Each branch gets fresh module instances, but the underlying Go resources (*sql.DB connection pool, etc.) are already designed for concurrent access.
3. Local variables — thread-local by definition in Starlark.

So the construction is: one new starlark.Thread + one new module set per branch, with frozen library structs shared read-only across all branches. This is exactly what triggers/schedule.go already does for scheduled task execution — buildRestrictedModules creates a fresh module set for each cron fire. batch.exec generalizes that pattern.

---

API

results, errors = batch.exec([
    lambda: jira.create_issue(issue_data),
    lambda: confluence.create_page(page_data),
    lambda: slack.post_message(channel, msg),
])

# results[0] = return value of jira.create_issue(), or None on error
# errors[0]  = None on success, or error string on failure
# All three ran concurrently.

Signature

batch.exec(callables, timeout=10) → (results: list, errors: list)

Parameters:

Param	Type	Description
callables	list[callable]	Starlark callables (lambdas, named functions, bound methods). Max length: 8.
timeout	int (optional)	Per-branch timeout in seconds. Default 10. Max 30. Inherits parent context deadline if shorter.

Returns: A 2-tuple of equal-length lists.

• results[i] — the return value of callables[i], or None if it errored.
• errors[i] — None if callables[i] succeeded, or a string error message if it failed (timeout, step limit, runtime error).

Errors (whole-call):

• callables is empty → error
• callables length > 8 → error
• Any element is not callable → error
• Permission batch.exec not granted → error

Permission

New extension permission: batch.exec. Declared in manifest:

{
  "permissions": ["batch.exec"]
}

This is a separate permission because concurrent execution has resource implications (goroutines, module construction overhead). Extensions that don't need it shouldn't pay for it. The permission doesn't imply any other permissions — the branch inherits whatever modules the calling package already has.

---

Execution Model

batch.exec([fn_a, fn_b, fn_c])
    │
    ├─── goroutine 1: Thread₁ + Modules₁ → fn_a() → result[0]
    ├─── goroutine 2: Thread₂ + Modules₂ → fn_b() → result[1]
    └─── goroutine 3: Thread₃ + Modules₃ → fn_c() → result[2]
    │
    sync.WaitGroup.Wait()
    │
    return (results, errors)

Per-Branch Construction

For each callable in the input list, the kernel:

1. Creates a new sandbox.Sandbox with the same Config as the parent (same MaxSteps limit — each branch gets its own step budget, not a shared one).
2. Calls runner.buildModulesWithLibCtx() with the same packageID, manifest, and RunContext as the parent invocation. This produces a fresh module set — new DBModuleConfig, new HTTPModuleConfig, etc. — pointing at the same underlying Go resources (*sql.DB, etc.).
3. The libContext is shared (read path only — cached frozen exports). Library exports are immutable. The loading map (cycle detection) is not relevant because libraries are already loaded before batch.exec runs. If a branch triggers a new lib.require(), it would need its own libContext — see Open Questions.
4. Creates a new starlark.Thread with the branch's print handler, step limit, and context-based cancellation.
5. Calls starlark.Call(thread, callable, nil, nil) — the callable is a zero-arg lambda that closes over its arguments.

Context & Cancellation

Each branch gets a child context derived from the parent with the per-branch timeout applied:

branchCtx, cancel := context.WithTimeout(parentCtx, branchTimeout)
defer cancel()

If the parent context is cancelled (e.g., HTTP request timeout), all branches are cancelled. If one branch exceeds its timeout, only that branch is cancelled — others continue.

Goroutine Cap

Hard limit: 8 concurrent branches. This is enforced at the API boundary (list length check), not via a semaphore. Rationale:

• 8 covers the real-world fan-out patterns (2-5 API calls, small batch operations). Nobody needs 50 concurrent Starlark branches.
• Each branch allocates a starlark.Thread + module instances. At 8 branches, overhead is bounded at ~8KB per thread + module construction time (~50μs per module set).
• No semaphore means no queuing surprises. You get 8, period.

---

Implementation

New File: sandbox/batch_module.go

// BuildBatchModule creates the "batch" module.
// Requires the Runner reference for per-branch module construction.
func BuildBatchModule(
    ctx context.Context,
    runner *Runner,
    packageID string,
    manifest map[string]any,
    rc *RunContext,
    lc *libContext,
) *starlarkstruct.Module

The module holds a reference to the Runner — same pattern as BuildLibModule. It needs the runner to call buildModulesWithLibCtx for each branch.

Core Implementation Sketch

func batchExec(ctx context.Context, runner *Runner, packageID string,
    manifest map[string]any, rc *RunContext, parentLC *libContext,
) func(*starlark.Thread, *starlark.Builtin, starlark.Tuple, []starlark.Tuple) (starlark.Value, error) {
    return func(thread *starlark.Thread, b *starlark.Builtin,
        args starlark.Tuple, kwargs []starlark.Tuple,
    ) (starlark.Value, error) {
        var callableList *starlark.List
        var timeout int = 10

        if err := starlark.UnpackArgs(b.Name(), args, kwargs,
            "callables", &callableList,
            "timeout?", &timeout,
        ); err != nil {
            return nil, err
        }

        n := callableList.Len()
        if n == 0 {
            return nil, fmt.Errorf("batch.exec: callables list is empty")
        }
        if n > 8 {
            return nil, fmt.Errorf("batch.exec: max 8 callables, got %d", n)
        }
        if timeout < 1 || timeout > 30 {
            timeout = 10
        }

        // Validate all elements are callable.
        callables := make([]starlark.Callable, n)
        for i := 0; i < n; i++ {
            c, ok := callableList.Index(i).(starlark.Callable)
            if !ok {
                return nil, fmt.Errorf("batch.exec: element %d is %s, not callable",
                    i, callableList.Index(i).Type())
            }
            callables[i] = c
        }

        // Execute concurrently.
        type branchResult struct {
            index int
            value starlark.Value
            err   error
        }

        results := make([]starlark.Value, n)
        errors := make([]starlark.Value, n)
        var wg sync.WaitGroup

        for i, callable := range callables {
            wg.Add(1)
            go func(idx int, fn starlark.Callable) {
                defer wg.Done()

                // Per-branch context with timeout.
                branchCtx, cancel := context.WithTimeout(ctx,
                    time.Duration(timeout)*time.Second)
                defer cancel()

                // Fresh module set for this branch.
                modules, err := runner.buildModulesWithLibCtx(
                    branchCtx, packageID, manifest, rc, parentLC)
                if err != nil {
                    results[idx] = starlark.None
                    errors[idx] = starlark.String(err.Error())
                    return
                }

                // Fresh sandbox + thread.
                sb := New(DefaultConfig())
                val, _, callErr := sb.Call(branchCtx, fn, nil, nil)

                if callErr != nil {
                    results[idx] = starlark.None
                    errors[idx] = starlark.String(callErr.Error())
                } else {
                    results[idx] = val
                    errors[idx] = starlark.None
                }
            }(i, callable)
        }

        wg.Wait()

        return starlark.Tuple{
            starlark.NewList(results),
            starlark.NewList(errors),
        }, nil
    }
}

Runner Wiring

In buildModulesWithLibCtx, add the permission case:

case models.ExtPermBatchExec:
    // Deferred — wired after module map is complete (needs runner ref).
    hasBatchExec = true

After the module map is assembled:

if hasBatchExec {
    modules["batch"] = BuildBatchModule(ctx, r, packageID, manifest, rc, lc)
}

Permission Constant

In models/permissions.go:

ExtPermBatchExec = "batch.exec"

Add to AllExtensionPermissions slice.

---

Callable Closure Semantics

The callables passed to batch.exec are typically lambdas that close over variables from the calling scope:

issue_data = {"summary": "Review Q3 report"}
page_data  = {"title": "Q3 Report", "body": content}

results, errors = batch.exec([
    lambda: jira.create_issue(issue_data),
    lambda: confluence.create_page(page_data),
])

The closed-over values (issue_data, page_data, jira, confluence) are references to Starlark values in the calling thread's scope. Two safety properties make this work:

1. Library exports (jira, confluence) are frozen. They were returned by lib.require() as starlarkstruct.FromStringDict() — deeply immutable. Safe to read from any goroutine.

2. Dict/list arguments may be mutable, but Starlark's execution model means the calling thread is blocked waiting for batch.exec to return. No concurrent mutation is possible because the caller can't execute while the branches are running.

This is the same safety model as Go's sync.WaitGroup pattern: the goroutine that calls wg.Wait() cannot proceed until all goroutines complete, so values passed to goroutines before wg.Add are safe to read without locks.

---

Open Questions

1. lib.require() Inside Branches

If a callable triggers a lib.require() that hasn't been cached yet, the shared libContext.cache would be written from a goroutine. Options:

A. Prohibit: branches cannot call lib.require(). The branch gets a nil libContext, so lib module is unavailable inside batch.exec. Libraries must be loaded before the batch call. Simplest, safest.

B. Per-branch libContext with shared read cache. Each branch gets its own libContext whose cache is pre-populated from the parent's cache (snapshot). New loads go into the branch's local cache only. Slightly wasteful if two branches load the same library (loaded twice), but safe.

C. Mutex-protected shared libContext. Add a sync.RWMutex to libContext. Reads use RLock, writes use Lock. Minimal overhead, but makes libContext aware of concurrency — violates its current assumptions.

Recommendation: Option A for v0.7.11, Option B as follow-up if needed. In practice, extensions call lib.require() at module scope (top of script), not inside request handlers. The lambdas passed to batch.exec call methods on already-loaded library structs. Option A covers all real-world patterns.

2. Print Output

Each branch has its own print buffer (the output strings.Builder in Sandbox.Call). Options:

A. Discard. Branch print output is lost. Simple, avoids interleaving.

B. Collect per-branch. Return a third list: (results, errors, outputs). Useful for debugging but clutters the API.

C. Merge into parent. Append all branch output to the parent thread's print buffer, prefixed with branch index. Requires passing the parent's outputMu and output builder — invasive.

Recommendation: Option A for v0.7.11. print() in Starlark is a debugging tool, not a production logging facility. Branch callables that need to report status should return structured data. If debugging demand emerges, Option B is a backward-compatible addition.

3. Step Limit Scope

Each branch gets its own MaxSteps budget (default 1M). Should the total across all branches be capped?

No. The per-branch cap is sufficient. 8 branches × 1M steps = 8M total, which completes in under a second on any modern hardware. The wall-clock timeout (per-branch, max 30s) is the real resource guard. Adding a cross-branch step budget creates coupling between independent execution paths — branch A's step count shouldn't affect branch B's ability to complete.

---

Testing

Test	Description
Parallel ordering	3 callables with different sleep durations. Results in input order, not completion order.
Partial failure	3 callables, middle one raises error. results = [val, None, val], errors = [None, "err msg", None].
Timeout per-branch	One callable sleeps beyond timeout. Others succeed. Timed-out branch returns error.
Parent cancellation	Cancel parent context mid-execution. All branches cancelled.
Cap enforcement	List of 9 callables → immediate error, nothing executed.
Empty list	batch.exec([]) → error.
Non-callable element	batch.exec([1, 2]) → error, nothing executed.
Permission gating	Package without batch.exec permission → module not available.
Frozen library sharing	Two branches call same frozen library function concurrently. No race.
Nested batch.exec	Callable inside batch.exec attempts batch.exec → error (module not injected in branch).
db module isolation	Two branches insert into same table concurrently. Both succeed, no corruption.
http module isolation	Two branches make HTTP calls with different headers. No cross-contamination.

---

Migration

No schema changes. No new tables. No new migrations.

New permission constant batch.exec added to models/permissions.go. Extensions must declare the permission in their manifest to access the batch module.