Changeset 0.38.0 (#233)
Co-authored-by: gobha <jasafpro@gmail.com> Co-committed-by: gobha <jasafpro@gmail.com>
This commit is contained in:
61
CHANGELOG.md
61
CHANGELOG.md
@@ -1,5 +1,66 @@
|
||||
# Changelog
|
||||
|
||||
## [0.38.0.0] — 2026-03-25
|
||||
|
||||
### Summary
|
||||
|
||||
Multi-file Starlark packages. Promotes Starlark scripts from a single
|
||||
inline blob in the DB manifest to a proper file tree with `load()` support.
|
||||
Prerequisite for library packages (v0.38.2) and large extensions.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Extraction:** `extractableRelPath()` now extracts `star/` directories
|
||||
and bare `*.star` files alongside `js/`, `css/`, `assets/`. Starlark
|
||||
scripts land on disk at install time instead of being serialized into
|
||||
the manifest JSONB.
|
||||
(`server/handlers/packages.go`)
|
||||
- **Sandbox:** New `ExecWithLoader()` accepts an optional `LoadFunc` for
|
||||
resolving `load()` statements. `Exec()` is now a thin wrapper that
|
||||
passes a nil loader (backward compatible).
|
||||
(`server/sandbox/sandbox.go`)
|
||||
- **Runner:** Disk-first script loading via `loadScript()` — reads from
|
||||
`{packagesDir}/{pkgID}/script.star` (or custom `entry_point`), falls
|
||||
back to legacy `_starlark_script` manifest field. Package-scoped
|
||||
`packageLoader()` enables `load()` within a package's directory with
|
||||
path traversal protection, `.star`-only enforcement, and circular
|
||||
dependency detection.
|
||||
(`server/sandbox/runner.go`)
|
||||
- **Wiring:** `SetPackagesDir()` called at startup with
|
||||
`cfg.StoragePath + "/packages"`. Build script includes `star/` in
|
||||
archives.
|
||||
(`server/main.go`, `packages/build.sh`)
|
||||
|
||||
### Added
|
||||
|
||||
- **Entry point validation:** Starlark-tier packages must include
|
||||
`script.star` (or the manifest's `entry_point` value) in the archive.
|
||||
Returns 400 if missing.
|
||||
- **`entry_point` manifest field:** Optional override for the default
|
||||
entry script name (`script.star`).
|
||||
- **SDK test runner:** New `packages` domain with 6 tests covering
|
||||
install validation (missing entry point, with star/, custom entry_point,
|
||||
browser-tier bypass).
|
||||
(`packages/sdk-test-runner/js/domains/packages.js`)
|
||||
|
||||
### Removed
|
||||
|
||||
- **`_starlark_script` injection:** The installer no longer reads
|
||||
`script.star` from the archive and stuffs it into the manifest JSON.
|
||||
Scripts are loaded from disk at runtime. Existing packages with
|
||||
`_starlark_script` in their DB row continue to work (legacy fallback).
|
||||
|
||||
### Documentation
|
||||
|
||||
- **ICD:** Multi-file Starlark section added to `extensions.md` with
|
||||
archive structure, `load()` constraints, and error codes.
|
||||
- **OpenAPI:** Install endpoint description updated with entry point
|
||||
validation and `star/` directory extraction.
|
||||
- **Surfaces ICD:** Archive format updated to include `script.star` and
|
||||
`star/` entries.
|
||||
|
||||
---
|
||||
|
||||
## [0.37.19.0] — 2026-03-24
|
||||
|
||||
### Summary
|
||||
|
||||
@@ -275,6 +275,68 @@ def on_tool_call(call):
|
||||
- All sandbox modules (including `db`) are available per granted permissions
|
||||
- No additional permission is required beyond package being `active`
|
||||
|
||||
### Multi-File Starlark Packages (v0.38.0)
|
||||
|
||||
Starlark packages support `load()` for splitting code across multiple
|
||||
files. The entry point script is `script.star` (or the `entry_point`
|
||||
manifest field). Submodules live in `star/`.
|
||||
|
||||
**Archive structure:**
|
||||
|
||||
```
|
||||
my-extension/
|
||||
├── manifest.json ← metadata only (no _starlark_script)
|
||||
├── script.star ← entry point (required for starlark tier)
|
||||
├── star/ ← optional submodules
|
||||
│ ├── auth.star
|
||||
│ ├── repos.star
|
||||
│ └── helpers.star
|
||||
├── js/ ← optional surface assets
|
||||
└── css/
|
||||
```
|
||||
|
||||
**Manifest `entry_point` field (optional):**
|
||||
|
||||
```json
|
||||
{
|
||||
"tier": "starlark",
|
||||
"entry_point": "script.star"
|
||||
}
|
||||
```
|
||||
|
||||
Default is `script.star`. The installer validates the entry point exists
|
||||
in the archive for starlark-tier packages (returns 400 if missing).
|
||||
|
||||
**`load()` in Starlark scripts:**
|
||||
|
||||
```python
|
||||
# script.star
|
||||
load("star/auth.star", "auth_headers")
|
||||
load("star/repos.star", "get_repos")
|
||||
|
||||
def on_request(req):
|
||||
headers = auth_headers(conn)
|
||||
repos = get_repos(conn)
|
||||
return {"status": 200, "body": json.encode(repos)}
|
||||
```
|
||||
|
||||
**Constraints:**
|
||||
- Package-scoped: can only load files within the package's own directory
|
||||
- `.star` files only — cannot load `.js`, `.json`, etc.
|
||||
- Path traversal (`..`) and absolute paths are rejected
|
||||
- Circular dependencies are detected and rejected
|
||||
- Loaded files get the same injected modules (`db`, `http`, etc.) as the entry point
|
||||
- All loaded files share the same step limit budget (1M ops total)
|
||||
|
||||
**Backward compatibility:** Existing packages with `_starlark_script`
|
||||
in their manifest continue to work. The runner tries disk first, then
|
||||
falls back to the manifest field. Reinstalling extracts `script.star`
|
||||
to disk.
|
||||
|
||||
**Install errors:**
|
||||
- `400` — `starlark package missing entry point "script.star"` (or
|
||||
custom `entry_point` value)
|
||||
|
||||
### Builtin Seeding
|
||||
|
||||
On startup, `SeedBuiltinExtensions()` scans `extensions/builtin/`
|
||||
|
||||
@@ -168,13 +168,15 @@ manifest.json — required (must have "id" and "title")
|
||||
js/ — optional (extracted to /data/surfaces/{id}/js/)
|
||||
css/ — optional (extracted to /data/surfaces/{id}/css/)
|
||||
assets/ — optional (icons, images, etc.)
|
||||
script.star — optional (Starlark entry point, v0.38.0)
|
||||
star/ — optional (Starlark submodules, v0.38.0)
|
||||
```
|
||||
|
||||
The archive may have a single top-level directory prefix (e.g.,
|
||||
`my-surface/manifest.json`) — the installer strips it when locating
|
||||
`manifest.json` and when extracting `js/`, `css/`, and `assets/`
|
||||
directories. Files outside these directories (e.g., `script.js` at
|
||||
root) are not extracted.
|
||||
`manifest.json` and when extracting `js/`, `css/`, `assets/`, `star/`
|
||||
directories and bare `.star` files. Files outside these directories
|
||||
(e.g., `script.js` at root) are not extracted.
|
||||
|
||||
See [EXTENSION-SURFACES.md](../EXTENSION-SURFACES.md) for the full
|
||||
authoring guide including manifest schema, platform API, and CSS
|
||||
|
||||
@@ -279,7 +279,7 @@ See design docs for full specifications.
|
||||
|
||||
| Version | Summary | Design Doc |
|
||||
|---------|---------|------------|
|
||||
| v0.38.0 | Multi-file Starlark | [DESIGN-MULTI-FILE-STARLARK.md](DESIGN-MULTI-FILE-STARLARK.md) — `load()` support, disk-based scripts, package-scoped loader. Prerequisite for libraries. 5 backend changesets. |
|
||||
| v0.38.0 ✅ | Multi-file Starlark | [DESIGN-MULTI-FILE-STARLARK.md](DESIGN-MULTI-FILE-STARLARK.md) — `load()` support, disk-based scripts, package-scoped loader. Prerequisite for libraries. 5 backend changesets. |
|
||||
| v0.38.1 | Extension Connections | [DESIGN-EXT-CONNECTIONS-LIBRARIES.md](DESIGN-EXT-CONNECTIONS-LIBRARIES.md) Part 1 — `ext_connections` table, scoped CRUD (personal → team → global resolution), `connections` Starlark module, 3 management UIs. |
|
||||
| v0.38.2 | Library Packages | Part 2 — `library` package type, `ext_dependencies` table, `lib.load()` with per-library permission context, dependency resolution, uninstall protection. |
|
||||
| v0.38.3 | Full Composition | Part 3 — Libraries declare connection types for consumers, reference `gitea-client` library. |
|
||||
|
||||
@@ -38,6 +38,7 @@ build_package() {
|
||||
[ -d "$dir/css" ] && dirs="$dirs css/"
|
||||
[ -d "$dir/assets" ] && dirs="$dirs assets/"
|
||||
[ -f "$dir/script.star" ] && dirs="$dirs script.star"
|
||||
[ -d "$dir/star" ] && dirs="$dirs star/"
|
||||
[ -d "$dir/migrations" ] && dirs="$dirs migrations/"
|
||||
|
||||
(cd "$dir" && zip -qr "$out" manifest.json $dirs)
|
||||
|
||||
237
packages/sdk-test-runner/js/domains/packages.js
Normal file
237
packages/sdk-test-runner/js/domains/packages.js
Normal file
@@ -0,0 +1,237 @@
|
||||
/**
|
||||
* SDK Test Runner — Domain: packages (v0.38.0)
|
||||
*
|
||||
* Tests package install validation for multi-file Starlark.
|
||||
* Requires admin role.
|
||||
*/
|
||||
(function () {
|
||||
'use strict';
|
||||
var T = window.SDKR;
|
||||
if (!T) return;
|
||||
|
||||
// ── Minimal zip builder ──────────────────────────────────
|
||||
// Creates a valid zip archive in memory from an array of
|
||||
// {name: string, content: string} entries. No compression
|
||||
// (STORED method) — sufficient for small test payloads.
|
||||
|
||||
function buildZip(entries) {
|
||||
var localHeaders = [];
|
||||
var centralHeaders = [];
|
||||
var offset = 0;
|
||||
|
||||
entries.forEach(function (entry) {
|
||||
var nameBytes = new TextEncoder().encode(entry.name);
|
||||
var dataBytes = new TextEncoder().encode(entry.content);
|
||||
|
||||
// Local file header (30 + name + data)
|
||||
var local = new Uint8Array(30 + nameBytes.length + dataBytes.length);
|
||||
var dv = new DataView(local.buffer);
|
||||
dv.setUint32(0, 0x04034b50, true); // signature
|
||||
dv.setUint16(4, 20, true); // version needed
|
||||
dv.setUint16(6, 0, true); // flags
|
||||
dv.setUint16(8, 0, true); // method: STORED
|
||||
dv.setUint16(10, 0, true); // mod time
|
||||
dv.setUint16(12, 0, true); // mod date
|
||||
dv.setUint32(14, crc32(dataBytes), true);
|
||||
dv.setUint32(18, dataBytes.length, true); // compressed
|
||||
dv.setUint32(22, dataBytes.length, true); // uncompressed
|
||||
dv.setUint16(26, nameBytes.length, true);
|
||||
dv.setUint16(28, 0, true); // extra length
|
||||
local.set(nameBytes, 30);
|
||||
local.set(dataBytes, 30 + nameBytes.length);
|
||||
localHeaders.push(local);
|
||||
|
||||
// Central directory header (46 + name)
|
||||
var central = new Uint8Array(46 + nameBytes.length);
|
||||
var cdv = new DataView(central.buffer);
|
||||
cdv.setUint32(0, 0x02014b50, true); // signature
|
||||
cdv.setUint16(4, 20, true); // version made by
|
||||
cdv.setUint16(6, 20, true); // version needed
|
||||
cdv.setUint16(8, 0, true); // flags
|
||||
cdv.setUint16(10, 0, true); // method
|
||||
cdv.setUint16(12, 0, true); // mod time
|
||||
cdv.setUint16(14, 0, true); // mod date
|
||||
cdv.setUint32(16, crc32(dataBytes), true);
|
||||
cdv.setUint32(20, dataBytes.length, true);
|
||||
cdv.setUint32(24, dataBytes.length, true);
|
||||
cdv.setUint16(28, nameBytes.length, true);
|
||||
cdv.setUint16(30, 0, true); // extra length
|
||||
cdv.setUint16(32, 0, true); // comment length
|
||||
cdv.setUint16(34, 0, true); // disk number
|
||||
cdv.setUint16(36, 0, true); // internal attrs
|
||||
cdv.setUint32(38, 0, true); // external attrs
|
||||
cdv.setUint32(42, offset, true); // local header offset
|
||||
central.set(nameBytes, 46);
|
||||
centralHeaders.push(central);
|
||||
|
||||
offset += local.length;
|
||||
});
|
||||
|
||||
// End of central directory
|
||||
var centralSize = centralHeaders.reduce(function (sum, c) { return sum + c.length; }, 0);
|
||||
var eocd = new Uint8Array(22);
|
||||
var edv = new DataView(eocd.buffer);
|
||||
edv.setUint32(0, 0x06054b50, true);
|
||||
edv.setUint16(4, 0, true);
|
||||
edv.setUint16(6, 0, true);
|
||||
edv.setUint16(8, entries.length, true);
|
||||
edv.setUint16(10, entries.length, true);
|
||||
edv.setUint32(12, centralSize, true);
|
||||
edv.setUint32(16, offset, true);
|
||||
edv.setUint16(20, 0, true);
|
||||
|
||||
var total = offset + centralSize + 22;
|
||||
var result = new Uint8Array(total);
|
||||
var pos = 0;
|
||||
localHeaders.forEach(function (h) { result.set(h, pos); pos += h.length; });
|
||||
centralHeaders.forEach(function (h) { result.set(h, pos); pos += h.length; });
|
||||
result.set(eocd, pos);
|
||||
|
||||
return new Blob([result], { type: 'application/zip' });
|
||||
}
|
||||
|
||||
// CRC-32 (IEEE)
|
||||
var crcTable = null;
|
||||
function crc32(bytes) {
|
||||
if (!crcTable) {
|
||||
crcTable = new Uint32Array(256);
|
||||
for (var n = 0; n < 256; n++) {
|
||||
var c = n;
|
||||
for (var k = 0; k < 8; k++) c = (c & 1) ? (0xEDB88320 ^ (c >>> 1)) : (c >>> 1);
|
||||
crcTable[n] = c;
|
||||
}
|
||||
}
|
||||
var crc = 0xFFFFFFFF;
|
||||
for (var i = 0; i < bytes.length; i++) crc = crcTable[(crc ^ bytes[i]) & 0xFF] ^ (crc >>> 8);
|
||||
return (crc ^ 0xFFFFFFFF) >>> 0;
|
||||
}
|
||||
|
||||
// ── Helpers ──────────────────────────────────────────────
|
||||
|
||||
function makeManifest(overrides) {
|
||||
var base = {
|
||||
id: 'sdk-test-pkg-' + Date.now(),
|
||||
title: 'SDK Test Package',
|
||||
type: 'extension',
|
||||
tier: 'starlark',
|
||||
version: '0.0.1',
|
||||
tools: [{ name: 'noop', description: 'test', parameters: { type: 'object', properties: {} } }]
|
||||
};
|
||||
if (overrides) {
|
||||
Object.keys(overrides).forEach(function (k) { base[k] = overrides[k]; });
|
||||
}
|
||||
return base;
|
||||
}
|
||||
|
||||
function makePkgFile(manifest, extraFiles) {
|
||||
var entries = [{ name: 'manifest.json', content: JSON.stringify(manifest) }];
|
||||
if (extraFiles) entries = entries.concat(extraFiles);
|
||||
var blob = buildZip(entries);
|
||||
return new File([blob], manifest.id + '.pkg', { type: 'application/zip' });
|
||||
}
|
||||
|
||||
// ── Tests ────────────────────────────────────────────────
|
||||
|
||||
T.registerDomain('packages', async function () {
|
||||
|
||||
if (!window.sw || !sw.isAdmin) {
|
||||
await T.test('packages', 'guard', 'skip — not admin', async function () {
|
||||
T.assert(true, 'skipped (user is not admin)');
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// ── v0.38.0: Starlark entry point validation ──
|
||||
|
||||
await T.test('packages', 'install-missing-star', 'install starlark pkg without script.star → 400', async function () {
|
||||
var manifest = makeManifest();
|
||||
var file = makePkgFile(manifest); // no script.star
|
||||
try {
|
||||
await sw.api.admin.packages.install(file);
|
||||
T.assert(false, 'should have rejected');
|
||||
} catch (e) {
|
||||
var msg = (e && e.message) || String(e);
|
||||
T.assert(
|
||||
msg.indexOf('400') >= 0 || msg.indexOf('missing entry point') >= 0 || msg.indexOf('error') >= 0,
|
||||
'expected 400 for missing entry point, got: ' + msg
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
await T.test('packages', 'install-with-star', 'install starlark pkg with script.star → 200', async function () {
|
||||
var manifest = makeManifest();
|
||||
var file = makePkgFile(manifest, [
|
||||
{ name: 'script.star', content: 'def on_tool_call(call):\n return {"ok": True}\n' }
|
||||
]);
|
||||
var r = await sw.api.admin.packages.install(file);
|
||||
T.assert(r && r.id === manifest.id, 'expected install success with id');
|
||||
|
||||
// Cleanup: delete the test package
|
||||
T.cleanup.push(async function () {
|
||||
try { await sw.api.admin.packages.del(manifest.id); } catch (_) {}
|
||||
});
|
||||
});
|
||||
|
||||
await T.test('packages', 'install-with-submodules', 'install starlark pkg with star/ submodules → 200', async function () {
|
||||
var manifest = makeManifest();
|
||||
var file = makePkgFile(manifest, [
|
||||
{ name: 'script.star', content: 'load("star/helpers.star", "greet")\ndef on_tool_call(call):\n return {"msg": greet()}\n' },
|
||||
{ name: 'star/helpers.star', content: 'def greet():\n return "hello"\n' }
|
||||
]);
|
||||
var r = await sw.api.admin.packages.install(file);
|
||||
T.assert(r && r.id === manifest.id, 'expected install success with id');
|
||||
|
||||
// Cleanup
|
||||
T.cleanup.push(async function () {
|
||||
try { await sw.api.admin.packages.del(manifest.id); } catch (_) {}
|
||||
});
|
||||
});
|
||||
|
||||
await T.test('packages', 'install-browser-tier', 'install browser-tier pkg without script.star → 200 (no validation)', async function () {
|
||||
var manifest = makeManifest({ tier: 'browser', type: 'surface', route: '/s/sdk-test-browser-' + Date.now() });
|
||||
// Remove tools — browser surfaces don't need them
|
||||
delete manifest.tools;
|
||||
var file = makePkgFile(manifest, [
|
||||
{ name: 'js/main.js', content: '// noop' }
|
||||
]);
|
||||
var r = await sw.api.admin.packages.install(file);
|
||||
T.assert(r && r.id === manifest.id, 'browser tier should not require script.star');
|
||||
|
||||
// Cleanup
|
||||
T.cleanup.push(async function () {
|
||||
try { await sw.api.admin.packages.del(manifest.id); } catch (_) {}
|
||||
});
|
||||
});
|
||||
|
||||
await T.test('packages', 'install-custom-entry', 'install starlark pkg with custom entry_point → 200', async function () {
|
||||
var manifest = makeManifest({ entry_point: 'main.star' });
|
||||
var file = makePkgFile(manifest, [
|
||||
{ name: 'main.star', content: 'def on_tool_call(call):\n return {"ok": True}\n' }
|
||||
]);
|
||||
var r = await sw.api.admin.packages.install(file);
|
||||
T.assert(r && r.id === manifest.id, 'custom entry_point should work');
|
||||
|
||||
// Cleanup
|
||||
T.cleanup.push(async function () {
|
||||
try { await sw.api.admin.packages.del(manifest.id); } catch (_) {}
|
||||
});
|
||||
});
|
||||
|
||||
await T.test('packages', 'install-custom-entry-missing', 'install starlark pkg with missing custom entry_point → 400', async function () {
|
||||
var manifest = makeManifest({ entry_point: 'main.star' });
|
||||
var file = makePkgFile(manifest, [
|
||||
{ name: 'script.star', content: 'x = 1\n' } // wrong name
|
||||
]);
|
||||
try {
|
||||
await sw.api.admin.packages.install(file);
|
||||
T.assert(false, 'should have rejected missing custom entry point');
|
||||
} catch (e) {
|
||||
var msg = (e && e.message) || String(e);
|
||||
T.assert(
|
||||
msg.indexOf('400') >= 0 || msg.indexOf('missing entry point') >= 0 || msg.indexOf('error') >= 0,
|
||||
'expected 400 for missing custom entry point'
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
})();
|
||||
@@ -74,6 +74,7 @@
|
||||
'domains/tasks.js',
|
||||
'domains/misc.js',
|
||||
'domains/admin.js',
|
||||
'domains/packages.js',
|
||||
// UI (must come last)
|
||||
'ui.js'
|
||||
];
|
||||
|
||||
@@ -241,31 +241,25 @@ func (h *PackageHandler) InstallPackage(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// v0.37.15: Read script.star from archive if present.
|
||||
// Keeps the manifest clean — authors write a file, installer injects
|
||||
// it as _starlark_script for the sandbox runner.
|
||||
if _, hasInline := manifest["_starlark_script"]; !hasInline {
|
||||
// v0.38.0: Entry point validation for starlark-tier packages.
|
||||
// Scripts are loaded from disk at runtime — no _starlark_script injection.
|
||||
if manifestTier, _ := manifest["tier"].(string); manifestTier == "starlark" {
|
||||
entryPoint := "script.star"
|
||||
if ep, ok := manifest["entry_point"].(string); ok && ep != "" {
|
||||
entryPoint = ep
|
||||
}
|
||||
found := false
|
||||
for _, f := range zr.File {
|
||||
name := f.Name
|
||||
base := filepath.Base(name)
|
||||
if base == "script.star" && !f.FileInfo().IsDir() {
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
data, err := io.ReadAll(rc)
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
script := string(data)
|
||||
if strings.TrimSpace(script) != "" {
|
||||
manifest["_starlark_script"] = script
|
||||
log.Printf("[packages] Injected script.star (%d bytes) into manifest", len(data))
|
||||
}
|
||||
base := filepath.Base(f.Name)
|
||||
if base == entryPoint && !f.FileInfo().IsDir() {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": fmt.Sprintf("starlark package missing entry point %q", entryPoint)})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Validate required fields
|
||||
@@ -487,17 +481,25 @@ func (h *PackageHandler) InstallPackage(c *gin.Context) {
|
||||
}
|
||||
|
||||
// extractableRelPath returns the relative path for a zip entry if it
|
||||
// belongs to an extractable static directory (js/, css/, assets/).
|
||||
// belongs to an extractable directory (js/, css/, assets/, star/) or
|
||||
// is a bare .star file at the archive root.
|
||||
// Returns "" if the file should be skipped.
|
||||
func extractableRelPath(name string) string {
|
||||
staticPrefixes := []string{"js/", "css/", "assets/"}
|
||||
staticPrefixes := []string{"js/", "css/", "assets/", "star/"}
|
||||
|
||||
// Direct match (flat archive without package-id/ wrapper)
|
||||
for _, p := range staticPrefixes {
|
||||
if strings.HasPrefix(name, p) {
|
||||
return name
|
||||
}
|
||||
}
|
||||
|
||||
// Bare .star file at root (e.g. "script.star")
|
||||
if strings.HasSuffix(name, ".star") && !strings.Contains(name, "/") {
|
||||
return name
|
||||
}
|
||||
|
||||
// Nested inside package-id/ directory
|
||||
idx := strings.Index(name, "/")
|
||||
if idx <= 0 {
|
||||
return ""
|
||||
@@ -510,6 +512,11 @@ func extractableRelPath(name string) string {
|
||||
}
|
||||
}
|
||||
|
||||
// Bare .star file inside package-id/ (e.g. "gitea-client/script.star")
|
||||
if strings.HasSuffix(rest, ".star") && !strings.Contains(rest, "/") {
|
||||
return rest
|
||||
}
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
|
||||
@@ -410,6 +410,10 @@ func main() {
|
||||
starlarkRunner.SetProviderResolver(handlers.NewProviderResolverAdapter(stores, keyResolver))
|
||||
// v0.29.2: db module — extension namespaced table access
|
||||
starlarkRunner.SetDB(database.DB, database.IsPostgres())
|
||||
// v0.38.0: disk-based script loading + load() support
|
||||
if cfg.StoragePath != "" {
|
||||
starlarkRunner.SetPackagesDir(cfg.StoragePath + "/packages")
|
||||
}
|
||||
|
||||
// Discover and register active Starlark pre-completion filters
|
||||
filters.DiscoverStarlarkFilters(context.Background(), filterChain, stores, starlarkRunner)
|
||||
|
||||
@@ -22,6 +22,9 @@ import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"go.starlark.net/starlark"
|
||||
|
||||
@@ -43,12 +46,13 @@ type RunContext struct {
|
||||
|
||||
// Runner executes Starlark package scripts with permission-gated modules.
|
||||
type Runner struct {
|
||||
sandbox *Sandbox
|
||||
stores store.Stores
|
||||
notifier NotificationSender // nil = notifications module unavailable
|
||||
resolver ProviderResolver // nil = provider module unavailable
|
||||
db *sql.DB // nil = db module unavailable
|
||||
dbPostgres bool // true = use $N placeholders; false = use ?
|
||||
sandbox *Sandbox
|
||||
stores store.Stores
|
||||
packagesDir string // v0.38.0: disk path for load() support
|
||||
notifier NotificationSender // nil = notifications module unavailable
|
||||
resolver ProviderResolver // nil = provider module unavailable
|
||||
db *sql.DB // nil = db module unavailable
|
||||
dbPostgres bool // true = use $N placeholders; false = use ?
|
||||
}
|
||||
|
||||
// NewRunner creates a runner with the given sandbox and dependencies.
|
||||
@@ -77,8 +81,17 @@ func (r *Runner) SetDB(db *sql.DB, isPostgres bool) {
|
||||
r.dbPostgres = isPostgres
|
||||
}
|
||||
|
||||
// ExecPackage loads a package's script from its manifest and executes it
|
||||
// with modules gated by the package's granted permissions.
|
||||
// SetPackagesDir sets the disk path where package archives are extracted.
|
||||
// Required for load() support — scripts are read from disk at runtime.
|
||||
// v0.38.0.
|
||||
func (r *Runner) SetPackagesDir(dir string) {
|
||||
r.packagesDir = dir
|
||||
}
|
||||
|
||||
// ExecPackage loads a package's script from disk (primary) or manifest
|
||||
// (legacy) and executes it with modules gated by granted permissions.
|
||||
//
|
||||
// v0.38.0: Disk-based loading + package-scoped load() support.
|
||||
//
|
||||
// The RunContext carries per-invocation state (user_id for provider
|
||||
// resolution). Pass nil if no per-invocation context is needed.
|
||||
@@ -94,10 +107,10 @@ func (r *Runner) ExecPackage(ctx context.Context, pkg *store.PackageRegistration
|
||||
return nil, fmt.Errorf("package %q is tier %s, not starlark", pkg.ID, pkg.Tier)
|
||||
}
|
||||
|
||||
// Extract script from manifest
|
||||
script, ok := pkg.Manifest["_starlark_script"].(string)
|
||||
if !ok || script == "" {
|
||||
return nil, fmt.Errorf("package %q has no _starlark_script in manifest", pkg.ID)
|
||||
// Load script from disk (primary) or manifest (legacy)
|
||||
script, err := r.loadScript(pkg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Build modules based on granted permissions
|
||||
@@ -106,9 +119,105 @@ func (r *Runner) ExecPackage(ctx context.Context, pkg *store.PackageRegistration
|
||||
return nil, fmt.Errorf("failed to build modules for %q: %w", pkg.ID, err)
|
||||
}
|
||||
|
||||
// Build package-scoped load callback
|
||||
loader := r.packageLoader(pkg.ID, modules)
|
||||
|
||||
log.Printf(" 🔧 runner: exec %s (%d modules granted)", pkg.ID, len(modules))
|
||||
|
||||
return r.sandbox.Exec(ctx, pkg.ID+".star", script, modules)
|
||||
return r.sandbox.ExecWithLoader(ctx, pkg.ID+"/script.star", script, modules, loader)
|
||||
}
|
||||
|
||||
// loadScript reads the entry point script from disk (primary path)
|
||||
// or falls back to the legacy _starlark_script manifest field.
|
||||
func (r *Runner) loadScript(pkg *store.PackageRegistration) (string, error) {
|
||||
// Primary: read from disk
|
||||
if r.packagesDir != "" {
|
||||
entryPoint := "script.star"
|
||||
if ep, ok := pkg.Manifest["entry_point"].(string); ok && ep != "" {
|
||||
entryPoint = ep
|
||||
}
|
||||
path := filepath.Join(r.packagesDir, pkg.ID, entryPoint)
|
||||
data, err := os.ReadFile(path)
|
||||
if err == nil && len(data) > 0 {
|
||||
return string(data), nil
|
||||
}
|
||||
// Fall through to legacy
|
||||
}
|
||||
|
||||
// Legacy: inline in manifest
|
||||
script, ok := pkg.Manifest["_starlark_script"].(string)
|
||||
if ok && script != "" {
|
||||
return script, nil
|
||||
}
|
||||
|
||||
return "", fmt.Errorf("package %q: no script.star on disk and no _starlark_script in manifest", pkg.ID)
|
||||
}
|
||||
|
||||
// loadEntry tracks cache and cycle detection for the package loader.
|
||||
type loadEntry struct {
|
||||
loading bool
|
||||
globals starlark.StringDict
|
||||
}
|
||||
|
||||
// packageLoader builds a LoadFunc scoped to a package's directory.
|
||||
// Returns nil if no packages directory is configured.
|
||||
func (r *Runner) packageLoader(pkgID string, modules map[string]starlark.Value) LoadFunc {
|
||||
if r.packagesDir == "" {
|
||||
return nil // no disk = no load support
|
||||
}
|
||||
|
||||
pkgDir := filepath.Join(r.packagesDir, pkgID)
|
||||
cache := make(map[string]*loadEntry) // dedup + cycle detection
|
||||
|
||||
return func(thread *starlark.Thread, module string) (starlark.StringDict, error) {
|
||||
// Security: reject path traversal
|
||||
if strings.Contains(module, "..") || filepath.IsAbs(module) {
|
||||
return nil, fmt.Errorf("load: path traversal not allowed: %q", module)
|
||||
}
|
||||
|
||||
// Resolve to package directory
|
||||
resolved := filepath.Join(pkgDir, module)
|
||||
if !strings.HasPrefix(filepath.Clean(resolved), filepath.Clean(pkgDir)) {
|
||||
return nil, fmt.Errorf("load: path escapes package directory: %q", module)
|
||||
}
|
||||
|
||||
// Must be a .star file
|
||||
if !strings.HasSuffix(resolved, ".star") {
|
||||
return nil, fmt.Errorf("load: only .star files can be loaded: %q", module)
|
||||
}
|
||||
|
||||
// Cache / cycle detection
|
||||
if entry, ok := cache[module]; ok {
|
||||
if entry.loading {
|
||||
return nil, fmt.Errorf("load: circular dependency: %q", module)
|
||||
}
|
||||
return entry.globals, nil
|
||||
}
|
||||
|
||||
entry := &loadEntry{loading: true}
|
||||
cache[module] = entry
|
||||
|
||||
// Read and execute
|
||||
data, err := os.ReadFile(resolved)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load: %q not found in package %q", module, pkgID)
|
||||
}
|
||||
|
||||
// Build predeclared with same modules as entry point
|
||||
predeclared := make(starlark.StringDict, len(modules)+1)
|
||||
for k, v := range modules {
|
||||
predeclared[k] = v
|
||||
}
|
||||
|
||||
globals, err := starlark.ExecFile(thread, module, string(data), predeclared)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load: error in %q: %w", module, err)
|
||||
}
|
||||
|
||||
entry.globals = globals
|
||||
entry.loading = false
|
||||
return globals, nil
|
||||
}
|
||||
}
|
||||
|
||||
// CallEntryPoint executes a package script and calls a named function.
|
||||
|
||||
@@ -79,18 +79,28 @@ func New(cfg Config) *Sandbox {
|
||||
return &Sandbox{config: cfg}
|
||||
}
|
||||
|
||||
// LoadFunc resolves load("path") calls to Starlark source.
|
||||
// Returns the module's globals. The sandbox caches results per
|
||||
// thread (Starlark handles this via thread.Load dedup).
|
||||
type LoadFunc func(thread *starlark.Thread, module string) (starlark.StringDict, error)
|
||||
|
||||
// Exec executes a Starlark script within the sandbox.
|
||||
// load() is disabled — all dependencies must be provided via modules.
|
||||
// For load() support, use ExecWithLoader.
|
||||
func (s *Sandbox) Exec(ctx context.Context, filename, source string, modules map[string]starlark.Value) (*Result, error) {
|
||||
return s.ExecWithLoader(ctx, filename, source, modules, nil)
|
||||
}
|
||||
|
||||
// ExecWithLoader executes a Starlark script within the sandbox with
|
||||
// an optional load callback. If loader is nil, load() returns an error.
|
||||
//
|
||||
// Parameters:
|
||||
// - ctx: context for timeout/cancellation
|
||||
// - filename: used in error messages and stack traces
|
||||
// - source: Starlark source code
|
||||
// - modules: predeclared modules injected into the script's namespace
|
||||
// (e.g., {"secrets": secretsModule, "notifications": notifModule})
|
||||
//
|
||||
// The script cannot use load() — all dependencies must be provided
|
||||
// via the modules parameter.
|
||||
func (s *Sandbox) Exec(ctx context.Context, filename, source string, modules map[string]starlark.Value) (*Result, error) {
|
||||
// - loader: resolves load() calls; nil = load() disabled
|
||||
func (s *Sandbox) ExecWithLoader(ctx context.Context, filename, source string, modules map[string]starlark.Value, loader LoadFunc) (*Result, error) {
|
||||
var output strings.Builder
|
||||
var outputMu sync.Mutex
|
||||
|
||||
@@ -101,6 +111,14 @@ func (s *Sandbox) Exec(ctx context.Context, filename, source string, modules map
|
||||
// json.encode / json.decode — universal, no permissions required (pure computation)
|
||||
predeclared["json"] = starlarkjson.Module
|
||||
|
||||
// Default: load() disabled. If loader provided, delegate to it.
|
||||
loadFn := func(_ *starlark.Thread, module string) (starlark.StringDict, error) {
|
||||
return nil, fmt.Errorf("load() is disabled in sandbox (requested %q)", module)
|
||||
}
|
||||
if loader != nil {
|
||||
loadFn = loader
|
||||
}
|
||||
|
||||
thread := &starlark.Thread{
|
||||
Name: s.config.Name,
|
||||
Print: func(_ *starlark.Thread, msg string) {
|
||||
@@ -109,9 +127,7 @@ func (s *Sandbox) Exec(ctx context.Context, filename, source string, modules map
|
||||
output.WriteString(msg)
|
||||
output.WriteByte('\n')
|
||||
},
|
||||
Load: func(_ *starlark.Thread, module string) (starlark.StringDict, error) {
|
||||
return nil, fmt.Errorf("load() is disabled in sandbox (requested %q)", module)
|
||||
},
|
||||
Load: loadFn,
|
||||
}
|
||||
|
||||
if s.config.MaxSteps > 0 {
|
||||
|
||||
@@ -8674,6 +8674,13 @@ paths:
|
||||
- Admin
|
||||
- Extensions
|
||||
summary: Install package from upload
|
||||
description: |
|
||||
Upload a .pkg/.surface/.zip archive containing manifest.json and
|
||||
optional assets. For starlark-tier packages, the archive must
|
||||
include `script.star` (or the manifest's `entry_point` value).
|
||||
Starlark submodules in `star/` are extracted alongside static
|
||||
assets (js/, css/, assets/). The manifest `entry_point` field
|
||||
overrides the default entry script name (v0.38.0).
|
||||
security:
|
||||
- bearerAuth: []
|
||||
responses:
|
||||
|
||||
@@ -152,7 +152,7 @@ export async function boot() {
|
||||
};
|
||||
|
||||
// Marker for idempotency
|
||||
sw._sdk = '0.37.19';
|
||||
sw._sdk = '0.38.0';
|
||||
|
||||
// 8. Expose globally
|
||||
window.sw = sw;
|
||||
|
||||
Reference in New Issue
Block a user