core/docs/DESIGN-EXT-CONNECTIONS-LIBRARIES.md

DESIGN — Extension Connections & Library Packages

Two platform primitives. Extensions that talk to external services need scoped credential management. Extensions that share logic and data need a dependency mechanism with clean boundaries.

---

Part 1: Extension Connections

Problem

Extensions integrating with external services need credentials and endpoint config. The current settings mechanism is flat key-value. It breaks when a user works with two instances of the same service, when teams share bot tokens alongside personal tokens, or when multiple extensions need the same credentials.

Design

A connection concept. Same scope/resolution pattern as LLM providers — scoped CRUD, resolution chain — but generic, owned by package manifests, and sharable across packages.

Manifest Declaration

{
  "id": "git-board",
  "connections": [
    {
      "type": "gitea",
      "label": "Gitea Instance",
      "fields": {
        "base_url":  {"type": "url",    "required": true,  "label": "Server URL"},
        "api_token": {"type": "secret", "required": true,  "label": "API Token"},
        "org":       {"type": "string", "required": false, "label": "Default Org"}
      },
      "scopes": ["global", "team", "personal"]
    },
    {
      "type": "github",
      "label": "GitHub",
      "fields": {
        "api_token": {"type": "secret", "required": true,  "label": "Personal Access Token"},
        "org":       {"type": "string", "required": false, "label": "Default Org"}
      },
      "scopes": ["global", "personal"]
    }
  ]
}

Field	Description
type	Connection type identifier. Shared by name — if two packages declare type: "gitea", users configure once and both packages resolve it. Prefix to namespace: "git-board:internal".
label	Human-readable name for the UI.
fields	Config form schema. Types: string, url, secret, number, boolean, select.
scopes	Allowed scopes. Subset of ["global", "team", "personal"].

Storage

CREATE TABLE ext_connections (
    id          TEXT PRIMARY KEY,
    type        TEXT NOT NULL,
    package_id  TEXT NOT NULL,          -- declaring package (UI attribution)
    scope       TEXT NOT NULL,          -- global | team | personal
    owner_id    TEXT NOT NULL,          -- '' for global, team_id, or user_id
    name        TEXT NOT NULL,          -- user label: "Work Gitea"
    config      TEXT NOT NULL,          -- JSON, secrets encrypted at rest
    is_active   BOOLEAN DEFAULT TRUE,
    created_at  TIMESTAMP,
    updated_at  TIMESTAMP,
    UNIQUE(type, scope, owner_id, name)
);

Secrets use the same vault pattern as provider_configs.api_key.

Resolution Chain

Personal → Team → Global. Same walk as provider resolution.

conn = connections.get("gitea")                        # scope chain
conn = connections.get("gitea", name="Work Gitea")     # explicit
all  = connections.list("gitea")                       # all available

API Endpoints

# Admin (global)
POST/GET/PUT/DELETE  /api/v1/admin/connections[/:id]

# Team
POST/GET/PUT/DELETE  /api/v1/teams/:teamId/connections[/:id]

# Personal
POST/GET/PUT/DELETE  /api/v1/connections[/:id]

# Resolution
GET  /api/v1/connections/resolve?type=gitea&name=Work+Gitea

UI

Three surfaces matching the provider config pattern:

• Admin → Connections — global. Type picker from installed packages.
• Team Admin → Connections — team-scoped.
• Settings → Connections — personal.

---

Part 2: Library Packages

Problem

Extensions duplicate everything. Two packages talking to Gitea write the same API client, the same auth logic, the same pagination. If they cache data, each creates its own tables with the same schema. There's no mechanism for sharing code, data access, or connection types across packages.

Core Principle

Libraries are services, not shared databases.

A library owns its data privately. Consumers access it through exported Starlark functions or REST endpoints — never by querying the library's tables directly. The library controls validation, access patterns, schema evolution, and data integrity. Consumers are decoupled from the physical storage.

Same principle as microservices vs shared databases: share the schema, share the coupling. Share the API, share the contract.

Design

A new package type: library. Libraries can provide:

• Starlark functions — server-side consumers call via lib.load()
• REST endpoints — browser-side consumers call via HTTP
• Connection types — inherited by consumers
• Private DB tables — optional, never exposed directly
• Any combination of the above

Libraries do not have: surfaces, LLM tools, or pipe filters.

Package Type Taxonomy

Type	Surface	Starlark exports	REST endpoints	LLM tools	Private DB	Depends on libs
surface	✓	—	—	—	—	via REST
extension	—	—	—	✓	✓	✓
library	—	✓	✓	—	✓	✓
full	✓	✓	✓	✓	✓	✓

Library Manifest

{
  "id": "gitea-client",
  "title": "Gitea API Client",
  "type": "library",
  "tier": "starlark",
  "version": "1.0.0",
  "description": "Shared Gitea client — connections, API, and data caching.",

  "permissions": ["api.http", "db.read", "db.write"],

  "exports": [
    "get_repos", "sync_repos",
    "get_issues", "get_issue", "create_issue", "update_issue",
    "get_prs", "get_ci_status",
    "search_cached_issues"
  ],

  "api_routes": [
    {"method": "GET",  "path": "/repos"},
    {"method": "GET",  "path": "/issues"},
    {"method": "GET",  "path": "/issues/*"},
    {"method": "POST", "path": "/issues"},
    {"method": "GET",  "path": "/prs"},
    {"method": "GET",  "path": "/ci/*"},
    {"method": "POST", "path": "/sync"}
  ],

  "connections": [
    {
      "type": "gitea",
      "label": "Gitea Instance",
      "fields": {
        "base_url":  {"type": "url",    "required": true},
        "api_token": {"type": "secret", "required": true},
        "org":       {"type": "string", "required": false}
      },
      "scopes": ["global", "team", "personal"]
    }
  ],

  "db_tables": [
    {
      "name": "repos",
      "columns": {
        "connection_id": "text", "full_name": "text",
        "owner": "text", "name": "text",
        "description": "text", "html_url": "text",
        "open_issues": "integer", "synced_at": "text"
      }
    },
    {
      "name": "issues",
      "columns": {
        "connection_id": "text", "repo_full_name": "text",
        "number": "integer", "title": "text", "state": "text",
        "labels": "text", "assignee": "text", "body": "text",
        "created_at": "text", "synced_at": "text"
      }
    }
  ],

  "schema_version": 1
}

Physical tables: ext_gitea_client_repos, ext_gitea_client_issues. Private to the library. The db module remains package-scoped. physicalTable() is unchanged — no cross-package table access.

script.star (library)

Two entry points. exports for Starlark consumers. on_request for REST consumers. Both use the same internal logic. The DB is an implementation detail.

# ═══════════════════════════════════════════
#  Exported functions (Starlark consumers)
# ═══════════════════════════════════════════

def get_repos(conn):
    """Fetch repos live from Gitea API."""
    url = conn["base_url"] + "/api/v1/repos/search?limit=50"
    resp = http.get(url=url, headers=_auth(conn))
    if int(resp["status"]) >= 400:
        return None
    return json.decode(resp["body"])

def sync_repos(conn):
    """Fetch repos and cache locally. Returns count."""
    repos = get_repos(conn)
    if repos == None:
        return 0
    conn_id = conn.get("id", "")
    for row in db.query("repos", filters={"connection_id": conn_id}):
        db.delete("repos", row["id"])
    for r in repos:
        db.insert("repos", {
            "connection_id": conn_id,
            "full_name": r.get("full_name", ""),
            "owner": r.get("owner", {}).get("login", ""),
            "name": r.get("name", ""),
            "description": r.get("description", ""),
            "html_url": r.get("html_url", ""),
            "open_issues": int(r.get("open_issues_count", 0)),
            "synced_at": "",
        })
    return len(repos)

def get_issues(conn, owner, repo, state="open"):
    """Fetch issues live from Gitea API."""
    url = (conn["base_url"] + "/api/v1/repos/" + owner + "/" + repo
           + "/issues?state=" + state + "&limit=50&type=issues")
    resp = http.get(url=url, headers=_auth(conn))
    if int(resp["status"]) >= 400:
        return None
    return json.decode(resp["body"])

def search_cached_issues(conn_id, repo_full_name, state=None):
    """Query locally cached issues. No API call."""
    filters = {"connection_id": conn_id, "repo_full_name": repo_full_name}
    if state:
        filters["state"] = state
    return db.query("issues", filters=filters, order="-synced_at", limit=200)

def create_issue(conn, owner, repo, title, body="", labels=None):
    """Create issue via Gitea API."""
    payload = {"title": title}
    if body:
        payload["body"] = body
    if labels:
        payload["labels"] = labels
    url = conn["base_url"] + "/api/v1/repos/" + owner + "/" + repo + "/issues"
    resp = http.post(url=url, body=json.encode(payload),
                     headers=_auth_json(conn))
    if int(resp["status"]) >= 400:
        return None
    return json.decode(resp["body"])

# ... update_issue, get_prs, get_ci_status same pattern ...


# ═══════════════════════════════════════════
#  REST endpoints (browser / external)
# ═══════════════════════════════════════════

def on_request(req):
    path   = req["path"]
    method = req["method"]
    conn   = connections.get("gitea")
    if not conn:
        return _resp(400, {"error": "no gitea connection configured"})

    if method == "GET" and path == "/repos":
        return _resp(200, {"data": get_repos(conn) or []})

    if method == "POST" and path == "/sync":
        return _resp(200, {"synced": sync_repos(conn)})

    if method == "GET" and path == "/issues":
        q = req.get("query", {})
        owner = _str(q.get("owner", ""))
        repo  = _str(q.get("repo", ""))
        if owner and repo:
            return _resp(200, {"data": get_issues(conn, owner, repo) or []})
        return _resp(200, {"data": db.query("issues", order="-synced_at", limit=200)})

    return _resp(404, {"error": "not found"})

def _auth(conn):
    return {"Authorization": "token " + conn.get("api_token", "")}

def _auth_json(conn):
    return {"Authorization": "token " + conn.get("api_token", ""),
            "Content-Type": "application/json"}

def _resp(status, data):
    return {"status": status, "body": json.encode(data),
            "headers": {"Content-Type": "application/json"}}

def _str(v):
    return str(v) if v != None else ""

The DB calls are all in the library's script. Consumers never see the physical tables.

Consumer: Starlark Path

{
  "id": "ci-monitor",
  "type": "extension",
  "tier": "starlark",
  "dependencies": {"gitea-client": ">=1.0.0"},
  "tools": [{"name": "check_ci", "description": "Check CI status",
             "parameters": {"owner": {"type": "string", "required": true},
                            "repo": {"type": "string", "required": true},
                            "ref": {"type": "string", "required": true}}}]
}

gitea = lib.load("gitea-client")

def on_tool_call(tool_name, params):
    conn = connections.get("gitea")
    if tool_name == "check_ci":
        return gitea.get_ci_status(conn, params["owner"],
                                   params["repo"], params["ref"])
    return {"error": "unknown tool"}

No db_tables. No connections declaration. No permissions for db access. The consumer calls functions.

Consumer: REST Path (browser-only)

A pure type: "surface" package calls the library's REST endpoints:

// git-board/js/main.js — no Starlark, no lib.load
var repos = await fetch(BASE + '/s/gitea-client/api/repos', {
    headers: { 'Authorization': 'Bearer ' + token }
}).then(r => r.json());

await fetch(BASE + '/s/gitea-client/api/sync', {
    method: 'POST',
    headers: { 'Authorization': 'Bearer ' + token }
});

This means git-board can be type: "surface" (browser-only). Tools come from the library. Data comes from the library's REST API. Connections come from the library. The surface just renders.

Permission Model

Libraries have their own permissions, granted by the admin:

• gitea-client gets api.http + db.read + db.write
• ci-monitor gets nothing beyond its own tools

When ci-monitor calls gitea.get_ci_status(conn, ...), the library function runs with the library's permissions. The library can make HTTP calls and write to its own tables regardless of what the consumer has.

The admin trusts the library (by granting permissions). The consumer trusts the library (by declaring a dependency). The library trusts nobody — it validates inputs and controls its own data.

lib.load() Mechanics

1. Check ext_dependencies: caller must declare dependency.
2. Load library's script.star.
3. Build library's module set using library's own permissions.
4. Execute in sandboxed context.
5. Extract globals listed in exports.
6. Return as frozen starlarkstruct.Struct.
7. Cache per sandbox invocation.

The library's db module resolves to ext_gitea_client_*. The consumer's db module (if any) resolves to ext_ci_monitor_*. No cross-contamination. physicalTable() unchanged.

Dependencies

CREATE TABLE ext_dependencies (
    consumer_id   TEXT NOT NULL,
    library_id    TEXT NOT NULL,
    version_spec  TEXT NOT NULL,
    resolved_ver  TEXT NOT NULL,
    PRIMARY KEY (consumer_id, library_id),
    FOREIGN KEY (consumer_id) REFERENCES package_registry(id)
        ON DELETE CASCADE,
    FOREIGN KEY (library_id)  REFERENCES package_registry(id)
        ON DELETE RESTRICT
);

ON DELETE RESTRICT: cannot uninstall a library with consumers. Libraries can depend on libraries. Circular deps rejected at install.

Schema Migrations

Libraries own their migrations. Consumers are unaffected. If a migration changes internal tables, the library's exported functions adapt. Private schema is NOT part of the API surface.

Breaking Changes

API surface = exports + REST endpoints + connection types.

• Remove/rename an export or endpoint → major version bump.
• Change a function's return shape → major version bump.
• Change connection type fields → major version bump.
• Add new exports, endpoints, fields → minor version bump.
• Internal DB changes, bug fixes → patch version bump.

Enforcement deferred. Social contract for now.

---

The Full Stack

┌──────────────────────────────────────────────┐
│           gitea-client (library)             │
│                                              │
│  connections: gitea                          │
│  exports: get_repos, sync_repos, ...         │
│  REST: /repos, /issues, /ci/*, /sync         │
│  private DB: repos, issues  (untouchable)    │
│                                              │
│  permissions: api.http, db.read, db.write    │
└────────┬────────────────┬────────────────┬───┘
         │                │                │
   lib.load()        lib.load()        fetch()
         │                │                │
   ┌─────┴──────┐  ┌──────┴───────┐  ┌────┴──────┐
   │ ci-monitor │  │ code-review  │  │ git-board │
   │ extension  │  │ extension    │  │ surface   │
   │ (starlark) │  │ (starlark)   │  │ (browser) │
   └────────────┘  └──────────────┘  └───────────┘

One library. N consumers. Zero schema coupling.

---

Implementation Phases

Phase 1: Extension Connections

Prerequisite: none.

• ext_connections table + store + handlers + UI
• connections Starlark module
• Three management surfaces (admin / team / personal)

Phase 2: Library Packages

Prerequisite: none. Independent of Phase 1.

• library package type in installer
• ext_dependencies table
• lib.load() with per-library permission context
• Library api_routes (already works)
• Dependency resolution + uninstall protection
• Admin UI: dependency tree

Phase 3: Full Composition

Prerequisite: Phase 1 + 2.

• Libraries declare connection types for consumers
• Build gitea-client reference library
• Migrate git-board to library-backed surface

---

Not Covered

• OAuth flows. Static credentials only.
• Package registry / marketplace. .pkg archives, no central registry.
• Runtime version pinning. Installed version wins.
• Cross-package events. Functions and REST, not event subscriptions.
• Library UI. No surface. Config via package settings + connections.