Jeffrey Smith
4a4160300b
Some checks failed
CI/CD / detect-changes (pull_request) Successful in 21s
CI/CD / test-frontend (pull_request) Successful in 5s
CI/CD / test-go-pg (pull_request) Successful in 2m35s
CI/CD / test-sqlite (pull_request) Successful in 3m20s
CI/CD / build-and-deploy (pull_request) Has been cancelled
CI/CD / test-runners (pull_request) Has been cancelled
In Gitea runner pods, docker compose port mapping (3000:80) doesn't expose to the runner container's localhost. Resolve the armature container IP via `docker inspect` and pass it directly to the health check and Playwright test runner. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
853 lines
37 KiB
YAML
853 lines
37 KiB
YAML
# .gitea/workflows/ci.yaml
|
|
# ============================================
|
|
# Armature - CI/CD Pipeline (v0.17.3)
|
|
# ============================================
|
|
# Single unified image (Go backend + nginx frontend).
|
|
# v0.1.0: Dropped FE/BE image split per ROADMAP design decision.
|
|
#
|
|
# Pipeline:
|
|
# 0. Detect changes (path-based gating for all downstream jobs)
|
|
# 1a. Frontend tests — skipped if only BE/docs changed
|
|
# 1b. Go unit tests — all non-DB packages + SQLite integration (race-enabled)
|
|
# 1c. Go test (PG) — PG store + handlers against Postgres (race-enabled)
|
|
# 2. Build + Deploy — skipped if docs-only change
|
|
#
|
|
# Test coverage mapping (no package tested by zero jobs):
|
|
# Unit packages (auto-discovered) → test-sqlite (race)
|
|
# ./handlers/... → test-sqlite (SQLite driver, race)
|
|
# + test-go-pg (PG driver, race)
|
|
# ./store/sqlite/... → test-sqlite (race)
|
|
# ./store/postgres/... → test-go-pg (race)
|
|
# CGO_ENABLED=0 build check → test-sqlite
|
|
#
|
|
# Path gating rules:
|
|
# src/, src/editor/ → frontend tests
|
|
# server/, scripts/db-* → backend tests (PG + SQLite)
|
|
# Dockerfile*, k8s/, .gitea/ → all tests (infra change)
|
|
# docs/, *.md → skip all tests + deploy
|
|
# VERSION, scripts/* → frontend + backend tests
|
|
# Tags (v*) → always full pipeline
|
|
#
|
|
# Deployment mapping (single domain, path-based):
|
|
# PR → FE + BE :dev → armature.DOMAIN/dev/ (DB wipe + fresh schema)
|
|
# Push to main → FE + BE :test → armature.DOMAIN/test/ (migrate only)
|
|
# Tag v* → FE + BE :latest → armature.DOMAIN/ (migrate only)
|
|
# → Unified :latest → Docker Hub (not deployed)
|
|
#
|
|
# Database lifecycle:
|
|
# CI: bootstrap (admin creds) → create DB + role + extensions
|
|
# CI: dev wipe (app creds) → drop tables for fresh install test
|
|
# BE: auto-migrate on startup → schema_migrations tracking
|
|
# BE: bootstrap admin from ARMATURE_ADMIN_* env vars (upsert on every restart)
|
|
#
|
|
# Shared PG safety:
|
|
# - Bootstrap uses IF NOT EXISTS (idempotent)
|
|
# - Dev wipe REFUSES to run on databases not ending in _dev
|
|
# - Admin creds never reach the backend pods
|
|
# - Each env has its own DB name (armature_{dev,test,})
|
|
#
|
|
# Required Gitea Variables:
|
|
# REGISTRY, NAMESPACE, DOMAIN, POSTGRES_HOST
|
|
# PROVIDER — LLM provider type for live tests: "venice", "openai", "anthropic"
|
|
# PROVIDER_URL — provider endpoint (optional, uses default for known providers)
|
|
# SEED_USERS — (optional) CSV: "user:pass:role,..." for dev/test seed accounts
|
|
# STORAGE_CLASS — StorageClass for PVC (e.g. "cephfs", required for file storage)
|
|
# STORAGE_SIZE — PVC capacity (e.g. "10Gi", default: "10Gi")
|
|
# STORAGE_BACKEND — "pvc" or "s3" (default: "pvc")
|
|
#
|
|
# Required Gitea Secrets:
|
|
# POSTGRES_USER, POSTGRES_PASSWORD
|
|
# POSTGRES_ADMIN_USER, POSTGRES_ADMIN_PASSWORD
|
|
# ARMATURE_ADMIN_USERNAME, ARMATURE_ADMIN_PASSWORD, ARMATURE_ADMIN_EMAIL
|
|
# ENCRYPTION_KEY — AES-256 key for API key encryption (openssl rand -base64 32)
|
|
# PROVIDER_KEY — API key for live provider integration tests (optional, tests skip if missing)
|
|
# DOCKERHUB_USERNAME, DOCKERHUB_TOKEN (optional)
|
|
# S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY, S3_SECRET_KEY (only when STORAGE_BACKEND=s3)
|
|
#
|
|
# Global Variables (Gitea org-level):
|
|
# CERT_ISSUER_PROD
|
|
# ============================================
|
|
|
|
name: CI/CD
|
|
|
|
on:
|
|
pull_request:
|
|
types: [opened, synchronize, reopened]
|
|
push:
|
|
branches: [main]
|
|
tags: ['v*']
|
|
|
|
concurrency:
|
|
group: ${{ gitea.workflow }}-${{ gitea.event.pull_request.number || gitea.ref }}
|
|
cancel-in-progress: true
|
|
|
|
env:
|
|
REGISTRY: ${{ vars.REGISTRY }}
|
|
NAMESPACE: ${{ vars.NAMESPACE || 'gobha-ai' }}
|
|
DOMAIN: ${{ vars.DOMAIN || 'gobha.ai' }}
|
|
CERT_ISSUER: ${{ vars.CERT_ISSUER_PROD || 'letsencrypt-prod' }}
|
|
POSTGRES_HOST: ${{ vars.POSTGRES_HOST || 'postgres-primary.postgres.svc.cluster.local' }}
|
|
POSTGRES_PORT: ${{ vars.POSTGRES_PORT || '5432' }}
|
|
IMAGE: ${{ vars.REGISTRY }}/armature
|
|
DOCKERHUB_IMAGE: ${{ vars.DOCKERHUB_IMAGE || 'gobha/armature' }}
|
|
|
|
jobs:
|
|
# ── Stage 0: Detect Changed Paths ──────────────
|
|
# Sets output flags so downstream jobs can skip when irrelevant.
|
|
# Tags always set all flags true (full pipeline for releases).
|
|
detect-changes:
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
frontend: ${{ steps.filter.outputs.frontend }}
|
|
backend: ${{ steps.filter.outputs.backend }}
|
|
infra: ${{ steps.filter.outputs.infra }}
|
|
docs_only: ${{ steps.filter.outputs.docs_only }}
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0 # need history for diff
|
|
|
|
- name: Detect changed paths
|
|
id: filter
|
|
run: |
|
|
# Tags = release → run everything
|
|
if [[ "${{ gitea.ref }}" == refs/tags/v* ]]; then
|
|
echo "frontend=true" >> "$GITHUB_OUTPUT"
|
|
echo "backend=true" >> "$GITHUB_OUTPUT"
|
|
echo "infra=true" >> "$GITHUB_OUTPUT"
|
|
echo "docs_only=false" >> "$GITHUB_OUTPUT"
|
|
echo "🏷️ Tag build — running full pipeline"
|
|
exit 0
|
|
fi
|
|
|
|
# Determine diff base
|
|
if [[ "${{ gitea.event_name }}" == "pull_request" ]]; then
|
|
BASE="${{ gitea.event.pull_request.base.sha }}"
|
|
HEAD="${{ gitea.event.pull_request.head.sha }}"
|
|
else
|
|
# Push to main — compare with previous commit
|
|
BASE="${{ gitea.event.before }}"
|
|
HEAD="${{ gitea.sha }}"
|
|
fi
|
|
|
|
echo "Comparing ${BASE:0:8}..${HEAD:0:8}"
|
|
CHANGED=$(git diff --name-only "${BASE}" "${HEAD}" 2>/dev/null || git diff --name-only HEAD~1 HEAD)
|
|
echo "Changed files:"
|
|
echo "${CHANGED}" | sed 's/^/ /'
|
|
|
|
# Classify
|
|
FE=false; BE=false; INFRA=false; DOCS=false; OTHER=false
|
|
while IFS= read -r file; do
|
|
[[ -z "$file" ]] && continue
|
|
case "$file" in
|
|
src/js/*|src/css/*|src/editor/*|src/index.html|src/sw.js|src/manifest.json|src/vendor/*)
|
|
FE=true ;;
|
|
server/*|scripts/db-*)
|
|
BE=true ;;
|
|
.gitea/*|k8s/*|Dockerfile*|docker-compose*|docker-entrypoint*|nginx.conf)
|
|
INFRA=true ;;
|
|
docs/*|*.md|CHANGELOG.md|LICENSE)
|
|
DOCS=true ;;
|
|
VERSION|scripts/*)
|
|
FE=true; BE=true ;;
|
|
*)
|
|
OTHER=true ;;
|
|
esac
|
|
done <<< "${CHANGED}"
|
|
|
|
# Docs-only: only docs changed, nothing else
|
|
if [[ "$DOCS" == "true" && "$FE" == "false" && "$BE" == "false" && "$INFRA" == "false" && "$OTHER" == "false" ]]; then
|
|
DOCS_ONLY=true
|
|
else
|
|
DOCS_ONLY=false
|
|
fi
|
|
|
|
echo "frontend=${FE}" >> "$GITHUB_OUTPUT"
|
|
echo "backend=${BE}" >> "$GITHUB_OUTPUT"
|
|
echo "infra=${INFRA}" >> "$GITHUB_OUTPUT"
|
|
echo "docs_only=${DOCS_ONLY}" >> "$GITHUB_OUTPUT"
|
|
|
|
echo ""
|
|
echo "━━━ Change Detection ━━━"
|
|
echo " frontend: ${FE}"
|
|
echo " backend: ${BE}"
|
|
echo " infra: ${INFRA}"
|
|
echo " docs_only: ${DOCS_ONLY}"
|
|
|
|
# ── Stage 1a: Frontend Tests ─────────────────
|
|
# API contract tests, model processing, policy wiring audits.
|
|
# Uses Node.js built-in test runner (node --test), zero npm deps.
|
|
#
|
|
# Runs when: frontend files changed, infra changed, or not docs-only.
|
|
# Skipped when: only backend or docs changed.
|
|
test-frontend:
|
|
runs-on: ubuntu-latest
|
|
needs: [detect-changes]
|
|
if: needs.detect-changes.outputs.frontend == 'true' || needs.detect-changes.outputs.infra == 'true'
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Verify Node.js
|
|
run: |
|
|
echo "Node $(node --version)"
|
|
NODE_MAJOR=$(node -v | sed 's/v//' | cut -d. -f1)
|
|
if [ "$NODE_MAJOR" -lt 21 ]; then
|
|
echo "❌ Node >= 21 required for built-in test runner (have v${NODE_MAJOR})"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Syntax check (all JS)
|
|
run: |
|
|
echo "Checking JS syntax..."
|
|
err=0
|
|
while IFS= read -r f; do
|
|
# Feed file as module to node --check via stdin
|
|
node --input-type=module --check < "$f" 2>/dev/null || {
|
|
echo " ✗ $f"
|
|
err=1
|
|
}
|
|
done < <(find src/js -name '*.js' -not -path '*/vendor/*' -not -path '*node_modules*')
|
|
[ "$err" -eq 0 ] && echo " ✓ All JS files passed syntax check"
|
|
exit $err
|
|
|
|
- name: Run frontend tests
|
|
run: node --test src/js/__tests__/*.test.js
|
|
|
|
# ── Stage 1b: Go Unit Tests + SQLite Integration ─────
|
|
# Covers ALL non-Postgres packages via auto-discovery, plus
|
|
# SQLite handler and store integration tests. Race-enabled.
|
|
#
|
|
# Coverage: unit packages (dynamic), ./handlers/... (SQLite),
|
|
# ./store/sqlite/..., CGO_ENABLED=0 build check.
|
|
#
|
|
# Runs when: backend files changed or infra changed.
|
|
# Skipped when: only frontend or docs changed.
|
|
test-sqlite:
|
|
runs-on: ubuntu-latest
|
|
needs: [detect-changes]
|
|
if: needs.detect-changes.outputs.backend == 'true' || needs.detect-changes.outputs.infra == 'true'
|
|
env:
|
|
GOPRIVATE: git.gobha.me/*
|
|
GONOSUMCHECK: git.gobha.me/*
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Setup Go
|
|
uses: actions/setup-go@v5
|
|
with:
|
|
go-version: '1.22'
|
|
|
|
- name: Download and tidy
|
|
working-directory: server
|
|
run: |
|
|
go mod download
|
|
go mod tidy
|
|
|
|
- name: Build check (CGO_ENABLED=0)
|
|
working-directory: server
|
|
run: |
|
|
echo "━━━ SQLite Backend Build Check ━━━"
|
|
CGO_ENABLED=0 go build -o /dev/null .
|
|
echo "✓ Binary compiles with SQLite backend (pure Go, no CGO)"
|
|
|
|
- name: Run unit tests (auto-discovered, no DB)
|
|
working-directory: server
|
|
run: |
|
|
echo "━━━ Unit Tests (all non-DB packages) ━━━"
|
|
|
|
# Auto-discover: everything except store/* and handlers/*
|
|
# This ensures new packages are never silently untested
|
|
UNIT_PKGS=$(go list ./... | grep -v -E '/(store|handlers)(/|$)')
|
|
|
|
echo "Packages under test:"
|
|
echo "${UNIT_PKGS}" | sed 's/^/ /'
|
|
echo ""
|
|
|
|
go test -v -race -count=1 -p 1 -timeout 8m ${UNIT_PKGS}
|
|
echo "✓ Unit tests complete"
|
|
|
|
- name: Run SQLite handler integration tests
|
|
working-directory: server
|
|
env:
|
|
DB_DRIVER: sqlite
|
|
run: |
|
|
echo "━━━ SQLite Integration Tests (handlers + stores) ━━━"
|
|
go test -v -race -count=1 -p 1 -timeout 8m \
|
|
./handlers/... \
|
|
./store/sqlite/...
|
|
echo "✓ SQLite integration tests complete"
|
|
|
|
# ── Stage 1c: Go Test (Postgres) ─────────────
|
|
# Tests ONLY Postgres-dependent packages: store/postgres and
|
|
# handlers (with PG driver). Runs in parallel with test-sqlite.
|
|
#
|
|
# Coverage: ./store/postgres/..., ./handlers/... (PG driver).
|
|
# Handlers are intentionally tested against BOTH drivers.
|
|
#
|
|
# Runs when: backend files changed or infra changed.
|
|
# Skipped when: only frontend or docs changed.
|
|
test-go-pg:
|
|
runs-on: ubuntu-latest
|
|
needs: [detect-changes]
|
|
if: needs.detect-changes.outputs.backend == 'true' || needs.detect-changes.outputs.infra == 'true'
|
|
env:
|
|
GOPRIVATE: git.gobha.me/*
|
|
GONOSUMCHECK: git.gobha.me/*
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Setup Go
|
|
uses: actions/setup-go@v5
|
|
with:
|
|
go-version: '1.22'
|
|
|
|
- name: Download and tidy
|
|
working-directory: server
|
|
run: |
|
|
go mod download
|
|
go mod tidy
|
|
|
|
- name: Bootstrap CI test database
|
|
env:
|
|
PGHOST: ${{ env.POSTGRES_HOST }}
|
|
PGPORT: ${{ env.POSTGRES_PORT }}
|
|
PGUSER: ${{ secrets.POSTGRES_ADMIN_USER }}
|
|
PGPASSWORD: ${{ secrets.POSTGRES_ADMIN_PASSWORD }}
|
|
APP_USER: ${{ secrets.POSTGRES_USER }}
|
|
APP_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
|
|
DB_NAME: armature_ci
|
|
run: |
|
|
echo "━━━ CI Test Database Setup ━━━"
|
|
# Create DB for Go integration tests (admin creds)
|
|
DB_EXISTS=$(psql -tAc "SELECT 1 FROM pg_database WHERE datname = '${DB_NAME}';" postgres || echo "0")
|
|
if [[ "${DB_EXISTS}" != "1" ]]; then
|
|
psql -c "CREATE DATABASE ${DB_NAME} OWNER ${APP_USER};" postgres
|
|
echo "✓ Created ${DB_NAME}"
|
|
else
|
|
echo "✓ ${DB_NAME} already exists"
|
|
fi
|
|
# Extensions
|
|
psql -d "${DB_NAME}" -c 'CREATE EXTENSION IF NOT EXISTS "uuid-ossp";'
|
|
psql -d "${DB_NAME}" -c 'CREATE EXTENSION IF NOT EXISTS "pgcrypto";'
|
|
psql -d "${DB_NAME}" -c 'CREATE EXTENSION IF NOT EXISTS "vector";'
|
|
# Grant
|
|
psql -d "${DB_NAME}" -c "GRANT ALL PRIVILEGES ON DATABASE ${DB_NAME} TO ${APP_USER};"
|
|
psql -d "${DB_NAME}" -c "GRANT ALL PRIVILEGES ON SCHEMA public TO ${APP_USER};"
|
|
psql -d "${DB_NAME}" -c "ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO ${APP_USER};"
|
|
psql -d "${DB_NAME}" -c "ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO ${APP_USER};"
|
|
echo "✓ CI test database ready"
|
|
|
|
- name: Run Postgres integration tests
|
|
working-directory: server
|
|
env:
|
|
PGHOST: ${{ env.POSTGRES_HOST }}
|
|
PGPORT: ${{ env.POSTGRES_PORT }}
|
|
PGUSER: ${{ secrets.POSTGRES_USER }}
|
|
PGPASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
|
|
run: |
|
|
echo "━━━ Postgres Integration Tests ━━━"
|
|
go test -v -race -count=1 -p 1 -timeout 12m \
|
|
./store/postgres/... \
|
|
./handlers/...
|
|
echo "✓ Postgres integration tests complete"
|
|
|
|
- name: Drop CI test database
|
|
if: always()
|
|
env:
|
|
PGHOST: ${{ env.POSTGRES_HOST }}
|
|
PGPORT: ${{ env.POSTGRES_PORT }}
|
|
PGUSER: ${{ secrets.POSTGRES_ADMIN_USER }}
|
|
PGPASSWORD: ${{ secrets.POSTGRES_ADMIN_PASSWORD }}
|
|
run: |
|
|
psql -c "DROP DATABASE IF EXISTS armature_ci;" postgres
|
|
echo "✓ Dropped CI test database"
|
|
|
|
# ── Stage 1d: Surface Test Runners ──────────
|
|
# Boots the server in Docker, runs all installed test-runner packages
|
|
# via Playwright, and asserts zero failures.
|
|
#
|
|
# Runs when: backend or frontend changed (runners test both tiers).
|
|
# Skipped when: only docs changed.
|
|
test-runners:
|
|
runs-on: ubuntu-latest
|
|
needs: [detect-changes, test-sqlite]
|
|
if: |
|
|
!cancelled() && !failure() &&
|
|
(needs.detect-changes.outputs.backend == 'true' ||
|
|
needs.detect-changes.outputs.frontend == 'true' ||
|
|
needs.detect-changes.outputs.infra == 'true')
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Build Docker image
|
|
run: docker compose build
|
|
|
|
- name: Boot server (SQLite)
|
|
env:
|
|
BUNDLED_PACKAGES: '*'
|
|
ARMATURE_ADMIN_USERNAME: admin
|
|
ARMATURE_ADMIN_PASSWORD: admin
|
|
ARMATURE_ADMIN_EMAIL: admin@test.local
|
|
run: docker compose up -d
|
|
|
|
# In DinD (Gitea runner pods), port-mapped localhost doesn't reach
|
|
# the compose container. Resolve the container IP on its Docker
|
|
# network so curl / Playwright can connect directly on port 80.
|
|
- name: Resolve container IP
|
|
id: container-ip
|
|
run: |
|
|
IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' armature)
|
|
echo "SERVER_URL=http://${IP}:80" >> "$GITHUB_OUTPUT"
|
|
echo "Resolved container IP → ${IP}"
|
|
|
|
- name: Wait for healthy
|
|
run: ./ci/wait-for-healthy.sh ${{ steps.container-ip.outputs.SERVER_URL }}
|
|
|
|
- name: Install Playwright
|
|
run: npx playwright install --with-deps chromium
|
|
|
|
- name: Run surface tests
|
|
env:
|
|
SERVER_URL: ${{ steps.container-ip.outputs.SERVER_URL }}
|
|
ADMIN_USER: admin
|
|
ADMIN_PASS: admin
|
|
run: ./ci/run-surface-tests.sh
|
|
|
|
- name: Teardown
|
|
if: always()
|
|
run: docker compose down -v
|
|
|
|
# ── Stage 2: Build, Database, Deploy ─────────
|
|
#
|
|
# Depends on all test jobs. Skipped jobs (due to path gating)
|
|
# are treated as successful — no blocking.
|
|
#
|
|
# Skipped entirely for docs-only changes (nothing to build).
|
|
build-and-deploy:
|
|
runs-on: ubuntu-latest
|
|
needs: [detect-changes, test-go-pg, test-frontend, test-sqlite, test-runners]
|
|
# Run unless: a needed job failed, the workflow was cancelled, or it's docs-only.
|
|
# Skipped test jobs (path-gated) are fine — they don't block.
|
|
if: |
|
|
!cancelled() && !failure() &&
|
|
needs.detect-changes.outputs.docs_only != 'true'
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Add /tools to PATH
|
|
run: echo "/tools" >> $GITHUB_PATH
|
|
|
|
# - name: Install tools
|
|
# run: |
|
|
# apt-get update -qq && apt-get install -y -qq gettext-base postgresql-client > /dev/null
|
|
|
|
# ── Determine environment ──────────────────
|
|
- name: Determine environment
|
|
id: setup
|
|
run: |
|
|
# All environments share one host: armature.DOMAIN
|
|
# Environments are separated by path prefix (BASE_PATH)
|
|
DEPLOY_HOST="armature.${DOMAIN}"
|
|
echo "DEPLOY_HOST=${DEPLOY_HOST}" >> "$GITHUB_OUTPUT"
|
|
|
|
if [[ "${{ gitea.event_name }}" == "pull_request" ]]; then
|
|
echo "ENVIRONMENT=dev" >> "$GITHUB_OUTPUT"
|
|
echo "GIN_MODE=debug" >> "$GITHUB_OUTPUT"
|
|
echo "IMAGE_TAG=dev" >> "$GITHUB_OUTPUT"
|
|
echo "BASE_PATH=/dev" >> "$GITHUB_OUTPUT"
|
|
echo "DEPLOY_SUFFIX=-dev" >> "$GITHUB_OUTPUT"
|
|
echo "DB_NAME=armature_dev" >> "$GITHUB_OUTPUT"
|
|
echo "DB_WIPE=true" >> "$GITHUB_OUTPUT"
|
|
echo "REPLICAS=1" >> "$GITHUB_OUTPUT"
|
|
echo "MEMORY_REQUEST=256Mi" >> "$GITHUB_OUTPUT"
|
|
echo "MEMORY_LIMIT=512Mi" >> "$GITHUB_OUTPUT"
|
|
echo "CPU_REQUEST=50m" >> "$GITHUB_OUTPUT"
|
|
echo "CPU_LIMIT=250m" >> "$GITHUB_OUTPUT"
|
|
echo "BUNDLED_PACKAGES=*" >> "$GITHUB_OUTPUT"
|
|
echo "env_label=dev (PR #${{ gitea.event.pull_request.number }})" >> "$GITHUB_OUTPUT"
|
|
elif [[ "${{ gitea.ref }}" == refs/tags/v* ]]; then
|
|
VERSION="${{ gitea.ref_name }}"
|
|
echo "ENVIRONMENT=production" >> "$GITHUB_OUTPUT"
|
|
echo "GIN_MODE=release" >> "$GITHUB_OUTPUT"
|
|
echo "IMAGE_TAG=latest" >> "$GITHUB_OUTPUT"
|
|
echo "EXTRA_TAG=${VERSION}" >> "$GITHUB_OUTPUT"
|
|
echo "BASE_PATH=" >> "$GITHUB_OUTPUT"
|
|
echo "DEPLOY_SUFFIX=" >> "$GITHUB_OUTPUT"
|
|
echo "DB_NAME=armature" >> "$GITHUB_OUTPUT"
|
|
echo "DB_WIPE=false" >> "$GITHUB_OUTPUT"
|
|
echo "REPLICAS=2" >> "$GITHUB_OUTPUT"
|
|
echo "MEMORY_REQUEST=256Mi" >> "$GITHUB_OUTPUT"
|
|
echo "MEMORY_LIMIT=512Mi" >> "$GITHUB_OUTPUT"
|
|
echo "CPU_REQUEST=100m" >> "$GITHUB_OUTPUT"
|
|
echo "CPU_LIMIT=500m" >> "$GITHUB_OUTPUT"
|
|
echo "BUNDLED_PACKAGES=notes,chat,chat-core,mermaid-renderer,schedules" >> "$GITHUB_OUTPUT"
|
|
echo "is_release=true" >> "$GITHUB_OUTPUT"
|
|
echo "env_label=production (${VERSION})" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "ENVIRONMENT=test" >> "$GITHUB_OUTPUT"
|
|
echo "GIN_MODE=debug" >> "$GITHUB_OUTPUT"
|
|
echo "IMAGE_TAG=test" >> "$GITHUB_OUTPUT"
|
|
echo "BASE_PATH=/test" >> "$GITHUB_OUTPUT"
|
|
echo "DEPLOY_SUFFIX=-test" >> "$GITHUB_OUTPUT"
|
|
echo "DB_NAME=armature_test" >> "$GITHUB_OUTPUT"
|
|
echo "DB_WIPE=false" >> "$GITHUB_OUTPUT"
|
|
echo "REPLICAS=1" >> "$GITHUB_OUTPUT"
|
|
echo "MEMORY_REQUEST=256Mi" >> "$GITHUB_OUTPUT"
|
|
echo "MEMORY_LIMIT=512Mi" >> "$GITHUB_OUTPUT"
|
|
echo "CPU_REQUEST=50m" >> "$GITHUB_OUTPUT"
|
|
echo "CPU_LIMIT=250m" >> "$GITHUB_OUTPUT"
|
|
echo "BUNDLED_PACKAGES=notes,chat,chat-core" >> "$GITHUB_OUTPUT"
|
|
echo "env_label=test (main)" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
# ── Database Bootstrap (admin creds) ───────
|
|
- name: Bootstrap database
|
|
env:
|
|
PGHOST: ${{ env.POSTGRES_HOST }}
|
|
PGPORT: ${{ env.POSTGRES_PORT }}
|
|
PGUSER: ${{ secrets.POSTGRES_ADMIN_USER }}
|
|
PGPASSWORD: ${{ secrets.POSTGRES_ADMIN_PASSWORD }}
|
|
APP_USER: ${{ secrets.POSTGRES_USER }}
|
|
APP_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
|
|
DB_NAME: ${{ steps.setup.outputs.DB_NAME }}
|
|
run: |
|
|
chmod +x scripts/db-bootstrap.sh
|
|
scripts/db-bootstrap.sh
|
|
|
|
# ── Dev: Upgrade Test (dev only) ───────────
|
|
- name: "Dev: test migration upgrade"
|
|
if: steps.setup.outputs.DB_WIPE == 'true'
|
|
env:
|
|
PGHOST: ${{ env.POSTGRES_HOST }}
|
|
PGPORT: ${{ env.POSTGRES_PORT }}
|
|
PGUSER: ${{ secrets.POSTGRES_USER }}
|
|
PGPASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
|
|
PGDATABASE: ${{ steps.setup.outputs.DB_NAME }}
|
|
run: |
|
|
echo "━━━ Upgrade test: applying migrations to existing dev DB ━━━"
|
|
|
|
psql -v ON_ERROR_STOP=1 <<'SQL'
|
|
CREATE TABLE IF NOT EXISTS schema_migrations (
|
|
version VARCHAR(255) PRIMARY KEY,
|
|
applied_at TIMESTAMPTZ DEFAULT NOW()
|
|
);
|
|
SQL
|
|
|
|
APPLIED=0
|
|
SKIPPED=0
|
|
FAILED=0
|
|
|
|
for migration in server/database/migrations/*.sql; do
|
|
[ -f "${migration}" ] || continue
|
|
VERSION=$(basename "${migration}")
|
|
|
|
ALREADY=$(psql -tAc "SELECT 1 FROM schema_migrations WHERE version = '${VERSION}';" || echo "0")
|
|
if [[ "${ALREADY}" == "1" ]]; then
|
|
echo " skip: ${VERSION}"
|
|
SKIPPED=$((SKIPPED + 1))
|
|
continue
|
|
fi
|
|
|
|
echo " apply: ${VERSION}..."
|
|
if psql -v ON_ERROR_STOP=1 -f "${migration}"; then
|
|
psql -c "INSERT INTO schema_migrations (version) VALUES ('${VERSION}');"
|
|
echo " ✓ ${VERSION}"
|
|
APPLIED=$((APPLIED + 1))
|
|
else
|
|
echo " ✗ ${VERSION} FAILED"
|
|
FAILED=$((FAILED + 1))
|
|
fi
|
|
done
|
|
|
|
echo ""
|
|
echo "━━━ Upgrade test: ${APPLIED} applied, ${SKIPPED} skipped, ${FAILED} failed ━━━"
|
|
if [[ ${FAILED} -gt 0 ]]; then
|
|
echo "❌ Migration upgrade test failed — fix before merge"
|
|
exit 1
|
|
fi
|
|
|
|
# ── Dev: Validate Schema ───────────────────
|
|
# - name: "Dev: validate schema"
|
|
# if: steps.setup.outputs.DB_WIPE == 'true'
|
|
# env:
|
|
# PGHOST: ${{ env.POSTGRES_HOST }}
|
|
# PGPORT: ${{ env.POSTGRES_PORT }}
|
|
# PGUSER: ${{ secrets.POSTGRES_USER }}
|
|
# PGPASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
|
|
# PGDATABASE: ${{ steps.setup.outputs.DB_NAME }}
|
|
# run: |
|
|
# chmod +x scripts/db-validate.sh
|
|
# scripts/db-validate.sh
|
|
|
|
# ── Dev Wipe (fresh install test) ──────────
|
|
- name: "Dev: wipe for fresh install test"
|
|
if: steps.setup.outputs.DB_WIPE == 'true'
|
|
env:
|
|
PGHOST: ${{ env.POSTGRES_HOST }}
|
|
PGPORT: ${{ env.POSTGRES_PORT }}
|
|
PGUSER: ${{ secrets.POSTGRES_USER }}
|
|
PGPASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
|
|
PGDATABASE: ${{ steps.setup.outputs.DB_NAME }}
|
|
run: |
|
|
DB="${{ steps.setup.outputs.DB_NAME }}"
|
|
|
|
if [[ "${DB}" != *_dev ]]; then
|
|
echo "❌ REFUSING to wipe '${DB}' — only *_dev databases can be wiped"
|
|
exit 1
|
|
fi
|
|
|
|
echo "⚠ Dev wipe: dropping all tables in ${DB}"
|
|
psql <<'SQL'
|
|
DO $$ DECLARE
|
|
r RECORD;
|
|
BEGIN
|
|
FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname = 'public') LOOP
|
|
EXECUTE 'DROP TABLE IF EXISTS public.' || quote_ident(r.tablename) || ' CASCADE';
|
|
END LOOP;
|
|
END $$;
|
|
SQL
|
|
echo "✓ Dev wipe complete — backend will rebuild schema on startup"
|
|
|
|
# ── Build Unified Image ──────────────────────
|
|
- name: Build image
|
|
run: |
|
|
docker build -f Dockerfile \
|
|
-t ${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }} \
|
|
.
|
|
|
|
# ── Tag release versions ─────────────────────
|
|
- name: Tag release versions
|
|
if: steps.setup.outputs.EXTRA_TAG != ''
|
|
run: |
|
|
docker tag ${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }} ${IMAGE}:${{ steps.setup.outputs.EXTRA_TAG }}
|
|
|
|
# ── Push to Internal Registry ────────────────
|
|
- name: Push image
|
|
run: |
|
|
docker push ${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }}
|
|
if [[ -n "${{ steps.setup.outputs.EXTRA_TAG }}" ]]; then
|
|
docker push ${IMAGE}:${{ steps.setup.outputs.EXTRA_TAG }}
|
|
fi
|
|
|
|
# ── Push to Docker Hub (release only) ────────
|
|
- name: Push to Docker Hub
|
|
if: steps.setup.outputs.is_release == 'true'
|
|
env:
|
|
DH_USER: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
DH_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
run: |
|
|
docker tag ${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }} ${DOCKERHUB_IMAGE}:latest
|
|
docker tag ${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }} ${DOCKERHUB_IMAGE}:${{ steps.setup.outputs.EXTRA_TAG }}
|
|
if [[ -n "${DH_TOKEN}" ]]; then
|
|
echo "${DH_TOKEN}" | docker login -u "${DH_USER}" --password-stdin
|
|
docker push ${DOCKERHUB_IMAGE}:latest
|
|
docker push ${DOCKERHUB_IMAGE}:${{ steps.setup.outputs.EXTRA_TAG }}
|
|
echo "✓ Pushed to Docker Hub"
|
|
else
|
|
echo "⚠ Docker Hub credentials not configured, skipping push"
|
|
fi
|
|
|
|
# ── Deploy to Kubernetes ─────────────────────
|
|
- name: Ensure namespace
|
|
run: |
|
|
kubectl create namespace ${NAMESPACE} 2>/dev/null || true
|
|
|
|
- name: Sync secrets
|
|
env:
|
|
PG_USER: ${{ secrets.POSTGRES_USER }}
|
|
PG_PASS: ${{ secrets.POSTGRES_PASSWORD }}
|
|
SW_ADMIN_USER: ${{ secrets.ARMATURE_ADMIN_USERNAME }}
|
|
SW_ADMIN_PASS: ${{ secrets.ARMATURE_ADMIN_PASSWORD }}
|
|
SW_ADMIN_EMAIL: ${{ secrets.ARMATURE_ADMIN_EMAIL }}
|
|
SEED_USERS: ${{ vars.SEED_USERS }}
|
|
ENCRYPTION_KEY: ${{ secrets.ENCRYPTION_KEY }}
|
|
run: |
|
|
kubectl create secret generic armature-db-credentials \
|
|
--namespace=${NAMESPACE} \
|
|
--from-literal=POSTGRES_USER="${PG_USER}" \
|
|
--from-literal=POSTGRES_PASSWORD="${PG_PASS}" \
|
|
--dry-run=client -o yaml | kubectl apply -f -
|
|
|
|
kubectl create secret generic armature-admin \
|
|
--namespace=${NAMESPACE} \
|
|
--from-literal=username="${SW_ADMIN_USER}" \
|
|
--from-literal=password="${SW_ADMIN_PASS}" \
|
|
--from-literal=email="${SW_ADMIN_EMAIL}" \
|
|
--dry-run=client -o yaml | kubectl apply -f -
|
|
|
|
kubectl create secret generic armature-seed-users \
|
|
--namespace=${NAMESPACE} \
|
|
--from-literal=users="${SEED_USERS}" \
|
|
--dry-run=client -o yaml | kubectl apply -f -
|
|
|
|
kubectl create secret generic armature-encryption \
|
|
--namespace=${NAMESPACE} \
|
|
--from-literal=ENCRYPTION_KEY="${ENCRYPTION_KEY}" \
|
|
--dry-run=client -o yaml | kubectl apply -f -
|
|
|
|
- name: Sync S3 secrets
|
|
if: vars.STORAGE_BACKEND == 's3'
|
|
env:
|
|
S3_ENDPOINT: ${{ vars.S3_ENDPOINT }}
|
|
S3_BUCKET: ${{ vars.S3_BUCKET }}
|
|
S3_ACCESS_KEY: ${{ secrets.S3_ACCESS_KEY }}
|
|
S3_SECRET_KEY: ${{ secrets.S3_SECRET_KEY }}
|
|
S3_REGION: ${{ vars.S3_REGION || 'us-east-1' }}
|
|
S3_PREFIX: ${{ vars.S3_PREFIX }}
|
|
run: |
|
|
kubectl create secret generic armature-s3 \
|
|
--namespace=${NAMESPACE} \
|
|
--from-literal=S3_ENDPOINT="${S3_ENDPOINT}" \
|
|
--from-literal=S3_BUCKET="${S3_BUCKET}" \
|
|
--from-literal=S3_ACCESS_KEY="${S3_ACCESS_KEY}" \
|
|
--from-literal=S3_SECRET_KEY="${S3_SECRET_KEY}" \
|
|
--from-literal=S3_REGION="${S3_REGION}" \
|
|
--from-literal=S3_PREFIX="${S3_PREFIX}" \
|
|
--dry-run=client -o yaml | kubectl apply -f -
|
|
|
|
- name: Render and apply manifests
|
|
env:
|
|
ENVIRONMENT: ${{ steps.setup.outputs.ENVIRONMENT }}
|
|
GIN_MODE: ${{ steps.setup.outputs.GIN_MODE }}
|
|
IMAGE_TAG: ${{ steps.setup.outputs.IMAGE_TAG }}
|
|
DEPLOY_SUFFIX: ${{ steps.setup.outputs.DEPLOY_SUFFIX }}
|
|
DEPLOY_HOST: ${{ steps.setup.outputs.DEPLOY_HOST }}
|
|
BASE_PATH: ${{ steps.setup.outputs.BASE_PATH }}
|
|
DB_NAME: ${{ steps.setup.outputs.DB_NAME }}
|
|
REPLICAS: ${{ steps.setup.outputs.REPLICAS }}
|
|
MEMORY_REQUEST: ${{ steps.setup.outputs.MEMORY_REQUEST }}
|
|
MEMORY_LIMIT: ${{ steps.setup.outputs.MEMORY_LIMIT }}
|
|
CPU_REQUEST: ${{ steps.setup.outputs.CPU_REQUEST }}
|
|
CPU_LIMIT: ${{ steps.setup.outputs.CPU_LIMIT }}
|
|
# Storage (v0.12.0+)
|
|
STORAGE_CLASS: ${{ vars.STORAGE_CLASS }}
|
|
STORAGE_SIZE: ${{ vars.STORAGE_SIZE || '10Gi' }}
|
|
STORAGE_BACKEND: ${{ vars.STORAGE_BACKEND || 'pvc' }}
|
|
BUNDLED_PACKAGES: ${{ steps.setup.outputs.BUNDLED_PACKAGES }}
|
|
run: |
|
|
# Render PVC first (must exist before backend references it)
|
|
if [[ -n "${STORAGE_CLASS}" ]]; then
|
|
envsubst < k8s/storage-pvc.yaml > /tmp/storage-pvc.yaml
|
|
fi
|
|
|
|
envsubst < k8s/armature.yaml > /tmp/armature.yaml
|
|
envsubst < k8s/middleware-retry.yaml > /tmp/middleware-retry.yaml
|
|
envsubst < k8s/ingress.yaml > /tmp/ingress.yaml
|
|
|
|
echo "━━━ Applying manifests for ${DEPLOY_HOST} ━━━"
|
|
|
|
# Apply PVC (idempotent — won't recreate if exists)
|
|
if [[ -n "${STORAGE_CLASS}" ]]; then
|
|
kubectl apply -f /tmp/storage-pvc.yaml
|
|
echo " ✓ Storage PVC (class: ${STORAGE_CLASS}, size: ${STORAGE_SIZE})"
|
|
else
|
|
echo " ⚠ STORAGE_CLASS not set — file storage disabled"
|
|
fi
|
|
|
|
kubectl apply -f /tmp/armature.yaml
|
|
|
|
# Traefik retry middleware (v0.28.8) — requires RBAC for traefik.io CRDs.
|
|
# First-time setup: kubectl apply -f k8s/rbac-traefik.yaml (cluster admin)
|
|
if kubectl apply -f /tmp/middleware-retry.yaml 2>/tmp/mw-err.txt; then
|
|
echo " ✓ Traefik retry middleware"
|
|
MW_READY=true
|
|
else
|
|
echo " ⚠ Traefik middleware skipped ($(head -1 /tmp/mw-err.txt))"
|
|
echo " First-time setup: kubectl apply -f k8s/rbac-traefik.yaml"
|
|
MW_READY=false
|
|
fi
|
|
|
|
kubectl apply -f /tmp/ingress.yaml
|
|
|
|
# Annotate ingress with middleware reference ONLY if middleware exists.
|
|
# Traefik invalidates the entire router if it references a missing middleware,
|
|
# which kills all routes for this path prefix.
|
|
if [[ "${MW_READY}" == "true" ]]; then
|
|
MW_REF="${NAMESPACE}-armature-retry${DEPLOY_SUFFIX}@kubernetescrd"
|
|
kubectl annotate ingress "armature${DEPLOY_SUFFIX}" \
|
|
"traefik.ingress.kubernetes.io/router.middlewares=${MW_REF}" \
|
|
--namespace="${NAMESPACE}" --overwrite
|
|
echo " ✓ Ingress annotated with retry middleware"
|
|
fi
|
|
|
|
- name: Rollout and verify
|
|
env:
|
|
SUFFIX: ${{ steps.setup.outputs.DEPLOY_SUFFIX }}
|
|
ENV: ${{ steps.setup.outputs.ENVIRONMENT }}
|
|
run: |
|
|
kubectl rollout restart deployment/armature${SUFFIX} -n ${NAMESPACE}
|
|
kubectl rollout status deployment/armature${SUFFIX} -n ${NAMESPACE} --timeout=180s
|
|
|
|
echo ""
|
|
echo "━━━ Pod Status ━━━"
|
|
kubectl get pods -n ${NAMESPACE} -l app=armature,env=${ENV}
|
|
|
|
READY=$(kubectl get deployment armature${SUFFIX} -n ${NAMESPACE} -o jsonpath='{.status.readyReplicas}')
|
|
|
|
if [[ "${READY:-0}" -gt 0 ]]; then
|
|
echo "✅ Healthy: ${READY} pods ready"
|
|
else
|
|
echo "❌ Unhealthy: ${READY:-0} pods ready"
|
|
kubectl logs -n ${NAMESPACE} -l app=armature,env=${ENV} --tail=30
|
|
exit 1
|
|
fi
|
|
|
|
# ── Post-deploy: verify schema ─────────────
|
|
- name: Verify schema migration
|
|
env:
|
|
EXPECTED_DB: ${{ steps.setup.outputs.DB_NAME }}
|
|
run: |
|
|
sleep 5
|
|
HOST="${{ steps.setup.outputs.DEPLOY_HOST }}"
|
|
BP="${{ steps.setup.outputs.BASE_PATH }}"
|
|
HEALTH=$(curl -sf "https://${HOST}${BP}/health" 2>/dev/null || echo "unreachable")
|
|
echo "Health: ${HEALTH}"
|
|
|
|
SCHEMA=$(echo "${HEALTH}" | grep -o '"schema_version":"[^"]*"' | cut -d'"' -f4 || true)
|
|
DB_OK=$(echo "${HEALTH}" | grep -o '"database":true' || true)
|
|
ACTUAL_DB=$(echo "${HEALTH}" | grep -o '"database_name":"[^"]*"' | cut -d'"' -f4 || true)
|
|
|
|
if [[ -n "${SCHEMA}" ]] && [[ -n "${DB_OK}" ]]; then
|
|
echo "✅ Schema: ${SCHEMA} | DB: ${ACTUAL_DB}"
|
|
else
|
|
echo "⚠ Could not verify schema (backend may still be starting)"
|
|
echo " Response: ${HEALTH:0:200}"
|
|
fi
|
|
|
|
# Verify the backend connected to the correct database
|
|
if [[ -n "${ACTUAL_DB}" ]] && [[ "${ACTUAL_DB}" != "${EXPECTED_DB}" ]]; then
|
|
echo "❌ DATABASE MISMATCH: expected=${EXPECTED_DB} actual=${ACTUAL_DB}"
|
|
echo " The backend is connected to the WRONG database!"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Summary
|
|
run: |
|
|
HOST="${{ steps.setup.outputs.DEPLOY_HOST }}"
|
|
BP="${{ steps.setup.outputs.BASE_PATH }}"
|
|
echo "## ✅ Deployed to ${{ steps.setup.outputs.env_label }}" >> $GITHUB_STEP_SUMMARY
|
|
echo "" >> $GITHUB_STEP_SUMMARY
|
|
echo "| | |" >> $GITHUB_STEP_SUMMARY
|
|
echo "|---|---|" >> $GITHUB_STEP_SUMMARY
|
|
echo "| **URL** | https://${HOST}${BP}/ |" >> $GITHUB_STEP_SUMMARY
|
|
echo "| **Image** | \`${IMAGE}:${{ steps.setup.outputs.IMAGE_TAG }}\` |" >> $GITHUB_STEP_SUMMARY
|
|
echo "| **Database** | \`${{ steps.setup.outputs.DB_NAME }}\` |" >> $GITHUB_STEP_SUMMARY
|
|
echo "| **DB Wipe** | ${{ steps.setup.outputs.DB_WIPE }} |" >> $GITHUB_STEP_SUMMARY
|
|
if [[ -n "${{ vars.STORAGE_CLASS }}" ]]; then
|
|
if [[ "${{ vars.STORAGE_BACKEND }}" == "s3" ]]; then
|
|
echo "| **Storage** | S3 (bucket: ${{ vars.S3_BUCKET }}, PVC scratch: ${{ vars.STORAGE_CLASS }}) |" >> $GITHUB_STEP_SUMMARY
|
|
else
|
|
echo "| **Storage** | PVC (class: ${{ vars.STORAGE_CLASS }}, size: ${{ vars.STORAGE_SIZE || '10Gi' }}) |" >> $GITHUB_STEP_SUMMARY
|
|
fi
|
|
else
|
|
echo "| **Storage** | Disabled (STORAGE_CLASS not set) |" >> $GITHUB_STEP_SUMMARY
|
|
fi
|
|
if [[ "${{ steps.setup.outputs.is_release }}" == "true" ]]; then
|
|
echo "| **Docker Hub** | \`${DOCKERHUB_IMAGE}:${{ steps.setup.outputs.EXTRA_TAG }}\` (unified) |" >> $GITHUB_STEP_SUMMARY
|
|
fi |