deckhearth/.github/workflows/seed-visual-baselines.yml
Randall Stillwell 014c01cb5e convoy: add seed-visual-baselines workflow (harden-visual-diff-gate brief 1)
Adds a workflow_dispatch-triggered job on the self-hosted axiom runner
that captures fresh `tests/visual/__screenshots__/*.png` against a
caller-provided URL and opens a `chore(visual): refresh baselines from
<url>` PR via peter-evans/create-pull-request@v6.

This is brief 1 of 2 of the harden-visual-diff-gate convoy. The
workflow exists but is not invoked by this PR — operator dispatches via
the GitHub UI or `gh workflow run seed-visual-baselines.yml` once they
want a fresh baseline against post-glass-redesign main.

Brief 2 (flip continue-on-error: true off visual-diff.yml, add 9th
forbidden-patterns check) is unblocked once a fresh baseline lands via
this workflow's auto-PR.

Convoy decision D1 ratified: Option B (workflow_dispatch + auto-PR)
chosen over Option A (ad-hoc SSH-into-CT-111 + manual commit). The
workflow gives the baseline regeneration a reviewable Git-native shape;
the auto-PR carries a checklist for visual sanity-check before merge.

Workflow shape:
- Inputs: base_url (required, no default to avoid wrong-target
  accidents), reason (optional, used in PR body).
- Permissions: contents: write + pull-requests: write — sufficient for
  default GITHUB_TOKEN; no PAT needed.
- Caches: shares the node_modules + Playwright browser caches with
  visual-diff.yml so the byte-equivalence guarantee holds without
  cache miss overhead.
- Idempotent: peter-evans/create-pull-request short-circuits to a
  ::notice:: annotation if the captures match the existing committed
  baselines (no PR opened).

Metrics: logged role-conductor + role-architect + role-implementer
events for this convoy in .convoys/.metrics.jsonl. Satisfies the
convoy-metrics-gate (PR #134, 9eef8d9) that fires on convoy:-titled
PRs requiring at least one new metrics row.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 21:29:43 -05:00

162 lines
6.6 KiB
YAML

name: Seed visual baselines
# Brief 1 of the `harden-visual-diff-gate` convoy.
#
# Repeatable Linux baseline regeneration for `tests/visual/__screenshots__/`.
# Runs on the self-hosted axiom runner (CT 111) so the resulting PNGs are
# byte-equivalent to what `visual-diff.yml` will see during PR comparisons —
# this is the same toolchain (`myoung34/github-runner` image + Chromium
# version pinned by `package-lock.json`).
#
# Dispatch:
# - GitHub UI → Actions → Seed visual baselines → Run workflow
# - Or via gh CLI:
# gh workflow run seed-visual-baselines.yml \
# -f base_url=https://<preview-or-prod>.vercel.app \
# -f reason="Glass redesign briefs landed — refresh"
#
# Output: if baselines changed, a `chore(visual): ...` PR is opened with
# the updated `tests/visual/__screenshots__/*.png` for human review. If
# the new captures match the existing committed baselines, no PR is
# opened (workflow exits with a `::notice::` annotation).
#
# Why not a Mac dev workflow? `playwright.config.js`'s
# `snapshotPathTemplate` has no `{platform}` token, so a Mac-generated
# PNG silently overwrites the canonical Linux baseline and the next CI
# diff will fail against it. See AGENTS.md § Visual baselines.
on:
workflow_dispatch:
inputs:
base_url:
description: 'URL to capture baselines against (production or a long-lived preview). Required — no default to avoid wrong-target accidents.'
required: true
reason:
description: 'Short reason for the refresh (appears in the resulting PR body).'
required: false
default: 'Periodic baseline refresh'
permissions:
contents: write
pull-requests: write
jobs:
seed:
name: Capture + commit new baselines
runs-on: [self-hosted, axiom]
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm
# Same caching shape as ci.yml / visual-diff.yml so the baseline-seed
# run reuses the existing CT 111 cache mounts.
- name: Cache node_modules
id: cache-node-modules
uses: actions/cache@v4
with:
path: node_modules
key: node-modules-${{ runner.os }}-node20-${{ hashFiles('package-lock.json') }}
- run: npm ci
if: steps.cache-node-modules.outputs.cache-hit != 'true'
- name: Resolve Playwright version
id: pw-version
run: |
VERSION=$(node -p "require('./package-lock.json').packages['node_modules/@playwright/test'].version")
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
- name: Cache Playwright browsers
id: cache-playwright
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}-chromium
- name: Install Playwright browsers
run: |
if [ "${{ steps.cache-playwright.outputs.cache-hit }}" = "true" ]; then
npx playwright install-deps chromium
else
npx playwright install --with-deps chromium
fi
# `--update-snapshots` writes new PNGs to `tests/visual/__screenshots__/`
# but does NOT fail the run when a baseline mismatches — that's the
# point. The diff-gate hardening (Brief 2) makes the matching
# `visual-diff.yml` invocation fail-on-mismatch.
- name: Capture + write baselines
env:
BASE_URL: ${{ inputs.base_url }}
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
run: npx playwright test --project=visual --update-snapshots
# `peter-evans/create-pull-request@v6` handles branch creation,
# commit, push, and PR open/update in a single idempotent step. If
# there are no changes to `tests/visual/__screenshots__/`, it
# short-circuits and leaves no branch/PR behind. Uses the default
# GITHUB_TOKEN (no PAT needed) — `permissions:` block above grants
# `contents: write` + `pull-requests: write` which are sufficient.
- name: Open baseline-refresh PR
id: cpr
uses: peter-evans/create-pull-request@v6
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: bot/visual-baselines-${{ github.run_id }}
delete-branch: true
title: "chore(visual): refresh baselines from ${{ inputs.base_url }}"
commit-message: |
chore(visual): refresh baselines from ${{ inputs.base_url }}
Reason: ${{ inputs.reason }}
Triggered by workflow_dispatch run ${{ github.run_id }} on
the axiom self-hosted runner (CT 111). Byte-equivalent
toolchain to visual-diff.yml — PNGs should match cleanly on
the follow-up Screenshot diff job.
body: |
Automated baseline refresh from the
[Seed visual baselines](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
workflow run.
- **Source URL:** `${{ inputs.base_url }}`
- **Reason:** ${{ inputs.reason }}
- **Runner:** CT 111 self-hosted axiom runner (byte-equivalent toolchain to `visual-diff.yml`).
## Review checklist
- [ ] Open the run's artifacts (or pull the PR locally and
open `tests/visual/__screenshots__/home.png`) and
visually confirm the new baseline matches an
intentional UI state.
- [ ] If matching intent: merge.
- [ ] If wrong (transient overlay, wrong URL, debug bar
visible): close the PR; investigate; re-dispatch with
a corrected URL or after fixing the source.
## Convoy context
Brief 1 of `harden-visual-diff-gate` (this workflow exists)
is now usable. Brief 2 (flip `continue-on-error: true` →
removed in `.github/workflows/visual-diff.yml`) is unblocked
once this PR merges with a fresh baseline.
labels: |
automated
visual-baselines
add-paths: |
tests/visual/__screenshots__/
- name: Report no changes
if: steps.cpr.outputs.pull-request-number == ''
run: |
echo "::notice::No baseline changes detected. The captures from ${{ inputs.base_url }} match the committed baselines — no PR opened."
- name: Report opened PR
if: steps.cpr.outputs.pull-request-number != ''
run: |
echo "::notice::Opened PR #${{ steps.cpr.outputs.pull-request-number }} with refreshed baselines: ${{ steps.cpr.outputs.pull-request-url }}"