Adds a workflow_dispatch-triggered job on the self-hosted axiom runner
that captures fresh `tests/visual/__screenshots__/*.png` against a
caller-provided URL and opens a `chore(visual): refresh baselines from
<url>` PR via peter-evans/create-pull-request@v6.
This is brief 1 of 2 of the harden-visual-diff-gate convoy. The
workflow exists but is not invoked by this PR — operator dispatches via
the GitHub UI or `gh workflow run seed-visual-baselines.yml` once they
want a fresh baseline against post-glass-redesign main.
Brief 2 (flip continue-on-error: true off visual-diff.yml, add 9th
forbidden-patterns check) is unblocked once a fresh baseline lands via
this workflow's auto-PR.
Convoy decision D1 ratified: Option B (workflow_dispatch + auto-PR)
chosen over Option A (ad-hoc SSH-into-CT-111 + manual commit). The
workflow gives the baseline regeneration a reviewable Git-native shape;
the auto-PR carries a checklist for visual sanity-check before merge.
Workflow shape:
- Inputs: base_url (required, no default to avoid wrong-target
accidents), reason (optional, used in PR body).
- Permissions: contents: write + pull-requests: write — sufficient for
default GITHUB_TOKEN; no PAT needed.
- Caches: shares the node_modules + Playwright browser caches with
visual-diff.yml so the byte-equivalence guarantee holds without
cache miss overhead.
- Idempotent: peter-evans/create-pull-request short-circuits to a
::notice:: annotation if the captures match the existing committed
baselines (no PR opened).
Metrics: logged role-conductor + role-architect + role-implementer
events for this convoy in .convoys/.metrics.jsonl. Satisfies the
convoy-metrics-gate (PR #134, 9eef8d9) that fires on convoy:-titled
PRs requiring at least one new metrics row.
Co-authored-by: Cursor <cursoragent@cursor.com>
162 lines
6.6 KiB
YAML
162 lines
6.6 KiB
YAML
name: Seed visual baselines
|
|
|
|
# Brief 1 of the `harden-visual-diff-gate` convoy.
|
|
#
|
|
# Repeatable Linux baseline regeneration for `tests/visual/__screenshots__/`.
|
|
# Runs on the self-hosted axiom runner (CT 111) so the resulting PNGs are
|
|
# byte-equivalent to what `visual-diff.yml` will see during PR comparisons —
|
|
# this is the same toolchain (`myoung34/github-runner` image + Chromium
|
|
# version pinned by `package-lock.json`).
|
|
#
|
|
# Dispatch:
|
|
# - GitHub UI → Actions → Seed visual baselines → Run workflow
|
|
# - Or via gh CLI:
|
|
# gh workflow run seed-visual-baselines.yml \
|
|
# -f base_url=https://<preview-or-prod>.vercel.app \
|
|
# -f reason="Glass redesign briefs landed — refresh"
|
|
#
|
|
# Output: if baselines changed, a `chore(visual): ...` PR is opened with
|
|
# the updated `tests/visual/__screenshots__/*.png` for human review. If
|
|
# the new captures match the existing committed baselines, no PR is
|
|
# opened (workflow exits with a `::notice::` annotation).
|
|
#
|
|
# Why not a Mac dev workflow? `playwright.config.js`'s
|
|
# `snapshotPathTemplate` has no `{platform}` token, so a Mac-generated
|
|
# PNG silently overwrites the canonical Linux baseline and the next CI
|
|
# diff will fail against it. See AGENTS.md § Visual baselines.
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
inputs:
|
|
base_url:
|
|
description: 'URL to capture baselines against (production or a long-lived preview). Required — no default to avoid wrong-target accidents.'
|
|
required: true
|
|
reason:
|
|
description: 'Short reason for the refresh (appears in the resulting PR body).'
|
|
required: false
|
|
default: 'Periodic baseline refresh'
|
|
|
|
permissions:
|
|
contents: write
|
|
pull-requests: write
|
|
|
|
jobs:
|
|
seed:
|
|
name: Capture + commit new baselines
|
|
runs-on: [self-hosted, axiom]
|
|
timeout-minutes: 20
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
cache: npm
|
|
|
|
# Same caching shape as ci.yml / visual-diff.yml so the baseline-seed
|
|
# run reuses the existing CT 111 cache mounts.
|
|
- name: Cache node_modules
|
|
id: cache-node-modules
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: node_modules
|
|
key: node-modules-${{ runner.os }}-node20-${{ hashFiles('package-lock.json') }}
|
|
|
|
- run: npm ci
|
|
if: steps.cache-node-modules.outputs.cache-hit != 'true'
|
|
|
|
- name: Resolve Playwright version
|
|
id: pw-version
|
|
run: |
|
|
VERSION=$(node -p "require('./package-lock.json').packages['node_modules/@playwright/test'].version")
|
|
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Cache Playwright browsers
|
|
id: cache-playwright
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: ~/.cache/ms-playwright
|
|
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}-chromium
|
|
|
|
- name: Install Playwright browsers
|
|
run: |
|
|
if [ "${{ steps.cache-playwright.outputs.cache-hit }}" = "true" ]; then
|
|
npx playwright install-deps chromium
|
|
else
|
|
npx playwright install --with-deps chromium
|
|
fi
|
|
|
|
# `--update-snapshots` writes new PNGs to `tests/visual/__screenshots__/`
|
|
# but does NOT fail the run when a baseline mismatches — that's the
|
|
# point. The diff-gate hardening (Brief 2) makes the matching
|
|
# `visual-diff.yml` invocation fail-on-mismatch.
|
|
- name: Capture + write baselines
|
|
env:
|
|
BASE_URL: ${{ inputs.base_url }}
|
|
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
|
|
run: npx playwright test --project=visual --update-snapshots
|
|
|
|
# `peter-evans/create-pull-request@v6` handles branch creation,
|
|
# commit, push, and PR open/update in a single idempotent step. If
|
|
# there are no changes to `tests/visual/__screenshots__/`, it
|
|
# short-circuits and leaves no branch/PR behind. Uses the default
|
|
# GITHUB_TOKEN (no PAT needed) — `permissions:` block above grants
|
|
# `contents: write` + `pull-requests: write` which are sufficient.
|
|
- name: Open baseline-refresh PR
|
|
id: cpr
|
|
uses: peter-evans/create-pull-request@v6
|
|
with:
|
|
token: ${{ secrets.GITHUB_TOKEN }}
|
|
branch: bot/visual-baselines-${{ github.run_id }}
|
|
delete-branch: true
|
|
title: "chore(visual): refresh baselines from ${{ inputs.base_url }}"
|
|
commit-message: |
|
|
chore(visual): refresh baselines from ${{ inputs.base_url }}
|
|
|
|
Reason: ${{ inputs.reason }}
|
|
|
|
Triggered by workflow_dispatch run ${{ github.run_id }} on
|
|
the axiom self-hosted runner (CT 111). Byte-equivalent
|
|
toolchain to visual-diff.yml — PNGs should match cleanly on
|
|
the follow-up Screenshot diff job.
|
|
body: |
|
|
Automated baseline refresh from the
|
|
[Seed visual baselines](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
|
|
workflow run.
|
|
|
|
- **Source URL:** `${{ inputs.base_url }}`
|
|
- **Reason:** ${{ inputs.reason }}
|
|
- **Runner:** CT 111 self-hosted axiom runner (byte-equivalent toolchain to `visual-diff.yml`).
|
|
|
|
## Review checklist
|
|
|
|
- [ ] Open the run's artifacts (or pull the PR locally and
|
|
open `tests/visual/__screenshots__/home.png`) and
|
|
visually confirm the new baseline matches an
|
|
intentional UI state.
|
|
- [ ] If matching intent: merge.
|
|
- [ ] If wrong (transient overlay, wrong URL, debug bar
|
|
visible): close the PR; investigate; re-dispatch with
|
|
a corrected URL or after fixing the source.
|
|
|
|
## Convoy context
|
|
|
|
Brief 1 of `harden-visual-diff-gate` (this workflow exists)
|
|
is now usable. Brief 2 (flip `continue-on-error: true` →
|
|
removed in `.github/workflows/visual-diff.yml`) is unblocked
|
|
once this PR merges with a fresh baseline.
|
|
labels: |
|
|
automated
|
|
visual-baselines
|
|
add-paths: |
|
|
tests/visual/__screenshots__/
|
|
|
|
- name: Report no changes
|
|
if: steps.cpr.outputs.pull-request-number == ''
|
|
run: |
|
|
echo "::notice::No baseline changes detected. The captures from ${{ inputs.base_url }} match the committed baselines — no PR opened."
|
|
|
|
- name: Report opened PR
|
|
if: steps.cpr.outputs.pull-request-number != ''
|
|
run: |
|
|
echo "::notice::Opened PR #${{ steps.cpr.outputs.pull-request-number }} with refreshed baselines: ${{ steps.cpr.outputs.pull-request-url }}"
|