name: Seed visual baselines # Brief 1 of the `harden-visual-diff-gate` convoy. # # Repeatable Linux baseline regeneration for `tests/visual/__screenshots__/`. # Runs on the self-hosted axiom runner (CT 111) so the resulting PNGs are # byte-equivalent to what `visual-diff.yml` will see during PR comparisons — # this is the same toolchain (`myoung34/github-runner` image + Chromium # version pinned by `package-lock.json`). # # Dispatch: # - GitHub UI → Actions → Seed visual baselines → Run workflow # - Or via gh CLI: # gh workflow run seed-visual-baselines.yml \ # -f base_url=https://.vercel.app \ # -f reason="Glass redesign briefs landed — refresh" # # Output: if baselines changed, a `chore(visual): ...` PR is opened with # the updated `tests/visual/__screenshots__/*.png` for human review. If # the new captures match the existing committed baselines, no PR is # opened (workflow exits with a `::notice::` annotation). # # Why not a Mac dev workflow? `playwright.config.js`'s # `snapshotPathTemplate` has no `{platform}` token, so a Mac-generated # PNG silently overwrites the canonical Linux baseline and the next CI # diff will fail against it. See AGENTS.md § Visual baselines. on: workflow_dispatch: inputs: base_url: description: 'URL to capture baselines against (production or a long-lived preview). Required — no default to avoid wrong-target accidents.' required: true reason: description: 'Short reason for the refresh (appears in the resulting PR body).' required: false default: 'Periodic baseline refresh' permissions: contents: write pull-requests: write jobs: seed: name: Capture + commit new baselines runs-on: [self-hosted, axiom] timeout-minutes: 20 steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' cache: npm # Same caching shape as ci.yml / visual-diff.yml so the baseline-seed # run reuses the existing CT 111 cache mounts. - name: Cache node_modules id: cache-node-modules uses: actions/cache@v4 with: path: node_modules key: node-modules-${{ runner.os }}-node20-${{ hashFiles('package-lock.json') }} - run: npm ci if: steps.cache-node-modules.outputs.cache-hit != 'true' - name: Resolve Playwright version id: pw-version run: | VERSION=$(node -p "require('./package-lock.json').packages['node_modules/@playwright/test'].version") echo "version=$VERSION" >> "$GITHUB_OUTPUT" - name: Cache Playwright browsers id: cache-playwright uses: actions/cache@v4 with: path: ~/.cache/ms-playwright key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}-chromium - name: Install Playwright browsers run: | if [ "${{ steps.cache-playwright.outputs.cache-hit }}" = "true" ]; then npx playwright install-deps chromium else npx playwright install --with-deps chromium fi # `--update-snapshots` writes new PNGs to `tests/visual/__screenshots__/` # but does NOT fail the run when a baseline mismatches — that's the # point. The diff-gate hardening (Brief 2) makes the matching # `visual-diff.yml` invocation fail-on-mismatch. - name: Capture + write baselines env: BASE_URL: ${{ inputs.base_url }} VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }} run: npx playwright test --project=visual --update-snapshots # `peter-evans/create-pull-request@v6` handles branch creation, # commit, push, and PR open/update in a single idempotent step. If # there are no changes to `tests/visual/__screenshots__/`, it # short-circuits and leaves no branch/PR behind. # # KNOWN LIMITATION (encountered in workflow run 27454132468, first # invocation 2026-06-13): the PR-open sub-step fails with # "GitHub Actions is not permitted to create or approve pull requests" # because stwl-labs has the org-level "Allow GitHub Actions to # create and approve pull requests" setting disabled (Settings → # Actions → General → Workflow permissions). The branch IS pushed # successfully even when this fails — so the operator can open the # PR manually: # # gh pr create --base main \ # --head bot/visual-baselines- \ # --title "chore(visual): refresh baselines from " \ # --body "..." # # `continue-on-error: true` on this step is intentional and scoped: # it lets the workflow as a whole succeed when the branch push # works but the PR-open step is blocked by the org setting. The # follow-up step below tells the operator exactly what to run. # This is NOT the same `continue-on-error` as the one we just # removed from `visual-diff.yml`'s screenshot capture (that one # silently hid real UI regressions; this one fronts a known org # limitation with a loud notice). - name: Open baseline-refresh PR id: cpr uses: peter-evans/create-pull-request@v6 continue-on-error: true with: token: ${{ secrets.GITHUB_TOKEN }} branch: bot/visual-baselines-${{ github.run_id }} delete-branch: true title: "chore(visual): refresh baselines from ${{ inputs.base_url }}" commit-message: | chore(visual): refresh baselines from ${{ inputs.base_url }} Reason: ${{ inputs.reason }} Triggered by workflow_dispatch run ${{ github.run_id }} on the axiom self-hosted runner (CT 111). Byte-equivalent toolchain to visual-diff.yml — PNGs should match cleanly on the follow-up Screenshot diff job. body: | Automated baseline refresh from the [Seed visual baselines](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) workflow run. - **Source URL:** `${{ inputs.base_url }}` - **Reason:** ${{ inputs.reason }} - **Runner:** CT 111 self-hosted axiom runner (byte-equivalent toolchain to `visual-diff.yml`). ## Review checklist - [ ] Open the run's artifacts (or pull the PR locally and open `tests/visual/__screenshots__/home.png`) and visually confirm the new baseline matches an intentional UI state. - [ ] If matching intent: merge. - [ ] If wrong (transient overlay, wrong URL, debug bar visible): close the PR; investigate; re-dispatch with a corrected URL or after fixing the source. ## Convoy context Brief 1 of `harden-visual-diff-gate` (this workflow exists) is now usable. Brief 2 (flip `continue-on-error: true` → removed in `.github/workflows/visual-diff.yml`) is unblocked once this PR merges with a fresh baseline. labels: | automated visual-baselines add-paths: | tests/visual/__screenshots__/ # Three possible outcomes from the cpr step: # 1. No baseline changes → cpr.outputs.pull-request-number == '' AND no bot/ branch on origin # 2. PR opened successfully → cpr.outputs.pull-request-number != '' # 3. Branch pushed, PR-open blocked by org setting → cpr.outputs.pull-request-number == '' BUT bot/ branch exists on origin # The branch-exists query disambiguates outcomes 1 and 3. - name: Disambiguate outcome id: outcome if: always() && steps.cpr.outputs.pull-request-number == '' run: | BRANCH="bot/visual-baselines-${GITHUB_RUN_ID}" if git ls-remote --exit-code --heads origin "$BRANCH" >/dev/null 2>&1; then echo "kind=branch-pushed-pr-blocked" >> "$GITHUB_OUTPUT" echo "branch=$BRANCH" >> "$GITHUB_OUTPUT" else echo "kind=no-changes" >> "$GITHUB_OUTPUT" fi - name: Report — no changes if: steps.outcome.outputs.kind == 'no-changes' run: | echo "::notice::No baseline changes detected. The captures from ${{ inputs.base_url }} match the committed baselines — no PR opened." - name: Report — PR opened if: steps.cpr.outputs.pull-request-number != '' run: | echo "::notice::Opened PR #${{ steps.cpr.outputs.pull-request-number }} with refreshed baselines: ${{ steps.cpr.outputs.pull-request-url }}" - name: Report — branch pushed, manual PR-open required if: steps.outcome.outputs.kind == 'branch-pushed-pr-blocked' run: | BRANCH="${{ steps.outcome.outputs.branch }}" BASE_URL="${{ inputs.base_url }}" REASON="${{ inputs.reason }}" echo "::warning::Branch \`$BRANCH\` pushed with refreshed baselines, but PR-open was blocked by stwl-labs org setting (\"Allow GitHub Actions to create and approve pull requests\" is OFF)." echo "" echo "Operator: run this from your local checkout to open the baseline PR:" echo "" echo " gh pr create --base main --head $BRANCH \\\\" echo " --title \"chore(visual): refresh baselines from $BASE_URL\" \\\\" echo " --body \"Refresh baselines (run $GITHUB_RUN_ID). Reason: $REASON\"" echo "" echo "See AGENTS.md § Testing § Screenshot diff for the full operator runbook." exit 1