deckhearth/.convoys/fix-vercel-deployment-protection-in-ci.md

name	classification	success_metric	skip	status	created	parent	addresses	depends_on
fix-vercel-deployment-protection-in-ci	convoy	`Playwright smoke` and `Screenshot diff` workflows reach their actual smoke / capture step on every PR (no more 401-from-Vercel-SSO 10-min timeouts). Both workflows complete in < 5 minutes. Failures, when they occur, are real assertion failures — not auth failures against the preview URL.	role-design-system-auditor role-a11y-auditor role-ux-reviewer role-ia-architect browser-smoke	queued	2026-05-24	ship-readiness	P0	fix-layout-default-user (shipped — PR PR

Fix Vercel Deployment Protection in CI

Plumb VERCEL_AUTOMATION_BYPASS_SECRET into the Playwright smoke and Screenshot diff workflows so anonymous GitHub Actions runners can actually GET the preview URL without hitting Vercel's SSO 401 challenge. Without this, both workflows permanently red on every PR — just slower red than before PR #16.

Why now

PR #16 (fix(ci): scoped permissions, squash commit 7e97254) added minimal scoped permissions: blocks to .github/workflows/preview-smoke.yml and .github/workflows/visual-diff.yml. That fixed the 5-second 403 "Resource not accessible by integration" failure both workflows hit when trying to call the GitHub deployments API. However, with permissions correct, both workflows now reach the actual deployment check and fail with a different error: a 10-minute timeout from patrickedqvist/wait-for-vercel-preview@v1.3.2's subsequent HTTP GET against the preview URL, which Vercel returns 401 for because Deployment Protection is on (anonymous GitHub-runner request → Vercel SSO challenge). Cost: ~10 minutes of runner time per workflow per PR — and zero signal, since neither workflow ever reaches its smoke step. This blocks PR #15's recurring follow-up convoys (visual-regression baselines, Playwright smoke for adopt-playwright-smoke) from getting any CI feedback.

The bypass token already exists locally as VERCEL_AUTOMATION_BYPASS_SECRET in .env.local (Protection Bypass for Automation, configured in the Vercel project). Documented in AGENTS.md § 7 — Deployment. The work here is plumbing it from operator-supplied repo secret → workflow env → wait-for-vercel-preview's path: input + the eventual Playwright BASE_URL so anonymous runner requests bypass the SSO challenge.

Operator action required (BEFORE this convoy can run)

This convoy CANNOT proceed without the operator first seeding the secret into GitHub Actions. The implementer has nothing to wire up if the secret isn't visible to the workflows.

1. Seed the secret:

   gh secret set VERCEL_AUTOMATION_BYPASS_SECRET --body "<value from local .env.local>"
   

   (The value is whatever VERCEL_AUTOMATION_BYPASS_SECRET=… says in .env.local. Do not paste it anywhere logged. Do not echo it from a workflow step.)
2. Confirm visibility:

   gh secret list
   

   Expect to see VERCEL_AUTOMATION_BYPASS_SECRET listed alongside the existing repo secrets. Note: gh secret list shows names only — never values — by design.
3. Notify the next agent that steps 1 + 2 are done. The convoy file's frontmatter status: should flip from queued to in-progress only after this notification.

This is the same pattern npm run setup-db's ADMIN_INITIAL_PASSWORD established (drop-public-setup Brief 1, commit ff80753): CI / scripts that need a secret get an actionable fail-loud error when the secret is missing, and the operator seeds it once per environment.

Decisions to ratify with operator

Queued; do not pre-decide.

1. Bypass via query param vs. request header.
   • Option A — query param. Append ?x-vercel-protection-bypass=...&x-vercel-set-bypass-cookie=true to the wait-action's path: input AND to the Playwright BASE_URL. The first request sets a _vercel_jwt cookie on the runner's ephemeral browser context; subsequent same-origin requests reuse it. Pro: works with any HTTP client, no custom config in Playwright. Con: the bypass token shows up in workflow run logs if any step echoes the URL (mitigation: never echo or cat a URL containing the token; log ${{ steps.wait.outputs.url }} only after stripping the query string).
   • Option B — request header (x-vercel-protection-bypass: <secret>). Cleaner — the token never appears in any URL. But requires custom HTTP-client config in playwright.config.js (extraHTTPHeaders) AND in wait-for-vercel-preview (the action's docs need confirming — header support may not be exposed via inputs).
2. CI assertion that bypass actually works. Should we add a step that explicitly asserts 200 on the preview URL during the wait- action's healthcheck phase, before handing off to Playwright / screenshot capture? This would surface bypass-misconfiguration as a fast-fail step instead of letting Playwright time out 8 minutes later on a different error. Cost: ~5 lines of YAML; benefit: clearer failure signal for the next operator-touch event.
3. Workflow concurrency cancellation. The workflows already use concurrency: keyed on github.ref. Confirm that the bypass-token wiring doesn't inadvertently break the cancel-stale behavior (e.g. if the secrets.VERCEL_AUTOMATION_BYPASS_SECRET reference is in a concurrency: expression, that's a syntax error and the implementer should pull it into a job-level env: instead).

Scope

In scope:

• .github/workflows/preview-smoke.yml — wire the bypass into the wait-for-vercel-preview step's path: input (Option A) OR add the bypass header via the action's input shape (Option B, pending confirmation that the action exposes header inputs).
• .github/workflows/visual-diff.yml — same treatment as preview-smoke (the two workflows have similar shapes; whatever pattern works for one should land in both).
• playwright.config.js (when it exists — the adopt-playwright-smoke convoy ships it) — add use: { extraHTTPHeaders: { 'x-vercel-protection-bypass': process.env.VERCEL_AUTOMATION_BYPASS_SECRET } } if Decision #1 picks Option B; OR build the BASE_URL with the query param (Option A).
• Any test-setup file or helper that constructs the preview URL for screenshot-diff-style workflows.

Out of scope:

• Writing new Playwright tests. Test authoring lives in adopt-playwright-smoke. This convoy only makes the existing smoke pipeline reachable.
• Broadening workflow permissions: blocks. PR #16 already landed the minimal scope; this convoy should not need to touch them again.
• Replacing patrickedqvist/wait-for-vercel-preview with a different action. The action retrieves the URL successfully (confirmed in PR #16's run logs); the failure is the subsequent HTTP GET, which is a configuration issue, not an action choice. A wholesale action swap is a deeper rewrite — separate convoy if/when it's needed.
• Authoring new visual-regression baselines. The screenshot diff workflow has nothing meaningful to compare against today; baseline authoring is its own convoy.
• Disabling Vercel Deployment Protection on the project. Operator may prefer to keep protected previews (cheap defense-in-depth against preview-URL leakage); this fix lets CI work around the protection without weakening it.

Known constraints

• wait-for-vercel-preview@v1.3.2 path: input is supported. PR #16's run logs confirm the action retrieves the URL successfully — the subsequent HTTP GET is what fails. The action's path: input accepts a full path including query string, so Option A (?x-vercel-protection-bypass=...) is mechanically straightforward. Whether the action exposes a way to inject custom request headers (Option B) needs to be confirmed by reading the action's source / README before the implementer commits to it.
• The same secret will need to be plumbed into Playwright's BASE_URL or into a request header in playwright.config.js when the adopt-playwright-smoke convoy ships. Coordinating shape now (this convoy) vs. shape later (when Playwright lands) saves churn — the implementer should pick whichever option keeps both call sites consistent.
• npm run setup-db's ADMIN_INITIAL_PASSWORD is a parallel precedent for "CI needs a secret the operator must seed." Same pattern applies: secret is repo-scoped, fail-loud (or fail-noisy) when unset, never echoed to logs. See drop-public-setup Brief 1 (commit ff80753).
• Token rotation. The Vercel bypass token can be rotated from the Vercel dashboard. If/when that happens, the operator must re-seed the GitHub secret (gh secret set ...). No automation here — this is a human responsibility per the same pattern as JWT_SECRET rotation.

Acceptance criteria

The convoy is shippable when ALL of the following hold:

1. Playwright smoke workflow reaches its actual smoke step on a fresh PR. It either passes (smoke green) OR fails on a real assertion (Playwright reports a test failure or a runtime error from the smoke spec). It does NOT fail with a 10-min timeout from the wait-for-vercel-preview step or with a 401 from the preview URL.
2. Screenshot diff workflow reaches its screenshot capture step and posts the "Visual Diff" comment to the PR (even if the diff itself is empty / first-run / null-baseline). Same constraint: no 10-min timeout, no 401.
3. Both workflows complete in < 5 minutes on a typical PR (the pre-PR-16 baseline was ~30 seconds for the workflow body; adding a bypass query string or header shouldn't materially affect runtime).
4. The bypass token does not appear in any workflow run log. Verify by downloading the raw log of a passing run and grepping for the token's first 8 chars.
5. Workflow YAML still passes basic actionlint review (actionlint .github/workflows/*.yml exits 0). PR #16's permissions blocks remain unchanged.
6. AGENTS.md § 7 deployment paragraph (the "Preview protection bypass for automation" line) still reflects reality after the change. May need a one-sentence update if the implementer picks Option B (x-vercel-protection-bypass header) vs. Option A (query string).

Anything flagged but not acted on (in advance)

These are real findings that the architect / implementer should NOT try to solve in this convoy. Each is queued separately if it warrants a fix.

• The wait-for-vercel-preview action is no longer maintained (last release Mar 2024; no v2). Could be replaced with a few lines of gh api + curl-loop in the workflow itself. Not in scope here — this convoy needs to fix the immediate auth failure, not rewrite the wait logic. Queue as replace-wait-for-vercel-preview if the action ages out further or has a security advisory.
• Playwright config doesn't exist yet. playwright.config.js, tests/smoke/, and @playwright/test all land in adopt-playwright-smoke (P1 #10 step 2 / launch sequence step 10). Until that convoy ships, the only Playwright smoke workflow body is a no-op. This convoy can pre-wire the bypass infrastructure (env var, workflow secrets) so adopt-playwright-smoke only needs to add the test files and the Playwright config — but it can't ship a real smoke-pass without that follow-up.
• Screenshot diff baseline authoring. Even after this convoy lands, the visual-diff workflow has nothing to compare against on its first run. That's expected and orthogonal — baseline authoring is a separate scope.
• Operator-rotation hygiene for VERCEL_AUTOMATION_BYPASS_SECRET. Vercel's bypass tokens don't auto-expire. If the team wants a periodic rotation policy, that's an ops-runbook concern outside this convoy.
• AGENTS.md § 7 wording. The current "Smoke/visual-diff workflows pass this header (x-vercel-protection-bypass)" line in § 7 is aspirational — it describes intent, not what was actually wired. After this convoy ships, that line becomes accurate. The doc-writer pass at convoy close should reword to past-tense reality.