Commit graph

2 commits

Author SHA1 Message Date
Randall Stillwell
014c01cb5e convoy: add seed-visual-baselines workflow (harden-visual-diff-gate brief 1)
Adds a workflow_dispatch-triggered job on the self-hosted axiom runner
that captures fresh `tests/visual/__screenshots__/*.png` against a
caller-provided URL and opens a `chore(visual): refresh baselines from
<url>` PR via peter-evans/create-pull-request@v6.

This is brief 1 of 2 of the harden-visual-diff-gate convoy. The
workflow exists but is not invoked by this PR — operator dispatches via
the GitHub UI or `gh workflow run seed-visual-baselines.yml` once they
want a fresh baseline against post-glass-redesign main.

Brief 2 (flip continue-on-error: true off visual-diff.yml, add 9th
forbidden-patterns check) is unblocked once a fresh baseline lands via
this workflow's auto-PR.

Convoy decision D1 ratified: Option B (workflow_dispatch + auto-PR)
chosen over Option A (ad-hoc SSH-into-CT-111 + manual commit). The
workflow gives the baseline regeneration a reviewable Git-native shape;
the auto-PR carries a checklist for visual sanity-check before merge.

Workflow shape:
- Inputs: base_url (required, no default to avoid wrong-target
  accidents), reason (optional, used in PR body).
- Permissions: contents: write + pull-requests: write — sufficient for
  default GITHUB_TOKEN; no PAT needed.
- Caches: shares the node_modules + Playwright browser caches with
  visual-diff.yml so the byte-equivalence guarantee holds without
  cache miss overhead.
- Idempotent: peter-evans/create-pull-request short-circuits to a
  ::notice:: annotation if the captures match the existing committed
  baselines (no PR opened).

Metrics: logged role-conductor + role-architect + role-implementer
events for this convoy in .convoys/.metrics.jsonl. Satisfies the
convoy-metrics-gate (PR #134, 9eef8d9) that fires on convoy:-titled
PRs requiring at least one new metrics row.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 21:29:43 -05:00
varutasu
9eef8d9575
feat(convoy-metrics): un-gitignore .metrics.jsonl + add CI gate on convoy PRs (#134)
The pipeline metrics shim worked correctly through Jun 4 (65 role events,
25 convoys recorded) but silently stopped capturing thereafter. 8 convoy
PRs merged Jun 5-11 (#126-#133) with zero rows logged to
.convoys/.metrics.jsonl. The roles' Metrics sections clearly instruct the
agents to call scripts/log-convoy-event.sh after every hand-off, but the
instruction was skipped during multitask audit fan-outs and longer
sessions where the Metrics section fell out of working context.

Two changes to prevent the silent gap from recurring:

1. .gitignore: drop the `.convoys/.metrics.jsonl` ignore line. Convoy
   telemetry now committed in git so gaps surface in PR review. The
   script comment was already clear that events contain metadata only —
   no code, no prompts.
2. .github/workflows/convoy-metrics-gate.yml: new CI gate that fails any
   PR titled `convoy:` if no rows were added to .convoys/.metrics.jsonl
   between base and head. Bypass with the `skip-metrics` label + a
   documented reason. Non-convoy PRs are no-op.

Also: commits the existing 65-event history to git so future analysis
(and the §10 measurement protocol in agent-pipeline's v0.4 plan) has a
stable baseline to compare against.

Runs on self-hosted axiom runner to inherit the lower GH minutes cost
the Jun 5-11 work already migrated to (PR #132).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 17:46:28 -05:00