feat(convoy-metrics): un-gitignore .metrics.jsonl + add CI gate on convoy PRs #134

Merged

varutasu merged 1 commit from feat/restore-convoy-metrics-gate into main

2026-06-12 18:46:28 -04:00

varutasu commented

2026-06-11 18:22:19 -04:00

(Migrated from github.com)

Why

The convoy metrics shim worked correctly through Jun 4 (65 role events, 25 convoys recorded in .convoys/.metrics.jsonl) but silently stopped capturing thereafter. 8 convoy PRs merged Jun 5-11 (#126-#133) with zero rows logged. The L2 roles' ## Metrics sections clearly instruct agents to call scripts/log-convoy-event.sh after every hand-off, but the instruction was skipped during multitask audit fan-outs and longer sessions where the Metrics section fell out of the agent's working context.

Without telemetry, the agent-pipeline analytics/v0.4-beta1-results.md §10 measurement protocol can't run, kill criteria can't trigger, and the closed-loop improvement story (Phases 6/7/8) has no fuel.

See agent-pipeline / docs/PHASE-2A-DESCOPE.md and analytics/v0.4-beta1-results.md §Observational data for the data trail.

What

Two changes:

.gitignore: drop the .convoys/.metrics.jsonl ignore line. Convoy telemetry is now committed in git so silent gaps surface in PR review. The shim header already documented "events contain only metadata — no code, no prompts" so this is safe.
.github/workflows/convoy-metrics-gate.yml: new CI workflow that fails any PR titled convoy: (case-insensitive) if no rows were added to .convoys/.metrics.jsonl between base and head. Bypass: add the skip-metrics label to the PR and document why in the body. Non-convoy PRs are no-op.

Also: commits the existing 65-event history so future baselines have a stable reference point.

Test plan

Shim works locally: ran bash scripts/log-convoy-event.sh role=role-test convoy=metrics-restoration-check duration_s=1 — file appended cleanly (test row removed before commit).
CI passes on this PR (does NOT trigger the new gate since title prefix is feat(convoy-metrics): not convoy:).
After merge: next convoy PR opens, gate fires, fails if no .metrics.jsonl change.

Behavior changes for future convoy work

Once merged, L2 roles MUST call bash scripts/log-convoy-event.sh role=role-X convoy=<slug> ... at every hand-off, exactly as their existing ## Metrics sections instruct. The CI gate will catch any convoy PR that skips it.

Bypass scenarios for the skip-metrics label:

Pure docs-only convoys that ship without a role pipeline (rare).
Manual cherry-picks where the agent loop wasn't involved.
Truly broken telemetry on a follow-up PR while we debug.

varutasu/agent-pipeline#1 — companion Phase 1a + descope-Phase-2a PR.
This PR is not a convoy PR (title prefix feat(convoy-metrics): not convoy:) so the new gate does not self-trigger.

Made with Cursor

## Why The convoy metrics shim worked correctly through Jun 4 (65 role events, 25 convoys recorded in `.convoys/.metrics.jsonl`) but **silently stopped capturing thereafter**. 8 convoy PRs merged Jun 5-11 (#126-#133) with zero rows logged. The L2 roles' `## Metrics` sections clearly instruct agents to call `scripts/log-convoy-event.sh` after every hand-off, but the instruction was skipped during multitask audit fan-outs and longer sessions where the Metrics section fell out of the agent's working context. Without telemetry, the agent-pipeline `analytics/v0.4-beta1-results.md` §10 measurement protocol can't run, kill criteria can't trigger, and the closed-loop improvement story (Phases 6/7/8) has no fuel. See [`agent-pipeline / docs/PHASE-2A-DESCOPE.md`](https://github.com/varutasu/agent-pipeline/blob/feat/phase-1a-design-skills/docs/PHASE-2A-DESCOPE.md) and [`analytics/v0.4-beta1-results.md` §Observational data](https://github.com/varutasu/agent-pipeline/blob/feat/phase-1a-design-skills/analytics/v0.4-beta1-results.md) for the data trail. ## What Two changes: 1. **`.gitignore`**: drop the `.convoys/.metrics.jsonl` ignore line. Convoy telemetry is now committed in git so silent gaps surface in PR review. The shim header already documented "events contain only metadata — no code, no prompts" so this is safe. 2. **`.github/workflows/convoy-metrics-gate.yml`**: new CI workflow that fails any PR titled `convoy:` (case-insensitive) if no rows were added to `.convoys/.metrics.jsonl` between base and head. Bypass: add the `skip-metrics` label to the PR and document why in the body. Non-convoy PRs are no-op. Also: commits the existing 65-event history so future baselines have a stable reference point. ## Test plan - [x] Shim works locally: ran `bash scripts/log-convoy-event.sh role=role-test convoy=metrics-restoration-check duration_s=1` — file appended cleanly (test row removed before commit). - [ ] CI passes on this PR (does NOT trigger the new gate since title prefix is `feat(convoy-metrics):` not `convoy:`). - [ ] After merge: next convoy PR opens, gate fires, fails if no `.metrics.jsonl` change. ## Behavior changes for future convoy work Once merged, L2 roles MUST call `bash scripts/log-convoy-event.sh role=role-X convoy=<slug> ...` at every hand-off, exactly as their existing `## Metrics` sections instruct. The CI gate will catch any convoy PR that skips it. Bypass scenarios for the `skip-metrics` label: - Pure docs-only convoys that ship without a role pipeline (rare). - Manual cherry-picks where the agent loop wasn't involved. - Truly broken telemetry on a follow-up PR while we debug. ## Related - `varutasu/agent-pipeline#1` — companion Phase 1a + descope-Phase-2a PR. - This PR is **not** a convoy PR (title prefix `feat(convoy-metrics):` not `convoy:`) so the new gate does not self-trigger. Made with [Cursor](https://cursor.com)

vercel[bot] commented

2026-06-11 18:22:25 -04:00

(Migrated from github.com)

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
tcg-vault	Ready	Preview, Comment	Jun 11, 2026 10:22pm

[vc]: #EltXSLAJr+ZuKpDun2vRmX2toY4WmybacFVWSnVTyIc=:eyJpc01vbm9yZXBvIjp0cnVlLCJ0eXBlIjoiZ2l0aHViIiwicHJvamVjdHMiOlt7Im5hbWUiOiJ0Y2ctdmF1bHQiLCJwcm9qZWN0SWQiOiJwcmpfRjZXOEVvRkd3Y0g3aWVGcnRvRlNlOXdVVkFhNSIsImxpdmVGZWVkYmFjayI6eyJyZXNvbHZlZCI6MCwidW5yZXNvbHZlZCI6MCwidG90YWwiOjAsImxpbmsiOiJ0Y2ctdmF1bHQtZ2l0LWZlYXQtcmVzdG9yZS1jLTBlYmMwYy1yYW5kYWxsLXN0aWxsd2VsbHMtcHJvamVjdHMudmVyY2VsLmFwcCJ9LCJpbnNwZWN0b3JVcmwiOiJodHRwczovL3ZlcmNlbC5jb20vcmFuZGFsbC1zdGlsbHdlbGxzLXByb2plY3RzL3RjZy12YXVsdC9IVzFOOHJ2enhBMjNtTnNiVlhGMkEyNUo4c0EyIiwicHJldmlld1VybCI6InRjZy12YXVsdC1naXQtZmVhdC1yZXN0b3JlLWMtMGViYzBjLXJhbmRhbGwtc3RpbGx3ZWxscy1wcm9qZWN0cy52ZXJjZWwuYXBwIiwibmV4dENvbW1pdFN0YXR1cyI6IkRFUExPWUVEIn1dLCJyZXF1ZXN0UmV2aWV3VXJsIjoiaHR0cHM6Ly92ZXJjZWwuY29tL3ZlcmNlbC1hZ2VudC9yZXF1ZXN0LXJldmlldz9vd25lcj1zdHdsLWxhYnMmcmVwbz10Y2ctdmF1bHQmcHI9MTM0In0= The latest updates on your projects. Learn more about [Vercel for GitHub](https://vercel.link/github-learn-more). | Project | Deployment | Actions | Updated (UTC) | | :--- | :----- | :------ | :------ | | [tcg-vault](https://vercel.com/randall-stillwells-projects/tcg-vault) | ![Ready](https://vercel.com/static/status/ready.svg) [Ready](https://vercel.com/randall-stillwells-projects/tcg-vault/HW1N8rvzxA23mNsbVXF2A25J8sA2) | [Preview](https://tcg-vault-git-feat-restore-c-0ebc0c-randall-stillwells-projects.vercel.app), [Comment](https://vercel.live/open-feedback/tcg-vault-git-feat-restore-c-0ebc0c-randall-stillwells-projects.vercel.app?via=pr-comment-feedback-link) | Jun 11, 2026 10:22pm | <a href="https://vercel.com/vercel-agent/request-review?owner=stwl-labs&repo=tcg-vault&pr=134" rel="noreferrer"><picture><source media="(prefers-color-scheme: dark)" srcset="https://agents-vade-review.vercel.sh/request-review-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://agents-vade-review.vercel.sh/request-review-light.svg"><img src="https://agents-vade-review.vercel.sh/request-review-light.svg" alt="Request Review"></picture></a>

github-actions[bot] commented

2026-06-11 18:22:39 -04:00

(Migrated from github.com)

Pipeline Health

Build + CI gates

Gate	Status
Vercel build (Preview)	✅ pass
CI: Lint	✅ pass
CI: Schema map fresh	❌ skipped
Preview smoke	✅ pass
Visual diff	⏭ skipped or pending

Build runs on Vercel; this CI runs lint and schema-map drift only (no duplicate build).

Role reports

Role	Status
Reviewer report	⏳ pending
A11y audit	⏳ pending
Design system audit	⏳ pending

See individual comments above for details. This rollup updates automatically.

## Pipeline Health ### Build + CI gates | Gate | Status | | --- | --- | | Vercel build (Preview) | ✅ pass | | CI: Lint | ✅ pass | | CI: Schema map fresh | ❌ skipped | | Preview smoke | ✅ pass | | Visual diff | ⏭ skipped or pending | _Build runs on Vercel; this CI runs lint and schema-map drift only (no duplicate build)._ ### Role reports | Role | Status | | --- | --- | | Reviewer report | ⏳ pending | | A11y audit | ⏳ pending | | Design system audit | ⏳ pending | See individual comments above for details. This rollup updates automatically.