51 lines
3.3 KiB
Markdown
51 lines
3.3 KiB
Markdown
---
|
|
brief: 3
|
|
convoy: repo-discovery-onboarding
|
|
role: role-implementer
|
|
title: Idempotent repository context ingestion
|
|
status: ready
|
|
priority: P1
|
|
slice_dependencies: [1, 2]
|
|
recommended_model: qwen-coder
|
|
model_tier: local
|
|
workspace_id: global
|
|
---
|
|
|
|
# Brief 3 — Idempotent repository context ingestion
|
|
|
|
## Goal
|
|
|
|
After repositories are connected, an explicit retryable ingestion action imports supported repository documentation and existing convoy manifests with provenance, without making remote API latency part of the batch-connect transaction.
|
|
|
|
Backlog task: `plans/Plan-automation-foundation/Epic-repo-discovery-onboarding/Task-repo-doc-ingestion.md`.
|
|
|
|
## Architecture constraints
|
|
|
|
- Use the canonical object type `document`, not `doc`.
|
|
- Do not import an unbounded repository tree. V1 paths are root `README.md`, `AGENTS.md`, `CONTRIBUTING.md`, and `.convoys/*.md`; recursive `docs/**` and `plans/**` are deferred until limits and conflict semantics are designed.
|
|
- Import is idempotent by `(workspace, connectedRepoId, sourcePath)`, and records source forge, repo, path, revision/sha, and last import result in metadata.
|
|
- Keep remote content read-only. Re-import updates the matching imported object only when the remote revision changes.
|
|
- Share convoy parsing/reconciliation logic from a neutral package; do not import `apps/mcp-server` source into `apps/web`.
|
|
- A missing optional file is not an error. Authentication, rate-limit, parse, and persistence failures are visible per repo and retryable.
|
|
|
|
## Files
|
|
|
|
- Shared connected-repo schemas/services — add bounded tree/list-file and fetch-text-file contracts with size/type validation.
|
|
- Forge adapters — implement GitHub and Forgejo/Gitea directory/file reads against the connected repo's default branch.
|
|
- Extract reusable convoy markdown parsing from `apps/mcp-server/src/lib/convoy-files.ts` into a package usable by web and MCP, preserving local-file behavior.
|
|
- Database schema/migration if needed — add durable repository-source identity and ingestion status. Every new tenant row includes `workspace_id` and an index.
|
|
- Web tRPC/service — add admin-only `startImport`/`importStatus` or an equivalent bounded background execution path. Do not perform all remote reads inside the `batchConnect` transaction.
|
|
|
|
## Acceptance
|
|
|
|
- Importing a repo creates or updates `document` objects for supported files with source provenance.
|
|
- Re-running an unchanged import creates no duplicates.
|
|
- Existing `.convoys/*.md` manifests reconcile to workspace-scoped convoy projects. Brief files are imported only when their schema is valid and can be linked safely.
|
|
- One bad file does not roll back successful files; the result reports imported, unchanged, skipped, and failed counts.
|
|
- File count and byte limits prevent accidental large imports.
|
|
- Unit/integration tests cover idempotency, missing files, malformed convoy frontmatter, tenant isolation, and partial failure.
|
|
- Generate and commit a new migration for schema changes; lint, type-check, test, and relevant build checks pass.
|
|
|
|
## Prompt for implementer
|
|
|
|
Implement only Brief 3 after Briefs 1 and 2. Read `config/CursorSync.md` before changing sync behavior. Preserve the local-first `.convoys/` contract and never mutate remote files. Keep ingestion bounded and retryable; do not silently invent conflict resolution for `plans/**`.
|