60 lines
2.8 KiB
Markdown
60 lines
2.8 KiB
Markdown
|
---
|
||
|
|
kind: task
|
||
|
|
slug: repo-doc-ingestion
|
||
|
|
title: Idempotent repository context ingestion
|
||
|
|
plan_slug: automation-foundation
|
||
|
|
epic_slug: repo-discovery-onboarding
|
||
|
|
status: ready
|
||
|
|
priority: P1
|
||
|
|
tenant_id: global
|
||
|
|
owner: unassigned
|
||
|
|
cursor_todo_id: null
|
||
|
|
updated_at: "2026-09-25"
|
||
|
|
---
|
||
|
|
|
||
|
|
# Task summary
|
||
|
|
|
||
|
|
Provide an explicit, retryable import that reads bounded repository documentation and existing convoy manifests through the forge API, stores source provenance, and avoids duplicate Echodo objects.
|
||
|
|
|
||
|
|
## Description
|
||
|
|
|
||
|
|
Remote reads should not lengthen or roll back the batch-connect transaction. After connection, an admin starts import and sees per-repo status. V1 scans only root `README.md`, `AGENTS.md`, `CONTRIBUTING.md`, and `.convoys/*.md`; recursive `docs/**` and `plans/**` import is deferred.
|
||
|
|
|
||
|
|
## Subtasks
|
||
|
|
|
||
|
|
- [ ] Add bounded file/list-directory forge contracts with response, media type, file-count, and byte-size validation.
|
||
|
|
- [ ] Extract pure convoy markdown parsing into a neutral shared package so web and MCP do not import each other's app source.
|
||
|
|
- [ ] Define durable source identity and import status by workspace, connected repo, and source path; generate a new migration if schema changes are needed.
|
||
|
|
- [ ] Store docs as the canonical `document` object type with forge/repo/path/revision metadata.
|
||
|
|
- [ ] Reconcile valid `.convoys/*.md` into workspace-scoped projects and valid brief metadata into child tasks.
|
||
|
|
- [ ] Add admin-only import start/status/retry APIs or an equivalent bounded background execution mechanism.
|
||
|
|
- [ ] Report imported, unchanged, skipped, and failed counts without aborting successful files.
|
||
|
|
- [ ] Test idempotency, malformed manifests, missing optional files, partial failure, limits, and tenant isolation.
|
||
|
|
|
||
|
|
## Security and sync constraints
|
||
|
|
|
||
|
|
- Remote files are read-only in this task.
|
||
|
|
- Re-import updates only objects with matching source identity and a changed revision.
|
||
|
|
- Missing files are `skipped`, while authentication/rate-limit/parse/persistence failures are visible.
|
||
|
|
- Read `config/CursorSync.md` before changing reconciliation behavior.
|
||
|
|
|
||
|
|
## Estimation
|
||
|
|
|
||
|
|
L
|
||
|
|
|
||
|
|
## Acceptance criteria
|
||
|
|
|
||
|
|
- [ ] Re-running an unchanged import creates no duplicate documents, convoys, or briefs.
|
||
|
|
- [ ] Imported docs use `type: "document"` and record source repo/path/revision.
|
||
|
|
- [ ] Valid convoy manifests appear in Mission Control; malformed manifests do not block other imports.
|
||
|
|
- [ ] File count and byte limits prevent unbounded repository ingestion.
|
||
|
|
- [ ] Failures are retryable and reported per repo/file.
|
||
|
|
- [ ] New schema changes include generated migration SQL.
|
||
|
|
- [ ] Lint, type-check, test, and relevant builds pass.
|
||
|
|
|
||
|
|
## Links
|
||
|
|
|
||
|
|
- Epic: `./Epic-repo-discovery-onboarding.md`
|
||
|
|
- Depends on: `./Task-forge-discovery-api.md`, `./Task-repo-picker-modal.md`
|
||
|
|
- Convoy brief: `../../../.convoys/repo-discovery-onboarding/brief-3-ingestion.md`
|