---
kind: task
slug: repo-doc-ingestion
title: Idempotent repository context ingestion
plan_slug: automation-foundation
epic_slug: repo-discovery-onboarding
status: ready
priority: P1
tenant_id: global
owner: unassigned
cursor_todo_id: null
updated_at: "2026-09-25"
---

# Task summary

Provide an explicit, retryable import that reads bounded repository documentation and existing convoy manifests through the forge API, stores source provenance, and avoids duplicate Echodo objects.

## Description

Remote reads should not lengthen or roll back the batch-connect transaction. After connection, an admin starts import and sees per-repo status. V1 scans only root `README.md`, `AGENTS.md`, `CONTRIBUTING.md`, and `.convoys/*.md`; recursive `docs/**` and `plans/**` import is deferred.

## Subtasks

- [ ] Add bounded file/list-directory forge contracts with response, media type, file-count, and byte-size validation.
- [ ] Extract pure convoy markdown parsing into a neutral shared package so web and MCP do not import each other's app source.
- [ ] Define durable source identity and import status by workspace, connected repo, and source path; generate a new migration if schema changes are needed.
- [ ] Store docs as the canonical `document` object type with forge/repo/path/revision metadata.
- [ ] Reconcile valid `.convoys/*.md` into workspace-scoped projects and valid brief metadata into child tasks.
- [ ] Add admin-only import start/status/retry APIs or an equivalent bounded background execution mechanism.
- [ ] Report imported, unchanged, skipped, and failed counts without aborting successful files.
- [ ] Test idempotency, malformed manifests, missing optional files, partial failure, limits, and tenant isolation.

## Security and sync constraints

- Remote files are read-only in this task.
- Re-import updates only objects with matching source identity and a changed revision.
- Missing files are `skipped`, while authentication/rate-limit/parse/persistence failures are visible.
- Read `config/CursorSync.md` before changing reconciliation behavior.

## Estimation

L

## Acceptance criteria

- [ ] Re-running an unchanged import creates no duplicate documents, convoys, or briefs.
- [ ] Imported docs use `type: "document"` and record source repo/path/revision.
- [ ] Valid convoy manifests appear in Mission Control; malformed manifests do not block other imports.
- [ ] File count and byte limits prevent unbounded repository ingestion.
- [ ] Failures are retryable and reported per repo/file.
- [ ] New schema changes include generated migration SQL.
- [ ] Lint, type-check, test, and relevant builds pass.

## Links

- Epic: `./Epic-repo-discovery-onboarding.md`
- Depends on: `./Task-forge-discovery-api.md`, `./Task-repo-picker-modal.md`
- Convoy brief: `../../../.convoys/repo-discovery-onboarding/brief-3-ingestion.md`