# Task: Wire Pipeline Orchestrator

## Current State

`src/data-pipeline/pipeline.py` exists but is not connected to Celery Beat or the backend. The four ingestion scripts (prices, sec, financials, news) run standalone — no dependency ordering, no error handling between stages, no visibility into pipeline health.

## Goal

Create a proper pipeline orchestrator that:
1. Chains ingestion tasks in correct dependency order
2. Provides health/status endpoints
3. Integrates with existing Celery infrastructure

## Requirements

1. Define dependency graph: prices → sec → financials → news (prices must run first)
2. Create `src/backend/tasks/pipeline.py` with orchestration logic
3. Add `/api/v1/sync/status` endpoint (already exists per README — verify it works)
4. Create Celery chain/group for full pipeline run
5. Add manual trigger endpoint: `POST /api/v1/sync/pipeline/run`
6. Add to docker-compose worker service

## Acceptance Criteria

- Full pipeline runs end-to-end via single API call
- Dependency ordering enforced (prices before sec, etc.)
- Pipeline status visible via `/api/v1/sync/status`
- Failed stage does not block other independent stages
- Pipeline health check returns last run time, status, errors

## Constraints

- Use Celery chains for ordered stages, Celery groups for parallel stages
- Follow existing task patterns — don't reinvent error handling
- Keep under 200 lines per framework rule
- Read existing migrations before writing schema

## Files to Create/Modify

- `src/backend/tasks/pipeline.py` (new)
- `src/backend/routers/data_sync.py` (add pipeline trigger endpoint)
- `src/backend/celery_app.py` (add pipeline schedule)
- `docker-compose.dev.yml` (verify worker includes task)

## Next Steps After This Task

Wire backtest integration (phase2-backtest-integration)