feat: multiple updates - alerts, auth, sectors, rotation service, financials ingestion, task specs, and agent framework

tasks/alert-service-cleanup/SPEC.md

@@ -0,0 +1,37 @@
+# Task: Cleanup alert router — remove DB calls, consolidate
+
+## Goal
+Remove all remaining direct database calls from `routers/alerts.py`. The router should now be a thin HTTP layer only.
+
+## Requirements
+
+### Changes to `src/backend/routers/alerts.py`
+
+1. Remove direct imports of `execute_query`, `execute_command`, `execute_one` from `database`
+2. Remove the `_require_watchlist_owner()` helper function (moved to AlertService)
+3. Ensure all endpoints use `AlertService` exclusively
+4. The router should only contain:
+   - Route decorators and signatures
+   - Auth extraction (`current_user["id"]`)
+   - Service method calls
+   - Response model wrapping
+5. The file should be ~60-80 lines (down from ~200+)
+
+### Service layer additions if needed
+If AlertService is missing a helper method the router needs, add it:
+- `build_watchlist_in_clause(user_id)` — returns (placeholders, params) for user's watchlists
+- Any other query helper needed by the listing endpoint
+
+## Acceptance Criteria
+1. `routers/alerts.py` has no direct `execute_query`/`execute_command`/`execute_one` calls
+2. File is under 100 lines
+3. All alert tests still pass: `pytest tests/test_alerts.py --tb=short`
+4. No import errors
+
+## Files to Modify
+- `src/backend/routers/alerts.py`
+- `src/backend/services/alert_service.py` (if new helpers needed)
+
+## Files to Read First
+- `src/backend/routers/alerts.py` — current state
+- `src/backend/services/alert_service.py` — current service

tasks/alert-service-create/SPEC.md

@@ -0,0 +1,42 @@
+# Task: Create AlertService class
+
+## Goal
+Create `src/backend/services/alert_service.py` with the `AlertService` class. This is Phase 1 of the alert refactor — only create the service, don't touch the router yet.
+
+## Requirements
+
+### Create `src/backend/services/alert_service.py`
+
+Implement an `AlertService` class with these methods:
+
+1. `get_user_alerts(watchlist_id, status, page, page_size, user_id)` — returns list of alert dicts + total count
+2. `create_alert(watchlist_id, type, trigger_type, message, severity, ticker, metadata, user_id)` — inserts alert, returns alert dict
+3. `get_alert(alert_id, user_id)` — returns alert dict or None
+4. `update_alert(alert_id, updates_dict, user_id)` — updates fields, returns alert dict
+5. `resolve_alert(alert_id, user_id)` — sets status='resolved', returns alert dict
+6. `dismiss_alert(alert_id, user_id)` — deletes alert, returns None
+7. `_require_watchlist_owner(watchlist_id, user_id)` — raises HTTPException if not owner
+
+### Implementation Details
+- Import `execute_query`, `execute_command`, `execute_one` from `database`
+- Import `HTTPException` from `fastapi`
+- All methods should be `async`
+- Use parameterized queries (no f-strings for SQL)
+- Handle the `watchlist_id IN (...)` pattern for listing alerts across multiple watchlists
+
+### Service Pattern
+Follow the existing `SentimentService` or `RotationService` pattern for class structure.
+
+## Acceptance Criteria
+1. File `src/backend/services/alert_service.py` exists
+2. `AlertService` class has all 7 methods listed above
+3. File is under 200 lines (per RULES.md)
+4. Can be imported without errors: `from services.alert_service import AlertService`
+
+## Files to Create
+- `src/backend/services/alert_service.py`
+
+## Files to Read First
+- `src/backend/routers/alerts.py` — extract logic from each endpoint
+- `src/backend/services/sentiment_service.py` — follow as pattern reference
+- `src/backend/services/rotation_service.py` — follow as pattern reference

tasks/alert-service-migrate-get/SPEC.md

@@ -0,0 +1,37 @@
+# Task: Migrate GET endpoints to AlertService
+
+## Goal
+Replace `routers/alerts.py` GET endpoints (`get_user_alerts`, `get_alert`) to use `AlertService` instead of direct DB calls.
+
+## Requirements
+
+### Changes to `src/backend/routers/alerts.py`
+
+1. Import `AlertService` from `services.alert_service`
+2. Create a service instance: `service = AlertService()`
+3. Replace `get_user_alerts()` body:
+   - Remove all DB query logic (the ~40 lines of query building)
+   - Call `service.get_user_alerts(watchlist_id, status, page, page_size, user_id)`
+   - Return the result wrapped in `AlertListResponse`
+4. Replace `get_alert()` body:
+   - Use `service.get_alert(alert_id, user_id)` 
+   - Keep the watchlist owner check using `service._require_watchlist_owner()`
+   - Return result wrapped in `AlertResponse`
+
+### Constraints
+- Keep the existing route signatures, decorators, and response models unchanged
+- Keep the `resolve_alert`, `dismiss_alert`, `update_alert`, and `create_alert` methods calling DB directly (they will be migrated in the next task)
+- File must stay under 200 lines total
+
+## Acceptance Criteria
+1. `get_user_alerts` uses `AlertService.get_user_alerts()` internally
+2. `get_alert` uses `AlertService.get_alert()` internally
+3. All existing GET tests still pass: `pytest tests/test_alerts.py -k "get" --tb=short`
+4. No import errors
+
+## Files to Modify
+- `src/backend/routers/alerts.py`
+
+## Files to Read First
+- `src/backend/services/alert_service.py` — the service created in the previous task
+- `src/backend/routers/alerts.py` — current GET endpoint implementations

tasks/alert-service-migrate-write/SPEC.md

@@ -0,0 +1,46 @@
+# Task: Migrate POST/PUT/DELETE endpoints to AlertService
+
+## Goal
+Replace `routers/alerts.py` write endpoints (`create_alert`, `update_alert`, `resolve_alert`, `dismiss_alert`) to use `AlertService`.
+
+## Requirements
+
+### Changes to `src/backend/routers/alerts.py`
+
+1. Replace `create_alert()` body:
+   - Call `service.create_alert(watchlist_id, type, trigger_type, message, severity, ticker, metadata, user_id)`
+   - Return `AlertResponse` wrapping the result
+   - Keep the watchlist owner check
+
+2. Replace `update_alert()` body:
+   - Use `service.get_alert()` to verify existence and ownership
+   - Build updates dict from `body` fields
+   - Call `service.update_alert(alert_id, updates_dict, user_id)`
+   - Return `AlertResponse` wrapping the result
+
+3. Replace `resolve_alert()` body:
+   - Use `service.get_alert()` to verify existence and ownership
+   - Call `service.resolve_alert(alert_id, user_id)`
+   - Return `AlertResponse` wrapping the result
+
+4. Replace `dismiss_alert()` body:
+   - Use `service.get_alert()` to verify existence and ownership
+   - Call `service.dismiss_alert(alert_id, user_id)`
+   - Return `None` (204 response)
+
+### Constraints
+- Keep all route signatures, decorators, and response models unchanged
+- Keep `check_all_alerts` endpoint unchanged (it calls `check_sentiment_alerts` directly)
+- File must stay under 200 lines total
+
+## Acceptance Criteria
+1. All write endpoints use `AlertService` methods
+2. All alert CRUD tests pass: `pytest tests/test_alerts.py -k "test_create or test_update or test_delete or test_resolve or test_dismiss" --tb=short`
+3. No import errors
+
+## Files to Modify
+- `src/backend/routers/alerts.py`
+
+## Files to Read First
+- `src/backend/services/alert_service.py`
+- `src/backend/routers/alerts.py` — current write endpoint implementations

tasks/alert-service-verify/SPEC.md

@@ -0,0 +1,32 @@
+# Task: Alert refactor verification
+
+## Goal
+Run the full test suite and verify the alert refactor is complete and correct.
+
+## Requirements
+
+1. Run `pytest tests/test_alerts.py --tb=short -v` — all tests must pass
+2. Run `pytest tests/ --tb=short -v` — no regressions in other test files
+3. Verify `routers/alerts.py` is clean:
+   - No direct DB calls
+   - Uses `AlertService` for all data access
+   - Under 100 lines
+4. Verify `services/alert_service.py` is clean:
+   - All methods are async
+   - Uses parameterized queries
+   - Under 200 lines
+5. Verify the `check_all_alerts` endpoint still works (calls `check_sentiment_alerts` via background task)
+
+## Acceptance Criteria
+1. All alert tests pass (23 tests)
+2. No regressions in other test files
+3. Router file is clean and under 100 lines
+4. Service file is clean and under 200 lines
+
+## Files to Read
+- `src/backend/routers/alerts.py`
+- `src/backend/services/alert_service.py`
+- `src/backend/tests/test_alerts.py`
+
+## Notes
+This is a verification-only task. No new code should be written unless a test fails, in which case fix the root cause and re-run.

tasks/complete-rotation-macro/SPEC.md

@@ -0,0 +1,48 @@
+# Task: Rotation macro data endpoint
+
+## Goal
+Add the macro data endpoint that provides the full sector rotation overview with all required fields for the frontend.
+
+## Requirements
+
+### Endpoint: `GET /api/v1/rotation/macro-full`
+
+Returns complete sector rotation data:
+- All sectors with their current ranking
+- Momentum scores (20d, 50d)
+- Relative strength vs benchmark
+- Rotation signals (bullish/bearish/neutral)
+- Rank changes from previous period
+
+### Response model `FullRotationResponse` in `schemas/rotation.py`
+
+```python
+class FullRotationResponse(BaseModel):
+    sectors: list[SectorRotationDetail]
+    benchmark: str
+    as_of: str  # ISO timestamp
+    period: str  # e.g., "20d"
+```
+
+### Service method
+
+Add `get_full_rotation(period="20d")` to `rotation_service.py`:
+- Queries all sector data
+- Joins with benchmark data
+- Returns formatted response
+
+## Acceptance Criteria
+1. `/api/v1/rotation/macro-full` returns complete rotation data
+2. All fields properly typed and validated
+3. Tests pass for new endpoint
+4. File stays under 200 lines
+
+## Files to Modify
+- `src/backend/services/rotation_service.py`
+- `src/backend/routers/rotation.py`
+- `src/backend/schemas/rotation.py`
+
+## Files to Read First
+- `src/backend/services/rotation_service.py` — current state
+- `src/backend/routers/rotation.py` — existing endpoints
+- `src/backend/schemas/rotation.py` — existing schemas

tasks/complete-rotation-ranking/SPEC.md

@@ -0,0 +1,45 @@
+# Task: Complete rotation ranking service
+
+## Goal
+Implement the ranking logic for sector rotation in `services/rotation_service.py`. The service currently has skeleton methods for ranking but needs the actual implementation.
+
+## Requirements
+
+### Implement ranking methods in `services/rotation_service.py`
+
+1. `rank_sectors(sector_data, timeframe="20d")` method:
+   - Accept a list of sector data dicts (from the database)
+   - Sort sectors by momentum score (descending)
+   - Assign rank_now to each sector
+   - Calculate rank_change compared to previous ranking
+   - Return ranked sector list
+
+2. `get_ranking_history(ticker, days=30)` method:
+   - Query historical rankings from the database (TimescaleDB hypertable)
+   - Return time-series of rank data for charting
+   - Use `execute_query` with parameterized queries
+
+3. `calculate_relative_strength(ticker, benchmark="SPY")` method:
+   - Calculate price ratio between sector ETF and benchmark
+   - Compute relative strength as a percentage change
+   - Return strength metric
+
+### Database interactions
+- Read from `sector_rankings` hypertable (TimescaleDB)
+- Use parameterized queries
+- Handle missing data gracefully (return empty lists)
+
+## Acceptance Criteria
+1. `rank_sectors()` returns properly ranked sector list
+2. `get_ranking_history()` returns time-series data
+3. `calculate_relative_strength()` returns strength metric
+4. All methods are async
+5. File stays under 200 lines
+
+## Files to Modify
+- `src/backend/services/rotation_service.py`
+
+## Files to Read First
+- `src/backend/services/rotation_service.py` — current state
+- `src/backend/database.py` — query patterns
+- `src/backend/tests/test_sector_rotation.py` — test expectations

tasks/complete-rotation-signals/SPEC.md

@@ -0,0 +1,46 @@
+# Task: Complete rotation signals and macro view
+
+## Goal
+Implement the rotation signal generation and macro view endpoints for the sector rotation service.
+
+## Requirements
+
+### 1. Signal generation in `services/rotation_service.py`
+
+Implement `generate_signals(ranked_sectors)` method:
+- For each sector, determine rotation signal based on:
+  - `rank_change > 0` → `"bullish"` (rising)
+  - `rank_change < 0` → `"bearish"` (falling)  
+  - `rank_change == 0` → `"neutral"` (unchanged)
+- Return list of signal dicts with sector info and signal type
+- Filter out sectors with insufficient data
+
+### 2. Macro view in `routers/rotation.py`
+
+Add `GET /api/v1/rotation/macro` endpoint:
+- Returns top 3 rising sectors and bottom 3 falling sectors
+- Includes sector name, ticker, momentum, and signal
+- Response model: `MacroRotationResponse` with `rising` and `falling` lists
+
+### 3. Add response model
+
+Add `MacroRotationResponse` schema in `schemas/rotation.py`:
+- `rising: list[RisingSector]`
+- `falling: list[FallingSector]`
+- Each sector: `sector_name`, `sector_ticker`, `momentum`, `signal`
+
+## Acceptance Criteria
+1. `generate_signals()` correctly classifies sectors
+2. `/api/v1/rotation/macro` endpoint returns correct data
+3. Response model is properly typed
+4. Tests pass for new endpoints
+
+## Files to Modify
+- `src/backend/services/rotation_service.py`
+- `src/backend/routers/rotation.py`
+- `src/backend/schemas/rotation.py`
+
+## Files to Read First
+- `src/backend/services/rotation_service.py` — current state
+- `src/backend/routers/rotation.py` — existing rotation endpoints
+- `src/backend/schemas/rotation.py` — existing schemas

tasks/fix-alert-test-and-complete-sector-rotation/SPEC.md

@@ -0,0 +1,135 @@
+# Task: Fix Alert Test & Complete Sector Rotation Service
+
+## Current State
+
+The invest-copilot project is at Phase 1-6 complete, with Phase 2 (Data Pipeline) and Phase 5 (Sector Rotation) marked as "In Progress".
+
+### Critical Issues Found
+
+1. **Failing Test**: `tests/test_alerts.py::TestAlertCRUD::test_get_all_alerts`
+   - Creates 2 alerts successfully (201)
+   - GET request returns `total: 0` instead of `total: 2`
+   - Root cause: watchlist fixture not properly linked to test user's watchlists
+
+2. **Incomplete Sector Rotation Service** (`services/rotation_service.py`)
+   - Hardcoded ETF list (XLK, XLF, XLE, etc.)
+   - No proper ranking algorithm (rank_now, rank_previous, rank_change missing)
+   - Missing rotation_signal field (bullish/bearish/neutral)
+   - No macro context or analysis summary
+   - Uses random data in some calculations
+   - Only stores 30 days of prices instead of full history
+
+### Project Structure
+
+- **Backend**: FastAPI, asyncpg, Pydantic v2, Celery + Redis
+- **Database**: PostgreSQL + TimescaleDB (hypertables for time-series)
+- **Frontend**: Next.js 16, React 19, TypeScript, Zustand, TanStack Query
+- **Testing**: pytest with 192 tests (1 currently failing)
+
+## Recommended Next Task
+
+### Phase 1: Fix Alert Test (Quick Win)
+
+**Goal**: Make all 192 tests pass
+
+**Steps**:
+1. Investigate `conftest.py` watchlist fixture
+2. Ensure watchlist is created with proper `user_id` ownership
+3. Verify alert creation links to user's watchlist
+4. Run full test suite to confirm all pass
+
+**Acceptance Criteria**:
+- `pytest tests/` returns 0 failures
+- All 192 tests pass
+
+### Phase 2: Complete Sector Rotation Service
+
+**Goal**: Implement a production-ready sector rotation detection system
+
+**Current Gaps**:
+- No ranking algorithm (needs rank_now, rank_previous, rank_change)
+- Missing rotation_signal (bullish/bearish/neutral based on momentum)
+- No macro context (S&P 500 benchmark comparison)
+- No analysis summary generation
+- Hardcoded ETF list (should be configurable)
+- Uses only 30 days of data (should use full history for ranking)
+
+**Implementation Plan**:
+
+1. **Add missing fields to rotation_service.py**:
+   - `rank_now`: Current rank among all sectors
+   - `rank_previous`: Previous period rank
+   - `rank_change`: Difference (positive = improving)
+   - `rotation_signal`: bullish/bearish/neutral based on momentum thresholds
+   - `macro_context`: S&P 500 vs sector comparison
+   - `analysis_summary`: Human-readable summary
+
+2. **Implement ranking algorithm**:
+   - Sort sectors by relative_strength
+   - Assign ranks
+   - Compare with previous day's ranks (from database)
+   - Calculate rank_change
+
+3. **Add rotation signal logic**:
+   - bullish: momentum_20d > 0.05 AND rank_change > 0
+   - bearish: momentum_20d < -0.05 AND rank_change < 0
+   - neutral: everything else
+
+4. **Add macro context**:
+   - Fetch SPY prices
+   - Compare sector momentum vs SPY momentum
+   - Store in macro_context field
+
+5. **Make ETF list configurable**:
+   - Read from database or config
+   - Allow dynamic sector ETF mapping
+
+6. **Add tests**:
+   - Test ranking algorithm
+   - Test rotation signal logic
+   - Test macro context calculation
+   - Test edge cases (missing data, single sector)
+
+**Acceptance Criteria**:
+- All rotation fields populated correctly
+- Ranking algorithm produces consistent results
+- Rotation signals match expected thresholds
+- Tests cover all new logic
+- Service can be called via Celery task for scheduled runs
+
+## Why This Task First?
+
+1. **Fixes a critical blocker**: The failing test indicates a data ownership bug that could affect other features
+2. **Completes a core feature**: Sector rotation is a key differentiator for this investment tool
+3. **Both are high-value**: The test fix is quick (1-2 hours), the rotation service is substantial but well-scoped
+4. **Aligns with project phases**: Directly addresses the "In Progress" items in the README
+
+## Files to Modify
+
+- `src/backend/tests/conftest.py` - Fix watchlist fixture
+- `src/backend/services/rotation_service.py` - Complete rotation logic
+- `src/backend/routers/sectors.py` - Update if needed
+- `src/backend/tasks/rotation_tasks.py` - Add Celery task for scheduled runs
+- `src/backend/tests/test_rotation.py` - Add new tests
+
+## Estimated Effort
+
+- Fix alert test: 1-2 hours
+- Complete sector rotation: 4-6 hours
+- Total: 5-8 hours
+
+## Risks & Mitigations
+
+- **Risk**: Watchlist fixture issue may affect other tests
+  - **Mitigation**: Run full test suite after fix, check all watchlist-related tests
+- **Risk**: Sector rotation logic may be complex
+  - **Mitigation**: Start with simple ranking, add macro context in second pass
+- **Risk**: Database schema may need updates for new fields
+  - **Mitigation**: Check init.sql, add migration if needed
+
+## Next Steps After This Task
+
+1. Complete data pipeline (Phase 2)
+2. Implement PWA features (Phase 7)
+3. Add integration tests
+4. Performance optimization

tasks/fix-alert-test-get-all/SPEC.md

@@ -0,0 +1,28 @@
+# Task: Fix test_get_all_alerts
+
+## Goal
+Fix the single failing test in `test_alerts.py::TestAlertCRUD::test_get_all_alerts`. The test creates 2 alerts but GET returns `total: 0`.
+
+## Root Cause
+The test uses `mock_db.execute_query.side_effect` with SQL string matching. The router's query for alerts uses parameterized queries with dynamic placeholders (`$1`, `$2`, etc.) that don't match the mock's simple string patterns. Specifically, the count query and the data query use different SQL patterns than what the mock expects.
+
+## Requirements
+1. Read `test_alerts.py` lines 30-80 to understand the mock side_effect logic
+2. Read `routers/alerts.py` lines 30-80 to understand the actual SQL queries being made
+3. Fix the mock's `alerts_query_side_effect` function to correctly match all SQL patterns the router uses:
+   - The COUNT query with `watchlist_id IN (...)` pattern
+   - The data query with `watchlist_id IN (...)` pattern
+   - Ensure the mock returns correct data for both queries
+4. Ensure the watchlist fixture's `user_id: "test-user-1"` matches the test user's ID
+
+## Acceptance Criteria
+- `pytest tests/test_alerts.py::TestAlertCRUD::test_get_all_alerts` passes
+- `pytest tests/test_alerts.py` passes all 23 tests (0 failures)
+
+## Files to Modify
+- `src/backend/tests/test_alerts.py` — Fix the mock side_effect function
+
+## Constraints
+- Do not change the router code
+- Do not change the conftest.py fixtures
+- Keep the test's existing structure and mock approach

tasks/fix-test-watchlist-fixture/SPEC.md

@@ -0,0 +1,35 @@
+# Task: Fix test watchlist fixture for sector rotation tests
+
+## Goal
+Fix the `watchlist` fixture in `tests/conftest.py` so that sector rotation tests can properly create and use watchlists with user ownership.
+
+## Root Cause
+The `watchlist` fixture in `conftest.py` sets `user_id: "test-user-1"` but the `test_user` fixture generates a UUID for `user_id`. When the rotation service calls `_require_watchlist_owner`, the user IDs don't match, causing 404 errors.
+
+## Requirements
+
+### Fix in `src/backend/tests/conftest.py`
+
+1. In the `watchlist` fixture, use the same user ID as `test_user`:
+   - Extract the user ID from `test_user` fixture's UUID
+   - Or use a fixed user ID like `"test-user-1"` in both fixtures
+2. Ensure the fixture returns a watchlist dict with:
+   - `id`: a consistent UUID (not random per test)
+   - `user_id`: matching the test user's ID
+   - `name`, `description`, `is_default`: reasonable defaults
+
+### Alternative: Fix in rotation service tests
+If the fixture approach is too invasive, fix the rotation service tests to use the correct user ID when calling endpoints.
+
+## Acceptance Criteria
+1. The `watchlist` fixture's `user_id` matches the `test_user` fixture's user ID
+2. Sector rotation tests that create watchlists pass
+3. No regressions in existing alert tests
+
+## Files to Modify
+- `src/backend/tests/conftest.py`
+
+## Files to Read First
+- `src/backend/tests/conftest.py` — the `watchlist` and `test_user` fixtures
+- `src/backend/tests/test_sector_rotation.py` — tests that depend on the fixture
+- `src/backend/services/rotation_service.py` — how ownership is checked

tasks/phase2-backtest-integration/SPEC.md

@@ -0,0 +1,47 @@
+# Task: Wire Backtest Integration
+
+## Current State
+
+`src/backend/services/backtest_service.py` (16,798 bytes) exists with backtesting logic. `POST /api/v1/strategies/{id}/backtest` endpoint exists per README. However, the backtest engine is not connected to the data pipeline — it runs against whatever data happens to be in the DB, with no guarantee of freshness or completeness.
+
+## Goal
+
+Connect the backtest engine to the data pipeline so:
+1. Backtests always run against a known-good, pipeline-fresh dataset
+2. Backtest results are stored and queryable
+3. Users can trigger backtests via the API and see results
+
+## Requirements
+
+1. Verify `backtest_service.py` reads from database correctly
+2. Add backtest result storage model (SQLAlchemy)
+3. Store backtest results: trades, metrics, equity curve
+4. Add `GET /api/v1/backtests/{id}` endpoint to retrieve results
+5. Add `GET /api/v1/backtests` endpoint to list past backtests
+6. Ensure backtest runs after pipeline completes (dependency on pipeline task)
+
+## Acceptance Criteria
+
+- Backtest endpoint returns results with trades, metrics, equity curve
+- Results persist in database and survive server restart
+- Past backtests queryable via list endpoint
+- Backtest uses data from the most recent pipeline run
+- Frontend can display backtest results on strategy page
+
+## Constraints
+
+- Use TimescaleDB hypertables for equity curve time-series data
+- Follow existing patterns — check `models/` for existing backtest model
+- Keep under 200 lines per framework rule
+- Read existing migrations before writing schema
+
+## Files to Create/Modify
+
+- `src/backend/models/` (check for existing backtest model, add if missing)
+- `src/backend/services/backtest_service.py` (add result storage)
+- `src/backend/routers/strategies.py` (add result endpoints)
+- `src/backend/tasks/pipeline.py` (add backtest trigger option)
+
+## Next Steps After This Task
+
+Phase 2 complete. Move to Phase 5 (Sector Rotation) or Phase 7 (PWA).

tasks/phase2-pipeline-orchestrator/SPEC.md

@@ -0,0 +1,47 @@
+# Task: Wire Pipeline Orchestrator
+
+## Current State
+
+`src/data-pipeline/pipeline.py` exists but is not connected to Celery Beat or the backend. The four ingestion scripts (prices, sec, financials, news) run standalone — no dependency ordering, no error handling between stages, no visibility into pipeline health.
+
+## Goal
+
+Create a proper pipeline orchestrator that:
+1. Chains ingestion tasks in correct dependency order
+2. Provides health/status endpoints
+3. Integrates with existing Celery infrastructure
+
+## Requirements
+
+1. Define dependency graph: prices → sec → financials → news (prices must run first)
+2. Create `src/backend/tasks/pipeline.py` with orchestration logic
+3. Add `/api/v1/sync/status` endpoint (already exists per README — verify it works)
+4. Create Celery chain/group for full pipeline run
+5. Add manual trigger endpoint: `POST /api/v1/sync/pipeline/run`
+6. Add to docker-compose worker service
+
+## Acceptance Criteria
+
+- Full pipeline runs end-to-end via single API call
+- Dependency ordering enforced (prices before sec, etc.)
+- Pipeline status visible via `/api/v1/sync/status`
+- Failed stage does not block other independent stages
+- Pipeline health check returns last run time, status, errors
+
+## Constraints
+
+- Use Celery chains for ordered stages, Celery groups for parallel stages
+- Follow existing task patterns — don't reinvent error handling
+- Keep under 200 lines per framework rule
+- Read existing migrations before writing schema
+
+## Files to Create/Modify
+
+- `src/backend/tasks/pipeline.py` (new)
+- `src/backend/routers/data_sync.py` (add pipeline trigger endpoint)
+- `src/backend/celery_app.py` (add pipeline schedule)
+- `docker-compose.dev.yml` (verify worker includes task)
+
+## Next Steps After This Task
+
+Wire backtest integration (phase2-backtest-integration)

tasks/phase2-wire-financials-ingestion/SPEC.md

@@ -0,0 +1,52 @@
+# Task: Wire Financials Ingestion as Celery Task
+
+## Status: COMPLETE
+
+## Current State
+
+`src/data-pipeline/ingest_financials.py` exists as a standalone script. It uses psycopg2 (sync) and runs as a CLI script.
+
+## Goal
+
+Make financial data ingestion runnable as a scheduled Celery task, integrated into the pipeline orchestrator.
+
+## Requirements
+
+1. Create `src/backend/tasks/ingest_financials.py` Celery task
+2. Reuse logic from `src/data-pipeline/ingest_financials.py` (don't duplicate)
+3. Task must be idempotent — deduplicate by (ticker, report_type, period)
+4. Task must store results into PostgreSQL/TimescaleDB using existing models
+5. Add to Celery Beat schedule in `celery_app.py`
+6. Add to docker-compose worker service
+
+## Acceptance Criteria
+
+- [x] `celery -A celery_app worker` processes financial ingestion tasks
+- [x] Running the task twice with same input produces no duplicate records (ON CONFLICT)
+- [x] Financial data visible in database after task completes
+- [x] Celery Beat runs it on a configurable schedule (default: daily at 2 AM UTC)
+
+## Implementation Details
+
+### Files Created
+- `src/backend/tasks/ingest_financials.py` — Celery task using yfinance + asyncpg
+  - Fetches quarterly + annual income statement, balance sheet, cash flow
+  - Extracts 20 metrics (revenue, net_income, eps, roe, roa, etc.)
+  - Idempotent upsert via `ON CONFLICT (ticker, filing_date, period)`
+  - 3 retries with exponential backoff for both yfinance fetches and DB writes
+
+### Files Modified
+- `src/backend/tasks/__init__.py` — Added `ingest_financials_task` export
+- `src/backend/celery_app.py` — Added `ingest-financials-daily` schedule (2 AM UTC)
+- `init.sql` — Added `financials` table + indexes (was missing from Docker init)
+- `src/backend/database.py` — Added `financials` table to `init_db()` (was missing)
+
+### Key Design Decisions
+- Used yfinance (already in backend requirements.txt) instead of duplicating the data-pipeline script
+- Converted from psycopg2 to asyncpg via `_execute_command()` wrapper
+- Extracted metrics via column-name matching (yfinance column names vary by ticker)
+- Derived metrics: debt_to_equity, roe, roa computed from base values
+
+## Next Steps After This Task
+
+Wire news ingestion (phase2-wire-news-ingestion)

tasks/phase2-wire-news-ingestion/SPEC.md

@@ -0,0 +1,44 @@
+# Task: Wire News Ingestion as Celery Task
+
+## Current State
+
+`src/data-pipeline/ingest_news.py` exists as a standalone script but is NOT wired as a Celery task. Only `ingest_prices` and `ingest_sec` exist in `src/backend/tasks/`.
+
+## Goal
+
+Make news/sentiment data ingestion runnable as a scheduled Celery task, integrated into the pipeline orchestrator.
+
+## Requirements
+
+1. Create `src/backend/tasks/ingest_news.py` Celery task
+2. Reuse logic from `src/data-pipeline/ingest_news.py` (don't duplicate)
+3. Task must be idempotent — deduplicate by (ticker, source, published_date)
+4. Task must store news articles and sentiment signals into PostgreSQL
+5. Add to Celery Beat schedule in `celery_app.py`
+6. Add to docker-compose worker service
+
+## Acceptance Criteria
+
+- `celery -A celery_app worker` processes news ingestion tasks
+- Running the task twice with same input produces no duplicate records
+- News articles and sentiment data visible in database after task completes
+- Celery Beat runs it on a configurable schedule (default: daily)
+- Frontend can query news via existing `/api/v1/stocks/{ticker}/sentiment` endpoint
+
+## Constraints
+
+- Use TimescaleDB hypertables for time-series news data
+- Follow existing task patterns from `ingest_prices.py` and `ingest_sec.py`
+- Keep under 200 lines per framework rule
+- Read existing migrations before writing schema
+
+## Files to Create/Modify
+
+- `src/backend/tasks/ingest_news.py` (new)
+- `src/backend/celery_app.py` (add schedule entry)
+- `docker-compose.dev.yml` (verify worker includes task)
+- `src/data-pipeline/ingest_news.py` (verify it's importable, not just executable)
+
+## Next Steps After This Task
+
+Wire pipeline orchestrator (phase2-pipeline-orchestrator)

tasks/pipeline-orchestrator-api/SPEC.md

@@ -0,0 +1,58 @@
+# Task: Pipeline orchestrator API endpoints
+
+## Goal
+Create the API endpoints for triggering and monitoring pipeline runs. This is Phase 2 — the API layer on top of the task definitions from the previous task.
+
+## Requirements
+
+### Create `src/backend/services/pipeline_orchestrator.py`
+
+Implement `PipelineOrchestrator` class:
+
+1. `run_pipeline(task_name: str, user_id: str) -> str` — start a pipeline run, return run_id
+2. `get_run_status(run_id: str) -> dict` — return run status (pending/running/completed/failed)
+3. `cancel_run(run_id: str)` — cancel a running pipeline
+4. Internal: execute tasks respecting dependency order
+5. Store run state in memory (dict) — no DB needed yet
+
+### Create `src/backend/routers/pipeline.py`
+
+Add endpoints:
+
+1. `POST /api/v1/pipeline/run` — trigger a pipeline run
+   - Body: `{"task": "news_ingestion"}` or `"full"` for all tasks
+   - Returns: `{"run_id": "...", "status": "pending"}`
+
+2. `GET /api/v1/pipeline/run/{run_id}` — get run status
+   - Returns: `{"run_id": "...", "status": "...", "tasks": [...]}`
+
+3. `POST /api/v1/pipeline/run/{run_id}/cancel` — cancel a run
+   - Returns: `{"message": "cancelled"}`
+
+4. `GET /api/v1/pipeline/tasks` — list available tasks
+   - Returns: list of registered tasks with descriptions
+
+### Response models in `schemas/pipeline.py`
+
+Create:
+- `PipelineRunRequest` — task name to run
+- `PipelineRunResponse` — run_id and status
+- `PipelineRunStatus` — detailed run status with task results
+- `PipelineTaskInfo` — task metadata
+
+## Acceptance Criteria
+1. All 4 endpoints work correctly
+2. Dependency order is respected when running full pipeline
+3. Run status updates in real-time
+4. Files stay under 200 lines each
+
+## Files to Create/Modify
+- `src/backend/services/pipeline_orchestrator.py`
+- `src/backend/services/pipeline_tasks.py` (from previous task)
+- `src/backend/routers/pipeline.py`
+- `src/backend/schemas/pipeline.py`
+
+## Files to Read First
+- `src/backend/services/pipeline_tasks.py` — task definitions
+- `src/backend/routers/alerts.py` — follow routing pattern
+- `src/backend/schemas/alert.py` — follow schema pattern

tasks/pipeline-orchestrator-tasks/SPEC.md

@@ -0,0 +1,51 @@
+# Task: Pipeline task definitions
+
+## Goal
+Create the task definition classes and registry for the pipeline orchestrator. This is Phase 1 — define what tasks exist and their metadata.
+
+## Requirements
+
+### Create `src/backend/services/pipeline_tasks.py`
+
+Implement:
+
+1. `PipelineTask` dataclass:
+   - `name: str` — unique task identifier
+   - `description: str` — human-readable name
+   - `func: Callable` — async function to execute
+   - `depends_on: list[str]` — task names that must complete first
+   - `timeout: int` — max execution time in seconds
+   - `retry_count: int` — number of retries on failure
+
+2. `TaskRegistry` class:
+   - `register(task: PipelineTask)` — add task to registry
+   - `get(task_name: str) -> PipelineTask` — lookup task
+   - `get_all() -> list[PipelineTask]` — list all registered tasks
+   - `get_dependencies(task_name: str) -> list[str]` — get task dependencies
+   - `get_ready_tasks( completed: set[str]) -> list[str]` — find tasks whose deps are met
+
+3. Register built-in tasks:
+   - `news_ingestion` — runs news data ingestion
+   - `financials_ingestion` — runs financials data ingestion
+   - `sector_rotation` — runs sector rotation analysis
+   - `price_update` — runs price data update
+   - `sentiment_analysis` — runs sentiment analysis
+
+### Constraints
+- File under 200 lines
+- Use existing async patterns
+- No real DB calls in task definitions
+
+## Acceptance Criteria
+1. `TaskRegistry` correctly tracks tasks and dependencies
+2. `get_ready_tasks()` returns correct task order
+3. All 5 built-in tasks are registered
+4. File is under 200 lines
+
+## Files to Create
+- `src/backend/services/pipeline_tasks.py`
+
+## Files to Read First
+- `src/backend/services/news_ingestion_service.py` — existing task functions
+- `src/backend/services/financials_ingestion_service.py` — existing task functions
+- `src/backend/services/rotation_service.py` — existing task functions

tasks/refactor-alert-crud-to-service/SPEC.md

@@ -0,0 +1,43 @@
+# SPEC: Refactor Alert CRUD to Service Layer
+
+## Goal
+Refactor the `src/backend/routers/alerts.py` file to move business logic and data access into a dedicated `src/backend/services/alert_service.py` file. This aligns the alerting module with the existing service-oriented architecture used in the rest of the project (e.g., `SentimentService`, `RotationService`).
+
+## Exact Requirements
+1. **Create `src/backend/services/alert_service.py`**:
+    - Implement an `AlertService` class containing methods for all current alert operations.
+    - Methods required:
+        - `get_user_alerts(watchlist_id, status, page, page_size, user_id)`
+        - `create_alert(body, user_id)`
+        - `get_alert(alert_id, user_id)`
+        - `update_alert(alert_id, body, user_id)`
+        - `resolve_alert(alert_id, user_id)`
+        - `dismiss_alert(alert_id, user_id)`
+2. **Encapsulate Authorization**:
+    - Move the ownership verification logic (`_require_watchlist_owner`) into the service layer or a shared security service.
+3. **Encapsulate Data Access & Transformation**:
+    - Move all `execute_query`, `execute_one`, and `execute_command` calls into the `AlertService`.
+    - Handle the mapping of database rows to `AlertResponse` and `AlertListResponse` models within the service.
+4. **Update `src/backend/routers/alerts.py`**:
+    - Remove direct database calls and business logic.
+    - Inject/instantiate `AlertService` and delegate all requests to it.
+    - Maintain the existing FastAPI route definitions and dependency injection (e.g., `get_current_user`).
+5. **Maintain Feature Parity**:
+    - The API behavior (endpoints, status codes, response models) must remain identical to the current implementation.
+
+## Acceptance Criteria
+1. **Code Structure**: `src/backend/routers/alerts.py` contains only routing and request/response handling.
+2. **Service Implementation**: `src/backend/services/alert_service.py` is the single source of truth for alert business logic.
+3. **Test Pass Rate**: All tests in `src/backend/tests/test_alerts.py` must pass (including the previously failing tests).
+4. **No Regressions**: All CRUD operations (Create, Read, Update, Delete, Resolve, Dismiss) must function exactly as before.
+
+## Constraints & Non-Goals
+- **Non-Goal**: Do not change the database schema.
+- **Non-Goal**: Do not change the external API contract (URLs, JSON structure, status codes).
+- **Constraint**: Maintain existing error handling patterns (e.g., 404 for missing resources, 403/404 for ownership issues).
+
+## Recommended Implementation Approach
+1. **Phase 1: Service Creation**: Implement the `AlertService` in `src/backend/services/alert_service.py` by copying logic from the router, but parameterizing it for the service methods.
+2. **Phase 2: Router Migration**: Replace the body of each router function with a call to the corresponding `AlertService` method.
+3. **Phase 3: Cleanup**: Remove the old `_require_watchlist_owner` helper from the router.
+4. **Phase 4: Verification**: Run the existing test suite.