Implement walk-forward cross-validation orchestrator for time-series model training

[Copilot]

Implements Phase 3 WFCV orchestrator to enable robust out-of-sample validation through fold-wise temporal splitting. Training automatically uses WFCV when walkforward.enabled=true in config, with backward compatibility when disabled.

Changes

BaseTrainer Interface

• Extended train() signature with optional feature_names, instrument, fold_id parameters
• Enables fold-specific metadata propagation for logging and debugging

WalkForwardOrchestrator

• New orchestrator class manages fold-wise training pipeline (600 LOC)
• Supports rolling (fixed window) and expanding (growing window) modes
• Implements best/average/ensemble aggregation strategies
• Generates temporal splits via existing WalkForwardValidator.split()
• Exports wfcv_summary.json with per-fold metrics and aggregated statistics

CLI Integration

• Modified _train_direction_or_regime_model() to conditionally route to orchestrator
• Reads cfg.walkforward and switches between WFCV/standard training
• Passes fold metadata through training stack

Training Reports

• Enhanced with WFCV section showing mean±std test accuracy, stability coefficient (CV), and best fold selection
• Distinguishes single-fold vs multi-fold validation in output

Usage

# config/config_improved_H1.yaml
walkforward:
  enabled: true
  mode: rolling          # or expanding
  n_splits: 5
  aggregate_method: best # selects highest val accuracy fold

# Training automatically uses WFCV
./bin/Buddy train -i EUR_USD

Output includes per-fold progress and aggregated metrics:

Fold 1/5 | Train: 3,000 | Val: 500 | Test: 500
  ✓ Fold 1: Val Acc=62.3%, Test Acc=61.8%

wfcv_mean_test_accuracy: 62.1% ± 2.4%
wfcv_best_fold: 3

Test Coverage

• 13 test cases covering rolling/expanding modes, aggregation strategies, edge cases
• Integration tests validate YAML config loading and realistic datasets

Original prompt

Start implementation

The user has attached the following file paths as relevant context:

• .github/copilot-instructions.md

[Chronological Review: The conversation began with the user outlining a request to implement Phase 3 of a project involving Walk-Forward Cross-Validation (WFCV). The user specified several tasks, including extending the BaseTrainer interface, creating a WalkForwardOrchestrator class, wiring the orchestrator into the CLI training, and updating the training report metrics. The user provided detailed instructions for each task, indicating a clear progression from planning to implementation.]

[Intent Mapping: The user explicitly requested the following:

1. "Add optional kwargs to BaseTrainer.train() in base.py: add feature_names: Optional[List[str]] = None, instrument: str = '', fold_id: Optional[int] = None to the abstract signature."
2. "Create WalkForwardOrchestrator class in src/training/walkforward_orchestrator.py."
3. "Wire orchestrator into CLI training in training.py."
4. "Make WFCV the reported metric in the training report."]

[Technical Inventory: The conversation involved several technical concepts and components:

• BaseTrainer interface in base.py
• WalkForwardOrchestrator class in src/training/walkforward_orchestrator.py
• WalkForwardValidator.split() and purged_kfold_split() methods in walkforward_validation.py
• CLI training modifications in training.py
• Training report metrics in training_report.md]

[Code Archaeology: The user discussed modifications to:

• base.py: extending the BaseTrainer interface
• src/training/walkforward_orchestrator.py: creating a new class
• training.py: integrating the orchestrator into the training process
• training_report.md: updating the metrics reported]

[Progress Assessment: The user has outlined the tasks to be completed but has not indicated any tasks as completed yet. All tasks are pending implementation.]

[Context Validation: All critical information for continuation is captured, including specific file modifications, class creations, and metric updates.]

[Recent Commands Analysis: The most recent commands involved the user detailing the tasks for Phase 3 of the project. No specific tool commands were executed, but the user provided a comprehensive outline of the tasks to be completed. The agent was actively working on summarizing the conversation when the token budget was exceeded.]

1. Conversation Overview: - Primary Objectives: The user requested to implement Phase 3 of the project, specifically focusing on Walk-Forward Cross-Validation (WFCV) with detailed tasks such as extending the BaseTrainer interface, creating a WalkForwardOrchestrator class, wiring it into the CLI training, and updating the training report metrics. - Session Context: The conversation has been focused on outlining the necessary steps for implementing WFCV, with the user providing explicit instructions for each component. - User Intent Evolution: The user's needs have remained consistent, focusing on the implementation of WFCV and its integration into the existing framework.

2. Technical Foundation:

• BaseTrainer: Interface in base.py that requires extension to support fold-aware training.
• WalkForwardOrchestrator: New class to be created in src/training/walkforward_orchestrator.py for managing the training pipeline.
• WalkForwardValidator: Methods used for generating fold indices and applying purging/embargo.
• CLI Training: Modifications in training.py to integrate the new orchestrator.

3. Codebase Status:

• base.py:
  • Purpose: To define the BaseTrainer interface.
  • Current State: Needs to be modified to include optional kwargs for fold-aware training.
• src/training/walkforward_orchestrator.py:
  • Purpose: To orchestrate the training process for WFCV.
  • Current State: New file to be created.
• training.py:
  • Purpose: To manage the training process.
  • Current State: Needs to be modified to route to WalkForwardOrchestrator.
• training_report.md:
  • Purpose: To report training metrics.
  • Current State: Needs to be updated to reflect WFCV metrics.

4. Problem Resolution:

• Issues Encountered: None reported yet as tasks are still pending.
• Solutions Implemented: Not applicable as no tasks have been completed.
• Debugging Context: No ongoing troubleshooting efforts reported.
• Lessons Learned: Not applicable at this stage.

5. Progress Tracking:

• Completed Tasks: None yet.
• Partially Complete Work: All tasks are outlined and pending implementation.
• Validated Outcomes: None yet.

6. Active Work State:

• Current Focus: The user is focused on implementing Phase 3 of the project, specifically WFCV.
• Recent Context: The user provided detailed instructions for extending the BaseTrainer interface, creating the WalkForwardOrchestrator, wiring it into the CLI, and updating the training report.
• Working Code: No specific code snippets were discussed recently.
• Immediate Context: The user was outlining task...

Created from VS Code.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.