Eliminate configuration drift: unify direction/inference defaults to single source of truth

[Copilot]

Configuration drift caused 5 locations to have diverging fallback defaults that didn't match config/config_improved_H1.yaml. Training used 0.005/12, data loaders used 0.001/6 and 0.005/6, backtest used 0.0015/24, and InferenceConfig defaulted to 0.55 instead of 0.60.

Changes

• Created src/core/constants.py - Single source of truth for defaults

  • DIRECTION_DEFAULTS: threshold=0.003, lookahead=24
  • INFERENCE_DEFAULTS: min_tcn_probability=0.60, etc.

• Unified 5 diverging locations to reference shared constants

  • cli/training.py - Training orchestration
  • src/core/modular_data_loaders.py - Both load_direction_data() and load_all_modular_data()
  • cli/commands.py - Backtest evaluation
  • src/core/modular_inference.py - InferenceConfig + 2 fallback sites

• Added tests/test_unified_defaults.py - Verifies constants match YAML config and are used correctly

Example

Before:

# cli/training.py
direction_threshold = cfg.get("direction_threshold", 0.005)  # Wrong fallback
direction_lookahead = cfg.get("direction_lookahead", 12)     # Wrong fallback

# src/core/modular_inference.py  
min_tcn_probability: float = 0.55  # Doesn't match YAML (0.60)

After:

from src.core.constants import DIRECTION_DEFAULTS, INFERENCE_DEFAULTS

# cli/training.py
direction_threshold = cfg.get("direction_threshold", DIRECTION_DEFAULTS['threshold'])
direction_lookahead = cfg.get("direction_lookahead", DIRECTION_DEFAULTS['lookahead'])

# src/core/modular_inference.py
min_tcn_probability: float = INFERENCE_DEFAULTS['min_tcn_probability']  # 0.60

All fallbacks now match the YAML config authoritative values. Future config changes require updating only src/core/constants.py.

Original prompt

Start implementation

The user has attached the following file paths as relevant context:

• .github/copilot-instructions.md

[Chronological Review: The conversation began with the user requesting to unify direction threshold/lookahead defaults and fix a configuration mismatch in the codebase. The user specified the exact locations in the code where changes were needed, indicating a clear direction for the implementation phase. The conversation then transitioned to the user providing context about the current repository and the active pull request, which is focused on refactoring the repository structure to standard Python conventions.]

[Intent Mapping: The user explicitly requested to create a single DIRECTION_DEFAULTS constant in a shared location and update five diverging fallback defaults to reference this constant. Additionally, the user requested to fix a mismatch in the InferenceConfig.min_tcn_probability.]

[Technical Inventory: The user is working within a Python repository named "ml_engine," specifically on the branch "copilot/refactor-repo-structure." The user is focused on standardizing configuration defaults and ensuring consistency across the codebase.]

[Code Archaeology: The user identified specific lines in the following files for modification:

• training.py: Lines 820-821 for direction thresholds
• modular_data_loaders.py: Lines 3666-3670 and 1681-1682 for fallback defaults
• commands.py: Line 2702 for fallback defaults
• modular_inference.py: Line 420 for fixing the min_tcn_probability mismatch.]

[Progress Assessment: The user has outlined specific tasks to be completed but has not yet indicated any completed tasks. The focus is on implementing the requested changes.]

[Context Validation: All necessary context for continuation is captured, including the specific files and lines to modify, as well as the overarching goal of eliminating silent configuration drift.]

[Recent Commands Analysis: The last agent commands involved summarizing the conversation history and capturing the user's requests regarding configuration changes. The tool results included the identification of the user's requests and the context of the repository. The agent was actively working on summarizing the conversation when the token budget was exceeded, which was triggered by the need to encapsulate the user's detailed requests and the repository context.]

1. Conversation Overview: - Primary Objectives: The user requested to "create a single DIRECTION_DEFAULTS constant in a shared location" and "fix InferenceConfig.min_tcn_probability mismatch." - Session Context: The conversation focused on standardizing configuration defaults in the "ml_engine" repository, with specific lines in various files identified for modification. - User Intent Evolution: The user's needs remained consistent, focusing on eliminating configuration drift and ensuring defaults are unified across the codebase.

2. Technical Foundation:

• Core Technology: Python, used for developing the "ml_engine" repository.
• Framework/Library: Not explicitly mentioned, but the context suggests standard Python conventions are being applied.
• Architectural Pattern: The user is refactoring the repository structure to improve maintainability and consistency.
• Environment Detail: The repository is hosted on GitHub, with an active pull request for refactoring.

3. Codebase Status:

• training.py:
  • Purpose: Contains training logic and configurations.
  • Current State: Lines 820-821 need to be updated to reference the new DIRECTION_DEFAULTS constant.
• modular_data_loaders.py:
  • Purpose: Handles data loading for the model.
  • Current State: Lines 3666-3670 and 1681-1682 need updates for fallback defaults.
• commands.py:
  • Purpose: Contains command definitions and configurations.
  • Current State: Line 2702 needs to be updated for fallback defaults.
• modular_inference.py:
  • Purpose: Manages inference configurations.
  • Current State: Line 420 needs to change the default min_tcn_probability from 0.55 to 0.60.

4. Problem Resolution:

• Issues Encountered: Diverging fallback defaults and a mismatch in configuration values.
• Solutions Implemented: Proposed changes to unify defaults and fix the configuration mismatch.
• Debugging Context: No ongoing troubleshooting efforts were mentioned, but the user is focused on implementing the requested changes.
• Lessons Learned: The importance of maintaining consistent configuration defaults to avoid silent drift.

5. Progress Tracking:

• Completed Tasks: None reported yet.
• Partially Complete Work: The user has outlined specific changes to be made but has not yet implemented them.
• Validated Outcomes: No features or code have been confirmed working through testing yet.

6. Active Work State:

• Current Focus: The user is focused on implementing the changes to unify configuration defaults and fix the mismatch in the InferenceConfig.
• Recent Context: The user provided specific ...

Created from VS Code.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.