Skip to content

Task 1.2 (b) Move Development Code related to the D2 layer #17

Description

@xandie985

D2 Layer Implementation for Time Series Dataset

Key Changes

  1. Architecture

    • Split into modules:
      • time_series_d2.py: Model-ready processing (TSDataModule)
    • Modularized data handling to separate D2 logic from D1, improving maintainability and clarity.
  2. D2 Layer (TSDataModule)

    • PyTorch Lightning integration
    • Sliding-window generation with NaN-aware indexing (preserves NaNs for downstream validation/model training)
    • Flexible train/val/test splits
  3. Integration with D1 Layer (MultiSourceTSDataSet)

    • Multi-source CSV loading, encoding/normalization, memory-efficient & pre-loaded modes
    • Compatibility tests ensuring D2 correctly consumes D1 outputs
  4. Code Quality

    • Docstrings, type hints, error handling, logging
  5. Testing

    • Unit tests for D2 layer
    • Tests covering D1→D2 integration

⚠️ Blocked by PR #16: Please merge PR #16 to main before continuing work on this PR.

Metadata

Metadata

Assignees

Labels

data_layerissues related to data layers (d1/d2/others data related tasks)documentationImprovements or additions to documentationenhancementNew feature or request

Type

Projects

Status
No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions