Skip to content

COMPETITION_STRATEGY.md

Latest commit

 

History

History
263 lines (212 loc) · 7.75 KB

COMPETITION_STRATEGY.md

File metadata and controls

263 lines (212 loc) · 7.75 KB

MITSUI&CO. Commodity Prediction Challenge - Competition Strategy

🏆 Competition Overview

Prize Pool: $100,000

  • 1st Place: $20,000
  • 2nd Place: $10,000
  • 3rd Place: $10,000
  • 4th-15th Place: $5,000 each

Timeline:

  • Entry Deadline: September 29, 2025
  • Team Merger Deadline: September 29, 2025
  • Final Submission Deadline: October 6, 2025

📋 Competition Rules Compliance

✅ Submission Requirements

  • Format: Notebook submissions only
  • Runtime: ≤ 8 hours (9 hours during forecasting phase)
  • Internet: Disabled during execution
  • Submissions: 5 per day maximum, 2 final submissions for judging
  • Team Size: Maximum 5 members

✅ Code Requirements

  • Reproducibility: Complete code delivery required for winners
  • Documentation: Detailed methodology and architecture description
  • License: Non-exclusive license granted to sponsor
  • External Data: Allowed if "reasonably accessible to all"

🚀 Competition Strategy

Phase 1: Foundation (Week 1-2)

  1. Data Understanding

    • Analyze 1977 features across LME, JPX, US, FX markets
    • Understand 424 target configurations
    • Identify data quality issues and patterns

  2. Baseline Development

    • Implement memory-efficient pipeline
    • Create robust cross-validation
    • Establish performance benchmarks

  3. Community Engagement

    • Join official Discord
    • Study public notebooks
    • Consider team formation

Phase 2: Optimization (Week 3-4)

  1. Feature Engineering

    • Domain-specific features (spreads, ratios, correlations)
    • Time-series features (lags, rolling statistics)
    • Cross-asset relationships

  2. Model Development

    • Gradient boosting optimization
    • Ensemble methods
    • Hyperparameter tuning

  3. Memory & Runtime Optimization

    • Feature selection (top 500-1000 features)
    • Data type optimization
    • Batch processing

Phase 3: Advanced Techniques (Week 5-6)

  1. Multi-Target Optimization

    • Target-specific feature engineering
    • Multi-output model tuning
    • Target correlation analysis

  2. Stability & Generalization

    • Overfitting prevention
    • Robust validation strategies
    • Model ensemble diversity

  3. Competition-Specific Tuning

    • Sharpe ratio variant optimization
    • Stability-focused evaluation
    • Cross-validation with gaps

Phase 4: Final Submission (Week 7-8)

  1. Submission Preparation

    • Notebook format conversion
    • Runtime optimization
    • Memory efficiency verification

  2. Quality Assurance

    • Reproducibility testing
    • Performance validation
    • Competition compliance check

🎯 Key Success Factors

1. Memory Management (CRITICAL)

# Memory optimization strategies
- Reduce data types (float64 → float32 → float16)
- Feature selection (correlation-based)
- Chunk processing for large datasets
- Batch predictions
- Garbage collection

2. Runtime Optimization (CRITICAL)

# Runtime optimization strategies
- Conservative model parameters
- Early stopping in gradient boosting
- Single-threaded processing
- Efficient data loading
- Minimal cross-validation during submission

3. Overfitting Prevention (HIGH)

# Overfitting prevention strategies
- Time-series CV with gaps
- Regularization (L1/L2)
- Feature sampling
- Data sampling
- Model ensemble diversity

4. Competition Metric Optimization (HIGH)

# Sharpe ratio variant optimization
- Mean Spearman correlation / Standard deviation
- Focus on stability (lower variance)
- Multi-target performance
- Robust evaluation across time periods

🔧 Technical Implementation

Memory-Efficient Pipeline

# Key components
from src.memory_optimization import create_memory_efficient_pipeline
from src.robust_validation import time_series_cv_robust
from src.multi_target import parse_target_pairs

# Usage
train_opt, labels_opt, test_opt, features = create_memory_efficient_pipeline(
    train_df, train_labels, target_pairs, test_df,
    max_features=500,  # Conservative for runtime
    max_targets=None   # Use all targets
)

Competition-Ready Submission

# Run competition submission
python run_competition_submission.py

# Or use notebook template
jupyter notebook notebooks/competition_submission_template.ipynb

📊 Performance Targets

Baseline Targets

  • Memory Usage: < 16 GB
  • Runtime: < 6 hours (2-hour buffer)
  • CV Score: > 0.5 (competition metric)
  • Stability: CV std < 0.1

Competitive Targets

  • Memory Usage: < 12 GB
  • Runtime: < 4 hours
  • CV Score: > 1.0
  • Stability: CV std < 0.05

🏅 Community Insights

From Discussion Analysis

  1. Memory Management: Major challenge (14 replies in "Reduce data size")
  2. Overfitting: Significant concern (4 comments)
  3. Team Formation: Active (multiple "Looking for team" posts)
  4. Getting Started: Official Discord available

Recommended Actions

  1. Download real competition data immediately
  2. Join official Discord for community support
  3. Study top public notebooks for insights
  4. Consider team formation for better results
  5. Focus on memory optimization first

📈 Evaluation Strategy

Public vs Private Leaderboard

  • Public: Based on public test set (visible during competition)
  • Private: Based on private test set (final ranking)
  • Strategy: Don't overfit to public leaderboard

Validation Strategy

# Robust validation approach
- Time-series CV with gaps (prevent data leakage)
- Multiple validation periods
- Stability-focused evaluation
- Overfitting detection

🚨 Risk Mitigation

Technical Risks

  1. Memory Issues: Implement aggressive memory optimization
  2. Runtime Exceeded: Conservative model parameters
  3. Overfitting: Robust validation and regularization
  4. Reproducibility: Complete documentation and code

Competition Risks

  1. Late Submission: Submit early and iterate
  2. Rule Violations: Review rules carefully
  3. Team Issues: Clear communication and agreements
  4. External Dependencies: Avoid expensive external data/tools

📚 Resources

Official Resources

Technical Resources

Community Resources

🎯 Success Metrics

Immediate Goals

  • Download and analyze real competition data
  • Implement memory-efficient baseline
  • Join official Discord community
  • Submit first entry

Short-term Goals (2 weeks)

  • Achieve baseline performance
  • Optimize for memory and runtime
  • Study top public notebooks
  • Consider team formation

Medium-term Goals (4 weeks)

  • Implement advanced feature engineering
  • Optimize for competition metric
  • Achieve competitive performance
  • Prepare final submission strategy

Long-term Goals (6 weeks)

  • Final model optimization
  • Competition submission preparation
  • Performance validation
  • Documentation completion

Remember: This is a marathon, not a sprint. Focus on stability, reproducibility, and incremental improvements rather than chasing the public leaderboard.