Skip to content

Difficulty reproducing forecasting results with identical config and pretrained weights #59

@Gwenielulu

Description

@Gwenielulu

Hi, I followed the official config (configs/global_forecast_climax.yaml) and used exactly the same hyperparameters as described in the paper. All training parameters — including learning rate, optimizer settings, and warmup epochs (set following the official code) are identical to the original setup. The pretrained checkpoint (5.625deg.ckpt) was loaded successfully, but the forecasting results, especially for geopotential_500, still deviate significantly from the reported values (≈3× higher RMSE), while other variables remain consistent or slightly better.Could you please confirm if there are any hidden preprocessing, normalization, or climatology adjustments affecting geopotential metrics?

My experiments are based on the WeatherBench2 ERA5 dataset, which may differ slightly from the original preprocessing used in the ClimaX paper. I wonder if this could partially explain the higher RMSE observed for geopotential fields.

Thanks for your great work and open-sourcing ClimaX!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions