Code For "Better Uncertainties Don't Guarantee Better Decisions"

This is the repository for the EIML@EurIPS 2025 workshop paper "Better Uncertainties Don't Guarantee Better Decisions".

🚀 Getting Started

All requirements can be found in the pyproject.toml. The probably fastest way to get started is to install uv and simply run

uv sync

📈 Running Experiments

All experiments from Section 4 of the paper can be found in the experiments folder.

Toy Examples

The toy example data is provided in this repository and the results can be reproduced via the provided Jupyter notebooks.

Experiments on Benchmark Datasets

We have separate folders for binary classification and regression experiments. First run preprocess_datasets.py in the respective folder to fetch and prepare the datasets. Then you can run the binary_classification.py/ regression.py from the terminal, specifying the dataset you want to use, or run all datasets with one command using orchestration.sh. The results will be stored and can be inspected using the corresponding results.ipynb Jupyter notebook.

Case Study

Note that you first need to download the data / create an API to be able to fetch the data. You can find corresponding instructions in the case study Jupyter notebook. After that, the experiment can be reproduced via running the Jupyter notebook.

🗨️ Questions or Problems?

Don't hesitate to reach out if you encounter bugs, or have general questions or feedback! 💫