This is a tutorial repository on how to get started with the MLE-Infrastructure:
- Google Colab walkthrough of the sub-packages:
- Google Colab walkthrough of the
mle-toolbox: - Example template project repository:
mle-project.
- 12/2021: Sprekeler lab group presentation slides
mle-logging: A Lightweight Logger for ML Experiments 📖mle-scheduler: A Lightweight Cluster/Cloud VM Job Management Tool 🚀mle-hyperopt: A Lightweight Hyperparameter Optimization Tool 🚂mle-monitor: A Lightweight Experiment & Resource Monitoring Tool 📺mle-toolbox: A Lightweight Tool to Manage Distributed ML Experiments 🛠
| Job Types | Description | |
|---|---|---|
| 📄 Single-Objective | multi-configs, hyperparameter-search |
Core experiment types. |
| 📄 Multi-Objective | hyperparameter-search |
Multi-objective tuning. |
| 📄 Multi Bash | multi-configs |
Bash-based jobs. |
| 📄 Quadratic PBT | hyperparameter-search |
PBT on toy quadratic surrogate. |
| 📄 Hyperband | hyperparameter-search |
Hyperband on toy polynomial problem. |
| Description | Colab | |
|---|---|---|
| 📓 Getting Started | Get started with the toolbox. |  |
| 📓 Subpackages | Get started with the toolbox subpackages. | |
📓 MLExperiment |
Introduction to MLExperiment wrapper. |
|
| 📓 Evaluation | Evaluation of gridsearch results. | |
| 📓 GIF Animations | Walk through a set of animation helpers. | |
| 📓 Testing | Perform hypothesis tests on logs. | |
Here is a set of published papers using parts of the mle-toolbox experiment infrastructure to generate their results:
- Vischer*, M. A., Lange*, R. T., & Sprekeler, H. (2021). On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning. arXiv preprint arXiv:2105.01648.
- Lange, R. T., & Sprekeler, H. (2020). Learning not to learn: Nature versus nurture in silico. Published as a conference paper at AAAI 2022.
And here is a set of blogs and other projects using parts of the mle-toolbox experiment infrastructure: