2024-06-30-aden-ali24a.md

title	section	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Majority-of-Three: The Simplest Optimal Learner?	Original Papers	Developing an optimal PAC learning algorithm in the realizable setting, where empirical risk minimization (ERM) is suboptimal, was a major open problem in learning theory for decades. The problem was finally resolved by Hanneke a few years ago. Unfortunately, Hanneke’s algorithm is quite complex as it returns the majority vote of many ERM classifiers that are trained on carefully selected subsets of the data. It is thus a natural goal to determine the simplest algorithm that is optimal. In this work we study the arguably simplest algorithm that could be optimal: returning the majority vote of three ERM classifiers. We show that this algorithm achieves the optimal in-expectation bound on its error which is provably unattainable by a single ERM classifier. Furthermore, we prove a near-optimal high-probability bound on this algorithm’s error. We conjecture that a better analysis will prove that this algorithm is in fact optimal in the high-probability regime.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	aden-ali24a	0	Majority-of-Three: The Simplest Optimal Learner?	22	45	22-45	22	false	Aden-Ali, Ishaq and H\o{}andgsgaard, Mikael M\o{}ller and Larsen, Kasper Green and Zhivotovskiy, Nikita	given family Ishaq Aden-Ali given family Mikael Møller Høandgsgaard given family Kasper Green Larsen given family Nikita Zhivotovskiy	2024-06-30		Proceedings of Thirty Seventh Conference on Learning Theory	247	inproceedings	date-parts 2024 6 30	https://proceedings.mlr.press/v247/aden-ali24a/aden-ali24a.pdf