Minimax Deviation Strategies for Machine Learning and Recognition with Short Learning Samples

Prof. Michail Schlesinger and
Evgeniy Vodolazskiy

Ukraine, International Research and Training Centre of Information Technologies and Systems, National Academy of Science of Ukraine, Cybernetica Centre
We formulate problems of learning and recognition in a common framework of complex hypothesis testing. Based on arguments from multi-criteria optimization, we identify strategies that are improper for solving these problems and derive a common form of the remaining strategies. We show that some widely used approaches to recognition and learning are improper in this sense. We then propose a generalized formulation of the recognition and learning problem that embraces the whole range of sizes of the learning sample, including zero size. Learning becomes a special case of recognition. We define the concept of minimax deviation Bayesian learning, being a solution to the formulated problem. In several illustrative cases, the strategy is shown to be superior to the widely used learning methods based on maximal likelihood estimation.