Intelligent Learning: Similarity Control and Knowledge Transfer

Prof. Vladimir Vapnik
USA, Columbia University, Facebook
During the last 50 years, a strong machine learning theory has been developed. This theory includes:
– the necessary and sufficient conditions of consistency of learning processes;
– the bounds on the rate of convergence which in general cannot be improved;
– the new inductive principle (SRM) which always achieves the smallest risk;
– the effective algorithms (such as SVM) that realize the SRM principle. It looked like general learning theory had been complied: it answered almost all standard questions that were asked in the statistical theory of inference.
Meanwhile, the common observation was that human students require much less examples for training than a learning machine. Why?
This talk attempts to answer this question. The answer is that it is because the human students have an intelligent teacher and that teacher-student interactions are based not only on the brute force methods of function estimation from observations. Speed of learning is also based on teacher-student interactions which have additional mechanisms that boost the learning process. To learn from a smaller number of observations, the learning machine has to use these mechanisms.
In this talk I will introduce a model of learning that includes the so-called Intelligent Teacher who during training session supplies the Student with intelligent (privileged) information, in contrast to the classical model where the student is given only outcomes y for events x. Based on additional privileged information x* for event x two mechanisms of Teacher-Student interactions are introduced:
– the mechanism to control Student’s concept of examples similarity; and
– the mechanism to transfer knowledge that can be obtained in the space of privileged information to the desired space of decision rules.
Privileged information exists for any inference problem and Student-Teacher interaction can be considered as the basic element of intelligent behavior.