Machine learning theory: additional chapters

I. Bayesian approach to the problem.

Lecture 1. General statement. Bayes formula. Bayesian strategy in the Game Theory.
Lecture 2. Regularization of the least square method on the basis of Bayesian approach.
Lecture 3. Inverse problems and their solution on the basis of Bayesian approach.
Lecture 4. Kriging method. Ridge regression.
Lecture 5. Criticism of Bayesian approach. Regularization as an approximate realization of the Bayesian approach. The problem of regularization parameters and basis functions selection.

II. The optimal model complexity choice.

Lecture 6. Structural risk minimization on the basis of uniform convergence. General approach.
Lecture 7. Application of Structural empirical risk minimization to pattern recognition problems.
Lecture 8. Application of Structural empirical risk minimization to model reconstruction. Relative estimates of the uniform closeness of means to expectations. Their application to Structural empirical risk minimization.
Lecture 9. Support Vector Machine.
Lecture 10. Recurrent Algorithms. Criterion of minimum description length.
Lecture 11. Combined Bayesian – Maximum Likelihood method.
Lecture 12. Examples of practical applications.