Copenhagen Image and Signal Processing
Graduate School

Main page > Events > Courses > A Statistical Physics Approach to Bayesian Inference
Main page
People
Events
     Courses
     Talks
     Workshops
Presentations
Software
Links
Contact

Ph.D. Course: A Statistical Physics Approach to Bayesian Inference

14., 21., 29. August and 2. September 2002
10.00 am
DTU/IMM, Richard-Petersens-Plads, Building 321, Room 133 (not room 119)
The 2. September will be in building 322, room 133.

The course will be given by Dörthe Malzahn at Wednesdays, 10.00 am, as a series of lectures. It starts Wednesday, 14.8.2002. The course will be given in English.

There will be 2 additional lectures on Thursday, 12.9.02 and 19.9.02. Time: 9-11 am Room: 133, Building 321, DTU/IMM

Aim

This course outlines a new statistical physics based approach to the theory of learning with kernel machines. Examples are Gaussian process models and Support Vector machines which can be obtained as posterior mean or maximum-a-posteriori (MAP) of a suitably defined Bayesian inference model. The course teaches central ideas of statistical physics - such as the concept of self-averaging quantities, order parameters and the variational principle. The latter enables us to develop a theory which contains the data density as a free parameter. I give two examples how the new theory can be used to derive tools for the evaluation of learning in practically relevant situations where the data density is either unknown or replaced by the empirical density (resampling methods).

Program

O. Role of statistical physics in learning theory
1. Basic concepts
1.1. Introduction of free energy F and partition function Z
1.2. The use of F
1.3. Statistical properties of F and Z
1.4. The Replica-trick
2. A ''classical'' statistical mechanics calculation for a Gaussian process regression model
(Order parameters, saddle point argument, covariance structure in replica space)
3. Alternative treatment
3.1. The variational principle: Feynman versus Kulback-Leibler
3.2. The grand-canonical ensemble
3.3. Order parameters, revisited
4. Bootstrap estimates
5. Error estimates
5.1. A simple exercise
5.2. Calculating empirical measures

For further information you can download two recent papers:
Paper1 (accepted for publication in Physical Review Letters)
Paper2 (appeared in Advances in Neural Information Processing Systems 14, MIT Press, 2002)

Contact

Dörthe Malzahn
Informatics and Mathematical Modelling, Technical University of Denmark (DTU)
Tel: 4525 3922
mail: dm@imm.dtu.dk
Room: 124, Richard-Petersens-Plads, Building 321

$Id: 20020814Statistical.html,v 1.4 2002/09/11 10:03:30 fnielsen Exp $, © 2001 IMM DTU