IP Logo		  

DAGM 2006

28th Annual Symposium of the German Association for Pattern Recognition
     















 

TM2: Appropximate Probabilistic Inference for Machine Learning
by Manfred Opper, Technical University Berlin, Germany


Abstract:


TBA


 Appropximate Probabilistic Inference for Machine Learning


Probabilistic Bayesian models find widespread applications in machine learning and artificial intelligence. They explain observed data using generative models with probability distributions over hidden, unobserved variables. In such a way it is possible to incorporate uncertainties and prior knowledge in the models. Predictions on unobserved variables and the learning of model parameters is conceptually simple in this approach and involves only basic manipulations (marginalisation) of probability distributions.


Unfortunately, the practical computations (sums, high-dimensional integrals) can become intractable when the number of hidden variables is large. Hence, approximation techniques are needed which allow for a fast and reliable inference in such cases. This tutorial aims at introducing a variety of ideas which have been developed in recent years to overcome this problem. These methods usually replace the intractable summations/integrations by a tracable optimisation problem.


=================================================================

Overview:


  • 1. Inference with probabilistic models
    • Introduction geometry (1/2h)
    • examples
    • computational problems.
  • 2. The variational method
    • mean field approximation
    • structured mean field approximations
    • variational EM
    • Applications
    • Limitations
  • 3. Free Energy approximations
    • belief propagation
    • Bethe approximation
    • Kikuchi hierarchy
    • expectation consistency (EP, EC)
    • applications
  • 4. Outlook
    • accuracy & reliability
    • convexify!
    • computing corrections

CV:


After a PhD in physics in 1987 (Giessen, Germany) and postdoctoral visits (ecole normale superieure, Paris and UC Santa Cruz) in 1989-90 Manfred Opper received a habilitation degree in theoretical physics in 1991 from Giessen university.


In 1992 he was awarded the Physics Prize of the German Physical Society for his work on learning in neural networks. After some time as a research associate (Giessen and Wuerzburg, Germany) he was awarded a Heisenberg fellowship in 1994 which he spent at the machine learning group of UC Santa Cruzand at the Weitzmann Institute in Israel. He joined the Neural Computing Research Group at Aston University in Birminngham (UK) as a reader in 1997.


In 2004 he moved to Southampton university to work in the signals, images, systems (ISIS) group, where he became a professor in 2005. Since 2006 he is a professor for artifical intelligence at the Technical University in Berlin, Germany. He has over 120 publications, mostly in the applications of statistical mechanics and related probabilistic methods to problems of machine learning and other complex systems.