This course will survey recent advances in applying nonparametric Bayesian methods to machine learning problems. The variety of approaches in the literature can be treated in a unified way using stochastic processes, and a major part of the course will be devoted to studying important stochastic processes such as Dirichlet Process, Gaussian Process and Beta Process. We will also discuss important and interesting applications in image processing and topic modeling.
Prerequisites:
College calculus, probability theory and some familiarity with basic machine learning methods.
Grading:
Implementation
Slice Sampling of Dirichlet Process. Data . Data in text .
The data contains 600 1D points drawn from a Gaussian mixture with five components. The variance of each component is set to 1. You need to identify the mean of each component. Note that your simulation result may indicate more than five components. However, you should be able to look at the result and identify at least 3-4 components easily.
This course does not have a required textbook since most of the materials will be taken out of recently published papers. However, there are several books that can serve as references throughout the semester:
For the brave ones, the book that covers most of what we will discuss in class this semester (and more) is, "Bayesian Nonparametrics" byJ.K. Ghosh and R.V. Ramamoorthi, published by Springer. Be warned that this book is not for faint-hearted.
For probability, a good book to have is "Probability" by Davar Khoshnevisan, published by American Mathematical Society.
For Gaussian Process, "Gaussian Processes for Machine Learning" by Carl Rasmussen and Christopher Williams, published by the MIT press (available in PDFs).
A good textbook for stochastic processes is "Essential of Stochastic Processes" by Rick Durret, published by Springer.