CAP 6610, Machine Learning, Spring 2007

Instructor:
Prof. Arunava Banerjee
Office: CSE E336.
E-mail: arunava@cise.ufl.edu.
Phone: 392-1476.
Office hours: Wednesday 2:00 p.m.-4:00 p.m. or by appointment.

TA:
Karthik Gurumoorthy:
Office: CSE E445.
E-mail: ksg@cise.ufl.edu.
Office hours: Monday 3:00 p.m.-5:00 p.m.(at CSE E309) or by appointment.

Pre-requisites:

• There are no official pre-requisites for this course. However, knowledge of calculus and linear algebra is necessary since we shall be touching on mathematical probability theory. In addition, since there will be several programming assignments, proficiency in some programming language is a must.

Textbook: Pattern Classification, 2nd Edition, Duda, Hart and Stork, John Wiley, ISBN 0-471-05669-3.

Tentative list of Topics to be covered

• Introduction to Mathematical Probability Theory
• Perceptron, Multi-layer neural networks, SVM
• Bayesian Decision Theory, Maximum likelihood, MAP
• PCA, ICA, HMMs
• Decision Trees

The above list is tentative at this juncture and the set of topics we end up covering might change due to class interest and/or time constraints.

Please return to this page at least once a week to check updates in the table below

Evaluation:

• Homework assignments (written and programming): 40%
• Two midterm exam: 30% each (2 hrs, in-class)
• There will be no makeup exams (Exceptions shall be made for those that present appropriate letters from the Dean of Students Office).

Course Policies:

• Late assignments: All homework assignments are due before class.
• Plagiarism: You are expected to submit your own solutions to the assignments. While the final project and presentation will be done in groups, each member will be required to demonstrate his/her contribution to the work.
• Attendance: Their is no official attendance requirement. If you find better use of the time spent sitting thru lectures, please feel free to devote such to any occupation of your liking. However, keep in mind that it is your responsibility to stay abreast of the material presented in class.
• Cell Phones: Absolutely no phone calls during class. Please turn off the ringer on your cell phone before coming to class.

Academic Dishonesty: See http://www.dso.ufl.edu/judicial/honestybrochure.htm for Academic Honesty Guidelines. All academic dishonesty cases will be handled through the University of Florida Honor Court procedures as documented by the office of Student Services, P202 Peabody Hall. You may contact them at 392-1261 for a "Student Judicial Process: Guide for Students" pamphlet.

Students with Disabilities: Students requesting classroom accommodation must first register with the Dean of Students Office. The Dean of Students Office will provide documentation to the student who must then provide this documentation to the Instructor when requesting accommodation.

HomeWorks

HomeWork	Due Date	HomeWork Solutions
HomeWork 1	Feb 8th 2007	HomeWork 1 Sol
HomeWork 2	Mar 1st 2007
HomeWork 3	Apr 3rd, 2007 (postponed from Mar 29th 2007)
HomeWork 4	Apr 19th, 2007

List of Topics covered

Week	Topic
Jan 08 - Jan 14	Introduction Probabilistic Framework for Machine learning; Generator, Supervisor, and the Learning machine The Risk functional approach to Pattern recognition and Regression estimation.
Jan 15 - Jan 21	The Risk functional approach to Density estimation The inductive hypothesis and the Empirical Risk minimization principle Informal description of the law of large numbers for functionals. Limit supremum and Limit infimum, Limits of sets
Jan 22 - Jan 28	Probability Space (Sample space, sigma algebra, Probability measure) The Borel Sigma algebra Various theorems in Mathematical Probability Theory: Boole's inequality, Convergence of probability measure, Borel-Cantelli Lemma Almost sure and Null events. Probability distribution functions and Probability density functions
Jan 29 - Feb 03	Random Variables Independence of Random variables Expectation of a random variable Simple functions and Lebesgue integration Convergence in probability and almost sure convergence
Feb 04 - Feb 10	Survey Paper on Mathematical Modeling of Learning Markov, Chebyshev Inequality Proof for Weak Law of Large Numbers Proof for Strong Law of Large Numbers (simpler proof under sufficient but not necessary conditions)
Feb 11 - Feb 17	Optimal discriminant boundary for normal distributions Perceptron Intro to Support Vector Machines Margin based VC dimension Paper
Feb 18 - Feb 24	Support Vector Machines continued Few chapters from Scholkopf and Smola's book Constrained optimization and Lagrange Multipliers
Feb 25 - Mar 03	Support Vector Machines continued Karush Kuhn Tucker Conditions Dual Formulation of SVMs Kernel SVMs
Mar 05 - Mar 11	Decision Trees Multilayer Neural Networks; back-propagation
Mar 12 - Mar 18	SPRING BREAK
Mar 19 - Mar 25	Finished back-prop Maximum Likelihood estimate Bias in Maximum Likelihood Began Expectation Maximization
Mar 26 - Apr 1	Expectation Maximization Midterm-1 on Thursday (Open Book; Open Notes).
Apr 2 - Apr 8	Expectation Maximization for Mixture of Gaussians: A Note explaining the derivation Started Principal Component Analysis
Apr 9 - Apr 15	Finished Principal Component Analysis Vector/Babach/Hilbert Space Singular Value decomposition
Apr 16 - Apr 22	Discrete Stochastic process, kth order Markov model Hidden Markov Model Evaluation, Decoding (Viterbi Algorithm) Learning using expectation maximization
Apr 23 - Apr 29	Finished HMM Learning using expectation maximization Midterm-2 on Thursday (Open Book; Open Notes).