CMPE 58K, Sp Top in CmpE: Bayesian Statistics and Machine Learning

Fall 2008-2009

Instructor: Ali Taylan Cemgil
Bogaziçi University, Department of Computer Engineering, Istanbul, Turkey

Announcement

Final Project Presentations: 19, 20 and 21 Jan 2009

The format:
We will meet at my room ETA 19 with a group of 6-8 for an informal roundtable meeting where you will describe what you have been doing on the whiteboard. You may prepare a few slides if you wish to do so, but you don't have to. You can also just go over your report.
You must bring your completed report to the meeting. After you are finished with your presentation you will also listen to what others have done.
You will talk about the following points:

The problem statement, Motivations, why you are interested into this problem	3 min
Your model	3 min
Your inference method	3 min
Your results in a nutshell, not too much detail	3 min
What would you do if you had more time on this?	1 min
Questions	max 5 min

Anyone, who has submitted a proposal is assigned to one of the following slots:

19 Jan 9:30-10:00

20 Jan 10:00-12:30

20 Jan 15:00-17:30

21 Jan 10:00-12:30

The schedule is here
If for some reason you are not allocated to any slot, let me know immediately
If you want to exchange time slots with others, you may do so.

Final Exam Deadline is extended: 19 Jan 2009, 10:00

Dataset for question 6.2 and 6.3 (matlab mat file)
Note there was a typo: A6.2 x_2 \in {1 .. N_2}
A6.5 We ask now for p(\hat{y}| y_{1:L}, x_{1:L}, \hat{x})
(previously was f instead of y) This should change your answers slightly and would correct the potential instabilities.

The Project deadline is not extended, submit your final version reports by the deadline time. There will be project presentations the week of 19th Jan. The presentations will take in groups of about 4-6 people. Further remarks and the schedule of the presentations will follow later.

Catalog Information

Class List, Including Quiz and Assignment Results

Midterm Results

Midterm Questions and Solutions

Lecture Outline, Slides, Assignments,

Date	Topic	Slides	Reading	Assignment	Solutions
Sep 24	No class		Check your math skills Sections 2.1-2.3 from David MacKay's book A short introduction to graphical models by Kevin Murphy
Oct 1	Bayram, No Class
Oct 8	Probability Theory Graphical Models	Lecture 1	Chapter 1 and 2 from Bishop	Problem Sheet 1
Oct 15	Bayesian Learning, Probability Distributions	Lecture 2	Chapter 1 and 2 from Bishop	Problem Sheet 2	Solutions
Oct 22	Conjugate Priors, Sequential Data	Lecture 3	Chapter 8 and 13 from Bishop	Problem Sheet 3	Solutions
Oct 29	No Class, Cumhuriyet Bayrami
Nov 5	Inference in HMM's, Multivariate Gaussians, Linear Dynamical Systems	Lecture 4	A technical note on Multivariate Gaussians	Problem Sheet 4	Solutions
Nov 12	Approximate Bayesian Inference, Sampling methods, Markov Chain Monte Carlo, Gibbs Sampler	Lecture 5	MacKay 29.1, 29.3, 29.4, 29.5, 29.6, 29.9, 30.2, 30.3 Bishop, 11.1, 11.2, 11.3	No new assignment sheet Do any missing assignments you like from sheets 1 - 4
Nov 19	Metropolis-Hastings, Gibbs sampler, Simulated Annealing, Assignments 1-4 Review,	No new slide set		Problem Sheet 5 (complete as of 10 Dec)
Nov 26	Importance Sampling, Nonlinear Dynamic Systems, Changepoint models	Lecture 6	A.Doucet and A. Johansen, Particle filtering and smoothing: Fifteen years later, 2008
Dec 3	Midterm review, Sequential Monte Carlo	No new slide set	A. Doucet, S.J. Godsill and C. Andrieu, On Sequential Monte Carlo sampling methods for Bayesian filtering, (section IV) Stat. Comp., 2000
Dec 10	No Class, Bayram
Dec 17	Introduction to Variational Methods, Mean field, Variational Bayes	Lecture 7 (updated)	MacKay 33.1, 33.4, 33.5, 33.6, 33.7, Bishop 10.1, 10.2, 10.3, 10.4
Dec 24	Variational Bayes, EM, ICM	No new slide set
Dec 31	Exact Inference, Junction Tree, Belief Propagation, Sum Product	Lecture 8		Final Problem Set
14 Jan, 23:59 GMT	Submission Deadline of Final Project Report

Past Announcements

Midterm : 29 Nov 2008, SATURDAY, 10:00-13:00, ETA 5

(Place may change to another room, but the exam will be in ET (CMPE) Building, Please watch eventual announcements at the main entrance)
Closed notes, closed books. Bring a pencil and an eraser. We won't need a calculator, laptop or a mobile phone.

There will be 8 questions for a total of 60 pts. Tentative outline of the midterm questions is as follows:

1 - General knowledge 12pts

1 Quiz style question 6 pts

2 Simple modelling questions, i.e. given verbal descriptions or some plotted data construct a probabilistic model 12 pts

2 questions regarding factorisations of Gaussians 12 pts

2 surprise questions 18 pts

Please also look at Problem Sheet 5 (part 1) . This sheet is included as additional exercise material. Solving some of these exercises before the examination is particulary encouraged.

New Deadline for handing in the project proposal : 21 Nov 2008, 10:00

The following proposal examples could give you an idea about the level of detail I like to see.
MCMC example
HMM example
You can also look for papers in the literature and include their abstracts + how you would apply them for an application you are interested in. The key here is to know what your goal is and what you aim to deliver. Remember, a good proposal is like a good proposal distribution; not too far, not too short.