Lectures / Schedule

Tentative Schedule (in progress)

Supervised Learning
R Jan 11
Introduction and course overview;
Binary classification and Bayes error
[slides] [lecture notes]
Bishop Ch 1
Duda, Hart & Stork, Pattern Classification
T Jan 16 [Class postponed]      
R Jan 18 [Class postponed]      
T Jan 23
Generative probabilistic models
for classification
[lecture notes]
Bishop Ch 4
PS1 out
R Jan 25
Discriminative probabilistic models
for classification
[lecture notes]
Bishop Ch 4
McCullagh & Nelder, Generalized Linear Models  
F Jan 26
Least squares regression
[lecture notes]
Bishop Ch 3
On your
Review: Constrained optimization
and Lagrangian duality
[review notes]
Boyd & Vandenberghe, Convex Optimization  
T Jan 30
Support vector machines
[lecture notes]
Bishop 7.1
Cristianini & Shawe-Taylor, Support Vector Machines  
R Feb 1
Kernel methods
[lecture notes]
Bishop Ch 6
Schoelkopf & Smola, Learning with Kernels  
F Feb 2
Neural networks/deep learning
[lecture notes]
Bishop Ch 5;
LeCun et al.'s review article on deep learning (Nature, 2015)
Goodfellow et al., Deep Learning;
Penn course CIS 680;
Other courses
T Feb 6
Decision trees;
Nearest neighbor methods
[lecture notes]
Bishop 14.4, 2.5.2
PS1 due;
PS2 out
R Feb 8 No class (university closed)      
F Feb 9 Discussion: PS1      
T Feb 13
[lecture notes]
Bishop 14.2-14.3
Schapire & Freund, Boosting  
R Feb 15 Tutorial: MATLAB      
T Feb 20
Performance measures
[lecture notes]
PS2 due;
PS3 out
R Feb 22
Understanding generalization error:
Bounds and decompositions
[lecture notes]
F Feb 23 Discussion: PS2      
T Feb 27 Midterm exam (in class)      
Unsupervised Learning
R Mar 1
Principal component analysis;
Auto-encoding neural networks
[lecture notes]
Bishop 12.1, 12.3,
Jolliffe, Principal Component Analysis
T Mar 6 No class (Spring Break)      
R Mar 8 No class (Spring Break)      
T Mar 13
K-means clustering;
Mixture models and EM algorithm
Bishop Ch 9
(primary reading)
PS3 due;
PS4 out;
proposals due
Probabilistic Graphical Models
R Mar 15
Hidden Markov models
[lecture notes]
Bishop 13.1-13.2
F Mar 16 Discussion: PS3 and Midterm      
T Mar 20
Hidden Markov models
[lecture notes]
R Mar 22
Bayesian networks
Jordan draft 2.1
(primary reading)
Koller & Friedman, Probabilistic Graphical Models  
F Mar 23 Project office hours      
T Mar 27
Markov networks;
Inference (variable elimination)
Jordan draft 2.2;
Jordan draft Ch 3
(primary reading)
Koller & Friedman, Probabilistic Graphical Models PS4 due;
PS5 out
R Mar 29
Inference (message-passing algorithms);
Topic models
Jordan draft 4.1, 4.3;
Review article on probabilistic topic models by Blei (Communications of the ACM, 2012)
(primary reading)
Koller & Friedman, Probabilistic Graphical Models
F Mar 30 Discussion: PS4      
Other Learning Frameworks
T Apr 3
Online learning
[lecture notes]
Cesa-Bianchi & Lugosi, Prediction, Learning, and Games  
R Apr 5
Semi-supervised learning;
Active learning
[lecture notes]
Zhu & Goldberg, Introduction to Semi-Supervised Learning;
Settles, Active Learning
F Apr 6 Project office hours      
T Apr 10
Reinforcement learning
Ng lecture notes Sec 1-3
(primary reading)
Sutton & Barto, Reinforcement Learning PS5 due;
PS6 out
Additional/Advanced Topics
R Apr 12
Structured prediction
[lecture notes]
Nowozin et al. (Eds), Advanced Structured Prediction  
F Apr 13 Discussion: PS5      
T Apr 17
Collaborative filtering
Review articles on recommender systems by Koren et al. (IEEE Computer, 2009) and Jannach et al. (Communications of the ACM, 2016)  
R Apr 19
Statistical consistency and PAC learning
Devroye et al., A Probabilistic Theory of Pattern Recognition;
Kearns & Vazirani, An Introduction to Computational Learning Theory;
Penn courses
CIS 620 and CIS 625
F Apr 20
Project office hours (Meyerson Hall B1)
slides due
Home Stretch
T Apr 24
Project spotlight presentations
PS6 due;
reports due
F Apr 27 Discussion: PS6 (Optional)      
F May 4 Final exam (9-11 AM, Huntsman Hall JMHH G06)