Lectures / Schedule


Tentative Schedule (in progress)

Date
 
 
Topic
 
 
Recommended
Additional
Reading
Further
Study
 
Notes
 
 
Supervised Learning
R Jan 11
 
 
Introduction and course overview;
Binary classification and Bayes error
[slides] [lecture notes]
Bishop Ch 1
 
 
Duda, Hart & Stork, Pattern Classification
 
 
 
 
T Jan 16 [Class postponed]      
R Jan 18 [Class postponed]      
T Jan 23
 
 
Generative probabilistic models
for classification
[lecture notes]
Bishop Ch 4
 
 
 
 
 
PS1 out
 
 
R Jan 25
 
 
Discriminative probabilistic models
for classification
[lecture notes]
Bishop Ch 4
 
 
McCullagh & Nelder, Generalized Linear Models  
F Jan 26
 
Least squares regression
[lecture notes]
Bishop Ch 3
 
 
 
 
 
On your
own
 
Review: Constrained optimization
and Lagrangian duality
[review notes]
 
 
 
Boyd & Vandenberghe, Convex Optimization  
 
 
T Jan 30
 
 
Support vector machines
[lecture notes]
 
Bishop 7.1
 
 
Cristianini & Shawe-Taylor, Support Vector Machines  
 
 
R Feb 1
 
Kernel methods
[lecture notes]
Bishop Ch 6
 
Schoelkopf & Smola, Learning with Kernels  
 
F Feb 2
 
 
 
 
Neural networks/deep learning
[lecture notes]
 
 
 
Bishop Ch 5;
LeCun et al.'s review article on deep learning (Nature, 2015)
Goodfellow et al., Deep Learning;
Penn course CIS 680;
Other courses
 
 
 
 
 
 
T Feb 6
 
 
Decision trees;
Nearest neighbor methods
[lecture notes]
Bishop 14.4, 2.5.2
 
 
 
 
 
PS1 due;
PS2 out
 
R Feb 8 No class (university closed)      
F Feb 9 Discussion: PS1      
T Feb 13
 
Boosting
[lecture notes]
Bishop 14.2-14.3
 
Schapire & Freund, Boosting  
 
R Feb 15 Tutorial: MATLAB      
T Feb 20
 
Performance measures
[lecture notes]
 
 
 
 
PS2 due;
PS3 out
R Feb 22
 
 
Understanding generalization error:
Bounds and decompositions
[lecture notes]
     
 
 
F Feb 23 Discussion: PS2      
T Feb 27 Midterm exam (in class)      
Unsupervised Learning
R Mar 1
 
 
Principal component analysis;
Auto-encoding neural networks
[lecture notes]
Bishop 12.1, 12.3,
12.4.2
 
Jolliffe, Principal Component Analysis
 
 
 
 
T Mar 6 No class (Spring Break)      
R Mar 8 No class (Spring Break)      
T Mar 13
 
 
 
K-means clustering;
Mixture models and EM algorithm
 
 
Bishop Ch 9
(primary reading)
 
 
 
 
 
 
PS3 due;
PS4 out;
Project
proposals due
Probabilistic Graphical Models
R Mar 15
 
Hidden Markov models
[lecture notes]
Bishop 13.1-13.2
 
 
 
 
 
F Mar 16 Discussion: PS3 and Midterm      
T Mar 20
 
Hidden Markov models
[lecture notes]
 
 
 
 
 
 
R Mar 22
 
 
Bayesian networks
 
 
Jordan draft 2.1
(primary reading)
 
Koller & Friedman, Probabilistic Graphical Models  
 
 
F Mar 23 Project office hours      
T Mar 27
 
 
Markov networks;
Inference (variable elimination)
 
Jordan draft 2.2;
Jordan draft Ch 3
(primary reading)
Koller & Friedman, Probabilistic Graphical Models PS4 due;
PS5 out
 
R Mar 29
 
 
 
 
 
 
Inference (message-passing algorithms);
Learning;
Topic models
 
 
 
 
Jordan draft 4.1, 4.3;
Review article on probabilistic topic models by Blei (Communications of the ACM, 2012)
(primary reading)
Koller & Friedman, Probabilistic Graphical Models
 
 
 
 
 
 
 
 
 
 
 
F Mar 30 Discussion: PS4      
Other Learning Frameworks
T Apr 3
 
 
Online learning
[lecture notes]
 
 
 
 
Cesa-Bianchi & Lugosi, Prediction, Learning, and Games  
 
 
R Apr 5
 
 
 
 
Semi-supervised learning;
Active learning
[lecture notes]
 
 
 
 
 
 
 
Zhu & Goldberg, Introduction to Semi-Supervised Learning;
Settles, Active Learning
 
 
 
 
 
F Apr 6 Project office hours      
T Apr 10
 
 
Reinforcement learning
 
 
Ng lecture notes Sec 1-3
(primary reading)
Sutton & Barto, Reinforcement Learning PS5 due;
PS6 out
 
Additional/Advanced Topics
R Apr 12
 
 
Structured prediction
[lecture notes]
 
 
 
 
Nowozin et al. (Eds), Advanced Structured Prediction  
 
 
F Apr 13 Discussion: PS5      
T Apr 17
 
 
 
 
 
 
Collaborative filtering
 
 
 
 
 
 
Review articles on recommender systems by Koren et al. (IEEE Computer, 2009) and Jannach et al. (Communications of the ACM, 2016)  
 
 
 
 
 
 
 
 
 
 
 
 
 
R Apr 19
 
 
 
 
 
 
 
 
 
Statistical consistency and PAC learning
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Devroye et al., A Probabilistic Theory of Pattern Recognition;
Kearns & Vazirani, An Introduction to Computational Learning Theory;
Penn courses
CIS 620 and CIS 625
 
 
 
 
 
 
 
 
 
 
F Apr 20
 
 
Project office hours (Meyerson Hall B1)
 
 
 
 
 
 
 
 
Project
spotlight
slides due
Home Stretch
T Apr 24
 
 
Project spotlight presentations
 
 
 
 
 
 
 
 
PS6 due;
Project
reports due
F Apr 27 Discussion: PS6 (Optional)      
F May 4 Final exam (9-11 AM, Huntsman Hall JMHH G06)