Summary
CIS 520 provides a modern introduction to the mathematics, algorithms, and practice of machine learning.
It is designed for students who want to understand not only what machine learning algorithms do and how they can be used, but also the fundamental principles behind how and why they work.
After taking the course, students will:
 have a strong foundation in machine learning;
 be able to apply machine learning methods to reallife problems, as well as modify/design new algorithms where needed;
 be prepared for advanced coursework/research in machine learning and related fields.
Prerequisites
The course is designed for beginning graduate students and advanced undergraduate students, and requires the following background:
 Probability at the level of ESE 301/STAT 510/ENM 503
 Linear algebra at the level of EAS 205/MATH 312
 Basic familiarity with algorithms and computational complexity analysis (e.g. bigO notation)
 Experience with computer programming
Syllabus (Tentative)
The course will cover roughly the following set of topics. We will also discuss several application case studies throughout.

Supervised Learning:
 Generative and discriminative probabilistic classifiers (naive Bayes, logistic regression);
 Least squares regression
 Support vector machines
 Neural networks/deep learning
 Decision trees
 Nearest neighbor methods
 Boosting

Learning Theory Basics:
 Model selection and regularization
 Structural risk minimization

Unsupervised Learning:
 Principal component analysis (PCA)
 kmeans clustering; mixture models; expectationmaximization (EM) algorithm

Probabilistic Graphical Models:
 Hidden Markov models (HMMs)
 Bayesian networks
 Markov networks

Other Learning Frameworks:
 Online learning
 Semisupervised and active learning
 Reinforcement learning

Possible Additional Topics:
 Structured prediction
 Complex performance measures
 Statistical consistency
Textbooks
We will not be following a single textbook; lecture notes (and occasionally papers/other selected reading materials) will be made available online. However the following textbook is strongly recommended for additional reading (we will be assigning readings from it):
 C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
Additional textbooks:
 T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2nd Edition, 2009.
 R.O. Duda, P.E. Hart and D.G. Stork. Pattern Classification. WileyInterscience, 2nd Edition, 2000.
 T. Mitchell, Machine Learning. McGraw Hill, 1997.
Grading
 6 problem sets: 24%
 Class participation: 6%
 Midterm exam: 20%
 Project: 25%
 Final exam: 25%
Other Machine Learning Courses at Penn
If this course does not fit your needs, there are also a couple of other introductory machine learning courses at Penn that you can consider taking:
 CIS 419/519: This course provides a more applicationoriented introduction to machine learning, and is best suited for those interested in machine learning primarily as endusers. Note that CIS 419/519 and CIS 520 are alternative courses, and do not in general form a sequence.
 STAT 553: This course provides a more statisticallyoriented introduction to machine learning.
