Course Description


Summary

CIS 520 provides a modern introduction to the mathematics, algorithms, and practice of machine learning. It is designed for students who want to understand not only what machine learning algorithms do and how they can be used, but also the fundamental principles behind how and why they work. After taking the course, students will:
  • have a strong foundation in machine learning;
  • be able to apply machine learning methods to real-life problems, as well as modify/design new algorithms where needed;
  • be prepared for advanced coursework/research in machine learning and related fields.

Prerequisites

The course is designed for beginning graduate students and advanced undergraduate students, and requires the following background:
  • Probability at the level of ESE 301/STAT 510/ENM 503
  • Linear algebra at the level of EAS 205/MATH 312
  • Basic familiarity with algorithms and computational complexity analysis (e.g. big-O notation)
  • Experience with computer programming

Syllabus (Tentative)

The course will cover roughly the following set of topics. We will also discuss several application case studies throughout.
  • Supervised Learning:
    • Generative and discriminative probabilistic classifiers (naive Bayes, logistic regression);
    • Least squares regression
    • Support vector machines
    • Neural networks/deep learning
    • Decision trees
    • Nearest neighbor methods
    • Boosting
  • Learning Theory Basics:
    • Model selection and regularization
    • Structural risk minimization
  • Unsupervised Learning:
    • Principal component analysis (PCA)
    • k-means clustering; mixture models; expectation-maximization (EM) algorithm
  • Probabilistic Graphical Models:
    • Hidden Markov models (HMMs)
    • Bayesian networks
    • Markov networks
  • Other Learning Frameworks:
    • Online learning
    • Semi-supervised and active learning
    • Reinforcement learning
  • Possible Additional Topics:
    • Structured prediction
    • Complex performance measures
    • Statistical consistency

Textbooks

We will not be following a single textbook; lecture notes (and occasionally papers/other selected reading materials) will be made available online. However the following textbook is strongly recommended for additional reading (we will be assigning readings from it):
  • C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
Additional textbooks:
  • T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, 2nd Edition, 2009.
  • R.O. Duda, P.E. Hart and D.G. Stork. Pattern Classification. Wiley-Interscience, 2nd Edition, 2000.
  • T. Mitchell, Machine Learning. McGraw Hill, 1997.

Grading
  • 6 problem sets: 24%
  • Class participation: 6%
  • Midterm exam: 20%
  • Project: 25%
  • Final exam: 25%

Other Machine Learning Courses at Penn

If this course does not fit your needs, there are also a couple of other introductory machine learning courses at Penn that you can consider taking:
  • CIS 419/519: This course provides a more application-oriented introduction to machine learning, and is best suited for those interested in machine learning primarily as end-users. Note that CIS 419/519 and CIS 520 are alternative courses, and do not in general form a sequence.

  • STAT 553: This course provides a more statistically-oriented introduction to machine learning.