Syllabus and Textbooks

The readings will be from select chapters of the following freely-available textbooks and lecture notes:

• Probabilistic Machine Learning by Kevin P. Murphy. MIT Press, 2023.

• Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. MIT Press, Second Edition, 2018.

• Deep Learning: Foundations and Concepts by Christopher Bishop and Hugh Bishop. Springer Nature, 2023.

Syllabus

W1. Introduction to ML

• Machine Learning definition, types, and applications

• K-Nearest Neighbors (KNNs)

• Course administration

• Example: KNN example

W2. Naïve Bayes and Logistic Regression

• Probabilistic classification

• Estimating probability distributions

• Contrasting Naïve Bayes (NB) and Logistic Regression

• Example: NB example

• Due: KNN

W3. Stochastic Gradient Descent (SGD)

• Convexity and optimization

• Step size selection

• Regularized conditional log-likelihood

• Example: SGD for Logistic Regression example

W4. Data Engineering and PAC learnability

• Consistent hypotheses and finite spaces

• Connection to Interval Learning

• Example: Californian in Boston

• Due: Logistic Regression I

W5. Support Vector Machines (SVMs)

• Vector space models and linear classifiers

• Theoretical guarantees of SVMs

• Slack variables in SVMs

• Kernel SVMs

• Example: Slack example

• Due: Logistic Regression II

W6. Boosting

• Training error and weak learners

• Gradient boosting

• Example: Boosting example

W7. Regression

• Linear regression

• Regularization techniques

• Example: Fitting a linear regression example

• Example: Predicting MPG example

• Due: SVM

• Midterm Review: Weeks 1–8

W8. Clustering I and Midterm

• k-Means

• Example: k-Means example

• Midterm: Assessing knowledge from Weeks 1–8

W9. Clustering II and Proposal Presentations

• Gaussian Mixture Models

• Limitations of k-means and GMMs

• Project: Proposal presentation: initial feedback and guidance

• Due: project proposal & slides

W10. Structured Perceptron and Multilayer Networks

• Online learning % seperate this from structured prediciton maybe

• Structured Perceptron

• Activation and objective functions

• Backpropigation algorithm

• Example: POS Example

• Example: Grid example

• Due: Boosting

W11. Fairness

• Introduction to fairness metrics and bias mitigation

• Debiasing techniques

• Example: Pneumonia example

W12. Training Paradigms

• Introduction to curriculum learning

• Sample difficulty and curriculum design

• Applications and use cases

• Performance metrics

• Example: SVMs convergence and generalization example

W13. Machine Unlearning

• Introduction to machine unlearning

• Algorithmic approaches

• Applications and use cases

• Performance metrics

• Example: User account unlearning example

• Due: Regression

W14. Final Project Presentations and Reports

• Project: Final presentation

• Due: Final report, project slides