MLPR class notes
When learning advanced material, you won't immediately understand everything
just from reading notes. Please sign up to the
forum, ask questions, and share insights and external materials that you
Keyboard shortcut: step through the notes using the left and right arrow keys.
Please annotate the HTML versions of the notes in the forum, to keep the
class's comments together. You can show/hide
links to PDF versions.
- w0a – Course administration, html, pdf.
- w0b – Books useful for MLPR, html, pdf.
- w0c – MLPR background self-test, html, pdf. Answers: html, pdf.
- w0d – Maths background for MLPR, html, pdf.
- w0e – Programming in Python, html, pdf.
- w0f – Expectations and sums of variables, html, pdf.
- w0g – Notation, html, pdf.
Week 1: Introduction to ML with Linear Regression
- w1a – Course Introduction, html, pdf.
- w1b – Linear regression, html, pdf.
- w1c – Plotting and understanding basis functions, html, pdf.
- w1d – Linear regression, overfitting, and regularization, html, pdf.
Week 2: ML fundamentals: generalization, error bars, Gaussians
- w2a – Training, Testing, and Evaluating Different Models, html, pdf.
- w2b – Univariate Gaussians, html, pdf.
- w2c – The Central Limit Theorem, html, pdf.
- w2d – Error bars, html, pdf.
- w2e – Multivariate Gaussians, html, pdf.
Week 3: Classification and gradient-based fitting
- w3a – Classification: Regression, Gaussians, and pre-processing, html, pdf.
- w3b – Regression and Gradients, html, pdf.
- w3c – Logistic Regression, html, pdf.
Week 4: Bayesian linear regression
- w4a – Bayesian regression, html, pdf.
- w4b – Bayesian inference and prediction, html, pdf.
Week 5: Bayesian model choice and Gaussian processes
Week 6: More detailed models: Gaussian process kernels, more non-Gaussian regression
- w6a – Gaussian Processes and Kernels, html, pdf.
- w6b – Overview: fitting probabilistic models, html, pdf.
- w6c – Softmax regression, html, pdf.
- w6d – A robust logistic regression model, html, pdf.
Week 7: Reading week
Week 8: Neural Networks
- w8a – Neural networks introduction, html, pdf.
- w8b – Neural network architectures, html, pdf.
- w8c – Fitting and initializing neural networks, html, pdf.
- w8d – Backpropagation of Derivatives, html, pdf.
Week 9: Autoencoders, PCA, Netflix Prize
- w9a – Autoencoders and Principal Components Analysis (PCA), html, pdf.
- w9b – Netflix Prize, html, pdf.
Week 10: Bayesian logistic regression, Laplace approximation
- w10a – Bayesian logistic regression, html, pdf.
- w10b – The Laplace approximation applied to Bayesian logistic regression, html, pdf.
Week 11: Sampling-based approximate Bayesian inference, variational inference
A coarse overview of major topics covered is below. Some principles aren't
taught alone as they're useful in multiple contexts, such as gradient-based
optimization, different regularization methods, ethics, and practical choices
such as feature engineering or numerical implementation.
- Linear regression and ML introduction
- Evaluating and choosing methods from the zoo of possibilities
- Multivariate Gaussians
- Classification, generative and discriminative models
- Bayesian machine learning: linear regression, Gaussian processes and kernels
- Neural Networks
- Learning low-dimensional representations
- Approximate Inference: Bayesian logistic regression, Laplace, Variational
- Time allowing: applying what you know; other ways to combine and regularize models.
You are encouraged to write your own outlines and summaries of the course.
Aim to make connections between topics, and imagine trying to explain to someone
else what the main concepts of the course are.