BST 235: Advanced Regression and Statistical Learning
Lecture Notes [PDF]
Syllabus
This course is an advanced course on the methods and theory of high dimensional statistics, statistical machine learning, and large-scale inference and optimization. We aim to quickly bring students to the frontier and interdisciplinary areas of statistics, optimization, probability, and machine learning. We are going to cover the topics in:
- Concentration inequality, Sub-Gaussian random variables, Chernoff bounds, Hoeffding's inequality, Maximal inequalities
- Ordinary least square, Compressed sensing, Lasso, Variations of Lasso including group lasso, fused lasso, adaptive lasso, etc., General high dimensional M- estimators, Variable selection consistency
- Convex geometry, Lagrange duality, Gradient descent, Proximal gradient descent, LARS, ADMM, Mirror descent, Stochastic optimization
- Linear model hypothesis testing, high dimensional inference, Chi-square test, maximal test, Higher criticism, False discovery rate control
- Principal component analysis, Nuclear norm, Matrix regression, Spectral Methods
BST 263: Statistical Learning
Lecture Notes [PDF]
Syllabus
This course introduces statistical machine learning under the big umbrella of data, sincere and modern analytics. More detailed topics include:
- OLS, Lasso, Ridge, Elastic-Net, Tree based methods, Random Forests
- Logistic regression, Support vector machine, Linear discriminant analysis, Naive Bayes, Boosting
- Mixture of Gaussians, Expectation maximization algorithm, K-means
- Deep learning, Reinforcement learning