Machine Learning and Statistical Learning Theory 1
Instructor: He Wang
he.wang@northeastern.edu
All course materials (homework/labs/tests/etc.) are on Canvas.
Lecture Notes:
1 Introduction Math7243Sec1Introduction
2 Linear Regressions, general least squares and matrix calculus.Math7243Sec2LinearReg
3 Locally weighted linear regression, Ridge and LASSO RegressionsMath7243Sec3RidgeLASSOReg
4 Statistics of linear regression, subset selection, Gauss-Markov Theorem Math7243Sec4StatisticsML
5 Gradient Descents, Newton’s methods, Descent with momentum, ADAM Math7243Sec5GrdientDescent
6. Logistics Regression, SoftMax Math7243Sec6LogReg
7 Gaussian Discriminant Analysis, LDA, QDA Math7243Sec7GDA_LDA
8 Resampling, Bias and Variance Tradeoff, Cross Validations, Bootstrap Math7243Sec8Resampling
9 Naive Bayes, Spam email classification Math7243Sec9NaiveBayes
10 Perceptron, Neural Network, Backpropagation Math7243Sec10ANN and Math7243Sec10BackPropagation
11 Convolutional Neural Network Math7243Sec11CNN
12 Recurrent Neural Network Math7243Sec12RNN
13 Support Vector Machine and Kernel methods Math7243Sec13SVM
14 k-Nearest Neighborhoods Math7243Sec14kNN
15 Tree-based Methods for regression and classification, bagging, random forest, Boosting Math7243Sec15TreeMethods
16 Clustering (K-means, Hierarchical, Density based, etc.) Math7243Sec16Clustering
17 PCA and SVD Math7243Sec17PCA and Math7243Sec17SVD
18 Mixture of Gaussians Math7243Sec18EM
19 Topological Data Analysis-introduction Math7243Sec19TDA
There are a few assignments and computer labs in Python or Matlab.
There are industry Experimental Network (XN) final projects for the course
- XN project with the partner company, Zeta Surgical (https://www.zetasurgical.com/ ).
XN project 1: Brain CT image hemorrhage segmentation
- XN project with Merck Research Laboratories:
XN project 2: Find Novel Intrinsic Oncology Targets and Biology
References:
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani, Jerome Friedman
- Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems – by Aurélien Géron
- Pattern Recognition and Machine Learning by Christopher Bishop
- Linear algebra and learning from data by Gilbert Strang
- Machine Learning: A Probabilistic Perspective by Kevin P. Murphy
- Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz, Shai Ben-David
Recent Comments