This course covers the mathematical aspects of data science, including aspects of clustering, compression, classification, and data completion, along with big-data algorithms like alternating minimization and ADMM. This course hopes to give technical knowledge of many recent concepts in data science which would be useful in further fundamental research in the area.
| Lecture Date | Topic | Reference Material |
| 8/23/2016 |
Introduction, Principal Component Analysis in High Dimensions, Classification |
Trevor Hastie, Robert Tibshirani, and Jerome Friedman, "The Elements of Statistical Learning - Data Mining, Inference, and Prediction," Springer Series in Statistics, Second Edition, 2009. |
| 8/30/2016 |
Regression and k-means Clustering |
Trevor Hastie, Robert Tibshirani, and Jerome Friedman, "The Elements of Statistical Learning - Data Mining, Inference, and Prediction," Springer Series in Statistics, Second Edition, 2009. |
| 9/6/2016 |
Spectral Clustering |
Ulrike von Luxburg, "A Tutorial on Spectral Clustering," arXiv:0711.0189, Nov. 2007 |
| 9/13/2016 |
Lecture by C. Quinn on Directed Information Graphs Concentration Inequalities, Scalar and Matrix Versions |
C. J. Quinn, N. Kiyavash, and T. P. Coleman, “Efficient methods to compute optimal tree approximations of directed information graphs,” Signal Processing, IEEE Transactions on, vol. 61, no. 12, pp. 3173-3182, 2013. |
| 9/20/2016 |
Johnson-Lindenstrauss Lemma and Gordons Theorem Applications of PCA for prediction |
Eric Bair, Trevor Hastie, Debashis Paul, and Robert Tibshirani, "Prediction by Supervised Principal Components," Journal of the American Statistical Association, 2006 |
| 9/27/2016 |
Applications of PCA for Anomoly Detection Tensors |
Y. J. Lee, Y. R. Yeh and Y. C. F. Wang, "Anomaly Detection via Online Oversampling Principal Component Analysis," in IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 7, pp. 1460-1470, July 2013. Wolfgang Hackbusch and Reinhold Schneider, "Tensor Spaces and Hierarchical Tensor Representations," Chapter 12 of Extraction of Quantifiable Information from Complex Systems, Sept 2014. S. Holtz, T. Rohwedder, and R. Schneider, "On Manifolds of Tensors of Fixed TT-Rank," Numerische Mathematik, 2011. A. Cichocki, N. Lee, I.V. Oseledets, A-H. Phan, Q. Zhao, D. Mandic, "Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1" srXiv, Sept 2016 |
| 10/4/2016 |
Tensors (Contd.) PCA for low TT-rank tensors Dictionary Learning for Tensor Data |
W. Wang, V. Aggarwal, and S. Aeron, in preparation. Z. Shakeri, W. U. Bajwa, and A. D. Sarwate, "Minimax Lower Bounds on Dictionary Learning for Tensor Data," arXiv:1608.02792v2, Aug 2016. |
| 10/18/2016 |
Alternating Minimization Alternating Direction Method of Multipliers |
A. Beck, "On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes", SIAM J. Optimization, 2015 W. Deng, M. Lai, Z. Peng, and W. Yin, "Parallel Multi-block ADMM with o(1/k) convergence" |
| 10/25/2016 | In-Network Nonconvex Large-scale Optimization | P. D. Lorenzo and G. Scutari, “NEXT: In-Network Nonconvex Optimization,” IEEE Trans. on Signal and Information Processing over Networks, vol. 2, no. 2, pp. 120-136, June 2016. |
| 11/1/2016 |
Parallel Direction Method of Multipliers Stochastic Alternate Direction Method |
H. Wang, A. Banerjee, and Z. Luo, "Parallel Direction Method of Multipliers," NIPS 2014 Shen-Yi Zhao, Wu-Jun Li, Zhi-Hua Zhou, "Scalable Stochastic Alternating Direction Method of Multipliers," Jul 2015 |
| 11/8/2016 |
Matrix Completion Matrix Completion using Alternating Minimization |
E. J. Candes and B. Recht, "Exact Matrix Completion via Convex Optimization," Foundations of Computational Mathematics, 2009. P. Jain, P. Netrapalli, and S. Sanghavi, "Low-rank matrix completion using alternating minimization," STOC 2013 R. Ge, J. D. Lee, and T. Ma, "Matrix Completion has No Spurious Local Minimum," arXiv, May 2016. W. Wang, V. Aggarwal, and S. Aeron, "Tensor Completion by Alternating Minimization under the Tensor Train (TT) Model," Submitted to IEEE Transactions on Signal Processing, Sept 2016. |
| 11/15/2016 |
Deterministic Sampling Patterms for Matrix Completion Algebraic Combinatorial Approaches to Matrix Completion |
D. Pimentel, N. Boston, and R. Nowak, "A Characterization of Deterministic Sampling Patterns for Low-Rank Matrix Completion," IEEE Journal on Special Topics in Signal Processing, 2016. Franz J.Kiraly, Louis Theran, and Ryota Tomioka,"The Algebraic Combinatorial Approach for Low-Rank Matrix Completion," Journal of Machine Learning Research, 2015. M. Ashraphijuo, V. Aggarwal, and X. Wang, "Deterministic Sampling Patterns for Tensor Completion," in preparation. |
| 11/22/2016 |
Matrix Completion (Contd.) Fundamental limits of Matrix Completion |
D. Y. Li, K. Lee, and Y. Bresler, "Optimal Sample Complexity for Stable Matrix Recovery," ISIT 2016 C. Suh, "Information Theory of Matrix Completion," ISIT 2014 V. Aggarwal and S. Aeron, "A note on Information-theoretic Bounds on Matrix Completion under Union of Subspaces Model," in Proc. Allerton, Sept 2016. |
| 11/29/2016 |
Sparse Subspace Clustering: Algorithm, Theory, and Applications Analysis of Subspace Clustering with Outliers
|
E. Elhamifar and R. Vidal, "Sparse Subspace Clustering: Algorithm, Theory, and Applications," IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013 M. Soltanolkotabi and E. J. Candes, "A Geometric Analysis of Subspace Clustering with Outliers," Annals of Statistics, 2012. |
| 12/6/2016 |
Abstract Algebraic Subspace Clustering Deterministic Conditions for Subspace Clustering under Missing Data |
M. C. Tsakiris and R. Vidal, “Abstract algebraic subspace clustering,” arXiv v2, Mar 2016.
W. Wang, S. Aeron, and V. Aggarwal,"On deterministic conditions for subspace clustering under missing data," ISIT 2016. |
Information regarding scribe can be seen by clicking here.
Course Grading:
Scribe:15%
Class Participation: 15%
Paper presentation: 30%
Final Project: 40%