Beginner's Guide to Tensors and Deep Learning

This page contains selected materials for beginners interested in the following areas:

* Tensor: from Linear Algebra to Multilinear Algebra

* Tensor implementations

* Big data analysis

* Deep learning and deep reinforcement learning

* Various applications

General Views for Machine Learning

* M. Jordan,  T. M. Mitchell. Machine learning: Trends, perspectives, and prospects. Science, 349, no. 6245, pp. 255-260, 2015.

* Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature 521.7553,  pp. 436-444, 2015.

General Views for Big Data Analysis

* R.G. Baraniuk. More is less: signal processing and the data deluge. Science, 331.6018, pp. 717-719, 2011.

* E. Papalexakis, F. Christos, N. D. Sidiropoulos. Tensors for data mining and data fusion: Models, applications, and scalable algorithms. ACM Transactions on Intelligent Systems and Technology (TIST), 8.2, 2017.

* X.-Y. Liu, X. Wang. LS-decomposition for robust recovery of sensory big data. IEEE Transactions on Big Data, 2017.

Tensor: from Linear Algebra to Multilinear Algebra

Linear Algebra

* Linear Algebra,  MIT Open Course by Prof. Gilbert Strang. 

* Matrix Computations, by Gene H. Golub and Charles F. Van Loan.

* The Matrix Cookbook, by Kaare Brandt Petersen, Michael Syskind Pedersen.

* YouTube videos for Essence of linear algebra

* BLAS (Basic Linear Algebra Subroutines)    BLAS Technical Forum  (PDF Documents)

  cuBLAS: CUDA Toolkit by NVIDIA

Tensor Basics

* T.G. Kolda, B. W. Bader. Tensor decompositions and applications. SIAM Review, 51.3,  pp. 455-500, 2009.

* Cichocki, A., Mandic, D., De Lathauwer, L., Zhou, G., Zhao, Q., Caiafa, C. and Phan, H.A. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. IEEE Signal Processing Magazine, 32(2), pp.145-163, 2015.

* Sidiropoulos, N.D., De Lathauwer, L., Fu, X., Huang, K., Papalexakis, E.E. and Faloutsos, C. Tensor decomposition for signal processing and machine learning. IEEE Transactions on Signal Processing, 65(13), pp.3551-3582, 2017.

Tensor Toolbox

* TensorLet in Python by our group.

* TensorLet in C++ by our group.

* TensorLet in CUDA: cuTensor by our group.

MATLAB Tensor Toolbox by Brett W. Bader, Tamara G. Kolda.

* TensorLab: A MATLAB package for tensor computations.

* TensorLy: Tensor Learning in Python.

Tensor Implementations

* N. Vervliet, O. Debals, L. De Lathauwer. November. Tensorlab 3.0—Numerical optimization strategies for large-scale constrained and coupled matrix/tensor factorization. In 50th Asilomar Conference on Signals, Systems and Computers, pp. 1733-1738, 2016.

Tensor Surveys

* Q. Song, H. Ge, J. Caverlee, X. Hu. Tensor completion algorithms in big data analytics. arXiv preprint arXiv:1711.10105, 2017.

Deep Learning and Deep Reinforcement Learning Basics

*  Deep LearningI. Goodfellow, Y. Bengio and A. Courville. An MIT Press book.

* Machine Learning Yearning. Andrew Ng.

AlphaGo  by Google DeepMind.

A Beginner’s Guide to Deep Reinforcement Learning

* cuDNNSharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, Evan Shelhamer. cuDNN: Efficient primitives for deep learning.

Tutorial on Hardware Architectures for Deep Neural Networks


Extended Readings

Alan Turing’s papers:

Turing, A.M., 2009. Computing machinery and intelligence. In Parsing the Turing Test (pp. 23-65). Springer, Dordrecht.

Turing, A.M., 1937. On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London mathematical society2(1), pp.230-265.

Turing, A.M., 1995. Computing machinery and intelligence. Brian Physiology and Psychology213.

Turing, A.M., 1954. Solvable and unsolvable problems. Penguin Books.

Recent Developments in AI

* V. Mnih, K. Kavukcuoglu, D. Silver, A.A. Rusu, J. Veness, M.G.  Bellemare, A. Graves, M. Riedmiller, A.K. Fidjeland, G. Ostrovski, S. Petersen. Human-level control through deep reinforcement learning. Nature518(7540), p.529, 2015.

* D. Silver, A.  Huang, C.J.  Maddison, A. Guez, A, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M.  Lanctot, S. Dieleman. Mastering the game of Go with deep neural networks and tree search. Nature529(7587), pp. 484, 2016.

* D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez,  T. Hubert, L. Baker, M. Lai, A. Bolton, Y. Chen. Mastering the game of Go without human knowledge.  Nature550(7676), p.354, 2017.

A good blog on NLP

Writing, Revising and Polishing Your Manuscripts

* To polish AI/ML/DL manuscripts that target at conferences (NIPS, CVPR, AAAI, ICML, ICLR, IJCAI, ACL, etc.),  please check review comments on Open Review

* Interesting Papers

Heisenberg’s invention of matrices. Pradeep Kumar, 2017.

* Videos for Animated math  

  a. Neural networks

  b. Fourier transform

  c. Visualizing high dimensional sphere

Close Menu