Some resources to start with Fundamentals of Machine Learning

With a number of courses, books and reading material out there here is a list of some which I personally find useful for building a fundamental understanding in Machine Learning.

Machine Learning at a higher level requires some mathematical prerequisites which are at the heart of it.

1. Learning Theory
2. Optimization
3. Statistical learning and high dimensional probability theory

Some really nice resources might be the ones below

1. Learning Theory
   1. Learning from Data - Caltech.
   2. The initial chapters from Foundations of Machine Learning - Mohri, or Part I from Understanding Machine Learning From Theory to Algorithms - Shai Shalev-Shwartz and Shai Ben-David.
2. Optimization for Machine Learning
   1. Large scale optimization for Machine Learning - Talks by Suvrit Sra - Part 1, Part 2, and Part 3 - Slides.
   2. Convex Optimization literature - Convex Optimization course by Stephen Boyd Slides, and the classical book on Introductory Lectures on Convex Programming - Yuri Nesterov.
   3. Non-convex Optimization for Machine Learning - Jain and Kar.
   4. OPTML++ page by Suvrit Sra.
3. Statistical Learning and Probabilistic Machine Lerning
   1. Introduction to Statistical Learning - Trevor Hastie and Robert Tibshirani - Introductory Book, Advanced Book.
   2. Machine Learning: A Probabilistic Perspective - Kevin P Murphy.