Mathematical Principles of Machine Learning
Spring 2018

Mathematical Principles of Machine Learning aims to introduce some theoretical foundations of machine learning. The course is roughly divided into two parts: the statistical principles, and the optimization principles. For the former, we will focus on introducing basics of statistical learning theory, where the main focus is what can be learned and how well a machine can learn. For the latter, we will focus on algorithmic aspects of optimization, which play a key role in machine learning. Towards the end of the lecture, we will also cover representability of various learning models.

• [06.12] (Exercises)
  Some exercises regarding optimization can be found in the end of Duchi.
• [06.12] (Reading assignment)
  Week 16 [06.13]: Chapter 6.1 - 6.4 of Bubeck.
• [06.10] (Reading assignment)
  Week 14, 15 [05.30, 06.06]: Chapter 3.3, 3.4, 3.7, 4.1, 4.2, and 4.3 of Bubeck.
• [05.27] (Reading assignment)
  Week 13 [05.23]: Chapter 1.1 and 1.2 of Nesterov and Chapter 3.1 and 3.2 of Bubeck.
• [05.27] (Reading assignment)
  Week 12 [05.16]: Lecture 4 V1.
• [05.26] (Homework)
  A draft of Homework 3 is posted. Check V0. Due: 06.13 in class.
• [05.13] (Reading assignment)
  Week 10, 11 [05.02, 05.09]: Lecture 4 V0 (up to Section 3).
• [05.09] (Homework)
  Homework 2 is finalized anb posted. Check V1. Due: 05.16 in class.
• [04.30] (Reading assignment)
  Week 9 [04.25]: Lecture 3 V1.
• [04.28] (Project)
  You can find the logistics of the final project here
• [04.28] (Homework)
  Parts of Homework 2 are posted. Check V0. It is due on 05.16 in class. The last problem will be posted by 05.04.
• [04.23] (Reading assignment)
  Week 8 [04.18]: Lecture 3 V0 (up to Section 3).
• [04.15] (Homework)
  HW1 P5 is modified. Check Lecture 2 V1.
• [04.15] (Reading assignment)
  Week 7 [04.11]: Lecture 2 V1 appendix.
• [04.02] (Homework)
  Homework 1 is finalized and posted. Check V2. Due: 04.11 in class.
• [04.01] (Homework)
  Parts of Homework 1 are posted. Check V1. It is due on 04.11 in class. The last problem will be posted by 04.02.
• [04.01] (Reading assignment)
  Week 4, 5 [03.21, 03.28]: Lecture 2 V0.
• [03.18] (Homework)
  Parts of Homework 1 are posted. Check V0.
• [03.18] (Reading assignment)
  Week 3 [03.14]: Lecture 1 V1 (up to Section 5).
• [03.11] (Reading assignment)
  Week 2 [03.07]: Lecture 1 (up to Section 2).
• [02.26]
  Welcome!

• Instructor:
  I-Hsiang Wang (王奕翔), ihwang AT ntu DOT edu DOT tw
• Office Location and Office Hours:
  MD-524 (明達館524室), Time: Tuesday 18:00 - 19:00.
• Lecture Room and Lecture Hours:
  MD-231 (明達館231室), Time: Wednesday 14:20 – 17:20.
• Teaching Assistants:
  Chung-Yi Lin (林宗毅), r05942127 AT ntu DOT edu DOT tw
    Office hours: Monday, 15:30 - 16:30, BL-524 (博理館524室)
  Wei-Ning Chen (陳偉寧), r05942078 AT ntu DOT edu DOT tw
    Office hours: Thursday, 14:30 - 15:30, BL-524 (博理館524室)

• References:
  
  1. S. Shalev-Shwartz and S. Ben-David, “Understanding Machine Learning: From Theory to Algorithms,” Cambridge University Press, 2014.
  2. Y. Nesterov, “Introductory Lectures on Convex Optimization: A Basic Course,” Kluwer Academic Publishers, 2004.
  3. M. Mohri, A. Rostamizadeh, and A. Talwalkar, “Foundations of Machine Learning,” the MIT Press, 2012.
  4. S. Boyd and L. Vandenberghe, “Convex Optimization,” Cambridge University Press, 2004.
• Prerequisites:
  Calculus; Linear algebra; Probability (mathematical maturity is required).
• Grading:
  Homework (40%), Exam (20%), Project (40%).

• Lecture 0 Logistics and Course Information
    Slide: [03.07] V0
• Lecture 1 Overview of Statistical Learning Theory
    Slide: [03.07] V0 [03.14] V1
    Note: [03.11] V0 [03.18] V1
• Lecture 2 Basics of Statistical Learning Theory
    Slide: [03.21] V0 [03.28] V1
    Concentration [04.15] Link
    Note: [04.01] V0 [04.15] V1
• Lecture 3 Uniform Convergence
    Slide: [03.28] V0_draft [04.17] V0 [04.24] V1
    Note: [04.23] V0 [04.30] V1
• Lecture 4 Stability
    Slide: [05.02] V0 [05.09] V1 [05.16] V2
    Note: [05.13] V0 [05.27] V1
• Lecture 5 Optimization
    Slide: [05.23] V0 [05.30] V1 [06.06] V2 [06.13] V3
    Note:

• Homework 1 Due: 04.11 (in class)
    Problems: [03.18] V0 [04.01] V1 [04.02] V2
    Solution:
• Homework 2 Due: 05.16 (in class)
    Problems: [04.28] V0 [05.09] V1
    Solution:
• Homework 3 Due: 06.13 (in class)
    Problems: [05.26] V0
    Solution:

• Time 06.20 in class (14:20 – 17:20).
• Range All lectures (except Week 16 (06.13)) and homeworks.
• Logistics Policy.

• Logistics [04.28] V0
• Report [06.25] Template