Mathematical Principles of Machine Learning
Spring 2018
Mathematical Principles of Machine Learning aims to introduce some theoretical foundations of machine learning. The course is roughly divided into two parts: the statistical principles, and the optimization principles. For the former, we will focus on introducing basics of statistical learning theory, where the main focus is what can be learned and how well a machine can learn. For the latter, we will focus on algorithmic aspects of optimization, which play a key role in machine learning. Towards the end of the lecture, we will also cover representability of various learning models.
News
 [06.12] (Exercises)
Some exercises regarding optimization can be found in the end of Duchi.
 [06.12] (Reading assignment)
Week 16 [06.13]: Chapter 6.1  6.4 of Bubeck.
 [06.10] (Reading assignment)
Week 14, 15 [05.30, 06.06]: Chapter 3.3, 3.4, 3.7, 4.1, 4.2, and 4.3 of Bubeck.
 [05.27] (Reading assignment)
Week 13 [05.23]: Chapter 1.1 and 1.2 of Nesterov and Chapter 3.1 and 3.2 of Bubeck.
 [05.27] (Reading assignment)
Week 12 [05.16]: Lecture 4 V1.
 [05.26] (Homework)
A draft of Homework 3 is posted. Check V0. Due: 06.13 in class.
 [05.13] (Reading assignment)
Week 10, 11 [05.02, 05.09]: Lecture 4 V0 (up to Section 3).
 [05.09] (Homework)
Homework 2 is finalized anb posted. Check V1. Due: 05.16 in class.
 [04.30] (Reading assignment)
Week 9 [04.25]: Lecture 3 V1.
 [04.28] (Project)
You can find the logistics of the final project here
 [04.28] (Homework)
Parts of Homework 2 are posted. Check V0. It is due on 05.16 in class. The last problem will be posted by 05.04.
 [04.23] (Reading assignment)
Week 8 [04.18]: Lecture 3 V0 (up to Section 3).
 [04.15] (Homework)
HW1 P5 is modified. Check Lecture 2 V1.
 [04.15] (Reading assignment)
Week 7 [04.11]: Lecture 2 V1 appendix.
 [04.02] (Homework)
Homework 1 is finalized and posted. Check V2. Due: 04.11 in class.
 [04.01] (Homework)
Parts of Homework 1 are posted. Check V1. It is due on 04.11 in class. The last problem will be posted by 04.02.
 [04.01] (Reading assignment)
Week 4, 5 [03.21, 03.28]: Lecture 2 V0.
 [03.18] (Homework)
Parts of Homework 1 are posted. Check V0.
 [03.18] (Reading assignment)
Week 3 [03.14]: Lecture 1 V1 (up to Section 5).
 [03.11] (Reading assignment)
Week 2 [03.07]: Lecture 1 (up to Section 2).
 [02.26]
Welcome!
Logistics
 Instructor:
IHsiang Wang
(王奕翔),
ihwang AT ntu DOT edu DOT tw
 Office Location and Office Hours:
MD524 (明達館524室), Time: Tuesday 18:00  19:00.

Lecture Room and Lecture Hours:
MD231 (明達館231室), Time: Wednesday 14:20 – 17:20.
 Teaching Assistants:
ChungYi Lin (林宗毅), r05942127 AT ntu DOT edu DOT tw
Office hours: Monday, 15:30  16:30, BL524 (博理館524室)
WeiNing Chen (陳偉寧), r05942078 AT ntu DOT edu DOT tw
Office hours: Thursday, 14:30  15:30, BL524 (博理館524室)
Course Information
 References:
 S. ShalevShwartz and S. BenDavid, “Understanding Machine Learning: From Theory to Algorithms,” Cambridge University Press, 2014.
 Y. Nesterov, “Introductory Lectures on Convex Optimization: A Basic Course,” Kluwer Academic Publishers, 2004.
 M. Mohri, A. Rostamizadeh, and A. Talwalkar, “Foundations of Machine Learning,” the MIT Press, 2012.
 S. Boyd and L. Vandenberghe, “Convex Optimization,” Cambridge University Press, 2004.

Prerequisites:
Calculus; Linear algebra; Probability (mathematical maturity is required).

Grading:
Homework (40%), Exam (20%), Project (40%).
Lecture Materials

Lecture 0 Logistics and Course Information
Slide:
[03.07]
V0

Lecture 1 Overview of Statistical Learning Theory
Slide:
[03.07]
V0
[03.14]
V1
Note:
[03.11]
V0
[03.18]
V1

Lecture 2 Basics of Statistical Learning Theory
Slide:
[03.21]
V0
[03.28]
V1
Concentration
[04.15]
Link
Note:
[04.01]
V0
[04.15]
V1

Lecture 3 Uniform Convergence
Slide:
[03.28]
V0_draft
[04.17]
V0
[04.24]
V1
Note:
[04.23]
V0
[04.30]
V1

Lecture 4 Stability
Slide:
[05.02]
V0
[05.09]
V1
[05.16]
V2
Note:
[05.13]
V0
[05.27]
V1

Lecture 5 Optimization
Slide:
[05.23]
V0
[05.30]
V1
[06.06]
V2
[06.13]
V3
Note:
Homeworks

Homework 1 Due: 04.11 (in class)
Problems:
[03.18]
V0
[04.01]
V1
[04.02]
V2
Solution:

Homework 2 Due: 05.16 (in class)
Problems:
[04.28]
V0
[05.09]
V1
Solution:

Homework 3 Due: 06.13 (in class)
Problems:
[05.26]
V0
Solution:
Exam

Time 06.20 in class (14:20 – 17:20).

Range All lectures (except Week 16 (06.13)) and homeworks.

Logistics Policy.
Project
Resources