(Tentative schedule for the course and deviations may be necessary as the course progresses.)
| Date |
Topic |
Notes |
Reading (Textbook or Other Materials) |
| Jan. 10 |
Introduction |
|
RL: Chapter 1, Nature, Basic calculus refresher |
| Jan. 12 |
Machine Learning Basics |
|
Machine learning math essentials |
| Jan. 17 |
Sequential Decision Making |
|
RL: Chapter 1 |
| Jan. 19 |
Markov Decision Processes (MDP) |
Project Details |
RL: Chapter 3 |
| Jan. 24 |
MDP Bellman Equations |
|
RL: Chapter 3 |
| Jan. 31 |
Policy Iteration, Value Iteration |
|
RL: Chapter 4 |
| Feb. 2 |
Tutorial on Numpy and PyTorch |
|
|
| Feb. 7 |
Model-free Prediction: MC vs. TD |
|
RL: Chapter 5,6,7 |
| Feb. 9 |
Model-free Control: SARSA, Q-Learning |
|
RL: Chapter 6 |
| Feb. 14 |
Function Approximation: Linear Classifiers |
|
RL: Chapter 9 |
| Feb. 16 |
Computational Graphs, Backpropagation |
|
DL: Chapter 8 |
| Feb. 21 |
Neural Networks, MLP |
|
|
| Feb. 23 |
Convolutional Neural Networks |
|
AlexNet, ResNet |
| Feb. 28 |
Training Neural Networks |
|
|
| Mar. 2 |
Recurrent Nerual Networks, LSTM |
|
|
| Mar. 7 |
Transformer |
|
Attention is all you need, tutorial |
| Mar. 9 |
Deep Q Network (DQN) |
|
|
| Mar. 21 |
Policy Gradient |
|
|
| Mar. 23 |
Actor-Critic |
Homework4 |
|
| Mar. 28 |
Sparse Reward |
|
Faulty reward, HER |
| Mar. 30 |
Imitation Learning |
|
IRL |
| Apr. 4 |
Model-based RL |
|
|
| Apr. 6 |
AlphaGo, AlphaGo Zero |
Notes on final exam and presentation |
Nature |
| Apr. 11 |
Review(office hours) |
|
|
| Apr. 13 |
Final Exam |
|
|
| Apr. 18 |
Presentation |
|
|
| Apr. 20 |
Presentation |
|
|
| Apr. 25 |
Presentation |
|
|
| Apr. 28 |
Project Report Due (by 11:59 pm) |
|
|