Lecture location: MCHU 206
Lecture times: TT 3:30-4:45 PM

Class Schedule

Course Description

This course introduces the basic concepts and algorithms of deep neural networks (DNNs) and its applications to computer vision and natural language processing. DNN architectures, such as MLP, CNN, LSTM, Transformer, etc., will be presented. Supervised learning and unsupervised (or self-supervised) learning will be discussed. Depending on the course progress, selected topics such as generative models and model compression will be covered. The class emphasizes the understanding of the state-of-the-art DL architectures as well as practical implementations of DNNs with Python.

• Background of artificial intelligence, machine learning, and deep learning will be discussed at the beginning.
• We will then learn the basics of neural networks and feed-forward neural networks, from shallow models to deep models.
• Optimization and regularization techniques are critical in building powerful deep neural networks. We will cover different optimization and regularization strategies, such as Adam, Dropout, BatchNorm, Initiations, and so on.
• Convolutional Neural Network (CNN) is one of the most important deep learning architectures. We will study CNN and its applications in the context of computer vision.
• Recurrent Neural Network (RNN) is another critical deep learning architecture that can process sequential data. We will learn RNN and its applications in the context of natural language processing.
• Unsupervised Autoencoder is one of powerful dimensionality reduction techniques and self-supervised learning algorithms. We will study its applications in unsupervised feature extraction.
• Attention Mechanism has been widely used in recent deep learning techniques. We will study the general attention mechanism and recent advances including multi-head attention, transformer, and so on.
• Depending on the course progress, selected topics such as generative models, graph neural networks, or model compression, etc. will be briefly discussed.
• Frameworks: We recommend using the popular deep-learning frameworks (e.g., PyTorch or TensorFlow) for course projects.

Textbook

Deep Learning, Ian Goodfellow, Aaron Courville, and Yoshua Bengio, MIT Press, 2016

Prerequisites

• This should not be your first machine learning class! You should have taken CSE 4820 or 5819 Machine Learning with a B or above.
• Proficiency in Python: All HWs will be in Python. You should have taken CSE 1010 Python Programming.
• Basic knowledge in calculus, linear algebra, and statistics.

Course Objectives

• The core and advanced methodologies, algorithms, and tools of deep learning techniques.
• The skills of implementing and applying deep learning algorithms and models.
• Ideally, developing deep learning algorithms and models to solve real-world problems.

Grading

• No late submissions accepted! All the assignment and project deadlines are 11:59 pm (EST) of the due dates.
• If you have doubts in your grading, please email TA and CC to the Instructor indicating the reason why you think it should be regraded.
• The regrading request should be submitted within 1 week after you receive your score.

Misc

• Course materials, assignments, projects and QAs are managed by HuskyCT.

Student Responsibilities and Resources

• The Student Code, Academic Integrity, Resources on Avoiding Cheating and Plagiarism
• Copyrighted Materials
• Netiquette and Communication
• Adding or Dropping a Course
• Academic Calendar
• Policy Against Discrimination, Harassment, and Inappropriate Romantic Relationships
• Sexual Assault Reporting