Course Description

Generative models are widely used in many subfields of AI and Machine Learning. Recent advances in parameterizing these models using deep neural networks, combined with progress in stochastic optimization methods, have enabled scalable modeling of complex, high-dimensional data including images, text, and speech. In this course, we will study the probabilistic foundations and learning algorithms for deep generative models, including variational autoencoders, generative adversarial networks, autoregressive models, normalizing flow models, energy-based models, and score-based models. The course will also discuss application areas that have benefitted from deep generative models, including computer vision, speech and natural language processing, graph mining, reinforcement learning, reliable machine learning, and inverse problem solving.

Lecture Attendance

While we do not require in-person lecture attendance, students are encouraged to join the live lecture. To accommodate various circumstances, lecture recordings will also be available on Canvas shortly following the lecture.

Stanford Honor Code

Students are free to form study groups and may discuss homework in groups. However, each student must write down the solutions and code from scratch independently, and without referring to any written notes from the joint session. In other words, each student must understand the solution well enough in order to reconstruct it by him/herself. It is an honor code violation to copy, refer to, or look at written or code solutions from a previous year, including but not limited to: official solutions from a previous year, solutions posted online, and solutions you or someone else may have written up in a previous year. Furthermore, it is an honor code violation to post your assignment solutions online, such as on a public git repo.

The Stanford Honor Code can be found here

The Stanford Honor Code pertaining to CS courses can be found here

Time & Location

Fall Quarter: Sept. - Dec., 2023 Lecture: Monday, Wednesday 4:30 PM - 5:50 PM Location: Gates B1

Office Hours

You can find an up-to-date list of times and locations here.

Grade Breakdown

• Three Homeworks: 15% each

• Midterm: 15%

• Course Project: 40%

• Proposal: 5%

• Progress Report: 10%

• Poster Presentation: 10%

• Final Report: 15%

Course Discussions

We use Ed for course communication.

Assignment Details

See here for more details concerning assignments.

Course Project Details

See here for more details concerning the course project.

FAQ

What are the pre-requisites?

Basic knowledge about machine learning from at least one of: CS 221, 228, 229 or 230.

Basic knowledge of probabilities and calculus: students will work with computational and mathematical models.

Proficiency in a programming language: preferably Python.

Can I audit or sit in?

In general we are very open to sitting-in guests if you are a member of the Stanford community (registered student, staff, and/or faculty). Out of courtesy, we would appreciate that you first email us or talk to the instructor after the first class you attend. If the class is too full and we’re running out of space, we would ask that you please allow registered students to attend.

Is there a textbook for this course?

We offer our own self-contained notes for this course. While there is no required textbook, we recommend "Deep Learning" by Ian Goodfellow, Yoshua Bengio, Aaron Courville. The online version available for free here.

Academic Accomodations

If you need an academic accommodation based on a disability, please register with the Office of Accessible Education (OAE). Professional staff will evaluate your needs, support appropriate and reasonable accommodations, and prepare an Academic Accommodation Letter for faculty. To get started, or to re-initiate services, please visit oae.stanford.edu.

If you already have an Academic Accommodation Letter, please make a private Ed post. OAE Letters should be sent to us at the earliest possible opportunity so that the course staff can partner with you and OAE to make the appropriate accommodations.

Acknowledgments. HTML taken from various CS courses given at Stanford: cs231n, cs231a, and cs229.