Notebooks
Table of contents
- Course syllabus and key points
- Preview: Solving spam classification with optimization
- Tentative course structure
- Expectations and learning outcomes
Course syllabus and key points
Welcome to STAT 4830: Numerical optimization for data science and machine learning. This course teaches you how to formulate optimization problems, select and implement algorithms, and use frameworks like PyTorch to build and train models. Below are some highlights of the syllabus to get you oriented:
Prerequisites
- Basic calculus and linear algebra (Math 2400).
- Basic probability (Stat 4300).
- Familiarity with Python programming.
- You do not need a background in advanced optimization or machine learning research. We’ll cover the fundamentals together.
Schedule and format
- This is primarily a lecture-based course that will also include frequent group meetings with the professor.
- We will introduce theory (convexity, gradient-based methods, constraints, etc.) and then apply it in Python notebooks using PyTorch (and occasionally other libraries like CVXPY).
Deliverables
- A single final project that you begin drafting by Week 3 and refine throughout the semester.
- Several “checkpoints” (drafts, presentations) so you can get feedback and improve incrementally.
- The final submission will consist of a GitHub repository (code, report, slides) plus a polished demonstration (e.g., a Google Colab notebook).
Why PyTorch?
- We are focusing on PyTorch because deep learning’s success has been driven in part by modern auto-differentiation frameworks.
- These frameworks allow for rapid experimentation with new model architectures and optimization algorithms—something that older solver-based tools (like CVX or early MATLAB packages) did not fully accommodate.
Who is this course for?
- Targeted at junior/senior undergrads, but also valuable for PhD students wanting to incorporate numerical optimization into their research. Students who have met the prerequisites are welcome to join.
- If you already have a research project that involves model fitting or data analysis, this course may deepen your toolkit and sharpen your understanding of optimization.
- We will keep refining the course content based on your interests. If you have a particular topic, domain, or application you’d like to see, let me know.
A Brief History of Optimization
EVOLUTION OF OPTIMIZATION
========================
Loading more...