CS2680 Modern AI Systems: Agents and System Optimizations
Fall 2026

Announcement

  • Aug 17, 2026: Welcome to CS2680 Modern AI Systems: Agents and System Optimizations! Class meets Tuesday and Thursday; our first meeting is Thu Sep 3, 2026. The syllabus below is tentative and will be updated before the term begins.

Overview

This course examines the systems that make modern AI possible, and the systems that agents are now forcing us to rethink. We start with LLMs and agents themselves — using them, then building one — so that the rest of the course has something concrete to optimize. We then work down the stack that sits between a model and the hardware it runs on: GPU programming and kernels, distributed training and fine-tuning, data pipelines, and the serving path, where batching, KV-cache and prefix reuse, quantization, speculative decoding, and routing decide what inference actually costs. We close by turning the relationship around and asking what LLMs can do for systems research. At every layer the question is the same: where do the time and the money actually go?

Prerequisites: At least one of CS61, CS1610, CS2620. Comfort with Python and PyTorch is assumed; prior exposure to machine learning (e.g. CS1810) is recommended but not required.
Textbook: No textbook is required. Background reading: Operating Systems: Three Easy Pieces and the Ultra-Scale Playbook
Class time and location: Tuesday and Thursday, 11:15am – 12:30pm, SEC LL2.221
Format: The course runs in three parts.
  • Part I — Introduction to LLMs and Agents. Modern ML from a systems view, then agents from both a user's and a designer's perspective. Two hands-on assignments: build something with an agent, then build an agent.
  • Part II — Systems for LLMs. The bulk of the course: GPU programming, kernels, training and fine-tuning, data pipelines, and the serving stack — batching and scheduling, KV-cache and prefix caching, quantization, speculative decoding, routing, and agent serving.
  • Part III — LLMs for Systems. Turning the relationship around: using LLMs to do systems research.
Lectures carry the fundamentals; the research-heavy topics run as student-led paper discussions, marked Students in the schedule. Students are expected to actively participate. See the paper discussion page for what that involves.
Grading:
  • Project: 45%
  • Paper Presentation: 15%
  • Class Participation: 10%
  • Cache Competition: 10%
  • Assignment 1 (build something with an agent): 10%
  • Assignment 2 (build an agent): 10%
  • Bonus: up to 10% (course feedback 5%; problems current AI cannot solve, 2% each)
There is no exam in this course — the assignments, the project, and the paper discussions carry the assessment. See policy page for details.
Course website: TBD

Staff

Office hours:
  • Instructor: TBD
  • TA: TBD

Schedule

Tentative and subject to change. Class meets Tuesday and Thursday. Dates follow the FAS Fall 2026 calendar; the Thanksgiving recess and the last day of classes should be confirmed against the registrar's calendar. Rows with Students as the speaker are student-led paper discussions — see the paper discussion page. Readings marked required should be read before class; everything else is optional depth.

Week Date Speaker Topic Course Materials Notes
Part I — Introduction to LLMs and Agents
1 2026-09-03 Juncheng Yang Course overview and logistics notes; optional readings Transformer, scaling laws, sustainable AI
2 2026-09-08 Juncheng Yang Modern ML basics I: transformers and the training loop, from a systems view notes; optional readings Transformer, PyTorch, TensorFlow
2 2026-09-10 Juncheng Yang Modern ML basics II: inference, scaling, and where the cost actually is notes; required reading efficiently scaling inference; optional reading Chinchilla
3 2026-09-15 Juncheng Yang Agents from a user’s perspective notes; optional readings MemGPT, RAG Assignment 1 out (build something with an agent)
3 2026-09-17 Juncheng Yang Agents from a designer’s perspective notes; required reading Parrot; optional readings SGLang, Ray Assignment 2 out (build an agent)
Part II — Systems for LLMs
4 2026-09-22 Juncheng Yang LLM training and serving basics required reading Megatron-LM; optional reading vLLM docs Paper presentation sign-up opens
4 2026-09-24 Juncheng Yang GPU programming for MLSys I: architecture and the memory hierarchy optional readings CUDA guide, dissecting Volta, TPU
5 2026-09-29 Juncheng Yang GPU programming for MLSys II: Triton, tiling, and fusion optional readings Triton docs, Triton, CUDA matmul Assignment 1 due, 11:59pm
5 2026-10-01 Juncheng Yang Efficient LLM computing: GPU kernels required readings FlashAttention, roofline; optional readings FlashAttention-2, making DL go brrr Presentation paper choice due Oct 2
6 2026-10-06 Juncheng Yang Efficient and reliable LLM training and fine-tuning I: parallelism and ZeRO required reading ZeRO; optional readings FSDP, GPipe, PipeDream, Alpa Project proposal opens; form teams
6 2026-10-08 Juncheng Yang Efficient and reliable LLM training and fine-tuning II: memory, failures, and scale required reading Llama 3 (§3, infrastructure); optional readings MegaScale, DeepSeek-V3, ZeRO-Infinity, sublinear memory Assignment 2 due, 11:59pm
7 2026-10-13 Juncheng Yang Data pipelines optional readings CheckFreq, GEMINI, 3FS
7 2026-10-15 Juncheng Yang Efficient LLM serving: batching and scheduling I required reading vLLM / PagedAttention; optional reading Orca
8 2026-10-20 Juncheng Yang Efficient LLM serving: batching and scheduling II — disaggregation required reading Sarathi-Serve; optional readings DistServe, Splitwise
8 2026-10-22 Students Efficient LLM serving: KV-cache optimization Mooncake, InfiniGen, ring attention Project proposal due Oct 22, 11:59pm
9 2026-10-27 Students Efficient LLM serving: prefix cache SGLang / RadixAttention, Parrot, Mooncake Cache competition opens
9 2026-10-29 Juncheng Yang Efficient LLM serving: pruning and quantization I required reading LLM.int8(); optional readings SmoothQuant, mixed precision
10 2026-11-03 Students Efficient LLM serving: pruning and quantization II GPTQ, AWQ, SmoothQuant
10 2026-11-05 Students Efficient LLM serving: speculative decoding speculative decoding, Medusa
11 2026-11-10 Juncheng Yang Efficient LLM serving: routing and load balancing required reading Llumnix; optional readings AlpaServe, Clipper, GShard, Switch Transformer
11 2026-11-12 Juncheng Yang Efficient agent serving systems I required reading Parrot; optional reading SGLang
12 2026-11-17 Students Efficient agent serving systems II MemGPT, Ray, RAG, FAISS Cache competition closes Nov 17, 11:59pm
Part III — LLMs for Systems
12 2026-11-19 Students LLM for Systems Research I learned index, measure one level deeper Project checkpoint due Nov 19, 11:59pm
13 2026-11-24 Students LLM for Systems Research II MLPerf, Gavel, Pollux, Tiresias
13 2026-11-26 Thanksgiving recess — no class
14 2026-12-01 Students Final project presentations I Peer evaluation
14 2026-12-03 Students Final project presentations II Peer evaluation; final report due Dec 9, 11:59pm

Compute Resources

Project compute is expected to come from a combination of the following. Access instructions will be posted on the projects page once confirmed.