CS2680 Modern AI Systems: Agents and System Optimizations
Paper Discussion

Paper Discussion Preparations

The second half of the course runs on paper discussions. Everyone reads; two to three students present each class.

Read the assigned papers before class. You are not expected to understand every detail — you are expected to have formed an opinion. Come with at least one thing you did not believe and one thing you would have done differently.

Questions to consider when reading each paper:

  • What problem is this solving, and who actually has that problem?
  • What changed in the hardware, the workload, or the models that made this problem worth solving now?
  • What is the key insight, in one sentence, without jargon?
  • What does the design give up in exchange? Every system trades something.
  • Is the baseline fair? What baseline would have made the result look worse?
  • What workload is being evaluated, and how representative is it?
  • Which numbers would change if you doubled the model size, the context length, or the request rate?
  • What would break this system in production that the paper does not discuss?
  • Is the result still true on today's hardware, or was it a fact about a specific GPU generation?

Class Structure

Each class we will discuss two papers, and will be structured as follows:

Instructor gives a brief context.
First presentation.
Open discussion.
Second presentation.
Open discussion.

For Presenters

You have 20 minutes. Structure them roughly like this.

1. Background & Motivation — about 5 minutes

Why should the room care?

Cover:

  • The problem, stated concretely — ideally with a number that shows it hurts.
  • What people did before this paper, and why it was not good enough.
  • What changed to make this the right moment for this idea.
Tip: assume the audience read the paper. Do not spend five minutes explaining what a KV cache is — spend them explaining why this paper's authors thought the existing approach was broken.

2. Key Idea & Design — about 8 minutes

What is the insight, and how is it realized?

Cover:

  • The central idea in one sentence, before any mechanism.
  • The mechanism: how the system is put together, using the paper's own figures where they are good and your own where they are not.
  • The one hard part — every systems paper has a place where the idea meets reality and gets complicated. Find it and explain it.
Tip: if you cannot explain the key idea without the paper's diagram, you probably do not have it yet. Try explaining it to someone outside the course first.

3. Evaluation — about 5 minutes

Do the numbers support the claims?

Cover:

  • The setup: hardware, models, workloads. Say it out loud — it is often where the story is.
  • The two or three figures that actually matter. Skip the rest.
  • The baseline. Is it the strongest available alternative, or a convenient one?
  • What is not measured.

Evaluation lens

For AI systems papers specifically: check whether throughput gains come at the cost of tail latency, whether the workload's request-length distribution is realistic, whether "GPU utilization" means occupancy or useful FLOPs, and whether the comparison holds the accuracy of the model fixed.

4. Discussion & Takeaways — about 2 minutes

Hand the room something to argue about.

Cover:

  • What you would keep from this paper if you were building a system tomorrow.
  • What has been superseded since publication.
  • Two or three prepared discussion questions — open-ended, not factual. The goal is disagreement, not recall.

Presentation Rubric

Presentations are graded on the following criteria (100 points total).

Criterion Excellent (90-100%) Good (75-89%) Adequate (60-74%) Needs Improvement (<60%) Weight
Content & Understanding Demonstrates deep understanding of the problem, approach, and contributions; clearly explains technical details Solid understanding of the paper; minor gaps in technical depth Covers the main points but misses important details or shows surface-level understanding Significant misunderstandings or major omissions 25%
Slide Quality & Delivery Well-organized slides with effective figures; clear, confident delivery; good pacing within time limit Good slides and delivery; minor issues with organization, pacing, or clarity Slides are text-heavy or disorganized; delivery is unclear or significantly over/under time Poorly prepared slides; hard to follow; major time issues 25%
Critical Analysis Identifies key strengths and weaknesses; offers thoughtful evaluation of methodology and results Provides reasonable critique but lacks depth in some areas Minimal critical analysis; mostly summarizes without evaluating No critical perspective; pure summary of the paper 25%
Discussion Leadership Prepares thought-provoking questions; engages the class effectively; handles audience questions well Asks reasonable questions; some audience engagement Few or superficial discussion questions; limited engagement No prepared questions; unable to lead discussion 25%

Send your slides to the instructor at least 24 hours before your session so we can catch problems while there is still time to fix them.

For the Audience

Participation is 10% of your grade, and it is earned here.

How to participate effectively:

  • Read the paper. Not the abstract — the paper.
  • Arrive with one question written down. Having it on paper makes it far more likely you will ask it.
  • Ask about the thing you did not understand. You are reliably not the only one.
  • Push on evaluation claims. "What was the baseline configured to do?" is almost always a good question.
  • Connect papers to each other and to the lectures — this is where the good discussions come from.
  • Disagree with the paper, with the presenter, and with the instructor. Do it about the ideas.

Quality counts more than volume. One sharp question is worth more than five comments.

Effective Paper Reading Strategy

Adapted from Keshav's three-pass approach. Budget your attention — not every paper deserves the same depth.

Pass 1 (10–30 min): Triage and Mental Map

Decide whether and how to keep reading.

Do:

  • Read the title, abstract, and introduction carefully.
  • Read section headings; ignore the body.
  • Look at every figure and read its caption.
  • Read the conclusion. Skim the references for what you recognize.

Outcome:

  • You can state the category, the context, the core claim, and whether the evaluation looks credible.
Tip: if you cannot say what problem the paper solves after pass 1, that is usually a fact about the paper's writing, not about you.

Pass 2 (1–2 hrs): Understand & Evaluate

Grasp the design and judge the evidence.

Identify:

  • The system's architecture and the one or two mechanisms doing the real work.
  • Every assumption the design depends on — about the workload, the hardware, the model.
  • The experimental setup, and which claims each figure actually supports.
  • What the authors chose not to measure.

Outcome:

  • You could summarize the paper to a colleague and defend or attack its central claim.

Pass 3 (Deep Dive): Re-derive & Stress-test

Read as though you were the author, and then as the reviewer.

Do:

  • Reconstruct the design yourself before reading how they did it; compare your choices to theirs.
  • Sanity-check the headline numbers against a back-of-the-envelope estimate — bandwidth, FLOPs, memory capacity. Do they even make sense?
  • Look for the unstated assumption that makes the result work.
  • Ask what the paper would look like if the result had been negative.

Outcome:

  • You could reimplement the system, and you know exactly where it would fall over.

Use pass 1 on everything, pass 2 on the papers assigned for discussion, and pass 3 on the paper you are presenting and on anything close to your project. Reading fewer papers more carefully beats skimming many.

Additional Resources: