01
Course Introduction: AI as Workload, AI as Designer
Instructor-selected background · no student presentation
Optional readings (2)
- Sep 12, 11:59 PM - background survey
Schedule
Computing for AI
AI for Computing
13 Friday meetings, 10:10 AM - 12:00 PM, 602 Northwest Corner. Presentation slides are due 11:59 PM the Thursday before class.
Module 1: Computing for AI
02
How to Study an AI Computing System + LLM Inference Fundamentals
Instructor-selected background · no student presentation
- The Tail at Scale CACM 2013
Optional readings (2)
- Sep 18, 11:59 PM - paper preference form
- Sep 23 - project bidding form
03
Serving Systems for LLMs and AI Agents
Guest lecture: LLM serving and efficiency.
Mini-lecture case study: DyServe - a workflow-aware serving layer (instructor's work)
Student-led papers
- KV-cache virtualization, fragmentation, batching, vLLM
- RadixAttention, KV reuse across program structure
Optional readings (9)
- Splitwise: Efficient Generative LLM Inference Using Phase Splitting (ISCA 2024)
- Agentix: An Efficient Serving Engine for LLM Agents as General Programs (NSDI 2026)
- A Workflow-Aware Serving Layer for Agentic Applications (DyServe, arXiv 2026)
- DistServe: Disaggregating Prefill and Decoding for Goodput-Optimized Large Language Model Serving (OSDI 2024)
- Mooncake: A KVCache-Centric Disaggregated Architecture for LLM Serving (FAST 2025)
- Parrot: Efficient Serving of LLM-Based Applications with Semantic Variable (OSDI 2024)
- Fast Inference from Transformers via Speculative Decoding (ICML 2023)
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve (OSDI 2024)
- MemGPT: Towards LLMs as Operating Systems (2023)
- Project teams formed
04
AI Hardware: Accelerators and System-on-Chips
Guest lecture: AI accelerator and SoC design.
Student-led papers
- read as a ten-year retrospective
- Columbia's open-source heterogeneous SoC platform; available for course projects
Optional readings (7)
- Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration (DAC 2021)
- TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings (ISCA 2023)
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks (ISCA 2016)
- Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A Data-Centric Approach Using MAESTRO (MICRO 2019)
- Timeloop: A Systematic Approach to DNN Accelerator Evaluation (ISPASS 2019)
- NVIDIA H100 Tensor Core GPU Architecture (Whitepaper, 2022)
- NVIDIA Blackwell Architecture Technical Brief (2024)
- Oct 2, 11:59 PM - P0 project proposal & team charter
05
Physical AI I: Inference Systems and Serving for Embodied AI
Student-led papers
- deployable embodied-AI runtime across heterogeneous robots
- overlapping inference with action execution; latency hiding
- analytical performance model; device/edge/cloud placement
Optional readings (6)
- π0: A Vision-Language-Action Flow Model for General Robot Control (RSS 2025)
- OpenVLA: An Open-Source Vision-Language-Action Model (CoRL 2024)
- DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution (NeurIPS 2024)
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (CoRL 2023)
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots (2025)
- Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability (ISPASS 2025)
06
Physical AI II: Hardware-Software Co-Design and Architecture
Student-led papers
- decoupling LLM inference from robotic control via trajectory prediction
- ReCA: Integrated Acceleration for Real-Time and Efficient Cooperative Embodied Autonomous Agents ASPLOS 2025cross-layer co-design for cooperative embodied agents
- HW/SW co-design for diffusion-based VLA on edge devices
Optional readings (6)
- Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification (MICRO 2026)
- OctoCache: Caching Voxels for Accelerating 3D Occupancy Mapping in Autonomous Systems (ASPLOS 2025)
- RobotPerf: An Open-Source, Vendor-Agnostic, Benchmarking Suite for Evaluating Robotics Computing System Performance (ICRA 2024)
- DaDu Series: Fast and Efficient Robot Accelerators (ICCAD 2020)
- NVIDIA Jetson AGX Orin Series Technical Brief (2022)
- NVIDIA Jetson AGX Thor Developer Kit User Guide (2026)
- Oct 16, 11:59 PM - P1 infrastructure & baselines
07
Accelerating Neuro-Symbolic and Compositional AI
Student-led papers
- CogSys: Efficient and Scalable Neurosymbolic Cognition System via Algorithm-Hardware Co-Design HPCA 2025workload characterization, reconfigurable processing
- REASON: Accelerating Probabilistic Logical Reasoning for Scalable Neuro-Symbolic Intelligence HPCA 2026unified DAGs, irregular reasoning acceleration
- compiling Datalog-based neurosymbolic programs to GPUs
Optional readings (4)
- Scallop: A Language for Neurosymbolic Programming (PLDI 2023)
- DeepProbLog: Neural Probabilistic Logic Programming (NeurIPS 2018)
- NSFlow: An End-to-End FPGA Framework with Scalable Dataflow Architecture for Neuro-Symbolic AI (DAC 2025)
- Towards Cognitive AI Systems: A Survey and Prospective on Neuro-Symbolic AI (ISPASS 2024)
Module 2: AI for Computing
08
AI for Software Systems: Compilers and GPU Kernels
Student-led papers
- algorithm discovery, reward design, validation
- evolutionary LLM code search across scheduling, TPU circuits, kernels
- benchmark design; discussed with the Sakana AI CUDA Engineer episode
Optional readings (5)
- CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research (CGO 2022)
- Large Language Models for Compiler Optimization (2023)
- Kevin: Multi-Turn RL for Generating CUDA Kernels (2025)
- Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization (Sakana AI, 2025)
- ECO: An LLM-Driven Efficient Code Optimizer for Warehouse Scale Computers (2025)
- Oct 30, 11:59 PM - P2 prototype & pilot results
09
Midterm Project Presentations
Team presentations; no mini-lecture and no assigned papers.
- Nov 5, 11:59 PM - midterm slides
10
AI for Computer Architecture I: Measuring and Exploring
Guest lecture: AI-assisted architecture design.
Student-led papers
- benchmarking LLM agents on architecture tasks
- AI inside the microarchitecture
Optional readings (7)
- ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design (ISCA 2023)
- AgentDSE: Reasoning-Augmented Architectural Design Space Exploration (MLArchSys 2026)
- LLM-DSE: Searching Accelerator Parameters with LLM Agents (2025)
- ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators Using Reinforcement Learning (MICRO 2020)
- Concorde: Fast and Accurate CPU Performance Modeling with Compositional Analytical-ML Fusion (ISCA 2025)
- QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture (ICML 2026)
- Learning Memory Access Patterns (ICML 2018)
11
AI for Computer Architecture II: Agentic Design Systems
Guest lecture: agentic hardware design frameworks.
Mini-lecture case study: ArchOrchestra - end-to-end agentic accelerator design (instructor's work) (arXiv link coming in September)
Student-led papers
- composable design loops for agent-driven HW/SW co-design
- Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization arXiv 2026LLM-driven microarchitecture evolution; seed quality bounds discovery
Optional readings (5)
- ArchOrchestra: End-to-End Agentic Accelerator Design (forthcoming, 2026)
- AgentDSE: Reasoning-Augmented Architectural Design Space Exploration (MLArchSys 2026)
- LLM-DSE: Searching Accelerator Parameters with LLM Agents (2025)
- FireSim: FPGA-Accelerated Cycle-Exact Scale-Out System Simulation in the Public Cloud (ISCA 2018)
- Chipyard: Integrated Design, Simulation, and Implementation Framework for Custom SoCs (IEEE Micro 2020)
- Nov 20, 11:59 PM - P3 evaluation plan & initial results
No class - Thanksgiving
12
AI for RTL and Chip Physical Design
In-class exercise: In-class critical reading: the RedWood white paper (Architect Labs 2026) audited with the evidence-centered checklist
Student-led papers
- A Graph Placement Methodology for Fast Chip Design (AlphaChip) Nature 2021 (+2024 addendum) co-led by a pair companion critiqueread with Markov's 'The False Dawn' critique; co-led by a pair
- ChipNeMo: Domain-Adapted LLMs for Chip Design NVIDIA 2024domain adaptation, retrieval, EDA scripts
- RTL benchmarks and their limitations
Optional readings (9)
- AssertLLM: Generating and Evaluating Hardware Verification Assertions from Design Specifications via Multi-LLMs (2024)
- Using LLMs to Facilitate Formal Verification of RTL (2023)
- DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement (DAC 2019)
- VerilogEval: Evaluating Large Language Models for Verilog Code Generation (ICCAD 2023)
- ChatEDA: A Large Language Model Powered Autonomous Agent for EDA (TCAD 2024)
- Automated Multi-Agent Workflows for RTL Design (VeriMaAS, 2025)
- SpecLLM: Exploring Generation and Review of VLSI Design Specification with Large Language Model (2024)
- Chip-Chat: Challenges and Opportunities in Conversational Hardware Design (MLCAD 2023)
- Redwood: A Frontier AI Accelerator Designed, Verified, and Deployed from Scratch in 2 Weeks by AI (2026)
- Dec 4, 11:59 PM - P4 main results & ablations
13
Final Project Poster Session
Poster showcase; no mini-lecture and no assigned papers.
- Dec 10, 11:59 PM - posters in shared folder
- Dec 11, 11:59 PM - P5 complete draft & artifact
- Dec 21, 11:59 PM - final paper & artifact
Registrar dates: Sep 18: Last day to add (Change of Program) · Oct 13: Drop deadline · Nov 19: Pass/fail & withdrawal deadline · Nov 27: No class - Thanksgiving · academic calendar