Next class · Week 01 ·
Course Introduction: AI as Workload, AI as Designer
Research question of the weekHow does the relationship between workloads and architectures change when AI can adapt both?
Required reading
COMS 6998 · Columbia University · Fall 2026
AI-Native Computing
Hardware for AI AI for Hardware
AI is transforming computing in two directions: emerging AI workloads demand new hardware and system architectures, while AI is becoming a powerful tool for designing computing systems themselves.
Module 1: Computing for AI
Profile, serve, schedule, map, accelerate, and make reliable emerging LLM, agentic, physical, and compositional AI workloads.
Module 2: AI for Computing
Use agents to design, optimize, and verify software, compilers, architectures, SoCs, RTL, EDA flows, and chips.
Shared methodology. Dynamic workflows, closed-loop feedback, cross-layer optimization, heterogeneous resources, quality-performance-cost tradeoffs, and evidence-driven evaluation.
What you'll learn
Profile and diagnose AI systems. Represent an AI application as a pipeline, dynamic DAG, or feedback loop; measure latency, throughput, utilization, energy, and cost; locate bottlenecks with roofline reasoning, queueing, and trace analysis.
Reason across the stack. Connect model, software, runtime, architecture, memory, accelerator, SoC, and deployment decisions, and evaluate joint quality-performance-energy-cost tradeoffs.
Serve and accelerate emerging workloads. LLM and agent serving, embodied and physical AI inference, neuro-symbolic acceleration, datacenter accelerators, and SoCs.
Build AI that designs computing systems. Formulate system design as an agent environment with state, actions, tools, and feedback; compare LLM agents, RL, Bayesian optimization, and classical heuristics under matched budgets.
Audit claims like a reviewer. Read papers and industry claims against baselines, budgets, ablations, and held-out evidence.
Produce conference-style research. A semester-long project with meaningful baselines, ablations, failure analysis, and a reproducible artifact.
Prerequisites & expectations
- Expected
- Basic computer organization or systems knowledge, familiarity with machine-learning concepts, and the ability to program and run quantitative experiments.
- Helpful, not required
- Experience with CUDA, compilers, digital design, RTL, EDA, robotics simulators, FPGA platforms, LLM agents, or research-paper reading. No one is expected to arrive with expertise across the entire stack.
- Project readiness
- Each team should bring enough complementary expertise to implement, measure, and evaluate its selected project.
Scope note. The course covers cross-layer computing systems, spanning computer architecture, software systems, and silicon, for emerging AI workloads such as physical, embodied, neuro-symbolic, and agentic AI; and agentic AI methods that design, optimize, and verify computing systems themselves. The two directions close a loop: better computing enables stronger AI, and stronger AI builds better computing.
Now
Start here this week
Next class · Week 02 ·
How to Study an AI Computing System + LLM Inference Fundamentals
Research question of the weekWhat makes a performance model useful when bottlenecks shift with workload, scale, and operating conditions?
Required reading
Next class · Week 03 ·
Serving Systems for LLMs and AI Agents
Guest lecture: Prof. Jovan Stojkovic (UT Austin)
Research question of the weekWhat is the right unit of optimization when an AI application is a workflow rather than a single model?
Required reading
Next class · Week 04 ·
AI Hardware: Accelerators and System-on-Chips
Guest lecture: Prof. Tushar Krishna (Georgia Tech)
Research question of the weekWhen does specialization create a lasting advantage, and when does it bind a system too closely to today's workloads?
Required reading
- In-Datacenter Performance Analysis of a Tensor Processing Unit ISCA 2017
- Open AI Jalapeño Chip Hot Chips 2026 slides
Next class · Week 05 ·
Physical AI I: Inference Systems and Serving for Embodied AI
Guest lecture: Prof. Yilun Du (Harvard)
Research question of the weekHow should computation be organized when the value of a prediction depends on when an action is taken?
Required reading
Next class · Week 06 ·
Physical AI II: Hardware-Software Co-Design and Architecture
Guest lecture: Prof. Luca Carloni (Columbia)
Research question of the weekWhere should the boundary between algorithm and hardware move when latency, energy, and task success must be optimized together?
Required reading
Next class · Week 07 ·
Accelerating Neuro-Symbolic and Compositional AI
Research question of the weekHow should computing abstractions change when intelligence combines learning, search, and symbolic reasoning?
Required reading
Next class · Week 08 ·
AI for Software Systems: Compilers and GPU Kernels
Research question of the weekWhat should an AI optimizer be free to change, and which properties must remain invariant?
Required reading
Next class · Week 09 ·
Midterm Project Presentations
Research question of the weekWhich uncertainty is most worth resolving before investing further in a research direction?
Required reading
No assigned readings.
Next class · Week 10 ·
AI for Computer Architecture I: Measuring and Exploring
Guest lecture: Prof. Sagar Karandikar (UC Berkeley)
Research question of the weekWhat makes a design environment informative enough for an AI agent to discover better architectures?
Required reading
Next class · Week 11 ·
AI for Computer Architecture II: Architecture 2.0
Guest lecture: Prof. Vijay Janapa Reddi (Harvard)
Research question of the weekWhat would it take for AI-native design to turn high-level intent into better, physically verified systems and chips?
Required reading
- ArchOrchestra: End-to-End Agentic Accelerator Design Forthcoming 2026
- Architecture 2.0: Principles of AI-Native System and Chip Design Online book, 2026
Next class · Week 12 ·
AI for RTL and Chip Physical Design
Research question of the weekHow should exploration and verification work together when AI makes consequential design decisions?
Required reading
- A Graph Placement Methodology for Fast Chip Design (AlphaChip) Nature 2021 (+2024 addendum)
- ChipNeMo: Domain-Adapted LLMs for Chip Design NVIDIA 2024
- Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification arXiv 2025
Next class · Week 13 ·
Final Project Poster Session
Research question of the weekWhat general principle does a research result establish, and where does that principle stop applying?
Required reading
No assigned readings.
Next deadline
background survey Please open with your Columbia email account.
project proposal (LaTeX template (ZIP))
P1 infrastructure & baselines
P2 prototype & pilot results
midterm slides
P3 evaluation plan & initial results
P4 main results & ablations
posters in shared folder
P5 complete draft & artifact
final paper & artifact
No upcoming deadlines.
Announcements
-
Project office hours
Discuss your project proposal, current progress, and next steps, and get feedback on any questions or challenges. Oct 14, 4-6 PM; Oct 15, 11 AM-12 PM & 3-5 PM (ET). CSB 522. Sign up
-
Classroom update
Our classroom is now MUDD Building 825 (to accommodate more students). We still meet Fridays, 10:10 AM-12:00 PM.
-
Course website is live
The complete schedule and reading list are now available. Material updates will also be announced in CourseWorks.
Schedule at a glance
All deadlines: 11:59 PM ET.