<!-- AUTO-GENERATED by build.py. Edit build.py or data/*.yaml, then rebuild. -->
# COMS 6998 | Fall 2026
# AI-Native Computing
## *Hardware for AI ⇄ AI for Hardware*

**Columbia University | Department of Computer Science**  
Instructor: Prof. Zishen Wan · Fridays 10:10 AM - 12:00 PM · 602 Northwest Corner  
Generated from the canonical course website on Monday, August 31, 2026.

> **Canonical-source notice.** This syllabus is generated from the same source as the course website. The live website and Canvas announcements govern later schedule or logistics updates.

## Course overview

> **Course thesis - AI is transforming computing in two directions: emerging AI workloads demand new hardware and system architectures, while AI is becoming a powerful tool for designing computing systems themselves.**

This advanced graduate lecture-seminar studies the two directions of AI-native computing:

- **Computing for AI.** Profile, serve, schedule, map, accelerate, and make reliable emerging LLM, agentic, physical, and compositional AI workloads.
- **AI for Computing.** Use agents to design, optimize, and verify software, compilers, architectures, SoCs, RTL, EDA flows, and chips.
- **Shared methodology.** Dynamic workflows, closed-loop feedback, cross-layer optimization, heterogeneous resources, quality-performance-cost tradeoffs, and evidence-driven evaluation.

## Course information

| Item | Details |
|---|---|
| Course | COMS 6998 - AI-Native Computing: Hardware for AI, AI for Hardware |
| Instructor | Prof. Zishen Wan · [zw3306@columbia.edu](mailto:zw3306@columbia.edu) · [webpage](https://zishenwan.github.io/) |
| Meeting | Fridays 10:10 AM - 12:00 PM |
| Location | 602 Northwest Corner |
| Office hours | Wednesdays 5-6 PM, CSB 522 |
| Enrollment | Cap 30 |
| Course platform | [Canvas](https://courseworks2.columbia.edu/courses/sis_course_id:COMSE6998_020_2026_3) |
| Format | Advanced graduate lecture-seminar with a semester-long research project; no exams and no problem sets |

## Learning outcomes

1. **Profile and diagnose AI systems.** Represent an AI application as a pipeline, dynamic DAG, or feedback loop; measure latency, throughput, utilization, energy, and cost; locate bottlenecks with roofline reasoning, queueing, and trace analysis.
2. **Reason across the stack.** Connect model, software, runtime, architecture, memory, accelerator, SoC, and deployment decisions, and evaluate joint quality-performance-energy-cost tradeoffs.
3. **Serve and accelerate emerging workloads.** LLM and agent serving, embodied and physical AI inference, neuro-symbolic acceleration, datacenter accelerators, and SoCs.
4. **Build AI that designs computing systems.** Formulate system design as an agent environment with state, actions, tools, and feedback; compare LLM agents, RL, Bayesian optimization, and classical heuristics under matched budgets.
5. **Audit claims like a reviewer.** Read papers and industry claims against baselines, budgets, ablations, and held-out evidence.
6. **Produce conference-style research.** A semester-long project with meaningful baselines, ablations, failure analysis, and a reproducible artifact.

## Prerequisites and expectations

- **Expected.** Basic computer organization or systems knowledge, familiarity with machine-learning concepts, and the ability to program and run quantitative experiments.
- **Helpful, not required.** Experience with CUDA, compilers, digital design, RTL, EDA, robotics simulators, FPGA platforms, LLM agents, or research-paper reading. No one is expected to arrive with expertise across the entire stack.
- **Project readiness.** Each team should bring enough complementary expertise to implement, measure, and evaluate its selected project.

## Course format

Nine seminar meetings provide 23 paper-lead slots; every student leads exactly once. Presentation slides are due at 11:59 PM ET on the Thursday before class.

### Regular week (110 minutes)

| Time | Activity |
|---|---|
| 10:10-10:35 | Instructor mini-lecture: concepts, methods, cross-paper connections |
| 10:35-11:00 | Paper 1 - presentation, critique, discussion |
| 11:00-11:10 | Break |
| 11:10-11:35 | Paper 2 - presentation, critique, discussion |
| 11:35-12:00 | Paper 3 - presentation, critique, discussion |

### Guest-speaker week

| Time | Activity |
|---|---|
| 10:10-10:35 | Paper 1 - presentation, critique, discussion |
| 10:35-11:00 | Paper 2 - presentation, critique, discussion |
| 11:00-11:10 | Break |
| 11:10-12:00 | Guest lecture + Q&A |

### Student-led paper presentation (25 minutes)

- **12 minutes:** problem, context, mechanism, and the minimum results needed to understand the paper.
- **8 minutes:** critical analysis of claims, baselines, assumptions, methodology, and missing evidence.
- **5 minutes:** facilitated discussion around two or three precise questions.

Presenters read the full paper, appendices, and artifact documentation. Everyone else reads the abstract, introduction, core method, principal results, and limitations of all assigned papers, and arrives with at least one discussion question. There are no weekly summary reports.

## Grading

| Component | Weight |
|---|---:|
| Paper presentation & discussion leadership | 15% |
| Engagement & participation | 10% |
| Project P0 - Proposal | 5% |
| Project check-ins P1-P5 (completion-based) | 10% |
| Midterm presentation | 10% |
| Final poster | 10% |
| Final paper & artifact | 40% |

Grades reflect research judgment, technical execution, evidence quality, communication, and reproducibility - not whether a project happens to beat the state of the art. A rigorous negative result can earn full credit.

### Evidence-centered discussion

1. What is the paper's exact central claim?
2. Which experiment most directly supports that claim?
3. What hidden assumption is most likely to break?
4. Is the baseline fair and budget-matched?
5. Which conclusion extends beyond the presented evidence?
6. What single additional experiment would most change confidence in the result?

## Semester-long research project

The project is the center of the course: a carefully scoped research effort that could mature into a top-tier architecture, systems, ML systems, robotics, or EDA paper. Teams of 1-2 are formed by bidding on a curated portfolio of directions; publication is an aspiration, not a grading requirement.

- **Track A: Computing for AI.** Profile, serve, schedule, map, accelerate, or make reliable an LLM, agentic, physical, or neuro-symbolic workload.
- **Track B: AI for Computing.** Build and rigorously evaluate an agent for software optimization, compilers, GPU kernels, architecture DSE, RTL/EDA, or verification.

### Minimum research standard

- A falsifiable research question and a precise intended claim.
- At least two meaningful baselines, including a classical or non-agentic baseline where applicable.
- A mechanism, not only a correlation or leaderboard result.
- At least one ablation or controlled intervention tied to the central claim.
- Joint reporting of task quality or success and relevant systems metrics.
- A held-out workload, configuration, system scale, or design budget.
- Explicit compute, GPU, API, token, simulation, and wall-clock budgets.
- Failure-case analysis, including invalid actions and tool failures for agentic design systems.
- Reproducible code, environment instructions, configurations, data, and figure-generation scripts.

### Milestones

| Date | ID | Deliverable |
|---|---|---|
| Oct 2 | P0 | Project proposal |
| Oct 16 | P1 | Infrastructure & baselines |
| Oct 30 | P2 | Prototype & pilot results |
| Nov 6 | Midterm | Midterm presentation (in class) |
| Nov 20 | P3 | Evaluation plan & initial results |
| Dec 4 | P4 | Main results & ablations |
| Dec 11 | P5 | Complete draft, artifact & poster; final poster session (in class) |
| Dec 21 | Final | Final paper & artifact |

All written deliverables are due at 11:59 PM ET. Check-ins P1-P5 are pacing devices, graded on completeness. No course deadline falls on the Thanksgiving holiday.

### Final submission

An eight- to ten-page conference-style paper (excluding references and appendices); a repository with pinned environment, one-command smoke test, and a documented reproduction path for one central result; machine-readable results with scripts regenerating principal figures; an experiment manifest covering seeds, configurations, models, machines, tool versions, and resource budgets; a response-to-feedback memo and individual contribution statements.

## Weekly schedule and readings

Required readings appear first. Optional readings are listed separately, one paper per line. Guest-speaker details remain tentative until announced.

### Week 01 · Friday, September 11, 2026

**Course Introduction: AI as Workload, AI as Designer** · Launch


**Instructor-selected background**

- [A New Golden Age for Computer Architecture](https://doi.org/10.1145/3282307) - CACM 2019
- [GenAI for Systems: Recurring Challenges and Design Principles from Software to Silicon](https://arxiv.org/abs/2602.15241) - arXiv 2026

**Optional readings**

- [The Bitter Lesson (2019)](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)
- [Architecture 2.0: Why Computer Architects Need a Data-Centric AI Gymnasium (ACM SIGARCH Blog, 2023)](https://www.sigarch.org/architecture-2-0-why-computer-architects-need-a-data-centric-ai-gymnasium/)

**Deadlines**

- Sep 12, 11:59 PM - background survey

### Week 02 · Friday, September 18, 2026

**How to Study an AI Computing System + LLM Inference Fundamentals** · Computing for AI


**Instructor-selected background**

- [Roofline: An Insightful Visual Performance Model for Multicore Architectures](https://doi.org/10.1145/1498765.1498785) - CACM 2009
- [The Tail at Scale](https://research.google/pubs/the-tail-at-scale/) - CACM 2013
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://arxiv.org/abs/2205.14135) - NeurIPS 2022

**Optional readings**

- [MLPerf Inference Benchmark (ISCA 2020)](https://arxiv.org/abs/1911.02549)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey (Proceedings of the IEEE 2017)](https://arxiv.org/abs/1703.09039)

**Deadlines**

- Sep 18, 11:59 PM - paper preference form
- Sep 23 - project bidding form

### Week 03 · Friday, September 25, 2026

**Serving Systems for LLMs and AI Agents** · Computing for AI

- **Guest lecture:** LLM serving and efficiency (Guest Speaker; confirmation pending).
- **Mini-lecture case study:** DyServe - a workflow-aware serving layer (instructor's work).

**Student-led papers**

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://arxiv.org/abs/2309.06180) - SOSP 2023; focus: KV-cache virtualization, fragmentation, batching, vLLM
- [SGLang: Efficient Execution of Structured Language Model Programs](https://arxiv.org/abs/2312.07104) - NeurIPS 2024; focus: RadixAttention, KV reuse across program structure

**Optional readings**

- [Splitwise: Efficient Generative LLM Inference Using Phase Splitting (ISCA 2024)](https://arxiv.org/abs/2311.18677)
- [Agentix: An Efficient Serving Engine for LLM Agents as General Programs (NSDI 2026)](https://www.usenix.org/conference/nsdi26/presentation/luo)
- [A Workflow-Aware Serving Layer for Agentic Applications (DyServe, arXiv 2026)](https://arxiv.org/abs/2607.02942)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-Optimized Large Language Model Serving (OSDI 2024)](https://arxiv.org/abs/2401.09670)
- [Mooncake: A KVCache-Centric Disaggregated Architecture for LLM Serving (FAST 2025)](https://arxiv.org/abs/2407.00079)
- [Parrot: Efficient Serving of LLM-Based Applications with Semantic Variable (OSDI 2024)](https://arxiv.org/abs/2405.19888)
- [Fast Inference from Transformers via Speculative Decoding (ICML 2023)](https://arxiv.org/abs/2211.17192)
- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve (OSDI 2024)](https://arxiv.org/abs/2403.02310)
- [MemGPT: Towards LLMs as Operating Systems (2023)](https://arxiv.org/abs/2310.08560)

**Deadlines**

- Project teams formed

### Week 04 · Friday, October 2, 2026

**AI Hardware: Accelerators and System-on-Chips** · Computing for AI

- **Guest lecture:** AI accelerator and SoC design (Guest Speaker; confirmation pending).

**Student-led papers**

- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://doi.org/10.1145/3079856.3080246) - ISCA 2017; focus: read as a ten-year retrospective
- [Agile SoC Development with Open ESP](https://arxiv.org/abs/2009.01178) - ICCAD 2020; focus: Columbia's open-source heterogeneous SoC platform; available for course projects; [project site](https://www.esp.cs.columbia.edu/)

**Optional readings**

- [Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration (DAC 2021)](https://arxiv.org/abs/1911.09925)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings (ISCA 2023)](https://arxiv.org/abs/2304.01433)
- [Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks (ISCA 2016)](https://dl.acm.org/doi/10.1145/3007787.3001177)
- [Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A Data-Centric Approach Using MAESTRO (MICRO 2019)](https://arxiv.org/abs/1805.02566)
- [Timeloop: A Systematic Approach to DNN Accelerator Evaluation (ISPASS 2019)](https://doi.org/10.1109/ISPASS.2019.00042)
- NVIDIA H100 Tensor Core GPU Architecture (Whitepaper, 2022) *(link pending)*
- NVIDIA Blackwell Architecture Technical Brief (2024) *(link pending)*

**Deadlines**

- Oct 2, 11:59 PM - P0 project proposal & team charter

### Week 05 · Friday, October 9, 2026

**Physical AI I: Inference Systems and Serving for Embodied AI** · Computing for AI


**Student-led papers**

- [Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots](https://arxiv.org/abs/2607.02501) - arXiv 2026; focus: deployable embodied-AI runtime across heterogeneous robots
- [VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference](https://arxiv.org/abs/2512.01031) - arXiv 2025; focus: overlapping inference with action execution; latency hiding
- [How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf](https://arxiv.org/abs/2602.18397) - arXiv 2026; focus: analytical performance model; device/edge/cloud placement

**Optional readings**

- [π0: A Vision-Language-Action Flow Model for General Robot Control (RSS 2025)](https://arxiv.org/abs/2410.24164)
- [OpenVLA: An Open-Source Vision-Language-Action Model (CoRL 2024)](https://arxiv.org/abs/2406.09246)
- [DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution (NeurIPS 2024)](https://arxiv.org/abs/2411.02359)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (CoRL 2023)](https://arxiv.org/abs/2307.15818)
- [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots (2025)](https://arxiv.org/abs/2503.14734)
- [Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability (ISPASS 2025)](https://arxiv.org/abs/2504.18945)

### Week 06 · Friday, October 16, 2026

**Physical AI II: Hardware-Software Co-Design and Architecture** · Computing for AI


**Student-led papers**

- [DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-Powered Robotic Manipulation](https://arxiv.org/abs/2407.04292) - ISCA 2025; focus: decoupling LLM inference from robotic control via trajectory prediction
- [ReCA: Integrated Acceleration for Real-Time and Efficient Cooperative Embodied Autonomous Agents](https://doi.org/10.1145/3676641.3716016) - ASPLOS 2025; focus: cross-layer co-design for cooperative embodied agents
- [Deltoris: Enabling Real-Time VLA Inference in Embodied AI via Bit-Level Sparsity and Speculative Inference](https://arxiv.org/abs/2608.04428) - MICRO 2026; focus: HW/SW co-design for diffusion-based VLA on edge devices

**Optional readings**

- [Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification (MICRO 2026)](https://arxiv.org/abs/2608.15636)
- [OctoCache: Caching Voxels for Accelerating 3D Occupancy Mapping in Autonomous Systems (ASPLOS 2025)](https://dl.acm.org/doi/10.1145/3676641.3716263)
- [RobotPerf: An Open-Source, Vendor-Agnostic, Benchmarking Suite for Evaluating Robotics Computing System Performance (ICRA 2024)](https://arxiv.org/abs/2309.09212)
- DaDu Series: Fast and Efficient Robot Accelerators (ICCAD 2020) *(link pending)*
- [NVIDIA Jetson AGX Orin Series Technical Brief (2022)](https://www.nvidia.com/content/dam/en-zz/Solutions/gtcf21/jetson-orin/nvidia-jetson-agx-orin-technical-brief.pdf)
- [NVIDIA Jetson AGX Thor Developer Kit User Guide (2026)](https://docs.nvidia.com/jetson/agx-thor-devkit/user-guide/latest/)

**Deadlines**

- Oct 16, 11:59 PM - P1 infrastructure & baselines

### Week 07 · Friday, October 23, 2026

**Accelerating Neuro-Symbolic and Compositional AI** · Computing for AI


**Student-led papers**

- [CogSys: Efficient and Scalable Neurosymbolic Cognition System via Algorithm-Hardware Co-Design](https://research.ibm.com/publications/cogsys-efficient-and-scalable-neurosymbolic-cognition-system-via-algorithm-hardware-co-design) - HPCA 2025; focus: workload characterization, reconfigurable processing
- [REASON: Accelerating Probabilistic Logical Reasoning for Scalable Neuro-Symbolic Intelligence](https://arxiv.org/abs/2601.20784) - HPCA 2026; focus: unified DAGs, irregular reasoning acceleration
- [Lobster: A GPU-Accelerated Framework for Neurosymbolic Programming](https://arxiv.org/abs/2503.21937) - ASPLOS 2026; focus: compiling Datalog-based neurosymbolic programs to GPUs

**Optional readings**

- [Scallop: A Language for Neurosymbolic Programming (PLDI 2023)](https://arxiv.org/abs/2304.04812)
- [DeepProbLog: Neural Probabilistic Logic Programming (NeurIPS 2018)](https://arxiv.org/abs/1805.10872)
- [NSFlow: An End-to-End FPGA Framework with Scalable Dataflow Architecture for Neuro-Symbolic AI (DAC 2025)](https://arxiv.org/abs/2504.19323)
- [Towards Cognitive AI Systems: A Survey and Prospective on Neuro-Symbolic AI (ISPASS 2024)](https://arxiv.org/abs/2401.01040)

### Week 08 · Friday, October 30, 2026

**AI for Software Systems: Compilers and GPU Kernels** · AI for Computing


**Student-led papers**

- [Faster Sorting Algorithms Discovered Using Deep Reinforcement Learning (AlphaDev)](https://www.nature.com/articles/s41586-023-06004-9) - Nature 2023; focus: algorithm discovery, reward design, validation
- [AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery](https://arxiv.org/abs/2506.13131) - DeepMind 2025; focus: evolutionary LLM code search across scheduling, TPU circuits, kernels
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://arxiv.org/abs/2502.10517) - ICML 2025; focus: benchmark design; discussed with the Sakana AI CUDA Engineer episode

**Optional readings**

- [CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research (CGO 2022)](https://arxiv.org/abs/2109.08267)
- [Large Language Models for Compiler Optimization (2023)](https://arxiv.org/abs/2309.07062)
- [Kevin: Multi-Turn RL for Generating CUDA Kernels (2025)](https://arxiv.org/abs/2507.11948)
- [Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization (Sakana AI, 2025)](https://sakana.ai/ai-cuda-engineer/)
- [ECO: An LLM-Driven Efficient Code Optimizer for Warehouse Scale Computers (2025)](https://arxiv.org/abs/2503.15669)

**Deadlines**

- Oct 30, 11:59 PM - P2 prototype & pilot results

### Week 09 · Friday, November 6, 2026

**Midterm Project Presentations** · Project

- **Format note:** Team presentations; no mini-lecture and no assigned papers.

**Deadlines**

- Nov 5, 11:59 PM - midterm slides

### Week 10 · Friday, November 13, 2026

**AI for Computer Architecture I: Measuring and Exploring** · AI for Computing

- **Guest lecture:** AI-assisted architecture design (Guest Speaker; confirmation pending).

**Student-led papers**

- [ArchEval: Measuring AI Agents as Computer Architects](https://arxiv.org/abs/2607.03601) - arXiv 2026; focus: benchmarking LLM agents on architecture tasks
- [PF-LLM: Large Language Model Hinted Hardware Prefetching](https://doi.org/10.1145/3779212.3790202) - ASPLOS 2026; focus: AI inside the microarchitecture

**Optional readings**

- [ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design (ISCA 2023)](https://arxiv.org/abs/2306.08888)
- [AgentDSE: Reasoning-Augmented Architectural Design Space Exploration (MLArchSys 2026)](https://arxiv.org/abs/2606.21836)
- [LLM-DSE: Searching Accelerator Parameters with LLM Agents (2025)](https://arxiv.org/abs/2505.12188)
- [ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators Using Reinforcement Learning (MICRO 2020)](https://arxiv.org/abs/2009.02010)
- [Concorde: Fast and Accurate CPU Performance Modeling with Compositional Analytical-ML Fusion (ISCA 2025)](https://arxiv.org/abs/2503.23076)
- [QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture (ICML 2026)](https://arxiv.org/abs/2510.22087)
- [Learning Memory Access Patterns (ICML 2018)](https://arxiv.org/abs/1803.02329)

### Week 11 · Friday, November 20, 2026

**AI for Computer Architecture II: Agentic Design Systems** · AI for Computing

- **Guest lecture:** agentic hardware design frameworks (Guest Speaker; confirmation pending).
- **Mini-lecture case study:** ArchOrchestra - end-to-end agentic accelerator design (instructor's work).

**Student-led papers**

- [CHIA: An Open-Source Framework for Principled, Agentic AI-Driven Hardware/Software Co-Design Research](https://arxiv.org/abs/2606.27350) - arXiv 2026; focus: composable design loops for agent-driven HW/SW co-design
- [Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization](https://arxiv.org/abs/2604.25083) - arXiv 2026; focus: LLM-driven microarchitecture evolution; seed quality bounds discovery

**Optional readings**

- ArchOrchestra: End-to-End Agentic Accelerator Design (forthcoming, 2026) *(link pending)*
- [AgentDSE: Reasoning-Augmented Architectural Design Space Exploration (MLArchSys 2026)](https://arxiv.org/abs/2606.21836)
- [LLM-DSE: Searching Accelerator Parameters with LLM Agents (2025)](https://arxiv.org/abs/2505.12188)
- [FireSim: FPGA-Accelerated Cycle-Exact Scale-Out System Simulation in the Public Cloud (ISCA 2018)](https://fires.im/)
- [Chipyard: Integrated Design, Simulation, and Implementation Framework for Custom SoCs (IEEE Micro 2020)](https://github.com/ucb-bar/chipyard)

**Deadlines**

- Nov 20, 11:59 PM - P3 evaluation plan & initial results

### Week 12 · Friday, December 4, 2026

**AI for RTL and Chip Physical Design** · AI for Computing

- **In-class exercise:** In-class critical reading: the RedWood white paper (Architect Labs 2026) audited with the evidence-centered checklist.

**Student-led papers**

- [A Graph Placement Methodology for Fast Chip Design (AlphaChip)](https://www.nature.com/articles/s41586-021-03544-w) - Nature 2021 (+2024 addendum); focus: read with Markov's 'The False Dawn' critique; co-led by a pair; [companion critique](https://arxiv.org/abs/2306.09633)
- [ChipNeMo: Domain-Adapted LLMs for Chip Design](https://arxiv.org/abs/2311.00176) - NVIDIA 2024; focus: domain adaptation, retrieval, EDA scripts
- [Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification](https://arxiv.org/abs/2506.14074) - arXiv 2025; focus: RTL benchmarks and their limitations

**Optional readings**

- [AssertLLM: Generating and Evaluating Hardware Verification Assertions from Design Specifications via Multi-LLMs (2024)](https://arxiv.org/abs/2402.00386)
- [Using LLMs to Facilitate Formal Verification of RTL (2023)](https://arxiv.org/abs/2309.09437)
- [DREAMPlace: Deep Learning Toolkit-Enabled GPU Acceleration for Modern VLSI Placement (DAC 2019)](https://github.com/limbo018/DREAMPlace)
- [VerilogEval: Evaluating Large Language Models for Verilog Code Generation (ICCAD 2023)](https://arxiv.org/abs/2309.07544)
- [ChatEDA: A Large Language Model Powered Autonomous Agent for EDA (TCAD 2024)](https://arxiv.org/abs/2308.10204)
- [Automated Multi-Agent Workflows for RTL Design (VeriMaAS, 2025)](https://arxiv.org/abs/2509.20182)
- [SpecLLM: Exploring Generation and Review of VLSI Design Specification with Large Language Model (2024)](https://arxiv.org/abs/2401.13266)
- [Chip-Chat: Challenges and Opportunities in Conversational Hardware Design (MLCAD 2023)](https://arxiv.org/abs/2305.13243)
- [Redwood: A Frontier AI Accelerator Designed, Verified, and Deployed from Scratch in 2 Weeks by AI (2026)](https://arxiv.org/abs/2608.26418)

**Deadlines**

- Dec 4, 11:59 PM - P4 main results & ablations

### Week 13 · Friday, December 11, 2026

**Final Project Poster Session** · Synthesis

- **Format note:** Poster showcase; no mini-lecture and no assigned papers.

**Deadlines**

- Dec 10, 11:59 PM - posters in shared folder
- Dec 11, 11:59 PM - P5 complete draft & artifact
- Dec 21, 11:59 PM - final paper & artifact

## Policies

### AI use and evidence

**AI use is permitted and encouraged when it is disclosed, reproducible, and independently verified. Agent output is not evidence by itself.**

- You may use AI throughout the course - brainstorming, literature discovery, coding, debugging, experiment orchestration, and writing assistance - with meaningful use disclosed.
- Every citation must be checked against a primary source, and every numerical result must trace to an actual experiment, simulator output, formal result, or cited source.
- AI-generated code must satisfy the same correctness, testing, performance, licensing, and provenance requirements as human-written code.

### Academic integrity

Students must follow Columbia academic-integrity policies. Fabricated citations, invented experiments, altered logs, undisclosed result selection, plagiarism, or presenting agent-generated claims as verified evidence are serious violations. When in doubt, disclose the tool, source, assistance, or collaboration.

### Collaboration & authorship

Course collaboration does not automatically establish publication authorship. If a project continues after the semester, authorship and ordering follow substantive intellectual and technical contributions, manuscript participation, accountability, and venue policies. Students retain credit for their work; continuation plans should be discussed transparently with the instructor and research mentors.

### Accessibility & accommodations

Students who require disability-related accommodations should contact Columbia Disability Services and inform the instructor as early as possible so approved accommodations can be implemented. Please communicate time-sensitive circumstances before deadlines whenever possible.

### Resource fairness

Projects report GPU, API, token, simulation, and wall-clock budgets. Grades are not based on access to the largest model or most GPUs. Every project defines a fallback experiment that remains valid if an API, simulator, board, robot, or cloud resource becomes unavailable. Curated starter environments and smoke tests are provided for officially supported directions when feasible.

### Late work

Each team may use one 48-hour grace pass on a written milestone, requested before the deadline. The grace pass does not apply to in-class presentations, the final poster, or the final submission. Other extensions require prior approval or documented circumstances.

### Changes to the syllabus

This is a first-offering advanced-topics course in a rapidly changing research area. Individual readings, project briefs, guest-speaker scheduling, or detailed deadlines may be updated when new work appears or infrastructure changes. Material changes will be announced clearly and will not retroactively disadvantage students.

## Registrar dates

- **Friday, September 18, 2026:** Last day to add (Change of Program)
- **Tuesday, October 13, 2026:** Drop deadline
- **Thursday, November 19, 2026:** Pass/fail & withdrawal deadline
- **Friday, November 27, 2026:** No class - Thanksgiving

See the [Columbia Registrar academic calendar](https://registrar.columbia.edu/content/academic-calendar) for the authoritative university calendar.
