mlip-cmu
Machine Learning in Production @ CMU
Shortcuts: The Course (Fall 2026) The Book Lecture recordings (Spring 2026)
Find resources related to teaching and research on how to build, deploy, assure, and maintain AI-powered software products, spanning classic machine learning, LLMs, and AI agents. For example, how to integrate a voice-to-text model and an LLM into a video conferencing product to create automated meeting summaries. This is not about chasing a benchmark or building a throwaway demo, but focuses on the hard parts: We cover the entire lifecycle, from a prototype model to an entire product deployed and operated in the real world under load, with real customers who depend on it. Because models make mistakes, we emphasize managing risk and the responsible engineering of such systems (safety, security, fairness, transparency), together with MLOps.
All materials (book, slides, assignments, video recordings, bibliography) are released under creative commons licenses. We hope that this fosters teaching and research on these topics.
Maintained by Christian Kaestner.
The Pitch
The following talk motivates the entire endeavor, explains why we engineer the entire system around the model, and runs through what this means from the lens of quality assurance (from model testing to system testing):
The Book
Print & Ebook version Online version
A book has been published MIT Press as open access. All author proceeds are donated to Evidence Action. The complete book is available online under a creative common license here.
The Course
We teach a 12-unit course at Carnegie Mellon University on this topic, open to undergraduates and graduate students. We expect some minimal machine learning background and some programming skills, but no prior software engineering experience. The course is offered regularly in both the spring and fall semesters.
- Fall 2026: F2026 website and F2026 on GitHub
- Spring 2026: S2026 website and S2026 on GitHub
- Fall 2025: F2025 website and F2025 on GitHub
- Spring 2025: S2025 website and S2025 on GitHub
- Fall 2024: F2024 website and F2024 on GitHub
- Spring 2024: S2024 website and S2024 on GitHub
- Spring 2023: S2023 website and S2023 on GitHub
- Fall 2022: F2022 website and F2022 GitHub branch
- Spring 2022: S2022 website and S2022 GitHub branch
- Spring 2021: S2021 website and S2021 GitHub branch
- Fall 2020: F2020 website and F2020 GitHub branch
- Summer 2020 (with video recordings): S2020 website and S2020 GitHub branch
- Fall 2019: F2019 website and F2019 GitHub branch.
For a description of topics covered and course structure, see learning goals.
Annotated Bibliography
An (opinionated) annotated bibliography of academic papers in this space, covering a wide range of topics from research on testing to requirements to notebooks: https://github.com/ckaestne/seaibib
Open-Source ML Products
A curated set of open source products that use machine learning. These are all end-user products that incorporate machine learning models, not libraries, research prototypes, or notebooks. We hope that this lists facilitates research on building products beyond the model-centric view of analyzing ML components: https://github.com/mlip-cmu/awesome-ml-products. For more applications also see this list of Open Source Android Apps with LLMs.
Related Courses at CMU
- 17-649 Artificial Intelligence for Software Engineering: This course focuses on how AI techniques can be used to build better software engineering tools and goes into more depth with regard to specific AI techniques, whereas we focus on how software engineering techniques can be used to build AI-enabled systems. Our application scenarios are typical web-based systems for end users, rather than tools for software developers.
- 05-318 Human-AI Interaction: Focuses on the HCI angle on designing AI-enabled products. Overlaps in some coverage on fairness, covers in much more detail user interface design and how to involving humans in ML-supported decisions, whereas this course focuses more on architecture design, requirements engineering, and deploying systems in production. Both courses are complementary.
- 17-646 DevOps: Modern Deployment, 17-647 Engineering Data Intensive Scalable Systems, and similar: These course cover techniques to build scalable, reactive, and reliable systems in depth. We will survey DevOps, and big data systems in the context of designing and deploying systems, but will not explore them in as much detail as a dedicated course can. We will look at MLOps as a ML-specific variant of DevOps.
- 10-601 Machine Learning, 15-381 Artificial Intelligence: Representation and Problem Solving, 05-834 Applied Machine Learning, 95-865 Unstructured Data Analytics, 10-718: Machine Learning in Practice, and many others: CMU offers many course that teach how machine learning and artificial intelligence techniques work internally or how to apply them to specific problems (including feature engineering and model evaluation), often on static data sets. We assume a basic understanding of such techniques and processes (see prerequisites) but focus on the engineering process for production ML systems.
- 17-691 Machine Learning in Practice: Applied machine learning with an focus on deployment and monitoring of models.
- 15-884 Machine Learning Systems, 10-714 Deep Learning Systems: Courses that focus on the systems aspect of building ML libraries, such as distributed learning and using hardware acceleration.
- 17-630 Prompt Engineering: Course focused specifically on prompt engineering for large language models.
- 10-613 Machine Learning, Ethics and Society, 16-735 Ethics and Robotics, [05-899 Fairness, Accountability, Transparency, & Ethics (FATE) in Sociotechnical Systems], and others dive much deeper into ethical issues and fairness in machine learning, in some cases diving deeper into statistical notions or policy. We will cover these topics in a two-week segment among many others.