Introduction: The Operating Problem

A model can be impressive in an isolated experiment and still fail as a product. Production introduces traffic, failures, shared resources, changing data, delayed feedback about correctness, cost limits, and deployment risk. When a result changes or a job fails, the team also needs to know which code, data, model, and settings produced it. Together, these constraints turn a one-time result into a system whose behavior and changes the team can explain.

The supporting systems store model versions, allocate compute, handle requests, and record what happened. Their design depends on the workload. A service can use a pretrained model without running its own training job. Training or adaptation becomes relevant when the existing model does not meet the task’s needs.

The running case: A grounded-answer service

Suppose a company offers its employees a question-answering service over internal policy documents. A typical question is how many remote-work days a contractor may take. The application checks which documents this employee may read, retrieves relevant passages from the current policies, and supplies the question and passages to a large language model (LLM), a model that generates text from the supplied input. The application returns an answer with citations to the passages it used. Policies change weekly, so the service must answer from documents that the model never saw during training.

  • Retrieval-augmented generation (RAG): A design in which the application retrieves passages from a document collection at request time and supplies them to the model as context for generating the answer.1

  • Groundedness: The degree to which every claim in an answer is supported by the passages supplied to the model. A grounded answer can still be wrong if the supplied passage is outdated or incorrect.2

  • Hallucination: Generated content that the supplied evidence does not support or that contradicts it, such as an answer stating a renewal date that appears in no retrieved passage.3

The model has different roles in three operating modes. In inference, the serving system turns a question and passages into an answer using the model’s learned numerical settings. Each adjustable setting is a parameter (weight). In training, a job compares model predictions with target outputs and updates those weights. In evaluation, a job scores answers against reference labels or evidence and records a verdict without changing any weights.

Scope boundaries

The starting knowledge is basic programming, familiarity with code and data files, and arithmetic with rates and percentages. The reader can recognize a trained model but need not know how training clusters, serving engines, vector search, or workflow systems operate. Those ideas are explained where they are used.

The book focuses on the systems around a model: allocating compute, moving and storing data, scheduling requests, finding relevant passages, measuring performance and answer quality, and checking releases. Low-level GPU kernel authoring, proprietary hardware compilation, and model architecture design are outside its scope. Software products illustrate component roles. They are not required purchases or a prescribed deployment stack.

Sources and acknowledgment

This book combines research papers with standards and software documentation. Five core papers support these explanations:

  • The ML Test Score (2017) for testing, monitoring, and release checks in Chapters 4, 7, and 9.4
  • The ZeRO paper (2020) for training memory and communication calculations in Chapter 2.5
  • Retrieval-Augmented Generation (NeurIPS 2020) for the retrieval-and-generation design of the running case.6
  • The PagedAttention paper (SOSP 2023) for Chapter 4’s explanation of how serving systems manage memory while generating answers.7
  • The TFX paper (KDD 2017) for coordinated steps and shared services in Chapter 8’s machine learning workflows.8

Chapter 4 also uses the SGLang paper (NeurIPS 2024) to explain reuse of earlier computation for requests with the same opening text.9 It uses Fast Inference from Transformers via Speculative Decoding (ICML 2023) to explain drafting text with a smaller model and checking it with a larger one.10

Standards and product documentation support the details about software behavior and data formats.

The book was inspired by Nebius Academy’s AI Performance Engineering course.11

Nine operating stages and handoffs

The book follows nine operating stages in three parts. Making training runs repeatable and recoverable (Part I, Chapters 1–3) establishes run identity, shared training work, and recoverable state. Serving answers and assessing their quality (Part II, Chapters 4–7) connects serving and retrieval to request evidence and quality evaluation. Running and improving the complete system (Part III, Chapters 8–9) connects repeatable workflows to incident diagnosis and release checks.

Table 1: The roadmap connects the three Parts and nine operating stages to their results.
Part / stage Problem solved Result passed forward
Part I: Making training runs repeatable and recoverable Turn changing code, data, and compute into repeatable, recoverable training runs. Reproducible inputs, a recoverable checkpoint, and a versioned training bundle
1. Reproducible runs Identify the exact code, data, configuration, environment, and outputs of one run. Tracked run with versioned inputs and artifacts
2. Distributed training Use many devices while controlling memory, communication, scheduling, and recovery. Checkpoint contents, recovery objectives, and efficiency evidence
3. Storage for training Keep GPUs fed with data and publish only complete checkpoints on the right storage. Committed checkpoint generation and measured storage path
Part II: Serving answers and assessing their quality Serve the trained bundle, supply it with authorized evidence, and measure answer quality. Measured service behavior with grounded, traceable answers
4. Predictable serving Meet latency and throughput targets while managing batching, KV-cache memory, and releases. Versioned serving bundle with performance gates
5. Retrieval for grounded answers Build rebuildable embeddings and indexes and retrieve authorized passages with source identity. Versioned retrieval collection and cited passages
6. Request behavior and change signals Connect operations across requests and detect changes in inputs and predictions. Request traces, aggregate measurements, and cases to investigate
7. Experiments and quality evaluation Compare recorded runs, evaluate answers against evidence, and check the evaluator. Raw results, calibrated verdicts, and regression cases
Part III: Running and improving the complete system Automate repeatable work and turn production failures into verified improvements. Run evidence, release decisions, and regression prevention
8. Repeatable workflows Schedule dependencies, retry safely, and commit outputs as one reproducible run. Immutable run manifest with task and artifact lineage
9. Safe system operation Trace a groundedness incident to its cause, repair it, and add a gate that catches the same failure before another release. Regression case and evidence for the next pipeline run

System map

Versioned model files enter serving, while retrieval supplies authorized source passages for each answer. Request traces connect the operations to their release identities. Experiment and evaluation records support a proposed change, and workflow records identify the tasks and inputs that built it. Production measurements guide investigation, and a diagnosed incident becomes a regression case for later evaluation and workflow runs.

Learning path

Chapters 1–3 explain how to identify a run, share its training work, and recover its data and saved state. Chapters 4–7 connect serving and retrieval to request records and quality evaluation. Chapters 8–9 use those results in repeatable workflows and a complete incident investigation.

Reference material and appendices

The Formula reference collects the quantitative relationships and links to their explanations. Questions and concise answers provide conceptual and calculation practice. The Glossary collects definitions, and References gives complete details for the cited research, standards, and documentation. Footnotes identify the claim each source supports and its limits.

Table 2: The back matter collects formulas, practice questions, definitions, and cited sources.
Reference section Title Focus and content Used in
Appendix A Formula reference Formulas with links to their explanations Chapters 1–9
Appendix B Questions and concise answers Conceptual and calculation questions with answers Chapters 1–9
Glossary Glossary Definitions of terms introduced in the book Chapters 1–9
References References Complete details for cited research, standards, and documentation Chapters 1–9

  1. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. Curran Associates. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html. The original RAG paper combines a retriever with a generator for knowledge-intensive tasks. Its model trains both parts jointly, while the service in this book retrieves with a separately built index.↩︎

  2. Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2023). Measuring attribution in natural language generation models. Computational Linguistics, 49(4), 777–840. DOI: 10.1162/coli_a_00486. https://doi.org/10.1162/coli_a_00486. The Attributable to Identified Sources framework judges whether generated statements are supported by an identified source.↩︎

  3. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. DOI: 10.1145/3571730. https://doi.org/10.1145/3571730. The survey defines hallucination as generated content that is nonsensical or unfaithful to the provided source. This book uses the source-unfaithful sense for answers built from retrieved passages.↩︎

  4. Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017). The ML test score: A rubric for ML production readiness and technical debt reduction. In 2017 IEEE International Conference on Big Data (pp. 1123–1132). IEEE. DOI: 10.1109/BigData.2017.8258038. https://storage.googleapis.com/gweb-research2023-media/pubtools/4156.pdf. The paper proposes tests and monitoring needs drawn from production systems. This book supplies its own examples, release thresholds, and stop rules.↩︎

  5. Rajbhandari, S., Rasley, J., Ruwase, O., & He, Y. (2020). ZeRO: Memory optimizations toward training trillion parameter models (arXiv:1910.02054v3, May 13, 2020). arXiv. https://arxiv.org/abs/1910.02054v3. The paper analyzes memory and communication in distributed training. The chapter states the assumptions used in its numerical examples.↩︎

  6. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. Curran Associates. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html. The original RAG paper combines a retriever with a generator for knowledge-intensive tasks. Its model trains both parts jointly, while the service in this book retrieves with a separately built index.↩︎

  7. Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023). Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (pp. 611–626). ACM. DOI: 10.1145/3600006.3613165. https://arxiv.org/abs/2309.06180. Sections 4.1–4.2 explain allocation and sharing of memory blocks.↩︎

  8. Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., Koo, C. Y., Lew, L., Mewald, C., Modi, A. N., Polyzotis, N., Ramesh, S., Roy, S., Whang, S. E., Wicke, M., … Zinkevich, M. (2017). TFX: A TensorFlow-based production-scale machine learning platform. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1387–1395). ACM. DOI: 10.1145/3097983.3098021. https://storage.googleapis.com/gweb-research2023-media/pubtools/4795.pdf. TFX describes the components and shared services of Google’s production platform.↩︎

  9. Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., Barrett, C., & Sheng, Y. (2024). SGLang: Efficient execution of structured language model programs. Advances in Neural Information Processing Systems, 37, 62557–62583. https://papers.nips.cc/paper_files/paper/2024/hash/724be4472168f31ba1c9ac630f15dec8-Abstract-Conference.html. Section 3 explains how the runtime saves and reuses computation for matching request prefixes.↩︎

  10. Leviathan, Y., Kalman, M., & Matias, Y. (2023). Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning (PMLR 202, pp. 19274–19286). PMLR. https://proceedings.mlr.press/v202/leviathan23a.html. The paper gives the sampling rules for checking drafts against the larger model’s probabilities. Speed gains depend on the draft model, hardware, and workload.↩︎

  11. Nebius Academy. (n.d.). AI Performance Engineering. Retrieved October 16, 2026, from https://academy.nebius.com/ai-engineering-il.↩︎