Part III: Running and improving the complete system

Updating a document collection can require validation, embedding, evaluation, and a controlled release. A failure between these steps leaves operators deciding which work can be repeated and which state is ready to serve.

Chapter 8 connects these steps through recorded dependencies, checked outputs, and retry rules. Chapter 9 combines the training, storage, serving, retrieval, and evaluation records to investigate a groundedness incident, check a repair, and decide whether to release it.

Chapters in this part

  • Running repeatable ML workflows: Every week the policy corpus changes. An update can require document validation and new embeddings before the index is rebuilt. Evaluation then informs the release decision. If a step fails halfway or runs twice, an operator needs its recorded inputs and output state to decide how to recover. The same records let another operator continue the work safely.
  • Keeping a grounded-answer service reliable: Suppose that after a routine weekly update, answers about newly added policies contain claims unsupported by the supplied passages and cite the wrong documents or none at all, while latency and error rates stay normal. The groundedness defined in the Introduction has fallen even though the service is available. The preceding chapters produced versioned artifacts, recoverable distributed jobs, durable storage, serving bundles, a versioned retriever, evaluation evidence, and retryable pipelines. This chapter operates them as one system whose release evidence connects such an alert to a verified recovery.