Part V - System cases, diagnosis, and release decisions

A release decision concerns the complete system, not one prompt, retriever, model, or tool in isolation. The evidence must show which component produced each result, which failures remain, and how the system behaves within its time, cost, and safety limits.

This Part combines task definitions, evaluation rows, retrieval traces, tool steps, saved-state changes, cost, and latency into system cases. Those records support choosing a sufficient design, investigating contributing failures, and deciding whether release requirements are met.

Chapter 12 follows evaluation, RAG, and support-agent cases through their records and checks, then tests a small runnable subset with local stand-ins. Chapter 13 connects architecture choices and failure diagnosis to release requirements for quality, evidence support, tool control, infrastructure, cost, and latency.

Chapter 12 shows a six-hour source versus a twelve-hour generated claim, a document-to-actual-request evidence path, and an authorized filter-to-complete-reference-to-count-to-check sequence ending at illustrative twenty-seven. A preview of at most twenty rows is associated with the record and is not counted. Chapter 13 shows optional components selected for a tested need, records and controlled diagnosis, release categories for used components, and illustrative ninety-three of one hundred below ninety-five of one hundred leading to HOLD. A separate PASS panel branches traffic concurrently to current and canary, with monitoring and a candidate stop or rollback route. Lower strips list all seven Chapter 12 sections and four Chapter 13 sections.
Figure 1: Task-specific records support diagnosis and release checks. The illustrative count uses the complete authorized result reference; dashed preview links are auxiliary associations, not count inputs. An assumed routing gate produces HOLD, while a separate conditional PASS permits monitored traffic shared between current and canary versions.

Chapters in this part