Part IV: Serving systems
When included, the optional decode and serving labs are in the Optional Labs appendix. They collect longer cache, trace and decoding probes without supplying setup required by the main chapters.
Chapters in this part
- 13. Request scheduling and prefix reuse: Trace admission, batching, compatible prefix reuse and prefill/decode handoff under a stated load.
- 14. Speculative and constrained decoding: Trace exact sampling correction separately from constraint-state validity.
- 15. Serving measurement and stack selection: Collect and interpret request outcomes/cost before selecting a service stack.
