Part V: Adapting and Running LLMs

A pretrained model may lack relevant information, fail a required format, or generate too slowly for a task. Changing context, changing parameters, and scheduling inference affect different objects. Each choice needs evidence against the intended task and resource limits.

Chapter 20 compares prompting, retrieved evidence, full fine-tuning, preference-guided objectives, and smaller trainable additions. It separates the objective guiding an update from the storage format of the frozen base. Quantization can reduce weight payload while changing the numerical values used in computation.

Chapter 21 follows changing token and attention state with fixed model parameters in its cache and scheduling traces. It also compares how query heads share key and value heads, which is a model architecture choice. Its cache timeline supports memory estimates, request latency, and block allocation. Part VI then connects other input representations, tensor storage, and experiment evidence.

Two connected chapter panels show task evidence choosing a context or parameter change, followed by prompt processing, repeated token generation, stored attention state, scheduling, and resource measurements.
Figure V.1: Chapter 20 separates changes to input context, trainable parameters, and stored weight values. Chapter 21 follows fixed parameters through prompt processing, token selection, cache growth, scheduling, and latency measurement.

Chapters in this part