Part V: Adapting and Running LLMs
A pretrained model may lack relevant information, fail a required format, or generate too slowly for a task. Changing context, changing parameters, and scheduling inference affect different objects. Each choice needs evidence against the intended task and resource limits.
Chapter 20 compares prompting, retrieved evidence, full fine-tuning, preference-guided objectives, and smaller trainable additions. It separates the objective guiding an update from the storage format of the frozen base. Quantization can reduce weight payload while changing the numerical values used in computation.
Chapter 21 follows changing token and attention state with fixed model parameters in its cache and scheduling traces. It also compares how query heads share key and value heads, which is a model architecture choice. Its cache timeline supports memory estimates, request latency, and block allocation. Part VI then connects other input representations, tensor storage, and experiment evidence.
Chapters in this part
- Model adaptation: Context, preferences, and trainable weights: Adaptation can change context, parameters, or both. Its objective, trainable state, and storage format are separate choices.
- KV cache: Generation state, latency, and allocation: Cached generation reuses attention state. Scheduling and allocation affect request latency and memory use.