Library Map
Library operations need links to their mathematical roles, executable examples, and full experiment requirements.
The examples in this book use Python libraries to prepare data, calculate model outputs, train models, and evaluate predictions. A library’s application programming interface (API) exposes operations through functions, classes, and their expected inputs and outputs. The links below connect each operation with its mathematical explanation and code. Supporting notebooks provide broader experiments, with common requirements in the execution reference and task procedures in Chapter 24.
| Role | Operation | Explanations and code |
|---|---|---|
| Token row selection | nn.Embedding |
Lookup and gradients |
| Affine class scores | nn.Linear |
Supplied-vector head |
| Aligned target loss | F.cross_entropy |
Ignored-target mean |
| Derivative accumulation | loss.backward() |
Fresh graphs |
| Parameter updates and gradient clearing | step(), zero_grad() |
Running-batch SGD |
Tabular data and tokenization. pandas.DataFrame keeps text-label pairs together in §1.1, with runnable preparation code. transformers.AutoTokenizer turns text into IDs and a padding mask in §2.2, with batch code requiring cached or downloaded tokenizer files. Chapter 3 develops tokenizer algorithms. Its WordPiece score calculation is not a complete tokenizer trainer. The text and recurrence assignment provides the BPE and WordPiece exercises associated with §24.2.
Fixed document features. scikit-learn vectorizers fit vocabulary and document-frequency information on training text, then reuse it for new rows. §4.2 distinguishes count and TF-IDF conventions and provides the executable vectorizer example. Its returned matrices use SciPy sparse storage. The logistic-regression notebook uses bag-of-words features for the SST-2 sentiment exercise.
Arrays and affine layers. PyTorch tensors hold the token batch, class-score matrix, and three-row affine calculation. The last two use explicit multiplication. torch.nn.Linear is explained as an affine layer in §5.3. Its first code use is in the runnable classifier head in §12.3. Chapter 23 develops array storage, numerical types, strides, and devices. NumPy arrays appear in the projection setup of §14.4 and the tensor conversions of §23.3.
Probabilities, loss, and updates. §6.2 contains sigmoid and softmax code. torch.optim supplies the update procedures explained in Chapter 10. §11.5 provides the complete hand calculation, the running-batch optimizer example using torch.optim.SGD, and the random-batch forward, backward, and update code. The masked target check in §8.5 verifies alignment and the included-position mean. The logistic-regression lab adds repeated batches, validation, and optimizer comparisons to these local checks.
Evaluation. §9.2 explains confusion counts and classification metrics. The metric code uses sklearn.metrics in §9.3, after ranking and calibration are explained. The text-classification exercise in §24.2 requires held-out results and error inspection in addition to these calculations.
Embeddings and count models. gensim trains the Word2Vec example in §14.2. The representation lab compares learned vectors and downstream classifiers. nltk connects the count estimator in §3.4 to the original count-model tutorial and §24.6, which covers counting and generation. The original tutorial continues through generation, saving, and reloading the corpus model. Smoothing and held-out perplexity are book extensions in §24.6.
Recurrence and attention plots. torch.nn.RNN and torch.nn.LSTM are discussed in Chapter 15, with an LSTM dimension example in §15.4 and a character-model exercise. Matplotlib and Seaborn render the attention heatmap code in §16.2. The supporting attention notebook supplies the controlled comparison in §24.3.
Generation, training interfaces, and adapters. GenerationConfig and model.generate are interface references in Chapter 7, whose code example selects one token rather than running a full generation loop. datasets supplies load_dataset, used with pinned dataset revisions in §24.1 and §24.2. peft and bitsandbytes appear in the adapter configuration fragment in §20.7. The supporting fine-tuning notebook supplies the wider data and model experiment associated with §24.4.
Other modalities and serving. Chapter 22 introduces image, audio, and table representations, including patch-construction code. The supporting multimodal notebook underlies the captioning and visual-question-answering work in §24.7. The serving interface reference records vLLM, SGLang and TensorRT-LLM. Chapter 21 develops cache and scheduling mechanisms, with vLLM as its documented implementation example.
The environment and supporting notebooks provide notebook access, dataset and checkpoint requirements, environment versions, and evidence from experiments. The Glossary supplies short definitions, and the reading paths locate the mathematical prerequisites.