Part I: From Text to Mathematical Objects

Text is a flexible sequence of characters, while model calculations require numbers arranged in explicit structures. The useful structure depends on the task. A classifier may use fixed feature coordinates, while a language model needs an ordered token sequence.

The introduction separated inference, training, and evaluation. Each mode still depends on an earlier preparation step: specify the input, define the required output, and retain any reference information needed to judge that output. This part constructs those objects before later parts calculate predictions and parameter updates.

Chapter 1 pairs inputs with targets and shows how individual prediction errors form an objective. Chapter 2 maps text units to integer IDs and explains rectangular batches. Chapter 3 constructs reusable subword pieces and a count-based prediction baseline. Chapter 4 turns document terms into fixed feature columns and class scores. The resulting token sequences and feature rows become the inputs for the score, probability, loss, and evaluation calculations in Part II. The count-based predictor is a sequence baseline, not a required stage before the document classifier.

Four numbered chapter panels form a branching map. Chapters 1 and 2 are on the left and center. Arrows then split to Chapter 3 above and Chapter 4 below, and both branches point to a lower box for scores, probabilities, loss, and evaluation. Panel 1 pairs inputs with targets and shows prediction loss. Panel 2 maps text to token IDs and pads two sequences with a validity mask. Panel 3 builds subword pieces and predicts the next token from counts for a fixed earlier context. Panel 4 places document counts and TF-IDF values in matrices and turns them into raw class scores.
Figure I.1: Chapter 1 pairs model inputs with the correct answers and combines prediction errors. Chapter 2 maps text to token IDs and batches unequal sequences with padding and a validity mask. Chapter 3 builds subword pieces and estimates next token probabilities from a fixed amount of earlier text. Chapter 4 turns document terms into fixed features and raw class scores. The sequence and document branches both lead to the probability, loss, and evaluation work in Part II.

Chapters in this part

  • Prediction targets and training pairs: In supervised and self-supervised training, inputs are paired with reference answers and prediction errors are combined.
  • Token IDs and batches: A shared vocabulary mapping makes token IDs consistent between tokenizer and model. Padding masks record added batch positions.
  • Subwords and sequence counts: Reusable subword pieces balance vocabulary size against sequence length, while continuation counts supply a fixed-history prediction baseline.
  • Count vectors and TF-IDF: Document terms need shared feature coordinates before the same classifier weights can score texts of different lengths.