11 Backpropagation
Parameter gradients need all contributing derivative paths before an optimizer can apply a consistent update.
A parameter can affect the loss through many intermediate operations. The optimizer needs their combined sensitivity, not just the derivative of the last operation. Backpropagation applies the derivative chain rule in reverse dependency order to compute these gradients.1
The local slopes and chain rule from §5.4 provide the foundation. Graphs organize longer calculations, and array derivatives account for multiple dependent coordinates. Computing a gradient and applying an optimizer step remain separate operations.
11.1 Forward dependencies and saved values
The output value alone does not identify how it depends on a parameter. At \(x=2\), both \(x^2\) and \(2x\) give 4, but their derivatives are 4 and 2. The backward pass uses the operations that produced the result, together with values required by their derivative rules. The derivative chain rule from §5.4 multiplies local sensitivities along each dependency path.
A computation graph records the operations and dependencies that produced a result. In a value-based drawing, nodes represent values and edges show which earlier values an operation uses. PyTorch’s automatic differentiation system, autograd, records differentiable operations during forward execution when gradient tracking is enabled.2
A leaf tensor requiring gradients has no recorded operation producing it. Directly created trainable weights commonly have this role. Results of tracked operations are non-leaf tensors, with operation history accessible through grad_fn.
For fixed scalar input \(x\), weight \(w\), bias \(b\), and the positive target \(y=1\), the calculation is
\[ z = wx+b,\quad p=\sigma(z),\quad L=-\log(p) \tag{11.1}\]
The logit \(z\) becomes probability \(p\), and the positive-target loss is \(L=-\log p\). A negative target would require \(-\log(1-p)\) instead. The graph connects \(L\) to both parameters through these intermediate values.
Example: Local derivatives in the scalar graph
Use \(x=2\), \(w=0.5\), \(b=0.1\) and positive target \(y=1\). Numerical results in this chapter are displayed to six decimal places when needed. Calculations retain their unrounded values.
Multiplication gives \(z_{\mathrm{mul}}=2(0.5)=1.0\). Adding the bias gives \(z=1.1\), probability \(p\approx0.750260\) and loss \(L\approx0.287335\) nats.
At these values, \(dL/dp=-1/p\approx-1.332871\) and \(dp/dz=p(1-p)\approx0.187370\). Their product is \(dL/dz=p-1\approx-0.249740\). The affine derivatives \(dz/dw=2\) and \(dz/db=1\) give \(dL/dw\approx-0.499480\) and \(dL/db\approx-0.249740\).
Using the rounded value 0.750 inside the logarithm would give \(-\log(0.750)\approx0.287682\), which rounds to 0.288 at three decimals. Using the unrounded probability gives 0.287 at three decimals.
Conclusion: Multiplying derivatives through the recorded operations recovers \(p-y\). The weight gradient includes the input factor, while the bias gradient does not. Rounding only the displayed results preserves that agreement.
Different derivative rules need different saved information. Multiplication may need its operands, while sigmoid’s derivative can use its output. Autograd saves the data required by the backward rules, not necessarily every intermediate tensor forever. Saving or recomputing intermediate activations costs memory and work.
Setting requires_grad=True requests gradient tracking for a tensor. It does not add that tensor to an optimizer’s parameter list. torch.no_grad() disables recording of operations in its context, which is useful for ordinary parameter updates. Detaching an intermediate removes its earlier path from the differentiated computation. In-place changes to saved operands can invalidate the backward calculation.
Saved graph data and .grad buffers serve different purposes. The first supports derivative computation. The second stores accumulated derivatives on selected tensors. Non-leaf gradients can be used during propagation without being retained in .grad afterward.
11.2 Array derivatives and vector-Jacobian products
An array operation can have many input-output dependencies, yet training usually needs the gradient of one scalar loss. Constructing every derivative entry explicitly would often waste memory and work.
For input \(x\in\mathbb R^n\) and output \(y\in\mathbb R^m\), the Jacobian is the \(m\)-by-\(n\) matrix
\[ J_{i,j}=\frac{\partial y_{i}}{\partial x_{j}} \tag{11.2}\]
Row \(i\) enumerates output \(y_i\) and column \(j\) enumerates input \(x_j\). The upstream gradient supplies the loss sensitivities to the outputs, calculated through operations that occur later in the forward calculation.
Using a row-gradient convention, \(dL/dy\) has shape \([1,m]\). Its vector-Jacobian product has shape \([1,n]\):
\[ \frac{dL}{dx} = \frac{dL}{dy}\,J_{y,x} \tag{11.3}\]
Each input coordinate receives a contribution from every output that depends on it:
\[ \frac{\partial L}{\partial x_{j}}=\sum_{i}\frac{\partial L}{\partial y_{i}}\,\frac{\partial y_{i}}{\partial x_{j}} \tag{11.4}\]
Here \(i\) runs over the \(m\) outputs and \(j\) selects one input. The multiplication expresses these sums. An implementation can compute the product directly without materializing the full Jacobian.
Example: Two output contributions to one input
For \(y=(x_1+2x_2,3x_1-x_2)\), the Jacobian is \(J=\begin{bmatrix}1&2\\3&-1\end{bmatrix}\). With upstream gradient \((2,3)\), the input gradient is \((2,3)J=(11,1)\).
The first coordinate receives \(2(1)+3(3)=2+9=11\). The second receives \(2(2)+3(-1)=4-3=1\).
Conclusion: Output sensitivities weight the Jacobian’s rows. Adding their contributions gives one derivative per input coordinate, with the declared output-by-input orientation.
A column-gradient convention would use \(J^{\top}\) times the upstream column. It describes the same calculation with transposed notation. For a nonscalar output, backward needs these upstream weights to specify the combination being differentiated.
11.3 Path sums and repeated backward calls
A shared input can feed several branches of a graph. Its loss sensitivity must include every branch. Within one backward pass, reverse differentiation multiplies local derivatives along paths and adds their contributions:
\[ \nabla_x L=\sum_{\text{paths }p}\operatorname{contribution}_{p} \tag{11.5}\]
The index \(p\) enumerates paths from the shared value \(x\) to the scalar loss. A path contribution is the product of its local derivatives, using the derivative chain rule from §5.4 on each edge. Reverse mode reuses each node’s accumulated sensitivity when propagating it to earlier operations. For a concrete branched graph, set \(a=x^2\), \(b=5x\), and \(L=a+b\). At \(x=1\), the forward values are \(a=1\), \(b=5\), and \(L=6\). The first branch contributes \((dL/da)(da/dx)=1(2x)=2\). The second contributes \((dL/db)(db/dx)=1(5)=5\). Their sum gives \(dL/dx=7\), agreeing with direct differentiation of \(x^2+5x\). Addition combines branches, while multiplication connects successive operations.
Gradient accumulation also occurs across backward calls because leaf .grad buffers retain their values. This storage behavior is distinct from adding branches inside one derivative calculation.
Ordinary backward normally frees saved graph data after using it. Calling backward again on the same graph can therefore fail. A fresh forward calculation constructs a new graph, but it does not clear the leaf’s existing gradient. Deliberate graph retention is another option when the same graph must be differentiated again, with an additional memory cost.
The runnable PyTorch example uses a one-element tensor x with value 2. It first computes y with value 13, then differentiates \(y=3x^2+1\) to obtain \(6x=12\).
Code example: Fresh forward graphs accumulate into the same leaf gradient buffer
import torch
x = torch.tensor([2.0], requires_grad=True)
y = 3 * x ** 2 + 1
y.backward()
print(x.grad) # tensor([12.]), since the derivative is 6x
y = 3 * x ** 2 + 1 # A fresh graph, with the same leaf x.
y.backward()
print(x.grad) # tensor([24.]), because its gradient was not cleared.The two printouts are tensor([12.]) and tensor([24.]). Recomputing y permits the second backward pass. Since x.grad was not cleared, the second derivative adds another 12. The tensor shape is [1], not scalar shape [].
A backward call without an explicit upstream gradient is valid for a result containing one element. For multiple elements, the caller supplies the upstream weights described in §11.2. Calling retain_grad() on a non-leaf tensor requests its .grad buffer for inspection. It is not required merely to propagate through that tensor.
Backpropagation changes gradient buffers, not the parameter values. The optimizer performs the parameter update afterward.
11.4 Parameters, gradient buffers, and optimizer state
After backward, a parameter can have a new gradient while retaining exactly its old value. An optimizer step reads that gradient and any stored update state to change the parameter.
One independent batch cycle has the following order:
optimizer.zero_grad()clears the tracked parameters’ previous gradients. The API can set them toNoneor fill existing buffers with zeros.- The forward calculation uses the current parameters to produce predictions and a scalar loss.
loss.backward()adds derivatives to the relevant leaf gradient buffers. A parameter unused by the graph need not receive a gradient.- Optional clipping modifies the completed gradients.
optimizer.step()then updates its parameter list using those buffers and any momentum or Adam state.
The next forward calculation uses the changed parameters. The previously calculated loss still describes the old version. The step does not compute a new loss, calculate missing derivatives, or automatically clear gradients.
Intentional accumulation can combine several batches before an update. The loss scaling must match the intended sum or mean, especially when batch sizes differ. Accidental accumulation changes the gradient the optimizer consumes.
The clipping rule from §10.7 can now be checked within this cycle. The runnable PyTorch example uses one weight of 2 and its squared loss. nn.Parameter makes it trainable, and nn.ParameterList exposes it through model.parameters(). Backward fills the gradient before clip_grad_norm_ limits its norm, then SGD applies the update.
Code example: A supplied scalar model, clipped gradient, SGD step, and gradient reset
import torch
import torch.nn as nn
model = nn.ParameterList([nn.Parameter(torch.tensor(2.0))])
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
loss = model[0].square()
loss.backward()
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
assert model[0].grad.norm().item() <= 1.0
optimizer.step()
assert torch.allclose(model[0], torch.tensor(1.9))
optimizer.zero_grad()
print(model[0].detach())Conclusion: The loss is 4 and the initial gradient is 4. Clipping reduces the gradient to approximately 1, so SGD at rate 0.1 changes the weight to approximately 1.9. The assertions check the bound and new parameter. The final reset clears the gradient buffer while retaining the changed weight.
The supporting gradient notebook connects these operations to a manual logistic training loop. Its repeated training experiment is separate from the small numerical examples below.
11.5 A complete classifier calculation and update
A classifier update can be checked by following the same input, target, and parameter version through prediction, loss, gradients, and the new prediction. Local derivatives must agree with the forward values. A batch mean must divide every parameter-gradient sum by the same example count. The calculations here connect the affine map from Chapter 5, probabilities from Chapter 6, and binary cross-entropy from Chapter 8.
The single-example calculation uses a feature vector \(x\), weights \(w\) of the same width, scalar bias \(b\), and target \(y\in\{0,1\}\). It applies §5.2’s affine score, §6.1’s sigmoid, and §8.1’s binary cross-entropy in that order. The update examples below follow these formulas at the original parameters and recompute the loss after changing them.
The binary loss uses the probability assigned to the reference class, with natural logarithms giving nats. Its logit derivative from §8.4 is \(\partial L/\partial z=p-y\). The remaining derivatives follow the affine map:
\[ \frac{dL}{dw}=\frac{dL}{dz}\,\frac{dz}{dw} \tag{11.6}\]
\[ \frac{dz}{dw}=x \tag{11.7}\]
In the scalar equation, \(x\) is held fixed when differentiating with respect to \(w\). For a feature vector, coordinate \(j\) instead gives \(\partial z/\partial w_j=x_j\). The bias derivative is 1, so its loss gradient is also \(p-y\).
Example: One two-feature logistic update
Let \(x=(2,-1)\), \(w=(0.5,-0.3)\), \(b=0.1\), and \(y=1\). These are the classification weights for this example, separate from the regression weights used for squared error in §5.4.
The score is \(z=2(0.5)+(-1)(-0.3)+0.1=1.4\). Its probability is \(p\approx0.802184\), and the positive-label loss is \(L=-\log p\approx0.220417\) nats.
The logit derivative is \(p-y\approx-0.197816\). Multiplication by the feature coordinates gives the weight gradient approximately \((-0.395632,0.197816)\). The bias gradient is approximately \(-0.197816\).
With learning rate \(\eta=0.1\), subtracting the unrounded gradients gives \(w'\approx(0.539563,-0.319782)\) and \(b'\approx0.119782\). On the same input, \(z'\approx1.518690\) and the recomputed loss is approximately 0.198030.
Conclusion: This step raises the score of the positive example and lowers its loss from about 0.220417 to 0.198030. All update values come from the original parameter version. Recomputing the loss verifies this finite step. It does not establish convergence or improvement on other examples.
For the running batch, let \(X\) have shape \([B,F]\), \(w\) shape \([F]\), and \(b\) be scalar. Then \(z=Xw+b\), probabilities \(p\), and labels \(y\) all have shape \([B]\). Using the mean binary loss gives logit sensitivities \((p-y)/B\). Consequently, the weight gradient is \(X^{\top}(p-y)/B\) and the bias gradient is \(\sum_i(p_i-y_i)/B\). The transpose sums contributions from every row to the shared feature weights.
Example: One update of the running batch
The running-batch loss in §8.3 uses \(X=[[1,0],[0,1],[1,1]]\), targets \(y=[1,0,1]\), weights \(w=[0.8,-0.3]\), and bias \(b=0.1\). That section calculated probabilities \(p\approx[0.710950,0.450166,0.645656]\) and mean loss approximately 0.458927 at this parameter version. This example calculates how the mean loss changes the shared parameters.
Before division by three, \(p-y\approx[-0.289050,0.450166,-0.354344]\). Rows 1 and 3 contribute to the first weight, giving mean gradient approximately \(-0.214465\). Rows 2 and 3 contribute to the second, giving approximately \(0.031941\). All rows contribute to the bias, whose mean gradient is approximately \(-0.064409\).
A rate of 0.1 gives \(w'\approx[0.821446,-0.303194]\) and \(b'\approx0.106441\). Repeating the forward and mean-loss calculations with the unrounded updated parameters gives mean loss approximately 0.453861.
Conclusion: The shared weights combine evidence from all three rows. Row 2 contributes against the other rows to the bias update, and averaging keeps the update consistent with the mean objective. This checked batch step reduces that objective from about 0.458927 to 0.453861, without a claim about held-out quality.
The runnable PyTorch code below executes this same three-row mean-loss update with torch.optim.SGD. Its learning rate is 0.1 and momentum is disabled. The printed loss is approximately 0.458927, followed by the updated weights and bias calculated above.
Code example: One optimizer step on the running logistic-regression batch
import torch
import torch.nn.functional as F
X = torch.tensor([[1.0, 0.0], [0.0, 1.0], [1.0, 1.0]])
y = torch.tensor([1.0, 0.0, 1.0])
w = torch.tensor([0.8, -0.3], requires_grad=True)
b = torch.tensor(0.1, requires_grad=True)
optimizer = torch.optim.SGD([w, b], lr=0.1)
optimizer.zero_grad()
loss = F.binary_cross_entropy_with_logits(X @ w + b, y)
loss.backward()
optimizer.step()
print(loss.item(), w.detach(), b.detach())The printed loss belongs to the pre-update forward calculation. step() changes the parameters but does not recalculate that stored value. This agrees with the hand calculation.
For this mean binary loss, logits and target labels must have matching shapes and the reduced loss is scalar. A stored nn.Linear weight receives a gradient with the same shape and orientation, as §5.3 specifies.
The backward, update, and reset cycle in §11.4 also applies to a manual update. The subtraction below runs inside torch.no_grad() so that update arithmetic is not recorded as part of the training graph.
The following supporting example uses a separate random batch with 32 rows and four features. Targets and logits both have shape [32,1], weights [4,1], bias [1], and the mean loss scalar shape []. It illustrates manual state changes rather than checking the supplied numerical batch.
Code example: A minimal binary logistic regression step
import torch
X = torch.randn(32, 4)
y = torch.randint(0, 2, (32, 1)).float()
w = torch.randn(4, 1, requires_grad=True)
b = torch.zeros(1, requires_grad=True)
logits = X @ w + b
loss = torch.nn.functional.binary_cross_entropy_with_logits(logits, y)
loss.backward()
with torch.no_grad():
w -= 0.1 * w.grad
b -= 0.1 * b.grad
w.grad.zero_(); b.grad.zero_()Conclusion: The manual update changes the weights and bias, then leaves both gradient buffers filled with zeros. The inputs and labels stay fixed. The stored loss still belongs to the earlier parameter values. Checking the new loss requires another forward calculation. This unseeded snippet has no printed numerical result, and its random inputs vary between runs.
Correct differentiation cannot add relationships that the model has no way to represent. An affine boundary still cannot separate every target pattern. Chapter 12 uses nonlinear transformations to address that limitation.
Chapter checkpoint
If the running batch used summed loss instead of mean loss, how would its gradients change? Would one lower training loss establish generalization?
Answer: With three examples, the sum’s parameter gradients are three times the mean’s gradients. At the same learning rate the proposed step would therefore be three times as large, so the checked mean-loss update cannot be reused unchanged. A lower loss on this batch establishes only that local result. Separate representative examples are needed to assess generalization.
Two branches connect a parameter to the loss. Their path contributions are 3 and -1. What gradient is computed for the parameter? If backward() is then called again without clearing its .grad buffer and contributes 2, what value is stored?
Answer: The branches add, so the first backward pass contributes 2. Gradient buffers accumulate across backward calls, so the second contribution raises the stored value to 4. Clearing the buffer and changing the parameter are separate operations.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536.↩︎
PyTorch Contributors. (2026). Autograd mechanics. PyTorch 2.12 documentation.↩︎