$L=\tfrac12(x_1+2x_2-3)^2$, now at $x=(0,0)$. Which step of length $\varepsilon=0.1$ lowers $L$ the most?

1. A$(\varepsilon,\,0)$
2. B$(0,\,\varepsilon)$✓
3. C$(-\varepsilon,\,0)$

New losses $4.205$, $3.92$, $4.805$, against $4.5$ where we started.

B is the best of these three. The steepest direction of all is not on offer.

$e\in\R^m$

prediction minus target

$L\in\R$

one number, and never negative

$\tfrac12$

convenience: it moves values, not minimisers

Lecture 3 measured $r=y^\star-\hat y$, so $e=-r$.

$x\in\R^3$, $W\in\R^{2\times3}$, $\hat y=Wx$, $L=\tfrac12\|\hat y-y^\star\|^2$. What shapes are $L$, $\nabla_xL$ and $\nabla_WL$?

1. A$1$, $3\times1$, $2\times3$✓
2. B$2\times1$, $3\times1$, $3\times2$
3. C$1$, $1\times3$, $3\times2$

A gradient carries the shape of the variable it differentiates. Row conventions exist; ours is columns.

1. Local slopes, gradients, Jacobians
2. The chain rule, in both directions
3. Differentials for vectors and matrices
4. Graphs, then a whole small network

One derivative, three checks: a tensor program, autograd, a finite difference.

$f$

differentiable at the expansion point

$h$

a small step, either sign

$f'(x)$

the slope at the expansion point

$o(|h|)$

the rest: divided by $|h|$, it tends to $0$

$f(x)=x^2$ at $x=1$. The tangent predicts $1+2h$; the curve gives $(1+h)^2$. How does actual minus predicted behave?

1. Alike $h$
2. Blike $h^2$✓
3. Czero for every $h$

$(1+h)^2-(1+2h)=h^2$: $0.25$ at $h=0.5$, $0.01$ at $h=0.1$. Exact only at $h=0$.

$s$

$x_1+2x_2-3$, the value inside

$\delta=\Delta s$

the change the step makes in $s$

first-order term

$s\delta$; it vanishes when $s=0$

Step $h$ in $x_1$: $\delta=h$, so $\partial_1L=s$.

Step $h$ in $x_2$: $\delta=2h$, so $\partial_2L=2s$.

At $x=(0,0)$, what is $\nabla L=(\partial_1L,\ \partial_2L)\T$?

1. A$(-3,-6)$✓
2. B$(-3,-3)$
3. C$(3,6)$

$s=-3$, and the slices give $s$ and $2s$. Both slopes are signed derivatives per unit of their own coordinate, not a ranking of importance when the units differ.

$\nabla L(x)$

partials at $x$

$dL$

loss change predicted by the linear model

$\nabla L\T\Delta x$

predicted change

$+\,L(x)$

predicted new loss

$v$

$\|v\|_2=1$

$\phi$

angle from $v$ to $\nabla L$

$\nabla L\ne0$

largest at $v=\nabla L/\|\nabla L\|_2$

Back at $x=(0,0)$. Which unit direction has $D_vL=0$?

1. A$(1,2)/\sqrt5$
2. B$(2,-1)/\sqrt5$✓
3. C$(-1,-2)/\sqrt5$

$\nabla L\T v$ gives $-3\sqrt5$, $0$, $3\sqrt5$. Here B does better than first order: $L$ is constant along that whole line.

$x,dx$

input and its increment, in $\R^n$

$y,dy$

output and its increment, in $\R^m$

$J_f$

$m\times n$, and it changes with $x$

$f:\R^3\to\R^2$. Which shape must $J_f$ have?

1. A$3\times2$(3×2)(3×1) ✗
2. B$2\times3$✓(2×3)(3×1) = 2×1
3. C$2\times2$(2×2)(3×1) ✗

$dy=J_f\,dx$ with $dx$ of size $3\times1$ and $dy$ of size $2\times1$: rows are outputs, columns are inputs.

$J_f$

at $x=(1,2)$

$J_g$

from $g=(y_1+y_2,2y_2)$ at $y=(5,2)$

$J_gJ_f$

$dx$ to $dz$

$dx=e_1$ gives $dy=(1,2)$

and then $dz=(3,4)$

$J_gJ_f=\begin{bmatrix}1&1\\0&2\end{bmatrix}\begin{bmatrix}1&2\\2&1\end{bmatrix}=\begin{bmatrix}3&3\\4&2\end{bmatrix}$

$k$

runs over every intermediate coordinate

shapes

$J_g:p\times m$, $J_f:m\times n$, product $p\times n$

order

$g$ after $f$ gives $J_gJ_f$, not the reverse

Multiply along a route, add across routes.

$f:\R^2\to\R^3$ then $g:\R^3\to\R$, with $L=g(f(x))$. Which product is $\partial L/\partial x$?

1. A$J_fJ_g$(3×2)(1×3) ✗
2. B$J_gJ_f$✓(1×3)(3×2) = 1×2
3. C$J_g+J_f$1×3 + 3×2 ✗

Inner dimensions must match, and a sum needs two equal shapes. Only B is defined, and it is $1\times2$, the shape of $(\nabla_xL)\T$.

$dy$

an output increment, $\R^m$

$\nabla_yL$

output sensitivity, $\R^m$

$\nabla_xL$

input sensitivity, $\R^n$

$=(\nabla_yL)\T J_f\,dx$ : put $dy=J_f\,dx$ in

$=(J_f\T\nabla_yL)\T dx$ : move $J_f$ across the pairing

since $(J_f\T\nabla_yL)\T=(\nabla_yL)\T J_f$

$\nabla_xL=J_f\T\nabla_yL$ : the coefficient of $dx$

$Jv$

JVP: $v$, an input direction, pushed forward

$u\T J$

VJP: $u=\nabla_yL$, an output loss sensitivity, pulled back

both

available without ever forming $J$

$J$ is $2\times3$ and $g=\nabla_yL\in\R^2$. Which product is $\nabla_xL$?

1. A$Jg$(2×3)(2×1) ✗
2. B$J\T g$✓(3×2)(2×1) = 3×1
3. C$J^{+}g$3×1, other question

The middle caption is the question. $J^{+}g$ is the minimum-norm least-squares input for a requested $g$, which need not be reachable. With a square $J$ all three shapes agree, so a shape check goes quiet.

$\bar L$

the seed $\partial L/\partial L=1$

cost

one forward sweep per input direction, one reverse sweep per output direction

memory

reverse mode saves or recomputes the forward values its local rules need

Both modes are the same chain rule.

Assign reverse mode or one forward JVP to jobs A, B and C.

1. 1A reverse · B forward · C forward✓
2. 2A forward · B reverse · C reverse
3. 3A reverse · B reverse · C reverse

C is one JVP along the supplied $v$, with no Jacobian built. Judge by the product you need, not by what networks usually do.

$x,\,dx\in\R^2$

the variable and a linear increment

$a=(2,-1)$

fixed coefficients

$dL\in\R$

the matching change in the loss

$\nabla_xL$

shaped like $x$

Match $dL=(\nabla_xL)\T dx$ for every $dx$: $dx=e_j$ reads off entry $j$. $d$ means differential, not a number to cancel.

$A,B\in\R^{m\times n}$

same shape, paired cell by cell

$\operatorname{tr}$

sum of the diagonal entries

$dL=\langle\nabla_WL,dW\rangle_F$

the matrix version of $a\T dx$, shaped like $W$

$L(W)=2W_{11}-W_{12}+3W_{21}+0\cdot W_{22}$. Which matrix is $\nabla_WL$?

1. A$\begin{bmatrix}2&3\\-1&0\end{bmatrix}$
2. B$\begin{bmatrix}2&-1\\3&0\end{bmatrix}$✓
3. C$2W$

Each cell holds the coefficient of its own entry. A transposes them; $2W$ is $\nabla_W\|W\|_F^2$.

$x\in\R^n$

the variable

$(dx)\T x=x\T dx$

each term is a number

$\nabla L=x$

shaped like $x$

Without the $\tfrac12$ the gradient would be $2x$.

$A\in\R^{n\times n}$

constant in $x$

$(dx)\T Ax$

$x$ as the left factor

$x\T A\,dx$

$x$ as the right factor

$\nabla$

two routes, added

So $\nabla(x\T Ax)=(A+A\T)x$; $A$ need not be symmetric, and $\nabla(\tfrac12x\T Hx)=Hx$ when $H=H\T$.

$L=2x_1^2+\tfrac12x_2^2=\tfrac12x\T Hx$ with $H$ diagonal. Which $H$?

1. A$\mathrm{diag}(2,\tfrac12)$
2. B$\mathrm{diag}(4,1)$✓
3. C$\mathrm{diag}(1,4)$

$\nabla L=Hx=(4x_1,x_2)$. In $x\T Kx$ the coefficient matrix is $K$ itself, and the Hessian is $2K$ for symmetric $K$.

$A\in\R^{3\times2}$

constant

$e\in\R^3$

prediction minus target

$\nabla_xL$

shaped like $x$

$de=A\,dx$

$dL=e\T de=e\T A\,dx$

$\nabla_xL=A\T(Ax-b)$

Lecture 3’s $r=b-Ax$ is $-e$, so $\nabla_xL=-A\T r$.

$r=b-Ax$

the residual of Lecture 3

$A\T r=0$

$r\perp$ every column, and $r\ne0$ in general

$\nabla L=0$

a stationary point

Curvature along any direction $v$ is $v\T A\T Av=\|Av\|^2\ge0$, so $L$ is convex and a stationary point is a global minimum. Unique unless some $v\ne0$ has $Av=0$.

$L(x)=\tfrac12[(x-3)^2+(x-1)^2]$ with scalar $x$. What is $L'(0)$?

1. A$-4$✓
2. B$+4$
3. C$0$

$L'(x)=2x-4$, so a small increase in $x$ lowers the loss.

Minimiser $x=2$, output $(2,2)$, residual $(1,-1)$. The gradient is one number in parameter space; the residual is a vector in output space.

Each row comes from writing $dL$ as a pairing with the increment.

$\nabla(a\T x)=a$ $n\times1$

$\nabla(\tfrac12\|x\|^2)=x$ $n\times1$

$\nabla(x\T Ax)=(A+A\T)x$ $n\times1$

$\nabla(\tfrac12\|Ax-b\|^2)=A\T(Ax-b)$ $n\times1$

$\nabla_W(\tfrac12\|W\|_F^2)=W$ $m\times n$

$a,A,b$ fixed. Row 3 holds for any $A$; $\nabla(\tfrac12x\T Hx)=Hx$ needs $H=H\T$.

$a,b,c$

the inputs, here $2,3,-1$

$u,v$

values the graph stores

$L$

one number at the end

$u=ab=6$

$v=u+c=5$

$L=v^2=25$

The graph records dependencies, and the backward pass reuses exactly these stored values.

Which triple is $(u,v,L)$, and at which node do the other two break?

1. A$(6,5,25)$✓
2. B$(6,7,49)$$v=u-c$
3. C$(5,6,36)$$u=a+b$

B breaks at the addition node; C breaks at the multiplication node and then again at the addition.

$\bar v$

how much $L$ moves per unit of $v$

$\bar L=1$

the seed, not the value $L=25$

$2v$

the local derivative of $L=v^2$

$\bar x$ and $g_x$ name the same object. Upstream means nearer the loss along the backward path.

$\bar z$

already known at this node

$\mathrel{+}=$

accumulate, never overwrite

$b,a$

the other input’s stored value

In the running graph $\bar v=10$ gives $\bar u=\bar c=10$.

$\bar v=10$ is already known. Which number returns to $a$?

1. A$10$
2. B$20$
3. C$30$✓

$\bar a=b\,\bar u=30$ and $\bar b=a\,\bar u=20$. A forgot the local factor; B used $a$ where the rule asks for $b$.

$u=2x,\ v=x^2$

two paths out of one $x$

$L=u+v$

where they meet

$\bar x$

one accumulator: start at $0$ and add

At $x=3$ a program reports $dL/dx=6$. What is wrong?

1. Athe result is correct
2. Bthe path through $u$ is missing✓
3. Cthe path through $v$ is missing

Both paths contribute: $2$ through $u$ and $6$ through $v$, so $dL/dx=8$. The reported $6$ is the $v$ path alone. Overwriting is one possible cause; a $u$ term never added is another.

$\bar z=1$

the seed

$x\,\bar z$

what each edge sends back

$\bar x$

the sum over edges

One shared node is not two copies. This is the product rule, and where the factor $2$ on a squared length came from.

$s:\,x_i\to x_j$

one edge per argument slot

$\partial x_j/\partial s$

vary slot $s$, hold the rest

$\bar x_j$

already finished

1. Store the forward values.
2. Seed $\bar L=1$.
3. Visit in reverse topological order.
4. Add one term per slot: $x\cdot x$ gives $x\bar z+x\bar z$.

Which of these is backpropagation itself?

1. Acompute $\nabla_\theta L$ from the stored values✓
2. Bchange $\theta$ with an update rule
3. Cboth together

Backpropagation produces the gradient and stops.

$x$

the layer’s input, $n\times1$

$W,\ b$

parameters, $m\times n$ and $m\times1$

$g_z$

sensitivity arriving from above, $m\times1$

Three arrows enter, so three gradients leave. With a bias the map is affine; “linear layer” is the usual name.

$dz$

the linear part of a change in $z$

$g_z\T dz$

all of $dL$ when $x,W,b$ reach $L$ only via $z$

shape

$(m\times1)(1\times n)=m\times n$

$W\,dx\ \Rightarrow\ \nabla_xL=W\T g_z$

$(dW)x\ \Rightarrow\ \nabla_WL=g_zx\T$

$db\ \Rightarrow\ \nabla_bL=g_z$

$x$ has $3$ entries and $z$ has $2$, so $W$ is $2\times3$. Which product has $W$’s shape?

1. A$xg_z\T$3 × 2
2. B$g_zx\T$✓2 × 3
3. C$g_z\T x$not defined

A shape check rejects a wrong formula; it never proves a right one.

$\odot$

entrywise product

$\partial a/\partial z$

$\mathrm{diag}(\mathbf 1_{z>0})$, $m\times m$, where $z_i\ne0$

$z_i=0$

no ordinary derivative; $0$ is our backward rule

ReLU clamps the value. Backward, each entry is multiplied by $0$ or $1$ — the diagonal Jacobian, applied entrywise.

A ReLU layer had $z=(-2,3)$ on the forward pass, and $g_a=(4,-5)$ arrives. Which is $g_z$?

1. A$(0,-5)$✓mask (0, 1)
2. B$(4,0)$mask (1, 0)
3. C$(0,5)$sign flipped

An active unit passes a negative sensitivity unchanged: ReLU clamps the value, not the gradient.

$e=\hat y-y^\star$

prediction minus target

cache

$x,z_1,a_1$ are kept; $W_1,W_2$ stay in the layers

Three numbers sit at the end of the forward pass. Which one is $g_{\hat y}=\partial L/\partial\hat y$?

1. A$6$the prediction
2. B$5$✓the error
3. C$12.5$the loss

$\dfrac{d}{d\hat y}\tfrac12(\hat y-1)^2=\hat y-1$, so this loss makes the first prediction sensitivity equal the error.

$\nabla_{W_2}L$

$1\times2$, the shape of $W_2$

$g_{a_1}$

$2\times1$, one entry per hidden unit

$a_1$

from the forward cache

The second weight’s gradient is zero because $a_{1,2}=0$, not because $g_{a_1,2}$ is.

The forward pass left $z_1=(3,-1)$, and $g_{a_1}=(10,-5)$ now arrives. Which is $g_{z_1}$?

1. A$(10,-5)$no mask at all
2. B$(10,0)$✓mask (1, 0)
3. C$(0,-5)$reversed forward mask $(0,1)$

The mask is read from the forward row, never from the sign of the sensitivity above.

$g_{z_1}x\T$

$(2\times1)(1\times2)=2\times2$

$\nabla_{b_1}L$

$=g_{z_1}$, one entry per unit

$\nabla_xL$

$2\times1$, how $L$ responds to a change in $x$

No parameter has moved: changing them is a separate job.

Why is the second row of $\nabla_{W_1}L$ zero here?

1. Athat row of $W_1$ is zeroit is $(1,-1)$
2. B$z_{1,2}<0$, so the path is blocked✓
3. Ca two-layer network always has oneit depends on $x$

Feed $x'=(2,1)$: now $z_1=(3,1)$, both units are active, and that row becomes $(-8,-4)$. A zero input or cancellation can zero a gradient too.

One sample gives $z=Wx+b$. Transposing turns $Wx$ into $x\T W\T$, so a row of $X$ goes through $W\T$. In code $b$ holds shape $(m,)$ and broadcasts.

$Y=XU$

$(B\times n)(n\times m)$

$G$

$\nabla_YL$, $B\times m$

$\operatorname{tr}(A\T B)$

$\langle A,B\rangle_F$

$\nabla_XL=GU\T$: the $dX$ coefficient.

$\nabla_UL=X\T G$; entrywise $\partial L/\partial U_{kj}=\sum_iX_{ik}G_{ij}$.

With $U=W\T$: $\nabla_XL=GW$ and $\nabla_WL=G\T X$.

$B=4$, $n=3$, $m=2$ and $Z=XW\T$, with $G=\nabla_ZL$. Which is $\nabla_WL$?

1. A$X\T G$3 × 2
2. B$G\T X$✓2 × 3
3. C$GX\T$not defined

$X\T G$ is the gradient of $U$ in $Y=XU$. Name the parameter first, then multiply.

A variable used in many places collects one contribution from each.

$G=\begin{bmatrix}1&2\\3&4\\-1&5\end{bmatrix}$ with $B=3$ and $m=2$. Which is $\nabla_bL$?

1. A$(3,7,4)$row sums
2. B$(1,2)$one sample only
3. C$(3,11)$✓column sums

Sum over samples, not over features: in code, G.sum(dim=0). The bias has one entry per feature.

“MSE” alone is ambiguous: at the same denominator, a framework MSE without the one-half has twice this gradient.

$x_i\T$

row $i$ of $X$

$g_i$

row $i$ of $G$, transposed: $m\times1$, $1/B$ inside

sum

over samples; do not divide a second time

$G$ is already $(Z-Y)/B$ for the same batch. The next line writes dW = G.T @ X / B. That result is:

1. Acorrectthat is $G\T X$
2. Btoo small by a factor $B$✓
3. Ctoo large by a factor $B$dividing shrinks it

$1/B$ appears twice: once inside $G$, once in the divide. Apply the loss scale exactly once.

$e_i$

only $\theta_i$ moves

$\varepsilon$

a finite step: two more evaluations

$\partial_iL$

the entry the backward pass returns

A finite difference measures. Reverse mode applies the chain rule to the operations as written.

$\phi(s)$

$L(\theta+sv)$; $\phi(0)=L(\theta)$

$v_j$

the rate of $\theta_j$ in $s$

cost

two evaluations, any $n$

A coordinate sweep costs two evaluations per coordinate. One direction that agrees is evidence, not proof.

Does a smaller $\varepsilon$ always give a better check?

1. Ayes, without limit
2. Bno — truncation and cancellation pull opposite ways✓
3. C$\varepsilon$ does not matter

Truncation here is $O(\varepsilon^2)$ for smooth enough $L$; the subtraction loses digits as $\varepsilon$ shrinks.

Sweep a few $\varepsilon$ in double precision; no one value fits all.

centred quotient

$\tfrac12$ for every $\varepsilon>0$

$z=0$

no ordinary derivative; our backward gives $0$

A mismatch at the corner is not evidence of a broken smooth derivative.

Which test shows a manual backward returns the right numbers?

1. Athe gradient is not None
2. Bvalues against AD and an independent finite difference✓
3. Cthe shape only

A and C are diagnostics; neither establishes a value. Use a smooth fixture.

The linear task we read checks shapes, the forward product and that gradients exist, never their values.

Inspected 2026-09-30: TorchCode torch_judge/tasks/linear.py blob 82fe0b4d: one task at one version, not every TorchCode test.

$X,W,b$

$B\times n$, $m\times n$, $m\times1$

$G$

$B\times m$: the reduction is already in it

Exercise shape after TorchCode, github.com/duoan/TorchCode; the fixtures and the checker are ours.

$X$ is $4\times3$, $W$ is $2\times3$, $G$ is $4\times2$. Which line gives $dX$?

1. AG @ W✓
2. BG @ W.Tinner $2\ne3$
3. CW @ Ginner $3\ne4$

$(4\times2)(2\times3)=4\times3$, the shape of $X$.

$dX_{ik}=\sum_jG_{ij}W_{jk}$ sums the $m$ outputs; the forward pass summed the $n$ inputs.

$\odot$

entrywise product

$\mathbf 1_{z>0}$

the ReLU slope; Boolean, read as $1$ and $0$

$z$

the cached forward value; at $z_i=0$ we use $0$

g_z = g_a * (z > 0) — the mask needs the cached forward $z$.

A test that only checks nonnegative forward outputs says nothing about the backward pass.

$z=(-2,3)$ and $g_a=(4,-5)$. Which line gives $g_z$?

1. Ag_a * (g_a > 0)$(4,0)$
2. Bg_a * (z > 0)✓
3. Crelu(g_a)$(4,0)$

B gives $(0,-5)$. A and C read the sign of the upstream gradient and delete a real negative entry; the mask belongs to the forward $z$.

Four columns, three checks of the hand column. ε = 1e−4 · autograd 0 (tol 1e−9) · fd ≤ 2.406e−12 (tol 2e−5)

Scaled error $|a-b|/\max(1,|a|,|b|)$: absolute near zero, not relative. No torch: autograd rows read skip, not pass.

Measured by python3 build/exercises/m5/check.py --verbose --require-autograd at 0b3feee, float64: 85 comparisons executed, 0 failed, 0 skipped. Transcript in build/exercises/m5/EVIDENCE.md.

$q$

the differentiated variable, $p=f(q)$ the tip

$e=p-p^\star$

$L=\tfrac12\|e\|^2$, so $dL=e\T de$

$J$

$m\times n$: the local map $dq\mapsto dp$

$\nabla_qL$

$n\times1$, shaped like $q$; a loss gradient, not a joint velocity

$J=\begin{bmatrix}1&2\\0&1\end{bmatrix}$, $e=(1,-1)$. What is $\nabla_qL=J\T e$?

1. A$(1,1)$✓columns of J
2. B$(-1,-1)$that is Je
3. C$(1,3)$signs of $e$ dropped

Each entry is one column of $J$ dotted with $e$: $1\cdot1+0\cdot(-1)$, then $2\cdot1+1\cdot(-1)$. Neither joint is larger.

$z=wx+b$

$L=\tfrac12(z-y^\star)^2$ and $e=z-y^\star$

$x,w,b$

one varies, the other two are held fixed

all three

are scalars: a shape check cannot tell them apart

$x=2,\ w=3,\ b=1,\ y^\star=4$, so $z=7$ and $e=3$. What is $(\partial L/\partial x,\ \partial L/\partial w)$?

1. A$(6,9)$swapped: ex, then ew
2. B$(9,6)$✓ew, then ex
3. C$(3,3)$e twice: w and x dropped

$\partial L/\partial x=ew=3\cdot3$ and $\partial L/\partial w=ex=3\cdot2$. One forward value, two partners.

$X$

$B\times n$ sample rows

$G=\nabla_ZL$

$B\times m$ upstream

$\nabla_WL$

$m\times n$; $X\T G$ is $n\times m$, equal here only because $n=m=2$

Both candidates are $2\times2$. Which test separates them?

1. Aassert the shape is $2\times2$passes anyway
2. Bcheck that every value is finitepasses anyway
3. Ccompare an entrywise derivative with the declared loss✓

With $L(W)=\langle G,XW\T\rangle_F$, $G$ is $\nabla_ZL$ for $Z=XW\T$, exactly. Perturb $W_{12}$: the slope is 5, not 0.

$B=2$

the declared reduction is a mean over samples

$x_i,y_i$

fixed data, not differentiated

$w,b$

shared by both branches: paths will add

$\hat y=(1,2)$

errors $e=(-1,0)$ at $w=1$, $b=0$

$\ell_i=\tfrac12e_i^2$

$0.5$ and $0$; the mean $L=0.25$

$e_i/B$

combined sensitivity

edges

one factor each, combined nowhere yet

With $\hat y=(1,2)$, $e=(-1,0)$ and $B=2$: what is $(\partial L/\partial w,\ \partial L/\partial b)$?

1. A$(-1,-1)$no $1/B$
2. B$(-0.5,-0.5)$✓$\sum e_ix_i/B$
3. C$(0.5,0.5)$sign of $e$

$(-1\cdot1+0\cdot2)/2$ and $(-1+0)/2$. Sample 2 contributes nothing: its error is zero.

$A$

$B\times2$, row $i$ is $(x_i,1)$

$\theta=(w,b)\T$

the parameters as one $2\times1$ vector

$A\theta-y$

$=(-1,0)$: the same errors as the graph

Duplicate both samples, so $B=4$, with the same mean loss and the same $w,b$. The gradient…

1. Adoublessum loss only
2. Bstays $(-0.5,-0.5)$✓both double
3. Chalves$B$ alone

$\sum e_ix_i$ goes from $-1$ to $-2$ and $B$ from 2 to 4. An unnormalised sum loss would double.

$b=0$

held fixed, so this is a 1-D slice

$\varepsilon=0.01$

the probe step, so the probes sit $0.02$ apart; a quotient is evidence

What actually travels backward?

1. Athe prediction error, copied through every layer
2. Bdownstream sensitivity × local derivatives, summed over paths✓
3. Cinverse matrices that undo the forward pass

Seed $\partial L/\partial L=1$. Then $\partial L/\partial\hat y=e$ for $L=\tfrac12e^2$, or $e_i/B$ for the batch mean.

1. 1$dy=J\,dx$ — a derivative is a local linear map
2. 2$\nabla_xL=J\T\nabla_yL$ — the transpose, on column gradients
3. 3$\nabla_WL=g_zx\T$ — one affine use: an outer product, shaped like $W$
4. 4a shared variable, or a batch, sums its paths

Next: new data, your choice of method.

Find $L$ and both gradients. Choose a method; check one slope.

1. A$(2,-\tfrac12)$✓
2. B$(4,-1)$mean omitted
3. C$(1,-\tfrac14)$mean counted twice