Notation and Conventions

Transpose, inverse and pseudoinverse are three different objects. $A^{\mathsf T}$ always exists and $A^{-1}$ usually does not. $A^{+}$ always exists: it returns the least-squares solution of smallest norm, and it agrees with $A^{-1}$ whenever the inverse exists.

For $f:\mathbb R^{n}\to\mathbb R^{m}$ the Jacobian is

One row per output, one column per input. Row $i$ says how output $i$ responds to every input; column $j$ says how input $j$ moves every output. The shape is the same statement: $J_f$ maps an input increment to an output increment, so it must be $m\times n$.

The robot Jacobian is this object with particular names. With joint angles $q$ and a tip position $p=f(q)$, $J$ is $\partial p/\partial q$ — the same rule with $q$ in place of $x$ and $p$ in place of $y$. A book may instead give a spatial or body Jacobian, which stacks angular velocity with linear velocity and is a larger object than the position-only one used here.

A scalar function of a vector has two standard arrangements of the same numbers, and this course uses both — named, and in the open.

The row is what the chain rule multiplies, because $\partial L/\partial x$ composes as $(\partial L/\partial y)(\partial y/\partial x)$. The column is what you add to a parameter, what a picture draws as an arrow, and what code stores beside the parameter it belongs to.

These two statements look alike and are not the same:

The first is an identity about a linear model. The second is a statement about the function, and it carries a remainder that vanishes faster than $\lVert\Delta x\rVert$ and is in general not zero — it is identically zero exactly when the function is affine. Changing the symbol from $dx$ to $\Delta x$ is not what makes it an approximation; the curvature is.

One sample. Inputs are columns and a weight matrix is output-by-input:

A batch. Samples are rows, which is what array libraries and data files do, so the weight matrix appears transposed:

With $G=\nabla_Z L$, the same shape as $Z$:

Indices that count different things get different letters: $i$ for a sample, $j$ and $k$ for coordinates, and a layer number as a subscript on the object ($W_1$, $z_1$, $a_1$), never as a free index. A code name such as dX is the gradient an implementation stores, not a differential $dX$; the lecture says which one it means.

Both appear because each makes its own chapter read naturally: the projection picture wants the vector that points from the prediction to the target, and a loss wants the quantity whose derivative flows backward. A gradient written with the wrong one is off by a sign, which is exactly the kind of error a sign convention is supposed to make visible rather than hide.

Lecture 5 writes the bridge where it switches. Lectures 1 to 4 use one of the two throughout and do not state the relation on the page, so when you carry a formula between them, convert it here first: the residual of the projection chapters and the error of the loss chapters are the same vector with opposite signs.

The target is named locally, and $y$ is used carefully: $y^\star$ or $b$ for a target, $y$ for an intermediate value $y=f(x)$, $\hat y$ for a prediction.

A loss is never written as "MSE" alone. Each one states its reduction and every factor:

The $\tfrac12$ in $\tfrac12\lVert e\rVert^2$ is there so the derivative comes out as $e$ rather than $2e$. A chapter that works with $\lVert e\rVert^2$ instead is not in conflict: it is the same problem with the same minimiser, and its gradient is twice as large. The earlier lectures state the objective but not that consequence, so check the factor before carrying a gradient out of one.

The same letter counts different things in different chapters. That is allowed; guessing is not. Read the role, not the letter.

So Lecture 4's data matrix with $N$ rows and $d$ columns and Lecture 5's $X\in\mathbb R^{B\times n}$ are the same arrangement: one row per sample. The letters differ because each chapter borrows the ones its own literature uses.

A second name is kept when it helps a chapter read naturally or when it is what you will meet elsewhere. The rule for new material is that each alias is bridged on the page where it first appears; in the earlier lectures some are bridged only in what the lecturer says, so this table is where to settle them.

Course-wide convention: Lectures 1–6, practice sheets and the midterm use $\operatorname{Col}(A)$, $\operatorname{Null}(A)$ and $\operatorname{Row}(A)$. The calligraphic names in older handouts and external books denote the same subspaces. A calligraphic $\mathcal C$ used as the name of a basis is a different object and is unchanged.

None of this is avoidable in a course that borrows from linear algebra, robotics, statistics and machine learning at once; the alphabet is not large enough. What matters is that a page defines the letter it is about to use, and that you look at the definition rather than at the letter.

Some code and some texts put inputs in rows and weights input-by-output — the transpose of the single-sample layout above. Both are correct. What is not correct is converting one of them and leaving the rest alone.

Writing $\cdot_c$ for the column layout of this page and $\cdot_r$ for the row layout:

When a book writes a gradient as a row, stacks samples in columns, or omits a $\tfrac12$, it is not wrong — it made a different choice. Three questions settle almost every mismatch:

• Is this a column or a row? Check the shape of the product it appears in.
• Which way does the error point? Find the definition, not the letter.
• What reduction does the loss use, and are all its factors written down?

The rule for new material is that a result adapted from a source using another convention says so — adapted to the course column-gradient convention — rather than being imported silently. The Textbooks and Reading Guide lists each source's category and the places its notation differs from this page.