Introduction and Linear Algebra I

Spaces, Maps, Rank, Solutions

Mathematics for Robotics · ROB-GY 6013 · NYU Tandon

Jing Zhang · Fall 2026 · September 2

Why Mathematics for Robotics?

The three panels are recurring questions, not one per box above.

Why Mathematics for Robotics?

The loop closes many times a second.

Why Mathematics for Robotics?

Calibration

locally, $Ax \approx b$

Robot motion

$\delta x \approx J\,\delta q$

Learning

$\arg\min_\theta \mathcal{L}(\theta)$

Different surface, same mathematical structure.

Why Mathematics for Robotics?

Robotics changes quickly.

The mathematical structure changes much more slowly.

Why Mathematics in the GenAI Era?

Google DeepMind · Advanced dexterity with Gemini Robotics 2, July 2026

Why Mathematics in the GenAI Era?

The pi-zero model: a pre-trained vision-language model of SigLIP 400M and Gemma 2.6B reading images through three ViTs and the words fold shirt, feeding a 300M action expert that also takes the joint state q sub t and noise, and emits the actions a sub t to a sub t plus H — A modern VLA is not replacing mathematics. It is built from mathematics.

Black, K. et al. π₀: A Vision-Language-Action Flow Model for General Robot Control, Figure 3. RSS, 2025. arXiv:2410.24164.

Why Mathematics in the GenAI Era?

$$\boxed{\;\frac{dA_\tau}{d\tau}=v_\theta(A_\tau,o,\tau)\;}\qquad A_0=\epsilon,\qquad \hat a=A_1$$

τ: flow time, 0 to 1; the field changes with it
v_θ: the trained vector field
A₀: the start: pure noise ε
â = A₁: the action chunk, read off at τ = 1

Black, K. et al. π₀. RSS, 2025. arXiv:2410.24164.

Why Mathematics in the GenAI Era?

AI can generate. You still have to judge.

Why Mathematics in the GenAI Era?

“…with AI assistance completely off.”

Sapora, S. ML Job Interviews: The Ultimate Guide, 2026 · silviasapora.github.io/blog/ml-interviews.html
Liu, A. Notes on the Industry Job Search, 2026 · alisawuffles.github.io/blog/job-search/

The Math We Keep Coming Back To

Different problems, the same mathematical ideas.

The Math We Keep Coming Back To

Four robot quantities

$q\in\R^{7}$	joint configuration
$p\in\R^{3}$	position
$I\in\R^{3\times H\times W}$	image
$a\in\R^{16\times 7}$	action chunk

Each is a vector, or an array that can be vectorized.

The Math We Keep Coming Back To

Map

$$f:\R^{n}\to\R^{m},\qquad u\;\mapsto\;y=f(u)$$

domain: $\R^{n}$, the allowed inputs
codomain: $\R^{m}$, the space the outputs live in
u ↦ y: the rule that carries one to the other

Forward kinematics: $\R^{7}\to\R^{3}$, joint angles to gripper position.

The Math We Keep Coming Back To

Local linearisation

$$\boxed{\;f(x+\delta x)\;\approx\;f(x)+J(x)\,\delta x\;}$$

x: the current point
δx: a small change
J(x): the Jacobian, $J(x)\in\R^{m\times n}$

Many nonlinear robotics problems become linear when viewed locally.

Left, the shrinking window. Right, what is inside it.

The Math We Keep Coming Back To

Where am I?

$$x_t=f(x_{t-1},u_t)+w_t,\qquad z_t=h(x_t)+v_t$$

xₜ: the latent state, usually not observed directly
uₜ, zₜ: the command, the measurement
f, h: motion and sensor models
wₜ, vₜ: noise

Thrun, S., Burgard, W., Fox, D. Probabilistic Robotics, Ch. 7. MIT Press, 2005.

The Math We Keep Coming Back To

$\|r\|_R^{2}=r\T R^{-1}r$, with $R$ the measurement covariance.

r: $z-h(x)$: measurement minus prediction; $R$ is positive definite

Barfoot, T. D. State Estimation for Robotics, Chapter 4. Cambridge University Press, 2017.

Course information

Everything on these four pages, plus the schedule and the reading: jingz6676.github.io/rob6013.html

Course information

One sheet of paper, both sides, prepared by you, and a scientific calculator. No phone, laptop, tablet, or watch.

Course information

Linear Algebra I

Spaces, Maps, Rank, Solutions

A matrix is a map

$$A\in\R^{m\times n},\qquad x\in\R^{n}\;\longmapsto\;Ax\in\R^{m}$$

A: the map itself, m rows by n columns
x: the input, a column of n numbers
Ax: the output, a column of m numbers
n: input dimension: the number of columns
m: output dimension: the number of rows; m and n need not be equal

Sanderson, G. Essence of Linear Algebra, ch. 3. 3Blue1Brown, 2016.

In numbers

A matrix is a map

$$\begin{bmatrix}2&1\\1&1\end{bmatrix}\begin{bmatrix}1\\2\end{bmatrix} =1\begin{bmatrix}2\\1\end{bmatrix}+2\begin{bmatrix}1\\1\end{bmatrix} =\begin{bmatrix}2+2\\1+2\end{bmatrix}=\begin{bmatrix}4\\3\end{bmatrix}$$

$\begin{bmatrix}2\\1\end{bmatrix}$: the first column, where e₁ lands
$\begin{bmatrix}1\\1\end{bmatrix}$: the second column, where e₂ lands

A matrix is a map

Linear map

$$\boxed{\,L(\alpha x+\beta y)=\alpha L(x)+\beta L(y)\,}$$

additivity: $L(x+y)=L(x)+L(y)$, the case $\alpha=\beta=1$
homogeneity: $L(\alpha x)=\alpha L(x)$, the case $\beta=0$
superposition: both at once, for all $x,y$ and all $\alpha,\beta\in\R$

Axler, S. Linear Algebra Done Right, 4th ed., Definition 3.1, p. 52. Springer, 2024. Open access.

Questions

Which of these three maps is linear?

A matrix is a map

$$e_j=(0,\ldots,1,\ldots,0)\T$$

the 1: sits in slot j; every other slot holds 0

Column rule

$$\boxed{\,Ae_j=a_j\,}$$

aⱼ: column j of A, a whole output vector

Axler, S. Linear Algebra Done Right, 4th ed.: standard basis 2.27(a), p. 39; columns 3.50, p. 76. Springer, 2024.

A matrix is a map

Linear combination of columns

$$Ax=\sum_{j=1}^{n}x_j\,(Ae_j)=x_1a_1+\cdots+x_na_n$$

xⱼ: coordinate j of x, used as an amount
Aeⱼ: column j, from the page before
aⱼ: column j of A, a whole direction

Axler, S. Linear Algebra Done Right, 4th ed., 3.50 linear combination of columns, p. 76. Springer, 2024.

A matrix is a map

$$\begin{bmatrix}1&0&1\\0&1&1\\2&1&3\end{bmatrix} \begin{bmatrix}3\\2\\0\end{bmatrix}=3a_1+2a_2=\begin{bmatrix}3\\2\\8\end{bmatrix}$$

a₁, a₂: (1,0,2) and (0,1,1), the first two columns
3, 2, 0: the amounts asked of the three columns

A matrix is a map

Do B first, then A

$$(AB)x=A(Bx)$$

shapes: $B\in\R^{n\times p}$, $A\in\R^{m\times n}$
Bx: where B sends x first
A(Bx): where A then sends that

Axler, S. Linear Algebra Done Right, 4th ed., 3.43 matrix of product of linear maps, p. 74. Springer, 2024.

A matrix is a map

Transpose

$$(A\T)_{k,j}=A_{j,k}$$

row j: of A becomes column j of Aᵀ
m × n: becomes n × m

It reverses a product

$$\boxed{\,(AB)\T=B\T A\T\,}$$

(AB)ᵀ: p × m, the shape the answer must have
BᵀAᵀ: p × n times n × m, so n meets n

Axler, S. Linear Algebra Done Right, 4th ed., 3.54 transpose, p. 77; (AC)ᵗ = CᵗAᵗ, p. 78. Springer, 2024.

Integrated worked example

A matrix is a map

$B=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix}$, $A=\begin{bmatrix}1&2&0\\0&1&1\end{bmatrix}$. Apply $B$, then $A$, to $x=(1,-1)\T$; check the transpose.

$$AB=\begin{bmatrix}1&2\\1&2\end{bmatrix}$$

reason: $\R^2\to\R^3\to\R^2$: the inner dimensions match.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

A matrix is a map

Continue; check the original map.

$$AB=\begin{bmatrix}1&2\\1&2\end{bmatrix}$$

reason: $\R^2\to\R^3\to\R^2$: the inner dimensions match.

$$Bx=(1,-1,0)\T,\quad A(Bx)=(-1,-1)\T=(AB)x$$

reason: Following the intermediate vector checks the product.

$$(AB)\T=B\T A\T=\begin{bmatrix}1&1\\2&2\end{bmatrix}$$

reason: BA is 3 by 3; it describes a different trip.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Independent transfer

A matrix is a map

Keep $B$. Now $A=\begin{bmatrix}1&0&2\\0&1&-1\end{bmatrix}$ and $x=(2,1)\T$. Find $AB$, its output and $(AB)\T$.

$$AB=\begin{bmatrix}3&2\\-1&0\end{bmatrix}$$

reason: Column $j$ is $A(Be_j)$.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

A matrix is a map

Continue; check the original map.

$$AB=\begin{bmatrix}3&2\\-1&0\end{bmatrix}$$

reason: Column $j$ is $A(Be_j)$.

$$Bx=(2,1,3)\T,\quad (AB)x=(8,-2)\T$$

reason: Check each output using the intermediate vector.

$$(AB)\T=B\T A\T=\begin{bmatrix}3&-1\\2&0\end{bmatrix}$$

reason: Transposition reverses the factor order.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Span and column space

Linear combination

$$w=c_1v_1+c_2v_2+\cdots+c_kv_k$$

$v_i$: the directions you are handed
$c_i$: any reals, negative and zero included
w: where those weights land you

Axler, S. Linear Algebra Done Right, 4th ed., 2.2 linear combination, p. 28. Springer, 2024.

Span and column space

A =

1 0 1
0 1 1
2 1 3

Span

$$\operatorname{span}(v_1,\dots,v_k)=\{c_1v_1+\cdots+c_kv_k\;:\;c_i\in\R\}$$

{ }: every w the weights can reach, all at once

Let both weights roam: the plane fills in.

Worked example

$v_1=(1,0,2)$, $v_2=(0,1,1)$: $3v_1+2v_2=(3,2,8)$ is in the span.

Axler, S. Linear Algebra Done Right, 4th ed., 2.4 span, p. 29. Springer, 2024.

Span and column space

$$\begin{aligned} \operatorname{span}(v_1)&=\{c\,v_1\}\\ \operatorname{span}(v_1,v_2)&=\{c_1v_1+c_2v_2\}\\ \operatorname{span}(v_1,v_2,v_3)&=\operatorname{span}(v_1,v_2) \end{aligned}$$

the conditions: dependent pair: a line, or $\{0\}$ if both zero
v₃: already a mixture of the first two

Sanderson, G. Essence of Linear Algebra, Chapter 2. 3Blue1Brown, 2016.

Span and column space

A =

1 0 1
0 1 1
2 1 3

$$b\in\operatorname{span}(v_1,\dots,v_k)\iff Vc=b\ \text{has a solution}$$

V: the arrows as columns
c: the unknown weights
b: the target vector

Is (3,2,9) in the span?

$c_1(1,0,2)+c_2(0,1,1)=(3,2,9)$

row 1 gives $c_1=3$, row 2 gives $c_2=2$

row 3 is then forced: $2(3)+2=8$, not 9

One lands, one misses.

Span and column space

A vector space is a set equipped with vector addition and scalar multiplication satisfying the vector-space axioms.

What we use

$$\boxed{\,u,v\in V,\ \alpha,\beta\in\R\ \Longrightarrow\ \alpha u+\beta v\in V\,}$$

closure: the one property this course leans on all term

Axler, S. Linear Algebra Done Right, 4th ed., 1.20 vector space, p. 12. Springer, 2024.

Span and column space

Addition

$$\boxed{\begin{aligned}u+v&=v+u & (u+v)+w&=u+(v+w)\\ v+0&=v & \exists\,w:\ v+w&=0\end{aligned}}$$

0 and $w$: an origin, and an undo for every vector

Scaling

$$\boxed{\begin{aligned}1v&=v & \alpha(\beta v)&=(\alpha\beta)v\\ \alpha(u+v)&=\alpha u+\alpha v & (\alpha+\beta)v&=\alpha v+\beta v\end{aligned}}$$

the payoff: obey all eight and every theorem here applies

Axler, S. Linear Algebra Done Right, 4th ed., 1.20 vector space, p. 12. Springer, 2024.

Span and column space

A subset $S\subseteq V$ is a subspace if it is itself a vector space under the same operations.

Equivalent test

$$\boxed{\,S\neq\varnothing\ \ \text{and}\ \ u,v\in S,\ \alpha,\beta\in\R\Rightarrow\alpha u+\beta v\in S\,}$$

nonempty: plus closure already gives $0\in S$: choose $\alpha=\beta=0$
next page: the same test, split into three checks

Axler, S. Linear Algebra Done Right, 4th ed., 1.33 subspace and 1.34 conditions for a subspace, p. 18. Springer, 2024.

Span and column space

Conditions for a subspace

$$0\in S;\qquad u+v\in S;\qquad \alpha u\in S$$

0: the zero vector is a member
u+v: any two members add to a member
alpha u: any multiple stays in, negatives included

Axler, S. Linear Algebra Done Right, 4th ed., 1.34 conditions for a subspace, p. 18. Springer, 2024.

Span and column space

Fastest disqualifier

$$0\notin S\ \Longrightarrow\ S\ \text{is not a subspace}$$

0 ∈ S: necessary, never sufficient: closure still has to hold

Axler, S. Linear Algebra Done Right, 4th ed., 1.35 example: subspaces, p. 19. Springer, 2024.

Span and column space

Claim

$$\boxed{\,v_1,\dots,v_k\in V\ \Longrightarrow\ \operatorname{span}(v_1,\dots,v_k)\ \text{is a subspace of}\ V\,}$$

S: every mixture of the list; the smallest subspace containing it

$$0=\sum_{i=1}^{k}0\,v_i\in S$$

every c_i = 0: the origin is a member

$$\alpha u+\beta w=\sum_{i=1}^{k}(\alpha c_i+\beta d_i)\,v_i\in S$$

u, w: any two members: $u=\sum_i c_iv_i$, $w=\sum_i d_iv_i$

Axler, S. Linear Algebra Done Right, 4th ed., 2.6 span is the smallest containing subspace, p. 29. Springer, 2024.

Span and column space

Column space

$$\boxed{\,\Col(A)=\{Ax:x\in\R^{n}\}\subseteq\R^{m}\,}$$

A: any $m\times n$ matrix; x runs over the whole of $\R^{n}$
$\subseteq\R^{m}$: a subspace of the codomain; if $m=n$ the roles still differ
also called: the image of A, or the range of A

Axler, S. Linear Algebra Done Right, 4th ed., 3.16 range and 3.18 the range is a subspace, p. 61. Springer, 2024.

Span and column space

Definition

$$\Col(A)=\{Ax:x\in\R^{n}\}$$

x: runs over every input

Column view

$$Ax=\sum_{j=1}^{n}x_j\,a_j$$

a_j: column j of A

⇓

$$\boxed{\,\Col(A)=\operatorname{span}(a_1,\dots,a_n)\,}$$

one set: reachable outputs and mixtures of the columns are the same thing

Axler, S. Linear Algebra Done Right, 4th ed., 3.16 range, p. 61; 3.50 linear combination of columns, p. 76. Springer, 2024.

Span and column space

Question

$$Ax=b\ ?$$

b: a target on the output side

$$Ax=b\ \text{has a solution}\iff\exists\,x:\ Ax=b\iff b\in\{Ax:x\in\R^{n}\}$$

∃ x: the whole question is whether one input reaches b

$$\boxed{\,Ax=b\ \text{is solvable}\iff b\in\Col(A)\,}$$

nothing new: the right side is the definition of Col(A), read backwards

Axler, S. Linear Algebra Done Right, 4th ed., 3.16 range, p. 61. Springer, 2024.

Span and column space

Solve

$$c_1\begin{bmatrix}1\\0\\2\end{bmatrix}+c_2\begin{bmatrix}0\\1\\1\end{bmatrix}=\begin{bmatrix}3\\2\\y_3\end{bmatrix}$$

row 1: $c_1+0=3$, so $c_1=3$
row 2: $0+c_2=2$, so $c_2=2$
row 3: $2c_1+c_2=2(3)+2=8$ must equal $y_3$

Target 1

$$\boxed{\,y_3=8:\ 8=8\ \checkmark\,}$$

so: $b_1=3a_1+2a_2\in\Col(A)$

Target 2

$$\boxed{\,y_3=9:\ 9\neq8\ \times\,}$$

so: no $c$ works: $b_2\notin\Col(A)$

Axler, S. Linear Algebra Done Right, 4th ed., 3.16 range, p. 61. Springer, 2024.

Span and column space

A =

1 0 1
0 1 1
2 1 3

$$\boxed{\,Ax=\sum_{j}x_j\,a_j\,}$$

x_j: the amount taken of column j

⇓

$$\boxed{\,\Col(A)=\operatorname{span}(a_1,\dots,a_n)\,}$$

span: every mixture, so every reachable output

⇓

$$\boxed{\,Ax=b\ \text{solvable}\iff b\in\Col(A)\,}$$

our A: $a_3=a_1+a_2$, so the third column adds nothing to the span

When do different columns carry independent information?

Axler, S. Linear Algebra Done Right, 4th ed., 2.15 linearly independent, p. 32. Springer, 2024.

Integrated worked example

Span and column space

$S=\{y\in\R^3:y_3=y_1+2y_2\}$. Prove $S$ is a subspace, give spanning vectors, and test $b=(2,-1,0)\T$.

$$y=(s,t,s+2t)\T=s(1,0,1)\T+t(0,1,2)\T$$

reason: Every member is a combination, and every combination satisfies the equation.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Span and column space

Continue; check the original map.

$$y=(s,t,s+2t)\T=s(1,0,1)\T+t(0,1,2)\T$$

reason: Every member is a combination, and every combination satisfies the equation.

$$S=\operatorname{span}\{(1,0,1)\T,(0,1,2)\T\}$$

reason: A span contains zero and is closed under linear combinations.

$$b=2(1,0,1)\T-(0,1,2)\T\in S$$

reason: The third coordinate checks: 2 minus 2 is zero.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Independent transfer

Span and column space

$T=\{y:y_3=2y_1-y_2\}$. Find spanning vectors. Is $(1,3,0)\T$ in $T$? Is $y_3=2y_1-y_2+1$ a subspace?

$$T=\operatorname{span}\{(1,0,2)\T,(0,1,-1)\T\}$$

reason: Use the first two coordinates as free coefficients.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Span and column space

Continue; check the original map.

$$T=\operatorname{span}\{(1,0,2)\T,(0,1,-1)\T\}$$

reason: Use the first two coordinates as free coefficients.

$$2(1)-3=-1\ne0$$

reason: The target fails the defining equation.

$$0\ne 2(0)-0+1$$

reason: The shifted set misses zero and is not a subspace.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Null Space and Linear Independence

So far

$$\Col(A)=\text{all reachable outputs}$$

settled: which targets can be hit, and by which mixtures

Now ask

$$\boxed{\,Ax_1=Ax_2,\qquad x_1\neq x_2\,}$$

one output: but more than one input that reaches it

How can two different inputs produce the same output?

Axler, S. Linear Algebra Done Right, 4th ed., 3.14 injective, p. 60; 3.11 null space, p. 59. Springer, 2024.

Null Space and Linear Independence

$$\begin{aligned} Ax_1&=Ax_2\\ Ax_1-Ax_2&=0\\ A(x_1-x_2)&=0 \end{aligned}$$

last step: linearity, and nothing else, is what pulls A outside

The difference

$$\boxed{\,z=x_1-x_2\ \Longrightarrow\ Az=0\,}$$

z: nonzero, since $x_1\neq x_2$, and A sends it to zero

Every collision $Ax_1=Ax_2$ is a solution $z=x_1-x_2$ of $Ax=0$.

Axler, S. Linear Algebra Done Right, 4th ed., 3.1 linear map, p. 52; 3.11 null space, p. 59. Springer, 2024.

Null Space and Linear Independence

Homogeneous system

$$\boxed{\,Ax=0\,}$$

homogeneous: the right-hand side is the zero vector
x = 0: always a solution, and the uninteresting one

Are there nonzero solutions?

One gives a line

$$\boxed{\,Az=0,\ z\neq0\ \Longrightarrow\ A(\alpha z)=0\ \ \forall\,\alpha\in\R\,}$$

αz: one invisible direction, so a whole line of invisible inputs

Axler, S. Linear Algebra Done Right, 4th ed., 3.26 homogeneous system of linear equations, p. 65. Springer, 2024.

Null Space and Linear Independence

Null space

$$\boxed{\,\Null(A)=\{x\in\R^{n}:Ax=0\}\,}$$

also called: the kernel of A, written $\ker(A)$
$Ax=0$: the inputs A cannot see

Shape

$$\boxed{\,\Null(A)\subseteq\R^{n}\,}\qquad\text{against}\qquad\Col(A)\subseteq\R^{m}$$

input side: Null(A) lives where x lives, Col(A) where Ax lives

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space and 3.13 the null space is a subspace, p. 59. Springer, 2024.

Null Space and Linear Independence

$$u,v\in\Null(A)\ \Longrightarrow\ Au=0,\ Av=0$$

and: $A0=0$, so the origin is in $\Null(A)$ to begin with

$$A(\alpha u+\beta v)=\alpha Au+\beta Av=0$$

so: $\alpha u+\beta v\in\Null(A)$ for every $\alpha,\beta$

Subspace

$$\boxed{\,\Null(A)\ \text{is a subspace of}\ \R^{n}\,}$$

flat: in $\R^{3}$: $\{0\}$, a line, a plane, or all of $\R^{3}$

Axler, S. Linear Algebra Done Right, 4th ed., 3.13 the null space is a subspace, p. 59. Springer, 2024.

Null Space and Linear Independence

A =

1 0 1
0 1 1
2 1 3

Our matrix

$$a_3=a_1+a_2\ \Longrightarrow\ a_1+a_2-a_3=0$$

read it: a mixture of the columns, weights $1,1,-1$, giving zero

$$A\begin{bmatrix}1\\1\\-1\end{bmatrix}=0\ \Longrightarrow\ \begin{bmatrix}1\\1\\-1\end{bmatrix}\in\Null(A)$$

and: next: solve for the whole set

The whole set

$$\boxed{\,\Null(A)=\operatorname{span}\left\{(1,1,-1)\T\right\}\,}$$

a line: through the origin

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Null Space and Linear Independence

Ax = 0

$$\begin{bmatrix}1&0&1\\0&1&1\\2&1&3\end{bmatrix}\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}=0$$

row 1: $1x_1+0x_2+1x_3=0$, so $x_1+x_3=0$
row 2: $0x_1+1x_2+1x_3=0$
row 3: $2x_1+x_2+3x_3=0$ is twice row 1 plus row 2

Solve

$$x_3=t:\ x_1=-t,\ x_2=-t,\ x=t(-1,-1,1)\T$$

why: one free unknown

Null space

$$\boxed{\,\Null(A)=\operatorname{span}\{(-1,-1,1)\T\}\,}$$

check: $-1+1,\ -1+1,\ -2-1+3$: all zero

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Null Space and Linear Independence

$$z\in\Null(A)\ \Longrightarrow\ Az=0\ \Longrightarrow\ A(x+z)=Ax+Az=Ax$$

x: any input at all, not a special one

What it means

$$\boxed{\,A(x+z)=Ax,\qquad z\in\Null(A)\,}$$

x + z: for our matrix, a line of inputs sharing one output

Null-space directions are invisible to the map.

Axler, S. Linear Algebra Done Right, 4th ed., 3.1 linear map, p. 52; 3.11 null space, p. 59. Springer, 2024.

Null Space and Linear Independence

On a robot

$$\dot x=J(q)\,\dot q$$

$\dot q$: joint velocities, what you command
$\dot x$: end-effector velocity, what the world sees

Redundancy

$$\boxed{\,z\in\Null(J(q))\ \Longrightarrow\ J(q)(\dot q+z)=J(q)\dot q\,}$$

z: a joint-velocity direction that produces no end-effector velocity

Lynch, K. M. and Park, F. C. Modern Robotics, §6.3 inverse velocity kinematics, p. 233. Cambridge University Press, 2017 preprint.

Null Space and Linear Independence

Linear independence

$$\boxed{\,c_1v_1+\cdots+c_kv_k=0\ \Longrightarrow\ c_1=\cdots=c_k=0\,}$$

read it: the all-zero weights are the only ones that cancel

Same thing

$$\boxed{\,\sum_{i=1}^{k}c_iv_i=0\ \Longrightarrow\ c=0\,}$$

c: the whole list of weights, read as one vector

Left, a nonzero walk comes home. Right, none does.

Axler, S. Linear Algebra Done Right, 4th ed., 2.15 linearly independent, p. 32. Springer, 2024.

Null Space and Linear Independence

Linear dependence

$$\boxed{\,\exists\,c\neq0:\ c_1v_1+\cdots+c_kv_k=0\,}$$

$c\neq0$: at least one weight is nonzero; say it is $c_j$

Solve for that one

$$\boxed{\,v_j=-\frac{1}{c_j}\sum_{i\neq j}c_iv_i\,}$$

so: $v_j$ is built from the others, and deleting it leaves the span unchanged

Strike the diagonal and the plane is untouched.

Axler, S. Linear Algebra Done Right, 4th ed., 2.17 linearly dependent and 2.19 linear dependence lemma, p. 33. Springer, 2024.

Null Space and Linear Independence

Stack them

$$V=[\,v_1\ \cdots\ v_k\,],\qquad Vc=\sum_i c_iv_i$$

Vc: the matrix times its weights

The anchor

$$\boxed{\begin{aligned}v_1,\dots,v_k\ \text{independent}&\iff\Null(V)=\{0\}\\v_1,\dots,v_k\ \text{dependent}&\iff\Null(V)\neq\{0\}\end{aligned}}$$

one question: redundancy in a list is a null space

Axler, S. Linear Algebra Done Right, 4th ed., 2.15 linearly independent, p. 32; 3.15 injectivity, p. 60. Springer, 2024.

Null Space and Linear Independence

A =

1 0 1
0 1 1
2 1 3

Our matrix

$$a_3=a_1+a_2\ \Longrightarrow\ A(1,1,-1)\T=0$$

one equation: the weights $1,1,-1$ cancel the three columns

Null space

$$(1,1,-1)\T\in\Null(A)$$

reading: an input A cannot see

Linear dependence

$$a_1,a_2,a_3\ \text{dependent}$$

reading: a column you may delete

Same equation, two interpretations.

Axler, S. Linear Algebra Done Right, 4th ed., 2.17 linearly dependent, p. 33; 3.11 null space, p. 59. Springer, 2024.

Null Space and Linear Independence

$$\boxed{\begin{aligned} \Null(A)&=\{x:Ax=0\}\\ z\in\Null(A)&\ \Longrightarrow\ A(x+z)=Ax\\ \Null(A)\neq\{0\}&\ \Longrightarrow\ \text{information is lost} \end{aligned}}$$

the chain: definition, meaning, cost

For the columns

$$\boxed{\,\Null(A)=\{0\}\iff\text{columns independent}\,}$$

settled: whether a direction is redundant

How many independent directions?

Axler, S. Linear Algebra Done Right, 4th ed., 2.26 basis, p. 39. Springer, 2024.

Basis, Dimension, and Rank

A =

1 0 1
0 1 1
2 1 3

We know

$$\Col(A)\ \text{what A can produce},\qquad \Null(A)\ \text{what A cannot see}$$

and: how to tell whether a direction is redundant

Now ask

$$\boxed{\,\text{How many independent directions are needed?}\,}$$

our A: $a_3=a_1+a_2$: three columns, but how many directions?

Axler, S. Linear Algebra Done Right, 4th ed., 2.26 basis, p. 39. Springer, 2024.

Basis, Dimension, and Rank

Basis

$$\boxed{\,\text{Basis}=\text{Spanning}+\text{Linear Independence}\,}$$

spans: $\operatorname{span}(b_1,\dots,b_k)=V$
independent: $c_1b_1+\cdots+c_kb_k=0\Rightarrow c_1=\cdots=c_k=0$

So

$$\text{enough to build everything, with nothing spare}$$

both halves: either one alone is not a basis

Axler, S. Linear Algebra Done Right, 4th ed., 2.26 basis, p. 39. Springer, 2024.

Basis, Dimension, and Rank

Too few cannot reach; too many give several addresses.

Basis

$$\boxed{\,\text{enough, but not too many}\,}$$

too few: here independent, but does not reach everything
too many: reaches everything, but carries a repeat

Worked example

In $\R^{2}$: $\{(1,0)\}$ is too few, $\{(1,0),(1,1)\}$ is a basis, adding $(2,1)$ is too many.

Axler, S. Linear Algebra Done Right, 4th ed., 2.26 basis, p. 39. Springer, 2024.

Basis, Dimension, and Rank

Unique weights

$$x=c_1b_1+\cdots+c_kb_k\ \text{in exactly one way}$$

why: independence: two different weightings would differ by a nonzero mix that vanishes

Coordinates

$$\boxed{\,[x]_B=(c_1,\dots,c_k)\T,\qquad x=B\,[x]_B\,}$$

one address: the vector does not change; only its coordinates do

Axler, S. Linear Algebra Done Right, 4th ed., 2.28 criterion for basis, p. 39; 3.73 matrix of a vector, p. 88. Springer, 2024.

Basis, Dimension, and Rank

Dimension

$$\boxed{\,\dim V=\text{the number of vectors in any basis of }V\,}$$

any: every basis of the same space has the same length

Reading

$$\text{dimension counts independent degrees of freedom}$$

finite: the vector spaces considered here have finite bases

$$\boxed{\,\dim\{0\}=0\,}$$

zero space: the empty list is a basis of it

Axler, S. Linear Algebra Done Right, 4th ed., 2.35 dimension, p. 44. Springer, 2024.

Basis, Dimension, and Rank

A =

1 0 1
0 1 1
2 1 3

One plane, two bases, both of size two.

Same size always

$$\boxed{\,\text{different bases, different arrows, the same number of them}\,}$$

this plane: every basis of it has exactly two vectors
so: dimension belongs to the space, not to a choice

Axler, S. Linear Algebra Done Right, 4th ed., 2.35 dimension, p. 44. Springer, 2024.

Basis, Dimension, and Rank

Reduce

$$A=\begin{bmatrix}1&0&1\\0&1&1\\2&1&3\end{bmatrix}\xrightarrow{r_3\leftarrow r_3-2r_1}\begin{bmatrix}1&0&1\\0&1&1\\0&1&1\end{bmatrix}\xrightarrow{r_3\leftarrow r_3-r_2}\begin{bmatrix}1&0&1\\0&1&1\\0&0&0\end{bmatrix}=R$$

step 1: $(2,1,3)-2(1,0,1)=(0,1,1)$
step 2: $(0,1,1)-(0,1,1)=(0,0,0)$
pivots: columns 1 and 2

Why R

$$R=EA,\ E\ \text{invertible}:\ Ax=0\iff Rx=0$$

so: $\Null(A)=\Null(R)$

Strang, G. 18.06, Lecture 7: pivot variables and special solutions. MIT OCW, 2010.

Basis, Dimension, and Rank

Pivot

$$\boxed{\,\text{the leading nonzero entry of a nonzero row of }R\,}$$

in R: pivot positions are read off the echelon form, not off $A$
pivot variable: a variable whose column carries a pivot
free variable: a variable whose column does not

Here

$$x_1,x_2\ \text{pivot};\qquad x_3\ \text{free}$$

count: 3 variables, 2 pivots, 1 free

Strang, G. 18.06, Lecture 7: pivot variables and special solutions. MIT OCW, 2010.

Basis, Dimension, and Rank

Pivot columns

$$\boxed{\,\text{the pivot columns of }A\ \text{are a basis for }\Col(A)\,}$$

here: $\Null(A)=\Null(R)$, while $\Col(A)\neq\Col(R)$
R's columns: $(1,0,0)\T,(0,1,0)\T$ span $y_3=0$; $a_1=(1,0,2)\T$ has $y_3=2$
A's columns: $a_1,a_2$ independent, and $a_3=a_1+a_2$, so they span $\Col(A)$

Axler, S. Linear Algebra Done Right, 4th ed., 3.52 column rank, p. 77. Springer, 2024.

Basis, Dimension, and Rank

A =

1 0 1
0 1 1
2 1 3

R =

1 0 1
0 1 1
0 0 0

Rows of R

$$x_1+x_3=0,\qquad x_2+x_3=0,\qquad 0=0$$

legal: $\Null(A)=\Null(R)$
free: column 3 has no pivot: $x_3=t$

Solve

$$x_1=-t,\quad x_2=-t\ \Longrightarrow\ x=(-t,-t,t)\T=t(-1,-1,1)\T$$

$t=1$: on $A$: $-1+1,\ -1+1,\ -2-1+3$, all zero

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Basis, Dimension, and Rank

All solutions

$$\Null(A)=\{\,t(-1,-1,1)\T:\ t\in\R\,\}$$

why: every solution is $t$ times one vector, nothing else
so: all multiples of one vector: a span

Basis and dimension

$$\boxed{\,\Null(A)=\operatorname{span}\{(-1,-1,1)\T\},\qquad\dim\Null(A)=1\,}$$

basis: one nonzero vector is independent, hence a basis
count: one free variable, one basis vector: $\dim\Null(A)=$ free variables

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59; 2.26 basis, p. 39. Springer, 2024.

Basis, Dimension, and Rank

A =

1 0 1
0 1 1
2 1 3

Rank

$$\boxed{\,\operatorname{rank}(A)=\dim\Col(A)\,}$$

our A: $\dim\Col(A)=2$, so $\operatorname{rank}(A)=2$

To compute

$$\boxed{\,\operatorname{rank}(A)=\text{the number of pivots}\,}$$

reading: how many independent output directions A can produce

Axler, S. Linear Algebra Done Right, 4th ed., 3.58 rank, p. 79; 3.52 column rank, p. 77. Springer, 2024.

Basis, Dimension, and Rank

Into three-space

$$r=3\ \text{volume};\quad r=2\ \text{plane};\quad r=1\ \text{line};\quad r=0\ \text{the point }\{0\}$$

rank: the dimension of the reachable output space

The input cube never changes. Each direction the matrix loses costs the image a whole dimension.

Sanderson, G. Essence of Linear Algebra, ch. 7. 3Blue1Brown, 2016.

Basis, Dimension, and Rank

A =

1 0 1
0 1 1
2 1 3

Nullity

$$\boxed{\,\operatorname{nullity}(A)=\dim\Null(A)\,}$$

our A: $\Null(A)=\operatorname{span}\{(-1,-1,1)\T\}$, so $\operatorname{nullity}(A)=1$

Reading

$$\text{independent input directions lost by }A$$

the pair: rank counts what survives, nullity counts what disappears

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Basis, Dimension, and Rank

A =

1 0 1
0 1 1
2 1 3

Rank–nullity

$$\boxed{\,\operatorname{rank}(A)+\operatorname{nullity}(A)=n\,}$$

n: the number of columns, the input dimension
reach: $\dim\Col(A)=r$: directions that survive
loss: $\dim\Null(A)=n-r$: directions that disappear
our A: $2+1=3$
a choice: $n-r$ basis vectors in $\Null(A)$, $r$ whose images span $\Col(A)$

Axler, S. Linear Algebra Done Right, 4th ed., 3.21 fundamental theorem of linear maps, p. 62. Springer, 2024.

Basis, Dimension, and Rank

The map

$$A:\R^{n}\to\R^{m},\qquad n=r+(n-r)$$

r: directions that survive
n - r: directions that vanish

Two survivors, one casualty, three directions accounted for.

So

$$\boxed{\,\text{rank--nullity counts the input side}\,}$$

not m: the null space is not in the output space

Axler, S. Linear Algebra Done Right, 4th ed., 3.21 fundamental theorem of linear maps, p. 62. Springer, 2024.

Basis, Dimension, and Rank

Exam question

$$B\in\R^{7\times10},\qquad \operatorname{rank}(B)=6$$

ask: $\dim\Col(B)$, $\dim\Null(B)$, and the number of free variables

$$\dim\Col(B)=6,\qquad \dim\Null(B)=10-6=4,\qquad \text{4 free variables}$$

the trap: use the 10, not the 7

$\dim\Null(B)=n-r=10-6=4$.

Axler, S. Linear Algebra Done Right, 4th ed., 3.21 fundamental theorem of linear maps, p. 62. Springer, 2024.

Basis, Dimension, and Rank

Full column rank

$$\boxed{\,r=n\,}$$

then: $\Null(A)=\{0\}$: no input direction is lost; needs $n\le m$

Full row rank

$$\boxed{\,r=m\,}$$

then: $\Col(A)=\R^{m}$: every output is reachable; needs $m\le n$

One is about losing nothing. The other is about reaching everything.

Axler, S. Linear Algebra Done Right, 4th ed., 3.21 fundamental theorem of linear maps, p. 62. Springer, 2024.

Basis, Dimension, and Rank

Where we are

$$\dim\Col(A)=r,\qquad \dim\Null(A)=n-r$$

so: the three questions become countable

$$\begin{aligned} \text{is }b\text{ reachable?}&\quad b\in\Col(A)\\ \text{is }x\text{ unique?}&\quad \Null(A)=\{0\}\\ \text{how many free directions?}&\quad n-r \end{aligned}$$

row two: uniqueness is a question only once a solution exists

Axler, S. Linear Algebra Done Right, 4th ed., 3.21 fundamental theorem of linear maps, p. 62. Springer, 2024.

Integrated worked example

Basis, Dimension, and Rank

$F=\begin{bmatrix}1&2&0&1\\0&0&1&-1\end{bmatrix}$. Find bases of $\Col(F)$ and $\Null(F)$, rank and nullity.

$$x_2=s,\ x_4=t:\quad x_1=-2s-t,\ x_3=t$$

reason: Pivot columns 1 and 3 determine two variables.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Basis, Dimension, and Rank

Continue; check the original map.

$$x_2=s,\ x_4=t:\quad x_1=-2s-t,\ x_3=t$$

reason: Pivot columns 1 and 3 determine two variables.

$$\Null(F)=\operatorname{span}\{(-2,1,0,0)\T,(-1,0,1,1)\T\}$$

reason: The two free-coordinate patterns prove independence.

$$\Col(F)=\R^2,\quad r=2,\quad n-r=2$$

reason: Original columns 1 and 3 form the output basis; 2+2=4.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Independent transfer

Basis, Dimension, and Rank

$G=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix}$. Find rank, nullity and $\Col(G)$. Can it reach $(1,2,4)\T$?

$$\operatorname{rank}G=2,\quad\operatorname{nullity}G=2-2=0$$

reason: Two input coordinates, two independent columns.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Basis, Dimension, and Rank

Continue; check the original map.

$$\operatorname{rank}G=2,\quad\operatorname{nullity}G=2-2=0$$

reason: Two input coordinates, two independent columns.

$$Gx=(x_1,x_2,x_1+x_2)\T$$

reason: Its image is the plane y3 = y1 + y2.

$$1+2=3\ne4$$

reason: Full column rank gives uniqueness when reachable, not existence for every target.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Linear Systems and Solution Sets

Return to

$$\boxed{\,Ax=b\,}$$

1 existence: does a solution exist?
2 uniqueness: if it exists, is it the only one?
3 structure: what are all of them?

With

$$\Col(A)\quad\text{and}\quad\Null(A)$$

nothing new: both spaces are already built and measured

Axler, S. Linear Algebra Done Right, 4th ed., 3.16 range, p. 61; 3.11 null space, p. 59. Springer, 2024.

Linear Systems and Solution Sets

Solution set

$$\boxed{\,\mathcal{S}(A,b)=\{x\in\R^{n}:Ax=b\}\,}$$

a solution: any x that produces the requested b
consistent: $\mathcal{S}(A,b)\neq\emptyset$; otherwise inconsistent

Three possibilities

$$\text{empty},\qquad \text{one point},\qquad \text{infinitely many}$$

never two: two solutions force infinitely many: a line, a plane, or higher

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Linear Systems and Solution Sets

A =

1 0 1
0 1 1
2 1 3

Recall

$$\Col(A)=\{Ax:x\in\R^{n}\}$$

so: b is reachable exactly when some x reaches it

Existence

$$\boxed{\,Ax=b\ \text{is consistent}\iff b\in\Col(A)\,}$$

one sentence: the column space controls existence

Worked example

$\Col(A)$ is $y_3=2y_1+y_2$: $(3,2,8)$ passes since $2(3)+2=8$; $(3,2,9)$ fails.

Axler, S. Linear Algebra Done Right, 4th ed., 3.16 range, p. 61. Springer, 2024.

Linear Systems and Solution Sets

Suppose

$$Ax_1=b,\quad Ax_2=b\ \Longrightarrow\ A(x_1-x_2)=0\ \Longrightarrow\ x_1-x_2\in\Null(A)$$

so: if $\Null(A)=\{0\}$ then $x_1=x_2$

Uniqueness

$$\boxed{\,\Null(A)=\{0\}\ \Longrightarrow\ \text{at most one solution}\,}$$

careful: at most one, not one
if consistent: then it means exactly one

Either only the origin, or at least a whole line of erased inputs.

Axler, S. Linear Algebra Done Right, 4th ed., 3.15 injectivity, p. 60. Springer, 2024.

Linear Systems and Solution Sets

Particular

$$\boxed{\,Ax_p=b\ \text{for one particular }x_p\,}$$

any one: it does not matter which solution you found

General

$$\boxed{\,\mathcal{S}(A,b)=x_p+\Null(A)=\{x_p+z:z\in\Null(A)\}\,}$$

recipe: find one solution, then add everything A ignores

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Linear Systems and Solution Sets

Each one works

$$A(x_p+z)=b+0=b$$

uses: linearity and $Az=0$

Nothing is missed

$$Ax=b\Rightarrow A(x-x_p)=0$$

so: $x=x_p+z$ with $z=x-x_p\in\Null(A)$

Equality

$$\boxed{\,\mathcal{S}(A,b)=x_p+\Null(A)\,}$$

both ways: one inclusion each, which is what an equality of sets needs

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Linear Systems and Solution Sets

Augment

$$\left[\begin{array}{ccc|c}1&0&1&3\\0&1&1&2\\2&1&3&8\end{array}\right]\xrightarrow{r_3\leftarrow r_3-2r_1}\left[\begin{array}{ccc|c}1&0&1&3\\0&1&1&2\\0&1&1&2\end{array}\right]\xrightarrow{r_3\leftarrow r_3-r_2}\left[\begin{array}{ccc|c}1&0&1&3\\0&1&1&2\\0&0&0&0\end{array}\right]$$

step 1: $(2,1,3|8)-2(1,0,1|3)=(0,1,1|2)$
step 2: $(0,1,1|2)-(0,1,1|2)=(0,0,0|0)$
row 3: $0=0$: consistent

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Linear Systems and Solution Sets

Read off

$$x_1+x_3=3,\quad x_2+x_3=2,\quad x_3=t\ \Longrightarrow\ x_1=3-t,\ x_2=2-t$$

free: no pivot in column 3: choose $x_3$
$t=0$: one particular solution, $x_p=(3,2,0)\T$

All of them

$$\boxed{\,x=(3,2,0)\T+t\,(-1,-1,1)\T,\qquad t\in\R\,}$$

$t=1$: $(2,1,1)\T$: $2+1=3$, $1+1=2$, $4+1+3=8$

$b=(3,2,9)\T$: same steps: $9-6=3$, $3-2=1$, so row 3 reads $0=1$: no solution

Axler, S. Linear Algebra Done Right, 4th ed., 3.11 null space, p. 59. Springer, 2024.

Linear Systems and Solution Sets

Affine set

$$\boxed{\,x_0+S=\{x_0+s:s\in S\}\ \text{for a subspace }S\,}$$

here: $\mathcal{S}(A,b)=x_p+\Null(A)$: the null space, translated
through 0?: only if $b=0$; then $\mathcal{S}=\Null(A)$, a subspace

The null space through the origin, the same line moved to a particular solution, and an output that never moves.

Axler, S. Linear Algebra Done Right, 4th ed., 1.34 conditions for a subspace, p. 18. Springer, 2024.

Linear Systems and Solution Sets

Notation

$$r=\operatorname{rank}(A)$$

unique: when consistent, means $r=n$

Cases

$$\boxed{\begin{aligned}b\notin\Col(A)&:\ \text{none}\\b\in\Col(A),\ \Null(A)=\{0\}&:\ \text{one}\\b\in\Col(A),\ \dim\Null(A)>0&:\ \text{infinitely many}\end{aligned}}$$

free: $n-r$ directions

Top: the demand. Bottom: the solutions.

Axler, S. Linear Algebra Done Right, 4th ed., 3.21 fundamental theorem of linear maps, p. 62. Springer, 2024.

Invertibility and Singular Matrices

Demand

$$\Col(A)=\R^{m}\ \text{and}\ \Null(A)=\{0\}$$

reach: $r=m$: nothing is missed
no loss: $r=n$ by rank--nullity

So

$$\boxed{\,m=n\ \text{and}\ r=n\,}$$

square: a conclusion, not a hypothesis

Both cases have three input directions to spend. Only the top spends one on the null space.

Axler, S. Linear Algebra Done Right, 4th ed., 3.21 fundamental theorem of linear maps, p. 62. Springer, 2024.

Invertibility and Singular Matrices

Inverse

$$\boxed{\,AA^{-1}=A^{-1}A=I\,}$$

shapes: $A,A^{-1}\in\R^{n\times n}$ and $I=I_n$
both sides: two-sided by definition; for square $A$ either half implies the other

Singular

$$\boxed{\,A\ \text{singular}\iff A\ \text{has no inverse}\,}$$

nonsingular: the same as invertible

Worked example

$\begin{bmatrix}2&1\\1&1\end{bmatrix}^{-1}=\begin{bmatrix}1&-1\\-1&2\end{bmatrix}$

Axler, S. Linear Algebra Done Right, 4th ed., 3.79 identity, p. 90; 3.80 invertible and the word singular, p. 91; 3.60 inverse is unique, p. 82. Springer, 2024.

Invertibility and Singular Matrices

In general

$$A\ \text{invertible}\iff A\ \text{injective and surjective}$$

injective: $Ax=Ay\Rightarrow x=y$
surjective: $\Col(A)=\R^{m}$

If A is square

$$\boxed{\,\Null(A)=\{0\}\iff \Col(A)=\R^{n}\iff A\ \text{invertible}\,}$$

saving: check one side, the other follows
why: rank--nullity: $r=n$ makes both true at once

Axler, S. Linear Algebra Done Right, 4th ed., 3.63 invertibility, p. 83; 3.65 injectivity equivalent to surjectivity, p. 84. Springer, 2024.

Invertibility and Singular Matrices

Square A

$$\boxed{\begin{aligned}A\ \text{invertible}&\iff r=n\iff\Null(A)=\{0\}\\&\iff\text{columns independent}\iff\Col(A)=\R^{n}\end{aligned}}$$

and: $\iff Ax=b$ has one solution for every $b$
det: a preview; the determinant is a later lecture

Axler, S. Linear Algebra Done Right, 4th ed., 3.63 invertibility, p. 83; 3.65, p. 84; 3.21, p. 62. Springer, 2024.

Invertibility and Singular Matrices

Singular

$$\boxed{\,\Null(B)\neq\{0\}\iff B\ \text{singular}\,}$$

B: square and rank deficient

Worked example

$B=\begin{bmatrix}1&2\\2&4\end{bmatrix}$ has $r=1$, and $B(2,-1)\T=0$: a nonzero input lands on zero.

The unit square flattens; the area, and the inverse, are gone.

Sanderson, G. Essence of Linear Algebra, ch. 7. 3Blue1Brown, 2016 · Axler, 4th ed., 3.80, p. 91.

Invertibility and Singular Matrices

Preview · least squares

$$X=\begin{bmatrix}1&2\\2&4\\3&6\end{bmatrix},\qquad X\T X=\begin{bmatrix}1+4+9&2+8+18\\2+8+18&4+16+36\end{bmatrix}=\begin{bmatrix}14&28\\28&56\end{bmatrix}$$

the fit: $\min_\beta\|X\beta-y\|^{2}\ \Rightarrow\ X\T X\beta=X\T y$

entries: $(X\T X)_{ij}$ is column $i$ dotted with column $j$; the off-diagonal pair agree

Goodfellow, I., Bengio, Y. and Courville, A. Deep Learning, ch. 2. MIT Press, 2016.

Invertibility and Singular Matrices

Singular

$$X\begin{bmatrix}2\\-1\end{bmatrix}=\begin{bmatrix}2-2\\4-4\\6-6\end{bmatrix}=0\ \Longrightarrow\ X\T X\begin{bmatrix}2\\-1\end{bmatrix}=0$$

collinear: column 2 is twice column 1: $(2,-1)\T\in\Null(X)$

Null spaces

$$\boxed{\,X\T Xz=0\ \Rightarrow\ \|Xz\|^{2}=z\T X\T Xz=0\ \Rightarrow\ Xz=0\,}$$

so: $\Null(X\T X)=\Null(X)$
the fit: $\beta+t(2,-1)\T$ keeps $X\beta$: not identifiable

Goodfellow, I., Bengio, Y. and Courville, A. Deep Learning, ch. 2. MIT Press, 2016.

Invertibility and Singular Matrices

At a singularity

$$\boxed{\,J(q)\ \text{loses rank}\ \Longrightarrow\ \text{a tool direction is lost}\,}$$

rank drop: $\Col(J)$ shrinks: some $\dot x$ has no $\dot q$
nearby: joint rates grow without bound for tool directions along the lost one

one matrix: four spaces, one rank

Lynch, K. M. and Park, F. C. Modern Robotics, §5.3 singularity analysis, p. 191. Cambridge University Press, 2017 preprint.

Integrated worked example

Invertibility and Singular Matrices

$H=\begin{bmatrix}1&2\\0&1\end{bmatrix}$, $b=(5,2)\T$. Solve $Hx=b$, find $H^{-1}$, and connect uniqueness to rank.

$$x_2=b_2,\quad x_1=b_1-2b_2$$

reason: Back-substitution works for an arbitrary target.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Invertibility and Singular Matrices

Continue; check the original map.

$$x_2=b_2,\quad x_1=b_1-2b_2$$

reason: Back-substitution works for an arbitrary target.

$$H^{-1}=\begin{bmatrix}1&-2\\0&1\end{bmatrix},\quad x=(1,2)\T$$

reason: The inverse collects the coefficients of b.

$$HH^{-1}=H^{-1}H=I,\quad\Null(H)=\{0\},\ r=2$$

reason: A reversible map reaches every target exactly once.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Independent transfer

Invertibility and Singular Matrices

$K=\begin{bmatrix}1&2\\2&4\end{bmatrix}$. Compare targets $(5,10)\T$ and $(5,11)\T$. Can an inverse exist?

$$b=(5,10)\T:\quad x=(5,0)\T+t(-2,1)\T$$

reason: The second equation repeats twice the first.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Invertibility and Singular Matrices

Continue; check the original map.

$$b=(5,10)\T:\quad x=(5,0)\T+t(-2,1)\T$$

reason: The second equation repeats twice the first.

$$b=(5,11)\T:\quad0=1$$

reason: Subtract twice row 1 from row 2: inconsistent.

$$K(-2,1)\T=0,\quad(-2,1)\T\ne0$$

reason: An inverse cannot recover two inputs that share an output.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

The Four Fundamental Subspaces

So far

$$A:\R^{n}\to\R^{m},\qquad \Col(A)\ \text{and}\ \Null(A)$$

but: $A\T:\R^{m}\to\R^{n}$ is a linear map too

So ask

$$\boxed{\,\text{what spaces come from }A\T?\,}$$

the answer: $\Col(A\T)$ and $\Null(A\T)$: two more, for free

Axler, S. Linear Algebra Done Right, 4th ed., 3.54 transpose, p. 77. Springer, 2024.

The Four Fundamental Subspaces

New names

$$\boxed{\,\operatorname{Row}(A)=\Col(A\T),\qquad \Null(A\T)=\{y:A\T y=0\}\,}$$

left null: $A\T y=0$ is $y\T A=0\T$: y multiplies from the left

Axler, S. Linear Algebra Done Right, 4th ed., 3.54 transpose, p. 77; 3.11 null space, p. 59. Springer, 2024.

The Four Fundamental Subspaces

Output side

$$\Col(A)=\operatorname{span}\{(1,0,2)\T,(0,1,1)\T\}$$

$\Null(A\T)$: $=\operatorname{span}\{(-2,-1,1)\T\}$

Input side

$$\operatorname{Row}(A)=\operatorname{span}\{(1,0,1)\T,(0,1,1)\T\}$$

$\Null(A)$: $=\operatorname{span}\{(-1,-1,1)\T\}$

So

$$\boxed{\,y=(-2,-1,1)\T,\qquad A\T y=0\,}$$

read it: the row weights

Axler, S. Linear Algebra Done Right, 4th ed., 3.57 column rank equals row rank, p. 78. Springer, 2024.

The Four Fundamental Subspaces

Rank equality

$$\boxed{\,\operatorname{rank}(A)=\operatorname{rank}(A\T)=r\,}$$

so: column rank equals row rank

All four

$$\boxed{\begin{aligned}\dim\Col(A)&=r&\dim\Null(A)&=n-r\\\dim\Col(A\T)&=r&\dim\Null(A\T)&=m-r\end{aligned}}$$

two sums: $r+(n-r)=n$ and $r+(m-r)=m$

Worked example

Our $A$: $r=2$, so $2+1=3$ columns and $2+1=3$ rows.

Axler, S. Linear Algebra Done Right, 4th ed., 3.57 column rank equals row rank, p. 78; 3.21, p. 62. Springer, 2024.

The Four Fundamental Subspaces

Lecture 1 in one line

$$\boxed{\,\dim=r,\ n-r,\ r,\ m-r\,}$$

later: the two pairs form orthogonal-complement decompositions

Axler, S. Linear Algebra Done Right, 4th ed., 3.21, p. 62; 3.57, p. 78. Springer, 2024.

Integrated whiteboard problem

Linear Systems and Four Subspaces

Given $C=\begin{bmatrix}1&2&0\\0&1&1\\1&3&1\end{bmatrix}$, $b=(4,1,5)\T$. Find reachability, all solutions and four-space dimensions.

$$r_3-r_1-r_2=0\ \Rightarrow\ b_3=b_1+b_2,\quad r=2$$

r: rank; two independent rows

$$x=(2,1,0)\T+t(2,-1,1)\T$$

t: any real; dimensions: $2,1,2,1$

Your turn · new matrix

Linear Systems and Four Subspaces

$D=\begin{bmatrix}1&0&2\\0&1&-1\end{bmatrix}$, $b=(3,1)\T$. Find all solutions. Is every target reachable? Is its input unique?

$$x_3=t,\quad x_1=3-2t,\quad x_2=1+t$$

t: the free third coordinate

$$x=(3,1,0)\T+t(-2,1,1)\T$$

rank 2: every target reached; nullity 1: inputs not unique

Selected references

Courses, books and lectures

Strang, G. 18.06 Linear Algebra, Lectures 3, 7 and 10. MIT OCW, 2010. CC BY-NC-SA 4.0.
Grizzle, J. W. ROB 501: Mathematics for Robotics, Michigan. CC BY-NC 4.0.
Axler, S. Linear Algebra Done Right, 4th ed., 2024. Open access.
Goodfellow, Bengio and Courville. Deep Learning, MIT Press, 2016, ch. 2.
Sanderson, G. Essence of Linear Algebra, ch. 2, 3 and 7. 3Blue1Brown, 2016.

ROB-GY 6013 course references · Notation and Conventions · Textbooks and Reading Guide

Selected references

Papers and books

Strang, G. The fundamental theorem of linear algebra. Amer. Math. Monthly, 1993.
Black, K. et al. π₀. RSS, 2025; arXiv:2410.24164, posted 2024.
Lipman, Y. et al. Flow Matching for Generative Modeling. ICLR, 2023. arXiv:2210.02747.
Lynch and Park. Modern Robotics, §5.3, §6.3, 2017.
Barfoot, T. D. State Estimation for Robotics, ch. 4. Cambridge, 2017.