Linear Algebra II

Coordinates, Determinant, Eigenstructure

Mathematics for Robotics · ROB-GY 6013 · NYU Tandon

Jing Zhang · Fall 2026 · September 9

One Map, Many Rulers

H =

1 0 1
0 1 1
2 1 3

Rank–nullity

$$\dim\ker H+\dim\operatorname{im}H=1+2=3$$

$\operatorname{im}H$: output plane, $\Col(H)$

$$\begin{gathered}\boxed{\,\Null(A)=\ker(A)=\{x\in\R^{n}:Ax=0\}\,}\\[3pt]\boxed{\,\Null(H)=\operatorname{span}\{[-1,-1,1]\T\}\,}\end{gathered}$$

$\Null(A)=\ker(A)$: the inputs $A$ sends to $0$
$x\in\R^{n}$: an $n$-entry input column
span: all its multiples: a line

One Map, Many Rulers

Which choice makes the matrix simple?

One Map, Many Rulers

Coordinates and Change of Basis

$$\mathcal B=(b_1,\dots,b_n)\ \text{is a basis of }\R^n\iff\text{independent and spans }\R^n$$

independent: none of them is a mixture of the rest
spans: every vector is some mixture of the list
n: a basis of $\R^n$ has exactly n vectors

Axler, S. Linear Algebra Done Right, 4th ed., 2.26 basis, p. 39. Springer, 2024. Open access.

Coordinates and Change of Basis

One recipe only

$$x=\alpha_1b_1+\dots+\alpha_nb_n\quad\text{in exactly one way}$$

$\alpha_j$: the amount of $b_j$ the recipe calls for
exists: because the list spans
only one: because the list is independent

Axler, S. Linear Algebra Done Right, 4th ed., 2.28 criterion for basis, p. 39. Springer, 2024. Open access.

Coordinates and Change of Basis

Coordinate vector

$$[x]_{\mathcal B}=\begin{pmatrix}\alpha_1\\\vdots\\\alpha_n\end{pmatrix}\ \text{where}\ x=\alpha_1b_1+\dots+\alpha_nb_n$$

x: the arrow itself, out in the world
$[x]_{\mathcal B}$: a vector in $\R^{n}$ holding the coefficients, not the arrow $x$
$\alpha_j$: how much of $b_j$ the one recipe uses

Axler, S. Linear Algebra Done Right, 4th ed., 3.73 the matrix of a vector, p. 88. Springer, 2024. Open access.

Coordinates and Change of Basis

Worked example

$[x]_{\mathcal E}=(4,2)\T$, $b_1=(1,1)$, $b_2=(1,-1)$. Then $\alpha_1+\alpha_2=4$ and $\alpha_1-\alpha_2=2$, so $\alpha_1=3$, $\alpha_2=1$. Check: $3b_1+b_2=(4,2)$.

$$[x]_{\mathcal E}=(4,2)\T\longrightarrow[x]_{\mathcal B}=(3,1)\T$$

$[x]_{\mathcal E}$: the given standard-coordinate column
$[x]_{\mathcal B}$: the same vector in $\mathcal B$: three $b_1$, one $b_2$

Coordinates and Change of Basis

Convert to standard coordinates

$$\boxed{[x]_{\mathcal E}=B\,[x]_{\mathcal B}},\qquad B=\bigl[\,[b_1]_{\mathcal E}\ \cdots\ [b_n]_{\mathcal E}\,\bigr]$$

B: columns are the basis vectors in standard coordinates
$[x]_{\mathcal E}$: written $x$ once the standard basis is understood

Coordinates and Change of Basis

Convert back with the inverse

$$\boxed{[x]_{\mathcal B}=B^{-1}[x]_{\mathcal E}}$$

$B^{-1}$: undoes $B$, back to $\mathcal B$-coordinates

Worked example

$M^{-1}=\tfrac1{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}$: swap, negate, divide by $\det M$.

$B=\begin{pmatrix}1&1\\1&-1\end{pmatrix}$, $\det B=-2$, so $B^{-1}=\tfrac12B$ and $[x]_{\mathcal B}=\tfrac12(6,2)\T=(3,1)\T$.

Coordinates and Change of Basis

Standard basis

$$\begin{gathered}A^{-1}A=AA^{-1}=I\\[3pt]e_1,\dots,e_n:\quad B=I\ \Rightarrow\ [x]_{\mathcal E}=I^{-1}x=x\end{gathered}$$

$A^{-1}$: undoes $A$ from either side
$e_j$: 1 in slot j, 0 everywhere else
I: its basis matrix, and its own inverse

Practice

Coordinates and Change of Basis

Same $\mathcal B=\{(1,1),(1,-1)\}$. What is $[y]_{\mathcal B}$ for $y=(6,0)$?

$[y]_{\mathcal B}=(3,3)\T$

Check it forwards: $3(1,1)+3(1,-1)=(6,0)$. Or read it off: $B^{-1}(6,0)=\tfrac12(6,6)=(3,3)$.

Coordinates and Change of Basis

$$[x]_{\mathcal C}=P_{\mathcal C\leftarrow\mathcal B}\,[x]_{\mathcal B},\qquad P_{\mathcal C\leftarrow\mathcal B}=C^{-1}B$$

P: turns B-coordinates into C-coordinates
C⁻¹B: rebuild the vector from B, then read it off in C

Axler, S. Linear Algebra Done Right, 4th ed., 3.84 change-of-basis formula, p. 93. Springer, 2024. Axler writes this matrix as M(I, …), the identity map read between two bases.

Coordinates and Change of Basis

B =

1 1
1 -1

Worked example

$\mathcal C=\{(1,0),(1,1)\}$, so $C^{-1}=\begin{pmatrix}1&-1\\0&1\end{pmatrix}$ and $P=C^{-1}B=\begin{pmatrix}0&2\\1&-1\end{pmatrix}$.

$b_1=0c_1+1c_2$ and $b_2=2c_1-1c_2$: the columns of $P$.

Coordinates and Change of Basis

$$x=(4,2)\ \text{in }\mathcal E,\qquad [x]_{\mathcal B}=(3,1),\qquad [x]_{\mathcal C}=(2,2)$$

(4,2): read in the standard basis E
(3,1): read in B = {(1,1), (1,-1)}
(2,2): read in C = {(1,0), (1,1)}

Worked example

$P\,[x]_{\mathcal B}=\begin{pmatrix}0&2\\1&-1\end{pmatrix}\!\begin{pmatrix}3\\1\end{pmatrix}=\begin{pmatrix}2\\2\end{pmatrix}$

Coordinates and Change of Basis

P =

0 2
1 -1

$$P_{\mathcal B\leftarrow\mathcal C}=P_{\mathcal C\leftarrow\mathcal B}^{-1}=B^{-1}C$$

left side: C-coordinates in, B-coordinates out
B⁻¹C: rebuild from C, then read off in B

$$P^{-1}=\begin{pmatrix}1/2&1\\1/2&0\end{pmatrix},\qquad P^{-1}(2,2)=(3,1)$$

(2,2): the C-column
(3,1): back to the B-column

Coordinates and Change of Basis

$$P_{\mathcal D\leftarrow\mathcal B}=P_{\mathcal D\leftarrow\mathcal C}\,P_{\mathcal C\leftarrow\mathcal B}$$

inner pair: C meets C and cancels
order: the right factor acts first

Strang, G. 18.06 Linear Algebra, Lecture 31, change of basis. MIT OpenCourseWare, 2010. CC BY-NC-SA 4.0.

Coordinates and Change of Basis

$$[Tx]_{\mathcal B}=[T]_{\mathcal B}\,[x]_{\mathcal B},\qquad\text{column }j=[Tb_j]_{\mathcal B}$$

T: a linear map from the space to itself
$[x]_{\mathcal B}$: coordinates in, coordinates out, same basis
column j: feed in $b_j$, write the answer in the same basis
$[T]_{\mathcal B}$: depends on $T$ and on the ordered basis $\mathcal B$

Axler, S. Linear Algebra Done Right, 4th ed., 3.31 matrix of a linear map M(T), p. 69. Springer, 2024.

Coordinates and Change of Basis

A =

2 1
1 2

Worked example

$Ab_1=(3,3)=3b_1+0b_2$, so column 1 is $(3,0)\T$. $Ab_2=(1,-1)=0b_1+1b_2$, so column 2 is $(0,1)\T$.

$$\boxed{[A]_{\mathcal B}=B^{-1}AB=\begin{pmatrix}3&0\\0&1\end{pmatrix}}$$

B⁻¹AB: rebuild, apply A, read off
3 and 1: the factor each basis direction is stretched by
off-diagonal: zero here: the two directions no longer mix

Coordinates and Change of Basis

$$[T]_{\mathcal C}=P\,[T]_{\mathcal B}\,P^{-1},\qquad P=P_{\mathcal C\leftarrow\mathcal B}$$

P: puts B-coordinates into C, the same P as before
$[T]_{\mathcal B}$: the operator acts, in $\mathcal B$
P⁻¹: takes the C-column into B, and acts first
similar: the name for two matrices related this way

Axler, S. Linear Algebra Done Right, 4th ed., 3.84 change of basis, p. 93. Springer, 2024. The word similar is Strang's: MIT 18.06, Lecture 28.

Coordinates and Change of Basis

C =

1 1
0 1

Worked example

$C^{-1}A=\begin{pmatrix}1&-1\\1&2\end{pmatrix}$, then $(C^{-1}A)\,C=\begin{pmatrix}1&0\\1&3\end{pmatrix}=[A]_{\mathcal C}$.

$$\begin{gathered}\boxed{\mathrm{tr}(A)=\textstyle\sum_i a_{ii}}\\[3pt]\boxed{\mathrm{tr}(B^{-1}AB)=\mathrm{tr}(A)}\end{gathered}$$

tr: sum of the diagonal entries
$B^{-1}AB$: new basis, same sum

Coordinates and Change of Basis

The same gains, written in two frames

Axler, S. Linear Algebra Done Right, 4th ed., 3.84 change-of-basis formula, p. 93. Springer, 2024. Open access.

Practice

Coordinates and Change of Basis

$[T]_{\mathcal B}=\begin{pmatrix}3&0\\0&1\end{pmatrix}$ with $\mathcal B=\{(1,1),(1,-1)\}$. What is $[T]_{\mathcal E}$?

$$\begin{gathered}[x]_B=B^{-1}[x]_E\\[2pt][Tx]_B=[T]_B[x]_B\\[2pt][Tx]_E=B[Tx]_B\\[3pt]\boxed{[T]_E=B[T]_BB^{-1}}\end{gathered}$$

the three lines: into $B$, act, back out; together they are the box

Rotations and Frames

Orthonormal

$$\langle q_i,q_j\rangle=\begin{cases}1&i=j\\0&i\neq j\end{cases}$$

$q_i$: the basis vectors, one for each axis
$i=j$: every one of them has length one
$i\neq j$: every pair meets at a right angle

Axler, S. Linear Algebra Done Right, 4th ed., 6.22 orthonormal, p. 197; 6.27 orthonormal basis, p. 199. Springer, 2024. Open access.

Rotations and Frames

$$[x]_{\mathcal Q}=Q\T x,\qquad\text{i.e.}\ \alpha_j=\langle x,q_j\rangle$$

$Q$: the $n\times n$ matrix with columns $q_1,\dots,q_n$
$Q\T$: exactly $Q^{-1}$: a transpose, nothing to solve
$\alpha_j$: how far $x$ reaches along axis $j$

Axler, S. Linear Algebra Done Right, 4th ed., 6.30(a) writing a vector as a linear combination of an orthonormal basis, p. 200. Springer, 2024. Open access.

Rotations and Frames

Orthogonal

$$Q\T Q=I\iff\text{the columns of }Q\text{ are orthonormal},\qquad Q^{-1}=Q\T$$

$(Q\T Q)_{ij}$: column $i$ dotted with column $j$
$I$: ones on the diagonal, zeros off it
$Q^{-1}=Q\T$: the inverse costs one transpose

Both ends are orthogonal; the frames between them are smaller.

Grizzle, J. W. ROB 501: Mathematics for Robotics, notes 11, symmetric and orthogonal matrices. University of Michigan, 2018. CC BY-NC 4.0.

Rotations and Frames

Worked example

$R=\tfrac15\begin{bmatrix}3&-4\\4&3\end{bmatrix}$. Column one with itself: $\tfrac{9}{25}+\tfrac{16}{25}=1$. Column one with column two: $\tfrac{-12}{25}+\tfrac{12}{25}=0$. So $R\T R=I$, with $\cos t=\tfrac35$, $\sin t=\tfrac45$, $t\approx53.13^\circ$.

Rotations and Frames

Rotation group

$$SO(n)=\{R\in\R^{n\times n}\ :\ R\T R=I,\ \det R=1\}$$

$R\T R=I$: lengths and angles survive the map
$\det R=1$: a turn, not a mirror; the number itself arrives next chapter
$n$: 2 for planar orientations, 3 for spatial ones

Lynch, K. M. & Park, F. C. Modern Robotics, Definitions 3.1 and 3.2, p. 70. Cambridge, 2017. SO(3) and SO(2) are defined separately there; the general n is our shorthand.

Rotations and Frames

R =

3/5 -4/5
4/5 3/5

Active

$$x'=Rx:\qquad R\,(5,0)=(3,4)$$

$x$: the arrow before the turn
$R$: the turn itself, the same for every arrow
$x'$: a different arrow, of the same length

Rotations and Frames

R =

3/5 -4/5
4/5 3/5

Passive

$$[x]_{\mathcal R}=R\T x:\qquad R\T(3,4)=(5,0)$$

$x$: one physical point; nothing about it moves
$R\T$: chapter 1's $B^{-1}x$, with $B=R$
$[x]_{\mathcal R}$: the same point's new address

Lynch, K. M. & Park, F. C. Modern Robotics, §3.2.1.2, uses of rotation matrices, p. 71. Cambridge University Press, 2017.

Rotations and Frames

Compose

$$R_{ac}=R_{ab}R_{bc}$$

$R_{ab}$: b-coordinates rewritten as a-coordinates
the inner b: cancels; read the product right to left

Worked example

$R_{ab}=\tfrac15\begin{bmatrix}3&-4\\4&3\end{bmatrix}$, $R_{bc}=\begin{bmatrix}0&-1\\1&0\end{bmatrix}$, so $R_{ac}=\tfrac15\begin{bmatrix}-4&-3\\3&-4\end{bmatrix}$ — still unit, still perpendicular.

Rotations and Frames

$$S=\begin{bmatrix}0&1\\1&0\end{bmatrix},\qquad S\T S=I$$

$S$: the mirror in the line $y=x$
$S\T S=I$: every length and every angle survives

Integrated worked example

Rotations and Frames

$R=\frac15\begin{bmatrix}3&-4\\4&3\end{bmatrix}$ maps sensor to world. Sensor position $p=(1,2)\T$; measured displacement $d_s=(5,0)\T$. Find the world point and recover $d_s$.

$$d_w=Rd_s=(3,4)\T,\quad q_w=p+d_w=(4,6)\T$$

reason: Rotate a displacement; then add the origin position.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Rotations and Frames

Continue; check the original map.

$$d_w=Rd_s=(3,4)\T,\quad q_w=p+d_w=(4,6)\T$$

reason: Rotate a displacement; then add the origin position.

$$R\T(q_w-p)=R\T(3,4)\T=(5,0)\T$$

reason: Subtract the origin before reversing the coordinates.

$$\|d_s\|=\|d_w\|=5$$

reason: Orthogonality preserves displacement length, not distance from an unrelated origin.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Independent transfer

Rotations and Frames

$Q=\begin{bmatrix}0&-1\\1&0\end{bmatrix}$ maps body to world, with origin $p=(2,-1)\T$. A world point is $q=(1,2)\T$. Find its body displacement and check.

$$d_w=q-p=(-1,3)\T$$

reason: Subtract positions in the same frame.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Rotations and Frames

Continue; check the original map.

$$d_w=q-p=(-1,3)\T$$

reason: Subtract positions in the same frame.

$$d_b=Q\T d_w=(3,1)\T$$

reason: The fixed point calls for a passive coordinate change.

$$p+Qd_b=(2,-1)\T+(-1,3)\T=q$$

reason: Forward reconstruction checks the frame convention.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

The Determinant

What number tells a turn from a mirror?

The Determinant

Definition

$$\det\begin{pmatrix}a&b\\c&d\end{pmatrix}=ad-bc$$

a, b, c, d: the four entries, columns first
ad − bc: the signed area of the parallelogram the columns span
signed: the sign records orientation, not size

Axler, S. Linear Algebra Done Right, 4th ed., 9.43 determinant of a matrix, p. 355; 9.47 the 2×2 formula, p. 356. Springer, 2024. Open access.

The Determinant

Worked example

$A=\begin{bmatrix}2&1\\1&2\end{bmatrix}$: $ad-bc=2\cdot2-1\cdot1=3$, so every area in the plane is tripled.

The Determinant

$$\det S=0\cdot 0-1\cdot 1=-1$$

$0\cdot 0$: the diagonal product
$1\cdot 1$: the off-diagonal product
−1: the plane has been turned over: a right-handed pair of axes is now left-handed

The square flattens, then reopens the other way round.

Sanderson, G. Essence of Linear Algebra, ch. 6: the determinant. 3Blue1Brown, 2016.

The Determinant

S =

0 1
1 0

$$\det R=\tfrac{3}{5}\cdot\tfrac{3}{5}-\left(-\tfrac{4}{5}\right)\cdot\tfrac{4}{5}=\tfrac{9}{25}+\tfrac{16}{25}=1$$

9/25: the diagonal product
+16/25: minus a negative, so the off-diagonal term adds
= 1: no stretch, no flip: a turn

Map	QᵀQ = I	det	Verdict
rotation R	✓	+1	SO(2)
mirror S	✓	−1	reflection

Lynch, K. M. & Park, F. C. Modern Robotics, Definitions 3.1 and 3.2, p. 70. Cambridge, 2017.

The Determinant

Rule	Why, in volume
$\det(AB)=\det A\cdot\det B$	do one map, then the other
$\det A\T=\det A$	same signed volume scale factor
$\det A^{-1}=1/\det A$	undoing a scaling by 3 scales by 1/3
$\det(cA)=c^{n}\det A$	all n directions scaled by c
triangular $T$: $\det T=t_{11}\cdots t_{nn}$	shearing moves the box, not its volume

Row five, not row one: the other four rules are not drawn.

Axler, S. Linear Algebra Done Right, 4th ed., 9.48 upper-triangular, p. 356; 9.49 multiplicative, p. 357; 9.56(a) transpose, p. 360. Springer, 2024. Open access.

The Determinant

$$\det A=a_{11}M_{11}-a_{12}M_{12}+a_{13}M_{13}$$

$a_{1j}$: the three entries of the first row
$M_{1j}$: the 2×2 determinant left when row 1 and column j are deleted
+ − +: a chequerboard of signs, starting at plus

Strang, G. 18.06 Linear Algebra, Lecture 19: determinant formulas and cofactors. MIT OpenCourseWare, 2010. CC BY-NC-SA 4.0.

The Determinant

H =

1 0 1
0 1 1
2 1 3

Worked example

$\det H=1(1\cdot3-1\cdot1)-0(0\cdot3-1\cdot2)+1(0\cdot1-1\cdot2)=2-0-2=0$

Last week $a_3=a_1+a_2$ by eye; one number says the same thing.

The Determinant

$$\det A=0\iff A\ \text{is singular}\iff\Null(A)\neq\{0\}\iff\operatorname{rank}A<n$$

det A = 0: the unit box is flattened
singular: no inverse exists
$\Null(A)$: the null space: some nonzero input is sent to zero
rank: fewer than n independent columns

Every point of the plane lands on one line through the origin.

Axler, S. Linear Algebra Done Right, 4th ed., 9.50 invertible if and only if nonzero determinant, p. 357. Springer, 2024. Open access.

Practice

The Determinant

$$K=\begin{bmatrix}2&1&0\\1&2&1\\0&1&2\end{bmatrix}\ \text{singular?}$$

singular: no inverse: some direction is destroyed

$\det K=4$, so no. $2(4-1)-1(2-0)+0=6-2=4$, so $Kx=b$ has exactly one solution for every $b$. Or eliminate — subtracting a multiple of one row leaves $\det$ alone: $r_2-\tfrac12r_1$, then $r_3-\tfrac23r_2$, leaves pivots $2,\tfrac32,\tfrac43$, product $4$.

The Determinant

$$\operatorname{vol}A(\Omega)=|\det A|\cdot\operatorname{vol}\Omega$$

$\Omega$: any region you like
$A(\Omega)$: its image under A
$|\det A|$: one factor, the same for every region
absolute value: volume has no sign; orientation does

Axler, S. Linear Algebra Done Right, 4th ed., 9.61 T changes volume by a factor of |det T|, p. 363. Springer, 2024. Open access.

The Determinant

$$\det J(\theta)=l_1l_2\sin\theta_2$$

J: joint rates to tip velocity, Lecture 1's Jacobian
$l_1,l_2$: the two link lengths, both fixed
$\theta_2$: the elbow angle
= 0: exactly when the elbow is straight or folded

Worked example

$l_1=l_2=1$: $\det J=\sin\theta_2$, and that set has area $\pi|\sin\theta_2|$ — $\pi$ at $90^\circ$, and $0$ when the elbow is straight.

Lynch, K. M. & Park, F. C. Modern Robotics, §5.3 singularities. Cambridge University Press, 2017.

The Determinant

$$\det(0.01\,I_{10})=(0.01)^{10}=10^{-20}$$

$0.01\,I_{10}$: one hundredth of the identity in ten dimensions
$(0.01)^{10}$: one factor from each diagonal entry
$10^{-20}$: small, and every direction still scaled alike
$\kappa$: $\dfrac{\text{largest stretch}}{\text{smallest stretch}}$, the number to ask for — Lecture 4

$0.2\,I$: $\kappa=1$

$\operatorname{diag}(1,0.04)$: $\kappa=25$

Integrated worked example

The Determinant

$S=\begin{bmatrix}1&2\\0&1\end{bmatrix}$, $D=\operatorname{diag}(-2,1/2)$. Apply $S$ then $D$. Find area factor, orientation and inverse existence.

$$A=DS=\begin{bmatrix}-2&-4\\0&1/2\end{bmatrix}$$

reason: Composition order follows the maps, not their names.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

The Determinant

Continue; check the original map.

$$A=DS=\begin{bmatrix}-2&-4\\0&1/2\end{bmatrix}$$

reason: Composition order follows the maps, not their names.

$$\det A=\det D\det S=(-1)(1)=-1$$

reason: Absolute determinant is area factor; the sign records orientation.

$$|\det A|=1,\quad A^{-1}\text{ exists},\quad\|Ae_1\|=2$$

reason: Area preservation does not imply length preservation or rotation.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Independent transfer

The Determinant

$T=\operatorname{diag}(2,1/4)$ acts on a region of area 6. Find its new area. Does every vector shrink? What changes if $1/4$ becomes 0?

$$\det T=1/2,\quad\operatorname{area}(T\Omega)=3$$

reason: Multiply the original area by the absolute determinant.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

The Determinant

Continue; check the original map.

$$\det T=1/2,\quad\operatorname{area}(T\Omega)=3$$

reason: Multiply the original area by the absolute determinant.

$$Te_1=2e_1$$

reason: Area shrinks, but the first axis expands.

$$\det\operatorname{diag}(2,0)=0$$

reason: The image lies on a line; area is zero and the inverse fails.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

The Determinant

$$\begin{gathered}\det(A-\lambda I)=0\\[4pt]A=\begin{bmatrix}2&1\\1&2\end{bmatrix}:\quad\det(A-\lambda I)=\begin{vmatrix}2-\lambda&1\\1&2-\lambda\end{vmatrix}=(2-\lambda)^{2}-1\end{gathered}$$

$A-\lambda I$: A with $\lambda$ subtracted along the diagonal
det = 0: this chapter's test for collapse
$\lambda$: unknown; the whole of the next chapter

Axler, S. Linear Algebra Done Right, 4th ed., 9.51, p. 358; 9.63 characteristic polynomial, p. 363. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

One rider keeps its dashed line. The other does not.

Eigenvalues and Eigenvectors

Eigenvector

$$Av=\lambda v,\qquad v\neq 0$$

A: the map: a square matrix
v: the eigenvector, a nonzero vector whose line A preserves
$\lambda$: the eigenvalue, the factor v is stretched by
$v\neq0$: required; $\lambda=0$ is allowed

Axler, S. Linear Algebra Done Right, 4th ed., 5.5 eigenvalue, p. 134; 5.8 eigenvector, p. 135. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

Allow v = 0 and every number is an eigenvalue of every matrix.

$$\lambda=0\ \text{is an eigenvalue}\iff\Null(A)\neq\{0\}\iff\det A=0$$

$\lambda=0$: $Av=0$ for some $v\neq0$
$\Null(A)$: Lecture 1's null space: what the map destroys
$\det A=0$: chapter 3's test for a collapse

The marked arrow goes to the origin.

Axler, S. Linear Algebra Done Right, 4th ed., 9.50 invertible if and only if nonzero determinant, p. 357. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

Eigenspace

$$E(\lambda,A)=\Null(A-\lambda I)=\{x:\ Ax=\lambda x\}$$

$E(\lambda,A)$: every vector A stretches by $\lambda$, and 0
$\Null(A-\lambda I)$: Lecture 1's null space, of a different matrix
closed: $A(\alpha x+\beta y)=\lambda(\alpha x+\beta y)$, so it is a subspace

Axler, S. Linear Algebra Done Right, 4th ed., 5.52 eigenspace, p. 164. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

$$\lambda\ \text{is an eigenvalue}\iff\exists\,v\neq0:\ (A-\lambda I)v=0\iff\det(A-\lambda I)=0$$

subtract: $\lambda v=\lambda Iv$, so both terms move to one side
$\exists v\neq0$: some nonzero v solves it, not the v you name
collapse: chapter 3: it has one exactly when its determinant is 0

Axler, S. Linear Algebra Done Right, 4th ed., 5.7 equivalent conditions, p. 135; 9.51 eigenvalue test, p. 358. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

A =

2 1
1 2

Characteristic polynomial

$$p(\lambda)=\det(A-\lambda I),\qquad \underbrace{p=\lambda^{2}-(\operatorname{tr}A)\lambda+\det A}_{2\times2\ \text{only}}$$

$p$: degree n in $\lambda$; its roots are the eigenvalues
$\operatorname{tr}A$: the sum of the diagonal entries

Axler, S. Linear Algebra Done Right, 4th ed., 9.63 characteristic polynomial, p. 363; 8.47 trace of a matrix, p. 326. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

A =

2 1
1 2

Worked example

$A-\lambda I=\begin{pmatrix}2-\lambda&1\\1&2-\lambda\end{pmatrix}$

$$\det(A-\lambda I)=(2-\lambda)^{2}-1=\lambda^{2}-4\lambda+3=(\lambda-3)(\lambda-1)$$

expand: $(2-\lambda)^{2}=4-4\lambda+\lambda^{2}$, then subtract 1
roots: $\lambda=3$ and $\lambda=1$
check: tr A = 4 and det A = 3 give the same polynomial

Eigenvalues and Eigenvectors

A =

2 1
1 2

Worked example

$\lambda=3$: solve $(A-3I)v=0$, last week's null-space computation.

$$A-3I=\begin{pmatrix}-1&1\\1&-1\end{pmatrix}\to\begin{pmatrix}1&-1\\0&0\end{pmatrix}$$

reduce: $r_2\to r_2+r_1$, then $-r_1$: one pivot, one free
read: $v_1-v_2=0$, so $v_2=t$ and $v=t(1,1)$
check: $A(1,1)=(2+1,\,1+2)=(3,3)=3(1,1)$

Strang, G. 18.06 Linear Algebra, Lecture 21: eigenvalues and eigenvectors. MIT OCW, 2010. CC BY-NC-SA 4.0.

Eigenvalues and Eigenvectors

A =

2 1
1 2

Worked example

$\lambda=1$: solve $(A-I)v=0$.

$$A-I=\begin{pmatrix}1&1\\1&1\end{pmatrix}\to\begin{pmatrix}1&1\\0&0\end{pmatrix}$$

reduce: $r_2\to r_2-r_1$: one pivot, one free
read: $v_1+v_2=0$, so $v=t(1,-1)$
check: $A(1,-1)=(2-1,\,1-2)=(1,-1)$, unchanged

Eigenvalues and Eigenvectors

A =

2 1
1 2

$$B^{-1}AB=\begin{pmatrix}3&0\\0&1\end{pmatrix},\qquad Ab_1=3b_1,\quad Ab_2=1\,b_2$$

$b_1=(1,1)$: column 1 of B, and the first diagonal entry is 3
$b_2=(1,-1)$: column 2 of B, and the second diagonal entry is 1
order: swap the columns of B and the diagonal swaps with them

Eigenvalues and Eigenvectors

$$\operatorname{tr}A=\lambda_1+\dots+\lambda_n,\qquad\det A=\lambda_1\cdots\lambda_n$$

$\operatorname{tr}A$: the sum of the diagonal entries of A
$\lambda_j$: the $n$ roots of $p$, over $\mathbb C$ and with multiplicity
over $\mathbb C$: required: the next page is real and has no real root

Worked example

$\operatorname{tr}A=2+2=4=3+1$ and $\det A=4-1=3=3\cdot1$.

Axler, S. Linear Algebra Done Right, 4th ed., 8.52 trace is the sum of the eigenvalues, p. 328; 9.55 the determinant is their product, p. 359. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

$$R=\tfrac15\begin{pmatrix}3&-4\\4&3\end{pmatrix},\quad p(\lambda)=\lambda^{2}-\tfrac65\lambda+1,\quad \Delta=\tfrac{36}{25}-4<0$$

$\operatorname{tr}R=6/5$: $3/5+3/5$
$\det R=1$: a rotation preserves area
$\Delta=b^{2}-4ac$: of $a\lambda^{2}+b\lambda+c$; negative here, so no real root and no real direction survives

No dashed line appears.

Lynch, K. M. & Park, F. C. Modern Robotics, Definition 3.2, SO(2), p. 70. Cambridge University Press, 2017.

Practice

Eigenvalues and Eigenvectors

What are the eigenvalues of $\begin{pmatrix}3&7\\0&5\end{pmatrix}$?

3 and 5: $\det(A-\lambda I)=(3-\lambda)(5-\lambda)$, and a triangular determinant is the product of the diagonal.

For $\lambda=3$, $v=(1,0)$. For $\lambda=5$ it is $(7,2)$, not $(0,1)$: $(A-5I)(7,2)=(-14+14,\,0)=0$.

Axler, S. Linear Algebra Done Right, 4th ed., 9.48 determinant of an upper-triangular matrix, p. 356. Springer, 2024. Open access.

Eigenvalues and Eigenvectors

Similar matrices

$$Av=\lambda v\ \Longrightarrow\ (P^{-1}AP)(P^{-1}v)=\lambda\,(P^{-1}v)$$

$P$: columns: the new basis in old coordinates, so $[x]_{\text{old}}=P[x]_{\text{new}}$
$P^{-1}v$: the same eigenvector, read in the new coordinates
$\lambda$: untouched: similar matrices share their eigenvalues

Strang, G. 18.06 Linear Algebra, Lecture 28: similar matrices and Jordan form. MIT OCW, 2010. CC BY-NC-SA 4.0.

Eigenvalues and Eigenvectors

Covariance: axes and spread of an ellipse — Lecture 8, PCA in Lecture 4
Loss curvature $\tfrac12x\T Hx$: the largest eigenvalue caps the step — Lecture 6
A linear iteration is stable or it is not — next chapter, Lecture 7
Principal moments of an inertia matrix
Power iteration: the turning arrows as an algorithm, one $|\lambda|$ strictly largest

$cf,\,c>0$: same minimiser, gradient $\times c$

Eigenvalues and Eigenvectors

A =

2 1
1 2

Worked example

$x_0=(1,0)$: $Ax_0=(2,1)$, $A^{2}x_0=(5,4)$, $A^{3}x_0=(14,13)$.

Divide by the first entry: $(1,0),(1,\tfrac12),(1,\tfrac45),(1,\tfrac{13}{14})$.

First entries grow by $2,\tfrac52,\tfrac{14}5$ toward 3, the strictly largest $|\lambda|$. From $(1,-1)$ nothing turns.

Directions only; each arrow is drawn to its own scale.

Integrated worked example

Eigenvalues and Eigenvectors

$A=\begin{bmatrix}4&2\\0&1\end{bmatrix}$. Find eigenvalues and one eigenvector for each. Express $x=(0,3)\T$ in those directions and compute $Ax$.

$$p(\lambda)=(4-\lambda)(1-\lambda)$$

reason: Triangular determinant gives eigenvalues 4 and 1.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Eigenvalues and Eigenvectors

Continue; check the original map.

$$p(\lambda)=(4-\lambda)(1-\lambda)$$

reason: Triangular determinant gives eigenvalues 4 and 1.

$$v_4=(1,0)\T,\quad v_1=(-2,3)\T$$

reason: Solve each shifted null-space problem; do not assume the coordinate axes work.

$$x=2v_4+v_1,\quad Ax=8v_4+v_1=(6,3)\T$$

reason: Linearity combines the modes; direct multiplication agrees.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Independent transfer

Eigenvalues and Eigenvectors

$N=\begin{bmatrix}2&1\\0&2\end{bmatrix}$. Find its eigenvalues and eigenspace. Can $(0,1)\T$ be built from eigenvectors of $N$?

$$p(\lambda)=(2-\lambda)^2$$

reason: One eigenvalue, 2, counted twice.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Complete the reasoning

Eigenvalues and Eigenvectors

Continue; check the original map.

$$p(\lambda)=(2-\lambda)^2$$

reason: One eigenvalue, 2, counted twice.

$$(N-2I)v=0\iff v_2=0$$

reason: All eigenvectors lie on the horizontal axis.

$$E(2,N)=\operatorname{span}\{(1,0)\T\},\quad(0,1)\T\notin E(2,N)$$

reason: Repeated roots do not guarantee enough independent directions.

Course-authored example. Definitions: Axler, Linear Algebra Done Right, 4th ed., 2024; Strang, MIT 18.06.

Eigenvalues and Eigenvectors

A =

2 1
1 2

Two directions and two factors. Not yet: the hundredth state without a hundred products.

$$x_{k+1}=Ax_k\ \Longrightarrow\ x_k=A^{k}x_0$$

$x_k$: the state after k steps
build $A^{k}$: one at a time, $k-1$ products, about $n^{3}$ each
only $A^{k}x_0$: $k$ matrix–vector products, about $n^{2}$ each

Diagonalization and Modes

Diagonalizable

$$A\ \text{is diagonalizable over }\R\iff\R^{n}\ \text{has a basis of eigenvectors of }A$$

a basis of eigenvectors: n independent directions, each one only stretched by A
the consequence: in that basis the matrix of the operator is diagonal

Axler, S. Linear Algebra Done Right, 4th ed., 5.50 diagonalizable, p. 163; 5.55 conditions equivalent to diagonalizability, p. 165. Springer, 2024. Open access.

Diagonalization and Modes

$$AV=V\Lambda\ \Longrightarrow\ A=V\Lambda V^{-1}$$

V: the eigenvectors as columns, invertible because they are a basis
Λ: the eigenvalues on the diagonal, in the same order as those columns
step two: multiply both sides on the right by $V^{-1}$

Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning, ch. 2.7 eigendecomposition. MIT Press, 2016.

Diagonalization and Modes

Worked example

$$V\Lambda=\begin{bmatrix}3&1\\3&-1\end{bmatrix},\qquad(V\Lambda)V^{-1}=\tfrac12\begin{bmatrix}4&2\\2&4\end{bmatrix}=A$$

$V\Lambda$: column j of V multiplied by $\lambda_j$, which is 3 and 1
the half: $V^{-1}=\tfrac12 V$ for this V, so it rides to the end

Diagonalization and Modes

A =

2 1
1 2

Change of basis

$$\Lambda=V^{-1}AV=[A]_{\mathcal B}$$

$V^{-1}AV$: similarity, run with the eigenvectors as the new basis
$[A]_{\mathcal B}$: the matrix of the operator in the basis $\mathcal B$
$\Lambda$: one operator, three names; this $V$ is p60's basis matrix $B$

The dashed lines are the columns of V — the axes Λ is written in.

Diagonalization and Modes

$$A^{k}=V\Lambda^{k}V^{-1},\qquad\Lambda^{k}=\operatorname{diag}(\lambda_1^{k},\dots,\lambda_n^{k})$$

the middle: $V^{-1}V=I$ cancels $k-1$ times
$\Lambda^{k}$: raise each diagonal entry; nothing else moves
the cost: after the factorization, all dependence on $k$ is $n$ scalar powers

Strang, G. 18.06 Linear Algebra, Lecture 22, diagonalization and powers of A. MIT OpenCourseWare, 2010. CC BY-NC-SA 4.0.

Diagonalization and Modes

A =

2 1
1 2

Worked example

$$A^{k}=V\Lambda^{k}V^{-1}=\tfrac12\begin{bmatrix}3^{k}+1&3^{k}-1\\3^{k}-1&3^{k}+1\end{bmatrix}$$

$k=3$: $3^{3}=27$, so $(27+1)/2=14$ and $(27-1)/2=13$
check: square A, then multiply by A once more

Diagonalization and Modes

V =

1 1
1 -1

Modal coordinates

$$\begin{gathered}x_{k+1}=Ax_k,\qquad z_k=V^{-1}x_k\\[3pt]z_{k+1}=V^{-1}Ax_k=(V^{-1}AV)\,z_k=\Lambda z_k\ \Longrightarrow\ z_k=\Lambda^{k}z_0\end{gathered}$$

$z$: the state in eigen-coordinates, one number per mode
$\Lambda=\operatorname{diag}(3,1)$: no coupling: row i mentions only $z_i$

Strang, G. 18.06 Linear Algebra, Lecture 22, diagonalization and powers of A. MIT OpenCourseWare, 2010. CC BY-NC-SA 4.0.

Diagonalization and Modes

Λ =

3 0
0 1

Worked example

$$x_k=V\Lambda^{k}z_0=3\cdot3^{k}\begin{bmatrix}1\\1\end{bmatrix}+1\cdot1^{k}\begin{bmatrix}1\\-1\end{bmatrix}$$

$z_0=(3,1)$: $V^{-1}x_0$, the coordinates already found for $x_0=(4,2)$
$k=1,2$: $9(1,1)+(1,-1)=(10,8)$, $27(1,1)+(1,-1)=(28,26)$
every $k$: the entries differ by 2: the dots sit on one straight line

Diagonalization and Modes

$$x_k=\sum_j c_j\lambda_j^{k}v_j,\qquad \lambda_1^{-k}x_k\longrightarrow c_1v_1$$

$\lambda_1$: strictly largest in absolute value, and nonzero
$c_1\neq0$: $x_0$ must carry some $v_1$
residual: here $x_k-3\cdot3^{k}(1,1)=(1,-1)$ at every $k$
at $k=10$: $3\cdot3^{10}=177{,}147$ against 1

Diagonalization and Modes

$$\rho(A)=\max_j|\lambda_j|;\qquad\text{diagonalizable }A:\ A^{k}x_0\to0\ \ \forall x_0\iff\rho(A)<1$$

$\rho$: the largest of the $|\lambda_j|$, over every eigenvalue
$x_0$: any starting state
strict: $\rho=1$ is not enough

Worked example

$A_d$ has 0.7 on the diagonal and 0.2 off it: the same eigenvectors, and eigenvalues 0.9 and 0.5.

Strang, G. 18.06 Linear Algebra, Lecture 22, diagonalization and powers of A. MIT OpenCourseWare, 2010. CC BY-NC-SA 4.0.

Diagonalization and Modes

Sh =

1 1
0 1

Defective

$$(\text{Sh}-I)v=\begin{bmatrix}0&1\\0&0\end{bmatrix}v=0\ \Rightarrow\ v_2=0\ \Rightarrow\ v\in\operatorname{span}\{(1,0)\}$$

$\text{Sh}$: the shear in the corner: $x$ slides by $y$
$\lambda=1$ twice: from $(1-\lambda)^{2}$, yet one direction solves it
defective: short of a basis of eigenvectors, so not diagonalizable

One dashed invariant line, where two would have made a basis.

Axler, S. Linear Algebra Done Right, 4th ed., 5.55 conditions equivalent to diagonalizability, p. 165. Springer, 2024. Open access.

Diagonalization and Modes

$$n\ \text{distinct real eigenvalues}\Rightarrow A\ \text{is diagonalizable over }\R$$

real: the rotation example: distinct complex roots, no real basis
one way only: the identity repeats an eigenvalue, yet is diagonal

Over $\mathbb C$ every square matrix has a Jordan form: diagonal plus ones just above it.

Axler, S. Linear Algebra Done Right, 4th ed., 5.58, p. 166. Springer, 2024. Jordan form quoted without proof: Horn, R. & Johnson, C. Matrix Analysis, 2nd ed. Cambridge, 2012.

Diagonalization and Modes

$$x_{k+1}=(A-BK)x_k\to 0\ \ \forall x_0\iff\rho(A-BK)<1$$

$A,B$: the plant; this $B$ is the input matrix, not the basis matrix of p10–24
$K$: the gain you design; it moves the eigenvalues
$\rho<1$: strictly inside; a repeated root on the circle can still grow
next door: Lecture 7 does $\operatorname{Re}\lambda<0$; Lecture 8 pushes a covariance through $A$

Recap

The five questions, answered

Integrated whiteboard problem

Coordinates, Modes and Stability

$B=\begin{bmatrix}1&1\\1&-1\end{bmatrix}$, $D=\operatorname{diag}(1/2,1/4)$, $x_0=(3,1)\T$. Find $A$, $x_k$, determinant and stability.

$$A=BDB^{-1},\quad z_0=B^{-1}x_0=(2,1)\T$$

z: basis coordinates; $z_{k+1}=Dz_k$

$$x_k=2(1/2)^k(1,1)\T+(1/4)^k(1,-1)\T$$

gains: $\det A=1/8$, $\rho(A)=1/2$: all states decay

Your turn

Coordinates, Modes and Stability

$B=\begin{bmatrix}1&1\\0&1\end{bmatrix}$, $D=\operatorname{diag}(2,3)$. With $x_0=(2,1)\T$, find $A$, $x_k$, determinant and stability.

$$B^{-1}=\begin{bmatrix}1&-1\\0&1\end{bmatrix},\quad A=BDB^{-1}=\begin{bmatrix}2&1\\0&3\end{bmatrix}$$

inverse: use $B^{-1}$, not $B\T$

$$\det A=6,\quad\rho(A)=3>1$$

conclusion: not all states decay

$$x_k=(2^k+3^k,3^k)\T$$

check: $z_0=B^{-1}x_0=(1,1)\T$

Selected references

Courses and books

Axler, S. Linear Algebra Done Right, 4th ed. Springer, 2024. Open access. Quotes include 3.84, 5.5, 5.52, 9.50, 9.61.
Strang, G. 18.06 Linear Algebra, L19, 20, 21, 22, 28, 31. MIT OCW, 2010. CC BY-NC-SA 4.0.
Lynch and Park. Modern Robotics. Cambridge, 2017.
Grizzle, J. ROB 501, notes 11. Michigan, 2018. CC BY-NC 4.0.
Sanderson, G. Essence of Linear Algebra, ch. 6. 3Blue1Brown, 2016.

Selected references

Carried forward

Goodfellow, Bengio and Courville. Deep Learning, ch. 2.7, 2.8. MIT Press, 2016.
Horn and Johnson. Matrix Analysis, 2nd ed. Cambridge, 2012. Jordan form, slide 80.
Strang, G. Linear Algebra and Learning from Data. Wellesley–Cambridge, 2019. Lectures 3, 4.
ROB-GY 6013. Lecture 1, Introduction and Linear Algebra I. Null space and rank.
Notation and Conventions · Textbooks and Reading Guide.