LECTURE 12 · FLOW MATCHING · DECEMBER 2
Flow Matching and VLA
From random vectors to robot actions
How can a learned rate turn noise into a possible action sequence?
Your turn
Flow Matching and VLA
Which clock is running?
$k$: physical robot step; $\tau\in[0,1]$: generator time. One action chunk is refined while the camera observation stays fixed.
- AEvery flow step moves the robot.
- BFlow steps refine a candidate chunk.✓
- CFlow time is the camera frame index.
Solution 1 / 1
Separate computation from execution
$k$: robot time; $\tau$: generation time; $A$: candidate action chunk.
- The generator changes the candidate A while holding the observation at robot step k fixed.
- The robot executes selected entries only after generation.
- B is correct. A confuses numerical refinement with execution; C identifies two different clocks.
Black et al., π₀ (2024/2026), §IV; App. A-D · original course examples
Problem
Probability Flows
The target is a distribution
$$X_0\sim p_0,\qquad X_1\sim p_{\rm data}$$
- where
- $X_0,X_1\in\mathbb R^d$: random vectors; $p_0$: easy noise law; $p_{\rm data}$: data law.
Lipman et al. (2023), §§2–4 · original course examples
Definition
Probability Flows
A learned velocity field
$$\frac{dx_\tau}{d\tau}=v_\theta(x_\tau,\tau),\quad x_0\sim p_0$$
- where
- $x_\tau,v_\theta\in\mathbb R^d$; $\theta$: fixed learned parameters during sampling.
Lipman et al. (2023), §§2–4 · original course examples
Definition
Probability Flows
The flow map collects trajectories
$$\phi_\tau(x_0)=x_\tau,\qquad \phi_0(x_0)=x_0$$
- where
- $\phi_\tau:\mathbb R^d\to\mathbb R^d$: map from an initial point to its state at time $\tau$.
Lipman et al. (2023), §§2–4 · original course examples
Worked example
Probability Flows
Worked example: translation
$$\begin{gathered}v(x,\tau)=2,\quad \phi_\tau(x_0)=x_0+2\tau \\[6pt] x_\tau-x_0=\int_0^\tau2\,ds=2\tau \\[6pt] \dot x_\tau=2,\quad\phi_0(x_0)=x_0;\quad X_\tau\sim\N(2\tau,1)\end{gathered}$$
- where
- $x_0,x_\tau\in\mathbb R$: positions; velocity $2$ is constant per unit generator time.
Lipman et al. (2023), §§2–4 · original course examples
Your turn
Probability Flows
Transfer: a different velocity
$X_0\sim\mathcal N(1,4)$ and $dx_\tau/d\tau=-3$. Find the law at $\tau=1$.
- AGaussian mean −2, variance 4✓
- BGaussian mean −2, variance 16
- CGaussian mean 4, variance 4
Solution 1 / 2
Integrate before transforming the law
$X_0$: initial random value; $X_1$: final value.
- $X_1=X_0+\int_0^1(-3)\,d\tau=X_0-3$.
- $\mathbb E[X_1]=1-3=-2$.
- $\operatorname{Var}(X_1)=\operatorname{Var}(X_0)=4$.
Solution 2 / 2
Choose A
$\mathcal N(\mu,\sigma^2)$ uses variance as its second argument.
- A follows because a constant shift leaves centered differences unchanged.
- B incorrectly scales the spread. C reverses the velocity sign.
Goodfellow et al. (2016), §§3.8–3.9.3 · original course examples
Definition
Probability Flows
Expansion changes density
$$\begin{gathered}x_\tau=e^{a\tau}x_0,\qquad p_\tau(x)=e^{-a\tau}p_0(e^{-a\tau}x) \\[6pt] \dot x=ax\ \Rightarrow\ x=e^{a\tau}x_0,\quad x_0=e^{-a\tau}x \\[6pt] p_\tau(x)\,dx=p_0(e^{-a\tau}x)e^{-a\tau}\,dx\end{gathered}$$
- where
- $a\in\mathbb R$: fixed rate; $p_\tau$: scalar density; $x$: evaluation point.
Lipman et al. (2023), §§2–4 · original course examples
Worked example
Probability Flows
Compression preserves total probability
$$X_\tau=e^{-\tau}X_0,\quad X_0\sim\mathcal N(0,1)$$
- where
- $X_\tau$: scalar random state; $\operatorname{Var}(X_\tau)=e^{-2\tau}$.
Lipman et al. (2023), §§2–4 · original course examples
Derivation
Probability Flows
Probability enters and leaves an interval
$$\frac{d}{d\tau}\int_a^bp_\tau(x)\,dx=j_\tau(a)-j_\tau(b)$$
- where
- $[a,b]$: fixed interval; $j_\tau(x)=p_\tau(x)v(x,\tau)$: probability flux.
Lipman et al. (2023), Appendix B · original course examples
Derivation
Probability Flows
Local probability conservation
$$\begin{gathered}\partial_\tau p_\tau(x)=-\partial_x[p_\tau(x)v(x,\tau)] \\[6pt] j(a)-j(b)=-\int_a^b\partial_xj(x)\,dx \\[6pt] \int_a^b[\partial_\tau p+\partial_x(pv)]\,dx=0 \\[6pt] \partial_\tau p=-\partial_x(pv),\qquad\partial_\tau p+\nabla\!\cdot(pv)=0\end{gathered}$$
- where
- $\partial_\tau$: time derivative at fixed $x$; $\partial_x$: spatial derivative.
Lipman et al. (2023), Appendix B · original course examples
Your turn
Probability Flows
Does constant velocity freeze density?
$p_\tau(x)=p_0(x-2\tau)$, $v=2$. At a fixed $x$, does density generally stay constant?
- AYes, because velocity is constant.
- BNo, because the density profile moves.✓
- CNo, because total probability grows.
Solution 1 / 2
Differentiate the translated profile
$p_0$ is differentiable; prime means derivative of its scalar argument.
- $\partial_\tau p_0(x-2\tau)=-2p_0^{\prime}(x-2\tau)$.
- $-\partial_x(2p_\tau)=-2p_0^{\prime}(x-2\tau)$.
- These sides match, so translation satisfies continuity.
Solution 2 / 2
Choose B
$x$ stays fixed while the profile passes it.
- B: the density at x can change even with constant velocity.
- A confuses constant velocity with constant density. C violates conserved total mass.
Lipman et al. (2023), Appendix B · original course examples
Worked example
Probability Flows
Integrated example: a moving Gaussian
$$\begin{gathered}X_\tau=2\tau+(1+\tau)X_0,\quad X_0\sim\N(0,1) \\[6pt] \dot x_\tau=2+x_0,\qquad x_0=\frac{x_\tau-2\tau}{1+\tau} \\[6pt] v(x,\tau)=2+\frac{x-2\tau}{1+\tau} \\[6pt] X_1=2+2X_0\sim\N(2,4)\end{gathered}$$
- where
- $\mu_\tau=2\tau$; $s_\tau=1+\tau>0$: standard deviation.
Lipman et al. (2023), §§2–4 · original course examples
Your turn
Probability Flows
Transfer: contracting toward a target mean
$X_\tau=3\tau+(1-\tau/2)X_0$, $X_0\sim\mathcal N(0,4)$. Find mean and variance at $\tau=1$.
- AMean 3, variance 1✓
- BMean 3, variance 2
- CMean 0, variance 1
Solution 1 / 2
Apply affine moment rules
$X_1=3+X_0/2$; initial variance is 4.
- $\mathbb E[X_1]=3+(1/2)0=3$.
- $\operatorname{Var}(X_1)=(1/2)^2(4)=1$.
- The endpoint scale 1/2 stays positive, so this affine map remains invertible.
Solution 2 / 2
Choose A
$\mathcal N(3,1)$ is the final Gaussian.
- A squares the scale when transforming variance.
- B multiplies variance by the scale only. C ignores translation.
Goodfellow et al. (2016), §§3.8–3.9.3 · original course examples
Problem
Conditional Paths
A training pair supplies both endpoints
$$(X_0,X_1)\sim\pi,\qquad \pi_0=p_0,\ \pi_1=p_{\rm data}$$
- where
- $\pi$: joint pairing law; $\pi_0,\pi_1$: its two marginals, not model names.
Tong et al. (2024), §§2.2–3.2 · original course examples
Definition
Conditional Paths
Straight interpolation defines a path
$$X_\tau=(1-\tau)X_0+\tau X_1,\quad 0\leq\tau\leq1$$
- where
- $X_0,X_1\in\mathbb R^d$: paired endpoints; $\tau$: scalar interpolation weight.
Tong et al. (2024), §§2.2–3.2 · original course examples
Derivation
Conditional Paths
The target velocity is an endpoint difference
$$\begin{gathered}U=\frac{dX_\tau}{d\tau}=-X_0+X_1 \\[6pt] \frac{d}{d\tau}[(1-\tau)x_0]=-x_0,\qquad\frac{d}{d\tau}[\tau x_1]=x_1 \\[6pt] u=x_1-x_0,\qquad\Delta x=\Delta\tau\,u\end{gathered}$$
- where
- $U\in\mathbb R^d$: target rate for a fixed pair; endpoints stay fixed in this derivative.
Tong et al. (2024), §§2.2–3.2 · original course examples
Worked example
Conditional Paths
Worked example: a 2D pair
$$\begin{gathered}x_0=\begin{bmatrix}-1\\2\end{bmatrix},\quad x_1=\begin{bmatrix}3\\0\end{bmatrix},\quad \tau=\tfrac14 \\[6pt] u=(3-(-1),\,0-2)^T=(4,-2)^T \\[6pt] x_{1/4}=(-1,2)^T+\tfrac14(4,-2)^T=(0,3/2)^T \\[6pt] x_{1/4}+\tfrac34u=(3,0)^T=x_1\end{gathered}$$
- where
- $x_0$: noise draw; $x_1$: data vector; $x_\tau$: training input; $u$: velocity target.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Conditional Paths
Transfer: another pair
$x_0=(2,-1)^T$, $x_1=(-2,3)^T$, $\tau=1/4$. Choose $(x_\tau,u)$.
- A((1, 0), (−4, 4))✓
- B((−1, 2), (−4, 4))
- C((1, 0), (−1, 1))
Solution 1 / 2
Compute the rate, then the state
$u=x_1-x_0$; $x_\tau=x_0+\tau u$.
- $u=(-2-2,3-(-1))^T=(-4,4)^T$.
- $x_{1/4}=(2,-1)^T+(1/4)(-4,4)^T=(1,0)^T$.
Solution 2 / 2
Choose A
$\tau u=(-1,1)^T$ is an increment, not the rate.
- A gives both quantities correctly.
- B evaluates at τ=3/4. C replaces the rate by a quarter-step increment.
Tong et al. (2024), §§2.2–3.2 · original course examples
Definition
Conditional Paths
Fixing data leaves a Gaussian noise cloud
$$X_\tau\mid X_1=y\sim\mathcal N(\tau y,(1-\tau)^2I)$$
- where
- $X_0\sim\mathcal N(0,I)$ independent of $X_1$; $y\in\mathbb R^d$ fixed; $\tau<1$.
Lipman et al. (2023), §§2–4 · original course examples
Your turn
Conditional Paths
Is the noise coefficient a variance?
$X_0\sim\mathcal N(0,1)$, fixed $y=4$, $X_\tau=(1-\tau)X_0+\tau y$. Use $\tau=1/2$.
- AMean 2, variance 1/2
- BMean 2, variance 1/4✓
- CMean 4, variance 1/4
Solution 1 / 2
Transform the standard Gaussian
$X_{1/2}=X_0/2+2$.
- $\mathbb E[X_{1/2}]=2$.
- $\operatorname{Var}(X_{1/2})=(1/2)^2\operatorname{Var}(X_0)=1/4$.
- The standard deviation is 1/2.
Solution 2 / 2
Choose B
$y=4$ is fixed; it supplies no conditional variance.
- B keeps the standard deviation and variance distinct.
- A uses the noise coefficient as variance. C uses the final rather than intermediate mean.
Lipman et al. (2023), §§2–4 · original course examples
Your turn
Conditional Paths
Can we use the endpoint during sampling?
Training stores $(x_0,x_1,\tau)$ and constructs $x_\tau$. A new generation call has only noise and an observation.
- AGive the network the unknown x₁.
- BUse x₁−x₀ directly at every step.
- CPredict velocity from xτ, τ and observation.✓
Solution 1 / 1
Separate supervision from inputs
$x_1$: training target; $v_\theta$: learned velocity.
- During training x₁ constructs the target u=x₁−x₀.
- At generation time x₁ is unknown, so u cannot be supplied directly.
- C is executable. A and B assume the answer is already known.
Tong et al. (2024), §§2.2–3.2 · original course examples
Derivation
Conditional Paths
A conditional field can eliminate the noise
$$\begin{gathered}u(x,\tau\mid y)=\frac{y-x}{1-\tau},\quad \tau<1 \\[6pt] x_0=\frac{x-\tau y}{1-\tau} \\[6pt] y-x_0=\frac{(1-\tau)y-x+\tau y}{1-\tau}=\frac{y-x}{1-\tau}\end{gathered}$$
- where
- $x=(1-\tau)x_0+\tau y$; $u=y-x_0$; fixed endpoint $y$.
Lipman et al. (2023), §§2–4 · original course examples
Problem
Conditional Paths
One input can have several target velocities
$$U=X_1-X_0,\qquad (X_\tau,\tau)=(x,\tau)$$
- where
- $U$: random target under endpoint pairing; $x$: shared current location.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Conditional Paths
Which average does the field need?
At a fixed $(x,\tau)$, velocity targets are $+2$ with probability $3/4$ and $-2$ with probability $1/4$.
- AThe unconditional endpoint average.
- BThe target mean given this x and τ.✓
- CAlways the most frequent target exactly.
Solution 1 / 1
Use the input-conditioned target law
$U\mid x,\tau$ is the stated two-valued random target.
- $\mathbb E[U\mid x,\tau]=(3/4)2+(1/4)(-2)=1$.
- B selects the relevant conditional distribution.
- A discards the input condition. C gives +2, which need not minimize squared error.
Tong et al. (2024), §§2.2–3.2 · original course examples
Definition
Learning the Velocity
Squared error compares rates
$$\mathcal L(\theta)=\mathbb E\|v_\theta(X_\tau,\tau)-U\|^2$$
- where
- $U=X_1-X_0$; expectation over endpoint pairs and sampled time; $\theta$: trainable weights.
Tong et al. (2024), §§2.2–3.2 · original course examples
Derivation
Learning the Velocity
Derivation: the best scalar prediction
$$\begin{gathered}R(a)=\E[(a-U)^2\mid x,\tau] \\[6pt] R(a)=a^2-2a\E[U\mid x,\tau]+\E[U^2\mid x,\tau] \\[6pt] R^{\prime}(a)=2a-2\E[U\mid x,\tau] \\[6pt] a^*=\E[U\mid x,\tau],\qquad R^{\prime\prime}(a)=2>0\end{gathered}$$
- where
- $a\in\mathbb R$: candidate prediction; conditional target has a finite second moment.
Tong et al. (2024), §§2.2–3.2 · original course examples
Result
Learning the Velocity
The vector result is coordinatewise
$$\begin{gathered}v^*(x,\tau)=\E[U\mid X_\tau=x,\tau] \\[6pt] \|a-U\|^2=\sum_{j=1}^d(a_j-U_j)^2 \\[6pt] \nabla_aR=2(a-\E[U\mid x,\tau])=0\end{gathered}$$
- where
- $v^*,U\in\mathbb R^d$; squared norm sums all coordinate risks.
Tong et al. (2024), §§2.2–3.2 · original course examples
Worked example
Learning the Velocity
Worked example: conflicting targets
$$\begin{gathered}R(a)=\tfrac34(a-2)^2+\tfrac14(a+2)^2 \\[6pt] R(a)=\tfrac34(a^2-4a+4)+\tfrac14(a^2+4a+4) \\[6pt] R(a)=a^2-2a+4=(a-1)^2+3 \\[6pt] R(1)=3<4=R(2)\end{gathered}$$
- where
- $a$: predicted scalar rate; targets $2,-2$ have conditional weights $3/4,1/4$.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Learning the Velocity
Transfer: changed conditional weights
$P(U=-1\mid x,\tau)=1/4$, $P(U=3\mid x,\tau)=3/4$. Minimize conditional squared error.
- APredict 1
- BPredict 2✓
- CPredict 3
Solution 1 / 2
Compute the conditional mean
$U$: scalar rate target; $a$: prediction.
- $a^*=(1/4)(-1)+(3/4)3=2$.
- $R(a)=(1/4)(a+1)^2+(3/4)(a-3)^2$.
- $R(a)=a^2-4a+7=(a-2)^2+3$.
Solution 2 / 2
Choose B
$R(2)=3$, while $R(1)=R(3)=4$.
- B minimizes the risk.
- A uses equal rather than stated weights. C selects the more frequent label rather than the mean.
Tong et al. (2024), §§2.2–3.2 · original course examples
Derivation
Learning the Velocity
Conditional supervision learns the marginal field
$$\begin{gathered}\E\|v-U\|^2=\E\|v-m\|^2+\E\|U-m\|^2 \\[6pt] v-U=(v-m)+(m-U) \\[6pt] \E[(v-m)^T(m-U)\mid X_\tau,\tau]=(v-m)^T\,0=0 \\[6pt] \mathcal L_{\rm CFM}=\mathcal L_{\rm FM}+C,\qquad\nabla_\theta C=0\end{gathered}$$
- where
- $m=\E[U\mid X_\tau,\tau]$; $v=v_\theta(X_\tau,\tau)$; finite second moments.
Tong et al. (2024), §§2.2–3.2 · original course examples
Algorithm
Learning the Velocity
Training is ordinary supervised regression
Sample endpoints and a generation time.
Construct the state and velocity target.
Predict velocity, compute squared error, backpropagate.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Learning the Velocity
Shapes and loss reduction
$B=2$ samples, $d=3$ coordinates. $V,U\in\mathbb R^{2\times3}$. Use $L=\frac1{Bd}\sum_{i,j}(V_{ij}-U_{ij})^2$.
- AGradient is 2(V−U)
- BGradient is (V−U)/3✓
- CGradient has shape 3×2
Solution 1 / 1
Differentiate the stated mean
$V$: predictions; $U$: fixed labels; $B d=6$.
- $\partial L/\partial V_{ij}=2(V_{ij}-U_{ij})/6$.
- $\nabla_VL=(V-U)/3\in\mathbb R^{2\times3}$.
- B is correct. A omits the reduction; C transposes the gradient shape.
Goodfellow et al. (2016), §§3.8–3.9.3 · original course examples
Worked example
Learning the Velocity
Worked example: a scalar linear field
$$\begin{gathered}v_w(x,\tau)=wx,\quad (x,u)=(1,2),(2,1) \\[6pt] L^{\prime}(w)=(w-2)+2(2w-1)=5w-4 \\[6pt] w_0=0,\quad\eta=.1,\quad w_1=0-.1(-4)=.4 \\[6pt] L(0)=2.5,\qquad L(.4)=1.3\end{gathered}$$
- where
- $w\in\mathbb R$: trainable weight; $L(w)=\frac12[(w-2)^2+(2w-1)^2]$.
Goodfellow et al. (2016), §§3.8–3.9.3 · original course examples
Your turn
Learning the Velocity
Transfer: a negative input
$v_w(x)=wx$, one pair $(x,u)=(-2,3)$, $L=\frac12(wx-u)^2$, $w=1$, $\eta=.1$.
- ANew w is 0✓
- BNew w is 2
- CNew w is 1.5
Solution 1 / 2
Differentiate before updating
$e=wx-u$: residual; $w$: weight; $x,u$: fixed input and target.
- $e=1(-2)-3=-5$.
- $dL/dw=e x=(-5)(-2)=10$.
- $w_{new}=1-.1(10)=0$.
Solution 2 / 2
Choose A
Old loss $25/2$; new loss $9/2$.
- A follows gradient descent and decreases this example loss.
- B uses the wrong update sign. C omits the input factor in the chain rule.
Goodfellow et al. (2016), §§3.8–3.9.3 · original course examples
Synthesis
Learning the Velocity
The training chain is complete
Paired endpoints define targets.
Conditional least squares defines one field.
A new random start still needs integration.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Sampling the Learned Flow
What remains fixed while sampling?
$dx_\tau/d\tau=v_\theta(x_\tau,\tau,c)$; draw $x_0$ once. The condition $c$ is one observation.
- AOptimize θ at every flow step.
- BHold θ and c fixed; update x and τ.✓
- CReplace x with fresh noise every step.
Solution 1 / 1
Read the differential equation
$\theta$: learned parameters; $c$: condition; $x_\tau$: generated state.
- The ODE evolves x with generation time τ under one fixed field.
- B follows the stated ODE. A adds test-time training; C defines a different stochastic process.
- The initial draw supplies randomness even with deterministic integration.
Tong et al. (2024), §§2.2–3.2 · original course examples
Recall
Sampling the Learned Flow
Euler multiplies velocity by a step size
$$x_{j+1}=x_j+h\,v_\theta(x_j,\tau_j,c),\quad \tau_{j+1}=\tau_j+h$$
- where
- $j$: numerical step index; $h>0$: step size; $τ_0=0$, $Nh=1$.
Lipman et al. (2023), §§2–4 · original course examples
Worked example
Sampling the Learned Flow
Worked example: two Euler steps
$$\begin{gathered}v(x,\tau)=x,\quad x_0=1,\quad h=\tfrac12 \\[6pt] x_1=1+\tfrac12(1)=1.5 \\[6pt] x_2=1.5+\tfrac12(1.5)=2.25 \\[6pt] x(1)=e\approx2.71828\end{gathered}$$
- where
- $x_j$: numerical state; exact solution $x(\tau)=e^\tau$.
Lipman et al. (2023), §§2–4 · original course examples
Your turn
Sampling the Learned Flow
Transfer: a time-dependent rate
$v(x,\tau)=2\tau$, $x_0=1$, $h=1/2$. Use two forward Euler steps from $\tau=0$ to $1$.
- A1.5✓
- B2
- C2.5
Solution 1 / 2
Evaluate at the beginning of each step
$x_j$: Euler state; $\tau_j=j/2$.
- $v(x_0,0)=0$, so $x_1=1+(1/2)0=1$.
- $v(x_1,1/2)=1$, so $x_2=1+(1/2)1=1.5$.
Solution 2 / 2
Choose A and compare the exact integral
$x(1)=1+\int_0^1 2\tau\,d\tau=2$.
- A is the forward Euler approximation.
- B is exact here, not two-step forward Euler. C uses right-end rates.
Lipman et al. (2023), §§2–4 · original course examples
Derivation
Sampling the Learned Flow
A midpoint estimate improves the local rate
$$\begin{gathered}k_1=v(x_j,\tau_j),\quad k_2=v(x_j+\tfrac h2k_1,\tau_j+\tfrac h2) \\[6pt] k_1=v(x_j,\tau_j),\qquad x_{\rm trial}=x_j+\tfrac h2 k_1 \\[6pt] k_2=v(x_{\rm trial},\tau_j+\tfrac h2) \\[6pt] x_{j+1}=x_j+hk_2\end{gathered}$$
- where
- $k_1,k_2$: rates; $x_j+(h/2)k_1$: trial midpoint state; condition $c$ held fixed.
Lipman et al. (2023), §§2–4 · original course examples
Worked example
Sampling the Learned Flow
Worked example: midpoint on the same field
$$\begin{gathered}v(x,\tau)=x,\quad x_0=1,\quad h=\tfrac12 \\[6pt] k_1=1,\quad x_{\rm trial}=1+.25(1)=1.25,\quad k_2=1.25 \\[6pt] x_1=1+.5(1.25)=1.625 \\[6pt] x_2=(1+h+h^2/2)^2=1.625^2=2.640625\end{gathered}$$
- where
- $k_1,k_2$: rates; each accepted step uses $x_{j+1}=x_j+hk_2$.
Lipman et al. (2023), §§2–4 · original course examples
Your turn
Sampling the Learned Flow
Transfer: a fixed evaluation budget
Budget: 8 calls to $v_\theta$. Euler uses 1 call/step; explicit midpoint uses 2. Both integrate $[0,1]$.
- AEuler h=1/8; midpoint h=1/4✓
- BBoth h=1/8
- CMidpoint always gives the smaller error
Solution 1 / 1
Count steps from calls per step
$N$: accepted steps; $h=1/N$.
- Euler: $N=8/1=8$, so $h=1/8$.
- Midpoint: $N=8/2=4$, so $h=1/4$.
- A uses the full budget. B needs 16 midpoint calls; C needs assumptions and an error comparison.
Lipman et al. (2023), §§2–4 · original course examples
Diagnostic
Sampling the Learned Flow
Different errors need different repairs
Field error: the learned velocity differs from the target.
Solver error: finite steps approximate that learned ODE.
Execution error: robot dynamics differ from assumptions.
Lipman et al. (2023), §§2–4 · original course examples
Interpretation
Sampling the Learned Flow
Random starts preserve sample diversity
$$X_0\sim p_0,\qquad X_1=\phi_1(X_0;c)$$
- where
- $c$: fixed observation; $\phi_1$: deterministic learned flow map for that condition.
Lipman et al. (2023), §§2–4 · original course examples
Your turn
Sampling the Learned Flow
Reversing the generation clock
$s=1-\tau$; $y_s=x_{1-s}$; $dx_\tau/d\tau=v(x_\tau,\tau)$. Find $dy_s/ds$.
- Av(yₛ,1−s)
- B−v(yₛ,1−s)✓
- C−v(yₛ,s) always
Solution 1 / 1
Apply the chain rule
$s$: reversed clock; $y_s=x_{1-s}$.
- $dy_s/ds=(dx_\tau/d\tau)(d\tau/ds)$.
- $d\tau/ds=-1$, hence $dy_s/ds=-v(y_s,1-s)$.
- B changes both the sign and time argument. A misses the sign; C generally uses the wrong time.
openpi: Pi0.compute_loss / sample_actions · original course examples
Definition
Pairing and Optimal Transport
A coupling changes pairs, not endpoint laws
$$\pi(X_0\in A)=p_0(A),\quad \pi(X_1\in B)=p_1(B)$$
- where
- $\pi$: joint endpoint law; $A,B$: measurable sets; $p_0,p_1$: fixed marginals.
Tong et al. (2024), §§2.2–3.2 · original course examples
Worked example
Pairing and Optimal Transport
Worked example: two pairings
$$\begin{gathered}x_0\in\{-1,1\},\quad x_1\in\{-2,2\} \\[6pt] C_{\rm aligned}=\tfrac12[(-2+1)^2+(2-1)^2]=1 \\[6pt] C_{\rm crossed}=\tfrac12[(2+1)^2+(-2-1)^2]=9\end{gathered}$$
- where
- Each endpoint has mass $1/2$; pair cost is squared displacement $(x_1-x_0)^2$.
Tong et al. (2024), §§2.2–3.2 · original course examples
Definition
Pairing and Optimal Transport
Optimal transport chooses the coupling
$$\pi^*\in\arg\min_{\pi\in\Pi(p_0,p_1)}\mathbb E_\pi\|X_1-X_0\|^2$$
- where
- $\Pi(p_0,p_1)$: couplings with the required marginals; cost uses Euclidean coordinates.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Pairing and Optimal Transport
Transfer: compute the assignment cost
Sources $0,2$; targets $1,4$; each has mass $1/2$. Compare aligned $(0→1,2→4)$ and crossed pairs.
- AAligned cost 2.5; crossed 8.5✓
- BAligned cost 5; crossed 17
- CBoth cost 2.5
Solution 1 / 2
Square each displacement, then average
$C$ is expected squared cost, not a sum.
- $C_{aligned}=[(1-0)^2+(4-2)^2]/2=(1+4)/2=2.5$.
- $C_{crossed}=[(4-0)^2+(1-2)^2]/2=(16+1)/2=8.5$.
Solution 2 / 2
Choose A
Both assignments preserve one-half mass at each endpoint.
- A includes the probability weights.
- B reports sums, not expected costs. C assumes fixed marginals determine the joint pairing.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Pairing and Optimal Transport
Are learned trajectories necessarily straight?
Training paths are straight. The learned field is $v^*(x,\tau)=\mathbb E[U\mid x,\tau]$.
- AYes, every generated path is one training segment.
- BNo; the conditional average can vary along a path.✓
- CEvery learned path must cross another path.
Solution 1 / 1
Track the field along the generated state
$U$: paired velocity label; $v^*$: conditional mean field.
- As x and τ change, the compatible hidden endpoint pairs can change.
- Their conditional average can therefore change direction along a generated trajectory.
- B is correct. A confuses paired paths with the learned flow; C contradicts uniqueness for a regular ODE.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Pairing and Optimal Transport
What does a conditional path establish?
For each data point y, Gaussian noise contracts toward y along a simple conditional path. The data point may be sampled independently.
- AThis alone proves globally optimal endpoint pairing.
- BA simple conditional path and global OT are different claims.✓
- CEvery coupling has equal expected displacement cost.
Solution 1 / 1
Identify what was optimized
$y$: fixed endpoint in a conditional path; $\pi$: joint endpoint pairing.
- A simple path conditional on y does not choose the best joint pairing π.
- Global quadratic OT additionally minimizes expected cost over couplings with fixed marginals.
- B is correct. A skips that optimization; C fails in the 1-versus-9 cost example.
Tong et al. (2024), §§2.2–3.2 · original course examples
Your turn
Pairing and Optimal Transport
Re-pair examples with different conditions
Condition c says “move left”. Its data actions are negative. A batch also contains “move right” examples.
- ARe-pair all targets freely and retain old c.
- BKeep targets compatible with each condition.✓
- CRemove c from training and still claim conditioning.
Solution 1 / 1
Preserve the conditional target law
$p_{data}(x_1\mid c)$: action distribution for a fixed condition.
- The conditional generator must learn endpoint actions paired with their observation c.
- B retains the intended conditional law.
- A can relabel a right action as left. C trains an unconditional model instead.
Tong et al. (2024), §§2.2–3.2 · original course examples
Definition
The Bridge to Diffusion
A Gaussian path can use different schedules
$$X_\tau=\alpha(\tau)Y+\sigma(\tau)\varepsilon$$
- where
- $Y$: data vector; $\varepsilon\sim\mathcal N(0,I)$ independent; $\alpha,\sigma$: differentiable scalars.
Lipman et al. (2023), §4 and Appendix D · original course examples
Derivation
The Bridge to Diffusion
The score recovers average noise
$$\begin{gathered}s_\tau(x)=\nabla_x\log p_\tau(x) \\[6pt] \nabla_x\log p_\tau(x\mid y)=-\frac{x-\alpha y}{\sigma^2}=-\frac\varepsilon\sigma \\[6pt] s_\tau(x)=\E[\nabla_x\log p_\tau(x\mid Y)\mid x] \\[6pt] \E[\varepsilon\mid x]=-\sigma s_\tau(x)\end{gathered}$$
- where
- $s_\tau$: score; $\sigma>0$; $Y$: data, $\varepsilon$: noise. Interchange justified.
Lipman et al. (2023), §4 and Appendix D · original course examples
Derivation
The Bridge to Diffusion
Score to velocity
$$\begin{gathered}X_\tau=\alpha Y+\sigma\varepsilon \\[6pt] v=\dot\alpha m+\dot\sigma n \\[6pt] m=(x-\sigma n)/\alpha,\quad n=-\sigma s \\[6pt] v=(\dot\alpha/\alpha)x+(\dot\alpha\sigma^2/\alpha-\sigma\dot\sigma)s\end{gathered}$$
- where
- $Y$: data; independent $\varepsilon\sim\N(0,I)$; $m=\E[Y|x]$, $n=\E[\varepsilon|x]$; $s$: score; $\alpha\ne0$, $\sigma>0$. Dots: τ-rates.
Lipman et al. (2023), §4 and Appendix D · original course examples
Derivation
The Bridge to Diffusion
The straight-path bridge
$$\begin{gathered}v(x,\tau)=\frac{x}{\tau}+\frac{1-\tau}{\tau}s_\tau(x),\quad 0<\tau<1 \\[6pt] \alpha=\tau,\quad\dot\alpha=1,\quad\sigma=1-\tau,\quad\dot\sigma=-1 \\[6pt] \frac{(1-\tau)^2}\tau+(1-\tau)=(1-\tau)\!\left[\frac{1-\tau}\tau+1\right]=\frac{1-\tau}\tau\end{gathered}$$
- where
- $\alpha=\tau$, $\dot\alpha=1$, $\sigma=1-\tau$, $\dot\sigma=-1$.
Lipman et al. (2023), §4 and Appendix D · original course examples
Your turn
The Bridge to Diffusion
Worked bridge: a known Gaussian path
Fixed data $Y=2$, $\tau=1/2$, $x=0$. Then $p_\tau=\mathcal N(1,1/4)$. Find score and velocity.
- As=4, v=4✓
- Bs=2, v=2
- Cs=−4, v=−4
Solution 1 / 2
Differentiate the Gaussian log density
$\mu=1$, $\sigma^2=1/4$; $s(x)=-(x-\mu)/\sigma^2$.
- $s(0)=-(0-1)/(1/4)=4$.
- $v(0,1/2)=0/(1/2)+[(1/2)/(1/2)]4=4$.
Solution 2 / 2
Check by the endpoint-conditioned field
$y=2$, $x=0$, $\tau=1/2$.
- $u=(y-x)/(1-\tau)=2/(1/2)=4$.
- A agrees with both derivations. B uses standard deviation as variance; C reverses the Gaussian score sign.
Lipman et al. (2023), §4 and Appendix D · original course examples
Your turn
The Bridge to Diffusion
Transfer: reverse the clock consistently
At course time $\tau=1/2$, velocity is $+4$. Code time is $s=1-\tau$ and a sampling step has $\Delta s=-.1$.
- ACode velocity −4, state increment +.4✓
- BCode velocity +4, state increment −.4
- CCode velocity −4, state increment −.4
Solution 1 / 1
Transform rate and increment together
$w(x,s)=-v(x,1-s)$; $\Delta s=-.1$.
- $w=-4$.
- $\Delta x\approx\Delta s\,w=(-.1)(-4)=+.4$.
- A preserves the same forward generation direction. B misses the field sign; C multiplies signs incorrectly.
openpi: Pi0.compute_loss / sample_actions · original course examples
Definition
Vision-Language-Action Models
An action chunk is one generated vector
$$A_k=\operatorname{vec}(a_k,\ldots,a_{k+H-1})\in\mathbb R^{Hd_a}$$
- where
- $k$: robot step; $H$: chunk length; $a_k\in\mathbb R^{d_a}$: one action.
Black et al., π₀ (2024/2026), §IV; App. A-D · original course examples
Definition
Vision-Language-Action Models
The condition describes the task now
$$c_k=(I_k^{1:C},\ell_k,q_k),\qquad v_\theta(A^\tau_k,\tau,c_k)$$
- where
- $I^{1:C}$: camera images; $\ell$: instruction; $q$: robot state; $A^\tau$: noisy candidate.
Black et al., π₀ (2024/2026), §IV; App. A-D · original course examples
Model instance
Vision-Language-Action Models
π₀ adds a continuous action expert
VLM representations supply image-language context.
A smaller action expert processes noisy actions.
Attention connects context to action refinement.
Black et al., π₀ (2024/2026), §IV; App. A-D · original course examples
Convention bridge
Vision-Language-Action Models
Official code uses the opposite clock
$$\begin{gathered}Z_s=(1-s)A+s\varepsilon,\quad U_s=\varepsilon-A,\quad s:1\to0 \\[6pt] Z_s=X_{1-s},\qquad\frac{dZ_s}{ds}=-\frac{dX_\tau}{d\tau} \\[6pt] \Delta s=-h,\quad U_s=-U_\tau\quad\Rightarrow\quad\Delta s\,U_s=hU_\tau\end{gathered}$$
- where
- $A$: data action chunk; $\varepsilon$: noise; $s=1-\tau$; step $\Delta s<0$.
openpi: Pi0.compute_loss / sample_actions · original course examples
Your turn
Vision-Language-Action Models
A code step toward the data
Action target $A=2$, noise $\varepsilon=-1$. Code clock starts $s=1$, $z=-1$; use exact target rate and $\Delta s=-1/4$.
- Au=−3, new z=−.25✓
- Bu=+3, new z=−1.75
- Cu=−3, new z=−1.75
Solution 1 / 2
Follow the code convention
$u=\varepsilon-A$; $z_{new}=z+\Delta s\,u$.
- $u=-1-2=-3$.
- $z_{new}=-1+(-1/4)(-3)=-1+3/4=-1/4$.
Solution 2 / 2
Check the interpolation at the new time
$s_{new}=3/4$; $Z_s=(1-s)A+s\varepsilon$.
- $Z_{3/4}=(1/4)2+(3/4)(-1)=-1/4$.
- A agrees. B reverses the target; C reverses the increment sign.
openpi: Pi0.compute_loss / sample_actions · original course examples
Your turn
Vision-Language-Action Models
Generation, execution, and replanning
Generate $H=6$ actions. Execute the first $r=2$, observe again, then generate a new chunk.
- ASix internal steps imply six robot actions.
- BPhysical time advances by two executed actions.✓
- CThe remaining four must always be executed.
Solution 1 / 1
Follow the physical loop
$H$: generated horizon; $r$: executed prefix length; $1\leq r\leq H$.
- The internal solver first produces a six-action candidate.
- Executing two entries advances the robot by two action periods.
- B is correct. A confuses clocks. C rules out the stated replanning procedure.
Black et al., π₀ (2024/2026), §IV; App. A-D · original course examples
Model instance
Vision-Language-Action Models
π₀.₅ connects semantic and action prediction
A high-level prediction identifies a subtask.
The action expert conditions on that subtask.
Training combines robot and non-robot sources.
Physical Intelligence, π₀.₅ (2025), §§IV-A–D · original course examples
Objective
Vision-Language-Action Models
Discrete and continuous losses
$$\begin{gathered}\mathcal L=\mathcal L_{\rm token}+\lambda\mathcal L_{\rm flow} \\[6pt] \mathcal L_{\rm token}=-\log p_\theta(y\mid\text{context}) \\[6pt] \mathcal L_{\rm flow}=\E\|v_\theta(Z_s,s,c)-U_s\|^2\end{gathered}$$
- where
- $\mathcal L_{token}$: next-token cross-entropy; $\mathcal L_{flow}$: velocity regression; $\lambda>0$: weight.
Physical Intelligence, π₀.₅ (2025), §§IV-A–D · original course examples
Your turn
Vision-Language-Action Models
What follows from a richer training mixture?
A VLA receives additional image-language and robot examples. Its low-level action model still uses flow matching.
- AThe ODE becomes physical robot dynamics.
- BRepresentation learning changes; the flow objective can remain.✓
- CGeneralization is guaranteed for every new home.
Solution 1 / 1
Separate the mathematical objective from evidence
VLA: vision-language-action model; ODE: numerical action-generation flow.
- Additional data changes the learned representation and parameters.
- It does not redefine generation time as physical time or prove every new task succeeds.
- B follows. A confuses the two systems; C overstates empirical evidence.
Physical Intelligence, π₀.₅ (2025), §§IV-A–D · original course examples
Worked example
From Data to an Executed Action
Integrated example: training a chunk
$$\begin{gathered}A=(2,0)^T,\quad \varepsilon=(-2,2)^T,\quad \tau=\tfrac14 \\[6pt] A^{1/4}=\tfrac34(-2,2)^T+\tfrac14(2,0)^T=(-1,1.5)^T \\[6pt] U=(4,-2)^T,\quad V=(3,-1)^T,\quad e=V-U=(-1,1)^T \\[6pt] L=\tfrac12(e_1^2+e_2^2)=1,\qquad\nabla_VL=e\end{gathered}$$
- where
- $H=2$, $d_a=1$; fixed condition $c$; $A^\tau=(1-\tau)\varepsilon+\tau A$.
Tong et al. (2024), §§2.2–3.2 · original course examples
Worked example
From Data to an Executed Action
Generate a chunk; execute a prefix
$$\begin{gathered}\varepsilon=(-2,2)^T,\quad v=(4,-2)^T,\quad h=\tfrac14 \\[6pt] x_j=(-2,2)^T+\tfrac j4(4,-2)^T \\[6pt] x_1=(-1,1.5)^T,\quad x_2=(0,1)^T,\quad x_3=(1,.5)^T,\quad x_4=(2,0)^T \\[6pt] H=2,\quad N=4,\quad H_{\rm exec}=1:\quad a_k=2\end{gathered}$$
- where
- Toy field $v$; $N=4$ solver steps; $H=2$ actions; execute first action.
Black et al., π₀ (2024/2026), §IV; App. A-D · original course examples
Your turn
From Data to an Executed Action
Transfer: the observation changes mid-execution
After one executed action, a new image shows an obstacle. Old condition c described a clear route.
- AMore ODE steps under old c incorporate the new image.
- BUpdate the condition and replan under the policy.✓
- CLow training MSE guarantees the old chunk is safe.
Solution 1 / 1
Identify the changed input
$c_{old}$ and $c_{new}$ represent different observations.
- Continuing integration under c_old keeps using the old information.
- B passes the new condition into a new policy computation.
- A confuses numerical accuracy with updated observations; C confuses regression fit with a safety guarantee.
Black et al., π₀ (2024/2026), §IV; App. A-D · original course examples
Synthesis
From Data to an Executed Action
The course ingredients meet in one policy
Linear algebra represents chunks and attention.
Calculus trains a velocity predictor.
Probability defines the target law.
ODE integration generates a candidate action sequence.
Tong et al. (2024), §§2.2–3.2 · original course examples