Multivariate Analysis Review

Lee Smooth Manifolds

In these notes we prove three foundational theorems in multivariate analysis: the inverse function theorem, implicit function theorem, and change of variables theorem. For both vectors and matrices $|\cdot |$ refers to the supremum norm while $\Vert \cdot \Vert$ refers to the Euclidean norm, following Munkres’ convention. Note we abuse the term $C^r$ diffeomorphism a bit here to mean “a bijective $C^r$ map with a $C^r$ inverse,” ignoring additional topological conditions of connectivity and such.

Theorem 1: Let $A\subset {\mathbb{R}}^n$ be open and $f:A\to {\mathbb{R}}^n$ be an injective $C^r$ map ($r\ge 1$). If $Df(x)$ is non-singular for every $x\in A$, then $f(A)$ is open in ${\mathbb{R}}^n$ and $f$ is a $C^r$ diffeomorphism from $A$ to $f(A)$.

Proof) First, because $f$ is $C^1$ and $Df(x)$ is always non-singular we can locally lower-bound the length contraction that $f$ incurs. Take any $a\in A$ and let $E=Df(a)$. We claim that there is a open ball $U$ of $a$ such that for any $x,y\in U$, $|f(x)-f(y)|>\alpha |x-y|$ where $\alpha =1/(2n|E^{-1}|)$. $n|E^{-1}|$ seems to be the most unmotivated constant here; it appears because it’s the minimum “contraction factor” of the nonsingular matrix $E$; for any $x,y$, $|x-y|=|E^{-1}(E(x-y))|\le n|E^{-1}||E(x-y)|\Rightarrow |E(x-y)|\ge |x-y|/(n|E^{-1}|)$.

The proof is quite intuitive. Take $h(x)=f(x)-E(x-a)$; then, $Dh(a)=0$, and thus there is a open ball $U$ of $a$ such that $|Dh(u)|<1/(2n|E^{-1}|)$ for all $u\in U$. Given any $x,y\in U$, for a fixed coordinate $1\le i\le n$, there exists some $u\in U$ in the segment connecting $x$ and $y$ such that

$$|h_i(x)-h_i(y)|=|D{h}_i(u)\cdot (x-y)|<|x-y|/(2n|E^{-1}|)$$

Thus, $|h(x)-h(y)|<|x-y|/(2n|E^{-1}|)$ for any $x,y\in U$. For these same $x,y$, $f$ must satisfy

$$|f(x)-f(y)|\ge |E(x-y)|-|h(x)-h(y)|>|x-y|/(2n|E^{-1|})=\alpha |x-y|$$

Note we can easily find an upper bound too as $|f(x)-f(y)|\le |E(x-y)|+|h(x)-h(y)|$, though this isn’t useful for our purposes.

Next we show that $B=f(A)$ is open in ${\mathbb{R}}^n$. Take any $a\in A$; it suffices to show there is some $\delta$ such that $B(b,\delta )\subset f(A)$, where $b=f(a)$. We utilize the fact that if $\varphi$ is a differentiable function that maps an open subset of ${\mathbb{R}}^m$ to $\mathbb{R}$, if $\varphi$ has a local extremum at some point $x$, $D\varphi (x)=0$.

Take a rectangle $Q\subset A$ such that $a\in Int(Q)$; $f(Bd(Q))$ is a compact subset of ${\mathbb{R}}^n$ disjoint from $b$, and thus there is some $\delta$ such that $B(b,2\delta )\cap f(Bd(Q))=\varnothing$. We claim that $B(b,\delta )\subset f(A)$. To prove this, take any arbitrary point $c\in B(b,\delta )$ and the function $F:Q\to \mathbb{R}$ defined as $F(q)=\Vert f(q)-c\Vert$. As $Q$ is compact, there is some point $d$ for which $F$ achieves a global minimum. $d$ cannot be on $Bd(Q)$, as if $d\in Bd(Q)$, $F(d)>\delta >\Vert f(a)-c\Vert =F(a)$. Thus, $d\in Int(Q)$, meaning $DF(d)=0$. As $Df(d)$ is nonsingular, one can check this is only possible if $F(d)=0$, implying $c=f(d)\in f(A)$.

We now consider the inverse function $g$ of $f$, which is guaranteed to exist by injectivity. By the argument above, $f$ must also map any open subset of $A$ into an open subset of ${\mathbb{R}}^n$, and thus $f$ is an open map, implying $g$ is continuous. We show that for any $a\in A$ and $b=f(a)$, $g$ is differentiable at $b$ and $Dg(b)=Df(a{)}^{-1}$. That is, we show that if $E=Df(a)$,

$$\underset{h\to 0}{\lim }\frac{g(b+h)-g(b)-E^{-1}h}{|h|}=\vec{0}$$

We can confirm with simple computation that

$$\frac{g(b+h)-g(b)-E^{-1}h}{|h|}=-E^{-1}\frac{(f(g(b+h))-f(g(b)))-E(g(b+h)-g(b))}{|h|}$$

If we let $\Delta (h)=g(b+h)-g(b)$,

$$=-E^{-1}\frac{(f(a+\Delta (h))-f(a))-E(\Delta (h))}{|h|}=-E^{-1}\frac{(f(a+\Delta (h))-f(a))-E(\Delta (h))}{\Delta (h)}\frac{\Delta (h)}{|h|}$$

The middle term converges to the vector $\vec{0}$ as $E=Df(a)$ and $\Delta (h)\to 0$ as $h\to 0$ by continuity. The right term is bounded within some neighborhood of $b$ as, by our previous argument, there exists some $\alpha$ and neighborhood $U$ of $a$ such that for any $x,y\in U$, $|f(x)-f(y)|\ge \alpha |x-y|$. All together, the expression is forced to converge to $\vec{0}$.

It suffices to show $g$ is $C^r$; we do so by induction. We’ve already shown that $g$ is $C^1$, as its partial derivatives are algebraic expressions of $C^{r-1}$ functions. Note that we can decompose the function $Dg$ as follows:

$$Dg=i_{{\mathbb{R}}^{n\times n}}\circ Df\circ g$$

where $i$ is the matrix inversion operator. Assume we have shown $g$ is $C^m$ for some $m\le n-1$; the entries of $Dg$ then become $C^m$, implying $g$ is $C^{m+1}$. Induction completes the proof.

Corollary (The Inverse Function Theorem): Let $A\subset {\mathbb{R}}^n$ be open and $f:A\to {\mathbb{R}}^n$ be a $C^r$ map. If $a\in A$ is such that $Df(a)$ is nonsingular, there are neighborhoods $U$ of $a$ and $V$ of $f(a)$ such that $f$ is a $C^r$ diffeomorphism from $U$ to $V$.

Proof) There is a neighborhood $U_1$ of $a$ in which $Df(x)$ is nonsingular, and by arguments in Theorem 1 there is a neighborhood $U_2$ such that $|f(x)-f(y)|>\alpha |x-y|$ for all $x,y\in U_2$ where $\alpha$ is some fixed constant, implying injectivity. Taking $U=U_1\cap U_2$ and applying Theorem 1 completes the proof.

Note that modifying the above theorem to make $U$ and $V$ connected neighborhoods of $a$ and $f(a)$ is quite simple; take the connected components of the original $U$ and $V$ containing $a$ and $f(a)$, respectively. As ${\mathbb{R}}^n$ is locally path-connected, these components are open, and thus the resulting restriction is still $C^r$ with a $C^r$ inverse.

Theorem 2 (Implicit Function Theorem): Let $A$ be open in ${\mathbb{R}}^{n+k}$; let $f:A\to {\mathbb{R}}^n$ be of class $C^r$. Write $f$ in the form $f(x,y)$, for $x\in {\mathbb{R}}^k$ and $y\in {\mathbb{R}}^n$. Suppose that $(a,b)$ is a point of $A$ such that $f(a,b)=0$ and

$$\det \frac{\partial f}{\partial y}(a,b)\ne 0$$

Then there is a neighborhood $B$ of $a$ in ${\mathbb{R}}^k$ such that there exists a unique continuous function $g:B\to {\mathbb{R}}^n$ such that $g(a)=b$ and $f(x,g(x))=0$ for all $x\in B$. This function $g$ is of class $C^r$.

Proof) Consider the function $f:{\mathbb{R}}^{n+k}\to {\mathbb{R}}^{n+k}$ such that $(x,y)\mapsto (x,f(x,y))$. $F$ is clearly $C^r$ and $DF$ is nonsingular at $(a,b)$. By the inverse function theorem, there are connected neighborhoods $U$ of $(a,b)$ and $V$ of $F(a,b)=(a,0)$ such that $F{|}_U$ is a $C^r$ diffeomorphism from $U$ to $V$; let $H:V\to U$ be its $C^r$ inverse. Take a connected neighborhood $W$ of $a$ such that $W\times 0\subset V$, and define $h:W\to {\mathbb{R}}^n$ such that $H(w,0)=(w,h(w))$. Such an $h(w)$ satisfies $f(w,h(w))=0$ for all $w\in W$, $h(a)=b$, and is $C^r$; we have found our desired function.

It suffices to show this function $h$ is unique over the neighborhood $W$. FTSOC, assume there is some other continuous function $g:W\to {\mathbb{R}}^n$ such that $f(w,g(w))=0$ for all $w\in W$ and $g(a)=b$. Note that $g(w)\in U$ iff $g(w)=h(w)$ as $F$ is a bijection between $U$ and $W$. Thus, the set $\{w\in W|g(w)=h(w)\}$ is open as it is equal to $g^{-1}(U)$. Meanwhile, as $g$ and $h$ are both continuous the set $\{w\in W|g(w)\ne h(w)\}$ is also open in $W$. If $g$ and $h$ are not equivalent functions these sets form a separation of the connected set $W$, an immediate contradiction.

Theorem 3: Let $g:A\to B$ be a diffeomorphism of open sets in ${\mathbb{R}}^n$. Then for every continuous function $f:B\to \mathbb{R}$ that is integrable over $B$, the function $(f\circ g)|\det Dg|$ is integrable over $A$, and

$${\int }_{B}f={\int }_A(f\circ g)|\det Dg|$$

Note these integrals are all improper integrals.

Proof) We first reduce the problem to something more tractable. First, if we show the Theorem holds for diffeomorphisms $g:U\to V$ and $h:V\to W$, the theorem must hold for $h\circ g$.

Wow this proof is like… really… really long and quite annoying to write down. See Munkres 20 for the full proof; it really is a lot of machinery that isn’t all that difficult in actuality.