Finding Least Squares Solution Review and Others

Rotation Matrix

The rotation matrix
$\displaystyle R=\begin{pmatrix} \cos\theta & -\sin\theta\\ \sin\theta & \cos\theta \end{pmatrix}$
rotates points in the $xy$ -plane counterclockwise through an angle $\theta$ about the origin.

For example rotating the vector $(1,0)$ 45 degrees counterclockwise gives us:
$\displaystyle \begin{pmatrix} \cos 45^\circ & -\sin 45^\circ\\ \sin 45^\circ & \cos 45^\circ \end{pmatrix} \begin{pmatrix} 1\\ 0 \end{pmatrix} = \begin{pmatrix} \frac{\sqrt{2}}{2}\\ \frac{\sqrt{2}}{2} \end{pmatrix}.$

Finding Least Squares Solution

Given $Ax=b$ (inconsistent system), solve
$\displaystyle A^TAx=A^Tb$ instead to get a least squares solution of the original equation.

Projection

If we know a least squares solution $\mathbf{u}$ of $A\mathbf{x}=\mathbf{b}$ , we can find the projection $\mathbf{p}$ of $\mathbf{b}$ onto the column space of $A$ by $\displaystyle \mathbf{p}=A\mathbf{u}.$

Dimension Theorem for Matrices (Also known as Rank-Nullity Theorem)

If $A$ is a matrix with $n$ columns, then $\displaystyle rank(A)+nullity(A)=n.$

( $rank(A)$ =number of pivot columns,

$nullity(A)$ =number of non-pivot columns.)

Linear Independence and the Wronskian
A set of vector functions $\vec{f_1}(x), \dots, \vec{f_n}(x)$ from $\mathbb{R}$ to $\mathbb{R}^n$ is linearly independent in the interval $(\alpha,\beta)$ if $\displaystyle W[\vec{f_1}(x),\dots,\vec{f_n}(x)]\neq 0$ for at least one value of $x$ in the interval $(\alpha,\beta)$ .