Hi just a quickish question: Show that the least squares solution of $\displaystyle Ax=b$(where x and b are column vectors) is not unique and solve the normal equations to find all the least squares solutions.


$\displaystyle A=\left( \begin{array}{cccc} 0&1&1&0 \\ 1&-1&1&-1\\1&0&1&0\\1&1&1&1 \end{array}\right)$

$\displaystyle b=\left( \begin{array}{c} 5\\3\\-1\\1 \end{array}\right)$

My solution: The least squares solution will not be unique since the columns of A are not linearly independent.(by least squares theorem)

i.e. we can find scalars $\displaystyle c_1,c_2,c_3,c_4 $ such that

$\displaystyle c_1\left( \begin{array}{c} 0\\1\\1\\1 \end{array}\right) + c_2\left( \begin{array}{c} 1\\-1\\0\\1 \end{array}\right) + c_3\left( \begin{array}{c} 1\\1\\1\\1 \end{array}\right) + c_4\left( \begin{array}{c} 0\\-1\\0\\1 \end{array}\right) =0 $ ok so i went through with this and found $\displaystyle c_1=-t, c_2=-t,c_3=t,c_4=t$ where t is a free variable

Now $\displaystyle A^TA=
\left( \begin{array}{cccc} 3&0&3&0 \\ 0&3&1&2 \\3&1&4&0\\0&2&0&2 \end{array}\right)$

and $\displaystyle A^Tb = \left( \begin{array}{c} 3\\3\\8\\-2 \end{array}\right)$

The normal equations are given by $\displaystyle A^Tx=A^Tb$

when i solved this i got $\displaystyle x_1=r,x_2=r,x_3=-r,x_4=r$ where r is a free variable


So this is the least squares approximation??

Thanks guys