Hi just a quickish question: Show that the least squares solution of Ax=b(where x and b are column vectors) is not unique and solve the normal equations to find all the least squares solutions.


A=\left( \begin{array}{cccc} 0&1&1&0 \\ 1&-1&1&-1\\1&0&1&0\\1&1&1&1 \end{array}\right)

b=\left( \begin{array}{c} 5\\3\\-1\\1 \end{array}\right)

My solution: The least squares solution will not be unique since the columns of A are not linearly independent.(by least squares theorem)

i.e. we can find scalars c_1,c_2,c_3,c_4 such that

c_1\left( \begin{array}{c} 0\\1\\1\\1 \end{array}\right) + c_2\left( \begin{array}{c} 1\\-1\\0\\1 \end{array}\right) + c_3\left( \begin{array}{c} 1\\1\\1\\1 \end{array}\right) + c_4\left( \begin{array}{c} 0\\-1\\0\\1 \end{array}\right) =0 ok so i went through with this and found c_1=-t, c_2=-t,c_3=t,c_4=t where t is a free variable

Now A^TA=<br /> \left( \begin{array}{cccc} 3&0&3&0 \\ 0&3&1&2 \\3&1&4&0\\0&2&0&2 \end{array}\right)

and A^Tb = \left( \begin{array}{c} 3\\3\\8\\-2 \end{array}\right)

The normal equations are given by A^Tx=A^Tb

when i solved this i got x_1=r,x_2=r,x_3=-r,x_4=r where r is a free variable


So this is the least squares approximation??

Thanks guys