# How to solve this best least squares fit problem

• Dec 7th 2009, 09:52 AM
Computer Science guy
How to solve this best least squares fit problem
Hello,

I find myself needing to solve a number of best least squares fit problems, and honestly I don't remember the specifics of how it's done. Therefore I'm trying to nail the solution to a single problem before moving on with the rest - and I'd really like some help with that. Here's the description:

It is assumed that there is a coherence between the variables \$\displaystyle x\$ and \$\displaystyle y\$ of the form \$\displaystyle y=c0+c1*x\$. Given the following data (table)
x -1 1 2 4
y 0 1 3 4
we wish to find the best possible values for the coefficients
\$\displaystyle c0\$ and \$\displaystyle c1\$.

Task 1: Construct an overdetermined system of equations of the form \$\displaystyle A*c=y\$ determining the vector \$\displaystyle c=(c0,c1)^T\$ (\$\displaystyle ^T\$ meaning transposed).

I immediately imagine the answer to this is simply the following table:
\$\displaystyle -1\$ \$\displaystyle 0\$
\$\displaystyle 1\$ \$\displaystyle 1\$
\$\displaystyle ( 2\$ \$\displaystyle 3 ) * ( x\$ \$\displaystyle y )^T = ( c0\$ \$\displaystyle c1 )^T\$ -- the first part here is a 4x2 matrix, but my formatting is awful - sorry!
\$\displaystyle 4\$ \$\displaystyle 4\$
But that doesn't seem to match the required form. What is the correct answer - and why?

Furthermore, I'm most uncertain as to how the following tasks are solved...

Task 2: Calculate the normal equations \$\displaystyle A^T*A*c=A^T*y\$.

Task 3: Find \$\displaystyle c0\$ and \$\displaystyle c1\$.

Could someone explain how it's done? I could really use some help as I pretty much don't have a clue here.

Lots of thanks!
• Dec 8th 2009, 02:54 AM
Computer Science guy
No help to find?
• Dec 8th 2009, 06:16 AM
Zocken
ok umm that looks really confusing but I'll take a shot since I'm doing least squares.

you have to find the form ax=b right.

the A form would be like

1 -1
1 1
1 2
1 4

x would be c0 and c1

b would be

0
1
3
4

so part 1 done. Then it wants you to find Atransposed times A, which I assume you know how to do. Then it wants you to calculate y (the least squares solution) for c0 and c1, which you just need to find the inverse of AtA times Aty
• Dec 8th 2009, 06:24 AM
Computer Science guy
I actually think I figured it out now... Somewhat:
Calculus.pdf