Weighted linear regression

May 2010
1
0
Hello all,

I have a couple of questions regarding weighted linear regression that i will be using for the data analysis of my research work. I work in the area of Analytical chemistry, but i have to do a lot of data analysis stuff that includes linear regression (finding the best fit line and the corresponding slope, intercept etc.).Please share your opinions regarding my following questions.

1. Suppose i have an independent variable "X" and a dependent variable "Y". For different values of X and Y, I can obtain a best fit line in Microsoft Excel without using any weighting. How can i obtain a best fit line if i use a weighting scheme such as 1/x , 1/x/x , etc. ? Are there any free Statistical softwares available for drawing these best fit lines?

2. Also, in my research i will be having some baseline value of "Y" when X=0 (for example, for x=0, y = 10 units). In such a case, how is the weighting considered when x=0 ? I have this question because 1/x, 1/x/x cannot be defined when x=0 . How do these statistical softwares consider weighting in such a case? Will these softwares ignore the point (x,y) when x=0? What is the right thing to do in such a scenario?

Please provide information to the concerned questions.

Thanks,
Sai
 
May 2010
1,034
272
disclaimer: I made this up.

Your regression line is
\(\displaystyle y = a + bx + e\)
y=dependant variable
x=independant variable
e=error



You want to minimise:
\(\displaystyle f(x) = \sum{w_i e_i^{2}} = \sum{w_i(y_i-a_i-bx_i)^{2}}\)
where \(\displaystyle w_i\) is the weight applying to data point i.
(subscripts are dropped for the rest of this post)

Take derivatives with respect to a and set to 0
\(\displaystyle \frac{\partial f}{\partial a} = -2 \sum{(yw - aw -bwx)} =0\)

\(\displaystyle \sum{(yw -bwx)} =a \sum{w}\)

\(\displaystyle \frac{\sum{(yw -bwx)}}{\sum{w}} =a \)



Take derivatives with respect to b and set to 0
\(\displaystyle \frac{\partial f}{\partial b} = -2 \sum{x(yw - aw -bwx)} =0\)

\(\displaystyle \sum{xyw} - \sum{awx} -\sum{bwx^{2}} =0\)

\(\displaystyle \sum{xyw} - a\sum{wx} -b\sum{wx^{2}} =0\)

\(\displaystyle \frac{\sum{xyw} - a\sum{wx}}{\sum{wx^{2}}} =b\)


if you solve those simultaneously i think(!) you get

\(\displaystyle b = \frac{\sum{w}\sum{xyw} - \sum{wx}\sum{wy}}{\sum{w}\sum{wx^{2}} - (\sum{wx})^{2}}\)

\(\displaystyle a = \frac{\sum{(yw -bwx)}}{\sum{w}}\)



So, to answer your question:
You can get the WLS fit line by solving those two equations simultaneously for a and b.

If you try to use a weighting of 1/x at x=0, you will get an undefined answer. You could choose a more suitable weighting function, manually set a very high weighting (eg 100000000000000) for x=0, or discard those data points.


Once again, i remind you that i made the above algebra up, and you should check it yourself!
 
Last edited:
May 2010
7
0
R is free, and with it you can do anything and easily.