Results 1 to 5 of 5

Math Help - Weighted linear regression

  1. #1
    Newbie
    Joined
    May 2010
    Posts
    1

    Weighted linear regression

    Hello all,

    I have a couple of questions regarding weighted linear regression that i will be using for the data analysis of my research work. I work in the area of Analytical chemistry, but i have to do a lot of data analysis stuff that includes linear regression (finding the best fit line and the corresponding slope, intercept etc.).Please share your opinions regarding my following questions.

    1. Suppose i have an independent variable "X" and a dependent variable "Y". For different values of X and Y, I can obtain a best fit line in Microsoft Excel without using any weighting. How can i obtain a best fit line if i use a weighting scheme such as 1/x , 1/x/x , etc. ? Are there any free Statistical softwares available for drawing these best fit lines?

    2. Also, in my research i will be having some baseline value of "Y" when X=0 (for example, for x=0, y = 10 units). In such a case, how is the weighting considered when x=0 ? I have this question because 1/x, 1/x/x cannot be defined when x=0 . How do these statistical softwares consider weighting in such a case? Will these softwares ignore the point (x,y) when x=0? What is the right thing to do in such a scenario?

    Please provide information to the concerned questions.

    Thanks,
    Sai
    Follow Math Help Forum on Facebook and Google+

  2. #2
    MHF Contributor
    Joined
    May 2010
    Posts
    1,030
    Thanks
    28
    edit deleted reply as it was incorrect
    Follow Math Help Forum on Facebook and Google+

  3. #3
    MHF Contributor
    Joined
    May 2010
    Posts
    1,030
    Thanks
    28
    disclaimer: I made this up.

    Your regression line is
    y = a + bx + e
    y=dependant variable
    x=independant variable
    e=error



    You want to minimise:
    f(x) = \sum{w_i e_i^{2}} = \sum{w_i(y_i-a_i-bx_i)^{2}}
    where w_i is the weight applying to data point i.
    (subscripts are dropped for the rest of this post)

    Take derivatives with respect to a and set to 0
    \frac{\partial f}{\partial a} = -2 \sum{(yw - aw -bwx)} =0

      \sum{(yw -bwx)} =a \sum{w}

     \frac{\sum{(yw -bwx)}}{\sum{w}} =a



    Take derivatives with respect to b and set to 0
    \frac{\partial f}{\partial b} =  -2 \sum{x(yw - aw -bwx)} =0

    \sum{xyw} - \sum{awx} -\sum{bwx^{2}}  =0

    \sum{xyw} - a\sum{wx} -b\sum{wx^{2}}  =0

    \frac{\sum{xyw} - a\sum{wx}}{\sum{wx^{2}}}  =b


    if you solve those simultaneously i think(!) you get

    b = \frac{\sum{w}\sum{xyw} - \sum{wx}\sum{wy}}{\sum{w}\sum{wx^{2}} - (\sum{wx})^{2}}

    a = \frac{\sum{(yw -bwx)}}{\sum{w}}



    So, to answer your question:
    You can get the WLS fit line by solving those two equations simultaneously for a and b.

    If you try to use a weighting of 1/x at x=0, you will get an undefined answer. You could choose a more suitable weighting function, manually set a very high weighting (eg 100000000000000) for x=0, or discard those data points.


    Once again, i remind you that i made the above algebra up, and you should check it yourself!
    Last edited by SpringFan25; May 22nd 2010 at 04:28 PM.
    Follow Math Help Forum on Facebook and Google+

  4. #4
    MHF Contributor
    Joined
    May 2010
    Posts
    1,030
    Thanks
    28
    google also suggests the following:

    an excel implementation of WLS:
    http://david.horner.faculty.noctrl.e...es_example.xls



    a formula for WLS:
    Google Answers: Weighted Least Linear Square Method simple example
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Newbie
    Joined
    May 2010
    Posts
    7
    R is free, and with it you can do anything and easily.
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. perpendicular linear span (linear regression)
    Posted in the Advanced Algebra Forum
    Replies: 0
    Last Post: October 22nd 2011, 03:56 PM
  2. Linear regression.
    Posted in the Statistics Forum
    Replies: 1
    Last Post: October 8th 2010, 02:08 AM
  3. Regression fundamentals (linear, non linear, simple, multiple)
    Posted in the Advanced Statistics Forum
    Replies: 13
    Last Post: November 12th 2009, 06:47 AM
  4. MLE of linear regression
    Posted in the Advanced Statistics Forum
    Replies: 4
    Last Post: September 4th 2009, 04:35 AM
  5. Regression SS in multiple linear regression
    Posted in the Advanced Statistics Forum
    Replies: 0
    Last Post: June 20th 2009, 12:23 PM

Search Tags


/mathhelpforum @mathhelpforum