Degrees of freedom

**TriKri** · October 14th 2009, 06:02 AM

Hi! Can someone explain this to me:

The definition of $\chi^2$ -distribution, taken from my statistics book, is:

If $X_1,\ X_2,... ,\ X_f$ are independent and $~N(0,\ 1)$ , then

$\sum_{i=1}^f X_i^2\ \sim\chi^2(f)$

f is the number of degrees of freedom.

The book also says (but it doesn't prove it) that if $X_1,\ X_2,... ,\ X_n$ are independent and $~N(0,\ 1)$ , then

$\sum_{i=1}^n (X_i-\bar{X})^2\ \sim\chi^2(n-1),$ where $\bar{X}=\frac{X_1+X_2+...+X_n}{n}$

I would really like to see what the proof looks like. How can this be proven?

**matheagle** · October 15th 2009, 05:25 PM

First of all that's a lame definition of a chi-square.

The real definition is $\chi^2_v=\Gamma(v/2,2)$

WHERE the dfs need not be an integer.

It's easy to prove that if you square a st normal you get a chi-square with 1 df and then via MGFs you can show that sums of independent chi-squares gives you a chi-square.

NOW, when you subtract the sample mean you do lose that 1 df.
It's not a simple proof and I couldn't find it on the web.
I'm sure it's here and I'll look again.

**TriKri** · October 16th 2009, 03:00 PM

I just realized it wasn't the definition of chi-square :P It was just a theorem; the definition was some function containing the gamma function, like you wrote. I think the definition was

$ f(x;k)= \begin{cases}\displaystyle \frac{1}{2^{k/2}\Gamma(k/2)}\,x^{(k/2) - 1} e^{-x/2}&\text{for }x>0,\\ 0&\text{for }x\le0, \end{cases} $

(the same as that on wikipedia). What I think is kind of strange - my book (our course literature) states a lot of things, but it proves few of them. Another thing that it states but it doesn't prove is that the test variable in the chi-square test is chi-square distributed:

If Z is distributed in r states with probabilities $p_1,\ p_2,\ ...\ ,\ p_r$ , and $X_i$ is the number of times Z, out of n observations, ended up in state i, then the test variable:

$Q=\sum_{i=1}^r \frac{(X_i-np_i)^2}{np_i}=\sum_{i=1}^r \frac{(X_i-E_i)^2}{E_i}$

is chi-square(r-1)-distributed (here $E_i$ is the expected number of times Z will end up in state i). The formula however is not motivated, although they prove it is chi-square(r-1) distributed for r = 2. If you look at a single term:

$\frac{(X_i-E_i)^2}{E_i}$

it doesn't look chi-square distributed. If $Y_i$ is 1 if Z ends up in state i, and 0 otherwise, $y_{i,j}$ has mean $p_i$ and variance $p_i(1-p_1)$ . The sum $\sum_{j=1}^n y_{i,j} = X_i$ will approximately get distributed by $N(np_i,\ np_i(1-p_1)) = N(E_i,\ E_i(1-p_i))$ . Now

$\frac{(X_i-E_i)^2}{E_i(1-p_i)}\sim\chi^2(1)$

approximately. That is why I wonder why the formula doesn't look like

$Q=\sum_{i=1}^r \frac{(X_i-E_i)^2}{E_i(1-p_i)}$

instead (which I think should be chi-square(r) or possibly chi-square(r-1) distributed), so for me it looks like someone has forgot a factor in the denominator (although I know that it's not the case). Anyone who knows why it looks as it does and how the formula has been obtained?

**matheagle** · October 16th 2009, 03:20 PM

I figured it was just a theorem.
But I don't have your book in front of me.
I teach out of wackerly and walpole all the time.
The second thing you posted is the pearson goodness of fit test.
http://en.wikipedia.org/wiki/Pearson's_chi-square_test
It is important to note that this is not exactly a chi-square as mentioned in that link
'the distribution of the test statistic is not exactly that of a chi-square random variable'
This link seems reasonable too...http://www.statsdirect.com/help/chi_...s/chi_good.htm
Note that the denominators are all different, see
http://www.stat.wisc.edu/~mchung/tea.../lecture24.pdf

**TriKri** · October 17th 2009, 02:54 AM

No, I know, it's an approximation and only good for large n, or for large E:s. Besides, this distribution is discrete (there is only a finite number of possible outcomes for each n), while the chi-square distribution is continuous. It seems to be difficult to prove that these sums are chi-square distributed, or in the latter case, distributed similarly to the chi-square distribution. I still think it's bad though that we are taught stuff that is not proven. It forces you to trust blindly in different mathematical expressions.

Thread: Degrees of freedom

LinkBack

Thread Tools

Display

Degrees of freedom

Similar Math Help Forum Discussions

Degrees of Freedom in Matrix?

error degrees of freedom

chi square degrees of freedom

chi square dist with zero degrees of freedom

P-value and Degrees of Freedom

Search Tags