# Sampling without replacement

• Mar 14th 2011, 01:21 PM
gustavodecastro
Sampling without replacement
Hi,

I'm trying to show that the sample variance obtained from a random sampling without replacement is

$\displaystyle \frac{\sigma^2}{n}\left(1 - \frac{n - 1}{N - 1}\right) \quad \mbox{where} \quad \sigma^2 = \frac{1}{N} \sum_i^N \left(x_i - \mu\right)^2$

and \mu is the sample mean. A lot of books cite the factor $\displaystyle \left(1 - \frac{n - 1}{N - 1}\right)$ as a correction for small samples, but I couldn't derive it. Could someone help me here, please?
• Mar 14th 2011, 07:18 PM
Sambit
Have you taken into account the Covariance term while calculating the variance? The Covariance term becomes 0 when sampling is done With Replacement; but that will not vanish in case of Without Replacement.
• Mar 18th 2011, 02:34 AM
gustavodecastro
Hi Sambit, thank you for your response.

Yes, I considered the covariance, but I'm probably doing something wrong. I got the following result

$\displaystyle Cov(x_i,x_j) = E(x_i x_j) - E(x_i)E(x_j) = x_i x_j \frac{n}{N}\frac{n-1}{N-1} - x_ix_j\left(\frac{n}{N}\right)^2.$

The problem is that it does not depend on $\displaystyle \sigma$. I'm missing something... or doing something completely wrong.
• Mar 18th 2011, 08:18 AM
Sambit
The right expression for Covariance is $\displaystyle -\frac{\sigma^2}{N-1}$. Consult any book, you will find it there.
• Mar 18th 2011, 01:42 PM
gustavodecastro
Thanks again, Sambit.

I saw this expression, but I don't know how to derive it. Most books only mention it but does not show how to obtain this formula. I was reading Kish's Survey Sampling book, but he just mention it as a correction for finite sample with replacement. I found some links from the University of Texas, but they also just mention the covariance without showing all calculations behind.

I'm stuck. I have no clue how to calculate this covariance. Could you give some hint, please?
This problem is driving me crazy!
• Mar 19th 2011, 06:33 AM
Sambit
$\displaystyle Cov(y_i,y_j)$$\displaystyle =E[\{y_i-\bar{Y}\}\{y_j-\bar{Y}\}]$$\displaystyle =\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y}).P(y_i=Y_{\alpha},y_j=Y_{\beta})$$\displaystyle =\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y})\frac{1}{N(N-1)} where\displaystyle \alpha\neq\beta \displaystyle =\frac{1}{N(N-1)}\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y}) where\displaystyle \alpha\neq\beta \displaystyle =\frac{1}{N(N-1)}[\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y})-\sum_{\alpha=1}^N(Y_{\alpha}-\bar{Y})^2]$$\displaystyle =\frac{1}{N(N-1)}\sum_{\alpha=1}^N(Y_{\alpha}-\bar{Y})\sum_{\beta=1}^N(Y_{\beta}-\bar{Y})-N\sigma^2$$\displaystyle =\frac{1}{N(N-1)}[0-N\sigma^2]$$\displaystyle =-\frac{\sigma^2}{N-1}$

Here $\displaystyle i,j$ denote sample members and $\displaystyle \alpha,\beta$ denote population members.
• Mar 19th 2011, 04:10 PM
gustavodecastro
Thank you very much for your help, Sambit. Now I can sleep in peace! :)