1. ## Sampling without replacement

Hi,

I'm trying to show that the sample variance obtained from a random sampling without replacement is

$\frac{\sigma^2}{n}\left(1 - \frac{n - 1}{N - 1}\right) \quad
\mbox{where} \quad \sigma^2 = \frac{1}{N} \sum_i^N \left(x_i - \mu\right)^2$

and \mu is the sample mean. A lot of books cite the factor $\left(1 - \frac{n - 1}{N - 1}\right)$ as a correction for small samples, but I couldn't derive it. Could someone help me here, please?

2. Have you taken into account the Covariance term while calculating the variance? The Covariance term becomes 0 when sampling is done With Replacement; but that will not vanish in case of Without Replacement.

3. Hi Sambit, thank you for your response.

Yes, I considered the covariance, but I'm probably doing something wrong. I got the following result

$
Cov(x_i,x_j) = E(x_i x_j) - E(x_i)E(x_j) = x_i x_j \frac{n}{N}\frac{n-1}{N-1} - x_ix_j\left(\frac{n}{N}\right)^2.
$

The problem is that it does not depend on $\sigma$. I'm missing something... or doing something completely wrong.

4. The right expression for Covariance is $-\frac{\sigma^2}{N-1}$. Consult any book, you will find it there.

5. Thanks again, Sambit.

I saw this expression, but I don't know how to derive it. Most books only mention it but does not show how to obtain this formula. I was reading Kish's Survey Sampling book, but he just mention it as a correction for finite sample with replacement. I found some links from the University of Texas, but they also just mention the covariance without showing all calculations behind.

I'm stuck. I have no clue how to calculate this covariance. Could you give some hint, please?
This problem is driving me crazy!

6. $Cov(y_i,y_j)$ $
=E[\{y_i-\bar{Y}\}\{y_j-\bar{Y}\}]$
$
=\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y}).P(y_i=Y_{\alpha},y_j=Y_{\beta})$
$
=\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y})\frac{1}{N(N-1)}$
where $\alpha\neq\beta$
$
=\frac{1}{N(N-1)}\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y})$
where $\alpha\neq\beta$
$
=\frac{1}{N(N-1)}[\sum_{\alpha=1}^N\sum_{\beta=1}^N(Y_{\alpha}-\bar{Y})(Y_{\beta}-\bar{Y})-\sum_{\alpha=1}^N(Y_{\alpha}-\bar{Y})^2]$
$
=\frac{1}{N(N-1)}\sum_{\alpha=1}^N(Y_{\alpha}-\bar{Y})\sum_{\beta=1}^N(Y_{\beta}-\bar{Y})-N\sigma^2$
$
=\frac{1}{N(N-1)}[0-N\sigma^2]$
$
=-\frac{\sigma^2}{N-1}$

Here $i,j$ denote sample members and $\alpha,\beta$ denote population members.

7. Thank you very much for your help, Sambit. Now I can sleep in peace!