The sample mean is just the average of the 4 values.

I guess you want a 2 by 2 sample variance matrix.

Put the sample variances in the diagonal positions and the sample covariance in the off diagonal spot.

NOW do we use n or n-1 in this case?

Not sure if you want the MLEs (n) or the unbiased estimators (n-1).