Thread: Cumulative standard deviation question

Hello,
I have a question about cumulative standard deviation.
So I am applying this statics equation to my project, which deals with a huge data set. For each second, my data increases so there is no way I can keep track of all the data values.
What I need to find out is the new mean and standard deviation for each second given the previous data (mean, standard deviation, and total number of data), and new data, say x. Of course, each second means I calculate mean and standard deviation for 0,1,2,...second so somewhat accumulating.

So for 1st second, I can calculate new mean using
mean_1 = ((mean_0*n0)+data_1)/(n0+1)
My question is that I am having hard time to find the equation for the new standard deviation, sigma_1 given only mean_0, data_1, n_0, and mean_1

Could anyone help me?

Thank you so much,
Sincerely,

To calculate the new standard deviation you are correct in saying you need to remember some information about the previous data set.

To calculate the mean of n+1 data points from the mean of n data points is easy, as you've shown.

To calculate the new std dev is also not hard. Remember the definition:
std dev = (1/n) (sum over all data points) ( x - mean ) ^ 2.

The sum can also be written as 2 sums:
(sum over all data points)(x^2) - (sum over all data points)(mean ^2).

You should be able to figure it out from here. Alternatively, you can "remember" the value of "(sum over all data points)(x^2)" and recalculate the std dev as each new data point appears.