Results 1 to 13 of 13
Like Tree7Thanks
  • 1 Post By ILikeSerena
  • 1 Post By ILikeSerena
  • 1 Post By ILikeSerena
  • 1 Post By ILikeSerena
  • 1 Post By ILikeSerena
  • 1 Post By ILikeSerena
  • 1 Post By ILikeSerena

Math Help - confidence that values are different

  1. #1
    Newbie
    Joined
    Jun 2011
    Posts
    12

    confidence that values are different

    Hi there,

    This isn't a homework question. I'm a PhD student, and I'm currently working on an area that would be a little lengthy and unnecessary to explain here, but I have summarised in the following analogy. Stats is not my strong point, so I would really appreciate it if someone could point me in the right direction:

    Imagine that you had a whole bunch of red balls and green balls. You suspect that the red balls may have a very slightly different diameter than the green balls.

    If you measure one of each, you would get a value for the diameter of each colour ball and an associated uncertainty with each measurement. For example:

    D(red) = 10.5 +/- 0.5
    D(green) = 10.6 +/- 0.5

    With these values, you can't really decide whether the balls DO have a different diameter or if they are the same.

    Now if you measure more and more balls of each colour, then the uncertainty of the diameter with become smaller and smaller. After say 100 measurements of the diameter of each colour of ball, we might have:

    mean D(red) = 10.51 +/- 0.05
    mean D(green) = 10.59 +/- 0.05

    My question is - how many measurements might we like to take until we can say with some degree of confidence that the red balls do (or don't) have a different diameter to the green balls. Obviously it depends to some extent how far apart the mean of the diameters are. If you had:

    mean D(red) = 10.5 +/- 0.5
    mean D(green) = 20.6 +/- 0.5

    Then you can be pretty sure they have different diameters!

    I guess in summary, I want to say if:

    mean D(red) = X(R) +/- Y(R)
    mean D(green) = X(G) +/- Y(G)

    Then with what confidence can I say that the diameters are the same or different.

    Any help at all with this would be gratefully received! Thank you in advance.
    Follow Math Help Forum on Facebook and Google+

  2. #2
    Super Member ILikeSerena's Avatar
    Joined
    Dec 2011
    Posts
    733
    Thanks
    121

    Re: confidence that values are different

    Hi jenny1988!

    What you are describing is a t-test.
    More specifically, an independent two-sample t-test with presumably equal variances.


    As an example, after 100 measurements of red balls, and 100 measurements of green balls, your t-score is:

    t = \frac {X(R) - X(G)} {\sqrt{Y(R)^2+Y(G)^2} / \sqrt{100}}

    This assumes you have properly determined Y(R) and Y(G).

    Then you would have to look it up in a t-test table, specifying df=2x100-2=198.
    This will give you the confidence level you ask for.


    The proper procedure is:
    1. Calculate the t-score with the given formula (see here).
    2. Look up p-value that corresponds with this t-score.
      You will need to specify the degrees of freedom (df) as given in the article.
      You can do this with a graphical calculator, SPSS, Excel, an online calculator, or whatnot.
    3. If this p-value is smaller than 0.05, you can say with a confidence of 95% that the diameters are different.
      Or with an even smaller p-value, you can say with a confidence of (1-2p)x100% that the diameters are different.
    Last edited by ILikeSerena; January 1st 2013 at 03:49 PM. Reason: correction for 2-sided
    Thanks from jenny1988
    Follow Math Help Forum on Facebook and Google+

  3. #3
    Newbie
    Joined
    Jun 2011
    Posts
    12

    Re: confidence that values are different

    That's awesome! Thank you so much for your help!
    Follow Math Help Forum on Facebook and Google+

  4. #4
    Super Member ILikeSerena's Avatar
    Joined
    Dec 2011
    Posts
    733
    Thanks
    121

    Re: confidence that values are different

    Just to be safe, how are you getting Y(R) and Y(G)?
    Since, if you use the wrong values for those, the result will be entirely off.
    Thanks from jenny1988
    Follow Math Help Forum on Facebook and Google+

  5. #5
    Newbie
    Joined
    Jun 2011
    Posts
    12

    Re: confidence that values are different

    Well there's quite a lot of error propagation calculations in my actual work before I arrive at the final uncertainty in the "diameters" - but I was careful and I'm pretty sure that my uncertainties are correct. Famous last words :-)
    Follow Math Help Forum on Facebook and Google+

  6. #6
    Super Member ILikeSerena's Avatar
    Joined
    Dec 2011
    Posts
    733
    Thanks
    121

    Re: confidence that values are different

    My point is that you probably already reduced Y(R) and Y(G) by a factor \sqrt{100}.
    Obviously you should not do that twice.

    And actually, Y(R) and Y(G) should be calculated with the formula for the standard deviation from the respective sets of measurements.
    This may well be larger than the (reduced or not) measurement precision.
    Not sure if you are aware of that.
    Thanks from jenny1988
    Follow Math Help Forum on Facebook and Google+

  7. #7
    Newbie
    Joined
    Jun 2011
    Posts
    12

    Re: confidence that values are different

    Ah I see. Well now that you've put a name to my problem, I can research how to do the test properly and I will carefully check that I'm doing everything correctly. Thank you again for the advice and for the link - it's saved me lots of time!
    Follow Math Help Forum on Facebook and Google+

  8. #8
    Newbie
    Joined
    Jun 2011
    Posts
    12

    Re: confidence that values are different

    Okay - so I see what you mean about using the factor of sqrt(100) twice.

    The problem is that I can't calculate the standard deviation using the set of measurements, because I don't actually have any measurements (and it's not possible to make any at this stage). To continue my analogy, imagine that the uncertainty in one measurement of the diameter of a ball is known from previous work - but I extrapolate this idea to imagine putting 100 balls end-to-end to, measuring this distance and, therefore, reduce this uncertainty, right? So I have an uncertainty value that has not come from multiple real data.

    With my reduced uncertainty, am I now not able to use the t test as quoted above, because as you said, the n = 100 has already played a part is getting my uncertainty?

    I should also say that although I have no measurements in the work I am doing, I do have something I can use as the mean, because I am inputting some values that I think would be roughly the mean I expect and then going through this analysis to find out how small the uncertainties would need to be to say that the measurements are difference with say 95% confidence.

    Hope this makes some sense! Thank you.

    (edit: as I look at it more though, doesn't the uncertainty (sigma) have n already ingrained into it, even if you do calculate it from real data?)
    Last edited by jenny1988; January 1st 2013 at 12:41 PM.
    Follow Math Help Forum on Facebook and Google+

  9. #9
    Super Member ILikeSerena's Avatar
    Joined
    Dec 2011
    Posts
    733
    Thanks
    121

    Re: confidence that values are different

    If you already know the uncertainty, you should use that value.
    Suppose the uncertainty of a single measurement is \sigma (this is the symbol typically used for the standard deviation of the population as opposed to the sample).
    And suppose this is the uncertainty for the red balls as well as the green balls.
    Then the uncertainty of the mean of 100 measurements is \sigma / \sqrt{100}.
    This is called the standard error (SE) as opposed to the standard deviation of a single measurement (denoted \sigma or s depending on its origin).

    It this is the case, you should use a z-test.

    z = \frac {X(R) - X(G)}{\sigma / \sqrt{100}}

    If this z-value is greater than 1.96 or less than -1.96, you have a confidence level of 95% that the diameters are different.

    Just to make sure, I would also calculate the unreduced standard deviations of your samples, to see if they are in the same range.
    Just to avoid nasty surprises.
    Thanks from jenny1988
    Follow Math Help Forum on Facebook and Google+

  10. #10
    Super Member ILikeSerena's Avatar
    Joined
    Dec 2011
    Posts
    733
    Thanks
    121

    Re: confidence that values are different

    Quote Originally Posted by jenny1988 View Post
    (edit: as I look at it more though, doesn't the uncertainty (sigma) have n already ingrained into it, even if you do calculate it from real data?)
    A couple of uncertainties are distinguished.

    \sigma - The standard deviation of the population.
    This one is usually unknown.
    If it is known, perhaps from very reliable earlier measurements, you should do the z-test.
    As you can see, n, the number of measurements in a sample, is not ingrained in it.


    s - The standard deviation of a sample of n measurements.
    The formula is s=\sqrt{\frac {\sum (x_k-\bar x)^2}{n - 1}}.
    This does indeed contain n, the number of measurements.
    I suspect this is the one you need to use.


    SE - The standard error, which is the standard deviation of the mean of a sample of n measurements.
    The formula is SE=\frac {\sigma}{\sqrt n} or SE=\frac {s}{\sqrt n} depending on whether you know \sigma or not.
    Last edited by ILikeSerena; January 1st 2013 at 01:30 PM.
    Thanks from jenny1988
    Follow Math Help Forum on Facebook and Google+

  11. #11
    Super Member ILikeSerena's Avatar
    Joined
    Dec 2011
    Posts
    733
    Thanks
    121

    Re: confidence that values are different

    Let me put it differently, since I suspect I am confusing you.

    If you have:

    mean D(red) = 10.51 +/- 0.05
    mean D(green) = 10.59 +/- 0.05

    where these numbers and uncertainties are the result of 100 measurements of each.

    Then the corresponding z-value is:

    z = \frac {10.59 - 10.51}{0.05} = 1.6

    Since this is less than 1.96, we have not enough evidence to be able to say that the diameters are different.
    So either they are not, or you need more measurements.

    The corresponding p-value is:

    p = 0.0548

    It means that, based on the sample, the chance that the diameters are actually different is (1-2p)x100% = 89.04%.
    However, this is usually considered insufficient to be able to say they are actually different.
    Last edited by ILikeSerena; January 1st 2013 at 03:51 PM. Reason: correction for 2-sided
    Thanks from jenny1988
    Follow Math Help Forum on Facebook and Google+

  12. #12
    Newbie
    Joined
    Jun 2011
    Posts
    12

    Re: confidence that values are different

    Fantastic - that's all really clear thank you!
    Follow Math Help Forum on Facebook and Google+

  13. #13
    Super Member ILikeSerena's Avatar
    Joined
    Dec 2011
    Posts
    733
    Thanks
    121

    Re: confidence that values are different

    Erm... just edited the numbers in my previous post, since I made a calculation error.

    Quote Originally Posted by jenny1988 View Post
    Fantastic - that's all really clear thank you!
    You're welcome!
    (Btw, did you notice the "Thanks" hyperlink at the right bottom in the posts?)
    Thanks from jenny1988
    Follow Math Help Forum on Facebook and Google+

Similar Math Help Forum Discussions

  1. Confidence intervals and large-sample confidence
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: November 18th 2012, 03:43 PM
  2. Replies: 1
    Last Post: June 1st 2011, 01:47 AM
  3. How to find zα/2 values for confidence interval
    Posted in the Advanced Statistics Forum
    Replies: 1
    Last Post: February 17th 2011, 05:06 PM
  4. confidence level and confidence interval?
    Posted in the Advanced Statistics Forum
    Replies: 2
    Last Post: July 12th 2009, 06:59 AM
  5. Replies: 1
    Last Post: May 24th 2009, 05:16 AM

Search Tags


/mathhelpforum @mathhelpforum