Thread: How to interpret results from a t-test

Hi,

I have a question regarding how to interpret values from a t-test. If you could help me it would be greatly appreciated.

I have two populations for which I sent a survey out to and got a number of responses back. The survey consisted of a question that had a rating scale of 1 to 5. I worked out the means of both populations so I can compare how they differ. Now I have come to do a t-test. I am reading different things, on what the t-test does, I have read that it will tell me:

- How statistically different two means are
- How likely the results are due to chance

I am doing a 2-tailed independent t-test, and my null hypothesis is that there is no difference in the means of the two populations. I have decided to use a 0.05 p-value threshold.

And this is where I get stuck, if my p-value is greater than 0.05, how do I interpret this, which one of these is correct:

- P-value greater than 0.05, therefore the null hypothesis is accepted and I say that the means are statistically the same.
Or
- P-value greater than 0.05, therefore theses results have occurred by chance

Which one is it? Is it both? If it is both, why are they related?

Thank you
Richard

A two sample t-test can indeed tell you things about two population/sample means. Remember though, that these various tests refer to the DISTRIBUTION of values across a spectrum. It is a good idea to review just how the graphs of our Z, T, Chi-squared and F charts look like, but for the moment, we're simply told that we need to use a T-chart to computer this particular Hypothesis Test.

One thing to note before starting, is that we can not PROVE a "null hypothesis". When we are doing a Hyp-Test, we can either reject or fail to reject a "null hypothesis", but we can not "prove" it. Knowing that, we can certainly create a test to determine whether or not our population differences are the result of chance, or whether there is something statistically significant going on to account for their difference.

I am doing a 2-tailed independent t-test, and my null hypothesis is that there is no difference in the means of the two populations. I have decided to use a 0.05 p-value threshold.

What is your alternate hypothesis; are the means "not equal", is Mean A greater than Mean B, or is Mean A less than Mean B? All these are valid alt. hypotheses, that need to be defined. In the case here, Mean A is not equal to Mean B would be our alt hypothesis, as you are saying you are told to use a two-tailed test.

I am doing a 2-tailed independent t-test, and my null hypothesis is that there is no difference in the means of the two populations. I have decided to use a 0.05 p-value threshold.

What is your critical value? Do you know how to calculate this? You can not just grab an arbitrary p-value (well I suppose you can, but they probably don't want you to do this). You are perhaps referring to a SIGNIFICANCE LEVEL of 0.05. And that's fine. However our critical value is going to determine what our P-value is.

Now, if the P-Value you manage to calculate is greater than the significance level of 0.05, then we fail to reject the null hypothesis (that our results are simply a matter a chance). If your P-Value is LESS than 0.05, then we can reject our null hypothesis, and say that there is evidence that. . ."lah blah blah" (whatever you set out to test).

Hopefully this isn't TOO cumbersome and you get something of value here. Let me know if you have any questions.

Thanks for the reply. To answer your questions:
1. alternative hypothesis is that the means are different
2. i have no idea what critical value is, nor how to work it out? I am simply entering my data into this tool and getting a p-value:
GraphPad QuickCalcs: t test calculator
you are right, I am referring to a significance level of 0.05.

Originally Posted by ANDS!

Now, if the P-Value you manage to calculate is greater than the significance level of 0.05, then we fail to reject the null hypothesis (that our results are simply a matter a chance). If your P-Value is LESS than 0.05, then we can reject our null hypothesis, and say that there is evidence that. . ."lah blah blah" (whatever you set out to test).

If P-Value > 0.05
Statisically the means are the same AND the results are simply a matter of chance

If P-Value < 0.05
Statisically the means are different AND the results are not due to chance

I guess the bit that confuses me is the relationship between chance and the means being the same. Why can't the means be the same and not due to chance?

Real quick - what math is this from? Is it from a Statistics class? Because if you are not able to interpret how to find critical values (and what it means), then the answers are not going to make much sense here.

If P-Value > 0.05
Statisically the means are the same AND the results are simply a matter of chance

Remember what a P-Value is; A P-value is the probability of getting the observed values by "chance" assuming we randomly sample from our population. P-Value on its own doesn't really tell us anything. However, a P-Value paired with a pre-chosen significance level, allows us to reject/fail to reject a hypothesis.

In this case, with a significance level of 0.05, if we have a P-Value greater than 0.05 (0.10 for instance), then we fail to reject the null hypothesis. In the case of this problem, your claim is that the means ARE different; however if we fail to reject the null hypothesis, then there is NOT enough evidence, at a 0.05 level of significance, to suggest that the means are different.

If however our P-Value is less than 0.05, say 0.01, then we reject the null hypothesis, and say there is a sufficient evidence to warrant rejection of the null hypothesis, and that it appears the sample means are different.

When you are solving statistical problem like this, you behave like a prosecutor.
H0: the accused is not guilty (the population means are equal)
H1: the accused is guilty (the population means are different)
In statistics and in law you always assume that the accused is not guilty (you always assume authenticity of H0). Your work is collecting proofs for authenticity H1. If you have enough strong proofs (if p<0,05), then you can say, that you reject H0 and accept H1. If your proofs isn’t strong enough you couldn’t say that you prove H0.You didn’t collect proofs for innocence. In that situation you always should tell that you fail to reject H0. So, you say that the means are statistically the same.

Originally Posted by ANDS!

Real quick - what math is this from? Is it from a Statistics class? Because if you are not able to interpret how to find critical values (and what it means), then the answers are not going to make much sense here.

This is actually for my dissertation which has nothing to do with statisics, I had to do a survery and I was told I should look at the t-tests as part of my analysis - hence why there are so many holes in my approach.

I think I am starting to get it, I guess the bit that was at the back of my mind was that when I looked at the means of the two groups, they are 2 and 4, which are clearly different. However when I did the t-test, the p-value was > 0.05 so I fail to reject my null hypothesis and so I concluded that the means must be the same - but I knew they were not the same!

Does that mean it would be better to make the null hypothesis: 'The relatedness of the means is unknown' ... because that in actual fact makes more sense.

I think I am starting to get it, I guess the bit that was at the back of my mind was that when I looked at the means of the two groups, they are 2 and 4, which are clearly different.

But are they STATISTICALLY different. Essentially that is what you are testing. Whether you can conclude that these two means represent truly DIFFERENT populations/samples. By assuming that the populations are the same, we can then formulate an alternative hypothesis.

In your case, YOUR claim (that the populations are different) becomes the alternative hypothesis; although it is just as normal if your claim was that the means are equal, then the null hypothesis because your claim, and we are deciding whether to reject/not-reject. Just remember, we can never PROVE the null hypothesis. It wouldn't be possible (you'd have to do A LOT of question asking. . .a lot).

Does that mean it would be better to make the null hypothesis: 'The relatedness of the means is unknown' ... because that in actual fact makes more sense.

Your Null Hypothesis has to have an equality in it; generally it is best to stay at "A=B" or some other similar hypothesis instead of using "greater/less-than or equal" signs.

However when I did the t-test, the p-value was > 0.05 so I fail to reject my null hypothesis and so I concluded that the means must be the same - but I knew they were not the same!

And it may well be that they aren't the same; you just don't have enough evidence to conclude that yet.

Let me give you an example:

Say I have a bin filled with red balls and blue balls in EQUAL quantity; 50/50 red and blue balls. Lets then say that I draw 20 balls, and out of those balls I have 8 red balls and 12 blue balls. The PROPORTION of red balls in this sample is 0.40. I then create a hypothesis test to determine whether my sample proportion is different than my population proportion.

Clearly in this case I can see that 0.40 is different than 0.50. However is it STATISTICALLY different. The probability that I will draw EXACTLY 10 red balls, and 10 blue balls is pretty damn low; however, because I draw a sample size of 0.40 red balls, may not be enough to conclude that this sample of red balls differs from the POPULATION (say for example you were selling packages of red and blue balls, and you wanted to make sure that clients were getting a healthy mix, and someone brings you a package and says "Hey look, 8 reds, 12 blue! Your packaging is biased!")

Thanks for your help guys, its been really useful.

Hi Guys,

I am almost near the end of writing my dissertation, a few questions have popped up regarding the t-test and I was wondering if you could help me interpret the t-test correctly.

Using:
Null hypo: no difference between the means
Alt hypo: means are statisically different i.e. not by chance
SL: 0.05

I have three sets of means to compare, here are the means, results and my interpretations:

Case 1
Mean 1: 4
Mean 2: 2
P-value: 0.0001
Conclusion: Means are statisically different

Case 2
Mean 1: 4.2
Mean 2: 4.0
P-value: 0.04
Conclusion: Means are statisically different

Case 3
Mean 1: 2.3
Mean 2: 2.1
P-value: 0.08
Conclusion: Fail to reject the null hypo - there is not enough statisical evidence to state that the observed difference did not occur by chance

I understand the t-test when I am comparing two means that I expect to be different (as I can see in case 1), however in cases 2 and 3, I can see the means are very close however according to my t-test, case 2 is statisically different (did not occur by chance) whereas in case 3 the t-test tells me that there is not sufficient evidence that the means are statistically different.

Does that mean that the data in case 3 is actually more closely related than the data in case 2 - is that what the t-test is telling me?