I am analyzing results from a survey, using ANOVA to determine differences between the means of different subgroups (sorted by variables such as Ethnicity, Job level, etc). My total sample size is 400, and the subgroup sample sizes range from 6 to 79. Does ANOVA take into consideration the size of the samples?
A colleague recommended that I use weighted means before running ANOVA. Would that produce accurate results?
The concern is that the group with 6 people only represents 1.5% of my total sample - yet their mean is different to a statistically significant amount. Is the subgroup size large enough to be significant? Are there guidelines as to how big a subgroup has to be before it's significant?