Start by looking just at the first five racers. Imagine that the first three are wearing red hats. There are four possibilities for the first child, then 3 for the second, two for the third. There are 4(3)(2)= 24 ways that can happen. Now, there are 8-non-red hats so 8(7)= 56 ways the fourth and fifth students can be "non-red hats". That would give a total of (56)(24)= 1344 ways the first 5 children can finish "red, red, red, non-red, non-red". But there are also 5!/(3!2!)= 10 different orders for "RRRNN" so, in fact, 13440 ways the first five children could be "3 red hat, 2 non-red hat". Now, we have 7 children remaining. There are 7! ways they can finish.

(This is assuming that we are talking about the ways individual children, not just colors of hats, can finish. That is, "Jimmy first, Sarah second" is different from "Sarah first, Jimmy second" even though both are wearing red hats.)