The wording of this question is quite subtle. It ask for us to count the number of ways that the balls can be put in the box such that any two boxes have more balls than the third.
I assume the question refers to using all the balls i.e. no ball can be left outside a box.
Hence in order for this requirement to be fulfilled no single box can have more than half the balls i.e. since there are
balls in total the maximum number of balls we can place in any single box must be
. It should be easy to see that this guarantees the requirement is fulfilled.
Now let's call the three distinct boxes
,
and
and the respective number of balls placed in each
,
and
.
If we start by placing a ball in
then
and
must have
balls each i.e. using the notation
to signify the different configurations we have that when
the configuration must be
and this is the only possible configuration for
since exchanging a ball from
to
would result in
and vice versa.
Now it should be easy to see that when we have
(
) one configuration we have is
and we can make
exchanges from
to
so in total (including the above configuration) we have
configurations of the balls when
.
Hence the total number of configurations,
, is given by
So using the well known result for this arithmetic progression we have that
.