If I understood correctly: you are cutting small squares from 4 corners (identical) and closing them up to make the open-box (open rectangular prism).

Let x be the side of small squares cut off the corners.

Then the volume: (20-2x)^2 * x = f(x)

To maximize f(x), take the derivative and equate it to zero. So: f'(x)=0 will give you the answer.

Scientific detail: You must take second derivative and check that it is always negative for all possible values of x (why?)

Unscientific detail: Somehow the height in similar questions often comes up to be 25% of the side of the base.

Please let us know of what you'll find.

-O