If I understood correctly: you are cutting small squares from 4 corners (identical) and closing them up to make the open-box (open rectangular prism).
Let x be the side of small squares cut off the corners.
Then the volume: (20-2x)^2 * x = f(x)
To maximize f(x), take the derivative and equate it to zero. So: f'(x)=0 will give you the answer.
Scientific detail: You must take second derivative and check that it is always negative for all possible values of x (why?)
Unscientific detail: Somehow the height in similar questions often comes up to be 25% of the side of the base.
Please let us know of what you'll find.