I see this problem in 2 steps.

Firstly a 2D problem looking down onto the top of the box. From here you a maximising a rectangle inside a semi circle.

You need to find the area of this rectangle as a function and solve it when its derivative is equal to zero.

If the rectangle in question has dimensions y and x then and then now find where

After you have this we can find the third deminsion.