I see this problem in 2 steps.
Firstly a 2D problem looking down onto the top of the box. From here you a maximising a rectangle inside a semi circle.
You need to find the area of this rectangle as a function and solve it when its derivative is equal to zero.
If the rectangle in question has dimensions y and x then and then now find where
After you have this we can find the third deminsion.