I think your order of integration is wrong.
where a and b are positive real numbers. (Here max(s,t) is defined as the larger of the two numbers s and t.)
how do i even begin this? I am not familiar with this max(s,t) function, how does one determine which is greater in this case (depends on x and y, right?).
a^2x^2> b^2y^2, with all numbers positive, if and only if ax/b> y
You will need to break this into two double integrals will. We have x going from 0 to a and, for each x, y going from 0 to b. Instead, do one integral with y going from 0 to ax/b and another with y going from ax/b to b. In the first integral, and in the other .
Once you have done that, see if changing the order of integration doesn't help.