Evaluate the triple integral where is the solid tetrahedon with vertices .
My question is, how do you find the equation of the plane z = ?
I need that to start solving the problem but i have no idea how to do that.
This is what it looks like to me: You need the equation of the gray plane in the figure below. To get it, you first calculate a normal which I've shown. Calculate the normal via the cross-product of two vectors which you can calculate emanating from the point (0,4,0). Once you get the normal, then the equation of the plane is $\displaystyle n_x(x-p_x)+n_y(y-p_y)+n_z(z-p_z)=0$
I get:
$\displaystyle I=\int_0^6\int_0^{f(x)}\int_0^{g(x,y)} xydzdydx$
with $\displaystyle f(x)=-2/3x+4$
$\displaystyle g(x,y)=1/24\left(-24x-36y-144\right)$
I'm not sure about this. Hopefully, this can get you in the ball-park.