Use the Divergence Theorem to find the flux of F across the surface sigma with outward orientation.
F(x,y,z) = 2xzi + yzj + z^2k, where sigma is the surface of the solid bounded above by z = sqrt(a^2-x^2-y^2) and below by the xy-plane.
Use the Divergence Theorem to find the flux of F across the surface sigma with outward orientation.
F(x,y,z) = 2xzi + yzj + z^2k, where sigma is the surface of the solid bounded above by z = sqrt(a^2-x^2-y^2) and below by the xy-plane.
$\displaystyle \text{div}(F) = 2z+z+2z=5z$
So you want to evaluate $\displaystyle 5\int\int\int z\,dV$ bounded above by a sphere of radius $\displaystyle a$ and bounded below by the plane $\displaystyle z=0$.
Use spherical coordinates (where $\displaystyle \rho$ is the radius, $\displaystyle \phi$ is the azimuth angle, and $\displaystyle \theta$ is the angle around the circle.
$\displaystyle z=\rho\cos\phi$
$\displaystyle dV=\rho^2\sin\phi\,d\rho\,d\phi\,d\theta$
The bounds:
$\displaystyle \rho=0..a$
$\displaystyle \phi=0..\pi/2$
$\displaystyle \theta=0..2\pi$
So you want to evaluate $\displaystyle 5\int_0^{2\pi}\int_0^{\pi/2}\int_0^a \rho^3\sin(\phi)\cos(\phi)\,d\rho\,d\phi\,d\theta = \frac{5}{2}\int_0^{2\pi}\int_0^{\pi/2}\int_0^a \rho^3\sin(2\phi)\,d\rho\,d\phi\,d\theta$
Evaluating this integral should be straightforward. I have included the final answer below.
Spoiler: