Your attempted working is incredibly vague but I shall take a shot at this.
First of all, judging by your equation, I believe you are using volumes of solids of revolution using circular discs, which uses the formula for revolutions about the y axis.
However, your equation is incorrect if you are to integrate between y = 0 to y = 5.
This is because you turned the cup up-side down with that equation. Integrating between y = 0 and 5 gives you the empty volume of the glass, not the volume of water.
To find the volume of water, either integrate between y = 5 to 10 with your equation, or flip the cup so it's base of 3 cm lies on the x axis. The equation for this is