(a) is ... weird Oo
First thing : if X and Y are independent, you can write the product
if not, you have to find the joint pdf of X and Y.
Second thing :
The whole formula is not very correct...
Assuming they're independent :
For the boundaries in (*), that's using the fact that
and that for any measurable function h, if X and Y are independent.*
erm... good luck
How did you get your equality (a) ?