As the hint suggests we will find the cdf (comulative distribution function) Y:

F(y) = P(Y <= y) = P(max(X1, X2, X3) <= y)

Now the important part is to notice that max(X1, X2, X3) <= y implies that each & every one of the r.v.'s (X1, X2, X3) is smaller than y, and thus:

P(max(X1, X2, X3) <= y) = P(X1 <= y)P(X2 <= y)P(X3 <= y) = Fx1(y)Fx2(y)Fx3(y) = y^3 0 <= y <= 1

now we can compute the pdf of Y by taking the derivative of it's cdf:

fy(Y) = 3*y^2

E(Y) = integ(y*fy(Y))[0,1] = integ(3*y^3)[0,1] = 0.75*y^4 |[0,1] = 0.75