1) think of the problem as this:

and

so the problem becomes

set u = cos(x) and du = -sin(x)dx

integral is then:

the end result should be and not tan(5x)

2) tan(x)^2 is really

first you need to use the trig identity and replace the sin(x)^2

then it becomes

separate each

which is then

and

end result being