Check the wording of the question. What you've written isn't a line.
Make x the subject of the line equation and substitute this into the hyperbola equation. Rearrange this as a quadratic in y. For the line to be a tangent we require this quadratic to have repeated roots. Applying this condition should produce the result stated.
So, do I substitute the value of y in the line equation with the value of y with respect to x?
In that case, the equation becomes,
So the line equation will become
I tried to simplify the equation and it is becoming messy. Am I doing things right?
(Sorry for the small image, laTEX is pain and I do not know how to put the special characters here. Tried to use an online latex editor but there seems to be some problem)
If you want one last fresh start try this. I'll write A instead of alpha. Line can be written y=-(cosA/sinA)x+p/sinA
This is y=mx+c with m=-cosA/sinA and c=p/sinA
Substitute y=mx+c into the hyperbola. So x^2/a^2-(mx+c)^2/b^2=1
Multiply throughout by a^2b^2 and remove the brackets and then rearrange as a quadratic in x
Apply the condition for this to have repeated roots
After some symplifying this gives b^2+c^2-(a^2m^2)=0
Put back in what m and c stood for to get the required result