Bit lost here, tried all sorts of converting but can't quite work it out, just need a push in the right direction.
the square outside the brackets is confusing me.\
If you cannot solve that simple quadratic equation, you have no business trying to solve this question.
As for the use of LaTeX:
Why not learn to post in symbols? You can use LaTeX tags
[tex] a^2-3a=10[/tex] gives