You're not doing it well, revise in your textbook how to multiply two matrices:
To multiply two vectors you're supposed to use the definition of dot product:
with this information try to check .
For the second part a possible way is to use cofactors along the second row, so that you get
and this is 0 if or (use the quadratic formula).