I've done some more work on the problem and have reduced it further. I believe this is a notational problem on my end, as I have two books for the class and the notation is inconsistent.
I am given that the "squared norm" is as given above. How do I extract from that the inner product?
I assume for a vector v = (x, y, z), ||v||^2 = x^2 + 2xy + 4y^2 + 8z^2 = <v, v>, but how would I define this for different vectors <v, u>?
I hope I'm being clear. Thanks for the help!