I'm surprised no one else commented on this. I think that's a nice result. My guess is that its probably well known, but still, its a cool result.
I tried proving that every quadratic residue had an inverse like you said, by some kind of counting argument, but I wasn't getting anywhere. Instead I noticed that since the group of units mod p is an abelian group, the function
f(a) = a^(-1)
is an isomorphism. Then if a is a quad residue, let b be such that b^2 == a mod p. then
f(a) = f(b^2) = f(b)^2 = a^(-1)
So that a^(-1) is congruent to f(b)^2, i.e. b^(-2).