Yes, you should take the gradient vector but you are taking the gradient of the wrong function. You want to take the gradient of the function F(x,y,z)= z- f(x,y)= z- 5y^2+ 2x^2+ xy= 0. The gradient of a function is always perpendicular to a "level curve" of the function.