I assume you've found the equation of the tangent plane correctly, although you didn't explicitly write it down (I don't know the convention you used when you wrote "finding L(x, y) at point (a,b)").

The equation of the tangent plane at (a,b) is:

, or

From that, you can read off that the plane is normal to the vector (6a, -8b, -1). So far, so good, if I understood you.

Now you tried to set (6a, -8b, -1) = (3, 2, 2) and solve for a and b. That was your mistake. Look at the z-component for instance. There's no solution on your reasoning.

Here's what's happening: If a plane is normal to a vector , then it's also normal to , and , and .

That's because to be normal to a plane means that the vector defined by the difference between any two points in the plane is perpendicular to . And if that's perpendicularity holds with , then it holds with , and , and . It holds with any (non-zero) multiple of !!!

So it's not that your normal vector, (6a, -8b, -1) at (a, b), and the problems normal vector, (3, 2, 2), should be "equal", but rather they should represent the same direction, so that one should be a (real number) multiple of the other.

So what you should've done is this:

Find (a, b) such that for some .

Then , so

, so

the 3rd equation gives , so .