First, you need and to have the same sign in order to be at an extremum. If they have opposite signs, then you are looking at a function that is convex in one variable and concave in the other. So, if , then you already know you're at a saddle point. And, the determinant of the Hessian will clearly be negative.
Second, you need to be larger than . If you view and as indicators of what will happen to the function if you perturb x or y alone by a small amount, then is an indicator of what will happen if you perturb both variables simultaneously by a small amount. To elaborate, implies that if both x and y move in the same direction, the function will tend to increase, and if they move in opposite directions, the function will tend to decrease. The opposite is true if . This is bad as far as hoping for an extremum is concerned, because at an extremum the function should either increase or decrease (but not both) in every direction. But, if the effects are small compared to the and effects, you can still be looking at an extremum. Hence, you want the determinant of the Hessian to be positive.
This is, of course, all assuming you are already at a point where .