How can you determine the type of critical point
f(x,y)=y*x^2+x*y^2
Has at (0,0)?
There is a version of the "second derivative test" that works for functions of two variables. The analog of df/dx for functions of two variables is the matrix
If the determinant of that matrix (at the given (x,y)) is positive then either:
1) if the point gives a minimum.
2) if the point gives a maximum.
If the determinant is negative, then the point gives a saddle point.
If the determinant is 0, the "second derivative test" won't give you an answer and, having written all of that, I now notice that is the case!
So, instead, look what happens on lines through (0, 0). On the line x= 0, f(0, y)= 0 for all y and similarly, on y= 0, f(x, 0)= 0. But on the line y= x, the graph goes up for x positive and down for x negative. That gives you the answer.
When trying different lines, you are looking for "counter-examples" and counter examples can only prove something is NOT true.
Your original example gave lines that went both up and down- the "down" part was a counter example to "minimum" and the "up" part was a counter example to "maximum"- and that left only "saddle point".
If you try different lines and find that they give increasing values, that is a counter example to "maximum" but it does not follow that it must be a minimum- there might be other lines that you did not try on which the function decreases- it could still be either a maximum or a minimum. But situations in which the "second derivative test" does not apply (the determinant is 0) are rare. What do you get when you apply that test here?