Well I hope it's not too late for your test but here goes
a) First thing I'd do is develop f so that the derivatives become easier, thus
So from this we see that f'(x,y)=0, so f is a critical point. However |f''(0,0)|=0, so the second derivative test fails.
b) let y=ax^2, thus f(x,y) becomes:
So, if a>2 or a<1, f obviously has a min. at x=0 (y is also 0 when x is 0). However if 1<a<2 then f has a max at x=0. Thus, f(x,y) has neither a max or min at (0,0)
b) let y=mx, where m is any real number. This is the family of straight lines through (0,0). f then becomes:
So, f'(0)=0, so it's a critical point, as we'd expect. Also, we have that
This is positive, and thus f has a min, for all values of m except for zero. If m=0, then,
In this case f obviously has a min at x=0. The only other case to examine is the straight line x=0, in that case, f is simply f(y)=y^2, which also has an obvious min at (0,0). Thus, f has a min at (0,0) along any straight line passing through (0,0).
Hope that helps.