What did you try?
Starty by solving and .
I've solved hundreds of these kind of problems and then I came across this one. Well, normally, I'd say that it cannot be solved in closed form, only to realise that it is given to first year undergraduate students. So I must be overseeing something
Find extremal points of f(x,y):
From fx=0, we have ... (1)
which gives ... (2)
From fy=0, we have ... (3)
Putting (1) & (2) in (3) gives :
which can be written as ... (4)
You can expand (4) and solve for x, but to make our life easier, (4) can be written
as
Let t=x+1, or :
... (5)
Can you solve (5) ?