Thread: How to get the solution of this simple optimization problem?

1. How to get the solution of this simple optimization problem?

How to get the solution of this simple optimization problem?

$
minimize~~\Phi = x_1^2 + x_2^2
$

$
subject~to~~x_1 +2x_2 = 5
$

I konw we may use Moore-Penrose inverse like,

$
Ax = b => x = (A^TA)^{-1}A^Tb
$

but what are $A$ and $x$ now?

if $A = (1~~2)$ and $A^T = (1~~2)^T$ then there is no inverse of $A^TA$. I am quite confused how to formulate these $A$ and $x$ to do the matrix multiplication.

Thanks a lot

2. Originally Posted by ggyyree
How to get the solution of this simple optimization problem?

$
minimize~~\Phi = x_1^2 + x_2^2
$

$
subject~to~~x_1 +2x_2 = 5
$

I konw we may use Moore-Penrose inverse like,

$
Ax = b => x = (A^TA)^{-1}A^Tb
$

but what are $A$ and $x$ now?

if $A = (1~~2)$ and $A^T = (1~~2)^T$ then there is no inverse of $A^TA$. I am quite confused how to formulate these $A$ and $x$ to do the matrix multiplication.

Thanks a lot
"Moore-Penrose" is much too advanced for me ("Moore" isn't so bad but I cringe when I hear "Penrose") so I would do it with Lagrange multipliers (I much better with "Lagrange").

You want to minimize $F(x,y)= x^2+ y^2$ subject to the constraint G(x,y)= x+ 2y= 5. The gradients are $\nabla F= 2x\vec{i}+ 2y\vec{j}$ and $\nabla G= \vec{i}+ 2\vec{j}$. Now we need to find (x,y) so that $2x\vec{i}+ 2y\vec{j}= \lambda(\vec{i}+ 2\vec{j})$. That means we must have $2x= \lambda$ and $2y= 2\lambda$. Dividing the second equation by the first give $\frac{y}{x}= 2$ or y= 2x. Putting that into the constraint x+ 2y= 5, x+ 2(2y)= x+ 4x= 5x= 5 or x= 1, y= 2.

3. Originally Posted by ggyyree
How to get the solution of this simple optimization problem?

$
minimize~~\Phi = x_1^2 + x_2^2
$

$
subject~to~~x_1 +2x_2 = 5
$
Why not solve the constraint for $x_1$ in terms of $x_2$ and substitute into the objective to reduce this to a 1-D optimisation??

$x_1=5-2x_2$, then:

$
\Phi=5x_2^2-20x_2+25
$

CB