Originally Posted by
apcalculus Ellen:
How much progress have you made with this problem?
Let x and y be the sides.
x will be the side of the rectangle along the triangle side of length 5, and y will be the side along the triangle side of length 12.
Identify the right triangle (lower right if side of length 5 is the base) with legs (5-x) and y. This triangle is similar to the big right triangle, so the following ratio is valid:
$\displaystyle \frac{y}{12} = \frac{5-x}{5}$
Solve for y:
$\displaystyle y = 12 - \frac{12x}{5}$ (*)
The objective function is the area of the rectangle:
$\displaystyle A = x y$
Write this area as a single variable function by using substitution (*) above, then apply differential calculus techniques to optimize.
I hope this helps.
Good luck!