1. Draw a sketch. Let x denote the base of the ladder and y the overhang.

2. Use proportions:

3. Use Pythagorean theorem:

4. Solve for y. That means: Calculate the overhang with respect to the base. You'll get a positive and a negative result. The negative result isn't very plausible here. Thus:

5. Calculate the first derivative (it's easier (for me!) to use product rule and chain rule):

6. Solve for x: y' = 0. I've got