You can do with or without calculus.

Without calculus method:

Let the lengths of the folded up sides be x.

Then the base will have width 26-2x

The area of the gutter is

Now you have a quadratic to solve.

To find the maximum capacity of the gutter, use x = -b/(2a).