You can do with or without calculus.
Without calculus method:
Let the lengths of the folded up sides be x.
Then the base will have width 26-2x
The area of the gutter is
Now you have a quadratic to solve.
To find the maximum capacity of the gutter, use x = -b/(2a).