If you draw the right-triangle, with an arbitrary inscribed rectangle,
that has sides against the base and vertical side, settling in the
label the base of the triangle 6, height 8, hypotenuse 10.
The rectangle base is x and the rectangle height y.
Now, observe the smaller identical triangle at the bottom corner.
It has base (6-x) and height y.
Hence we can write an equation
Rectangle area is
in terms of x only, allowing us to differentiate the area and equate the derivative to zero to find the rectangle of maximum area
which equals zero when x=3