The diagonal of the rectangle will be the diameter and I believe the shape is a square
We can always set up a coordinate system so that the origin is at the center of the circle and the x-axis is parallel to one side of the rectangle. In that coordinate system, the circle has equation for some R (the radius of the circle). If we call the corner of the square in the first quadrant , then, because the vertex lies oh the circle, so .
The lengths of the sides of the rectangle are, by symmetry, just twice those x and y values so the area of the rectangle is . Replace by x and find the value of x that maximizes .