Find the image of the first quadrant under the mapping
It's an interesting problem and I'd like to try:
I believe the image can be determined by considering both values of the square root in the integrand and using an analytic branch of the multi-valued antiderivative where we can use the principal-valued logarithm and square root. Below are the sheets of the integrand and note in the first quadrant (Q), the sheets have the same sign. Both sheets are either negative or positive so consider the expansion:
so that the imaginary part of the integral will be positive if the integration is over the blue branch or negative if the integration is over the red branch. This then splits the image into two separate rectangular regions extending to which I've colored the same as the corresponding branch. The single vertical lines on either side of the images represent the two values of the integral for real and greater than one. Perhaps though I should explain more how the rectangular regions are obtained in terms of the logarithm expression.
Not sure but that's what I think it is.
Your approach is valid, but probably more work than necessary. Why not look at the path which the image point describes as the point travels along the boundary of ? The image of will then be either the inside or the outside of the domain delimited by that path.
You might not even require the antiderivative!