I don't understand how to do these transformations as depicted on the picture.
There's 2R = pi/3 and R/2=0 How to get these pi/3 and 0?
There are a number of typographical errors in your post (at a glance, there's two dx's and a missing term in the transformation Jacobian).
From what I can tell, the region of integration is the upper half of the circle $\displaystyle (x - R)^2 + y^2 = R^2$ between the line x = R/2 and the y-axis. Do you realise this?
A transformation is then made to polar coordinates. Do you realise that? Are you familiar with them?
Do you know how to describe the region of integration using polar coordinates? In particular:
1. Do you see how to express the line x = R/2 and the circle in polar coordinates.
2. Do you see how to get the integral terminals for the angle?
$\displaystyle x = \frac{R}{2} \Rightarrow r \cos \phi = \frac{R}{2} \Rightarrow r = \frac{R}{2 \cos \phi}$.
$\displaystyle y = \sqrt{2Rx - x^2} \Rightarrow y^2 = 2 R x - x^2$
$\displaystyle \Rightarrow r^2 \sin^2 \phi = 2 R r \cos \phi - r^2 \cos^2 \phi \Rightarrow r = 2 R \cos \phi$. (r = 0 is rejected - why?)
At the point (2R, 0) $\displaystyle \phi = 0$.
The line $\displaystyle x = \frac{R}{2}$ cuts the semi-circle at $\displaystyle \left( \frac{R}{2}, ~ \frac{R \sqrt{3}}{2}\right)$. At this point, $\displaystyle \tan \phi = \frac{R \sqrt{3}/2}{R/2} = \sqrt{3} \Rightarrow \phi = \frac{\pi}{3}$.