One way to do this:

1) translate so that R is the origin by subtracting R from each vector.

P becomes and Q becomes

2) Rotate through by 2.3 radians by multiplying those vectors by the matrix .

3) Translatebackby adding R to each vector.

As for "homogeneous coordinates" do you know what they are? We represent the point (x, y) as the three-vector (x, y, 1) with the understanding that if, after some transformation, the last component is not 1, we divide through by it. That is, (x, y, a) represents the same point as (x/a, y/a, 1). In homogeneous coordinates, a translation of (x, y) by (a, b) is given by the matrix multiplication while rotation through angle is given by the matrix multiplication .

Find the matrices corresponding to the translation of R to the origin, rotation by 2.3 radians, and translation back and multiply them (in the correct order).