So, I think the most important thing for you to do at this point is to choose angles to input for theta (I'm hoping inside the parentheses, you are supposed to have theta and not x). Once you pic the angles, I suggest picking an (x,y) that are easy to see.

For example, for theta=pi and (1,2)=(x,y), you have...

[cos(pi)*1-sin(pi)*2]=[x']

[sin(pi)*1-cos(pi)*2]=[y']

That's what happens under a 180 degree angle. (Which isn't a terrible amount of an angle.) What about if theta is 45 degrees? 90 degrees? 135 degrees?