Well you are solving the system of equations
y=2x-2a+b
x=y
so just substitute to get
but then you have x=y, so this is also the y coordinate
This gives you the system you have.
P' is the point of intersection of the line y=x and the line of slope 2 that passes through P.
[Sol] Letting (a,b) and (x',y') be the coordinates of P and P', respectively,
Calculating the equation of the line of slope 2 that passes through P.
y-b=2(x-a)
y=2x-2a+b
Since the lines y=x and y=2x-2a+b intersect at P'(x',y'),
x'=2a-b
y'=2a-b (That's my problem, how do you get x' and y'?)
I didn't read the title of this, if you want them to be solved as like a matrix equation, you need to first put the system in standard form.
so you have
y=2x-2a+b
x=y
This yields
-2x+y=-2a+b
x-y =0
Which is the same as
So then you can solve this by row reduction, or finding the inverse of this matrix, or whatever method you want, you will get the same solution as above.