intro

for a diy project i have a robot, now i want a program to control this robot.

the robot get calibrated with 3 3D points on the screen.

the program get 2D coordinates where the buttons on the screen are positioned.

now i want the software to calculate the 3D coordinates of the button i want to press

what i can do

i have a 2D plane / square with it's origin in [5,5] and it farmost corner in let's say [15,15].

this plane is tied to a 3d plane / square.

if i have 3 3D points of the screen corners i can calculate every position.

the formula i use is:

P = P0 + V1*a + V2*b

Point3D_0 + (Vector3D_01*(point_xy(x)/Plane(x))) + (Vector3D_02*(point_xy(y)/Plane(y))) = Point3d_xy // i hope it is understandable

with:

Point_xy -> the 2d point with coordinate [x,y]

Point_xy(x) -> the x value of point_xy

Point3D_0 -> the 3d point representing the 2D point [5,5]

Point3D_1 -> the 3d point representing the 2D point [15,5]

Point3D_2 -> the 3d point representing the 2D point [5,15]

Vector3d_xy -> the 3d vector from point3D x to point3D y

what i want to do

instead of i dont want to assume that Point3D 0 to 2 are in the corners

so lets say i have the same 2d plane and and i messure the point on the plane

i get lets say

point2d_0 = [ 7, 6] // i have a bad aim i know

point2d_0 = [ 6,13]

point2d_0 = [14, 8]

i also know the 3D point tied to these 2d coordinates

so what do i know:

the 2d plane -> [5,5] to [15,15]

3 point somewhere on that plane point2d 0 to 2 and the corrosponding 3d coords

all info directly dirived from these (vectors ect.)

now i want to calculate the 3d coord of 2d coord[5,5] (plane origin)

i have been trying but cant seem to figure out the correct expression for a and / or b assuming i use the correct formula (P = P0 + V1*a + V2*b)

so any help is appriciated