I am working on a project for a tabletop computer application. We are using a magnetic system (Polhemus Liberty - LIBERTY Electromagnetic Motion Tracking System) that returns a 6DOF location of a stylus in this magnetic coordinate system (it's curved). I would like to map from this strange coordinate system to real 3d euclidian space.

Basically, I take a few sample locations on the table to define the size and Z coordinate of the table, and now I need to map from all arbitrary points onto the 2d surface defined by the table (would also be useful to know the height (or Z coord) of this point, because I'd like to know when the pen is touching the table). The brute force method of simply assuming that this magnetic coordinate system is flat and rectangular has blown up in my face, and does not provide me with fine grained enough results.

Any suggestions about how I should tackle this problem?