
3D prism problem
The cross section of a right prism is an equilateral triangle. The rectangular face ABCD of the prism lies on the plane $\displaystyle x+y+5z=\sqrt{2}$, where A and B are the points $\displaystyle (0,\sqrt{2},0)$ and $\displaystyle (\sqrt{2},0,0)$ respectively. EF is the edge in which the other two rectangular faces ADEF and BCEF meet.
Prove that the equation of the plane containing A, B, F is
$\displaystyle 5x+5y2z=5\sqrt{2}$.
If the origin lies inside the prism, determine the equations of the line EF.
I have proven the first part, but I can't get the equations of EF.
F(x,y,z)
AF=BF from this x=y
AB=AF=BF=2
Perpendicular distance of F from plane ABCD=$\displaystyle \sqrt 3=AF(\cos \theta)$
Where $\displaystyle \theta$ is the angle of elevation of F from A.
$\displaystyle \cos\theta=[(\frac{1}{3\sqrt 3})(\frac{x\sqrt 2}{\surd (x^22\sqrt 2x+2+y^2+z^2)})+(\frac{1}{3\sqrt 3})(\frac{y}{\surd (x^22\sqrt 2x+2+y^2+z^2)})+(\frac{5}{3\sqrt 3})(\frac{z}{\surd(x^22\sqrt 2x+2+y^2+z^2)})]$
Fill in the values for the perpendicular distance.
Then we have $\displaystyle 9=\frac{2x2\sqrt 2+2y+10z}{\surd (x^22\sqrt 2x+2+y^2+z^2)}$
BF=$\displaystyle \surd (x^22\sqrt 2x+2+y^2+z^2)=2$
So
$\displaystyle 9=\frac{2x2\sqrt 2+2y+10z}{2}$
$\displaystyle 9=x\sqrt 2+y+5z$
$\displaystyle 5x+5y=45+5\sqrt 225z=5\sqrt 2+2z$ from the plane ABF
$\displaystyle z=\frac{5}{3}$
now x=y
$\displaystyle 2x=9+\sqrt 25(\frac{5}{3})$
$\displaystyle x=y=\frac{1}{3}+\frac{\sqrt 2}{2}$
So $\displaystyle F(\frac{1}{3}+\frac{\sqrt 2}{2}, \frac{1}{3}+\frac{\sqrt 2}{2}, \frac{5}{3})$
The direction ratios of the line are 5:5:2 since it is perpendicular to the plane ABF
the equations for the line I arrive at are
$\displaystyle 2x=2y=5z9\sqrt 2$
the answer is supposed to be $\displaystyle 2x=2y=5z+9\sqrt 2$
I can't find where I'm wrong.
Thanks!

A unit vector perpendicular to ABCD is $\displaystyle \frac1{3\sqrt3}(1,1,5)$. The midpoint of AB is $\displaystyle P = \bigl(\frac{\sqrt2}2,\frac{\sqrt2}2,0\bigr)$. The point F is at a distance $\displaystyle \sqrt3$ from P along this vector. Therefore F is the point $\displaystyle \bigl(\frac{\sqrt2}2\pm\frac13,\frac{\sqrt2}2\pm\f rac13,\pm\frac53\bigr)$. Notice the plus/minus signs. They are there because F could be on either side of the plane ABCD. But we are told that the origin lies inside the prism. Therefore F and the origin must lie in the same halfspace $\displaystyle x+y+5z<\sqrt{2}$ or $\displaystyle x+y+5z>\sqrt{2}$. The origin clearly lies in the first of these, hence so does F, which means that we must take the minus signs. Thus $\displaystyle F = \bigl(\frac{\sqrt2}2\frac13,\frac{\sqrt2}2\frac13,\frac53\bigr)$.
That should account for the change of sign from –9 to 9 in the final result.