Thread: Finding best hyperplane through points?

1. Finding best hyperplane through points?

Hi!

If you have points $\overline{x}_1,\ ...\ ,\ \overline{x}_n$ in $\Bbb{R}^d$, how can you find the "best fitting" hyperplane $\pi$ so that $\sum_{k=1}^n d\left(\pi,\ \overline{x}_k\right)^2$ gets as small as possible?

$d\left(\pi,\ \overline{x}_k\right)$ is the distance from the hyperplane $\pi$ to the k:th point. If the equation of $\pi$ is $\overline{a}\cdot\overline{x} = b$, where $\overline{a}$ is the normal of the hyperplane, then the distance is defined as

$d\left(\pi,\ \overline{x}\right)\ =\ \frac{\left|\overline{a}\cdot\overline{x} - b\right|}{\left|\overline{a}\right|}$

2. Alternatively, I would like to get help finding a vector $\overline{a}$ so that the sum $\sum_{k=1}^n\ \left(\frac{\overline{a}\cdot\overline{x}_k}{\left |\overline{a}\right|}\right)^2$ is maximized, where $\sum_{k=1}^n \ \overline{x}_k\ =\ \overline{0}$. The vectors $\overline{a}$ and $\overline{x}_k$ are still in $\Bbb{R}^d$.

3. Ok, both problems are relevant when finding a transformation matrix $A$ for a random vector $Z$, so you can describe a random vector $X$ by the formula

$X=AZ+\mu$

$X$ has the dimension d, and so does $Z$. Each element in $X$ is normally distributed, but not independent of each other. $Z$ on the other hand, has it's elements normal distributed around 0 with the standard derivation 1, and independently of each other. $A$ is a $d\times d$ matrix with its columns pairwise orthogonal to each other, of which some could be $\overline{0}$. $\mu$ is the average of the random vectors and is simple to find for a set of sample random $X$ vectors, the difficulty is to find $A$. The distribution of $X$ is called multivariate normal distribution.

If I, for example have about 100 points in a three-dimensional room, I may want to find the direction in which the points differ the most and how much they may differ in that direction. Do you have any idea of how to find that direction? Or maybe the transformation matrix $A$ from the problem above?

Any suggestions would be appreciated.