this has nothing to do with differential geometry (we're not considering differentiability of anything). where you're going wrong is in thinking about which direction L is going. L is perpendicular to the hyperplane H, in physics, one often hears the term "normal vector (at 0)" to describe L. in other words, L is coming "straight up" (perpendicular) out of the plane. for example, if H was the xy-plane in euclidean 3-space, then L would be the z-axis.

Plane + perpendicular line = 3-space.

actually, we could use ANY line, not lying in the plane H but then we lose orthogonality. and orthogonality makes the arithmetic easier (it makes our basis for V look like the standard basis "rotated" to line up with H and L).

in other words, one way to describe a point in 3-space, is to pick a point in a plane (any plane will do, imagine it being the xy-plane "tilted" by some amount), and to pick a distance "up off the plane" (how far along the line L we've come).