Why is an orthogonal basis important?
Because it is particularly easy to write any vector in terms of an orthogonal basis (even easier if it's orthonormal). Here's an application: thinking of the vector space of functions, you can solve differential equations formed by means of, say, an Hermitian operator, by writing down the solution as a sum of eigenvectors of the operator. Physicists do this kind of thing all the time to solve ODE's or even PDE's.
Another application is diagonalization. Suppose you have the first-order system of DE's: . A solution is . But how do you compute Well, if is diagonalizable (you can find an orthogonal matrix such that for some diagonal matrix ), then which is much easier to compute. And the process of diagonalization, or finding the orthogonal matrix , is a process of finding an orthonormal (which is automatically orthogonal) basis.
If a basis, is orthonormal (all vectors are orthogonal and have length 1), then the vector v can be written as . That is, the coefficients of the vector is just the inner product of v with .
If the basis is orthogonal but the vectors are not of unit length, the coefficient is .
Also, if is an orthogonal basis of an euclidean space and , then, the inner product can be expressed by . If is an orthonormal basis, that is, the inner product works as in the usual case.
By the Gram-Schmidt Theorem always is possible to find an orthonormal basis of . This means that (technically but nor conceptually) things work as if "there is only one inner product" in .