I think you have to look at infinite dimensional spaces as a separate case. The area that looks at this is known as infinite dimensional vector spaces, or Hilbert Spaces as they are known. In addition to this you look at operator algebras on Hilbert-spaces which include more areas including Von-Neumann and C* algebras.
The infinite dimensional cases when you first study them look at all these individual cases where you look at various kinds of convergence. Basically its the same kind of thing that you study when you look at infinite series, except in this case you are looking at inner products of infinite dimensional vectors. When you have infinity, the intuition is obviously different.
I took a survey course on this stuff many years ago, but I do know that this is the area that you want to look at to get more and specific information. I'm not a pure mathematician (in fact I'm very bad at the higher pure stuff), but I do know that this field will help you.
You should look at infinite dimensional spaces in terms of the spectrum which is an eigen-value type analysis but in the infinite-dimensional space (its also used in the finite case but the infinite dimensional analysis looks at in more general and advanced ways).