This section features one video on the topic of Hilbert Spaces and Hilbert bases. A Hilbert space is a vector space (usually infinite dimensional) with an inner product, and with a topology that comes from that inner product. (Technically, it is a complete normed vector space, a.k.a. Banach space, with the norm coming from an inner product.) By "topology" I mean that we have a notion on limits and convergence.

If $\{{\bf x_1, x_2, \ldots}\}$ is a sequence of vectors in a Hilbert space $\mathcal{H}$, we say that the sequence converges to a vector ${\bf y}$ if (and only if) $|{\bf y - x_i}| \to 0$, or equivalently if $$\langle {\bf y - x_i | y - x_i} \rangle \to 0.$$ Once we understand convergence, we can talk about infinite sums and infinite linear combinations. As with numbers, an infinite sum $\sum_{n=1}^\infty c_n {\bf b}_n$ of vectors is the limit of a finite sum $$ \sum_{n=1}^\infty c_n {\bf b}_n = \lim_{N \to \infty} \sum_{n=1}^N c_n {\bf b}_n,$$ as long as the sum converges. The sum converges absolutely if $\sum |c_n| |{\bf b}_n|$ converges, and there's a theorem (that we won't prove) that absolute convergence implies convergence.

Now that we're allowed to take infinite linear combinations, we can use them to expand vectors using an infinite basis. A Hilbert basis is a set $\{{\bf b_1, b_2, \ldots}\}$ of vectors in our Hilbert space $\mathcal{H}$ such that

1. Every vector in $\mathcal{H}$ can be written as a linear combination $\sum_{n=1}^\infty c_n {\bf b_n}$, and
2. The only way to write $0 = \sum c_n {\bf b_n}$ is if all of the coefficients $c_n$ are zero.

The three examples are:

1. $\ell_2$ is the space of infinite sequences ${\bf x} = (x_1, x_2, x_3, \ldots, )$ of complex numbers such that $\sum |x_n|^2$ converges. The inner product is $\langle {\bf x} | {\bf y} \rangle = \sum_{j=1}^\infty \bar x_j y_j$. This is like the standard inner product on $\mathbb{C}^n$, only with infinite sums instead of finite sums. A Hilbert basis consists of ${\bf e_1} = (1,0,0,\ldots)$, ${\bf e_2} = (0,1,0,0,\ldots)$, etc.
2. $L^2(\mathbb{R})$ is the space of all reasonable functions $f(t)$ on the real line such that $\int_{-\infty}^\infty |f(t)|^2 dt$ converges. (We'll ignore the technical questions of what sorts of functions qualify as reasonable. See the book for a more careful treatment.) The inner product is $\langle {\bf f} | {\bf g} \rangle = \int_{-\infty}^\infty \overline{f(t)} g(t) dt.$ It turns out that the functions $t^n e^{-t^2/2}$, where $n$ ranges from 0 to $\infty$, form a Hilbert basis. (This is not obvious, and is related to finding the energy levels of a harmonic oscillator in quantum mechanics.)
3. $L^2([0,L])$ is like $L^2(\mathbb{R})$, except that we only consider functions on the interval $[0,L]$ and do our integrals from $t=0$ to $t=L$. It turns out that the functions $\xi_n(t) = \sin(n \pi t/L)$ form an orthogonal Hilbert basis for this space. Expanding vectors with respect to this basis is called Fourier Series, and is the topic of our next section.