Real inner products are generalizations of the familiar dot product
for ${\bf R}^3$ and ${\bf R}^2$. So before we launch into inner products in
general, let's review the dot product.
By the Pythagorean Theorem, the length of a vector
${\bf x} = x_1 {\bf e_1} + x_2 {\bf e_2}
+ x_3 {\bf e_3}$ is $$|{\bf x}| =\sqrt{x_1^2 + x_2^2 + x_3^2}.$$
The dot product of two vectors ${\bf x}$ and ${\bf y}$ is
defined to be
$$ {\bf x} \cdot {\bf y} = |{\bf x}| \, |{\bf y}|\, \cos(\theta),$$
where $\theta$ is the angle between the two vectors. The motivation
for this formula, and the use of dot products to get projections,
is explained in the following video. (Note: This video was made for M408M,
so the notation is slightly different than we are using in M346.)
It's not hard to show that the dot product has four useful properties:
$(a {\bf x} + b{\bf y}) \cdot {\bf z} = a ({\bf x} \cdot {\bf z}) + b ({\bf y} \cdot {\bf z})$.
That is, the dot product is a linear function of the first factor, when we hold
the second factor fixed.
${\bf x} \cdot (a {\bf y} + b{\bf z}) = a ({\bf x} \cdot {\bf y}) + b ({\bf x} \cdot {\bf z})$.
That is, the dot product is a linear function of the second factor, when we hold
the first factor fixed.
${\bf x} \cdot {\bf y} = {\bf y} \cdot {\bf x}$. Unlike the cross product,
the dot product is symmetric.
If ${\bf x} \ne 0$, then ${\bf x} \cdot {\bf x} > 0$. This is called
positivity. In fact, ${\bf x} \cdot {\bf x} = |{\bf x}|^2$, since the
angle between ${\bf x}$ and itself is zero.
From these properties, we derive a very simple formula for the dot product. Since
${\bf e_i} \cdot {\bf e}_j$ equals 1 if $i=j$ and 0 if $i \ne j$ (insofar as $\cos(\theta)$
is either 1 or 0), we have
\begin{eqnarray*} {\bf x} \cdot {\bf y} & = & (x_1 {\bf e_1} + x_2 {\bf e_2} + x_3 {\bf e_3}) \cdot
(y_1 {\bf e_1} + y_2 {\bf e_2} + y_3 {\bf e_3}) \cr
&=& \phantom{+} x_1 y_1 {\bf e_1} \cdot {\bf e_1} + x_1 y_2 {\bf e_1} \cdot {\bf e_2} +
x_1 y_3 {\bf e_1} \cdot {\bf e_3} \cr
&& + x_2 y_1 {\bf e_2} \cdot {\bf e_1} + x_2 y_2 {\bf e_2} \cdot {\bf e_2} +
x_2 y_3 {\bf e_1} \cdot {\bf e_3} \cr
&& + x_3 y_1 {\bf e_3} \cdot {\bf e_1} + x_3 y_2 {\bf e_3} \cdot {\bf e_2} +
x_2 y_3 {\bf e_3} \cdot {\bf e_3} \cr
& = & x_1 y_1 + x_2 y_2 + x_3 y_3.
\end{eqnarray*}
Now let's move on to inner products on arbitrary real vector spaces. Let
$V$ be a real vector space. If ${\bf x, y}$ are vectors in $V$, we write
$\langle {\bf x}| {\bf y} \rangle$ for the inner product of ${\bf x}$
and ${\bf y}$. (This bracket notation is standard in physics, but in many
math texts you'll see the notation $({\bf x}, {\bf y})$ instead.) We assume
the the function $\langle \; | \; \rangle: V \times V \to {\bf R}$ satisfies
four axioms for arbitrary vectors ${\bf x,y}$ and arbitrary scalars $a,b$:
$ \langle a {\bf x} + b{\bf y} | {\bf z} \rangle = a \langle {\bf x} | {\bf z}\rangle
+ b \langle{\bf y} | {\bf z}\rangle$.
This is called linearity in the first factor.
$\langle {\bf x} |a {\bf y} + b{\bf z} \rangle = a \langle {\bf x} | {\bf y}\rangle + b \langle{\bf x} | {\bf z}\rangle$.
This is called linearity in the second factor.
$\langle {\bf x} | {\bf y}\rangle = \langle {\bf y} | {\bf x}\rangle$.
This is called symmetry.
If ${\bf x} \ne 0$, then $\langle {\bf x} | {\bf x} \rangle > 0$. This is called
positivity. We then define $|{\bf x}|$ to be
$\sqrt{\langle {\bf x}|{\bf x} \rangle}$.
The most common examples are:
$V = {\bf R}^n$ and $\langle {\bf x} | {\bf y} \rangle = {\bf x}^T {\bf y} = \sum_i x_i y_i$.
This is called the standard inner product on ${\bf R}^n$. There are plenty of
other inner products that satisfy the axioms, but this is the simplest.
$V$ is a space of functions, say on the interval $[0,1]$, and
$\langle f | g \rangle = \int_0^1 f(t) g(t) dt$. This is called the $L^2$ inner
product. It comes up a lot in quantum mechanics.
$V$ is a space of $n \times m$ matrices, and $\langle A | B \rangle = Tr(A^T B)$.
Note that the standard inner product on ${\bf R}^n$ is a special case of this with
$m=1$.
No matter what $V$ is, and no matter what the inner product is, we can derive
certain properties from the axioms. The most important is the Schwarz inequality:
$$|\langle {\bf x}| {\bf y} \rangle | \le |{\bf x}|\, |{\bf y}|.$$
This allows us to define the angle between two vectors to be the angle
whose cosine is $\frac{\langle {\bf x}|{\bf y}\rangle}{|{\bf x}|\, |{\bf y}|}$.
This brings us back, full circle, to $|{\bf x}|\, |{\bf y}| \cos(\theta)$.
Only now we can speak about the angle between two functions, or between two
matrices, as well as the angle between two vectors in ${\bf R}^3$.
Real inner products and the Schwarz inequality are explained
in the following video: