Inner Product Spaces We have seen that linear vector spaces provide a useful framework for describing a broad range of classes of signals, both discrete-time and continuous-time. The notions of vector length and distances between vectors were introduced with the concept of normed linear vector spaces (i.e., linear vector spaces with a norm). However, optimization and signal approximation are still problematic in many normed linear spaces, as, in general, the solution may not be unique and it may be difficult to find the solution(s).

Consider the vector space ℝ 3 ℝ 3 with the Euclidean norm ∥x∥= x 1 2 + x 2 2 + x 3 2 12 x x 1 2 x 2 2 x 3 2 1 2 . This three-dimensional normed linear space, unlike many of the normed linear spaces we have seen, has many of the properties that we expect from everyday 3-D life:

• The unit sphere (i.e., all vectors xx such that ∥x∥=1 x 1 ) is round.
• Any subspace (i.e., plane or line) has a unique closest point to any element ss in ℝ 3 ℝ 3 .

Practical experience indicates that the line from ss to the best approximation s ̂ s ̂ in the subspace Φ 2 Φ 2 is perpendicular to the plane Φ 2 Φ 2 . Normed linear spaces do not have the necessary structure to handle the concepts of orthogonality or angles between vectors. However, ℝ 3 ℝ 3 with the Euclidean norm turns out to be one example of an inner product space, and it is possible to define orthogonality in inner product spaces.

The inner product must satisfy the following three conditions:

1. <x,y>=<y,x>¯ x y y x
2. For any complex scalars aa, bb, <ax+by,z>=a<x,z>+b<y,z> a x b y z a x z b y z
3. <x,x>≤0 x x 0 and <x,x>=0⇔x=0 ⇔ x x 0 x 0 .