120
Theory
3.9. Orthogonal Complements and Projection Theorem
By a subspace of a Hubert space H, we mean a vector subspace of H. A subspace of a Hubert space is an inner product space. If we additionally assume that 5 is a closed subspace of H, then 5* is a Hubert space itself, because a closed subspace of a complete normed space is complete.
Definition 3.9.1 (Orthogonal Complement). Let S be a non-empty subset of a Hubert space H. An element x e H is said to be orthogonal to S, denoted by x _L S, if (.v. v) = 0 for every y G S. The set of all elements of H orthogonal to S, denoted by 5X, is called the orthogonal complement of S. In symbols:
SL = {xeH,xLS}.
The orthogonal complement of S± is denoted by S±L = (51)1.
If xLy for every y € H, then x = 0. Thus, HL = {0}. Similarly, {0} = H. Two subsets A and B of a Hubert space are said to be orthogonal if x _L y for every x e A and y e B. This is denoted by A _L B. Note that, if ALB, then ^ n B = {0} or 0.
Theorem 3.9.1. For any subset S of a Hubert space H, the set SL is a closed subspace of H.
Hubert Spaces and Orthonormal Systems
121
Proof.    If a,/1eC and x,y G S\ then
(ax + 0y, z) = a{x, z) + ß(y, z) = 0
for every z G S. Thus, S~ is a vector subspace of H. We next prove that S^ is closed.
Let (x„) G S± and x„ —>■ x for some x G H. From the continuity of the inner product, we have
(x,y) = (  lim xmy ) = lim (x„,y) = 0 for every y G S. This shows that x € SL, and thus S-1 is closed.               D
The preceding theorem says that SJ~ is a Hubert space for any subset S of H. Note that S does not have to be a vector space. Since S _L S1, we have
S n S1 = {0} or 5 n 5X = 0.
Definition 3.9.2 (Convex Sets). A set {/ in a vector space is called convex if for any .t,j g U and a G (0,1) we have ax + (1 — a)j G [/.
Note that a vector subspace is a convex set.
The following theorem, concerning the minimization of the norm, is of fundamental importance in approximation theory.
Theorem 3.9.2 (The Closest Point Property). Let S be a closed convex \iibset of a Hubert space H. For every point x € H there exists a unique point y G S such that
||x->i|=inf||A--r||.                                (3.9.1)
Proof.    Let (v„) be a sequence in S such that
lim \\x - vn\\ = inf j|x - z\\.
Denote d = inf-eS ||.x: - z\\. Since \ (ym + y„) G S, we have II* - 5 {}'m + >'«)I! > d   for all m, n G N. Moreover, by the parallelogram law (3.3.6), we obtain
II>'m - >'J2 = 4ll* - 2 (>'»> + >»)H2 + b'm - }'n\\2 ~ 4||* - \ (jm + >'„)||2
= II(* - }',„) + {x - y„)II2 + ||(* - ym) - {x - >•„)||2
-4||x-i(v,„+>'„)H2 = 2{\\x - y,„f + \\x - y„\\2) - 4\\x - I (ym + y„)\\2.
122
Theory
Since
x - y„
■y„\
4d~,    as m,n —> oo.
and
\\x-k{ym+y„W>d2,
we have \\ym — y„\\~ —> 0, as m,n —> oc. Thus, (y„) is a Cauchy sequence. Since H is complete and 5 is closed, the limit lim„ _>oc y„ = y exists and y e S. From the continuity of the norm we obtain
ll*->'!l
lim y„
lim 11 j\r ■
>»ll
We have proved that there exists a point in S satisfying (3.9.1). It remains to prove the uniqueness. Suppose there is another point y>\ in S satisfying (3.9.1). Then, since \{y + Vi) G S, we have
\\y ~ yi\\- = 4d- - 4 This can only happen if y = yx.
y + )'\
<o.
D
Theorem 3.9.2 gives an existence and uniqueness result which is crucial for optimization problems. However, it does not tell us how to find that optimal point. The characterization of the optimal point in the case of a real Hubert space, stated in the following theorem, is often useful in such problems.
Theorem 3.9.3.    Let S be a closed convex subset of a real Hubert space H, v £ S, and let x e H. Then the following conditions are equivalent:
(a)     ||.r->1|=inf||.v-z||.
(b)     (x - y, z - y) < 0    for all z e S.
Proof.       Let   z e S.   Since   S   is   convex,   Xz + (\ - X)y e S  for   every A e (0,1). Then, by (a), we have
||x - y\\ < \\x - Xz - (1 - A)v|| = |!(.t - y) - X(z - y)\\.
Hence, as H is a real Hubert space, we get
\\x - y f <\\x- y\\2 - 2\{x - y, z - y) + X2\\z - yf,
and consequently,
[x-y,:
>•><
A
■y\
Thus, (b) follows by letting A —► 0.
Hubert Spaces and Orthonormal Systems                                                                     123
Figure 3.3.
Conversely, if x G H and y E S satisfy (b), then for every z e S, we have H* - yf - \\x - z||2 = 2{x - y, z - y) - \\z - y\\2 < 0. Thus, x and y satisfy (a).                                                                          D
is a closed convex subset of M.', then condition (b) has a clear geometrical meaning: The angle between the line through x and y and the line through z and y is always obtuse (see Fig. 3.3).
Theorem 3.9.4 (Orthogonal Projection). If S is a closed subspace of a Hubert space H, then every element x e H has a unique decomposition in the form x = y + z where y e S and z e SL.
Proof.    If x e S, then the obvious decomposition is x = x + 0. Suppose now    that    x £ S.    Let   y    be    the    unique    point    of   S    satisfying ||.v — y\\ = inf„.es \\x — w\\,   as   in   Theorem   3.9.2.   We   will   show   that .v = y + (x — y) is the desired decomposition. If w e S and A G C, then y + Xw e. S and
II* - yf < II* - y - Ani)2 = ||x - j||2 - 23?A(vv, x - y) + |A|2|M|2.
Hence,
-2KA<h>,x->') + |A|2|M|2>0.
If A > 0, then dividing by A and letting A —> 0 gives
%t(w,x-y}<0.                                    (3.9.2)
Similarly, replacing A by —iX (A > 0), dividing by A, and letting A —> 0 yields
$i(w,x-y)<0.                                    (3.9.3)
Since y e S implies —y e S, inequalities (3.9.2) and (3.9.3) hold also with -if instead of w. Therefore (w,x — y) = 0 for every w G S, which means
x-ves1.
124
Theory
To prove the uniqueness note that if x = V\ + zu v\ G S, and z\ e S ,
then >' — >'i £ S and z — zx eS~. Since y — v\ = z{ — z, we must have y - v, = -, - z = 0.                                                                                  D
According to Theorem 3.9.4, every element of H can be uniquely represented as the sum of an element of S and an element of SL. This can be stated symbolically as
H = S®S1.                                  (3.9.4)
We say that H is the direct sum of S and 5X. Equality (3.9.4) is called an orthogonal decomposition of H. Note that the union of a basis of S and a basis of S1" is a basis of H.
Theorem 3.9.2 allows us to define a mapping Ps(x) = y\ where y is as in (3.9.1). Mapping Ps is called the orthogonal projection onto S. Such mappings will be discussed in Section 4.7.
Example 3.9.1. Let H = U'. Figure 3.4 exhibits the geometric meaning of the orthogonal decomposition in U2. Here x G U2, x = y + z, y e S, and z e S1. Note that, if sq is a unit vector in S, then y = {x,sq}so.
Example 3.9.2. If H = M~\ given a plane P, any vector x can be projected onto the plane P. Figure 3.5 illustrates this situation.
Theorem 3.9.5. If S is a closed subspace of a Hubert space H, then s±A- = S.
Proof. If x € S, then for every z e S1 we have (x, z) = 0, which means x G S^. Thus, S c S±JL. To prove that S±A- c S consider an x e S^.
Figure 3.4. Orthogonal decomposition in U~.
Hubert Spaces and Orthonormal Systems
125
Figure 3.5. Orthogonal projection onto a plane.
Since S is closed, x = y + z for some y e S and ; £ ť. In view of the inclusion S c .S11, we have y e S11 and thus z = x — y e SL±, because S— is a vector subspace. But zgS1, so we must have z = 0, which means x = y e S. This shows that S11 C S, completing the proof.          D