APPENDIX H

Convex Analysis

H.1. Convex sets and cones

Definition H.1. A set C ⊆ Rn is called convex if for any x, y ∈ C, 0 < λ < 1.

λx + (1 − λ)y ∈ C

convex set ������, ���”��”���, ���������� C ����

��, ������������� C �. � Figure H.1 and Figure H.2.

x

y

Figure H.1. convex set

Example H.2. Let b be a nonzero Rn-vector, β ∈ R.

The closed half-spaces

{x ∈ Rn : 〈x, b〉 ≤ β} and {x ∈ R

n : 〈x, b〉 ≥ β}

and the open half-spaces

{x ∈ Rn : 〈x, b〉 < β} and {x ∈ R

n : 〈x, b〉 > β}349

350 H. CONVEX ANALYSIS

x

y

Figure H.2. not convex set

and the hyperplane

{x ∈ Rn : 〈x, b〉 = β}

are convex

(�� R2 � R

3 �����������)

Theorem H.3. The intersecton of an arbitrary collection of convex sets is convex.

Theorem H.4. A closed convex set is the intersection of the closed half-spaces con-

taining it.

��� Figure H.3.

Figure H.3

H.2. CONVEX FUNCTIONS 351

Definition H.5. (1) A set K ⊆ Rn is called cone if it is closed under positive

scalar multiplication, i.e., λx ∈ K when x ∈ K, λ > 0.

(2) A convex cone is a cone which is a convex set.

Example H.6. ��������������. � Figure H.4 � Figure H.5.

Figure H.4. convex cone

Figure H.5. cone, but not a convex cone

Theorem H.7. The intersection of an arbitrary collection of convex cone is a convex

cone.

H.2. Convex functions

Let C be a convex set on Rn.

352 H. CONVEX ANALYSIS

Definition H.8. (1) A function f : C −→ R is said to be convex if the set

{(x, y) : x ∈ C, y ≥ f(x)}

is convex on Rn+1.

(2) A function f : C −→ R is concave if −f is a convex function.

(3) An affine function on C is a function which is finite, convex and concave.

������������� R, �� R ∪ {∞,−∞}. ����, �������

���.

Theorem H.9. The following statements are equivalent.

(1) f : C −→ R is convex.

(2) f(αx + (1 − α)y) ≤ αf(x) + (1 − α)f(y) for all 0 < α < 1, x, y ∈ C.

(3) f(λ1x1 + λ2x2 + · · · + λmxm) ≤ λ1f(x1) + λ2f(x2) + · · · + λmf(xm)

for all λ1, λ2, · · · , λm ≥ 0, λ1 + λ2 + · · · + λm = 1.

������������ convex���,���� Figure H.6����.

Theorem H.10. If C = (a, b) ⊆ R and f : C −→ R is convex, then

(1) f is continuous on C.

(2) f is differentiable at all but at most countably many points.

(������, � ����.)

Theorem H.11. (1) Suppose f : (a, b) −→ R is twice continuously differentiable.

Then f is convex if and only if f ′′(x) ≥ 0 for all x ∈ (a, b).

H.2. CONVEX FUNCTIONS 353

x ycx+(1-c)y

f(x)

f(y)

f(cx+(1-c)y)

cf(x) + (1-c)f(y)

f

Figure H.6

(2) Suppose f : C(⊆ Rn) −→ R is twice continuously differentiable. Then f is convex

if and only if its Hessian matrix (qij(x))1≤i,j≤n with

qij(x) =∂2

∂xi∂xj

f(x1, x2, · · · , xn)

is positive semi-definite for every x ∈ C.

Remark H.12. A matrix A = (aij)1≤i,j≤n is positive semidefinite if xT Ax ≥ 0 for all

x = (x1, x2, · · · , xn)T ∈ Rn.

���� :

det(a11) ≥ 0

det

⎛⎜⎝a11 a12

a21 a22

⎞⎟⎠ ≥ 0

...

354 H. CONVEX ANALYSIS

det

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎝

a11 a12 · · · a1n

a21 a22 · · · a2n

......

. . ....

an1 an2 · · · ann

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎠

≥ 0

H.3. Construction of convex functions

� �� convex functions or convex sets ������� convex function, ���

�����������.

����� : convex functions ��.

Trial 1: f, g : R −→ R : convex?

=⇒ f + g : convex.

Yes, by Theorem 9.

Trial 2: f : R −→ R : convex, c : constant?

=⇒ cf : convex.

Yes, if c > 0.

Trial 3: f, g : R −→ R : convex?

=⇒ f · g : convex.

No, in general, e.g., g(x) = −1

Trial 4: f, g : R −→ R : positive convex?

=⇒ f · g : convex.

No, e.g., f(x) = x2, g(x) = ex, then

(f ◦ g)(x) = (ex − 1)2.

However, since

(f ◦ g)′(x) = 2ex(ex − 1)

(f ◦ g)′′(x) = 2ex(2ex − 1)

(f ◦ g)′′(

ln1

4

)= 2 · 1

4·(

2 · 1

4− 1

)= −1

4< 0

H.3. CONSTRUCTION OF CONVEX FUNCTIONS 355

����������������.

Theorem H.13. Suppose f, g : C(⊆ Rn) −→ R are convex, then f + g and cf (c > 0)

are convex.

Theorem H.14. Suppose g : Rn −→ R is convex, f : R −→ R is increasing and

convex, then f ◦ g is convex.

Proof. Since g is convex, we have

g(αx + (1 − α)y) ≤ αg(x) + (1 − α)g(y)

for all α ∈ (0, 1), and x, y ∈ Rn. Thus,

(f ◦ g)(αx + (1 − α)y) = f(g(αx + (1 − α)y))

≤f : increasing

f(αg(x) + (1 − α)g(y))

≤f : convex

αf(g(x)) + (1 − α)f(g(y))

= α(f ◦ g)(x) + (1 − α)(f ◦ g)(y).

Thus, f ◦ g is convex. �

Theorem H.15. Let A be a convex set on Rn+1, and let

f(x) = inf{xn+1 : (x1, x2, · · · , xn, xn+1) = (x, xn+1) ∈ A}.

Then f is a convex function on Rn+1.

������������, ����������.

Example H.16. � Figure H.7, Figure H.8, Figure H.9.

356 H. CONVEX ANALYSIS

f

Figure H.7. f is a convex function

A

C

Figure H.8. f(x) = g(x), if x ∈ C, and f(x) = +∞, if x ∈ C.

Theorem H.17. f1, f2, · · · , fm : Rn −→ R are convex. Let

f(x) = inf{f1(x1) + f2(x2) + · · · + fm(xm) : x1 + x2 + · · · + xm = x} (H.1)

Then f is a convex function on Rn.

H.3. CONSTRUCTION OF CONVEX FUNCTIONS 357

Figure H.9. f(x) = −∞ for all x ∈ R

Example H.18. (1) Let

f1(x) = x2, f2(x) = (x − 1)2,

then for x1 + x2 = x,

f1(x1) + f2(x2) = x21 + (x2 − 1)2 = x2

1 + (x − x1 − 1)2

the minimum occurs at x1 =1

2(x − 1). This implies that

f(x) =1

2(x − 1)2.

(2) Consider

f1(x) = ex, f2(x) = e−x,

then f(x) = 0.

Theorem H.19. The function f in (H.1) is called infimal convolution (Analogous to

”integral convolution”).