Math 5593 Linear Programming Homework 2

Unit II: Theory and Foundations (Convex Analysis)

University of Colorado Denver, Fall 2013

Short Solutions and Proofs

1. Let S and T be two convex sets in Rn, and let α and β be two real numbers. Prove:

(a) The set αS + βT := {x ∈ Rn : x = αx1 + βx2, x1 ∈ S, x2 ∈ T} is convex.

(b) The set S ∩ T is convex.

Proof. For (a), we need to show that for any x, y ∈ αS+ βT and 0 ≤ λ ≤ 1, we have λx+ (1−λ)y ∈ αS+ βT .Hence, let x, y, λ be given so that x = αx1 + βx2 and y = αy1 + βy2 for some x1, y1 ∈ S and x2, y2 ∈ T . Then

λx+ (1− λ)y = λ(αx1 + βx2) + (1− λ)(αy1 + βy2)= α(λx1 + (1− λ)y1) + β(λx2 + (1− λ)y2)

where λx1+(1−λ)y1 ∈ S and λx2+(1−λ)y2 ∈ T because S and T are convex, and thus λx+(1−λ)y ∈ αS+βT .

For (b), pick any two points in S ∩ T . By definition of set intersection, these points are also in S and T and soare all their convex combinations, because S and T are convex. In particular, then they are also in S ∩ T .

2. Prove that the convex hull of a set S in Rn is the (smallest) set that contains all convex combinations of pointsin S:

conv(S) =

{z ∈ Rn : z =

k∑i=1

αizi, αi ≥ 0,k∑

i=1

αi = 1, zi ∈ S

}.

Proof. Some you found result and proof in the book by Vanderbei (Theorem 10.2). That’s enough for me!

3. Prove that a cone K is convex if and only if K +K ⊆ K.

Proof. First, let K be a convex cone and let x ∈ K + K, i.e., x = k1 + k2 for some k1, k2 ∈ K. BecauseK is a cone, we also have (1/2)k1, (1/2)k2 ∈ K, and because K is convex and 1/2 + 1/2 = 1, it follows that(1/2)(k1 + k2) ∈ K and thus 2(1/2)(k1 + k2) = k1 + k2 ∈ K again because K is a cone. Thus K +K ⊆ K.

Next, let K be a cone such that K +K ⊆ K, and let k1, k2 ∈ K and 0 ≤ λ ≤ 1. Because K is a cone, we alsohave λk1, (1− λ)k2 ∈ K, and because K +K ⊆ K, it follows that λk1 + (1− λ)k2 ∈ K. Thus K is convex.

4. Prove that a set is convex if and only if it is the intersection of all its supporting halfspaces, under suitableassumptions.

Proof. We will prove that a closed set S is convex if and only if it is the intersection of all its (closed) supportinghalfspaces. For convenience, let us denote this intersection by T . First, if S is convex, then it is clear that Sis contained in each (and thus the intersection of all its) supporting halfspaces, i.e., S ⊆ T . To show that alsoT ⊆ S, suppose to the contrary that there exists x ∈ T \S and consider the singleton {x}, which can be strictlyseparated from S using a supporting hyperplane or halfspace because S is closed and convex. This contradictsthat T is the intersection of all supporting halfspaces of S, and thus S = T . Then, for the second direction, itsuffices to observe that every halfspace is convex and that the intersection of convex sets is again convex.

5. Give a general definition for unboundedness of a set, and prove that a convex set S is unbounded if and onlyif there exists a halfline in S:

x = y + αd ∈ S for some y ∈ S, d ∈ Rn \ {0} and all α ≥ 0

Definition of Bounded Sets in a Metric Space (favorite class choice): A set S in a metric space (V, d) is boundedif it is contained in a ball of finite radius, i.e., if there exists x ∈ V and r > 0 such that for all x ∈ S, we haved(x, s) < r. V is called a bounded metric space (or d is a bounded metric) if V is bounded as a subset of itself.

Math 5593 LP Homework 2 Solutions (Unit II: Theory & Foundations), UC Denver, Fall 2013 2

Proof. Unlike the majority of you who used concepts of boundedness or unboundedness in terms of distancesor norms in a metric or normed space, the following proof uses the more general concept of boundedness ina topological vector space (due to von Neumann): a set S is said to be bounded if for every neighborhood Naround zero, there exists a scalar α ≥ 0 so that S ⊆ αN ; otherwise it is said to be unbounded (draw a picture!).

For our proof, then let S be convex and first suppose that S contains a halfline x = y + αd. To show that Sis unbounded, we will show that there exists a neighborhood N around zero such that for any α ≥ 0, we canfind β ≥ 0 such that x(β) := y + βd /∈ αN . In fact, we can show this for any neighborhood N , so let N andα ≥ 0 be given arbitrarily. If y /∈ αN , then we can choose β = 0 and are done. Otherwise, if y ∈ αN , then wecan consider the set αN −{y} := {αx− y : x ∈ N} which is another neighborhood around zero. It follows thatthere exists a (sufficiently large) scalar β ≥ 0 such that βd /∈ αN − {y} (by the concept of neighborhood) andthus y + βd /∈ αN . Note that this direction does not use that S is convex and thus also holds for general sets.

Next, to show that an unbounded convex set contains a halfline, we will consider the contrapositive and provethat a convex set S without a halfline must be bounded, i.e., that for every neighborhood N around zero thereis α ≥ 0 such that S ⊆ αN . Hence, let N be given arbitrarily and S be convex without a halfline, i.e., forevery y ∈ S and d ∈ N \ {0} there be a (sufficiently large) β = β(y, d) > 0 (dependent on y and d) such thaty + βd /∈ S. In particular, because S is convex, we see that also y + γd /∈ S for any γ ≥ β(y, d) (otherwise wecould write y+αd = (β/γ)(y+γd)+(1−β/γ)y ∈ S because also y ∈ S and 0 ≤ β/γ ≤ 1 – this is where we needthat S is convex, but what’s really needed here is a picture!). It follows that we can choose y ∈ S arbitrarily,and α ≥ 0 sufficiently large such that y, y + βd ∈ αN for all d ∈ N and thus S ⊆ {y + βd : d ∈ N} ⊆ αN .

6. Prove the Fundamental Theorem of LP: Let A ∈ Rm×n with rank(A) = n ≤ m, b ∈ Rm, and c ∈ Rn be givenand consider the linear optimization problem

(P) minimize cTx subject to Ax ≥ b.(a) If there is a feasible point for (P), then there is a feasible vertex.(b) If there is an optimal point for (P), then there is an optimal vertex.

Proof. If rank(A) = n ≤ m, then the feasible region S = {x ∈ Rn : Ax ≥ b} is a bounded convex polyhedron (orpolytope, based on your preferred terminology). Now we can apply the representation theorem for convex sets:every point in a bounded convex set is a convex combination of the extreme points of the set, which we provedin class to be equivalent to the vertices if the underlying convex set is polyhedral (i.e., an intersection of finitelymany halfspaces). Hence, for (a), feasibility of (P) is equivalent with nonemptyness of S which immediatelyimplies existence of an extreme point or feasible vertex. Similar, for (b), every optimal point x∗ ∈ S for (P)is either an optimal vertex itself, or it can be written as (strict) convex combination of two or more (optimal)vertices xi with λi > 0, i.e., x∗ =

∑ki=1 λixi where cTx∗ = cTxi for all i = 1, . . . , k because x∗ is optimal.

7. Prove that a feasible point x is optimal for (P) if and only if there exists no feasible descent directions at xwith respect to c.

Proof. First, if x∗ ∈ S is optimal for (P), then cTx∗ ≤ cTx for all x ∈ S and thus cT d ≥ 0 for all feasibledirections d = α(x− x∗) with α ≥ 0. Second, if x∗ ∈ S was not optimal for (P), then there would exist x ∈ Swith cTx < cTx∗ and thus d = α(x− x∗) would be a feasible descent direction at x∗ for c and any α ≥ 0.

8. Prove Fredholm’s Theorem of the Alternative (Problem 3 on the June 2011 Linear Algebra Prelim): ForA ∈ Rm×n a matrix and b ∈ Rm a vector, exactly one of the following two systems has a solution:

(I) Ax = b (II) AT y = 0, bT y 6= 0.

Proof. We convert the Fredholm systems (I) and (II) to equivalent Farkas systems of the form (FI) Ax = b andx ≥ 0; and (FII) AT y ≥ 0 and bT y < 0, respectively. For (I), it suffices to introduce two auxiliary variables and

write x = x1−x2 so that (I) Ax = b is equivalent to (FI)[A −A

] [x1

x2

]= b and (x1, x2)T ≥ 0. The dual Farkas

system to (FI) is then given by (FII)[AT

−AT

]y ≥ 0 and bT y < 0, or equivalently for the former, by AT y ≥ 0

and AT y ≤ 0 and thus AT y = 0. Moreover, if AT y = 0 and bT y < 0, then AT (−y) = 0 and bT (−y) > 0 andvice versa, and thus bT y 6= 0 is analogous to yT b < 0. Hence, the result now follows from Farkas Lemma.

Please let me know if you find any mistakes or if you have any other general comments or questions!