Wolfe’s Method

Jamie Haddock

April 6, 2017

Graduate Group in Applied Mathematics

UC Davis

Minimum Norm Point in Polytope

We are interested in solving the problem (MNP(P)):

min x∈P ||x ||2

where P is a polytope.

Reminder: A polytope, P, is the convex hull of points p1, p2, ..., pm,

P =

{ m∑ i=1

λipi : m∑ i=1

λi = 1, λi ≥ 0 for all i = 1, 2, ...,m } .

1

Minimum Norm Point in Polytope

We are interested in solving the problem (MNP(P)):

min x∈P ||x ||2

where P is a polytope.

Reminder: A polytope, P, is the convex hull of points p1, p2, ..., pm,

P =

{ m∑ i=1

λipi : m∑ i=1

λi = 1, λi ≥ 0 for all i = 1, 2, ...,m } .

1

Minimum Norm Point in Polytope

p1 p2

p3

p4

p5

P

O

2

Minimum Norm Point in Polytope

p1 p2

p3

p4

p5

P

O

2

Formulation as QP

This is a quadratic programming problem (QP).

Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m. Then our problem is which can be solved using interior-point methods.

3

Formulation as QP

This is a quadratic programming problem (QP).

Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m.

Then our problem is which can be solved using interior-point methods.

3

Formulation as QP

This is a quadratic programming problem (QP).

Given p1, p2, ..., pm ∈ Rn, define P = [p1p2...pm] ∈ Rn×m. Then our problem is

min ||Py ||22 s.t. 1T y = 1

y ≥ 0

which can be solved using interior-point methods.

3

Optimality Condition

Theorem (Wolfe)

Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if

xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.

4

Optimality Condition

Theorem (Wolfe)

Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if

xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.

p1 p2

p3

p4

p5

P

O

4

Optimality Condition

Theorem (Wolfe)

Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if

xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.

p1 p2

p3

p4

p5

P

O

4

Optimality Condition

Theorem (Wolfe)

Let P = {p1, p2, ..., pm}. Then x ∈ conv(P) is MNP(P) if and only if

xTpj ≥ ||x ||22 for all j = 1, 2, ...,m.

p1 p2

p3

p4

p5

P

O

{x : pT4 x = ||p4||22}

4

Intuition and Definitions

Idea: Exploit linear information about the problem in order to progress

towards the nonlinear (quadratic) solution.

Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral if MNP(conv(Q)) ∈ relint(conv(Q)).

q1 q2

O

q1 q2

q3

O

Note: Singletons are corrals.

Note: There is a corral of points in P whose convex hull contains

MNP(P).

5

Intuition and Definitions

Idea: Exploit linear information about the problem in order to progress

towards the nonlinear (quadratic) solution.

Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral if MNP(conv(Q)) ∈ relint(conv(Q)).

q1 q2

O

q1 q2

q3

O

Note: Singletons are corrals.

Note: There is a corral of points in P whose convex hull contains

MNP(P).

5

Intuition and Definitions

Idea: Exploit linear information about the problem in order to progress

towards the nonlinear (quadratic) solution.

Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral if MNP(conv(Q)) ∈ relint(conv(Q)).

q1 q2

O

q1 q2

q3

O

Note: Singletons are corrals.

Note: There is a corral of points in P whose convex hull contains

MNP(P).

5

Intuition and Definitions

Idea: Exploit linear information about the problem in order to progress

towards the nonlinear (quadratic) solution.

Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral if MNP(conv(Q)) ∈ relint(conv(Q)).

q1 q2

O

q1 q2

q3

O

Note: Singletons are corrals.

Note: There is a corral of points in P whose convex hull contains

MNP(P).

5

Intuition and Definitions

Idea: Exploit linear information about the problem in order to progress

towards the nonlinear (quadratic) solution.

Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral if MNP(conv(Q)) ∈ relint(conv(Q)).

q1 q2

O

q1 q2

q3

O

Note: Singletons are corrals.

Note: There is a corral of points in P whose convex hull contains

MNP(P).

5

Intuition and Definitions

Idea: Exploit linear information about the problem in order to progress

towards the nonlinear (quadratic) solution.

Def: An affinely independent set of points Q = {q1, q2, ..., qk} is a corral if MNP(conv(Q)) ∈ relint(conv(Q)).

q1 q2

O

q1 q2

q3

O

Note: Singletons are corrals.

Note: There is a corral of points in P whose convex hull contains

MNP(P). 5

Sketch of Method

x ∈ P = {p1, p2, ..., pm} C = {x} while x is not MNP(P)

pj ∈ {p ∈ P : xTp < ||x ||22} C = C ∪ {pj} y = MNP(aff(C ))

while y 6∈ conv(C ) z = argmin

z∈conv(C)∩xy ||z − y ||2

C = C − {pi} where pi , z are on different faces of

conv(C )

x = z

y = MNP(aff(C ))

x = y

return x

p1 = (0, 2)

p2 = (3, 0)

p3 = (−2, 1)

p1

p3

p2

O

P

6

Sketch of Method

x ∈ P = {p1,p2, ...,pm} C = {x} while x is not MNP(P)

pj ∈ {p ∈ P : xTp < ||x ||22} C = C ∪ {pj} y = MNP(aff(C ))

while y 6∈ conv(C ) z = argmin

z∈conv(C)∩xy ||z − y ||2

C = C − {pi} where pi , z are on different faces of

conv(C )

x = z

y = MNP(aff(C ))

x = y

return x

p1 = (0, 2)

p2 = (3, 0)

p3 = (−2, 1)

p1 = x

p3

p2

O

P

6

Sketch of Method

x ∈ P = {p1, p2, ..., pm} C = {x} while x is not MNP(P)

pj ∈ {p ∈ P : xTp < ||x ||22} C = C ∪ {pj} y = MNP(aff(C ))

while y 6∈ conv(C ) z = argmin

z∈conv(C)∩xy ||z − y ||2

C = C − {pi} where pi , z are on different faces of

conv(C )

x = z

y = MNP(aff(C ))

x = y

return x

p1 = (0, 2)

p2 = (3, 0)

p3 = (−2, 1)

p1 = x

p3

p2

O

P

C = {p1}

6

Sketch of Method

x ∈ P = {p1, p2, ..., pm} C = {x} while x is not MNP(P)

pj ∈ {p ∈ P : xTp < ||x ||22} C = C ∪ {pj} y = MNP(aff(C ))

while y 6∈ conv(C ) z = argmin

z∈conv(C)∩xy ||z − y ||2

C = C − {pi} where pi , z are on different faces of

conv(C )

x = z

y = MNP(aff(C ))

x = y

return x

p1 = (0, 2)

p2 = (3, 0)

p3 = (−2, 1)

p1 = x

p3

p2

O

P

C = {p1}

6

Sketch of Method

x ∈ P = {p1, p2, ..., pm} C = {x} while x is not MNP(P)

pj ∈ {p ∈ P : xTp < ||x||22} C = C ∪ {pj} y = MNP(aff(C ))

while y 6∈ conv(C ) z = argmin

z∈conv(C)∩xy ||z − y ||2

C = C − {pi} where pi , z are on different faces of

conv(C )

x = z

y = MNP(aff(C ))

x = y

return x

p1 = (0, 2)

p2 = (3, 0)

p3 = (−2, 1)

p1 = x

p3

p2

O

P

C = {p1}

6

Sketch of Method

x ∈ P = {p1, p2, ..., pm} C = {x} while x is not MNP(P)

pj ∈ {p ∈ P : xTp < ||x ||22} C = C∪ {pj} y = MNP(aff(C ))

while y 6∈ conv(C ) z = argmin

z∈conv(C)∩xy ||z − y ||2

C = C − {pi} where pi , z are on different faces of

conv(C )

x = z

y = MNP(aff(C ))

x = y

return x

p1 = (0, 2)

p2 = (3, 0)

p3 = (−2, 1)

p1 = x

p3

p2

O

P

C = {p1,p2}

6

Sketch of Method

x ∈ P = {p1, p2, ..., pm} C = {x} while x is not MNP(P)

pj ∈ {p ∈ P : xTp < ||x ||22} C = C ∪ {pj} y = MNP(aff(C))

while y 6∈ conv(C ) z = argmin

z∈conv(C)∩xy ||z − y ||2

C = C − {pi} where pi , z are on different faces of

conv(C )

x = z

y = MNP(aff(C ))

x = y

return x

p1 = (0, 2)

p2 = (3, 0)

p3 = (−2, 1)

p1 = x

p3

p2

O

P y

C = {p1, p2}

6

Sketch of Method