Mixed Integer Nonlinear Programming

IMA New Directions Short Course on Mathematical OptimizationJeff Linderoth and Jim Luedtke

Department of Industrial and Systems EngineeringUniversity of Wisconsin-Madison

August 12, 2016

Introduction

Mixed-Integer Nonlinear Optimization

Mixed Integer Nonlinear Program: (MINLP)

zminlp = minx

subject to gi (x) 0, i = 1, . . . ,mf (x) x X , xI Z|I |

f : Rn R, g : Rn Rm smooth, sometimes convex functions.X Rn bounded, polyhedral set, e.g. X = {x : l AT x u}I {1, . . . , n} subset of integer variablesWe may assume that f is a linear function.

NP-Super-Hard

Combines challenges of handling nonlinearitieswith combinatorial explosion of integer variables

Introduction

The MINLP Family

The great watershed inoptimization isnt betweenlinearity and nonlinearity,but convexity andnonconvexity.

- R. Tyrrell Rockafellar

If f and g are convex functions,then we have a convex MINLP

If f and g are not convex, thenwe have a nonconvex MINLP

Optimization 101

Know the convexity properties of your instance!

Algorithms for two different classes of problems are different

We can solve significantly larger convex MINLP instances thatnonconvex MINLP instances

Algorithms designed for convex-MINLP are often used as heuristicsfor nonconvex MINLPs

Introduction

Specializations

Functional Form Problem Type

gi (x) = Aix + bi2 pTi x + qi MISOCPgi (x) = polynomial MIPP

gi (x) = xTQix + p

Ti + qi MIQP

gi (x) = aTi x bi MILP

MISOCP: Mixed Integer Second Order Cone ProgramAre convex-MINLP

MIPP: Mixed Integer Polynomial ProgramMIQP: Mixed Integer Quadratic Program

May be convex or nonconvexConvex MIQP is a special case of MISOCPIf f is convex quadratic and c is an affine mapping, then there arespecialized algorithms for convex-MIQP

MILP: Mixed Integer Linear Program

Introduction

Aside MISOCP

The feasible regions of surprisingly many MINLP problems can beexpressed as MISOCP:

Hyperbolic constraints: Product of (non-negative) variables anorm2.

wTw xy , x 0, y 0 [ 2wx y

] x + yConvex Quadratic: ci (x) = x

TQix + pTi x + qi t

Qi is positive semidefinite, so it has Cholesky factorization Qi = LTL

y = Lx g(x) = yT y + pTi x + qi t

{(x , t) Rn+1 | xTQx + pTi x + qi t} =Proj(x ,t){(x , y , t, p) R2n+2 | y = Lx , p + qT x + r t, p y2}

Introduction

MINLP Tree

MINLP

NonconvexMINLP

ConvexMINLP

NonconvexNLP

ConvexNLP

PolynomialOpt.

MI-Polynomial

Opt.MISOCP SOCP

NonconvexQP

NonconvexMIQP

ConvexMIQP

Convex QP

MILP LP

Introduction

How Big Is It?

The Leyffer-Linderoth-Luedtke (LLL) Measure of Complexity

You have a problem of class X that has Y decision variables? What is thelargest value of Y for which one of Sven, Jim, and Jeff would be willingto bet $50 that a state-of-the-art solver could solve the problem?

Convex NonconvexProblem Class (X ) # Var (Y ) Problem Class (X ) # Var (Y )

MINLP 500 MINLP 100NLP 5 104 NLP 100

MISOCP 1000 MIPP 150SOCP 105 PP 150MIQP 1000 MIQP 300

QP 5 105 QP 300MILP 2 104

LP 5 107

Introduction Software Tools and Online Resources

Solvers for MINLP

Convex Nonconvex-ECP -BBBONMIN ANTIGONEDICOPT BARONFilMINT COCONUTGUROBI COUENNEKNITRO CPLEXMILANO LGOMINLPBB LindoGlobalMINOPT SCIPMINOTAURSBBXpressMIQP MIQP/convex MIQP

Open-source and commercial solvers

Introduction Software Tools and Online Resources

Software for MINLP

Convex MINLP

ALPHA-ECP, Bonmin, DICOPT, FilMINT, KNITRO, MINLP-BB,SBB

(Convex) MIQP

CPLEX, GUROBI, MOSEK, XPRESS

(Nonconvex) MIQP

GLOMIQO

General Global Optimization

BARON, Couenne, LINDOGlobal, SCIP

Try em on NEOS

http://www.neos-server.org/neos/solvers/index.html

Introduction Basic Concepts

Simple Example

(x1 2)2 + (x2 + 1)2 363x1 x2 6

2.5(x1 1)2 x2 40 x1, x2 5

x2 Z

x1

x2

Doubly NP-Hard!

Feasible region is not convex for two reasonsIntegrality of x2 andnonconvexity of constraint

Introduction Basic Concepts

Q: What to Do?!

Relax!

Remove some constraints.Solution of relaxed problem provide a lower bound on optimal solutionvalue

Introduction Basic Concepts

Natural Relaxation

Natural relaxation is toremove constraints xi Z i IBut this still leaves a nonconvexNLP relaxation

x1

x2

Introduction Basic Concepts

Relax More

We need a mechanism forrelaxing the nonconvexconstraints.

Secant underestimator forconcave functions

Easy to get polyhedralouterapproximation ofnonconvex constraints

Ideally, we would like to get theconvex hull of the feasible points

x1

x2

Introduction Basic Concepts

Linear Objective Is Important Here!

min(x1 1/2)2 + (x2 1/2)2

s.t. x1 {0, 1}, x2 {0, 1}

(x1 1/2)2 + (x2 1/2)2

x1

x2

(x1, x2)

Without linear objective, optimal solution may be interior to theconvex hull convexifying may do you no good!

Introduction Basic Concepts

Algorithm Ingredients

1 Relaxation

Used to compute a lower bound onthe optimum valueObtained by enlarging feasible set; e.g.ignore constraintsShould be tractable and tight

2 Constraint Enforcement

Exclude solutions from relaxations not feasible to MINLPRefine or tighten of relaxationBranching and Cutting

3 Upper Bounds

Obtained from any feasible point; e.g. solve NLP for fixed xI .(NLP(xI )).Other heuristic techniques have been developed

Modeling and Applications

Models are Important!

Writing good models is extremelyimportant in MILP

Probably more important for MINLP

More Complexity

Tradeoff between physical accuracy and mathematical tractability

Convex reformulations? Linearize expressions?

MINLP Modeling Preference

We want strong formulationsRelaxations that can be relaxed tosomething that closely approximates the convex hull

We prefer linear over convex over nonconvex formulations

Modeling and Applications

Some MINLP Applications

1 Supply ChainConvex MINLPDiscrete: Fixed charges for opening facilitiesNonlinear: Nonlinear transportation costs

2 Gas/Water Network DesignNonconvex MINLPDiscrete: Pipe connections/sizesNonlinear: Pressure Loss

3 Pooling/PetrochemicalNonconvex MINLPDiscrete: Which process to use?Nonlinear: Product Blending (among others)

Luedtke Theorem

97.5% of all practical MINLPs have integer variables for one of twopurposes...

1 Binary variables to make a multiple choice selection2 Binary indicator variables that turn on/off continuous variables

and/or constraints.

Modeling and Applications Supply Chain

Uncapacitated Facility Location

Problem introduced by Gunluk, Lee,and Weismantel (07) and classes ofstrong cutting planes derived

M: Facility

N: Customer

xij : percentage of customer j N demand met by facility i Mzi = 1 facility i M is builtFixed cost for opening facility i MQuadratic cost for meeting demand j N from facility i M

Modeling and Applications Supply Chain

Quadratic Uncapacitated Facility Location Problem

A very simple MIQP

zdef= min

iM

cizi +iM

jN

qijx2ij

subject to

xij zi i M,j NiM

xij = 1 j N

xij 0 i M, j Nzi {0, 1} i M

Modeling and Applications Supply Chain

Base Case of Induction to Luedtke Theorem

Note that binary variables are used as indicators: zi = 0 xij = 0If z = 1, then we need to model the epigraph of x2ij

Intager Programmerz r Smrt

Mixed Integer Linear Programmerscarefully study simple problemstructures to come up withgood formulations for problems

Good formulations closelyapproximate convex hull offeasible solutions

Important to understand thestructure of a special MINLP withindicator variables

Modeling and Applications Supply Chain

A Very Simple Structure

Rdef={

(x , y , z) R2 B | y x2, 0 x uz}

z = 0 x = 0, y 0z = 1 x u, y x2

x

y

z = 1

z

y x2

Deep Insights

conv(R) line connecting (0, 0, 0) to y = x2 in the z = 1 plane

Modeling and Applications Supply Chain

Characterization of Convex Hull

Deep Theorem #1

R ={

(x , y , z) R2 B | y x2, 0 x uz}

conv(R) ={

(x , y , z) R3 | yz x2, 0 x uz , 0 z 1, y 0}

x2 yz , y , z 0

Second Order Cone Programming

There are effective, robust algorithms for optimizing over conv(R)

Modeling and Applications Supply Chain

In GeneralGiving You Some Perspective

For a convex function f : Rn R, the perspective functionP : Rn+1 R of f is

P(x , z) def={

0 if z = 0zf (x/z) if z > 0

The epigraph of P(x , z) is a cone pointed at the origin whose lowershape is f (x)

Exploiting Your Perspective

If zi is an indicator that the (nonlinear, convex) inequalityf (x) 0 must hold, (otherwise x = 0), replace the inequality withits perspective version:

zi f (x/zi ) 0

The resulting (convex) inequality is a much tighter relaxation ofthe feasible region.

Modeling and Applications Supply Chain

Perspectivizing Quadratic Uncapacitated Facility Location

zdef= min

iM

cizi +iM

jN

qijx2ijyij