Coordinate-Exchange Simulated Annealing Algorithm for … · Lulu Kang, Department of Applied...
1
Motivations Coordinate-Exchange Simulated Annealing Algorithm for Constrained Optimal Design Lulu Kang, Department of Applied Mathematics, Illinois Institute of Technology, Chicago, IL Coordinate-Exchange SA Algorithm Coordinate-Exchange Algorithm Set the tuning parameters of SA algorithm T 0 , β, and Tε. Construct the initial design D. Compute the row deletion function d r (x i ) for each row x i and column deletion function d c (X j ) for each column X j . Random choose the row i* and column j* with probabilities proportional to d r (x i ) and d c (X j ). Denote the delta function Δ(x i*, j* , x) as the measurement of improvement of the design criterion by exchanging the coordinate x i*, j* with another value x. Solve the optimization problem: Min or Max Δ(x i*, j* , x) s.t. x in Ω j* (x i* ), where Ω j* (x i* ) is the projected constraints for x=x i* . Deonte the optimal solution as x*. • Physical Experiments: mixture experiments. (Cornell, 2002) DESIGN-EXPERT Plot StdErr of Design Design Points X1 = A: A X2 = B: B X3 = C: C A: A 0.85 B: B 0.65 C: C 0.60 0.05 0.10 0.30 StdErr of Design 2 2 2 2 2 2 2 2 Linear Constraints only! • Computer Experiments 2 4 6 8 10 0 5 10 15 20 v t Temperature E.g. Nonlinear Constrain for the Mathematical Model for the Low Pressure Chamber. More Complex Constraints! Literature on Constrained DOE • Physical Experiments: o Mainly are mixture experiments (Cornell 2002) o Montgomery et al. (2002): bond strength of adhesive o Hung et al. (2010) and Hung (2011) • Computer Experiments: o Sasena et al. (2002): applied optimization o Trosset (1999) and Stinstra et al (2003) o Draguljie, Dean, and Santner (2012) All methods are for linear constraints only! Can work for nonlinear Constraints. But need advanced NLP solver! Can work for nonlinear Constraints. But need to generate huge candidate set! Not accessible or Inefficient! • Finding the optimal coordinate is a one-dimension optimization problem, easier and less computational! • No sophisticated NLP solvers are needed. • Coordinate-exchange method can handle broader range of constraints. • Fix all the other p-1 dimensional variable value and vary only one variable value. • If the p-dimensional constraints are convex, then the projected one- dimension feasible region becomes a single-interval set. • If the p-dimensional design space are non-convex or disconnected, then the one-dimension feasible region contains multiple intervals. • Exchanging one coordinate of the design leads to less computation in updating the objective function. Simulated Annealing (SA) Algorithm • Randomize the selection of coordinate D i,j to exchange. • Always accept improvement, but also accept the setback with a probability. • Simulate the metal cooling process, in order to avoid local optimum. • Initial temperature T 0 : large, specified according to the setting of the design. • Rate of temperature decreasing β between [0.96, 0.99]. Exchange the optimal solution x* with x i*, j* according to the probability π=min{1, exp(-Δ(x i*, j* , x) /T}. Then update all the deletion functions and design matrix. Update the temperature TTβ. Iterate Step 1-3 until T<Tε. Return the current design D as the optimal design with the optimal criterion ψ(D). D: the design matrix of size n-by-d; Ω is the d-dimension constrained space; x i is the design point in Ω; ψ(.) is design criterion. (a) r e Frequency 0.935 0.945 0.955 0.965 0 10 20 30 40 (b) r e Frequency 0.970 0.980 0 5 10 15 20 25 30 (c) r e Frequency 0.9 1.0 1.1 1.2 1.3 0 5 10 15 20 25 Histogram of efficiencies of optimal designs returned by proposed method w.r.t. the ones returned by JMP. (a) D-optimal (b) A-optimal (c) ϕ p space filling design. (with box constraints only!). Three design-criterion • D-optimal: ψ(D)=det(F T F) • Linear-optimal: ψ(D)=tr(M(F T F) -1 ) • ϕ p space filling: ψ(D)=ϕ p (D)= 0 @ n X i,j =1 d(x i , x j ) -p 1 A 1/p Need to derive the delta function, deletion functions, and updating formula for different criteria. Examples • Convex Constraint: x 2 1 + x 2 2 1 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 x 1 x 2 4 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 x 1 x 2 6 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 x 1 x 2 D-optimal A-optimal Space Filling • Non-convex Constraint: (0.7x 1 ) 2 +(x 2 - 0.5) 2 ≥ 0.25 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x 1 x 2 6 6 6 6 5 6 6 9 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x 1 x 2 6 6 6 6 5 6 6 9 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x 1 x 2
Transcript of Coordinate-Exchange Simulated Annealing Algorithm for … · Lulu Kang, Department of Applied...
Motivations
Coordinate-Exchange Simulated Annealing Algorithm for Constrained Optimal Design
Lulu Kang, Department of Applied Mathematics, Illinois Institute of Technology, Chicago, IL
Coordinate-Exchange SA Algorithm
Coordinate-Exchange Algorithm
Set the tuning parameters of SA algorithm T0, β, and Tε. Construct the initial design D.
Compute the row deletion function dr(xi) for each row xi and column deletion function dc(Xj) for each column Xj. Random choose the row i* and column j* with probabilities proportional to dr(xi) and dc(Xj).
Denote the delta function Δ(xi*, j*, x) as the measurement of improvement of the design criterion by exchanging the coordinate xi*, j* with another value x. Solve the optimization problem:
Min or Max Δ(xi*, j*, x) s.t. x in Ωj*(xi*),
where Ωj*(xi*) is the projected constraints for x=xi*. Deonte the optimal solution as x*.
• Physical Experiments: mixture experiments. (Cornell, 2002)
DESIGN-EXPERT Plot
StdErr of DesignDesign Points
X1 = A: AX2 = B: BX3 = C: C
A: A0.85
B: B0.65
C: C0.60
0.05 0.10
0.30
StdErr of Design
22
22 22
22
Linear Constraints
only!
• Computer Experiments
2 4 6 8 10
05
1015
20
vt
Temperature
E.g. Nonlinear Constrain for the Mathematical Model for the Low Pressure Chamber.
More Complex Constraints!
Literature on Constrained DOE • Physical Experiments:
o Mainly are mixture experiments (Cornell 2002)
o Montgomery et al. (2002): bond strength of adhesive
o Hung et al. (2010) and Hung (2011)
• Computer Experiments:
o Sasena et al. (2002): applied optimization
o Trosset (1999) and Stinstra et al (2003)
o Draguljie, Dean, and Santner (2012)
All methods are for linear
constraints only!
Can work for nonlinear Constraints. But need advanced NLP solver!
Can work for nonlinear Constraints. But need to generate huge candidate
set! Not accessible or Inefficient!
• Finding the optimal coordinate is a one-dimension optimization problem, easier and less computational!
• No sophisticated NLP solvers are needed. • Coordinate-exchange method can handle broader range of constraints.
• Fix all the other p-1 dimensional variable value and vary only one variable value.
• If the p-dimensional constraints are convex, then the projected one-dimension feasible region becomes a single-interval set.
• If the p-dimensional design space are non-convex or disconnected, then the one-dimension feasible region contains multiple intervals.
• Exchanging one coordinate of the design leads to less computation in updating the objective function.
Simulated Annealing (SA) Algorithm • Randomize the selection of coordinate Di,j to exchange. • Always accept improvement, but also accept the setback with a
probability. • Simulate the metal cooling process, in order to avoid local optimum. • Initial temperature T0: large, specified according to the setting of the
design. • Rate of temperature decreasing β between [0.96, 0.99].
Exchange the optimal solution x* with xi*, j* according to the probability π=min{1, exp(-Δ(xi*, j*, x) /T}. Then update all the deletion functions and design matrix. Update the temperature TçTβ.
Iterate Step 1-3 until T<Tε. Return the current design D as the optimal design with the optimal criterion ψ(D).
D: the design matrix of size n-by-d; Ω is the d-dimension constrained space; xi is the design point in Ω; ψ(.) is design criterion.
(a)
re
Frequency
0.935 0.945 0.955 0.965
010
2030
40
(b)
re
Frequency
0.970 0.980
05
1015
2025
30
(c)
re
Frequency
0.9 1.0 1.1 1.2 1.3
05
1015
2025
Histogram of efficiencies of optimal designs returned by proposed method w.r.t. the ones returned by JMP. (a) D-optimal (b) A-optimal (c) ϕp space filling design. (with box constraints only!).
Three design-criterion • D-optimal: ψ(D)=det(FTF) • Linear-optimal: ψ(D)=tr(M(FTF)-1)
• ϕp space filling: ψ(D)=ϕp(D)=
0
@nX
i,j=1
d(xi,xj)�p
1
A1/p
Need to derive the delta function, deletion functions, and updating formula for different criteria. Examples
• Convex Constraint: x
21 + x
22 1
-1.0 -0.5 0.0 0.5 1.0
-1.0
-0.5
0.0
0.5
1.0
x1
x 2 4
-1.0 -0.5 0.0 0.5 1.0
-1.0
-0.5
0.0
0.5
1.0
x1
x 2 6
-1.0 -0.5 0.0 0.5 1.0
-1.0
-0.5
0.0
0.5
1.0
x1
x 2
D-optimal A-optimal Space Filling
• Non-convex Constraint: (0.7x1)2 + (x2 � 0.5)2 � 0.25
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
x1
x 2
6 6 6
6
566
9
0.0 0.2 0.4 0.6 0.8 1.00.0
0.2
0.4
0.6
0.8
1.0
x1
x 26 6 6
6
566
9
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
x1
x 2
