Model Reduction (Approximation) of Large-Scale Systems ... ·...

Introduction Operators Balanced truncation Computing the Gramians Summary

Model Reduction (Approximation) of Large-Scale Systems

Truncation and SVD based techniquesLecture 3

C. Poussot-Vassal, P. Vuillemin & I. Pontes Duff

EDSYS, April 4-7th, 2016 (Toulouse, France)

moremoreΣ

(A,B,C,D)i

Σ

Σ

(A, B, C, D)i

model reduction toolbox

Kr(A,B)

AP + PAT + BBT = 0

WTV

DAE/ODE

State x(t) ∈ Rn, n large orinfinite

Data

ReducedDAE/ODE

Reduced state x(t) ∈ Rrwith r � n(+) Simulation(+) Analysis(+) Control(+) Optimization

Case 1u(f) = [u(f1) . . . u(fi)]y(f) = [y(f1) . . . y(fi)]

Case 2Ex(t) = Ax(t) +Bu(t)y(t) = Cx(t) +Du(t)

Case 3H(s) = e−τs

C. Poussot-Vassal, P. Vuillemin & I. Pontes Duff [Onera - DCSD] Model Reduction (Approximation) of Large-Scale Systems


IntroductionRealization based approximation methods overviewProjecion-based problem

OperatorsHankel Singular ValuesSystem Gramians

Balanced truncationBalanced realizationThe balanced truncation (BT)The singular perturbation approximation (SPA)Illustration and limitations

Computing the GramiansNumerical aspectsDense Lyapunov solvers (exact)Sparse Lyapunov solvers (approximate)

Summary



Outlines

IntroductionRealization based approximation methods overviewProjecion-based problem

Operators

Balanced truncation

Computing the Gramians

Summary



IntroductionThe big picture


DAE/ODE


Data

ReducedDAE/ODE






IntroductionRealization based approximation methods overview

Realization-based ap-proximation methods

Non-projection methods

Linear systems• LMIs• Vector fitting• Complex optimization (DARPO)

Projection-based methods

Linear systems (truncation)• SVD (BT, SPA)• Modal truncation (SDPA)• Hankel approximation (H)

Linear systems (iterative)• Rational Interpolation (IRKA)• Tangential (ITIA)

Linear systems (mixed)• Krylov/SVD (ISRKA, ISTIA)• Krylov/Eigenvectors (IETIA)

Non linear systems• POD• Empirical Gramians



IntroductionProjecion-based problem

Let H : C→ Cny×nu be a nu inputs ny outputs, complex-valued function describinga LTI dynamical system as a DAE of order n, with realization H:

H :{

Ex(t) = Ax(t) +Bu(t)y(t) = Cx(t) (1)

the approximation problem consists in finding V,W ∈ Rn×r (with r � n) spanning Vand W subspaces and forming a projector ΠV,W = VWT , such that

H :{

WTEV ˙x(t) = WTAV x(t) +WTBu(t)y(t) = CV x(t) (2)

well approximates H.

I Small approximation error and/or global error boundI Stability / passivity preservationI Numerically stable & efficient procedure



IntroductionProjecion-based problem


Krylovmethods

• Arnoldi• Lanczos• Rational Interpolation• Tangential

SVDmethods

Linear systems• Balanced truncation• Hankel approximation


Mixedmethods

• Krylov/SVD• Sylvester

• Numerical efficiency• n� 103

• Matching points• Optimality • Stability

• Error bound• n ≈ 103



Outlines

Introduction

OperatorsHankel Singular ValuesSystem Gramians

Balanced truncation


Summary



OperatorsHankel Singular Values

Definition: Singular Value DecompositionGiven a matrix A ∈ Rn×m, its singular value decomposition is defined as follows:

A = USV T where S = diag(σ1, . . . , σn) ∈ Rn×m (3)

where σ1(A) ≥ · · · ≥ σn(A) ≥ 0 are the singular values, and where σ1(A) isthe 2-induced norm of A. Then, the columns of the unitary orthogonal matricesU = (u1, . . . , un) and V = (v1, . . . , vm) are the left and right singular vector of Arespectively. The SVD induces the Dyadic decomposition of A:

A = σ1u1vT1 + σ2u2v

T2 + · · ·+ σrurv

Tr (4)

with rank(A) = r.




Definition: Hankel Singular ValueLet H be a nu inputs ny outputs LTI continuous-time dynamical system of order n,the controllability (P) and observability (Q) Gramians are defined as the solution of thetwo Lyapunov functions, {

AP + PAT +BBT = 0ATQ+QA+ CTC = 0 (5)

Then, the Hankel singular values are defined as,

σi(Σ) =√λi(PQ) (6)

where λi denotes the eigenvalue operator indexing the ith value. Let denote the largestsingular value as,

||H||H = σ1(Σ) =√λ1(PQ) (7)




How to compute the Hankel Singular ValueFirst, solve the two Lyapunov equations:


Compute an upper triangular U and a lower triangular L such that (with Choleskyfactorization), {

P = UUT

Q = LLT(9)

Then, the Hankel singular values are computed through the SVD of UTL as,

PQ = U UTL︸︷︷︸ZSY T

LT

= UZSY TLT(10)

where S is directly the singular values sorted in decreasing magnitude.



OperatorsSystem Gramians

Assume a stable system. The impulse response, t ≥ 0 such that h(t) = CeAtB

I Input-to-state map x : eAtBI State-to-output map η : CeAt

I δ(t)→ x(t) = eAtB

I Initial condition x0 → y(t) = CeAtx0

Corresponding to Gramians:

P =∑t

x(t)xT (t) =∫ ∞

0eAtBBT eA

T tdt

Q =∑t

η(t)ηT (t) =∫ ∞

0eA

T tCTCeAtdt

(11)

State transformation x = Tx implies P = TPTT and Q = TQTT , then,

λi(PQ) (12)



OperatorsSystem Gramians

Definition: GramiansLet consider system H, the controllability (P > 0) and observability (Q > 0) Gramiansare defined as the solution of the Lyapunov equations,{


or equivalently,

P =∫ ∞

0eAtBBT eA

T tdt

Q =∫ ∞

0eA

T tCTCeAtdt

(14)

I Also useful for sensor placement, controllability and reachabilityI The minimal energy required to steer the state from 0 to x as t→∞ is xTP−1x

I The maximal energy produced by observing the output of the systemcorresponding to an initial state x0 when no input is applied is xT0 Qx0



Outlines

Introduction

Operators

Balanced truncationBalanced realizationThe balanced truncation (BT)The singular perturbation approximation (SPA)Illustration and limitations


Summary



Balanced truncationBalanced realization

Definition: Balanced realizationLet H be a nu inputs ny outputs LTI continuous-time dynamical system of order n.Then, H is said to be balanced iff.

P = Q = S = diag(σ1, . . . , σn) (15)

where P and Q are the solution of the Lyapunov equations,





Theorem: Balanced transformationLet H be a nu inputs ny outputs LTI continuous-time dynamical system of order n. Itsbalanced realization H⊥ is obtained as follows,

Σ⊥ =[WTAV WTBCV D

]=[A⊥ B⊥C⊥ D

]∈ R(n+ny)×(n+nu) (17)

where V,W ∈ Rn×n are defined as:

V = UZS−1/2

W = LY S−1/2 (18)

with P = UUT and Q = LLT (Cholesky) are the solutions of the two Lyapunovequations and UTL = ZSY T (SVD).

I Sort states that are "the more" simultaneously easy to reach and to observe




ProofLet consider the initial minimal system H. Its associated Lyapunov equations,{

AP + PA+BBT = 0ATQ+QA+ CTC = 0

⇔{

(WTAV )(V −1PW ) + (WTPV −T )(V TATW ) +WTBBTW = 0(V TATW )(W−1QV ) + (V TQW−T )(WTAV ) + V TCTCV = 0

⇔

A⊥ V

−1PW︸︷︷︸P⊥

+WTPV −T︸︷︷︸P⊥

AT⊥ +B⊥BT⊥ = 0

AT⊥W−1QV︸︷︷︸Q⊥

+V TQW−T︸︷︷︸Q⊥

A⊥ + CT⊥C⊥ = 0

(19)We are looking for a transformation couple V and W such that,

P⊥ = Q⊥ = S = diag(σ1, . . . , σn) (20)




(V −1PW )(W−1QV ) = S2

⇔ V −1PQV = S2

⇔ V −1UUTLLTV = S2 where U,L are obtained by a Cholesky⇔ V −1UZSY TLTV = S2 where ZSY T = UTL by an SVD

(21)

and in W ,

(WTPV −T )(V TQW−T ) = S2

⇔ WTPQW−T = S2

⇔ WTUUTLLTW−T = S2 where U,L are obtained by a Cholesky⇔ WTUZSY TLTW−T = S2 where ZSY T = UTL by an SVD

(22)Then, we obtain the following results for V and W (note that S is diagonal):

V = UZS−1/2 = L−TY −TS1/2

W = LY S−1/2 = U−TZ−TS1/2 (23)

the balanced realization objective, i.e. P⊥ = Q⊥ = S are satisfied.



Balanced truncationThe balanced truncation (BT)

Theorem: Balanced truncation approximationLet H be a nu inputs ny outputs stable LTI continuous-time dynamical system oforder n, and Vr,Wr ∈ Rn×r be the first r columns of V,W , the balanced realizationprojectors, then

H =[WTr AVr WT

r BCVr D

]∈ R(r+ny)×(r+nu) (24)

is called the rth balanced truncation approximation of H.

The resulting reduced order system H, has the following properties,I H is asymptotically stable.I The error between original and reduced systems is upper-bounded by the

following relation,

σr ≤ ||H− H||H∞ ≤ 2(σr+1 + · · ·+ σn) (25)

where σi, i = 1, . . . n are the singular values of the original system.




ProofLet H⊥ be balanced with Gramians equals to,

P⊥ = Q⊥ = S =[S1 00 S2

](26)

where S1 ∈ Rr×r and S2 ∈ R(n−r)×(n−r) contains all the highest and lowest singularvalues, respectively. Then, according to the Balanced realization theorem, the balancedsystem H⊥ can be expressed as,

H⊥ =[A⊥ B⊥C⊥ D

]=

[A11 A12 B1A21 A22 B2C1 C2 D

](27)

where A11 ∈ Rr×r, A12 ∈ Rr×(n−r), A21 ∈ R(n−r)×r, A22 ∈ R(n−r)×(n−r),B1 ∈ Rr×nu , B2 ∈ R(n−r)×nu , C1 ∈ Rny×r and C2 ∈ Rny×(n−r).



Balanced truncationThe balanced truncation (BT)1

Then the balanced reduced order system is obtained by truncating the balancedrealization, hence,

H =[A11 B1C1 D

](28)

which has a rth order Lyapunov equation of solution S1. This truncated realization isobtained by projecting the initial system Σ onto Vr and Wr, the r first columns of theprojectors V and W .

H =[WTr AVr WT

r BCVr D

](29)

1 A.C. Antoulas, "Approximation of Large-Scale Dynamical Systems", Advanced Design and Control,SIAM, Philadelphia, 2005.




Require: A ∈ Rn×n, B ∈ Rn×nu , C ∈ Rny×n, r ∈ N∗1: Solve AP + PAT +BBT = 02: Solve ATQ+QA+ CTC = 03: P = UUT and Q = LLT

4: SVD decomposition: [Z, S, Y ] = SVD(UTL)5: Set V = UZS−1/2

6: Set W = LY S−1/2

7: Apply projectors V and W and obtain

H⊥ =

[A11 A12 B1A21 A22 B2C1 C2 D

](30)

8: Approximation is obtained by H =[A11 B1C1 D

]Ensure: Small approximation error



Balanced truncationThe singular perturbation approximation (SPA)

Theorem: Singular perturbation approximationLet H⊥ be a nu inputs ny outputs balanced stable LTI continuous-time dynamicalsystem of order n, defined as,

H⊥ =[A⊥ B⊥C⊥ D

]=

[A11 A12 B1A21 A22 B2C1 C2 D

](31)

then,

H =[A11 −A12A

−122 A21 B1 −A12A

−122 B2

C1 − C2A−122 A21 D − C2A

−122 B2

](32)

is called the rth singular perturbation approximation of H. The resulting reduced ordersystem H, has the following properties,

I H is asymptotically stable.I H(0) = H(0), ensuring static gain matching.




ProofFrom equation,

H⊥ =[A⊥ B⊥C⊥ D

]=

[A11 A12 B1A21 A22 B2C1 C2 D

](33)

the reduced order model is obtained by setting:

x2 = A21x1 +A22x2 +B2u = 0 (34)

Then it turns that, if A22 is full rank,

H =[A11 −A12A

−122 A21 B1 −A12A

−122 B2

C1 − C2A−122 A21 D − C2A

−122 B2

](35)

Note that H may become ill-conditioned due to the potentially poorly ranked A22.




Require: A ∈ Rn×n, B ∈ Rn×nu , C ∈ Rny×n, r ∈ N∗1: Solve AP + PAT +BBT = 02: Solve ATQ+QA+ CTC = 03: P = UUT and Q = LLT

4: SVD decomposition: [Z, S, Y ] = SVD(UTL)5: Set V = UZS−1/2

6: Set W = LY S−1/2

7: Apply projectors V and W and obtain

H⊥ =

[A11 A12 B1A21 A22 B2C1 C2 D

](36)

8: Approximation is obtained by H =[A11 −A12A

−122 A21 B1 −A12A

−122 B2

C1 − C2A−122 A21 D − C2A

−122 B2

]Ensure: Small approximation error



Balanced truncationTruncation and SVD based techniques

Lecture 3

Hankel Singular Values of the Beam original model (n = 348).

0 50 100 150 200 250 300 35010

−40

10−35

10−30

10−25

10−20

10−15

10−10

10−5

100

Normalized Hankel Singular Value vs. Order

k

σ/σ m

ax



Balanced truncationIllustration and limitations

Comparison of the Bode diagrams of Beam original (n = 348) andapproximated (r = 5) models.

10−3

10−2

10−1

100

101

102

103

−80

−60

−40

−20

0

20

40

60

80M

agni

tude

(dB

)

k = 5

Frequency (Hz)

OriginalSVD truncationSVD singular





10−3

10−2

10−1

100

101

102

103

−80

−60

−40

−20

0

20

40

60

80M

agni

tude

(dB

)

k = 10

Frequency (Hz)






10−3

10−2

10−1

100

101

102

103

−80

−60

−40

−20

0

20

40

60

80M

agni

tude

(dB

)

k = 25

Frequency (Hz)




Balanced truncationIllustration and limitations2

The general idea consists then in solving two Lyapunov equations and guaranteeing thediagonal structure of the solution. Then, using the same concept, there exists othertype of Lyapunov equation which can enforce other system properties such that:

I Lyapunov balancing, which preserves stability


I Positive real balancing, which preserves passivity

AP + PAT + (PCT −B)(D +DT )−1(CP −BT ) = 0ATQ+QA+ (CT −QB)(D +DT )−1(C −BTQ) = 0 (38)

I Weighted balanced approximation, which consists in focusing on a specificfrequency range.

2 A.C. Antoulas, "Approximation of Large-Scale Dynamical Systems", Advanced Design and Control,SIAM, Philadelphia, 2005.




√Stability preserved

√Global error bound

||H− H||H∞ ≤ 2(σk+1 + · · ·+ σn) (39)√

Efficient functions available in Matlab, SLICOT & LAPACK√

Possibility to extend to weighted, BR, PR SVD× Local results may be unsatisfactory× Requite the entire transformed system before reducing× Solution of 2 Lyapunov equations (hard for very large-scale systems)× Gramians are ill conditioned× Require a lot of memory and operations

Well adapted to large-scale systems (but not very large ones)



Outlines

Introduction

Operators

Balanced truncation

Computing the GramiansNumerical aspectsDense Lyapunov solvers (exact)Sparse Lyapunov solvers (approximate)

Summary



Computing the GramiansNumerical aspects

Need to solve, for the Gramians,


and then, find the "dominant subspaces" X and Y such that,

PQX = XΛ2

Y ∗PQ = Λ2Y ∗

Y ∗X = In

(41)

Problem: rapid Hankel singular values decay and Gramians are ill conditioned.



Computing the GramiansNumerical aspects

Direct methodsI Vectorization of the Lyapunov equationI Bartels-Stewart method, for Sylvester and Lyapunov equation

(the one implemented in Matlab)I Hammarling’s method, for stable Lyapunov equationsI (Hessenberg-Schur method, for Sylvester equations)

Approximate methodsI Alternating Direction Implicit methods (ADI)I Krylov-like approaches

These methods are not covered here (note that they rely on iterative procedures)



Computing the GramiansDense Lyapunov solvers (exact)

(Naive) Vectorization approachTransform

AP + PAT +BBT = 0, (42)

into (In ⊗A+A⊗ In

)︸︷︷︸A

vect(P)︸︷︷︸x

= vect(−BBT )︸︷︷︸b

. (43)

Then solveAx = b. (44)

Thus the complexity of such n× n problem is O((n2)3

).

» startLyapunovSolver




Bartels-Stewart methodConsider the Lyapunov function,

AP + PAT +BBT = 0 (45)

where A ∈ Rn×n, B ∈ Rn×nu , P = PT ∈ Rn×n, (A,B) reachable and A has noeigenvalues on the imaginary axis.Then, applying A← TATT (Schur factorization), where T is a block triangular matrix.

A =[A11 A12

0 A22

], B =

[B1B2

], P =

[ P11 P12PT12 P22

](46)




Bartels-Stewart methodConsider the Lyapunov function,

AP + PAT +BBT = 0 (47)

Based on the factorization A = QTQT , and left right multiplying by Q, QT leads to

QTAPQ+QTPATQ+QTBBTQ = 0 (48)

Then, by noticing that QQT = I, then

QTAQQTPT +QTPQQTATQ+QTBBTQ = 0 (49)

Or simplyQTAQ︸︷︷︸

T

QTPT︸︷︷︸Y

+QTPQ︸︷︷︸Y

QTATQ︸︷︷︸TT

+QTBBTQ = 0 (50)

where Y = QTPQ.




Then, Y = QTPQ.TY + Y TT +QTBBTQ = 0 (51)

Then one should note

T =[T11 T120 T22

], Y =

[Y11 Y12Y T12 Y22

], QTBBTQ =

[C11 C12C21 C22

](52)

Where the last block of T ∈ Rnr . Then solve the equation in a backward fashion.An iterative algorithm can then be stated with complexity of O

(n3). The solution is

given as:P = QTY Q (53)




Hammarling’s methodIf A is stable then P > 0. In this case in the Schur basis, we can write P = UUT ,where U is upper triangular. Hammarling observed that U can be computed withoutexplicitly computing P first. We have,

P =[P1/2

11 P12P−1/222

0 P1/222

][P1/2

11 0P12P−1/2

22 P1/222

]= UUT (54)

The problem is now to successfully compute the smaller part of P, i.e. P22 ∈R(n−r)×(n−r), P12 ∈ Rr×(n−r) and finally, P11 ∈ Rr×r, where r � n. To thateffect, three equations have to be solved:

A22P22 + P22AT22 +B2B

T2 = 0

A11P12 + P12

(P−1

22 A22P22 + P−122 B2B

T2

)+A12P22 +B1B

T2 = 0

A11P11 + P11AT11 + B1B

T1 = 0

(55)




The problem is now to successfully compute the smaller part of P, i.e.P22 ∈ R(n−r)×(n−r), P12 ∈ Rr×(n−r) and finally, P11 ∈ Rr×r, where r � n.

A22P22 + P22AT22 +B2B

T2 = 0

A11P12 + P12

(P−1

22 A22P22 + P−122 B2B

T2

)+A12P22 +B1B

T2 = 0

A11P11 + P11AT11 + B1B

T1 = 0

(56)

We can thus solve the first (Lyapunov) equation for P22. Then, we solve the secondequation which is a Sylvester equation for P12. Then, since (A11, B1) is reachable,the problem of solving a Lyapunov equation of size n is reduced to that of solving aLyapunov equation of smaller size r < n.LemmaUnder the above assumptions, the following hold:

I Pair (A22, B2) is reachableI P22 is nonsingularI Pair (A11, B1) is reachable, where B1 = B1 − P12P−1

22 B2




Lyapunov equationsThe Lyapunov equation solvers is still a very active field of research within the numericalcommunity. Today, Matlab used the BLAS (Basic Linear Algebra Subroutines) libraryfor basic routines such as eig or to solve Ax = b and LAPACK for Lyapunov equations.




Stewart’s Lyapunov solverI Transform to Schur basisI Solve for columns of Q recursivelyI Return to original coordinates P = QTY Q

Hammarling’s methodI Transform to Schur basis and write the Cholesky factorization, P = UUT :

AUUT + UUTAT +BBT = 0 (57)

A, U in upper triangular formI Solve U recursivelyI Return to original coordinates P



Computing the GramiansSparse Lyapunov solvers (approximate)

Some of them...I Low rank Alternating Direction Implicit methods (LR-ADI)I Smith iteration

These methods are not covered here (note that they rely on iterative procedures)



Computing the GramiansSparse Lyapunov solvers (approximate)

A Krylov based solver (Petrov-Galerkin indeed)I Set

v1 = orth(BBT ) (58)I For k = 1, ..., compute Yk, solution to

(V Tk AVk)Y + Y (V Tk ATVk) + V Tk BB

TVk = 0 (59)

I If converged Xk = VkYkVTk and stop

I Arnoldi procedure for the next basis block:

vk+1 = Avk (60)

and orthogonalize it wrt {v1, ..., vk}I Vk+1 = [Vk, vk+1]



Outlines

Introduction

Operators

Balanced truncation


Summary



SummaryLecture’s highlights

About this lectureI GramiansI Balanced realization, TruncationI Stability and (pessimistic) global bound (startBTerror)I Lyapunov equations

OK, so is everything based on Lyapunov? What about this Krylov?



Summary

Realization-based ap-proximation methods

Non-projection methods

Linear systems• LMIs• Vector fitting• Complex optimization (DARPO)


Linear systems (truncation)• SVD (BT, SPA)• Modal truncation (SDPA)• Hankel approximation (H)

Linear systems (iterative)• Rational Interpolation (IRKA)• Tangential (ITIA)

Linear systems (mixed)• Krylov/SVD (ISRKA, ISTIA)• Krylov/Eigenvectors (IETIA)




SummaryApproximation methods: an overview


Krylovmethods

• Arnoldi• Lanczos• Rational Interpolation• Tangential

SVDmethods

Linear systems• Balanced truncation• Hankel approximation


Mixedmethods

• Krylov/SVD• Sylvester

• Numerical efficiency• n� 103

• Matching points• Optimality • Stability

• Error bound• n ≈ 103



Model Reduction (Approximation) of Large-Scale Systems

Truncation and SVD based techniquesLecture 3

C. Poussot-Vassal, P. Vuillemin & I. Pontes Duff

EDSYS, April 4-7th, 2016 (Toulouse, France)

moremoreΣ

(A,B,C,D)i

Σ

Σ

(A, B, C, D)i

model reduction toolbox

Kr(A,B)

AP + PAT + BBT = 0

WTV

DAE/ODE


Data

ReducedDAE/ODE






Model Reduction (Approximation) of Large-Scale Systems ... ·...

Documents

Transcript of Model Reduction (Approximation) of Large-Scale Systems ... ·...