Lecture Lecture – ––– 19 119919 SDRE and Designs - D...Lecture Lecture – ––– 19...

Lecture Lecture Lecture Lecture –––– 19191919

SDRE and DesignsSDRE and DesignsSDRE and DesignsSDRE and Designs

Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi

Dept. of Aerospace Engineering

Indian Institute of Science - Bangalore

Optimal Control, Guidance and Estimation

θ - D

OPTIMAL CONTROL, GUIDANCE AND ESTIMATION

Prof. Radhakant Padhi, AE Dept., IISc-Bangalore2

Topics

State-Dependent Riccati Equation

(SDRE) Design

Design

Benchmark Examples

Dθ −

StateStateStateState----Dependent Dependent Dependent Dependent RiccatiRiccatiRiccatiRiccati Equation Equation Equation Equation (SDRE) Design(SDRE) Design(SDRE) Design(SDRE) Design

Dr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiAssociate Professor





References J. R. Cloutier, “State-Dependent Riccati Equation Techniques: An

Overview”, Proceedings of the American Control Conference, Albuquerque, New Mexico, USA, 1997.

J. R. Cloutier and D. T. Stansbery, “The Capabilities and Art of State-Dependent Riccati Equation-Based Design”, Proceedings of the American Control Conference, Anchorage, AK, USA, 2002.

J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett, State-Dependent Riccati Equation Techniques: Theory and Applications.Workshop Notes: American Control Conference, June, 1998.

T. Cimen, “State-Dependent Riccati Equation (SDRE) Control: A Survey”, Proceedings of 17th IFAC World Congress, Seoul, Korea, 2008.

T. Cimen, “Systematic and Effective Design of Nonlinear Feedback Controllers via the State-Dependent Riccati (SDRE) Method”, Annual Reviews in Control, Vo. 34, 2010, pp.32-51.



SDRE Design: Usage

Nonlinear suboptimal control design

• Regulator design

• Servo (tracking) design

• Robust control design

Nonlinear suboptimal observer design

Nonlinear suboptimal filters design

(Essentially, wherever Riccati equation appears,

SDRE concept can be brought in)

( )2/H H∞



SDRE Design: Problem Statement

Performance Index: (to minimize)

System Dynamics: (control affine)

Conditions:

•••••

( ) ( )( )0

1

2

T T

t

J X Q X X U R X U dt

∞

= +∫

( ) ( )X f X B X U= +ɺ

( ) ( ) ( ) ( ) ( ), , , 1kf X B X Q X R X C k∈ ≥

( )0 0f =

( ) 0 (domain of interest)B X X≠ ∀ ∈Ω

( ) ( )( ) is globally convex True when , 0J Q X R X >

( ) ( ) ( ) ( ) , , is point-wise stabilizablef X A X X A X B X=



SDRE Design: Procedure

Cost Function

Write the system dynamics

in state-dependent coefficient (SDC) form

Solve the state-dependent

Riccati equation

Construct the controller

( ) ( )X A X X B X U= +ɺ

( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )1 0

T

T

P X A X A X P X Q X

P X B X R X B X P X−

+ +

− =

( ) ( ) ( )

( )

1 TU R X B X P X X

K X X

− = −

= −

( ) ( )( )0

1

2

T T

t

J X Q X X U R X U dt

∞

= +∫



Implementation Issue

Solve the Riccati equation symbolically by long hand algebra.

Use symbolic software package to solve the Riccati equation symbolically

(e.g. Maple, Mathematica, Matcad etc.)

Solve the Riccati equation online with a high speed computer.

Obtain off line point solution and use gain scheduling.



Example - 1

Reference:J. R. Cloutier, C. P. Mracek, D. B. Ridgely, and K. D. Hammett,

State-Dependent Riccati Equation Techniques: Theory and Applications,Workshop Notes: American Control Conference, June, 1998.



Example - 1



x

u(0,0)



Definitions:

Controllability / Observability

Controllability

Observability

( )

( ) ( )

is an observable (detectable) parameterization of the

nonlinear system in a region if the pair ,

is point-wise observable (detectable) in the linear sense .

A X

C X A X

X

Ω

∀ ∈Ω

( )

( ) ( )

is an controllable (stabilizable) parameterization of the

nonlinear system in a region if the pair ,

is point-wise controllable (stabilizable) in the linear sense .

A X

A X B X

X

Ω

∀ ∈Ω

( )Output: Z C X X=



SDRE Design: Useful Results

In addition to the conditions mentioned earlier, if and it is both a detectable and stabilizable parameterization, then the SDRE approach produces a closed loop system that is “locally asymptotically stable”.

For scalar problems, the resulting SDRE nonlinear controller satisfies all the necessary conditions of optimality; i.e. for scalar problems it always leads to the optimal solution (this is not true for vector case however).

( ) ( )1kA X C k∈ ≥



SDRE Design: Useful Results

Out of the three necessary conditions,

the optimal control equation is

always satisfied

However, the costate equation

is satisfied only asymptotically (under

certain additional mathematical

conditions). This is the reason for sub-

optimality of the controller in general.

/ 0H U∂ ∂ =

( )/H Xλ = − ∂ ∂ɺ



Convergence of Costate Equation

( )

( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

Let 0, be an arbitrarily large open ball centred

at the origin with radious . Assume that the functions

, , , , along with their gradients

, , , , , 1, are

bounded in

i i i i ix x x x x

r

r

A X B X P X Q X R X

A X B X P X Q X R X i n

< ∞

= …

B

B ( )

( )

0, . Then, in SDRE nonlinear regulation,

under asymptotic stability (i.e. as 0), the necessary condition

/ is asymptotically satisfied at a quadratic rate.

r

X

H Xλ

→

= − ∂ ∂ɺ



SDRE Design: Capabilities

Can directly specify and affect performance through the selection of appropriate state dependent state and control weighting matrices

Can incorporate hard bounds on state and control

Can directly handle unstable and/or non-minimum phase systems

Can preserve beneficial nonlinearities



Can be used to design servo (tracking)

control

Can be applied to a broad class of

nonlinear systems

Can incorporate extra degree of freedom

(a design parameter) to enhance

performance of the suboptimal controller

SDRE Design: Capabilities



SDRE Design: Limitations

Can be applied only to a class of

nonlinear problems

Suboptimality of the controller

Non-uniqueness of the parameterization

of the system dynamics

Applicable for infinite-time problems only



Demands solution of Riccati equation

online, which may not be feasible for

high-dimensional problems (since Riccati

equation is nonlinear)

No analytical guarantee of global stability

for the resulting controller in general.

SDRE Design: Limitations



SDRE Design: Some Useful Tricks

Presence of state-independent terms

Presence of state-dependent terms that excludes the origin

Uncontrollable and Unstable but Bounded State dynamics

( )

Constant bias:

0 1b bα α= − <ɺ ≪

( )11 1

1

cos 1cos 1 bias

xx x

x

−⇒ +

( )( )1 1

Add a stabilizing term

x xα= −ɺ i

( )0 1α< ≪



Extra Degree of Freedom

( ) ( )

( ) ( ) ( ) ( )

1 2

3 1

Claim:

Assume that and are two SDC parameterizations.

Then another SDC parameterization can be constructed as a

convex combination of these two parameterizations as follows:

1

A X A X

A X X A X X Aα α = + − ( ) ( )2 , 0 1X Xα≤ ≤

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

1 2

1 2

Proof:

1

1

1

X A X X A X X

X A X X X A X X

X f X X f X f X

α α

α α

α α

+ −

= + −

= + − =



Example - 2





Example - 2



u

t

x2

x1



Example - 2



u

t

x2

x1



Example - 2



u

t

x2

x1

α(t)

t

Suboptimal Control DesignSuboptimal Control DesignSuboptimal Control DesignSuboptimal Control Design

Dr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiDr. Radhakant PadhiAssociate Professor



θ - D

Courtesy: Ming Xin, Department of Aerospace Engineering

Mississippi State University, USA

OPTIMAL CONTROL, GUIDANCE AND ESTIMATION Prof.

Radhakant Padhi, AE Dept., IISc-Bangalore

26

1. Xin M. and Balakrishnan S. N., “A New Method for Suboptimal

Control of a Class of Nonlinear Systems,” Optimal Control

Applications and Methods, 26(2): 55-83, 2005.

2. Xin M., Balakrishnan S. N., Stansbery D. T., and Ohlmeyer E.J.,

“Nonlinear Missile Autopilot Design with θ - D Technique,” AIAA

Journal of Guidance, Control and Dynamics, 27, 406-417, 2004.

3. Drake D., Xin, M. and Balakrishnan S. N., “A New Nonlinear Control

Technique for Ascent Phase of Reusable Launch Vehicles,” AIAA

Journal of Guidance, Control and Dynamics, 27(6): 938-948, 2004.

4. Radhakant Padhi, Ming Xin and S. N. Balakrishnan, “Suboptimal

Control of a One-dimensional Nonlinear Heat Equation Using POD

and θ - D Techniques”, Optimal Control Applications and Methods,

29(3): pp.191-224, 2008.

References


Prof. Radhakant Padhi, AE Dept., IISc-Bangalore

27

Optimal Control Problem

Objective:

( ) ( )0

1

2

ft

T T

t

J x Q x x u R x u dt

→∞

= + ∫

Find a controller u to minimize a cost function

( )( ) ( ) ( )x t f x t Bu t= +ɺ

System Dynamics:

This is an infinite-horizon optimal control problem

control affine form



28

Solution to the Optimal Control Problem

min*

uV J=

• Solve the Hamilton-Jacobi-Bellman (HJB) equation

*1 T V− ∂

= −∂

u R Bx

( )T *T

1 T T1 10

2 2

* *V V V−∂ ∂ ∂

− + =∂ ∂ ∂

f x BR B x Qxx x x

• A closed-form solution is very difficult to obtain

where

Challenge



29

• Make Approximations

Summary of Technique

0

1

2

T TJ dt

∞ = + +

∑∫

i

i

i=1

x Q x u uD θ R∞∞∞∞

*

0

( , ) i

i

i

Vθ θ

∞

=

∂=

∂∑T x x

x

( ) ( )

= + = + = 0

( )ɺ

A xx f x Bu F x x Bu A + θ x + Bu

θ

Assume:

• Solve perturbed Hamilton-Jacobi-Bellman equation

* * *1

1

1 1( ) 0

2 2

T TT T i

i

i

V V Vθ

∞−

=

∂ ∂ ∂ − + + = ∂ ∂ ∂

∑f x BR B x Q D xx x x

θ - D

Recall:*

1 T V− ∂= −

∂u R B

x



30

0 0 0 0

1

0 00

T TA A BRT T T TB Q

−−−−+ − + =+ − + =+ − + =+ − + =

T

c cF x DA AT TT

1 10 10 0 1( , , )θθθθ+ = −+ = −+ = −+ = −

⋮

• Closed-form Optimal Control

0T

1T

nT

Algebraic Riccati Equation

Linear Equation with

constant coefficients

Linear Equation with

constant coefficients⋮

n n n

T

cn ncF xT T TA T DA

0 1 10( , , , , )θθθθ−−−−

+ = −+ = −+ = −+ = −⋯⋯⋯⋯

1

0

( , )n

T i

i

i

T θ θ−

=

= − ∑u R B x x

1

0 0 0

T

cA A BR B T−−−−= −= −= −= −

Note: θ will be cancelled in the final control calculation

Substitute in HJB equation and equate coefficients:



31

Construction of

iD (((( ))))0 1

( ), , , ,il t

i i i iD k e F A x T T θθθθ

−−−−

−−−−==== ⋯⋯⋯⋯is constructed as

such that

i i i i i iF A x T T D t F A x T T

0 1 0 1( ( ), , , , ) ( ) ( ( ), , , , )θ ε θθ ε θθ ε θθ ε θ− −− −− −− −− = ⋅− = ⋅− = ⋅− = ⋅⋯ ⋯⋯ ⋯⋯ ⋯⋯ ⋯

( ) 1 il t

i it k eεεεε

−−−−= −= −= −= −with a small number

iD

Note:

TT A x A x TF x T 0 0

1 0

( ) ( )( , , )θθθθ

θ θθ θθ θθ θ= − −= − −= − −= − −

T nTn n

n n j n j

j

T A x A x TF x T T T BR B T

111 1

1 1

1

( ) ( )( , , , , )θθθθ

θ θθ θθ θθ θ

−−−−−−−−− −− −− −− −

− −− −− −− −

====

= − − += − − += − − += − − +∑∑∑∑⋯⋯⋯⋯



32

Motivation of Using i

D

Prove the convergence of the series.

Guarantee semi-global asymptotic stability

Reduce the initial control level

Adjust the system transient performance

ik and are primary design parameters to tune the system

performance

il

( ) 1 il t

i it k eεεεε

−−−−= −= −= −= − is used to:



33

• Systematic Method of Selecting and Parameters

Ideally, on the optimal path, the Hamiltonian

Procedure is as follows

– An initial value of (ki, li) is given.

– Controller is run, computing H value at each time.

– Iteratively change (ki, li) to minimize H in the least-

square sense.

– This procedure is run offline.

( )*

1 12 2

0T

T T VH x Q x u R u f x B u

∂= + + + = ∂x

ili

k



34

Benchmark Example (Kokotovic,1994)

Scalar Problem:

uxxx +−= 3ɺ

dtuxJ )(2

1 2

0

2 += ∫∞

Cost function:

The optimal solution

22)( 243 +−−−−= xxxxxu

Feedback Linearization solution3 2flu x x= − Feedback linearization cancels the beneficial nonlinearity

and results in large control effort when the state is large!

System dynamics:



35

SolutionDθ −

2

0 11, ( )A A x x= = −

• Factorize nonlinear term f(x) as

with Q = 1, R = 1

0 1 2T = +

solutionDθ −

2

1 1

1 (1 2)2

2 2

xT D

θ

⋅ += − −

22 2

2

2 1 1 22

1 1 (1 2) 1 (1 2)2 2 2

82 2 2 2

x xT x D D D

θθ

− ⋅ + ⋅ + = − ⋅ − + − −



36

SolutionDθ −

2 0 01

( ) ( )0 .98

Tt T A x A x T

D eθ θ

− = − −

0.9 1 12

( ) ( )0 .98

Tt T A x A x T

D eθ θ

− = − −

Di terms in the method play key role.Dθ −



37

Figure 3: Scalar problem: x0= [10,10]



38

Comparison Between

SDRE and θ - D Methods

Requires adjustment of both weighting matrix and

other design parameters

Requires adjustment of weighting matrices

Same requirement

Needs state dynamics in

terms of state dependant coefficient form

Lesser computational timeHigher Computational Time

Solving a set of Lyapunov equations online

Solving the Riccati equation online

SuboptimalSuboptimal

θθθθ - D MethodSDRE Method

OPTIMAL CONTROL, GUIDANCE

AND ESTIMATION Prof. Radhakant

Padhi, AE Dept., IISc-Bangalore

39

Thanks for the Attention….!!

Lecture Lecture – ––– 19 119919 SDRE and Designs - D...Lecture Lecture – ––– 19...

Documents

Transcript of Lecture Lecture – ––– 19 119919 SDRE and Designs - D...Lecture Lecture – ––– 19...