1

Discrete Nonlinear Optimal Controlof S/C Formations Near the L1 and L2 Points

of the Sun-Earth/Moon SystemB. G. Marchand and K. C. Howell

School of Aeronautics and AstronauticsPurdue University

J. T. BettsTechnical Fellow

Mathematics and Engineering AnalysisThe Boeing Company

2

Formations Near the Libration Points

Moon

1L

2L

x

( )ˆ inertialXB

θ

y

Chief S/C Path(Lissajous Orbit)

EPHEM = Sun + Earth + Moon Motion From Ephemeris w/ SRPCR3BP = Sun + Earth/Moon barycenter Motion Assumed Circular w/o SRP

Deputy S/C(Orbit Chief Vehicle)

3

Formation Keeping viaNonlinear Optimal Control

• Incorporate nonlinearities into control design process • Allows for the addition of control and path constraints

– Upper and lower bounds on thrust output– Specifications on relative path error– Allow for thruster on-off times while min. the impact on the path

• Min. # of assumptions better assessment of feasibility

4

Optimal Control Solution• Method #1: Partial Discretization

– Divide Trajectory into Segments and Nodes– Numerically integrate node states – Impulsive Control at Nodes (or Constant Thrust Between Nodes)– Numerically integrated gradients– Solve Using Subspace Trust Region Method

• Method #2: Transcription and Nonlinear Programming– Divide Trajectory Into Segments and Nodes– Solve using Sparse Optimal Control Software (SOCS)

• Use Hermite-Simpson discretization (others available)• Jacobian and Hessian computed via Sparse Finite Differencing.• Estimate cost index to second order • Use SQP algorithm

5

Identification of Startup Solution• Possible Startup Solution Options

– Non-Natural Arcs IFL/OFL Nonlinear Control • Specify some nominal motion• Apply IFL/OFL control to achieve desired nominal• Use results as initial guess to optimal control process with

– Natural Arcs Floquet Analysis of Chief S/C Linearized Equations• Deputy dynamics modeled as a perturbation relative to chief path• Floquet controller applied to establish natural relative formation• Transition into NL system via 2-level corrector• Use results as initial guess to optimal control process with

( )u t 0≠

( )u t 0=

6

Method #1: Optimal Controlby Partial Discretization

( ) ( ) ( ) ( )11 1

0 0min , , , ,φ φ

+− −

= =

= + = +∑ ∑ ∫ j

j

tN N

N j j j Nj j t

J x L t x u x L t x u dt

Subject to:

( ) ( )1 0, , ; Subject to 0+ = = =j j j j ix F t x u x x x

( ) ( ) ( )

( )( )

( ) ( )( )

1

11

1 1 1

00

min

, ,, , ;

, ,

Subject to 0

φ φ+

+

++ + +

= + =

= = = +

=

N n N N

j j jjj j j j

n j n j j j j

J x x t x

F t x uxx F t x u

x t x t L t x u

xx

Equivalent Representation as Augmented Nonlinear System:

7

Euler-Lagrange Optimality Conditions(Based on Calculus of Variations)

( )1

1

Condition #1: 1

Condition #2: 0 ; 0, , 1

φλ λ λ

λ

+

+

∂ ∂ = = → = ∂ ∂ ∂

= = =

∂

∂∂

∂−

∂

j NT T Tj j N

j

j

j

j N

j Tj

j j

H xx x

jFu

HN

u

Fx

( )1 , ,λ +=

Tj j j j jH F t x u

Identify and from augmented linear system.∂ ∂

∂ ∂

j j

j j

F Fx u

8

Identification of Gradients From the Augmented Linearized Model

( ) ( ) ( ) ( ) ( )x t A t x t B t u tδ δ δ= +

( )( ) 0

0

= ∂ ∂

dA tA t L

x( )

3

3

0

0T

B t I =

( )( )

( )( )

0

11

0, ,;

0 0, , nn

x xf t x uxxL t x ux ++

= =

Nonlinear System:

Linear System:

9

Solution to Linearized Equations

( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( )0

0 0

0 0 0 0 7

, ,

, , ; ,

t

t

x t t t x t B u d

t t A t t t t t I

δ δ τ τ δ τ τ= Φ + Φ

Φ = Φ Φ =

∫

∂∂

j

Fx

Relation to Gradients in E-L Optimality Conditions:

( ) ( ) ( ) ( )1

1 1 1, ,j

j

t

j j j j jt

x t t x t B u dδ δ τ τ δ τ τ+

+ + += Φ + Φ∫

10

Control Gradient for Impulsive Control

( ) ( ) ( ) ( )1

1 1 1, ,j

j

t

j j j j jt

x t t x t B u dδ δ τ τ δ τ τ+

− ++ + += Φ + Φ∫

( )( )( ) ( )

1

1 1

,

, ,

j j j j

j j j j j j

t t x B V

t t x t t B V

δ

δ

−+

−+ +

= Φ + ∆

= Φ +Φ ∆

∂∂

j

Fu

( )1,+

∂= Φ

∂

j jj

F t t Bu

11

Control Gradient for Constant Thrust Arcs

( ) ( ) ( )1

1 1 1, ,j

j

t

j j j j j jt

x t t x t B d uδ δ τ τ τ δ+

+ + +

= Φ + Φ ∫

∂∂

j

Fu

( ) ( ) ( )1

1

1, ,j

j

t

j j jj t

F t t t B du

τ τ τ+

−

+

∂ = Φ Φ∂

∫

( )( )

( )( )

( ) ( )( ) ( )

1

1*

, ,, ,

,,

, ,

n

jj

j j

f t x uxL t x ux

A t t tt t

t t t t B τ

+

−

= ΦΦ Φ Φ

( )

( )

Only , available from numerical integration

Use STM properties to rewrite in terms of , .

τ

τ

Φ

∂Φ

∂

j

j

t

F tu

12

Numerical Solution Process

( )0(1) ; ( 0,1,..., 1)

(2)1-Scalar Equation to Optimize in 3 1 Control Variables

Use optimizer to identify optimal

Input , , and ini

given .

During each it

tial guess for

e ati n

r o

= −

−

∂∂

N i

ii

i

i NN

Huu

x t u

( )( ) Sequence (by numerical integration) forward and stor

of the optimizer, the following steps are followed:

( ) Evaluate cost functional,

( ) Evaluat

e; 1,

e

, 1

φ

φλ

= … −

=

∂=

N

N

i

T

a x i

b J x

c

N

x( )

( )

( ) Sequence backward and compute the search direction ; 1, ,1

and used in next update of control input. (Subspace Trust Region Method)

1

φ

λ

∂= ∂ ∂

= −∂

∂∂

∂

ii

i

i

i

N N

N N

Hd i Nu

He Ju

x x

13

State Corrector vs. Nonlinear Optimal Control:Magnitude of Radial Error

( ) ˆ5000 km Zρ = ( ) ( )11

0

1min2

+−

=

= − −∑ ∫

j

j

tN T

i t

J x x Q x x d t

14

State Corrector vs. Nonlinear Optimal Control:Cost Function

15

State Corrector vs. Nonlinear Optimal Control:Impulsive Maneuver Differences

16

Method #2: Nonlinear Programming

• General Nonlinear Programming (NLP) Problem

– Sequential Programming Solution Algebraic System• Approximate Lagrangian to 2nd Order

• Approximate constraints as linear• Iterative solution via globalized Newton methods

( )( ) ( )= ≤ ≤ ≤ ≤min ; and L U L UJ F x c c x c x x x

( ) ( ) ( )

( ) ( ) ( ) ( )

λ

∂∂

∂

= −

≈ − + − −∂

∂ ∂

** *

*

*12

T

T

L x F x c x

L x x x x x Lx x

xx

xL

JacobianMatrix Hessian

MatrixSearchDirection

17

Dynamic Optimization via Nonlinear Programming

• Divide trajectory into phases (segments)• Define objective function• For each phase, define

– Dynamic variables– State equation– Nonlinear constraints– State Vector Limits– Control Vector Limits– Phase boundary conditions

• Approximate State Equations by Direct Transcription• Use SOCS SQP algorithm to solve

( ) ( ) ( )= [ , ]k k kz y u( ) ( ) ( ) ( ) ( ) ( ) ( )= [ , , , ]k k k k ky f y t u t p t( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )≤ ≤[ , , , ]k k k k k k

l ug g y t u t p t g( ) ( ) ( ) ( )≤ ≤k k k

l uy y t y( ) ( ) ( ) ( )≤ ≤k k k

l uu u t uΨ ≤ Ψ ≤ Ψl u

18

Direct TranscriptionExample: Hermite-Simpson Discretization

( ) ( )

( )

( ) ( )

ς

ς

τ

+ + +

+ + +

+ + +

′= − − + +

′ ′ ′= +

′ = + + −

= ∆

1 1 1

1 1 1

1 1 1

Given an initial guess for , at each node, the defect ( ) at node:

4 6

where

, ,2

12 8

k k thk

kk k k k k k

kk k k k

kk k k k k

k k

y u khy y f f f

hf f y u t

hy y y f f

h t

• Treat the defects as a constraint, ςk = 0, imposed on the cost function!• The partials of the defect equations lead to large sparse matrices.• Use SOCS (Sparse Optimal Control Software) to ID solution.

+1y

−2y

+2y

−3y

1t 2t 3t

( )y t

ς k+

3y

19

Sample Startup Solution:Slowly Drifting Vertical Orbit

Origin = Chief S/C

( )0r ( )fr t

100 Revolutions = 18,000 days

y

z

20

Example 2: Continuous Optimal ControlGoal Periodicity

( ) ( )( )

( )

1

kf

ki

N t T

tk

J u t u t dt=

= ∑∫

( ) fixed, for 1, , 4= = …kit k( ) fixed, for 1, , 4= = …k

ft k( )( )10 = −i ir t r

( )( )10 = iu t

( )( ) ( )( )10 , 1, ,3+= − =

k kf iz t z t k

( )( ) ( )( )1 40 i fy t y t= −

( )

10,000 km 10,000 km10,000 km 10,000 km10,000 km 10,000 km

4 m/sec 4 m/sec4 m/sec 4 m/sec4 m/sec 4 m/sec

− − −

≤ ≤ −

−

−

y t

( )

2 2

2 2

2 2

4 m/s 4 m/s4 m/s 4 m/s4 m/s 4 m/s

− − ≤ ≤ −

u t

Constraints: Search Space:

21

Numerical Issues• Relative Scaling Problems

– Non-convergence– All constraints met except for control acceleration continuity

• Source– Small control accelerations trick the software into convergence

• Solution– Chief S/C path pre-determined and stored using B-splines– Internal rescaling of variables– Use dimensional form of relative equations of motion

22

EX2: Periodicity Via Continuous Control

Startup SolutionConverged Solution

23

Conclusions• Direct Transcription Method

– With proper variable scaling, responds well to dynamical sensitivity of n-body problem.

– Accuracy issues overcome through mesh refinement.– Availability of higher order representations may be

useful in reducing mesh refinement iterations. These methods not currently present in SOCS.

• Partial Discretization Method– Similar optimization scheme in some respects– No constraints presently included in the formulation– Solution speed hindered by sequencing– Accuracy controlled by integrator selection

24

Backups

25

Distributed S/C Systems• Generic formulation Application Independent

– Formation Flight• Vehicles share and exchange information to accomplish mission

– Central vehicle Chief S/C– Other vehicles Deputies– Examples

» Interferometry» Surface Imaging» Radar» Geolocation

– Vehicle Rendezvous & Docking• Resources and information may be transferred (application dependent)

– Central “chief” vehicle not necessarily aware of the presence or activities of other spacecraft or “deputies”.

– Deputy vehicles perform operations on or in the vicinity of the chief– Examples:

» Resource transfer (fuel, equipment, etc.) between vehicles» Unmanned on-orbit servicing of satellites» Space based threat assessment and handling

26

Relative Dynamics(Frame Independent Formulation)

( )( ) ( )

=

= +

c c

d d

y f y

y f y Bu t

( ) ( ) ( )( ) ( ) ( )( ), ,

= − +

= + − +

=

d c

c c

c

x f y f y u t

x f y x f y u t

x F x u y

( ) ( ) ( )( ) ( ) ( ) ( )

δ δ

δ δ δ

=

= +

c c c

d d d

y t A t y t

y t A t y t B u t( ) ( ) ( )

( ) ( ) ( )

δ δ δ

δ δ

∂ ∂= +∂ ∂

= +

d

F Fx t x t u tx u

A t x t B u t

Absolute Dynamical Model Relative Dynamical Model

Linear System

Define the mathematical model that preserves generality for all apps/systems.

Linear System

Nonlinear System Nonlinear System

27

Impulsive Radial Optimal Control

( )11 2

0

1min2

+−

=

= −∑ ∫

j

j

tN

j t

J q r r dt

28

Radial Optimal Control:Cost Functional

29

Radial Optimal Control:Maneuver History

30

( ) ( ) ( )

( )

> > >

′ =

* *1 2 1 2

Direct

Find a sequence of points , , , such that .

This only requires a comparison of the objective function at each point.

IndirectIdentify the root of the necessary condition 0. T

z z z F z F z F z

F z

→

his requires that the usercompute the derivative of the cost index and determine if it meets the specifiedtolerance.

From an optimal control perspective:Indirect Optimization Identify the roots of th

→e KKT Conditions (Euler-Lagrange)

Direct Optimization Does NOT require the explicit derivation and constructionof the necessary conditions (i.e. the adjoint equations, the control equations, or the transversality conditions) that are required by the Euler-Lagrange equations.

Dynamic Optimization Approach

31

Direct vs. Indirect• If the optimality conditions are already determined by the EL-

equations, why not use an indirect method? For the formation keeping problem, this approach has been investigated but is not recommended.– The partial derivative matrices, in this case, involve a matrix quadrature

of a function of the STM. This is computationally intensive of course.– Requires a good estimate of the constrained trajectory arc to start the

optimization process.– In general, the numerical process is extremely sensitive (ill-conditioned)

to the initial guess for the Lagrange multipliers. This problem is even more difficult to deal with in the n-body problem.

• Since, in the n-body problem, an exact solution is not available for the KKT equations, a direct method is better suited for nonlinear optimization in this case.

32

Direct Methods: Nonlinear Programming

• Parameter Optimization– Finite dimensional– Solution Globalized Newton Methods

• Functional Optimization Optimal Control– Infinite dimensional– Solution Transcribe into finite dimensional problem

• Represent dynamical system in terms of finite set of variables• Solve the finite dimensional problem using NLP

– Reduces problem to solving an algebraic system of eqns.• Assess accuracy of finite dimensional approximation• If needed, refine grid and repeat first two steps

33

Optimal Control Preliminaries• Observations

– The cost index depends on point functions and quadrature functions– Nonlinear point functions can include variables from all phases– Quadrature functions are evaluated along the length of the phase by

augmenting the state vector:

– The boundary conditions also depend on variables from all phases– Each phase is divided into N mesh points for the discretization– Each interior grid point is assigned a control variable,

( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

=

, , ,, , ,

, , ,

k k k

k k k

k k k

f y t u t p tF y t u t p t

w y t u t p t

( ) ( ) ( )=k kj ju t u

34

Define Optimal Control Problem

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )( )

=

= Φ

+ ∑ ∫

( )

1 1 1 1 1

1

min , , , , , , , , , ,

, ,j

F

jI

N N N N Ni i f f i i f f

tNj j j j

j t

J z t z t p z t z t p

w z t t p dt

( ) ( ) ( ) ( ) ( )( ) ( ) ( )

( ) ( ) ( ) ( ) ( )( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

= ≤ ≤

≤ ≤

≤ ≤

Ψ ≤ Ψ ≤ Ψ

1 1 1 1 1

Each phase is subject to:

, , ;

, ,

The phases are linked by boundary conditions of the form:

, , , , , , , , , ,

k k k k ki f

k k k k kl u

k k kl u

N N N N Nl i i f f i i f f u

y t f z t p t t t t

g g z t p t g

z z t z

z t z t p z t z t p

( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( )

= For each phase, , define a vector of dynamic variables, ,

that includes both the state vector, , and the control input vector, .

k k k

k k

k z t y t u t

y t u t

35

Optimizer Test• Identification of a good startup solution for the optimizer

is necessary to ensure convergence.– Determine non-periodic but bounded relative orbits in the

linearized system using the Floquet Controller.– Employ a 2-level differential corrections process to converge the

solution in the nonlinear system.– Transfer this solution as an initial guess to the nonlinear optimal

control process.– Choose mathematical model that is consistent with ephemeris

formulation for later transition into the Generator FORMATION tool.

– Impose closed-path constraint as a test case.

36

Impulsive Optimal ControlMinimize State Error with End-State Weighting

( ) ( ) ( ) ( )11

0

1 1min2 2

+−

=

= − − + − −∑ ∫

j

j

tNT T

N N N Ni t

J x x W x x x x Q x x d t

( ) ˆ500 mρ = Y

37

EX1: Impulsive Optimal Control:Closed Relative Path (Small)

Chief S/C Halo OrbitAz = 300,000 km

Deputy S/C Relative Orbitrmax = 82 km

38

EX3: Sensitivity of Solution to Initial Guess

0

0

8254 kmGoal: Determine min( V) for f

rr r

=

∆ =

Given a bad initial guess for the optimizer …

39

Chief S/C Halo OrbitAz = 300,000 km

Deputy S/C Relative Orbitrmax = 8254 km

Converged Periodic Solution(Max. Amplitude 8254 km)

… the numerical process is still able to identify the desired solution

40

Example #1: Impulsive Optimal Controlto Achieve Closed Path

0 0Cost Index: min = ∆ ∆TJ V V

( ) ( )

( )( ) ( )( )

0

0

0

0 0 0

1

Dynamical Constraint:

, ; 0

Terminal Path Constraint:0

Initial Velocity Constraint:0

Continuity Constraints:

0

−

− +

+

= = =

− =

+ ∆ − ∆ =

− =

c

f

k kf i

r ry f y y x

VV

r r

V V V

y t y t

41

Chief S/C Halo OrbitAz = 300,000 km

Deputy S/C Relative Orbitrmax = 81057.8 km

EX1: Impulsive Optimal Control:Closed Relative Path (Large)

2 m/sec

ΔV

42

Deputy S/C Path

Chief S/C @ Origin

43

80, 81

160, 161

240, 241

320

1

Discontinuities in Control Acceleration

44

EX2: Control Acceleration Profile