dReach: δ-Reachability Analysis for Hybrid Systems · In this section, we propose a hybrid...

dReach: δ-Reachability Analysis for Hybrid Systems

Soonho [email protected] Mellon University

*

Joint work with Sicun Gao(MIT), Wei Chen(CMU), and Edmund Clarke(CMU)*

mailto:[email protected]

ODE Visualization with dReach

0 5 10 15 20 25 30 35

Click and drag above to zoom / pan the data

Mode1 Mode2 Mode3 Mode1 Mode2 Mode3

0 5 10 15 20 25 30 35

-5-4-3-2-10

z

0 5 10 15 20 25 30 35

-2.0

-1.5

-1.0

-0.5

0.0

y

0 5 10 15 20 25 30 35

-1.5

-1.0

-0.5

x

0 5 10 15 20 25 30 35

0

2

4

6

8

tau

0 5 10 15 20 25 30 35

0.0

0.5

1.0

1.5

2.0

2.5

omega2

0 5 10 15 20 25 30 35

0.5

1.0

1.5

2.0

omega1

3-mode Oscillator

Discrete Control + Continuous Dynamics

dReach: δ-Reachability Analysis of Hybrid Systems

flow1:

du

dt= ! !

u

"o1

dv

dt=1! v

"v1

!

dw

dt=

1!u

"w

"!w

"w1

!+

"w2

!!"

w1

!

1+ e!2kw

!(u!uw

!)

flow2:

du

dt= ! !

u

"o2

dv

dt= !

v

"v2

!

dw

dt=

w"

*!w

"w1

!+

"w2

!!"

w1

!

1+ e!2kw

!(u!uw

!)

flow3:

du

dt= ! !

1

"so1+

"so2!"

so1

1+ e!2kso (u!uso )

+w

2 " (1+ e!2ks (u!us ) ) ""

si

dv

dt= !

v

"v2

!

dw

dt= !

w

"w

+

flow4:

du

dt= ! +

v(u!"v )(uu !u)

# fi

!1

# so1 +# so2 !# so11+ e

!2kso (u!uso )

+w

2 " (1+ e!2ks (u!us ) ) "# si

dv

dt= !

v

# v2!

dw

dt= !

w

# w+

u !!o

u <!o

Mode 1

u !!w

u <!w

u !!v

u <!v

Mode 2 Mode 3 Mode 4

(b)

flow1:

du

dt= ! !

u

"o1

dv

dt=1! v

"v1

!

dw

dt=

1!u

"w

"!w

"w1

!+

"w2

! !"w1

!

1+ e!2kw

!(u!uw

!)

ds

dt=

1

1+ e!2ks (u!us )

! s#

$%

&

'(1

"s1

flow2:

du

dt= ! !

u

"o2

dv

dt= !

v

"v2

!

dw

dt=

w"* !w

"w1

!+

"w2

! !"w1

!

1+ e!2kw

!(u!uw

!)

ds

dt=

1

1+ e!2ks (u!us )

! s#

$%

&

'(1

"s1

flow3:

du

dt= ! !

1

"so1+

"so2!"

so1

1+ e!2kso (u!uso )

+w " s

"si

dv

dt= !

v

"v2

!

dw

dt= !

w

"w

+

ds

dt=

1

1+ e!2ks (u!us )

! s#

$%

&

'(1

"s1

flow4:

du

dt= ! +

v(u!"v )(uu !u)

# fi

!1

# so1 +# so2 !# so11+ e

!2kso (u!uso )

+w " s

# si

dv

dt= !

v

# v2!

dw

dt= !

w

# w+

ds

dt=

1

1+ e!2ks (u!us )

! s#

$%

&

'(1

# s1

u !!o

u <!o

Mode 1

u !!w

u <!w

u !!v

u <!v

Mode 2 Mode 3 Mode 4

(a)

Cardiac Cell Model




depict the dynamical changes of proliferation rates inducedby perturbing androgen levels that are di�cult for previousmodels (e.g. [20]) to capture. It also addresses the variabil-ity in individual patients and is able to accurately reproducethe datasets of di↵erent patients.

– Second, we obtain interesting insights on CRC prolifera-tion dynamics through analysis of the nonlinear model. Ourresults support the hypothesis that the physiological level ofandrogen reduce CRCs [20], while rule out other hypotheses,for instance, CRCs proliferate at a constant rate [32].

– Third, we propose a computational framework for iden-tifying patient-specific IAS schedules for postponing the po-tential cancer relapse. Specifically, we obtain personalizedmodel parameters by fitting to the clinical data in orderto characterize individual patients. We then use �-decisionproduces and bounded model checking to predict therapeu-tic strategies.

Through this case study, we aim to highlight the oppor-tunity for solving realistic biomedical problems using formalmethods. In particular, methods based on �-reachabilityanalysis suggest a very promising direction to proceed.

Related Work. We perform parameter synthesis, which re-quires the computation of concrete trajectories and param-eter values. This can not be done by simply computingan over-approximation of the forward reachable set. Con-sequently, reachable set computation tools such as SpaceEx[11] and Flow* [7] can not be directly used. There exists vari-ous approaches for performing parameter synthesis throughextra refinement on the reachable sets [10, 2, 12], but arerestricted to dynamics that are much simpler than the mod-els we encounter here. On the other hand, other SMT-basedmethods for hybrid systems [8, 9], which can perform param-eter synthesis in a similar manner, mostly focus on e�cienthandling of complex discrete transitions but are restrictedto models with simpler continuous dynamics.

The rest of the paper is organized as follows. We de-scribe our model in Section 2 and present preliminaries on�-reachability analysis in Section 3. In Section 4, we presentthe biological insights we gained through this case study, aswell as the model-predicted treatment schemes for individ-ual patients. In the final section, we summarize the paperand discuss future work.

2. A HYBRID MODEL OF PROSTATE CAN-CER PROGRESSION

In this section, we propose a hybrid automata based modelin order to reproduce the clinical observations [4, 5] of prostatecancer cell dynamics in response to the IAS therapy. It isknown that the proliferation and survival of prostate cancercells depend on the levels of androgens, specifically testos-terone and 5↵-dihydrotestosterone (DHT). Here we considertwo distinct subpopulations of prostate cancer cells: hor-mone sensitive cells (HSCs) and castration resistant cells(CRCs). Androgen deprivation can lead to remarkable de-creases of the proliferation and survival rates of HSCs, butalso up-regulates the conversion from HSCs to CRCs, whichwill keep proliferating under low androgen level. The corre-sponding hybrid automata model is shown in Figure 1.

Our model is based on previous models developed by [22,21, 20]. It takes into account the population of HSCs, thepopulation of CRCs, as well as the serum androgen concen-

dx

dt=

!x

1

1+ e!(z!k1 )k2

"

#$

%

&'!"x

1

1+ e!(z!k3 )k4

"

#$

%

&'

!m11!

z

z0

"

#$

%

&'!#x

"

#

$$$$

%

&

''''

x +µx

dy

dt=m

11!

z

z0

"

#$

%

&'x + !y 1! d

z

z0

"

#$

%

&'!"y

"

#$$

%

&''y

dv

dt=

!x

1

1+ e!(z!k1 )k2

"

#$

%

&'!"x

1

1+ e!(z!k3 )k4

"

#$

%

&'

!m11!

z

z0

"

#$

%

&'!#x

"

#

$$$$

%

&

''''

x +µx

+m11!

z

z0

"

#$

%

&'x + !y 1! d

z

z0

"

#$

%

&'!"y

"

#$$

%

&''y

jump1!2 :

x + y " r0

#dx

dt+dy

dt< 0

$w % tmax 0

:jump

1

12

>+!"+

#

dt

dy

dt

dxryx

Mode 1 (on-treatment)

Mode 3 (dummy)

jump1!3 :

v =v(0)

2

dz

dt=!z

!+µ

z

dz

dt=z0! z

!+µ

z

Mode 2 (off-treatment)

Control (cancer therapy) Plant (cancer progression)

||

Figure 1: A hybrid automaton model for prostate

cancer hormone therapy. Symbol “||” denotes the

parallel composition of the two automata.

tration, represented as x(t), y(t), and z(t), respectively. Inaddition, it also includes the serum prostate-specific antigen(PSA) level v(t), which is a commonly used biomarker forassessing the total population of prostate cancer cells. Themodel has two modes: on-treatment mode and o↵-treatmentmode (note that the auxiliary Mode 3 will only be usedin Section 4.2). Following [20], in the o↵-treatment mode(Mode 2), the androgen concentration is maintained at thenormal level z0 by homeostasis. In the on-treatment (Mode1), the androgen is cleared at a rate 1

⌧

. Further, we alsointroduce a basal androgen production rate µ

z

, in order toreproduce the measured basal testosterone levels in responseto androgen suppression [4, 5].The net growth rate of x(t) equals to (prolif

x

� apopx

�conv

x

)·x(t), where prolifx

, apopx

and convx

denote the pro-liferation, apoptosis and conversion rates, respectively. Inprevious studies such as [22, 21, 20], the prolif

x

and apopx

were modeled using Michaelis-Menten-like (MML) functions,

in the form of Vmax

+ (1� Vmax

) z(t)z(t)+Km

, where Vmax

andK

m

are kinetic parameters. This approach will result in an-drogen response curves as shown in Figure 2(a). In particu-lar, when one decreases the androgen level starting from thenormal level, prolif

x

(or apopx

) begins to decrease (or in-crease) first slowly and then fast until a su�ciently low levelof androgen is reached. However, this is inconsistent withthe clinical observations presented in [4, 5]. The data showthat for most of the patients, androgen suppression aroundnormal level will induce an immediate decrease of the PSAlevel, which implies an fast decrease (or increase) of prolif

x

(or apopx

). Therefore, instead of the MML functions, weadopt sigmoid functions, in the form of 1

1+exp(�(z(t)�k1)·k2),

to model prolifx

and apopx

. The corresponding androgenresponse curves are shown in Figure 2(b). Following [20],we model the conversion rate, proliferation rate and theapoptosis rate of y(t) as m1(1 � z(t)

z0), ↵

y

(1 � d z(t)z0

) and

�y

, respectively. The PSA level v (ng ml�1) is defined asv(t) = c1 · x(t) + c2 · y(t).The transitions between two modes depends on the val-

ues of v, dv/dt and an auxiliary variable w, which measuresthe time taken in a mode. Specifically, for each patient westarts with mode 1 to apply the treatment. When the PSAlevel drops to certain threshold r0 or w hits time out thresh-old t

max

, the treatment will be suspended. When the PSA

Prostate Cancer Model



Can a hybrid system run into an unsafe region of its state space?


⟦H⟧

⟦unsafe⟧

⟦H⟧

⟦unsafe⟧

Can a hybrid system run into an unsafe region of its state space?

The standard bounded reachability problems for simple hybrid systems are undecidable.

Unreachable(Safe)

Reachable(Unsafe)



1. Give up 2. Don’t give Up A. Find a decidable fragment and solve it B. Use approximation


Given δ ∈ ℚ⁺, ⟦H⟧ and ⟦unsafe⟧ over-approximate ⟦H⟧ and ⟦unsafe⟧δ δ

δ-reachability problem asks for one of the following answers:

- Decidable for a wide range of nonlinear hybrid systems - polynomials, log, exp, trigonometric functions, …

Unreachable(Safe)

δ-reachable(Unsafe)

⟦H⟧

⟦unsafe⟧

δ

δ

⟦H⟧

⟦unsafe⟧

δ

δ


Given δ ∈ ℚ⁺, ⟦H⟧ and ⟦unsafe⟧ over-approximate ⟦H⟧ and ⟦unsafe⟧δ δ

δ-reachability problem asks for one of the following answers:

- Decidable for a wide range of nonlinear hybrid systems - Reasonable complexity bound (PSPACE-complete)

Unreachable(Safe)


⟦H⟧

⟦unsafe⟧

δ

δ

⟦H⟧

⟦unsafe⟧

δ

δ


Unreachable(Safe)

1. “Unreachable” answers is sound.

⟦H⟧

⟦unsafe⟧

δ

δ


2. Analysis is parameterized with δ

⟦H⟧

⟦unsafe⟧

δ

δ


⟦H⟧

⟦unsafe⟧

δ

δ

Unreachable with smaller δ (Safe)


3. Robustness: If your system is δ-reachable under a reasonably small δ, then a small error can lead your system to an unsafe state

Unreachable (Unsafe)

⟦H⟧

⟦unsafe⟧

δ

δ


“δ-reachability analysis checks robustness which implies safety.”


dReach

Hybrid System Model + Specification

(drh format)BMC

Encoder

dReal (δ-complete SMT solver)

SMT2formula

Numerical Error (δ)

Unrolling Depth(k)

δ-SAT

UNSAT

δ-reachable+ Counterexample

(Visualization)

Unreachable

DPLL<T>

SATSolver

ICP Solver

ODESolver

NonlinearConstraint

Solver

- Open Source (GPL3), available at https://dreal.github.io- Support polynomials, transcendental functions and nonlinear ODEs- Formulas with 100+ ODEs have been solved.

https://dreal.github.io

– A mode definition consists of mode id, mode invariant, flow, and jump. id isa unique positive interger assigned to a mode. An invariant is a conjuctionof logic formulae which must always hold in a mode. A flow describes thecontinuous dynamics of a mode by providing a set of ODEs. The first formulaof jump is interpreted as a guard, a logic formula specifying a condition tomake a transition. Note that this allows a transition but does not force it.The second argument of jump, n denotes the target mode-id. The last one isreset, a logic formula connecting the old and new values for the transition.

– initial-condition specifies the initial mode of a hybrid system and its initialconfiguration. goal shares the same syntactic structure of initial-condition.

#define D 0.45#define K 0.9[0, 15] x;[9.8] g;[-18, 18] v;[0, 3] time;

{ mode 1;invt: (v <= 0);

(x >= 0);flow: d/dt[x] = v;

d/dt[v] = -g - (D * v ˆ 2);jump: (x = 0) ==> @2 (and (x’ = x) (v’ = - K * v)); }

{ mode 2;invt: (v >= 0);

(x >= 0);flow: d/dt[x] = v;

d/dt[v] = -g + (D * v ˆ 2);jump: (v = 0) ==> @1 (and (x’ = x) (v’ = v)); }

init: @1 (and (x >= 5) (v = 0));goal: @1 (and (x >= 0.45));

Fig. 3: An example of drh format: Inelastic bouncing ball with air resistance.Lines 1 and 2 define a drag coefficientD = 0.45 and an elastic coefficientK = 0.9.Line 3 declares variables x, g, v, and time. At lines 4 - 7 and 8 - 11, we define twomodes – the falling and the bouncing-back modes respectively. At line 12, wespecify the hybrid system to start at mode 1 (@1) with initial condition satisfyingx ≥ 5 ∧ v = 0. At line 13, it asks whether we can have a trajectory ending atmode 1 (@1) while the height of the ball is higher than 0.45.

4.2 Command Line Options

dReach follows the standard unix command-line usage:

dReach <options> <drh file>

It has the following options:

Inelastic bouncing ball with air resistance

Input Format (drh) for Hybrid System

Logical Encoding of Reachability Problem

…

~x0

~x

t0

~x1~x

t1

~x2

~x

tk�1

~xk

~x

tk

Unsafe

Init

step 0 step 1 … step k

modeq0 modeq1 modeqk

flowq0(~x0, ~xt0, t0)


flowqk(~xk, ~xtk, tk)

jumpq0!q1(~xt0, ~x1)


jumpqk�1!qk(~xtk�1, ~xk)

9~x0, ~x1, . . . , ~xk9~xt0, ~x

t1, . . . , ~x

tk9t0, t1, . . . , tk

Init(~x0) ^ flowq0(~x0, ~xt0, t0) ^ jumpq0!q1(~x

t0, ~x1)^

flowq1(~x1, ~xt1, t1) ^ jumpq1!q2(~x

t1, ~x2)^

. . .

f lowqk(~xk, ~xtk, tk) ^ Unsafe(~xk)


step i

modeqi

flowqi(~xi, ~xti, ti)

~xi

~x

ti

8t 2 [0, ti] 8~x 2 X flowqi(~xi, ~x, t) =) invqi(~x)

How to encode a mode invariant


…

~x0

~x

t0

~x1~x

t1

~x2

~x

tk�1

~xk

~x

tk

Unsafe

Init

step 0 step 1 … step k

modeq0 modeq1 modeqk



flowqk(~xk, ~xtk, tk)



jumpqk�1!qk(~xtk�1, ~xk)

9~x0, ~x1, . . . , ~xk9~xt0, ~x

t1, . . . , ~x

tk9t0, t1, . . . , tk

Init(~x0) ^ flowq0(~x0, ~xt0, t0) ^ 8t 2 [0, t0] 8~x 2 X flowq0(~x0, ~x, t) =) invq0(~x) ^ jumpq0!q1(~x

t0, ~x1)^

flowq1(~x1, ~xt1, t1) ^ 8t 2 [0, t1] 8~x 2 X flowq1(~x1, ~x, t) =) invq1(~x) ^ jumpq1!q2(~x

t1, ~x2)^

. . .

f lowqk(~xk, ~xtk, tk) ^ 8t 2 [0, tk] 8~x 2 X flowqk(~xk, ~x, t) =) invqk(~x) ^ Unsafe(~xk)

dReach


(drh format)BMC

Encoder


SMT2formula


Unrolling Depth(k)

δ-SAT

UNSAT


(Visualization)

Unreachable

DPLL<T>

SATSolver

ICP Solver

ODESolver

NonlinearConstraint

Solver

l1 ^ l2 ^ · · · ^ ln

Input: - Box (search space) - List of constraints

Output: δ-sat or Unsat

How to Solve

Theory Solver

Main Algorithm: Interval Constraint Propagation

Pruning Branch

Main Algorithm: Interval Constraint Propagation

δ-sat Unsat

δ

pruning on Xt (Forward)

Pruning using ODEs

t

XtX0

T

Pruning using ODEs

t

XtX0

T�t �t �t


Pruning using ODEs

t

XtX0

T


Pruning using ODEs

t

XtX 0

tX0

T


t

T

Xt

X0

X 00

Pruning using ODEs

Pruning on X0

(Backward)

Invariant

t

Xt

X 0t

T

X0

Pruning using ODEs

Pruning with Mode Invariant

Visualization of Counterexample

dReach


(drh format)BMC

Encoder


SMT2formula


Unrolling Depth(k)

δ-SAT

UNSAT


(Visualization)

Unreachable

DPLL<T>

SATSolver

ICP Solver

ODESolver

NonlinearConstraint

Solver

Visualization of CounterexampleODE Visualization with dReach

0 5 10 15 20 25 30 35

Click and drag above to zoom / pan the data

Mode1 Mode2 Mode3 Mode1 Mode2 Mode3

0 5 10 15 20 25 30 35

-5-4-3-2-10

z

0 5 10 15 20 25 30 35

-2.0

-1.5

-1.0

-0.5

0.0

y

0 5 10 15 20 25 30 35

-1.5

-1.0

-0.5

x

0 5 10 15 20 25 30 35

0

2

4

6

8

tau

0 5 10 15 20 25 30 35

0.0

0.5

1.0

1.5

2.0

2.5

omega2

0 5 10 15 20 25 30 35

0.5

1.0

1.5

2.0

omega1

3-mode Oscillator Model

Demo(1 min)

Thank You

See You @ Tool Market (16:30-18:00, Octagon)

http://dreal.github.io

http://dreal.github.io

dReach: δ-Reachability Analysis for Hybrid Systems · In this section, we propose a hybrid...

Documents

Transcript of dReach: δ-Reachability Analysis for Hybrid Systems · In this section, we propose a hybrid...