Newtons Method: EquationsNewtons method solves a system of nonlinear equations of the form,F(u) = 0,by a sequence of steps including the linear problemJk(uk)uk= F(uk),where,Ji ,j(uk) = Fiukj,and,uk+1= uk+ uk.This is continued untilF(uk) < tolnF(u0)where toln is an input nonlinear tolerance.Newtons Method: Basics

Newtons method is a rst-order Taylor series approximationF(u + u) = 0,F(u) + uFu 0Ju = F(u),

Forming J can be challenging for many problems. Analytic ornumerical evaluation ?Ji ,j = Fiuj,

Radius of convergence can be small. This is a function of the Taylorseries approximation. Damping / Globalization is often required.Choosing d ?uk+1= uk+ duk.Newton Iteration: Implementation1. Dene Tk=12. Do k = 1,Nnewt3. Evaluate F(Tk) (if F(Tk) 2< tol then exit)4. Evaluate J(Tk)5. Solve J(Tk)T = F(Tk)6. Update Tk+1= Tk+ T7. End DoStandard Newton-Krylov Methods

Using a Krylov method to solve Jkuk= F(uk)

Form both J and F(u)

Send J and F(u) to Krylov solver and use standard matrix vectormultiply.

Extract preconditioning directly from JInexact Newton Methods

Inexact Newton tolerance on Krylov solver for each Newton iterationlimits excessive Krylov iterations. Does prevent quadraticconvergence. Jkuk+ F(uk) 2< F(uk) 2, (1)

Under solving in early Newton iterations can increase radius ofconvergence.Jacobian-Free Newton-Krylov

Using a Krylov method to solve Jkuk= F(uk)

Newton-Krylov methods only need the action of the Jacobian matrixto construct the mthlinear iterate of ukukm = 0r0 + 1Jr0 + 2J2r0 + . . . + mJmr0wherer0 = Jkuk0 + F(uk)

The Matrix-vector product required by the Krylov method can beapproximated with a single function evaluationJv F(u + v) F(u)

The Jacobian-vector Product

Consider the two coupled nonlinear equationsF1(u1, u2) = 0, F2(u1, u2) = 0. The Jacobian matrix isJ =__F1u1F1u2F2u1F2u2__.

Working backwardsF(u + v) F(u)

=_____F1(u1+v1,u2+v2)F1(u1,u2)

F2(u1+v1,u2+v2)F2(u1,u2)

_____.The Jacobian-vector Product, cont.

Approximating F(u + v) with a rst-order Taylor seriesF(u + v) F(u)

F1(u1,u2)+v1F1u1+v2F1u2F1(u1,u2)

F2(u1,u2)+v1F2u1+v2F2u2F2(u1,u2)

,

which simplies tov1F1u1+v2F1u2v1F2u1+v2F2u2= Jv.Jacobian-Free Newton-Krylov : Refs

Standard PDE motivated references:P. N. Brown and Y. Saad, SIAM J. Sci. Stat. Comput., 11, pp.450-481 (1990)Tony F. Chan and Kenneth R. Jackson, SIAM J. Sci. Stat.Comput., 5, pp. 533-542 (1984)

Also see the monograph,C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations,SIAM, Philadelphia, 1995

Recent JFNK review article from the application perspectiveD.A. Knoll and D.E. Keyes, Jacobian-free Newton-Krylov methods:a survey of approaches and applications, J. Comput. Phys., 193, pp.357-397 (2004)Choosing

Choosing for a numerical derivative of f (x)f (x)x f (x + ) f (x)

= a(1.0 + |x|) where a is on order square root of round-o.a 1.0e 6 to 1.0e 8.

Choosing for the JFNK matvec. Many algorithms work forchoosing (see refs), one simple one is = 1N||v||2N

i =1a(1.0 + |ui |)where v is the Krylov vector and N is the system dimension. Lookslike an average of what would be done individually.

Why we use GMRESJacobian Times a Vector:Implementation

We needJv = ResT(T + v) ResT(T)

We have ResT(T) and v

Must evaluate an , then evaluate ResT(T + v), and nallyevaluate Jv.Evaluating y = Jv, MatrixFreeJv

Subroutine MatrixFreeJv(v,y,N)1. evaluate 2. evaluate Tpert = T + v3. evaluate w = ResT(Tpert) (call ResTemp)4. Do i = 1,N5. y(i ) = w(i )ResT(i )

6. End Do

This routine / function provides the same service to GMRES as thefunction AFUN in Matlab. This is a matrix-free AFUN which allowsyou to do JFNK with GMRES in Matlab.Standard Newton-Krylov vs JFNK

Standard: Using a Krylov method to solve Jkuk= F(uk)

Form both J and F(u)

Send J and F(u) to Krylov solver and use standard matrix vectormultiply.

Extract preconditioning directly from J

JFNK: Using a Krylov method to solve Jkuk= F(uk)

Form only F(u)

Send only F(u) to Krylov solver and use Matrix-free matrix vectormultiply.

Extract preconditioning from something less costly than JPreconditioning: Concept and Choices

In left preconditioning the Krylov method is solving the alteredsystemM1Ax = M1b

What is the meaning of M1vs that of A1?

We have two choices to make1. Do we use the same discrete operator to form M as we are using tosolve Ax = b ?2. How do we approximately invert M ?

GOAL: Reduce the number of Krylov iterations from N1 to N2 witha preconditioning process which cost less than (N1N2) Kryloviterations.Preconditioning JFNK (1 of 2)

Preconditioning is the KEY for ecient application to multiphysicsengineering problems.

Using right preconditioning one solves(JM1)(Mu) = F(u). (1)M symbolically represents the preconditioning matrix (or process)and M1the inverse of preconditioning matrix.

Actually realized through a two-step process. First solve(JM1)w = F(u), (2)for w with the Krylov methods. Then solve for u,u = M1w. (3)Preconditioning JFNK (2 of 2)

Right-preconditioned matrix-free version is:JM1v [F(u + M1v) F(u)] / . (4)

Required in step 1 of previous slide

Actually done in two steps (v is given):1. Preconditioning: Solve (approximately) for y in y = M1v .2. Perform matrix-free product Jy [F(u + y) F(u)] / .

Only the matrix elements required for the action of M1are formed.Preconditioning JFNK: Options (1 of 2)

Form J every Newton iteration, M J, and approximately invert Mas preconditioning.

There is NO advantage to this !!

Should perform the same as standard Newton Krylov

Form J every N Newton iterations, M J, and approximately invertM as preconditioning.

This can signicantly reduce the cost of forming J.

Strong Newton convergence maintained since GMRES is using mostcurrent version of J in matrix-vector multiply.

Use of same J for a number of Newton iterations is often referred toas Modied Newton Krylov (MNK).

We will consider this as a solver and a preconditioner.Preconditioning JFNK: Options (2 of 2)

Do not form J. Form M from a Picard linearization, andapproximately invert M as preconditioning.

Here M can typically be formed with less eort.

Often the matrix from Picard linearization has better properties withrespect to linear algebra.

A key concept in physics-based preconditioning.

Do not form J. Form M from operators which control numericalstiness (spread in eigenvalues) and approximately invert M aspreconditioning.

Here M is formed from targeted physical processes such as diusion,advection, reaction ...

This can often result optimal preconditioning cost-benet.