Nonlinear Equations: Root Finding

J. B. Snively (EP501)

Root Finding

For the continuous nonlinear function f(x), find the value x=α such that f(α)=0.

Or, transform to solve x=g(x), and find x=α, such that α=g(α) and f(α)=0.

Root FindingTwo Step Process:

1) Bound the solution2) Refine the solution

Closed Domain Methods

Open Domain Methods

• Interval Halving • False Position

• Fixed-point iteration • Newton’s Method • Secant Method

Trial and Error Methods

Root FindingTwo Step Process:

1) Bound the solution2) Refine the solution

Trial and Error Methods

Trial and Errora = minimum guess value

δa= guess increment

while |f(a)|>tol do (not converged)

a=a+δa

end.

We advance a until f(a) is sufficiently close to 0.

Trial and Errora = minimum guess value

δa= guess increment

while |f(a)|>tol do (not converged)

a=a+δa

end.

Advance a until f(a) is satisfactorily close to 0.Try for textbook problem (in degrees):

Let a = 30º to start, tol = 0.0001.

Root FindingTwo Step Process:

1) Bound the solution2) Refine the solution

Closed Domain Methods• Interval Halving • False Position

Bisection Methoda = lower bound guess value

b = upper bound guess value

while |f(c)|>tol do (while not converged)

c=(a+b)/2

if f(a)*f(c)<0 then (if different sign)

b=c

else

a=c

end.

We advance a until f(a) is sufficiently close to 0, or until (b-a)>tol, i.e., we can define tolerance in terms of how f(a) is to the root or how close a and b are to a single value.

Let a = 30º, b = 40º to start, tol = 0.0001.

False Position Methoda = lower bound guess value

b = upper bound guess value

while |f(c)|>tol do (while not converged)

c=b-f(b)(b-a)/(f(b)-f(a))

if f(a)*f(c)<0 then (if different sign)

b=c

else

a=c

end.

We advance a until f(a) is sufficiently close to 0, or until (b-a)>tol, i.e., we can define tolerance in terms of how f(a) is to the root or how close a and b are to a single value. Here, c is a midpoint calculation defined by a linear interpolation!

Root FindingTwo Step Process:

1) Bound the solution2) Refine the solution

Open Domain Methods• Fixed-point iteration • Newton’s Method • Secant Method

Root Finding

For the continuous nonlinear function f(x), find the value x=α such that f(α)=0.

Or, transform to solve x=g(x), and find x=α, such that α=g(α) and f(α)=0.

Fixed-Point Iteration

[Courtesy of A. Z. Liu]

Fall 2015 EP501 Numerical Methods Lecture 5

y

xx1

x2

x3

g(x1

)

g(x2

)g(x

3

)

y = x

y = g(x)

↵

y

xx1

x2

x3

g(x1

)

g(x2

)

g(x3

)

y = x

y = g(x)

↵

Figure 2: Illustration of fixed-point iteration. True solution is at the cross point between y = x and y = g(x).On the left arrows indicate how the iteration converges to the true solution while on the right, the iterationdoes not converge.

The convergence criterion is ����ei+1

ei

���� = |g0(⇠)| < 1. (1.16)

Convergence is linear since ei+1

is linearly dependent on ei

. Rapid convergence occurs when this ratio isclose to zero. Re-arranging f(x) to get a di↵erent g(x) may reduce |g0(x)| for faster convergence (but stilllinear).

Example: Solve f(x) = x2 � sin x = 0 in the range x = 0.5 to 1.We can define

x = g(x) = x2 � sin x + x (1.17)

and getg0(x) = 2x � cos x + 1 > 1 when x > 0.5. (1.18)

This g does not give a convergent iteration. Alternatively, we can define x = g(x) =p

sin x, and get

g0(x) =cos x

2p

sin x< 1. (1.19)

Using this g the iteration convergences.

The fixed point iteration is not a desired method because of its unreliability in convergence. It is introducedhere because the Newton’s method described next is also an iteration process and has some similar properties.

b. Newton’s Method

This is the most well-known and powerful method. The new estimate of the root xn+1

is calculated as

xi+1

= xi

� f(xi

)

f 0(xi

). (1.20)

This process continues until|x

i+1

� xi

| "1

and/or |f(xi+1

)| "2

. (1.21)

Graphical illustrationNewton’s method can also be obtained from the Taylor series

f(xi+1

) = f(xi

) + f 0(xi

)(xi+1

� xi

) + · · · . (1.22)

10

Converging Diverging

Fixed-Point Iterationx1 = guess value

g(xi)= where g(xi)=xi when f(xi)=0

while |f(xi)|>tol do

xi+1=g(xi)

end.

Advance xi until f(xi) is satisfactorily close to 0.

Let phi = 30º to start, tol = 0.0001.

Need to Define Problem:Try for textbook problem (in degrees):

Found Root

? (deg.)25 30 35 40

f(?)

25

30

35

40

a=32.0156f(a)=9.6426e-06k=4

Need to Define Problem:Try for textbook problem (in degrees):

Another fixed point solution, but leads to a different result, finding another root that may not be relevant to the problem.

Found (Wrong) Root

? (deg.)-20 -10 0 10 20 30 40

f(?)

-20

-10

0

10

20

30

40

a=-9.7467f(a)=-8.5049e-06k=17

Newton’s Method Iteration

x1 = guess value

while |f(xi)|>tol do

xi+1=xi-f(xi)/f’(xi)

end.

Advance xi until f (xi) is satisfactorily close to 0.

Have option to evaluate f ’(xi) numerically.

Secant Method Iterationx0 = guess value 0

x1 = guess value 1

while |f(xi)|>tol do

slope=(f(xi)-f(xi-1))/(xi-xi-1)

xi+1=xi-f(xi)/slope

end.

Advance xi until f(xi) is satisfactorily close to 0.