Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of...

100
Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm - 6 pm 1

Transcript of Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of...

Page 1: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Title: Multivariate Continuity and differentiation

Md. Masum MurshedLecturer

Department of Mathematics, R.U.25 November, 2011, Friday

Time: 4:50 pm - 6 pm

1

Page 2: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Limit of a Function

2

Figure- 1

f(x)

L

L - ε

L + ε

a a - δ a + δ

Page 3: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Limit of a FunctionFind the value of delta () and Epsilon ()

3Figure- 2

Page 4: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

4

q

q - r

q + r

p p - r p + r

(p , q)

Rectangular Neighborhood of a point in R2 Nr(p) = {x ϵ R : |x - p| < r} = (p – r, p + r).

Nr(q) = {y ϵ R : |y - q| < r} = (q – r, q + r).

Page 5: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

5

q

p

Circular Neighborhood of a point in R2

Nr((p, q)) = {x, y ϵ R : (x – p)2 + (y – q) 2 < r2}

(p , q)

r

Page 6: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

6

Circular Neighborhood of a point in R2

..

Page 7: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

7

Norm on a Vector Space

V

Vector Space

R

k

Norm

Page 8: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

8

Norm in Rn

Rn

Euclidean Norm is given by

Page 9: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

0(0, 0)

Br(0)

r

+ x = Br(x)

x

Mathematically

Mathematically

In E = R2

Open Ball and Open sets

9

Page 10: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Continuous Function

10

A function f(x) is said to be continuous x = a, if for every neighborhood Nε(f(a)) there is some neighborhood Nδ(a) such that if x ϵ Nδ(a) then f(x) ϵ Nε(L).

A function f(x) is said to be continuous x = a, if for every neighborhood Nε(f(a)) there is some neighborhood Nδ(a) such that if x ϵ Nδ(a) then f(x) ϵ Nε(L).

A function f(x) is said to be continuous x = a, if

(i) f(x) is well defined at x = a, and

(ii)

A function f(x) is said to be continuous x = a, if

(i) f(x) is well defined at x = a, and

(ii)

A function f (x) is said to be continuous x = a, if for every number ε > 0 (however small) there is some number δ > 0 (usually depending on ε and a) such that

| f(x) – f(a)| < ε whenever |x-a| < δ

A function f (x) is said to be continuous x = a, if for every number ε > 0 (however small) there is some number δ > 0 (usually depending on ε and a) such that

| f(x) – f(a)| < ε whenever |x-a| < δ

Page 11: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Continuous Function

11

Figure- 3

f(x)

f(a)

f(a) - ε

f(a) + ε

a a - δ a + δ

Page 12: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

CONTINUOUS FUNCTIONS

Observe that |x – x0| < δ means that x0 - δ < x < x0 + δ, or equivalently that x

belongs to the open interval (x0 - δ , x0 + δ).

Similarly, |f(x) - f(x0)| < ε means that f(x) belongs to the open interval (f(x0) - ε, f(x0) + ε).

Accordingly, the statement|x – x0| < δ implies |f(x) - f(x0)| < ε

is equivalent to the statement x ∈ (x0 - δ , x0 + δ) implies f(x) ∈ (f(x0) - ε, f(x0) + ε)

which is equivalent to the statement f [(x0 - δ , x0 + δ)] is contained in (f(x0) - ε, f(x0) + ε)

which is equivalent to the statement (x0 - δ , x0 + δ) is contained in f -1(f(x0) - ε, f(x0) + ε)

Page 13: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

CONTINUOUS FUNCTIONS Hence we can restate the previous definition as follows.A function f : ℝ → ℝ is continuous at a point p ∈ ℝ if for any open set Vf(p), containing f(p) there exists an open set UP containing p such that f [UP] Vf(p) or UP f -1[Vf(p)]. ℝ ℝ

Continuous at p

f(p)p

f

Figure- 4

Page 14: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Continuous Function

14

A function f is continuous x = a, if and only if for every open set S containing f(a) , f -1(S) is an open set containing a.A function f is continuous x = a, if and only if for every open set S containing f(a) , f -1(S) is an open set containing a.

A function f is continuous x = a, if and only if for every ε > 0 there exists δ > 0 such that |x-a| < δ implies | f(x) – f(a)| < εA function f is continuous x = a, if and only if for every ε > 0 there exists δ > 0 such that |x-a| < δ implies | f(x) – f(a)| < ε

A function f is continuous x = a, if and only if for all {xn}, x ≠ a

xn → a implies f (xn ) → f (a)

A function f is continuous x = a, if and only if for all {xn}, x ≠ a

xn → a implies f (xn ) → f (a)

For f : ℝ → ℝ we know

Page 15: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Contiuity of Multivariate FunctionHow can we extend this definition for scalar/vector/ matrix/function valued functions of vector/matrix/function keeping important properties intact??

For examples, i) f1(x)=Ax, ii) f2(A)=A-1, iii) g(A)=|A|, iv) μ(f)=∫xf(x)dx, f3(x)=xTAx etc.

We need norm /metric in both domain and co-domain.

Norm is better. Why??

f is continuous at a if and only if for all ε > 0, there exists δ > 0 s.t. ||x-a||1 < δ implies ||f(x)-f(a)||2 < ε

f is continuous at a if and only if for all ε > 0, there exists δ > 0 s.t. d1(x,a) < δ implies d2( f(x), f(a)) < ε

f is continuous at a if and only if for all {xn}, xn ≠a

Page 16: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Our Well-known Derivative

For f : I→R , where I, an interval of R , x ε interior of I

How can we extend this definition for scalar/vector/ matrix/function valued functions of vector/matrix/function keeping important properties intact??

In co-domain there must exist the concept of subtraction. General vector spaces does have.

For examples, i) f1(x)=Ax, ii) f2(A)=A-1, iii) g(A)=|A|, iv) μ(f)=∫xf(x)dx, f3(x)=xTAx etc.

In vector/matrix/function space we can define limit with metric but not division. We cannot define division in general vector spaces

Page 17: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Math

ematical C

on

cepts

Figure- 5

Page 18: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

A function f of n variables is a rule that assigns to each ordered n-tuple of real numbers (x1,…, xn) in a set D a unique real number denoted by f (x1,…, xn).

Study of Multivariate Functions

We often write z= f (x1,…, xn) to make explicit the value taken on by f at the general point (x1,…, xn).

The set D is the domain of f and its range is the set of values that f takes on, that is,

The variables x’s are independent variables and z is the dependent variable.

DEFINITION

Page 19: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Study of Multivariate Functions

0

D

y

(x, y)

(a, b)

0 zx

f (a, b)f

f (x, y)

Domain of f

y

0-1

-1

x =1

x + y + 1 =0

x

Figure- 6

Figure- 7

Page 20: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Study of Multivariate Functions2:f R R

Figure- 9

Figure- 8

Page 21: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

DEFINITION

Graph of Multivariate Functions

Figure 11

If f is a function of two variables with domain D, then the graph of is the set of all points (x, y, z) in R3 such that z=f (x, y) and (x, y) is in D.

Figure- 10

Page 22: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Graph of Multivariate Functions

Figure- 12 Figure- 13

Page 23: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Graph of Multivariate Functions

Figure 14

Page 24: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Given a function f :Rn R and a real number c, we call the set, L = {(x1, x2,…., xn)| f(x1, x2,…., xn) = c} a level set of f at level c.

Graph of Multivariate FunctionsDEFINITION

We also call L a contour of f. When n = 2, we call L a level curve of f and when n = 3 we call L a level surface of f.

A plot displaying level sets for several different levels is called a contour plot.

Figure- 15

Page 25: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Figure 16: Level surface of f(x, y)=2x2 +y2

Figure 17: The parabloid f(x, y)=2x2 +y2

Figure- 18

Page 26: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Graph of Multivariate Functions

Figure- 19Figure- 20

Page 27: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

The graph of h (x, y)=4x2+y2

is formed by lifting the level curves.

Graph of Multivariate Functions

Figure- 21

Page 28: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Standardard Ellipsoid

X2+2y2+3z2 =1

Very im

portant in Multivariate

Statistics

Figure- 22

Page 29: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

29

Bivariate Gaussian

Its Importance in Statistics

Page 30: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

30

Spherical, diagonal, full covariance

UMFigure- 23

Page 31: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

April 20, 2023Md. Masum Murshed , Dept. of

Mathematics, RU31

Limits of functions f : R2→R

Note that in order for

to exist, it must have the same value regardless of the approach of (x, y) to (x0, y0).

It follows that if two different approaches give different values, the limit cannot exist.

For a limit in one variable to exist, the limits from both sides must yield the same value.

The analogous result for 2 variable limits is that the limit exists only if the limits along every curve through (x0, y0) are the same.

Page 32: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Continuity Checking

Figure- 24

Page 33: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Continuity Checking

Figure- 25

Page 34: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Line in Vector Space

y + x

y - x

y

x

-x

Λ(y-x)

x+Λ(y-x) = (1- Λ)x+ Λy

Figure- 26

Page 35: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

x

y

Line through x and yLine through x and y (1-λ) (1-λ) xx + λ + λ yy

λ = 0

λ = 1

λ = 1/2

λ = -0.3

λ = 1.3

Figure- 27

Page 36: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Line in Vector Space

X

Y

a

U||u||=1

a+hu

a+h(0,1)

a+h(1,0)

Figure- 28

Page 37: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Study of Multivariate FunctionsGiven this function, we can move in the direction of the

red, black or green vector and can measure rate change of the

but how??.

function in the direction,

Figure- 29

Page 38: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

2x2 + y2 = 3(zero slope)

(x,y)=(1,1)

direction (-1,-1)steep rate of ascent

direction (1,1)steep rate of descent

In order to talk about the slopeof a function we must specify direction

Study of Multivariate Functions

Let F(x,y) = 4 – 2x2 –y2

Figure- 30

Page 39: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Study of Multivariate FunctionsIf we can measure rate change of the function in the direction or in other direction, can we measure the rate of change of the function all around from rates of range in all directions?

Figure- 31

Page 40: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Study of Multivariate FunctionsFor example, suppose a bug moves in the direction of v = (-1, -1). If we let

Note: f(x, y) = 4 – 2x2 –y2

Page 41: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Directional Derivative

(black arrow).

Figure- 32

Page 42: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Directional Derivative

0

( ) ( )( ) limu

h

f c hu f cD f c

h

Where ||u||=1.

The definition is valid even if f is vector valued.

Given a point c and a unit vector u, then the directional derivative of f in the direction of u at c is given by

Definition

Page 43: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Partial DerivativesIf we consider f as a function of x = (x1,x2,….,xn) and let ek be the k-th standard basis vector, k=1,2,…, then we call the partial derivative of f with respect to xk at c, which is also denoted as

Now suppose f : ℝn → ℝ and define g : ℝ → ℝ by g(t)=f(t, x2,….,xn).

Page 44: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

1

1 2 1 1 21 2

0

1 2 1 2

0

1 2 1 2

0

1 1

0

(( , ,......, ) ) ( , ,......, )' ( , ,......, ) lim

(( , ,......, ) ( ,0,0,...0)) ( , ,......, )lim

( , ,......, ) ( , ,......, )lim

( ) ( )lim

n nx n

h

n n

h

n n

h

h

f x x x he f x x xf x x x

hf x x x h f x x x

hf x h x x f x x x

hg x h g x

h

1'( )g x

Partial Derivatives

In other words, we may compute the partial derivative by treating x2,…..xn as constants and differentiating with respect to x1 as we would in single-variable calculus

Page 45: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

If f is a function of two variables, its partial derivatives are the functions fx and fy defined by

h

yxfyhxfyxf

hx

),(),(lim),(

0

h

yxfhyxfyxf

hy

),(),(lim),(

0

RULE FOR FINDING PARTIAL DERIVATIVES OF z=f (x, y)1. To find fx, regard y as a constant and differentiate f (x, y) with respect to x.2. To find fy, regard x as a constant and differentiate f (x, y) with

respect to y.

Page 46: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

NOTATIONS FOR PARTIAL DERIVATIVES

fDfDfx

zyxf

xx

ffyxf xxx

11),(),(

fDfDfy

zyxf

yy

ffyxf yyy

22),(),(

If Z=f (x, y) , we write

The definition of PARTIAL DERIVATIVES is valid even if f is vector valued.

Page 47: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Partial Derivatives: Emamples

For examplef(x, y)=3x2 – 4xy2

Treating y as a constant and differentiating with respect to x

2( , ) 6 4x

ff x y x y

x

Treating x as a constant and differentiating with respect to y

( , ) 8y

ff x y xy

y

2 2

8ff

yx y y x

Page 48: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

The partial derivatives of f at (a, b) arethe slopes of the tangents to C1 and C2.

Geometric Meanings of Partial Derivatives

Figure- 33

Page 49: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

The Gradient

That is, we may view the gradient as a function which takes an n-dimensional vector for input and returns another n-dimensional vector. We call a function of this type a vector field.

Each partial derivative may be regarded as a multivariate function of x.

The n-vector of these partial derivatives is then a vector function of x, termed the gradient vector of F, and will be denoted by ∇ F(x) or g(x):

For example, if F(x) is defined by F(x)=x1x2

2+x2cos(x1), then

Notice that if f : ℝn → ℝ then ∇f : ℝn → ℝn

Page 50: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

The Gradient of a Linear Function

• If the gradient of F is a constant vector, F is said to be a linear function of x:

( ) TF x x ( )

constant

F x

1 2

1 2

( ) 5 3 10

5, 3, 10

F x x x

F F

x x

Example

Page 51: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Gradients and Lines and PlanesWe recall that a one-dimensional derivative defines the slope of the

tangent line to the curve defined by F(x).

Similarly, for a differentiable multivariate function the tangent hyperplane at the point is defined by the gradient vector.

Figure- 34 Figure- 35

Page 52: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Figure- 36

Page 53: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Geometric Description of Gradient

For example, consider the quadratic form f(x) = ½ xTAx – bTx + c

This quadratic form is simply a scalar, quadratic function of a vector. 3 2 2

02 6 8

A b c

Figure- 37Figure- 38

Page 54: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Geometric Description of GradientFor example, consider the quadratic form

f(x) = ½ xTAx – bTx + c

2 2

3 2 2 0

2 6 8

1( , ) (3 6 4 ) 2 8

2

A b c

F x y x y xy x y

3 2 2 3 2 2( , )

6 2 8 2 6 8

x y xF x y

y x y

. .,

0(2, 2)

0

2(0,0)

8

e g

F

F

Page 55: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Geometric Description of Gradient

Note that the gradient vectors in the middle figure are orthogonal to the level curves of the contour plot (remember that the gradient is defined in the xy-plane).

Show that where ( ) 2f x Ax ( ) Tf x x Ax

Figure- 39Figure- 40Figure- 41

Page 56: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Geometric Description of GradientThis generalizes to higher dimensions. The gradient is normal to the level surface at that point.

For example, in this figure we see the level surface x2+y2+z2=1 (of the function F(x,y,z)= x2+y2+z2, which we obviously cannot plot in three dimensions).

Since at the level surface F(x,y,z)=constant, the function values don't change with change in x, y or z, thus the directional derivative is zero for all directions u that are tangential to the level surface:

All these tangential u's also define a tangent plane. Since this implies that the gradient is orthogonal to u and thus to the tangent plane.

Next slide

Page 57: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Geometric Description of Gradientgradient: orthogonal tosurface

Directional derivativein direction of u (tangent to surface) iszero

Figure- 42

Page 58: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Relation between the Gradient and the Directional Derivative

Later we shall show that

By C-S inequality

Page 59: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Relation between gradient and directional derivative

The gradient vector points in the direction of the maximum rate of increase of the function, and the negative of the gradient vector points in the direction of the maximum rate

of decrease of the function. The length of the gradient vector tells us the rate of increase/decrease

Page 60: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

The gradient at a point has the following properties:

•The gradient direction at a point is in the direction of steepest ascent at that point

•The gradient magnitude is the steepness of that ascent

•The gradient direction is the normal to the level curve (or surface) at that point

•The gradient defines the tangent curve (or plane) at that point

•The gradient can be used to calculate the first derivative (i.e., directional derivative) in any direction.

Relation between Gradient and Directional Derivative

Page 61: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Relation between Gradient and Directional Derivative

( , ) ( 4 , 2 )f x y x y .

Thus

.

(1,1) ( 4, 2)f

Page 62: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Hence, if a bug standing above (1,1) in the xy-plane on the graph of f wants to head in the direction of most rapid ascent, it should move in the direction

of the unit vector (1,1) 1(2,1)

(1,1) 5

fu

f

.

If the bug wants to head in the direction of most rapid descent, it should move in the direction of the unit vector –u. Moreover,

2 2(1,1) (1,1) 4 2 20

(1,1) (1,1) 20

u

u

D f f

D f f

Relation between Gradient and Directional Derivative

Page 63: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Relation between the Gradient and the Directional Derivative

Scaled gradient vectors for f(x,y)=4-2x2-y2

f(x,y)=4-2x2-y2 with path of most rapid ascent from (1,1,1) y x

Figure- 43

Figure- 44

Page 64: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Recall that when we zoom in on a “sufficiently nice” function of two variables, we see a plane.

Relation between Continuity, Gradient and Directional Derivative

Figure- 45

Page 65: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

What is meant by “sufficiently nice”?

Suppose we zoom in on the function z=f(x,y) centering our zoom on the point (a,b) and we see a plane. What can we say about the plane?

• The partial derivatives for the plane at the point must be the same as the partial derivatives for the function.

• Therefore, the equation for the tangent plane is

( , ) ( , ) ( , )( ) ( , )( )f f

L x y f a b a b x a a b y bx y

Page 66: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

In particular. . .The Partial Derivatives Must Exist

If the partial derivatives don’t exist at the point (a,b), the function f cannot be locally planar at (a,b).

Example: A cone with vertex at the origin cannot be locally planar there, as it is clear that the x and y cross sections are not differentiable there.

Figure- 46

Page 67: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Not enough: A Puny Condition

Suppose we have a function

1 if 0 or 0( , )

0 if neither nor is 0

x yf x y

x y

Notice several things:

•Both partial derivatives exist at x=0. •The function is not locally planar at x=0.

•The function is not continuous at x=0.

Whoa! The existence of the

partial derivatives doesn’t even

guarantee continuity at the point!

Page 68: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Directional Derivatives?

A surprising fact is that a function can have direction derivative in every direction at a point without being continuous at that point

2 2 4( , ) 0

0( , )

xy x y x y

Otherwisef x y

0

( ) ( )( ) limu

h

f c hu f cD f c

h

1 2

2

1 2 1 22 2 4( , ) 0 01 2

( , )((0,0))

a a h h

f ha ha aaD f Lt Lt

h a ha

Page 69: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Our Well-known Derivative

(1) )()(

)( lim0 h

xfhxfxf

h

For f: I→R , where I, an interval of R , xε interior of I

0)()()(

0)(

where),()()()(

0

0

h

hxfxfhxfLt

h

hoLthohxfxfhxf

h

h

It can be easily shown, is a linear map. It can be shown,.

RRxf :)(

under usual norm every linear map between finite-dimensional spaces is always continuous ( shown later)

Page 70: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Our Well-known Derivative

0||

|)()()(|

0)()()(

0

0

h

hxfxfhxfLt

h

hxfxfhxfLt

h

h

It can be easily generalized to Banach space valued function, f: 2211 ,, BB

0||||

||)()()(||

1

2

0

h

hxfxfhxfLt

h

is a (continuous)linear map. 1 2

( ) :f x B B

It can be shown,.every linear map between infinite-dimensional spaces is not always continuous.

Called F

réchet derivative

Page 71: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Main Results of Fréchet Derivative

Page 72: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

UA Banach Space

V,AnotherB-space

f,a nonlinear function

What is Differentiation?

Differentiation is nothing but local linearization

In differentiation we approximate a non-linear function locally by a (continuous) linear function

Page 73: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Linear Map and Matrices

Vn is a finite-dimensional vector space. Let v belongs to Vn

v=x1v1+x2v2+--------+xnvn

(x1, x2, --------- xn)nR

Every Vn is isomorphic to Rn

nn RV Its

Significance??

This isom

orphism is basis

dep

end

ent

Page 74: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Linear Map and Matrices

Every Vnm is isomorphic to

nmnmnm RRV

Its Significance??

This isom

orphism is basis

dep

end

ent

VnWm

k1v1+k2v2

k1L(v1)+k2L(v2))

L

L(k1v1+k2v2)

Vnm, the set of all such L’s is a vector space of dimension nm

nmR

=

Page 75: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Linear Map and Matrices

Linear mappings are almost omnipresent

If both domain and co-domain are both finite-dimensional vector space, each linear mapping can be uniquely represented by a matrix w.r.t. specific couple of bases

We intend to study properties of linear mapping from properties of its matrix

Page 76: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Normed spacesDefine the notion of the size of f, an element in F , a vector spaceNorm || f ||, || ||: F→[0,∞)1)||f||=0↔f=02) ||kf||=|k|||f||3) ||f+g|| <=||f||+||g||

0,f 0 0f f

1 2 1 2f f f f 1 2 1 2,d f f f f

Both R and Z are normed spaces are spaces with | |

Both Rn and are normed spaces are spaces with Euclidean norm,|| ||

n mR

Normed spaces become Banach spaces where every Cauchy sequence converges

Page 77: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

A norm on the vector space is a mapping, ,

satisfying the following axioms:

Vector Norms

. : 0; 0 0

. : | | ;

. : ;

a Positivity v v v

b Linearity cv c v

c Triangleinequality v w v w

1v

n RR

Page 78: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Examples of Vector NormsnRvvvv ,),(|||| 21.Euclidean Norm

ni Cvnivv },,...,1:||{max||||

2.p Norm ),1[,,||||||/1

1

pCvvv npn

k

pkp

3,Max Norm

4nnCA nonsingular ppA Avv |||||||| ,

is a vector norm on nC for any ],1[ p

5.Show that ||f||= sup|f(x)| defines a norm on B(X).

Page 79: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Properties of Norms

|| || 0 ||| || || || | 0

|| || || || 0

|| || || ||

n n

n

n

v u v u

v u

v u

, ,v u V

( ): f v vf V R

thenTheorem 1 If

Proofs

| || || || || | || ||v u v u

|| || || || || ||,|| || || || || || (2)u u v v v v u u

(1)

(2) Is a continuous function

| || || || || | || ||n nv u v u

Page 80: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

For any two norms and , defined in a finite

dimensional vector space, there exist positive constants

C≥c>0, such that:

Norms Equivalence

A B

BAB vCvvc,v

Proof (outline):

Due to property (b) of norms, it is enough to show (2) on the

unit sphere, . Due to property (c) the norm on

this sphere is a continuous function, thus it has the minimum

m and maximum M, which are both positive due to property

(a). Taking C=1/m and c = 1/M satisfies (2).

(2)

1vA

Page 81: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Norms Equivalence

|| || on S { :|| || 1}

|| |||| ||

B A A

B

A

m v M v v v

vm M

v

1|| ||

|| ||1 1 1

|| |||| ||

|| || || |||| ||

B

A

A

B

B BA

m v Mv

vM v mv v

vM m

xnxx1Show that

Page 82: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Matrix NormConsider the vector space M22 of 2 2 matrices. Let u and v defined as follows be arbitrary 2 2 matrices.

Prove that the following function is an inner product on M22.<u, v>= ae + bf + cg + dh

Determine the inner product of the matrices .

hg

fe

dc

bavu ,

Solution

Axiom 1:,<u, v>= ae + bf + cg + dh = ea + fb + gc + hd =<v, u>

Axiom 3: Let k be a scalar. Then

<ku, v>= kae + kbf + kcg + kdh = k(ae + bf + cg + dh) = k<u, v>

4)01()90()23()52( ,09

25

10

32

09

25 and

10

32

Page 83: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Consider the vector space M22 of 2 2 matrices. Let u and v defined as follows be arbitrary 2 2 matrices.

It is known that the function <u, v>= ae + bf + cg + dh is an inner product on M22 by Example 2.

The norm of the matrix is

hg

fe

dc

bavu ,

Matrix Norm

2 2 2 2, a b c d || u || u u

This norm is called Frobenius norm, denoted by || ||F.

Remember that it is generated from an inner product.We can define other norms in matrix spacesShow that

2|| || ( ) ( ) sum of eigen values of /T T T T

FA tr A A tr AA A A AA

Page 84: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Matrix Norms

Given ANY vector norm

nnC

on||||a CORRESPONDING matrix norm on

we can construct

}1||||:||||max{|||| vAvA

by

nC

Theorem .

0vProof. If

nnn CvCAvAAv ,||,||||||||||

then ||||||||0||0|||||| vAAv

If 0v then ||||||

||||||||||||

||||||||||

v

vAv

v

vvAAv

||||||||||||||||||

|| vAvv

vA

From this theorem you show that A is always continuous.

Page 85: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Matrix Norms

nnCBA ,

|||||||||||| BABA

thenTheorem 2 If

Proofs|||||||||||| BABA

|||||||||||||| BABA

||||||||||||max||||max||)(||max||||1||||1||||1||||

BABvAvvBABAvvv

)2(||||||||||||,|||||||||||| AABBBBAA

(1)

(2)

(3)

||||||||||)(||max||||1||||

BAvABABv

Page 86: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Examples of Induced Matrix Norms

}||{max||||11

n

j ijni aATheorem 2

i

Proof.

Choose

hencenjau ijj ,...,1),sgn(

then constructthat maximizes

n

j ija1

||

|| || 1v

by

}||{max||111

n

j ijni

n

j ij aa

1|||| unCu

}||{max||||11

n

j ijni aAv

hence }||{max||||11

n

j ijni aA

and n

j jiji uaAuAu1

|||)(|||||

Page 87: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

pxp

p

xp

Axx

AxA

p 10supsup

For instance,

m

iij

ja

11

maxA

m

jij

ia

1maxA

(max column sum) (max row sum)

21

max2)]([ AAA T AAT of eigenvalue maximum

Examples of Induced Matrix Norms

There are bridges between matrix induced norm :

1111

2

2

2

21

1

)(

1)(

AAvAAA

AvAA

vvvAA

T

T

T

Proof

AAA12

Page 88: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Definition

Page 89: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Theorem 1

Page 90: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Theorem 2

Page 91: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Theorem 2

Page 92: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Theorem 3

Page 93: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Theorem 4

Page 94: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Theorem 4

Page 95: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Page 96: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Theorem 4 shows that

2

Theorem 5: Identification Theorem

Page 97: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Fréchet Derivative of f:Rn→Rm

Page 98: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Sufficient Condition for Differentiability

No.

No.

No.

No.

Theorem 6

Page 99: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

Process of generalization of differentiation

Page 100: Title: Multivariate Continuity and differentiation Md. Masum Murshed Lecturer Department of Mathematics, R.U. 25 November, 2011, Friday Time: 4:50 pm -

100

Thank You All