Diffusion equations:from Euclidean space to Graphs

MATH 697 AM:ST

November 7th, 2017

Warm upConvolutions and differentiability

Consider the following:

A function K : Rd 7→ R which is infinitely differentiable, with

K(y) ≥ 0 ∀ y, K(y) = 0 if |y| ≥ 1,

∫RdK(y) dy = 1,

and which is spherically symmetric, that is

K(y) = K(y′) if |y| = |y′|.

For δ > 0, define

Kδ(y) = δ−dK(δ−1y).

Warm upConvolutions and differentiability

Consider a domain D ⊂ Rd, and f : D 7→ R an integrablefunction, ∫

D|f(x)| dx <∞

For δ > 0 define the domain

Dδ = {x | d(x, ∂D) > δ}

Warm upConvolutions and differentiability

Lemma

For x ∈ Dδ, define

fδ(x) = (f ∗Kδ)(x)

Then, fδ is infinitely differentiable in Dδ, and

∂αfδ(x) = (f ∗ ∂αKδ)(x)

Minima of FunctionalsRECAP

Hilbert’s 19th Problem (1900)

Show that the minima for the functional

J (f) =

∫DL(∇f, f, x) dx

are always analytic functions of x (under certain specificconditions we will not specify).

Minima of FunctionalsRECAP

Hilbert’s 19th Problem (1900)

1. Schauder (1930’s): If there exists a C1,α minimizer, thenthis minimizer must be analytic.

2. De Giorgi (1957), Nash (1958) showed that there were C1,α

minimizers. Their method relied greatly on new regularityestimates for partial differential equations.

Minima of FunctionalsRECAP

The Dirichlet Energy

Given a bounded domain D ⊂ Rd with smooth boundary, and acontinuous function

φ : ∂D 7→ R

ProblemFind a function f : D 7→ R equal to φ on ∂D, minimizing

J (f) =1

2

∫D|∇f |2 dx

Minima of FunctionalsRECAP

The Dirichlet Energy

Theorem

If φ is differentiable, there is a unique continuous function

f : D 7→ R, f = φ on ∂D,

which is infinitely differentiable in the interior of D, and

∆f = 0.

Mean Value PropertyRECAPThe MVP

Theorem

Let f : D 7→ R be a continouous function which is twicedifferentiable and harmonic in its interior.

Then, if x0 is an interior point of D and r is strictly smallerthan the distance from x0 to ∂D, we have that

(Average of f over ∂Br(x0)) = f(x0).

Harmonic Functions and the MVPHigher differentiability of harmonic functions

Suppose we are given a continuous function f : D 7→ R which istwice differentiable and harmonic in D.

Lemma

If x ∈ D is a distance larger than δ from ∂D, then

f(x) = (f ∗Kδ)(x)

Harmonic Functions and the MVPHigher differentiability of harmonic functions

Theorem

A continuous function f : D 7→ R which is twice differentiableand harmonic in D is always infinitely differentiable.

Furthermore, for any index α there is a constant C(d, α) suchthat in Dδ we have the estimate

|∂αf(x)| ≤ C(d, α)δ−|α|‖f‖L1(D).

Harmonic Functions and the MVPStability properties of Harmonic Functions

An important consequence of the higher differentiability andthe estimate above, is the following stability property forharmonic functions (hinted at last time).

If a sequence of harmonic functions converge (locally) uniformlyto a function, then this function is itself harmonic and thederivatives also converge (locally) uniformly.

The Fractional Laplacian

Last time, we defined the fractional Laplacian of orderα ∈ [0, 2], by the formula

Lαf(x) := C(d, α)

∫Rd

f(y)− f(x)

|y − x|d+αdy

the constant C(d, α) is explicitly defined but we will notconcern ourselves with its exact form.

We observed that (−L1) = (−∆)12 , and that in general

−Lα = (−∆)α2

The Fractional Laplacian

Last time, we defined the fractional Laplacian of orderα ∈ [0, 2], by the formula

Lαf(x) := C(d, α)

∫Rd

f(y)− f(x)

|y − x|d+αdy

the constant C(d, α) is explicitly defined but we will notconcern ourselves with its exact form.

We observed that (−L1) = (−∆)12 , and that in general

−Lα = (−∆)α2

The Fractional Laplacian

The Nonlocal Dirichlet Energy: given f : Rd 7→ R

Jα(f) =1

2C(d, α)

∫Rd

∫Rd

|f(x)− f(y)|2

|x− y|d+αdxdy

Minimization of Jα leads to the equation Lαf = 0, just as forthe case of the Laplacian and the L2 norm of |∇f |.

GraphsVertices, Edges, and weights

A (finite) weighted graph G is most typically, described as

G = (V,E,w)

V = a (finite) set, the set of vertices

E = a subset of V × V, the set of edges

w : E 7→ R the weight function

It is said that x ∼ y if (x, y) ∈ E,

GraphsVertices, Edges, and weights

Some simplificationIt is usually preferable to think simply of the graph as being aset G (forget about distinguishing V ), coupled with a(non-negative) weight function w,

w : G×G 7→ R

with w(x, y) denoted sometimes as wxy for any x, y ∈ G.

One can think as E being the set of (x, y)’s such that wxy > 0.

GraphsExamples

1. Subsets of Rp.

Let V = {x1, . . . , xN} be a subset of Rp, let

w(xi, xj) = h(xi − xj)

A popular choice is for h to be spherically symmetric, e.g.

1

Zσe−|xi−xj |

2

σ , χBσ(0)(xi − xj)

where σ > 0 is a tuning parameter.

GraphsExample

2. Subsets of a metric space.

Let V = {x1, . . . , xN} be a subset of (X, ρ), a metric space.Then, define

w(xi, xj) = h

(ρ(xi, xj)

2

σ2

)Popular selections for h are

h(t) =√t

h(t) = e−t

h(t) = χ[0,1](t)

GraphsSetup

Given a vertex x, it’s degree is the number

d(x) =∑y∈V

wxy

The normalized weight function is then defined by

K(x, y) :=1

d(x)wxy

so that for fixed x, K(x, ·) is a probability distribution over G,∑y∈V

K(x, y) = 1.

The Dirichlet Problem on graphs andit’s use in Semi-supervised learning

MATH 697 AM:ST

November 9th, 2017

Warm up

A path in a graph G is a sequence of points xi (i = 1, . . . ,m)such that

xi ∼ xi+1 for i = 1, 2, . . . ,m− 1

Two points x, y ∈ G are said to be connected if there exists apath with x1 = x and xm = y.

A subset D of G is said to be connected if given two points inD then they can be connected by a path exclusively made outof points in D.

Warm up

We consider the Laplacian of a function f : G 7→ R

∆f(x) =∑y∈G

wxy(f(y)− f(x)).

Then, we observe the following:

f(x0) ≥ f(x) ∀ x ∈ G⇒ ∆f(x0) ≤ 0.

Moreover, if f(x0) > f(x1) for at least one x1 ∼ x0, then

∆f(x0) < 0.

Graph LaplaciansRECAP

We define various operators all with equal claims to be called aLaplacian.

First, the Combinatorial Laplacian (or just the Laplacian)

∆f(x) =∑y∈G

(f(y)− f(x))wxy

Graph LaplaciansRECAP

We define various operators all with equal claims to be called aLaplacian.

Secondly, we have the Random Walk Laplacian

∆f(x) =∑y∈G

(f(y)− f(x))K(x, y)

Graph LaplaciansRECAP

We define various operators all with equal claims to be called aLaplacian.

Last but not least, we have the Symmetric Laplacian

∆f(x) =1√d(x)

∑y∈G

wxy

(g(y)√d(x)

− g(x)√d(x)

)

Graph LaplaciansRECAP

The Combinatorial Laplacian (or just the Laplacian)

∆f(x) =∑y∈G

(f(y)− f(x))wxy

The Random Walk Laplacian

∆f(x) =∑y∈G

(f(y)− f(x))K(x, y)

The Symmetric Laplacian

∆f(x) =1√d(x)

∑y∈G

wxy

(g(y)√d(x)

− g(x)√d(x)

)

The (strong) Maximum Principle

Consider a function f : G 7→ R such that

∆f(x) =∑y∈G

wxy(f(y)− f(x)) = 0 ∀ x ∈ G.

Let M = maxx∈G

f(x), and let x0 ∈ G be such that

f(x0) = M

Claim:

If x is connected to x0 then we have f(x0) = M

The (strong) Maximum Principle

Theorem

Let G be a connected graph, then the only functions f : G 7→ Rwhich are harmonic in G are the constants.

In other words: for a connected graph, the kernel of the linearmap ∆ is one dimensional and given by the constant function.

To have any interesting harmonic functions, we must ask theybe harmonic only in some portion of G.

The Comparison Principle

Theorem

Let G be a graph, and D ⊂ G a connected subset of the graph.Then, if f1, f2 : G 7→ R are such that

∆f1 = ∆f2 = 0 in D, and f1 ≤ f2 in G \D

Then, we have

f1 ≤ f2 in D

Measuring Smoothness

Let f : G 7→ R, and define the norms

‖f‖L2 :=

(∑x∈G|f(x)|2

) 12

‖f‖Hw :=

∑x∈G|f(x)|2 +

∑x,y∈G

|f(x)− f(y)|2ωxy

12

as well as

‖f‖Hw :=

∑x,y∈G

|f(x)− f(y)|2ωxy

12

Measuring Smoothness

While ‖f‖L2 measures simply the average “size” of f , ‖f‖Hwmeasures the average size of its oscillations.

Observe that if ‖f‖Hw = 0, then f must be constant in eachconnected component of the graph.

Thus, the larger ‖f‖Hw is with respect to ‖f‖L2 , the more thefunction f is oscillating with respect to its size.

The Laplacian is a symmetric operator

We introduce an inner product for functions in G,

〈f, g〉 =∑x∈G

f(x)g(x)

Henceforth assume that G has symmetric weights, that is

wxy = wyx

The Laplacian is a symmetric operator

Let ∆ be the combinatorial Laplacian, and observe that

∑x∈G

∆f(x)g(x) =∑x∈G

∑y∈G

(f(y)− f(x))wxy

g(x)

The Laplacian is a symmetric operator

Then, expanding this sum, we have

〈∆f, g〉 =∑x,y∈G

f(y)g(x)wxy −∑x∈G

f(x)g(x)d(x)

where d(x) =∑y∈G

wxy. Since wxy = wyx, it follows that

〈∆f, g〉 = 〈f,∆g〉

Therefore, ∆ is symmetric with respect to this inner product.

The Laplacian is a symmetric operatorPositivity

One further expression, which is reminiscent of the Greenidentity, is the following (to be proved later)

−〈∆f, g〉 = −12

∑x,y∈G

wxy(f(y)− f(x))(g(y)− g(x))

It follows that for any f : G 7→ R,

−〈∆f, f〉 ≥ 0

The Laplacian is a symmetric operatorPositivity

Furthermore, if G is connected, then

−〈∆f, f〉 = 0

if and only if f is equal to a constant.

With this knowledge in hand, we obtain a good picture of theeigenfunction decomposition associated to ∆.

Eigenfunctions of ∆

There is a family of functions

{φn}Nn=0, φn : G 7→ R

as well as numbers 0 = λ0 ≤ λ1 ≤ λ2 ≤ . . . ≤ λN , such that

−∆φn(x) = λnφn(x) ∀ x ∈ G.

Moreover, the φn are orthonormal,

〈φn, φm〉 = δnm.

Eigenfunctions of ∆Decrasing Smoothness of φn

Observe that

‖φn‖1Hw =∑x,y∈G

wxy(φn(y)− φn(x))2

= −2〈∆φn, φn〉

Therefore,

‖φn‖1Hw = 2λn‖φn‖2L2

As the λn is increasing with n, we see that the φn becomeincreasingly more oscillatory as n gets larger and larger.

The Heat Equation

We consider the heat equation in all of G, with some initialdatum u0 : G 7→ R,

u = ∆u in G× (0,∞),

u = u0 at t = 0.

The Heat Equation

By the orthogonality of the φn, it follows that u0 can beexpressed as

u0 =

N∑n=0

αnφn where αn = 〈φn, u0〉.

Therefore, for t > 0, the solution to the heat equation is

u(t) =

N∑n=0

αne−tλnφn

The Heat Equation

By the orthogonality of the φn, it follows that u0 can beexpressed as

u0 =

N∑n=0

αnφn where αn = 〈φn, u0〉.

Therefore, for t > 0, the solution to the heat equation is

u(t) =

N∑n=0

e−tλn〈φn, u0〉φn

The Heat Equation

By the orthogonality of the φn, it follows that u0 can beexpressed as

u0 =

N∑n=0

αnφn where αn = 〈φn, u0〉.

Therefore, for t > 0, the solution to the heat equation is

u(x, t) =

N∑n=0

∑y∈G

e−tλnφn(y)u0(y)φn(x)

The Heat Equation

This leads to an analogue of the heat kernel H(t, x, y), namely,a function such that

u(x, t) =∑y∈G

H(t, x, y)u0(y)

This H(t, x, y) is given via the eigenfunction decomposition

H(t, x, y) =

N∑n=0

e−tλnφn(x)φn(y)

The Dirichlet Problem Revisited

Fix some subset D ⊂ G, and suppose we are given a function

g : G \D 7→ R

Then, we aim to find f : G 7→ R, such that

f = g in G \D,

and with f minimizing the discrete Dirichlet energy

J(f) :=1

2

∑x,y∈G

wxy(f(x)− f(y))2

The Dirichlet Problem Revisited

Let fs = f + sφ, then

d

dsJ(fs) =

d

ds

1

2

∑x,y∈G

wxy(fs(x)− fs(y)2

=∑x,y∈G

wxy(fs(x)− fs(y)(φ(x)− φ(y))

At s = 0, this results in∑x,y∈G

wxy(f(x)− f(y)(φ(x)− φ(y))

The Dirichlet Problem Revisited

∑x,y∈G

wxy(f(x)− f(y)(φ(x)− φ(y))

=

∑x,y∈G

wxy(f(x)− f(y))φ(x)

− ∑x,y∈G

wxy(f(x)− f(y))φ(y)

Since wxy = wyx means that∑

x,y∈Gwxy(f(x)− f(y))φ(y) =

∑x,y∈G

wxy(f(y)− f(x))φ(x)

The Dirichlet Problem Revisited

∑x,y∈G

wxy(f(x)− f(y)(φ(x)− φ(y))

= 2∑x,y∈G

wxy(f(x)− f(y))φ(x)

= −2∑x∈G

φ(x)∑y∈G

wxy(f(y)− f(x))

= −2∑x∈G

φ(x)∆f(x)

The Dirichlet Problem Revisited

In conclusion,

d

ds |s=0J(fs) = −2

∑x∈G

φ(x)∆f(x)

In particular, ∑x∈G

φ(x)∆f(x) = 0

for any function φ : G 7→ R with φ(x) = 0 if x ∈ G \D.

The Dirichlet Problem Revisited

There is a unique minimizer f , and it is characterized by{Lf(x) = 0 if x ∈ D,f(x) = g(x) if x ∈ G \D.

Compare with the Dirichlet problem for the Laplacian

min

∫D|∇f(x)|2 dx, f = g on ∂D.

and the one for the fractional Laplacian (α ∈ (0, 2)),

min

∫R2d

(f(x)− f(y)|2|x− y|−d−2α dx, f = g in Rd \D.

The Dirichlet Problem Revisited

There is a unique minimizer f , and it is characterized by{Lf(x) = 0 if x ∈ D,f(x) = g(x) if x ∈ G \D.

Compare with the Dirichlet problem for the Laplacian

min

∫D|∇f(x)|2 dx, f = g on ∂D.

and the one for the fractional Laplacian (α ∈ (0, 2)),

min

∫R2d

(f(x)− f(y)|2|x− y|−d−2α dx, f = g in Rd \D.

The Dirichlet Problem Revisited

Thanks to the variational characterization of harmonicfunctions and the comparison theorem, we have the following:

Theorem

Let D ⊂ G be a connected subset of G. Then, giveng : G \D 7→ R there exists one, and exactly one, function fwhich solves {

Lf(x) = 0 if x ∈ D,f(x) = g(x) if x ∈ G \D.

The Dirichlet Problem Revisited

The solution to {Lf(x) = 0 if x ∈ D,f(x) = g(x) if x ∈ G \D.

Is given by

f(x) =∑

y∈G\D

g(y)P (x, y)

for a certain kernel P (x, y) computable from the wxy.