Solving Multi-Linear Systems with M-Tensors · 2016. 7. 14. · complementarity problem can be...

Solving Multi-Linear Systems with M-Tensors

Yimin WEI

School of Mathematical SciencesFudan University, Shanghai, China

Joint work with Weiyang DING

[email protected] M-Equations

M-Tensor I

Definition (Z-tensor)We call a tensor A as a Z-tensor, if all of its off-diagonal entriesare non-positive.

Spectral radius of a tensor is defined by

ρ(A) ={|λ| : Axm−1 = λx[m−1], λ ∈ C, x ∈ Cn \ {0}

},

where

(Axm−1)i =n∑

i2,...,im=1

aii2...imxi2 . . . xim

andx[m−1] = [xm−11 , x

m−12 , . . . , x

m−1n ]

>.


M-Tensor II

Definition (M-tensor)We call a Z-tensor A = sI − B (B ≥ 0) as an M-tensor ifs ≥ ρ(B); We call it as a nonsingular M-tensor if s > ρ(B).

K. Chang, L. Qi, and T. Zhang, A survey on the spectral theory ofnonnegative tensors. Numer. Linear Algebra Appl., 20(6),891–912, 2013.

W. Ding, L. Qi, and Y. Wei. M-tensors and nonsingularM-tensors. Linear Algebra Appl., 439(10):3264–3278, 2013.

L. Zhang, L. Qi, and G. Zhou. M-tensors and some applications.SIAM J. Matrix Anal. Appl., 35(2):437–452, 2014.


If A is a Z-tensor, then the following conditions areequivalent:

(D1) A is a nonsingular M-tensor;(D2) Every real eigenvalue of A is positive;(D3) The real part of each eigenvalue of A is positive;(D4) A is semi-positive; that is, there exists x > 0 with Axm−1 > 0;(D5) There exists x ≥ 0 with Axm−1 > 0;(D6) A has all positive diagonal entries, and there exists a positive

diagonal matrix D such that ADm−1 is strictly diagonallydominant;

(D7) A has all positive diagonal entries, and there exist two positivediagonal matrices D1 and D2 such that D1ADm−12 is strictlydiagonally dominant;

(D8) There exists a positive diagonal tensor D and a nonsingularM-tensor C with A = DC;

(D9) There exists a positive diagonal tensor D and a nonnegative tensorE such that A = D − E and there exists x > 0 with(D−1E)xm−1 < x[m−1][email protected] M-Equations

Multi-Linear Equations I

Let A be an mth-order tensor in Cn×n×···×n and b be a vector inCn.

Then a multi-linear equation can be expressed as

Axm−1 = b.

E.g., if A ∈ R2×2×2, then Ax2 = b is a condense form of{a111x

21 + (a112 + a121)x1x2 + a122x

22 = b1,

a211x21 + (a212 + a221)x1x2 + a222x

22 = b2.


Multi-Linear Equations II

Denote the solution set of the multi-linear equation Axm−1 = b as

A−1b :={x ∈ Cn : Axm−1 = b

}.

Furthermore, when A and b are both real, we define the realsolution set, the nonnegative solution set, and the positive solutionset as

(A−1b)R :={x ∈ Rn : Axm−1 = b

},

(A−1b)+ :={x ∈ Rn+ : Axm−1 = b

},

(A−1b)++ :={x ∈ Rn++ : Axm−1 = b

}.

An M-equation is referred to

Axm−1 = b, (1)

where A = sI − B is an M-tensor. Particularly, when theright-hand side b is nonnegative or positive, we also require anonnegative or positive solution, respectively.


Applications I

Find one of the sparsest solutions to a tensor complementarityproblem

min ‖x‖0, s.t. Axm−1−b ≥ 0, x ≥ 0, x>(Axm−1−b) = 0;(2)

This is generally an NP-hard problem. Luo, Qi and Xiu (Optimization Letters, 2016, online) suggest that if the tensor A isa Z-tensor, then a sparsest solution of the above tensorcomplementarity problem can be achieved by solving the followingpolynomial programming problem

min ‖x‖1, s.t. Axm−1 = b, x ≥ 0. (3)

They prove that if A is a nonsingular M-tensor, then the problem(3) is uniquely solvable and the unique solution is also an optimalsolution to the problem (2).


Applications II

Discretize the partial differential equation (nonlinearKlein-Gordon equation) with Dirichlet’s boundary conditionlike {

u(x)m−2 ·∆u(x) = −f (x) in Ω,u(x) = g(x) on ∂Ω,

(m = 3, 4, . . . )

Analyze a necessary and sufficient condition for existence of apositive Perron vector. ( S. Hu and L. Qi, arXiv:1511.07759,SIMAX, 2016 )


Diagonal Systems I

The diagonal of a tensor A contains the entries aii ...i withi = 1, 2, . . . , n, and other entries are called off-diagonal.

A tensor is called diagonal if all its off-diagonal entries are zeros.

A diagonal equation is referred to

Dxm−1 = b,

where the coefficient mth-order tensor D is diagonal.


Diagonal Systems II

When m is even, the real solution set (D−1b)R has a uniqueelement x with xi = (bi/dii ...i )

1/(m−1).

When the diagonal of D and the vector b are positive, the solutionabove is also the unique element in (D−1b)++.

When m is odd, the real solution set (D−1b)R has at most 2nelements x with xi = ±(bi/dii ...i )1/(m−1) if dii ...ibi ≥ 0 for all i , orelse the real solution set is empty.

Further when the diagonal of D and the vector b are positive, thepositive solution set (D−1b)++ has a unique element x withxi = (bi/dii ...i )

1/(m−1).


Triangular Systems

The lower triangular part of a tensor A contains the entries ai1i2...imwith i1 = 1, 2, . . . , n and i2, . . . , im ≤ i1, and other entries are saidto be in the off-lower triangular part. The strictly lower partconsists of the entries ai1i2...im with i1 = 1, 2, . . . , n andi2, . . . , im < i1.

A tensor is called lower triangular if all its entries in the off-lowertriangular part are zeros.

A lower triangular equation is referred to

Lxm−1 = b,

where the coefficient tensor L is lower triangular.

The (strictly) upper triangular part of a tensor, upper triangulartensors and upper triangular equations can be defined analogously.


Forward Substitution

Algorithm (Forward Substitution)

If L ∈ Cn×n×···×n is lower triangular and b ∈ Cn, then thisalgorithm overwrites b with one of the solutions to Lxm−1 = b.b1 = one of the (m − 1)-st roots of b1/l11···1for i = 2 : n

for k = 1 : mpk =

∑{lii2...im ·

∏l=2,...,m

l 6=p1,...,pk−1

bil : i2, . . . , im ≤ i ,

ip1 , . . . , ipk−1 are the only k − 1 indices equal i}

endbi = one of the roots of p1 + p2t + · · ·+ pmtm−1 = bi

end


Triangular M-Equations

Now we consider the triangular equations satisfying that thecoefficient tensor is a nonsingular M-tensor.

For a triangular tensor, it is a nonsingular M-tensor if and only ifits diagonal entries are positive and its off-diagonal entries arenonpositive.

Proposition

Let L be an mth-order n-dimensional lower triangular M-tensor. Ifb is a nonnegative vector, then Lxm−1 = b has at least onenonnegative solution. Furthermore, if b is a positive vector, thenLxm−1 = b has a unique positive solution.


Triangular M-EquationsWhen we solve the equation, the coefficients p1, p2, . . . , pm−1 arenonpositive and pm is positive in each step. Thus the companionmatrix of p1 + p2t + · · ·+ pmtm−1 = bi , i.e.,

Ci =

0 1 0 · · · 00 0 1 · · · 0...

......

. . ....

0 0 0 · · · 1bi−p1pm

−p2pm

−p3pm

· · · −pm−1pm

,

is a nonnegative matrix when bi ≥ 0, and it is an irreduciblenonnegative matrix, which is not similar via a permutation to ablock upper triangular matrix, when bi > 0.Thus the polynomial equation in each step has at least onenonnegative solution xi = ρ(Ci ) if b is nonnegative (ρ(Ci ) is thespectral radius of matrix Ci ), and it has a unique positive solutionif b is positive.


A Fixed-Point Iteration

An M-equation is referred to

Axm−1 = b,

where A = sI − B is an M-tensor. When the right-hand side b isnonnegative, we require a nonnegative solution.

Consider the fixed-point iteration

xk+1 = Ts,B,b(xk) := (s−1Bxm−1k + s

−1b)[1/(m−1)], k = 0, 1, 2, . . . ,

where v[α] = [vα1 , vα2 , . . . , v

αn ]> denotes the componentwise power

of a vector.

It is easy to understand that each fixed point of this iteration is asolution of the above nonsingular M-equation, and vice versa.


Cones

Let E be a real Banach space. If P is a nonempty closed andconvex set in E and satisfies that

x ∈ P and λ ≥ 0 imply λx ∈ P, andx ∈ P and −x ∈ P imply x = 0,

then P is called a cone in E.

Furthermore, a cone P induces a semi-order in E, i.e., x ≤ y ify − x ∈ P. Let {xn} be an arbitrary increasing series in E with anupper bound, i.e., there exists y ∈ E such that

x1 ≤ x2 ≤ · · · ≤ xn ≤ · · · ≤ y.

If there must be x∗ ∈ E such that ‖xn − x∗‖ → 0 (n→∞), thenwe call P a regular cone.

Let T : D→ E be a map, where D is a subset in E. If x ≤ y(x, y ∈ D) implies T (x) ≤ T (y), then we call T an increasing mapon D.


A Fixed-Point Theorem

Theorem (Amann 1976)

Let P be a regular cone in an ordered Banach space E and[u, v] ⊂ E be a bounded order interval. Suppose thatT : [u, v]→ E is an increasing continuous map which satisfies

u ≤ T (u) and v ≥ T (v).

Then T has at least one fixed point in [u, v]. Moreover, thereexists a minimal fixed point x∗ and a maximal fixed point x∗ in thesense that every fixed point x̄ satisfies x∗ ≤ x̄ ≤ x∗. Finally, theiteration method

xk+1 = T (xk), k = 0, 1, 2, . . .

converges to x∗ from below if x0 = u, i.e., u = x0 ≤ x1 ≤ · · · ≤ x∗,and converges to x∗ from above if x0 = v, i.e.,v = x0 ≥ x1 ≥ · · · ≥ x∗.


Existence and Uniqueness of Positive Solutions I

By the above fixed-point theorem, we can study the existence ofthe positive solutions of the M-equations.

Theorem

If A is a nonsingular M-tensor, then for every positive vector b themulti-linear system of equations Axm−1 = b has a unique positivesolution.

We can rewrite the above theorem into an equivalent condition fornonsingular M-tensors, which generalizes the ‘nonnegative inverse’property of M-matrix to the tensor case.


Existence and Uniqueness of Positive Solutions II

Theorem

Let A be a Z-tensor. Then it is a nonsingular M-tensor if andonly if (A−1b)++ has a unique element for every positive vector b.

We introduce the notation A−1++b to denote the unique positivesolution to Axm−1 = b for a nonsingular M-tensor A and apositive vector b. Then A−1++b̂ ≥ A−1++b̃ > 0 if b̂ ≥ b̃ > 0.


Proof of existence

Recall that a nonnegative solution of Axm−1 = b is a fixed point of

Ts,B,b : Rn+ → Rn+, x 7→(s−1Bxm−1 + s−1b

)[1/(m−1)].

Note that Rn+ is a regular cone and Ts,B,b is an increasing continuousmap. When A is a nonsingular M-tensor, i.e., s > ρ(B), there exists apositive vector z ∈ Rn++ such that Azm−1 > 0. Denote

γ = mini=1,2,...,n

bi(Azm−1)i

and γ = maxi=1,2,...,n

bi(Azm−1)i

.

Then γAzm−1 ≤ b ≤ γAzm−1, which indicates that

γ1/(m−1)z ≤ Ts,B,b(γ1/(m−1)z

)and γ1/(m−1)z ≥ Ts,B,b

(γ1/(m−1)z

).

By the above fixed-point theorem, there exists at least one fixed point x̄of Ts,B,b with

γ1/(m−1)z ≤ x̄ ≤ γ1/(m−1)z,

which is obviously a positive vector when b is also positive.


Proof of uniqueness

Furthermore, we can prove that the positive fixed point is unique when bis positive. Assume that there are two positive fixed points x and y, i.e.,

Ts,B,b(x) = x > 0 and Ts,B,b(y) = y > 0.

Denote η = mini=1,2,...,n

xiyi

, thus x ≥ ηy and xj = ηyj for some j . If η < 1,

then A(ηy)m−1 = ηm−1b < b, which indicates

Ts,B,b(ηy) =[s−1B(ηy)m−1 + s−1b

][1/(m−1)]> ηy.

However, since Ts,B,b is nonnegative and increasing, we have

Ts,B,b(ηy)j ≤ Ts,B,b(x)j = xj = ηyj .

This is a contradiction. Thus η ≥ 1, which implies x ≥ y. Similarly, wecan also show that y ≥ x, so x = y. Therefore, the positive fixed point ofTs,B,b is unique, and equivalently the positive solution to Axm−1 = b isunique.


Systems with General M-Tensors

Theorem

Let A be an M-tensor. If there exists a nonnegative vector v suchthat Avm−1 ≥ b, then (A−1b)+ is nonempty.

Generally speaking, the nonnegative solution set (A−1b)+ in theabove theorem has more than one element, and these nonnegativesolutions lay on a hypersurface in Rn.

E.g., we construct a 3× 3× 3 singular M-tensor A = I − B,where B is a nonnegative tensor with B12 = 1, so that ρ(B) = 1,and bijk = 0 if i ∈ {2, 3} and either j = 1 or k = 1. Apparently, foreach right-hand side b = (b1, 0, 0)>, the nonnegative solutions toAx2 = b have the form x = (α, β, β)>. We display the proceduresof the iteration xk+1 = (Bx2k + b)[1/2] with two kinds of initialpoints (β, β, β)> and (2− β, β, β)> (0 ≤ β ≤ 1).


Systems with General M-Tensors

Figure: The iterations for a singular M-equation.


Non-Positive Right-Hand Side I

We discuss the nonsingular M-equations with non-positiveright-hand sides.

When the coefficient tensor is of even order, the situation issimple.

Assume that A is an even-order nonsingular M-tensor and b is anon-positive vector. Then the equation Axm−1 = b is equivalentto A(−x)m−1 = −b, which is a nonsingular M-equation withnonnegative right-hand side. However, the case is totally differentwhen the coefficient tensor is of odd order.


Non-Positive Right-Hand Side II

Theorem

Let A be a Z-tensor. Then A is a nonsingular M-tensor if andonly if A does not reverse the sign of any vector; that is, if x 6= 0and b = Axm−1, then for some subscript i ,

xm−1i bi > 0.

From the above theorem, we can easily understand that there is noreal vector x such that b = Axm−1 is non-positive when m is odd,since x[m−1] is always nonnegative.


Non-Homogeneous Left-Hand Side I

All the multi-linear equations that we discuss above havehomogeneous left-hand sides.

However, similar results can be established for some specialequations with non-homogeneous left-hand sides. Consider thefollowing equation

Axm−1 − Bm−1xm−2 − · · · − B2x = b > 0,

where A = sI − Bm is an mth-order nonsingular M-tensor and Bpis a pth-order nonnegative tensor for p = 2, 3, . . . ,m.


Non-Homogeneous Left-Hand Side II

Theorem

Let A be an mth-order Z-tensor and Bp be a pth-ordernonnegative tensor for p = 2, 3, . . . ,m. Then the equation

Axm−1 − Bm−1xm−2 − · · · − B2x = b (4)

has a unique positive solution for every positive vector b if andonly if A is a nonsingular M-tensor.


Absolutely M-Equations

A tensor is called an absolutely M-tensor if it can be written asA = sI − B with s > ρ(|B|), i.e., sI − |B| is a nonsingularM-tensor.It can be verified directly that an absolutely M-tensor must be anonsingular H-tensor, since s − |bii ...i | ≤ |s − bii ...i |. An absolutelyM-equation is referred to

Axm−1 = b,

where A is an absolutely M-tensor.An absolutely M-equation is referred to

Axm−1 = b,

where A is an absolutely M-tensor.


Employ the Brouwer fixed-point theorem

Theorem (Brouwer)

Let Ω be a bounded closed convex set in Rn. If map F : Ω→ Ω iscontinuous, then there exists x∗ ∈ Ω such that F (x∗) = x∗.

When m is even, we also consider the fixed-point iteration

xk = F (xk−1) := (s−1Bxm−1k−1 + s

−1b)[1/(m−1)], k = 1, 2, . . . .

Applying Brouwer’s fixed point theorem, we can prove that F (x)must have a fixed point in Rn.

Theorem

An even order absolutely M-equation has at least one real solutionfor every real right-hand side.


Decay Property of Banded Nonsingular M-tensor

The inverse matrix of a banded nonsingular M-matrix has aso-called decay property that is, its entries decay exponentiallyfrom the diagonal to the corners. It is interesting that a bandednonsingular M-tensor has a similar property.

Although there is no “inverse tensor” for a general M-tensor, wecan express this property as that the entries of the minimalnonnegative solution to Axm−1 = ep decay exponentially from thep-th entries to both the ends, where ep is the p-th orthonormalcolumn vector.


Banded M-EquationLet A = I − B be an mth-order nonsingular M-tensor satisfyingthat A is strictly diagonally dominant, i.e., A12 > 0. Furthermore,we also assume that B is d-banded, i.e., bi1i2...im = 0 if(i1, i2, . . . , im) lays out of

Ωi1 :={

(i1, i2, . . . , im) : |is − it | ≤ d for all s, t = 1, 2, . . . ,m}.

The minimal nonnegative solution solution x∗ of Axm−1 = epsatisfies

(x∗)i ≤ (x∗)p · ξ̂|i−p|.

Figure: The decay property of the minimal nonnegative solution toAx2 = e25.


The Classical Iterations

Split the coefficient tensor A into A =M−N satisfying that theequations with coefficient tensor M are easy to solve and N isnonnegative. Then the iteration

xk =M−1++(Nxm−1k−1 + b), k = 1, 2, . . .

offers a nonnegative solution to the equation above if it converges.

Different iterative methods for nonsingular M-equations.

Iterative Methods Alternatives of M

Jacobi diagonal partforward G-S lower triangular part

simplified forward G-S strictly lower triangular part & diagonal partbackward G-S upper triangular part

simplified backward G-S strictly upper triangular part & diagonal part


Convergence of the Classical Iterations I

Consider an operator φ : Rn → Rn. Let x∗ be a fixed point ofφ(x). Then we call x∗ an attracting fixed point, if there existsδ > 0 such that the sequence {xk} defined by xk+1 = φ(xk)converges to x∗ for any x0 such that ‖x0 − x∗‖ ≤ δ.

Theorem ( Rheinboldt 1998)

Let x∗ be a fixed point of φ : Rn → Rn, and let ∇φ : Rn → Rn×nbe the Jacobian of φ. Then x∗ is an attracting fixed point ifσ := ρ

(∇φ(x∗)

)< 1; further, if σ > 0, then the convergence of

xk+1 = φ(xk) to x∗ is linear with rate σ.


Convergence of the Classical Iterations I

Derive the Jacobian of the operator

φ(x) =M−1++(Nxm−1 + b).

take gradients on both sides of

Mφ(x)m−1 = Nxm−1 + b,

and we obtain

Mφ(x)m−2 · ∇φ(x) = Nxm−2.

When we take x as the positive fixed point x∗, then the matrixMφ(x∗)m−2 =Mxm−2∗ is a nonsingular M-matrix, since x∗ > 0and

Mxm−2∗ · x∗ = Nxm−1∗ + b ≥ b > 0.


Convergence of the Classical Iterations II

The Jacobian of φ(x) at x∗ is

∇φ(x∗) =(Mxm−2∗

)−1Nxm−2∗ ,which is a nonnegative matrix. Since

Nxm−2∗ · x∗ =Mxm−1∗ − b ≤ θMxm−1∗

with 0 ≤ θ < 1, then ∇φ(x∗) · x∗ ≤ θ1/(m−1)x∗.

Therefore the spectral radius ρ(∇φ(x∗)

)≤ θ1/(m−1) < 1, which

indicates that x∗ is an attracting fixed point of φ.


Convergence of the Classical Iterations III

Since A is a nonsingular M-tensor, we can take an initial vector x0satisfying that

0 < Axm−10 ≤ b,

then we shall prove that the iteration

xk =M−1++(Nxm−1k−1 + b), k = 1, 2, . . .

converges to the solution to the positive nonsingular M-equationwith a positive right-hand side.


An SOR-Like Acceleration

An SOR-like acceleration can also be applied to this iterativemethod. For instance, we can choose a proper ω > 0 so that theiterative scheme

xk = (M− ωI)−1++[(N − ωI)xm−1k−1 + b

], k = 1, 2, . . .

converges faster than the original scheme. The acceleration effectis due to a smaller α in the above discussion. There are somerestrictions when choosing the parameter ω, such as

1 ω > 0,

2 M− ωI is still a nonsingular M-tensor,3 (N − ωI)xm−1k−1 + b > 0 for all k = 1, 2, . . . .

However, whether there is an optimal parameter ω and how tochoose it still remain as open questions.


A Numerical Example

We construct a 3rd-order nonsingular M-tensor A = sI − B asfollows. First, we generate a nonnegative tensor B ∈ Rn×n×n+containing random values drawn from the standard uniformdistribution on (0, 1). Next, set the scalar

s = (1 + ε) · maxi=1,2,...,n

(B12)i , ε > 0,

where 1 = (1, 1, . . . , 1)>. Obviously, A is a diagonally dominantZ-tensor, i.e., A12 > 0. Thus A is a nonsingular M-tensor. Inthis example, we take n = 10 and ε = 0.01.The way we select the acceleration parameter is

ω = τ · mini=1,2,...,n

aii ...i , 0 < τ < 1.

This ensures the first two conditions discussed in the abovesubsection and the last condition when xk−1 is close to thesolution. In this experiment, we take τ = 0.35, which is chosen byexperience.


A Numerical Example I

Figure: The comparison of different classical iterative methods forM-equations.


The Newton Method I

When the coefficient tensor A is a symmetric nonsingularM-tensor, computing the positive solution is equivalent to solvingthe optimization problem

minx>0

ϕ(x) :=1

mAxm − x>b,

where

∇ϕ(x) = Axm−1 − b =: −r and ∇2ϕ(x) = (m − 1)Axm−2.

Note that when Axm−1 > 0, matrix Axm−2 is obviously asymmetric Z -matrix and

Axm−2 · x = Axm−1 > 0.

Therefore, Axm−2 is a symmetric nonsingular M-matrix, and thusa positive definite matrix.


The Newton Method II

Then the Newton step

pk = −[∇2ϕ(xk)

]−1∇ϕ(xk) = 1m−1(Axm−2k )−1rkis ensured to be a descending direction.

Then the iterations are as followsMk = Axm−2k ,rk = b−Mkxk ,pk =

1m−1M

−1k rk ,

xk+1 = xk + pk ,

k = 0, 1, 2, . . .


Numerical Example II

We construct a symmetric M-tensor of size 10× 10× 10 by thefollowing way. Let B ∈ R10×10×10 be a nonnegative tensor with

bi1i2i3 = | tan(i1 + i2 + i3)|.

It can be computed that ρ(B) ≈ 1450.3. Thus A = 1500I −B is asymmetric nonsingular M-tensor. We apply the Newton method,the accelerated Jacobi method, the accelerated Gauss-Seidelmethod, and the accelerated simplified Gauss-Seidel method tothis equation, respectively. And the acceleration parameter for theJacobi and Gauss-Seidel method is taken as τ = 400, which isexperimentally optimal. The figure shows the comparison on thenumber of iteration steps, from which we can see that the Newtonmethod converges much faster than other algorithms in thissituation.


Numerical Example II

Figure: Comparison of the Newton method and other iterative methods.


A Toy Example

Consider the ordinary differential equation

d2x(t)

dt2= − f (t)

x(t)2in (0, 1),

with Dirichlet’s boundary conditions

x(0) = c0, x(1) = c1.

Assume that f (t) > 0 on [0, 1], c0 > 0, c1 > 0, and we require apositive solution x(t) on [0, 1]. This equation can describe aparticle’s movement under the gravitation

md2x

dt2= −G Mm

x2→ x2 · d

2x

dt2= −GM,

where G ≈ 6.67× 10−11N ·m2/kg2 is the gravitational constantand M ≈ 5.98× 1024kg is the mass of the earth.


A Toy Example

After the discretization, we get a system of polynomial equationsx31 = c

30 ,

2x3i − x2i xi−1 − x2i xi+1 =GM

(n−1)2 , i = 2, 3, . . . , n − 1,x3n = c

31 .

Then this can be rewritten into a multi-linear equation Ax3 = b,where the coefficient tensor A is a tensor with seven diagonals,i.e., A is 1-banded, and

a1,1,1,1 = an,n,n,n = 1,ai ,i ,i ,i = 2, i = 2, 3, . . . , n − 1,ai ,i−1,i ,i = ai ,i ,i−1,i = ai ,i ,i ,i−1 = −1/3, i = 2, 3, . . . , n − 1,ai ,i+1,i ,i = ai ,i ,i+1,i = ai ,i ,i ,i+1 = −1/3, i = 2, 3, . . . , n − 1,

and the right-hand side is a positive vector withb1 = c

20 ,

bi =GM

(n−1)2 , i = 2, 3, . . . , n − 1,bn = c

21 .

The coefficient tensor A is a nonsingular [email protected] M-Equations

A Toy Example

Figure: A particle’s movement under the earth’s gravitation.


Inverse Iteration I

The power method for matrix eigenproblems is extended tononnegative tensor eigenproblems, often called the NQZ method,and its convergence is widely studied.

L. Elsner. Inverse iteration for calculating the spectral radius of anon-negative irreducible matrix. Linear Algebra and Appl.,15(3):235–242, 1976.

Z. Jia, W.-W. Lin, and C.-S. Liu. A positivity preserving inexact Nodaiteration for computing the smallest eigenpair of a large irreducibleM-matrix. Numer. Math., 130:645–679, 2015.

M. Ng, L. Qi, and G. Zhou. Finding the largest eigenvalue of anonnegative tensor. SIAM J. Matrix Anal. Appl., 31(3):1090–1099, 2009.

K.-C. Chang, K. J. Pearson, and T. Zhang. Primitivity, the convergence

of the NQZ method, and the largest eigenvalue for nonnegative tensors.

SIAM J. Matrix Anal. Appl., 32(3):806–819, 2011.


Inverse Iteration II

Algorithm (Inverse Iteration)

If B is an mth-order n-dimensional nonnegative tensor, then thisalgorithm gives the spectral radius of B and the correspondingeigenvector when converges.x0 = an approximated eigenvector

s0 = (1 + ε) ·maxi

(Bxm−10 )i/(x[m−1]0 )i

for k = 1, 2, . . .

yk = (sk−1I − B)−1++(x[m−1]k−1

)if ∠

(y[m−1]k , sk−1y

[m−1]k − x

[m−1]k−1

)is small enough

breakendsk = (1 + ε) ·

[sk−1 −

(mini

(xk−1)i/(yk)i)m−1]

xk = yk/‖yk‖end


Numerical Examples

I. We generate a nonnegative tensor B ∈ R10×10×10+ containingrandom values drawn from the standard uniform distribution on(0, 1). This is an irreducible example.

II. We also generate a nonnegative tensor B ∈ R10×10×10+ containingrandom values drawn from the standard uniform distribution on(0, 1) first. Then set bi1i2i3 = 0 if i1 ≥ 7 and i2, i3 < 7. Thus thistensor is reducible.

III. The third test tensor is a nonnegative tensor B ∈ R10×10×10+ withbi1i2i3 = | tan(i1 + i2 + i3)|, which is symmetric.

IV. Let B ∈ R3×3×3+ with b133 = b233 = b311 = b322 = 1 and otherentries being zeros. Since the power method does not work for thisexample, we compare the inverse iteration with the shifted powermethod. And we select an experimental optimal shift. Moreover, themulti-linear equation (sI − B)x2 = b is equivalent to a linearequation s 0 −10 s −1

−1 −1 s

·x21x22

x23

=b1b2

b3

,which can be solved by LU [email protected] M-Equations

Numerical Example III

Compare of the inverse iteration and the (shifted) power method.

I II III IV

Steps Time Steps Time Steps Time Steps Time

Inverse 4 7.38 6 11.00 5 9.89 5 0.58Power 11 2.65 33 7.93 41 9.90 NC NCShifted – – – – – – 25 1.41


Some Remarks

Each algorithm’s number of iteration steps and the running time of1000 times of experiments are listed in Table. Since the shiftedpower method has no apparent advance to the original powermethod for the first three examples, we just compare the inverseiteration with the power method. From Table, we can concludethat the inverse iteration always converges faster than the powermethod or the shifted power method.

The following Figure displays the convergence of the inverseiteration and the power method for the third example. From thisFigure, we can guess that the inverse iteration convergesquadratically, and this is true for the matrix case. Nevertheless, wecannot prove this conjecture and remain it as an open question.


The comparison of inverse iteration and power method

Figure: The comparison of inverse iteration and power [email protected] M-Equations

Consider the following the ordinary differential equation withDirichlets boundary condition{

u(x)m−2 · u′′(x) = −f (x), x ∈ (0, 1),u(0) = g0, u(1) = g1,

where f (x) > 0 in (0, 1) and g0, g1 > 0.Partition the interval [0, 1] into n − 1 small intervals with the samelength h = 1/(n − 1), and denote

uh =[u(0), u(h), . . . , u

((n − 1)h

)]>,

vh =[− u(4)(0),−u(4)(h), . . . ,−u(4)

((n − 1)h

)]>,

fh =[g0/h

2, f (h), . . . , f((n − 2)h

), g1/h

2]>,

where u(x) is the exact solution of the above boundary-valueproblem.


The discretization tensor Lh of the operator u 7→ um−2 · u′′ isintroduced in the first section, and our numerical solution ûh isobtained by solving the unique positive solution of an M-equationLhûm−1h = fh. It is well-known that the truncated error of thediscretization

u(x−h)−2u(x)+u(x+h)h2

− u′′(x) = h212u(4)(x) + O(h4).

Thus we have

Lhum−1h − Lhûm−1h = Lhu

m−1h − fh =

h2

12 · u[m−2]h ◦ vh + O(h

4),

which further implies that

dh(uh, ûh) :=∥∥Lhum−1h −Lhûm−1h ∥∥∞ ≤ h212 ·‖u‖m−2L∞ ·‖u(4)‖L∞+O(h4).

It can be verified that dh(·, ·) is a metric in the cone{x > 0 : Lhxm−1 > 0}. Then we can say that the numericalsolution ûh is very close to the exact solution uh when theparameter h is small enough. Next, we shall estimate theconvergence of the discretization scheme.


Note that Lhum−1h is also a positive vector when h is small enough,then the matrix Lhum−2h is a nonsingular M-matrix as discussed inSection 4. Hence we have the first order approximation

uh − ûh ≈ (Lhum−2h )−1(Lhum−1h − Lhû

m−1h )

when h is small enough, and thus

‖uh − ûh‖∞ .∥∥(Lhum−2h )−1∥∥∞ · ∥∥Lhum−1h − Lhûm−1h ∥∥∞.

We thus need to bound the ∞-norm of the inverse of theM-matrix Lhum−2h . First denote Lh = shI − Ah, where sh = 2/h

2

and Ah is nonnegative. Then we can write

(Lhum−2h )−1 =

((shI − Ah)um−2h

)−1=(shU

m−2h −Ahu

m−2h

)−1= s−1h Uh

[I − s−1h U

−(m−1)h (Ahu

m−2h )Uh

]−1U−(m−1)h ,

where Uh = diag((uh)1, (uh)2, . . . , (uh)n

).


Denote Wh = U−(m−1)h (Ahu

m−2h )Uh, which is a nonnegative

matrix. Note that (Ahum−2h )Uh1 = Ahum−1h , thus the summations

of all the rows of Wh are

Wh1 = U−(m−1)h Ahu

m−1h ≤ 1 · maxi=1:n

(Ahum−1h )i(uh)

m−1i

= 1 · maxi=1:n

sh(uh)m−1i − (Lhu

m−1h )i

(uh)m−1i

= 1 ·[

sh − mini=1:n

(Lhum−1h )i(uh)

m−1i

]=: 1 · (sh − γh).

Similarly, we have W kh 1 ≤Wk−1h 1 · (sh− γh) ≤ · · · ≤ 1 · (sh− γh)

k .Also, because Wh1 ≤ 1 · (sh − γh) < 1 · sh and Wh is an irreduciblenonnegative matrix, we have ρ(s−1h Wh) < 1.


Employ the Taylor expansion of the matrix(I − X )−1 = I + X + X 2 + . . . for ρ(X ) < 1, and we can obtainthat(I−s−1h Wh

)−11 =

∞∑k=0

(s−1h Wh

)k1 ≤

∞∑k=0

1·(1−γh/sh)k = 1·(sh/γh).

Finally, we get a upper bound of the ∞-norm of the nonnegativematrix (Lhum−2h )

−1 that

‖(Lhum−2h )−1‖∞ = max

i=1:n

((Lhum−2h )

−11)i

= maxi=1:n

(s−1h Uh

(I − s−1h Wh

)−1U−(m−1)h 1

)i

≤(

mini=1:n

uh)−(m−1)

· maxi=1:n

(uh)m−1i

(Lhum−1h )i· maxi=1:n

uh

≈(

mini=1:n

uh)−(m−1)

· maxi=1:n

(uh)m−1i

(fh)i· maxi=1:n

uh

≤ maxx u(x)m

minx u(x)m−1 ·minx f (x).


Note that u(x) can also be regarded as the solution of the ellipticproblem {

u′′(x) = f1(x) := −f (x)/u(x)m−2, x ∈ (0, 1),u(0) = g0, u(1) = g1.

So we know that u(x) ≥ min{g0, g1} since f (x) > 0 andg0, g1 > 0. Then

‖uh− ûh‖∞ .‖u‖mL∞

min{g0,g1}m−1·minx f (x) ·h2

12 · ‖u‖m−2L∞ · ‖u

(4)‖L∞ =: Kh2,

where the constant K is independent with the parameter h.Therefore, the sequence {ûh} converges to the exact solution whenh→ 0.


Conclusions

Let A be a Z-tensor. Then the following three conditions areequivalent:

1 A is a nonsingular M-tensor;2 (A−1b)++ has a unique element for every positive vector b;3 A does not reverse the sign of any vector; that is, if x 6= 0

and b = Axm−1, then xm−1i bi > 0 for some subscript i .

We also extend some algorithms and concepts in matrixcomputations, such as the Jacobi method, the Gauss-Seidelmethod, the Newton method, the inverse iteration, etc., to thehigher-order tensor cases.

Furthermore, many difficulties appear when extending the resultsfrom the linear case to the multi-linear case, many of which are leftas open questions throughout this paper.


Thank You

Thank You for Your Attention!


Solving Multi-Linear Systems with M-Tensors · 2016. 7. 14. · complementarity problem can be...

Documents

Transcript of Solving Multi-Linear Systems with M-Tensors · 2016. 7. 14. · complementarity problem can be...