EE611 Deterministic Systems

16
EE611 Deterministic Systems Vector Spaces and Basis Changes Kevin D. Donohue Electrical and Computer Engineering University of Kentucky

Transcript of EE611 Deterministic Systems

Page 1: EE611 Deterministic Systems

EE611Deterministic Systems

Vector Spaces and Basis ChangesKevin D. Donohue

Electrical and Computer EngineeringUniversity of Kentucky

Page 2: EE611 Deterministic Systems

Matrix Vector MultiplicationLet x be an nx1 (column) vector and y be a 1xn (row) vector:Dot (inner) Product: yx=c = |x||y|cos(θ) where c is a scalar (1x1) and θ is angle between y and xProjection: Projection of y onto x is denoted by (yx)x = |y|cos(θ) = yx/|x|Outer Product: xy=A where A is an nxn matrix.Matrix-Vector Multiplication: Let x be an nx1 vector and A be an nxn matrix:

where ' denotes transpose, and vectors denote a row vector partition in the first expression and a column vector partition in the second expression.

A x=[a١

⋮aN]x=[a١ x

a٢ x⋮

aN x ] x ' A=x ' [a١ a٢ ... aN ]=[x 'a١ x ' a٢ ... x 'aN]

ai

Page 3: EE611 Deterministic Systems

Linear IndependenceConsider an n-dimensional real linear space ℜn. A set of vectors {x1, x2, ... xm}∈ℜn are linearly dependent (l.d.) iff ∃ a set of real numbers {α1,α2, ... αm} not identically equal to 0 ∋

Otherwise the vectors are linearly independent (l.i.)

Show that if a set of vectors are l.d., then at least one of the vectors can be expressed as a linear combination of the others.

The dimension of the linear space is the maximal number of l.i. vectors in the space.

١ x١٢ x٢...m xm=٠

Page 4: EE611 Deterministic Systems

Basis and RepresentationA set of l.i. vectors in ℜn is a basis iff every vector in ℜn can be expressed as a unique linear combination of these vectors.

Given a basis for ℜn {q1, q2, ..., qn}, then every vector in ℜn

can be expressed as:

where is called the representation of

x=١q١٢q٢...n qn=[q١ q٢ ... qn ][١

٢

⋮n]=Q

x

Page 5: EE611 Deterministic Systems

ExampleFind the representation of noted point with orthonormal basis Q in terms of basis P.

q[١٠]=١ q[٠١]=٢ p[١٠]=١ p٢=[ ١٢

١٢ ]

[−٠.٥١.٥ ] [??]

Page 6: EE611 Deterministic Systems

NormsThe generalization of magnitude or length is given by a metric referred to as a norm. Any real valued function qualifies as a norm provided it satisfies:

∥x∥≥٠∀ x and ∥x∥=٠ iff x=٠

∥ x∥=∣∣∥x∥ for any real scalar

Non-negative

Scalable Consistency

∥x١x٢∥≤∥x١∥∥x٢∥ ∀ x١ ,x٢ Triangular Inequality

Page 7: EE611 Deterministic Systems

Popular NormsGiven

The 1-norm is defined by

The 2-norm (Euclidean norm)

The infinite-norm

Why do you think this is called the infinity norm? Hint: What would a 3-norm, … 100-norm look like?

∥x∥١ :=∑i=١

n

∣xi∣

x=[ x١ x٢ ... xn ] '

∥x∥٢:=٢∑i=١

n

x i ٢=x ' x

∥x∥∞ :=maxi∣xi∣

Page 8: EE611 Deterministic Systems

OrthonormalVector x is normalized, iff its Euclidean norm is 1(self dot product is 1). Vectors xi and xj are orthogonal iff their dot product is 0.

A set of (column) vectors {x1, x2, ... xm} are orthonormal iff

xi ' x j={٠ if i≠ j١ if i= j

∀ i , j

Page 9: EE611 Deterministic Systems

OrthonormalizationGiven a set of l.i. vectors {e1, e2, ... en}, the Schmidt orthonormalization procedure can be used to derive an orthonormal set of vectors {q1, q2, ... qn} forming a basis for the same linear space:

u١:=e١ q١:= ١∥u١∥u١

u٢:=e٢−q١' e٢q١ q٢:= ١∥u٢∥u٢

u٣:=e٣−q١' e٣q١−q٢ ' e٣q٢ q٣:= ١∥u٣∥u٣

un :=en−∑k=١

n−١

qk ' enqk qn := ١∥un∥un

Project and subtract Normalize

................................................

Page 10: EE611 Deterministic Systems

Linear Algebraic EquationsConsider a set of m linear equations with n unknowns:

Range space of A is the set all vectors resulting from all possible linear combinations of the columns of A.

The rank of coefficient matrix A ( rank(A) ) is equal to its number of l.i columns (or rows). rank(A) is also denoted as ρ(A)➢If rank(A) = n, a unique solution x exists given any y➢If rank(A) ≤ m < n, many solutions x exist given any y (underdetermined)➢If rank(A) ≤ n < m no solutions x may exist for some y (overdetermined)

y=A x

y١=a١١ x١a١٢ x٢...a١n xn

y٢=a٢١ x١a٢٢ x٢...a٢n xn

...............................ym=am١ x١am٢ x٢...amn xn

y=a١ x١a٢ x٢...an xn ai=[a١i a٢i ... a٣i ] '

Page 11: EE611 Deterministic Systems

NullityThe vector x is a null vector of A iff Ax=0

The null space of A is the set of all null vectors.

The nullity of A is the maximum number of l.i. vectors in its null space (i.e. dimension of null space).

nullity (A) = n - ρ(A)

Page 12: EE611 Deterministic Systems

Conditions for Solution ExistenceGiven mxn matrix A and mx1 vector y, an nx1 solution vector x exists for y=Ax iff y is in range space of A.

Given matrix A, a solution vector x exists for y=Ax, ∀ y iff A is full row rank (ρ(A) = m).

A= [A⋮y ] ⇔ ∃x such that A x=y

∃x such that A x=y ∀ y⇔A =m

Page 13: EE611 Deterministic Systems

Conditions for Unique SolutionGiven mxn matrix A and mx1 vector y, let xp be a solution for y=Ax. If ρ(A) = n (nullity k= 0), then xp is unique, and if nullity k > 0 then for any set of real αi's, x given below is a solution.

where vector set {n1, n2, ... nk} is a basis for the null space.

The above solution is also referred to as a parameterization of all solutions.

x=xp١ n١٢ n٢...k nk

Page 14: EE611 Deterministic Systems

Singular MatrixA square matrix is singular if its determinant is 0.

Given nxn non-singular matrix A, then for every y, a unique solution for y=Ax exists and is given by A-1y=x.

The homogeneous equation 0=Ax has a non zero solution iff A is singular, otherwise x=0 is the only solution.

Page 15: EE611 Deterministic Systems

Change of BasisDenote a representation of x with respect to (wrt) basis as , and representation wrt As . Note that basis vectors are assumed wrt the orthonormal basis. Find a change of basis transformation such that

Show that

{ e١, e٢, ... en}{e١, e٢, ... en}

=P =Q

P=Q−١=E−١E where E=[e١, e٢, ... en ] and E=[ e١, e٢, ... en ]

ei

Page 16: EE611 Deterministic Systems

Similarity TransformationConsider nxn matrix A as a linear operator that maps ℜn into itself. The vector representations are wrt . Determine the new representation of the linear operator wrt basis

Show that: where

The operation that changes the basis of the linear operator using a pre and post multiplication of a matrix and its inverse is referred as a similarity transform.

{ e١, e٢, ... en}{e١, e٢, ... en}

A=P A Q

P=Q−١=E−١E with E=[e١, e٢, ... en ] and E=[ e١, e٢, ... en ]