Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V,...

14
Question 1a

Transcript of Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V,...

Page 1: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 1a

Page 2: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 1a

The leaf variables are W 1, b1, W 2, b2, V , γ, xi, yi

Page 3: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Each leaf in the graph has a single directed path leading to theoutput o. To calculate the partial derivative, take the product ofJacobians along the path

∂o

∂V

∣∣∣∣f̄o

=∂o

∂q

∣∣∣∣fo

× ∂q

∂l

∣∣∣∣fq

× ∂l

∂V

∣∣∣∣f l

Page 4: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 1b

∂o

∂W 1

∣∣∣∣f̄o

=∂o

∂q

∣∣∣∣fo

× ∂q∂l

∣∣∣∣fq

× ∂l

∂h2

∣∣∣∣f l

× ∂h2

∂z2

∣∣∣∣fh2

× ∂z2

∂h1

∣∣∣∣fz2

× ∂h1

∂z1

∣∣∣∣fh1

× ∂z1

∂W 1

∣∣∣∣fz1

Page 5: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 1b

∂o

∂W 2

∣∣∣∣f̄o

=∂o

∂q

∣∣∣∣fo

× ∂q

∂l

∣∣∣∣fq

× ∂l

∂h2

∣∣∣∣f l

× ∂h2

∂z2

∣∣∣∣fh2

× ∂z2

∂W 2

∣∣∣∣fz2

Page 6: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 2a

Page 7: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 2a

We get valuesu3 = u1 × u2 = 12

u4 = u1 + u2 = 7

u5 = 2× u3 × u4 = 168

Page 8: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 2b

un = 2×u3×u4 = 2×(u1×u2)×(u1+u2) = 2×(u1)2×u2+2×u1×(u2)2

hence

f̄n(u1, u2) = 2× (u1)2 × u2 + 2× u1 × (u2)2

hence∂un

∂u1= 4× u1 × u2 + 2× (u2)2 = 80

∂un

∂u2= 2× (u1)2 + 4× u1 × u2 = 66

Page 9: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

J1→3(A3) = u2 = 4

J2→3(A3) =∂

∂u2

(u1 × u2

)= u1 = 3

J1→4(A4) =∂

∂u1

(u1 + u2

)= 1

J2→4(A4) =∂

∂u2

(u1 + u2

)= 1

J3→5(A5) =∂

∂u3

(2× u3 × u4

)= 2× u4 = 14

J4→5(A5) =∂

∂u4

(2× u3 × u4

)= 2× u3 = 24

Page 10: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

p5 = 1

p4 = p5 × J4→5(A5) = 1× 24 = 24

p3 = p5 × J3→5(A5) = 1× 14 = 14

p2 = p3 × J2→3(A3) + p4 × J2→4(A4) = 14× 3 + 24× 1 = 66

p1 = p3 × J1→3(A3) + p4 × J1→4(A4) = 14× 4 + 24× 1 = 80

Page 11: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 3a

Equations:

z1 ∈ Rm = Wxi,1 + b

h1 ∈ Rm = g(z1)

l1 ∈ RK = V h1 + γ

q1 ∈ RK = LS(l1)

z2 ∈ Rm = Wxi,2 + b

h2 ∈ Rm = g(z2)

l2 ∈ RK = V h2 + γ

q2 ∈ RK = LS(l2)

o ∈ R = −q1yi,1 − q2

yi,2

Page 12: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 3a

Page 13: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 3b

There are two directed paths from V to o in the computationalgraph. To calculate the partial derivative, take a sum of productsof Jacobians on these paths

∂o

∂V

∣∣∣∣f̄o

=∂o

∂q1

∣∣∣∣fo

× ∂q1

∂l1

∣∣∣∣fq1

× ∂l1

∂V

∣∣∣∣f l1

+∂o

∂q2

∣∣∣∣fo

× ∂q2

∂l2

∣∣∣∣fq2

× ∂l2

∂V

∣∣∣∣f l2

Page 14: Question 1amcollins/cs4705-spring2020/... · Question 1a The leaf variables are W1, b1, W2, b2, V, , xi, yi. Each leaf in the graph has a single directed path leading to the output

Question 3c

f z(W,xi,1, xi,2, b) = [Wxi,1 + b;Wxi,2 + b]

fh([z1; z2]) = [g(z1); g(z2)]

f l([h1;h2]) = [V h1 + γ;V h2 + γ]

f q([l1; l2]) = [LS(l1); LS(l2)]

f o([q1; q2], yi,1, yi,2) = −q1yi,1 − q2

yi,2