1. 2014.07.27 PRML PRML 2.4

2. (2.194) (2.195) 2.4 2 1 p(x)=h(x)g()exp{ T u(x)} p(x)d x=1 g()h(x)exp{ T u(x)}d x=1 3. (2.196) (2.197) (2.194) (2.198) 2.4 1 : (2.194) p(x)=Bern(x)= x (1() 1(x p(x)=exp{x ln +(1(x)ln(1()} =exp{ln(1()+x ln 1( } =(1()exp{ln( 1( )x} =ln( 1( ) p(x)=h(x)g()exp{ T u(x)} 4. (2.198) (2.199) 2.4 () =ln( 1( ) e = 1( e ((1+e )=0 = e 1+e = e ( e e ( (1+e ) = 1 1+e ( = () 1(=1( 1 1+e ( = e ( 1+e ( = 1 1+e = (() 5. (2.200) (2.194) (2.201) (2.202) (2.203) 2.4 (2.194) p(x)= (()exp(x) p(x)=h(x)g()exp{ T u(x)} u(x)=x h(x)=1 g()= (() 6. (2.204) (2.205) (2.194) (2.206) (2.207) (2.208) 2.4 2 : (2.194) p(x)= M k=1 k xk =exp{ M k=1 xk ln k } k=ln k =(1,,M ) T p(x)=exp( T x) p(x)=h(x)g()exp{ T u(x)} u(x)=x h(x)=1 g()=1 7. (2.209) (2.210) (2.211) 2.4 k=1 M k=1 0k1 k=1 m(1 k1 k=1 M xk=1 p(x)=exp( k=1 M xk ln k )=exp{ k=1 M (1 xk ln k+xM ln M } =exp{ k=1 M (1 xk ln k+(1( k=1 M (1 xk)ln(1( k=1 M (1 k)} =exp{ k=1 M (1 xk ln( k 1(j=1 M (1 j )+ln(1( k=1 M (1 k )} 8. (2.212) (2.213-1) (2.213-2) (2.213) 2.4 k t t (2.213-1) (2.213-2) ln( k 1(j=1 M (1 j )=k k=(1(j j)e k =t e k t=1(j j=1(j t e j =1(t j e j t=1/(1+j e j ) k= e k 1+j e j 9. (2.211) (2.214) (2.194) (2.215) (2.216) (2.216) 2.4 (2.211) t (2.194) p(x)=exp{ k=1 M (1 xk ln( k 1(j=1 M (1 j )+ln(1( k=1 M (1 k)} =(1(k k)exp{k xk ln( k 1(j j )} p(x)=1/(1+k e k )exp( T x) =(1,,M (1 ,0) T p(x)=h(x)g ()exp{ T u(x)} u(x)=x h(x)=1 g()=1/(1+ k=1 M (1 e k ) 10. (2.218) (2.219) (2.194) (2.220) (2.221) (2.222) (2.222) 2.4 3 : (2.194) p(x , 2 )= 1 (2 2 ) 1/2 exp{ (1 2 2 (x() 2 } = 1 (2 2 ) 1/2 exp{ (1 2 2 x 2 + 2 x( 1 2 2 2 } p(x)=h(x)g ()exp{T u(x)} = ( / 2 (1/2 2) u(x)= (x x 2) h(x)=(2) (1/2 g ()=((22) 1/2 exp( 1 2 42 ) = 1 (2 2 ) 1/2 exp( (1 2 2 2 )exp {( /2 (1/2 2) T (x x2)} 11. 2.4 2.57 -1 N (x , )= 1 (2) D/2 1 1/2 exp{ (1 2 (x() T ((((1 (x()} = 1 (2)D/2 1 1/2 exp{ (1 2 (x T ((((1 x(((( T ((((1 x((((x T ((((1 ++++ ((((1 )} T ((((1 x====x T ((((1 = 1 (2) D/2 1 1/2 exp{ (1 2 T ((((1 }exp{ (1 2 x T ((((1 x++++ T ((((1 x} = 1 (2) D/2 1 1/2 exp{ (1 2 T ((((1 }exp {( T ((((1 , (1 2 ((((1 )( x xx T)} 12. (2.195) (2.224) (2.225) (2.226) 2.4.1 1 (2.195) *g()/g() 2 g() u(x) g ()h(x)exp{ T u(x)}d x=1 g()h(x)exp{T u(x)}d x +g ()h(x)exp{ T u(x)}u(x)d x=0 g ()/g ()+g()h(x)exp{ T u(x)}u(x)d x=0 ( g()/ g()=g()h(x)exp{ T u(x)}u(x)d x=E[u(x)] p(x)u((((x))))d x (ln f (x))'= f ' (x)/ f (x) ( ln g ()=E[u(x)] 13. (a) (b) 2.4.1 2.58 (2.226) ( ln g() =(E[u(x)]) T =(g()h(x)exp{ T u(x)}u(x)d x) T = g ()(h(x)exp{T u(x)}u(x)d x) T +g()h(x)exp{ T u(x)}u(x)u(x) T d x (a)=( g ()/g ())E[u(x) T ] = ln g ()E[u(x) T ] =(E[u(x)]E[u(x) T ] (b)= p(x)u((((x))))u((((x)))) T d x =E[u(x)u(x) T ] ( ln g()=E[u(x)u(x) T ](E[u(x)]E[u(x) T ]=cov[u(x) T ] 14. (2.227) (2.228) 2.4.1 0 N (2.228) (2.226) X={x1,..., xN} p(X)= ( N n=1 h(xn) )g()N exp {T N n=1 u((((xn)))) } ln p(X)= n=1 N ln h(xn)+N ln g()+ T n=1 N u((((xn)))) ln p(XML)=0+N ln g(ML)+ n=1 N u((((xn))))=0 ( ln g(ML)= 1 N n=1 N u((((xn)))) n u((((xn)))) n u((((xn)))) E[u(x)] ML 15. (2.194) (2.229) (2.230) 2.4.2 (2.194) g (2.194) f (2.229) (2.227) (2.229) p( , )= f ( ,)g() exp{ T } p(x)=h(x)g ()exp{T u(x)} p(X) p( ,)= ( N n=1 h(xn) )f ( ,)g () N + exp { T ( N n=1 u((((xn))))+ ) } p(X , ,) g() N + exp { T ( N n=1 u((((xn))))+ ) } 16. (2.231) 2.4.3 K 1/K 1. 2.(1.27) =^2 p()= p()d d = p( 2 )2 17. (2.232) (2.233) (2.234) (2.235) 2.4.3 1 x A B p() p(x)= f (x() x=x+c p(x)= f ( x( ) =+c A B A(c B(c A B p()d =A(c B(c p()d =A B p( (c)d p((c)= p() 18. (2.141) (2.142) 2.4.3 N = 2 N0 2 + 2 0+ N0 2 N0 2 + 2 ML N = 0 2 0 ( 2 N0 2 + 2 0+ N0 2 N0 2 + 2 ML)= 0 2 0 ( 2 /0 2 N + 2 /0 2 0+ N N + 2 /0 2 ML)=ML 1 N 2 = 1 0 2 + N 2 1 N 2 = 0 2 (1 0 2 + N 2 )= N 2 19. (2.236) (2.237) (2.238) 2.4.3 2 x A B p() 1/ c p 1/c p(x)= 1 f (x ) (>0) x=cx p( x)= 1 f (x ) =c A B A/c B/c A B p()d =A/c B/c p()d =A B p( 1 c ) 1 c d p()= p(1 c )1 c 20. (2.240) 2.4.3 (1.27) p() 1/ p(ln ) p()= p(ln )d ln d = p(ln ) 1 p(ln )= p() N (x , 2 ) (1 exp{(( x/) 2 } ( x=x() =1/ 2 p()= 1 exp{(( x/) 2 }d a d x= 1 exp{(x 2 }(( 3 /2)= (1 2 (1 exp{(x 2 } Gam(a0,b0) a0=b0=0 21. (2.236) 2.4.3 2.59 f(x) (2.236) p(x)= 1 f (x ) ( p(x)d x=( 1 f (x )d x=( 1 f (y)d x d yd y y=x/ =( 1 f ( y) d y=( f ( y)d y=1 22. (2.262) (2.10) 2.3 (N m )+( N m(1 )=(N +1 m ) (N m ) N ! (N (m)! m! (N m )+( N m(1 )= N ! (N (m)! m! + N ! (N+1(m)!(m(1)! = (N +1(m)N ! (N +1(m)(N (m)! m! + N ! m (N +1(m)(N (m)! m! = N ! (N (m)! m(m(1)! + N ! (N +1(m)(N (m)!(m(1)! = (N+1) N ! (N +1(m)(N(m)! m! = (N +1)! (N +1(m)! m! =(N +1 m ) 23. (2.263) (1)N=0 N=0 (2.263) (2)N=k (2.263) N=k+1 (2.263) (1)(2) N(N>0) (2.263) (1+x) N = m=0 N (N m)x m (1+x) 0 = m=0 0 (0 m)x m =1 (1+x) k+1 =(1+x) k +x(1+x) k = m=0 k (k m)x m + m=0 k (k m)x m+1 = m=1 k (k m)x m +1+ m=1 k ( k m(1)x m +x k+1 = m=1 k (k+1 m )x m +1+x k+1 = m=0 k+1 (k+1 m )x m 24. (2.264) (2.263) m=0 N (N m) m (1() N(m =1 m=0 N (N m) m (1() N(m =(1() N m=0 N (N m) m (1() N (m (1() (N =(1()N m=0 N (N m)( 1( ) m =(1()N (1+ 1( ) N = {(1() (1+ 1( )} N =(1(+) N =1 N =1 25. (2.285) 2.20 (2.285) u (2.45) (2.285) a a u (2.285) a T a>0 ui=i ui ui T ui=ui T i ui=iui a= a1 u1++ aD uD a T a=( a1 u1 T ++ aD uD T )( a1 u1++ aD uD) =( a1 u1 T ++ aD uD T )( a1 1 u1++ aD D uD) = a1 2 1u1++ aD 2 DuD