Κατηγοριοποίηση ΙΙ I

Click here to load reader

  • date post

    20-Jan-2016
  • Category

    Documents

  • view

    40
  • download

    0

Embed Size (px)

description

Κατηγοριοποίηση ΙΙ I. Κατηγοριοποιητές Κανόνων. Κατηγοριοποίηση με Κανόνες. Κατηγοριοποίηση των εγγραφών με βάση ένα σύνολο από κανόνες της μορφής “if…then…” Κανόνας : ( Συνθήκη )  y όπου Συνθήκη ( Condition ) είναι σύζευξη συνθηκών στα γνωρίσματα y η ετικέτα της κλάσης - PowerPoint PPT Presentation

Transcript of Κατηγοριοποίηση ΙΙ I

  • I

  • ifthen

    : () y (Condition) y

    LHS: rule antecedent () condition ()RHS: rule consequent ( )

    :

    (Blood Type=Warm) (Lay Eggs=Yes) Birds

    (Taxable Income < 50K) (Refund=Yes) Cheat=No

  • R1: (Give Birth = no) (Can Fly = yes) BirdsR2: (Give Birth = no) (Live in Water = yes) FishesR3: (Give Birth = yes) (Blood Type = warm) MammalsR4: (Give Birth = no) (Can Fly = no) ReptilesR5: (Live in Water = sometimes) Amphibians (Rule Set)

  • R1: (Give Birth = no) (Can Fly = yes) BirdsR2: (Give Birth = no) (Live in Water = yes) FishesR3: (Give Birth = yes) (Blood Type = warm) MammalsR4: (Give Birth = no) (Can Fly = no) ReptilesR5: (Live in Water = sometimes) Amphibians R1 hawk ( hawk / (trigger) ) => Bird R3 grizzly bear => Mammals r (covers) ()

  • (Status=Single) No Coverage = 40%, Accuracy = 50% - Coverage: LHS

    - Accuracy: LHS RHS

    Tid

    Refund

    Marital

    Status

    Taxable

    Income

    Class

    1

    Yes

    Single

    125K

    No

    2

    No

    Married

    100K

    No

    3

    No

    Single

    70K

    No

    4

    Yes

    Married

    120K

    No

    5

    No

    Divorced

    95K

    Yes

    6

    No

    Married

    60K

    No

    7

    Yes

    Divorced

    220K

    No

    8

    No

    Single

    85K

    Yes

    9

    No

    Married

    75K

    No

    10

    No

    Single

    90K

    Yes

    10

  • : R1: (Give Birth = no) (Can Fly = yes) BirdsR2: (Give Birth = no) (Live in Water = yes) FishesR3: (Give Birth = yes) (Blood Type = warm) MammalsR4: (Give Birth = no) (Can Fly = no) ReptilesR5: (Live in Water = sometimes) Amphibians Lemur R1 Turtle R4, R5 Godfish -

  • (Mutually exclusive rules) ,

    (Exhaustive rules) (coverage)

    :

  • (conflict resolution) : , (Decision list) , . () , () / , () (size ordering) : , , + (misclassification cost) default LHS :

  • : ..: RIPPER, CN2, Holtes 1R

    : ( ) ..: C4.5 :

  • RepeatGrow a rule using the Learn-One-Rule functionRemove training records covered by the ruleAdd rule to the set until stopping condition : (sequential covering)

  • : :

  • : : Rule-Growing Strategy R: {} y

  • ? ? : : Remove Training Records R1 12/15 80%, R2 7/10 20%, R3 8/12 70% R1, () ? R2 R3?

  • ( ) , :

  • : - (RHS) :

  • Rule Setr1: (P=No,Q=No) ==> -r2: (P=No,Q=Yes) ==> +r3: (P=Yes,R=No) ==> +r4: (P=Yes,R=Yes,Q=No) ==> -r5: (P=Yes,R=Yes,Q=Yes) ==> +PQRQ-++-+NoNoNoYesYesYesNoYes : :

  • : (Refund=No) (Status=Married) No : (Status=Married) No ( LHS ) :

    Tid

    Refund

    Marital

    Status

    Taxable

    Income

    Cheat

    1

    Yes

    Single

    125K

    No

    2

    No

    Married

    100K

    No

    3

    No

    Single

    70K

    No

    4

    Yes

    Married

    120K

    No

    5

    No

    Divorced

    95K

    Yes

    6

    No

    Married

    60K

    No

    7

    Yes

    Divorced

    220K

    No

    8

    No

    Single

    85K

    Yes

    9

    No

    Married

    75K

    No

    10

    No

    Single

    90K

    Yes

    10

  • ():

    :

  • 1: Induction Step 2: Deduction Step Eager Learners vs Lazy Learners Instance Based Classifiers ( )

  • :Rote-learner (Memorizes)

    Nearest neighbor k closest (nearest neighbors)

  • k- x k- x

  • Basic idea: If it walks like a duck, quacks like a duck, then its probably a duck

  • : k - .., (majority vote)

  • Distance Metric k,

  • :

    weight factor, w = 1/d2

  • k:k , k , k = sqr(n), n , , , default, k=10

    X

  • : 1.5m 1.8m 90lb 300lb $10K $1M , ( ) ( k-)

  • ( )

  • 2- kd- R2

    : x y () ( )

  • 2- kd-

  • 2- kd-

  • 2- kd-

  • 2- kd-

  • 2- kd-

  • 2- kd-

  • 2- kd- u u

  • 2- kd- R2

    : O(n) O(logn) O(nlogn) 2 Binary Space Partitioning

  • Bayes

  • X, Y (Conditional probability): Pr(Y=y | X=x) Bayes Bayes : Pr(X=x,Y=y) (joint) (conditional)

  • Bayes " C", " " 4/5. " C" 1/3 " " 2/3

    ; " " " C"; Bayes: 1

  • Bayes 2 , 0 1 0 65% 0, 30% 175% 1

    1 0 , ; Bayes: 2

  • ; Bayes

  • X=(Home Owner=No, Marital Status=Married, AnnualIncome=120K): Pr(Yes|X), Pr(No|X) No Yes, ; BayesX: Y: ()

    Y X (non-determininstic) P(Y|X) : Posterior probability ( )P(Y): Prior probability ( )

  • Bayes : Pr(Y|X) X Y

    : X, Y Pr(Y|X)

    , P(X) ( - evidence)P(Y): , Y ( )Pr(X|Y)?

  • Bayes Pr(X|Y) :

  • Bayes , ?: 1 ()

  • Bayes: P(homeOwner = yes|No) = 3/7

    P(MaritalStatus = Single| Yes) = 2/3

    Categorical attribute XiPr(Xi = xi|Y=y): y xi i-

  • 35 , ? : P(, 35| ) (, 35| ) Bayes:

  • X = {X1,,Xd} d

    Conditional independence ( ):X Y, Z :

    P(X|Y,Z) = P(X|Z) P(X,Y|Z) = P(X|Z) P(Y|Z)

    Bayes ()

  • Bayes X = {X1,,Xd} d

  • 35 , ? Bayes: ()

  • Bayes

  • BayesP(A|M)P(M) > P(A|N)P(N)=> MammalsA: M: mammalsN: non-mammals

    animals2

    NameGive BirthLay EggsCan FlyLive in WaterHave LegsClass

    humanyesnononoyesmammals

    pythonnoyesnononoreptiles

    salmonnoyesnoyesnofishes

    whaleyesnonoyesnomammals

    frognoyesnosometimesyesamphibians

    komodonoyesnonoyesreptiles

    batyesnoyesnoyesmammals

    pigeonnoyesyesnoyesbirds

    catyesnononoyesmammals

    leopard sharkyesnonoyesnofishes

    turtlenoyesnosometimesyesreptiles

    penguinnoyesnosometimesyesbirds

    porcupineyesnononoyesmammals

    eelnoyesnoyesnofishes

    salamandernoyesnosometimesyesamphibians

    gila monsternoyesnonoyesreptiles

    platypusnoyesnonoyesmammals

    owlnoyesyesnoyesbirds

    dolphinyesnonoyesnomammals

    eaglenoyesyesnoyesbirds

    NameGive BirthLay EggsCan FlyLive in WaterHave LegsClass

    humanyesnononoyesmammals

    pythonnoyesnonononon-mammals

    salmonnoyesnoyesnonon-mammals

    whaleyesnonoyesnomammals

    frognoyesnosometimesyesnon-mammals

    komodonoyesnonoyesnon-mammals

    batyesnoyesnoyesmammals

    pigeonnoyesyesnoyesnon-mammals

    catyesnononoyesmammals

    leopard sharkyesnonoyesnonon-mammals

    turtlenoyesnosometimesyesnon-mammals

    penguinnoyesnosometimesyesnon-mammals

    porcupineyesnononoyesmammals

    eelnoyesnoyesnonon-mammals

    salamandernoyesnosometimesyesnon-mammals

    gila monsternoyesnonoyesnon-mammals

    platypusnoyesnonoyesmammals

    owlnoyesyesnoyesnon-mammals

    dolphinyesnonoyesnomammals

    eaglenoyesyesnoyesnon-mammals

    NameGive BirthLay EggsCan FlyLive in WaterHave LegsClass

    humanyesnonoyesno?

    animals2

    NameGive BirthLay EggsCan FlyLive in WaterHave LegsClass

    humanyesnononoyesmammals

    pythonnoyesnononoreptiles

    salmonnoyesnoyesnofishes

    whaleyesnonoyesnomammals

    frognoyesnosometimesyesamphibians

    komodonoyesnonoyesreptiles

    batyesnoyesnoyesmammals

    pigeonnoyesyesnoyesbirds

    catyesnononoyesmammals

    leopard sharkyesnonoyesnofishes

    turtlenoyesnosometimesyesreptiles

    penguinnoyesnosometimesyesbirds

    porcupineyesnononoyesmammals

    eelnoyesnoyesnofishes

    salamandernoyesnosometimesyesamphibians

    gila monsternoyesnonoyesreptiles

    platypusnoyesnonoyesmammals

    owlnoyesyesnoyesbirds

    dolphinyesnonoyesnomammals

    eaglenoyesyesnoyesbirds

    NameGive BirthLay EggsCan FlyLive in WaterHave LegsClass

    humanyesnononoyesmammals

    pythonnoyesnonononon-mammals

    salmonnoyesnoyesnonon-mammals

    whaleyesnonoyesnomammals

    frognoyesnosometimesyesnon-mammals

    komodonoyesnonoyesnon-mammals

    batyesnoyesnoyesmammals

    pigeonnoyesyesnoyesnon-mammals

    catyesnononoyesmammals

    leopard sharkyesnonoyesnonon-mammals

    turtlenoyesnosometimesyesnon-mammals

    penguinnoyesnosometimesyesnon-mammals

    porcupineyesnononoyesmammals

    eelnoyesnoyesnonon-mammals

    salamandernoyesnosometimesyesnon-mammals

    gila monsternoyesnonoyesnon-mammals

    platypusnoyesnonoyesmammals

    owlnoyesyesnoyesnon-mammals

    dolphinyesnonoyesnomammals

    eaglenoyesyesnoyesnon-mammals

  • Bayes (discretization) -> ->

  • Bayes Gauss () () (2)

  • Bayes To ij () yi

  • Bayes

    (Income, Class=No):sample mean = 110sample variance = 2975

  • Bayes

  • Bayes

  • X = (HomeOwner = No, MaritalStatus = Married, Income=120K) Pr(Y|X), Pr(Y)xPr(X|Y)But P(X|Y) isY = No:P(HO=No|No) x P(MS=Married|No) x P(Inc=120K|No) = 4/7 x 4/7 x 0.0072 = 0.0024Y=Yes:P(HO=No|Yes) x P(MS=Married|Yes) x P(Inc=120K|Yes) = 1 x 0 x 1.2x10-9 = 0 Bayes

  • X = (HomeOwner = No, MaritalStatus = Married, Income=120K) Pr(X|Y = Yes) 0! -> nc: yj xin: yj

    m: (equivalent sample size) ( (nc/n) (p)

    p: ( xi Xi yi) Bayes

  • - Xi (irrelevant), P(Xi|Y) uniform

    () (correlated attributes)

    ,

    ,

    Bayes

  • (Support Vector Machines)

  • - ( ) SVM

  • SVM

    B1

  • SVM

    B2

  • SVM

  • B1 B2? ; ; SVM

    B1

    B2

  • - (margin) => B1 B2 () SVM

    B1

    B2

    b11

    b12

    b21

    b22

    margin

  • SVM SVM1 -1

  • :

    :

    (constraints):

    (constrained optimization problem) SVM

  • SVM

  • (slack variables) :

    : SVM

  • SVM