1

20070510 Chap9 1

Chapter9

Inference in First-Order Logic

20070510 Chap9 2

Inference Rules Involving Quantifiers

• SubstitutionSUBST (θ, α) refers to applying the substitution or binding list θ to the sentence α.

SUBST ({ x / Sam , y / Pam } , Likes ( x , y )) = Likes ( Sam , Pam )

• Inference rulesModus PonensAnd-EliminationAnd-Introduction + 3 New Inference Rules with QuantifiersOr-IntroductionResolution

2

20070510 Chap9 3

Three New Inference Rules with Quantifiers• Universal Instantiation (UI, or Universal Elimination) :

a universally quantified variable can be replaced with any real instance of that type

∀v α ∀ x Likes(x, IceCream)

SUBST({v/g}, α) Likes(Ben, IceCream)

• Existential Instantiation (EI, or Existential Elimination) :an existentially quantified variable can be replaced with a new symbol (i.e. Skolem constant, give it a name that has not yet been used)

∃v α ∃ x Likes(x, IceCream)

SUBST({v/g}, α) Likes(Person1, IceCream)

20070510 Chap9 4

Three New Inference Rules with Quantifiers (cont.)

• Existential Introduction : a real object name can be replaced by an existentially quantifiedvariable

α Likes(Jerry, IceCream)

∃v SUBST({g/v}, α) ∃ x Likes(x, IceCream)

3

20070510 Chap9 5

Reduction to Propositional Inference• By instantiating the universal sentences in all ways, we can

have the new KB is propositionalized.

∀x King(x) ∧ Greedy(x) ⇒ Evil(x) King(John) ∧ Greedy(John) ⇒ Evil(John)King(Richard) ∧ Greedy(Richard) ⇒ Evil(Richard)

King(John) King(John)Greedy(John) Greedy(John)

Brother(Richard, John) Brother(Richard, John)

• A ground sentence is entailed by new KB iff entailed by original KB.

• Every FOL KB can be propositionalized so as to preserve entailment.

• Idea: propositionalized KB and query, apply resolution, return result

20070510 Chap9 6

Reduction to Propositional Inference (cont.-1)

• Problem: When the KB includes a function symbol, the set of possible ground term substitutions is infinite.

e.g. Father(Father(Father(John)))

• Theorem: Herbrand (1930)If a sentence is entailed by the original FOL KB, there is a proof involving just a finite subset of the propositionalized KB.

• Idea: For n = 0 to ∞ doCreate a proposional KB by instating with depth-n terms see if the sentence α is entailed by this KB.

4

20070510 Chap9 7

• Problem: works if α is entailed, loops if α is not entailed.

• Theorem: Turing (1936), Church (1936)Entailment in FOL is semidecidable.i.e. algorithms exists that say yes to every entailed sentence, but no algorithm exists that also say no to every nonentailed sentence.

Reduction to Propositional Inference (cont.-2)

20070510 Chap9 8

Problems with Propositionalization• Propositionalization approach is rather inefficient.

it seems to generate lots of irrelevant sentences.∀x King(x) ∧ Greedy(x) ⇒ Evil(x)King(John)∀x Greedy(x)Brother(Richard, John)

e.g. The following two are also generated:King(Richard) ∧ Greedy(Richard) ⇒ Evil(Richard)Greedy(Richard)

• With p k-ary predicates and n constants, there are p*nk instantiations.

5

20070510 Chap9 9

Unification• To take two atomic sentences p and q and return a

substitution θ would make p and q look the same.

Unify(p, q) = θwhere Subst(θ, p) = Subst(θ, q)

θ is called the unifier of the two sentences.

20070510 Chap9 10

Let our knowledge base contain the following sentences:

Knows(John, Jane)Knows(y, Leonid)Knows(y, Mom(y))Knows(x, Elizabeth)

p q θ

Knows(John, x) Knows(John, Jane) {x/Jane}Knows(John, x) Knows(y, Leonid) {x /Leonid, y/John}Knows(John, x) Knows(y, Mom(y)) {y/John, x/Mom(John)}Knows(John, x) Knows (x, Elizabeth) fail

Unification (cont.)

6

20070510 Chap9 11

Standardize Apart

• To rename the variables of one or both to avoid name clashes.i.e. To eliminate overlap of variables

p q θ

Knows(John, x) Knows (x, Elizabeth) fail

Knows(John, x1) Knows (x2, Elizabeth) {x1/ Elizabeth, x2/John}

20070510 Chap9 12

Most General Unifier (MGU)To unify Knows(John, x) and Knows (y, z),

θ = {y/John, x/z} or {y/John, x/z, w/Freda}or {y/John, x/John, z/John} or ….

The first unifier is more general than the second.

• The substitution that makes the least commitment about the bindings variables.

• There is a single MGU that is unique up to renaming of variables.

MGU is {y/John, x/z}

7

20070510 Chap9 13

Unification Algorithm

When matching a variable against a complex term, one must check whether the variable itself occurs inside the term; if it does, the match fails.

20070510 Chap9 14

Generalized Modus Ponens (GMP)For atomic sentences pi , pi’ , and q , where there is a substitution θ such that Subst(θ , pi’ ) = Subst(θ , pi ) for all i :

p1’ Owns(Nono, M1)…

pn’ Missile(M1)

p1 ∧ …∧ pn ⇒ q Owns(Nono , x) ∧ Missile(x) ⇒Sells(West, Nono, x)

Subst(θ , q ) Sells(West, Nono, M1)

8

20070510 Chap9 15

Generalized Modus Ponens (cont.)

• The Generalized Modus Ponens is a lifted version of Modus Ponens --- it raises Modus Ponensfrom propositional to first-order logic.

• The generalized modus ponens is sound.

• The generalized modus ponens is complete for Horn databases.

• The generalized modus ponens is efficient.

20070510 Chap9 16

Soundness of GMP• Need to show that

p1', …, pn', (p1 ∧ … ∧ pn ⇒ q) |= qθ

provided that pi'θ = piθ for all I

• LEMMA: For any sentence p, we have p |= pθ by UI

(p1 ∧ … ∧ pn ⇒ q) |= (p1 ∧ … ∧ pn ⇒ q)θ = (p1θ ∧ … ∧ pnθ ⇒ qθ)p1', …, pn' |= p1' ∧ … ∧ pn' |= p1'θ ∧ … ∧ pn'θFrom 1 and 2, qθ follows by ordinary Modus Ponens.

9

20070510 Chap9 17

First-Order Definite Clause

To use the generalized Modus Ponens in inference, each sentence in the KB must be either

• an atomic sentence , or • an implication whose antecedent is a conjunction of positive literals and whose consequent is a single positive literal.

i.e. a1 ∧ …∧ am ⇒ b

20070510 Chap9 18

First-Order Definite Clause (cont.)

• Not every knowledge base can be converted into a set of definite clauses, because of the single-positive literal restriction.

• Convert sentences into definite clause when they are first entered into the knowledge base using

- Existential Elimination- And-Elimination

e.g, ∃x Owns(Nono, x) ∧ Missile(x)is converted into two atomic definite clauses:

Owns(Nono, M1) Missile(M1)

10

20070510 Chap9 19

Example knowledge base• The law says that it is a crime for an American to

sell weapons to hostile nations. The country Nono, an enemy of America, has some missiles, and all of its missiles were sold to it by Colonel West, who is American.

• Prove that Col. West is a criminal

20070510 Chap9 20

Example knowledge base (cont.)

... it is a crime for an American to sell weapons to hostile nations:American(x) ∧ Weapon(y) ∧ Sells(x,y,z) ∧ Hostile(z) ⇒ Criminal(x)

Nono … has some missiles, i.e., ∃x Owns(Nono,x) ∧ Missile(x ):Owns(Nono,M1) ∧ Missile(M1)

… all of its missiles were sold to it by Colonel WestMissile(x) ∧ Owns(Nono,x) ⇒ Sells(West,x,Nono)

Missiles are weapons:Missile(x) ⇒ Weapon(x)

An enemy of America counts as "hostile“:Enemy(x,America) ⇒ Hostile(x)

West, who is American … American(West)

The country Nono, an enemy of America … Enemy(Nono,America)

11

20070510 Chap9 21

Revised Example ProofTo solve problem using Generalized Modus Ponens, we need to put the original knowledge into definite clause form.

1. American(x) ∧ Weapon(y) ∧ Nation(z) ∧ Hostile(z) ∧ Sells(x , y, z) ⇒ Criminal (x)2. Owns(Nono, M1) 3. Missile(M1)4. Owns(Nono, x) ∧ Missile(x) ⇒ Sells(West, Nono, x)5. Missile(x) ⇒ Weapon(x)6. Enemy(x, America) ⇒ Hostile(x) 7. American(West)8. Nation(Nono)9. Enemy(Nono, America)10. Nation(America)11. Criminal(West) ?

MP(3, 5)=12 Weapon(M1)MP(6, 9) =13 Hostile(Nono) MP(2,3,4)=14 Sells(West, Nono, M1)MP(7, 12, 8, 13, 14, 1) Criminal(West)

20070510 Chap9 22

Inference Chaining• Generalized Modus Ponens gives us a natural,

intuitive, and reasonably powerful tool to use for inference.

• There are two types of inference that differ simply in direction.

• Forward Chaining starts with new premises and tries to generate all new conclusions.

• Backward Chaining begins from a desired conclusion and attempts to realize the necessary implications and premises required to arrive at it.

12

20070510 Chap9 23

Inference Procedures

• Forward Chaining- Data-driven- Triggered by adding a new fact- Premises ⇒ Consequent(s)- Renaming- Composition:

Subst(Compose(θ1 , θ2) , p ) =Subst(θ2 , Subst(θ1 , p))

• Backward Chaining- Goal-driven- Triggered by a query , i.e. Ask - Premises ⇐ Consequent- Build up the unifier as it goes

20070510 Chap9 24

Forward Chaining Algorithm

13

20070510 Chap9 25

Forward Chaining Proof

20070510 Chap9 26

Forward Chaining Proof (cont.-1)

14

20070510 Chap9 27

Forward Chaining Proof (cont.-2)

20070510 Chap9 28

Properties of Forward Chaining• FC is sound and complete for first-order definite

clauses• Datalog = first-order definite clauses +

no function symbolsFC terminates for Datalog in poly iterations:

at most p * nk literals• May not terminate in general if α is not entailed.

Entailment with datalog is decidable.Entailment with definite clauses is semi-decidable.

• FC is widely used for Production systems.

15

20070510 Chap9 29

Efficiency of forward chaining• Incremental forward chaining: no need to match

a rule on iteration k if a premise wasn't added on iteration k-1- match each rule whose premise contains a newly

added positive literal.

• Matching itself can be expensive• Database indexing allows O(1) retrieval of known

facts- e.g., query Missile(x) retrieves Missile(M1)

• Forward chaining is widely used in deductive databases

20070510 Chap9 30

Backward Chaining Algorithm

16

20070510 Chap9 31

Backward Chaining Proof

20070510 Chap9 32

Backward Chaining Proof (cont.-1)

17

20070510 Chap9 33

Backward Chaining Proof (cont.-2)

20070510 Chap9 34

Backward Chaining Proof (cont.-3)

18

20070510 Chap9 35

Backward Chaining Proof (cont.-4)

20070510 Chap9 36

Backward Chaining Proof (cont.-5)

19

20070510 Chap9 37

Backward Chaining Proof (cont.-6)

20070510 Chap9 38

Properties of Backward Chaining• BC is depth-first recursive proof search:

space is linear in size of proof• BC is incomplete due to infinite loops.

⇒ fix by checking current goal against every goal on stack• BC is inefficient due to repeated subgoals.

⇒ fix by using caching of previous results (need extra space)• BC is widely used for logic programming.

20

20070510 Chap9 39

Logic programming• Logic programming

- Identify problem- Assemble

information- <coffee break>- Encode info in KB- Encode problem

instances as facts- Ask queries- Find false facts.

• Procedural programming- Identify problem- Assemble information- Figure out solution- Program solution- Encode problem

instance as data- Apply program to data- Debug procedural

errors

Should be easier to debug Capital(NY, US) than x=x+2

20070510 Chap9 40

Logic programming: Prolog• BASIS: backward chaining with Horn clauses + bells & whistles

Widely used in Europe, Japan (basis of 5th Generation project)

• Program = set of clauses= head :- literal1, … literaln.

criminal(X) :- american(X), weapon(Y), sells(X,Y,Z), hostile(Z).

• Efficient unification and retrieval of matching clauses.• Depth-first, left-to-right backward chaining• Built-in predicates for arithmetic etc., e.g., X is Y*Z+3• Built-in predicates that have side effects (e.g., input and output

predicates, assert/retract predicates)• Closed-world assumption ("negation as failure")

- e.g., given alive(X) :- not dead(X).- alive(joe) succeeds if dead(joe) fails

21

20070510 Chap9 41

Prolog• Appending two lists to produce a third:

append([],Y,Y). append([X|L],Y,[X|Z]) :- append(L,Y,Z).

• query: append(A,B,[1,2]) ?

• answers: A=[] B=[1,2]A=[1] B=[2]A=[1,2] B=[]

20070510 Chap9 42

Resolution Proof: Definite Clauses

22

20070510 Chap9 43

Resolution: Proof Tree

• Forward / Backward chaining.• The procedure is incomplete.• Need the factoring inference rule.

20070510 Chap9 44

Resolution: Proof Tree (cont.)

• Resolution is refutation complete.• It is also called proof by contradiction or reductio ad absurdum.• To prove P , we assume P is false and prove a contradiction.

23

20070510 Chap9 45

Resolution Rule of Inference

Generalized Resolution (disjunctions):

For literals pi and qi, where Unify(pj , ¬qk) = θ:

p1 ∨ …∨ pj ∨ … ∨ pm

q1 ∨ …∨ qk∨ … ∨ qn

Subst(θ, (p1 ∨ …∨ pj-1 ∨ pj+1 ∨ … ∨ pm ∨ q1 ∨ …∨ qk-1∨ qk+1 ∨ … ∨ qn ))

20070510 Chap9 46

Resolution Rule of Inference (cont.)

Generalized Resolution (implications):

For atoms pi , qi , ri , si , where UNIFY(pj , qk ) = θ:

p1 ∧ …∧ pj ∧ … ∧ pn1 ⇒ r1 ∨ …∨ rn2

s1 ∧ …∧ sn3 ⇒ q1 ∨ … ∨ qk ∨ … ∨ qn4

Subst(θ, (p1 ∧ … ∧ pj-1 ∧ pj+1 ∧ … ∧ pn1 ∧ s1 ∧ …∧ sn3

⇒ r1 ∨ …∨ rn2 ∨ q1 ∨ …∨ qk-1 ∨ qk+1 ∨ … ∨ qn ))

24

20070510 Chap9 47

Resolution: brief summary• Full first-order version:

l1 ∨ ··· ∨ lk, m1 ∨ ··· ∨ mn

(l1 ∨ ··· ∨ li-1 ∨ li+1 ∨ ··· ∨ lk ∨ m1 ∨ ··· ∨ mj-1 ∨ mj+1 ∨ ··· ∨ mn)θ

where Unify(li, ¬mj) = θ.

• The two clauses are assumed to be standardized apart so that they share no variables.

• For example,¬Rich(x) ∨ Unhappy(x)

Rich(Ken)Unhappy(Ken)

with θ = {x/Ken}

• Apply resolution steps to CNF(KB ∧ ¬α); complete for FOL

20070510 Chap9 48

Conversion to CNF

To translate a sentence p into normal form :

1. Eliminate implications ⇒, using the fact that ( p ⇒ q ) is equivalent to ¬ p ∨ q.

2. Move negations inwards.3. Standardize variables apart.4. Eliminate existentials using Skolemization.5. Move universal quantifiers to the left.6. Drop the prefix.7. Distribute ∧ over ∨ , i.e. into conjunction of disjunctions (CNF).8. Flatten nested conjunctions and disjunctions.9. (Combine negated terms and reintroduce implication ⇒.)

10. Standardize variables apart again.

25

20070510 Chap9 49

Example: Conversion to CNFEveryone who loves all animals is loved by someone.

∀x [∀y animal(y) ⇒ loves(x, y)] ⇒ [∃ y loves (y,x)]

1. ∀x [¬(∀y¬ animal(y) ∨ loves(x, y))] ∨ [∃ y loves (y, x)]2. ∀x [∃ y ¬¬ animal(y) ∧ ¬ loves(x, y)] ∨ [∃ y loves (y, x)]3. ∀x [∃ y animal(y) ∧ ¬ loves(x, y)] ∨ [∃ y loves (y, x)]4. ∀x [∃ y animal(y) ∧ ¬ loves(x, y)] ∨ [∃ z loves (z, x)]5. ∀x [animal(F(x)) ∧ ¬ loves(x, F(x))] ∨ [loves (G(x), x)]6. [animal(F(x)) ∧ ¬ loves(x, F(x))] ∨ [loves (G(x), x)]7. [animal(F(x)) ∨ loves (G(x), x)] ∧ [¬ loves(x, F(x)) ∨ loves

(G(x), x)]

20070510 Chap9 50

Conversion to CNF• Everyone who loves all animals is loved by someone:

∀x [∀y Animal(y) ⇒ Loves(x,y)] ⇒ [∃y Loves(y,x)]

• Eliminate biconditionals and implications∀x [¬∀y ¬Animal(y) ∨ Loves(x,y)] ∨ [∃y Loves(y,x)]

• Move ¬ inwards: ¬∀x p ≡ ∃x ¬p, ¬ ∃x p ≡ ∀x ¬p∀x [∃y ¬(¬Animal(y) ∨ Loves(x,y))] ∨ [∃y Loves(y,x)] ∀x [∃y ¬¬Animal(y) ∧ ¬Loves(x,y)] ∨ [∃y Loves(y,x)]∀x [∃y Animal(y) ∧ ¬Loves(x,y)] ∨ [∃y Loves(y,x)]

26

20070510 Chap9 51

Conversion to CNF (cont.)

Standardize variables: each quantifier should use a different one:∀x [∃y Animal(y) ∧ ¬Loves(x,y)] ∨ [∃z Loves(z,x)]

Skolemize: a more general form of existential instantiation.Each existential variable is replaced by a Skolemfunction of the enclosing universally quantified variables:∀x [Animal(F(x)) ∧ ¬Loves(x,F(x))] ∨ Loves(G(x),x)

Drop universal quantifiers:[Animal(F(x)) ∧ ¬Loves(x,F(x))] ∨ Loves(G(x),x)

Distribute ∨ over ∧ :[Animal(F(x)) ∨ Loves(G(x),x)] ∧ [¬Loves(x,F(x)) ∨ Loves(G(x),x)]

20070510 Chap9 52

SkolemizationLet p be a quantified sentence of the form:

where• the variables are universally quantified , • the variables x is existentially quantified , and • the variables appear in the component sentence q .

The Skolemization of p is the sentence

where is a new function constant that does not appear elsewhere in the database .

x),x.q(vvv in1 ∃∀ K

iv

))v,,(vs,.q(vvv n1nkin1 KK −∀

nks −

27

20070510 Chap9 53

Skolemization (cont.)

Theorem A database D and its Skolemization are

equivalent. i.e. For any query q containing no Skolem constants,

D |= q iff |= q

(D)SK

(D)SK

20070510 Chap9 54

Skolemization : Example

• John has a house .

• Everyone has a house .

John),house(SJohn)h.house(h,

1k −

∃

p)of(p),-usep.house(hop)(p),p.house(S

p)h.house(h,p2k

∀∀

∃∀−

28

20070510 Chap9 55

Skolemization : Example (cont.)

• There is a house that belongs to everyone.

• Big houses are a lot of work to maintain.

p),p.house(Sp)p.house(h,h

3k −∀∀∃

work(h)]p) house(h,hp.[big(h)work(h)]p) house(h,big(h)hp.[

work(h)]p) house(h,p.big(h)h.[work(h)]p)]p.house(h,[big(h)h.[

)( work(h)]p)p.house(h,h.[big(h)

→∧∀∨¬∨¬∀

∨¬∀∨¬∀∨∃∧¬∀

∗→∃∧∀

20070510 Chap9 56

Removing Existential Quantifiers

• Can we transform (*) into the following ?

The answer is No.

Consider a conceptualization of two persons A and B , as well as one house that is A’s house . The sentence (**) is true in the following mode !

while the sentence (*) is false.

)( work(h)]p)house(h,p.[big(h)h ∗∗→∧∃∀

)}work(HA),,house(H),{big(H AAA ¬

AH

29

20070510 Chap9 57

Removing Existential Quantifiers (cont.)

• Note: One should be extra careful when dealing with quantifiers.

For example, the following two sentences are equivalent

• On the other hand , the following are not :

x.q(x)x.p(x)q(x)]x.[p(x)

∀∧∀∧∀

x.q(x)x.p(x)q(x)]x.[p(x)

∀∨∀∨∀

20070510 Chap9 58

Resolution ProcedureGiven a database D and a query q,1. Transform all the sentences in D to normal form.2. Negate q and convert the result to normal form .

Add it to the set of sentences obtained in Step 1.3. Repeat until either a contradiction is found ,

no progress can be made, or a predetermined amount of effort has been expended.

(a) Select two sentences S1 and S2 in normal form such that some premise of S1 can unify with some consequent of S2.

(b) Resolve them together by applying the resolution rule.If there are more than one pair of atoms that can be unified,only one pair should be resolved away.

(c ) If the resolvent is , a contradiction has been found.Otherwise, add it to set and continue.

FT →

30

20070510 Chap9 59

Example Proof - Resolution by Refutation

Jack owns a dog.Every dog owner is an animal lover.No animal lover kills an animal.Either Jack or Curiosity killed the cat, who is named Tuna. Did Curiosity kill the cat?

A. ∃x Dog(x) ∧ Owns(Jack, x) B. ∀x (∃ y Dog(y) ∧ Owns(x, y)) ⇒ AnimalLover(x) C. ∀x AnimalLover(x) ∧ ∀y Animal(y) ⇒ ¬ Kills(x, y) D. Kills(Jack, Tuna) ∨ Kills(Curiosity, Tuna) E. Cat(Tuna) F. ∀x Cat(x) ⇒ Animal(x)

20070510 Chap9 60

Example Proof - Resolution by Refutation (cont.-1)

A1. Dog(D)

A2. Owns(Jack, D)

B. ∀x (∃ y Dog(y) ∧ Owns(x, y)) ⇒ AnimalLover(x)

C. AnimalLover(x) ∧ Animal(y) ∧ Kills(x, y) ⇒ False

D. Kills(Jack, Tuna) ∨ Kills(Curiosity, Tuna)

E. Cat(Tuna)

F. Cat(x) ⇒ Animal(x)

31

20070510 Chap9 61

Example Proof - Resolution by Refutation (cont.-2)

A1. Dog(D)A2. Owns(Jack, D) B. Dog(y) ∧ Owns(x, y) ⇒ AnimalLover(x)C. AnimalLover(x) ∧ Animal(y) ∧ Kills(x, y) ⇒ FalseD. Kills(Jack, Tuna) ∨ Kills(Curiosity, Tuna)E. Cat(Tuna)F. Cat(x) ⇒ Animal(x)

20070510 Chap9 62

Example Proof - Resolution by Refutation (cont.-3)

• We wish to proveCuriosity killed the cat.

• So we make the supposition that Curiosity did not kill the cat.

• We end up with a contradiction Jack killed Tuna vs. Jack did not kill Tuna.

• This contradiction means that our original supposition was incorrect, hence

Curiosity in fact killed Tuna!

32

20070510 Chap9 63

Resolution Strategies• Unit resolution - at least one of the parent clauses is a unit clause.

It is generally not complete.

e.g.

• Input resolution - at least one of the parent clauses is a member ofthe initial database.

• Linear resolution ( also, ancestry-filtered) - at least one of theparents is either in the initial database or is an ancestor of the otherparent.

• Set of support resolution - at least one parent is selected from the set of support or its descendants. A subset of a set is calleda set of support for is and only if is satisfiable.

QPQ,PQ,P Q,P ¬∨¬¬∨∨¬∨

Γ ΔΔ Γ−Δ

20070510 Chap9 64

Resolution Strategies (cont.)

• Ordered resolution - resolution is permitted only on thefirst literal of each clause.

• Directed resolution - a specialization of ordered resolution in which the query takes the form of a conjunction of positive literals , and the database consists entirely of directed clauses.

33

20070510 Chap9 65

Answer Extraction in Resolution

Given a query and a logical database D,

• Tautology --- Add to D.

• Answer literal --- Add to D.

x.q(x)∃

q(x)q(x) ∨¬

Ans(x)q(x) ∨¬