Page 44 - Data Science Algorithms in a Week

Naive Bayes

Rearranging these relationships, we get the following:

P(A∩B)=P(A|B)*P(B)
P(A∩B)=P(B|A)*P(A)

P(A|B)*P(B)=P(B|A)*P(A)

This is, in fact, Bayes' theorem:
P(A|B)=P(B|A)*P(A)/P(B)

This concludes the proof.

Extended Bayes' theorem

We can extend Bayes' theorem taking into consideration more probability events. Suppose
that the events B ,...,B are conditionally independent given A. Let ~A denote the
n
1
complement of A. Then:
P(A|B ,...,B ) = P(B ,...,B |A) * P(A) / P(B ,...,B )
n
1
1
n
1
n
= [P(B |A) * ... * P(B |A) * P(A)] / [P(B |A) * ... * P(B |A) * P(A) + P(B |~A) * ... * P(B |~A) *
n
1
1
n
1
n
P(~A)]
Proof:
Since the events B ,...,B are conditionally independent given A (and also given ~A), we have
1
n
the following:
P(B ,...,B |A)=P(B |A) * ... * P(B |A)
n
n
1
1
Applying the simple form of Bayes' theorem and this fact, we thus have the following:
P(A|B ,...,B ) = P(B ,...,B |A) * P(A) / P(B ,...,B )
n
1
1
n
n
1
= P(B |A) * ... * P(B |A) * P(A) / [P(B ,...,B |A)*P(A)+P(B ,...,B |~A)*P(~A)]
1
n
n
1
n
1
= [P(B |A) * ... * P(B |A) * P(A)] / [P(B |A) * ... * P(B |A) * P(A) + P(B |~A) * ... * P(B |~A) *
1
n
n
1
n
1
P(~A)]
This completes the proof as required.
[ 32 ]