# Naive Bayes（Simple Example）

xiaoxiao2021-02-28  15

### 文章目录

1 假设2 Notion3 Simple Example4 基于最小错误率的贝叶斯决策5 基于最小风险贝叶斯决策

# 2 Notion

i i 表示 l a b e l label 的类别数

j j 也表示 l a b e l label 的类别数，只是为了区别于 i i

eg: 对于一个二分类问题， y e s   o r   n o yes \ or \ no ， 对应的贝叶斯公式如下

P ( Y e s ∣ X ) = P ( X ∣ Y e s ) ⋅ P ( Y e s ) P ( X ) = P ( X ∣ Y e s )   ⋅   P ( Y e s ) P ( X ∣ Y e s ) ⋅ P ( Y e s ) + P ( X ∣ N o ) ⋅ P ( N o ) P(Yes|X)= \frac {P(X|Yes) \cdot P(Yes)} {P(X)} = \frac {P(X|Yes) \ \cdot \ P(Yes)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)}

P ( N o ∣ X ) = P ( X ∣ N o ) ⋅ P ( N o ) P ( X ) = P ( X ∣ N o )   ⋅   P ( N o ) P ( X ∣ Y e s ) ⋅ P ( Y e s ) + P ( X ∣ N o ) ⋅ P ( N o ) P(No|X)= \frac {P(X|No) \cdot P(No)} {P(X)} = \frac {P(X|No) \ \cdot \ P(No)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)}

# 3 Simple Example

X = { G e n d e r = F e m a l e , I n c o m e = H i g h , A g e = M i d d l e } X = \left \{ Gender = Female,Income = High, Age = Middle \right \} 计算分类结果 Y e s   o r   N o Yes \ or \ No

P ( Y e s ) = 3 / 6 P(Yes) = 3 / 6

P ( G e n d e r = F e m a l e ∣ Y e s ) = 2 / 3 P\left (Gender = Female \mid Yes\right ) = 2/3

P ( I n c o m e = H i g h ∣ Y e s ) = 3 / 3 P\left ( Income = High \mid Yes\right ) = 3/3

P ( A g e = M i d d l e ∣ Y e s ) = 1 / 3 P\left ( Age = Middle \mid Yes\right ) = 1/3

P ( X ∣ Y e s ) ⋅ P ( Y e s ) = P ( G e n d e r = F e m a l e ∣ Y e s ) ⋅ P ( I n c o m e = H i g h ∣ Y e s ) ⋅ P ( A g e = M i d d l e ∣ Y e s ) ⋅ P ( Y e s ) = 2 3 × 3 3 × 1 3 × 3 6 ≈ 0.111 P\left ( X\mid Yes \right ) \cdot P(Yes)= P\left (Gender = Female \mid Yes\right )\cdot P\left ( Income = High \mid Yes\right )\cdot P\left ( Age = Middle \mid Yes\right ) \cdot P(Yes) = \frac{2}{3}\times\frac{3}{3}\times\frac{1}{3}\times\frac{3}{6} \approx 0.111

P ( N o ) = 3 / 6 P(No) = 3/6

P ( G e n d e r = F e m a l e ∣ N o ) = 1 / 3 P\left (Gender = Female \mid No\right ) = 1/3

P ( I n c o m e = H i g h ∣ N o ) = 1 / 3 P\left ( Income = High \mid No\right ) = 1/3

P ( A g e = M i d d l e ∣ N o ) = 2 / 3 P\left ( Age = Middle \mid No\right ) = 2/3

P ( X ∣ N o ) ⋅ P ( N o ) = P ( G e n d e r = F e m a l e ∣ N o ) ⋅ P ( I n c o m e = H i g h ∣ N o ) ⋅ P ( A g e = M i d d l e ∣ N o ) ⋅ P ( N o ) = 1 3 × 1 3 × 2 3 × 3 6 = 0.037 P\left ( X\mid No \right ) \cdot P(No)= P\left (Gender = Female \mid No\right )\cdot P\left ( Income = High \mid No\right )\cdot P\left ( Age = Middle \mid No\right ) \cdot P(No) = \frac{1}{3}\times\frac{1}{3}\times\frac{2}{3}\times\frac{3}{6} = 0.037

P ( Y e s ∣ X ) = P ( X ∣ Y e s )   ⋅   P ( Y e s ) P ( X ∣ Y e s ) ⋅ P ( Y e s ) + P ( X ∣ N o ) ⋅ P ( N o ) = 0.111 0.111 + 0.037 = 75 % P(Yes|X)= \frac {P(X|Yes) \ \cdot \ P(Yes)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)} = \frac{0.111}{0.111+0.037} = 75 \%

P ( N o ∣ X ) = P ( X ∣ N o )   ⋅   P ( N o ) P ( X ∣ Y e s ) ⋅ P ( Y e s ) + P ( X ∣ N o ) ⋅ P ( N o ) = 0.037 0.111 + 0.037 = 25 % P(No|X)= \frac {P(X|No) \ \cdot \ P(No)} {P(X|Yes) \cdot P(Yes) + P(X|No) \cdot P(No)} = \frac{0.037}{0.111+0.037} = 25 \%

P ( Y e s ∣ X ) > P ( N o ∣ X ) P(Yes|X)>P(No|X)

# 4 基于最小错误率的贝叶斯决策

（1）

（2）

x ∈ w i x \in w_i

（3）似然比

x ∈ w 1 x \in w_1 否则 x ∈ w 2 x \in w_2

(4) 似然比的负对数 -ln

P ( e ) = ∫ R 1 P ( x ∣ w 2 ) P ( w 2 ) d x + ∫ R 2 P ( x ∣ w 1 ) P ( w 1 ) d x P(e) = \int_{R_1}P(x|w_2)P(w_2)dx + \int_{R_2}P(x|w_1)P(w_1)dx

P ( e ) = ∫ R 1 P ( w 2 ∣ x ) P ( x ) d x + ∫ R 2 P ( w 1 ∣ x ) P ( x ) d x P(e) = \int_{R_1}P(w_2|x)P(x)dx + \int_{R_2}P(w_1|x)P(x)dx

# 5 基于最小风险贝叶斯决策

（3）决策：选择风险最小的决策，即：