Seminar 7
Definition 1 (Naive Bayes Classifier) Naive Bayes (NB) Classifier assumes that the
effect of the value of a predictor 𝑥 on a given class 𝑐 is class conditional independent.
Bayes theorem provides a way of calculating the posterior probability 𝑃(𝑐|𝑥) from class prior
probability 𝑃(𝑐), predictor prior probability 𝑃(𝑥) and probability of the predictor given the
class 𝑃(𝑥|𝑐)
𝑃(𝑐|𝑥) =
𝑃(𝑥|𝑐)𝑃(𝑐)
𝑃(𝑥)
and for a vector of predictors 𝑋 = (𝑥1, . . . , 𝑥 𝑛)
𝑃(𝑐|𝑋) = 𝑃(𝑥1|𝑐) . . . 𝑃(𝑥 𝑛|𝑐)𝑃(𝑐).
The class with the highest posterior probability is the outcome of prediction.
Exercise 1
Considering the table of observations, use the Naive Bayes classifier to recommend whether
to Play Golf given a day with Outlook = Rainy, Temperature = Mild, Humidity = Normal
and Windy = True. Do not deal with the zero-frequency problem.
Outlook Temperature Humidity Windy Play Golf
Rainy Hot High False No
Rainy Hot High True No
Overcast Hot High False Yes
Sunny Mild High False Yes
Sunny Cool Normal False Yes
Sunny Cool Normal True No
Overcast Cool Normal True Yes
Rainy Mild High False No
Rainy Cool Normal False Yes
Sunny Mild Normal False Yes
Rainy Mild Normal True Yes
Overcast Mild High True Yes
Overcast Hot Normal False Yes
Sunny Mild High True No
Table 1: Exercise.
1
Definition 2 (Support Vector Machines Classifier (two-class, linearly separable))
Support Vector Machines (SVM) finds the hyperplane that bisects and is perpendicular to
the connecting line of the closest points from the two classes. The separating (decision)
hyperplane is defined in terms of a normal (weight) vector w and a scalar intercept term 𝑏 as
𝑓(𝑥) = w · x + 𝑏
where · is the dot product of vectors. Finally, the SVM classifier becomes
𝑐𝑙𝑎𝑠𝑠(𝑥) = 𝑠𝑔𝑛(𝑓(𝑥)).
Exercise 2
Build the SVM classifier for the training set {([1, 1], −1), ([2, 0], −1), ([2, 3], +1)}.
2