Learning Naive Bayes by Inventing It

You will discover the Naive Bayes classifier yourself — step by step, from counting to a working ML model.

Part 1: Counting and Probability

Before any algorithm, let's learn to count carefully. That's all probability is — careful counting.

Meet the Data

Here are 14 days of weather observations. For each day we recorded the outlook, temperature, humidity, and wind — and whether we played tennis.

Day Outlook Temp Humidity Wind Play?
1 Sunny Hot High Weak No
2 Sunny Hot High Strong No
3 Overcast Hot High Weak Yes
4 Rainy Mild High Weak Yes
5 Rainy Cool Normal Weak Yes
6 Rainy Cool Normal Strong No
7 Overcast Cool Normal Strong Yes
8 Sunny Mild High Weak No
9 Sunny Cool Normal Weak Yes
10 Rainy Mild Normal Weak Yes
11 Sunny Mild Normal Strong Yes
12 Overcast Mild High Strong Yes
13 Overcast Hot Normal Weak Yes
14 Rainy Mild High Strong No

Tasks (pen & paper):

  1. Count the number of Yes days and No days.
  2. What fraction of all days resulted in "Yes"?
  3. Without looking at any weather features, if someone asked "Will we play tomorrow?", what would you guess?

Your First Prediction Rule

Imagine you have no weather information at all. A friend calls and asks: "Should I come to play tennis?"

  1. What would you predict? Why?
  2. What is the probability of Play = Yes? Express it as a fraction.
  3. What is the probability of Play = No?
  4. Do these two probabilities add up to 1? Why must they?

Your answers:

  1. Prediction: ...
  2. P(Yes) = ...
  3. P(No) = ...
  4. Sum = ... because ...

Prior Probability

P(Play = Yes) = 9/14. In probability and machine learning, this is called a "prior" probability. Why is it called "prior"?

A. Because it was computed before computers existed

B. Because it uses the first column of the data

C. It's what you'd predict *before* looking at any features — your default belief based only on past outcomes

D. It always equals 0.5

Part 2: One Feature — Does Weather Help?

The prior says "Yes" 64% of the time. But surely the weather matters. Let's see how one feature changes our prediction.

Counting with a Condition

Look at the data table and answer by hand (just count the rows):

  1. Of the 9 Yes days, how many have Outlook = Sunny?
  2. Of the 5 No days, how many have Outlook = Sunny?
  3. Compute the fractions: "Among Yes days, what fraction is Sunny?" and "Among No days, what fraction is Sunny?"

Your answers:

  1. Sunny & Yes days: ... out of 9
  2. Sunny & No days: ... out of 5
  3. Fraction among Yes: ... / 9 = ...    Fraction among No: ... / 5 = ...

The Key Question

It's a Sunny day. You want to decide: is it more likely a Yes day or a No day?

You have two pieces of information:

  • From Ex 2.1: Among Yes days, 2/9 are Sunny. Among No days, 3/5 are Sunny.
  • From Ex 1.2: P(Yes) = 9/14, P(No) = 5/14.

Tasks:

  1. Compute: (fraction of Sunny among Yes) × P(Yes) = ?
  2. Compute: (fraction of Sunny among No) × P(No) = ?
  3. Which product is larger? What would you predict for a Sunny day?

Your calculation:

  1. Score(Yes) = (2/9) × (9/14) = ...
  2. Score(No) = (3/5) × (5/14) = ...
  3. Winner: ...

Try Other Outlooks

Repeat the same calculation from Ex 2.2 for:

  1. Overcast days: compute Score(Yes) and Score(No). What do you predict?
  2. Rainy days: compute Score(Yes) and Score(No). What do you predict?

You'll need to count how many Overcast and Rainy days appear among Yes days and No days first.

Code It: `count_where`

Now let's automate the counting. Write a function that counts how many rows match a set of conditions.

def count_where(data, conditions):
    """
    Count rows in data where ALL conditions are met.
    conditions: dict like {"Outlook": "Sunny", "Play": "Yes"}
    Returns: int
    """
    # Your code here
    pass

Code It: `probability`

Write a function that computes a conditional probability using count_where.

def probability(data, event, given):
    """
    Compute P(event | given).
    event: dict of conditions for the event, e.g. {"Outlook": "Sunny"}
    given: dict of conditions for the given, e.g. {"Play": "Yes"}
    Returns: float
    """
    # Your code here
    pass

Predict with One Feature

Write a function that predicts the class using a single feature:

def predict_one_feature(data, feature_name, feature_value, label_col, labels):
    """
    For each label, compute:  score = P(feature_value | label) * P(label)
    Return the label with the highest score.
    """
    # Your code here
    pass

Part 3: Two Features — Combining Evidence

One feature helped. Can two features help more? Let's combine evidence.

Two Features by Hand

It's a Sunny and Hot day. Should we play?

We already know how to use one feature. For two features, try this: multiply the contribution of each feature separately, along with the prior.

Compute by hand:

  1. From the data, find:
    • P(Sunny | Yes) = ?
    • P(Hot | Yes) = ?
    • P(Yes) = ?
  2. Multiply them: Score(Yes) = P(Sunny | Yes) × P(Hot | Yes) × P(Yes)
  3. Do the same for No: Score(No) = P(Sunny | No) × P(Hot | No) × P(No)
  4. Which score is higher? What do you predict?

Your calculation:

  1. P(Sunny|Yes) = ..., P(Hot|Yes) = ..., P(Yes) = ...
  2. Score(Yes) = ... × ... × ... = ...
  3. P(Sunny|No) = ..., P(Hot|No) = ..., P(No) = ...
  4. Score(No) = ... × ... × ... = ...
  5. Prediction: ...

Try Another Combination

Compute Score(Yes) and Score(No) for a Rainy and Cool day.

  1. Count P(Rainy | Yes), P(Cool | Yes), P(Rainy | No), P(Cool | No).
  2. Compute Score(Yes) and Score(No).
  3. What do you predict?

The Big Assumption

In Exercise 3.1, you multiplied P(Sunny | Yes) × P(Hot | Yes). This multiplication is only mathematically valid under a specific assumption. What is it?

A. Outlook and Temperature must have the same number of possible values

B. Outlook and Temperature are *independent* given the label — knowing one doesn't tell you about the other (once you already know whether we played)

C. The dataset must have at least 100 rows

D. All feature probabilities must be greater than 0.5

Code It: `predict_two_features`

Write a function that predicts using two features:

def predict_two_features(data, f1_name, f1_val, f2_name, f2_val, label_col, labels):
    """
    For each label, compute:
      score = P(f1_val | label) * P(f2_val | label) * P(label)
    Return the label with the highest score.
    """
    # Your code here
    pass

Part 4: N Features — The General Algorithm

You've done it for 1 feature and 2 features. Now let's generalize to any number of features.

Three Features by Hand

It's Sunny, Hot, and High humidity. Compute Score(Yes) and Score(No) using all three features.

  1. You already have P(Sunny|Yes) and P(Hot|Yes). Now find P(High|Yes) and P(High|No) from the data.
  2. Score(Yes) = P(Sunny|Yes) × P(Hot|Yes) × P(High|Yes) × P(Yes)
  3. Score(No) = P(Sunny|No) × P(Hot|No) × P(High|No) × P(No)
  4. Which class wins?

Generalize to N Features

You've seen the pattern: for each label, multiply the prior by the conditional probability of each feature. Now write the general version:

def predict(data, features, label_col, labels):
    """
    features: dict mapping feature_name -> feature_value,
              e.g. {"Outlook": "Sunny", "Temperature": "Hot", "Humidity": "High"}
    label_col: name of the label column, e.g. "Play"
    labels: list of possible labels, e.g. ["Yes", "No"]

    For each label, compute:
      score = P(label) * P(f1_val|label) * P(f2_val|label) * ... * P(fn_val|label)
    Return the label with the highest score.
    """
    # Your code here
    pass

Test on All Training Rows

Run your predict function on every row in the dataset and compare to the actual label.

  1. How many does it get right?
  2. Which rows (if any) does it get wrong? Look at those rows — can you see why?

Part 5: The Zero-Frequency Problem

Your algorithm has a fatal flaw. Let's find it and fix it.

Break Your Code

Look at the data: Outlook = Overcast appears on days 3, 7, 12, 13. What is the Play label for all of those days?

Now answer:

  1. What is P(Overcast | No)?
  2. If you try to predict for a new day with Outlook = Overcast and any other features, what happens to Score(No)?
  3. Is this a reasonable prediction? What if you had Overcast + Strong Wind + High Humidity — conditions that historically lean toward No?

Fix It: Laplace Smoothing

The fix is beautifully simple: pretend you've seen every combination at least once.

The idea: Instead of computing P(Overcast | No) = 0/5, add 1 to the numerator (pretend you saw one extra Overcast-No day) and add k to the denominator (where k is the number of possible values for that feature).

For Outlook, k = 3 (Sunny, Overcast, Rainy). So:

  • P_smooth(Overcast | No) = (0 + 1) / (5 + 3) = 1/8 = 0.125

Tasks:

  1. Compute P_smooth(Sunny | No) with k=3. Compare to the unsmoothed version.
  2. Write probability_smooth(data, event, given, k):
def probability_smooth(data, event, given, k):
    """
    Smoothed conditional probability.
    Adds 1 to the numerator and k to the denominator.
    k = number of possible values for the event's feature.
    """
    # Your code here
    pass

Update Your Predict Function

Write predict_smooth — like your predict from Part 4, but using probability_smooth instead of probability.

You'll need to know k (number of possible values) for each feature. Compute this from the data.

def predict_smooth(data, features, label_col, labels):
    """
    Like predict(), but uses Laplace smoothing.
    For each feature, k = number of distinct values of that feature in the data.
    """
    # Your code here
    pass

Why Smoothing Works

Laplace smoothing adds 1 to every count. What would happen if you added 1000 instead of 1?

A. The classifier would become more accurate because it has more "data"

B. Nothing would change — the relative ordering of scores stays the same

C. All smoothed probabilities would approach 1/k (uniform) — the fake counts would overwhelm the real data, and the classifier would ignore the actual evidence

D. Python would crash with an overflow error

Part 6: The NaiveBayes Class

You have all the pieces. Now wrap them into a reusable class with the standard ML interface: fit and predict.

Design the Interface

Before writing code, think about what fit(X, y) needs to compute and store so that predict(X) can work later without re-scanning all the training data.

Think about:

  1. What are the possible labels? (e.g., Yes and No)
  2. For each label, how many training examples had that label? (the priors)
  3. For each label AND each feature AND each feature value, how many times did that combination appear? (the conditional counts)
  4. For each feature, how many distinct values does it have? (needed for smoothing)

Your design:

After fit(X, y), self should contain:

  • self.labels = ...
  • self.label_counts = ...
  • self.feature_counts = ...
  • self.feature_values = ...
  • self.total = ...

Implement `NaiveBayes`

Implement the full class:

class NaiveBayes:
    def __init__(self, smoothing=1):
        """smoothing: Laplace smoothing parameter (default 1)."""
        pass

    def fit(self, X, y):
        """
        X : list of dicts (each dict maps feature_name -> feature_value)
        y : list of labels
        Precompute and store all the counts needed for prediction.
        """
        pass

    def predict(self, X):
        """
        X : list of dicts
        Returns: list of predicted labels, one per row.
        """
        pass

A Bigger Dataset

Test your NaiveBayes class on a spam detection problem. Each email is represented by the presence/absence of certain words:

Tasks:

  1. Train your NaiveBayes on this dataset.
  2. Compute training accuracy.
  3. Predict on these new emails:
new_emails = [
    {"free": "yes", "money": "yes", "urgent": "no", "meeting": "no"},   # likely spam?
    {"free": "no",  "money": "no",  "urgent": "no", "meeting": "yes"},  # likely ham?
    {"free": "yes", "money": "no",  "urgent": "yes", "meeting": "yes"}, # tricky!
]
  1. Do the predictions match your intuition?

Part 7: Bonus Challenges

Predict Probabilities

Right now, predict returns just the winning label. Add a predict_proba method that returns the probability of each class.

To convert scores to probabilities, divide each score by the sum of all scores:

For example, if Score(Yes) = 0.03 and Score(No) = 0.07, then:

  • P(Yes) = 0.03 / (0.03 + 0.07) = 0.3
  • P(No) = 0.07 / (0.03 + 0.07) = 0.7
def predict_proba(self, X):
    """
    X : list of dicts
    Returns: list of dicts, each mapping label -> probability.
    e.g. [{"Yes": 0.3, "No": 0.7}, ...]
    """
    pass

Test: The probabilities for each row should sum to 1.0.

Compare with scikit-learn

Scikit-learn has a built-in CategoricalNB. Compare your results:

from sklearn.preprocessing import OrdinalEncoder
from sklearn.naive_bayes import CategoricalNB
import numpy as np

# Encode string features as integers
enc = OrdinalEncoder()
X_encoded = enc.fit_transform([[row[f] for f in ["Outlook","Temperature","Humidity","Wind"]]
                                for row in data])
y_encoded = [1 if row["Play"]=="Yes" else 0 for row in data]

clf = CategoricalNB(alpha=1.0)  # alpha=1.0 is Laplace smoothing
clf.fit(X_encoded, y_encoded)

# Compare predictions
sk_preds = clf.predict(X_encoded)
print("sklearn predictions:", ["Yes" if p==1 else "No" for p in sk_preds])
print("Your predictions:   ", nb.predict(X))

Do the predictions match? If not, investigate why.

What About Continuous Features?

Your classifier works with categorical features (Sunny, Rainy, etc.). But what if a feature is a number, like actual temperature (72°F, 85°F, etc.)?

  1. Why can't you directly count P(Temperature=72.3 | Yes)? (How many training rows have that exact value?)
  2. One approach: assume numerical features follow a bell curve (Gaussian distribution) within each class. Instead of counting, you'd compute the mean and standard deviation per class, then use the bell curve to get the probability.
  3. Another approach: bin the numbers into categories (e.g., Cold: <60, Mild: 60-80, Hot: >80) and use your existing categorical classifier.

Try the binning approach on a dataset of your choice.

What you just built:

You started by counting rows in a table. From there, you:

  1. Discovered that prior probability (just counting labels) gives a baseline prediction.
  2. Found that conditional probabilities (counting features within each class) improve predictions.
  3. Learned to combine multiple features by multiplying their conditional probabilities — under the naive independence assumption.
  4. Discovered and fixed the zero-frequency problem with Laplace smoothing.
  5. Wrapped it all in a reusable class with fit and predict.

This is the Naive Bayes classifier — one of the oldest and most practical algorithms in machine learning. Despite its "naive" assumption, it excels at text classification (spam detection, sentiment analysis), medical diagnosis, and any problem with many categorical features.

The mathematical name for what you computed is Bayes' theorem: updating a prior belief using evidence. You invented it from scratch.