Train and Predict with Log Probabilities Practice Problem

This data science coding problem helps you practice Naive Bayes, train and predict with log probabilities, and implementation skills. Read the problem statement, write your solution, and strengthen your understanding of Naive Bayes.

• Problem ID: 195
• Problem key: 195-train-and-predict-with-log-probabilities
• URL: https://datacrack.app/solve/195-train-and-predict-with-log-probabilities
• Difficulty: hard
• Topic: Naive Bayes
• Module: Supervised Learning

Problem Statement

# 🧩 Train and Predict with Log Probabilities

---

### 🎯 Goal

Train a Laplace-smoothed Naive Bayes model, then predict by adding log probabilities instead of multiplying many small probabilities.

---

### 📖 Introduction

The previous problem fixed zero probabilities with Laplace smoothing.

Now we keep the same workflow:

1. Train from `X` and `y`.
2. Store class priors and smoothed likelihood tables.
3. Predict using only the stored model.

The only new idea is how we combine probabilities during prediction.

Naive Bayes normally multiplies:

$$
Score(class)=P(class)\times\prod_i P(feature_i=value_i\mid class)
$$

When there are many features, this product can become extremely small. Real implementations often use logarithms because logs turn multiplication into addition:

$$
\log(Score(class))=\log(P(class))+\sum_i \log(P(feature_i=value_i\mid class))
$$

where:

- $i$ refers to the position of a feature in the query. For example, in `["red", "small"]`, one position is `color=red` and the next is `size=small`.
- $\prod$ means multiply probabilities.
- $\sum$ means add values.
- $\log$ means logarithm. In Python, use the natural log with `math.log`.

The class with the largest log score is still the predicted class. Logs change the calculation scale, not the decision rule.

---

### 📖 Example Training Data

| color | size | class |
|---|---|---|
| red | small | A |
| red | large | A |
| blue | small | B |
| blue | large | B |

For query `["red", "small"]`, the smoothed probability score for class `A` is:

$$
0.5\times0.75\times0.5=0.1875
$$

The log score is:

$$
\log(0.5)+\log(0.75)+\log(0.5)=-1.673976
$$

---

### 💻 Task

Implement `log_smoothed_naive_bayes_predict(X, y, query)`.

Your nested training helper should create a model with this structure:

```python
{
    "classes": [...],
    "priors": {class_label: ...},
    "likelihoods": {
        class_label: [
            {value_1: probability, value_2: probability, ...},  # one dictionary for feature column 0
            {value_1: probability, value_2: probability, ...},  # one dictionary for feature column 1
            ...
        ]
    }
}
```

Your function should:

- Train a model from `X` and `y`.
- Store class priors and Laplace-smoothed likelihood tables.
- During prediction, calculate one log score for every class.
- Add log probabilities instead of multiplying raw probabilities.
- Use only the trained model during prediction.
- Return all log scores rounded to 6 decimals.
- Return the class with the highest log score.
- Break ties alphabetically.
- Return `{"log_scores": {}, "prediction": None}` when there are no classes.

Assume query values already appeared in the training data.

---

### 📥 Input / 📤 Output

**Input:** categorical feature matrix `X`, labels `y`, and one categorical `query`.

**Output:**

```python
{
    "log_scores": {"A": ..., "B": ...},
    "prediction": ...
}
```

---

### 🧩 Starter Code

```python
def log_smoothed_naive_bayes_predict(X, y, query):
    def train_model():
        # Learn priors and smoothed likelihood tables.
        pass

    def predict_with_model(model):
        # Add log probabilities from the learned model.
        pass

    # Train once, then predict with that model.
    pass
```

---

### 💡 Example

```python
X = [["red", "small"], ["red", "large"], ["blue", "small"], ["blue", "large"]]
y = ["A", "A", "B", "B"]

log_smoothed_naive_bayes_predict(X, y, ["red", "small"])
```

Expected output:

```python
{"log_scores": {"A": -1.673976, "B": -2.772589}, "prediction": "A"}
```

---

### ⚠️ Common Mistakes

- Taking the log after rounding probabilities.
- Multiplying logs instead of adding them.
- Forgetting that the largest log score wins, even though the values are negative.
- Recalculating probabilities from `X` and `y` inside prediction.