Does PLA Care How Wrong It Is? Practice Problem

This data science coding problem helps you practice Perceptron, does pla care how wrong it is?, and implementation skills. Read the problem statement, write your solution, and strengthen your understanding of Perceptron.

• Problem ID: 218
• Problem key: 218-does-pla-care-how-wrong-it-is
• URL: https://datacrack.app/solve/218-does-pla-care-how-wrong-it-is
• Difficulty: easy
• Topic: Perceptron
• Module: Introduction to Machine Learning

Problem Statement

# 🧩 Does PLA Care How Wrong It Is?

---

### 🎯 Goal

In this problem, we will look at **one Perceptron training step** and ask an important question:

If the model makes a **small error** versus a **much larger error**, does the Perceptron Learning Algorithm make a different-sized update?

We will compare the update rule in both cases to see whether the size of the error affects how much the weights and bias change.

This problem exposes an important limitation of the classical Perceptron Learning Algorithm.

---

### 📖 Introduction

PLA updates when a sample is misclassified:

$$
w \leftarrow w+\eta y_i x_i
$$

$$
b \leftarrow b+\eta y_i
$$

Notice what appears in the update:

- the learning rate $\eta$
- the true label $y_i$
- the sample $x_i$

The score value itself does not directly appear in the update size.

So if the **same sample** is misclassified with:

$$
z=-0.1
$$

or with:

$$
z=-10
$$

PLA still uses the same update expression as long as $x_i$, $y_i$, and $\eta$ are the same.

---

### 🔍 Important Distinction

This does **not** mean different samples get the same update.

Different samples have different $x_i$, so they can create different updates.

The point here is narrower:

> For the same sample, PLA does not make a larger update just because the wrong score is farther from zero.

---

### 📥 Input

- `sample`: one feature vector.
- `true_label`: either `-1` or `1`.
- `learning_rate`: update size.
- `scores`: a list of score values produced by different current model states for the same sample.

In this problem, the scores are already given. You do not need to calculate them from weights. The point is to compare how PLA reacts once those scores lead to predictions.

### 📤 Output

Return a dictionary with:

- `updates`: one update summary for each score.
- `same_update`: `True` if all update changes are identical, otherwise `False`.

Each update summary should include:

- `score`
- `prediction`
- `mistake`
- `delta_weights` ($w_{\text{new}} - w_{\text{old}} = \eta y_i x_i$)
- `delta_bias` ($b_{\text{new}} - b_{\text{old}} = \eta y_i$)

Round numeric update values to 6 decimals.

---

### 💻 Task

Implement `compare_pla_update_magnitudes(sample, true_label, learning_rate, scores)`.

For each score, determine whether the Perceptron makes a mistake.

If it makes a mistake, calculate **how much the weights and bias would change**:

$$
\Delta w = \eta y_i x_i
$$

$$
\Delta b = \eta y_i
$$

where:

- $\Delta w$ is the change in the weights ($w_{\text{new}} - w_{\text{old}}$).
- $\Delta b$ is the change in the bias ($b_{\text{new}} - b_{\text{old}}$).

So the full update would be:

$$
w_{\text{new}} = w_{\text{old}} + \Delta w
$$

$$
b_{\text{new}} = b_{\text{old}} + \Delta b
$$

Your function should:

1. Convert each score into a prediction.
2. Check whether the prediction is wrong.
3. If it is wrong, calculate $\Delta w$ and $\Delta b$.
4. If it is correct, use zero for both changes.
5. Compare the changes produced by the different scores.
6. Return each result and whether the update changes are identical.
---

### 🧩 Starter Code

```python
def compare_pla_update_magnitudes(sample, true_label, learning_rate, scores):
    # Store the update information for each score

    # Go through each score

        # Convert the score into a prediction (-1 or 1)

        # Check whether the prediction is wrong

        # If there is a mistake, calculate the change in each weight

        # If there is a mistake, calculate the change in the bias

        # If the prediction is correct, there is no update

        # Store the score, prediction, mistake, and update changes

    # Compare the update changes across all scores

    # Return the update summaries and whether they are the same
    pass
```

---

### 💡 Example

```python
compare_pla_update_magnitudes(
    sample=[2, 1],
    true_label=1,
    learning_rate=0.5,
    scores=[-0.1, -10]
)
```

Both scores are negative, so both predictions are `-1`.

The true label is `1`, so both are mistakes.

For both cases:

$$
\Delta w = 0.5(1)[2,1] = [1,0.5]
$$

$$
\Delta b = 0.5(1)=0.5
$$

Expected output:

```python
{
    "updates": [
        {
            "score": -0.1,
            "prediction": -1,
            "mistake": True,
            "delta_weights": [1.0, 0.5],
            "delta_bias": 0.5
        },
        {
            "score": -10,
            "prediction": -1,
            "mistake": True,
            "delta_weights": [1.0, 0.5],
            "delta_bias": 0.5
        }
    ],
    "same_update": True
}
```

---

### ⚠️ Common Mistakes

- Scaling the update by the score magnitude.
- Saying PLA gives the same update for different samples.
- Forgetting that correct predictions produce no update.
- Teaching Logistic Regression here. This problem only motivates the next idea.