Entropy Practice Problem
This data science coding problem helps you practice Decision Trees, entropy, and implementation skills. Read the problem statement, write your solution, and strengthen your understanding of Decision Trees.
- Problem ID: 206
- Problem key: 206-entropy
- URL: https://datacrack.app/solve/206-entropy
- Difficulty: easy
- Topic: Decision Trees
- Module: Supervised Learning
Problem Statement
# 🧩 Entropy
---
### 🎯 Goal
Calculate the uncertainty of one decision-tree node using **entropy**.
Entropy measures how hard it is to guess the class label of a sample inside a node. A value of `0` means the node is pure. Higher values mean the labels are more mixed.
---
### 📖 Introduction
A decision tree makes predictions by asking questions and sending samples into smaller groups called **nodes**.
A node contains training samples and their labels. If all labels inside the node are the same, the node is easy to predict from. If labels are mixed, the node is uncertain.
Entropy measures that uncertainty:
$$
Entropy = -\sum_{j=1}^{k} p_j\log_2(p_j)
$$
where:
- $k$ is the number of classes in the node.
- $j$ is the class index.
- $p_j$ is the proportion of samples that belong to class $j$.
- $\log_2$ measures information in base 2.
For example, if `class_counts = [2, 2]`, the node is evenly split between two classes, so entropy is `1.0`. For binary classification, this is the maximum entropy value.
---
### 💻 Task
Implement `entropy(class_counts)` using the entropy formula.
Your function should:
- Take a list of class counts.
- Compute the proportion of each nonzero class.
- Compute entropy using base-2 logarithms.
- Return entropy rounded to 6 decimals.
- Return `0.0` if the node has no samples.
---
### 📥 Input / 📤 Output
**Input**
- `class_counts`: list of counts, one count per class
**Output**
- float: entropy rounded to 6 decimals
---
### 🧩 Starter Code
```python
import math
def entropy(class_counts):
# Your code here
pass
```
---
### 💡 Example
```python
entropy([2, 2])
```
Expected Output:
```python
1.0
```
---
### ⚠️ Common Mistakes
- Calling `log2(0)`, which is undefined.
- Forgetting the negative sign in the formula.
- Dividing by the number of classes instead of the number of samples.Starter Code
import math
def entropy(class_counts):
# Your code here
pass