# The Machine Learning Metrics

When I started my machine learning journey, it wasn't very straightforward to find my way among all the existing metrics. What does each of these metrics represent ? What do they tell us about my training and my output model ? These are all the questions I was asking myself. Today, we will dive deep into these metrics and understand the meaning of those metrics.

## Accuracy

Let's start with the most common one: the **accuracy.**

### Formula

$$Accuracy = \frac{\text{correct prediction}} {\text{total prediction}}$$

### Meaning

This metric tells us how many predictions the model has made correctly among the samples in test dataset.

### Limit

The accuracy only tells us how many predictions the model gets right. However, for classification problems, it can be misleading.

Imagine your dataset contains 1,000 samples:

- 950 samples belong to class 0.
- 50 samples belong to class 1. A model that always predicts class 0 would achieve 95% accuracy, even though it never identifies a single sample from class 1.

This is why we need other metrics to better understand the performance of our model.

## Precision

### Formula

$$Precision = \frac{\text{TP}} {\text{FP + TP}}$$

Where:

- TP (True Positive): The model correctly predicts a positive class.
- FP (False Positive): The model predicts a positive class, but the actual class is negative.

It is the number of **correct predictions for a class among the total predictions made for that class.**

### Meaning

Precision tells us, among all the samples that our model predicted for a class, how many were actually positive.

The precision answers to the following question:

<div data-node-type="callout">
<div data-node-type="callout-emoji">❓</div>
<div data-node-type="callout-text"><p>When my model predicts a class, how often is it correct?</p></div>
</div>

Example Imagine we are building a phishing email detection model.

Our model identifies 100 emails as phishing, but only 80 of them are actually phishing emails.

For this example, the precision would be:

$$Precision = \frac{80} {100} = 0.8 = 80\%$$

It means that 80% of the emails identified as phishing were actually phishing emails.

### Limit

Precision doesn't tell us how many of a class samples our model failed to identify.

For example, if our model predicts only 10 emails as phishing and all those 10 emails are really phishing then it achieve 100% precision. Whereas it might have missed 90 other emails that are phishing attempts.

This is why we also need recall.

## Recall

### Formula

$$Recall = \frac{TP} {TP + FN}$$

Where:

- TP (True Positive): The model correctly predicts a positive class.
- FN (False Negative): The actual class is positive, but the model predicts it as another class.

### Meaning

Recall tells us, among all the samples that actually belong to a class, how many our model successfully identified.

It answers the following question:

<div data-node-type="callout">
<div data-node-type="callout-emoji">❓</div>
<div data-node-type="callout-text"><p>How many samples of a class can my model find?</p></div>
</div>

Example Let's continue with our phishing detection model.

Imagine our dataset contains 100 phishing emails. Our model successfully identifies 80 of them but misses the remaining 20.

The value of the recall here would be:

$$Recall = \frac{80} {100} = 0.8 = 80\%$$

It means that our model successfully identifies 80% of all phishing emails present in the test dataset.

### Limit

Recall doesn't tell us how many negative samples our model incorrectly classified as positive.

A model could achieve 100% recall by simply classifying every email as phishing. However, this would generate many false alarms.

This is why we need a metric that takes both precision and recall into consideration.

## F1-Score

### Formula

$$F1 = 2 × \frac{\text{Precision × Recall}} {\text{Precision + Recall}}$$

The F1 score is the **harmonic mean** of precision and recall, providing a single metric that balances both by penalizing extreme differences between them. It ranges from **0** (worst) to **1** (best).

### Meaning

The F1-score tells us how well our model balances precision and recall.

It is useful when we want our model to both:

- make as few false positive predictions as possible (high precision),
- find as many actual positive samples as possible (high recall).

For example, imagine our phishing detection model has:

- Precision = 80%
- Recall = 80% Then:

$$F1 = 2 × \frac{0.8 × 0.8}{0.8 + 0.8} = 0.8 = 80\%$$

This could mean that the model has a good balance between its ability to correctly identify phishing emails and its ability to avoid incorrectly classifying legitimate emails as phishing.

The F1-score is particularly useful when both precision and recall are important and we want to summarize them with a single metric.

### Note

The F1-score doesn't tell us whether the problem comes from precision or recall.

For example, two models can have a similar F1-score while having very different precision and recall values. One model might have high precision but low recall, while another might have low precision but high recall.

Therefore, we should not look at the F1-score alone. It is important to also look at **precision and recall separately** to understand how the model is performing.

## To recap

- A precision of 80% for a given class means that our model correctly predicts 80% of all examples that it **predicted as belonging to that class**.
- A recall of 80% for a given class means that our model correctly predicts 80% of all examples in the test dataset that **actually belong to that class**.
- An F1-score of 80% means that our model has a good **balance between precision and recall**, with both metrics being taken into account.