Confusion Matrix for Model Selection

Before we jump into calculating Accuracy, Precision, and Recall for our classification model, we first need to understand what a Confusion matrix is.

In machine learning, Classification is used to split data into categories. But after cleaning and preprocessing the data and training our model, how do we know if our classification model performs well?

That is where a confusion matrix comes into the picture.

The confusion matrix is in the form of a square matrix where the column represents the actual values and the row depicts the predicted value of the model and vice versa. Specifically;

1. A confusion matrix presents the ways in which a classification model becomes confused while making predictions.”
2. A good matrix (model) will have large values across the diagonal and small values off the diagonal.
3. Measuring a confusion matrix provides better insight in particular if our classification model is getting correct and what types of errors it is creating.

The confusion matrix has two parameters:

• Actual
• Predicted

Actual values are true binary values “0” and”1″. The prediction value that comes after fitting the model is also confusing because it does not predict all values properly. 

So, these four terms are born to know the evaluation performance:

1. TP is a true positive value means the predicted value is the same as the actual value, which is an outcome where the model correctly predicts the positive class.
2. TN is a true negative value means the actual value is “0” and the model predicted it the same also, which is an outcome where the model incorrectly predicts the negative class when the actual class is positive.

3. FP is a false positive value means the actual value is “0” but it predicted “1” , which is an outcome where the model incorrectly predicts the positive class when the actual class is negative, and is also called a type 1 error.

4. FN is a false negative value means the actual value is “1” but it predicted “0”, is an outcome where the model incorrectly predicts the negative class when the actual class is positive, and also called a type 2 error.

Now, let’s understand this with an example:

Case: A simple story of Boy and a wolf, 

For having fun, a boy shouted out “Wolf”, even though there is no wolf, villagers ran out to save themselves but soon got angry when they realized the boy was playing a joke. 

One day, the boy saw a wolf in reality and called out “Wolf is coming”, but villagers denied being fooled again and stayed at home. And then, the hungry wolf demolished the village, destroyed their crops.

After that, the entire village suffered many problems.

 Making definitions: 

• “Wolf” is a positive class
• “No wolf” is a negative class

Now, a wolf-prediction can be designed using a 2×2 confusion matrix that could reflect all four possible conditions;

Additional Reads:

[1] Please refer to this interesting article from WordPress on Confusion Matrix:

[2] You know the theory – now let’s put it into practice. You can code a confusion matrix with the Scikit-learn (sklearn) library in Python. This article from AnalyticsVidhya will walk you through the practical implementation of the confusion matrix.