Accuracy Score

A function for computing for computing basic classifcation accuracy, per-class accuracy, and average per-class accuracy.

from mlxtend.evaluate import accuracy_score

Example 1 -- Standard Accuracy

The "overall" accuracy is defined as the number of correct predictions (true positives TP and true negatives TN) over all samples n:

import numpy as np
from mlxtend.evaluate import accuracy_score


y_targ = [0, 0, 0, 1, 1, 1, 2, 2, 2]
y_pred = [1, 0, 0, 0, 1, 2, 0, 2, 2]

accuracy_score(y_targ, y_pred)
0.5555555555555556

Example 2 -- Per-Class Accuracy

The per-class accuracy is the accuracy of one class (defined as the pos_label) versus all remaining datapoints in the dataset.

import numpy as np
from mlxtend.evaluate import accuracy_score


y_targ = [0, 0, 0, 1, 1, 1, 2, 2, 2]
y_pred = [1, 0, 0, 0, 1, 2, 0, 2, 2]

std_acc = accuracy_score(y_targ, y_pred)
bin_acc = accuracy_score(y_targ, y_pred, method='binary', pos_label=1)

print(f'Standard accuracy: {std_acc*100:.2f}%')
print(f'Class 1 accuracy: {bin_acc*100:.2f}%')
Standard accuracy: 55.56%
Class 1 accuracy: 66.67%

Example 3 -- Average Per-Class Accuracy

Overview

The "overall" accuracy is defined as the number of correct predictions (true positives TP and true negatives TN) over all samples n:

in a binary class setting:

In a multi-class setting, we can generalize the computation of the accuracy as the fraction of all true predictions (the diagonal) over all samples n.

Considering a multi-class problem with 3 classes (C0, C1, C2)

let's assume our model made the following predictions:

We compute the accuracy as:

Now, in order to compute the average per-class accuracy, we compute the binary accuracy for each class label separately; i.e., if class 1 is the positive class, class 0 and 2 are both considered the negative class.

import numpy as np
from mlxtend.evaluate import accuracy_score


y_targ = [0, 0, 0, 1, 1, 1, 2, 2, 2]
y_pred = [1, 0, 0, 0, 1, 2, 0, 2, 2]

std_acc = accuracy_score(y_targ, y_pred)
bin_acc = accuracy_score(y_targ, y_pred, method='binary', pos_label=1)
avg_acc = accuracy_score(y_targ, y_pred, method='average')

print(f'Standard accuracy: {std_acc*100:.2f}%')
print(f'Class 1 accuracy: {bin_acc*100:.2f}%')
print(f'Average per-class accuracy: {avg_acc*100:.2f}%')
Standard accuracy: 55.56%
Class 1 accuracy: 66.67%
Average per-class accuracy: 70.37%

References

API

accuracy_score(y_target, y_predicted, method='standard', pos_label=1, normalize=True)

General accuracy function for supervised learning. Parameters

Returns

score: float