Lift Score

Scoring function to compute the LIFT metric, the ratio of correctly predicted positive examples and the actual positive examples in the test dataset.

from mlxtend.evaluate import lift_score

Overview

In the context of classification, lift [1] compares model predictions to randomly generated predictions. Lift is often used in marketing research combined with gain and lift charts as a visual aid [2]. For example, assuming a 10% customer response as a baseline, a lift value of 3 would correspond to a 30% customer response when using the predictive model. Note that lift has the range .

There are many strategies to compute lift, and below, we will illustrate the computation of the lift score using a classic confusion matrix. For instance, let's assume the following prediction and target labels, where "1" is the positive class:

Then, our confusion matrix would look as follows:

Based on the confusion matrix above, with "1" as positive label, we compute lift as follows:

Plugging in the actual values from the example above, we arrive at the following lift value:

An alternative way to computing lift is by using the support metric [3]:

where is the number of samples in the datset. Plugging the values from our example into the equation above, we arrive at

References

Example 1 - Computing Lift

This examples demonstrates the basic use of the lift_score function using the example from the Overview section.

import numpy as np
from mlxtend.evaluate import lift_score

y_target =    np.array([0, 0, 1, 0, 0, 1, 1, 1, 1, 1])
y_predicted = np.array([1, 0, 1, 0, 0, 0, 0, 1, 0, 0])

lift_score(y_target, y_predicted)
1.1111111111111112

Example 2 - Using lift_score in GridSearch

The lift_score function can also be used with scikit-learn objects, such as GridSearch:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.svm import SVC
from sklearn.metrics import make_scorer

# make custom scorer
lift_scorer = make_scorer(lift_score)


iris = load_iris()
X, y = iris.data, iris.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=123)

hyperparameters = [{'kernel': ['rbf'], 'gamma': [1e-3, 1e-4],
                     'C': [1, 10, 100, 1000]},
                   {'kernel': ['linear'], 'C': [1, 10, 100, 1000]}]

clf = GridSearchCV(SVC(), hyperparameters, cv=10,
                   scoring=lift_scorer)
clf.fit(X_train, y_train)

print(clf.best_score_)
print(clf.best_params_)
3.0
{'gamma': 0.001, 'kernel': 'rbf', 'C': 1000}

API

lift_score(y_target, y_predicted, binary=True, positive_label=1)

Lift measures the degree to which the predictions of a classification model are better than randomly-generated predictions.

The in terms of True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN), the lift score is computed as: [ TP/(TP+FN) ] / [ (TP+FP) / (TP+TN+FP+FN) ]

Parameters

  • y_target : array-like, shape=[n_samples]

    True class labels.

  • y_predicted : array-like, shape=[n_samples]

    Predicted class labels.

  • binary : bool (default: True)

    Maps a multi-class problem onto a binary, where the positive class is 1 and all other classes are 0.

  • positive_label : int (default: 0)

    Class label of the positive class.

Returns

  • score : float

    Lift score in the range [0, ]

Examples

For usage examples, please see http://rasbt.github.io/mlxtend/user_guide/evaluate/lift_score/