Sequential Feature Selector
Implementation of sequential feature algorithms (SFAs)  greedy search algorithms  that have been developed as a suboptimal solution to the computationally often not feasible exhaustive search.
from mlxtend.feature_selection import SequentialFeatureSelector
Overview
Sequential feature selection algorithms are a family of greedy search algorithms that are used to reduce an initial ddimensional feature space to a kdimensional feature subspace where k < d. The motivation behind feature selection algorithms is to automatically select a subset of features that is most relevant to the problem. The goal of feature selection is twofold: We want to improve the computational efficiency and reduce the generalization error of the model by removing irrelevant features or noise. A wrapper approach such as sequential feature selection is especially useful if embedded feature selection  for example, a regularization penalty like LASSO  is not applicable.
In a nutshell, SFAs remove or add one feature at the time based on the classifier performance until a feature subset of the desired size k is reached. There are 4 different flavors of SFAs available via the SequentialFeatureSelector
:
 Sequential Forward Selection (SFS)
 Sequential Backward Selection (SBS)
 Sequential Forward Floating Selection (SFFS)
 Sequential Backward Floating Selection (SBFS)
The floating variants, SFFS and SBFS, can be considered as extensions to the simpler SFS and SBS algorithms. The floating algorithms have an additional exclusion or inclusion step to remove features once they were included (or excluded), so that a larger number of feature subset combinations can be sampled. It is important to emphasize that this step is conditional and only occurs if the resulting feature subset is assessed as "better" by the criterion function after removal (or addition) of a particular feature. Furthermore, I added an optional check to skip the conditional exclusion steps if the algorithm gets stuck in cycles.
How is this different from Recursive Feature Elimination (RFE)  e.g., as implemented in sklearn.feature_selection.RFE
? RFE is computationally less complex using the feature weight coefficients (e.g., linear models) or feature importance (treebased algorithms) to eliminate features recursively, whereas SFSs eliminate (or add) features based on a userdefined classifier/regression performance metric.
The SFAs are outlined in pseudo code below:
Sequential Forward Selection (SFS)
Input:
 The SFS algorithm takes the whole dimensional feature set as input.
Output: , where
 SFS returns a subset of features; the number of selected features , where , has to be specified a priori.
Initialization: ,
 We initialize the algorithm with an empty set ("null set") so that (where is the size of the subset).
Step 1 (Inclusion):
Go to Step 1
 in this step, we add an additional feature, , to our feature subset .
 is the feature that maximizes our criterion function, that is, the feature that is associated with the best classifier performance if it is added to .
 We repeat this procedure until the termination criterion is satisfied.
Termination:
 We add features from the feature subset until the feature subset of size contains the number of desired features that we specified a priori.
Sequential Backward Selection (SBS)
Input: the set of all features,
 The SBS algorithm takes the whole feature set as input.
Output: , where
 SBS returns a subset of features; the number of selected features , where , has to be specified a priori.
Initialization: ,
 We initialize the algorithm with the given feature set so that the .
Step 1 (Exclusion):
Go to Step 1
 In this step, we remove a feature, from our feature subset .
 is the feature that maximizes our criterion function upon re,oval, that is, the feature that is associated with the best classifier performance if it is removed from .
 We repeat this procedure until the termination criterion is satisfied.
Termination:
 We add features from the feature subset until the feature subset of size contains the number of desired features that we specified a priori.
Sequential Backward Floating Selection (SBFS)
Input: the set of all features,
 The SBFS algorithm takes the whole feature set as input.
Output: , where
 SBFS returns a subset of features; the number of selected features , where , has to be specified a priori.
Initialization: ,
 We initialize the algorithm with the given feature set so that the .
Step 1 (Exclusion):
Go to Step 2
 In this step, we remove a feature, from our feature subset .
 is the feature that maximizes our criterion function upon re,oval, that is, the feature that is associated with the best classifier performance if it is removed from .
Step 2 (Conditional Inclusion):
if J(x_k + x) > J(x_k + x):
Go to Step 1
 In Step 2, we search for features that improve the classifier performance if they are added back to the feature subset. If such features exist, we add the feature for which the performance improvement is maximized. If or an improvement cannot be made (i.e., such feature cannot be found), go back to step 1; else, repeat this step.
Termination:
 We add features from the feature subset until the feature subset of size contains the number of desired features that we specified a priori.
Sequential Forward Floating Selection (SFFS)
Input: the set of all features,
 The SFFS algorithm takes the whole feature set as input, if our feature space consists of, e.g. 10, if our feature space consists of 10 dimensions (d = 10).
Output: a subset of features, , where
 The returned output of the algorithm is a subset of the feature space of a specified size. E.g., a subset of 5 features from a 10dimensional feature space (k = 5, d = 10).
Initialization: ,
 We initialize the algorithm with an empty set ("null set") so that the k = 0 (where k is the size of the subset)
Step 1 (Inclusion):
Go to Step 2
Step 2 (Conditional Exclusion):
:
Go to Step 1
 In step 1, we include the feature from the feature space that leads to the best performance increase for our feature subset (assessed by the criterion function). Then, we go over to step 2

In step 2, we only remove a feature if the resulting subset would gain an increase in performance. If or an improvement cannot be made (i.e., such feature cannot be found), go back to step 1; else, repeat this step.

Steps 1 and 2 are repeated until the Termination criterion is reached.
Termination: stop when k equals the number of desired features
References

Ferri, F. J., Pudil P., Hatef, M., Kittler, J. (1994). "Comparative study of techniques for largescale feature selection." Pattern Recognition in Practice IV : 403413.

Pudil, P., Novovičová, J., & Kittler, J. (1994). "Floating search methods in feature selection." Pattern recognition letters 15.11 (1994): 11191125.
Example 1  A simple Sequential Forward Selection example
Initializing a simple classifier from scikitlearn:
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_iris
iris = load_iris()
X = iris.data
y = iris.target
knn = KNeighborsClassifier(n_neighbors=4)
We start by selection the "best" 3 features from the Iris dataset via Sequential Forward Selection (SFS). Here, we set forward=True
and floating=False
. By choosing cv=0
, we don't perform any crossvalidation, therefore, the performance (here: 'accuracy'
) is computed entirely on the training set.
from mlxtend.feature_selection import SequentialFeatureSelector as SFS
sfs1 = SFS(knn,
k_features=3,
forward=True,
floating=False,
verbose=2,
scoring='accuracy',
cv=0)
sfs1 = sfs1.fit(X, y)
[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 4 out of 4  elapsed: 0.0s finished
[20171101 18:46:21] Features: 1/3  score: 0.96[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 3 out of 3  elapsed: 0.0s finished
[20171101 18:46:21] Features: 2/3  score: 0.973333333333[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 2 out of 2  elapsed: 0.0s finished
[20171101 18:46:21] Features: 3/3  score: 0.973333333333
Via the subsets_
attribute, we can take a look at the selected feature indices at each step:
sfs1.subsets_
{1: {'avg_score': 0.95999999999999996,
'cv_scores': array([ 0.96]),
'feature_idx': (3,)},
2: {'avg_score': 0.97333333333333338,
'cv_scores': array([ 0.97333333]),
'feature_idx': (2, 3)},
3: {'avg_score': 0.97333333333333338,
'cv_scores': array([ 0.97333333]),
'feature_idx': (1, 2, 3)}}
Furthermore, we can access the indices of the 3 best features directly via the k_feature_idx_
attribute:
sfs1.k_feature_idx_
(1, 2, 3)
Finally, the prediction score for these 3 features can be accesses via k_score_
:
sfs1.k_score_
0.97333333333333338
Example 2  Toggling between SFS, SBS, SFFS, and SBFS
Using the forward
and floating
parameters, we can toggle between SFS, SBS, SFFS, and SBFS as shown below. Note that we are performing (stratified) 4fold crossvalidation for more robust estimates in contrast to Example 1. Via n_jobs=1
, we choose to run the crossvalidation on all our available CPU cores.
# Sequential Forward Selection
sfs = SFS(knn,
k_features=3,
forward=True,
floating=False,
scoring='accuracy',
cv=4,
n_jobs=1)
sfs = sfs.fit(X, y)
print('\nSequential Forward Selection (k=3):')
print(sfs.k_feature_idx_)
print('CV Score:')
print(sfs.k_score_)
###################################################
# Sequential Backward Selection
sbs = SFS(knn,
k_features=3,
forward=False,
floating=False,
scoring='accuracy',
cv=4,
n_jobs=1)
sbs = sbs.fit(X, y)
print('\nSequential Backward Selection (k=3):')
print(sbs.k_feature_idx_)
print('CV Score:')
print(sbs.k_score_)
###################################################
# Sequential Forward Floating Selection
sffs = SFS(knn,
k_features=3,
forward=True,
floating=True,
scoring='accuracy',
cv=4,
n_jobs=1)
sffs = sffs.fit(X, y)
print('\nSequential Forward Floating Selection (k=3):')
print(sffs.k_feature_idx_)
print('CV Score:')
print(sffs.k_score_)
###################################################
# Sequential Backward Floating Selection
sbfs = SFS(knn,
k_features=3,
forward=False,
floating=True,
scoring='accuracy',
cv=4,
n_jobs=1)
sbfs = sbfs.fit(X, y)
print('\nSequential Backward Floating Selection (k=3):')
print(sbfs.k_feature_idx_)
print('CV Score:')
print(sbfs.k_score_)
Sequential Forward Selection (k=3):
(1, 2, 3)
CV Score:
0.972756410256
Sequential Backward Selection (k=3):
(1, 2, 3)
CV Score:
0.972756410256
Sequential Forward Floating Selection (k=3):
(1, 2, 3)
CV Score:
0.972756410256
Sequential Backward Floating Selection (k=3):
(1, 2, 3)
CV Score:
0.972756410256
In this simple scenario, selecting the best 3 features out of the 4 available features in the Iris set, we end up with similar results regardless of which sequential selection algorithms we used.
Example 3  Visualizing the results in DataFrames
For our convenience, we can visualize the output from the feature selection in a pandas DataFrame format using the get_metric_dict
method of the SequentialFeatureSelector object. The columns std_dev
and std_err
represent the standard deviation and standard errors of the crossvalidation scores, respectively.
Below, we see the DataFrame of the Sequential Forward Selector from Example 2:
import pandas as pd
pd.DataFrame.from_dict(sfs.get_metric_dict()).T
avg_score  ci_bound  cv_scores  feature_idx  std_dev  std_err  

1  0.952991  0.0660624  [0.974358974359, 0.948717948718, 0.88888888888...  (3,)  0.0412122  0.0237939 
2  0.959936  0.0494801  [0.974358974359, 0.948717948718, 0.91666666666...  (2, 3)  0.0308676  0.0178214 
3  0.972756  0.0315204  [0.974358974359, 1.0, 0.944444444444, 0.972222...  (1, 2, 3)  0.0196636  0.0113528 
Now, let's compare it to the Sequential Backward Selector:
pd.DataFrame.from_dict(sbs.get_metric_dict()).T
avg_score  ci_bound  cv_scores  feature_idx  std_dev  std_err  

3  0.972756  0.0315204  [0.974358974359, 1.0, 0.944444444444, 0.972222...  (1, 2, 3)  0.0196636  0.0113528 
4  0.952991  0.0372857  [0.974358974359, 0.948717948718, 0.91666666666...  (0, 1, 2, 3)  0.0232602  0.0134293 
We can see that both SFS and SBFS found the same "best" 3 features, however, the intermediate steps where obviously different.
The ci_bound
column in the DataFrames above represents the confidence interval around the computed crossvalidation scores. By default, a confidence interval of 95% is used, but we can use different confidence bounds via the confidence_interval
parameter. E.g., the confidence bounds for a 90% confidence interval can be obtained as follows:
pd.DataFrame.from_dict(sbs.get_metric_dict(confidence_interval=0.90)).T
avg_score  ci_bound  cv_scores  feature_idx  std_dev  std_err  

3  0.972756  0.0242024  [0.974358974359, 1.0, 0.944444444444, 0.972222...  (1, 2, 3)  0.0196636  0.0113528 
4  0.952991  0.0286292  [0.974358974359, 0.948717948718, 0.91666666666...  (0, 1, 2, 3)  0.0232602  0.0134293 
Example 4  Plotting the results
After importing the little helper function plotting.plot_sequential_feature_selection
, we can also visualize the results using matplotlib figures.
from mlxtend.plotting import plot_sequential_feature_selection as plot_sfs
import matplotlib.pyplot as plt
sfs = SFS(knn,
k_features=4,
forward=True,
floating=False,
scoring='accuracy',
verbose=2,
cv=5)
sfs = sfs.fit(X, y)
fig1 = plot_sfs(sfs.get_metric_dict(), kind='std_dev')
plt.ylim([0.8, 1])
plt.title('Sequential Forward Selection (w. StdDev)')
plt.grid()
plt.show()
[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 4 out of 4  elapsed: 0.0s finished
[20171101 18:46:24] Features: 1/4  score: 0.96[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 3 out of 3  elapsed: 0.0s finished
[20171101 18:46:24] Features: 2/4  score: 0.966666666667[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 2 out of 2  elapsed: 0.0s finished
[20171101 18:46:24] Features: 3/4  score: 0.953333333333[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=1)]: Done 1 out of 1  elapsed: 0.0s finished
[20171101 18:46:24] Features: 4/4  score: 0.973333333333
Example 5  Sequential Feature Selection for Regression
Similar to the classification examples above, the SequentialFeatureSelector
also supports scikitlearn's estimators
for regression.
from sklearn.linear_model import LinearRegression
from sklearn.datasets import load_boston
boston = load_boston()
X, y = boston.data, boston.target
lr = LinearRegression()
sfs = SFS(lr,
k_features=13,
forward=True,
floating=False,
scoring='neg_mean_squared_error',
cv=10)
sfs = sfs.fit(X, y)
fig = plot_sfs(sfs.get_metric_dict(), kind='std_err')
plt.title('Sequential Forward Selection (w. StdErr)')
plt.grid()
plt.show()
Example 6  Using the Selected Feature Subset For Making New Predictions
# Initialize the dataset
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.33, random_state=1)
knn = KNeighborsClassifier(n_neighbors=4)
# Select the "best" three features via
# 5fold crossvalidation on the training set.
from mlxtend.feature_selection import SequentialFeatureSelector as SFS
sfs1 = SFS(knn,
k_features=3,
forward=True,
floating=False,
scoring='accuracy',
cv=5)
sfs1 = sfs1.fit(X_train, y_train)
print('Selected features:', sfs1.k_feature_idx_)
Selected features: (1, 2, 3)
# Generate the new subsets based on the selected features
# Note that the transform call is equivalent to
# X_train[:, sfs1.k_feature_idx_]
X_train_sfs = sfs1.transform(X_train)
X_test_sfs = sfs1.transform(X_test)
# Fit the estimator using the new feature subset
# and make a prediction on the test data
knn.fit(X_train_sfs, y_train)
y_pred = knn.predict(X_test_sfs)
# Compute the accuracy of the prediction
acc = float((y_test == y_pred).sum()) / y_pred.shape[0]
print('Test set accuracy: %.2f %%' % (acc * 100))
Test set accuracy: 96.00 %
Example 7  Sequential Feature Selection and GridSearch
# Initialize the dataset
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.33, random_state=1)
Use scikitlearn's GridSearch
to tune the hyperparameters inside and outside the SequentialFeatureSelector
:
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from mlxtend.feature_selection import SequentialFeatureSelector as SFS
import mlxtend
knn = KNeighborsClassifier(n_neighbors=2)
sfs1 = SFS(estimator=knn,
k_features=3,
forward=True,
floating=False,
scoring='accuracy',
cv=5)
pipe = Pipeline([('sfs', sfs1),
('knn', knn)])
param_grid = [
{'sfs__k_features': [1, 2, 3, 4],
'sfs__estimator__n_neighbors': [1, 2, 3, 4]}
]
gs = GridSearchCV(estimator=pipe,
param_grid=param_grid,
scoring='accuracy',
n_jobs=1,
cv=5,
refit=False)
# run gridearch
gs = gs.fit(X_train, y_train)
... and the "best" parameters determined by GridSearch are ...
print("Best parameters via GridSearch", gs.best_params_)
Best parameters via GridSearch {'sfs__estimator__n_neighbors': 1, 'sfs__k_features': 3}
Obtaining the best k feature indices after GridSearch
If we are interested in the best k feature indices via SequentialFeatureSelection.k_feature_idx_
, we have to initialize a GridSearchCV
object with refit=True
. Now, the grid search object will take the complete training dataset and the best parameters, which it found via crossvalidation, to train the estimator pipeline.
gs = GridSearchCV(estimator=pipe,
param_grid=param_grid,
scoring='accuracy',
n_jobs=1,
cv=5,
refit=True)
gs = gs.fit(X_train, y_train)
After running the grid search, we can access the individual pipeline objects of the best_estimator_
via the steps
attribute.
gs.best_estimator_.steps
[('sfs', SequentialFeatureSelector(clone_estimator=True, cv=5,
estimator=KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',
metric_params=None, n_jobs=1, n_neighbors=1, p=2,
weights='uniform'),
floating=False, forward=True, k_features=3, n_jobs=1,
pre_dispatch='2*n_jobs', scoring='accuracy', verbose=0)),
('knn',
KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',
metric_params=None, n_jobs=1, n_neighbors=2, p=2,
weights='uniform'))]
Via subindexing, we can then obtain the bestselected feature subset:
print('Best features:', gs.best_estimator_.steps[0][1].k_feature_idx_)
Best features: (0, 1, 3)
During crossvalidation, this feature combination had a CV accuracy of:
print('Best score:', gs.best_score_)
Best score: 0.94
gs.best_params_
{'sfs__estimator__n_neighbors': 1, 'sfs__k_features': 3}
Alternatively, if we can set the "best grid search parameters" in our pipeline manually if we ran GridSearchCV
with refit=False
. It should yield the same results:
pipe.set_params(**gs.best_params_).fit(X_train, y_train)
print('Best features:', pipe.steps[0][1].k_feature_idx_)
Best features: (0, 1, 3)
Example 8  Selecting the "best" feature combination in a krange
If k_features
is set to to a tuple (min_k, max_k)
(new in 0.4.2), the SFS will now select the best feature combination that it discovered by iterating from k=1
to max_k
(forward), or max_k
to min_k
(backward). The size of the returned feature subset is then within max_k
to min_k
, depending on which combination scored best during cross validation.
X.shape
(150, 4)
from mlxtend.feature_selection import SequentialFeatureSelector as SFS
from sklearn.neighbors import KNeighborsClassifier
from mlxtend.data import wine_data
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
X, y = wine_data()
X_train, X_test, y_train, y_test= train_test_split(X, y,
stratify=y,
test_size=0.3,
random_state=1)
knn = KNeighborsClassifier(n_neighbors=2)
sfs1 = SFS(estimator=knn,
k_features=(3, 10),
forward=True,
floating=False,
scoring='accuracy',
cv=5)
pipe = make_pipeline(StandardScaler(), sfs1)
pipe.fit(X_train, y_train)
print('best combination (ACC: %.3f): %s\n' % (sfs1.k_score_, sfs1.k_feature_idx_))
print('all subsets:\n', sfs1.subsets_)
plot_sfs(sfs1.get_metric_dict(), kind='std_err');
best combination (ACC: 0.992): (0, 1, 2, 3, 6, 8, 9, 10, 11, 12)
all subsets:
{1: {'feature_idx': (6,), 'cv_scores': array([ 0.84615385, 0.6 , 0.88 , 0.79166667, 0.875 ]), 'avg_score': 0.7985641025641026}, 2: {'feature_idx': (6, 9), 'cv_scores': array([ 0.92307692, 0.88 , 1. , 0.95833333, 0.91666667]), 'avg_score': 0.93561538461538463}, 3: {'feature_idx': (6, 9, 12), 'cv_scores': array([ 0.92307692, 0.92 , 0.96 , 1. , 0.95833333]), 'avg_score': 0.95228205128205123}, 4: {'feature_idx': (3, 6, 9, 12), 'cv_scores': array([ 0.96153846, 0.96 , 0.96 , 1. , 0.95833333]), 'avg_score': 0.96797435897435891}, 5: {'feature_idx': (3, 6, 9, 10, 12), 'cv_scores': array([ 0.92307692, 0.96 , 1. , 1. , 1. ]), 'avg_score': 0.97661538461538466}, 6: {'feature_idx': (2, 3, 6, 9, 10, 12), 'cv_scores': array([ 0.92307692, 0.96 , 1. , 0.95833333, 1. ]), 'avg_score': 0.96828205128205125}, 7: {'feature_idx': (0, 2, 3, 6, 9, 10, 12), 'cv_scores': array([ 0.92307692, 0.92 , 1. , 1. , 1. ]), 'avg_score': 0.96861538461538466}, 8: {'feature_idx': (0, 2, 3, 6, 8, 9, 10, 12), 'cv_scores': array([ 1. , 0.92, 1. , 1. , 1. ]), 'avg_score': 0.98399999999999999}, 9: {'feature_idx': (0, 2, 3, 6, 8, 9, 10, 11, 12), 'cv_scores': array([ 1. , 0.92, 1. , 1. , 1. ]), 'avg_score': 0.98399999999999999}, 10: {'feature_idx': (0, 1, 2, 3, 6, 8, 9, 10, 11, 12), 'cv_scores': array([ 1. , 0.96, 1. , 1. , 1. ]), 'avg_score': 0.99199999999999999}}
API
SequentialFeatureSelector(estimator, k_features=1, forward=True, floating=False, verbose=0, scoring=None, cv=5, n_jobs=1, pre_dispatch='2n_jobs', clone_estimator=True)*
Sequential Feature Selection for Classification and Regression.
Parameters

estimator
: scikitlearn classifier or regressor 
k_features
: int or tuple or str (default: 1)Number of features to select, where k_features < the full feature set. New in 0.4.2: A tuple containing a min and max value can be provided, and the SFS will consider return any feature combination between min and max that scored highest in crossvalidtion. For example, the tuple (1, 4) will return any combination from 1 up to 4 features instead of a fixed number of features k. New in 0.8.0: A string argument "best" or "parsimonious". If "best" is provided, the feature selector will return the feature subset with the best crossvalidation performance. If "parsimonious" is provided as an argument, the smallest feature subset that is within one standard error of the crossvalidation performance will be selected.

forward
: bool (default: True)Forward selection if True, backward selection otherwise

floating
: bool (default: False)Adds a conditional exclusion/inclusion if True.

verbose
: int (default: 0), level of verbosity to use in logging.If 0, no output, if 1 number of features in current set, if 2 detailed logging i ncluding timestamp and cv scores at step.

scoring
: str, callable, or None (default: None)If None (default), uses 'accuracy' for sklearn classifiers and 'r2' for sklearn regressors. If str, uses a sklearn scoring metric string identifier, for example {accuracy, f1, precision, recall, roc_auc} for classifiers, {'mean_absolute_error', 'mean_squared_error'/'neg_mean_squared_error', 'median_absolute_error', 'r2'} for regressors. If a callable object or function is provided, it has to be conform with sklearn's signature
scorer(estimator, X, y)
; see http://scikitlearn.org/stable/modules/generated/sklearn.metrics.make_scorer.html for more information. 
cv
: int (default: 5)Scikitlearn crossvalidation generator or
int
. If estimator is a classifier (or y consists of integer class labels), stratified kfold is performed, and regular kfold crossvalidation otherwise. No crossvalidation if cv is None, False, or 0. 
n_jobs
: int (default: 1)The number of CPUs to use for evaluating different feature subsets in parallel. 1 means 'all CPUs'.

pre_dispatch
: int, or string (default: '2*n_jobs')Controls the number of jobs that get dispatched during parallel execution if
n_jobs > 1
orn_jobs=1
. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be: None, in which case all the jobs are immediately created and spawned. Use this for lightweight and fastrunning jobs, to avoid delays due to ondemand spawning of the jobs An int, giving the exact number of total jobs that are spawned A string, giving an expression as a function of n_jobs, as in2*n_jobs

clone_estimator
: bool (default: True)Clones estimator if True; works with the original estimator instance if False. Set to False if the estimator doesn't implement scikitlearn's set_params and get_params methods. In addition, it is required to set cv=0, and n_jobs=1.
Attributes

k_feature_idx_
: arraylike, shape = [n_predictions]Feature Indices of the selected feature subsets.

k_score_
: floatCross validation average score of the selected subset.

subsets_
: dictA dictionary of selected feature subsets during the sequential selection, where the dictionary keys are the lengths k of these feature subsets. The dictionary values are dictionaries themselves with the following keys: 'feature_idx' (tuple of indices of the feature subset) 'cv_scores' (list individual crossvalidation scores) 'avg_score' (average crossvalidation score)
Methods
fit(X, y)
Perform feature selection and learn model from training data.
Parameters

X
: {arraylike, sparse matrix}, shape = [n_samples, n_features]Training vectors, where n_samples is the number of samples and n_features is the number of features.

y
: arraylike, shape = [n_samples]Target values.
Returns
self
: object
fit_transform(X, y)
Fit to training data then reduce X to its most important features.
Parameters

X
: {arraylike, sparse matrix}, shape = [n_samples, n_features]Training vectors, where n_samples is the number of samples and n_features is the number of features.
Returns
Reduced feature subset of X, shape={n_samples, k_features}
get_metric_dict(confidence_interval=0.95)
Return metric dictionary
Parameters

confidence_interval
: float (default: 0.95)A positive float between 0.0 and 1.0 to compute the confidence interval bounds of the CV score averages.
Returns
Dictionary with items where each dictionary value is a list with the number of iterations (number of feature subsets) as its length. The dictionary keys corresponding to these lists are as follows: 'feature_idx': tuple of the indices of the feature subset 'cv_scores': list with individual CV scores 'avg_score': of CV average scores 'std_dev': standard deviation of the CV score average 'std_err': standard error of the CV score average 'ci_bound': confidence interval bound of the CV score average
get_params(deep=True)
Get parameters for this estimator.
Parameters

deep
: boolean, optionalIf True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns

params
: mapping of string to anyParameter names mapped to their values.
set_params(params)
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects
(such as pipelines). The latter have parameters of the form
<component>__<parameter>
so that it's possible to update each
component of a nested object.
Returns
self
transform(X)
Reduce X to its most important features.
Parameters

X
: {arraylike, sparse matrix}, shape = [n_samples, n_features]Training vectors, where n_samples is the number of samples and n_features is the number of features.
Returns
Reduced feature subset of X, shape={n_samples, k_features}