Feature Selection¶

Featuretools provides users with the ability to remove features that are unlikely to be useful in building an effective machine learning model. Reducing the number of features in the feature matrix can both produce better results in the model as well as reduce the computational cost involved in prediction.

Featuretools enables users to perform feature selection on the results of Deep Feature Synthesis with three functions:

ft.selection.remove_highly_null_features
ft.selection.remove_single_value_features
ft.selection.remove_highly_correlated_features

We will describe each of these functions in depth, but first we must create an entity set with which we can run ft.dfs.

[1]:

import pandas as pd
import featuretools as ft

from featuretools.selection import (
    remove_highly_correlated_features,
    remove_highly_null_features,
    remove_single_value_features,
)

from featuretools.primitives import NaturalLanguage
from featuretools.demo.flight import load_flight

es = load_flight(nrows=50)
es

Downloading data ...

[1]:

Entityset: Flight Data
  DataFrames:
    trip_logs [Rows: 50, Columns: 21]
    flights [Rows: 6, Columns: 9]
    airlines [Rows: 1, Columns: 1]
    airports [Rows: 4, Columns: 3]
  Relationships:
    trip_logs.flight_id -> flights.flight_id
    flights.carrier -> airlines.carrier
    flights.dest -> airports.dest

Remove Highly Null Features¶

We might have a dataset with columns that have many null values. Deep Feature Synthesis might build features off of those null columns, creating even more highly null features. In this case, we might want to remove any features whose null values pass a certain threshold. Below is our feature matrix with such a case:

[2]:

fm, features = ft.dfs(entityset=es,
                          target_dataframe_name="trip_logs",
                          cutoff_time=pd.DataFrame({
                              'trip_log_id':[30, 1, 2, 3, 4],
                              'time':pd.to_datetime(['2016-09-22 00:00:00']*5)
                                                  }),
                          trans_primitives=[],
                          agg_primitives=[],
                          max_depth=2)
fm

[2]:

	flight_id	dep_delay	taxi_out	taxi_in	arr_delay	diverted	air_time	distance	carrier_delay	weather_delay	national_airspace_delay	security_delay	late_aircraft_delay	canceled	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	NaN	NaN	NaN	NaN	<NA>	NaN	600.0	NaN	NaN	NaN	NaN	NaN	<NA>	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
1	AA-494:CLT->PHX	NaN	NaN	NaN	NaN	<NA>	NaN	1773.0	NaN	NaN	NaN	NaN	NaN	<NA>	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
2	AA-494:CLT->PHX	NaN	NaN	NaN	NaN	<NA>	NaN	1773.0	NaN	NaN	NaN	NaN	NaN	<NA>	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
3	AA-494:CLT->PHX	NaN	NaN	NaN	NaN	<NA>	NaN	1773.0	NaN	NaN	NaN	NaN	NaN	<NA>	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
4	NaN	NaN	NaN	NaN	NaN	<NA>	NaN	NaN	NaN	NaN	NaN	NaN	NaN	<NA>	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

We look at the above feature matrix and decide to remove the highly null features

[3]:

ft.selection.remove_highly_null_features(fm)

[3]:

	flight_id	distance	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	600.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
1	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
2	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
3	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
4	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

Notice that calling remove_highly_null_features didn’t remove every feature that contains a null value. By default, we only remove features where the percentage of null values in the calculated feature matrix is above 95%. If we want to lower that threshold, we can set the pct_null_threshold paramter ourselves.

[4]:

remove_highly_null_features(fm, pct_null_threshold=.2)

[4]:


trip_log_id
30
1
2
3
4

Remove Single Value Features¶

Another situation we might run into is one where our calculated features don’t have any variance. In those cases, we are likely to want to remove the uninteresting features. For that, we use remove_single_value_features.

Let’s see what happens when we remove the single value features of the feature matrix below.

[5]:

fm

[5]:

	flight_id	dep_delay	taxi_out	taxi_in	arr_delay	diverted	air_time	distance	carrier_delay	weather_delay	national_airspace_delay	security_delay	late_aircraft_delay	canceled	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	NaN	NaN	NaN	NaN	<NA>	NaN	600.0	NaN	NaN	NaN	NaN	NaN	<NA>	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
1	AA-494:CLT->PHX	NaN	NaN	NaN	NaN	<NA>	NaN	1773.0	NaN	NaN	NaN	NaN	NaN	<NA>	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
2	AA-494:CLT->PHX	NaN	NaN	NaN	NaN	<NA>	NaN	1773.0	NaN	NaN	NaN	NaN	NaN	<NA>	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
3	AA-494:CLT->PHX	NaN	NaN	NaN	NaN	<NA>	NaN	1773.0	NaN	NaN	NaN	NaN	NaN	<NA>	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
4	NaN	NaN	NaN	NaN	NaN	<NA>	NaN	NaN	NaN	NaN	NaN	NaN	NaN	<NA>	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

Note

A list of feature definitions such as those created by dfs can be provided to the feature selection functions. Doing this will change the outputs to include an updated list of feature definitions.

[6]:

new_fm, new_features = remove_single_value_features(fm, features=features)
new_fm

[6]:

	flight_id	distance	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	600.0	RSW	Fort Myers, FL	FL	CLT	3	Charlotte, NC	NC
1	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	Phoenix, AZ	AZ
2	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	Phoenix, AZ	AZ
3	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	Phoenix, AZ	AZ
4	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

Now that we have the features definitions for the updated feature matrix, we can see that the features that were removed are:

[7]:

set(features) - set(new_features)

[7]:

{<Feature: air_time>,
 <Feature: arr_delay>,
 <Feature: canceled>,
 <Feature: carrier_delay>,
 <Feature: dep_delay>,
 <Feature: diverted>,
 <Feature: flights.carrier>,
 <Feature: flights.flight_num>,
 <Feature: late_aircraft_delay>,
 <Feature: national_airspace_delay>,
 <Feature: security_delay>,
 <Feature: taxi_in>,
 <Feature: taxi_out>,
 <Feature: weather_delay>}

With the function used as it is above, null values are not considered when counting a feature’s unique values. If we’d like to consider NaN its own value, we can set count_nan_as_value to True and we’ll see flights.carrier and flights.flight_num back in the matrix.

[8]:

new_fm, new_features = remove_single_value_features(fm, features=features, count_nan_as_value=True)
new_fm

[8]:

	flight_id	distance	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	600.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
1	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
2	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
3	AA-494:CLT->PHX	1773.0	CLT	Charlotte, NC	NC	PHX	8	AA	494	Phoenix, AZ	AZ
4	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

The features that were removed are:

[9]:

set(features) - set(new_features)

[9]:

{<Feature: air_time>,
 <Feature: arr_delay>,
 <Feature: canceled>,
 <Feature: carrier_delay>,
 <Feature: dep_delay>,
 <Feature: diverted>,
 <Feature: late_aircraft_delay>,
 <Feature: national_airspace_delay>,
 <Feature: security_delay>,
 <Feature: taxi_in>,
 <Feature: taxi_out>,
 <Feature: weather_delay>}

Remove Highly Correlated Features¶

The last feature selection function we have allows us to remove features that would likely be redundant to the model we’re attempting to build by considering the correlation between pairs of calculated features.

When two features are determined to be highly correlated, we remove the more complex of the two. For example, say we have two features: col and -(col).

We can see that -(col) is just the negation of col, and so we can guess those features are going to be highly correlated. -(col) has has the Negate primitive applied to it, so it is more complex than the identity feature col. Therefore, if we only want one of col and -(col), we should keep the identity feature. For features that don’t have an obvious difference in complexity, we discard the feature that comes later in the feature matrix.

Let’s try this out on our data:

[10]:

fm, features = ft.dfs(entityset=es,
                          target_dataframe_name="trip_logs",
                          trans_primitives=['negate'],
                          agg_primitives=[],
                          max_depth=3)
fm.head()

[10]:

	flight_id	dep_delay	taxi_out	taxi_in	arr_delay	diverted	air_time	distance	carrier_delay	weather_delay	national_airspace_delay	security_delay	late_aircraft_delay	canceled	-(air_time)	-(arr_delay)	-(carrier_delay)	-(dep_delay)	-(distance)	-(late_aircraft_delay)	-(national_airspace_delay)	-(security_delay)	-(taxi_in)	-(taxi_out)	-(weather_delay)	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	-11.0	12.0	10.0	-12.0	False	88.0	600.0	0.0	0.0	0.0	0.0	0.0	False	-88.0	12.0	-0.0	11.0	-600.0	-0.0	-0.0	-0.0	-10.0	-12.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
38	AA-495:ATL->PHX	-6.0	28.0	5.0	1.0	False	224.0	1587.0	0.0	0.0	0.0	0.0	0.0	False	-224.0	-1.0	-0.0	6.0	-1587.0	-0.0	-0.0	-0.0	-5.0	-28.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ
46	AA-495:CLT->ATL	-2.0	18.0	8.0	-3.0	False	50.0	226.0	0.0	0.0	0.0	0.0	0.0	False	-50.0	3.0	-0.0	2.0	-226.0	-0.0	-0.0	-0.0	-8.0	-18.0	-0.0	CLT	Charlotte, NC	NC	ATL	1	AA	495	Atlanta, GA	GA
31	AA-494:RSW->CLT	0.0	11.0	10.0	-3.0	False	87.0	600.0	0.0	0.0	0.0	0.0	0.0	False	-87.0	3.0	-0.0	-0.0	-600.0	-0.0	-0.0	-0.0	-10.0	-11.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
39	AA-495:ATL->PHX	-4.0	26.0	3.0	10.0	False	235.0	1587.0	0.0	0.0	0.0	0.0	0.0	False	-235.0	-10.0	-0.0	4.0	-1587.0	-0.0	-0.0	-0.0	-3.0	-26.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ

Note that we have some pretty clear correlations here between all the features and their negations.

Now, using remove_highly_correlated_features, our default threshold for correlation is 95% correlated, and we get all of the obviously correlated features removed, leaving just the less complex features.

[11]:

new_fm, new_features = remove_highly_correlated_features(fm, features=features)
new_fm.head()

[11]:

	flight_id	dep_delay	taxi_out	taxi_in	arr_delay	diverted	air_time	carrier_delay	weather_delay	national_airspace_delay	security_delay	late_aircraft_delay	canceled	-(security_delay)	-(weather_delay)	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	-11.0	12.0	10.0	-12.0	False	88.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
38	AA-495:ATL->PHX	-6.0	28.0	5.0	1.0	False	224.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ
46	AA-495:CLT->ATL	-2.0	18.0	8.0	-3.0	False	50.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	CLT	Charlotte, NC	NC	ATL	1	AA	495	Atlanta, GA	GA
31	AA-494:RSW->CLT	0.0	11.0	10.0	-3.0	False	87.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
39	AA-495:ATL->PHX	-4.0	26.0	3.0	10.0	False	235.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ

The features that were removed are:

[12]:

set(features) - set(new_features)

[12]:

{<Feature: -(air_time)>,
 <Feature: -(distance)>,
 <Feature: distance>,
 <Feature: -(taxi_out)>,
 <Feature: -(taxi_in)>,
 <Feature: -(late_aircraft_delay)>,
 <Feature: -(national_airspace_delay)>,
 <Feature: -(carrier_delay)>,
 <Feature: -(dep_delay)>,
 <Feature: -(arr_delay)>}

Change the correlation threshold¶

We can lower the threshold at which to remove correlated features if we’d like to be more restrictive by using the pct_corr_threshold parameter.

[13]:

new_fm , new_features = remove_highly_correlated_features(fm, features=features, pct_corr_threshold=.9)
new_fm.head()

[13]:

	flight_id	dep_delay	taxi_out	taxi_in	arr_delay	diverted	air_time	carrier_delay	weather_delay	security_delay	late_aircraft_delay	canceled	-(security_delay)	-(weather_delay)	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	-11.0	12.0	10.0	-12.0	False	88.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
38	AA-495:ATL->PHX	-6.0	28.0	5.0	1.0	False	224.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ
46	AA-495:CLT->ATL	-2.0	18.0	8.0	-3.0	False	50.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	CLT	Charlotte, NC	NC	ATL	1	AA	495	Atlanta, GA	GA
31	AA-494:RSW->CLT	0.0	11.0	10.0	-3.0	False	87.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
39	AA-495:ATL->PHX	-4.0	26.0	3.0	10.0	False	235.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ

The features that were removed are:

[14]:

set(features) - set(new_features)

[14]:

{<Feature: -(air_time)>,
 <Feature: -(distance)>,
 <Feature: national_airspace_delay>,
 <Feature: distance>,
 <Feature: -(taxi_out)>,
 <Feature: -(taxi_in)>,
 <Feature: -(late_aircraft_delay)>,
 <Feature: -(national_airspace_delay)>,
 <Feature: -(carrier_delay)>,
 <Feature: -(dep_delay)>,
 <Feature: -(arr_delay)>}

Check a Subset of Features¶

If we only want to check a subset of features, we can set features_to_check to the list of features whose correlation we’d like to check, and no features outside of that list will be removed.

[15]:

new_fm, new_features = remove_highly_correlated_features(fm, features=features, features_to_check=['air_time', 'distance', 'flights.distance_group'])
new_fm.head()

[15]:

	flight_id	dep_delay	taxi_out	taxi_in	arr_delay	diverted	air_time	carrier_delay	weather_delay	national_airspace_delay	security_delay	late_aircraft_delay	canceled	-(air_time)	-(arr_delay)	-(carrier_delay)	-(dep_delay)	-(distance)	-(late_aircraft_delay)	-(national_airspace_delay)	-(security_delay)	-(taxi_in)	-(taxi_out)	-(weather_delay)	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	-11.0	12.0	10.0	-12.0	False	88.0	0.0	0.0	0.0	0.0	0.0	False	-88.0	12.0	-0.0	11.0	-600.0	-0.0	-0.0	-0.0	-10.0	-12.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
38	AA-495:ATL->PHX	-6.0	28.0	5.0	1.0	False	224.0	0.0	0.0	0.0	0.0	0.0	False	-224.0	-1.0	-0.0	6.0	-1587.0	-0.0	-0.0	-0.0	-5.0	-28.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ
46	AA-495:CLT->ATL	-2.0	18.0	8.0	-3.0	False	50.0	0.0	0.0	0.0	0.0	0.0	False	-50.0	3.0	-0.0	2.0	-226.0	-0.0	-0.0	-0.0	-8.0	-18.0	-0.0	CLT	Charlotte, NC	NC	ATL	1	AA	495	Atlanta, GA	GA
31	AA-494:RSW->CLT	0.0	11.0	10.0	-3.0	False	87.0	0.0	0.0	0.0	0.0	0.0	False	-87.0	3.0	-0.0	-0.0	-600.0	-0.0	-0.0	-0.0	-10.0	-11.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
39	AA-495:ATL->PHX	-4.0	26.0	3.0	10.0	False	235.0	0.0	0.0	0.0	0.0	0.0	False	-235.0	-10.0	-0.0	4.0	-1587.0	-0.0	-0.0	-0.0	-3.0	-26.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ

The features that were removed are:

[16]:

set(features) - set(new_features)

[16]:

{<Feature: distance>}

Protect Features from Removal¶

To protect specific features from being removed from the feature matrix, we can include a list of features_to_keep, and these features will not be removed

[17]:

new_fm, new_features = remove_highly_correlated_features(fm, features=features, features_to_keep=['air_time', 'distance', 'flights.distance_group'])
new_fm.head()

[17]:

	flight_id	dep_delay	taxi_out	taxi_in	arr_delay	diverted	air_time	distance	carrier_delay	weather_delay	national_airspace_delay	security_delay	late_aircraft_delay	canceled	-(security_delay)	-(weather_delay)	flights.origin	flights.origin_city	flights.origin_state	flights.dest	flights.distance_group	flights.carrier	flights.flight_num	flights.airports.dest_city	flights.airports.dest_state
trip_log_id
30	AA-494:RSW->CLT	-11.0	12.0	10.0	-12.0	False	88.0	600.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
38	AA-495:ATL->PHX	-6.0	28.0	5.0	1.0	False	224.0	1587.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ
46	AA-495:CLT->ATL	-2.0	18.0	8.0	-3.0	False	50.0	226.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	CLT	Charlotte, NC	NC	ATL	1	AA	495	Atlanta, GA	GA
31	AA-494:RSW->CLT	0.0	11.0	10.0	-3.0	False	87.0	600.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	RSW	Fort Myers, FL	FL	CLT	3	AA	494	Charlotte, NC	NC
39	AA-495:ATL->PHX	-4.0	26.0	3.0	10.0	False	235.0	1587.0	0.0	0.0	0.0	0.0	0.0	False	-0.0	-0.0	ATL	Atlanta, GA	GA	PHX	7	AA	495	Phoenix, AZ	AZ

The features that were removed are:

[18]:

set(features) - set(new_features)

[18]:

{<Feature: -(air_time)>,
 <Feature: -(distance)>,
 <Feature: -(taxi_out)>,
 <Feature: -(taxi_in)>,
 <Feature: -(late_aircraft_delay)>,
 <Feature: -(national_airspace_delay)>,
 <Feature: -(carrier_delay)>,
 <Feature: -(dep_delay)>,
 <Feature: -(arr_delay)>}

Generating Feature Descriptions

Resources

Table of Contents

Previous topic

Next topic

This Page

Feature Selection¶

Remove Highly Null Features¶

Remove Single Value Features¶

Remove Highly Correlated Features¶

Change the correlation threshold¶

Check a Subset of Features¶

Protect Features from Removal¶

Table of Contents

Previous topic

Next topic

This Page

Quick search

Feature Selection¶

Remove Highly Null Features¶

Remove Single Value Features¶

Remove Highly Correlated Features¶

Change the correlation threshold¶

Check a Subset of Features¶

Protect Features from Removal¶