Deep Feature Synthesis#
Deep Feature Synthesis (DFS) is an automated method for performing feature engineering on relational and temporal data.
Input Data#
Deep Feature Synthesis requires structured datasets in order to perform feature engineering. To demonstrate the capabilities of DFS, we will use a mock customer transactions dataset.
Note
Before using DFS, it is recommended that you prepare your data as an EntitySet. See Representing Data with EntitySets to learn how.
[1]:
import featuretools as ft
es = ft.demo.load_mock_customer(return_entityset=True)
es
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/woodwork/type_sys/utils.py:33: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
pd.to_datetime(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/woodwork/type_sys/utils.py:33: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
pd.to_datetime(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/woodwork/type_sys/utils.py:33: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
pd.to_datetime(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/woodwork/type_sys/utils.py:33: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
pd.to_datetime(
[1]:
Entityset: transactions
DataFrames:
transactions [Rows: 500, Columns: 6]
products [Rows: 5, Columns: 3]
sessions [Rows: 35, Columns: 5]
customers [Rows: 5, Columns: 5]
Relationships:
transactions.product_id -> products.product_id
transactions.session_id -> sessions.session_id
sessions.customer_id -> customers.customer_id
Once data is prepared as an .EntitySet, we are ready to automatically generate features for a target dataframe - e.g. customers.
Running DFS#
Typically, without automated feature engineering, a data scientist would write code to aggregate data for a customer, and apply different statistical functions resulting in features quantifying the customer’s behavior. In this example, an expert might be interested in features such as: total number of sessions or month the customer signed up.
These features can be generated by DFS when we specify the target_dataframe as customers and "count" and "month" as primitives.
[2]:
feature_matrix, feature_defs = ft.dfs(
entityset=es,
target_dataframe_name="customers",
agg_primitives=["count"],
trans_primitives=["month"],
max_depth=1,
)
feature_matrix
[2]:
| zip_code | COUNT(sessions) | MONTH(birthday) | MONTH(join_date) | |
|---|---|---|---|---|
| customer_id | ||||
| 5 | 60091 | 6 | 7 | 7 |
| 4 | 60091 | 8 | 8 | 4 |
| 1 | 60091 | 8 | 7 | 4 |
| 3 | 13244 | 6 | 11 | 8 |
| 2 | 13244 | 7 | 8 | 4 |
In the example above, "count" is an aggregation primitive because it computes a single value based on many sessions related to one customer. "month" is called a transform primitive because it takes one value for a customer transforms it to another.
Note
Feature primitives are a fundamental component to Featuretools. To learn more read Feature primitives.
Creating “Deep Features”#
The name Deep Feature Synthesis comes from the algorithm’s ability to stack primitives to generate more complex features. Each time we stack a primitive we increase the “depth” of a feature. The max_depth parameter controls the maximum depth of the features returned by DFS. Let us try running DFS with max_depth=2
[3]:
feature_matrix, feature_defs = ft.dfs(
entityset=es,
target_dataframe_name="customers",
agg_primitives=["mean", "sum", "mode"],
trans_primitives=["month", "hour"],
max_depth=2,
)
feature_matrix
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function mean at 0x7f2e223639d0> is currently using SeriesGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function sum at 0x7f2e2235ea60> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function mean at 0x7f2e223639d0> is currently using SeriesGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function sum at 0x7f2e2235ea60> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function sum at 0x7f2e2235ea60> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function mean at 0x7f2e223639d0> is currently using SeriesGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
to_merge = base_frame.groupby(
[3]:
| zip_code | MODE(sessions.device) | MEAN(transactions.amount) | MODE(transactions.product_id) | SUM(transactions.amount) | HOUR(birthday) | HOUR(join_date) | MONTH(birthday) | MONTH(join_date) | MEAN(sessions.MEAN(transactions.amount)) | MEAN(sessions.SUM(transactions.amount)) | MODE(sessions.HOUR(session_start)) | MODE(sessions.MODE(transactions.product_id)) | MODE(sessions.MONTH(session_start)) | SUM(sessions.MEAN(transactions.amount)) | MODE(transactions.sessions.device) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| customer_id | ||||||||||||||||
| 5 | 60091 | mobile | 80.375443 | 5 | 6349.66 | 0 | 5 | 7 | 7 | 78.705187 | 1058.276667 | 0 | 3 | 1 | 472.231119 | mobile |
| 4 | 60091 | mobile | 80.070459 | 2 | 8727.68 | 0 | 20 | 8 | 4 | 81.207189 | 1090.960000 | 1 | 1 | 1 | 649.657515 | mobile |
| 1 | 60091 | mobile | 71.631905 | 4 | 9025.62 | 0 | 10 | 7 | 4 | 72.774140 | 1128.202500 | 6 | 4 | 1 | 582.193117 | mobile |
| 3 | 13244 | desktop | 67.060430 | 1 | 6236.62 | 0 | 15 | 11 | 8 | 67.539577 | 1039.436667 | 5 | 1 | 1 | 405.237462 | desktop |
| 2 | 13244 | desktop | 77.422366 | 4 | 7200.28 | 0 | 23 | 8 | 4 | 78.415122 | 1028.611429 | 3 | 3 | 1 | 548.905851 | desktop |
With a depth of 2, a number of features are generated using the supplied primitives. The algorithm to synthesize these definitions is described in this paper. In the returned feature matrix, let us understand one of the depth 2 features
[4]:
feature_matrix[["MEAN(sessions.SUM(transactions.amount))"]]
[4]:
| MEAN(sessions.SUM(transactions.amount)) | |
|---|---|
| customer_id | |
| 5 | 1058.276667 |
| 4 | 1090.960000 |
| 1 | 1128.202500 |
| 3 | 1039.436667 |
| 2 | 1028.611429 |
For each customer this feature
calculates the
sumof all transaction amounts per session to get total amount per session,then applies the
meanto the total amounts across multiple sessions to identify the average amount spent per session
We call this feature a “deep feature” with a depth of 2.
Let’s look at another depth 2 feature that calculates for every customer the most common hour of the day when they start a session
[5]:
feature_matrix[["MODE(sessions.HOUR(session_start))"]]
[5]:
| MODE(sessions.HOUR(session_start)) | |
|---|---|
| customer_id | |
| 5 | 0 |
| 4 | 1 |
| 1 | 6 |
| 3 | 5 |
| 2 | 3 |
For each customer this feature calculates
The
hourof the day each of his or her sessions started, thenuses the statistical function
modeto identify the most common hour he or she started a session
Stacking results in features that are more expressive than individual primitives themselves. This enables the automatic creation of complex patterns for machine learning.
Note
You can graphically visualize the lineage of a feature by calling featuretools.graph_feature() on it. You can also generate an English description of the feature with featuretools.describe_feature(). See Generating Feature Descriptions for more details.
Changing Target DataFrame#
DFS is powerful because we can create a feature matrix for any dataframe in our dataset. If we switch our target dataframe to “sessions”, we can synthesize features for each session instead of each customer. Now, we can use these features to predict the outcome of a session.
[6]:
feature_matrix, feature_defs = ft.dfs(
entityset=es,
target_dataframe_name="sessions",
agg_primitives=["mean", "sum", "mode"],
trans_primitives=["month", "hour"],
max_depth=2,
)
feature_matrix.head(5)
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function sum at 0x7f2e2235ea60> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function mean at 0x7f2e223639d0> is currently using SeriesGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function sum at 0x7f2e2235ea60> is currently using SeriesGroupBy.sum. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "sum" instead.
to_merge = base_frame.groupby(
/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-featuretools/envs/stable/lib/python3.9/site-packages/featuretools/computational_backends/feature_set_calculator.py:781: FutureWarning: The provided callable <function mean at 0x7f2e223639d0> is currently using SeriesGroupBy.mean. In a future version of pandas, the provided callable will be used directly. To keep current behavior pass the string "mean" instead.
to_merge = base_frame.groupby(
[6]:
| customer_id | device | MEAN(transactions.amount) | MODE(transactions.product_id) | SUM(transactions.amount) | HOUR(session_start) | MONTH(session_start) | customers.zip_code | MODE(transactions.HOUR(transaction_time)) | MODE(transactions.MONTH(transaction_time)) | customers.MODE(sessions.device) | customers.MEAN(transactions.amount) | customers.MODE(transactions.product_id) | customers.SUM(transactions.amount) | customers.HOUR(birthday) | customers.HOUR(join_date) | customers.MONTH(birthday) | customers.MONTH(join_date) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| session_id | ||||||||||||||||||
| 1 | 2 | desktop | 76.813125 | 3 | 1229.01 | 0 | 1 | 13244 | 0 | 1 | desktop | 77.422366 | 4 | 7200.28 | 0 | 23 | 8 | 4 |
| 2 | 5 | mobile | 74.696000 | 5 | 746.96 | 0 | 1 | 60091 | 0 | 1 | mobile | 80.375443 | 5 | 6349.66 | 0 | 5 | 7 | 7 |
| 3 | 4 | mobile | 88.600000 | 1 | 1329.00 | 0 | 1 | 60091 | 0 | 1 | mobile | 80.070459 | 2 | 8727.68 | 0 | 20 | 8 | 4 |
| 4 | 1 | mobile | 64.557200 | 5 | 1613.93 | 0 | 1 | 60091 | 0 | 1 | mobile | 71.631905 | 4 | 9025.62 | 0 | 10 | 7 | 4 |
| 5 | 4 | mobile | 70.638182 | 5 | 777.02 | 1 | 1 | 60091 | 1 | 1 | mobile | 80.070459 | 2 | 8727.68 | 0 | 20 | 8 | 4 |
As we can see, DFS will also build deep features based on a parent dataframe, in this case the customer of a particular session. For example, the feature below calculates the mean transaction amount of the customer of the session.
[7]:
feature_matrix[["customers.MEAN(transactions.amount)"]].head(5)
[7]:
| customers.MEAN(transactions.amount) | |
|---|---|
| session_id | |
| 1 | 77.422366 |
| 2 | 80.375443 |
| 3 | 80.070459 |
| 4 | 71.631905 |
| 5 | 80.070459 |
Improve feature output#
To learn about the parameters to change in DFS read Tuning Deep Feature Synthesis.