{ "cells": [ { "cell_type": "markdown", "id": "ba92172a", "metadata": {}, "source": [ "# Specifying Primitive Options\n", "\n", "By default, DFS will apply primitives across all dataframes and columns. This behavior can be altered through a few different parameters. Dataframes and columns can be optionally ignored or included for an entire DFS run or on a per-primitive basis, enabling greater control over features and less run time overhead." ] }, { "cell_type": "code", "execution_count": null, "id": "106d36a3", "metadata": {}, "outputs": [], "source": [ "import featuretools as ft\n", "from featuretools.tests.testing_utils import make_ecommerce_entityset\n", "\n", "es = make_ecommerce_entityset()\n", "\n", "features_list = ft.dfs(\n", " entityset=es,\n", " target_dataframe_name=\"customers\",\n", " agg_primitives=[\"mode\"],\n", " trans_primitives=[\"weekday\"],\n", " features_only=True,\n", ")\n", "features_list" ] }, { "cell_type": "markdown", "id": "29ae225d", "metadata": {}, "source": [ "## Specifying Options for an Entire Run\n", "\n", "The `ignore_dataframes` and `ignore_columns` parameters of DFS control dataframes and columns that should be ignored for all primitives. This is useful for ignoring columns or dataframes that don't relate to the problem or otherwise shouldn't be included in the DFS run." ] }, { "cell_type": "code", "execution_count": null, "id": "2d481527", "metadata": {}, "outputs": [], "source": [ "# ignore the 'log' and 'cohorts' dataframes entirely\n", "# ignore the 'birthday' column in 'customers' and the 'device_name' column in 'sessions'\n", "features_list = ft.dfs(\n", " entityset=es,\n", " target_dataframe_name=\"customers\",\n", " agg_primitives=[\"mode\"],\n", " trans_primitives=[\"weekday\"],\n", " ignore_dataframes=[\"log\", \"cohorts\"],\n", " ignore_columns={\"sessions\": [\"device_name\"], \"customers\": [\"birthday\"]},\n", " features_only=True,\n", ")\n", "features_list" ] }, { "cell_type": "markdown", "id": "4a9bd7e2", "metadata": {}, "source": [ "DFS completely ignores the `log` and `cohorts` dataframes when creating features. It also ignores the columns `device_name` and `birthday` in `sessions` and `customers` respectively. However, both of these options can be overridden by individual primitive options in the `primitive_options` parameter.\n", "\n", "## Specifying for Individual Primitives\n", "Options for individual primitives or groups of primitives are set by the `primitive_options` parameter of DFS. This parameter maps any desired options to specific primitives. In the case of conflicting options, options set at this level will override options set at the entire DFS run level, and the include options will always take priority over their ignore counterparts.\n", "\n", "Using the string primitive name or the primitive type will apply the options to all primitives of the same name. You can also set options for a specific instance of a primitive by using the primitive instance as a key in the `primitive_options` dictionary. Note, however, that specifying options for a specific instance will result in that instance ignoring any options set for the generic primitive through options with the primitive name or class as the key. \n", "\n", "### Specifying Dataframes for Individual Primitives\n", "Which dataframes to include/ignore can also be specified for a single primitive or a group of primitives. Dataframes can be ignored using the `ignore_dataframes` option in `primitive_options`, while dataframes to explicitly include are set by the ``include_dataframes`` option. When ``include_dataframes`` is given, all dataframes not listed are ignored by the primitive. No columns from any excluded dataframe will be used to generate features with the given primitive." ] }, { "cell_type": "code", "execution_count": null, "id": "8bcbf11a", "metadata": {}, "outputs": [], "source": [ "# ignore the 'cohorts' and 'log' dataframes, but only for the primitive 'mode'\n", "# include only the 'customers' dataframe for the primitives 'weekday' and 'day'\n", "features_list = ft.dfs(\n", " entityset=es,\n", " target_dataframe_name=\"customers\",\n", " agg_primitives=[\"mode\"],\n", " trans_primitives=[\"weekday\", \"day\"],\n", " primitive_options={\n", " \"mode\": {\"ignore_dataframes\": [\"cohorts\", \"log\"]},\n", " (\"weekday\", \"day\"): {\"include_dataframes\": [\"customers\"]},\n", " },\n", " features_only=True,\n", ")\n", "features_list" ] }, { "cell_type": "markdown", "id": "b5cbbff0", "metadata": {}, "source": [ "In this example, DFS would only use the `customers` dataframe for both `weekday` and `day`, and would use all dataframes except `cohorts` and `log` for `mode`.\n", "\n", "### Specifying Columns for Individual Primitives\n", "\n", "Specific columns can also be explicitly included/ignored for a primitive or group of primitives. Columns to\n", "ignore is set by the `ignore_columns` option, while columns to include are set by `include_columns`. When the\n", "`include_columns` option is set, no other columns from that dataframe will be used to make features with the given primitive." ] }, { "cell_type": "code", "execution_count": null, "id": "f9e42358", "metadata": {}, "outputs": [], "source": [ "# Include the columns 'product_id' and 'zipcode', 'device_type', and 'cancel_reason' for 'mean'\n", "# Ignore the columns 'signup_date' and 'cancel_date' for 'weekday'\n", "features_list = ft.dfs(\n", " entityset=es,\n", " target_dataframe_name=\"customers\",\n", " agg_primitives=[\"mode\"],\n", " trans_primitives=[\"weekday\"],\n", " primitive_options={\n", " \"mode\": {\n", " \"include_columns\": {\n", " \"log\": [\"product_id\", \"zipcode\"],\n", " \"sessions\": [\"device_type\"],\n", " \"customers\": [\"cancel_reason\"],\n", " }\n", " },\n", " \"weekday\": {\"ignore_columns\": {\"customers\": [\"signup_date\", \"cancel_date\"]}},\n", " },\n", " features_only=True,\n", ")\n", "features_list" ] }, { "cell_type": "markdown", "id": "88ea7094", "metadata": {}, "source": [ "Here, `mode` will only use the columns `product_id` and `zipcode` from the dataframe `log`, `device_type`\n", "from the dataframe `sessions`, and `cancel_reason` from `customers`. For any other dataframe, `mode` will use all\n", "columns. The `weekday` primitive will use all columns in all dataframes except for `signup_date` and `cancel_date`\n", "from the `customers` dataframe.\n", "\n", "\n", "### Specifying GroupBy Options\n", "\n", "GroupBy Transform Primitives also have the additional options `include_groupby_dataframes`, `ignore_groupby_dataframes`, `include_groupby_columns`, and `ignore_groupby_columns`. These options are used to specify dataframes and columns to include/ignore as groupings for inputs. By default, DFS only groups by foreign key columns. Specifying `include_groupby_columns` overrides this default, and will only group by columns given. On the other hand, `ignore_groupby_columns` will continue to use only the foreign key columns, ignoring any columns specified that are also foreign key columns. Note that if including non-foreign key columns to group by, the included columns must be categorical columns. " ] }, { "cell_type": "code", "execution_count": null, "id": "1c1046b5", "metadata": {}, "outputs": [], "source": [ "features_list = ft.dfs(\n", " entityset=es,\n", " target_dataframe_name=\"log\",\n", " agg_primitives=[],\n", " trans_primitives=[],\n", " groupby_trans_primitives=[\"cum_sum\", \"cum_count\"],\n", " primitive_options={\n", " \"cum_sum\": {\"ignore_groupby_columns\": {\"log\": [\"product_id\"]}},\n", " \"cum_count\": {\n", " \"include_groupby_columns\": {\"log\": [\"product_id\", \"priority_level\"]},\n", " \"ignore_groupby_dataframes\": [\"sessions\"],\n", " },\n", " },\n", " features_only=True,\n", ")\n", "features_list" ] }, { "cell_type": "markdown", "id": "10616725", "metadata": {}, "source": [ "We ignore `product_id` as a groupby for `cum_sum` but still use any other foreign key columns in that or any other dataframe. For `cum_count`, we use only `product_id` and `priority_level` as groupbys. Note that `cum_sum` doesn't use\n", "`priority_level` because it's not a foreign key column, but we explicitly include it for `cum_count`. Finally, note that specifying groupby options doesn't affect what features the primitive is applied to. For example, `cum_count` ignores the dataframe `sessions` for groupbys, but the feature `` is still made. The groupby is from the target dataframe `log`, so the feature is valid given the associated options. To ignore the `sessions` dataframe for `cum_count`, the `ignore_dataframes` option for `cum_count` would need to include `sessions`.\n", "\n", "\n", "## Specifying for each Input for Multiple Input Primitives\n", "\n", "For primitives that take multiple columns as input, such as `Trend`, the above options can be specified for each input by passing them in as a list. If only one option dictionary is given, it is used for all inputs. The length of the list provided must match the number of inputs the primitive takes." ] }, { "cell_type": "code", "execution_count": null, "id": "2e808749", "metadata": {}, "outputs": [], "source": [ "features_list = ft.dfs(\n", " entityset=es,\n", " target_dataframe_name=\"customers\",\n", " agg_primitives=[\"trend\"],\n", " trans_primitives=[],\n", " primitive_options={\n", " \"trend\": [\n", " {\"ignore_columns\": {\"log\": [\"value_many_nans\"]}},\n", " {\"include_columns\": {\"customers\": [\"signup_date\"], \"log\": [\"datetime\"]}},\n", " ]\n", " },\n", " features_only=True,\n", ")\n", "features_list" ] }, { "cell_type": "markdown", "id": "53d5d207", "metadata": {}, "source": [ "Here, we pass in a list of primitive options for trend. We ignore the column `value_many_nans` for the first input\n", "to `trend`, and include the column `signup_date` from `customers` for the second input." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.2" } }, "nbformat": 4, "nbformat_minor": 5 }