Create or edit a connection
The connection creation and edit workflow experience brings all the dataset and connection configuration settings to the center of the screen with assistive workflow. It provides detailed dataset selection, configuration, and review experience with critical information like dataset type, size, schema, dataset id, batch status, backfill status, Person IDs, and much more to reduce the risk of wrong connection configuration. Here is an overview of the capabilities:
- You can enable a rolling data retention window when you create the connection.
- You can add to and remove datasets from a connection. (Removing a dataset removes it from the connection and impacts any associated data views and underlying Analysis Workspace projects.)
- You can enable and request backfill data per dataset.
- You can edit datasets, for example to request another backfill.
- You can import existing data per dataset.
Prerequisites
The maximum number of datasets you can add to a connection is capped at 100. The mix depends on which Customer Journey Analytics package your company has purchased.
Contact your administrator if you’re unsure which Customer Journey Analytics package you have.
Create and configure the connection create-connection
-
In Customer Journey Analytics, click the Connections tab.
-
Click Create new connection.
-
Configure the connection settings.
table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2 6-row-2 7-row-2 8-row-2 9-row-2 10-row-2 11-row-2 12-row-2 13-row-2 14-row-2 15-row-2 layout-auto Setting Description Connection name Enter a unique name for the connection. Connection description Describe the purpose of this connection. Sandbox Choose a sandbox in Experience Platform that contains the dataset/s to which you want to create a connection.
Adobe Experience Platform provides sandboxes which partition a single Platform instance into separate virtual environments to help develop and evolve digital experience applications. You can think of sandboxes as “data silos” that contain datasets. Sandboxes are used to control access to datasets.
Once you have selected the sandbox, the left rail shows all the datasets in that sandbox that you can pull from.
Enable rolling data window This checkbox, if checked, lets you define Customer Journey Analytics data retention as a rolling window in months (1 month, 3 months, 6 months, and so on), at the connection level.
Data retention is based on event dataset timestamps and applies to event datasets only. No rolling data window setting exists for profile or lookup datasets, since there are no applicable timestamps. However, if your connection includes any profile or lookup datasets (besides one or more event datasets), that data is retained for the same time period.
The main benefit is that you store or report only on data that is applicable and useful and delete older data that is no longer useful. It helps you stay under your contract limits and reduces the risk of overage cost.
If you leave the default (unchecked), the retention period is superseded by the Adobe Experience Platform data retention setting. If you have 25 months’ worth of data in Experience Platform, Customer Journey Analytics gets 25 months of data through backfill. If you deleted 10 of those months in Platform, Customer Journey Analytics would retain the remaining 15 months.
Add datasets (see below) Add datasets if no datasets appear in your dataset listing. Dataset name Select one or more datasets that you want to pull into Customer Journey Analytics and click Add.
(If you have many datasets to choose from, you can search for the right one(s) using the Search datasets search bar above the list of datasets.)
Last updated For event datasets only, this setting is automatically set to the default timestamp field from event-based schemas in Experience Platform. “N/A” means that this dataset contains no data. Schema The schema based on which the dataset was created in Adobe Experience Platform. Dataset type For each dataset that you added to this connection, Customer Journey Analytics automatically sets the dataset type based on the data coming in. There are 3 different dataset types: Event data, Profile data, and Lookup data. See the table below for an explanation of dataset types. Person ID Select a person ID from the drop-down list of available identities. These identities were defined in the dataset schema in the Experience Platform. See below for information on how to use Identity Map as a Person ID.
IMPORTANT: If there are no person IDs to choose from, that means one or more person IDs have not been defined in the schema. View this video on how to define an identity in Experience Platform.
Key For lookup datasets only (such as _id). Matching Key For lookup datasets only (such as _id). Import new data Set to On or Off. Backfill data You can request to backfill the data in a dataset based on event timestamps. For example, you can request to backfill the last 7 days worth of data, configure the right Person ID and test your connection for correct configuration. If everything looks good, you can backfill all the remaining data with ease.
In addition, you can enable the import of new data by dataset.
Backfill status Indicates whether any backfill data is processing.
Add and configure datasets add-dataset
The new workflow lets you add an Experience Platform dataset when you create a connection.
-
In the Connection settings dialog, select Add datasets.
-
In the Select datasets step, select one or more datasets and select Next. At least one event dataset must be part of the connection.
- To change the columns displayed for the list of datasets, select
- To search for a specific dataset, use the
- To toggle between showing or hiding the selected datasets, select
- To remove a dataset from the list of selected datasets, use
- To change the columns displayed for the list of datasets, select
-
Now configure the datasets one by one.
table 0-row-2 1-row-2 2-row-2 3-row-2 4-row-2 5-row-2 6-row-2 7-row-2 8-row-2 9-row-2 10-row-2 11-row-2 12-row-2 13-row-2 14-row-2 15-row-2 layout-auto Setting Description Person ID Only available for event and profile datasets. Select a person ID from the drop-down list of available identities. These identities were defined in the dataset schema in the Experience Platform. See below for information on how to use Identity Map as a Person ID.
If there are no person IDs to choose from, that means one or more person IDs have not been defined in the schema. View this video on how to define an identity in Experience Platform.
Timestamp For event datasets only, this setting is automatically set to the default timestamp field from event-based schemas in Experience Platform. Key Only available for lookup datasets. The key to use for a Lookup dataset. Matching key Only available for lookup datasets. The matching key to join on in one of the event datasets. If this list is empty you probably haven’t added or configured an event dataset. Data source type Select a type of data source.
Types of data sources include:- Web data
- Mobile App data
- POS data
- CRM data
- Survey data
- Call Center data
- Product data
- Accounts data
- Transaction data
- Customer Feedback data
- Other
This field is used to survey the types of data sources in use.
Import new data Select this option if you want to establish an ongoing connection, so that any new data batches that get added to the datasets in this connection automatically flow into Workspace. Can be set to On or Off. Dataset backfill Select Request backfill to backfill historical data.
- You can backfill each dataset individually.
- You prioritize new data added to a dataset in the connection, so this new data has the lowest latency.
- Any backfill (historical) data is imported at a slower rate. The latency is influenced by how much historical data that you have.
- The Analytics source connector imports up to 13 months of data (irrespective of size) for production sandboxes. The backfill in non-production sandboxes is limited to 3 months.
Backfill status Possible status indicators are:
- Success
- X backfill(s) processing
- Off
Dataset ID This ID is automatically generated. Description The description given to this dataset when it was created. Dataset size The dataset’s size. Schema The schema based on which the dataset was created in Adobe Experience Platform. Dataset The name of the dataset. Preview: dataset name Previews the dataset with date, my ID, and Identifier columns. Remove You can delete or remove the dataset and change the Person ID without deleting the whole connection. Deleting or removing reduces the costs involved in data ingestion and the cumbersome process of recreating the whole connection and associated data views.
Connection preview preview
To preview the connection that you have created, click Connection preview in the Connection settings dialog.
This preview contains some columns listing the connection configuration. What column types are shown depends on your individual datasets.
Dataset types dataset-types
For each dataset that you add to this connection, Customer Journey Analytics automatically sets the dataset type based on the data coming in.
There are three different dataset types: Event data, Profile data, and Lookup data.
Use numeric fields as lookup keys and lookup values numeric
This lookup functionality is useful if you want to add a numeric field such as a cost or margin to a string-based key field. It allows numeric values to be part of lookups, either as keys or as values. In your lookup schema, you might have numeric values tied to, for example, your product names, COGS, campaign marketing cost, or margins. Here is an example lookup schema in Adobe Experience Platform:
You now support bringing in these values as metrics or dimensions into Customer Journey Analytics reporting. When you set up your connection and pull in lookup datasets, you can edit the datasets to select the Key and Matching Key:
When you set up a data view based on this connection, you add the numeric values as components to the data view. Any project based on this data view can then report on these numeric values.
Use Identity Map as a Person ID id-map
Customer Journey Analytics supports the ability to use the Identity Map for its Person ID. Identity Map is a map data structure that allows you to upload key -> value pairs. The keys are identity namespaces and the value is a structure that holds the identity value. The Identity Map exists on each row/event uploaded and is populated for each row accordingly.
The Identity Map is available for any dataset that uses a schema based on the ExperienceEvent XDM class. When you select such a dataset to be included in a Customer Journey Analytics Connection, you have the option of selecting either a field as the primary ID or the Identity Map:
If you select Identity Map, you get two additional configuration options:
Identity Map edge cases id-map-edge
This table shows the two configuration options when edge cases are present and how they are handled:
Calculate the average number of daily events average-number
This calculation is done for every dataset in the connection.
-
Go to Adobe Experience Platform Query Services and create a query.
The query would look like this:
code language-none Select AVG(A.total_events) from (Select DISTINCT COUNT (*) as total_events, date(TIMESTAMP) from analytics_demo_data GROUP BY 2 Having total_events>0) A;In this example, “analytics_demo_data” is the name of the dataset.
-
To show all the datasets that exist in Adobe Experience Platform, perform the
Show Tablesquery .
Algorithmic pruning of large lookup datasets
When creating a connection, you can add large datasets for lookup purposes. For example, a dataset representing a product catalog so descriptive product information can be looked up when building reports and visualizations. Such a large lookup dataset can exceed the maximum of 10 million unique lookups currently implemented as as guardrail, resulting in additional data being skipped.
You can request an algorithmic pruning of a large lookup dataset. This algorithmic pruning only keeps data in the lookup dataset that matches the keys in your event dataset. This way, you don’t need to load the entire unpruned lookup dataset. Old or less frequently used items are removed, which might slightly affect reports but brings significant benefits. The algorithm looks back 90 days and updates weekly.
Contact your Adobe support team for further information and to enable this capability.