Map a CSV file to an XDM schema
In order to ingest CSV data into Adobe Experience Platform, the data must be mapped to an Experience Data Model (XDM) schema. This tutorial covers how to map a CSV file to an XDM schema using the Platform user interface.
In addition, the appendix to this tutorial provides further information regarding the use of mapping functions .
This tutorial requires a working understanding of the following components of Platform:
- Experience Data Model (XDM System) : The standardized framework by which Platform organizes customer experience data.
- Batch ingestion : The method by which Platform ingests data from user-supplied datafiles.
This tutorial also requires that you have already created a dataset to ingest your CSV data into. For steps on creating a dataset in the UI, see the data ingest tutorial .
Choose a destination
Log in to Adobe Experience Platform and then select Workflows from the left navigation bar to access the Workflows workspace.
From the Workflows screen, select Map CSV to XDM schema under the Data ingestion section and then select Launch .
The Map CSV to XDM schema workflow appears, starting on the Destination step. Choose a dataset for inbound data to be ingested into. You can either use an existing dataset or create a new one.
Use an existing dataset
To ingest your CSV data into an existing dataset, select Use existing dataset . You can either retrieve an existing dataset using the search function or by scrolling through the list of existing datasets in the panel.
To ingest your CSV data into a new dataset, select Create new dataset and enter a name and description for the dataset in the fields provided. Select a schema by using either the search function or by scrolling through the list of schemas provided. Select Next to proceed.
The Add data step appears. Drag-and-drop your CSV file into the space provided, or select Choose files to manually input your CSV file.
The Sample data section appears once the file is uploaded, showing the first ten rows of data. Once you have confirmed that the data has uploaded as expected, select Next .
Map CSV fields to XDM schema fields
The Mapping step appears. The columns of the CSV file are listed under Source Field , with their corresponding XDM schema fields listed under Target Field . Unselected target fields are outlined in red. You can use the filter fields option to narrow down the list of source fields available.
To map a CSV column to an XDM field, select the schema icon next to the column's corresponding target field.
The Select schema field window appears. Here you can navigate the structure of the XDM schema and locate the field you wish to map the CSV column to. Click an XDM field to select it, then click Select .
The Mapping screen reappears, with the selected XDM field now appearing under Target Field .
If you do not wish to map a particular CSV column, you can remove the mapping by clicking the remove icon next to the target field. You can also remove all mappings by selecting the Clear all mappings button .
If you want to add a new mapping, select Add new mapping at the top of the Source Field list.
When mapping fields, you can also include functions to compute values based on input source fields. See the mapping functions section in the appendix for more information.
Add calculated field
Calculated fields allow for values to be created based on the attributes in the input schema. These values can then be assigned to attributes in the target schema and be provided a name and description to allow for easier reference.
Select the Add calculated field button to proceed.
The Create calculated field panel appears. The left dialog box contains the fields, functions, and operators supported in calculated fields. Select one of the tabs to start adding functions, fields, or operators to the expression editor.
The fields tab lists fields and attributes available in the source schema.
The functions tab lists the functions available to transform the data.
The operators tab lists the operators that are available to transform the data.
You can manually add fields, functions, and operators using the expression editor at the center. Select the editor to start creating an expression.
Select Save to proceed.
The mapping screen reappears with your newly created source field. Apply the appropriate corresponding target field and select Finish to complete the mapping.
Monitor your dataflow
Once your CSV file is mapped and created, you can monitor the data that is being ingested through it. For more information on monitoring dataflows, see the tutorial on monitoring streaming dataflows .
By following this tutorial, you have successfully mapped a flat CSV file to an XDM schema and ingested it into Platform. This data can now be used by downstream Platform services such as Real-time Customer Profile. See the overview for Real-time Customer Profile for more information.
The following section provides additional information for mapping CSV columns to XDM fields.
Certain mapping functions can be used to compute and calculate values based on what is entered in source fields. To use a function, type it in under Source Field with appropriate syntax and inputs.
For example, to concatenate city and country CSV fields and assign them to the city XDM field, set the source field as concat(city, ", ", county) .
The following table lists all supported mapping functions, including sample expressions and their resulting outputs.
Concatenates given strings.
concat("Hi, ", "there", "!")
Splits the string based on a regex and returns an array of parts.
explode("Hi, there!", " ")
Returns the location/index of a substring.
instr("adobe .com", "com")
Replaces the search string if present in original string.
replacestr("This is a string re test", "re", "replace")
"This is a string replace test"
Returns a substring of a given length.
substr("This is a substring test", 7, 8)
" a subst"
Converts a string to lowercase.
Converts a string to uppercase.
Splits an input string on a separator.
split("Hello world", " ")
Joins a list of objects using the separator.
join(" ", ["Hello", "world"] )
Returns the first non-null object in a given list.
coalesce(null, null, null, "first", null, "second")
Given a key and a list of key value pairs flattened as an array, the function returns the value if key is found or return a default value if present in the array.
decode("k2", "k1", "v1", "k2", "v2", "default")
Evaluates a given boolean expression and returns the specified value based on the result.
iif("s".equalsIgnoreCase("S"), "True", "False")
Returns the minimum of the given arguments. Uses natural ordering.
min(3, 1, 4)
Returns the maximum of the given arguments. Uses natural ordering.
max(3, 1, 4)
Retrieves the first given argument.
first("1", "2", "3")
Retrieves the last given argument.
last("1", "2", "3")
Generates a pseudo-random ID.
Retrieves the current time.
Retrieves the current Unix time.
Formats the input date according to a specified format.
format(, "yyyy-MM-dd HH:mm:ss")
Converts a timestamp to a date string according to a specified format.
dformat(1571829875, "dd-MMM-yyyy hh:mm")
Converts a date string into a ZonedDateTime object (ISO 8601 format).
Retrieves the parts of the date. The following component values are supported:
date_part(date("2019-10-17 11:55:12"), "MM")
Replaces a component in a given date. The following components are accepted:
set_date_part("m", 4, date("2016-11-09T11:44:44.797")
Creates a date from parts.
make_date_time(2019, 10, 17, 11, 55, 12, 999, "America/Los_Angeles")
Returns the current timestamp.
Returns the current date without a time component.