Create an Apache HDFS source connector in the UI
The Apache HDFS connector is in beta. See the Sources overview for more information on using beta-labelled connectors.
Source connectors in Adobe Experience Platform provide the ability to ingest externally sourced data on a scheduled basis. This tutorial provides steps for authenticating an Apache Hadoop Distributed File System (hereinafter referred to as "HDFS") source connector using the Platform user interface.
This tutorial requires a working understanding of the following components of Platform:
- Experience Data Model (XDM) System : The standardized framework by which Experience Platform organizes customer experience data.
- Real-time Customer Profile : Provides a unified, real-time consumer profile based on aggregated data from multiple sources.
If you already have an HDFS connection, you may skip the remainder of this document and proceed to the tutorial on configuring a dataflow .
Gather required credentials
In order to authenticate your HDFS source connector, you must provide values for the following connection property:
The URL defines auth params required for connecting to HDFS anonymously. For more information on how to obtain this value, refer to the following document on HTTPS authentication for HDFS .
Connect your HDFS account
Once you have gathered your required credentials, you can follow the steps below to create a new HDFS account to connect to Platform.
Log in to Adobe Experience Platform and then select Sources from the left navigation bar to access the Sources workspace. The Catalog screen displays a variety of sources for which you can create an inbound account with, and each source shows the number of existing accounts and dataflows associated with them.
You can select the appropriate category from the catalog on the left-hand side of your screen. Alternatively, you can find the specific source you wish to work with using the search option.
Under the Cloud storage category, select Apache HDFS click on the + icon (+) to create a new HDFS connector.
The Connect to HDFS page appears. On this page, you can either use new credentials or existing credentials.
If you are using new credentials, select New account . On the input form that appears, provide the connection with a name, an optional description, and your File Storage credentials. When finished, select Connect to source and then allow some time for the new account to establish.
To connect an existing account, select the HDFS account you want to connect with, then select Next to proceed.
By following this tutorial, you have established a connection to your HDFS account. You can now continue on to the next tutorial and configure a dataflow to bring data from your cloud storage into Platform .