Show Menu

Create an Apache Spark on Azure HDInsights source connector in the UI

The Apache Spark on Azure HDInsights connector is in beta. See the Sources overview for more information on using beta-labelled connectors.
Source connectors in Adobe Experience Platform provide the ability to ingest externally sourced data on a scheduled basis. This tutorial provides steps for creating an Apache Spark on Azure HDInsights source connector using the Platform user interface.

Getting started

This tutorial requires a working understanding of the following components of Adobe Experience Platform:
If you already have a valid Spark connection, you may skip the remainder of this document and proceed to the tutorial on configuring a dataflow

Gather required credentials

In order to access your Spark account on Platform, you must provide the following values:
The IP address or hostname of the Spark server.
The username that you use to access the Spark server.
The password that corresponds to the user.
For more information about getting started refer to this Spark document .

Connect your Spark account

Once you have gathered your required credentials, you can follow the steps below to create a new Spark account to connect to Platform.
Log in to Adobe Experience Platform and then select Sources from the left navigation bar to access the Sources workspace. The Catalog screen displays a variety of sources for which you can create inbound account, and each source shows the number of existing accounts and dataset flows associated to them.
You can select the appropriate category from the catalog on the left-hand side of your screen. Alternatively, you can find the specific source you wish to work with using the search option.
Under the Databases category, select Spark to expose an information bar on the right-hand side of your screen. The information bar provides a brief description for the selected source as well as options to connect with the source or view its documentation. To create a new inbound connection, select Connect source .
The Connect to Spark page appears. On this page, you can either use new credentials or existing credentials.

New account

If you are using new credentials, select New account . On the input form that appears, provide the connection with a name, an optional description, and your Spark credentials. When finished, select Connect and then allow some time for the new account to establish.

Existing account

To connect an existing account, select the Spark account you want to connect with, then select Next to proceed.

Next steps

By following this tutorial, you have established a connection to your Spark account. You can now continue on to the next tutorial and configure a dataflow to bring data into Platform .