DocumentationExperience PlatformDestinations Guide

Amazon S3 connection s3-connection

Last update: Fri Mar 08 2024 00:00:00 GMT+0000 (Coordinated Universal Time)

CREATED FOR:

  • Admin
  • User

Destination changelog changelog

View changelog
table 0-row-3 1-row-3 2-row-3 layout-auto
Release month Update type Description
January 2024 Functionality and documentation update The Amazon S3 destination connector now supports a new assumed role authentication type. Read more about it in the authentication section.
July 2023 Functionality and documentation update

With the July 2023 Experience Platform release, the Amazon S3 destination provides new functionality, as listed below:

Connect to your Amazon S3 storage through API or UI connect-api-or-ui

Supported audiences supported-audiences

This section describes which types of audiences you can export to this destination.

Audience origin
Supported
Description
Segmentation Service
✓
Audiences generated through the Experience Platform Segmentation Service.
Custom uploads
✓
Audiences imported into Experience Platform from CSV files.

Export type and frequency export-type-frequency

Refer to the table below for information about the destination export type and frequency.

Item
Type
Notes
Export type
Profile-based
You are exporting all members of a segment, together with the desired schema fields (for example: email address, phone number, last name), as chosen in the select profile attributes screen of the destination activation workflow.
Export frequency
Batch
Batch destinations export files to downstream platforms in increments of three, six, eight, twelve, or twenty-four hours. Read more about batch file-based destinations.

Amazon S3 profile-based export type highlighted in the UU.

Export datasets export-datasets

This destination supports dataset exports. For complete information on how to set up dataset exports, read the tutorials:

File format of the exported data file-format

When exporting audience data, Platform creates a .csv, parquet, or .json file in the storage location that you provided. For more information about the files, see the supported file formats for export section in the audience activation tutorial.

When exporting datasets, Platform creates a .parquet or .json file in the storage location that you provided. For more information about the files, see the verify successful dataset export section in the export datasets tutorial.

Connect to the destination connect

IMPORTANT
To connect to the destination, you need the View Destinations and Manage Destinations access control permissions. Read the access control overview or contact your product administrator to obtain the required permissions.

To connect to this destination, follow the steps described in the destination configuration tutorial. In the destination configuration workflow, fill in the fields listed in the two sections below.

Authenticate to destination authenticate

To authenticate to the destination, fill in the required fields and select Connect to destination. The Amazon S3 destination supports two authentication methods:

  • Access key and secret key authentication
  • Assumed role authentication

Access key and secret key authentication

Use this authentication method when you want to input your Amazon S3 access key and secret key to allow Experience Platform to export data to your Amazon S3 properties.

Image of the required fields when selecting access key and secret key authentication.

  • Amazon S3 access key and Amazon S3 secret key: In Amazon S3, generate an access key - secret access key pair to grant Platform access to your Amazon S3 account. Learn more in the Amazon Web Services documentation.

  • Encryption key: Optionally, you can attach your RSA-formatted public key to add encryption to your exported files. View an example of a correctly formatted encryption key in the image below.

    Image showing an example of a correctly formatted PGP key in the UI.

Assumed role assumed-role-authentication

Image of the required fields when selecting assumed role authentication.

Use this authentication type if you prefer not to share account keys and secret keys with Adobe. Instead, Experience Platform connects to your Amazon S3 location using role-based access.

To do this, you need to create in the AWS console an assumed user for Adobe with the right required permissions to write to your Amazon S3 buckets. Create a Trusted entity in AWS with the Adobe account 670664943635. For more information, refer to the AWS documentation on creating roles.

  • Role: Paste the ARN of the role that you created in AWS for the Adobe user. The pattern is similar to arn:aws:iam::800873819705:role/destinations-role-customer.
  • Encryption key: Optionally, you can attach your RSA-formatted public key to add encryption to your exported files. View an example of a correctly formatted encryption key in the image below.

Fill in destination details destination-details

To configure details for the destination, fill in the required and optional fields below. An asterisk next to a field in the UI indicates that the field is required.

  • Name: Enter a name that will help you identify this destination.

  • Description: Enter a description of this destination.

  • Bucket name: Enter the name of the Amazon S3 bucket to be used by this destination.

  • Folder path: Enter the path to the destination folder that will host the exported files.

  • File type: Select the format Experience Platform should use for the exported files. When selecting the CSV option, you can also configure the file formatting options.

  • Compression format: Select the compression type that Experience Platform should use for the exported files.

  • Include manifest file: Toggle this option on if you’d like the exports to include a manifest JSON file that contains information about the export location, export size, and more. The manifest is named using the format manifest-<<destinationId>>-<<dataflowRunId>>.json. View a sample manifest file. The manifest file includes the following fields:

    • flowRunId: The dataflow run which generated the exported file.
    • scheduledTime: The time in UTC when the file was exported.
    • exportResults.sinkPath: The path in your storage location where the exported file is deposited.
    • exportResults.name: The name of the exported file.
    • size: The size of the exported file, in bytes.
TIP
In the connect destination workflow, you can create a custom folder in your Amazon S3 storage per exported audience file. Read Use macros to create a folder in your storage location for instructions.

Enable alerts enable-alerts

You can enable alerts to receive notifications on the status of the dataflow to your destination. Select an alert from the list to subscribe to receive notifications on the status of your dataflow. For more information on alerts, see the guide on subscribing to destinations alerts using the UI.

When you are finished providing details for your destination connection, select Next.

Required Amazon S3 permissions required-s3-permission

To successfully connect and export data to your Amazon S3 storage location, create an Identity and Access Management (IAM) user for Platform in Amazon S3 and assign permissions for the following actions:

  • s3:DeleteObject
  • s3:GetBucketLocation
  • s3:GetObject
  • s3:ListBucket
  • s3:PutObject
  • s3:ListMultipartUploadParts

Activate audiences to this destination activate

IMPORTANT

See Activate audience data to batch profile export destinations for instructions on activating audiences to this destination.

Validate successful data export exported-data

To verify if data has been exported successfully, check your Amazon S3 storage and make sure that the exported files contain the expected profile populations.

IP address allowlist ip-address-allow-list

Refer to the IP address allowlist article if you need to add Adobe IPs to an allowlist.

recommendation-more-help
7f4d1967-bf93-4dba-9789-bb6b505339d6