Data feeds

We support publishing data feeds to an Amazon S3 bucket or SFTP endpoint. This allows you to export contacts as they enter (or exit) a flow, and record the occurrence of other events. The file path, type, naming, schema, control protocol, and optional conditions for Simon data feeds are broken out below.

Usage

  1. When setting up a new flow, scroll to the bottom of the page to section titled "Configure Data Feed (Optional).
  2. Select "Yes".
  3. Fill in the setup fields and save or launch the flow.
990990

Feed configuration

Data format

Select the format of the data export from the following options:

  • CSV
  • TSV
  • JSONL

(Optional) Hash identity

Optionally, select the checkbox if you want a contact's identity information to be hashed using a SHA-256.

Data target

Select S3 or FTP.

Host/bucket

Enter the name of the S3 bucket or SFTP endpoint you wish to write to. Do not include the scheme (e.g. S3://) or a trailing slash (/) in the name.

Path

Enter the file path you wish to write to. Do not include a beginning or trailing slash (/) in the name. Files are uploaded to the following path: /path/year/month/day.

Personalized content

Add fields to the data feed in addition to the default values in the schema. Add a name and value using the same steps to setup Custom Contexts.

Feed format

File types

Two types of files are produced: data files and control files.

File name

Data files are named by the client_flow_operation_random_YmdHMS with an extension that indicates the file type (either ".csv" or ".jsonl").

Data file schema

The file will include, for each contact, one entry for the data feed and one for each applicable flow action.

Parameter

Description

identifier

Email Address

timestamp

Time of export, with format: Y-M-D HH:MM:SS (e.g. "2017-03-14 17:54:13")

segment_name

Name of the segment

flow_name

Name of the flow

action

Flow action associated with the entry

variant

Experiment variant (null if there is not one)

operation

Type of operation performed (either 'add' or 'remove')

Custom Contexts (optional)

Control files

Control files are named identically to their corresponding data file with the prefix control_ and are always JSON files. They contain a single JSON object with the following structure:

Parameter

Description

delta

Reserved for future use; always true

file_count

Reserved for future use; always 1

files

A single-element array of metadata; see below

The files array will contain a single JSON object that provides information about the corresponding data file. It contains:

bucket

The specified bucket

path

The specified path

file_type

Either "csv" or "jsonl"

file

Full URL of data file

rows

The number of rows in the data file

flow_ID

The ID of the flow

flow_name

The name of the flow

Currently this produces one control file for every data file, but in the future it may produce a single control file for many data files.


Did this page help you?