CDPGuidesSupport
SourcesReverse ETL

Connecting Reverse ETL Sources

How each Reverse ETL source connects to Ours Privacy: Snowflake tables and queries, your own S3 bucket, SFTP drops, data lake tables through your query engine, and paginated HTTP APIs, with the read-only access each one needs.

BetaThis is in beta. The basics are stable, and more capabilities are on the way.

Use this page to prepare access for a Reverse ETL source. Every source is read with the narrowest credential that works, and your account team configures the connection with you.

Beta: Reverse ETL is set up with your account team today. Hosted SFTP is currently available. The other connection types below are limited beta and are not generally available. Confirm access with your account representative before preparing credentials.


Snowflake

Ours Privacy reads a table or the result of a single SELECT query.

  • Access: a Snowflake user with a read-only role scoped to the schema you want to share. Authenticate with a password, a key pair, or an OAuth token.
  • What to prepare: the account identifier, warehouse, database, schema, and either the table name or the query. For incremental reads, name a primary key column and a timestamp column that changes when a row changes.
  • Good to know: the query runs in your warehouse on your compute. Keep it narrow, and add any column the transform needs rather than joining inside Ours Privacy.

Your Own Amazon S3 Bucket

Ours Privacy reads CSV or JSON Lines files from a prefix in a bucket you own.

  • Access: a read-only IAM role in your account that trusts a role Ours Privacy issues for your source. You grant list and read on one prefix, and decrypt on the KMS key if the bucket uses one. No access keys change hands, and the role appears in your own CloudTrail under the source's name.
  • What to prepare: the bucket, the prefix, the file format, and how your pipeline names and delivers files. Upload under a temporary name and rename into place when complete, so a half-written file is never read.
  • Good to know: each file version is imported once. A replaced file is read again as a new version.

SFTP

Two options: a drop Ours Privacy hosts for you, or your own SFTP server.

  • Ours-hosted drop: you provide an SSH public key and the IP ranges your sender uploads from. Ours Privacy returns the host, username, and host key fingerprint. Upload CSV or JSON Lines files under a temporary name and rename them into the ready folder when complete.
  • Your server: you provide the host, port, a username, and a password or private key. Ours Privacy pins your server's host key and reads from the folder you name.
  • Good to know: this is the usual path for a vendor export, such as a nightly appointment file, and the fastest path from a warehouse that is not yet a direct source.

Data Lake Tables

Ours Privacy reads an open-format data lake table, or a query over one, through the distributed SQL query engine you already run in front of it.

  • Access: a query engine user with a password or bearer token and read-only access to the catalog and schema.
  • What to prepare: the endpoint, catalog, schema, and the table or query, plus a primary key and timestamp column for incremental reads.

HTTP APIs

Ours Privacy reads a paginated collection from an internal service or a vendor API.

  • Access: a bearer token or basic auth credential with read access to the endpoint.
  • What to prepare: the URL, the pagination style (cursor or offset), and where the records and the next-page marker sit in the response.
  • Good to know: HTTP sources read the full collection on every run, so keep the endpoint filtered to the records you want to import.

Warehouses Not Yet Available as Sources

BigQuery, Databricks, Redshift, and Postgres are not direct sources yet. Schedule an export from your warehouse to an S3 bucket or an SFTP drop in the format above, and connect that as the source. Tell your account representative which warehouse you use; direct connectors are prioritized by request.

Your warehouse can still be a destination. See Amazon S3, Google BigQuery, Snowflake, and Databricks for sending processed event, dispatch, and visitor data back out.


Next Steps

How is this guide?

On this page