flow
Search…
Endpoint configurations
How to configure endpoints for captures
An endpoint is an external system from which a Flow collection may be captured, or to which a Flow collection may be materialized.
Setting up endpoints for captures and materializations is similar in principle, but configuration methods differ. This page discusses captures.
Endpoints in Flow captures are objects that use the following entity structure:
1
captures:
2
my/capture/name:
3
# Required, type: object
4
endpoint:
5
# The value of an endpoint must be an object that defines only one of the following
6
# top-level properties. Each of these top-level properties corresponds to a specific type of
7
# external system that you want to connect to, and holds system-dependent connection
8
# information. Each object here also permits any additional properties, which
9
# will simply be ignored.
10
specific_endpoint_type:
11
specific_endpoint_configuration: value
12
specific_endpoint_configuration2: value2
13
14
my/other/name:
15
endpoint:
16
# As a concrete example, airbyteSource only requires `image`
17
# and `config` keys as its configuration.
18
airbyteSource:
19
image: ghcr.io/estuary/source-kinesis:1d76c51
20
config: {}
Copied!
Flow currently supports the following configurations. Required values for each type are provided below, as well as external documentation for each system.

Airbyte Source configuration

Any Airbyte Specification compatible source-connector is supported as a capture endpoint, regardless of the creator.
  • Estuary maintains several connectors. The docker images can be found on GitHub here.
  • A list of other connectors can be found on the Airbyte docker hub. You can use any item whose name begins with source-.
Learn more about Airbyte Source.
1
# An Airbyte Source endpoint
2
# To be nested under <endpoint> in a catalog spec.
3
airbyteSource:
4
5
# The image is a path to a Docker image.
6
# Required, type: string
7
image: ghcr.io/estuary/source-kinesis:1f26eb0
8
9
# Each Airbyte Source has required configuration that varies by the source itself.
10
# Required, type: object
11
config:
12
13
# The Estuary Kinesis connector requires an access key to authenticate the
14
# connection and a region to send its requests.
15
awsAccessKeyId: your-aws-access-key-id
16
awsSecretAccessKey: your-aws-secret-key
17
region: us-east-2
Copied!
Last modified 3mo ago