Skip to main content

Amazon Kinesis

This connector captures data from Amazon Kinesis streams.

It is available for use in the Flow web application. For local development or open-source workflows, provides the latest version of the connector as a Docker image. You can also follow the link in your browser to see past image versions.


To use this connector, you'll need:

  • One or more Amazon Kinesis streams. For a given capture, all streams must:

    • Contain JSON data only
    • Be in the same AWS region
  • An IAM user with the following permissions:

    • ListShards on all resources
    • GetRecords on all streams used
    • GetShardIterator on all streams used
    • DescribeStream on all streams used
    • DescribeStreamSummary on all streams used

    These permissions should be specified with the kinesis: prefix in an IAM policy document. For more details and examples, see Controlling Access to Amazon Kinesis Data in the Amazon docs.

  • The AWS access key and secret access key for the user. See the AWS blog for help finding these credentials.


You configure connectors either in the Flow web app, or by directly editing the catalog specification file. See connectors to learn more about using connectors. The values and specification sample below provide configuration details specific to the Amazon Kinesis source connector.



/awsAccessKeyIdAWS access key IDPart of the AWS credentials that will be used to connect to Kinesis.stringRequired, "example-aws-access-key-id"
/awsSecretAccessKeyAWS secret access keyPart of the AWS credentials that will be used to connect to Kinesis.stringRequired, "example-aws-secret-access-key"
/endpointAWS endpointThe AWS endpoint URI to connect to, useful if you're capturing from a kinesis-compatible API that isn't provided by AWS.string
/regionAWS regionThe name of the AWS region where the Kinesis stream is located.stringRequired, "us-east-1"


/streamStreamStream name.stringRequired


A minimal capture definition will look like the following:

awsAccessKeyId: "example-aws-access-key-id"
awsSecretAccessKey: "example-aws-secret-access-key"
region: "us-east-1"
- resource:
stream: ${STREAM_NAME}

Your capture definition will likely be more complex, with additional bindings for each Kinesis stream.

Learn more about capture definitions..