This connector materializes Flow collections into tables in an Amazon DynamoDB.
It is available for use in the Flow web application. For local development or open-source workflows,
provides the latest version of the connector as a Docker image. You can also follow the link in your
browser to see past image versions.
To use this connector, you'll need:
An IAM user with the following permissions:
BatchGetItemon all resources
BatchWriteItemon all resources
CreateTableon all resources
These permissions should be specified with the
dynamodb:prefix in an IAM policy document. For more details and examples, see Using identity-based policies with Amazon DynamoDB in the Amazon docs.
The AWS access key and secret access key for the user. See the AWS blog for help finding these credentials.
Materialized collections can have at most 2 collection keys.
By default, the materialized tables will include the collection keys as the DynamoDB partition key
and sort key, and the root document. The root document is materialized as
an alternate projection is configured for the source
collection. Additional fields may be included, but DynamoDB has a 400KB size limit on an individual
row so selecting too many fields of a collection with large documents will cause errors if the row
size exceeds that.
The root document is materialized as a DynamoDB
Map type, and the fields of the document must be
To resolve issues with collections with more than 2 keys, excessively large documents, or incompatible field names, use a derivation to derive a new collection and materialize that collection instead.
You configure connectors either in the Flow web app, or by directly editing the catalog specification file. See connectors to learn more about using connectors. The values and specification sample below provide configuration details specific to the DynamoDB materialization connector.
|Access Key ID
|AWS Access Key ID for materializing to DynamoDB.
|Secret Access Key
|AWS Secret Access Key for materializing to DynamoDB.
|Region of the materialized tables.
|The AWS endpoint URI to connect to. Use if you're materializing to a compatible API that isn't provided by AWS.
|The name of the table to be materialized to.
|Should updates to this table be done via delta updates. Default is false.