Skip to main content

Amazon EventBridge

This connector materializes Estuary collections into events on an Amazon EventBridge event bus. Each document is published as an EventBridge event using the AWS PutEvents API.

Prerequisites

To use this connector, you'll need:

  • An existing EventBridge event bus to publish to. The connector does not create the bus; it must exist before the materialization is applied. To use your account's default bus, set the bus name to default.

  • An IAM user or role with the following permissions on the target event bus:

    • PutEvents
    • DescribeEventBus

    These permissions should be specified with the events: prefix in an IAM policy document scoped to the bus ARN. For more details and examples, see Using identity-based policies with Amazon EventBridge in the Amazon docs.

  • AWS Credentials. One of the following types:

    • The AWS access key and secret access key for the user. See the AWS blog for help finding these credentials.
    • To authenticate using an AWS Role, you'll need the region and the role arn. Follow the steps in the AWS IAM guide to setup the role.

Configuration

You configure connectors either in the Estuary web app, or by directly editing the catalog specification file. See connectors to learn more about using connectors. The values and specification sample below provide configuration details specific to the EventBridge materialization connector.

Properties

Endpoint

PropertyTitleDescriptionTypeRequired/Default
/regionAWS RegionRegion of the EventBridge event bus.stringRequired
/event_bus_nameEvent Bus NameName or ARN of the EventBridge event bus to publish to. Use default for the account's default bus. Verified via DescribeEventBus on Apply.stringRequired, default
/credentialsAuthenticationCredentials for authentication.CredentialsRequired
/advanced/endpointAWS EndpointOverride the AWS endpoint URL. Used to direct requests at a compatible API such as LocalStack.string

Credentials

Credentials for authenticating with AWS. Use one of the following sets of options:

PropertyTitleDescriptionTypeRequired/Default
/credentials/auth_typeAuth TypeMethod to use for authentication.stringRequired: AWSAccessKey
/credentials/aws_access_key_idAWS Access Key IDAWS Access Key ID for publishing to EventBridge.stringRequired
/credentials/aws_secret_access_keyAWS Secret Access KeyAWS Secret Access Key for publishing to EventBridge.stringRequired
PropertyTitleDescriptionTypeRequired/Default
/credentials/auth_typeAuth TypeMethod to use for authentication.stringRequired: AWSIAM
/credentials/aws_role_arnAWS Role ARNIAM Role to assume.stringRequired
/credentials/aws_regionAWS RegionAWS Region to authenticate in.stringRequired

Bindings

Each binding publishes to the configured event bus. The source and detail_type values are written verbatim to the corresponding fields of every event produced by that binding, and can be used by EventBridge rules to route events to downstream targets.

PropertyTitleDescriptionTypeRequired/Default
/sourceEvent SourceSource field set on every event published from this binding.stringRequired, estuary.flow
/detail_typeDetail TypeDetailType field set on every event published from this binding.stringRequired, Document Published

Sample

materializations:
${PREFIX}/${MATERIALIZATION_NAME}:
endpoint:
connector:
image: ghcr.io/estuary/materialize-eventbridge:v1
config:
region: us-east-1
event_bus_name: default
credentials:
auth_type: AWSAccessKey
aws_access_key_id: example-aws-access-key-id
aws_secret_access_key: example-aws-secret-access-key
bindings:
- resource:
source: estuary.flow
detail_type: Document Published
source: ${PREFIX}/${COLLECTION_NAME}

Event payload

Each materialized document is published as a single EventBridge event. The connector populates the following fields on each PutEventsRequestEntry:

EventBridge fieldValue
EventBusNameThe bus configured at the endpoint level.
SourceThe source configured on the binding (defaults to estuary.flow).
DetailTypeThe detail_type configured on the binding (defaults to Document Published).
DetailThe full root document of the materialized collection, as a JSON string.

Time and Resources are not set; EventBridge assigns the event time on receipt. The materialized document's content is not modified — downstream rules and targets see exactly what was in the source collection.

You can publish multiple Estuary collections to the same event bus by adding multiple bindings to the materialization. Distinct source / detail_type values let downstream EventBridge rules route each collection's events independently.

Delta updates

Because EventBridge is an event-streaming system that has no concept of stored rows or keys, this connector uses only delta updates. Every document in the source collection results in one event published to the bus.

Delivery is at-least-once: in rare cases, a document may be published more than once. Downstream consumers should be prepared to deduplicate if exactly-once semantics are required.

Document size limit

EventBridge limits each event's combined Source + DetailType + Detail payload to 256 KB. If a document in the source collection produces an entry exceeding this limit, the materialization will fail with an error identifying the offending binding.

If your documents are large, use field selection on the binding to project down to only the fields needed by downstream consumers.