Observability
Estuary provides several observability features to keep track of your data flows and let you investigate anomalies, either within Estuary's platform or via an external integration.
Dashboard
Tasks provide an overview of their stats on the task details page. You can view data or docs in or out, task status, and other details specific to the task type.
The dashboard also provides information on current alerts and task logs for connectors and derivations.
See docs on the capture, collection, and materialization details pages for more information on observability in the dashboard.
Alerts and Notifications
You can set alerts to notify you when anomalies occur in your data flows. Estuary will email an address of your choice (such as a mailing list or address connected to a Slack channel) when anomalous behavior reaches a certain threshold. For example, when data movement stalls for a certain amount of time or when a connector sees a set number of failures within a specific timeframe.
Learn about available alert types and setting up alert subscriptions.
OpenMetrics API
Estuary surfaces task metrics with an OpenMetrics API. This lets you plug Estuary's stats into common observability and monitoring platforms like Datadog and Prometheus, or create your own custom integrations.
Find setup guides and available metrics.
flowctl CLI
Users who are comfortable with Estuary's flowctl CLI can also retrieve stats
using flowctl commands.
Available task statistics are very similar to those surfaced through
OpenMetrics; flowctl is simply a different way to retrieve the same
underlying information.