Skip to main content
DASH NYC, June 9-10 | AI + Observability.

Back to Catalog

Presented by Databricks: Unifying Observability and Data Intelligence with Databricks and Datadog to Power Index Platforms at Scale

About this Session

Enterprise data pipelines don't fail quietly — they impact the business users and clients who depend on them. In this session, learn how the Databricks Data Intelligence Platform integrates with Datadog to deliver unified observability across your lakehouse and cloud infrastructure.

 

This session covers how Databricks Jobs Monitoring, cloud cost monitoring, and data quality monitoring pair with Datadog to give teams a single view of pipeline health — across serverless and classic compute, alongside AWS services.

 

Hear directly from a leading global financial exchange that standardized their Databricks estate across business units to power critical index data pipelines. Their platform team unified monitoring across Spark streaming jobs and Databricks compute into a single Datadog view — enabling real-time alerting, health dashboards, and AI-generated executive summaries of pipeline status using the Datadog MCP integration.

 

The session also explores where this is headed: Databricks Genie connected to Datadog's MCP server, so business users querying data are proactively informed of upstream pipeline failures — right inside their AI conversation, without escalating to engineering. The session includes a live demo of these announcements.