Skip to content

Rtrieu/docs 12615 update data obs #22129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rtrieu wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Rtrieu/docs 12615 update data obs #22129

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
08e390a
Add Data Obs reference to Databricks
rtrieu Dec 15, 2025
6aed862
add steps too
rtrieu Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions databricks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ Datadog offers several Databricks monitoring capabilities.

[Reference Tables][32] allow you to import metadata from your Databricks workspace into Datadog. These tables enrich your Datadog telemetry with critical context like workspace names, job definitions, cluster configurations, and user roles.

[Data Observability][36] helps data teams detect, resolve, and prevent issues affecting data quality, performance, and cost. It monitors anomalies in volume, freshness, null rates, and distributions, and integrates with pipelines to correlate issues with job runs, data streams, and infrastructure events.

Model serving metrics provide insights into how your Databricks model serving infrastructure is performing. With these metrics, you can detect endpoints that have high error rate, high latency, are over/under provisioned, and more.
## Setup

Expand Down Expand Up @@ -89,6 +91,12 @@ First, [connect a new Databricks workspace](#connect-to-a-new-databricks-workspa

5. Click **Save**.

#### Data Observability

1. Connect a workspace in Datadog's Databricks integration tile.
2. In the **Select products to set up integration** section, set **Data Observability** to **Enabled** to monitor data quality, freshness, and volume anomalies.
3. See [the docs for Data Observability][36] for more details on configuration and features.

#### Permissions
For Datadog to access your Databricks cost data in Data Jobs Monitoring or [Cloud Cost Management][34], the user or service principal used to query [system tables][35] must have the following permissions:
- `CAN USE` permission on the SQL Warehouse.
Expand Down Expand Up @@ -133,5 +141,6 @@ You can troubleshoot issues yourself by enabling the [Databricks web terminal][1
[33]: https://docs.datadoghq.com/data_jobs/databricks
[34]: https://docs.datadoghq.com/cloud_cost_management/
[35]: https://docs.databricks.com/aws/en/admin/system-tables/
[36]: https://docs.datadoghq.com/data_observability/
[8]: https://docs.datadoghq.com/integrations/spark/#metrics
[9]: https://docs.datadoghq.com/integrations/spark/#service-checks
Loading