Datadog Metrics Reference
This page lists Onehouse metrics that are exposed to Datadog when you follow the Datadog integration setup steps. These are the same metrics that power the pre-built Onehouse Grafana dashboards in Advanced Monitoring, surfaced through the Spark driver's OpenMetrics endpoint at http://<driver>:9091/metrics (namespace onehouse in Datadog).
DeltaStreamer (ingestion) jobs emit metrics with the ohds_ prefix. SQL cluster jobs emit the equivalent per-action metrics with the ohsql_ prefix. The action source (deltastreamer, deltacommit, compaction, replacecommit, clean) follows the prefix, then the metric suffix. Each metric carries table and database labels so you can filter and aggregate per table.
DeltaStreamer Metrics
Track ingestion lag, sync timing, and failures.
| Description | Metric Name |
|---|---|
| Number of Kafka source messages pending to be ingested | ohds_deltastreamer_kafkaDelayCount |
| Number of consecutive sync lags | ohds_deltastreamer_numConsecutiveSyncLags |
| Sync Frequency | ohds_deltastreamer_syncIntervalSecs |
| Sync Duration | ohds_deltastreamer_syncDurationSecs |
| Absolute time of the last sync | ohds_deltastreamer_lastSync |
| Number of sync failures | ohds_deltastreamer_numFailedCommits |
Write Stats
Track records and bytes written by ingestion commits.
| Description | Metric Name |
|---|---|
| Total bytes written | ohds_deltacommit_totalBytesWritten |
| Records Written over time | ohds_deltacommit_totalRecordsWritten |
| Records Upserted Over Time | ohds_deltacommit_totalUpdateRecordsWritten |
| Records Inserted Over Time | ohds_deltacommit_totalInsertRecordsWritten |
| Total Number of Records Deleted | ohds_deltacommit_totalRecordsDeleted |
| Total partitions written in Ingestion Commits | ohds_deltacommit_totalPartitionsWritten |
Compaction Metrics
Track compaction service runs.
| Description | Metric Name |
|---|---|
| Time taken by Compaction Commits | ohds_compaction_duration |
| Number of bytes compacted by Compaction Commits | ohds_compaction_totalBytesWritten |
| Number of partitions touched by Compaction Commits | ohds_compaction_totalPartitionsWritten |
Clustering Stats
Track clustering service runs.
| Description | Metric Name |
|---|---|
| Time taken by Clustering Commits | ohds_replacecommit_duration |
| Total Bytes Written by Clustering Commits | ohds_replacecommit_totalBytesWritten |
| Total Partitions Written By Clustering Commits | ohds_replacecommit_totalPartitionsWritten |
Cleaner Service Stats
Track cleaner service runs.
| Description | Metric Name |
|---|---|
| Cleaner Duration | ohds_clean_duration |
| Number of Clean Files Deleted | ohds_clean_numFilesDeleted |
SQL Cluster Metrics
SQL clusters emit the same per-action write, compaction, clustering, and cleaner metrics as OneFlow, but with the ohsql_ prefix instead of ohds_. Use these to monitor table-level activity driven by SQL workloads.
| Description | Metric Name |
|---|---|
| Total bytes written by SQL commits | ohsql_deltacommit_totalBytesWritten |
| Records written over time by SQL commits | ohsql_deltacommit_totalRecordsWritten |
| Records upserted over time by SQL commits | ohsql_deltacommit_totalUpdateRecordsWritten |
| Records inserted over time by SQL commits | ohsql_deltacommit_totalInsertRecordsWritten |
| Total number of records deleted by SQL commits | ohsql_deltacommit_totalRecordsDeleted |
| Total partitions written by SQL commits | ohsql_deltacommit_totalPartitionsWritten |
| Time taken by SQL Compaction Commits | ohsql_compaction_duration |
| Number of bytes compacted by SQL Compaction Commits | ohsql_compaction_totalBytesWritten |
| Number of partitions touched by SQL Compaction Commits | ohsql_compaction_totalPartitionsWritten |
| Time taken by SQL Clustering Commits | ohsql_replacecommit_duration |
| Total bytes written by SQL Clustering Commits | ohsql_replacecommit_totalBytesWritten |
| Total partitions written by SQL Clustering Commits | ohsql_replacecommit_totalPartitionsWritten |
| SQL Cleaner Duration | ohsql_clean_duration |
| Number of Clean Files Deleted by SQL Cleaner | ohsql_clean_numFilesDeleted |
SQL clusters do not emit ohsql_deltastreamer_* family. Use Copy-On-Write equivalents (ohsql_commit_*) if your SQL workloads write to COW tables.
If you need additional metrics beyond the ones listed here, file a support ticket with Onehouse.