Skip to main content

Open-Format Sync Troubleshooting

Onetable (powered by Apache XTable) keeps Apache Iceberg and Delta Lake metadata in sync with your Apache Hudi tables. Because the sync writes a parallel metadata layer rather than rewriting data files, certain edits and edge cases can leave the metadata referencing files that no longer exist, or in a state the catalog refuses to read.

This page covers recovery for those scenarios. For initial setup, see the Onetable page.

When to suspect a sync problem​

  • Iceberg / Delta queries fail while Hudi queries against the same table succeed.
  • The query engine reports a missing data file (e.g. FileNotFoundException on a parquet file path) even though the Hudi .hoodie/ timeline is healthy.
  • Metadata integrity checks fail with file-count mismatches (Delta log integrity check failed: expected N files, found M).
  • File listings under <table_base_path>/metadata/ or <table_base_path>/_delta_log/ are inconsistent with what's actually in the data directory.

Recovery: Iceberg metadata references files that have been cleaned​

The Hudi cleaning service can remove old data files before Iceberg metadata catches up, leaving Iceberg snapshots pointing at deleted paths. This typically surfaces as FileNotFoundException from Iceberg readers.

To recover:

  1. Pause the Flow writing to the table to prevent further drift while you repair.
  2. Open a support ticket and request a full snapshot sync for the affected table. Snapshot sync rebuilds the Iceberg metadata from the current Hudi timeline, dropping references to cleaned files.
  3. Resume the Flow once the snapshot sync completes.

To reduce recurrence, lengthen the time travel retention on the cleaning service so files survive long enough for Iceberg sync to catch up, or sync more frequently.

Recovery: Delta _delta_log corruption or integrity errors​

Delta Lake transaction logs can drift from the actual file inventory if the table's data layer is rewritten outside the normal sync path — for example, manually deleting metadata, restoring from backup, or running a write tool that doesn't update the _delta_log consistently. The Unity Catalog and other Delta consumers will reject reads against a corrupted log.

To recover:

  1. Back up <table_base_path>/_delta_log/ before any repair attempt. The log is small but irreplaceable if a repair makes things worse.
  2. Pause the Flow.
  3. Contact support to request a snapshot-sync rebuild of the Delta log. Onehouse regenerates the log from the current Hudi state.
  4. After the rebuild, pause and resume the Metadata Sync table service so downstream catalogs (Unity, etc.) pick up the new state.
  5. Resume the Flow.
Avoid manual edits to the metadata layer

Manually deleting or recreating files under metadata/ (Iceberg) or _delta_log/ (Delta) breaks the sync's understanding of where it left off and almost always requires a snapshot-sync rebuild to restore. Contact support before attempting any manual repair.

Recovery: S3 base path changed (Iceberg only)​

Apache Iceberg metadata stores absolute paths to data files, while Hudi stores relative paths. If you move or copy a table to a new S3 bucket or prefix, the copied Iceberg metadata still references the old path and queries fail with file-not-found errors. Hudi reads continue to work because the relative paths resolve correctly at the new location.

To repair Iceberg metadata after a base-path change:

  1. Delete the <new_base_path>/metadata/ directory from S3.
  2. Trigger a metasync rebuild — Onehouse will write a new Iceberg metadata tree containing the correct absolute paths.

Delta Lake is unaffected by base-path changes because its transaction log uses relative paths.

XTable sync errors at startup​

These errors usually indicate a configuration problem rather than corruption, and fixing the configuration is enough — no snapshot rebuild required.

INVALID_CONFIGURATION on partition fields​

The partition field configuration in hoodie.properties has malformed input — most commonly a leading space, a trailing comma, or a typo. Open the table's hoodie.properties (visible via DESCRIBE TABLE) and clean up the partition field list.

Data-type mismatch across files​

Iceberg requires a single column type per field across all files. If different writers produced files with different types for the same column (int vs long, string vs bytes), Iceberg sync fails. Either standardize the type upstream and rewrite the affected files, or use schema evolution to widen the column to a common super-type.

XTable sync takes hours on first run​

First-run XTable sync time is dominated by reading the Hudi metadata table and column statistics. Verify the metadata table is enabled and column stats are populated; if they aren't, the sync has to scan every data file. Subsequent runs are incremental and much faster.

Sync after a schema or partition change​

If you change a table's partition fields while a sync is configured, Iceberg / Delta sync will fail until the table service refreshes its view of the schema:

  1. Pause the Metadata Sync table service.
  2. Resume the Metadata Sync table service. The pause/resume cycle forces the service to re-read the current schema and partition definition before its next run.

This is the same pattern used for credential rotation on sources — pause/resume is non-destructive and the standard way to make a service pick up a configuration change.