Skip to main content

Flattening

Flattening recursively expands all nested data structures into a flat schema, converting deeply nested fields into top-level columns.

When to Use​

Use Flattening when:

  • Your source data has deeply nested structures that are difficult to query.
  • You want a fully flat schema without manually specifying which structs to flatten.
  • You need to simplify data for downstream consumers that do not support nested types.

How It Differs from Flatten Struct​

  • Flatten Struct targets a specific struct field and promotes its immediate child properties to top-level columns with concatenated names.
  • Flattening operates on the entire record, recursively expanding all nested structures throughout the schema.

Configuration​

When adding a Flattening transformation to a Flow:

  1. Select Flattening from the list of available transformations.
  2. No additional configuration is required — the transformation automatically flattens all nested structures in the record.

Usage Notes​

  • Flattened field names are constructed by concatenating parent and child field names (e.g., address.city becomes address_city).
  • Applying Flattening to deeply nested schemas may produce a large number of top-level columns.