Skip to main content

Transformations

Transformations can be added in a Flow to transform data from a source as you ingest it into a Onehouse table.

No-code transformations​

Onehouse offers the following no-code transformations that you can add to your Flows:

  • Add Current Timestamp — adds a new field with the timestamp of when the transformation runs.
  • Add Derived Date — converts an existing date field to a string with a custom format.
  • Column Filtering — selects or excludes specific columns from the source data, reducing the schema to only the fields you need in the destination table.
  • Convert CDC Data — processes CDC data from operational databases to update records based on insert/update/delete operations.
  • Explode Array — converts each element from an array into a new row.
  • Filter Rows — filters rows that match specified conditions using various operators.
  • Flatten Struct — converts struct properties into top-level fields with concatenated names.
  • Flattening — flattens nested data structures, recursively expanding all nested fields into a flat schema.
  • Generate Vector Embeddings — creates vector embeddings from text data using OpenAI or VoyageAI models.
  • Mask Data — masks sensitive data by hashing or truncating fields for privacy.
  • Parse JSON — parses JSON string fields into structured columns, enabling extraction of nested data from JSON-encoded source fields.

Custom transformations​

Custom transformations allow you to inject custom Java code into your Flows. Follow these instructions to create a custom transformation.