Skip to main content

oh transformation

Manage row-level transformations applied to records during ingestion.

Transformation types: CONVERT_CDC ROW_FILTERING DERIVED_DATE ADD_CURRENT_TIMESTAMP USER_PROVIDED DATA_MASKING L_PARSE_JSON FLATTEN_STRUCT EXPLODE_ARRAY VECTOR_EMBEDDING

oh transformation show​

Lists all transformations.

oh transformation show

oh transformation describe NAME​

Describes a transformation.

oh transformation describe mask-pii

oh transformation create NAME​

Creates a transformation.

Flags​

FlagRequiredDescription
--typeYesTransformation type
--options KEY=VALUENoWITH clause parameters; repeatable (see Special parameters)

Special parameters​

All WITH clause parameters from the CREATE TRANSFORMATION API are passed via --options KEY=VALUE.

Examples​

Row filtering​

Filter records where a field equals a value. Valid operators: EQ, NEQ, GT, GEQ, LT, LEQ.

oh transformation create filter-test-users \
  --type ROW_FILTERING \
  --options row.filtering.transformation.field=is_test_user \
  --options row.filtering.transformation.sql.operator=EQ \
  --options row.filtering.transformation.value=true \
  --options row.filtering.transformation.value.type=string

Convert CDC​

Converts records from a CDC format. Supported formats: postgresql, mysql, oracle, mongodb.

# Postgres / MySQL / Oracle CDC (no schema required)
oh transformation create pg-cdc \
  --type CONVERT_CDC \
  --options convert_cdc.cdc_format=postgresql

# MongoDB CDC (target schema required)
oh transformation create mongo-cdc \
  --type CONVERT_CDC \
  --options convert_cdc.cdc_format=mongodb \
  --options convert_cdc.target_schema=my_schema

Data masking​

Mask fields by hashing or truncation.

oh transformation create mask-pii \
  --type DATA_MASKING \
  --options data_masking.type=HASHING \
  --options data_masking.fields=email,phone,ssn

Derived date field​

Extract and reformat a date from an existing field into a new field.

oh transformation create extract-date \
  --type DERIVED_DATE \
  --options derived.date.transformation.source.field=created_at \
  --options derived.date.transformation.source.input.format=epoch_millis \
  --options derived.date.transformation.new.field=created_date \
  --options derived.date.transformation.output.format=yyyy-MM-dd

Add current timestamp​

Add a new field populated with the current ingestion timestamp.

oh transformation create add-ingest-time \
  --type ADD_CURRENT_TIMESTAMP \
  --options current.timestamp.transformation.new.field=ingested_at \
  --options current.timestamp.transformation.output.format=yyyy-MM-dd HH:mm:ss

Parse JSON​

Parse JSON string fields into structured columns.

oh transformation create parse-payload \
  --type L_PARSE_JSON \
  --options l_parse_json.fields=payload,metadata \
  --options l_parse_json.field_schema=my_schema \
  --options l_parse_json.nested_fields=payload.event,payload.properties

Flatten struct​

Flatten nested Struct fields into their parent scope.

# Flatten the entire schema recursively (default)
oh transformation create flatten-all \
  --type FLATTEN_STRUCT

# Flatten all nested Structs under a specific path
oh transformation create flatten-home \
  --type FLATTEN_STRUCT \
  --options flatten_struct.operation_mode=recursive \
  --options flatten_struct.selected_column=home

# Flatten only the direct children of a specific Struct (one level)
oh transformation create flatten-address \
  --type FLATTEN_STRUCT \
  --options flatten_struct.operation_mode=selective \
  --options flatten_struct.selected_column=home.address

Explode array​

Explode Array fields into individual rows.

# Explode all top-level Arrays recursively (default)
oh transformation create explode-all \
  --type EXPLODE_ARRAY

# Explode a specific Array recursively
oh transformation create explode-scores \
  --type EXPLODE_ARRAY \
  --options explode_array.operation_mode=recursive \
  --options explode_array.selected_column=scores

# Explode only the specified Array, one level
oh transformation create explode-scores-shallow \
  --type EXPLODE_ARRAY \
  --options explode_array.operation_mode=selective \
  --options explode_array.selected_column=scores

Vector embeddings​

Generate vector embeddings from a text column using an external model.

oh transformation create embed-descriptions \
  --type VECTOR_EMBEDDING \
  --options vector.embedding.transformation.model.name=text-embedding-ada-002 \
  --options vector.embedding.transformation.selected.column=description \
  --options vector.embedding.transformation.new.embedding.column=description_embedding \
  --options vector.embedding.transformation.api.key=MY_API_KEY

Custom (user-provided) transformation​

Use a custom transformation class from a JAR uploaded via the Onehouse console.

oh transformation create my-custom \
  --type USER_PROVIDED \
  --options user.provided.transformation.class.name=com.example.MyTransformer \
  --options user.provided.transformation.properties=key1=value1,key2=value2

oh transformation delete NAME​

Deletes a transformation.

oh transformation delete mask-pii