Skip to main content

Parse JSON (Coming Soon)

Description​

Converts a JSON string to a Struct.

Parameters​

  • Fields to Parse: Specify a list of fields containing a JSON string to parse. All fields must share the same field schema.
  • Field Schema: Specify the schema of the JSON string. Only include the schema for the JSON string; exclude any other fields in the table.

Input Requirements​

Field(s) to parse must be string type and contain a JSON string.

Expected Output​

Changes each parsed field from a String to a Spark Struct.

Examples​

Example 1: Parsing nested JSON​

// Input
// Fields to Parse = "info"
// Field Schema =
//      |-- name: string (nullable = true)
//      |-- age: long (nullable = true)
//      |-- address: struct (nullable = true)
//      |    |-- city: string (nullable = true)
//      |    |-- state: string (nullable = true)

{
  "id": 123,
  "info": "{"name":"John","age":30,"address":{"city":"New York","state":"NY"}}"
}

// Output
{
  "id": 123,
  "info": {
    "name": "John",
    "age": 30,
    "address": {
        "city": "New York",
        "state": "NY"
    }
  }
}

Example 2: Parsing nested String​

// Input
// Fields to Parse = "info"
// Field Schema =
//      |-- name: string (nullable = true)
//      |-- age: long (nullable = true)
//      |-- address: string (nullable = true) 

{
  "id": 123,
  "info": "{"name":"John","age":30,"address":"{\\"city\\":\\"New York\\",\\"state\\":\\"NY\\"}" }"
}

// Output
{
  "id": 123,
  "info":
    {
      "name": "John",
      "age": 30,
      "address: "{\\"city\\":\\"New York\\",\\"state\\":\\"NY\\"}"
    }
}