Skip to main content

ALTER CLUSTER

Description​

Modify an existing Cluster.

Usage Tip: Only include the parameters you want to change. You must include at least 1 of the optional parameters.

Note that the SQL statement does not end with ;

Important considerations
  • Triggering a Job run is disabled while the Cluster is being modified.
  • Reducing the Max OCU of a Cluster may disrupt Jobs running on the Cluster if there are insufficient resources.

Syntax​

ALTER CLUSTER `<cluster_name>` SET
  [ STATE = { 'STOP' | 'START' } ]
  [ MAX_OCU = <int> ]
  [ MIN_OCU = <int> ]
  [ NEW_NAME = <string> ]
  WITH 'key1' = 'value1', 'key2' = 'value2' ....

Sample response

Examples​

Update max OCU, min OCU, and Cluster name:

ALTER CLUSTER `managed_cluster_retail` SET
  MAX_OCU = 30
  MIN_OCU = 5
  NEW_NAME = 'bigger_managed_cluster_retail'

Stop a running Cluster:

ALTER CLUSTER `managed_cluster_retail` SET
  STATE = 'STOP'

Update compute limits for Ray Open Engines Cluster:

ALTER CLUSTER `ray_cluster` SET
  MAX_OCU = 8
  MIN_OCU = 2
  WITH 'open_engines.ray.max_cpu_units' = '5', 'open_engines.ray.min_cpu_units' = '2', 'open_engines.ray.max_gpu_units' = '3'

Update Lakegres Cluster configuration:

ALTER CLUSTER `lakegres_cluster` SET
  WITH 'lakegres.engine_size' = 'large', 'lakegres.min_engines' = '2', 'lakegres.max_engines' = '10'

Required parameters​

  • <cluster_name>: Specify the name of the Cluster to modify.

Optional parameters​

Set state​

  • STATE: Start or stop the Cluster.

Set capacity​

  • MAX_OCU: Update the maximum OCU for the Cluster. May be set for all Cluster types except Lakegres.
  • MIN_OCU: Update the minimum OCU for the Cluster. May be set for all Cluster types except Lakegres.
  • NEW_NAME: Change the name of the Cluster.

Special parameters​

Include special parameters and advanced configs after WITH as type String.

Instance type parameters​

  • worker.type: Specify the worker instance type as a String. Must be a standard Onehouse instance type (eg. 'oh-general-4') or a custom instance type that's been enabled for the project (eg. 'm8g.xlarge'). View more on instance types.
  • worker.spot: Specify 'TRUE' or 'FALSE' to enable/disable spot instances for workers.
  • driver.type: Specify the driver instance type as a String. Default is 'Auto', which uses the same instance type as workers.

Quanton Engine parameters​

  • quanton.enabled: Specify 'true' or 'false' to enable/disable Quanton engine for the workloads in cluster.
    • This parameter is only applicable for SQL, Spark and Notebook Clusters.

Open Engines parameters​

  • open_engines.ray.max_cpu_units: Specify the maximum CPU units the Cluster can scale to. The max CPU Units and max GPU units must sum to the Max OCU you set above.
  • open_engines.ray.min_cpu_units: Specify the minimum CPU units the Cluster will always run. This must be the same value you set for the Min OCU above.
  • open_engines.ray.max_gpu_units: Specify the maximum GPU units the Cluster can scale to. The max CPU Units and max GPU units must sum to the Max OCU you set above.

Attached storage parameters​

For AWS projects only.

  • attached_storage.driver_storage_size: Optionally, specify a custom amount of attached storage (in GB) to provision for drivers in the Cluster. Select from any storage size in this table.
  • attached_storage.worker_storage_size: Optionally, specify a custom amount of attached storage (in GB) to provision for workers in the Cluster. Select from any storage size in this table.

Lakegres parameters​

Only allowed for the Lakegres cluster type.

  • lakegres.engine_size: Specify the engine size as one of: '2x-small', 'x-small', 'small', 'medium', 'large', 'x-large', '2x-large', '3x-large', '4x-large'. This determines the compute resources for each Spark cluster.
  • lakegres.min_engines: Specify the minimum number of Spark clusters that will always be running.
  • lakegres.max_engines: Specify the maximum number of Spark clusters the Cluster can scale to.
  • lakegres.query_result_cache_enabled: boolean ('true' / 'false'). When 'true', enables the query result cache for this Lakegres cluster, reducing latency for repeated deterministic queries. Default: 'false'. Takes effect on the next cluster restart.