AutoscalingPolicy

name string

The unique name of the resource.

args AutoscalingPolicyArgs

The arguments to resource properties.

opts CustomResourceOptions

Bag of options to control resource's behavior.

resource_name str

The unique name of the resource.

opts ResourceOptions

A bag of options that control this resource's behavior.

ctx Context

Context object for the current deployment.

name string

The unique name of the resource.

args AutoscalingPolicyArgs

The arguments to resource properties.

opts ResourceOption

Bag of options to control resource's behavior.

name string

The unique name of the resource.

args AutoscalingPolicyArgs

The arguments to resource properties.

opts CustomResourceOptions

Bag of options to control resource's behavior.

PolicyId string

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

BasicAlgorithm AutoscalingPolicyBasicAlgorithmArgs

Basic algorithm for autoscaling. Structure is documented below.

Location string

The location where the autoscaling poicy should reside. The default value is global.

Project string

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

SecondaryWorkerConfig AutoscalingPolicySecondaryWorkerConfigArgs

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

WorkerConfig AutoscalingPolicyWorkerConfigArgs

Describes how the autoscaler will operate for primary workers. Structure is documented below.

PolicyId string

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

BasicAlgorithm AutoscalingPolicyBasicAlgorithm

Basic algorithm for autoscaling. Structure is documented below.

Location string

The location where the autoscaling poicy should reside. The default value is global.

Project string

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

SecondaryWorkerConfig AutoscalingPolicySecondaryWorkerConfig

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

WorkerConfig AutoscalingPolicyWorkerConfig

Describes how the autoscaler will operate for primary workers. Structure is documented below.

policyId string

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

basicAlgorithm AutoscalingPolicyBasicAlgorithm

Basic algorithm for autoscaling. Structure is documented below.

location string

The location where the autoscaling poicy should reside. The default value is global.

project string

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

secondaryWorkerConfig AutoscalingPolicySecondaryWorkerConfig

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

workerConfig AutoscalingPolicyWorkerConfig

Describes how the autoscaler will operate for primary workers. Structure is documented below.

policy_id str

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

basic_algorithm Dict[AutoscalingPolicyBasicAlgorithm]

Basic algorithm for autoscaling. Structure is documented below.

location str

The location where the autoscaling poicy should reside. The default value is global.

project str

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

secondary_worker_config Dict[AutoscalingPolicySecondaryWorkerConfig]

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

worker_config Dict[AutoscalingPolicyWorkerConfig]

Describes how the autoscaler will operate for primary workers. Structure is documented below.

Id string

The provider-assigned unique ID for this managed resource.

Name string

The “resource name” of the autoscaling policy.

Id string

The provider-assigned unique ID for this managed resource.

Name string

The “resource name” of the autoscaling policy.

id string

The provider-assigned unique ID for this managed resource.

name string

The “resource name” of the autoscaling policy.

id str

The provider-assigned unique ID for this managed resource.

name str

The “resource name” of the autoscaling policy.

name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

state

Any extra arguments used during the lookup.

opts

A bag of options that control this resource's behavior.

resource_name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

state

Any extra arguments used during the lookup.

opts

A bag of options that control this resource's behavior.

name

The unique name of the resulting resource.

id

The unique provider ID of the resource to lookup.

state

Any extra arguments used during the lookup.

opts

A bag of options that control this resource's behavior.

BasicAlgorithm AutoscalingPolicyBasicAlgorithmArgs

Basic algorithm for autoscaling. Structure is documented below.

Location string

The location where the autoscaling poicy should reside. The default value is global.

Name string

The “resource name” of the autoscaling policy.

PolicyId string

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

Project string

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

SecondaryWorkerConfig AutoscalingPolicySecondaryWorkerConfigArgs

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

WorkerConfig AutoscalingPolicyWorkerConfigArgs

Describes how the autoscaler will operate for primary workers. Structure is documented below.

BasicAlgorithm AutoscalingPolicyBasicAlgorithm

Basic algorithm for autoscaling. Structure is documented below.

Location string

The location where the autoscaling poicy should reside. The default value is global.

Name string

The “resource name” of the autoscaling policy.

PolicyId string

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

Project string

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

SecondaryWorkerConfig AutoscalingPolicySecondaryWorkerConfig

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

WorkerConfig AutoscalingPolicyWorkerConfig

Describes how the autoscaler will operate for primary workers. Structure is documented below.

basicAlgorithm AutoscalingPolicyBasicAlgorithm

Basic algorithm for autoscaling. Structure is documented below.

location string

The location where the autoscaling poicy should reside. The default value is global.

name string

The “resource name” of the autoscaling policy.

policyId string

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

project string

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

secondaryWorkerConfig AutoscalingPolicySecondaryWorkerConfig

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

workerConfig AutoscalingPolicyWorkerConfig

Describes how the autoscaler will operate for primary workers. Structure is documented below.

basic_algorithm Dict[AutoscalingPolicyBasicAlgorithm]

Basic algorithm for autoscaling. Structure is documented below.

location str

The location where the autoscaling poicy should reside. The default value is global.

name str

The “resource name” of the autoscaling policy.

policy_id str

The policy id. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.

project str

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

secondary_worker_config Dict[AutoscalingPolicySecondaryWorkerConfig]

Describes how the autoscaler will operate for secondary workers. Structure is documented below.

worker_config Dict[AutoscalingPolicyWorkerConfig]

Describes how the autoscaler will operate for primary workers. Structure is documented below.

See the input and output API doc for this type.

See the input and output API doc for this type.

See the input and output API doc for this type.

YarnConfig AutoscalingPolicyBasicAlgorithmYarnConfigArgs

YARN autoscaling configuration. Structure is documented below.

CooldownPeriod string

Duration between scaling events. A scaling period starts after the update operation from the previous event has completed. Bounds: [2m, 1d]. Default: 2m.

YarnConfig AutoscalingPolicyBasicAlgorithmYarnConfig

YARN autoscaling configuration. Structure is documented below.

CooldownPeriod string

Duration between scaling events. A scaling period starts after the update operation from the previous event has completed. Bounds: [2m, 1d]. Default: 2m.

yarnConfig AutoscalingPolicyBasicAlgorithmYarnConfig

YARN autoscaling configuration. Structure is documented below.

cooldownPeriod string

Duration between scaling events. A scaling period starts after the update operation from the previous event has completed. Bounds: [2m, 1d]. Default: 2m.

yarnConfig Dict[AutoscalingPolicyBasicAlgorithmYarnConfig]

YARN autoscaling configuration. Structure is documented below.

cooldownPeriod str

Duration between scaling events. A scaling period starts after the update operation from the previous event has completed. Bounds: [2m, 1d]. Default: 2m.

See the input and output API doc for this type.

See the input and output API doc for this type.

See the input and output API doc for this type.

GracefulDecommissionTimeout string

Timeout for YARN graceful decommissioning of Node Managers. Specifies the duration to wait for jobs to complete before forcefully removing workers (and potentially interrupting jobs). Only applicable to downscaling operations. Bounds: [0s, 1d].

ScaleDownFactor double

Fraction of average pending memory in the last cooldown period for which to remove workers. A scale-down factor of 1 will result in scaling down so that there is no available memory remaining after the update (more aggressive scaling). A scale-down factor of 0 disables removing workers, which can be beneficial for autoscaling a single job. Bounds: [0.0, 1.0].

ScaleUpFactor double

Fraction of average pending memory in the last cooldown period for which to add workers. A scale-up factor of 1.0 will result in scaling up so that there is no pending memory remaining after the update (more aggressive scaling). A scale-up factor closer to 0 will result in a smaller magnitude of scaling up (less aggressive scaling). Bounds: [0.0, 1.0].

ScaleDownMinWorkerFraction double

Minimum scale-down threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2 worker scale-down for the cluster to scale. A threshold of 0 means the autoscaler will scale down on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

ScaleUpMinWorkerFraction double

Minimum scale-up threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2-worker scale-up for the cluster to scale. A threshold of 0 means the autoscaler will scale up on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

GracefulDecommissionTimeout string

Timeout for YARN graceful decommissioning of Node Managers. Specifies the duration to wait for jobs to complete before forcefully removing workers (and potentially interrupting jobs). Only applicable to downscaling operations. Bounds: [0s, 1d].

ScaleDownFactor float64

Fraction of average pending memory in the last cooldown period for which to remove workers. A scale-down factor of 1 will result in scaling down so that there is no available memory remaining after the update (more aggressive scaling). A scale-down factor of 0 disables removing workers, which can be beneficial for autoscaling a single job. Bounds: [0.0, 1.0].

ScaleUpFactor float64

Fraction of average pending memory in the last cooldown period for which to add workers. A scale-up factor of 1.0 will result in scaling up so that there is no pending memory remaining after the update (more aggressive scaling). A scale-up factor closer to 0 will result in a smaller magnitude of scaling up (less aggressive scaling). Bounds: [0.0, 1.0].

ScaleDownMinWorkerFraction float64

Minimum scale-down threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2 worker scale-down for the cluster to scale. A threshold of 0 means the autoscaler will scale down on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

ScaleUpMinWorkerFraction float64

Minimum scale-up threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2-worker scale-up for the cluster to scale. A threshold of 0 means the autoscaler will scale up on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

gracefulDecommissionTimeout string

Timeout for YARN graceful decommissioning of Node Managers. Specifies the duration to wait for jobs to complete before forcefully removing workers (and potentially interrupting jobs). Only applicable to downscaling operations. Bounds: [0s, 1d].

scaleDownFactor number

Fraction of average pending memory in the last cooldown period for which to remove workers. A scale-down factor of 1 will result in scaling down so that there is no available memory remaining after the update (more aggressive scaling). A scale-down factor of 0 disables removing workers, which can be beneficial for autoscaling a single job. Bounds: [0.0, 1.0].

scaleUpFactor number

Fraction of average pending memory in the last cooldown period for which to add workers. A scale-up factor of 1.0 will result in scaling up so that there is no pending memory remaining after the update (more aggressive scaling). A scale-up factor closer to 0 will result in a smaller magnitude of scaling up (less aggressive scaling). Bounds: [0.0, 1.0].

scaleDownMinWorkerFraction number

Minimum scale-down threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2 worker scale-down for the cluster to scale. A threshold of 0 means the autoscaler will scale down on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

scaleUpMinWorkerFraction number

Minimum scale-up threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2-worker scale-up for the cluster to scale. A threshold of 0 means the autoscaler will scale up on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

gracefulDecommissionTimeout str

Timeout for YARN graceful decommissioning of Node Managers. Specifies the duration to wait for jobs to complete before forcefully removing workers (and potentially interrupting jobs). Only applicable to downscaling operations. Bounds: [0s, 1d].

scaleDownFactor float

Fraction of average pending memory in the last cooldown period for which to remove workers. A scale-down factor of 1 will result in scaling down so that there is no available memory remaining after the update (more aggressive scaling). A scale-down factor of 0 disables removing workers, which can be beneficial for autoscaling a single job. Bounds: [0.0, 1.0].

scaleUpFactor float

Fraction of average pending memory in the last cooldown period for which to add workers. A scale-up factor of 1.0 will result in scaling up so that there is no pending memory remaining after the update (more aggressive scaling). A scale-up factor closer to 0 will result in a smaller magnitude of scaling up (less aggressive scaling). Bounds: [0.0, 1.0].

scaleDownMinWorkerFraction float

Minimum scale-down threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2 worker scale-down for the cluster to scale. A threshold of 0 means the autoscaler will scale down on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

scaleUpMinWorkerFraction float

Minimum scale-up threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2-worker scale-up for the cluster to scale. A threshold of 0 means the autoscaler will scale up on any recommended change. Bounds: [0.0, 1.0]. Default: 0.0.

See the input and output API doc for this type.

See the input and output API doc for this type.

See the input and output API doc for this type.

MaxInstances int

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

MinInstances int

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

Weight int

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.

MaxInstances int

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

MinInstances int

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

Weight int

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.

maxInstances number

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

minInstances number

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

weight number

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.

max_instances float

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

minInstances float

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

weight float

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.

See the input and output API doc for this type.

See the input and output API doc for this type.

See the input and output API doc for this type.

MaxInstances int

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

MinInstances int

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

Weight int

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.

MaxInstances int

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

MinInstances int

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

Weight int

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.

maxInstances number

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

minInstances number

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

weight number

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.

max_instances float

Maximum number of instances for this group. Note that by default, clusters will not use secondary workers. Required for secondary workers if the minimum secondary instances is set. Bounds: [minInstances, ). Defaults to 0.

minInstances float

Minimum number of instances for this group. Bounds: [0, maxInstances]. Defaults to 0.

weight float

Weight for the instance group, which is used to determine the fraction of total workers in the cluster from this instance group. For example, if primary workers have weight 2, and secondary workers have weight 1, the cluster will have approximately 2 primary workers for each secondary worker. The cluster may not reach the specified balance if constrained by min/max bounds or other autoscaling settings. For example, if maxInstances for secondary workers is 0, then only primary workers will be added. The cluster can also be out of balance when created. If weight is not set on any instance group, the cluster will default to equal weight for all groups: the cluster will attempt to maintain an equal number of workers in each group within the configured size bounds for each group. If weight is set for one group only, the cluster will default to zero weight on the unset group. For example if weight is set only on primary workers, the cluster will use primary workers only and no secondary workers.