Class Job

Provides a Glue Job resource.

Glue functionality, such as monitoring and logging of jobs, is typically managed with the default_arguments argument. See the Special Parameters Used by AWS Glue topic in the Glue developer guide for additional information.

Example Usage

Python Job

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
public MyStack()
{
    var example = new Aws.Glue.Job("example", new Aws.Glue.JobArgs
    {
        Command = new Aws.Glue.Inputs.JobCommandArgs
        {
            ScriptLocation = $"s3://{aws_s3_bucket.Example.Bucket}/example.py",
        },
        RoleArn = aws_iam_role.Example.Arn,
    });
}

}

Scala Job

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
public MyStack()
{
    var example = new Aws.Glue.Job("example", new Aws.Glue.JobArgs
    {
        Command = new Aws.Glue.Inputs.JobCommandArgs
        {
            ScriptLocation = $"s3://{aws_s3_bucket.Example.Bucket}/example.scala",
        },
        DefaultArguments = 
        {
            { "--job-language", "scala" },
        },
        RoleArn = aws_iam_role.Example.Arn,
    });
}

}

Enabling CloudWatch Logs and Metrics

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
public MyStack()
{
    var exampleLogGroup = new Aws.CloudWatch.LogGroup("exampleLogGroup", new Aws.CloudWatch.LogGroupArgs
    {
        RetentionInDays = 14,
    });
    var exampleJob = new Aws.Glue.Job("exampleJob", new Aws.Glue.JobArgs
    {
        DefaultArguments = 
        {
            { "--continuous-log-logGroup", exampleLogGroup.Name },
            { "--enable-continuous-cloudwatch-log", "true" },
            { "--enable-continuous-log-filter", "true" },
            { "--enable-metrics", "" },
        },
    });
}

}
Inheritance
System.Object
Resource
CustomResource
Job
Inherited Members
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: Pulumi.Aws.Glue
Assembly: Pulumi.Aws.dll
Syntax
public class Job : CustomResource

Constructors

View Source

Job(String, JobArgs, CustomResourceOptions)

Create a Job resource with the given unique name, arguments, and options.

Declaration
public Job(string name, JobArgs args, CustomResourceOptions options = null)
Parameters
Type Name Description
System.String name

The unique name of the resource
JobArgs args

The arguments used to populate this resource's properties
CustomResourceOptions options

A bag of options that control this resource's behavior

Properties

View Source

AllocatedCapacity

DEPRECATED (Optional) The number of AWS Glue data processing units (DPUs) to allocate to this Job. At least 2 DPUs need to be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory.

Declaration
public Output<int> AllocatedCapacity { get; }
Property Value
Type Description
Output<System.Int32>
View Source

Arn

Amazon Resource Name (ARN) of Glue Job

Declaration
public Output<string> Arn { get; }
Property Value
Type Description
Output<System.String>
View Source

Command

The command of the job. Defined below.

Declaration
public Output<JobCommand> Command { get; }
Property Value
Type Description
Output<JobCommand>
View Source

Connections

The list of connections used for this job.

Declaration
public Output<ImmutableArray<string>> Connections { get; }
Property Value
Type Description
Output<System.Collections.Immutable.ImmutableArray<System.String>>
View Source

DefaultArguments

The map of default arguments for this job. You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes. For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide. For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.

Declaration
public Output<ImmutableDictionary<string, object>> DefaultArguments { get; }
Property Value
Type Description
Output<System.Collections.Immutable.ImmutableDictionary<System.String, System.Object>>
View Source

Description

Description of the job.

Declaration
public Output<string> Description { get; }
Property Value
Type Description
Output<System.String>
View Source

ExecutionProperty

Execution property of the job. Defined below.

Declaration
public Output<JobExecutionProperty> ExecutionProperty { get; }
Property Value
Type Description
Output<JobExecutionProperty>
View Source

GlueVersion

The version of glue to use, for example "1.0". For information about available versions, see the AWS Glue Release Notes.

Declaration
public Output<string> GlueVersion { get; }
Property Value
Type Description
Output<System.String>
View Source

MaxCapacity

The maximum number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. Required when pythonshell is set, accept either 0.0625 or 1.0.

Declaration
public Output<double> MaxCapacity { get; }
Property Value
Type Description
Output<System.Double>
View Source

MaxRetries

The maximum number of times to retry this job if it fails.

Declaration
public Output<int?> MaxRetries { get; }
Property Value
Type Description
Output<System.Nullable<System.Int32>>
View Source

Name

The name you assign to this job. It must be unique in your account.

Declaration
public Output<string> Name { get; }
Property Value
Type Description
Output<System.String>
View Source

NotificationProperty

Notification property of the job. Defined below.

Declaration
public Output<JobNotificationProperty> NotificationProperty { get; }
Property Value
Type Description
Output<JobNotificationProperty>
View Source

NumberOfWorkers

The number of workers of a defined workerType that are allocated when a job runs.

Declaration
public Output<int?> NumberOfWorkers { get; }
Property Value
Type Description
Output<System.Nullable<System.Int32>>
View Source

RoleArn

The ARN of the IAM role associated with this job.

Declaration
public Output<string> RoleArn { get; }
Property Value
Type Description
Output<System.String>
View Source

SecurityConfiguration

The name of the Security Configuration to be associated with the job.

Declaration
public Output<string> SecurityConfiguration { get; }
Property Value
Type Description
Output<System.String>
View Source

Tags

Key-value map of resource tags

Declaration
public Output<ImmutableDictionary<string, object>> Tags { get; }
Property Value
Type Description
Output<System.Collections.Immutable.ImmutableDictionary<System.String, System.Object>>
View Source

Timeout

The job timeout in minutes. The default is 2880 minutes (48 hours).

Declaration
public Output<int?> Timeout { get; }
Property Value
Type Description
Output<System.Nullable<System.Int32>>
View Source

WorkerType

The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.

Declaration
public Output<string> WorkerType { get; }
Property Value
Type Description
Output<System.String>

Methods

View Source

Get(String, Input<String>, JobState, CustomResourceOptions)

Get an existing Job resource's state with the given name, ID, and optional extra properties used to qualify the lookup.

Declaration
public static Job Get(string name, Input<string> id, JobState state = null, CustomResourceOptions options = null)
Parameters
Type Name Description
System.String name

The unique name of the resulting resource.
Input<System.String> id

The unique provider ID of the resource to lookup.
JobState state

Any extra arguments used during the lookup.
CustomResourceOptions options

A bag of options that control this resource's behavior
Returns
Type Description
Job