Class Environment

An environment for running orchestration tasks.

Environments run Apache Airflow software on Google infrastructure.

To get more information about Environments, see:

Warning: We STRONGLY recommend you read the GCP guides as the Environment resource requires a long deployment process and involves several layers of GCP infrastructure, including a Kubernetes Engine cluster, Cloud Storage, and Compute networking resources. Due to limitations of the API, This provider will not be able to automatically find or manage many of these underlying resources. In particular:

  • It can take up to one hour to create or update an environment resource. In addition, GCP may only detect some errors in configuration when they are used (e.g. ~40-50 minutes into the creation process), and is prone to limited error reporting. If you encounter confusing or uninformative errors, please verify your configuration is valid against GCP Cloud Composer before filing bugs against this provider.
  • Environments create Google Cloud Storage buckets that do not get cleaned up automatically on environment deletion. More about Composer's use of Cloud Storage.

Example Usage

Basic Usage

using Pulumi;
using Gcp = Pulumi.Gcp;

class MyStack : Stack
{
public MyStack()
{
    var test = new Gcp.Composer.Environment("test", new Gcp.Composer.EnvironmentArgs
    {
        Region = "us-central1",
    });
}

}

With GKE and Compute Resource Dependencies

using Pulumi;
using Gcp = Pulumi.Gcp;

class MyStack : Stack
{
public MyStack()
{
    var testNetwork = new Gcp.Compute.Network("testNetwork", new Gcp.Compute.NetworkArgs
    {
        AutoCreateSubnetworks = false,
    });
    var testSubnetwork = new Gcp.Compute.Subnetwork("testSubnetwork", new Gcp.Compute.SubnetworkArgs
    {
        IpCidrRange = "10.2.0.0/16",
        Region = "us-central1",
        Network = testNetwork.Id,
    });
    var testAccount = new Gcp.ServiceAccount.Account("testAccount", new Gcp.ServiceAccount.AccountArgs
    {
        AccountId = "composer-env-account",
        DisplayName = "Test Service Account for Composer Environment",
    });
    var composer_worker = new Gcp.Projects.IAMMember("composer-worker", new Gcp.Projects.IAMMemberArgs
    {
        Role = "roles/composer.worker",
        Member = testAccount.Email.Apply(email => $"serviceAccount:{email}"),
    });
    var testEnvironment = new Gcp.Composer.Environment("testEnvironment", new Gcp.Composer.EnvironmentArgs
    {
        Region = "us-central1",
        Config = new Gcp.Composer.Inputs.EnvironmentConfigArgs
        {
            NodeCount = 4,
            Node_config = 
            {
                { "zone", "us-central1-a" },
                { "machineType", "n1-standard-1" },
                { "network", testNetwork.Id },
                { "subnetwork", testSubnetwork.Id },
                { "serviceAccount", testAccount.Name },
            },
        },
    });
}

}

With Software (Airflow) Config

using Pulumi;
using Gcp = Pulumi.Gcp;

class MyStack : Stack
{
public MyStack()
{
    var test = new Gcp.Composer.Environment("test", new Gcp.Composer.EnvironmentArgs
    {
        Config = new Gcp.Composer.Inputs.EnvironmentConfigArgs
        {
            SoftwareConfig = new Gcp.Composer.Inputs.EnvironmentConfigSoftwareConfigArgs
            {
                AirflowConfigOverrides = 
                {
                    { "core-loadExample", "True" },
                },
                EnvVariables = 
                {
                    { "FOO", "bar" },
                },
                PypiPackages = 
                {
                    { "numpy", "" },
                    { "scipy", "==1.1.0" },
                },
            },
        },
        Region = "us-central1",
    });
}

}
Inheritance
System.Object
Resource
CustomResource
Environment
Inherited Members
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: Pulumi.Gcp.Composer
Assembly: Pulumi.Gcp.dll
Syntax
public class Environment : CustomResource

Constructors

View Source

Environment(String, EnvironmentArgs, CustomResourceOptions)

Create a Environment resource with the given unique name, arguments, and options.

Declaration
public Environment(string name, EnvironmentArgs args = null, CustomResourceOptions options = null)
Parameters
Type Name Description
System.String name

The unique name of the resource
EnvironmentArgs args

The arguments used to populate this resource's properties
CustomResourceOptions options

A bag of options that control this resource's behavior

Properties

View Source

Config

Configuration parameters for this environment Structure is documented below.

Declaration
public Output<EnvironmentConfig> Config { get; }
Property Value
Type Description
Output<EnvironmentConfig>
View Source

Labels

User-defined labels for this environment. The labels map can contain no more than 64 entries. Entries of the labels map are UTF8 strings that comply with the following restrictions: Label keys must be between 1 and 63 characters long and must conform to the following regular expression: a-z?. Label values must be between 0 and 63 characters long and must conform to the regular expression (a-z?)?. No more than 64 labels can be associated with a given environment. Both keys and values must be <= 128 bytes in size.

Declaration
public Output<ImmutableDictionary<string, string>> Labels { get; }
Property Value
Type Description
Output<System.Collections.Immutable.ImmutableDictionary<System.String, System.String>>
View Source

Name

Name of the environment

Declaration
public Output<string> Name { get; }
Property Value
Type Description
Output<System.String>
View Source

Project

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

Declaration
public Output<string> Project { get; }
Property Value
Type Description
Output<System.String>
View Source

Region

The location or Compute Engine region for the environment.

Declaration
public Output<string> Region { get; }
Property Value
Type Description
Output<System.String>

Methods

View Source

Get(String, Input<String>, EnvironmentState, CustomResourceOptions)

Get an existing Environment resource's state with the given name, ID, and optional extra properties used to qualify the lookup.

Declaration
public static Environment Get(string name, Input<string> id, EnvironmentState state = null, CustomResourceOptions options = null)
Parameters
Type Name Description
System.String name

The unique name of the resulting resource.
Input<System.String> id

The unique provider ID of the resource to lookup.
EnvironmentState state

Any extra arguments used during the lookup.
CustomResourceOptions options

A bag of options that control this resource's behavior
Returns
Type Description
Environment