FirehoseDeliveryStream

Provides a Kinesis Firehose Delivery Stream resource. Amazon Kinesis Firehose is a fully managed, elastic service to easily deliver real-time data streams to destinations such as Amazon S3 and Amazon Redshift.

For more details, see the Amazon Kinesis Firehose Documentation.

Example Usage

Extended S3 Destination

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
    public MyStack()
    {
        var bucket = new Aws.S3.Bucket("bucket", new Aws.S3.BucketArgs
        {
            Acl = "private",
        });
        var firehoseRole = new Aws.Iam.Role("firehoseRole", new Aws.Iam.RoleArgs
        {
            AssumeRolePolicy = @"{
  ""Version"": ""2012-10-17"",
  ""Statement"": [
    {
      ""Action"": ""sts:AssumeRole"",
      ""Principal"": {
        ""Service"": ""firehose.amazonaws.com""
      },
      ""Effect"": ""Allow"",
      ""Sid"": """"
    }
  ]
}

",
        });
        var lambdaIam = new Aws.Iam.Role("lambdaIam", new Aws.Iam.RoleArgs
        {
            AssumeRolePolicy = @"{
  ""Version"": ""2012-10-17"",
  ""Statement"": [
    {
      ""Action"": ""sts:AssumeRole"",
      ""Principal"": {
        ""Service"": ""lambda.amazonaws.com""
      },
      ""Effect"": ""Allow"",
      ""Sid"": """"
    }
  ]
}

",
        });
        var lambdaProcessor = new Aws.Lambda.Function("lambdaProcessor", new Aws.Lambda.FunctionArgs
        {
            Code = new FileArchive("lambda.zip"),
            Handler = "exports.handler",
            Role = lambdaIam.Arn,
            Runtime = "nodejs8.10",
        });
        var extendedS3Stream = new Aws.Kinesis.FirehoseDeliveryStream("extendedS3Stream", new Aws.Kinesis.FirehoseDeliveryStreamArgs
        {
            Destination = "extended_s3",
            ExtendedS3Configuration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamExtendedS3ConfigurationArgs
            {
                BucketArn = bucket.Arn,
                ProcessingConfiguration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationArgs
                {
                    Enabled = true,
                    Processors = 
                    {
                        new Aws.Kinesis.Inputs.FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorArgs
                        {
                            Parameters = 
                            {
                                new Aws.Kinesis.Inputs.FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorParameterArgs
                                {
                                    ParameterName = "LambdaArn",
                                    ParameterValue = lambdaProcessor.Arn.Apply(arn => $"{arn}:$LATEST"),
                                },
                            },
                            Type = "Lambda",
                        },
                    },
                },
                RoleArn = firehoseRole.Arn,
            },
        });
    }

}

Coming soon!

import pulumi
import pulumi_aws as aws

bucket = aws.s3.Bucket("bucket", acl="private")
firehose_role = aws.iam.Role("firehoseRole", assume_role_policy="""{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}

""")
lambda_iam = aws.iam.Role("lambdaIam", assume_role_policy="""{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}

""")
lambda_processor = aws.lambda_.Function("lambdaProcessor",
    code=pulumi.FileArchive("lambda.zip"),
    handler="exports.handler",
    role=lambda_iam.arn,
    runtime="nodejs8.10")
extended_s3_stream = aws.kinesis.FirehoseDeliveryStream("extendedS3Stream",
    destination="extended_s3",
    extended_s3_configuration={
        "bucketArn": bucket.arn,
        "processingConfiguration": {
            "enabled": "true",
            "processors": [{
                "parameters": [{
                    "parameterName": "LambdaArn",
                    "parameterValue": lambda_processor.arn.apply(lambda arn: f"{arn}:$LATEST"),
                }],
                "type": "Lambda",
            }],
        },
        "role_arn": firehose_role.arn,
    })

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("bucket", {
    acl: "private",
});
const firehoseRole = new aws.iam.Role("firehose_role", {
    assumeRolePolicy: `{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
`,
});
const lambdaIam = new aws.iam.Role("lambda_iam", {
    assumeRolePolicy: `{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
`,
});
const lambdaProcessor = new aws.lambda.Function("lambda_processor", {
    code: new pulumi.asset.FileArchive("lambda.zip"),
    handler: "exports.handler",
    role: lambdaIam.arn,
    runtime: "nodejs8.10",
});
const extendedS3Stream = new aws.kinesis.FirehoseDeliveryStream("extended_s3_stream", {
    destination: "extended_s3",
    extendedS3Configuration: {
        bucketArn: bucket.arn,
        processingConfiguration: {
            enabled: true,
            processors: [{
                parameters: [{
                    parameterName: "LambdaArn",
                    parameterValue: pulumi.interpolate`${lambdaProcessor.arn}:$LATEST`,
                }],
                type: "Lambda",
            }],
        },
        roleArn: firehoseRole.arn,
    },
});

S3 Destination

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
    public MyStack()
    {
        var bucket = new Aws.S3.Bucket("bucket", new Aws.S3.BucketArgs
        {
            Acl = "private",
        });
        var firehoseRole = new Aws.Iam.Role("firehoseRole", new Aws.Iam.RoleArgs
        {
            AssumeRolePolicy = @"{
  ""Version"": ""2012-10-17"",
  ""Statement"": [
    {
      ""Action"": ""sts:AssumeRole"",
      ""Principal"": {
        ""Service"": ""firehose.amazonaws.com""
      },
      ""Effect"": ""Allow"",
      ""Sid"": """"
    }
  ]
}

",
        });
        var testStream = new Aws.Kinesis.FirehoseDeliveryStream("testStream", new Aws.Kinesis.FirehoseDeliveryStreamArgs
        {
            Destination = "s3",
            S3Configuration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamS3ConfigurationArgs
            {
                BucketArn = bucket.Arn,
                RoleArn = firehoseRole.Arn,
            },
        });
    }

}

package main

import (
    "fmt"

    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/iam"
    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/kinesis"
    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/s3"
    "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        bucket, err := s3.NewBucket(ctx, "bucket", &s3.BucketArgs{
            Acl: pulumi.String("private"),
        })
        if err != nil {
            return err
        }
        firehoseRole, err := iam.NewRole(ctx, "firehoseRole", &iam.RoleArgs{
            AssumeRolePolicy: pulumi.String(fmt.Sprintf("%v%v%v%v%v%v%v%v%v%v%v%v%v%v", "{\n", "  \"Version\": \"2012-10-17\",\n", "  \"Statement\": [\n", "    {\n", "      \"Action\": \"sts:AssumeRole\",\n", "      \"Principal\": {\n", "        \"Service\": \"firehose.amazonaws.com\"\n", "      },\n", "      \"Effect\": \"Allow\",\n", "      \"Sid\": \"\"\n", "    }\n", "  ]\n", "}\n", "\n")),
        })
        if err != nil {
            return err
        }
        _, err = kinesis.NewFirehoseDeliveryStream(ctx, "testStream", &kinesis.FirehoseDeliveryStreamArgs{
            Destination: pulumi.String("s3"),
            S3Configuration: &kinesis.FirehoseDeliveryStreamS3ConfigurationArgs{
                BucketArn: bucket.Arn,
                RoleArn:   firehoseRole.Arn,
            },
        })
        if err != nil {
            return err
        }
        return nil
    })
}

import pulumi
import pulumi_aws as aws

bucket = aws.s3.Bucket("bucket", acl="private")
firehose_role = aws.iam.Role("firehoseRole", assume_role_policy="""{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}

""")
test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="s3",
    s3_configuration={
        "bucketArn": bucket.arn,
        "role_arn": firehose_role.arn,
    })

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("bucket", {
    acl: "private",
});
const firehoseRole = new aws.iam.Role("firehose_role", {
    assumeRolePolicy: `{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
`,
});
const testStream = new aws.kinesis.FirehoseDeliveryStream("test_stream", {
    destination: "s3",
    s3Configuration: {
        bucketArn: bucket.arn,
        roleArn: firehoseRole.arn,
    },
});

Redshift Destination

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
    public MyStack()
    {
        var testCluster = new Aws.RedShift.Cluster("testCluster", new Aws.RedShift.ClusterArgs
        {
            ClusterIdentifier = "tf-redshift-cluster-%d",
            ClusterType = "single-node",
            DatabaseName = "test",
            MasterPassword = "T3stPass",
            MasterUsername = "testuser",
            NodeType = "dc1.large",
        });
        var testStream = new Aws.Kinesis.FirehoseDeliveryStream("testStream", new Aws.Kinesis.FirehoseDeliveryStreamArgs
        {
            Destination = "redshift",
            RedshiftConfiguration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamRedshiftConfigurationArgs
            {
                ClusterJdbcurl = Output.Tuple(testCluster.Endpoint, testCluster.DatabaseName).Apply(values =>
                {
                    var endpoint = values.Item1;
                    var databaseName = values.Item2;
                    return $"jdbc:redshift://{endpoint}/{databaseName}";
                }),
                CopyOptions = "delimiter '|'",
                DataTableColumns = "test-col",
                DataTableName = "test-table",
                Password = "T3stPass",
                RoleArn = aws_iam_role.Firehose_role.Arn,
                S3BackupConfiguration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationArgs
                {
                    BucketArn = aws_s3_bucket.Bucket.Arn,
                    BufferInterval = 300,
                    BufferSize = 15,
                    CompressionFormat = "GZIP",
                    RoleArn = aws_iam_role.Firehose_role.Arn,
                },
                S3BackupMode = "Enabled",
                Username = "testuser",
            },
            S3Configuration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamS3ConfigurationArgs
            {
                BucketArn = aws_s3_bucket.Bucket.Arn,
                BufferInterval = 400,
                BufferSize = 10,
                CompressionFormat = "GZIP",
                RoleArn = aws_iam_role.Firehose_role.Arn,
            },
        });
    }

}

package main

import (
    "fmt"

    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/kinesis"
    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/redshift"
    "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        testCluster, err := redshift.NewCluster(ctx, "testCluster", &redshift.ClusterArgs{
            ClusterIdentifier: pulumi.String(fmt.Sprintf("%v%v%v", "tf-redshift-cluster-", "%", "d")),
            ClusterType:       pulumi.String("single-node"),
            DatabaseName:      pulumi.String("test"),
            MasterPassword:    pulumi.String("T3stPass"),
            MasterUsername:    pulumi.String("testuser"),
            NodeType:          pulumi.String("dc1.large"),
        })
        if err != nil {
            return err
        }
        _, err = kinesis.NewFirehoseDeliveryStream(ctx, "testStream", &kinesis.FirehoseDeliveryStreamArgs{
            Destination: pulumi.String("redshift"),
            RedshiftConfiguration: &kinesis.FirehoseDeliveryStreamRedshiftConfigurationArgs{
                ClusterJdbcurl: pulumi.All(testCluster.Endpoint, testCluster.DatabaseName).ApplyT(func(_args []interface{}) (string, error) {
                    endpoint := _args[0].(string)
                    databaseName := _args[1].(string)
                    return fmt.Sprintf("%v%v%v%v", "jdbc:redshift://", endpoint, "/", databaseName), nil
                }).(pulumi.StringOutput),
                CopyOptions:      pulumi.String("delimiter '|'"),
                DataTableColumns: pulumi.String("test-col"),
                DataTableName:    pulumi.String("test-table"),
                Password:         pulumi.String("T3stPass"),
                RoleArn:          pulumi.String(aws_iam_role.Firehose_role.Arn),
                S3BackupConfiguration: &kinesis.FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationArgs{
                    BucketArn:         pulumi.String(aws_s3_bucket.Bucket.Arn),
                    BufferInterval:    pulumi.Int(300),
                    BufferSize:        pulumi.Int(15),
                    CompressionFormat: pulumi.String("GZIP"),
                    RoleArn:           pulumi.String(aws_iam_role.Firehose_role.Arn),
                },
                S3BackupMode: pulumi.String("Enabled"),
                Username:     pulumi.String("testuser"),
            },
            S3Configuration: &kinesis.FirehoseDeliveryStreamS3ConfigurationArgs{
                BucketArn:         pulumi.String(aws_s3_bucket.Bucket.Arn),
                BufferInterval:    pulumi.Int(400),
                BufferSize:        pulumi.Int(10),
                CompressionFormat: pulumi.String("GZIP"),
                RoleArn:           pulumi.String(aws_iam_role.Firehose_role.Arn),
            },
        })
        if err != nil {
            return err
        }
        return nil
    })
}

import pulumi
import pulumi_aws as aws

test_cluster = aws.redshift.Cluster("testCluster",
    cluster_identifier="tf-redshift-cluster-%d",
    cluster_type="single-node",
    database_name="test",
    master_password="T3stPass",
    master_username="testuser",
    node_type="dc1.large")
test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="redshift",
    redshift_configuration={
        "clusterJdbcurl": pulumi.Output.all(test_cluster.endpoint, test_cluster.database_name).apply(lambda endpoint, database_name: f"jdbc:redshift://{endpoint}/{database_name}"),
        "copyOptions": "delimiter '|'",
        "dataTableColumns": "test-col",
        "dataTableName": "test-table",
        "password": "T3stPass",
        "role_arn": aws_iam_role["firehose_role"]["arn"],
        "s3BackupConfiguration": {
            "bucketArn": aws_s3_bucket["bucket"]["arn"],
            "bufferInterval": 300,
            "bufferSize": 15,
            "compressionFormat": "GZIP",
            "role_arn": aws_iam_role["firehose_role"]["arn"],
        },
        "s3BackupMode": "Enabled",
        "username": "testuser",
    },
    s3_configuration={
        "bucketArn": aws_s3_bucket["bucket"]["arn"],
        "bufferInterval": 400,
        "bufferSize": 10,
        "compressionFormat": "GZIP",
        "role_arn": aws_iam_role["firehose_role"]["arn"],
    })

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const testCluster = new aws.redshift.Cluster("test_cluster", {
    clusterIdentifier: "tf-redshift-cluster-%d",
    clusterType: "single-node",
    databaseName: "test",
    masterPassword: "T3stPass",
    masterUsername: "testuser",
    nodeType: "dc1.large",
});
const testStream = new aws.kinesis.FirehoseDeliveryStream("test_stream", {
    destination: "redshift",
    redshiftConfiguration: {
        clusterJdbcurl: pulumi.interpolate`jdbc:redshift://${testCluster.endpoint}/${testCluster.databaseName}`,
        copyOptions: "delimiter '|'", // the default delimiter
        dataTableColumns: "test-col",
        dataTableName: "test-table",
        password: "T3stPass",
        roleArn: aws_iam_role_firehose_role.arn,
        s3BackupConfiguration: {
            bucketArn: aws_s3_bucket_bucket.arn,
            bufferInterval: 300,
            bufferSize: 15,
            compressionFormat: "GZIP",
            roleArn: aws_iam_role_firehose_role.arn,
        },
        s3BackupMode: "Enabled",
        username: "testuser",
    },
    s3Configuration: {
        bucketArn: aws_s3_bucket_bucket.arn,
        bufferInterval: 400,
        bufferSize: 10,
        compressionFormat: "GZIP",
        roleArn: aws_iam_role_firehose_role.arn,
    },
});

Elasticsearch Destination

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
    public MyStack()
    {
        var testCluster = new Aws.ElasticSearch.Domain("testCluster", new Aws.ElasticSearch.DomainArgs
        {
        });
        var testStream = new Aws.Kinesis.FirehoseDeliveryStream("testStream", new Aws.Kinesis.FirehoseDeliveryStreamArgs
        {
            Destination = "elasticsearch",
            ElasticsearchConfiguration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamElasticsearchConfigurationArgs
            {
                DomainArn = testCluster.Arn,
                IndexName = "test",
                ProcessingConfiguration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationArgs
                {
                    Enabled = true,
                    Processors = 
                    {
                        new Aws.Kinesis.Inputs.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorArgs
                        {
                            Parameters = 
                            {
                                new Aws.Kinesis.Inputs.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameterArgs
                                {
                                    ParameterName = "LambdaArn",
                                    ParameterValue = $"{aws_lambda_function.Lambda_processor.Arn}:$LATEST",
                                },
                            },
                            Type = "Lambda",
                        },
                    },
                },
                RoleArn = aws_iam_role.Firehose_role.Arn,
                TypeName = "test",
            },
            S3Configuration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamS3ConfigurationArgs
            {
                BucketArn = aws_s3_bucket.Bucket.Arn,
                BufferInterval = 400,
                BufferSize = 10,
                CompressionFormat = "GZIP",
                RoleArn = aws_iam_role.Firehose_role.Arn,
            },
        });
    }

}

package main

import (
    "fmt"

    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/elasticsearch"
    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/kinesis"
    "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        testCluster, err := elasticsearch.NewDomain(ctx, "testCluster", nil)
        if err != nil {
            return err
        }
        _, err = kinesis.NewFirehoseDeliveryStream(ctx, "testStream", &kinesis.FirehoseDeliveryStreamArgs{
            Destination: pulumi.String("elasticsearch"),
            ElasticsearchConfiguration: &kinesis.FirehoseDeliveryStreamElasticsearchConfigurationArgs{
                DomainArn: testCluster.Arn,
                IndexName: pulumi.String("test"),
                ProcessingConfiguration: &kinesis.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationArgs{
                    Enabled: pulumi.Bool(true),
                    Processors: kinesis.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorArray{
                        &kinesis.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorArgs{
                            Parameters: kinesis.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameterArray{
                                &kinesis.FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameterArgs{
                                    ParameterName:  pulumi.String("LambdaArn"),
                                    ParameterValue: pulumi.String(fmt.Sprintf("%v%v%v%v", aws_lambda_function.Lambda_processor.Arn, ":", "$", "LATEST")),
                                },
                            },
                            Type: pulumi.String("Lambda"),
                        },
                    },
                },
                RoleArn:  pulumi.String(aws_iam_role.Firehose_role.Arn),
                TypeName: pulumi.String("test"),
            },
            S3Configuration: &kinesis.FirehoseDeliveryStreamS3ConfigurationArgs{
                BucketArn:         pulumi.String(aws_s3_bucket.Bucket.Arn),
                BufferInterval:    pulumi.Int(400),
                BufferSize:        pulumi.Int(10),
                CompressionFormat: pulumi.String("GZIP"),
                RoleArn:           pulumi.String(aws_iam_role.Firehose_role.Arn),
            },
        })
        if err != nil {
            return err
        }
        return nil
    })
}

import pulumi
import pulumi_aws as aws

test_cluster = aws.elasticsearch.Domain("testCluster")
test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="elasticsearch",
    elasticsearch_configuration={
        "domainArn": test_cluster.arn,
        "indexName": "test",
        "processingConfiguration": {
            "enabled": "true",
            "processors": [{
                "parameters": [{
                    "parameterName": "LambdaArn",
                    "parameterValue": f"{aws_lambda_function['lambda_processor']['arn']}:$LATEST",
                }],
                "type": "Lambda",
            }],
        },
        "role_arn": aws_iam_role["firehose_role"]["arn"],
        "typeName": "test",
    },
    s3_configuration={
        "bucketArn": aws_s3_bucket["bucket"]["arn"],
        "bufferInterval": 400,
        "bufferSize": 10,
        "compressionFormat": "GZIP",
        "role_arn": aws_iam_role["firehose_role"]["arn"],
    })

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const testCluster = new aws.elasticsearch.Domain("test_cluster", {});
const testStream = new aws.kinesis.FirehoseDeliveryStream("test_stream", {
    destination: "elasticsearch",
    elasticsearchConfiguration: {
        domainArn: testCluster.arn,
        indexName: "test",
        processingConfiguration: {
            enabled: true,
            processors: [{
                parameters: [{
                    parameterName: "LambdaArn",
                    parameterValue: pulumi.interpolate`${aws_lambda_function_lambda_processor.arn}:$LATEST`,
                }],
                type: "Lambda",
            }],
        },
        roleArn: aws_iam_role_firehose_role.arn,
        typeName: "test",
    },
    s3Configuration: {
        bucketArn: aws_s3_bucket_bucket.arn,
        bufferInterval: 400,
        bufferSize: 10,
        compressionFormat: "GZIP",
        roleArn: aws_iam_role_firehose_role.arn,
    },
});

Splunk Destination

using Pulumi;
using Aws = Pulumi.Aws;

class MyStack : Stack
{
    public MyStack()
    {
        var testStream = new Aws.Kinesis.FirehoseDeliveryStream("testStream", new Aws.Kinesis.FirehoseDeliveryStreamArgs
        {
            Destination = "splunk",
            S3Configuration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamS3ConfigurationArgs
            {
                BucketArn = aws_s3_bucket.Bucket.Arn,
                BufferInterval = 400,
                BufferSize = 10,
                CompressionFormat = "GZIP",
                RoleArn = aws_iam_role.Firehose.Arn,
            },
            SplunkConfiguration = new Aws.Kinesis.Inputs.FirehoseDeliveryStreamSplunkConfigurationArgs
            {
                HecAcknowledgmentTimeout = 600,
                HecEndpoint = "https://http-inputs-mydomain.splunkcloud.com:443",
                HecEndpointType = "Event",
                HecToken = "51D4DA16-C61B-4F5F-8EC7-ED4301342A4A",
                S3BackupMode = "FailedEventsOnly",
            },
        });
    }

}

package main

import (
    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/kinesis"
    "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        _, err := kinesis.NewFirehoseDeliveryStream(ctx, "testStream", &kinesis.FirehoseDeliveryStreamArgs{
            Destination: pulumi.String("splunk"),
            S3Configuration: &kinesis.FirehoseDeliveryStreamS3ConfigurationArgs{
                BucketArn:         pulumi.String(aws_s3_bucket.Bucket.Arn),
                BufferInterval:    pulumi.Int(400),
                BufferSize:        pulumi.Int(10),
                CompressionFormat: pulumi.String("GZIP"),
                RoleArn:           pulumi.String(aws_iam_role.Firehose.Arn),
            },
            SplunkConfiguration: &kinesis.FirehoseDeliveryStreamSplunkConfigurationArgs{
                HecAcknowledgmentTimeout: pulumi.Int(600),
                HecEndpoint:              pulumi.String("https://http-inputs-mydomain.splunkcloud.com:443"),
                HecEndpointType:          pulumi.String("Event"),
                HecToken:                 pulumi.String("51D4DA16-C61B-4F5F-8EC7-ED4301342A4A"),
                S3BackupMode:             pulumi.String("FailedEventsOnly"),
            },
        })
        if err != nil {
            return err
        }
        return nil
    })
}

import pulumi
import pulumi_aws as aws

test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="splunk",
    s3_configuration={
        "bucketArn": aws_s3_bucket["bucket"]["arn"],
        "bufferInterval": 400,
        "bufferSize": 10,
        "compressionFormat": "GZIP",
        "role_arn": aws_iam_role["firehose"]["arn"],
    },
    splunk_configuration={
        "hecAcknowledgmentTimeout": 600,
        "hecEndpoint": "https://http-inputs-mydomain.splunkcloud.com:443",
        "hecEndpointType": "Event",
        "hecToken": "51D4DA16-C61B-4F5F-8EC7-ED4301342A4A",
        "s3BackupMode": "FailedEventsOnly",
    })

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const testStream = new aws.kinesis.FirehoseDeliveryStream("test_stream", {
    destination: "splunk",
    s3Configuration: {
        bucketArn: aws_s3_bucket_bucket.arn,
        bufferInterval: 400,
        bufferSize: 10,
        compressionFormat: "GZIP",
        roleArn: aws_iam_role_firehose.arn,
    },
    splunkConfiguration: {
        hecAcknowledgmentTimeout: 600,
        hecEndpoint: "https://http-inputs-mydomain.splunkcloud.com:443",
        hecEndpointType: "Event",
        hecToken: "51D4DA16-C61B-4F5F-8EC7-ED4301342A4A",
        s3BackupMode: "FailedEventsOnly",
    },
});

Create a FirehoseDeliveryStream Resource

new FirehoseDeliveryStream(name: string, args: FirehoseDeliveryStreamArgs, opts?: CustomResourceOptions);

def FirehoseDeliveryStream(resource_name, opts=None, arn=None, destination=None, destination_id=None, elasticsearch_configuration=None, extended_s3_configuration=None, kinesis_source_configuration=None, name=None, redshift_configuration=None, s3_configuration=None, server_side_encryption=None, splunk_configuration=None, tags=None, version_id=None, __props__=None);

func NewFirehoseDeliveryStream(ctx *Context, name string, args FirehoseDeliveryStreamArgs, opts ...ResourceOption) (*FirehoseDeliveryStream, error)

public FirehoseDeliveryStream(string name, FirehoseDeliveryStreamArgs args, CustomResourceOptions? opts = null)

name string: The unique name of the resource.
args FirehoseDeliveryStreamArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

resource_name str: The unique name of the resource.
opts ResourceOptions: A bag of options that control this resource's behavior.

ctx Context: Context object for the current deployment.
name string: The unique name of the resource.
args FirehoseDeliveryStreamArgs: The arguments to resource properties.
opts ResourceOption: Bag of options to control resource's behavior.

name string: The unique name of the resource.
args FirehoseDeliveryStreamArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

FirehoseDeliveryStream Resource Properties

To learn more about resource properties and how to use them, see Inputs and Outputs in the Programming Model docs.

Inputs

The FirehoseDeliveryStream resource accepts the following input properties:

Destination string: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
Arn string: The Amazon Resource Name (ARN) specifying the Stream
DestinationId string
ElasticsearchConfiguration FirehoseDeliveryStreamElasticsearchConfigurationArgs: Configuration options if elasticsearch is the destination. More details are given below.
ExtendedS3Configuration FirehoseDeliveryStreamExtendedS3ConfigurationArgs: Enhanced configuration options for the s3 destination. More details are given below.
KinesisSourceConfiguration FirehoseDeliveryStreamKinesisSourceConfigurationArgs: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
Name string: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
RedshiftConfiguration FirehoseDeliveryStreamRedshiftConfigurationArgs: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
S3Configuration FirehoseDeliveryStreamS3ConfigurationArgs: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
ServerSideEncryption FirehoseDeliveryStreamServerSideEncryptionArgs: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
SplunkConfiguration FirehoseDeliveryStreamSplunkConfigurationArgs
Tags Dictionary<string, string>: A map of tags to assign to the resource.
VersionId string: Specifies the table version for the output data schema. Defaults to LATEST.

Destination string: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
Arn string: The Amazon Resource Name (ARN) specifying the Stream
DestinationId string
ElasticsearchConfiguration FirehoseDeliveryStreamElasticsearchConfiguration: Configuration options if elasticsearch is the destination. More details are given below.
ExtendedS3Configuration FirehoseDeliveryStreamExtendedS3Configuration: Enhanced configuration options for the s3 destination. More details are given below.
KinesisSourceConfiguration FirehoseDeliveryStreamKinesisSourceConfiguration: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
Name string: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
RedshiftConfiguration FirehoseDeliveryStreamRedshiftConfiguration: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
S3Configuration FirehoseDeliveryStreamS3Configuration: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
ServerSideEncryption FirehoseDeliveryStreamServerSideEncryption: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
SplunkConfiguration FirehoseDeliveryStreamSplunkConfiguration
Tags map[string]string: A map of tags to assign to the resource.
VersionId string: Specifies the table version for the output data schema. Defaults to LATEST.

destination string: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
arn string: The Amazon Resource Name (ARN) specifying the Stream
destinationId string
elasticsearchConfiguration FirehoseDeliveryStreamElasticsearchConfiguration: Configuration options if elasticsearch is the destination. More details are given below.
extendedS3Configuration FirehoseDeliveryStreamExtendedS3Configuration: Enhanced configuration options for the s3 destination. More details are given below.
kinesisSourceConfiguration FirehoseDeliveryStreamKinesisSourceConfiguration: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
name string: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
redshiftConfiguration FirehoseDeliveryStreamRedshiftConfiguration: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
s3Configuration FirehoseDeliveryStreamS3Configuration: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
serverSideEncryption FirehoseDeliveryStreamServerSideEncryption: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
splunkConfiguration FirehoseDeliveryStreamSplunkConfiguration
tags {[key: string]: string}: A map of tags to assign to the resource.
versionId string: Specifies the table version for the output data schema. Defaults to LATEST.

destination str: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
arn str: The Amazon Resource Name (ARN) specifying the Stream
destination_id str
elasticsearch_configuration Dict[FirehoseDeliveryStreamElasticsearchConfiguration]: Configuration options if elasticsearch is the destination. More details are given below.
extended_s3_configuration Dict[FirehoseDeliveryStreamExtendedS3Configuration]: Enhanced configuration options for the s3 destination. More details are given below.
kinesis_source_configuration Dict[FirehoseDeliveryStreamKinesisSourceConfiguration]: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
name str: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
redshift_configuration Dict[FirehoseDeliveryStreamRedshiftConfiguration]: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
s3_configuration Dict[FirehoseDeliveryStreamS3Configuration]: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
server_side_encryption Dict[FirehoseDeliveryStreamServerSideEncryption]: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
splunk_configuration Dict[FirehoseDeliveryStreamSplunkConfiguration]
tags Dict[str, str]: A map of tags to assign to the resource.
version_id str: Specifies the table version for the output data schema. Defaults to LATEST.

Outputs

All input properties are implicitly available as output properties. Additionally, the FirehoseDeliveryStream resource produces the following output properties:

Id string: The provider-assigned unique ID for this managed resource.

Id string: The provider-assigned unique ID for this managed resource.

id string: The provider-assigned unique ID for this managed resource.

id str: The provider-assigned unique ID for this managed resource.

Look up an Existing FirehoseDeliveryStream Resource

Get an existing FirehoseDeliveryStream resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

public static get(name: string, id: Input<ID>, state?: FirehoseDeliveryStreamState, opts?: CustomResourceOptions): FirehoseDeliveryStream

static get(resource_name, id, opts=None, arn=None, destination=None, destination_id=None, elasticsearch_configuration=None, extended_s3_configuration=None, kinesis_source_configuration=None, name=None, redshift_configuration=None, s3_configuration=None, server_side_encryption=None, splunk_configuration=None, tags=None, version_id=None, __props__=None);

func GetFirehoseDeliveryStream(ctx *Context, name string, id IDInput, state *FirehoseDeliveryStreamState, opts ...ResourceOption) (*FirehoseDeliveryStream, error)

public static FirehoseDeliveryStream Get(string name, Input<string> id, FirehoseDeliveryStreamState? state, CustomResourceOptions? opts = null)

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

resource_name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

The following state arguments are supported:

Arn string: The Amazon Resource Name (ARN) specifying the Stream
Destination string: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
DestinationId string
ElasticsearchConfiguration FirehoseDeliveryStreamElasticsearchConfigurationArgs: Configuration options if elasticsearch is the destination. More details are given below.
ExtendedS3Configuration FirehoseDeliveryStreamExtendedS3ConfigurationArgs: Enhanced configuration options for the s3 destination. More details are given below.
KinesisSourceConfiguration FirehoseDeliveryStreamKinesisSourceConfigurationArgs: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
Name string: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
RedshiftConfiguration FirehoseDeliveryStreamRedshiftConfigurationArgs: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
S3Configuration FirehoseDeliveryStreamS3ConfigurationArgs: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
ServerSideEncryption FirehoseDeliveryStreamServerSideEncryptionArgs: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
SplunkConfiguration FirehoseDeliveryStreamSplunkConfigurationArgs
Tags Dictionary<string, string>: A map of tags to assign to the resource.
VersionId string: Specifies the table version for the output data schema. Defaults to LATEST.

Arn string: The Amazon Resource Name (ARN) specifying the Stream
Destination string: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
DestinationId string
ElasticsearchConfiguration FirehoseDeliveryStreamElasticsearchConfiguration: Configuration options if elasticsearch is the destination. More details are given below.
ExtendedS3Configuration FirehoseDeliveryStreamExtendedS3Configuration: Enhanced configuration options for the s3 destination. More details are given below.
KinesisSourceConfiguration FirehoseDeliveryStreamKinesisSourceConfiguration: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
Name string: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
RedshiftConfiguration FirehoseDeliveryStreamRedshiftConfiguration: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
S3Configuration FirehoseDeliveryStreamS3Configuration: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
ServerSideEncryption FirehoseDeliveryStreamServerSideEncryption: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
SplunkConfiguration FirehoseDeliveryStreamSplunkConfiguration
Tags map[string]string: A map of tags to assign to the resource.
VersionId string: Specifies the table version for the output data schema. Defaults to LATEST.

arn string: The Amazon Resource Name (ARN) specifying the Stream
destination string: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
destinationId string
elasticsearchConfiguration FirehoseDeliveryStreamElasticsearchConfiguration: Configuration options if elasticsearch is the destination. More details are given below.
extendedS3Configuration FirehoseDeliveryStreamExtendedS3Configuration: Enhanced configuration options for the s3 destination. More details are given below.
kinesisSourceConfiguration FirehoseDeliveryStreamKinesisSourceConfiguration: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
name string: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
redshiftConfiguration FirehoseDeliveryStreamRedshiftConfiguration: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
s3Configuration FirehoseDeliveryStreamS3Configuration: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
serverSideEncryption FirehoseDeliveryStreamServerSideEncryption: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
splunkConfiguration FirehoseDeliveryStreamSplunkConfiguration
tags {[key: string]: string}: A map of tags to assign to the resource.
versionId string: Specifies the table version for the output data schema. Defaults to LATEST.

arn str: The Amazon Resource Name (ARN) specifying the Stream
destination str: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
destination_id str
elasticsearch_configuration Dict[FirehoseDeliveryStreamElasticsearchConfiguration]: Configuration options if elasticsearch is the destination. More details are given below.
extended_s3_configuration Dict[FirehoseDeliveryStreamExtendedS3Configuration]: Enhanced configuration options for the s3 destination. More details are given below.
kinesis_source_configuration Dict[FirehoseDeliveryStreamKinesisSourceConfiguration]: Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
name str: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
redshift_configuration Dict[FirehoseDeliveryStreamRedshiftConfiguration]: Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
s3_configuration Dict[FirehoseDeliveryStreamS3Configuration]: Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
server_side_encryption Dict[FirehoseDeliveryStreamServerSideEncryption]: Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
splunk_configuration Dict[FirehoseDeliveryStreamSplunkConfiguration]
tags Dict[str, str]: A map of tags to assign to the resource.
version_id str: Specifies the table version for the output data schema. Defaults to LATEST.

Supporting Types

FirehoseDeliveryStreamElasticsearchConfiguration

DomainArn string: The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming RoleARN. The pattern needs to be arn:.*.
IndexName string: The Elasticsearch index name.
RoleArn string: The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. The pattern needs to be arn:.*.
BufferingInterval int: Buffer incoming data for the specified period of time, in seconds between 60 to 900, before delivering it to the destination. The default value is 300s.
BufferingSize int: Buffer incoming data to the specified size, in MBs between 1 to 100, before delivering it to the destination. The default value is 5MB.
CloudwatchLoggingOptions FirehoseDeliveryStreamElasticsearchConfigurationCloudwatchLoggingOptionsArgs: The CloudWatch Logging Options for the delivery stream. More details are given below
IndexRotationPeriod string: The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data. Valid values are NoRotation, OneHour, OneDay, OneWeek, and OneMonth. The default value is OneDay.
ProcessingConfiguration FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationArgs: The data processing configuration. More details are given below.
RetryDuration int: After an initial failure to deliver to Amazon Elasticsearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
S3BackupMode string: Defines how documents should be delivered to Amazon S3. Valid values are FailedDocumentsOnly and AllDocuments. Default value is FailedDocumentsOnly.
TypeName string: The Elasticsearch type name with maximum length of 100 characters.

DomainArn string: The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming RoleARN. The pattern needs to be arn:.*.
IndexName string: The Elasticsearch index name.
RoleArn string: The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. The pattern needs to be arn:.*.
BufferingInterval int: Buffer incoming data for the specified period of time, in seconds between 60 to 900, before delivering it to the destination. The default value is 300s.
BufferingSize int: Buffer incoming data to the specified size, in MBs between 1 to 100, before delivering it to the destination. The default value is 5MB.
CloudwatchLoggingOptions FirehoseDeliveryStreamElasticsearchConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
IndexRotationPeriod string: The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data. Valid values are NoRotation, OneHour, OneDay, OneWeek, and OneMonth. The default value is OneDay.
ProcessingConfiguration FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
RetryDuration int: After an initial failure to deliver to Amazon Elasticsearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
S3BackupMode string: Defines how documents should be delivered to Amazon S3. Valid values are FailedDocumentsOnly and AllDocuments. Default value is FailedDocumentsOnly.
TypeName string: The Elasticsearch type name with maximum length of 100 characters.

domainArn string: The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming RoleARN. The pattern needs to be arn:.*.
indexName string: The Elasticsearch index name.
roleArn string: The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. The pattern needs to be arn:.*.
bufferingInterval number: Buffer incoming data for the specified period of time, in seconds between 60 to 900, before delivering it to the destination. The default value is 300s.
bufferingSize number: Buffer incoming data to the specified size, in MBs between 1 to 100, before delivering it to the destination. The default value is 5MB.
cloudwatchLoggingOptions FirehoseDeliveryStreamElasticsearchConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
indexRotationPeriod string: The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data. Valid values are NoRotation, OneHour, OneDay, OneWeek, and OneMonth. The default value is OneDay.
processingConfiguration FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
retryDuration number: After an initial failure to deliver to Amazon Elasticsearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
s3BackupMode string: Defines how documents should be delivered to Amazon S3. Valid values are FailedDocumentsOnly and AllDocuments. Default value is FailedDocumentsOnly.
typeName string: The Elasticsearch type name with maximum length of 100 characters.

domainArn str: The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming RoleARN. The pattern needs to be arn:.*.
indexName str: The Elasticsearch index name.
role_arn str: The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. The pattern needs to be arn:.*.
bufferingInterval float: Buffer incoming data for the specified period of time, in seconds between 60 to 900, before delivering it to the destination. The default value is 300s.
bufferingSize float: Buffer incoming data to the specified size, in MBs between 1 to 100, before delivering it to the destination. The default value is 5MB.
cloudwatch_logging_options Dict[FirehoseDeliveryStreamElasticsearchConfigurationCloudwatchLoggingOptions]: The CloudWatch Logging Options for the delivery stream. More details are given below
indexRotationPeriod str: The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data. Valid values are NoRotation, OneHour, OneDay, OneWeek, and OneMonth. The default value is OneDay.
processingConfiguration Dict[FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfiguration]: The data processing configuration. More details are given below.
retryDuration float: After an initial failure to deliver to Amazon Elasticsearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
s3BackupMode str: Defines how documents should be delivered to Amazon S3. Valid values are FailedDocumentsOnly and AllDocuments. Default value is FailedDocumentsOnly.
typeName str: The Elasticsearch type name with maximum length of 100 characters.

FirehoseDeliveryStreamElasticsearchConfigurationCloudwatchLoggingOptions

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled boolean: Enables or disables the logging. Defaults to false.
logGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
logStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled bool: Enables or disables the logging. Defaults to false.
logStreamName str: The CloudWatch log stream name for logging. This value is required if enabled is true.
log_group_name str: The CloudWatch group name for logging. This value is required if enabled is true.

FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfiguration

Enabled bool: Enables or disables data processing.
Processors List<FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorArgs>: Array of data processors. More details are given below

Enabled bool: Enables or disables data processing.
Processors []FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessor: Array of data processors. More details are given below

enabled boolean: Enables or disables data processing.
processors FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessor[]: Array of data processors. More details are given below

enabled bool: Enables or disables data processing.
processors List[FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessor]: Array of data processors. More details are given below

FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessor

Type string: The type of processor. Valid Values: Lambda
Parameters List<FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameterArgs>: Array of processor parameters. More details are given below

Type string: The type of processor. Valid Values: Lambda
Parameters []FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameter: Array of processor parameters. More details are given below

type string: The type of processor. Valid Values: Lambda
parameters FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameter[]: Array of processor parameters. More details are given below

type str: The type of processor. Valid Values: Lambda
parameters List[FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameter]: Array of processor parameters. More details are given below

FirehoseDeliveryStreamElasticsearchConfigurationProcessingConfigurationProcessorParameter

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName str: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue str: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

FirehoseDeliveryStreamExtendedS3Configuration

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamExtendedS3ConfigurationCloudwatchLoggingOptionsArgs: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
DataFormatConversionConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationArgs: Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
ErrorOutputPrefix string: Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
ProcessingConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationArgs: The data processing configuration. More details are given below.
S3BackupConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfigurationArgs: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
S3BackupMode string: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamExtendedS3ConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
DataFormatConversionConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfiguration: Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
ErrorOutputPrefix string: Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
ProcessingConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
S3BackupConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfiguration: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
S3BackupMode string: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

bucketArn string: The ARN of the S3 bucket
roleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval number: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize number: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatchLoggingOptions FirehoseDeliveryStreamExtendedS3ConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
dataFormatConversionConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfiguration: Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
errorOutputPrefix string: Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name.
kmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
processingConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
s3BackupConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfiguration: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
s3BackupMode string: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

bucketArn str: The ARN of the S3 bucket
role_arn str: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval float: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize float: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options Dict[FirehoseDeliveryStreamExtendedS3ConfigurationCloudwatchLoggingOptions]: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat str: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
dataFormatConversionConfiguration Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfiguration]: Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
errorOutputPrefix str: Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name.
kms_key_arn str: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix str: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
processingConfiguration Dict[FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfiguration]: The data processing configuration. More details are given below.
s3BackupConfiguration Dict[FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfiguration]: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
s3BackupMode str: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

FirehoseDeliveryStreamExtendedS3ConfigurationCloudwatchLoggingOptions

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled boolean: Enables or disables the logging. Defaults to false.
logGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
logStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled bool: Enables or disables the logging. Defaults to false.
logStreamName str: The CloudWatch log stream name for logging. This value is required if enabled is true.
log_group_name str: The CloudWatch group name for logging. This value is required if enabled is true.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfiguration

InputFormatConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationArgs: Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
OutputFormatConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationArgs: Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
SchemaConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationSchemaConfigurationArgs: Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
Enabled bool: Defaults to true. Set it to false if you want to disable format conversion while preserving the configuration details.

InputFormatConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfiguration: Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
OutputFormatConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfiguration: Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
SchemaConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationSchemaConfiguration: Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
Enabled bool: Defaults to true. Set it to false if you want to disable format conversion while preserving the configuration details.

inputFormatConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfiguration: Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
outputFormatConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfiguration: Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
schemaConfiguration FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationSchemaConfiguration: Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
enabled boolean: Defaults to true. Set it to false if you want to disable format conversion while preserving the configuration details.

inputFormatConfiguration Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfiguration]: Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
outputFormatConfiguration Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfiguration]: Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
schemaConfiguration Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationSchemaConfiguration]: Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
enabled bool: Defaults to true. Set it to false if you want to disable format conversion while preserving the configuration details.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfiguration

Deserializer FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerArgs: Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.

Deserializer FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializer: Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.

deserializer FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializer: Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.

deserializer Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializer]: Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializer

HiveJsonSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerHiveJsonSerDeArgs: Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
OpenXJsonSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerOpenXJsonSerDeArgs: Nested argument that specifies the OpenX SerDe. More details below.

HiveJsonSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerHiveJsonSerDe: Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
OpenXJsonSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerOpenXJsonSerDe: Nested argument that specifies the OpenX SerDe. More details below.

hiveJsonSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerHiveJsonSerDe: Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
openXJsonSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerOpenXJsonSerDe: Nested argument that specifies the OpenX SerDe. More details below.

hiveJsonSerDe Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerHiveJsonSerDe]: Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
openXJsonSerDe Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerOpenXJsonSerDe]: Nested argument that specifies the OpenX SerDe. More details below.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerHiveJsonSerDe

TimestampFormats List<string>: A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime’s DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don’t specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.

TimestampFormats []string: A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime’s DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don’t specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.

timestampFormats string[]: A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime’s DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don’t specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.

timestampFormats List[str]: A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime’s DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don’t specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationInputFormatConfigurationDeserializerOpenXJsonSerDe

CaseInsensitive bool: When set to true, which is the default, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them.
ColumnToJsonKeyMappings Dictionary<string, string>: A map of column names to JSON keys that aren’t identical to the column names. This is useful when the JSON contains keys that are Hive keywords. For example, timestamp is a Hive keyword. If you have a JSON key named timestamp, set this parameter to { ts = "timestamp" } to map this key to a column named ts.
ConvertDotsInJsonKeysToUnderscores bool: When set to true, specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. For example, if the JSON contains a key whose name is “a.b”, you can define the column name to be “a_b” when using this option. Defaults to false.

CaseInsensitive bool: When set to true, which is the default, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them.
ColumnToJsonKeyMappings map[string]string: A map of column names to JSON keys that aren’t identical to the column names. This is useful when the JSON contains keys that are Hive keywords. For example, timestamp is a Hive keyword. If you have a JSON key named timestamp, set this parameter to { ts = "timestamp" } to map this key to a column named ts.
ConvertDotsInJsonKeysToUnderscores bool: When set to true, specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. For example, if the JSON contains a key whose name is “a.b”, you can define the column name to be “a_b” when using this option. Defaults to false.

caseInsensitive boolean: When set to true, which is the default, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them.
columnToJsonKeyMappings {[key: string]: string}: A map of column names to JSON keys that aren’t identical to the column names. This is useful when the JSON contains keys that are Hive keywords. For example, timestamp is a Hive keyword. If you have a JSON key named timestamp, set this parameter to { ts = "timestamp" } to map this key to a column named ts.
convertDotsInJsonKeysToUnderscores boolean: When set to true, specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. For example, if the JSON contains a key whose name is “a.b”, you can define the column name to be “a_b” when using this option. Defaults to false.

caseInsensitive bool: When set to true, which is the default, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them.
columnToJsonKeyMappings Dict[str, str]: A map of column names to JSON keys that aren’t identical to the column names. This is useful when the JSON contains keys that are Hive keywords. For example, timestamp is a Hive keyword. If you have a JSON key named timestamp, set this parameter to { ts = "timestamp" } to map this key to a column named ts.
convertDotsInJsonKeysToUnderscores bool: When set to true, specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. For example, if the JSON contains a key whose name is “a.b”, you can define the column name to be “a_b” when using this option. Defaults to false.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfiguration

Serializer FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerArgs: Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.

Serializer FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializer: Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.

serializer FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializer: Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.

serializer Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializer]: Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializer

OrcSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerOrcSerDeArgs: Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
ParquetSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerParquetSerDeArgs: Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.

OrcSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerOrcSerDe: Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
ParquetSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerParquetSerDe: Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.

orcSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerOrcSerDe: Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
parquetSerDe FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerParquetSerDe: Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.

orcSerDe Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerOrcSerDe]: Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
parquetSerDe Dict[FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerParquetSerDe]: Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerOrcSerDe

BlockSizeBytes int: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
BloomFilterColumns List<string>: A list of column names for which you want Kinesis Data Firehose to create bloom filters.
BloomFilterFalsePositiveProbability double: The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
Compression string: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
DictionaryKeyThreshold double: A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
EnablePadding bool: Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.
FormatVersion string: The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.
PaddingTolerance double: A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size. For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task. Kinesis Data Firehose ignores this parameter when enable_padding is false.
RowIndexStride int: The number of rows between index entries. The default is 10000 and the minimum is 1000.
StripeSizeBytes int: The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

BlockSizeBytes int: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
BloomFilterColumns []string: A list of column names for which you want Kinesis Data Firehose to create bloom filters.
BloomFilterFalsePositiveProbability float64: The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
Compression string: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
DictionaryKeyThreshold float64: A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
EnablePadding bool: Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.
FormatVersion string: The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.
PaddingTolerance float64: A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size. For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task. Kinesis Data Firehose ignores this parameter when enable_padding is false.
RowIndexStride int: The number of rows between index entries. The default is 10000 and the minimum is 1000.
StripeSizeBytes int: The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

blockSizeBytes number: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
bloomFilterColumns string[]: A list of column names for which you want Kinesis Data Firehose to create bloom filters.
bloomFilterFalsePositiveProbability number: The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
compression string: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
dictionaryKeyThreshold number: A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
enablePadding boolean: Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.
formatVersion string: The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.
paddingTolerance number: A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size. For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task. Kinesis Data Firehose ignores this parameter when enable_padding is false.
rowIndexStride number: The number of rows between index entries. The default is 10000 and the minimum is 1000.
stripeSizeBytes number: The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

blockSizeBytes float: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
bloomFilterColumns List[str]: A list of column names for which you want Kinesis Data Firehose to create bloom filters.
bloomFilterFalsePositiveProbability float: The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
compression str: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
dictionaryKeyThreshold float: A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
enablePadding bool: Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.
formatVersion str: The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.
paddingTolerance float: A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size. For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task. Kinesis Data Firehose ignores this parameter when enable_padding is false.
rowIndexStride float: The number of rows between index entries. The default is 10000 and the minimum is 1000.
stripeSizeBytes float: The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationOutputFormatConfigurationSerializerParquetSerDe

BlockSizeBytes int: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
Compression string: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
EnableDictionaryCompression bool: Indicates whether to enable dictionary compression.
MaxPaddingBytes int: The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
PageSizeBytes int: The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
WriterVersion string: Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

BlockSizeBytes int: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
Compression string: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
EnableDictionaryCompression bool: Indicates whether to enable dictionary compression.
MaxPaddingBytes int: The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
PageSizeBytes int: The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
WriterVersion string: Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

blockSizeBytes number: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
compression string: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
enableDictionaryCompression boolean: Indicates whether to enable dictionary compression.
maxPaddingBytes number: The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
pageSizeBytes number: The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
writerVersion string: Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

blockSizeBytes float: The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
compression str: The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
enableDictionaryCompression bool: Indicates whether to enable dictionary compression.
maxPaddingBytes float: The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
pageSizeBytes float: The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
writerVersion str: Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.

FirehoseDeliveryStreamExtendedS3ConfigurationDataFormatConversionConfigurationSchemaConfiguration

DatabaseName string: Specifies the name of the AWS Glue database that contains the schema for the output data.
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
TableName string: Specifies the AWS Glue table that contains the column information that constitutes your data schema.
CatalogId string: The ID of the AWS Glue Data Catalog. If you don’t supply this, the AWS account ID is used by default.
Region string: If you don’t specify an AWS Region, the default is the current region.
VersionId string: Specifies the table version for the output data schema. Defaults to LATEST.

DatabaseName string: Specifies the name of the AWS Glue database that contains the schema for the output data.
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
TableName string: Specifies the AWS Glue table that contains the column information that constitutes your data schema.
CatalogId string: The ID of the AWS Glue Data Catalog. If you don’t supply this, the AWS account ID is used by default.
Region string: If you don’t specify an AWS Region, the default is the current region.
VersionId string: Specifies the table version for the output data schema. Defaults to LATEST.

databaseName string: Specifies the name of the AWS Glue database that contains the schema for the output data.
roleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
tableName string: Specifies the AWS Glue table that contains the column information that constitutes your data schema.
catalogId string: The ID of the AWS Glue Data Catalog. If you don’t supply this, the AWS account ID is used by default.
region string: If you don’t specify an AWS Region, the default is the current region.
versionId string: Specifies the table version for the output data schema. Defaults to LATEST.

database_name str: Specifies the name of the AWS Glue database that contains the schema for the output data.
role_arn str: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
table_name str: Specifies the AWS Glue table that contains the column information that constitutes your data schema.
catalog_id str: The ID of the AWS Glue Data Catalog. If you don’t supply this, the AWS account ID is used by default.
region str: If you don’t specify an AWS Region, the default is the current region.
version_id str: Specifies the table version for the output data schema. Defaults to LATEST.

FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfiguration

Enabled bool: Enables or disables data processing.
Processors List<FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorArgs>: Array of data processors. More details are given below

Enabled bool: Enables or disables data processing.
Processors []FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessor: Array of data processors. More details are given below

enabled boolean: Enables or disables data processing.
processors FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessor[]: Array of data processors. More details are given below

enabled bool: Enables or disables data processing.
processors List[FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessor]: Array of data processors. More details are given below

FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessor

Type string: The type of processor. Valid Values: Lambda
Parameters List<FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorParameterArgs>: Array of processor parameters. More details are given below

Type string: The type of processor. Valid Values: Lambda
Parameters []FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorParameter: Array of processor parameters. More details are given below

type string: The type of processor. Valid Values: Lambda
parameters FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorParameter[]: Array of processor parameters. More details are given below

type str: The type of processor. Valid Values: Lambda
parameters List[FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorParameter]: Array of processor parameters. More details are given below

FirehoseDeliveryStreamExtendedS3ConfigurationProcessingConfigurationProcessorParameter

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName str: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue str: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfiguration

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfigurationCloudwatchLoggingOptionsArgs: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

bucketArn string: The ARN of the S3 bucket
roleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval number: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize number: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatchLoggingOptions FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

bucketArn str: The ARN of the S3 bucket
role_arn str: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval float: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize float: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options Dict[FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfigurationCloudwatchLoggingOptions]: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat str: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kms_key_arn str: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix str: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

FirehoseDeliveryStreamExtendedS3ConfigurationS3BackupConfigurationCloudwatchLoggingOptions

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled boolean: Enables or disables the logging. Defaults to false.
logGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
logStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled bool: Enables or disables the logging. Defaults to false.
logStreamName str: The CloudWatch log stream name for logging. This value is required if enabled is true.
log_group_name str: The CloudWatch group name for logging. This value is required if enabled is true.

FirehoseDeliveryStreamKinesisSourceConfiguration

KinesisStreamArn string: The kinesis stream used as the source of the firehose delivery stream.
RoleArn string: The ARN of the role that provides access to the source Kinesis stream.

KinesisStreamArn string: The kinesis stream used as the source of the firehose delivery stream.
RoleArn string: The ARN of the role that provides access to the source Kinesis stream.

kinesisStreamArn string: The kinesis stream used as the source of the firehose delivery stream.
roleArn string: The ARN of the role that provides access to the source Kinesis stream.

kinesisStreamArn str: The kinesis stream used as the source of the firehose delivery stream.
role_arn str: The ARN of the role that provides access to the source Kinesis stream.

FirehoseDeliveryStreamRedshiftConfiguration

ClusterJdbcurl string: The jdbcurl of the redshift cluster.
DataTableName string: The name of the table in the redshift cluster that the s3 bucket will copy to.
Password string: The password for the username above.
RoleArn string: The arn of the role the stream assumes.
Username string: The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.
CloudwatchLoggingOptions FirehoseDeliveryStreamRedshiftConfigurationCloudwatchLoggingOptionsArgs: The CloudWatch Logging Options for the delivery stream. More details are given below
CopyOptions string: Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter. For valid values, see the AWS documentation
DataTableColumns string: The data table columns that will be targeted by the copy command.
ProcessingConfiguration FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationArgs: The data processing configuration. More details are given below.
RetryDuration int: The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value of DurationInSeconds is 0 (zero) or if the first delivery attempt takes longer than the current value.
S3BackupConfiguration FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationArgs: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
S3BackupMode string: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

ClusterJdbcurl string: The jdbcurl of the redshift cluster.
DataTableName string: The name of the table in the redshift cluster that the s3 bucket will copy to.
Password string: The password for the username above.
RoleArn string: The arn of the role the stream assumes.
Username string: The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.
CloudwatchLoggingOptions FirehoseDeliveryStreamRedshiftConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
CopyOptions string: Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter. For valid values, see the AWS documentation
DataTableColumns string: The data table columns that will be targeted by the copy command.
ProcessingConfiguration FirehoseDeliveryStreamRedshiftConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
RetryDuration int: The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value of DurationInSeconds is 0 (zero) or if the first delivery attempt takes longer than the current value.
S3BackupConfiguration FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfiguration: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
S3BackupMode string: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

clusterJdbcurl string: The jdbcurl of the redshift cluster.
dataTableName string: The name of the table in the redshift cluster that the s3 bucket will copy to.
password string: The password for the username above.
roleArn string: The arn of the role the stream assumes.
username string: The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.
cloudwatchLoggingOptions FirehoseDeliveryStreamRedshiftConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
copyOptions string: Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter. For valid values, see the AWS documentation
dataTableColumns string: The data table columns that will be targeted by the copy command.
processingConfiguration FirehoseDeliveryStreamRedshiftConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
retryDuration number: The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value of DurationInSeconds is 0 (zero) or if the first delivery attempt takes longer than the current value.
s3BackupConfiguration FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfiguration: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
s3BackupMode string: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

clusterJdbcurl str: The jdbcurl of the redshift cluster.
dataTableName str: The name of the table in the redshift cluster that the s3 bucket will copy to.
password str: The password for the username above.
role_arn str: The arn of the role the stream assumes.
username str: The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.
cloudwatch_logging_options Dict[FirehoseDeliveryStreamRedshiftConfigurationCloudwatchLoggingOptions]: The CloudWatch Logging Options for the delivery stream. More details are given below
copyOptions str: Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter. For valid values, see the AWS documentation
dataTableColumns str: The data table columns that will be targeted by the copy command.
processingConfiguration Dict[FirehoseDeliveryStreamRedshiftConfigurationProcessingConfiguration]: The data processing configuration. More details are given below.
retryDuration float: The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value of DurationInSeconds is 0 (zero) or if the first delivery attempt takes longer than the current value.
s3BackupConfiguration Dict[FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfiguration]: The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
s3BackupMode str: The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

FirehoseDeliveryStreamRedshiftConfigurationCloudwatchLoggingOptions

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled boolean: Enables or disables the logging. Defaults to false.
logGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
logStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled bool: Enables or disables the logging. Defaults to false.
logStreamName str: The CloudWatch log stream name for logging. This value is required if enabled is true.
log_group_name str: The CloudWatch group name for logging. This value is required if enabled is true.

FirehoseDeliveryStreamRedshiftConfigurationProcessingConfiguration

Enabled bool: Enables or disables data processing.
Processors List<FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessorArgs>: Array of data processors. More details are given below

Enabled bool: Enables or disables data processing.
Processors []FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessor: Array of data processors. More details are given below

enabled boolean: Enables or disables data processing.
processors FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessor[]: Array of data processors. More details are given below

enabled bool: Enables or disables data processing.
processors List[FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessor]: Array of data processors. More details are given below

FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessor

Type string: The type of processor. Valid Values: Lambda
Parameters List<FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessorParameterArgs>: Array of processor parameters. More details are given below

Type string: The type of processor. Valid Values: Lambda
Parameters []FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessorParameter: Array of processor parameters. More details are given below

type string: The type of processor. Valid Values: Lambda
parameters FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessorParameter[]: Array of processor parameters. More details are given below

type str: The type of processor. Valid Values: Lambda
parameters List[FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessorParameter]: Array of processor parameters. More details are given below

FirehoseDeliveryStreamRedshiftConfigurationProcessingConfigurationProcessorParameter

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName str: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue str: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfiguration

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationCloudwatchLoggingOptionsArgs: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

bucketArn string: The ARN of the S3 bucket
roleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval number: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize number: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatchLoggingOptions FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

bucketArn str: The ARN of the S3 bucket
role_arn str: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval float: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize float: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options Dict[FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationCloudwatchLoggingOptions]: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat str: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kms_key_arn str: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix str: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

FirehoseDeliveryStreamRedshiftConfigurationS3BackupConfigurationCloudwatchLoggingOptions

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled boolean: Enables or disables the logging. Defaults to false.
logGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
logStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled bool: Enables or disables the logging. Defaults to false.
logStreamName str: The CloudWatch log stream name for logging. This value is required if enabled is true.
log_group_name str: The CloudWatch group name for logging. This value is required if enabled is true.

FirehoseDeliveryStreamS3Configuration

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamS3ConfigurationCloudwatchLoggingOptionsArgs: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

BucketArn string: The ARN of the S3 bucket
RoleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
BufferInterval int: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
BufferSize int: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
CloudwatchLoggingOptions FirehoseDeliveryStreamS3ConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
CompressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
KmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
Prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

bucketArn string: The ARN of the S3 bucket
roleArn string: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval number: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize number: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatchLoggingOptions FirehoseDeliveryStreamS3ConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat string: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kmsKeyArn string: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix string: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

bucketArn str: The ARN of the S3 bucket
role_arn str: The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
bufferInterval float: Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize float: Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options Dict[FirehoseDeliveryStreamS3ConfigurationCloudwatchLoggingOptions]: The CloudWatch Logging Options for the delivery stream. More details are given below
compressionFormat str: The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kms_key_arn str: Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix str: The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket

FirehoseDeliveryStreamS3ConfigurationCloudwatchLoggingOptions

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled boolean: Enables or disables the logging. Defaults to false.
logGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
logStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled bool: Enables or disables the logging. Defaults to false.
logStreamName str: The CloudWatch log stream name for logging. This value is required if enabled is true.
log_group_name str: The CloudWatch group name for logging. This value is required if enabled is true.

FirehoseDeliveryStreamServerSideEncryption

Enabled bool: Whether to enable encryption at rest. Default is false.

Enabled bool: Whether to enable encryption at rest. Default is false.

enabled boolean: Whether to enable encryption at rest. Default is false.

enabled bool: Whether to enable encryption at rest. Default is false.

FirehoseDeliveryStreamSplunkConfiguration

HecEndpoint string: The HTTP Event Collector (HEC) endpoint to which Kinesis Firehose sends your data.
HecToken string: The GUID that you obtain from your Splunk cluster when you create a new HEC endpoint.
CloudwatchLoggingOptions FirehoseDeliveryStreamSplunkConfigurationCloudwatchLoggingOptionsArgs: The CloudWatch Logging Options for the delivery stream. More details are given below.
HecAcknowledgmentTimeout int: The amount of time, in seconds between 180 and 600, that Kinesis Firehose waits to receive an acknowledgment from Splunk after it sends it data.
HecEndpointType string: The HEC endpoint type. Valid values are Raw or Event. The default value is Raw.
ProcessingConfiguration FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationArgs: The data processing configuration. More details are given below.
RetryDuration int: After an initial failure to deliver to Splunk, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
S3BackupMode string: Defines how documents should be delivered to Amazon S3. Valid values are FailedEventsOnly and AllEvents. Default value is FailedEventsOnly.

HecEndpoint string: The HTTP Event Collector (HEC) endpoint to which Kinesis Firehose sends your data.
HecToken string: The GUID that you obtain from your Splunk cluster when you create a new HEC endpoint.
CloudwatchLoggingOptions FirehoseDeliveryStreamSplunkConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below.
HecAcknowledgmentTimeout int: The amount of time, in seconds between 180 and 600, that Kinesis Firehose waits to receive an acknowledgment from Splunk after it sends it data.
HecEndpointType string: The HEC endpoint type. Valid values are Raw or Event. The default value is Raw.
ProcessingConfiguration FirehoseDeliveryStreamSplunkConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
RetryDuration int: After an initial failure to deliver to Splunk, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
S3BackupMode string: Defines how documents should be delivered to Amazon S3. Valid values are FailedEventsOnly and AllEvents. Default value is FailedEventsOnly.

hecEndpoint string: The HTTP Event Collector (HEC) endpoint to which Kinesis Firehose sends your data.
hecToken string: The GUID that you obtain from your Splunk cluster when you create a new HEC endpoint.
cloudwatchLoggingOptions FirehoseDeliveryStreamSplunkConfigurationCloudwatchLoggingOptions: The CloudWatch Logging Options for the delivery stream. More details are given below.
hecAcknowledgmentTimeout number: The amount of time, in seconds between 180 and 600, that Kinesis Firehose waits to receive an acknowledgment from Splunk after it sends it data.
hecEndpointType string: The HEC endpoint type. Valid values are Raw or Event. The default value is Raw.
processingConfiguration FirehoseDeliveryStreamSplunkConfigurationProcessingConfiguration: The data processing configuration. More details are given below.
retryDuration number: After an initial failure to deliver to Splunk, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
s3BackupMode string: Defines how documents should be delivered to Amazon S3. Valid values are FailedEventsOnly and AllEvents. Default value is FailedEventsOnly.

hecEndpoint str: The HTTP Event Collector (HEC) endpoint to which Kinesis Firehose sends your data.
hecToken str: The GUID that you obtain from your Splunk cluster when you create a new HEC endpoint.
cloudwatch_logging_options Dict[FirehoseDeliveryStreamSplunkConfigurationCloudwatchLoggingOptions]: The CloudWatch Logging Options for the delivery stream. More details are given below.
hecAcknowledgmentTimeout float: The amount of time, in seconds between 180 and 600, that Kinesis Firehose waits to receive an acknowledgment from Splunk after it sends it data.
hecEndpointType str: The HEC endpoint type. Valid values are Raw or Event. The default value is Raw.
processingConfiguration Dict[FirehoseDeliveryStreamSplunkConfigurationProcessingConfiguration]: The data processing configuration. More details are given below.
retryDuration float: After an initial failure to deliver to Splunk, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
s3BackupMode str: Defines how documents should be delivered to Amazon S3. Valid values are FailedEventsOnly and AllEvents. Default value is FailedEventsOnly.

FirehoseDeliveryStreamSplunkConfigurationCloudwatchLoggingOptions

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

Enabled bool: Enables or disables the logging. Defaults to false.
LogGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
LogStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled boolean: Enables or disables the logging. Defaults to false.
logGroupName string: The CloudWatch group name for logging. This value is required if enabled is true.
logStreamName string: The CloudWatch log stream name for logging. This value is required if enabled is true.

enabled bool: Enables or disables the logging. Defaults to false.
logStreamName str: The CloudWatch log stream name for logging. This value is required if enabled is true.
log_group_name str: The CloudWatch group name for logging. This value is required if enabled is true.

FirehoseDeliveryStreamSplunkConfigurationProcessingConfiguration

Enabled bool: Enables or disables data processing.
Processors List<FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessorArgs>: Array of data processors. More details are given below

Enabled bool: Enables or disables data processing.
Processors []FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessor: Array of data processors. More details are given below

enabled boolean: Enables or disables data processing.
processors FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessor[]: Array of data processors. More details are given below

enabled bool: Enables or disables data processing.
processors List[FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessor]: Array of data processors. More details are given below

FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessor

Type string: The type of processor. Valid Values: Lambda
Parameters List<FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessorParameterArgs>: Array of processor parameters. More details are given below

Type string: The type of processor. Valid Values: Lambda
Parameters []FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessorParameter: Array of processor parameters. More details are given below

type string: The type of processor. Valid Values: Lambda
parameters FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessorParameter[]: Array of processor parameters. More details are given below

type str: The type of processor. Valid Values: Lambda
parameters List[FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessorParameter]: Array of processor parameters. More details are given below

FirehoseDeliveryStreamSplunkConfigurationProcessingConfigurationProcessorParameter

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

ParameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
ParameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName string: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue string: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

parameterName str: Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
parameterValue str: Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.

Package Details

Repository: https://github.com/pulumi/pulumi-aws
License: Apache-2.0
Notes: This Pulumi package is based on the aws Terraform Provider.

The Pulumi Platform

Get Started

Migrate to Pulumi

Solutions for All Teams and Engineers

Any Code

Any Cloud