This page documents the language specification for the aws package. If you're looking for help working with the inputs, outputs, or functions of aws resources in a Pulumi program, please see the resource documentation for examples and API reference.

kinesis¶

This provider is a derived work of the Terraform Provider distributed under MPL 2.0. If you encounter a bug or missing feature, first check the pulumi/pulumi-aws repo; however, if that doesn’t turn up anything, please consult the source terraform-providers/terraform-provider-aws repo.

class pulumi_aws.kinesis.AnalyticsApplication(resource_name, opts=None, cloudwatch_logging_options=None, code=None, description=None, inputs=None, name=None, outputs=None, reference_data_sources=None, tags=None, __props__=None, __name__=None, __opts__=None)¶

Provides a Kinesis Analytics Application resource. Kinesis Analytics is a managed service that allows processing and analyzing streaming data using standard SQL.

For more details, see the Amazon Kinesis Analytics Documentation.

import pulumi
import pulumi_aws as aws

test_stream = aws.kinesis.Stream("testStream", shard_count=1)
test_application = aws.kinesis.AnalyticsApplication("testApplication", inputs={
    "kinesisStream": {
        "resource_arn": test_stream.arn,
        "role_arn": aws_iam_role["test"]["arn"],
    },
    "name_prefix": "test_prefix",
    "parallelism": {
        "count": 1,
    },
    "schema": {
        "recordColumns": [{
            "mapping": "$.test",
            "name": "test",
            "sqlType": "VARCHAR(8)",
        }],
        "recordEncoding": "UTF-8",
        "recordFormat": {
            "mappingParameters": {
                "json": {
                    "recordRowPath": "$",
                },
            },
        },
    },
})

Parameters

resource_name (str) – The name of the resource.
opts (pulumi.ResourceOptions) – Options for the resource.
cloudwatch_logging_options (pulumi.Input[dict]) – The CloudWatch log stream options to monitor application errors. See CloudWatch Logging Options below for more details.
code (pulumi.Input[str]) – SQL Code to transform input data, and generate output.
description (pulumi.Input[str]) – Description of the application.
inputs (pulumi.Input[dict]) – Input configuration of the application. See Inputs below for more details.
name (pulumi.Input[str]) – Name of the Kinesis Analytics Application.
outputs (pulumi.Input[list]) – Output destination configuration of the application. See Outputs below for more details.
reference_data_sources (pulumi.Input[dict]) – An S3 Reference Data Source for the application. See Reference Data Sources below for more details.
tags (pulumi.Input[dict]) – Key-value map of tags for the Kinesis Analytics Application.

The cloudwatch_logging_options object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
logStreamArn (pulumi.Input[str]) - The ARN of the CloudWatch Log Stream.
role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to send application messages.

The inputs object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
kinesisFirehose (pulumi.Input[dict]) - The Kinesis Firehose configuration for the streaming source. Conflicts with kinesis_stream. See Kinesis Firehose below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Firehose delivery stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
kinesisStream (pulumi.Input[dict]) - The Kinesis Stream configuration for the streaming source. Conflicts with kinesis_firehose. See Kinesis Stream below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
name_prefix (pulumi.Input[str]) - The Name Prefix to use when creating an in-application stream.
parallelism (pulumi.Input[dict]) - The number of Parallel in-application streams to create. See Parallelism below for more details.
- count (pulumi.Input[float]) - The Count of streams.
processingConfiguration (pulumi.Input[dict]) - The Processing Configuration to transform records as they are received from the stream. See Processing Configuration below for more details.
- lambda_ (pulumi.Input[dict]) - The Lambda function configuration. See Lambda below for more details.
  - resource_arn (pulumi.Input[str]) - The ARN of the Lambda function.
  - role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the Lambda function.
schema (pulumi.Input[dict]) - The Schema format of the data in the streaming source. See Source Schema below for more details.
- recordColumns (pulumi.Input[list]) - The Record Column mapping for the streaming source data element. See Record Columns below for more details.
  - mapping (pulumi.Input[str]) - The Mapping reference to the data element.
  - name (pulumi.Input[str]) - Name of the column.
  - sqlType (pulumi.Input[str]) - The SQL Type of the column.
- recordEncoding (pulumi.Input[str]) - The Encoding of the record in the streaming source.
- recordFormat (pulumi.Input[dict]) - The Record Format and mapping information to schematize a record. See Record Format below for more details.
  - mappingParameters (pulumi.Input[dict]) - The Mapping Information for the record format. See Mapping Parameters below for more details.
    - csv (pulumi.Input[dict]) - Mapping information when the record format uses delimiters. See CSV Mapping Parameters below for more details.
      - recordColumnDelimiter (pulumi.Input[str]) - The Column Delimiter.
      - recordRowDelimiter (pulumi.Input[str]) - The Row Delimiter.
    - json (pulumi.Input[dict]) - Mapping information when JSON is the record format on the streaming source. See JSON Mapping Parameters below for more details.
      - recordRowPath (pulumi.Input[str]) - Path to the top-level parent that contains the records.
  - recordFormatType (pulumi.Input[str]) - The type of Record Format. Can be CSV or JSON.
startingPositionConfigurations (pulumi.Input[list])
- starting_position (pulumi.Input[str])
streamNames (pulumi.Input[list])

The outputs object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
kinesisFirehose (pulumi.Input[dict]) - The Kinesis Firehose configuration for the destination stream. Conflicts with kinesis_stream. See Kinesis Firehose below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Firehose delivery stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
kinesisStream (pulumi.Input[dict]) - The Kinesis Stream configuration for the destination stream. Conflicts with kinesis_firehose. See Kinesis Stream below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
lambda_ (pulumi.Input[dict]) - The Lambda function destination. See Lambda below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Lambda function.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the Lambda function.
name (pulumi.Input[str]) - The Name of the in-application stream.
schema (pulumi.Input[dict]) - The Schema format of the data written to the destination. See Destination Schema below for more details.
- recordFormatType (pulumi.Input[str]) - The Format Type of the records on the output stream. Can be CSV or JSON.

The reference_data_sources object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
s3 (pulumi.Input[dict]) - The S3 configuration for the reference data source. See S3 Reference below for more details.
- bucketArn (pulumi.Input[str]) - The S3 Bucket ARN.
- fileKey (pulumi.Input[str]) - The File Key name containing reference data.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to send application messages.
schema (pulumi.Input[dict]) - The Schema format of the data in the streaming source. See Source Schema below for more details.
- recordColumns (pulumi.Input[list]) - The Record Column mapping for the streaming source data element. See Record Columns below for more details.
  - mapping (pulumi.Input[str]) - The Mapping reference to the data element.
  - name (pulumi.Input[str]) - Name of the column.
  - sqlType (pulumi.Input[str]) - The SQL Type of the column.
- recordEncoding (pulumi.Input[str]) - The Encoding of the record in the streaming source.
- recordFormat (pulumi.Input[dict]) - The Record Format and mapping information to schematize a record. See Record Format below for more details.
  - mappingParameters (pulumi.Input[dict]) - The Mapping Information for the record format. See Mapping Parameters below for more details.
    - csv (pulumi.Input[dict]) - Mapping information when the record format uses delimiters. See CSV Mapping Parameters below for more details.
      - recordColumnDelimiter (pulumi.Input[str]) - The Column Delimiter.
      - recordRowDelimiter (pulumi.Input[str]) - The Row Delimiter.
    - json (pulumi.Input[dict]) - Mapping information when JSON is the record format on the streaming source. See JSON Mapping Parameters below for more details.
      - recordRowPath (pulumi.Input[str]) - Path to the top-level parent that contains the records.
  - recordFormatType (pulumi.Input[str]) - The type of Record Format. Can be CSV or JSON.
table_name (pulumi.Input[str]) - The in-application Table Name.

arn: pulumi.Output[str] = None¶: The ARN of the Kinesis Analytics Appliation.

cloudwatch_logging_options: pulumi.Output[dict] = None¶

The CloudWatch log stream options to monitor application errors. See CloudWatch Logging Options below for more details.

id (str) - The ARN of the Kinesis Analytics Application.
logStreamArn (str) - The ARN of the CloudWatch Log Stream.
role_arn (str) - The ARN of the IAM Role used to send application messages.

code: pulumi.Output[str] = None¶: SQL Code to transform input data, and generate output.

create_timestamp: pulumi.Output[str] = None¶: The Timestamp when the application version was created.

description: pulumi.Output[str] = None¶: Description of the application.

inputs: pulumi.Output[dict] = None¶

Input configuration of the application. See Inputs below for more details.

id (str) - The ARN of the Kinesis Analytics Application.
kinesisFirehose (dict) - The Kinesis Firehose configuration for the streaming source. Conflicts with kinesis_stream. See Kinesis Firehose below for more details.
- resource_arn (str) - The ARN of the Kinesis Firehose delivery stream.
- role_arn (str) - The ARN of the IAM Role used to access the stream.
kinesisStream (dict) - The Kinesis Stream configuration for the streaming source. Conflicts with kinesis_firehose. See Kinesis Stream below for more details.
- resource_arn (str) - The ARN of the Kinesis Stream.
- role_arn (str) - The ARN of the IAM Role used to access the stream.
name_prefix (str) - The Name Prefix to use when creating an in-application stream.
parallelism (dict) - The number of Parallel in-application streams to create. See Parallelism below for more details.
- count (float) - The Count of streams.
processingConfiguration (dict) - The Processing Configuration to transform records as they are received from the stream. See Processing Configuration below for more details.
- lambda_ (dict) - The Lambda function configuration. See Lambda below for more details.
  - resource_arn (str) - The ARN of the Lambda function.
  - role_arn (str) - The ARN of the IAM Role used to access the Lambda function.
schema (dict) - The Schema format of the data in the streaming source. See Source Schema below for more details.
- recordColumns (list) - The Record Column mapping for the streaming source data element. See Record Columns below for more details.
  - mapping (str) - The Mapping reference to the data element.
  - name (str) - Name of the column.
  - sqlType (str) - The SQL Type of the column.
- recordEncoding (str) - The Encoding of the record in the streaming source.
- recordFormat (dict) - The Record Format and mapping information to schematize a record. See Record Format below for more details.
  - mappingParameters (dict) - The Mapping Information for the record format. See Mapping Parameters below for more details.
    - csv (dict) - Mapping information when the record format uses delimiters. See CSV Mapping Parameters below for more details.
      - recordColumnDelimiter (str) - The Column Delimiter.
      - recordRowDelimiter (str) - The Row Delimiter.
    - json (dict) - Mapping information when JSON is the record format on the streaming source. See JSON Mapping Parameters below for more details.
      - recordRowPath (str) - Path to the top-level parent that contains the records.
  - recordFormatType (str) - The type of Record Format. Can be CSV or JSON.
startingPositionConfigurations (list)
- starting_position (str)
streamNames (list)

last_update_timestamp: pulumi.Output[str] = None¶: The Timestamp when the application was last updated.

name: pulumi.Output[str] = None¶: Name of the Kinesis Analytics Application.

outputs: pulumi.Output[list] = None¶

Output destination configuration of the application. See Outputs below for more details.

id (str) - The ARN of the Kinesis Analytics Application.
kinesisFirehose (dict) - The Kinesis Firehose configuration for the destination stream. Conflicts with kinesis_stream. See Kinesis Firehose below for more details.
- resource_arn (str) - The ARN of the Kinesis Firehose delivery stream.
- role_arn (str) - The ARN of the IAM Role used to access the stream.
kinesisStream (dict) - The Kinesis Stream configuration for the destination stream. Conflicts with kinesis_firehose. See Kinesis Stream below for more details.
- resource_arn (str) - The ARN of the Kinesis Stream.
- role_arn (str) - The ARN of the IAM Role used to access the stream.
lambda_ (dict) - The Lambda function destination. See Lambda below for more details.
- resource_arn (str) - The ARN of the Lambda function.
- role_arn (str) - The ARN of the IAM Role used to access the Lambda function.
name (str) - The Name of the in-application stream.
schema (dict) - The Schema format of the data written to the destination. See Destination Schema below for more details.
- recordFormatType (str) - The Format Type of the records on the output stream. Can be CSV or JSON.

reference_data_sources: pulumi.Output[dict] = None¶

An S3 Reference Data Source for the application. See Reference Data Sources below for more details.

id (str) - The ARN of the Kinesis Analytics Application.
s3 (dict) - The S3 configuration for the reference data source. See S3 Reference below for more details.
- bucketArn (str) - The S3 Bucket ARN.
- fileKey (str) - The File Key name containing reference data.
- role_arn (str) - The ARN of the IAM Role used to send application messages.
schema (dict) - The Schema format of the data in the streaming source. See Source Schema below for more details.
- recordColumns (list) - The Record Column mapping for the streaming source data element. See Record Columns below for more details.
  - mapping (str) - The Mapping reference to the data element.
  - name (str) - Name of the column.
  - sqlType (str) - The SQL Type of the column.
- recordEncoding (str) - The Encoding of the record in the streaming source.
- recordFormat (dict) - The Record Format and mapping information to schematize a record. See Record Format below for more details.
  - mappingParameters (dict) - The Mapping Information for the record format. See Mapping Parameters below for more details.
    - csv (dict) - Mapping information when the record format uses delimiters. See CSV Mapping Parameters below for more details.
      - recordColumnDelimiter (str) - The Column Delimiter.
      - recordRowDelimiter (str) - The Row Delimiter.
    - json (dict) - Mapping information when JSON is the record format on the streaming source. See JSON Mapping Parameters below for more details.
      - recordRowPath (str) - Path to the top-level parent that contains the records.
  - recordFormatType (str) - The type of Record Format. Can be CSV or JSON.
table_name (str) - The in-application Table Name.

status: pulumi.Output[str] = None¶: The Status of the application.

tags: pulumi.Output[dict] = None¶: Key-value map of tags for the Kinesis Analytics Application.

version: pulumi.Output[float] = None¶: The Version of the application.

static get(resource_name, id, opts=None, arn=None, cloudwatch_logging_options=None, code=None, create_timestamp=None, description=None, inputs=None, last_update_timestamp=None, name=None, outputs=None, reference_data_sources=None, status=None, tags=None, version=None)¶

Get an existing AnalyticsApplication resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters

resource_name (str) – The unique name of the resulting resource.
id (str) – The unique provider ID of the resource to lookup.
opts (pulumi.ResourceOptions) – Options for the resource.
arn (pulumi.Input[str]) – The ARN of the Kinesis Analytics Appliation.
cloudwatch_logging_options (pulumi.Input[dict]) – The CloudWatch log stream options to monitor application errors. See CloudWatch Logging Options below for more details.
code (pulumi.Input[str]) – SQL Code to transform input data, and generate output.
create_timestamp (pulumi.Input[str]) – The Timestamp when the application version was created.
description (pulumi.Input[str]) – Description of the application.
inputs (pulumi.Input[dict]) – Input configuration of the application. See Inputs below for more details.
last_update_timestamp (pulumi.Input[str]) – The Timestamp when the application was last updated.
name (pulumi.Input[str]) – Name of the Kinesis Analytics Application.
outputs (pulumi.Input[list]) – Output destination configuration of the application. See Outputs below for more details.
reference_data_sources (pulumi.Input[dict]) – An S3 Reference Data Source for the application. See Reference Data Sources below for more details.
status (pulumi.Input[str]) – The Status of the application.
tags (pulumi.Input[dict]) – Key-value map of tags for the Kinesis Analytics Application.
version (pulumi.Input[float]) – The Version of the application.

The cloudwatch_logging_options object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
logStreamArn (pulumi.Input[str]) - The ARN of the CloudWatch Log Stream.
role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to send application messages.

The inputs object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
kinesisFirehose (pulumi.Input[dict]) - The Kinesis Firehose configuration for the streaming source. Conflicts with kinesis_stream. See Kinesis Firehose below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Firehose delivery stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
kinesisStream (pulumi.Input[dict]) - The Kinesis Stream configuration for the streaming source. Conflicts with kinesis_firehose. See Kinesis Stream below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
name_prefix (pulumi.Input[str]) - The Name Prefix to use when creating an in-application stream.
parallelism (pulumi.Input[dict]) - The number of Parallel in-application streams to create. See Parallelism below for more details.
- count (pulumi.Input[float]) - The Count of streams.
processingConfiguration (pulumi.Input[dict]) - The Processing Configuration to transform records as they are received from the stream. See Processing Configuration below for more details.
- lambda_ (pulumi.Input[dict]) - The Lambda function configuration. See Lambda below for more details.
  - resource_arn (pulumi.Input[str]) - The ARN of the Lambda function.
  - role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the Lambda function.
schema (pulumi.Input[dict]) - The Schema format of the data in the streaming source. See Source Schema below for more details.
- recordColumns (pulumi.Input[list]) - The Record Column mapping for the streaming source data element. See Record Columns below for more details.
  - mapping (pulumi.Input[str]) - The Mapping reference to the data element.
  - name (pulumi.Input[str]) - Name of the column.
  - sqlType (pulumi.Input[str]) - The SQL Type of the column.
- recordEncoding (pulumi.Input[str]) - The Encoding of the record in the streaming source.
- recordFormat (pulumi.Input[dict]) - The Record Format and mapping information to schematize a record. See Record Format below for more details.
  - mappingParameters (pulumi.Input[dict]) - The Mapping Information for the record format. See Mapping Parameters below for more details.
    - csv (pulumi.Input[dict]) - Mapping information when the record format uses delimiters. See CSV Mapping Parameters below for more details.
      - recordColumnDelimiter (pulumi.Input[str]) - The Column Delimiter.
      - recordRowDelimiter (pulumi.Input[str]) - The Row Delimiter.
    - json (pulumi.Input[dict]) - Mapping information when JSON is the record format on the streaming source. See JSON Mapping Parameters below for more details.
      - recordRowPath (pulumi.Input[str]) - Path to the top-level parent that contains the records.
  - recordFormatType (pulumi.Input[str]) - The type of Record Format. Can be CSV or JSON.
startingPositionConfigurations (pulumi.Input[list])
- starting_position (pulumi.Input[str])
streamNames (pulumi.Input[list])

The outputs object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
kinesisFirehose (pulumi.Input[dict]) - The Kinesis Firehose configuration for the destination stream. Conflicts with kinesis_stream. See Kinesis Firehose below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Firehose delivery stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
kinesisStream (pulumi.Input[dict]) - The Kinesis Stream configuration for the destination stream. Conflicts with kinesis_firehose. See Kinesis Stream below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Kinesis Stream.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the stream.
lambda_ (pulumi.Input[dict]) - The Lambda function destination. See Lambda below for more details.
- resource_arn (pulumi.Input[str]) - The ARN of the Lambda function.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to access the Lambda function.
name (pulumi.Input[str]) - The Name of the in-application stream.
schema (pulumi.Input[dict]) - The Schema format of the data written to the destination. See Destination Schema below for more details.
- recordFormatType (pulumi.Input[str]) - The Format Type of the records on the output stream. Can be CSV or JSON.

The reference_data_sources object supports the following:

id (pulumi.Input[str]) - The ARN of the Kinesis Analytics Application.
s3 (pulumi.Input[dict]) - The S3 configuration for the reference data source. See S3 Reference below for more details.
- bucketArn (pulumi.Input[str]) - The S3 Bucket ARN.
- fileKey (pulumi.Input[str]) - The File Key name containing reference data.
- role_arn (pulumi.Input[str]) - The ARN of the IAM Role used to send application messages.
schema (pulumi.Input[dict]) - The Schema format of the data in the streaming source. See Source Schema below for more details.
- recordColumns (pulumi.Input[list]) - The Record Column mapping for the streaming source data element. See Record Columns below for more details.
  - mapping (pulumi.Input[str]) - The Mapping reference to the data element.
  - name (pulumi.Input[str]) - Name of the column.
  - sqlType (pulumi.Input[str]) - The SQL Type of the column.
- recordEncoding (pulumi.Input[str]) - The Encoding of the record in the streaming source.
- recordFormat (pulumi.Input[dict]) - The Record Format and mapping information to schematize a record. See Record Format below for more details.
  - mappingParameters (pulumi.Input[dict]) - The Mapping Information for the record format. See Mapping Parameters below for more details.
    - csv (pulumi.Input[dict]) - Mapping information when the record format uses delimiters. See CSV Mapping Parameters below for more details.
      - recordColumnDelimiter (pulumi.Input[str]) - The Column Delimiter.
      - recordRowDelimiter (pulumi.Input[str]) - The Row Delimiter.
    - json (pulumi.Input[dict]) - Mapping information when JSON is the record format on the streaming source. See JSON Mapping Parameters below for more details.
      - recordRowPath (pulumi.Input[str]) - Path to the top-level parent that contains the records.
  - recordFormatType (pulumi.Input[str]) - The type of Record Format. Can be CSV or JSON.
table_name (pulumi.Input[str]) - The in-application Table Name.

translate_output_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

translate_input_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

class pulumi_aws.kinesis.AwaitableGetStreamResult(arn=None, closed_shards=None, creation_timestamp=None, id=None, name=None, open_shards=None, retention_period=None, shard_level_metrics=None, status=None, tags=None)¶

class pulumi_aws.kinesis.FirehoseDeliveryStream(resource_name, opts=None, arn=None, destination=None, destination_id=None, elasticsearch_configuration=None, extended_s3_configuration=None, kinesis_source_configuration=None, name=None, redshift_configuration=None, s3_configuration=None, server_side_encryption=None, splunk_configuration=None, tags=None, version_id=None, __props__=None, __name__=None, __opts__=None)¶

Provides a Kinesis Firehose Delivery Stream resource. Amazon Kinesis Firehose is a fully managed, elastic service to easily deliver real-time data streams to destinations such as Amazon S3 and Amazon Redshift.

For more details, see the Amazon Kinesis Firehose Documentation.

import pulumi
import pulumi_aws as aws

bucket = aws.s3.Bucket("bucket", acl="private")
firehose_role = aws.iam.Role("firehoseRole", assume_role_policy="""{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}

""")
lambda_iam = aws.iam.Role("lambdaIam", assume_role_policy="""{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}

""")
lambda_processor = aws.lambda_.Function("lambdaProcessor",
    code=pulumi.FileArchive("lambda.zip"),
    handler="exports.handler",
    role=lambda_iam.arn,
    runtime="nodejs8.10")
extended_s3_stream = aws.kinesis.FirehoseDeliveryStream("extendedS3Stream",
    destination="extended_s3",
    extended_s3_configuration={
        "bucketArn": bucket.arn,
        "processingConfiguration": {
            "enabled": "true",
            "processors": [{
                "parameters": [{
                    "parameterName": "LambdaArn",
                    "parameterValue": lambda_processor.arn.apply(lambda arn: f"{arn}:$LATEST"),
                }],
                "type": "Lambda",
            }],
        },
        "role_arn": firehose_role.arn,
    })

import pulumi
import pulumi_aws as aws

bucket = aws.s3.Bucket("bucket", acl="private")
firehose_role = aws.iam.Role("firehoseRole", assume_role_policy="""{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}

""")
test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="s3",
    s3_configuration={
        "bucketArn": bucket.arn,
        "role_arn": firehose_role.arn,
    })

import pulumi
import pulumi_aws as aws

test_cluster = aws.redshift.Cluster("testCluster",
    cluster_identifier="tf-redshift-cluster-%d",
    cluster_type="single-node",
    database_name="test",
    master_password="T3stPass",
    master_username="testuser",
    node_type="dc1.large")
test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="redshift",
    redshift_configuration={
        "clusterJdbcurl": pulumi.Output.all(test_cluster.endpoint, test_cluster.database_name).apply(lambda endpoint, database_name: f"jdbc:redshift://{endpoint}/{database_name}"),
        "copyOptions": "delimiter '|'",
        "dataTableColumns": "test-col",
        "dataTableName": "test-table",
        "password": "T3stPass",
        "role_arn": aws_iam_role["firehose_role"]["arn"],
        "s3BackupConfiguration": {
            "bucketArn": aws_s3_bucket["bucket"]["arn"],
            "bufferInterval": 300,
            "bufferSize": 15,
            "compressionFormat": "GZIP",
            "role_arn": aws_iam_role["firehose_role"]["arn"],
        },
        "s3BackupMode": "Enabled",
        "username": "testuser",
    },
    s3_configuration={
        "bucketArn": aws_s3_bucket["bucket"]["arn"],
        "bufferInterval": 400,
        "bufferSize": 10,
        "compressionFormat": "GZIP",
        "role_arn": aws_iam_role["firehose_role"]["arn"],
    })

import pulumi
import pulumi_aws as aws

test_cluster = aws.elasticsearch.Domain("testCluster")
test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="elasticsearch",
    elasticsearch_configuration={
        "domainArn": test_cluster.arn,
        "indexName": "test",
        "processingConfiguration": {
            "enabled": "true",
            "processors": [{
                "parameters": [{
                    "parameterName": "LambdaArn",
                    "parameterValue": f"{aws_lambda_function['lambda_processor']['arn']}:$LATEST",
                }],
                "type": "Lambda",
            }],
        },
        "role_arn": aws_iam_role["firehose_role"]["arn"],
        "typeName": "test",
    },
    s3_configuration={
        "bucketArn": aws_s3_bucket["bucket"]["arn"],
        "bufferInterval": 400,
        "bufferSize": 10,
        "compressionFormat": "GZIP",
        "role_arn": aws_iam_role["firehose_role"]["arn"],
    })

import pulumi
import pulumi_aws as aws

test_stream = aws.kinesis.FirehoseDeliveryStream("testStream",
    destination="splunk",
    s3_configuration={
        "bucketArn": aws_s3_bucket["bucket"]["arn"],
        "bufferInterval": 400,
        "bufferSize": 10,
        "compressionFormat": "GZIP",
        "role_arn": aws_iam_role["firehose"]["arn"],
    },
    splunk_configuration={
        "hecAcknowledgmentTimeout": 600,
        "hecEndpoint": "https://http-inputs-mydomain.splunkcloud.com:443",
        "hecEndpointType": "Event",
        "hecToken": "51D4DA16-C61B-4F5F-8EC7-ED4301342A4A",
        "s3BackupMode": "FailedEventsOnly",
    })

Parameters

resource_name (str) – The name of the resource.
opts (pulumi.ResourceOptions) – Options for the resource.
arn (pulumi.Input[str]) – The Amazon Resource Name (ARN) specifying the Stream
destination (pulumi.Input[str]) – This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
elasticsearch_configuration (pulumi.Input[dict]) – Configuration options if elasticsearch is the destination. More details are given below.
extended_s3_configuration (pulumi.Input[dict]) – Enhanced configuration options for the s3 destination. More details are given below.
kinesis_source_configuration (pulumi.Input[dict]) – Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
name (pulumi.Input[str]) – A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
redshift_configuration (pulumi.Input[dict]) – Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
s3_configuration (pulumi.Input[dict]) – Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
server_side_encryption (pulumi.Input[dict]) – Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
tags (pulumi.Input[dict]) – A map of tags to assign to the resource.
version_id (pulumi.Input[str]) – Specifies the table version for the output data schema. Defaults to LATEST.

The elasticsearch_configuration object supports the following:

bufferingInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds between 60 to 900, before delivering it to the destination. The default value is 300s.
bufferingSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs between 1 to 100, before delivering it to the destination. The default value is 5MB.
cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
domainArn (pulumi.Input[str]) - The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming RoleARN. The pattern needs to be arn:.*.
indexName (pulumi.Input[str]) - The Elasticsearch index name.
indexRotationPeriod (pulumi.Input[str]) - The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data. Valid values are NoRotation, OneHour, OneDay, OneWeek, and OneMonth. The default value is OneDay.
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
retryDuration (pulumi.Input[float]) - After an initial failure to deliver to Amazon Elasticsearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
role_arn (pulumi.Input[str]) - The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. The pattern needs to be arn:.*.
s3BackupMode (pulumi.Input[str]) - Defines how documents should be delivered to Amazon S3. Valid values are FailedDocumentsOnly and AllDocuments. Default value is FailedDocumentsOnly.
typeName (pulumi.Input[str]) - The Elasticsearch type name with maximum length of 100 characters.

The extended_s3_configuration object supports the following:

bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
dataFormatConversionConfiguration (pulumi.Input[dict]) - Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
- enabled (pulumi.Input[bool]) - Defaults to true. Set it to false if you want to disable format conversion while preserving the configuration details.
- inputFormatConfiguration (pulumi.Input[dict]) - Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
  - deserializer (pulumi.Input[dict]) - Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.
    - hiveJsonSerDe (pulumi.Input[dict]) - Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
      - timestampFormats (pulumi.Input[list]) - A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime’s DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don’t specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.
    - openXJsonSerDe (pulumi.Input[dict]) - Nested argument that specifies the OpenX SerDe. More details below.
      - caseInsensitive (pulumi.Input[bool]) - When set to true, which is the default, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them.
      - columnToJsonKeyMappings (pulumi.Input[dict]) - A map of column names to JSON keys that aren’t identical to the column names. This is useful when the JSON contains keys that are Hive keywords. For example, timestamp is a Hive keyword. If you have a JSON key named timestamp, set this parameter to { ts = "timestamp" } to map this key to a column named ts.
      - convertDotsInJsonKeysToUnderscores (pulumi.Input[bool]) - When set to true, specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. For example, if the JSON contains a key whose name is “a.b”, you can define the column name to be “a_b” when using this option. Defaults to false.
- outputFormatConfiguration (pulumi.Input[dict]) - Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
  - serializer (pulumi.Input[dict]) - Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.
    - orcSerDe (pulumi.Input[dict]) - Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
      - blockSizeBytes (pulumi.Input[float]) - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
      - bloomFilterColumns (pulumi.Input[list]) - A list of column names for which you want Kinesis Data Firehose to create bloom filters.
      - bloomFilterFalsePositiveProbability (pulumi.Input[float]) - The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
      - compression (pulumi.Input[str]) - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
      - dictionaryKeyThreshold (pulumi.Input[float]) - A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
      - enablePadding (pulumi.Input[bool]) - Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.
      - formatVersion (pulumi.Input[str]) - The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.
      - paddingTolerance (pulumi.Input[float]) - A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size. For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task. Kinesis Data Firehose ignores this parameter when enable_padding is false.
      - rowIndexStride (pulumi.Input[float]) - The number of rows between index entries. The default is 10000 and the minimum is 1000.
      - stripeSizeBytes (pulumi.Input[float]) - The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.
    - parquetSerDe (pulumi.Input[dict]) - Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.
      - blockSizeBytes (pulumi.Input[float]) - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
      - compression (pulumi.Input[str]) - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
      - enableDictionaryCompression (pulumi.Input[bool]) - Indicates whether to enable dictionary compression.
      - maxPaddingBytes (pulumi.Input[float]) - The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
      - pageSizeBytes (pulumi.Input[float]) - The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
      - writerVersion (pulumi.Input[str]) - Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.
- schemaConfiguration (pulumi.Input[dict]) - Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
  - catalog_id (pulumi.Input[str]) - The ID of the AWS Glue Data Catalog. If you don’t supply this, the AWS account ID is used by default.
  - database_name (pulumi.Input[str]) - Specifies the name of the AWS Glue database that contains the schema for the output data.
  - region (pulumi.Input[str]) - If you don’t specify an AWS Region, the default is the current region.
  - role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
  - table_name (pulumi.Input[str]) - Specifies the AWS Glue table that contains the column information that constitutes your data schema.
  - version_id (pulumi.Input[str]) - Specifies the table version for the output data schema. Defaults to LATEST.
errorOutputPrefix (pulumi.Input[str]) - Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name.
kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupConfiguration (pulumi.Input[dict]) - The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
- bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
- bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
- bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
- cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
  - enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
  - log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
  - logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
- compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
- kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
- prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
- role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupMode (pulumi.Input[str]) - The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

The kinesis_source_configuration object supports the following:

kinesisStreamArn (pulumi.Input[str]) - The kinesis stream used as the source of the firehose delivery stream.
role_arn (pulumi.Input[str]) - The ARN of the role that provides access to the source Kinesis stream.

The redshift_configuration object supports the following:

cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
clusterJdbcurl (pulumi.Input[str]) - The jdbcurl of the redshift cluster.
copyOptions (pulumi.Input[str]) - Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter. For valid values, see the AWS documentation
dataTableColumns (pulumi.Input[str]) - The data table columns that will be targeted by the copy command.
dataTableName (pulumi.Input[str]) - The name of the table in the redshift cluster that the s3 bucket will copy to.
password (pulumi.Input[str]) - The password for the username above.
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
retryDuration (pulumi.Input[float]) - The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value of DurationInSeconds is 0 (zero) or if the first delivery attempt takes longer than the current value.
role_arn (pulumi.Input[str]) - The arn of the role the stream assumes.
s3BackupConfiguration (pulumi.Input[dict]) - The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
- bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
- bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
- bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
- cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
  - enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
  - log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
  - logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
- compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
- kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
- prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
- role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupMode (pulumi.Input[str]) - The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.
username (pulumi.Input[str]) - The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.

The s3_configuration object supports the following:

bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.

The server_side_encryption object supports the following:

enabled (pulumi.Input[bool]) - Whether to enable encryption at rest. Default is false.

The splunk_configuration object supports the following:

cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
hecAcknowledgmentTimeout (pulumi.Input[float]) - The amount of time, in seconds between 180 and 600, that Kinesis Firehose waits to receive an acknowledgment from Splunk after it sends it data.
hecEndpoint (pulumi.Input[str]) - The HTTP Event Collector (HEC) endpoint to which Kinesis Firehose sends your data.
hecEndpointType (pulumi.Input[str]) - The HEC endpoint type. Valid values are Raw or Event. The default value is Raw.
hecToken (pulumi.Input[str]) - The GUID that you obtain from your Splunk cluster when you create a new HEC endpoint.
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
retryDuration (pulumi.Input[float]) - After an initial failure to deliver to Splunk, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
s3BackupMode (pulumi.Input[str]) - Defines how documents should be delivered to Amazon S3. Valid values are FailedEventsOnly and AllEvents. Default value is FailedEventsOnly.

arn: pulumi.Output[str] = None¶: The Amazon Resource Name (ARN) specifying the Stream

destination: pulumi.Output[str] = None¶: This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.

elasticsearch_configuration: pulumi.Output[dict] = None¶

Configuration options if elasticsearch is the destination. More details are given below.

bufferingInterval (float) - Buffer incoming data for the specified period of time, in seconds between 60 to 900, before delivering it to the destination. The default value is 300s.
bufferingSize (float) - Buffer incoming data to the specified size, in MBs between 1 to 100, before delivering it to the destination. The default value is 5MB.
cloudwatch_logging_options (dict) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (bool) - Enables or disables the logging. Defaults to false.
- log_group_name (str) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (str) - The CloudWatch log stream name for logging. This value is required if enabled is true.
domainArn (str) - The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming RoleARN. The pattern needs to be arn:.*.
indexName (str) - The Elasticsearch index name.
indexRotationPeriod (str) - The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data. Valid values are NoRotation, OneHour, OneDay, OneWeek, and OneMonth. The default value is OneDay.
processingConfiguration (dict) - The data processing configuration. More details are given below.
- enabled (bool) - Enables or disables data processing.
- processors (list) - Array of data processors. More details are given below
  - parameters (list) - Array of processor parameters. More details are given below
    - parameterName (str) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (str) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (str) - The type of processor. Valid Values: Lambda
retryDuration (float) - After an initial failure to deliver to Amazon Elasticsearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
role_arn (str) - The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. The pattern needs to be arn:.*.
s3BackupMode (str) - Defines how documents should be delivered to Amazon S3. Valid values are FailedDocumentsOnly and AllDocuments. Default value is FailedDocumentsOnly.
typeName (str) - The Elasticsearch type name with maximum length of 100 characters.

extended_s3_configuration: pulumi.Output[dict] = None¶

Enhanced configuration options for the s3 destination. More details are given below.

bucketArn (str) - The ARN of the S3 bucket
bufferInterval (float) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize (float) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options (dict) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (bool) - Enables or disables the logging. Defaults to false.
- log_group_name (str) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (str) - The CloudWatch log stream name for logging. This value is required if enabled is true.
compressionFormat (str) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
dataFormatConversionConfiguration (dict) - Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
- enabled (bool) - Defaults to true. Set it to false if you want to disable format conversion while preserving the configuration details.
- inputFormatConfiguration (dict) - Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
  - deserializer (dict) - Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.
    - hiveJsonSerDe (dict) - Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
      - timestampFormats (list) - A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime’s DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don’t specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.
    - openXJsonSerDe (dict) - Nested argument that specifies the OpenX SerDe. More details below.
      - caseInsensitive (bool) - When set to true, which is the default, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them.
      - columnToJsonKeyMappings (dict) - A map of column names to JSON keys that aren’t identical to the column names. This is useful when the JSON contains keys that are Hive keywords. For example, timestamp is a Hive keyword. If you have a JSON key named timestamp, set this parameter to { ts = "timestamp" } to map this key to a column named ts.
      - convertDotsInJsonKeysToUnderscores (bool) - When set to true, specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. For example, if the JSON contains a key whose name is “a.b”, you can define the column name to be “a_b” when using this option. Defaults to false.
- outputFormatConfiguration (dict) - Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
  - serializer (dict) - Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.
    - orcSerDe (dict) - Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
      - blockSizeBytes (float) - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
      - bloomFilterColumns (list) - A list of column names for which you want Kinesis Data Firehose to create bloom filters.
      - bloomFilterFalsePositiveProbability (float) - The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
      - compression (str) - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
      - dictionaryKeyThreshold (float) - A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
      - enablePadding (bool) - Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.
      - formatVersion (str) - The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.
      - paddingTolerance (float) - A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size. For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task. Kinesis Data Firehose ignores this parameter when enable_padding is false.
      - rowIndexStride (float) - The number of rows between index entries. The default is 10000 and the minimum is 1000.
      - stripeSizeBytes (float) - The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.
    - parquetSerDe (dict) - Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.
      - blockSizeBytes (float) - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
      - compression (str) - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
      - enableDictionaryCompression (bool) - Indicates whether to enable dictionary compression.
      - maxPaddingBytes (float) - The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
      - pageSizeBytes (float) - The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
      - writerVersion (str) - Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.
- schemaConfiguration (dict) - Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
  - catalog_id (str) - The ID of the AWS Glue Data Catalog. If you don’t supply this, the AWS account ID is used by default.
  - database_name (str) - Specifies the name of the AWS Glue database that contains the schema for the output data.
  - region (str) - If you don’t specify an AWS Region, the default is the current region.
  - role_arn (str) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
  - table_name (str) - Specifies the AWS Glue table that contains the column information that constitutes your data schema.
  - version_id (str) - Specifies the table version for the output data schema. Defaults to LATEST.
errorOutputPrefix (str) - Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name.
kms_key_arn (str) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix (str) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
processingConfiguration (dict) - The data processing configuration. More details are given below.
- enabled (bool) - Enables or disables data processing.
- processors (list) - Array of data processors. More details are given below
  - parameters (list) - Array of processor parameters. More details are given below
    - parameterName (str) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (str) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (str) - The type of processor. Valid Values: Lambda
role_arn (str) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupConfiguration (dict) - The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
- bucketArn (str) - The ARN of the S3 bucket
- bufferInterval (float) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
- bufferSize (float) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
- cloudwatch_logging_options (dict) - The CloudWatch Logging Options for the delivery stream. More details are given below
  - enabled (bool) - Enables or disables the logging. Defaults to false.
  - log_group_name (str) - The CloudWatch group name for logging. This value is required if enabled is true.
  - logStreamName (str) - The CloudWatch log stream name for logging. This value is required if enabled is true.
- compressionFormat (str) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
- kms_key_arn (str) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
- prefix (str) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
- role_arn (str) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupMode (str) - The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

kinesis_source_configuration: pulumi.Output[dict] = None¶

Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.

kinesisStreamArn (str) - The kinesis stream used as the source of the firehose delivery stream.
role_arn (str) - The ARN of the role that provides access to the source Kinesis stream.

name: pulumi.Output[str] = None¶: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.

redshift_configuration: pulumi.Output[dict] = None¶

Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.

cloudwatch_logging_options (dict) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (bool) - Enables or disables the logging. Defaults to false.
- log_group_name (str) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (str) - The CloudWatch log stream name for logging. This value is required if enabled is true.
clusterJdbcurl (str) - The jdbcurl of the redshift cluster.
copyOptions (str) - Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter. For valid values, see the AWS documentation
dataTableColumns (str) - The data table columns that will be targeted by the copy command.
dataTableName (str) - The name of the table in the redshift cluster that the s3 bucket will copy to.
password (str) - The password for the username above.
processingConfiguration (dict) - The data processing configuration. More details are given below.
- enabled (bool) - Enables or disables data processing.
- processors (list) - Array of data processors. More details are given below
  - parameters (list) - Array of processor parameters. More details are given below
    - parameterName (str) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (str) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (str) - The type of processor. Valid Values: Lambda
retryDuration (float) - The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value of DurationInSeconds is 0 (zero) or if the first delivery attempt takes longer than the current value.
role_arn (str) - The arn of the role the stream assumes.
s3BackupConfiguration (dict) - The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
- bucketArn (str) - The ARN of the S3 bucket
- bufferInterval (float) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
- bufferSize (float) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
- cloudwatch_logging_options (dict) - The CloudWatch Logging Options for the delivery stream. More details are given below
  - enabled (bool) - Enables or disables the logging. Defaults to false.
  - log_group_name (str) - The CloudWatch group name for logging. This value is required if enabled is true.
  - logStreamName (str) - The CloudWatch log stream name for logging. This value is required if enabled is true.
- compressionFormat (str) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
- kms_key_arn (str) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
- prefix (str) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
- role_arn (str) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupMode (str) - The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.
username (str) - The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.

s3_configuration: pulumi.Output[dict] = None¶

Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.

bucketArn (str) - The ARN of the S3 bucket
bufferInterval (float) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize (float) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options (dict) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (bool) - Enables or disables the logging. Defaults to false.
- log_group_name (str) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (str) - The CloudWatch log stream name for logging. This value is required if enabled is true.
compressionFormat (str) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kms_key_arn (str) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix (str) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
role_arn (str) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.

server_side_encryption: pulumi.Output[dict] = None¶

Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.

enabled (bool) - Whether to enable encryption at rest. Default is false.

tags: pulumi.Output[dict] = None¶: A map of tags to assign to the resource.

version_id: pulumi.Output[str] = None¶: Specifies the table version for the output data schema. Defaults to LATEST.

static get(resource_name, id, opts=None, arn=None, destination=None, destination_id=None, elasticsearch_configuration=None, extended_s3_configuration=None, kinesis_source_configuration=None, name=None, redshift_configuration=None, s3_configuration=None, server_side_encryption=None, splunk_configuration=None, tags=None, version_id=None)¶

Get an existing FirehoseDeliveryStream resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters

resource_name (str) – The unique name of the resulting resource.
id (str) – The unique provider ID of the resource to lookup.
opts (pulumi.ResourceOptions) – Options for the resource.
arn (pulumi.Input[str]) – The Amazon Resource Name (ARN) specifying the Stream
destination (pulumi.Input[str]) – This is the destination to where the data is delivered. The only options are s3 (Deprecated, use extended_s3 instead), extended_s3, redshift, elasticsearch, and splunk.
elasticsearch_configuration (pulumi.Input[dict]) – Configuration options if elasticsearch is the destination. More details are given below.
extended_s3_configuration (pulumi.Input[dict]) – Enhanced configuration options for the s3 destination. More details are given below.
kinesis_source_configuration (pulumi.Input[dict]) – Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
name (pulumi.Input[str]) – A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
redshift_configuration (pulumi.Input[dict]) – Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
s3_configuration (pulumi.Input[dict]) – Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
server_side_encryption (pulumi.Input[dict]) – Encrypt at rest options. Server-side encryption should not be enabled when a kinesis stream is configured as the source of the firehose delivery stream.
tags (pulumi.Input[dict]) – A map of tags to assign to the resource.
version_id (pulumi.Input[str]) – Specifies the table version for the output data schema. Defaults to LATEST.

The elasticsearch_configuration object supports the following:

bufferingInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds between 60 to 900, before delivering it to the destination. The default value is 300s.
bufferingSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs between 1 to 100, before delivering it to the destination. The default value is 5MB.
cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
domainArn (pulumi.Input[str]) - The ARN of the Amazon ES domain. The IAM role must have permission for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming RoleARN. The pattern needs to be arn:.*.
indexName (pulumi.Input[str]) - The Elasticsearch index name.
indexRotationPeriod (pulumi.Input[str]) - The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data. Valid values are NoRotation, OneHour, OneDay, OneWeek, and OneMonth. The default value is OneDay.
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
retryDuration (pulumi.Input[float]) - After an initial failure to deliver to Amazon Elasticsearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
role_arn (pulumi.Input[str]) - The ARN of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. The pattern needs to be arn:.*.
s3BackupMode (pulumi.Input[str]) - Defines how documents should be delivered to Amazon S3. Valid values are FailedDocumentsOnly and AllDocuments. Default value is FailedDocumentsOnly.
typeName (pulumi.Input[str]) - The Elasticsearch type name with maximum length of 100 characters.

The extended_s3_configuration object supports the following:

bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
dataFormatConversionConfiguration (pulumi.Input[dict]) - Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
- enabled (pulumi.Input[bool]) - Defaults to true. Set it to false if you want to disable format conversion while preserving the configuration details.
- inputFormatConfiguration (pulumi.Input[dict]) - Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
  - deserializer (pulumi.Input[dict]) - Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.
    - hiveJsonSerDe (pulumi.Input[dict]) - Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
      - timestampFormats (pulumi.Input[list]) - A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime’s DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don’t specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.
    - openXJsonSerDe (pulumi.Input[dict]) - Nested argument that specifies the OpenX SerDe. More details below.
      - caseInsensitive (pulumi.Input[bool]) - When set to true, which is the default, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them.
      - columnToJsonKeyMappings (pulumi.Input[dict]) - A map of column names to JSON keys that aren’t identical to the column names. This is useful when the JSON contains keys that are Hive keywords. For example, timestamp is a Hive keyword. If you have a JSON key named timestamp, set this parameter to { ts = "timestamp" } to map this key to a column named ts.
      - convertDotsInJsonKeysToUnderscores (pulumi.Input[bool]) - When set to true, specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. For example, if the JSON contains a key whose name is “a.b”, you can define the column name to be “a_b” when using this option. Defaults to false.
- outputFormatConfiguration (pulumi.Input[dict]) - Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
  - serializer (pulumi.Input[dict]) - Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.
    - orcSerDe (pulumi.Input[dict]) - Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
      - blockSizeBytes (pulumi.Input[float]) - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
      - bloomFilterColumns (pulumi.Input[list]) - A list of column names for which you want Kinesis Data Firehose to create bloom filters.
      - bloomFilterFalsePositiveProbability (pulumi.Input[float]) - The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
      - compression (pulumi.Input[str]) - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
      - dictionaryKeyThreshold (pulumi.Input[float]) - A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
      - enablePadding (pulumi.Input[bool]) - Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.
      - formatVersion (pulumi.Input[str]) - The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.
      - paddingTolerance (pulumi.Input[float]) - A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size. For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task. Kinesis Data Firehose ignores this parameter when enable_padding is false.
      - rowIndexStride (pulumi.Input[float]) - The number of rows between index entries. The default is 10000 and the minimum is 1000.
      - stripeSizeBytes (pulumi.Input[float]) - The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.
    - parquetSerDe (pulumi.Input[dict]) - Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.
      - blockSizeBytes (pulumi.Input[float]) - The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
      - compression (pulumi.Input[str]) - The compression code to use over data blocks. The possible values are UNCOMPRESSED, SNAPPY, and GZIP, with the default being SNAPPY. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.
      - enableDictionaryCompression (pulumi.Input[bool]) - Indicates whether to enable dictionary compression.
      - maxPaddingBytes (pulumi.Input[float]) - The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.
      - pageSizeBytes (pulumi.Input[float]) - The Parquet page size. Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.
      - writerVersion (pulumi.Input[str]) - Indicates the version of row format to output. The possible values are V1 and V2. The default is V1.
- schemaConfiguration (pulumi.Input[dict]) - Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
  - catalog_id (pulumi.Input[str]) - The ID of the AWS Glue Data Catalog. If you don’t supply this, the AWS account ID is used by default.
  - database_name (pulumi.Input[str]) - Specifies the name of the AWS Glue database that contains the schema for the output data.
  - region (pulumi.Input[str]) - If you don’t specify an AWS Region, the default is the current region.
  - role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
  - table_name (pulumi.Input[str]) - Specifies the AWS Glue table that contains the column information that constitutes your data schema.
  - version_id (pulumi.Input[str]) - Specifies the table version for the output data schema. Defaults to LATEST.
errorOutputPrefix (pulumi.Input[str]) - Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name.
kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupConfiguration (pulumi.Input[dict]) - The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
- bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
- bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
- bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
- cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
  - enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
  - log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
  - logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
- compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
- kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
- prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
- role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupMode (pulumi.Input[str]) - The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.

The kinesis_source_configuration object supports the following:

kinesisStreamArn (pulumi.Input[str]) - The kinesis stream used as the source of the firehose delivery stream.
role_arn (pulumi.Input[str]) - The ARN of the role that provides access to the source Kinesis stream.

The redshift_configuration object supports the following:

cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
clusterJdbcurl (pulumi.Input[str]) - The jdbcurl of the redshift cluster.
copyOptions (pulumi.Input[str]) - Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter. For valid values, see the AWS documentation
dataTableColumns (pulumi.Input[str]) - The data table columns that will be targeted by the copy command.
dataTableName (pulumi.Input[str]) - The name of the table in the redshift cluster that the s3 bucket will copy to.
password (pulumi.Input[str]) - The password for the username above.
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
retryDuration (pulumi.Input[float]) - The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. The default value is 3600 seconds (60 minutes). Firehose does not retry if the value of DurationInSeconds is 0 (zero) or if the first delivery attempt takes longer than the current value.
role_arn (pulumi.Input[str]) - The arn of the role the stream assumes.
s3BackupConfiguration (pulumi.Input[dict]) - The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
- bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
- bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
- bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
- cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
  - enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
  - log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
  - logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
- compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
- kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
- prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
- role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.
s3BackupMode (pulumi.Input[str]) - The Amazon S3 backup mode. Valid values are Disabled and Enabled. Default value is Disabled.
username (pulumi.Input[str]) - The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.

The s3_configuration object supports the following:

bucketArn (pulumi.Input[str]) - The ARN of the S3 bucket
bufferInterval (pulumi.Input[float]) - Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300.
bufferSize (pulumi.Input[float]) - Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher.
cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
compressionFormat (pulumi.Input[str]) - The compression format. If no value is specified, the default is UNCOMPRESSED. Other supported values are GZIP, ZIP & Snappy. If the destination is redshift you cannot use ZIP or Snappy.
kms_key_arn (pulumi.Input[str]) - Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used.
prefix (pulumi.Input[str]) - The “YYYY/MM/DD/HH” time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket
role_arn (pulumi.Input[str]) - The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren’t allowed.

The server_side_encryption object supports the following:

enabled (pulumi.Input[bool]) - Whether to enable encryption at rest. Default is false.

The splunk_configuration object supports the following:

cloudwatch_logging_options (pulumi.Input[dict]) - The CloudWatch Logging Options for the delivery stream. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables the logging. Defaults to false.
- log_group_name (pulumi.Input[str]) - The CloudWatch group name for logging. This value is required if enabled is true.
- logStreamName (pulumi.Input[str]) - The CloudWatch log stream name for logging. This value is required if enabled is true.
hecAcknowledgmentTimeout (pulumi.Input[float]) - The amount of time, in seconds between 180 and 600, that Kinesis Firehose waits to receive an acknowledgment from Splunk after it sends it data.
hecEndpoint (pulumi.Input[str]) - The HTTP Event Collector (HEC) endpoint to which Kinesis Firehose sends your data.
hecEndpointType (pulumi.Input[str]) - The HEC endpoint type. Valid values are Raw or Event. The default value is Raw.
hecToken (pulumi.Input[str]) - The GUID that you obtain from your Splunk cluster when you create a new HEC endpoint.
processingConfiguration (pulumi.Input[dict]) - The data processing configuration. More details are given below.
- enabled (pulumi.Input[bool]) - Enables or disables data processing.
- processors (pulumi.Input[list]) - Array of data processors. More details are given below
  - parameters (pulumi.Input[list]) - Array of processor parameters. More details are given below
    - parameterName (pulumi.Input[str]) - Parameter name. Valid Values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds
    - parameterValue (pulumi.Input[str]) - Parameter value. Must be between 1 and 512 length (inclusive). When providing a Lambda ARN, you should specify the resource version as well.
  - type (pulumi.Input[str]) - The type of processor. Valid Values: Lambda
retryDuration (pulumi.Input[float]) - After an initial failure to deliver to Splunk, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0.
s3BackupMode (pulumi.Input[str]) - Defines how documents should be delivered to Amazon S3. Valid values are FailedEventsOnly and AllEvents. Default value is FailedEventsOnly.

translate_output_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

translate_input_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

class pulumi_aws.kinesis.GetStreamResult(arn=None, closed_shards=None, creation_timestamp=None, id=None, name=None, open_shards=None, retention_period=None, shard_level_metrics=None, status=None, tags=None)¶

A collection of values returned by getStream.

arn = None¶: The Amazon Resource Name (ARN) of the Kinesis Stream (same as id).

closed_shards = None¶: The list of shard ids in the CLOSED state. See Shard State for more.

creation_timestamp = None¶: The approximate UNIX timestamp that the stream was created.

id = None¶: The provider-assigned unique ID for this managed resource.

name = None¶: The name of the Kinesis Stream.

open_shards = None¶: The list of shard ids in the OPEN state. See Shard State for more.

retention_period = None¶: Length of time (in hours) data records are accessible after they are added to the stream.

shard_level_metrics = None¶: A list of shard-level CloudWatch metrics which are enabled for the stream. See Monitoring with CloudWatch for more.

status = None¶: The current status of the stream. The stream status is one of CREATING, DELETING, ACTIVE, or UPDATING.

tags = None¶: A map of tags to assigned to the stream.

class pulumi_aws.kinesis.Stream(resource_name, opts=None, arn=None, encryption_type=None, enforce_consumer_deletion=None, kms_key_id=None, name=None, retention_period=None, shard_count=None, shard_level_metrics=None, tags=None, __props__=None, __name__=None, __opts__=None)¶

Provides a Kinesis Stream resource. Amazon Kinesis is a managed service that scales elastically for real-time processing of streaming big data.

For more details, see the Amazon Kinesis Documentation.

import pulumi
import pulumi_aws as aws

test_stream = aws.kinesis.Stream("testStream",
    retention_period=48,
    shard_count=1,
    shard_level_metrics=[
        "IncomingBytes",
        "OutgoingBytes",
    ],
    tags={
        "Environment": "test",
    })

Parameters

resource_name (str) – The name of the resource.
opts (pulumi.ResourceOptions) – Options for the resource.
arn (pulumi.Input[str]) – The Amazon Resource Name (ARN) specifying the Stream (same as id)
encryption_type (pulumi.Input[str]) – The encryption type to use. The only acceptable values are NONE or KMS. The default value is NONE.
enforce_consumer_deletion (pulumi.Input[bool]) – A boolean that indicates all registered consumers should be deregistered from the stream so that the stream can be destroyed without error. The default value is false.
kms_key_id (pulumi.Input[str]) – The GUID for the customer-managed KMS key to use for encryption. You can also use a Kinesis-owned master key by specifying the alias alias/aws/kinesis.
name (pulumi.Input[str]) – A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
retention_period (pulumi.Input[float]) – Length of time data records are accessible after they are added to the stream. The maximum value of a stream’s retention period is 168 hours. Minimum value is 24. Default is 24.
shard_count (pulumi.Input[float]) – The number of shards that the stream will use. Amazon has guidelines for specifying the Stream size that should be referenced when creating a Kinesis stream. See Amazon Kinesis Streams for more.
shard_level_metrics (pulumi.Input[list]) –
A list of shard-level CloudWatch metrics which can be enabled for the stream. See Monitoring with CloudWatch for more. Note that the value ALL should not be used; instead you should provide an explicit list of metrics you wish to enable.
tags (pulumi.Input[dict]) – A map of tags to assign to the resource.

arn: pulumi.Output[str] = None¶: The Amazon Resource Name (ARN) specifying the Stream (same as id)

encryption_type: pulumi.Output[str] = None¶: The encryption type to use. The only acceptable values are NONE or KMS. The default value is NONE.

enforce_consumer_deletion: pulumi.Output[bool] = None¶: A boolean that indicates all registered consumers should be deregistered from the stream so that the stream can be destroyed without error. The default value is false.

kms_key_id: pulumi.Output[str] = None¶: The GUID for the customer-managed KMS key to use for encryption. You can also use a Kinesis-owned master key by specifying the alias alias/aws/kinesis.

name: pulumi.Output[str] = None¶: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.

retention_period: pulumi.Output[float] = None¶: Length of time data records are accessible after they are added to the stream. The maximum value of a stream’s retention period is 168 hours. Minimum value is 24. Default is 24.

shard_count: pulumi.Output[float] = None¶: The number of shards that the stream will use. Amazon has guidelines for specifying the Stream size that should be referenced when creating a Kinesis stream. See Amazon Kinesis Streams for more.

shard_level_metrics: pulumi.Output[list] = None¶: A list of shard-level CloudWatch metrics which can be enabled for the stream. See Monitoring with CloudWatch for more. Note that the value ALL should not be used; instead you should provide an explicit list of metrics you wish to enable.

tags: pulumi.Output[dict] = None¶: A map of tags to assign to the resource.

static get(resource_name, id, opts=None, arn=None, encryption_type=None, enforce_consumer_deletion=None, kms_key_id=None, name=None, retention_period=None, shard_count=None, shard_level_metrics=None, tags=None)¶

Get an existing Stream resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters

resource_name (str) – The unique name of the resulting resource.
id (str) – The unique provider ID of the resource to lookup.
opts (pulumi.ResourceOptions) – Options for the resource.
arn (pulumi.Input[str]) – The Amazon Resource Name (ARN) specifying the Stream (same as id)
encryption_type (pulumi.Input[str]) – The encryption type to use. The only acceptable values are NONE or KMS. The default value is NONE.
enforce_consumer_deletion (pulumi.Input[bool]) – A boolean that indicates all registered consumers should be deregistered from the stream so that the stream can be destroyed without error. The default value is false.
kms_key_id (pulumi.Input[str]) – The GUID for the customer-managed KMS key to use for encryption. You can also use a Kinesis-owned master key by specifying the alias alias/aws/kinesis.
name (pulumi.Input[str]) – A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
retention_period (pulumi.Input[float]) – Length of time data records are accessible after they are added to the stream. The maximum value of a stream’s retention period is 168 hours. Minimum value is 24. Default is 24.
shard_count (pulumi.Input[float]) –
The number of shards that the stream will use. Amazon has guidelines for specifying the Stream size that should be referenced when creating a Kinesis stream. See Amazon Kinesis Streams for more.
shard_level_metrics (pulumi.Input[list]) –
A list of shard-level CloudWatch metrics which can be enabled for the stream. See Monitoring with CloudWatch for more. Note that the value ALL should not be used; instead you should provide an explicit list of metrics you wish to enable.
tags (pulumi.Input[dict]) – A map of tags to assign to the resource.

translate_output_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

translate_input_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

class pulumi_aws.kinesis.VideoStream(resource_name, opts=None, data_retention_in_hours=None, device_name=None, kms_key_id=None, media_type=None, name=None, tags=None, __props__=None, __name__=None, __opts__=None)¶

Provides a Kinesis Video Stream resource. Amazon Kinesis Video Streams makes it easy to securely stream video from connected devices to AWS for analytics, machine learning (ML), playback, and other processing.

For more details, see the Amazon Kinesis Documentation.

import pulumi
import pulumi_aws as aws

default = aws.kinesis.VideoStream("default",
    data_retention_in_hours=1,
    device_name="kinesis-video-device-name",
    media_type="video/h264",
    tags={
        "Name": "kinesis-video-stream",
    })

Parameters

resource_name (str) – The name of the resource.
opts (pulumi.ResourceOptions) – Options for the resource.
data_retention_in_hours (pulumi.Input[float]) – The number of hours that you want to retain the data in the stream. Kinesis Video Streams retains the data in a data store that is associated with the stream. The default value is 0, indicating that the stream does not persist data.
device_name (pulumi.Input[str]) – The name of the device that is writing to the stream. In the current implementation, Kinesis Video Streams does not use this name.
kms_key_id (pulumi.Input[str]) – The ID of the AWS Key Management Service (AWS KMS) key that you want Kinesis Video Streams to use to encrypt stream data. If no key ID is specified, the default, Kinesis Video-managed key (aws/kinesisvideo) is used.
media_type (pulumi.Input[str]) – The media type of the stream. Consumers of the stream can use this information when processing the stream. For more information about media types, see Media Types. If you choose to specify the MediaType, see Naming Requirements for guidelines.
name (pulumi.Input[str]) – A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
tags (pulumi.Input[dict]) – A map of tags to assign to the resource.

arn: pulumi.Output[str] = None¶: The Amazon Resource Name (ARN) specifying the Stream (same as id)

creation_time: pulumi.Output[str] = None¶: A time stamp that indicates when the stream was created.

data_retention_in_hours: pulumi.Output[float] = None¶: The number of hours that you want to retain the data in the stream. Kinesis Video Streams retains the data in a data store that is associated with the stream. The default value is 0, indicating that the stream does not persist data.

device_name: pulumi.Output[str] = None¶: The name of the device that is writing to the stream. In the current implementation, Kinesis Video Streams does not use this name.

kms_key_id: pulumi.Output[str] = None¶: The ID of the AWS Key Management Service (AWS KMS) key that you want Kinesis Video Streams to use to encrypt stream data. If no key ID is specified, the default, Kinesis Video-managed key (aws/kinesisvideo) is used.

media_type: pulumi.Output[str] = None¶: The media type of the stream. Consumers of the stream can use this information when processing the stream. For more information about media types, see Media Types. If you choose to specify the MediaType, see Naming Requirements for guidelines.

name: pulumi.Output[str] = None¶: A name to identify the stream. This is unique to the AWS account and region the Stream is created in.

tags: pulumi.Output[dict] = None¶: A map of tags to assign to the resource.

version: pulumi.Output[str] = None¶: The version of the stream.

static get(resource_name, id, opts=None, arn=None, creation_time=None, data_retention_in_hours=None, device_name=None, kms_key_id=None, media_type=None, name=None, tags=None, version=None)¶

Get an existing VideoStream resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters

resource_name (str) – The unique name of the resulting resource.
id (str) – The unique provider ID of the resource to lookup.
opts (pulumi.ResourceOptions) – Options for the resource.
arn (pulumi.Input[str]) – The Amazon Resource Name (ARN) specifying the Stream (same as id)
creation_time (pulumi.Input[str]) – A time stamp that indicates when the stream was created.
data_retention_in_hours (pulumi.Input[float]) – The number of hours that you want to retain the data in the stream. Kinesis Video Streams retains the data in a data store that is associated with the stream. The default value is 0, indicating that the stream does not persist data.
device_name (pulumi.Input[str]) – The name of the device that is writing to the stream. In the current implementation, Kinesis Video Streams does not use this name.
kms_key_id (pulumi.Input[str]) – The ID of the AWS Key Management Service (AWS KMS) key that you want Kinesis Video Streams to use to encrypt stream data. If no key ID is specified, the default, Kinesis Video-managed key (aws/kinesisvideo) is used.
media_type (pulumi.Input[str]) –
The media type of the stream. Consumers of the stream can use this information when processing the stream. For more information about media types, see Media Types. If you choose to specify the MediaType, see Naming Requirements for guidelines.
name (pulumi.Input[str]) – A name to identify the stream. This is unique to the AWS account and region the Stream is created in.
tags (pulumi.Input[dict]) – A map of tags to assign to the resource.
version (pulumi.Input[str]) – The version of the stream.

translate_output_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

translate_input_property(prop)¶

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters: prop (str) – A property name.
Returns: A potentially transformed property name.
Return type: str

pulumi_aws.kinesis.get_stream(name=None, tags=None, opts=None)¶

Use this data source to get information about a Kinesis Stream for use in other resources.

For more details, see the Amazon Kinesis Documentation.

import pulumi
import pulumi_aws as aws

stream = aws.kinesis.get_stream(name="stream-name")

Parameters

name (str) – The name of the Kinesis Stream.
tags (dict) – A map of tags to assigned to the stream.

The Pulumi Platform

Get Started

Migrate to Pulumi

Solutions for All Teams and Engineers

Any Code

Any Cloud

kinesis¶

On This Page