Skip to main content
Data Storage Modules 0.40.0Last updated in version 0.39.0

Redshift Module

View Source Release Notes

This module creates an Amazon Redshift cluster that you can use as a data warehouse. The cluster is managed by AWS and automatically handles leader nodes, worker nodes, backups, patching, and encryption.

imgimg

About Redshift

Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. Refer to the following resources to get more information:

Serverless

Amazon Redshift Serverless makes it convenient for you to run and scale analytics without having to provision and manage data warehouses. Use the var.enable_serverless to enable serverless and the var.serverless_base_capacity to set teh base Redshift Processing Units (RPU) for serving queries.

Refer to the Amazon Redshift Serverless page for more information.

Note: it seems like the terraform destroy command is not working smoothly for Redshift serverless feature yet (hashicorp/terraform-provider-aws#29962). Users would likely encounter error when trying to delete a workgroup associated with the Redshift serverless namespace. As a workaround, you can re-run the destroy command once the workspace gets deleted completely (e.g., after 30 mins).

Sample Usage

main.tf

# ------------------------------------------------------------------------------------------------------
# DEPLOY GRUNTWORK'S REDSHIFT MODULE
# ------------------------------------------------------------------------------------------------------

module "redshift" {

source = "git::git@github.com:gruntwork-io/terraform-aws-data-storage.git//modules/redshift?ref=v0.40.0"

# ----------------------------------------------------------------------------------------------------
# REQUIRED VARIABLES
# ----------------------------------------------------------------------------------------------------

# The name used to namespace all resources created by these templates,
# including the DB instance (e.g. drupaldb). Must be unique for this region.
# May contain only lowercase alphanumeric characters, hyphens.
name = <string>

# A list of subnet ids where the database should be deployed. In the standard
# Gruntwork VPC setup, these should be the private persistence subnet ids.
subnet_ids = <list(string)>

# The id of the VPC in which this DB should be deployed.
vpc_id = <string>

# ----------------------------------------------------------------------------------------------------
# OPTIONAL VARIABLES
# ----------------------------------------------------------------------------------------------------

# A list of CIDR-formatted IP address ranges that can connect to this DB.
# Should typically be the CIDR blocks of the private app subnet in this VPC
# plus the private subnet in the mgmt VPC. This is ignored if
# create_subnet_group=false.
allow_connections_from_cidr_blocks = []

# A list of Security Groups that can connect to this DB.
allow_connections_from_security_groups = []

# Indicates whether major version upgrades (e.g. 9.4.x to 9.5.x) will ever be
# permitted. Note that these updates must always be manually performed and
# will never automatically applied.
allow_major_version_upgrade = true

# A list of CIDR-formatted IP address ranges that this DB can connect. Use
# this if the database needs to connect to certain IP addresses for special
# operation
allow_outbound_connections_from_cidr_blocks = []

# Specifies whether any cluster modifications are applied immediately, or
# during the next maintenance window.
apply_immediately = false

# Indicates that minor engine upgrades will be applied automatically to the DB
# instance during the maintenance window. If set to true, you should set
# var.engine_version to MAJOR.MINOR and omit the .PATCH at the end (e.g., use
# 5.7 and not 5.7.11); otherwise, you'll get Terraform state drift. See
# https://www.terraform.io/docs/providers/aws/r/db_instance.html#engine_version
# for more details.
auto_minor_version_upgrade = true

# If true, the cluster can be relocated to another availabity zone, either
# automatically by AWS or when requested. Default is false. Available for use
# on clusters from the RA3 instance family.
availability_zone_relocation_enabled = null

# The description of the aws_db_security_group that is created. Defaults to
# 'Security group for the var.name DB' if not specified.
aws_db_security_group_description = null

# The name of the aws_db_security_group that is created. Defaults to var.name
# if not specified.
aws_db_security_group_name = null

# How many days to keep backup snapshots around before cleaning them up. Must
# be 1 or greater to support read replicas.
backup_retention_period = 21

# The description of the cluster_subnet_group that is created. Defaults to
# 'Subnet group for the var.name DB' if not specified.
cluster_subnet_group_description = null

# The name of the cluster_subnet_group that is created, or an existing one to
# use if cluster_subnet_group is false. Defaults to var.name if not specified.
cluster_subnet_group_name = null

# If false, the DB will bind to aws_db_subnet_group_name and the CIDR will be
# ignored (allow_connections_from_cidr_blocks)
create_subnet_group = true

# Timeout for DB creating
creating_timeout = "75m"

# A map of custom tags to apply to the RDS Instance and the Security Group
# created for it. The key is the tag name and the value is the tag value.
custom_tags = {}

# The name for your database of up to 8 alpha-numeric characters. If you do
# not provide a name, Amazon RDS will not create a database in the DB cluster
# you are creating.
db_name = "dev"

# Timeout for DB deleting
deleting_timeout = "40m"

# Elastic IP that will be associated with the cluster
elastic_ip = null

# Whether to enable serverless feature or not. Refer to
# https://docs.aws.amazon.com/redshift/latest/gsg/new-user-serverless.html for
# more information.
enable_serverless = false

# If true , enhanced VPC routing is enabled. Forces COPY and UNLOAD traffic
# between the cluster and data repositories to go through your VPC.
enhanced_vpc_routing = false

# The name of the final_snapshot_identifier. Defaults to
# var.name-final-snapshot if not specified.
final_snapshot_name = null

# A list of IAM Role ARNs to associate with the cluster. A Maximum of 10 can
# be associated to the cluster at any time.
iam_roles = null

# The instance type to use for the db (e.g. dc2.large). This field is
# mandatory for provisioned Redshift.
instance_type = null

# The ARN of a KMS key that should be used to encrypt data on disk. Only used
# if var.storage_encrypted is true. If you leave this blank, the default RDS
# KMS key for the account will be used.
kms_key_arn = null

# Required when log_destination_type is s3. Name of an existing S3 bucket
# where the log files are to be stored. Must be in the same region as the
# cluster and the cluster must have read bucket and put object permissions.
logging_bucket_name = null

# Boolean to toggle database audit logging.
logging_enable = false

# Type of the rule group. Valid values: "s3", "cloudwatch"
logging_log_destination_type = null

# Required when log_destination_type is cloudwatch. Collection of exported log
# types. See variable definition for details
logging_log_exports = null

# Required when log_destination_type is s3. Prefix applied to the log file
# names.
logging_s3_key_prefix = null

# The weekly day and time range during which system maintenance can occur
# (e.g. wed:04:00-wed:04:30). Time zone is UTC. Performance may be degraded or
# there may even be a downtime during maintenance windows.
maintenance_window = "sun:07:00-sun:08:00"

# The password for the master user. If var.snapshot_identifier is non-empty,
# this value is ignored. Required unless var.replicate_source_db is set.
master_password = null

# The username for the master user. Required unless var.replicate_source_db is
# set.
master_username = null

# The number of nodes in the cluster. This field is mandatory for provisioned
# Redshift.
number_of_nodes = null

# Name of a Redshift parameter group to associate.
parameter_group_name = null

# The port the DB will listen on (e.g. 3306)
port = 5439

# WARNING: - In nearly all cases a database should NOT be publicly accessible.
# Only set this to true if you want the database open to the internet.
publicly_accessible = false

# This setting specifies the base data warehouse capacity Amazon Redshift uses
# to serve queries. Base capacity is specified in RPUs. You can set a base
# capacity in Redshift Processing Units (RPUs). One RPU provides 16 GB of
# memory. You can adjust the Base capacity setting from 8 RPUs to 512 RPUs in
# units of 8. This field is mandatory for serverless.
serverless_base_capacity = null

# Determines whether a final DB snapshot is created before the DB instance is
# deleted. Be very careful setting this to true; if you do, and you delete
# this DB instance, you will not have any backups of the data!
skip_final_snapshot = false

# If non-null, the name of the cluster the source snapshot was created from.
snapshot_cluster_identifier = null

# Configuration of automatic copy of snapshots from one region to another. See
# https://registry.terraform.io/providers/hashicorp/aws/5.40.0/docs/resources/redshift_cluster#snapshot_copy
# for more detail
snapshot_copy = null

# If non-null, the Redshift cluster will be restored from the given Snapshot
# ID. This is the Snapshot ID you'd find in the Redshift console, e.g:
# rs:production-2015-06-26-06-05.
snapshot_identifier = null

# Required if you are restoring a snapshot you do not own, optional if you own
# the snapshot. The AWS customer account used to create or copy the snapshot.
snapshot_owner_account = null

# Automatic snapshot schedule definition. See
# https://registry.terraform.io/providers/hashicorp/aws/5.40.0/docs/resources/redshift_snapshot_schedule#definitions
# for more detail
snapshot_schedule_definitions = []

# Specifies whether the DB instance is encrypted.
storage_encrypted = true

# Timeout for DB updating
updating_timeout = "75m"

}


Reference

Required

namestringrequired

The name used to namespace all resources created by these templates, including the DB instance (e.g. drupaldb). Must be unique for this region. May contain only lowercase alphanumeric characters, hyphens.

subnet_idslist(string)required

A list of subnet ids where the database should be deployed. In the standard Gruntwork VPC setup, these should be the private persistence subnet ids.

vpc_idstringrequired

The id of the VPC in which this DB should be deployed.

Optional

A list of CIDR-formatted IP address ranges that can connect to this DB. Should typically be the CIDR blocks of the private app subnet in this VPC plus the private subnet in the mgmt VPC. This is ignored if create_subnet_group=false.

[]

A list of Security Groups that can connect to this DB.

[]

Indicates whether major version upgrades (e.g. 9.4.x to 9.5.x) will ever be permitted. Note that these updates must always be manually performed and will never automatically applied.

true

A list of CIDR-formatted IP address ranges that this DB can connect. Use this if the database needs to connect to certain IP addresses for special operation

[]
apply_immediatelystringoptional

Specifies whether any cluster modifications are applied immediately, or during the next maintenance window.

false

Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window. If set to true, you should set engine_version to MAJOR.MINOR and omit the .PATCH at the end (e.g., use 5.7 and not 5.7.11); otherwise, you'll get Terraform state drift. See https://www.terraform.io/docs/providers/aws/r/db_instance.html#engine_version for more details.

true

If true, the cluster can be relocated to another availabity zone, either automatically by AWS or when requested. Default is false. Available for use on clusters from the RA3 instance family.

null

The description of the aws_db_security_group that is created. Defaults to 'Security group for the name DB' if not specified.

null

The name of the aws_db_security_group that is created. Defaults to name if not specified.

null

How many days to keep backup snapshots around before cleaning them up. Must be 1 or greater to support read replicas.

21

The description of the cluster_subnet_group that is created. Defaults to 'Subnet group for the name DB' if not specified.

null

The name of the cluster_subnet_group that is created, or an existing one to use if cluster_subnet_group is false. Defaults to name if not specified.

null

If false, the DB will bind to aws_db_subnet_group_name and the CIDR will be ignored (allow_connections_from_cidr_blocks)

true
creating_timeoutstringoptional

Timeout for DB creating

"75m"
custom_tagsmap(string)optional

A map of custom tags to apply to the RDS Instance and the Security Group created for it. The key is the tag name and the value is the tag value.

{}
db_namestringoptional

The name for your database of up to 8 alpha-numeric characters. If you do not provide a name, Amazon RDS will not create a database in the DB cluster you are creating.

"dev"
deleting_timeoutstringoptional

Timeout for DB deleting

"40m"
elastic_ipstringoptional

Elastic IP that will be associated with the cluster

null
enable_serverlessbooloptional

Whether to enable serverless feature or not. Refer to https://docs.aws.amazon.com/redshift/latest/gsg/new-user-serverless.html for more information.

false

If true , enhanced VPC routing is enabled. Forces COPY and UNLOAD traffic between the cluster and data repositories to go through your VPC.

false
final_snapshot_namestringoptional

The name of the final_snapshot_identifier. Defaults to name-final-snapshot if not specified.

null
iam_roleslist(string)optional

A list of IAM Role ARNs to associate with the cluster. A Maximum of 10 can be associated to the cluster at any time.

null
instance_typestringoptional

The instance type to use for the db (e.g. dc2.large). This field is mandatory for provisioned Redshift.

null
kms_key_arnstringoptional

The ARN of a KMS key that should be used to encrypt data on disk. Only used if storage_encrypted is true. If you leave this blank, the default RDS KMS key for the account will be used.

null
logging_bucket_namestringoptional

Required when log_destination_type is s3. Name of an existing S3 bucket where the log files are to be stored. Must be in the same region as the cluster and the cluster must have read bucket and put object permissions.

null
logging_enablebooloptional

Boolean to toggle database audit logging.

false

Type of the rule group. Valid values: 's3', 'cloudwatch'

null
logging_log_exportslist(string)optional

Required when log_destination_type is cloudwatch. Collection of exported log types. See variable definition for details

null
logging_s3_key_prefixstringoptional

Required when log_destination_type is s3. Prefix applied to the log file names.

null
maintenance_windowstringoptional

The weekly day and time range during which system maintenance can occur (e.g. wed:04:00-wed:04:30). Time zone is UTC. Performance may be degraded or there may even be a downtime during maintenance windows.

"sun:07:00-sun:08:00"
master_passwordstringoptional

The password for the master user. If snapshot_identifier is non-empty, this value is ignored. Required unless replicate_source_db is set.

null
master_usernamestringoptional

The username for the master user. Required unless replicate_source_db is set.

null
number_of_nodesnumberoptional

The number of nodes in the cluster. This field is mandatory for provisioned Redshift.

null
parameter_group_namestringoptional

Name of a Redshift parameter group to associate.

null
portnumberoptional

The port the DB will listen on (e.g. 3306)

5439

WARNING: - In nearly all cases a database should NOT be publicly accessible. Only set this to true if you want the database open to the internet.

false

This setting specifies the base data warehouse capacity Amazon Redshift uses to serve queries. Base capacity is specified in RPUs. You can set a base capacity in Redshift Processing Units (RPUs). One RPU provides 16 GB of memory. You can adjust the Base capacity setting from 8 RPUs to 512 RPUs in units of 8. This field is mandatory for serverless.

null

Determines whether a final DB snapshot is created before the DB instance is deleted. Be very careful setting this to true; if you do, and you delete this DB instance, you will not have any backups of the data!

false

If non-null, the name of the cluster the source snapshot was created from.

null
snapshot_copymap(any)optional

Configuration of automatic copy of snapshots from one region to another. See https://registry.terraform.io/providers/hashicorp/aws/5.40.0/docs/resources/redshift_cluster#snapshot_copy for more detail

Any types represent complex values of variable type. For details, please consult `variables.tf` in the source repo.
null
snapshot_identifierstringoptional

If non-null, the Redshift cluster will be restored from the given Snapshot ID. This is the Snapshot ID you'd find in the Redshift console, e.g: rs:production-2015-06-26-06-05.

null

Required if you are restoring a snapshot you do not own, optional if you own the snapshot. The AWS customer account used to create or copy the snapshot.

null
storage_encryptedbooloptional

Specifies whether the DB instance is encrypted.

true
updating_timeoutstringoptional

Timeout for DB updating

"75m"