A Handy Guide to AWS CloudFormation Snippets
AWS CloudFormation is a robust service that allows users to model and set up their Amazon Web Services resources. This article provides handy CloudFormation snippets for quick reference.
Table of Contents
EC2:
Launch an EC2 Instance with an IAM Role:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
VpcId:
Type: AWS::EC2::VPC::Id
Description: Select a VPC for the instance
SubnetId:
Type: AWS::EC2::Subnet::Id
Description: Select a subnet for the instance
ImageId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Description: ID of the AMI to use for the instance
Default: Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
InstanceType:
Type: String
Description: EC2 instance type
Default: t2.medium
AllowedValues:
- t2.micro
- t2.small
- t2.medium
KeyName:
Type: AWS::EC2::KeyPair::KeyName
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
ConstraintDescription: Must be the name of an existing EC2 KeyPair
AllowedIp:
Type: String
Description: Your public IP address
Default: 10.0.0.0/8
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
KeyName: !Ref KeyName
SubnetId: !Ref SubnetId
SecurityGroupIds:
- !Ref InstanceSecurityGroup
IamInstanceProfile: !Ref EC2InstanceProfile
Tags:
- Key: Name
Value: !Sub "${AWS::StackName}-ec2"
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
yum update -y
#pip3 install boto3
#pip3 install requests
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: "/"
Roles:
- !Ref EC2InstanceRole
EC2InstanceRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub "${AWS::StackName}-role"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
Policies:
- PolicyName: EC2InstancePolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: "*"
InstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: "Enable SSH access and HTTP access on the inbound port"
VpcId: !Ref VpcId
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: !Ref AllowedIp
IAM:
Basic Role with Policy:
AWSTemplateFormatVersion: 2010-09-09
Resources:
MyRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub "${AWS::StackName}-role"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
Policies:
- PolicyName: EC2InstancePolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: "*"
S3:
Create an S3 Bucket with SSE:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
DeliveryBucketName:
Description: 'Bucket name'
Type: String
Default: 'my-bucket'
Resources:
MyBucket:
Type: "AWS::S3::Bucket"
Properties:
BucketName: !Ref BucketName
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
Lambda:
Lambda With Inline Code:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Environment:
Type: String
Default: 'Development'
AllowedValues:
- 'Development'
- 'Production'
Resources:
LambdaRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- logs:*
Resource: '*'
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Role: !GetAtt LambdaRole.Arn
Runtime: python3.9
Timeout: 900
MemorySize: 128
Handler: index.lambda_handler
Code:
ZipFile: |
def lambda_handler(event, context):
return "Hello world!"
Environment:
Variables:
Environment: !Ref Environment
Tags:
- Key: Lambda
Value: MyLambdaFunction
- Key: Environment
Value: !Ref Environment
Kinesis:
Kinesis Data Stream:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
KinesisStreamName:
Description: This will be used to name the Kinesis DataStream
Type: String
Default: 'data-stream'
KinesisStreamRetensionHours:
Description: This will be used to set the retension hours
Type: Number
Default: 24
KinesisStreamShardCount:
Description: This will be used to set the shard count
Type: Number
Default: 1
Environment:
Type: String
Default: 'Development'
AllowedValues:
- 'Development'
- 'Production'
Resources:
KinesisDataStream:
Type: AWS::Kinesis::Stream
Properties:
Name: !Sub ${AWS::StackName}-${KinesisStreamName}
RetentionPeriodHours: !Ref KinesisStreamRetensionHours
ShardCount: !Ref KinesisStreamShardCount
StreamEncryption:
EncryptionType: KMS
KeyId: alias/aws/kinesis
Tags:
- Key: Environment
Value: !Ref Environment
Kinesis Firehose with Direct Put:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
DeliveryStreamName:
Description: This will be used to name the Kinesis delivery stream
Type: String
Default: 'delivery-stream'
DeliveryBucketName:
Description: 'Bucket where kinesis delivery stream will store data. It should be a name, not Arn'
Type: String
Default: 'delivery-data'
Resources:
KinesisDeliveryStreamIAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: ''
Effect: Allow
Principal:
Service: firehose.amazonaws.com
Action: sts:AssumeRole
KinesisDeliveryStreamIAMPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: !Sub ${AWS::StackName}-${DeliveryStreamName}
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- s3:PutObject
Resource: !Join [ '', ['arn:aws:s3:::', !Ref DeliveryBucketName, '/*'] ]
Roles:
- !Ref KinesisDeliveryStreamIAMRole
KinesisDeliveryStreamDirectPut:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
DeliveryStreamName: !Ref DeliveryStreamName
DeliveryStreamType: DirectPut
ExtendedS3DestinationConfiguration:
BucketARN: !Join [ '', ['arn:aws:s3:::', !Ref DeliveryBucketName] ]
BufferingHints:
SizeInMBs: 128
IntervalInSeconds: 900
CompressionFormat: UNCOMPRESSED
ErrorOutputPrefix: table/error/!{firehose:error-output-type}/dt=!{timestamp:yyyy'-'MM'-'dd}/h=!{timestamp:HH}/
Prefix: YYYY=!{partitionKeyFromQuery:YYYY}/MM=!{partitionKeyFromQuery:MM}//DD=!{partitionKeyFromQuery:DD}/HH=!{partitionKeyFromQuery:HH}/REGION=!{partitionKeyFromQuery:REGION}/SITEID=!{partitionKeyFromQuery:SITEID}/
RoleARN: !GetAtt KinesisDeliveryStreamIAMRole.Arn
DynamicPartitioningConfiguration:
Enabled: true
RetryOptions:
DurationInSeconds: 300
ProcessingConfiguration:
Enabled: true
Processors:
- Type: MetadataExtraction
Parameters:
- ParameterName: MetadataExtractionQuery
ParameterValue: '{YYYY : (.ts/1000) | strftime("%Y"), MM : (.ts/1000)
| strftime("%m"), DD : (.ts/1000) | strftime("%d"), HH: (.ts/1000)
| strftime("%H")}'
- ParameterName: JsonParsingEngine
ParameterValue: JQ-1.6
- Type: AppendDelimiterToRecord
Parameters:
- ParameterName: Delimiter
ParameterValue: "\\n"
Kinesis Firehose with Kinesis Stream as Source:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
DeliveryStreamName:
Description: This will be used to name the Kinesis delivery stream
Type: String
Default: 'delivery-stream'
DeliveryBucketName:
Description: 'Bucket where kinesis delivery stream will store data. It should be a name, not Arn'
Type: String
Default: 'delivery-data'
KinesisDataStream:
Description: Kinesis data stream ARN where from data will be read'
Resources:
KinesisDeliveryStreamIAMRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: ''
Effect: Allow
Principal:
Service: firehose.amazonaws.com
Action: sts:AssumeRole
KinesisDeliveryStreamIAMPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: !Sub ${AWS::StackName}-${DeliveryStreamName}
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- s3:PutObject
Resource: !Join [ '', ['arn:aws:s3:::', !Ref DeliveryBucketName, '/*'] ]
- Effect: Allow
Action:
- kinesis:DescribeStream
- kinesis:GetShardIterator
- kinesis:GetRecords
Resource:
- !GetAtt KinesisDataStream.Arn
Roles:
- !Ref KinesisDeliveryStreamIAMRole
KinesisDeliveryStreamWithSource:
Type: AWS::KinesisFirehose::DeliveryStream
Properties:
DeliveryStreamName: !Ref DeliveryStreamName
DeliveryStreamType: KinesisStreamAsSource
KinesisStreamSourceConfiguration:
KinesisStreamARN: !GetAtt KinesisDataStream.Arn
RoleARN: !GetAtt KinesisDeliveryStreamIAMRole.Arn
ExtendedS3DestinationConfiguration:
BucketARN: !Join [ '', ['arn:aws:s3:::', !Ref DeliveryBucketName] ]
BufferingHints:
IntervalInSeconds: 60
SizeInMBs: 50
CompressionFormat: UNCOMPRESSED
Prefix: !Ref DeliveryBucketPrefix
ErrorOutputPrefix: !Join [ '', ['error/', "!{firehose:error-output-type}/"] ]
RoleARN: !GetAtt KinesisDeliveryStreamIAMRole.Arn
DependsOn:
- KinesisDeliveryStreamIAMRole
Glue:
Glue Database:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
GlueDatabaseName:
Description: 'Name of Glue database'
Type: String
Default: 'data-db'
Resources:
GlueDatabase:
Type: AWS::Glue::Database
Properties:
CatalogId: !Ref AWS::AccountId
DatabaseInput:
Name: !Ref GlueDatabaseName
Description: !Sub Database to hold tables for ${AWS::StackName}
Glue Crawler:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
GlueDatabaseName:
Description: 'Name of Glue database'
Type: String
Default: 'data-db'
CrawlerName:
Description: 'Name of Glue crawler'
Type: String
Default: 'raw-data-crawler'
CrawlerPath:
Description: 'Bucket path where crawler will run'
Type: String
Default: 's3://my-bucket/raw/'
TablePrefix:
Description: 'Prefix for table created by crawler'
Type: String
Default: 'tbl_'
Resources:
CrawlerRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Principal:
Service:
- "glue.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
Policies:
-
PolicyName: "root"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action: "*"
Resource: "*"
RawDataCrawlerWithBucket:
Type: AWS::Glue::Crawler
Properties:
Name: !Sub ${AWS::StackName}-${CrawlerName}
Role: !GetAtt CrawlerRole.Arn
Description: AWS Glue crawler to crawl raw data
#Schedule: none, use default run-on-demand
DatabaseName: !Ref GlueDatabaseName
Targets:
S3Targets:
- Path: !Ref CrawlerPath
TablePrefix: !Ref TablePrefix
SchemaChangePolicy:
UpdateBehavior: "UPDATE_IN_DATABASE"
DeleteBehavior: "LOG"
Configuration: "{\"Version\":1.0,\"CrawlerOutput\":{\"Partitions\":{\"AddOrUpdateBehavior\":\"InheritFromTable\"},\"Tables\":{\"AddOrUpdateBehavior\":\"MergeNewColumns\"}}}"