Introduction

Scattered over various blog posts and pieces of documentation are various tricks that I've yet to see collected anywhere, but that make working with CloudFormation variously less an exercise in a Matrix-like comprehension of never-ending braces, and in some cases possible at all when you mix and match features. This article documents a couple that have helped us out.

Readable Inline UserData

The AWS::EC2::Instance type can include a UserData property. A common use for this is to pass a small shell script in order to bootstrap a node into a configuration management system of some kind. According to the reference documentation, that property takes a String. UserData must be Base64 encoded, and that function also takes a string.

Due to the pure JSON syntax of a CloudFormation stack, attempting to make this readable by inserting line-breaks and the like will cause the parser to throw a malformed JSON exception. There is a slightly ugly hack (what in CloudFormation isn't?) that enables you to have something more readable. This might be moot if you're generating your stack manifests programatically from some other more palatable DSL, but it also can't hurt for those times when you want to read and understand what is going on.

The Fn::Join intrinsic function can be co-opted/abused to join an array of "lines" of script into a multi-line shell-script proper. For example:

{
  "Type": "AWS::EC2::Instance",
  "Properties": {
    "InstanceType": "m2.4xlarge",
    ...
    "UserData": { "Fn::Base64": { "Fn::Join": [ "", [
      "#!/bin/bash -e\n",
      "wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef_11.6.2-1.ubuntu.12.04_amd64.deb\n",
      "dpkg -i chef_11.6.2-1.ubuntu.12.04_amd64.deb\n"
    ] ] } }
  }
}

By using a null string as the delimeter and ensuring we specify escape sequences for the line endings, we can present something that is approaching readable whilst still satisfying the syntactic requirements of CloudFormation.

The VPC Security Group ID Requirement

Again taking AWS::EC2::Instance as an example, the documentation states that when defining security groups, the property that you should populate (and what that property should contain) is dependent on whether you are using VPC or not. Plain EC2 security groups should be passed in the SecurityGroups property as a "list of strings". VPC security groups should be passed using the SecurityGroupIds property, again a list of strings - but this time a list of IDs not names.

The most obvious way to accomplish this is to use the Ref intrinsic function against a AWS::EC2::SecurityGroup type to determine the dynamically-generated ID. Except the documentation states that this type, when passed to Ref as an argument, will return the GroupName. A more promising place you might think to look is the documentation for the Fn::GetAtt intrinsic function, given that you can pass it both the logical name of a resource (obtainable via Ref, for example), and the name of an attribute you would like returned.

The kicker here is that the table (at the time of writing) in the documentation that lists the types that Fn::GetAtt can be used against and the attributes that are returned does not include security groups of any kind nor their IDs.

Despite being undocumented, something similar to:

{
  "Type": "AWS::EC2::Instance",
  "Properties": {
    ...
    SecurityGroupIds: [
      { 
        "Fn::GetAtt": [
          { "Ref": "MyVpcSecurityGroup" },
          "GroupId"
        ]
      }
    ],
    "InstanceType": "m2.4xlarge"
}

will work. The describe sub-commands of the aws cli tools are your friend; they return CloudFormation-palatable JSON. Manually configuring a given AWS entity that you are not sure how to encode and describing it in this way can reveal a whole host of attributes not documented in the reference material.