Beginning with AWS CloudFormation – Part 2

In this post we are going to build on the previous template and add the ability to take input and produce output.  Sometimes you want to strictly define inputs in your template, but sometimes you want the ability for people to give their own values instead of writing tons of very specific templates for unique workflows.  And we will also start looking at intrinsic functions as well, so plenty of good content here.

Input – Parameters

Inputs for templates in CloudFormation are called parameters.  Like resources they have their own section in the template.

Here is the AWS documentation on parameters for reference:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html

Back in the CloudFormation designer, load up your previously created VPC workflow (or just drag a VPC resource to the canvas).  Notice that with the VPC object selected, the bottom pane on the Components tab will show you specific items related to the VPC resource.  Instead, on the canvas, click anywhere on the blank canvas to deselect the VPC object.  You should see the bottom pane change and have several different tabs show up.  The first one should be Parameters.

parameters

Here we can add in any items we would like the user to put in.  Specifically, I want the user to be able to define a name for the VPC this workflow creates.  Again keeping it simple, I’ll add in a very basic parameter definition so my section will look like this:

"Parameters": {
    "VPCNameParameter" : {
       "Type": "String",
       "Description": "Enter a name for the VPC"
    }
 }

So I just named the parameter, gave it a type and a description.  You may have a lot of questions like what other types are there, what other fields are there.  I would encourage you to take a look at the previously linked AWS doc as this has a lot of important details you’ll be interested in.

But we can’t move on to deployment yet.  This will allow the user to input a value for the VPC name, but we aren’t actually doing anything with it.  Right now if we executed this template, the stack would have a VPC in it but still not have a name.  We solve this by modifying the VPC resource to reference the parameter.

Click on the VPC resource to bring that back up.  We need to answer 2 questions.  First, what actually is property for the name of the VPC, and how do we set it?  For this we reference the AWS CloudFormation VPC documentation:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html

You’ll notice that there is no property that is something like VPCName.  But if you look carefully at the Tags section, you’ll see that the VPC name is really just a Tag with a key value of Name.  So we can do something like this to reference the VPC name:

"TestVPC": {
    "Type": "AWS::EC2::VPC",
    "Properties": {
       "CidrBlock": "172.10.0.0/16",
       "Tags": [
          {
             "Key": "Name", 
             "Value": "VPCNameParameter"
          }
       ]
    }
 }

So question #1 is answered.  This is a correctly formatted template which names the VPC, and you can use the Validate Template button if you don’t believe me.  However, this isn’t going to do what we want.  Feel free to deploy the stack yourself (save first!) but I’ll save you the trouble by showing you here.

createstack2

Here I’m creating this stack.  You see here that I’m naming the stack but it is also prompting me for the parameter name in the template.  Success!  Kind of…

After proceeding through the stack deployment, there were no errors, and it did create a VPC.  However, here is my VPC output:

vpcbadname3

Well it named it, but not what I wanted.  I wanted it to be named MyTestVPC.  This is because our tag referenced the specific string “VPCNameParameter” but not the actual parameter VPCNameParameter.   In other words, the template did exactly what we told it to, but not what we wanted it to.  Certainly a common problem when coding!

To reference the parameter value, we need to use what is called an intrinsic function.  We will cover more of these in a later post, but this is a way to have dynamic functionality in your templates instead of having to hard code everything.  In our case we want to reference the value of the parameter, so we need to use the intrinsic function called Ref. (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html)  Ref will return the value of a parameter when used on one, and will return a usable ID (say VPC ID) when used on a resource.  Intrinsic functions are key to writing good, reusable CloudFormation templates.

So a quick modification of our VPC resource gives us:

"TestVPC": {
   "Type": "AWS::EC2::VPC",
   "Properties": {
      "CidrBlock": "172.10.0.0/16",
      "Tags": [
         {
            "Key": "Name", 
            "Value": { "Ref": "VPCNameParameter" }
         }
      ]
   }
 }

That should do it.  Now we’ll properly reference the VPCNameParameter input value.

Save your template and give the deployment a shot.  It should allow you to input a name, and then you should see that name referenced on the VPC.

Now I’m going to be bad for a minute.  Tags in AWS have a maximum value of 255 characters.  I’m going to deploy this template and use a reeeeeeeeeally absurdly long name value of 256 characters.  What will happen?

badnamestack4

Well, the template will still validate no problem.

The deployment will also start, no problem either.  When you input the parameter, CloudFormation has no idea what you are using it for, so it being long or short or having weird characters doesn’t really bother it.

stackerror5

So notice here in our event log that the deployment failed and was rolled back.  Why?  It tells you right there, the value we tried to use for our tag exceeded the 255 character limit.

This is a kind of silly example as it is unlikely that someone would use a 256 character name for a VPC but it highlights a problem which is, whenever you allow users to input data, you have to be able to validate that data.  Sure we can modify the Description field and put “must be less than 255 characters, blah blah blah,” but this still allows a user to purposefully or accidentally put in the wrong data, and then they have to wait for the deploy to fail and do it all over again.  A better idea would be to actually do validation on the data.  And guess what?  You can do just that!

Back on the Parameters documentation you can find several options for limiting user input as well as an AllowedPattern property which even lets you use regular expressions for validation.  And there is a ConstraintDescription property which lets you give the user a message when they oops.  These are key when doing parameters in CloudFormation.

So in my template I will modify our parameter entry to:

"VPCNameParameter": {
   "Type": "String",
   "Description": "Enter a name for the VPC",
   "MaxLength": 255,
   "ConstraintDescription": "VPC Name must be less than 255 characters"
}

Now when I try to execute this template in CloudFormation with a 256 character tag, I get this error message:

vpcerror6

Which tells me exactly what the problem is, and I can go back and modify my inputs to satisfy the parameters.

Additionally, obviously, you don’t have to conform your input validation to AWS maximums.  Maybe you only want VPC names to be 20 characters max.  Maybe you want VPC names to start with “vpc-“.  You can do all this and more with the input validation in CloudFormation.

This is a pretty simple example but definitely review the parameter documentation and see how they can help improve your templates.

Output

Outputs in CloudFormation are actually called – wait for it – outputs! And they go in their own section as well.  Outputs are useful if you want to report some information back to the caller after they execute a template.

Here is the doc for outputs:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html

In our template, we don’t actually let the user define the VPC CIDR block so let’s report what the CIDR block is back to them after the resource is created.

In the designer again, click on the canvas to deselect the VPC object and you should see a few tabs again (Parameters being one).  The last one should be Outputs.  Click here and we can add in a section to do the output we want.

"Outputs": {
   "VPCCidrBlockOutput" : {
      "Description": "The Cidr Block of the VPC", 
      "Value" : { "Fn::GetAtt" : [ "TestVPC", "CidrBlock" ]}
   }
 }

This should look pretty familiar by now except Fn::GetAtt.  There is an identifier “VPCCidrBlockOutput,” and a description of what this is.  Then there is a value.

We need to use another intrinsic function as we want to dynamically get the CIDR block attribute from the resource after it has been created.  Sure we could just type in the CIDR block value, but what if later we altered our template to change the CIDR block, or maybe let the user enter it.  For the most part you will want your outputs to be dynamic.

This function is Fn::GetAttr (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-getatt.html) and it gets an attribute of a resource.  In our case the resource is the logical ID of our VPC, or “TestVPC”, and the attribute we want is “CidrBlock”.  How do we know what the attributes are named?  Consult the CloudFormation VPC documentation and there is a section called Return Values that will tell you what Ref will return as well as what attributes you can return via Fn::GetAttr.

Now that we’ve got this output in place, all we need is to execute the template again.  Once my stack is created, I can see these values in the Outputs section of the console.

vpcoutput7

Again this value is the value from the resource itself, so if the template were to change in the future, the correct value will still be reported back to the user executing the stack.

I hope this has been another useful intro to CloudFormation and some of the ways you can leverage it to be more dynamic.  More to come!

Beginning with AWS CloudFormation – Part 1

One of my few new goals for this year is to get back to blogging regularly about stuff I’m learning or interested in.  Keep a look out here for (hopefully!) more content this year than previous years which might have had just a handful of posts.

AWS CloudFormation is a utility that allows you to define AWS “infrastructure” as code in text files called Templates.  You can use it to deploy almost anything via JSON or YAML scripts.  The deployed resources are collectively called stacks.  There are other IaC options here as well, like Terraform, but I think it is handy to know the native toolset as well.  Plus if you are going for AWS certifications you’ll need to be familiar with it.

Continue reading

VMworld 2017 – PKS: Pivotal Container Services

The big announcement during the 2nd general session at VMworld was around Pivotal Container Services, or PKS, which is an on-prem, managed kubernetes offering.  PKS is targeted at customers who want a private kubernetes environment, operated at scale, and enterprise grade (meaning manageable, upgradeable, etc.).

I wanted to provide some context around several things that I think a lot of people aren’t aware of.  I’m going to try to define all the acronyms as well, as this gets confusing fast.  Give it a couple of reads if you have to! Continue reading

VMworld 2017 Day 2 – VMware Cloud on AWS

The hits just keep on coming here at VMworld 2017.  Lots of big announcements and great sessions today.  VMware Cloud on AWS is a big focus this year and you can expect to see continued integration points between VMware and Amazon (as well as other cloud providers). Continue reading

VMworld 2017 Day 1 – VMware Cloud Management Platform

My first session at VMworld 2017 was “What’s New and What’s Next for VMware Cloud Management Platform.”  There are a ton of great sessions and overlap at this conference, but I thought this would be a good one to attend (and blog on) as they touched on a lot of things VMware is doing outside of itself.

A big issue in the tech field today is that the world is moving “open.”  Open source, open APIs, commodity hardware, etc.  Customers are very wary of vendor lock-in that prevents them from either using new solutions or moving to other solutions later.

VMware has clearly recognized this, as a primary focus of late is cloud and product interoperability.  This is huge for a massive company like VMware who has TONS of proprietary stuff. Continue reading

Exploring Docker Networking – Ports and Forwarding

whale20logo332_5

If you’ve followed along in my other network posts about bridge and the rest of the default networks, you may have noticed that I keep spinning up CentOS containers.  This is well and good for demos and instructional purposes but you might have also noticed that they don’t really do anything.  I mean they exist, but that isn’t why you run things in your environment.  You don’t have a bunch of server OSes spun up doing nothing, right?  RIGHT?

Well, we hope not anyway.  Ideally your server actually does something like hosts an app, database, or webserver.  In this post we are going to explore some aspects of bridge networking and get a little more into the underpinnings of Docker.  And in order do that, hey we need a container that actually does something! Continue reading

Exploring Docker Networking – Host, None, and MACVLAN

whale20logo332_5

As with our previous post on bridge, the intent here is to explore the remaining standard (mostly non-swarm) network configurations.  Those configurations are host, none, and the newer MACVLAN.

Unlike bridge these are non-default configurations which means you must manually specify the network somewhere.  In our case we are just going to explicitly list the network when we run the container.  We will also explore the configuration elements that we looked at earlier.   Continue reading