No Open Blockers

With the announcement of RHEL’s offering on Amazon Web Services, I wanted to write up some notes from the work I’ve done with EC2 and python. Amazon provides a capable web console, but (not surprisingly) I’d rather do most of my work through a programmable API. The rest of this blog covers the steps necessary to launch an instance, along with some other random notes from my experience.

Boto

The first step is to grab the boto library. Boto is a python interface to Amazon’s web services (not just EC2 but S3 as well). Their site provides downloads, installation instructions, and source, so I won’t go into any more detail besides saying I use it.

Gather Amazon Information

There are a few things needed from your Amazon AWS account in order create and connect to instances, all of which can be retrieved from the AWS web console.

Account Access Keys

The account access keys are effectively your username/password when connecting through boto. From the AWS console, click the Account tab at the top and navigate to Security Credentials. In the middle of the page you’ll find “Access Key ID” and “Secret Access Key”. Make note of these but be sure to keep them safe; these pretty much give full access to your environment.

Key Pairs

The key pairs are used for SSH authentication when connecting to your instances. These are generated through the AWS console itself (the Account tab from the previous section will simply link you back to the AWS console). It’s pretty self-explanatory how to generate a key pair, just be sure to download and then keep the private key safe; there is no way to retrieve a private key from Amazon other than that initial download link.

Image ID

To create an instance, you have to specify which Amazon Machine Image (AMI) to base the instance on. These can be found under the AMIs section of the web console. Determine which image you want based on what it provides and make a note of the ID. It will look something like “ami-12345678″.

Existing Red Hat customers can find more information on Red Hat’s Cloud Access page.

Connect to Amazon

There are two ways of passing your AWS Key and Secret Key to boto, either through environment variables or as arguments to the connect calls. If you choose the environment variable route, they must be named:

AWS_ACCESS_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=bar

Once those are set, create a connection to EC2 in python with the following snippet:

import boto
ec2conn = boto.connect_ec2()

If you choose to skip the environment variables, the keys can be passed directly to the connect call:

import boto
ec2conn = boto.connect_ec2(aws_access_key_id='foo', aws_secret_access_key='bar')

In either case, it is important to realize that these calls default to the US east EC2 region. If you want to make this explicit or, more likely, connect to one of the other two regions, you can pass the optional region argument:

region = # one of 'us-east', 'us-west', 'eu-west'
ec2conn = boto.connect_ec2(region=region)

That’s the main connection to EC2 and the one we’ll use for creating instances. There are others with different purposes, such as connecting to S3, the AWS load balancer features, and so on. They are all named “connect_”, so looking through the help for boto will give you a good idea of what’s available.

Create a new Security Group

A security group is basically Amazon’s firewall to your instances. The default security group is pretty restrictive, so we’ll create a new one that allows us access to SSH and HTTP:

name = 'SSH and HTTP Security Group'
description = 'Test security group'
 
ec2conn.create_security_group(name, description)
group = ec2conn.get_all_security_groups(groupnames=[name])[0]
 
group.authorize(ip_protocol='tcp', from_port='22', to_port='22', cidr_ip='0.0.0.0/0')
group.authorize(ip_protocol='tcp', from_port='80', to_port='80', cidr_ip='0.0.0.0/0')

Note: The create_security_group call returns a handle to the group, but I wanted to demonstrate retrieving an existing group as well.

The above should be pretty self-explanatory. The biggest thing to note is the line where the group is retrieved. Since a list is passed to groupnames we get back a list of matching groups. I can’t tell you how many times I attempted to act on the returned result without indexing a specific group inside of it. This is a common pattern all over boto, so you’d think I’d have learned after the first 30 times.

After this is complete, the web console will show a new security group with the firewall holes we created. This will come in handy when we want to SSH into our instance to, ya know, actually do stuff.

Create the Instance

We’re now ready to actually create an instance.

ami_id = 'ami-12345678'
ami = ec2conn.get_all_images([ami_id])[0]
 
ssh_key_name = # name of the key pair created above
security_groups = # name of the security group created above; must be a list
instance_size = # 'm1.large', 'm1.xlarge', etc. see amazon docs for more info
 
reservation = ami.run(key_name=ssh_key_name, security_groups=security_groups,
                      instance_type=instance_size)
print('New instance [%s]' % reservation.instances[0].public_dns_name)

The call is pretty simple at this point, we just need to pass in the data we’ve been collecting. Remember the security_groups argument must be a list. Also, keep in mind a reservation is returned from the create call, not the instance itself. The boto documentation can provide more information on the distinction.

SSH into the Instance

The above code should have output the public DNS name of the newly created instance. Once it’s finished starting (you can watch the progress in the web console or there are ways to do it in boto, I just haven’t included them here) you can SSH into it by passing the SSH key created earlier (substitute in the relevant information):

ssh -i $SSH_KEY root@$INSTANCE_DNS

Conclusion

As you’d expect, there is a lot more to boto than just creating instances, such as creating/attaching Elastic Block Storage (EBS) volumes, creating/configuring Elastic Load Balancers (ELB), and adding Autoscaling Groups to load balancers. Many of the APIs look similar to the code used in creating an instance, so it’s just a matter of figuring out what you want to do.