No Open Blockers

With the announcement of RHEL’s offering on Amazon Web Services, I wanted to write up some notes from the work I’ve done with EC2 and python. Amazon provides a capable web console, but (not surprisingly) I’d rather do most of my work through a programmable API. The rest of this blog covers the steps necessary to launch an instance, along with some other random notes from my experience.

Boto

The first step is to grab the boto library. Boto is a python interface to Amazon’s web services (not just EC2 but S3 as well). Their site provides downloads, installation instructions, and source, so I won’t go into any more detail besides saying I use it.

Gather Amazon Information

There are a few things needed from your Amazon AWS account in order create and connect to instances, all of which can be retrieved from the AWS web console.

Account Access Keys

The account access keys are effectively your username/password when connecting through boto. From the AWS console, click the Account tab at the top and navigate to Security Credentials. In the middle of the page you’ll find “Access Key ID” and “Secret Access Key”. Make note of these but be sure to keep them safe; these pretty much give full access to your environment.

Key Pairs

The key pairs are used for SSH authentication when connecting to your instances. These are generated through the AWS console itself (the Account tab from the previous section will simply link you back to the AWS console). It’s pretty self-explanatory how to generate a key pair, just be sure to download and then keep the private key safe; there is no way to retrieve a private key from Amazon other than that initial download link.

Image ID

To create an instance, you have to specify which Amazon Machine Image (AMI) to base the instance on. These can be found under the AMIs section of the web console. Determine which image you want based on what it provides and make a note of the ID. It will look something like “ami-12345678″.

Existing Red Hat customers can find more information on Red Hat’s Cloud Access page.

Connect to Amazon

There are two ways of passing your AWS Key and Secret Key to boto, either through environment variables or as arguments to the connect calls. If you choose the environment variable route, they must be named:

AWS_ACCESS_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=bar

Once those are set, create a connection to EC2 in python with the following snippet:

import boto
ec2conn = boto.connect_ec2()

If you choose to skip the environment variables, the keys can be passed directly to the connect call:

import boto
ec2conn = boto.connect_ec2(aws_access_key_id='foo', aws_secret_access_key='bar')

In either case, it is important to realize that these calls default to the US east EC2 region. If you want to make this explicit or, more likely, connect to one of the other two regions, you can pass the optional region argument:

region = # one of 'us-east', 'us-west', 'eu-west'
ec2conn = boto.connect_ec2(region=region)

That’s the main connection to EC2 and the one we’ll use for creating instances. There are others with different purposes, such as connecting to S3, the AWS load balancer features, and so on. They are all named “connect_”, so looking through the help for boto will give you a good idea of what’s available.

Create a new Security Group

A security group is basically Amazon’s firewall to your instances. The default security group is pretty restrictive, so we’ll create a new one that allows us access to SSH and HTTP:

name = 'SSH and HTTP Security Group'
description = 'Test security group'
 
ec2conn.create_security_group(name, description)
group = ec2conn.get_all_security_groups(groupnames=[name])[0]
 
group.authorize(ip_protocol='tcp', from_port='22', to_port='22', cidr_ip='0.0.0.0/0')
group.authorize(ip_protocol='tcp', from_port='80', to_port='80', cidr_ip='0.0.0.0/0')

Note: The create_security_group call returns a handle to the group, but I wanted to demonstrate retrieving an existing group as well.

The above should be pretty self-explanatory. The biggest thing to note is the line where the group is retrieved. Since a list is passed to groupnames we get back a list of matching groups. I can’t tell you how many times I attempted to act on the returned result without indexing a specific group inside of it. This is a common pattern all over boto, so you’d think I’d have learned after the first 30 times.

After this is complete, the web console will show a new security group with the firewall holes we created. This will come in handy when we want to SSH into our instance to, ya know, actually do stuff.

Create the Instance

We’re now ready to actually create an instance.

ami_id = 'ami-12345678'
ami = ec2conn.get_all_images([ami_id])[0]
 
ssh_key_name = # name of the key pair created above
security_groups = # name of the security group created above; must be a list
instance_size = # 'm1.large', 'm1.xlarge', etc. see amazon docs for more info
 
reservation = ami.run(key_name=ssh_key_name, security_groups=security_groups,
                      instance_type=instance_size)
print('New instance [%s]' % reservation.instances[0].public_dns_name)

The call is pretty simple at this point, we just need to pass in the data we’ve been collecting. Remember the security_groups argument must be a list. Also, keep in mind a reservation is returned from the create call, not the instance itself. The boto documentation can provide more information on the distinction.

SSH into the Instance

The above code should have output the public DNS name of the newly created instance. Once it’s finished starting (you can watch the progress in the web console or there are ways to do it in boto, I just haven’t included them here) you can SSH into it by passing the SSH key created earlier (substitute in the relevant information):

ssh -i $SSH_KEY root@$INSTANCE_DNS

Conclusion

As you’d expect, there is a lot more to boto than just creating instances, such as creating/attaching Elastic Block Storage (EBS) volumes, creating/configuring Elastic Load Balancers (ELB), and adding Autoscaling Groups to load balancers. Many of the APIs look similar to the code used in creating an instance, so it’s just a matter of figuring out what you want to do.

Most of the details here taken from Jesus’ post.

Features & Enhancements

• Support for packages with checksums other than MD5. Spacewalk server and proxy can serve and accept rpms with SHA256 digests, show the checksums in webUI and accept them in API calls.
• Moved from mod_python to mod_wsgi on Fedora 11 and Fedora 12, which promises better performance.
• Improved performance of SSM and API.

Known Issues

• PostgreSQL support still does not work. We will need help with moving this forward.
• Provider GPG key is not sometimes recognized and packages shows up as unknown Provider.
• Documentation search does not work, other search are unaffected.
• Cobbler 2.0 is known to work, but full tests haven’t been done yet.

Congrats to (my old stomping ground) the Spacewalk team on the release.

Credit to Jaap A. Haitsma’s WPTouch blog entry for first mentioning this plugin (at least, the first place I saw it, since it seems to be making rounds on other blogs as well). With the simple activation of this plugin, your blog gets automatic formatting on mobile devices. Once it was installed I took a quick peek on my iPhone and it looks awesome. Very cool stuff for such a simple installation.

I recently came into some new hardware, an Alienware M9700 laptop. The thing is a beast, both in processing power and sheer size. Disappointingly, Dell seems to have tainted Alienware, as what would normally ship as a clean, optimized installation is now cluttered with shit.

I have unavoidable needs for Windows on this box, so I decided to dual boot it. With as new as the hardware is, I figured XP would just be asking for trouble, so I decided to go with Windows 7. I figured with both the newness and hype of Windows 7, it actually contained drivers that were written this millennium.

When I first booted into Windows, I noticed the sound card wasn’t detected. It wasn’t hard to spot, as the system tray icon was huge due to the 800×600 resolution it was running in since the video card wasn’t detected either. Amazingly, the network cards were picked up, which didn’t really matter since Alienware doesn’t have Windows 7 64-bit drivers for… well, anything.

Seriously Dell, what did you do to that company? I used to have nothing but great things to say about them.

Dejected, I rebooted into my Fedora installation disk. A few minutes later, I was listening to music and trying to not go blind at the 1920×1200 resolution. Both of which worked fine out of box. And for the record, my video card is an NVidia GeForce Go.

This makes my head hurt. This laptop is for my wife, who is non-technical (that’s putting it mildly). If I were to sit here down to do the installation with both the Windows and Fedora installers, it’s pretty clear she’d end up with a much more usable system running Fedora. Apparently, when the people on the commercials were saying what they wanted in Windows 7, no one mentioned drivers.

I’ve been getting into Django recently. I’ll go into it more in another entry, but I ran into a small issue where my database seemed to get out of sync with my model. Running syncdb didn’t throw any errors, but when I tried to access the model from the server I’d get an error about “no such column”, even though I could see it created in the generated DDL.

It took me a bit of digging (in other words, it wasn’t in the tutorial), but there’s a manage command to reset the database for a particular app. Running that and re-syncing my database got me moving again.

python manage.py reset [appname]
python manage.py syncdb

RHQ… now with 100% more awesome.

The release can be downloaded and full release notes found at the RHQ wiki.

Changes in this release include:

• Repo Synchronization – The ability to schedule package synchronization for specific repos as well as cancel a running sync.
• HTTP Content Access – The ability to browse content repositories and retrieve packages via HTTP.
• Server-Side Plugin Dynamic Configuration Values – Added the ability to indicate the enumerated values for a configuration property of a server-side plugin be pulled from the database. For more information see Server Plugin Descriptor Configuration.
• Raw Configuration Support – Significant enhancements made to raw configuration support.
• Notification Templates – It is now possible to create bundles of pre-configured alert senders that can be applied to individual alert definitions.
• Alert UI Revamp
• Server Side Plugins Operations – Server-side plugins now support operations.

I’ve used two monitors for years now on a kinda cheap card with one DVI and one VGA port. Naturally, I use the DVI one as the primary monitor, keeping the VGA as a supplement for extra space. However, my card detects the VGA as the “primary” monitor and puts my Gnome panels on that monitor. I learned to live with it, almost enjoying the fact that my primary workspace had no decorations at all.

However, I got curious and dug into it to find the setting to change this:

gconftool-2 -s /apps/panel/toplevels/top_panel/monitor --type int 1

Alternatively, use gconf-editor to navigate to that location and change monitor from 0 to 1 from there. I didn’t even need to re-login, it just hopped across to the other monitor.

UPDATE: Ya, so apparently you can just alt + left click to move the panel between monitors. The interwebs is awesome, thanks to mbooth in the comments for that one.