Whereas launching a server from an AMI is analogous to burning a CD from an ISO, a ServerTemplate is analogous to burning a CD from a playlist and essentially just the definition of the composition and configuration of your cloud servers. It's a lot easier to manipulate your components with the granularity of a script oriented playlist then it is by constantly bundling and “sprawling” images that represent “mostly tweaks”. Instead, you can save revisions of your Template and the underlying scripts themselves. Overall this model allows for quicker tweaking and repurposing of cloud servers and simplifies lifecycle management of cloud servers.

Also, it's portable! You don't have to reconfigure various instance sizes if you want to scale vertically, just replace the base AMI that contains the OS. You can also migrate templates from one cloud to another without having to reconfigure the bulk of the server deployment process. Of course we still have to play by each cloud's own rules, network level configurations, image types, etc, but you avoid the work of reconfiguring your servers for a new infrastructure provider. This feature is currently live and available in our free Developer Edition for EC2 US to EC2 EU migrations there and back, which are essentially entirely separate clouds.

I guess a key point is to have deployment processes that have to document purpose and expected life cycle of an instance when it is deployed — something we never really did in hardware.

So, if we do with virtual what we did with physical hardware we face sprawl. But would you agree that with new processes and better management tools that does not have to be the case?

That was 90 total AMIs created over the life of the CloudScale project, most of which are still there mothballed and could probably be deleted. I’ll save that for a weekend project in the near future.

The vast majority of those images were revisions of the baseline gold master that CloudScale used. There were revisions for 32-bit and 64-bit of CentOS, RHEL4, RHEL5, and Ubuntu. So, basically, this was 2 baselines x 4 OS versions x roughly 10 different attempts and revisions. Mostly tweaks to make sure that Puppet and the CloudScale node agent software did the right thing at boot.

Obviously, I couldn’t agree with you more about the role that Puppet plays in managing this problem. Using virtual machine images as a configuration management system is inherently flawed and completely unscalable.

Did you have 90 active images or that was over the life of the Cloudscale and some were retired? If you had 90 active, why did you need so many? I know you used Puppet, to build some systems, which makes me even more curious.

Image sprawl is a royal pain in the you-know-what and the fact that you get more and more of them is only part of the problem. Unless, you have a lot of discipline and take really good notes, you have no idea what is on ami XXXXXXX vs. YYYYYYY after they have been around for a while and even then it is nearly impossible to easily compare them. (that’s a not so new problem that also exists on physical servers as well)

As long as we are going to use virtualization, there will be some images, but there are tools and strategies to minimize the sprawl while providing other benefits as well. For example, Puppet can help minimize the number of necessary images and provide meaning to the services you are running. I blogged about it a bit here:
http://stochasticresonance.wordpress.com/2008/09/01/semantics-matter-or-i-finally-get-it/

Regards,
Andrew