Life, IT & the Rest

Nice talk “Infrastructure in the cloud area” from the O’Reilly Velocity Conference

Conclusion

• Hard things remain hard.
• Using an API for system administration: more scaleable, better to handle.
• Automation, configuration management, self healing is cool.
• New efficiencies opten lead to new hard problems.

It splits up the layers

1. Bootstrapping: get hardware, network, OS in place
2. Configuration: Get application and monitoring running
3. Command and control: daily business, run & deploy code, rerun configuration

1. Bootstrapping

Cloud computing does not save money, it saves time (to market) and gives flexibility. In traditional data centers it takes a long time and a complicated process (4-8 weeks) to get hardware in place. Hardware resources are wasted. Using a cloud it takes 5-10 min to boot another server instance and there are less humans involved (buerocratics and tech staff). The drawback is that the application has to fit to the cloud.

2. Configuration

The old way was to hack 1 week on a server to make it production ready: a graveyard of state. Manual server configuration is always the base for automation, but is error prone and unstructured. A new way is to create a “golden” base image and an deploy packages, services, files to it with a configuration management tool like puppet or chef (examples to deploy “sudo” functionality to a server).

By describung “infratructure with code” you get a repeatable, agile, self documenting server environment. But that takes time to learn and hard things remain hard. Using an API for system administration an dautomation makes it more scaleable and better to handle.

3. Command and control

How nice would it be to trigger the configuration management system to create/repair a server, integrate it into the monitoring system, maybe automatically? The old way “meatcould” (many humans operting the servers) will be replaced with frameworks like nanite (Ruby) or control tier (Java). A two way communication (query version a apache, do update if necessary, restart service) and a messaging bus that works across data centers is the base for these frameworks.

The Book “Deploying Rails Applications” gives an good overview and tutorials on the infrastructure needed for a successfull web project. The central tool is the domain specific language for deployment capistrano which is inspired by rake, written in Ruby by Jamis Buck and integrated with the Ruby on Rails framework.

I really liked reading this clear and progressive book by Ezra Zygmuntowicz, Bruce Tate, and Clinton Begin published by Pragmatic Programmers. The chosen analogy “moving into a new house” helps to understand why it is a good idea to automate server handling and web deployment. The main parts are

1. “bootstrap subversion” chapter on version control
2. hosting environments on shared and dedicated hosts.
3. The chapter on capistrano is the only printed documentation on that tool by now.
   Capistrano offers default deployment “tasks” for Ruby on Rails, but can be adopted e.g. for PHP Projects or admin tasks like key deployment, installation/compilation of software. Commands get pushed to servers in bash syntax via SSH in parallel. The servers can be grouped in “roles” like “web”, “db”, “all” or “most fancy customer”.
4. mongrel servers
5. scaling, performance
6. deployment on windows