Chef in 21 days. Part I. Introduction

Greetings, readers! The weather outside is miserable, tonsillitis is clawing at my enflamed throat, so what better time than now to write an article? This is my first attempt at writing here, so please don’t judge me too harshly. The title was inspired by a huge variety of similarly titled books. Here, I will try to describe the path of the automation warrior for all young padawans, though I sometimes still feel like one. I will help to quickly familiarize you with the cross-platform automation tool called CHEF. With considerable effort, you will master it well enough for your first serious experiments. This article is a guideline of sorts, for people not well acquainted with automation.

Step 1. Meeting CHEF

What is Chef?

Chef is a toolkit for automatically managing the configuration of your computers (known in Chef as “nodes”). Nodes communicate with the Chef server which contains descriptions of the actions the nodes must perform. These actions are described in a cookbook, the main parts of which are recipes (sets of actions for nodes), attributes (information about the node stored in the JSON format) and templates (embedded Ruby files generated on the node).

Think of a Chef cookbook, recipe, etc. as all the steps of a typical administrator’s installation runbook, but instead of an admin manually checking off steps as they’re performed, Chef handles this automatically.  As an added benefit of this automation, any defined set up steps are enforced.  This means that if someone later alters a node manually, Chef will enforce the settings as you initially set, reverting back any unauthorized changes.

Step 2. Preparing the environment

What are we going to need?

The minimum set, sufficient for a test lab or development environment, consists of two virtual machines with network support. One is for installing the server (open source Chef), the other will be the client, or node. 

For Chef Server, you will want either Ubuntu or RedHat Enterprise Linux (if using CentOS, you can use the same RHEL packages). For the client, or node, choose whichever operating system you prefer, but keep in mind that the less common your OS choice is, the more customization of recipes you’ll need to do for functionality that may not be handled “out of the box”.  It is recommended to test with Ubuntu or RHEL/CentOS. Currently, my humble experience shows, that the most reliable options are Debian-based distribution packages.

If you don’t wish to install Chef Server, your second option is using preview.opscode.com – a ready-to-use Chef server where you can register and create an organization with a unique name, to which you will then add your nodes and cookbooks. After creating an account and an organization, you will be able to download the so-called “start kit” – a ready-to-use configuration for the chef-administrator containing certificates and configuration files. This kit will enable you to manage the server from your PC.

Step 3. Adding nodes

With the environment now ready, we can now start adding our first nodes. There are two basic ways to do that: using the server’s web interface or the console of the chef-administrator. Using the web interface is not our style, so we’ll be using the console. The toolkit that will help us do it is called knife.

Knife allows creating cookbooks, adding nodes, managing execution lists for each node and has many other capabilities. The process of adding a node, is called bootstrap in this case, and, in fact, includes the chef-client installation and initial launch on the node. During this process, you can also transmit the initial launch settings for the new nodes as JSON attributes. The bootstrap needs you to have access to an administrator account and to provide its credentials. After validation, the new node will appear in the list of available nodes. This list can be accessed via the web interface or using the knife node console command.

At this point, the primary installation stage is complete and it’s high time for a well-deserved coffee break before we get to the most exciting part – the automation.

Step 4. What are we automating?

…Is a completely valid question. All this adding nodes does not look like automation at all, more like a bunch of manual labor. You’re absolutely right, but this is just one step, required so we know what to automate, but now we need to know how to automate. I have not told you everything yet, o young padawan. In fact, even this process of setting up nodes can also be automated. Let me digress for a moment.

When we have a group of PCs and we are performing the initial installation of an operating system on them (via cloning or unattended installs or even provisioning cloud instances), we can also install our Chef clients there. This would finish the first stage of software installation and mean that our chef-servers and clients are running and waiting for instructions (configuring them to work continuously as daemons or services is a topic for a separate article).  We don’t even need to fully configure the chef-clients.  As long as we have enough pre-configured to initiate at least one chef-client run, we can have Chef Server automatically reconfigure the client settings.

Our wonderful knife toolkit allows to perform client parameter (attribute) search. The results can be used to perform some primary actions with the client, for example, to install one package or another. The attributes themselves, as previously mentioned, are taken from a JSON file during the client installation. An attribute, in fact, is a tag that consists of a name and a value (like everything in automation, it’s as simple as that). As a result, we get nodes that are identified by the given tags, for example, node1(“tag1” = “value1”). When a server runs a node search, we can define a target for the search – an attribute and a value it’s supposed to correspond to. This is done using knife exec and nodes.find (check out the links at the end of the article to learn more). So, to get to the point, the search provides a list of nodes corresponding to the defined criterion, and this list can be used to do stuff – save it, feed it to the server, so that the server can order the nodes to install something according to the recipes, etc.

After these simple tweaks, we should have the architecture containing a server and a necessary number of registered nodes, communicating with it on a scheduled basis (please note that this must be implemented on the node using available means, such as cron, as an in init daemon, or Windows Services). Now, let’s return to our main goal – automating the node configuration.

Chef is a powerful tool that enables you to manipulate your nodes in many various ways. I’m pretty sure that I don’t even know its full capabilities. You can learn about the functionality on the official Chef website, in the Docs section, but even there the information is not entirely complete.

Need to install a dozen packages and dependencies? Not a problem! Restart the services? Consider it done! Generate a file using a template? Piece of cake! Run console commands? As easy as ABC!

Everything is specified in the cookbooks and the recipes. Creating and using them is the actual primary skill of the automation jedi. If you want to become a true master, I highly suggest you to get acquainted with Ruby and to delve into the concept of Chef itself. In a few weeks you will fall in love with it, and it will love you back and make you happy, only occasionally bothering you with bugs and workarounds.

By my modest estimation, Part I should take about two days. Of course, it depends on your persistence – so be brave, go forth and conquer!

As promised, see some useful links down below.
Official documents and manuals — docs.opscode.com/
Chef server installation (Enterprise) — docs.opscode.com/install_server_oec.html
Chef server installation (Open source) — docs.opscode.com/install_server.html
Chef client installation — docs.opscode.com/install_workstation.html
Knife toolkit — docs.opscode.com/knife.html
Useful image with knife options — docs.opscode.com/_images/qr_knife_web.png
Basic info on cookbooks — docs.opscode.com/essentials_cookbooks.html
Chef details — docs.opscode.com/chef_overview.html.

P.S. My sore throat is getting the best of me, so I bid you farewell until we meet again in Part II, which will be dedicated to the basics of “cooking”. I am open to any criticism, questions and suggestions. Thank you!