Migrating to Chef Server 11

Initially, we used Chef Server 10 for our EPAM Private Cloud virtual machine autoconfiguration.

The number of supported roles exceeded 60, including both simple and quite complex cluster solutions.
When the number of server clients reached 750, we noticed a significant decrease in performance.
Increasing the capacity of the virtual machine with Chef Server 10 installed was not rational, as it was already of a pretty high level (2x Intel® Xeon® CPU L5640 @ 2.27GHz and 8Gb RAM).
Trying to tune chef-solo and chef-expander didn't provide the expected performance boost either.

That was when we decided to migrate from Chef 10 to Chef 11.
The opscode.com website claims it is faster and easier to scale, configure, and manage.

Let’s start with a short tutorial on how to install Chef Server 11 on Ubuntu12.04.

#Download and install the package

wget -O chef-server-11.deb https://opscode-omnitruck-release.s3.amazonaws.com/ubuntu/12.04/x86_64/chef-server_11.0.6-1.ubuntu.12.04_amd64.deb sudo dpkg -i chef-server-11.deb

#Configure and launch

sudo chef-server-ctl reconfigure

#Check that the installation was successful

sudo chef-server-ctl test

As soon as the server is installed and running, we should create a user with the administrator rights. Create the .chef directory in the user home folder and copy the necessary keys.

mkdir ~/.chef

cp /etc/chef-server/admin.pem ~/.chef

cp /etc/chef-server/chef-validator.pem ~/.chef

Run the setup of the configuration file for knife.

knife configure -i

#As a result, we should get something like this:

cat ~/.chef/khife.rb log_level :info log_location STDOUT node_name 'chefuser' client_key '/home/chefuser/.chef/chefuser.pem' validation_client_name 'chef-validator' validation_key '/home/chefuser/.chef/chef-validator.pem' chef_server_url 'https://192.168.0.1' syntax_check_cache_path '/home/chefuser/.chef/syntax_check_cache'

Data transfer from Chef 10

First, we must install knife-essentials to extend the functionality of the knife tool:

/opt/chef-server/embedded/bin/gem install knife-essentials

Next, let’s create a directory to be used as a transit point. Here, we will create a folder for the knife configuration files.

mkdir -p ~/transfer/.chef

After that, we create separate knife configuration files for Chef 10 and Chef 11:

/home/chefuser/transfer/.chef/knife-chef10.rb

transfer_repo = File.expand_path('..', File.dirname(__FILE__))
chef_server_url «chef-10.example.com:4000»
node_name 'chef-webui'
client_key "#{transfer_repo}/.chef/chef-webui.pem"
repo_mode 'everything'
versioned_cookbooks true
chef_repo_path transfer_repo
cookbook_path nil

/home/chefuser/transfer/.chef/knife-chef11.rb

transfer_repo = File.expand_path('..', File.dirname(__FILE__))
chef_server_url «chef-11.example.com»
node_name 'admin'
client_key "#{transfer_repo}/.chef/admin.pem"
repo_mode 'everything'
versioned_cookbooks true
chef_repo_path transfer_repo
cookbook_path nil

All we have left to do is to copy webui.pem from Chef Server 10 to /home/chefuser/transfer/.chef/chef-webui.pem and/home/chefuser/.chef/chefuser.pem in /home/chefuser/transfer/.chef/admin.pem.

When this is done, we can try downloading the information about nodes, clients, roles, etc.

/opt/chef-server/embedded/bin/knife download -c .chef/knife-chef10.rb /

#As a result, we create folders /nodes /clients /roles /cookbooks and so on with the corresponding content.

ls transfer/ clients cookbooks data_bags environments nodes roles users

In the same manner we can download information about a specific node or client, for example:

/opt/chef-server/embedded/bin/knife download -c .chef/knife-chef10.rb /nodes/server1.json

Before loading the data to Chef Server 11, I recommend checking that you don't have clients or nodes with matching names on both servers. Otherwise, the information about them may be lost during the transfer.
To load the data, use the following command:

/opt/chef-server/embedded/bin/knife upload -c .chef/knife-chef11.rb /

You can also load a single node, client, etc.

Switching clients from Chef 10 to Chef 11

This is probably the most exciting stage of the migration.
There should be no problems if you used chef_server_url as the DNS-name . Just reassign it to the new server.
However, if, just as in our case, you used the name of the virtual machine that is unique and cannot be assigned to another machine, you would have to find the solution.
I solved this issue by creating a simple cookbook to change the value of chef_server_url to the necessary one and to restart the chef-client.
If anyone is interested in this method, the cookbook will be available on GitHub.

Conclusion: Migrating to Chef 11 resulted in a performance increase by several times. The old server was constantly “on the verge of a swap” and could take up to two minutes to process a simple role list query. The current server with the same number of clients has 3Gb of free RAM and queries are processed within seconds.

If you have any questions, feel free to leave them in the comments or message me, I will be happy to reply.