Safari Blog
Safari Blog - technology, business, and design
What is Chef Provisioning?
Chef Provisioning is a drop-in library for Chef that gives developers and infrastructure teams an added dimension of automated system configuration: the ability to bootstrap and install a series of OS and configuration deployments onto “bare metal”. There are a variety of drivers that can be used as bare metal abstractions, including Docker, LXC, Fog (EC2 / DigitalOcean / OpenStack), AWS, Azure, Vagrant, VSphere, DigitalOcean, Hanlon, OpenCrowbar, and SSH. Chef Provisioning, combined with your choice of these drivers, provides a number of new abilities:
Define and deploy a host using the machine resource
Run parallel tasks to deploy or configure multiple machines using machine_batch
Run a command on a machine using machine_execute
Copy a file to a machine using machine_file
Define a machine template using machine_image
This elevates Chef’s potential role beyond its historic use for managing resources on individual servers. Chef is a proven platform to manage components, such as installed packages, sensitive data (for example, SSL keys, encrypted passwords, API keys), configuration templates, user accounts, and other aspects of server management. With Chef Provisioning, it can now define and manage the servers themselves.
How can I use it?
Chef Provisioning can be useful for local, decentralized development of:
Application software
Chef cookbooks
Docker repositories
System configuration (user authentication, alerts, logs, stats, NTP, etc.)
Load balancer health checks and settings
Chef Provisioning promises that you can take the same provisioning script you use for your local Docker deployment (see deploy-environment.rb below) and run it in one of the other platforms listed above. You can also mix and match platforms to run machines in different mediums by using multiple drivers. The deploy-environment.rb on your machine can be shared with others, to provide them a copy of your full software stack configuration — leaving out the site-specific configuration (such as passwords), if you desire.
Show me an example
Using Docker with Chef Provisioning is the quickest way to get up and running. Here’s an example using the Docker driver that creates a network of nodes on your local workstation. There are 3 types of nodes, spanning 4 instances: 1 load balancer, 2 application servers, and 1 or 2 database servers, depending on the environment. While the example here won’t do much beyond install packages and set Chef node data, it should be enough to get you started.
Goals:
Define all machines in the environment, including their Chef node data and recipe run_list.
Be able to add, query, update, and delete these machines via the environment configuration file.
Enable sharing of the environment configuration file, so others can run their own copy of the environment (with or without site-specific information such as passwords).
Version-control the complete environment configuration.
Steps:
1. Copy the below example to a file on your local machine: ~/deploy-environment.rb:
2. Download and install Docker
boot2docker for your Mac or Windows machine
or Docker on Linux
3. Download and install ChefDK
4. Install the chef-provisioning-docker library for Chef
5. Set up your local machine to serve cookbooks via Chef Zero
6. Choose Docker as your driver, and go!
When all is done, you should be able to run docker images and see output similar to the following:
It’s like Chef is completing Docker, and Docker is completing Chef.
Note: One nice side-effect of Docker is data de-duplication. So while the “VIRTUAL SIZE” of the images totals 1912 MB, the actual size on disk is only 1135 MB.
Upon launch, these images will talk to a defined Chef server, which can be either your workstation’s local Development Chef Zero server or a production Chef server. For this example, we are using our local machine as the Chef server, so we need to launch it before the docker instances can check in.
Now you can launch the docker images by running one or all of the following:
This is a powerful pattern for application development. Sharing a single environment config file between your colleagues enables everyone to run everything on your local machine. Testing cookbook interoperability can be done quickly in local environments to spare resources on central hardware. Application and Chef Cookbook administrators may find this is a quick way to upload code and have it deployed in one or more fully networked environments.
Troubleshooting
1. The first time I ran through the above script, I got an error:
The problem was that the boot2docker VM running on my machine could not resolve my host machine’s hostname (
). I added an entry in the VM’s /etc/hosts file, and the issue went away.
2. If you receive an error like
, this Pull-Request may fix the problem.
3. If you make changes and they aren’t picked up, you can delete the node data from your Chef Zero cache by running
or delete the Chef Zero server data by running
(a cleaner way to do this would be to
).
Cleaning up
To delete all of your local containers, run
.
If it’s running, shut down Chef Zero via Control-C or
.
Next Steps
If you are interested in staying up to date with the latest developments in Docker, I recommend browsing through the upcoming Docker Cookbook. If you aren’t very comfortable with Chef, you can get a great introduction by reading Learning Chef. These are just two excellent resources out of thousands that are available at Safari Books Online. Please let me know in the comments if you have trouble with the example or have other feedback.
Building Docker Images with Chef Provisioning
Shane Ramey