Have you ever heard the following? “Welcome to the team! Here's a list of 15 applications to install, the instructions are in the team room, somewhere. See you in a week!” Or: “What do you mean it broke production, it runs fine on my machine?” Or: “Why is this working on her machine and his machine, but not my machine?”
Development environments are becoming more complex, with more moving parts and tricky dependencies. Virtualization has been a huge boon for the IT industry in saving costs, increasing flexibility and maintaining control over complex environments. Rather than focusing on virtualization on the delivery side, let's look at how you can provide that flexibility and control to developers to manage multiple development environments easily using Vagrant.
Vagrant is an open-source (MIT) tool for building and managing virtualized development environments developed by Mitchell Hashimoto and John Bender. Vagrant manages virtual machines hosted in Oracle VirtualBox, a full x86 virtualizer that is also open source (GPLv2).
A virtual machine is a software implementation of a computer, running a complete operating system stack on a virtualizer. It is a full implementation of a computer with a virtual disk, memory and CPU. The machine running the virtualizer is the Host system. The virtual machine running on the virtualizer is the Guest system. As far as the Guest operating system is concerned, it is running on real hardware. From the perspective of the Host, all of the Guest's resources are used by the virtualizer program. A Box, or base image, is the prepackaged virtual machine that Vagrant will manage.
Starting in version 1.0, Vagrant provides two installation methods: packaged installers for supported platforms or a universal install with Ruby Gems. This article covers installation using Gems. This method has three parts: 1) install VirtualBox, 2) install Ruby and 3) install Vagrant itself.
VirtualBox is available from the VirtualBox home page with builds for Windows, OS X, Linux and Solaris. Note that Oracle provides the Oracle VM VirtualBox Extension Pack on the Download site that provides additional features to the virtualizer. The Extension Pack has a separate license (Personal Use and Evaluation License) and is not needed to use Vagrant, but if the Box you are using was created using the Extension Pack, you will need to install the Extension Pack as well.
Ruby is a popular dynamically typed object-oriented scripting language. Ruby is available out of the box in OS X, and most Linux distributions also have a Ruby package available. For Windows users, the RubyInstaller Project provides an easy way to install the Ruby runtime.
Ruby libraries and applications are available in packages called RubyGems or Gems. Ruby comes with a package management tool called gem. To install Vagrant, run the gem command:
> gem install vagrant
Vagrant is a command-line tool. Calling vagrant without additional arguments will provide the list of available arguments. I'll visit most of these commands within this article, but here's a quick overview:
init — create the base configuration file.
up — start a new instance of the virtual machine.
suspend — suspend the running guest.
halt — stop the running guest, similar to hitting the power button on a real machine.
resume — restart the suspended guest.
reload — reboot the guest.
status — determine the status of vagrant for the current Vagrantfile.
provision — run the provisioning commands.
destroy — remove the current instance of the guest, delete the virtual disk and associated files.
box — the set of commands used to add, list, remove or repackage box files.
package — used for the creation of new box files.
ssh — ssh to a running guest.
The last thing you need to do in your installation is set up a base image. A Box, or base image, is the prepackaged virtual machine that Vagrant will manage. Use the box command to add the Box to your environment. The vagrant box add command takes two arguments, the name you use to refer to the Box and the location of the Box:
> vagrant box add lucid32 http://files.vagrantup.com/lucid32.box
This command adds a new Box to the system called “lucid32” from a remotely hosted site over HTTP. Vagrant also will allow you to install a Box from the local filesystem:
> vagrant box add rhel5.7 rhel5.7-20120120-1223.box [vagrant] Downloading with Vagrant::Downloaders::File... [vagrant] Copying box to temporary location... [vagrant] Extracting box... [vagrant] Verifying box... [vagrant] Cleaning up downloaded box... >
Now there are two Boxes installed:
>vagrant box list lucid32 rhel5.7
It is important to note that you can reuse these base images. A Box can be the base for multiple projects without contamination. Changes in any one project will not change the other projects that share a Box. As you'll see, changes in the base can be shared to projects easily. One of the powerful concepts about using Vagrant is that the development environment is now totally disposable. You persist your critical work on the Host, while the Guest can be reloaded quickly and provisioned from scratch.
Create a directory on the Host as your starting point. This directory is your working directory. Vagrant will share this directory between the Guest and the Host automatically. Developers can edit files from their preferred Editors or IDEs without updating the Guest. Changes made in the Host or Guest are immediately visible to the user from either perspective:
> mkdir ProjectX > cd ProjectX
To get started, you need a Vagrantfile. The Vagrantfile is to similar to a Makefile, a set of instructions that tells Vagrant how to build the Guest. Vagrant uses Ruby syntax for configuration. The simplest possible Vagrantfile would be something like this:
Vagrant::Config.run do |config| config.vm.box = "lucid32" end
This configuration tells Vagrant to use all the defaults and the Box called lucid32. There actually are four Vagrantfiles that are read: the local version in the current directory, a user version in ~/.vagrant.d/, a Box version in the Box file and an initial config installed with the Gem. Vagrant reads these files starting with the Gem version and finishing with the current directory. In the case of conflicts, the most recent version wins, so the current directory overrides the ~/.vagrant.d, which overrides the Box version and so on. Users can create a new Vagrantfile simply by running:
> vagrant init
This creates a Vagrantfile in the current directory. The generated Vagrantfile has many of the common configuration parameters with comments on their use. The file tries to be self-documenting, but additional information is available from the Vagrant Web site. To run most Vagrant commands, you need to be in the same directory as the Vagrantfile.
Let's give it a try:
$ vagrant up [default] Importing base box 'lucid32'... [default] The guest additions on this VM do not match the install version of VirtualBox! This may cause things such as forwarded ports, shared folders, and more to not work properly. If any of those things fail on this machine, please update the guest additions and repackage the box. Guest Additions Version: 4.1.0 VirtualBox Version: 4.1.8 [default] Matching MAC address for NAT networking... [default] Clearing any previously set forwarded ports... [default] Forwarding ports... [default] -- 22 => 2222 (adapter 1) [default] Creating shared folders metadata... [default] Clearing any previously set network interfaces... [default] Booting VM... [default] Waiting for VM to boot. This can take a few minutes. [default] VM booted and ready for use! [default] Mounting shared folders... [default] -- v-root: /vagrant
Let's break this down. I'm running with VirtualBox 4.1.8 and a Guest at 4.1.0. In this case it works smoothly, but the warning is there to help troubleshoot issues should they appear. Next, it sets up the networking for the Guest:
[default] Matching MAC address for NAT networking... [default] Clearing any previously set forwarded ports... [default] Forwarding ports... [default] -- 22 => 2222 (adapter 1)
I used the default network setting of Network Address Translation with Port Forwarding. I am forwarding port 2222 on the Host to port 22 on the Guest. Vagrant starts the Guest in headless mode, meaning there is no GUI interface that pops up. For users who want to use the GUI version of the Guest, the option is available in the Vagrantfile to run within a window. Once the network is set up, Vagrant boots the virtual machine:
[default] VM booted and ready for use! [default] Mounting shared folders... [default] -- v-root: /vagrant
After the Guest has started, Vagrant will add the shared folder. Vagrant uses the VirtualBox extensions to mount the current folder (ProjectX in this case) as /vagrant. Users can copy and manipulate files from the Host OS in the ProjectX directory, and all the files and changes will be visible in the Guest. If the shared folder isn't performing well because you have a large number of files, Vagrant does support using NFS. However, it does require that NFS is supported by both the Guest and Host systems. At this time, NFS is not supported on Windows Hosts.
To access the Guest, Vagrant Boxes have a default user with username:vagrant, password:vagrant and a default shared insecure ssh-key. Vagrant users usually have sudo rights if they need to make changes to the Guest OS. You can ssh to the Guest by using ssh vagrant@localhost 2222 on the Host or use the vagrant shortcut:
>vagrant ssh
Although you can automatically reach your Guest using vagrant ssh, sometimes it is useful to use scp or git, which refers to your SSH setup. To make life easier, you can update the entry in your ~/.ssh/config with information specific to the Guest (Windows users can achieve similar results by making similar updates to their PuTTY configuration):
>vagrant ssh-config Host default HostName 127.0.0.1 User vagrant Port 2222 UserKnownHostsFile /dev/null StrictHostKeyChecking no PasswordAuthentication no IdentityFile /Users/jay/.vagrant.d/insecure_private_key IdentitiesOnly yes
By plugging this information in to ~/.ssh/config, you can access your Guest with scp to move a file my_file from /vagrant to the Host desktop:
> scp default:/vagrant/my_file.txt ~/Desktop/my_file.txt
What if you want to run a Web server on your Guest that's accessible by the Host? Let's add one. Edit your Vagrant file, adding the following line:
config.vm.forward_port 80, 8080
The forward_port parameter allows the Host to forward ports from the Host to the Guest. In this case, you're forwarding port 8080 on the Host to 80 on the Guest. By default, Vagrant uses Network Address Translation networking, meaning the Guest can access the network through the Host. Machines on the Host network cannot reach the Guest directly unless the ports are forwarded. Vagrant also provides Host-only networking, where a Guest can access only the host or other Guests on the Host, and bridge networking where the Guest will be on the same network as the Host. Once you update the Vagrantfile, restart the system:
>vagrant reload
Make sure that when you open port forwarding in Vagrant you also are opening the appropriate firewall permissions on the Guest! If you try to make a connection to a Guest via port forwarding with a firewall turned on, the initial handshake will work with the host port (8080 in the example here) but fail if port 80 is blocked by the Guest firewall.
Once you're done with your work, you have a few choices. If you're just done for the day, or trying to get a few resources back for the Host, you can simply suspend Vagrant:
>vagrant suspend
when you want to restart. All state is maintained, and nothing is lost. Once an environment is completely finished, or if you want to start over from a clean slate, you can run vagrant destroy. This will remove the Guest and any changes to content outside of the shared folders (/vagrant). Shared folders reside on the Host and will survive the destruction of the Guest.
Creating and maintaining virtual images can be a delicate balance in terms of configuration. Over-configure the image, and it gets stale faster as applications are replaced with patches. Under-configure the image with a simple base, and each time a new instance is created, the user must spend the time configuring the Box to the exacting specifications of the project. With Vagrant, there is a good middle ground. A base image can be created and minimally updated to manage OS-level patches and libraries, while applications can be provisioned using tools like Chef, Puppet or shell scripts.
Many systems administrators are already familiar with Chef and Puppet, two configuration management tools that allow for consistent creation and maintenance of a system. By plugging Chef and Puppet into Vagrant, users can take immediate advantage of the scripts that already have been created by system administrators to create up-to-date images for development.
Chef and Puppet are both supported to work in solo/standalone mode and server mode. In Chef Solo or Puppet standalone, the configuration for the images are all self-contained. In server mode, Chef or Puppet Server users can leverage the existing Chef or Puppet infrastructure already established in their companies.
This article doesn't go into great detail about Chef, but let's look at a simple example of how using Chef can be helpful. Vagrant can run without additional servers in Chef-solo mode. To set up Chef, you first need a Cookbook. Cookbooks are a collection of Recipes. A Recipe is a description of how you would like the system configured. Recipes also are written in Ruby. For this example, I've downloaded Cookbooks available from the Opscode community site.
In this example, let's install Apache2 and MySQL on our Guest. Here's the setup. Create a directory called “cookbooks” under the ProjectX directory. Download the Cookbooks for Apache2 and MySQL from community.opscode.com (see Resources). In addition, you'll need two dependencies: the apt recipe to get the latest updates and openssl, which is required for the MySQL recipe.
Extract these files and save them to the cookbooks directory:
> ls -l cookbooks/ total 0 drwxr-xr-x@ 10 jay staff 340 Feb 16 18:49 apache2 drwxr-xr-x@ 9 jay staff 306 Feb 14 12:00 apt drwxr-xr-x@ 9 jay staff 306 Feb 16 18:23 mysql drwxr-xr-x@ 7 jay staff 238 Jun 3 2011 openssl
In my Vagrantfile, I have added the following lines:
config.vm.provision :chef_solo do |chef|
chef.cookbooks_path = "cookbooks"
chef.add_recipe("apt")
chef.add_recipe("openssl")
chef.add_recipe("mysql::server")
chef.add_recipe("apache2")
# You may also specify custom JSON attributes:
chef.json = {
:mysql => {
:server_root_password => "MYSQL PASSWORD"
}
}
end
When Vagrant runs the provisioning step (included in vagrant up but also available on a running Guest by using vagrant provision), it will install apt, openssl, mysql and apache2. Recipes can take arguments as well. In this example, I've included a parameter for the mysql recipe that sets the server root password. The parameters available for a given recipe are included in the documentation. The parameters are passed as JSON attributes to Vagrant, which will pass them along to Chef.
Managing this with Vagrant allows you to enforce change controls around your environments. Vagrantfiles and Cookbooks can be checked into source control and managed just like code resources. Branching and tagging environment configurations can be used to track changes as projects develop or requirements and packages are updated.
The traditional method is to use VirtualBox to create a new virtual machine image using the standard operating system installer. Because operating system steps vary, I'm not going to go into the details here, but here are some key steps:
When choosing the Virtual Disk storage details, choose “Dynamically Allocated”—you want your disk to start small but be able to increase as needed.
If you want to use Chef or Puppet with your application, you'll need to install the Chef or Puppet applications when creating the base image.
Install the VirtualBox Guest Additions. This is necessary to support shared folders!
Let's set up the vagrant user. Create a user with the user name vagrant. Add this user to an admin group. Update the sudoers file to make sure that users in the admin group do not need a password to access sudo. For a group called “admin”, it should look like this:
%admin ALL=NOPASSWD:ALL
Also make sure that the Defaults requiretty is commented out. You also want to make sure you can ssh into the Vagrant account. By default, Vagrant will try to use the insecure SSH key with vagrant ssh. To add the key to your Vagrant account, do the following as the vagrant user:
> mkdir .ssh > chmod 755 .ssh > curl -L http://github.com/mitchellh/vagrant/raw/master/ ↪keys/vagrant.pub > > .ssh/authorized_keys > chmod 644 .ssh/authorized_keys
Once the image is complete, you'll need to package the image into a Box. To create a set of default configurations for the Box, you can create a new Vagrantfile. Use vagrant package, which takes two arguments --base (the name of the image in VirtualBox that you created) and --include (which takes the files you wish to include in your Box):
> vagrant package --base RHEL-5.7-64 --include Vagrantfile
In this example, you created a package with the base image called RHEL-5.7-64 and included your custom Vagrantfile.
Vagrant is a powerful tool for creating and managing flexible development environments. In this article, I covered the basic use and creation of Vagrant images.