Building a Vagrant Box from Start to Finish

Note: This blog was originally published on tylerbird.org earlier this year. It was recently updated for publication on the Engine Yard blog.

The goal of Vagrant is to make it so simple to create a local development environment, you’d never want to do it another way again. With two simple commands you can quickly setup your first vagrant environment and with a third command, be connected into your first vagrant box in under a few minutes.

vagrant init precise32 http://files.vagrantup.com/precise32.box
vagrant up
vagrant ssh # you're in!

This power is amazing and so useful for freelancers who work on many different projects, or companies that need to bring new members of a team up to speed as quickly as possible. With the use of Vagrant, the configuration of the development and the production environment can be mirrored as closely as possible. And errors like “works on my machine” can become a thing of the past.

If you’re already a fan a Vagrant, I’m probably preaching to the choir. For more about what Vagrant can do check out the amazing documentation. Or do what I did and pick up the O’Reilly book by Vagrant’s author, Mitchell Hashimoto, Vagrant: Up and Running and read it. It’s a short but jam packed review of the details you need to get to know Vagrant, inside and out.

Why Build A Box?

There are a bunch of amazing boxes out there available on sites like vagrentbox.es and vagrantcloud.com so why would you want to build your own box?

Maybe you want to add a few extra things to your base (a runtime or two like Julia, Erlang, JVM or Python, etc.) then start this as your new “base”.

Maybe you want your box to have more ram or you need your boxes to more closely mirror production and you are building a ram enriched, multiple server cluster with multiple provisioners, we get it… you’ve got a mountain to climb, you just need your gear.

Definitions

What is a package.box file? When using the VirtualBox provider, it’s a tarred, gzip file containing the following:

Vagrantfile
box-disk.vmdk
box.ovf
metadata.json

The Vagrantfile has some information that will be merged into your Vagrantfile that is created when you run vagrant init boxname in a folder.

The box-disk.vmdk is the virtual hard disk drive.

The box.ovf defines the virtual hardware for the box.

The metadata.json tells vagrant what provider the box works with.

NOTE: These contents would be different for the VMWare provider, etc. For more about that please refer to the vagrant docs on boxes.

Shout Out

I’m a fan of Vagrant and all things automation, they make my life a lot easier. First I’d like to give credits to my main sources which are:

• The blog article Creating a Custom Box from Scratch by Ryan Skoblenick.
• And the book Vagrant: Up and Running by Mitchell Hashimoto.

So why did I write this blog article, if Ryan’s existed? Good question.

Well it only got me 90% of the way there. I had problems getting the Guest Tools installed and configuring the SSH User for the operating system using his article.

So my hope is that this article will improve upon that process and add a voice of sanity to those who need some help getting a box built by hand.

Getting Prepared

If you don’t already have Vagrant and VirtualBox, grab those.

Download Vagrant installer for your operating system.

Download VirtualBox installer for your operating system.

NOTE: Grab the VirtualBox Extension Pack as well. For more about the functionality it provides read here.

Prior to the installation packages, Vagrant was distributed as a RubyGem. Installation packages are now the preferred way to install Vagrant, so you should uninstall the RubyGem version and follow the instructions for your platform. The RubyGem-based installation is still supported for Vagrant 1.0.x, but is deprecated and will not be supported in any future versions.

Page 8, “Vagrant: Up and Running by Mitchell Hashimoto (O’Reilly). Copyright 2013 Mitchell Hashimoto, 978-1-449-33583-0.”

RUBY USERS: If you had Vagrant installed as a rubygem, uninstall it before you install vagrant:

gem uninstall vagrant

Run the Vagrant installer and the VirtualBox and VirtualBox Extension Pack installers, once they are installed, if you feel like a reboot would help, go ahead.

Download ubuntu server, the rest of the instructions will be for ubuntu, but if you’re choosing another distro you’re probably fine to adapt the commands that follow.

Build a Box

We’re going to use VirtualBox to build an ubuntu server from scratch. The reason for this is because Vagrant has native support for VirtualBox. There are more plugins out there for other providers like VMWare, Parallels or Vagrant-LXC, etc. We’ll stick to VirtualBox for this guide.

When we setup our ubuntu server, it prompts us to setup a default user. We’re going to name that user vagrant so it’s the default user as well. This will make it the default SSH user and streamline the process.

Configure the Virtual Hardware

Create a new Virtual Machine with the following settings:

• Name: vagrant-ubuntu64
• Type: Linux
• Version: Ubuntu64
• Memory Size: 512MB (to taste)
• New Virtual Disk: [Type: VMDK, Size: 40 GB]

Modify the hardware settings of the virtual machine for performance and because SSH needs port-forwarding enabled for the vagrant user:

• Disable audio
• Disable USB
• Ensure Network Adapter 1 is set to NAT
• Add this port-forwarding rule: [Name: SSH, Protocol: TCP, Host IP: blank, Host Port: 2222, Guest IP: blank, Guest Port: 22]

Mount the Linux Distro ISO and boot up the server.

Install The Operating System

Setting up ubuntu is very simple. Follow the on-screen prompts. And when prompted to setup a user, set the user to vagrant and the password to vagrant. It will give you a passive-aggressive guilt trip about it being a weak sauce password. Don’t let that shake you, be strong and soldier through.

Set Root Password

In order to setup the Super User, aka root user, you’ll need to be able to sign in as that user. Since I asked you to make vagrant the default user while installing the Operating System in the last step, these commands should help you set the root password and then sign in as root in order to make the next configuration changes below.

sudo passwd root

This will prompt you to type the password twice, where I’d suggest the password “vagrant”. Now sign in as the root user in order to Setup the Super User next.

su -

Setup the Super User

Vagrant must be able run sudo commands without a password prompt and if you’re not configuring ubuntu at this point, just make sure that requiretty is disabled for the vagrant user.

The most efficient way I’ve found to setup the vagrant user so it’s able to use sudo without being prompted for a password is to add it to the sudoers list like this:

sudo visudo -f /etc/sudoers.d/vagrant

Anything in the /etc/sudoers.d/* folder is included in the “sudoers” privileges when created by the root user. So that’s why we created this as the root user and in that folder.

With that file open add this to the file then save it and exit.

# add vagrant user
vagrant ALL=(ALL) NOPASSWD:ALL

Now you can test that it works by running a simple command:

sudo pwd

It will return the home folder without prompting you for a password if everything is setup correctly. If you are prompted for a password, something’s out of wack and things won’t work right. This step was CRUCIAL for me, so please ensure this test passes for you.

Updating The Operating System

One of the reasons we are building this box is so we can save time by already being as up to date as the box was built. So let’s get up-to-date first.

sudo apt-get update -y
sudo apt-get upgrade -y

Usually if there are kernel updates you’ll want to reboot the server. So do that.

sudo shutdown -r now

Install the Vagrant Key

The only way that all the vagrant commands will be able to communicate over ssh from the host machine to the guest server is if the guest server has this “insecure vagrant key” installed. It’s called “insecure” because essentially everyone has this same key and anyone can hack into everyone’s vagrant box if you use it.

But at the same time, we’re hoping you’re not running around with all your most valuable company data on your vagrant boxes, right? RIGHT? OK. Good.

mkdir -p /home/vagrant/.ssh
chmod 0700 /home/vagrant/.ssh
wget --no-check-certificate \
    https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub \
    -O /home/vagrant/.ssh/authorized_keys
chmod 0600 /home/vagrant/.ssh/authorized_keys
chown -R vagrant /home/vagrant/.ssh

Install and Configure OpenSSH Server

If you didn’t install SSH while installing the operating system you can do it now: sudo apt-get install -y openssh-server

We need to edit the /etc/ssh/sshd_config file:

sudo nano /etc/ssh/sshd_config

Find and uncomment the following line because we added the Vagrant key above to the authorized_keys file:

AuthorizedKeysFile %h/.ssh/authorized_keys

Then restart ssh:

sudo service ssh restart

Installing Guest Tools

Guest Tools help the operating system handle shared folders and “optimize the guest operating system for better performance and usability.” 2

A compiler is required to install the Guest Tools, use this command:

sudo apt-get install -y gcc build-essential linux-headers-server

In VirtualBox browse to the Devices menu at the top, then in the drop-down list at the bottom, click on Insert Guest Additions CD Image.

This will add an ISO image to the virtual CDROM running in your server. Run these commands to mount your cdrom and then run the script. NOTE: The message about the cdrom being read-only is fine.

sudo mount /dev/cdrom /mnt 
cd /mnt
sudo ./VBoxLinuxAdditions.run

Package the Box

Before you package the box you’ll want to “zero out” the drive. “This fixes fragmentation issues with the underlying disk, which allows it to compress much more efficiently later.” 3

sudo dd if=/dev/zero of=/EMPTY bs=1M
sudo rm -f /EMPTY

Now we’re ready to package the box. I usually make a folder to hold my boxes like so:

mkdir ~/code/personal/vagrant_boxes
cd ~/code/personal/vagrant_boxes

This is the command that finally packages up the box for you as we defined above into the compressed gzip tarball file, it also generates and includes the Vagrantfile and the metadata.json file.

vagrant package --base vagrant-ubuntu64

Vagrant will then check VirtualBox for any instances of the name vagrant-ubuntu64 and attempt to ssh into them and control them.

→ vagrant package --base vagrant-ubuntu64
[vagrant-ubuntu64] Attempting graceful shutdown of VM...
[vagrant-ubuntu64] Forcing shutdown of VM...
[vagrant-ubuntu64] Clearing any previously set forwarded ports...
[vagrant-ubuntu64] Exporting VM...
[vagrant-ubuntu64] Compressing package to: /Users/tbird/code/personal/virtual_boxes/package.box

You are left with the package.box file in your ~/code/personal/vagrant_boxes folder.

Test Your Box

From your same vagrant_boxes folder you can run these final test commands. All the heavy lifting is really done at this point. If you’ve screwed up something it’s probably in a step up above.

You should be in ~/code/personal/vagrant_boxes/ and type:

vagrant box add ubuntu64 package.box
vagrant init ubuntu64
vagrant up

Connect to the server you created from start to finish!

vagrant ssh

You’ve won, you deserve to get your high five on.

Packer

Just when you thought it was safe to do things manually at the beginning … in comes Packer. What is Packer, you say? Well Packer automates everything we just did. /FACEPALM

Anyway… Once you have Vagrant, VirtualBox, and Packer installed you can define a quick-start.json file like:

{
  'builders': [{
    'type': 'amazon-ebs',
    'access_key': 'YOUR KEY HERE',
    'secret_key': 'YOUR SECRET KEY HERE',
    'region': 'us-east-1',
    'source_ami': 'ami-de0d9eb7',
    'instance_type': 't1.micro',
    'ssh_username': 'ubuntu',
    'ami_name': 'packer-example '
  }]
}

After changing the Access and Secret Key you could run the packer command with the quick-start.json file:

packer build quick-start.json

Packer will then automate the creation of that quick-start.json machine image for Amazon EC2. That can optionally include the creation of a Vagrant box.

What Packer aims to do is to make their images compatible with all these providers: Amazon EC2 (AMI), DigitalOcean, Docker, Google Compute Engine, OpenStack, QEMU for KVM or Xen instances, or even VirtualBox or VMWare software. (See platforms for the entire list.)

What Have We Learned

What have we learned from this process of building boxes for Vagrant?

The process of automation is the process of taking yourself out of the equation. “Auto” is defined as self, and the suffix “ation” is defined as action. So if we think of the job of automating your development environment or production environment so it runs without your intervention, then your actions are doing it right!

And even if we want to do it by hand or use Packer, knowing how things work is a lot better than guessing how they work. And I like to experience things a few times before I start to automate those boring parts away, and then focus on new more interesting challenges.

Deploy and run your Ruby apps on Engine Yard
{{cta(‘5019e5e3-8348-4b05-b8bf-ede6bb67025e’)}}