Building a Vagrant Box from Start to Finish

     

building a vagrant box

Note: This article was originally published at tylerbird.org. It was recently updated for publication on the Engine Yard blog.

The goal of Vagrant is to make it simple to create a local development environment so that you’d never want to do it another way again. You can quickly set up your first Vagrant environment with two simple commands. And with a third, you’re connected to your first Vagrant box—all in under a few minutes.

vagrant init precise32 http://files.vagrantup.com/precise32.box
vagrant up
vagrant ssh # you're in!

This power is amazing and applicable for freelancers who work on many different projects—or companies that need to onboard new members of a team as quickly as possible. With Vagrant, the configuration of development and the production environment can be mirrored as closely as possible. Errors like “works on my machine” become a thing of the past.

Already a fan of Vagrant? Well, I’m preaching to the choir. For more about what Vagrant can do, check out the amazing documentation. Or do what I did and pick up Vagrant: Up and Running by Vagrant’s creator, Mitchell Hashimoto. It’s a short review, but it’s jam-packed with the details you need to get to know Vagrant.

Creating a Custom Box from Scratch, a blog written by Ryan Skoblenick, is another fantastic resource—but it only got me 90% of the way there. I had problems getting the Guest Tools installed and configuring the SSH User for the operating system. Thus, it’s my hope that this article improves upon that process and adds a voice of sanity to those who need some help getting a box built by hand.

So, Why Build a Box?

There are a bunch of amazing boxes out there available on sites like vagrantbox.es and Vagrant Cloud. Why would you want to build your own box?

Maybe you want to add a few extra things to your base and then start this as your new “base.” Maybe you want your box to have more RAM. Or perhaps you need boxes to more closely mirror production, and you are building a RAM-enriched, multiple-server cluster with multiple provisioners.

We get it…you’ve got a mountain to climb.

Definitions

What’s a package.box file? When using the VirtualBox provider, it’s a tarred, gzip file that contains the following:

Vagrantfile
box-disk.vmdk
box.ovf
metadata.json

The Vagrantfile has some information that will be merged into your Vagrantfile that’s created when you run vagrant init boxname in a folder.

  • The box-disk.vmdk is the virtual hard disk drive.
  • The box.ovf defines the virtual hardware for the box.
  • The metadata.json tells Vagrant what provider the box works with.

These contents would be different for the VMWare provider, etc. For more about this, refer to the Vagrant docs on boxes.

Getting Prepared

If you don’t already have Vagrant and VirtualBox, grab those.

Be sure to grab VirtualBox extension packs as well. For more about the functionality that these provide, read here.

“Prior to the installation packages, Vagrant was distributed as a RubyGem. Installation packages are now the preferred way to install Vagrant, so you should uninstall the RubyGem version and follow the instructions for your platform. The RubyGem-based installation is still supported for Vagrant 1.0.x, but is deprecated and will not be supported in any future versions.”

Page 8, Vagrant: Up and Running by Mitchell Hashimoto (O’Reilly). Copyright 2013 Mitchell Hashimoto, 978-1-449-33583-0.

RUBY USERS: If you had Vagrant installed as a RubyGem, uninstall it before you install Vagrant:

gem uninstall vagrant

Run the Vagrant installer and the VirtualBox and VirtualBox extension pack installers; once they are installed, if you feel like a reboot would help, go ahead.

Download Ubuntu Server. The rest of the instructions will be for Ubuntu, but if you’re choosing another distro, you’re probably fine to adapt the commands that follow.

Build a Box

We’re going to use VirtualBox to build an Ubuntu server from scratch. The reason for this is because Vagrant has native support for VirtualBox. There are more plug-ins out there for other providers like VMWare, Parallels, and Vagrant-LXC. We’ll stick to VirtualBox for this guide.

When we set up our Ubuntu server, it prompts us to set up a default user. We’re going to name that user vagrant so it’s the default user as well. This will make it the default SSH user and streamline the process.

CONFIGURE THE VIRTUAL HARDWARE

Create a new virtual machine with the following settings:

  • Name: vagrant-ubuntu64
  • Type: Linux
  • Version: Ubuntu64
  • Memory Size: 512MB (to taste)
  • New Virtual Disk: [Type: VMDK, Size: 40 GB]

Modify the hardware settings of the virtual machine for performance and because SSH needs port-forwarding enabled for the Vagrant user:

  • Disable audio
  • Disable USB
  • Ensure Network Adapter 1 is set to NAT
  • Add this port-forwarding rule: [Name: SSH, Protocol: TCP, Host IP: blank, Host Port: 2222, Guest IP: blank, Guest Port: 22]

Mount the Linux Distro ISO and boot up the server.

INSTALL THE OPERATING SYSTEM

Setting up Ubuntu is simple. Follow the on-screen prompts, and when prompted to set up a user, set the user to vagrant and the password to vagrant. It will respond with a passive-aggressive comment about it being a weak password. Don’t let that shake you. Be strong and soldier through.

SET ROOT PASSWORD

To set up the super user—aka root user—you’ll need to be able to sign in as that user. Because I asked you to make vagrant the default user while installing the operating system in the last step, these commands should help you set the root password and then sign in as root to make the next configuration changes below.

sudo passwd root

This will prompt you to type the password twice—where I’d suggest the password vagrant. Sign in as the root user to set up the super user next.

su -

SET UP THE SUPER USER

Vagrant must be able to run sudo commands without a password prompt, and if you’re not configuring Ubuntu at this point, just make sure that requiretty is disabled for the Vagrant user.

The most efficient way to set up the Vagrant user so that it’s able to use sudo without being prompted for a password is to add it to the sudoers list like this:

sudo visudo -f /etc/sudoers.d/vagrant

Anything in the /etc/sudoers.d/* folder is included in the sudoers privileges when created by the root user. That’s why we created this as the root user and in that folder.

With that file open, add this to the file and then save it and exit.

# add vagrant user
vagrant ALL=(ALL) NOPASSWD:ALL

You can now test that it works by running a simple command:

sudo pwd

It will return the home folder without prompting you for a password if everything is set up. If you’re prompted for a password, something is wrong. And things won’t work right. This step was CRUCIAL for me, so please ensure that this test passes for you.

UPDATING THE OPERATING SYSTEM

One of the reasons we are building this box is so that we can save time by already being as up to date as when the box was built. Let’s get up to date first.

sudo apt-get update -y
sudo apt-get upgrade -y

Usually if there are kernel updates, you’ll want to reboot the server. So, do that.

sudo shutdown -r now

INSTALL THE VAGRANT KEY

The only way that all the Vagrant commands can communicate over SSH from the host machine to the guest server is if the guest server has this “insecure Vagrant key” installed. It’s called “insecure” because essentially everyone has this same key, and anyone can hack into everyone’s Vagrant box if you use it.

At the same time, we’re hoping you’re not running around with all your most valuable company data on your Vagrant boxes, right?

RIGHT? OK. Good.

mkdir -p /home/vagrant/.ssh
chmod 0700 /home/vagrant/.ssh
wget --no-check-certificate \
          https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub \
          -O /home/vagrant/.ssh/authorized_keys
chmod 0600 /home/vagrant/.ssh/authorized_keys
chown -R vagrant /home/vagrant/.ssh

INSTALL AND CONFIGURE OPENSSH SERVER

If you didn’t install SSH while installing the operating system, you can do it now: sudo apt-get install -y openssh-server

We need to edit the /etc/ssh/sshd_config file:

sudo nano /etc/ssh/sshd_config

Find and uncomment the following line because we added the Vagrant key above to the authorized_keys file:

AuthorizedKeysFile %h/.ssh/authorized_keys

Then restart SSH:

sudo service ssh restart

INSTALLING GUEST TOOLS

Guest Tools help the operating system handle shared folders and, according to the manual, “optimize the guest operating system for better performance and usability.”

A compiler is required to install the Guest Tools. Use this command:

sudo apt-get install -y gcc build-essential linux-headers-server

In VirtualBox, browse to the Devices menu at the top and then in the drop-down list at the bottom, click on Insert Guest Additions CD Image.

This will add an ISO image to the virtual CD-ROM running in your server. Run these commands to mount your CD-ROM and then run the script.

NOTE: The message about the CD-ROM being read-only is fine.

sudo mount /dev/cdrom /mnt
cd /mnt
sudo ./VBoxLinuxAdditions.run

PACKAGE THE BOX

Before you package the box, you’ll want to “zero out” the drive. According to Vagrant: Up and Running: “This fixes fragmentation issues with the underlying disk, which allows it to compress much more efficiently later.”

sudo dd if=/dev/zero of=/EMPTY bs=1M
sudo rm -f /EMPTY

Now we’re ready to package the box. I usually make a folder to hold my boxes—like so:

mkdir ~/code/personal/vagrant_boxes
cd ~/code/personal/vagrant_boxes

This command finally packages up the box for you, as we defined above, into the compressed gzip tarball file. It also generates and includes the Vagrantfile and the metadata.json file.

vagrant package --base vagrant-ubuntu64

Vagrant will then check VirtualBox for any instances of the name vagrant-ubuntu64 and attempt to SSH into them and control them.

→ vagrant package --base vagrant-ubuntu64
[vagrant-ubuntu64] Attempting graceful shutdown of VM...
[vagrant-ubuntu64] Forcing shutdown of VM...
[vagrant-ubuntu64] Clearing any previously set forwarded ports...
[vagrant-ubuntu64] Exporting VM...
[vagrant-ubuntu64] Compressing package to: /Users/tbird/code/personal/virtual_boxes/package.box

You are left with the package.box file in your ~/code/personal/vagrant_boxes folder.

TEST YOUR BOX

From your same vagrant_boxes folder, you can run these final test commands. All the heavy lifting is done at this point—and if you’ve screwed up, it’s probably in a step up above.

You should be in ~/code/personal/vagrant_boxes/ and type:

vagrant box add ubuntu64 package.box
vagrant init ubuntu64
vagrant up

Connect to the server you created from start to finish!

vagrant ssh

You’ve won. You deserve a high-five.

Packer

Just when you thought it was safe to do things manually at the beginning… in comes Packer. What is Packer, you ask?

Well, Packer automates everything we just did. /FACEPALM

Anyway… once you have Vagrant, VirtualBox, and Packer installed, you can define a quick-start.json file like:

{
    "builders": [{
    "type": "amazon-ebs",
    "access_key": "YOUR KEY HERE",
    "secret_key": "YOUR SECRET KEY HERE",
    "region": "us-east-1",
    "source_ami": "ami-de0d9eb7",
          "instance_type": "t1.micro",
    "ssh_username": "ubuntu",
    "ami_name": "packer-example "
    }]
}

After changing the Access and Secret Key, you can run the Packer command with the quick-start.json file:

packer build quick-start.json

Packer will then automate the creation of that quick-start.json machine

image for Amazon EC2. That can optionally include the creation of a Vagrant box.

What Packer aims to do is to make its images compatible with all these providers: Amazon EC2 (AMI), DigitalOcean, Docker, Google Compute Engine, OpenStack, QEMU for KVM or Xen instances, or even VirtualBox or VMWare software. (See platforms for the entire list.)

What Have We Learned?

What have we learned from this process of building boxes for Vagrant?

The process of automation is the process of taking yourself out of the equation. If we think of the job of automating your development environment or production environment so that it runs without your intervention, then your actions are doing it right!

Even if we want to do it by hand—or use Packer—knowing how things work is a lot better than guessing how they work. Plus, I like to experience things a few times before I start to automate those boring parts so that I can focus on new, more interesting challenges.

Why Engine Yard Is Better Than Do It Yourself

Tyler Bird

Comments

Subscribe Here!