Blog

Getting Hired: fog Edition

By | November 30th, 2010 at 11:11AM

Having Ruby experience makes you hirable; but how can you stand out? You need to demonstrate your abilities. What better way than using Ruby and “the cloud” to store and serve your resume!

In this blog post you will learn to use fog – the cloud computing library – to upload your resume to Amazon’s Simple Storage Service (S3), Rackspace’s CloudFiles or Google’s Storage for Developers.

Here’s my out of date resume stored on S3, CloudFiles and Google Storage; programmatically stored in the cloud using this tutorial. NOTE: my boss would like me to add that I’m not currently looking for a new gig ;)

Check out those cloud-specific URLs! You could put all three in your job application, add the Ruby source for how you did it, and have your choice of Ruby jobs for being so awesome!

How? The all-clouds-in-one library of choice is fog.

Installing fog

fog is distributed as a RubyGem:
gem install fog

Or add it in your application’s Gemfile:

gem "fog"

Using Amazon S3 and fog

Sign up for an account here and copy down your secret access key and access key id from here. We are about to get into the code samples, so be sure to fill in anything in ALL_CAPS with your own values!

First, create a connection with your new account:

require 'rubygems'
require 'fog'
 
# create a connection
connection = Fog::Storage.new(
  :provider                 => 'AWS',
  :aws_secret_access_key    => YOUR_SECRET_ACCESS_KEY,
  :aws_access_key_id => YOUR_SECRET_ACCESS_KEY_ID
)
 
# First, a place to contain the glorious details
directory = connection.directories.create(
  :key    => "fog-demo-#{Time.now.to_i}", # globally unique name
  :public => true
)
 
# list directories
p connection.directories
 
# upload that resume
file = directory.files.create(
  :key    => 'resume.html',
  :body   => File.open("/path/to/my/resume.html"),
  :public => true
)

If you are anything like me, you will continually tweak your resume. Pushing updates is easy:

file.body = File.open("/path/to/my/resume.html")
file.save

As you can see, cloud storage files in fog are a lot like an ActiveRecord model. Attributes that can be changed and a `#save` method that creates or updates the stored file in the cloud.

But if it took you longer to realize the mistake you might not still have file around, but you’ve got options.

directory = connection.directories.get("proclamations1234567890")
 
# get the resume file
file = directory.files.get('resume.html')
file.body = File.open("/path/to/my/resume.html")
file.save
 
# also, create(attributes) is just new(attributes).save, so you can also do:
file = directory.files.new(
  :key => 'resume.html',
  :body => 'improvements',
  :public => true
)
file.save

Alright, so you (eventually) become satisfied enough to send it off, what is the URL endpoint to your resume?

puts file.public_url

Pop that link in an email and you should be ready to cruise job ads and send your resume far and wide (Engine Yard is hiring, so check us out!). Now you are set, unless you are interviewing for Google, or Rackspace… Both of these companies have their own cloud storage services, so using Amazon S3 might not be the foot in the door you hoped for.

More clouds? How much extra stuff will you have to do for these services!?! Hardly anything needs to change, you just have to pass slightly different credentials in, but I’m getting ahead of myself.

Google Storage for Developers

Sign up here and get your credentials here.
connection = Fog::Storage.new(
  :provider => 'Google',
  :google_storage_secret_access_key => YOUR_SECRET_ACCESS_KEY,
  :google_storage_access_key_id => YOUR_SECRET_ACCESS_KEY_ID
)

Rackspace CloudFiles

Rackspace has Cloud Files and you can sign up here and get your credentials here.
connection = Fog::Storage.new(
  :provider => 'Rackspace',
  :rackspace_username => RACKSPACE_USERNAME,
  :rackspace_api_key => RACKSPACE_API_KEY
)
Then create, save, destroy as per fog-for-AWS. The `:public => true` option when creating directories (see above) is important for Rackspace; your folder and files won’t be shared to Rackspace’s CDN and hence your users without it. Similarly the `:public => true` on files is important for AWS and Google or they will be private.

Local Storage

While you are working out the kinks you might not want to do everything live though, ditto for while you are running tests, so you have a couple options to try before you buy. First, you can use a local provider to store things in a directory on your machine.
connection = Fog::Storage.new(
  :provider => 'Local',
  :local_root => '~/fog'
)

Mocking out Cloud Storage

Of course when you are testing or developing you can always just use the mocks (at least for AWS and Google, Rackspace still needs mocks implemented if you are looking for somewhere to contribute). They emulate the behavior of the external systems without actually using them. It is as simple as:
Fog.mock!
connection = Fog::Storage.new(config_hash)

Cleaning up

Fog takes care of the rest so you can focus on your cover letter. And with the awesome cover letter and cloud delivered resume you are probably a shoe-in. So all that is left is to cleanup that leftover job hunt residue.
file.destroy
directory.destroy

Summary

All done. Try out all the different options and let me know if you have any bugs or issues. I also wrote up a more consolidated example as a script that you can use for reference.

Bonus, note the `Fog.mock!` command. In your tests you can easily mock out calls to cloud providers.

Please let me know in the comments if you got a new Ruby job because you hosted your CV on 3 different Cloud Stores without getting your hands dirty.

Have questions or comments? Hop into #ruby-fog on freenode, ping @fog or @geemus.

And please always remember that I accept high fives and contributions!

  • Melissa

    Job hunters and folks hiring, you might also take advantage of this great *free* resource to help in your job search or candidate search: http://jobs.engineyard.com/

  • Federico

    I have a better idea: https://github.com/febuiles/s3share and "s3.rb resume.pdf" :P

  • http://s3.cloudberrylab.com/ Andy, CloudBerry lab

    I always enjoy learning what other people think about Amazon Web Services and how they use them. Check out my very own tool CloudBerry Explorer that helps manage S3 on Windows . It is freeware. http://s3.cloudberrylab.com/ We also have a similar product for Google Storage.

  • http://rubyinair.com Lakshmanan

    Cool Library !!

  • http://www.antiveb.com Uros Jurglic

    Thanks for intro, but unfortunately it didnt worked very well for me.

    First the code here doesnt work with fog 0.3.26 (and ruby 1.9.2). The keys for connection should strangely be aws_access_key_id and aws_secret_access_key. After some trial/error I got that figured out, but couldnt find any docs describing the required keys. Is there any useful docs for fog? I just couldnt find it, and the rdocs doesnt reveal much (altough would be nice to put a link to rdocs into github readme).

    Playing around some more, it worked sporadically. I found that if my bucket is in europe, the file doesnt get saved, and I dont get any error as well. Tried setting region/location on connection, but to no success since I couldnt find any docs describing these parameters. Then the file.public_url didnt work as well. Sometimes I got an error, sometimes not even that.

    I really like the idea of fog and would be glad to use it, but it doesnt seem ready yet.

  • geemus

    Sorry about the difficulties you faced. I'll try to respond to each in turn.

    First off, sorry about the key thing. I did it correctly in my extended example <a href="http://(https://gist.github.com/710869)” target=”_blank”>(https://gist.github.com/710869) but I guess the wires got crossed when I was trying to add all that stuff into the blog post. You can see which parameters are required and recognized(optional) for aws storage here: https://gist.github.com/710869 (sorry it isn't proper documentation at present, but better than nothing).

    If you pass :region =&gt; 'eu-west-1' it should create a connection to europe for you that should otherwise work as normal. Otherwise many requests will automatically follow redirects to do things in Europe where necessary.

    I'm not sure outside that why you would have sporadic errors related to saving files. One thing to note is that when you use files.new it simply makes a local representation. It requires calling save in order to actually persist the file, similarly you can use files.create to do new+save in one step. Outside that if something is going wrong on amazon's side I would expect an error to be raised.

    I hope that helps clear some of that up for you. If not please follow up (if you can provide an example of how you are using it that can often help me clear things up much quicker). I'm glad you like the idea of fog and I am sorry that the process of getting started can still be a bit rocky. I have been doing my best to improve it but I know I still have a ways to go. Let me know if there is anything more I can help clear up for you in the mean time. Thanks!

  • http://www.antiveb.com Uros Jurglic

    HI Wesley, thanks for prompt response!

    I've tried again, but I still got many errors. If I follow your example, and don't use any specific region, I can access/create the S3 bucket, and push the file. The first problem arises when I call file.public_url. – I get an error: Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
    . Which is very strange, because I call this in the same code with the same connection I uploaded the file, meaning with the same credentials. Here you can look at the code, it's basically your example: https://gist.github.com/729611

    As for the region, I also get an error:
    <Code>IllegalLocationConstraintException</Code><Message>The unspecified location constraint is incompatible for the region specific endpoint this request was sent to.</Message>

    Also, may I ask where did you find the value 'eu-west-1' region? I tried looking at the s3 docs, it says just 'EU', but it doesn't work as well. I get na uknown region error. http://docs.amazonwebservices.com/AmazonS3/latest…

    There, I'll try some more when I have time. I must admit, I didn't use much S3 before this testing, so maybe I'm missing something. But on the other hand the api is really simple, not much I could do terribly wrong :)

  • geemus

    First I have to admit that I haven't messed with different regions too terribly much. It is probably one of the most painful things to run into because it is sometimes difficult to know how to do things properly and the error messages are often pretty cryptic. That said, as long as you always do all the requests in the same region it is usually much more helpful. I wonder if perhaps you accidently created that bucket in a different region previously, so when you create it now it isn't causing an error (recreating buckets is valid in s3), but that when it is recreated it isn't over writing the location constraint.

    eu-west-1 is a more direct reference to the endpoint url that it uses for that location constraint. You can see the endpoints here: http://docs.amazonwebservices.com/AmazonS3/latest…

    If you want to create a bucket specifically in eu you should still use 'EU' as the location for it. So you would probably use 'eu-west-1' to make the connection but then do something like:
    storage.directories.new(:key => 'foobarbaz', :public => true, :location => 'EU')

    I would suggest trying to start from a clean slate and see if you can get the workflow to work. So either delete the bucket you were testing with before, or choose a new name to try. Then make sure that the region of the connection and the region set for the bucket agree (setting neither would default to us-east, which is maybe easiest). Once you've gotten that to work you can try again with EU and a new bucket and hopefully get the same results. If that still isn't working out for you let me know though (I'm trying to finish getting a big presentation together for tomorrow so I don't really have time to dig in too much but I wanted to try to provide some ideas for you). So if you are still stuck let me know and I'll try again (just maybe not until tomorrow after the presentation). Thanks!

  • http://ksylvest.com/ Kevin Sylvestre

    Hi Wesley,

    Cool looking gem. I had a few quick questions:

    How does the fog S3 performance compare against the AWS::S3 gem?
    Does fog support ActionDispatch::Http::UploadedFile as the body of transfers (the Rails upload file)?
    Do you support renaming (without a copy and delete operation)?

    Thanks!

  • geemus

    thanks!

  • geemus

    Thanks!

    Performance wise I haven't done a direct comparison to AWS::S3 recently, but when I last did fog was comparable or better in nearly every case.

    fog is mostly geared toward doing either String or File bodies for uploads. But if ActionDispatch::Http::UploadedFile instances will return true with is_a?(File) then it should just work. I don't happen to know off hand if this is the case or not. If it ends up that they don't work correctly we can work to remedy the situation.

    As for renaming I think you are just stuck with copy/delete. It would be nice to do it all at once but to the best of my knowledge that is not an operation that s3 provides.