My son, John Presley, may be amongst the world’s most start-up savvy 13 year olds. Engine Yard was started when he was just 7. Being a precocious start-up, it quickly became a major influence on my family’s daily routine: I quickly became far less available for “family time” than I previously had been.

Children are very good at getting what they want, however, and John quickly discovered that if he talked to me about Engine Yard, I was far more talkative than if he started a conversation about a subject more to his liking, such as Sponge Bob or Mario Kart.

So it happened that part of his bedtime routine, after a bit of reading and some storytelling, was a brief discussion about Engine Yard. Believe me, as I write this, I find that every bit as weird as I suspect you do, yet it’s proof that truth is stranger than fiction. :-)

One of the points that we frequently visited was how many customers Engine Yard had. John loved to ask the question, because it always made me happy: Engine Yard grew very, very quickly, which made this a very happy subject for me, and by extension, for him as well.

I remember how surprised he was when we reached 100 customers. As a child, it seemed an impossibly large number! As he grew our routine changed. But once in a while he still asks. Knowing his reaction at 100, I should have predicted his surprise when I recently answered that we now have over 2,000 customers! After all, why shouldn’t he be surprised? It amazes me that we’re doubling our customer base yearly and growing monthly recurring revenue 125% per year!

While the raw growth makes me happy, our firm footing is even more exciting. Development pace is swift: we deliver product improvements more than once a day!

We were founded on the premise that Ruby on Rails deserved a world-class deployment environment. Clearly the community agrees! With last year’s migration away from our legacy hardware infrastructure to cloud infrastructure behind us, we are now 100% focused on the cloud based deployment needs of our customers, which will allow them to continue to focus on their applications, instead of their infrastructure, stack and 24x7 operations.

Today, I’m pleased to announce three important additions to our offering. These features are now available in Beta and will be fully rolled out over the next month: **See update above

• Collaboration allows users to easily switch between multiple accounts. This is a big deal for our 200+ development partners who regularly access their customer accounts, but any customer with more than one account and/or more than one person deploying will enjoy the benefits of this feature.
• Multi-Region allows applications to be deployed in the Amazon EC2 region of choice, delivering on our desire to provide low-latency application performance to the worldwide Rails community. This feature was also particularly well-timed, as it allowed our customers to re-deploy their applications into an alternate EC2 region during Amazon’s recent US-East service interruption.
• All customers will also soon have a basic level of support which includes the ability to interact with our support team by submitting tickets to the same system that paid support customers use. This will quicken response time and issue resolution. As CTO, the technical features are a bulls-eye in sphere of interest. But as a co-founder, I know that each and every customer now has enough support to succeed, which is most important of all.

It’s difficult to express how proud I am of the Engine Yard team or how confident I am that I’ll again be able to surprise John next time he asks how many customers Engine Yard has! Surprising and delighting our children is, after all, one of life’s greatest pleasures.

I was immediately struck by that staggering number, but not in an awe-inspired "Wow, that platform is amazing!" sort of way, but instead in a "They must be joking!" sort of way. :-)

Big numbers get attention, particularly at Salesforce. I can understand why the Force team feels the need to put up some big numbers: last year's acquisition of Ruby on Rails PaaS startup Heroku for an extremely high multiple of revenue has to have shaken the Force.com team's confidence. The deal delivered some very clear messages:

• Force was not performing to expectations
• Salesforce felt the need to plot a new future
• Large, meaningless numbers mesmerize the Salesforce management

It seems as though the Force.com team decided to join in with it's own large but meaningless number!

Reading the blog post and applying some simple arithmetic begs a few questions:

• 8 million tests. That sounds impressive, but...
• 125 LOC/test is quite low by modern standards.
• Do the BILLION lines of APEX code represent just customer applications?
• With more than a MILLION lines per day average (2 million a day now, assuming linear growth), why buy Heroku?

Clearly, something is fishy.

Given that demand for APEX programmers is about 1/3 that of Ruby developers, you'd think that there would be three times as much code being written in Ruby as APEX. Yet I'm confident that Ruby developers are writing a lot less code and getting a lot more done. That's what I call a win/win situation! :-)

Let me state this clearly: lines of code is a generally meaningless metric. Given that, few will argue that higher line counts are better than lower! You don't have to take my word on this: if you want to see into the future, check out the incredible work Alan Kay and his colleagues are doing at Viewpoints Research Institute by reading their 2010 summary. They're working on building an entirely new operating system, including productivity software, in just a few tens of thousands of lines of code!

One of the undeniable truths of publishing a competitive metric is that it needs to be unassailable. After all, once you choose to bloviate about a billion lines of code, it'd be highly embarrassing if a competitor quickly bested the number. With this in mind, I suspect that Force.com made a very wise choice in making lines of code their number. Who else is going to compete against a metric which, rather than being heralded, better belongs in the development hall of shame?

P.S. I also found it highly amusing that the press release states that APEX is a language for the cloud since it's multi-tenanted, and there are now just 16 ways (down from 70!) that your application can be "governor throttled." That is one of the most ridiculous bits of spin I've ever heard! I've already commented on SFDC's fascination with multi-tenancy; this is yet another reason that multi-tenant environments are not well suited to mission critical applications.

Engine Yard is helping for the simplest of reasons: Japan has experienced a terrible natural disaster and the people there need help.

On another level, Ruby on Rails was built on Ruby, a fundamental technology that originated in Japan. We have been committed to Ruby on Rails for over five years, owe a debt of gratitude for that alone.

But the most important reason is deeply personal. Over the years we've met, befriended, and appreciated our Japanese friends for their talent, dedication, skill, integrity and honor through our interactions with them at Ruby conferences and events throughout the world. This makes them family, and we believe family is most important of all. So we're doing what we can and thinking of them constantly as they make their way through this terrible tragedy.

For these reasons Engine Yard is honored to donate and match employee donations before March 30, 2011.

We encourage you and your team to contribute too. Please donate if you're able, and help direct people to http://www.rubyistsforjapan.com/.

We even share a CEO! Our CEO, John Dillon, was employee #10 at Salesforce.com and helped build and grow their business model.

When Salesforce.com’s purchase of Heroku was announced, I had yet another reason to pay close attention! Suddenly Salesforce and Engine Yard were much closer competitors than we previously had been.

Yet with all these good reasons to pay attention to what was happening, I kept getting distracted by the use of the term “Cloud 2.” Surely a major version upgrade to the Cloud must be accompanied by a new and earth shaking vision, something that I must pay close attention to or risk hitting the proverbial iceberg! Right?

Let’s face it: one of the most irritating things in cloud computing is the never quite satisfactorily answered question “What is Cloud Computing?” I believe the answer to that question is that Cloud Computing is the set of new business models enabled by the Internet. After all, since the earliest days of the Internet everyone has drawn clouds on whiteboards to represent the Internet itself.

Common to all of these new business models is the fundamental shift in realization that once every computing system can communicate with every other, there’s less of a need to be concerned with physical concerns such as hardware locality. To me, that is without a doubt THE big idea of Cloud Computing.

Given that, how does Cloud 2 differ from the Cloud? Frankly, I don’t see anything revolutionary at all. I suppose we’ll just have to pass it off as typical enterprise marketing hype. Does Salesforce.com honestly believe that Cloud is already passé, and that we’re on the brink of the next big thing already?

There’s one idea that we can clearly agree upon: Ruby on Rails is the future.

Salesforce.com is preaching to the choir on this. We’ve been clear for 5 years that Ruby is the next big language. It’s fully open source and freely licensed and will never lock in a customer.

Yet there are two ideas that we severely disagree with:

1) OS multi-tenancy is NOT the ideal architecture for a PaaS

Multi-tenancy within a SaaS application makes a lot of sense. It’s very efficient and convenient to have a single code base and application instance to manage. Contrary to popular belief, however, what’s good for the goose is not necessarily good for the gander.

OS multi-tenancy in a PaaS doesn’t make sense except for the least important applications.

Efficient virtualization is allowing IaaS vendors to deliver astonishingly efficient IaaS offerings. With security and quality of service constantly ranking at the top of cloud adoption concerns, it’s clear that for important applications, virtualized single tenancy is the way forward.

Think of it from a belt-and-suspenders mindset: Security and quality of service are too important to not take advantage of pervasive and inexpensive technology to help ensure that both are maintained at all times.

2) It’s a BAD idea to store data in a proprietary data store

While Salesforce.com’s sudden interest in Ruby on Rails framework might make it appear that they have abandoned the obvious lock-in tactics of Force.com’s Apex, database.com makes it perfectly clear that lock-in is still the name of their game.

Why in the world would anyone choose to store their data in a proprietary service when fantastic open source SQL databases and key/value datastores exist? What really surprises me about this is that Salesforce.com should understand this implicitly!

I’m confident they would be delighted to avoid their gigantic Oracle license fees, and I’m certain their users would be delighted if the cost of Salesforce.com products were substantially lower.

I’m confident that customers will prefer to downgrade to old school cloud computing rather than rushing into the digital quicksand and marketing hype that Cloud 2 appears to embody.

Tammer Saleh joined us 3 weeks ago as Director of Application Development. Most recently Tammer operated his own consultancy practice, and he is a well known and respected member of the Ruby community. He's already identified a number of quick wins that will continue the rapid fire development of AppCloud. While the AppCloud team has been absolutely killing it, I have confidence that Tammer's skills and techniques will further press the pace.

As we continue to grow, we felt the time was right to reaffirm our commitment to open source.  We don't want newcomers to the community to think we're a mere commercial entity, as opposed to the open source symbiote that long time members know us to be! Today I'm announcing two hires that will, I believe, make our commitment abundantly clear.

Dr. Nic Williams will be arriving from Australia to take the role of VP of Technology. His primary responsibility will be to organize and guide Engine Yard's open source efforts. He has already blogged about his pending move; perhaps I should have left off the Mark Twain quote? Thank you for your sacrifice, Mrs. Dr. Nic! Hopefully you and my wife, Elizabeth, will become fast friends! I find San Francisco to be a friendly and wonderful place to live and suspect you will too! :-)

Roger Levy will be joining us later in the month to oversee engineering, support and product management in his role as SVP of Products. Roger, who managed the SUSE Linux business at Novell, certainly has the open source experience and credentials to continue to reinforce Engine Yard's commitment to open source.

Finally, we've also added Sara Gardner as VP of Marketing and Steve Gross as VP of Business Development. There are so many things to inform the community about, and so many great companies to partner with, that Sara and Steve are already busy! I welcome them to Engine Yard and anxiously anticipate their unique contributions.

Startups that grow quickly place a LOT of stress on their founders and early employees. Many founders thrive on this, I know that Lance and I did. Finally, as a company grows, priorities and roles change. I won't argue with those who say that staying small is beautiful, as I agree with much of what has been said on that subject. We, however, chose a different path: go big or go home! :-)

Perhaps, then, it should not be a surprise that Ezra and his family have decided to move to Portland to be closer to family. I wish my good friend a very fond farewell, and a reduction of the stress that he has endured along with the rest of the founders and early employees of Engine Yard.

I cannot express how exciting and fulfilling it has been to steer Engine Yard over the last 4 years, from an early advocate of Ruby on Rails and its community, to the force that it is today. As the last remaining founder, I must admit that I'm very proud of what we have achieved during that time, both at Engine Yard and beyond! Lance Walley is now CEO of Chargify, an uber-cool recurring billing service. Jayson Vantuyl has created a successful consulting business and is up to something sneaky as well! And while only time will tell what Ezra shall choose to pursue next, I'm certain that additional success awaits him.

Finally, I'd like to close with something that I wish I could say every time I open my mouth: Thank you to our nearly 1,500 customers and all of my hard working, talented and dedicated employees. Perhaps the highest praise one person can offer another is "you make my dreams possible" and for making mine possible, I'll forever be grateful to each and every one of you. :-)

For Engine Yard Cloud (Amazon Web Services) customers, this move will have no impact on you whatsoever.

When we opened for business more than three years ago, racking and stacking our own hardware wasn’t really a choice: being self-funded well before the concept of cloud computing existed, doing it ourselves was the only way we could introduce our customers to our vision for application deployment and management.

How times have changed! Infrastructure vendors now agree with most of the concepts that we pioneered back then, eliminating the need for us to do it ourselves. We’ve always felt that specializing in Ruby on Rails and the surrounding stack would allow us to make deploying and scaling Rails applications as easy and efficient as it is to create those applications.

Today’s announcement will allow us to further focus on enabling our customers to leverage today’s and tomorrow’s rapidly evolving infrastructure and providing the best Rails Platform-as-a-Service technologies and support.

While there are many advantages to the Terremark infrastructure, we’re most excited about their  sophisticated fibre-channel storage area network. The Terremark SAN affords greater reliability and substantially higher throughput than our current storage system; we know that our customers will see great benefit and peace of mind from this.

Terremark has an excellent track record supporting the needs of large enterprise and federal government agencies. Their datacenters have SAS 70 level II, PCI and HIPAA certifications, and we’re confident that our private cloud customers will find this new infrastructure meets the most demanding application requirements.

Over the next six months, we will migrate all current Slice, Fractional Cluster and Dedicated Cluster environments that currently reside on the Engine Yard private cloud to Terremark.

At a high level, not much will change for our private cloud customers. In particular, I want to emphasize that there are no changes in your support team or support processes.

Based on our extensive planning with Terremark, we expect migrations to require minimal effort for our private cloud customers.

If you’re a private cloud customer, you will hear from your Engine Yard account manager in the next few weeks to discuss a migration plan that makes sense for you.

The model for the directory was that any firm could add an account, and once they received three customer endorsements, they would be checked as "confirmed" and prioritized in search results. Firms could also add up to three "showcase" projects so that customers could click through and see the quality of work. The goal was to let the work (and the customer endorsements) speak for themselves.

So far, the directory has been pretty successful. It now has about 250 development firms listed. Although we built in a "request for work" submission feature, it's mostly being used by potential development customers as a way to research firms, rather than as a way to submit work requests, but we have high hopes here. So far, we've had a few problems with spam accounts and bogus customer endorsements but not many. As far as development details, the site was built from scratch in Ruby on Rails in 3 weeks, then added 3 weeks of polish before launch.

There are 272 customer endorsements for member work, and as you'll see from the code, currently, the formula used to return search results is pretty basic:

1. first it includes companies with matching or lower minimum budgets
2. then it includes companies with matching locations (by state in the US and by country everywhere else - this filter is optional)
3. then it orders by whether a company is confirmed (has 3 endorsements)
4. then it orders by how many endorsements a company has
5. then it returns results randomly.

If your company profile is confirmed and you have lots of endorsements, all else being equal, you will rank higher in the search results. But in our experience, matching by budget and location is very important, as it is a qualifying step for most potential customers.

Continuing our support of open source projects and initiatives, and after discussing the directory's future with some of the community, we felt it made sense to release this directory application as open source, and today we're officially releasing the source code of the application in a generalized form that can quickly and easily be customized and put into use by anyone that needs a directory site.

You can find the Rails Directory source code released under the MIT License on GitHub at http://github.com/engineyard/rails_dev_directory.

Feel free to fork the repo and review it, play with it, or modify it as you see fit. If you're interested in contributing to the source code, or have any feedback, drop us a line at info@railsdevelopment.com.

Credit for developing the application in large part goes to Paul Campbell, Nick French and Ben De Jesus here at Engine Yard.

Rails and Deployment

Yehuda Katz talked about the challenges of refactoring Rails, (in two posts) and then about his five favorite things about Rails 3. His vote: a new bundler, a new ActionController architecture, the new Responder object, ActiveModel (which brings ORM pluggability) and much better performance. Carl Lerche also chipped in with some tips for how to boost Rails performance. Top tips: eager loading, leveraging the database and keeping the request cycle simple.

One of the most popular posts of the summer was Sudara Williams take on app server bloat, and bad ActiveRecord behavior that can get you in trouble. One interesting point was that looking for Rails or Ruby memory leaks as the source of big memory consumption jumps is usually the very last thing you should worry about. There were some great comments and extra tips in the comments, so be sure to read them all.

Taylor Weibley blogged about managing background jobs in Rails. The big takeaway, benchmark your workload and track completion and failure rates like a hawk. And Greg Nokes talked about sharding for Rails -- to be avoided as long as you can because it complicates life greatly, but is eventually inevitable.

All Things Ruby

Charlie Nutter kicked off our JRuby series with a brief introduction, and then wrote about how to script Java libraries using JRuby, and built a MIDI keyboard app to show it in action. Tom Enebo then went over how to bring your favorite Ruby features to Java APIs when you're accessing those libraries from JRuby, and covered decoration, delegation, blocks. Brian Ford kicked off the summer with a tour of RubySpec and later went through the mechanics of a Ruby compile from text to bytecode. Stay tuned for the next post on how a Ruby compiler goes from bytecode to machine code.

Key Value Stores & Search

Kirk Haines kicked off his series on key-value stores for Ruby with an introduction to this class of non-relational stores, and why you might look at them as an alternative to the tried and true relational database.

First up was an overview of Tokyo Cabinet, a family of key-value stores created by mixi -- the Facebook of Japan. Tokyo Cabinet is more than just simple key-value stores though, it also offers b-tree storage and fixed length array storage. In Kirk's evaluation, it looks super-fast as well as nice and mature.

Next up was Cassandra, an eventually consistent, fully distributed, highly scalable data store created by the Facebook crew. Cassandra's model is a little bit of twist on a simple key value store, but it can really be the right solution if you think you're going to need Amazon or Facebook like datastore scaling at a compelling cost. Finally, Kirk did an introduction to Redis, great if you need speed, persistence and you can tolerate some data loss. Stay tuned for more entries in Kirk's list. Next up: MongoDB!

Another interesting post was Avrohom Katz's post explaining five ways to improve your use of Sphinx text-search.

Development Tools and Testing

Dave Astels wrote an excellent series on Cucumber, from introductory to intermediate users and on to expert tips. And Joe Arnold talked about the pair-programming approach at Engine Yard.

And stay tuned for more great technology posts for the fall!

Why did we feel so strongly? I'd like to explain this on the record as many of our customers will no doubt wonder "Wow, that was a pretty good deal?" :-)

My number one reason was that I felt that distributed version control was critical to keeping the open source community centered around the Ruby on Rails cutting edge. The social implications of Git's fully distributed nature ensure that open source developers must listen to their community or risk losing control of their project to a fork. This is why we moved Rubinius and Merb to Git so early in the game!

When Rails itself announced the move to GitHub, we were overjoyed and entirely supportive. We then moved GitHub from a small slice environment to a larger cluster as we felt it critically important that GitHub scale smoothly with the substantial increase in traffic that the Rails source code would bring. That transition went so smoothly that an impromptu party developed to celebrate its success; it was quite an evening!

Shortly thereafter, as often happens when a site reaches a certain level of success, things began to slow down. We identified the bottlenecks and supported GitHub and the community by making patches to ssh to allow key lookup in MySQL rather than a text file. That remains, to this day, one of the finest examples of Engine Yard support and it makes me extremely proud just thinking of it.

The Ruby and Rails communities have wholeheartedly agreed with us and endorsed GitHub. In fact, the transition from Subversion to Git has been one of the most aggressive technology transitions that I've ever seen! This made the ever increasing resources required to deliver GitHub a fair trade for Engine Yard for a number of reasons:

1. More GitHub traffic meant more exposure for Engine Yard
2. Git allowed the Ruby community to be more agile, and more GitHub adoption meant a more robust Ruby community

Recently, however, GitHub has begun to grow very rapidly in non-Ruby communities. This is wonderful for GitHub, but made adding new resources on a free basis far less attractive to Engine Yard. GitHub offers the largest free storage quota among the big SCM hosters, and we came to the conclusion that we didn't want to subsidize that quota for non-Ruby developers.

We discussed options extensively with GitHub and made our best attempt to offer an easy ramp from free to paid. However, in the end, the best arrangement for them was moving.

We use GitHub each and every day for every project that we support. We worked incredibly hard for them since the beginning, and placed a substantial amount of resources at their disposal, in hardware, bandwidth, and support. Like all partnerships, we had our occasional disagreements, but I believe there were never hard feelings on either side. Accordingly, we are currently working directly with their next provider to maintain smooth service as they cut over.

For Private Cloud customers, nothing will change: our classic, fractional and dedicated offerings will continue to include a source code management account (Github or Beanstalk) as part of their service. For existing Engine Yard Cloud customers, we will continue to provide free Github through the end of this year. For new Engine Yard Cloud customers, there will no longer be a free Github account included in the service.

We’re confident that GitHub will continue its tremendous success in enabling the kind of collaboration that forms the heart of the open source software movement. We wish them the best and will continue to think of them as an important part of our future.