What is Rack?

Rack defines a standard for interaction between web servers and Ruby web applications. It's based on the CGI standard, but stripped of CGI's global characteristics (such as its use of environment variables and standard output).

Servers send requests to Rack applications by calling their #call method with an environment Hash containing information as a number of standardized keys. For instance, it provides the requested path in the PATH_INFO variable. Applications send responses back to the server as an Array of three elements: the status code, a Hash of the headers, and the body. Applications may return any object that responds to #each as the body, and each iteration must return a String object.

Here are a couple of simple Rack applications:

# A Proc responds to #call
proc do |env|
  [200, {"Content-Type" => "text/html"}, ["Hello world"]]
end

# Other objects that respond to #call work too
class MyApp
  def self.call(env)
    [
      200,
      {"Content-Type" => "text/html"},
      ["Hello world"]
    ]
  end
end

Rails 2.3

In Rails 2.3, Rails became a fully valid Rack application, leveraging Rack in a number of places:

• ActionController::Dispatcher.new is the top-level Rack application
• The parameter parser is implemented as middleware
• The router is a Rack application that dispatches to controllers
• Each controller is a Rack application

This was a modest step toward leveraging Rack, and made Rails applications work anywhere Rack worked. However, the internals of some of these pieces were messy, and an overhaul would be needed to take full advantage of the Rack ecosystem and the power of Rack's architecture.

Enter Rails 3: The Application

First, rather than having a global Dispatcher object pointing at a global router, Rails 3 introduces the idea of the Rails Application. The Application, an object holding the router and other configuration, is itself a Rack application.

The following is a valid config.ru.

# Set up load paths as desired

require "action_controller/railtie"

class FooController < ActionController::Base
  def bar
    self.response_body = "HELLO"
  end
end

class MyApp < Rails::Application
   config.session_store :disabled

   routes.draw do
     match "/foo" => "foo#bar"
  end
end

run MyApp

As you can see, a Rails application is a valid Rack application. The above config.ru file will run with rackup, Passenger, Glassfish, or any other Rack server.

ActionDispatch and Rack Middleware

Rails 3 introduces a new framework called ActionDispatch, a library extending the features of the Rack library for Rails' use. This library can also be used standalone in any Rack application.

The new ActionDispatch::Request and ActionDispatch::Response objects now inherit from Rack::Request and Rack::Response. Integration tests now use Rack::Test and can now be used to test any Rack application.

Rails now includes a number of new general purpose middlewares:

• A middleware to run preparation callbacks, which Rails runs once in production mode, or once per request in development mode
• A middleware to set up and write cookies
• A middleware to clean up flash notices
• A middleware to handle HEAD requests
• A middleware to handle IP spoof checking
• A middleware to serve static assets
• Middlewares to handle rescuing exceptions down the stack
• Middlewares for various session stores

Rails also uses a number of middlewares that ship with the Rack library:

• A middleware to synchronize non-threadsafe requests
• A middleware to measure and set the runtime of requests
• A middleware to implement Send-File semantics in various web servers
• A middleware to handle PUT and DELETE requests coming in via POST requests

The middleware stack object, available as config.middleware, supports reordering these middlewares, or inserting new middleware relative to default ones.

One nice thing about implementing the functionality as middleware is that it forces them to be simple, standalone objects, and understanding, patching or even replacing them becomes much easier than trying to modify a monolithic controller or request object.

The Router

The Rails 3 router was written from the ground up by Josh Peek as the Rack::Mount project, which provides a general-purpose Rack router. Rack::Mount recognizes URLs and dispatches them to any Rack application. As a result, the router now natively supports matching any URL to any arbitrary Rack application--even a Sintra application.

Rails.application.routes.draw do
  match "/blog" => Rack::Blog # Someone should write it!
end

The router is itself a Rack application, so it's possible to mount a router inside the main router. Carl Lerche's Astaire project uses this to provide the Sinatra routing DSL inside a Rails controller. The following is a valid config.ru:

$:.unshift File.expand_path('../../lib', __FILE__)

require 'rubygems'
require 'astaire'

class OmgController < ActionController::Base
  include Astaire::DSL
  append_view_paths File.expand_path('../views', __FILE__)

  get "/goodbye" do
    render :text => "goodbye"
  end

  get "/hello" do
    render "hello"
  end
end

run OmgController

Astaire uses ActionDispatch::Routing::RouteSet internally, a Rails wrapper around Rack::Mount.

Actions

Because the router dispatches directly to Rack applications, each controller action is its own mini-application. To get a valid Rack application for a controller's action, use the ArticlesController.action(:index) syntax.

The Rails router uses the syntax match "/articles" => "articles#index" to match a URL to a controller/action pair. That syntax is identical to match "/articles" => ArticlesController.action(:index).

Because actions are their own Rack applications, you can add middleware between the router and controller actions:

class ArticlesController < ApplicationController
  use MyMiddleware
end

This will insert the middleware between the router and the dispatching of an action. The following is a valid config.ru:

class ArticlesController < ActionController::Base
  append_view_path "/path/to/views"

  def index
    render
  end

  # implicit actions, such as "articles/show" would work here
end

run ArticlesController.action(:index)

This endpoint would behave exactly like a normal action, including callbacks and rendering. In fact, it's roughly identical to what happens internally inside the router.

You can also call into an action using this syntax:

env = Rack::MockRequest.env_for("/articles")
status, headers, body = ArticlesController.action(:index).call(env)

# Create a response object, if you want one
response = ActionDispatch::Response.new(status, headers, body)

You could also use Rack::Test:

class TestArticles < Test::Unit::TestCase
  include Rack::Test::Methods

  def app
    ArticlesController.action(:index)
  end

  def test_get
    get "/articles"
    assert_equal "Hello world", last_response.body
  end
end

# This would include the application's middleware,
# which you probably want as it includes sessions
# and cookies
class TestApplication < Test::Unit::TestCase
  include Rack::Test::Methods

  def app
    MyApplication
  end

  def test_get
    get "/articles"
    assert_equal "Hello world", last_response.body
  end
end

You would probably use Rails integration tests, which wrap Rack::Test with some additional conveniences, in practice. It's nice to know that there's very little magic going on, and that all of the individual pieces of Rails are just standard Rack.

Conclusion

While Rails 2 had started down the path of supporting Rack, Rails 3 took a few dozen steps forward, deeply integrating the framework with the Rack standard. In addition to improving the general interaction with servers, Rails 3 takes much more advantage of the Rack library, exposes its own functionality as reusable Rack components, and wires up the stack using the Rack architecture. This means that Rails itself integrates well with tools that work with Rack applications, and can also incorporate other applications or middleware following the Rack standard.

Stay tuned for more Rails 3 content to come—the series may be over, but there's plenty left to talk about.

Today I'm going to talk about the features we added to Rails 3 to provide ORM agnosticism. When we first started, the idea of such agnosticism was pretty fuzzy. As we approached the beta, we became convinced that we would shoot for the moon and give DataMapper, Sequel and other ORMs first-class access to all of the same parts of the framework that ActiveRecord had. Again, this sounds pretty fuzzy, so let me lay it out for you.

ActiveRecord is a Framework Extension

You read that right. While Rails itself ships with ActiveRecord as a default, the Railties gem, responsible for bootstrapping Rails, knows nothing about ActiveRecord. In order to achieve this, we had to make that bootstrapping process much more pluggable. While this is amazing for extensions, like DataMapper, that want to replace a built-in framework, it also exposes more functionality to any Rails extension distributed as a gem.

Rails::Railtie

In short, Railties now coordinates the initialization process, which involves the activation of a number of individual Railtie subclasses. Cute, no? Every Rails framework has its own Railtie subclass, which provides a number of useful pieces of functionality:

• The ability to add a key (like config.active_record) to an Application's configuration, and assign it defaults that will exist before Application configuration. For instance, a plugin could use this functionality to set a configuration key as an Array, so the Application could simply do config.action_view.view_paths << "my_path". While this isn't exactly earth-shattering, it levels the playing field between Rails components, like ActiveRecord, and third-party extensions, like DataMapper or Haml
• The ability to create additional generators that hook into Rails' default generators. For instance, Rails' scaffold generator invokes sub-generators for stylesheets, template engine, test framework, helpers, and ORM. For instance, when a user installs Haml as a gem, it can register itself as the handler for template engine generators, and provide replacements for each case where Rails provides a default implementation. In this case, Haml would replace the template engine generators provide by ActionController's Railtie
• The ability to supply Rake tasks to load when the user invokes the Rake command. If DataMapper supplies the same named Rake tasks as those supplied by the ActiveRecord Railtie, the user can completely remove ActiveRecord and replace it with DataMapper, retaining all of the named Rake tasks. This would allow other tasks, such as those used in testing, to automatically prepare the database and other tasks it performs by invoking tasks with certain names (like db:create)
• Supply a log subscriber to integrate seamlessly with the uniform request logging. This allows extensions like DataMapper to add their timings into the log output. In this case, DataMapper simply replaces the "Model" timing that ActiveRecord provides, allowing a level of very tight integration with the rest of the framework
• Specify initializers that should run at specific points in the initialization process. This allows extensions to set things up very early in the boot process, but then defer some setup until after the user has configured their application, or after specific parts of the initialization process have occurred. Each initializer also receives an instance of the Application object, giving it full access to the user's configuration

In fact, if you take a look at ActiveRecord's deep integration with the rest of the framework (which is roughly equivalent to its highly coupled implementation in Rails 2.3), you'll find that it looks extremely similar to DataMapper's equally deep integration. In fact, a major goal of the work we put into improving modularity in Rails 3 was to maintain the same level of stack integration we've historically had, while exposing the same toolchain for those who wanted to replace only certain elements of the stack.

The ActiveModel API

Giving DataMapper access to the same level of integration with Railties as ActiveRecord is one of two major pillars in making Rails truly ORM agnostic. The second part is decoupling Rails' historic connection between ActiveRecord, its ORM, and ActionPack, its controller and view layer. In this area, Rails takes a holistic, very conventional approach to linking the model and the view using REST principles.

In particular, domain objects, persisted via the ORM, can have canonical URLs that are used pervasively throughout the controller and view layer. The specific URLs can be configured via the router, and Rails 3's router is particularly powerful, but once configured, objects do have canonical URLs.

@post = Post.first

redirect_to @post           #=> Location: /posts/orm-agnosticism
redirect_to Post            #=> Location: /posts

form_for @post              #=>

form_for Post.new           #=>

error_messages_for @post    #=> a representation of the validation errors that exist on the object

link_to @post.title, @post  #=> 
                            #     Rails and Merb Merge: ORM Agnosticism (Part 5 of 6)
                            #   

posts_path                  #=> "/posts"
post_path(@post)            #=> "/posts/orm-agnosticism"

@posts = Post.where(:author => "wycats")

render @post                #=> renders "_post.html.erb", passing in @post as a local
render @posts               #=> renders a collection of "_post.html.erb", passing in each
                            #   Post instance as a local in turn

respond_with @post          #=> performs content negotiation between the formats
                            #   the client is willing to Accept (xml, json, html),
                            #   and the formats the @post object is willing to
                            #   provide. The available providable formats are
                            #   transparently determined by introspecting the @post
                            #   object. Additionally, if the object has not been persisted,
                            #   redirect it back to the editing page for further modification.
                            #   If it has been persisted, redirect it back to the index (or
                            #   some other appropriate path).

In short, Rails 2.3 provided a fair bit of integrated functionality between ActiveRecord and ActionPack. For instance, to determine what an object's canonical URL was, Rails called an internal .class.naming method on it, which provided the information needed to form the canonical URLs. And while other ORMs could attempt to emulate these internals, there was no guarantee that they'd be used in the same way from release to release (and, in fact, they evolved quite a bit, making them a moving target at best).

In Rails 3, we opened them up, exposing an explicit public API via ActiveModel. The ActiveModel API requires that objects respond to to_model, returning an Object that complies with the larger ActiveModel API. This allows objects that already implement methods with the same names as those required by the API to create a facade that presents itself to ActionPack as a fully compliant model.

The API itself is fairly small, with a few methods around validation, the ability to determine whether an object has persisted or not (which can be safely stubbed by objects without persistence), and a number of methods that tell ActionPack how to convert the object into a canonical URL or template name. By doing this, Rails has completely decoupled ActionPack from ActiveRecord directly, and ActiveRecord becomes just one of many ORMs to implement the ActiveModel API.

The DataMapper ORM has already released a Rails extension (called dm-rails) that implements both the Railtie functionality and ActiveModel, making it a full drop-in replacement for ActiveRecord. The Sequel ORM is also hard at work on their Railtie and ActiveModel adapter.

The really great thing about all of this is that we didn't need to sacrifice one bit of the (some have said overly) tight integration between various parts of Rails, to provide the ability to swap in entirely different components with the same apparent "coupling" that the Rails frameworks have with each other.

As always, questions and comments are welcome!

First of all, you're probably sick of hearing this, but we've done far, far more than we ever expected. A lot of that happened in the last few weeks.

One of the things that most surprised and impressed me is the Rails core team's (and especially DHH's) attention to detail and the experience of the beta release. For weeks, we've been "this close" to releasing, but the experience of starting up a new Rails app or upgrading from a Rails 2.3 app still felt too unpleasant. In this kind of situation, it's tempting to say "it's just a beta—people who use beta software know what they're getting," but that would have been a major cop-out. For many people on the leading edge, a poor beta experience will shape their perception of the product as a whole. So we waited a bit, but now we're finally here.

In addition to a surprising (in a good way) amount of polish, we touched nearly every part of Rails: I'll walk you through the top-level blocks.

Dependencies

Rails gets a bunch of new dependencies, stepping out of the "virtually zero dependencies" box it has lived in since the beginning. In order to achieve this, we wrote a brand new dependency management system (called Bundler) from scratch, remedying many of the problems with Rubygems that have made using a number of dependencies difficult (and which we encountered in Merb when taking a more-dependencies approach). Taking another step into the future, we built Bundler itself as a standalone component, so you can use it for Rails, Merb, Sinatra, or even a plain old Ruby script.

And even though the Rails happy path uses Bundler to manage dependencies, that happy path is limited to the generated code only. The Rails library itself does not rely on Rubygems, Bundler, or any package manager. This makes using a different package manager, like Debian or rip, as simple as changing a few lines of the generated code.

Instrumentation

A number of companies and projects have sprung up over the years attempting to provide useful information about the runtime status of a Rails process. In order to facilitate this work going forward, we've baked instrumentation into Rails itself. Instead of having to monkey-patch an appropriate method to learn about which templates Rails renders, for instance, simply listen for an event. While Rails has never officially supported the monkey-patching approach, we will commit to supporting instrumentation events in the same way as we support any other public APIs, making them future-proof stable ways to learn about what's happening inside of Rails.

Better yet, Rails' logging system uses these very events to emit information into the log!

Every Framework is an Extension

In what is possibly the biggest conceptual shift in Rails 3, we've exposed an extension API that allowed us to make ActiveRecord, ActionController, and ActionMailer simple Rails extensions. In other words, the DataMapper ORM can take a look at the part of ActiveRecord that implements its Rails extension, implement similar features, and know for sure that it'll work. Additionally, users can completely remove the ActiveRecord gem, and be rid of the generators, rake tasks and other peripheral elements. An example: if DataMapper implements a db:test:prepare rake task, a Rails developer can replace ActiveRecord with DataMapper, and the test:units rake task will use DataMapper's new tasks.

In short, we have completely removed references to other frameworks from Railties itself, making the frameworks of Rails true extensions to Rails itself. This makes it possible to replace any part of Rails without having to worry about entanglements with Railties.

Rack

Rails 2.3 took a very firm stance in favor of Rack. Rails 3 completes the story. In Rails 3, the Application itself, the router, and controller actions themselves are all Rack applications. Rails ships with a standard config.ru, and can boot using the rackup tool that ships with Rack, or with Passenger or Glassfish in standard Rack mode. In fact, any Rails host or server that supports the Rack protocol can run a Rails 3 application unaltered.

Because so many parts of Rails are Rack, you can also insert standard Rack middleware in config.ru, in your Rails application, in the router, or even in a controller's actions. Rails itself ships with a number of standard middlewares, for exception handling, cookies, parameter parsing, flash notifications, sessions and a bunch more. You can use any of these middlewares in standard Rack applications (without Rails).

ActionController

ActionController got a major overhaul, dividing up a monolithic component into multiple pieces.

ActionDispatch

You can think of ActionDispatch as rack-ext. Essentially, we packaged up all of the parts of ActionController involved in the Rack dispatching process into a single reusable component. This includes ActionDispatch::Request, ActionDispatch::Response, our mime type handling, a bunch of middleware, and integration testing support.

It also includes a wrapper around rack-mount a new powerful routing library. Josh Peek, a member of the Rails Core Team, wrote rack-mount as a standalone component, so that it could be used in standard Rack projects without any Rails dependencies. By default, it simply dispatches inbound requests to Rack endpoints.

Because Rails 3 controller actions are Rack endpoints, Josh could easily wire up a thin shim around rack-mount to dispatch inbound requests to Rails controller actions. But because we built the Rails router on top of rack-mount, Rails applications can route inbound requests to any Rack application, like Sinatra or Cloudkit.

AbstractController

The disparity between ActionController and ActionMailer has long been a personal pet peeve of mine. For Rails 3, we refactored ActionController and ActionMailer to inherit from a common superclass: AbstractController. AbstractController includes the basic concept of controllers, rendering, layouts, helpers, and callbacks, but nothing about HTTP or mail delivery. This means that ActionMailer can no longer drift away from ActionController, leaving us with two distant cousin APIs.

ActionController

ActionController got a big overhaul, completely eliminating alias_method_chain in favor of a module-based system that extension author can more easily extend. For instance, we exposed a Renderer interface, allowing extension authors to easily add render :pdf or render :csv without having to override the render method in brittle ways.

While the API did not change much in general, we did add a few new features, such as the ability to add middleware directly to a controller's actions or the ability to wrap up common idioms more cleanly using responders. Additionally, we improved the overall architecture to reduce the need for certain hacks.

For instance, ActionController will now look up the layout for an action using the details (such as format and locale) that its template used. In Rails 2.3, an RJS template could end up using an HTML layout, requiring us to exempt RJS templates from layouts altogether. However, this was a band-aid (what if you wanted an RJS layout?) and the improved architecture eliminates the possibility for other, similar problems.

ActionView

ActionView got a significant overhaul, improving its internal architecture, increasing performance, adding pervasive XSS protection, and making all JavaScript helpers unobtrusive.

In terms of architecture, Rails 3 defines a clear boundary between ActionController and ActionView. Now, ActionView exposes a single API entry point for rendering templates and partials, making the code path easier to understand, eliminating a number of latent (but unusual) bugs, and making it easier to reason about caching lookups for performance.

When doing a performance audit of Rails applications, ActiveRecord and ActionView (specifically, template rendering) pop up as major costs. In particular, ActionView has had a fair bit of overhead that's impossible to avoid without throwing certain features out the window. For instance, partials and especially collections of partials had enough overhead to become noticeable on profiles of real applications.

In Rails 3, we optimized both the template lookup logic (which should now take up extremely small amounts of time) and the logic inside the loop of partial collections. In particular, partial collections should often be much faster in Rails 3 than a manual loop, and not much slower than manually inlining and unrolling the loop. Finally, the improved architecture has made additional optimizations possible, so expect further performance improvements in Rails 3.1.

We also switched to Erubis, a fully compatible ERB implementation with much better extensibility characteristics. In particular, it allows us to easily hook in the code that it generates for any ERB construct. This will allow compile-time performance optimizations in the future and enables the next feature: pervasive XSS protection.

Rails has always shipped with a simple way to escape any user-generated content. Unfortunately, this required constant vigilance. A number of highly visible, successful attacks against websites on a number of platforms taught the world that successful mitigation of this threat requires treating all Strings as suspicious, and allowing developers to mark Strings as safe. Rails 3 escapes all Strings, allowing developers to convert Strings into safe Strings easily. We have also modified all of our internal helpers to mark Strings that we create as safe, while not modifying Strings passed in by the application developer.

Finally, we have overhauled all of the JavaScript helpers to emit plain HTML markup instead of JavaScript. As part of the process, we have created drivers for jQuery and Prototype that bind events to the appropriately marked up elements in the DOM. This means that Rails can now support any JavaScript library as a first-class citizen. We will support and maintain the markup and JavaScript developers can develop pure-JavaScript libraries to make use of the markup (without ever having to touch Ruby). If other frameworks emit the same markup, the same JavaScript libraries will work with their helpers.

ActiveModel

Over the years, people have created a number of ActiveRecord lookalike libraries for things like MongoDB or CouchDB. In Rails 3, we have extracted elements of ActiveRecord unrelated to RDBMS persistence, such as validations, callbacks, dirty tracking, and serialization into a separate library. Other libraries, or even plain old Ruby code, can use these components on an à la carte basis, choosing to look as much or as little like ActiveRecord as they wish.

Additionally, ActiveModel defines an API that any object can implement in order to work seamlessly with ActionPack. It provides drop-in implementations of the API, but Ruby libraries that wish to participate may implement the APIs themselves. This means that ActionPack itself is agnostic as to the ORM used, and even supports hand-rolled models.

ActiveRecord

ActiveRecord has undergone a large internal change, integrating Nick Kallen's ActiveRelation codebase. This replaces the internal ad-hoc query generation with query generation based on relational algebra. In essence, it means that you can chain queries for refinement before Rails builds the actual SQL query.

For instance, instead of Post.scoped immediately connecting to the database and retrieving all posts, it creates a relation object representing the idea of "all posts". You can then refine further, such as Post.scoped.where(:author => "Yehuda"). This means that you can pass around relations, refining them as appropriate in helper methods before the view finally kicks off the query when it iterates over the objects.

This also unifies similar concepts in ActiveRecord, like named scopes, into a single relational concept that pervades the entire ORM.

ActiveSupport

In order to facilitate use of ActiveSupport by other libraries, ActiveSupport 3.0 makes it easy for developers to cherry-pick specific parts of the library they wish to use. In particular, we have made all dependencies between ActiveSupport files explicit, so that if you require a particular extension, you can be sure that you get everything that you need. In the past, in order to use ActiveSupport extensions reliably, you needed to require the full library.

ActiveSupport also comes with some new and retooled modules, such as ActiveSupport::Concern (which makes it easier to manage dependencies between modules) and a significantly faster ActiveSupport::Callbacks.

Railties

We made the biggest user-facing changes in Railties. A Rails 3 application is an object (of the class Rails::Application), which also doubles as a valid Rack application. Instead of configuring your application in a block, you configure the Application itself (which now lives in config/application.rb).

The Rails::Application class also inherits from an Engine class, making Engines behave almost identically to Applications, as appropriate. And Engines are a special kind of Railtie, again simplifying and unifying the concepts in the Rails glue code.

The rails command itself now has special powers inside of Rails applications, replacing the commands in script with subcommands of rails. For instance, use rails server instead of script/server.

In Conclusion...

It's been an incredibly busy and productive year. Rails 3 is the culmination of thousands of man-hours by numerous dedicated open source contributors. It's been a long journey, filled with debate, feedback, and writing code, refactoring it and  then refactoring it yet again. While this release is just a beta, it represents the bulk of things to come in a stable and tested state.

Plugin authors should take a look at the new exposed APIs and start porting plugins to work on Rails 3. The community is excited, and we expect adoption to be rapid, aggressive and just as importantly, vocal. We'll be listening to that feedback to help get Rails 3 to its final perfected state, and you should be listening too, so your plugins can make the jump with us.

Leave comments here and on the official Ruby on Rails blog; we'll be here to respond!

In isolation, each of these pieces of code made a lot of sense as they were added, but as the pieces accumulated, plugin authors, and even Rails itself, had to often resort to contortions to get the job done. Our goal in Rails 3 was to solve these problems.

An Application Object

The first thing we addressed was that a Rails 2.3 application had pieces spread out in a number of places. For example, it stored the configuration on a global configuration object, stored the routes on a global router object, and booted up the server and console from this collection of global state. It also looked up plugins using PluginLocators and PluginLoaders. While all of this worked, we found it difficult to reason about and it was easy to trip over when implementing new features. We wanted to unify these pieces.

Second, we wanted to make it possible to run multiple applications in a single process. This meant we had to remove as much global state as possible. We will likely be continuing to work toward this goal over the next several releases, but the nice thing is that removing global state has already resulted in improvements in code quality and we have found that it makes code easier to understand.

If you take a look at a newly generated Rails 3 app, you now find a named application.

module YourApp
  class Application < Rails::Application
    # Settings in config/environments/* take precedence over those specified here.
    # Application configuration should go into files in config/initializers
    # -- all .rb files in that directory are automatically loaded.

    # Add additional load paths for your own custom dirs
    # config.load_paths += %W( #{config.root}/extras )

    # Only load the plugins named here, in the order given (default is alphabetical).
    # :all can be used as a placeholder for all plugins not explicitly named
    # config.plugins = [ :exception_notification, :ssl_requirement, :all ]

    # Activate observers that should always be running
    # config.active_record.observers = :cacher, :garbage_collector, :forum_observer

    # Set Time.zone default to the specified zone and make Active Record auto-convert to this zone.
    # Run "rake -D time" for a list of tasks for finding time zone names. Default is UTC.
    # config.time_zone = 'Central Time (US & Canada)'

    # The default locale is :en and all translations from config/locales/*.rb,yml are auto loaded.
    # config.i18n.load_path += Dir[Rails.root.join('my', 'locales', '*.{rb,yml}')]
    # config.i18n.default_locale = :de

    # Configure generators values. Many other options are available, be sure to check the documentation.
    # config.generators do |g|
    #   g.orm             :active_record
    #   g.template_engine :erb
    #   g.test_framework  :test_unit, :fixture => true
    # end
  end
end

This looks a whole lot like the initializer block in Rails 2.3, but the Application object, not a global configuration object, now has the configuration information. Similarly, routes.rb now begins YourApp::Application.routes.draw. The application object lives in the center of Railties in Rails 3, providing a core object that the rest of the frameworks can build on.

Improving Initialization

Rails 2.3 initialized a Rails application by calling a number of methods in sequence. Correct operation relied heavily on the order of those methods, and Rails ran some methods multiple times (apparently to hack around long-forgotten issues). As a next step towards getting a grip on Railties, we refactored the initializers so that Rails declared each initializer separately in a way that users could easily hook into.

This idea originated in Merb, where the framework defined each initializer as a separate class. Merb also provided a mechanism for declaring in the class body, that the framework should run a particular initializer before or after another one. By breaking the initializers up into reusable chunks, we provided a convenient API for extensions to hook in at whatever part of the process they wanted to.

In Rails 3, we have defined a slightly different API, inspired by the same ideas.

initializer :load_all_active_support do
  require "active_support/all" unless config.active_support.bare
end

# Set the $LOAD_PATH based on the value of
# Configuration#load_paths. Duplicates are removed.
initializer :set_load_path do
  config.paths.add_to_load_path
  $LOAD_PATH.uniq!
end

These first initializers in the Rails bootup process require active_support/all and set the load paths. Each initializer has a name, which allows other initializers (in plugins) to hook in at the appropriate time. Initializers also have access to the configuration object.

Making initializers their own isolated chunks definitely improves the code, and exposed a number of issues in the existing initializers. With initializers now living on the brand new application object, we then took a hard look at plugins.

Unifying Plugins

We wanted to unify plugins into one coherent system. Rails 2.3 supported several different kinds of plugins: vendor/plugins, gem plugins, vendor/plugins engines, and gem engines.

Rails 2.3 had a number of objects to find and load each kind of plugin, that resulted in a certain amount of magical behavior in each case. For instance, it automatically ran init.rb for plugins in vendor/plugins at the appropriate time. For Rails 3, we have created a new class, called Rails::Plugin, that can itself set configuration and have initializers. Rails 3 now merges in those initializers, giving plugins the ability to directly participate in the initialization process.

Each plugin in vendor/plugins automatically become a Rails::Plugin, but we also allow gems to explicitly create a subclass and add their own initialization. This makes plugins more powerful, giving them the ability to hook in at more precise parts of the process.

initializer :load_init_rb, :before => :load_application_initializers do |app|
  file   = "#{@path}/init.rb"
  config = app.config
  eval File.read(file), binding, file if File.file?(file)
end

Here, vendor/plugins defines an initializer called :load_init_rb, which runs before the application initializers. The block receives the application object, so it can take a look at the configuration, or other parts of the application in question.

Railties (Or, How Frameworks Became Plugins)

After making all of this progress, we revisited an earlier mission set out at the beginning of the merge: make all frameworks, like ActiveRecord and ActionController, behave like regular plugins. This would allow other plugins, like DataMapper, to use exactly the same APIs used by ActiveRecord. By definition, other projects could then replace all the functionality in ActiveRecord in a self-contained gem.

After moving each initializer into its own block, we noticed that a large number of the initializers were wrapped in if config.frameworks.include?(:active_record) and the like. If we fixed this, we could make good on our promise to make each framework a plugin. So now, instead of having Railties inspect the list of available frameworks and run appropriate initializers, the frameworks themselves add their own initializers, but only when loaded. When the user loads ActiveRecord, its plugin adds its initializers; when the user loads DataMapper, its plugin adds its initializers.

The monolithic Railties framework is now a set of individual Railties, each inheriting from a new class, Rails::Railtie. Railties has more power than simple plugins. For instance, a Railtie's name also serves as its key in the configuration object. And a Railtie can also specify rake tasks. For instance, if the user chooses DataMapper instead of ActiveRecord, he only sees DataMapper's rake tasks.

Because Railties merely specifies how Rails initializes them, requiring them has no side-effects. Taking a look at the ActiveRecord Railtie, you can see that the file first requires "rails" and "action_controller/railtie", ensuring that Rails runs the ActionController initializers before the ActiveRecord ones. It also allows ActiveRecord to define initializers that run before or after ActionController initializers.

Exposing an API to allow ActiveRecord to run decoupled from Railties forced us to enable functionality for third parties. For instance, because Rails loads Railties before initialization, this allows third-party plugins to hook into the very earliest stages of the process. By making the keys on config more general, we now can have config.data_mapper as well as config.active_record. By moving rake tasks and generators to the frameworks, and triggering them through the Railtie, Rails guarantees that the user can remove all artifacts of these frameworks by installing an alternative plugin. It also means that any plugin can add these artifacts.

Most usefully, it moves everything about how Rails should initialize ActiveRecord in a different context, such as booting up an app, running rake tasks, or running generators, into a single location that we can more easily maintain and that others can more easily copy. We think this will result in people being able to easily write Rails extensions that behave similarly to builtin Rails frameworks.

In short, Rails frameworks are now plugins to Rails.

In Inside Rails, Yehuda Katz, Rails expert and core team member, and Carl Lerche, Rails expert and full-time contributor, present expert advice and insight on the Rails platform and Rails development.

Rails 3 makes rendering options a first class citizen, and uses the same new system internally that plugins authors are expected to use. Before we get into how you can use this feature yourself, let's take a look at how Rails uses it:

ActionController.add_renderer :json do |json, options|
 json = ActiveSupport::JSON.encode(json) unless json.respond_to?(:to_str)
 json = "#{options[:callback]}(#{json})" unless options[:callback].blank?
 self.content_type ||= Mime::JSON
 self.response_body = json
end

Here, we are creating a new render :json option, which behaves exactly like render :json in Rails 2.3. In the render_json method, we use the same lower-level (but still public) APIs that are used by Rails itself to set the MIME type and response body. Using a render option entirely skips the rest of the render pipeline, so you don't have to worry about blocking normal template selection and rendering, or any other internal changes that might break your added option.

Next, let's take a look at adding a new render :pdf option. We'll use JRuby and the Flying Saucer library, which takes HTML and CSS and converts it to a PDF file. The syntax for the new option will be render :pdf => "template_name", :css => %w(main.css print.pdf), :layout => "print".

In order to understand how to do this, let's first take a look at the code necessary to use Flying Saucer in JRuby. First, you'll need to grab the Flying Saucer jars (you can get them from my Muse git repo). Next, let's write the code necessary to convert HTML and CSS to PDF with Flying Saucer:

# Both of these are .jar files
require "/path/to/itext"
require "/path/to/core-renderer"

module PdfUtils
 def self.string_to_pdf(input_string)
  io = StringIO.new
  renderer = org.xhtmlrenderer.pdf.ITextRenderer.new
  renderer.set_document_from_string_input input_string
  renderer.layout
  renderer.create_pdf(io.to_outputstream)
  io.string # the PDF file in a String
 end
end

It's a bit verbose, but it does work. Now that we have a way to make the PDF, let's wire it up into a render option.

ActionController.add_renderer :pdf do |template, options|
 css_files = Array.wrap(options.delete(:css))
 css = css_files.map {|file| File.read(file) }.join("\n")

 # Reuse the render semantics to get a string from the
 # template and options
 string = render_to_string template, options

 # Drop in the CSS before the  in a style tag.
 # In practice, you would probably cache the file reads
 # and the merging of the CSS and HTML.
 string.gsub!(%r{}, "


")
 send_data PdfUtils.string_to_pdf(string), :type => Mime::PDF
end

You'd also need to register the PDF mime type:

Mime::Type.register "application/pdf", :pdf

You could now do the following in a controller:

class PostsController < ActionController::Base
 def show
  @post = Post.find(params[:id])

  respond_to do |format|
   format.html
   format.pdf
   # or
   format.pdf { render :pdf => "show", :css => %w(application print) }
  end
 end
end

The tricky bits of this process are now reserved to figuring out how to build and return the output, not how to inject your option into render. Pretty cool, no?

When we announced the Rails/Merb merge,  we promised to bring a more stable plugin API to Rails 3.  Merb had strong opinions about an explicitly exposed plugin API, so we hoped to end the complications of alias_method_chain in favor of exposed APIs which plugin authors could rely on across versions.

Dogfooding

By far the most important element in developing the plugin APIs for Rails 3 was to eliminate the "secret" APIs used in Rails itself. Instead, we wanted to build Rails on top of the same core components that we would ask plugin authors to build on. The focal point of this effort (because we already had experience doing this with Merb) was ActionPack, and specifically ActionController.

ActionController presented us with another excellent opportunity: because ActionMailer reused a lot of the controller functionality, we had a built-in opportunity to create components that could be used in both ActionController and ActionMailer. In a sense, ActionController and ActionMailer would become plugins of the core system we were creating.

We also took a second important conceptual step. In Rails 2.3, the idea of "metal" was that you could skip the entire ActionController chain and write raw Rack handlers where needed for performance. In Rails 3, we are exposing an incremental series of components that, in sum, make up ActionController::Base. These components include rendering, layouts, conditional get, respond_to, streaming, and others.

In Rails 3, ActionController::Metal is the simplest possible controller, with no additional components added in. ActionController::Base is simply a subclass of ActionController::Metal with all components included. Again, we are dog-fooding our componentized APIs, not simply giving you a watered down version of what we use internally.

Now, let's dig into a few specific examples of new APIs.

ActionController::Renderers

One of the most common reasons people extend controllers is to expose additional renderers. For instance, you may want to render a PDF as well as HTML. The Rails 3 API that we expose to plugin authors for this purpose is identical to the one that we use internally.  (As you probably know, in addition to being able to render templates, you can render :json, :xml, or :js in normal Rails controllers.)

If you take a look at the source for ActionController::Renderers, here's how we do it:

add :json do |json, options|
  json = ActiveSupport::JSON.encode(json) unless json.respond_to?(:to_str)
  json = "#{options[:callback]}(#{json})" unless options[:callback].blank?
  self.content_type ||= Mime::JSON
  self.response_body  = json
end

This method is in ActionController::Renderers, so it's like saying ActionController::Renderers.add(:json) do |json, options|. You can add additional renderers easily in Rails 3 plugins, and you don''t have to figure out how to hook into Rails at the appropriate time. Like other parts of the public API, we're committing to keeping this API unchanged across multiple versions of Rails. If and when when the API DOES change, the change will be flagged for you first, via the normal Rails API deprecation process.

ActionController::Metal

In Rails 3, there's a supported way to use a simple, stripped down controller, and opt into the functionality you want. The overhead of the completely stripped down controller is a mere 8 microseconds (not much more than the overhead for a pure Rack app), compared with 100 or so microseconds of overhead for the full ActionController::Base (as I said last time, this itself is significantly less than Rails 2.3).

Here's an example of using ActionController::Metal:

class SimpleController < ActionController::Metal
  abstract!
end

class HelloController < SimpleController
  def index
    self.response_body = "Hello World"
  end
end

A few things to note here. First of all, you don't need a superclass, but you can build one to re-use in multiple places. The abstract! declaration tells Rails 3 not to include public methods on the superclass as controller actions. It's not strictly needed here (because there are no public methods), but it's a good habit for custom ActionController::Metal subclasses. Second of all, because this is a controller, you can route to it through the normal Rails router. From the perspective of the rest of Rails, this is a perfectly normal controller.

Finally, the self.response_body usage is the simple, low-level API used in ActionController itself (for instance, during template rendering), but it's now 100% first-class and available for your use. You can also set status, content_type and headers directly, regardless of whether you use the Request and Response helpers or not.

If you want to add rendering in, you can do something like this:

class SimpleController < ActionController::Metal
  abstract!
  include ActionController::Rendering
  append_view_path "#{Rails.root}/app/views"
end

class HelloController  "hello"
  end
end

You've now added in a new module, which provides rendering support. This keeps things simple by not including layout support, because you don't need it in this case. With this capability, you can add the features that you want to add, leaving out what you don't. This means that as a metal grows, you don't need to completely rewrite it as a "normal" controller. Instead, you can pull in the specific features you need. This process is seamless, and again, you can route to it as a normal controller throughout.

If you look at the source for ActionController::Base, you can see that we aren't doing anything particularly special anymore:

module ActionController
  class Base < Metal
    abstract!

    include AbstractController::Callbacks
    include AbstractController::Layouts

    include ActionController::Helpers
    helper :all # By default, all helpers should be included

    include ActionController::HideActions
    include ActionController::UrlFor
    include ActionController::Redirecting
    include ActionController::Rendering
    include ActionController::Renderers::All
    include ActionController::ConditionalGet
    include ActionController::RackDelegation
    include ActionController::Logger
    include ActionController::Configuration
    ...

Here, ActionController::Base inherits from Metal, and then proceeds to pull in all of the functionality not exposed as separate modules. Although Base pulls them all in, you don't have to worry about dependencies between these modules; handling dependencies is built into the way these modules work.

View Contexts

As one last example, the way that ActionController communicates with ActionView itself has become a fully defined, exposed API. From the perspective of ActionController, a view implements three APIs:

• Context.for_controller(controller)
• Context#render_partial(options)
• Context#render_template(template, layout, options)

By implementing this API, you can drop in any object in place of ActionView. How would you use this? Well to date, I can think of two examples: rspec_rails is going to use this functionality to implement isolation tests, and you can use this to implement a Merb shim in which the controller and view are the same object (an implementation using an earlier version of the API).

In addition to the usefulness of exposing this functionality via a stable Public API, reducing the boundary between ActionController and ActionView significantly improved the architecture, and exposed areas of brittleness that benefited quite a bit from the definition.

Conclusion

The APIs provided here, while stable at this point, are not final. If you feel that they (or any other part of Rails 3) aren't adequate for a plugin you're building, please let us know by commenting.

This post is just a taste of the new ways you can extend Rails 3. In the weeks ahead, expect a lot more documentation and discussion about the exposed plugin hooks.

For Rails 3, we wanted to take the performance optimizations in Merb and bring them over to Rails. In this post, I'll talk about just a few of the performance optimizations we've added to Rails 3: reducing general controller overhead and (greatly) speeding up rendering a collection of partials.

For our initial performance work, we focused on a few specific but commonly used parts of Rails:

• General overhead (the router plus the cost of getting in and out of a controller)
• render :text
• render :template
• render :partial
• rendering a number (10 and 100) of the same partials in a loop
• rendering a number (10 and 100) of the same partials via the collection feature

This was definitely a limited evaluation, but it covered most of the cases where performance might be at a premium and the Rails developer was unable to do anything about it.

General Controller Overhead

The first thing was improving the general overhead of a Rails controller. Rails 2.3 doesn't have any way to test this, because you're forced to use render :string to send back text to the client, which implicates the render pipeline. Still, we wanted to reduce it as much as possible.

When doing this work, we used Stefan Kaes' fork of ruby-prof that comes with the CallStackPrinter (the best way I've ever seen to visualize profile data from a Ruby application.) We also wrote a number of benchmarks that could double as profile runs if I wanted to zero in and get more precise data.

When we looked at overhead, it was dominated by setting the response. Digging a bit deeper, it turned out that ActionController was setting headers directly, which then needed to be re-parsed before returning the response to get additional information. A good example of this phenomenon was in the Content-Type header, which had two components (the content-type itself and an optional charset). The two components were available on the Response object as getters and setters:

def content_type=(mime_type)
  self.headers["Content-Type"] =
    if mime_type =~ /charset/ || (c = charset).nil?
      mime_type.to_s
    else
      "#{mime_type}; charset=#{c}"
    end
end

# Returns the response's content MIME type, or nil if content type has been set.
def content_type
  content_type = String(headers["Content-Type"] || headers["type"]).split(";")[0]
  content_type.blank? ? nil : content_type
end

# Set the charset of the Content-Type header. Set to nil to remove it.
# If no content type is set, it defaults to HTML.
def charset=(charset)
  headers["Content-Type"] =
    if charset
      "#{content_type || Mime::HTML}; charset=#{charset}"
    else
      content_type || Mime::HTML.to_s
    end
end

def charset
  charset = String(headers["Content-Type"] || headers["type"]).split(";")[1]
  charset.blank? ? nil : charset.strip.split("=")[1]
end

As you can see, the Response object was working directly against the Content-Type header, and parsing out the part of the header as needed. This was especially problematic because as part of preparing the response to be sent back to the client, the Response did additional work on the headers:

def assign_default_content_type_and_charset!
  self.content_type ||= Mime::HTML
  self.charset ||= default_charset unless sending_file?
end

So before sending the response, Rails was once again splitting the Content-Type header over semicolon, and then doing some more String work to put it back together again. And of course, Response#content_type= was used in other parts of Rails, so that it was correctly set based on the template type or via respond_to blocks.

This was not costing hundreds of milliseconds per request, but in applications that are extremely cache-heavy, the overhead cost could be larger than the cost of pulling something out of cache and returning it to the client.

The solution in this case was to store the content type and charset in instance variables in the response, and merge them in a quick, simple operation when preparing the response.

attr_accessor :charset, :content_type

def assign_default_content_type_and_charset!
  return if headers[CONTENT_TYPE].present?

  @content_type ||= Mime::HTML
  @charset      ||= self.class.default_charset

  type = @content_type.to_s.dup
  type < < "; charset=#{@charset}" unless @sending_file

  headers[CONTENT_TYPE] = type
end

So now, we're just looking up instance variables and creating a single String. A number of changes along these lines got overhead down from about 400usec to 100usec. Again, not a huge amount of time, but it could really add up in performance-sensitive applications.

Render Collections of Partials

Rendering collections of partials presented another good opportunity for optimization. And this time, the improvement ranked in milliseconds not microseconds!

First, here was the Rails 2.3 implementation:

def render_partial_collection(options = {}) #:nodoc:
  return nil if options[:collection].blank?

  partial = options[:partial]
  spacer = options[:spacer_template] ? render(:partial => options[:spacer_template]) : ''
  local_assigns = options[:locals] ? options[:locals].clone : {}
  as = options[:as]

  index = 0
  options[:collection].map do |object|
    _partial_path ||= partial ||
      ActionController::RecordIdentifier.partial_path(object, controller.class.controller_path)
    template = _pick_partial_template(_partial_path)
    local_assigns[template.counter_name] = index
    result = template.render_partial(self, object, local_assigns.dup, as)
    index += 1
    result
  end.join(spacer).html_safe!
end

The important part here is what happened inside the loop, which could occur hundreds of times in a large collection of partials. Here, Merb had a higher performance implementation which we were able to bring over to Rails. This is the Merb implementation.

with = [opts.delete(:with)].flatten
as = (opts.delete(:as) || template.match(%r[(?:.*/)?_([^\./]*)])[1]).to_sym

# Ensure that as is in the locals hash even if it isn't passed in here
# so that it's included in the preamble.
locals = opts.merge(:collection_index => -1, :collection_size => with.size, as => opts[as])
template_method, template_location = _template_for(
  template,
  opts.delete(:format) || content_type,
  kontroller,
  template_path,
  locals.keys)

# this handles an edge-case where the name of the partial is _foo.* and your opts
# have :foo as a key.
named_local = opts.key?(as)

sent_template = with.map do |temp|
  locals[as] = temp unless named_local

  if template_method && self.respond_to?(template_method)
    locals[:collection_index] += 1
    send(template_method, locals)
  else
    raise TemplateNotFound, "Could not find template at #{template_location}.*"
  end
end.join

sent_template

Now this wasn't perfect by a long shot. There was a lot going on here (and I'd personally like to have seen the method refactored). But the interesting part is what happened inside the loop (starting from sent_template = with.map). Unlike ActionView, which figured out the name of the template, got the template object, got the counter name, and so on, Merb limited the activity inside the loop to setting a couple of Hash values and calling a method.

For a collection of 100 partials, this could be the difference between overhead of around 10ms and overhead of around 3ms. For a collection of small partials, this could be significant (and a reason to inline partials that were appropriate to be partials in the first place).

In Rails 3, we've improved performance by reducing what happens inside the loop. Unfortunately,there was a specific feature of Rails that made it a bit harder to optimize this generically. Specifically, you could render a partial with a heterogenous collection (a collection containing Post, Article and Page objects, for instance) and Rails would render the correct template for each object (Article objects render _article.html.erb, etc.). This means that it was not always possible to determine the template to render up front.

In order to deal with this problem, we haven't been able to optimize the heterogenous case completely, but we have made render :partial => "name", :collection => @array faster. In order to achieve this, we split the code paths, with a fast path for when we knew the template, and a slow path for where it had to be determined based on the object.

So now, here's what rendering a collection looks like, when we know the template:

def collection_with_template(template = @template)
  segments, locals, as = [], @locals, @options[:as] || template.variable_name

  counter_name  = template.counter_name
  locals[counter_name] = -1

  @collection.each do |object|
    locals[counter_name] += 1
    locals[as] = object

    segments < < template.render(@view, locals)
  end

  @template = template
  segments
end

Importantly, the loop is now tiny (even simpler than what happened in Merb inside the loop). Something else worth mentioning is that in improving the performance of this code, we created a PartialRenderer object to track state. Even though you might expect that creating a new object would be expensive, it turns out that object allocations are relatively cheap in Ruby, and objects can provide opportunities for caching that are more difficult in procedural code.

For those of you want to see the improvements in pictures, here are a few things to look at: first, we have the improvement between Rails 2.3 and Rails 3 edge on Ruby 1.9 (smaller is faster).

And here it is for more expensive operations:

Last we've got a comparison of Rails 3 across four implementations (Ruby 1.8, Ruby 1.9, Rubinius, and JRuby):

You can see that Rails 3 is significantly faster than Rails 2.3 across the board, and that all implementations (including Rubinius!) are significantly improved over Ruby 1.8. All in all, a great year for Ruby!

Next post, I'll talk about improvements in the Rails 3 API for plugin authors—keep an eye out, and as always, leave your comments!

A year has gone by, so it's a good time to reflect on how well we've done at achieving those expectations. Over the next few days, I'll take each bullet point that I provided in my original post, and go into detail about the progress we've made on that front.

I've given a few recent talks on these topics, so some of you may already have seen some of this, but I wanted to get it down in writing for those who hadn't. I've also added new information, some of which was omitted because it was difficult to explain in a talk, and some of which is too current for any of my recent talks.

Modularity

Rails will become more modular, starting with a rails-core, and including the ability to opt in or out of specific components. We will focus on reducing coupling across Rails, and making it possible to replace parts of Rails without disturbing other parts. This is exactly what Merb means when it touts "modularity".

We've spent a significant amount of time on this step which has been really fruitful. I'll give a few specific examples.

ActiveSupport

First, we've gone through ActiveSupport, making it viable to cherry-pick specific elements. This means that using ActiveSupport's inflector, time extensions, class extensions, or anything your heart desires is now possible without having to personally track the dependency graph. Here's an example of what I mean, from the to_sentence method in ActiveSupport from Rails 2.3:

module ActiveSupport #:nodoc:
  module CoreExtensions #:nodoc:
    module Array #:nodoc:
      module Conversions
        def to_sentence(options = {})
          ...
          options.assert_valid_keys(:words_connector, :two_words_connector, :last_word_connector, :locale)
          ...
        end
        ...
      end
    end
  end
end

As you can see, to_sentence has an implicit requirement on assert_valid_keys, which means that in order to cherry-pick active_support/core_ext/array/conversions, you are forced to work through the file, find any unsatisfied dependencies, and be sure to require them as well. And of course, the structure of these dependencies could easily change in a future version of Rails, so relying on what you'd found would be unsafe. In Rails 3, the top of that same file looks like:

require 'active_support/core_ext/hash/keys'
require 'active_support/core_ext/hash/reverse_merge'
require 'active_support/inflector'

This is because we've gone through the entire ActiveSupport library, found the unsatisfied dependencies, and made them explicit. As a result, you can pull out the specific libraries you want for a small project, and not get the full weight of ActiveSupport.

Even better, other parts of Rails now explicitly declare the dependencies they have on ActiveSupport. So for instance, the code that adds logging support to ActionController has the following lines on top:

require 'active_support/core_ext/logger'
require 'active_support/benchmarkable'

This means that all of Rails knows what parts of ActiveSupport are needed. For simplicity, Rails 3 ships with all of ActiveSupport still provided, so you'll be able to use things like 3.days or 3.kilobytes without interruption. However, if you want more control over what gets included, that's possible. You can declare config.active_support.bare = true in your configuration and we'll pull in only the parts of ActiveSupport explicitly needed for the parts of Rails that you use. You'll still need to include the fancy parts if you want to use them - 3.days wont work out of the box with bare enabled.

ActionController

Another area that really needed an overhaul was ActionController. Previously, ActionController had a number of disparate elements all in one place. When we looked closely, we found that there were really three discrete components masquerading as one.

First, there was the dispatching functionality. This included the dispatcher itself, routing, middleware, and rack extensions. Second there was generic controller code that was meant to be reused elsewhere, and was in fact reused in ActionMailer. Finally, there was the subset of controller code that brought those two concerns together: code that handled requests and responses through a controller architecture.

In Rails 3, each of those components has been separated out. The dispatcher functionality has been moved into ActionDispatch, with the code inside tightened up and really made a conceptual component. The parts of ActionController that were meant to be reused by non-HTTP controllers was moved into a new component called AbstractController, which both ActionController and ActionMailer inherit from.

Finally, ActionController itself has gotten a significant overhaul. Essentially, we've isolated every standalone component, and made it possible to start with a stripped-down controller and pull in just the components you want. Our old friend ActionController::Base simply starts with that same stripped-down controller and pulls everything in. For instance, take a look at the beginning of the new version of that class:

module ActionController
  class Base < Metal
    abstract!

    include AbstractController::Callbacks
    include AbstractController::Logger

    include ActionController::Helpers
    include ActionController::HideActions
    include ActionController::UrlFor
    include ActionController::Redirecting
    include ActionController::Rendering
    include ActionController::Renderers::All
    include ActionController::Layouts
    include ActionController::ConditionalGet
    include ActionController::RackDelegation
    include ActionController::Logger
    include ActionController::Benchmarking
    include ActionController::Configuration

All we're doing here is pulling in every available module, so the default experience of Rails is the same as before. However, the real power of what we've done here is the same as what we've done in ActiveSupport: every module declares its dependencies on other modules, so you can pull in Rendering, for instance, without having to wonder what other modules need to be included and in what order.

The following is a perfectly valid controller in Rails 3:

class FasterController < ActionController::Metal
  abstract!

  # Rendering would be pulled in by layouts, but I include
  # it here for clarity
  include ActionController::Rendering
  include ActionController::Layouts

  append_view_path Rails.root.join("app/views")
end

class AwesomeController < FasterController
  def index
    render "so_speedy"
  end
end

And then, in your routes, it would be perfectly valid to do:

MyApp.routes.draw do
  match "/must_be_fast", :to => "awesome#index"
end

Essentially, ActionController::Base has become just one way to express your controllers. Think of it like Rails Classic, with the ability to roll your own if you're not so into that taste. It's really easy to mix and match too: if you wanted to pull in before_filter functionality to FasterController, we could simply include AbstractController::Callbacks.

Note that without doing anything else, including those modules pulled in AbstractController::Rendering (the subset of rendering functionality shared with ActionMailer), AbstractController::Layouts, and ActiveSupport::Callbacks.

This makes it really possible to trivially pull in just the specific functionality you need in performance-sensitive cases without having to use an entirely different API. If you need additional functionality, you can easily just pull in additional modules or eventually upgrade to the full ActionController::Base without needing to rip anything apart along the way.

This, in fact, is a core idea of Rails 3: there are no monolithic components, only modules that work seamlessly together in a great package of defaults. This allows people to continue using Rails as they have used it successfully in previous versions, but really leverage the codebase for alternative uses. No more functionality locked away in non-reusable forms.

One nice immediate benefit of all of this is that ActionMailer gets all of the functionality of ActionController in a clean, intentional way. Everything from layouts and helpers to filters is using the identical code that ActionController uses, so ActionMailer can never again drift away from the functionality of ActionController (as ActionController itself evolves).

Middleware gets a helping hand too. ActionController::Middleware, which is middleware with all of the powers of ActionController, allows you to pull in whatever ActionController features you want (like Rendering, ConditionalGet, robust Request and Response objects, and more) as needed. Here's an example:

# The long way
class AddMyName < ActionController::Middleware
  def call(env)
    status, headers, body = @app.call(env)
    headers["X-Author"] = "Yehuda Katz"
    headers["Content-Type"] = "application/xml"

    etag = env["If-None-Match"]
    key = ActiveSupport::Cache.expand_cache_key(body + "Yehuda Katz")
    headers["ETag"] = %["#{Digest::MD5.hexdigest(key)}"]
    if headers["ETag"] == etag
      headers["Cache-Control" = "public"]
      return [304, headers, [" "]]
    end

    return status, headers, body
  end
end

# Using extra Rack helpers
class AddMyName < ActionController::Middleware
  include ActionController::RackDelegation

  def call(env)
    self.status, self.headers, self.response_body = @app.call(env)

    headers["X-Author"] = "Yehuda Katz"

    # but you can do more nice stuff now
    self.content_type = Mime::XML # delegates to the response
    response.etag = "#{response.body}Yehuda Katz"
    response.cache_control[:public] = true

    self.status, self.response_body = 304, nil if request.fresh?(response)

    response.to_a
  end
end

# Using ConditionalGet helpers
class AddMyName < ActionController::Middleware
  # pulls in RackDelegation
  include ActionController::ConditionalGet

  def call(env)
    self.status, self.headers, self.response_body = @app.call(env)

    headers["X-Author"] = "Yehuda Katz"

    self.content_type = Mime::XML
    fresh_when :etag => "#{response.body}Yehuda Katz", :public => true

    response.to_a
  end
end

In all, I really think we've delivered on our promise to bring significant modularity improvements to Rails (and then some). In fact, I think the level of success that we've had with this version exceeds most people's expectations of a year ago, and is solidly in golden unicorn territory. Enjoy!

Next, I'll talk about bringing performance improvements to Rails 3. Hopefully it won't go by too fast. :)

In Inside Rails, Yehuda Katz, Rails expert and core team member, and Carl Lerche, Rails expert and full-time contributor, present expert advice and insight on the Rails platform and Rails development.

These tips are not extremely difficult to implement yourself with another framework, but Rails and Engine Yard believe in making your websites fast by default: you shouldn't need to understand all the minutiae in order to get a snappy application.

Use Far-Future Expires

YSlow recommends using an Expires header set far into the future. This means that browsers don't need to ask for your images, JavaScripts, or stylesheets again after the first request. However, this introduces an additional problem: what happens if you (inevitably) change these files? Won't the browser be stuck with the old file forever?

The solution is to include a last modified timestamp with your assets (something like <img src="/images/myasset.png?84392578943" alt="" />). When the file is modified, you update the timestamp, and the browser will know to ask for the file again.

When you use Rails, this timestamping is the default behavior; all you need to do is set up your Nginx or Apache config to serve with far future headers and you're ready to go. And if you use Engine Yard, we set up those headers for you.

GZip Components

YSlow recommends GZipping JavaScript, CSS, and other textual assets. This reduces their file size over the wire by approximately 70%. This can be set up using your Apache or Nginx config. If you use Engine Yard, your assets are gzipped by default.

Split Components Across Domains

Some older browsers (*cough* IE *cough*) will only download two assets from a host at a time. This means that if you have two JavaScript files, two CSS files, and 10 images, these assets will come down two at a time, even though the user's connection is fast enough to download them in parallel. The solution to this problem is to split your assets up over multiple hosts (assets0.yourapp.com, assets1.yourapp.com, and so on).

When using server-side code you wrote yourself, or using a traditional server-side framework, you'd be forced to go through all of the URLs you generated and update them to use an asset host helper. With Rails, you can simply add a single line to your configuration and all of your generated URLs will start using the asset hosts you specified:

config.action_controller.asset_host = "assets%d.yourapp.com"

Of course, you'll need to set up a CNAME entry in your DNS to point the new virtual domains at the same host, but that should be trivial.

This also single-handedly resolves another YSlow recommendation: "Use Cookie-free Domains for Components." If your assets and application are on the same domain, the browser and server will pass cookies back and forth (up to 4k worth) for each asset request. By using asset hosts for your assets, you get the side benefit of cookie-free domains for your assets.

When comparing frameworks, it's tempting to look at something like "Etag Support" as a feature and simply check it off. Looking under the surface, however, there are complex considerations that are begging to be abstracted away from you. Why should you have to know what the header name in the request is called (If-None-Match) or how to generate a reliable unique key for an object?

Rails makes working with these ideas drop-dead simple, so you won't have to stress about following web best practices; when using Rails, you'll be doing the right thing and hardly know it. Stay tuned for a follow up post on more of the ways Rails makes fast page loading a breeze.

I know a lot of you are eager for information on Rails 3. While this isn't a whole lot of code... think of it more like a movie trailer, and keep an eye out here, on the rubyonrails.org blog and on my personal blog for more information as we get closer to the release.

1. New Bundler

A lot of you have had trouble with the various efforts to make it possible to declare gem dependencies in your application and ensure that they're available on your development and production environment. Merb had an early solution to this problem that got about 90% of the way there, but didn't handle a large number of gems very well. This was exposed by the Engine Yard Cloud team, who had some trouble with the Merb gem bundler with a large number of potentially interdependent gems.

The new bundler handles virtually all issues we could think of around bundling gems in Rails applications, and then some.

Rubygems Resolution

This problem arises when you have multiple gems that depend on overlapping versions of some other gem. For instance, let's say you depend on ActiveMerchant 1.4.2 and Rails 2.3.2. ActiveMerchant depends on activesupport >= 2.3.2. Rails 2.3.2 depends on activesupport = 2.3.2. Now let's say you have Rails 2.3.2 and Rails 2.3.3 on your system.

If you go ahead and require "activemerchant", it will see the >= 2.3.2. Since you have 2.3.3 on your system, it will pull it next. Next, you gem "action_controller", "= 2.3.2". Since ActionController depends on activesupport = 2.3.2, it'll see that 2.3.3 was already activated and fail. You'll get the dreaded Gem::LoadError: can't activate activesupport (= 2.3.2, runtime), already activated activesupport-2.3.3.

This problem arises because gems are currently resolved linearly instead of doing an algorithmic dependency resolution. It turns out that doing dependency resolution with overlapping arbitrary ranges over a large set of potential dependencies is a hard problem. It took us a number of iterations over about a year to get it right, but our current solution is simple, elegant, reasonably fast, and most importantly, correct.

Limit Network Dependencies for Deploy

This is something we felt very strongly about with the Merb bundler, and most people seem to be coming around. The basic idea is that you should store your app's dependencies inside your application, not require a connection to Rubyforge, Github, and who knows what else at deploy time.

In addition to the possible risk of a network resource being down, there is the risk that you declared a dependency on something like nokogiri, and a new version was released to Rubyforge since staging, which breaks your code. You could solve this by always hardcoding all of your versions, but it's nicer to be able to declare your conceptual dependencies ("I want Nokogiri -- any version"). Once you pull the latest versions of your declared dependencies into your application, they should stay put, rather than possibly changing at deploy time.

Consistent Environment

By bundling specific .gem files inside of your application, you ensure that every developer, staging, and production are all running with the same environment.

Native Gems

By storing just the .gem file in the application, we can have a command that compiles the native gems for the architecture in question.

2. ActionController Architecture

In Rails 3, ActionController::Base is built on top of ActionController::Metal, a stripped down version of ActionController with support for simple Rack semantics. We then include a series of modules to add support for things like callbacks, rendering, layouts, helpers, and on and on. This means that it's easy for you to start with ActionController::Metal and pull in just the features you want, paying just the performance cost you want.

We even implemented deprecated 2.3 features as a module that gets mixed in to ActionController::Base. If you aren't using any deprecated features, you can simply stop including the module and get some performance back. You also know that you're not using any features you shouldn't be.

3. Responder

Since we got RESTful controllers, it has been very common to see idiomatic controllers that have actions looking like this:

def create
  @user = User.new(params[:user])

  respond_to do |format|
    if @user.save
      flash[:notice] = 'User was successfully created.'
      format.html { redirect_to(@user) }
      format.xml { render :xml => @user, :status => :created, :location => @user }
    else
      format.html { render :action => "new" }
      format.xml { render :xml => @user.errors, :status => :unprocessable_entity }
    end
  end
end

In Rails 3, this will be shrunk down to:

respond_to :html, :xml # class level

def create
  @user = User.new(params[:user])
  flash[:notice] = 'User was successfully created.' if @user.save
  respond_with(@user)
end

What we do is move the idiomatic logic into a new object called a Responder. By default, we use the Responder that ships with Rails, but you can use your own subclass the same way you would set a controller-wide (or application-wide) layout. That way, you can package up your own RESTful idioms into objects and use those.

4. ActiveModel

You've probably heard that things like validations are now available to any Ruby object by simply including a module: it's true. Take a look:

class Person
  include ActiveModel::Validations

  validates_presence_of :name
  attr_accessor :name
  def initialize(name)
    @name = name
  end
end

Person.new.valid?         #=> false
Person.new.errors         #=> {:name => ["cannot be blank"]} # localizable of course
Person.new("matz").valid? #=> true

This is pretty cool, and provides a lot of ActiveRecord functionality for other ORMs. What's really exciting to me, however, is that ActiveModel provides an API that can be used by ORM plugins to hook directly into ActionPack. If an object complies with the ActiveModel API, it will work seamlessly with the rest of Rails, just as well as ActiveRecord works in Rails 2.3.

This requires:

• Object#to_model: A method that returns an ActiveModel compliant version of the object. In most cases, this will return self
• Model#new_record?: true if the record has not yet been persisted
• Model#valid?: true if the record is valid (or the object has no concept of validations)
• Model#errors: a valid errors object. The errors object should respond to #[](field) and #full_messages
• Model#class.model_name: a valid naming object that responds to singular, plural, element, collection, partial_path, and human. This is used in routing, URL generation, and a number of other areas, and is possibly the most important integration point.

If your object implements these APIs, it will work with ActionPack with the same level of support as ActiveRecord. Pretty exciting!

5. Performance

Over the past several weeks, I've spent a lot of time working on reducing Rails' overhead. There's still a lot to be done, but I was able to reduce the overhead of Rails between two and five times, with especially good gains on Ruby 1.9. While this does not improve the time spent in your code, it reduces the overhead in calling partials, rendering collections, rendering templates, and rendering the entire response. This will be especially apparent for action-cached pages where most of the cost of the request is in Rails, not your request.

I've also started to look at general bottlenecks, like stylesheet_link_tag (surprising!) and URL generation in Paperclip, two things that bubbled up in a recent performance analysis of a real application.

Most of the improvement came from finding bottlenecks and then caching them, which can be tricky because of the possibility of dynamic changes. For instance, it is possible to modify the view_paths during any request, so it's hard to cache view lookup, since someone might prepend a path to the lookup order. In this case, my current approach has been to take a slower path, which means that doing dynamic things in Rails 3 is noticeably more expensive than the same operations in Rails 2.3 (where none of this was cached). I'll have some details on what kinds of things to avoid for optimal performance in a later post.

For now, until the next batch of awesomeness, there are my top upcoming features of Rails 3. Feel free to comment with questions on these or other Rails 3 features and topics. As always, I'm happy to help however I can.