Timing ruby methods

I recently started working on an old project (a couple years of dev with several different teams and different preferences on style).  It has a full test suite which is slow (~25 minutes).  And there are pages which suffer from over-wrought code that makes ruby spin and spin to get data.

In an effort to speed things up, I developed several little timing tests to see what ruby does well, and how we can improve things by using the right ruby convention or method.

I thought I should share the learnings.

The basic takeaways are:

  1. join strings with “+”, not join or interpolation (https://github.com/rcode5/ruby-join-strings-speed-test)
  2. keyword arguments for methods is slower (by ~15%) (https://github.com/rcode5/ruby-keyed-arguments-speed-test)
  3. services that are not purely class methods are slower (https://github.com/rcode5/ruby-class-v-instance-speed-test)
  4. matching strings with == (and downcase) is faster than regex (https://github.com/rcode5/ruby-match-strings-speed-test)
  5. .sum is equivalent to .reduce or .inject until ruby 2.4 where .sum is way faster (https://github.com/rcode5/ruby-sum-numbers-speed-test)

Check out the repositories for more details.

Upgrading Ruby for Rails3 – Installment 3

If you’re following along, I’ve been trying to get an old up from Rails 2.3 up to Rails 3.x (previous installments 1, 2).

After a bunch of refactoring and clean up, I was able to get my coverage report to an acceptable level. The app was running successfully on Rails3.0. Now to try to upgrade ruby.

  • Update my Gemfile & .ruby-version files to use ruby 1.9.3
  • Dump rcov from the Gemfile (eventually I’ll replace that with simplecov
  • bundle and run tests
  • clean up the mess
  • undefined method `each' for String

    No problem. I dug in to find a test trying to parse an http response body by newlines. Apparently ruby-1.8.7 String would handle this.

    The fix was simple

    -    resp.body.each do |row|
    +    resp.body.split("\n").each do |row|

    Scopes with arguments

    I’m not proud to admit it but I had a scope defined which both took an argument (not such a bad thing) but then wrote custom SQL based on that parameter. A combo of changes related to how lambda arguments are handled by scopes and the way the arguments were passed into the scope caused troubles. The fix was something like this (note: the model names have been changed to protect the innocent):

    -  scope :participants, lambda { |*key|
    -    q = key.blank? ? Conf.participant_key : key
    -    joins(:extra_info).where("extra_infos.participation like '%%{q}%%'")
    +  scope :participants, lambda { |key|
    +    q = (key.blank? ? Conf.participant : key).to_s
    +    joins(:extra_info).where("extra_infos.participation like '%#{q}%'")

    If you forget about the fact that I was pushing a string unprotected into a SQL query, the two things that seemed to cause issues were the splat on the lambda’s input args and the double ‘%’ in the query string. This change at least made the code behave as it used to.

    This code is actually doing a bunch of work that could be handled with serialize :participation, Array. I’ve added a chore for myself to make that change before I go further. More about serialize can be found here.

    Even after fixing that, the scope worked fine with arguments, but was very unhappy in cases where I had nil args which lead to more failing tests. In the end, I refactored the scope into a method and it made the whole thing much simpler.

    def self.participants(key = nil) 
      q = key || Conf.participant
      joins(:extra_info).where("extra_infos.participation like '%#{q}%'")

    Symbols to Strings

    The final straw was updating a few tests that counted on public_instance_methods. The test tried to grab all controller methods and run simple checks that methods existed. public_instance_methods returned strings in Ruby 1.8.7 and symbols in the new.

    1.8.7>  CatalogController.public_instance_methods(false)
    => ["index"]
    1.9.3>  CatalogController.public_instance_methods(false)
    => [:index]

    A quick fix (map the result to strings) got everything back in line.

    Well that was easy, how about ruby2.0.0?

    Once I had that rolling, what’s the harm in running all the way to ruby2.0.0?

    Following the same steps, I updated the Gemfile and .ruby-version. Rebundled, and started fixing issues.


    It turns out that ruby2.0.0, String#start_with?does not allow a nil input. This causes stylesheet_link_tag to fail because in Rails 3.0.20, it does the following:

    # actionpack-3.0.20/lib/action_view/helpers/asset_tag_helper.rb:749
    if has_request && include_host && !source.start_with?(controller.config.relative_url_root)

    and it seems that Rails 3.0.20 does not set relative_url_root by default. (more discussion here on stackoverflow)

    After adding a quick patch to my application.rb

    ActionController::Base.config.relative_url_root = ''

    everything was back in line.

    insert_record not available on Array

    This was the last big error that came up during tests. It seems that to get around this, I needed to move to Rails 3.1. Feeling like I was on a roll, I happily started more migrating.

    • Updated Rails to 3.1.12
    • Updated mysql2 gem and removed the activerecord-mysql2-adapter
    • added simplecov
    • removed tmail gem
    • remove config.action_view.debug_rjs from Rails config files
    • moved restful_authentication files into lib (from vendor/plugins)
    • upgraded restful_authentication to use AASM directly (instead of acts_as_state_machine method
    • Update references to Fixtures to use ActiveRecord::Fixtures

    And with that, I was on Ruby-2.0 and Rails-3.1. Not too shabby.

    I’m still not using the asset pipeline, but feel like I’m in a good place to make that jump to Rails3.2 and only have to worry about getting the pipeline in place.

rbenv and rvm plugins for Jenkins

For any of you folks still managing your own Jenkins CI box like I am, you may be interested to know that the rbenv and RVM plugins are great no-nonsense plugins that work.

My current Jenkins instance has been running for several years. Jenkins itself is getting upgraded with standard Ubuntu package upgrades, but it’s been a long time since I looked into plugin upgrades.

Older projects had some ugly setup script to get the rvm environment correct for the build. Stuff like:

. "$HOME/.rvm/scripts/rvm"
rvm use --create 1.9.3-p194@selectorschoice
gem install bundler
bundle exec rake

Much of this was came from old rvm docs or elsewhere on the web.

Well, I took the time to update my plugins recently and tried both the RVM and rbenv plugins. They both work smoothly. Simply add them to your Jenkins system. Then configure the build to run in either RVM or rbenv environment. Now my test scripts have been trimmed down to

bundle exec rake

And the log outputs are much cleaner so it’s easier to dig in when there are issues.

Check them out:

RVM Plugin version 0.4
rbenv Plugin version 0.0.15

Upgrading Rails2 -> Rails3 – installment 2

Installment 2 of my upgrade from Rails2 -> Rails3

Ack – I depend on all these gems!

I spoke with a co-worker about this. He’d been through the upgrade and mentioned that Gem upgrades was a major part of the move from Rails2 -> Rails3.

As I mentioned before, I’d taken steps to get my app as close to Rails3 ready as possible (read installment 1). Now it was time to pull the trigger.

I updated my Gemfile to require
gem "rails", "3.0.20"
bundle update rails

I immediately ran into issues. Out of date gems!

Since I was moving forward, I decided to do a full bundle update. This would force everything (within dependencies) to the latest. I then could go back and figure out issues. In the end, I had to pin a few gems down. I got to upgrade a few. I had to dump a few. And the rest (at least at this point), look like they’ll work out as is.


gem 'nokogiri', '~> 1.5.0' # At 1.6.0, this gem is no longer ready for 1.8.7. Pin it to 1.5.x.
gem 'gibbon', '~> 0.3.5' # mailchimp api
gem 'browser', "= 0.1.6" # 0.2.x does not support ruby 1.8.7


gem 'dalli' # now I can use 2.x
gem 'rspec' # now I can use 2.3.x
gem 'will_paginate' # move up from version 2.3.16


Moved mobile-fu to mobile-fu-rails3

gem 'mobile_fu-rails3'

Removed (looking for replacements)

has_many_polymorphs, ym4r_gm

Of those removed, I should add a few notes.


This one caused me all kinds of headaches. I did some searching and was able to find a replacement (here) that said it was Rails3 ready. I neglected to read the not so fine print from the README

This is a hacky attempt to port has_many_polymorphs to work under Rails 3. It is intended solely for my use (and use as a gem) but is offered here in case it helps anyone else develop it into a more complete port.

After sticking this into the Gemfile (and doing some other mods), my test suite started failing hard. belongs_to and has_many were failing on class load. I tore my hair out for a bit (read more here). Then some deep debugging revealed that the has_many_polymorphs update was Rails3.2 compatible, but not Rails3.0.

I’ve torn it out completely. It was relatively easily replaced by using the :polymorphic setting for Rails associations. You can read more about that here.


Strictly speaking, ym4r is not a gem, it’s a plugin. At the time I’d built this site, we wanted a map of Users. Using geokit and geokit-rails with a the Ym4r_gm plugin, I could generate a Google Map from ruby and drop it on the view. It seems that the Ym4r plugin has fallen away in favor of Google Maps For Rails. For now, I’ve decided to live without the Map (it’s only one page) until I get to Rails 3.2 and then I’ll check my options. I may just write the client side (JS) integration from scratch with Googles JS api. Less dependence on gems.

Other gem related mods

While making all these changes, and trusting my test suite, I decided to make a few more updates.

I removed mocha which I was using for mocking in tests. The new Rspec does a fine job with Double and has good support for stubs. This meant combing through my specs replacing:

before (mocha) after (rspec-*)
mock('thing') double('thing')
MyModel.any_instance.stubs(:method => val) MyModel.any_instance.stub(:method => val)

I also installed a much needed rcov to get some sense of my test coverage. I was able to stick this on the Rails2.3 branch and the Rails3.0 branch to validate that in all my mucking around, I hadn’t lost any tests. Happily, the old branch was 89% covered (not too bad, considering this was my Rails learning project). And I was able to validate that after updating tests to not rely on mocha I still had 89 percent coverage. In fact, in the process, the coverage went up, because I did some code-cruft clean up and added tests in a few spots.

As I find out more, I’ll scribble some more notes. Though these posts maybe a bit sloppily written, I hope they save a few folks out there who may still be going through this upgrade a headache or two.

Upgrading Rails2 -> Rails3

The past few months, among normal work stuff, I’ve been busy working to convert an old (5years and running) app from Rails 2.3.x to Rails 3 with the hope of finally getting to Rails 3.2 and Ruby 1.9.x.

It’s been a bit of an uphill slog, as you might imagine.

I’m going to start scribbling some of my notes here in the blog in a series of posts in hopes that others don’t rathole on the same issues that I struggled with.

Pre Rails3

After a chunk of reading online (much thanks to the airbnb post), I started by trying to update the codebase without updating Rails so that when I did move, I’d have as much of the simple stuff resolved in the Rails2 branch.

Step 1: rails_xss

I started by installing the rails_xss gem. This switches the default view generators to escape strings which is inline with what Rails3 views do.

I added to my Gemfile :
gem 'rails_xss'
gem 'erubis'

Thankfully, I had a pretty strong view test suite and was able to find and fix issues based on my test failures.

Step 2: Dump RAILS_ROOT

This was pretty much a search-replace task. Replace RAILS_ROOT with Rails.root

Step 3: include jQuery with noConflict

This app was still on PrototypeJS. It seems like it’s finally time to make the move. The first step is to simply get jQuery in the codebase with noConflict. This will allow me to convert, over time, the Prototype plugins and code to jQuery without having to move it all in one chunk.

This was pretty straight forward. Add the jQuery source from http://jquery.com/ and include it in my HAML layouts like so (at the top, before including prototype and other library js):

= javascript_include_tag "thirdparty/jquery-1.8.2.min.js"
= jQuery.noConflict();

With these mods, and getting my test suite sorted out, I was able to push out a new version still on Rails 2.3.x but much closer to being Rails3 ready that it was before these mods.

I’ll continue to add notes in subsequent posts.


Easy Postgres Backup/Restore from Heroku with PGBackups and Rake

Most of my projects lately have been deployed on Heroku. They’ve developed a really nice set of tools to get your Rails (and other) apps from a git repo out into the world. They do really smart things regarding database connections to make things easy to push live. If you follow the standard setup, you’ll be running on Postgres hosted at Heroku.

Often, we want to take data that may be out on a live app (staging or production level) and setup a development machine to have that data. For complex data models and complex data setups, this can be the only way to debug issues that may not have been covered by standard unit/integration tests. With PGBackups, a heroku add-on, and a couple small Rake tasks, this is a snap.

First, make sure you have pgbackups setup for your app:

% heroku addons:add pgbackups

With this enabled, you can immediately grab a snapshot of the database

heroku pgbackups:capture

You can also easily setup auto-weekly or auto-monthly snapshots

heroku addons:add pgbackups:auto-week
heroku addons:add pgbackups:auto-month

To grab the database, you ask for the url

heroku pgbackups:url

which you can then pull down with curl or by some other means. Then use pg_restore to jam it into your local database (if you want).

To make this even easier, I wrote a couple rake tasks which i bring along from project to project. To make it even easier.

Drop this file in your lib/tasks/ directory, update the namespace and development database name and you’re off to the races:

With this in place, you should be able to do:

rake my_app_namespace:db:fetch app=my_heroku_app_name
rake my_app_namespace:restore dbfile=the_dump_file_from_the_previous_command

The other task in there may not suit everyone’s purposes, but if you have sensitive User data, you can use that task to clean it up. Currently it just changes passwords to ‘monkey’. But you could do other things, like set all emails to or remove phone numbers or any other bits that you might not want to be real on your development system.

Now that I’ve done this a couple times, the next step to improve things is to get the script to read the local database.yml to grab the development database name automagically. An easy addition if you want to streamline things even more.

Custom RSpec Matchers for arrays

More than once, I’ve had to check an array for it’s order in RSpec. Usually, I just sort the array, and compare it to itself. The spec might look something like

describe MyObject
  describe "scope #ordered" do
    it "returns results in most recent order"
      expect(MyObject.ordered.all.sort(&:attr_to_order_by)).to eql my_objects

This may seem a bit opaque (as a reader) – how does the sorting relate to the test? Also, if you looking at timestamps (like sort by created_at), and you’re using fixtures or FactoryGirl.create, you may end up with duplicate timestamps. Suddenly your sort will be ill defined because the secondary sort key is undefined. A better method would be to check that the :created_at time is monotonically decreasing.

An array x is monotonically decreasing if and only if:
x_i \geq x_j \forall i < j

For these kinds of cases, I decided to write a some RSpec matchers. Following the RSpec Wiki docs here, it was a snap. I wrote matchers for monotonically increasing/decreasing and strictly increasing/decreasing tests.

The following four matchers (available from this gist (included below)) give you


So now I can rewrite my spec

describe MyObject
  describe "scope #ordered" do
    it "returns results in most recent order"
      # the most recent object should have the largest created_by timestamp
      expect(MyObject.ordered.map(&:created_by)).to be_monotonically_decreasing

These matchers follow the definitions from NIST for monotonically/strictly increasing/decreasing (see links below). The short story is:

Monotonically Increasing means:
x_i \leq x_j \forall < j

Strictly Increasing means:
x_i < x_j \forall i < j

Monotonically Decreasing means:
x_i \geq x_j \forall i < j

Strictly Decreasing means:
x_i > x_j \forall i < j

To use these, put the gist (below) custom_array_matchers.rb file in your spec/support directory and you’re off to the races. You can start using those matchers in your specs. The gist also includes specs that test the matchers. If the matcher names aren’t clear, check out the specs and hopefully that’ll reveal what they do. The algorithm itself is pretty simple – take a derivative of the array values and make sure that it’s always positive (or negative) in slope for increasing (or decreasing) checks.

The code is built for RSpec 2.something (at a minimum). They also assume your sort attribute is an number. I haven’t tried it with strings, but I think it wouldn’t take much to modify things to handle more general cases. Perhaps I’ll run into that problem and have an update. Until then, they are what they are.



Google Event Tracking with jQuery

As with lots of little bits of code, if they are useful, you may write and re-write them a several times in several projects before you take the time to package them up into a reusable chunk. That finally happened to me around Google Analytics Event tracking.

Almost every project that I’ve been on in the past year has added Google Analytics (simple page tracking) right up front, and then near the end of the work, realized that they also want event/click tracking. On this last project, I decided to make it right, and package it up.

I’ve built a jQuery plugin (https://github.com/rcode5/ga_event_tracker) that makes it easy to setup click tracking all over your site. The event tracker takes 3 parameters: category, action, and label. The plugin will either find these values as attributes on the tracked DOM element (using data-category, data-action and data-label), or use values passed in when the plugin is initialized, or evaluate functions passed in on initialization. This allows for a lot of flexibility. We were able to setup tracking for most pages on a full site with a couple lines of Javascript.

Assuming you add the class ‘trackit’ to every element you want tracked, you can have something like this:


And clicking all those elements will get tracked using the values found in the element attributes “data-category”, “data-action”, and “data-label”.

The source is available on Github with a few more examples. A test suite is included, in case you want to add your own extras and submit a pull request.

I hope this saves you a little time down the road, as I’m sure it will for me.

Handling Vanity URLs & Legacy Routes in Rails 3.2

Handling Vanity URLs is not a new problem. Website Owners want their URLs to be simple or catchy or elegant. For marketing purposes, they may need to be short(ish) and possibly temporary. Imagine a URL like


which in April points to /parties/april, then May points to nothing (May’s a dull month) and then in June it points to /parties/june. We need something that acts sort of like Rack::Rewrite but that looks into an easily editable chunk of data (read: database) for the list of paths and redirects.

If you comb the web (as I did) looking for Vanity URL solutions, most of them solve the issue for pretty URLs for a given Model/Controller pair. They solve the issue that numeric IDs are not informative, but words with hyphens can be more informative. If that is the problem you’re hoping to solve here, read no further. Go straight to FriendlyID (or similar).

If, instead, you’re trying to build a simple redirect/route-fallback system that allows for root level temporary Vanity URLs that can be managed within the running application (not by a static configuration file), then you’ve come to the right place.

First, we need a place to store the paths and redirects. Let’s start by adding a simple VanityUrl model

class VanityUrl < ActiveRecord::Base
  attr_accessible :active, :path, :url
  validates :url, link: true, presence: true
  validates :path, presence: true, uniqueness: true
  class << self
    def active
      where(active: true)

    def routes
      Hash[VanityUrl.active.map{|v| [v.path, v.url]}]

path is the path we will be watching for

url is the url we’ll redirect to, if we hit that path

active is a boolean (in case we want to easily turn the redirect off or on).

Looking ahead a bit, we’ve added an #active scoping method and #routes which returns all active VanityUrls. You’ll see where this is used shortly.

Exercise for the reader

Build controller and views to show/list/edit these guys in your application.

Now we want these to hit, only if other routes have not been matched. So at the end of our config/routes.rb file, we add a new catch-most route.

# in config/routes.rb

MyApp::Application.routes.draw do
  # your apps routes are defined here
  # then way down at the end... 
  # as the last defined route

  # this routes to the Vanity controller which handles the actual redirect
  # format:false allows us to recognize paths like '/this/old/page.htm
  match "/*vanity", to: 'vanity#routing', constraints: RoutingConstraints::VanityUrls.new, format: false

This last line is catch-most because it will only match if the constraint defined in RoutingConstraints::VanityUrls is hit. And with format: false is telling the router to ignore the any extension (as the comment says – so we can match page.htm if need be).

The RoutingConstraints::VanityUrls object needs to have a #matches? method (more in the Rails Routing Docs). So we can add the following:

# in app/lib/routing_constraints.rb
module RoutingConstraints
  class VanityUrls
    def matches?(request)
      routes = VanityUrl.routes
      vanity_url = request.params['vanity']
      if routes.has_key? vanity_url
          return true
        rescue URI::InvalidURIError
          # vanity url is no good
          return false

Notice here, we’re using the VanityUrl::routes method. Now, as routes are processed by Rails, *IF* they don’t match any of our normal routes *AND* they match one of our VanityURLs by path, *THEN* the router will pass them off to the VanityController#routing method which we told it to do in the routes.rb file.

Let’s add that method:

class VanityController
  def routing
    vanity_routes = VanityUrl.routes
    vanity_url = params['vanity']
    if vanity_routes.has_key? vanity_url
      redirect_to URI.parse(vanity_routes[vanity_url]).to_s and return
    raise "Unable to find requested VanityUrl for #{params}"

Notice, this is not the VanityUrlController which you’ve written to handle VanityUrls in the database. We make a new controller which will only have this one method to handle the rerouting of requests.

That’s basically it. If you’re app gets a lot of traffic, you’ll probably want to cache the VanityUrl collection (since it should be read-often and write-occasionally). Without a cache, every request is doing a full table retrieval of the VanityUrl table.

Initially, we added this feature to support true VanityUrls for a client e.g. http://example.com/party, http://example.com/temporary_coupon etc. And for this purpose, it works great. But it also saved our butts in a way we hadn’t forseen. The application we built was replacing an existing site almost entirely. When we actually launched, we starting getting piles of 404’s. The bits of the old site that were still up were asking for resources, images, stylesheets etc that were not duplicated in the new site. With a handful of VanityUrls, we were able to point those misdirected requests to the old assets (or wherever they needed to go) without modifying code or adding Rack::Rewrite directives. It made launch day a relative breeze.

acts_as_taggable_on + select2 = Super Easy Tags in Rails

I’ve had two projects in the past little bit where I wanted user input tags for objects. I also wanted to have typed tags – tags of different categories applied to the same object. I found that acts_as_taggable_on (Rails gem) and select2 (jQuery plugin) work really nicely together and make this an easy task.

Let’s assume that we have an ActiveRecord model for a Post. Follow directions to install acts_as_taggable_on (add to Gemfile, initialize and run migration). Then add two different tag types to Post like this:

class Post < ActiveRecord::Base
  acts_as_taggable # basic usage - gives us 'tags'
  acts_as_taggable_on :topics

With this, the Post model is ready to store tags and topics.

irb> Post.first.tags << 'super fun'

To push this to the users, we want a nice UI for users to enter tags/topics. This is where Select2 comes in. In your form for adding/editing posts, add two input boxes, one for topics and one for tags

# using haml templating
  = f.label :tags
  = f.text_field :tag_list
  = f.label :topics
  = f.text_field :topics_list

Then attach the select2 plugin to the form elements. Make sure you’ve got jQuery and select.js included on the page.

// in your javascript
$(function() {
  $('.input-row #post_topics_list, .input-row #post_tags_list').select2({tags:[]})

And you get a beautiful input box with handy tag handling.
Tags and Topics form

If you haven’t found it, there’s a nice package that puts select2 in the asset pipeline – https://github.com/argerim/select2-rails.