Flexible Rails Modeling with ActiveRecord::Store – Part II

The quick recap – we need a flexible model structure to handle similar but distinct Blocks (read Part I).

Now that we’ve got the base model setup and the ability to define model attributes with simple YAML files, we need to figure out how to enter them in the system (controllers/forms) and how to display them (views).

Controllers

We first require that when we create a new Block we specify the desired type. In the controller:

class BaseBlocksController < ApplicationController
  before_filter :type_is_required, :only => [:index, :new]

  protected
  def type_is_required
    message = "You must specify a block type."
    redirect_to('/', :flash => {:error => message}) unless params[:type]
  end
end

Then we can define our new endpoints to this type to build a new Block.

  def new
    @block = Block.new :block_type => params[:type]
  end

We also need to update our #update and #create methods to use the :without_protection param when building our new objects. This will avoid the ActiveModel::MassAssignmentSecurity::Errorthat we would get since our custom attributes don’t have attr_accessors defined in the normal way.

def create
    @block = Block.new(params[:block], :without_protection => true)
    if @block.save
      redirect_to @block, notice: 'Block was successfully created.'
    else
      render action: "new"
    end
  end

  def update
    @block = Block.find(params[:id])
    if @block.update_attributes(params[:block], :without_protection => true)
      redirect_to @block, notice: 'Block was successfully updated.'
    else
      render action: "edit"
    end
  end

You’ll notice, in the model initialization (in Part I) that we protected our database defined fields in the #add_block_type so we can’t accidentally override database fields with the custom fields.

Forms

We’ve already written some model code that figures out how to handle the dynamic fields using ActiveRecord::Store. With a little help from ActionView::Helpers::FormBuilder and SimpleForm, we can setup dynamic form fields pretty easily.

To build custom forms, we write a small helper:

module BlocksHelper
  def render_dynamic_form_fields(form, block, opts={})
    s = ''
    begin
      (Block.block_types[block.block_type] || {}).each do |k,v| 
        opts.merge!({as: v[:datatype], label: v[:display_name]})
        s += form.send('input', k, opts)
      end
      s.html_safe
    rescue Exception => ex
      Rails.logger.warn 'Failed to add form fields for block [%s]' % block.inspect
      Rails.logger.warn 'Check the YAML file for that block and make sure it\'s properly configured'
      Rails.logger.warn ex
      ''
    end
  end
end

With this helper in place, we can update our Block form (views/blocks/_form.html.slim) to look like this:

= simple_form_for(@block, :html => {:class => 'form-horizontal' }) do |f|

  .form-inputs
    = f.hidden_field :block_type
    = f.input :title, placeholder: 'Insert a title'
    = render_dynamic_form_fields(f, @block)

  .form-actions
    = f.button :submit, class: 'btn btn-primary'
    '
    = link_to 'Back', blocks_path, class: 'btn'

Our new helper will build input fields for our custom fields using their datatype (defined in the block YAML) to figure out what kind of field it is.

Views

Finally, to show our new widget, we use a custom template that knows about our custom fields. For this example, I put the following slim template in views/blocks/_rock_block.slim

dl
  dt Title
  dd= @block.title
  dt Rock
  dd= @block.rock
  dt Roll 
  dd= @block.roll

And our corresponding show template simply renders that partial (using a model method that derives the template name from the block type)

= render @block.block_type.tableize.singularize

Tada

That pretty much does it. We’ve got custom blocks which we can define easily with a simple YAML file. They can have custom fields. They all sit in the same database table (kind of like STI). We have a single controller and single edit form with some smarts to draw the custom fields. And each block gets a custom template to render itself which knows about the custom fields.

If you want to try it out, most of the code here was taken from my sample app, custo_blocks, available on github.

Flexible Rails Modeling with ActiveRecord::Store – Part I

The current project I’m on needed a pretty flexible model structure that could support something that acted kind of like a Block or a Widget.

We wanted it to be easy to add a new type (which might have new fields and a new view), but not require the need for a new model, controller, full set of views etc. Additionally, we wanted all the widgets to sit in the same table so we could more easily present them to a user for selection, creation, and modification.

Initially, I thought Single Table Inheritance(STI) might be the answer (nice write up here, by Alex Reisner). But I quickly realized that STI would mean every time we want a new field for a new block type, we’d be running a migration.
And if that field is unused by the other block types, then we’re adding more logic in views and edit screens to manage this new field for this one block type.

Using separate models for each new block seemed similarly heavyweight. It would alleviate the logic for special fields based on block type. But it would mean lots of duplicate code as we add new model/controller/view code for each new type.

The method we came up with was to setup a model with a flexible field that is serialized and stored using ActiveRecord::Store. For any of you who’ve used the SpringFramework in Java, it has a similar feel. It allows us to flexibly add fields as necessary based on the block type and still share code for forms, controllers and views.

We start by setting up a base model (for this discussion, we’ll be building a Block). This model can be initialized with a minimal set of fields that you know will be universal for all Blocks. Let’s start with one field: title. We’ll also include a field to manage type: block_type. We don’t use type only because Rails has reserved that field name for STI models. We also add a text field called extensions. This is where we’ll stick all the custom fields.

# -- db/migrate/<date>_create_block.rb --
class CreateBlock < ActiveRecord::Migration
  def change
    create_table :blocks do |t|
      t.string :title
      t.string :block_type
      t.text :extensions
      t.timestamps
    end
  end
end

The model looks as you might expect. Using ActiveRecord::Store, we define the extensions field to be a ‘store’.

# -- app/models/block.rb --
class Block < ActiveRecord::Base
  RESERVED_FIELDS = [ :title, :block_type, :extensions ]
  attr_accessible *RESERVED_FIELDS

  store :extensions
end

You’ll see later why we’ve setup the RESERVED_FIELDS constant.

We’ll define our custom blocks using YAML. We can add a new block type by dropping a YAML in the config/blocks directory.
It might look something like this:

# -- config/blocks/rock_block.yml --
rock:
  display_name: Rock
  type: string
roll:
  display_name: Roll
  type: text

With this in place, we’ve just made our first custom block.

Now we tell the model to figure out what blocks it can handle and what their custom fields are. We add the following to the Block definition in models/block.rb.

  # determine possible block types by reading config files
  cattr_accessor :block_types

  @@block_types = {}

  def self.add_block_type key, block_data
    @@block_types[key] = HashWithIndifferentAccess.new(block_data)
    @@block_types[key].each do |k,v|
      unless RESERVED_FIELDS.include? k.to_sym
        store_accessor :extensions, k.to_sym
      end
    end
  end

  # initialize
  Dir.glob(File.join(Rails.root, 'config', 'blocks','*_block.yml')).each do |f|
    puts "Reading #{f}"
    begin
      key = File.basename(f).gsub(/\.yml$/,'').classify
      add_block_type key, YAML.load(File.open(f))
   rescue Exception => ex
      puts ex
      Rails.logger.warn "Failed to import block #{f}"
      Rails.logger.warn ex
    end
  end  

Now we can start constructing RockBlocks. Fire up a Rails console:

irb> block = Block.new(block_type: "RockBlock", title: "Sabbath")
=> #<Block id: nil, title: "Sabbath", block_type: "RockBlock", extensions: {}, created_at: nil, updated_at: nil>
irb> block.rock = "granite"
=> "granite"
irb> block.roll = "it's what wheels do"
=> "it's what wheels do"
irb> block.save
   (0.1ms)  BEGIN
  SQL (48.9ms)  INSERT INTO "blocks" ("block_type", "created_at", "extensions", "title", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"  [["block_type", "RockBlock"], ["created_at", Fri, 23 Nov 2012 23:03:40 UTC +00:00], ["extensions", "---\n:rock: granite\n:roll: it's what wheels do\n"], ["title", "Sabbath"], ["updated_at", Fri, 23 Nov 2012 23:03:40 UTC +00:00]]
   (0.7ms)  COMMIT
=> true
irb> Block.last.roll
  Block Load (0.9ms)  SELECT "blocks".* FROM "blocks" ORDER BY "blocks"."id" DESC LIMIT 1
=> "it's what wheels do"

Note: going this route means that the custom fields will not be easily searchable since they’re stored in a serialized field. As long as that is not going to impact your application, then this way can make things very flexible.

To get controllers/views to work smoothly we did a little more magic which i wrote up in the next post.

[read more | check out the code]

pry it open with pry

Wow. Did i already mention how i love pry?

I like crowbars and hammers and this is the perfect tool to get into the guts of your code – wherever you want to. Follow these simple steps.

Add to your Gemfile (assuming ruby>1.9.2)

group :development do
  gem 'pry'
  gem 'pry-debugger'
end

Then pry your code open.
Given a method

def rock_an_item(item)
   do_this(item)
   do_that(item)
end

add binding.pry,

def rock_an_item(item)
   binding.pry
   do_this(item)
   do_that(item)
end

Run again and voilá

From: testit.rb @ line 10 Object#rock_an_item:

    10: def rock_an_item(item)
 => 11:   binding.pry
    12:   do_this(item)
    13:   do_that(item)
    14: end

[1] pry(main)> 

you’re dropped right into an irb console at the point of the break. With pry-debugger installed you have step, next, continue methods (and more). Investigate your item, check out your environment, when it’s all done, continue. Fix the bug, remove binding.pry and move on. Super simple.

Sadly, the debugger tie-in and 1.8.7 is not quite as seamless. I’m still working on that. But assuming most of your projects have moved into the 1.9.x realm, you should be in good shape.

Moving to rbenv from rvm

RVM has been a great tool for managing different projects with different ruby versions – for you python folks you may be aware of virtualenv. But that has not come without it’s issues. Setting up Jenkins CI and RubyMine can require some extra setup steps that are a little frustrating sometimes. I’ve started moving all my development environments to rbenv and have been very happy. rbenv is a bit less intrusive and simpler to manage.

Here are the steps I took to make that move. Hopefully this will make things easy when you decide to make the switch.

The steps here assume you’re on a Mac with homebrew. If not, you may need to do some installation from source.

First – blow away rvm.

rvm implode
rm ~/.rvmrc
rm -r ~/.rvm

Install rbenv.

# install rbenv with homebrew
brew install rbenv

Install ruby-build. This plugin gives rbenv the install action to install rubies.

git clone git://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build

Install rbenv-bundler. This will make rbenv bundler aware. Without, you may run into permissions issues because without it ‘gem install bundler’ tries to install into /usr/bin.

git clone git://github.com/carsomyr/rbenv-bundler.git ~/.rbenv/plugins/bundler

Add shims to every shell by adding this line to your .profile or .bashrc

eval "$(rbenv init -)"

And while you’re in there, you might also remove references to rvm. Mine was trying to run ~/.rvm/script/rvm and had ~/.rvm/bin in the PATH. I removed all that and started with a clean shell for the rest.

Install your rubies

rbenv install 1.8.7-p334
rbenv install 1.9.2-p180
# ... add others that you use here

You can run

rbenv install --list

to figure out what rubies ruby-build knows about.

Pick a system ruby

rbenv global 1.9.2-p180

Setup bundler. I’m not certain that you need to do this for each ruby, but I had problems without doing that. So for good measure, and since it only has to happen once:

rbenv shell 1.8.7-p334
gem install bundler
rbenv shell 1.9.2-p190
gem install bundler
rbenv rehash

Tell rbenv that you’ve got new stuff

rbenv rehash

Set rubies for your different levels of usage. You can set global, local, and shell. I primarily use global and local, where global is for the machine, and local is for the projects that I work on.

cd /projects/my-1.8.7-project/
rbenv local 1.8.7-p334
cd /projects/my-1.9.2-project/
rbenv local 1.9.2-p190

That should be it.

You’ll need to re-bundle on your projects but from there on out you should be happily moved onto rbenv.

I’ve only just begun the process. For my development machines, this has been an easy switch. I’m hoping it will help me clean up my config for Jenkins.

Also, keep in mind that if you share projects with RVM users, no problem. These two systems, though they don’t work well together on the same machine, will not conflict across boxes. So you can safely run rbenv while your co-workers use rvm.

Custom Image Upload Modal with Bootstrap-Wysihtml5

My current project is, among other things, a custom CMS.

In several places, we needed a Wysiwyg editor with an image uploader that would pull from our application’s media library.
We started off with bootstrap-wysihtml5. This project is a nice Bootstrap wrapper on wysihtml5 which is an open source html5 rich text editor project. Out of the box, the bootstrap-wysihtml5 works nicely. But it’s image insert functionality simply asks for an URL which it wraps in an image tag. We wanted it to pull that URL from our in-application library of images.

To achieve this, I modified the editor to allow inserting custom templates for the modal, and hooked up an Ajax call (at /attachable_images) that would pull the images (as json) allowing the dialog for inserting images to display and choose from our image library. The changes that allow this customization have been merged into the project (http://jhollingworth.github.com/bootstrap-wysihtml5/). To enable this, we added the following on our side:

First, we setup the customTemplates that we want to use for the image modal:

// override options
var wysiwygOptions = {
  customTemplates: {
    image: function(locale) {
      return "<li>" +
        "<div class='bootstrap-wysihtml5-insert-image-modal modal hide fade'>" +
        "<div class='modal-header'>" +
        "<a class='close' data-dismiss='modal'>&times;</a>" +
        "<h3>" + locale.image.insert + "</h3>" +
        "</div>" +
        "<div class='modal-body'>" +
        "<div class='chooser_wrapper'>" +
        "<table class='image_chooser images'></table>" +
        "</div>" +
        "</div>" +
        "<div class='modal-footer'>" +
        "<a href='#' class='btn' data-dismiss='modal'>" + locale.image.cancel + "</a>" +
        "</div>" +
        "</div>" +
        "<a class='btn' data-wysihtml5-command='insertImage' title='" + locale.image.insert + "'><i class='icon-picture'></i></a>" +
        "</li>";
    }
  }
};

These options are used when you fire up the editor:

$(function() {
 $('textarea.wysiwyg').each(function() {
    $(this).wysihtml5($.extend(wysiwygOptions, {html:true, color:false});
  });
});

Then we override the insertImage functions defined in the editor.

var xhrFetchingImages;

wysiHelpers = {
  getImageTemplate: function() {
    /* this is what goes in the wysiwyg content after the image has been chosen */
    var tmpl;
    var imgEntry = "<img src='<%= url %>' alt='<%= caption %>'>";
    tmpl = _.template("<div class='shrink_wrap'>" +
                      imgEntry +
                      "</div>" +
                      "<p class='credit'><%= caption %></p>" +
                      "<hr>");
    return tmpl;
  }
}
bootWysiOverrides: {
  initInsertImage: function(toolbar) {
    var self = this;
    var insertImageModal = toolbar.find('.bootstrap-wysihtml5-insert-image-modal');
    var urlInput = insertImageModal.find('.bootstrap-wysihtml5-insert-image-url');
    var insertButton = insertImageModal.find('a.btn-primary');
    var initialValue = urlInput.val();
  
    var chooser = insertImageModal.find('.image_chooser.images'),
    /* this is the template we put in the image dialog */
    var optionTemplate = _.template(
      "<tr><td data-type='image' data-caption='<%= title %>' data-url='<%= urls.content %>'>" +
        "<img src='<%= urls.icon %>'>"+
        "<div class='type'>Image</div>" +
        "<div class='title'><%= title %></div>" + 
        "<div class='caption'><%= caption %></div>" + 
        "</td></tr>");

    var helpers = wysiHelpers;
    
    // populate chooser
    // TODO: this get's called once for each wysiwyg on the page.  we could 
    //       be smarter and cache the results after call 1 and use them later.
    if (!xhrFetchingImages) {
      $.ajax({
        url:'/attachable_images',
        success: function(data) {
          xhrFetchingImages = false;
          // populate dropdowns
          _.each(data, function(img) {
            chooser.append(optionTemplate(img));
          });
        }
      });
    }

    var insertImage = function(imageData) {
      if(imageData.url) {
        var clz = 'image_container';
        var doc = self.editor.composer.doc;
        var tmpl = helpers.getImageTemplate(!!imageData.caption);
        var chunk = tmpl(imageData);
        self.editor.composer.commands.exec("insertHTML", chunk);
      }
    };
    
    chooser.on('click', 'td', function(ev) {
      var $row = $(ev.currentTarget);
      insertImage($row.data());
      insertImageModal.modal('hide');
    });
    
    insertImageModal.on('hide', function() {
      self.editor.currentView.element.focus();
    });
    
    toolbar.find('a[data-wysihtml5-command=insertImage]').click(function() {
      var activeButton = $(this).hasClass("wysihtml5-command-active");
      
      if (!activeButton) {
        insertImageModal.modal('show');
        insertImageModal.on('click.dismiss.modal', '[data-dismiss="modal"]', function(e) {
          e.stopPropagation();
        });
        return false;
      }
      else {
        return true;
      }
    });
  }
};

$.extend($.fn.wysihtml5.Constructor.prototype, bootWysiOverrides);

Now we’re ready to roll. This code will now populate the chooser table with a list of images that come from our system (by way of the Ajax call) and the optionTemplate template. When you click on the image, it’s inserted into the editor window using the DOM defined in the getImageTemplate (which is customized depending on whether or not the image has a caption).

Though the overrides may appear a bit complex, you’ll notice that they look a lot like the original bootstrap-wysihtml5 source where he handles inserting an image – the primary mod is populating the dialog with something more than a simple input box.

Another thing to notice, in the initial options, we add the customStyles and customTags entries. These are to prevent the HTML cleanser from ripping out styles and tags that we are using in our new image DOM. Without those definitions, your new elements will get cleansed – possibly to the point of deletion.

With these simple-ish modifications, we were able to get our application images into the content blocks with a very nice UI.

See it in action (and get the source code) here.
Much thanks to James Hollingworth for merging in my pull requests so I can share this enhancement with others.

UPDATE (4/11/2013): it looks like someone’s done one better (at first glance). You may want to check out http://mindmup.github.io/bootstrap-wysiwyg/. He’s got a nice image upload image picker right out of the gate.