I wanted to build a page with nice in-place add/delete of some items, and I wanted something nicer-looking than the usual Javascript alert popup for confirmations. Here’s what I came up with; this is in a Rails-powered app using (of course) JQuery. I’ve used Impromptu in the past as a powerful JQuery-based popup library, and I used it again in this case. I won’t go through the setup steps for it, since the site provides a good walk-through.

I started with a basic table showing the items; for the sake of this example, let’s say these are books in my library, displayed in a table. I want to remove books and add books via AJAX, and update the table dynamically.

<table id="books" width="90%" class="booklist">
    <th>Book #</th>
  <% @library.books.each do |book| -%>
    <tr id="<%= book.number %>" class="<%= cycle("even", "odd") -%>">
      <td><%= book.number %></td>
      <td><%= book.title %></td>
      <td align="center">
        <a href="#" onClick="removeBook('<%= book.number %>');">
        <%= image_tag('delete.png', :width => '20') %>
  <% end -%>

<form id="add_form" action="#">
  <input type="text" id="newbook" size="20" />
  <%= image_tag 'button-add.png', :id => 'add_button', :width => '20', :style => 'vertical-align:middle' %>

Pretty straightforward: After the header row, we loop through the library, and for each book we output a row. Note that we use the book’s number (pretend with me that it’s something like the ISBN that has no spaces or weird characters) as the id of each row — this number is also passed in to the removeBook() function when someone clicks the delete button, which allows us to dynamically remove the table row.

After the table, we have a little form for adding a new book — this assumes that we already have a database of books, and we’re just going to look up the new book by number, and add it to our library. All we have here is a button image, to which we will attach a JQuery click action. It would be nice to use the JQuery click action on the remove button as well, but it complicates things when it comes to determining which row was clicked on (yes, we could put the number in as an id on the image and look it up, which I’ll probably look at doing later).

So, when someone clicks on the remove button, what happens? The removeBook() function is called:

  function removeBook(number) {
    $.prompt('Are you sure you want to remove book #' + number +
        ' from the library?<input type="hidden" name="num" id="num" value="' + number + '"/>',{
      callback: removeCallback,
      buttons: { Yes: 'Remove', No: 'Cancel' }

  function removeCallback(button, msg, formvals) {
      if (button == 'Remove') {
        // Call server to remove the book
            type: "PUT",
            url: "/libraries/<%= %>.js",
            data: ({remove : 1, book_num : formvals.num}),
            success: function(msg){
                // Remove the row from the table
                $('#' + formvals.num).remove();
            error: function(response, textStatus, errorThrown){
                var msg = (response.status == 403) ? response.responseText : "unknown error.";
                $.prompt('Error removing book from library: ' + msg);

The removeBook() function uses the Impromptu library to display a message via $.prompt(), specifying the removeCallback() callback function and two buttons. If the user clicks the “No” button, the callback won’t do anything. If they click the “Yes” button, then the callback will make a JQuery AJAX call to the server.

The Impromptu callbacks receive three parameters: the button clicked, the message if any, and the values of any form fields, if any. We were tricky and placed a hidden form field into the popup in the removeBook() function, called “num”. Note that Impromptu uses the NAME of the field, not the ID, for looking up the values; in this case both are “num” to make life easy. We then check the button value to see if it’s “Remove”, and we get the book’s number from the form using formvals.num.

Being somewhat RESTful, we do a PUT to the libraries controller with the ID of the library we’re changing. Note that we specify “.js” for the call, so that our controller will respond appropriately, since this is calling the update() action. As the AJAX call’s data, we pass in “remove” as a flag, and “book_num” to tell it which book to remove. If everything goes well, the success function is called. All it does is to remove the row from the table, using the fact (as noted earlier) that the row id is the book number.

The relevant lines from the controller’s update() method:

  # Have we been called to remove a book?
  if (params[:remove])
    success = @library.remove_book(params[:book_num])
  elsif (params[:add])
    # We've been called to add a book
    success = @library.add_book(params[:book_num])
    if (success)
      # Now let's look up the book so we can get the title for display
      book = Book.find(params[:book_num])
      if (book)
        result = { "number", book.number, "title", book.title}
        # It's a book we don't have info about, but that's okay
        result = { "number", params[:book_num], "title", "Unknown"}

  . . .  # Do other work as needed

respond_to do |format|
  if (success)
    flash[:notice] = 'Library was successfully updated.'
    format.html { redirect_to(@library) }
    format.xml  { head : ok }
    format.js   { render :text => "OK" }
    format.json { render :json => result }
    format.html { render :action => "edit" }
    format.xml  { render : xml => @library.errors, :status => :unprocessable_entity }
    format.js   { render :text => "Failed to save update", :status => 403 }
    format.json { render :json => { "msg", "Failed to save update" }, :status => 403 }

(Note that in lines 25 and 30 I had to add a space after the colons, because WordPress is stupid and for some reason thinks that I want damn smilies inside a sourcecode block. Uh, okay, sure.)

This is treating a book removal as a specialized variation of an update; an argument could certainly be made that instead it should be a new action. My jury’s still out, but there are things I like about doing it this way, including keeping the routes simple and encapsulating all library update-related activities in one place. Regardless, if the removal goes well, we return an “OK” — if not, we return a failure message, via the format.js within the respond block. Notice that in the Javascript above, the error-handler will display response.responseText if the response code is 403, and here we set the status to 403 if an application error occurred during the update. In that case the text we return will be displayed.

While you’re looking at this, check out the JSON responses, because we’ll be using those for adding a new book to the library, below.

That’s it. When a user clicks on the Remove button, it will invoke the removeBook() function, which will display a nice-looking confirmation popover using Impromptu. If the user confirms, then the callback function will make the AJAX call. The controller will remove the book from the library, and return a text “OK” if all goes well. The success function will then remove the associated row from the table, and we’re done!

On to adding a new book, which is trickier, though it uses much the same ideas of course. A user enters a book number into the form and clicks the button. What makes something happen then? The JQuery click function we’ve associated with it. Here’s the Javascript:

  $('#add_button').click(function() {
      var add_number = $('#newbook').val();
      if ((!add_number) || (add_number == '')) {
          return false;
      // Call server to add the book to the library
          type: "PUT",
          url: "/libraries/<%= %>.json",
          data: ({add : 1, book_num : add_number}),
          success: function(data){
              // Add a new row to the table
              addTableRow('#books', data['number'], data['title']);
              $('#newbook').val('');   // Clear the input field
          error: function(response, textStatus, errorThrown){
              var msg = (response.status == 403) ? response.responseText : "unknown error.";
              $.prompt('Error adding book to library: ' + msg);

  function addTableRow(book_table, book_number, book_title){
      var row_class = $('tr:last', book_table).attr("class");
      var new_class = (row_class == 'odd' ? 'even' : 'odd');
      var tds = '<tr id="' + book_number + '" class="' + new_class + '">';
      tds += '<td>' + book_number + '</td>';
      tds += '<td>' + book_title + '</td>';
      tds += '<td align="center">' +
        '<a href="#" onClick="removeBook(\'' + book_number +
        '\');"><img src="/images/delete.png" width="20"></a></td></tr>';
      if($('tbody', book_table).length > 0){
          $('tbody', book_table).append(tds);
      }else {

Whew, that’s a fair amount of Javascript; but it’s pretty straightforward, nonetheless. The first block is of course the click function that we attach to the button. When a user enters a number into the text field and clicks the button, this function will be called, and the first thing it does is to grab the value from the text field. And excuse me while I take a brief moment to rant in a minor way about the fact that the function is called val() instead of value(). Really, are two more letters too much to ask in exchange for better clarity? Every single time I have to write something like this I start with value() and waste a few minutes reminding myself that it’s shortened for no apparent reason. Okay, rant over, sorry.

We do a quick check to make sure that there’s actually something in the text field and return if not. Otherwise, we make our AJAX call, to the same URL as the remove function but with parameters “add” and “book_num”. If you go back now and look at the controller code again, you’ll see where it checks for params[:add] and, if it’s there, it adds the book to the library. After that it does a quick query to grab the book, so that it can return the title. It actually makes a Hash called “result” with the number and the title, which is then used in the render :json => result line. Within render, it will actually JSONize the Hash and send it back to the caller.

Just to prevent any confusion, I’ll take a second here to note that in most cases you probably won’t need to do the Book.find call that’s shown here, because you’ll likely be using ActiveRecord and you might be able to look in the library instance or use some other workaround to have the book available. In my case (remembering that this is a ‘cleansed’ version of my real work, which isn’t about books and libraries at all), I’m using Redis as the data store so adding a book to a library is actually a matter of adding its key to a set, so I don’t have the actual object until I do the find — which is super-fast anyway.

Okay, so if all goes well, the success function is called, which needs to add the new row to the table (you can see that it also clears the value of the text field so it’s empty again). To add the row, it calls the addTableRow() function, passing in the table, book number, and title. To give credit where it’s due, this function is a stripped-down version of the one shown in this blog post, which was quite helpful. This version does some things specifically for this purpose, and isn’t as generic as the original.

The first two lines determine what the class should be for the row, because the table alternates ‘even’ and ‘odd’ in order to have a new striped appearance. Here we grab the class of the last row, and then set our new_class to be whatever the current last row isn’t. That is, if the last row is ‘even’ then our new one will be ‘odd’ and vice-versa. Then we build the HTML for our new row, carefully setting the id of the row to the book number, inserting the number and title, and then adding on the somewhat ugly final table element to have the Remove button with its proper onClick. The last few lines determine whether the table has a tbody or not, and appends our new row accordingly to the body or the table itself, so this works if we start with an empty library.

And that is that — when a user enters a number and clicks the add button, JQuery has attached the click event to it, so our “add” function is called, which gets the value, does the AJAX call which does the server-side adding and returns the book number and title as JSON. Our function takes that and gets a row added to the bottom of the table accordingly.

There’s only one minor improvement which I will make to this shortly, though it’s a minor use case: when a user deletes a book, the corresponding table row is removed but the classes of the remaining rows aren’t being updated. That means we lose our nice even/odd striping. As an exercise for the reader, I’ll let you add the code to the success function inside removeCallback() to iterate the table rows and set the class attributes after removing the row.

I hope this proves useful to some folks out there. The combination of Impromptu for pretty confirmation popovers (and error messaging), with in-place add/remove, provides a really nice user experience. And the patterns here end up being useful all over the place, so you’ll likely want to do the next step, which is to make this code more generic and put it into partials that you can re-use in multiple pages. Enjoy.

I ran into a bit of strangeness when trying to get RESTful forms working with non-ActiveRecord models — in this case, models that I was using to represent data stored in Redis. It turned out to not be straightforward. I wanted to create a simple form for creating a new object. Because I created things using the generators, I had a skeleton page already (even if I’m not using ActiveRecord it’s still a bit of a time-saver):


<h1>New Thing</h1>

<% form_for(@mymodel) do |f| %>
    <%= f.error_messages %>
      <%= f.label :name %><br/>
      <%= f.text_field :name %>
      <%= f.label :a_link %><br/>
      <%= f.text_field :a_link %>
      <%= f.submit 'Create' %>
<% end %>

Pretty ordinary. What happens with this, though, is that when you load the page, it calls the controller’s new() method first. The page expects that method to pass in an instance of the model, @mymodel. The form_for() call then ends up introspecting that instance to determine what to do with it. When the form is submitted, then the controller’s create() method is called…or is it? Actually, if form_for() determines that the instance is brand-new, then yes, create() is called. Otherwise, the controller’s update() method is invoked.

This is what was throwing me off. I put things together, and discovered that update() was being invoked instead of create. Why? Well, I ended up having to dig into the Rails source code to find out, specifically in the apply_form_for_options() method. The trick turns out that it really wants to call a method on the model called new_record?() to determine whether or not the instance is brand new. If the model doesn’t respond to that method, or if it returns false, then form_for() decides that this must be an existing instance, so it sets up the form to invoke the update() action.

Once figured out, it was easy enough. I added to my model:

  def new_record?
    (id.nil? || (id == 0))

And that fixed the problem. If my instance has a nil or zero id, then it’s new. I do have to point out that while this form_for() behavior is perhaps convenient for coding, it’s not a very efficient thing to do. The side effect is that every time the form is loaded a model instance is created, which is then thrown away after the page is delivered; clearly, that’s a waste. It would make more logical sense to allow form_for() to operate on a nil object, treating it as obviously new.

In any case, hopefully this info will help save someone else some time.

After spending time to get some data into Redis (as documented in some of my previous posts here), I not surprisingly wanted to make the data searchable. After looking around at some of the full-text search solutions available for Ruby, I really liked the look of Sunspot. Well-presented, well-designed, and it even has decent documentation. It uses Solr underneath, which is a very respectable search engine, so that’s all good. Of course, it didn’t take me long to discover that the sunspot_rails plugin makes things drop-and-go when using ActiveRecord, but those of us branching off into alternatives have to put in more effort. Hence, I’ll document my findings here to hopefully make it easier for others.

I won’t bother going into the details of getting things set up, as the Sunspot wiki does a fine job of that. Suffice it to say that we install the gem (and the sunspot_rails gem if you’re going to have some ActiveRecord models as well), start the Solr server, and that’s about it. We’ve got Redis already going, right? So now it’s time to get our model indexed and searchable!

There are a few steps that we need to follow to make this happen. First, we put code in the model to tell Sunspot what fields should be indexed, which ones are just for ordering/filtering, and which ones should be stored if desired for quicker display:

class Book
  require 'sunspot'
  require 'sunspot_helper'

  # Pretend some attributes like number, title, etc are defined here

  Sunspot.setup(Book) do
    text :number, :boost => 2.0
    text :title, :boost => 2.0
    text :excerpt
    text :authors
    string :title, :stored => true
    string :number, :stored => true
    date :publication_date

  def save
    book_key = "book:#{number}:data"
    @redis[book_key] = json_data
    @redis.set_add 'books', number
    # Make searchable
    Sunspot.index( self )

  def self.find_by_number(redis, number)

First, note that we need to require 'sunspot' to get access to the Sunspot class. This isn’t required for ActiveRecord models, but since we’re on our own, we have to specify that. Then, we call setup, passing the name of our model. In the code block, we specify a few text fields: the number, title, excerpt, and authors. Those fields will be indexed and searchable. Then we specify title and number again as strings, asking that they be stored for quicker retrieval. This is so we can display just that data without fetching the whole object, if we want — I won’t get into the details of doing that here because, well, fetching the objects in Redis is so fast that I found it didn’t matter. Last, the publication date is also listed, so we can filter and order by it if we want.

In our save() method, after we store a book in Redis, we tell Sunspot to index it, and commit the updated index. So far, so good. In theory, we should be able to create a Book, save it, and then search for it. Alas, if this were an ActiveRecord model we’d be pretty much done (and wouldn’t even have to do the index/commit part because those are automagically triggered on create and update). Unfortunately, we have some harder work ahead of us.

Sunspot uses what it calls “adapters” to tell it what to do when it wants to identify an object, and when it wants to fetch an object given an id. We have to provide the adapters for our model. To give credit where it’s due, this Linux Magazine article helped me figure out what to do, and then reading through the Sunspot adapter source code filled in the blanks. If you look back at our model, you’ll see that it requires ‘sunspot_helper’. That’s where we’ll put our adapters:


require 'rubygems'
require 'sunspot'

module SunspotHelper

  class InstanceAdapter < Sunspot::Adapters::InstanceAdapter
    def id
      @instance.number  # return the book number as the id

  class DataAccessor < Sunspot::Adapters::DataAccessor
    def load( id ), id)))

    def load_all( ids )
      redis = { |id|, id))) }


So, what’s going on here? We provide two adapters for Sunspot: the InstanceAdapter, and the DataAccessor. The InstanceAdapter just provides a method that returns the ID of the object. Easy enough, we just return the book’s number, which is the unique identifier. The DataAccessor has to provide two methods, load() and load_all(), that take an id and a list of ids, respectively, and expect objects back. In my case, the objects are serialized JSON, so we just call our find_by_number() method to get each object, call JSON.parse() to get the Hash of data, and construct a new Book object. (Note: obviously this requires having an initializer that can take a Hash and create the object, which I’ll leave as an exercise) Now we just register our adapters, by adding a couple of lines of code right before the call to Sunspot.setup():

  Sunspot::Adapters::InstanceAdapter.register(SunspotHelper::InstanceAdapter, Book)

  Sunspot::Adapters::DataAccessor.register(SunspotHelper::DataAccessor, Book)

Now we should be good to go, right? Okay, we construct a Book object, and call save…then search for it:

b ={ "number" => 8888888, "title" => "My test title"})
=> #<Book:blahblah...
=> nil
search = { keywords 'test' }
=> <Sunspot::Search:{:rows=>1, blahblah…
r = search.results
=> [#<Book:blahblah...
=> "My test title"

And we’re good! Congratulations. So now we want to add the search capability to our controller, right?

# In a view, put in a search form. I have a little search image, so excuse the image_submit_tag:
<% form_for(:book, :url => { :action => "search" }) do |f| %>
      <%= f.label "Search for:" %>
      <input type="text" name="searchterm" id="searchterm" size="20">
      <%= image_submit_tag('search.png', :width => '30', :alt => 'Search', :style => 'vertical-align:middle') %>
<% end %>

# Now in the controller. Note the pagination, which is why we store the search in the session,
# so we can grab it out again if they click forward/back through the pages.
  def search
    @search_term = params[:searchterm] || session[:searchterm]
    if (@search_term)
      session[:searchterm] = @search_term
    page_number = params[:page] || 1
    search = do |query|
      query.keywords @search_term
      query.paginate :page => page_number, :per_page => 30
      query.order_by :number, :asc

    @books = search.results

# And then in our search view, display the results:
<% @books.each do |book| %>
    <li><%= book.number %>: <%= book.title %></li>
<br />
Found: <%= @books.total_entries %> - <%= will_paginate @books %>

Yes, Sunspot is so cool that it integrates automatically with will_paginate. So, looking through the above, we have a form that posts to our action (assuming you set the routes up, which you did, yes?). The action then takes the searchterm parameter if it’s there, or extracts it from the session if it’s not there. Note that this is not robust code — if it’s called with no parm and nothing in the session, it will end up searching for an empty string, which will return every book. In any case, we store the search term in the session, so that when someone clicks through to page 2, we can re-run the search to get the second page. The more important code here, though, is the call to search.

I will give a thousand thanks to this blog post, specifically the fourth item! I was doing this:

    search = do
      keywords @search_term

And it didn’t work — it was fetching every object, even though I knew that @search_term was getting set properly. As that blog post notes, though, the search is done in a new scope, so this didn’t work. The code I showed above, using the query argument, fixes that problem. It certainly took me a while to figure that out, though, because nothing is said about it anywhere in the examples in the Sunspot wiki.

So now you should be all set. Put “test” into the form, submit it, the controller will do the search, return the book, and your view will list it. You are searching! Not so bad, and the fetches from Redis are so fast that the whole thing really speeds along. Pretty simple free-text search against any objects that you put into Redis.

    A Warning

I had one other hitch when I was working on this, which mysteriously went away. I hate that. So, in case someone else encounters here, I wanted to document the issue. When I got the adapters in place for the Book model, and tried to work with it, I got an error saying that there was no adapter registered for String. I was very puzzled, wondering if something about the fact that Redis was returning a JSON String was confusing Sunspot. So I made a quick change to the InstanceAdapter:

  class InstanceAdapter < Sunspot::Adapters::InstanceAdapter
    def id
      if (@instance.class.to_s == "String")
        @instance.number  # return the book number as the id

And changed the register lines in my model:

  Sunspot::Adapters::InstanceAdapter.register(SunspotHelper::InstanceAdapter, Book, String)

  Sunspot::Adapters::DataAccessor.register(SunspotHelper::DataAccessor, Book, String)

And that did the trick. I didn’t like it, and intended to try to figure out what was going on. But after getting all the rest of it working, when I put the code back to its pre-String-adapter state, the error didn’t return. Like I said, I hate that. Hopefully it was just due to something that I was unknowingly doing wrong which I fixed along the way, but…just in case, now the quick-fix is documented here for anyone else who runs into the problem.

This afternoon I wanted to add another quick report to the system I’m building, and it was so easy that I thought I’d share some of the details. As I’ve written here before, I’m using the RaphaelJS library for simple charts, and it makes it very simple to create bar charts and pie charts. So I’ve already got a couple of those, using common code as I described in that earlier posting.

When I wanted to create a new chart, then, I knew I could leverage that. First, though, I needed to get at my data. What I was getting was essentially a list of categories, and the number of items in each category. Since this is stored in Redis, the items are key-value entries, and each category is a Set to which items belong. In this particular case, each item belongs to only one set.

So for the sake of an example, let’s say that we have books, divided into categories. We’ll store the books with keys like book:#:title and each category set will be called cat:name:

  • book:1234:title => “Technical Book”
  • book:5678:title => “Another Tech Book”
  • book:9012:title => “Gardening Book”
  • cat:technical => 1234, 5678
  • cat:gardening => 9012

So, for charting purposes, we want to get a list of the categories, and then for each one we want to fetch the number of books in it. In my reports_controller I have this:

  def books_by_category
    @chart = params[:chart] || "bar"
    @chart_title = "Books by Category"

    redis =
    # Get the list of the categories, which are the labels for the graph
    keys = redis.keys("cat:*")

    # Now let's iterate through the categories and get the counts
    @labels = []
    @values = []
    keys.sort.each do |cat|
      count = redis.set_count(cat)
      @values << count
      cat_name = cat.scan(/cat:(.*)/)[0][0]
      if (@chart == 'pie')  # Need to add counts to the labels for pie charts
        cat_name << " (#{count})"
      @labels << cat_name

First we get a Redis connection, and ask it for all of the category keys, using the pattern chosen: redis.keys("cat:*"). I want to stress something here: if you read the Redis docs (which of course you should, in depth) you’ll see that they say to never use this command in a production app! Obviously, if you have a lot of keys in the database, this is not a good command. In this particular case, I know that the database being used will not have too many keys, so I’m comfortable doing this — but be careful and make sure it’s okay for your case! If not, the solution is to create a new set that contains the names of all of the categories. Grab that set using SORT and work from there, which is simple. I also want to stress that, as with any reporting, if your data set grows (i.e. you start to have lots and lots of categories), you don’t want to run this frequently! Do the count occasionally and cache the results, create roll-up data from which to do your reporting, etc. This is a very simple case, but is a nice example of some tools.

Okay, so then we have the categories, and we iterate through them. For each, we get the count of entries using redis.set_count(cat). The redis-rb library aliases “set_count” to be the Redis command SCARD, which returns the cardinality of the set, i.e. the number of entries. We add that onto the @values array, and then create the category name by taking everything after “cat:” from the key. If we’re making a pie chart, we add the count to the labels, simply because I found that it’s very friendly that way. We add the category name to the labels array, and continue.

That’s pretty much it then! Using the previous reporting code, I just had to create a new view, which includes the partials I created before — the partials expect the @labels and @values arrays, so they’ll just graph whatever they get. Here’s the actual view:

<%= render :partial => "report_chart" %>


<h1>Reports : Books by Category</h1>
<%= render :partial => "report_links" %>


<div id="holder"></div>

If you refer back to my earlier post about RaphaelJS graphing, the report_chart partial contains the Javascript to generate the chart. The report_links partial simply has code to create links to the various chart types for this data: pie, bar, and csv. The holder div is where the RaphaelJS Javascript will render the chart.

And that’s all there is to it. Thanks to the ease of Redis sets, getting the data sliced and diced as needed was extremely simple, and thanks to easy Javascript reporting from RaphaelJS, the plain old label/value charting couldn’t be much quicker.

Well, this has been a pain. I have a case where I need to have the ID of a resource be a string, not an integer. Rails routes don’t like that — if you search a bit, you’ll find a number of arguments about the practice, with the Rails core folks coming down on the side of integer-only IDs. Shame that, and people have come up with various workarounds. For me, I ended up doing this:

  map.resources :workers
  map.inventor 'worker/:id', :controller => 'workers', :action => 'show'

  # In the view, I can then do this, which isn't pretty but is temporary, okay?
  link_to worker_name, worker_path("#{CGI::escape(worker_name)}")

I have my nice RESTful routes created for workers, which is nice for the usual index, update, destroy, etc which I can do using IDs most of the time. But for show, I need to be able to pass in a name rather than an integer. That’s where the unfortunate additional named route comes in, so I can have a URL like /worker/your_name_here. It’s human-readable and SEO-friendly, plus for various reasons I’m working with String keys instead of integers in this case (using Redis as the data store).

So that worked…except that I discovered some cases where the strings contained periods. Which is another Rails no-no! Sigh. So, I could muck around with additional special cases in my routes file, which is not nice. Thankfully, though, my searching stumbled on a couple of bugs filed in the Rails core, most particularly this one, which provided a workaround that’s not too awful. I changed my route to the following:

  map.inventor 'worker/:id', :requirements => { :id => /.*/ }, :controller => 'workers', :action => 'show'

Et voila, there we go. Adding the requirements parameter instructs Rails to allow periods, and it works. But I still hate routes.

I’m looking at the options available for agile project management, and thought I’d put a list here to help out anyone else who might be looking as well.

  • AgileBuddy: Works either hosted or as a local install, which can be handy. I like this tool, and it feels fairly close to PivotalTracker in usability. There’s a free edition for up to 3 users on one project; after that, the basic edition is $9.95/user/month, the team edition is $12.95/user/month, and adds things like connectors and a REST API. For only another $3/month I’m not sure why you’d cut corners and go for the basic edition, actually.
  • Agile on Demand: A more complex tool, this supports many projects/teams so might be good for an enterprise, with multiple roles, multiple dashboards, etc. Up to 5 users is free, after that it’s $32/user/month or more.
  • AgileTrack: More of an issue/release tracker, which doesn’t seem to manage stories/sprints as much as the other tools, but as a tracker could be interesting. Provides web services for integration, supports file attachments. Costs $39/user annual or less for more users.
  • ExtremePlanner: The dashboard shows backlog, iterations, and active tasks, with drag and drop to move stories from backlog into release, a fairly standard feature. Can attach files and test cases to stories. License is $149/user, plus $40 annual maintenance per user.
  • Mingle: Mingle is sort of a super-tool, with an integrated wiki, RSS feeds, configurable templates, test management, defect tracking, and much more. For a big company I can see it, but for a startup it feels like overkill. Would probably take some time to learn it and configure it. You can try it free for 5 users or less; above that it’s $566/user annually.
  • Pivotal Tracker: Perhaps my favorite of the tools I’ve looked at. Free, very lean and focused, it lacks bells and whistles and that’s purposeful. If you’re looking for a tool for a large enterprise with multiple departments, it might not do what you want, but for a small team, it’s great. I might not be able to use it at my current company because of some legal concerns about the terms and conditions, though.
  • ProjectCards: This comes as a downloadable installer and uses Tomcat on Linux; support for Mac is unclear. There’s a free version for 2 users and 4 projects with 200 cards max per project; the regular version is $145/user for 1-10, $140 for 11+ users.
  • Rally: A big tool, Rally provides the usual backlog/iteration support, for multiple teams and projects, as well as defect tracking and more built in. There’s a free community edition for one project and up to 10 seats; the Enterprise edition is $35/user/month, with an annual commitment required.
  • ScrumDesk: This one supports multiple teams, provides like most a story board view, allows attachments and tags for stories, but feels a bit rough compared to some others. Up to 5 licenses free, after that $15/user/month or more.
  • ScrumNinja: I played with this one when it was in Beta, and though the features were minimal it was clean and easy to use. Compared to the free PivotalTracker, though, I didn’t feel at the time that it was as strong. It looks a bit fuller now, and has a REST API as well. Not sure if attachments for stories are supported. Free for one user, then $10/user/month or less for larger teams.
  • ScrumWorks: A fairly large tool, it provides product management, milestones, weighting, sprint and user management, drag and drop task board and attachments, and so on, including a web service API. There’s a free basic edition that lacks many of the features; the Pro edition is $500/user fully-paid, or $289/user per annum subscription.
  • Tackle: This is an open-source tool using ASP.Net and SQL Server, because of which I didn’t look any more closely. If that’s your tech stack, though, check it out.
  • TinyPM: The interface on this one is a bit rough, but it’s a nice small tool, with the usual project backlog, task board, stories with tags, plus a built-in wiki. The community edition is a free download for up to 5 users, while the standard edition can be either installed or hosted, and is 12,50€/user/month.
  • XPlanner: An open-source project, this can be downloaded and uses Java with MySQL. Browser-based, has the usual UI to manage iterations and stories with tasks, though it doesn’t seem to support attachments. It looks to be somewhat specialized for XP, not just Scrum, with minimal features compared to some of the other tools.

Please feel free to point to tools you may have used and liked, and share opinions on the above. I haven’t provided lots of details here, because most of them are unique enough that anyone should look them over to make your own choices. Plus, I’m still investigating so I don’t have a lot of info about all of them. More later when I make a choice and use it for a while!

As discussed in Part 1, I reached a certain point with MongoDB, and decided that rather than fussing with things I’d move over to Redis and see how things went. The first thing was to change the model — rather than using the mongoid “:field” definitions, for Redis the model becomes a simple PORO (Plain Old Ruby Object). I chose to borrow a nice initialization technique from StackOverflow here so that I didn’t have to hand-code all of the attributes, but basically my initialize() method just sets the attributes and then creates a Redis connection via @redis = So the changes to the model were easy. The harder part was working out how the relationships between objects would work.

Rather than the document-style storage of MongoDB, Redis is purely based on key-value, but with more advanced data structures placed on the top. For my purposes, after some fantastic answers from Salvatore (author of Redis) and folks on the redis mailing list, I worked out how to use Sets to access the data in the ways I needed. So let’s say we have three books, ISBN numbers 123, 456, and 789. Book 123 references book 456, and book 789 references both 123 and 456. We have two authors, “Matsumoto,Yukihiro” who wrote 123 and 456, and “Flanagan,David” who wrote 456 and 789. How do we handle this in a key-value store? By using Sets:

  • Create entries for each book, with key pattern “book::data”. The value is a JSON string of data like title, price, etc (see below for note on this).
  • Create set called “books” which contains the number of every book.
  • Create sets called “backrefs:” that contain the numbers of books referenced by book #.
  • Create a set called “authors” which contains all of the authors.
  • Create sets called “author:” that contains the numbers of books written by the author.

Using the set operations in Redis, then, I can display all of the books by using the “books” set; I can display all of the books by a given author by using the “author:” set; I can display all of the books referenced by a given book by using the “backrefs:” set. In the latter case, you might be thinking that I could just keep an array in the JSON string — and yes, that could work, but I wouldn’t be able to use some of the other interesting set operations, such as intersections to determine references for a given author, for example. Note that right now, since an author is just a name, there actually is no longer any Author model! If I add more meta-data about authors in the future, I can add that easily.

About that JSON string: this has advantages and disadvantages that I’m still considering. Some would say that every individual attribute (or “column” in RDBMS-speak) should be a separate key-value pair. In that approach, for example, if I have a book title and price, I’d have book:123:title => “The Ruby Programming Language” and book:123:price => “39.99”. Obviously I can then do things like add a book to sets like “Under $50″ by adding the price item to the set. The big advantage noted by some is that attributes can be added instantly by just saving a new key. Using a JSON string, adding an attribute requires reading/updating all of the existing keys. On the other hand, it is tidy to have a single key, and working with JSON is easy. For the time being, I’m giving it a try by using “book:123:data” to store the “data” about the book, and separating out certain attributes if it makes sense to use them in other data structures like sets and lists. Is this the best of both worlds or the worst of both? I’m not sure yet.

A quick note here before getting into the code: I did this using the redis-rb plugin, which has a lot of functionality but is definitely lacking in documentation. However, the code is extremely clear and easy to read through, so I strongly recommend reading through it, particularly the main lib/redis.rb file. Using it’s pretty much just a matter of installing the plugin and then calling

So, my save() method looks like this:

def save
    book_key = "book:#{number}:data"
    @redis[book_key] = json_data		# creates JSON string
    @redis.set_add "books", number	# add to global books set
    if (back_references)
      back_references.each do |ref|
        @redis.set_add "backrefs:#{ref}", number
    if (authors) then
      authors.each do |a|
        a = CGI::escape(a)
        @redis.set_add "authors", a		# add to global authors set
        @redis.set_add "author:#{a}", number

Improvements to be made here include handling the author names in a better way; doing a CGI::escape works, but a proper hash would be better. During prototyping, the escaping is nice because I can go in with the redis client and see human-readable names, but it makes the keys too long in my opinion.

So now the index() action in the Books controller looks like this:

  def index
    redis =
    @entries = redis.set_count 'books'
    @pager =, 20) do |offset, per_page|
      redis.sort('books', { :limit => [ offset, per_page ], : order => "alpha asc" })
    @keys =[:page])

    @books =
    @keys.each do |k|
      @books[k] = redis["book:#{k}:data"]

Here we get a redis connection, and use Paginator to do its thing — we have to get a count of the set, and then we use sort. This is a big part of the magic, and something that took me some time to work out. The sort command in redis (doc here) is the entry point to doing a lot of the interesting operations once you have things in a set. You’ll notice that in the save() method, all I do is add the book number to the set, not the actual key. That’s much more efficient (Redis is especially good with integers), and is enough. In the case above, all it does is call sort on the “books” set, with the “limit” and “order” options — “limit” as shown takes an offset and number of entries to return, which makes pagination a cinch. For “order” you’ll see that I use “alpha asc” which might seem confusing here since we’re dealing with numbers. In my actual use case the “numbers” can have alphanumerics, and I decided to leave this here because it’s a useful variant to see. In reality, the default for the sort command is ascending numeric so you wouldn’t need to even specify the option here.

Once the keys are retrieved, then I iterate on each one and get the actual data. This is very quick with Redis, but still not ideal. Redis supports an MGET command to retrieve multiple items in a single command, but it doesn’t return the keys, which would mean I’d have to data but not know which book number each one goes with. The redis-rb library provides a great mapped_mget() method, but at the moment it doesn’t support passing in an array. I would have to iterate each key and build a string of them. Presumably a fix can be made to accept an array, in which case this can all be collapsed down to a one-liner: @books = redis.mapped_mget(@keys). (By the way, in case you’re wondering why @keys is an instance variable, it’s because it contains Paginator metadata like page number, to display in my view).

Hopefully it’s pretty obvious that showing a book is pretty straightforward:

    book_data = redis["book:#{@book_number}:data"]
    if (book_data)
      @book = JSON.parse(book_data)

Also simple, here’s the code to get the list of books which reference the current book — that is, the books that have the current book as one of their backward references:

      references = redis.sort("backrefs:#{number}")

That’s pretty easy, isn’t it? Obviously you can add in an “order” option and even a “limit” if necessary. More interesting, here we get the list of authors, with the list of books written by each:

    alist = redis.sort("authors", { : order => "alpha asc" })
    @authors =
    alist.each do |a|
        @authors[CGI::unescape(i)] = redis.sort("author:#{a}")

First we do an initial call to sort to get the authors, sorted in ascending alphabetical order (note that this will be a little undependable given my current implementation since the names are CGI::escaped). Then we iterate each one and do a further sort to get each one’s books. This is fine, but it just returns the number of each book by the author — they key, not the value. Do we have to iterate yet again and do a third call to get the data for each book? Not at all, and this is one of the magic bits of the Redis sort command. If instead of the above sort call we can ask sort to return the values to us, instead of the keys. Using the redis client, the difference is like so:

$ ./redis-cli sort authors:Smith%3B+Bob limit 0 5
1. 123456789
2. 465768794
3. 344756635
4. 436485606
5. 347634767

$ ./redis-cli sort authors:Smith%3B+Bob limit 0 5 get book:*:data
1. {"title":"My Book","price":"19.99"}

The second command, as you can see, adds a “get” option. This is a somewhat magic option that instructs Redis to get the values of the keys matching the pattern provided. So what happens, in a sense, is that Redis does the sort, and gets the keys. It then takes the keys and plugs them into the pattern, and does a get. So the first sort command is augmented with a “get 123456789″ and so on for the others, and the results are returned. This is all done on the Redis side, very quickly indeed. It is, clearly, extremely powerful. So if we change our code to get the data for the list of books, rather than just the keys:

    alist = redis.sort("authors", { : order => "alpha asc" })
    @authors =
    alist.each do |a|
      books =
      a_data = redis.sort("author:#{a}", { :get => "book:*:data" })
      if (a_data)
        a_data.each do |data|
          books << (JSON.parse(data))
      @authors[CGI::unescape(i)] = books

With this, my controller is passing @authors to the view, which is a Hash keyed off the unescaped author names. The value of each entry in the Hash is an Array of data (which is actually another Hash, created by the JSON.parse call). In the view, I can do something like this rather silly example:

<% @authors.keys.sort.each do |author| %>
  <% books = @authors[author] %>
  <tr class="<%= cycle("even", "odd") -%>">
    <td><%= author %></td>
      <% if (books.length > 0) -%>
        <%= books.length %> :
        <% books.each do |b| -%>
        (<%= truncate(b["title"], :length => 25) %>) |
      <% end -%>
      <% else -%>
      <% end -%>

This page simply iterates through the authors, and for each one it displays the number of books they’ve written, and the first 25 characters of each title. If they didn’t write any books, it shows a zero.

There is one problem here, and it’s one that I’m working on a solution for: the “sort” with “get” is very cool, but it returns the value of each entry instead of the key. That means that in the above view, I have access to the book’s title, price, etc — but NOT the number! That’s because the number is embodied in the key. This is obviously a problem, since I need to display the book number. Right now, I’m working around this by storing the number in the JSONified data, but that’s not the right thing to do. Ideally, there would be a way to have the “sort get” return the key along with the data, though I’m not certain what that would look like. Alternately, the app can get the keys, and use them to do an MGET for the data. We’ll see.

In any case, we’re now able to display the books and the authors, approaching the objects from either direction to access the others. I’ll post more and/or update this post as I experiment further, but I hope this and the first part serve as a useful introduction to people interested in exploring MongoDB and Redis. For my purposes, I plan to continue forward with Redis rather than MongoDB, but as I’ve shown, they’re not at all the same thing — I can easily see cases where MongoDB might be a better fit. It’s clearly worthwhile to do quick prototyping to make sure you understand your problem set, and then see what the best tool is. One of the most exciting things about the so-called “NoSQL” data stores is that developers now have more tools to work with. If I get the time, I hope to play with Cassandra and Tokyo Cabinet to see how they might fit in. It’s always great to have more options in the tool box.

For the new project I’m working on, after doing some initial very simple prototyping using MySQL (mainly because I could get from 0 to somewhere very quickly with ActiveScaffold and a few simple migrations), I started to look at alternate data stores. There are real reasons given the type of data being managed, but I have to admit that at least some of it was my desire to get a bit of hands-on experience with some of the new kids on the block, too. After exploring the alternatives, I settled on doing some prototyping with both MongoDB, and Redis. There are obviously others that are equally interesting, particularly Cassandra, but there simply isn’t time for everything! I selected Redis because I’d already done some playing with it, understood its basic concepts, and felt that its support for sets would be valuable for what I’m working on. I chose MongoDB as another option after doing some reading on it and finding it to be an interesting combination of key-value with some relational-style support. I also thought the mongoid was a nice bit of work that would be nice to use.

I want to note that I purposely did not call this “MongoDB vs Redis” — they’re different tools, and have different uses, which is one of the things I hope will be clear from these posts. This isn’t a competition, but just a summary of my experiments in looking at how I might approach my needs using the two.

The “problem” to be solved

I’m not at liberty to divulge the details of what I’m working on, so I have a sort of parallel-world simulation of the problem that replicates the types of issues I have to take care of. The idea, then, is to model a reference library, where we have Books and Authors. A Book can have multiple Authors, while an Author may have written multiple Books, so in a relational schema there would be a many-to-many relationship between them. In addition, a Book can contain references to other Books. We want to build a web app that will:

  • Show all of the Books
  • Show all of the Authors
  • For a Book, show all of the Authors
  • For a Book, show all of the Books that it references
  • For a Book, show all of the Books that reference it
  • For an Author, show all of the Books they’ve authored


As I mentioned above, I liked the look of the mongoid plugin to work with MongoDB, though I did do an initial pass using MongoMapper as well. I just felt that mongoid was a bit smoother, had more support for associations, and had somewhat more documentation, but they both did the job. Using Mongoid, my models looked something like this:

class Book
  include Mongoid::Document

  field :number
  field :title
  field :back_references, :type => Array
  field :forward_references, :type => Array
  index :number
  has_many :authors

class Author
    include Mongoid::Document

    field :name
    belongs_to :book, :inverse_of => :authors

As you can see, much like with ActiveRecord, you simply specify the fields you want persisted, and use a has_many/belongs_to pair to create an association. Do note that instead of extending a class as you would with AR, for mongoid you simply include Mongoid::Document. When I want to create a Book, it goes something like the following, assuming that I have the book number/title and an array of author names:

    the_book =
                        :number => book_number,
                        :title => book_title
    authors.each do |a|
      the_book.authors << => a)

But what about the references, then? In the Book model above, I have two arrays, back_references (a list of books that reference this one) and forward_references (a list of books that are referenced by this one). Actually, all it takes for these is to create arrays containing the book numbers, assign them to the instance, and save. That’s one of the nice things about MongoDB, as we’ll see: you can query for items in embedded arrays.

A quick note here: I’ve glossed over the setup and configuration of MongoDB here, somewhat on purpose. Once you’ve installed it, if you’re using mongoid there are very clear instructions on setting up your Rails app to use the db so there’s not much need for me to repeat things here. Let’s just say we’re using a db called “books-development” which will then contain our collection, which is called “books”. Wait, shouldn’t we have another collection called “authors” since we have an Author model? Well, no, because the way we set up the has_many/belongs_to it means that Authors are embedded objects within Books. Let’s see what an entry looks like when we persist it. Running the mongo shell:

> db.books.find({number : "1234567890"});
{ "_id" : "4b58f90c69bef38f8f000720", "number" : "1234567890", "forward_references" : [
], "back_references" : [
], "authors" : [
                "_id" : "4b58f90569bef38f8f000091",
                "name" : "Matsumoto,Yukihiro",
                "_type" : "Author"
                "_id" : "4b58f90569bef38f8f000092",
                "name" : "Flanagan,David",
                "_type" : "Author"
],  "_type" : "Book", "title" : "The Ruby Programming Language" }

From this, you can see that Mongo has assigned “_id” values to each object, the references are both just arrays of book numbers, and the authors have become embedded objects with their own “_id” and “_type” (used by mongoid). As we’ll see in a bit, the fact that the authors are embedded objects is convenient for some purposes, but problematic for others due to the queries I needed to do. For now, though, let’s see what our queries look like for the various activities listed above.

  # Inside books_controller.rb, index action to list the books
  def index
    @entries = Book.count
    @pager =, 20) do |offset, per_page|
      Book.criteria.skip(offset).limit(per_page).order_by([[:title, :asc]])
    @books =[:page]) 

  # show action to display a single book's details
  def show
    @book = Book.find(:first,  :conditions => { :number => params[:number] })

Pretty straightforward stuff, even when bringing Paginator into the picture. Being able to chain the criteria with mongoid is a nice bonus to using it. So when a single book is displayed, the page can show the list of author names by simply iterating the array:

    <td class="label">Authors</td>
    <td class="show">
      <% if (@book.authors)
         @book.authors.each do |author| -%>
        <%= %> |
      <% end -%>
      <% else -%>
      <% end -%>

The backward references are exactly the same way. However, I discovered while writing the data entry scripts that the forward references (i.e. the books that reference the current book) were not available. No problem, I figured, instead of storing that I’ll just query it:

  def referenced_by
    Book.find(:all, :conditions => { :back_references => number }) 

There’s some nice MongoDB magic. Very simply, that will return any Book entry that contains “number” in its “back_references” attribute — even though that attribute is an array! That ability to query for contents of an array comes in very handy, needless to say. As an aside, I came across a reference that I sadly can’t find now to link to it, but it showed me how to add a super simple search. To make the books searchable, I just took the title and the author, did a split(), and created an array containing each word. I called that “search_words” and made it a new array-type attribute. The search is then a simple query:

  def search_books(search_term)
    Book.find(:all, :conditions => { :search_words => search_term }) 

This is obviously a very simplistic search, but given that it takes about 2 minutes to implement, who’s complaining?

The Author problem

So now we come to where I began to find problems with the approach. I wanted to display the list of all authors. Hmm, the authors are embedded documents within the books. Okay, it is possible:

  def get_author_list
    results = Books.criteria.only(:authors)
    author_list =
    results.each do |book|
      book.authors.each do |a|
        if (!author_list.has_key?(a))
          author_list[a] = Book.where(:authors => a)
    return author_list

Pretty ugly, ain’t it? It queries all of the books and gets just the authors attribute, then iterates each book, then iterates the authors. For each one, it does a query to get the list of books (so our page can show each author followed by their books), and creates a Hash with key=author, value=books array. This obviously doesn’t do any pagination, which would make it even messier, plus the results aren’t sorted yet. Nope, I didn’t like it.

The alternative seems to be to make authors a first-level document, and link explicitly with book numbers, which isn’t horrible but means, again, multiple queries to get our list of authors with their books. This was beginning to look like it might be too relational a problem for MongoDB to make sense.

Update: as noted in the comment below by module0000, using distinct(“author”) solves this particular problem in a much cleaner way — thanks for the comment! I’ll still stand by the thought that this is really a relational problem and a document database has shortcomings in that regard (and of course strengths in other ways).

So, I set this aside, since as a prototype it did work. I made a new branch (thanks, git) and converted it to use Redis. Which I’ll cover in part 2, shortly.

This took me some time to figure out — and I’m still not entirely sure what the whole issue is about. But I figured I’d post the solution here to help anyone else, since I didn’t actually find the solution spelled out anywhere.

I wanted to do a simple JSON.generate from a Hash, because I have a class with a number of attributes and I only want to include some of them. So I didn’t want to rely on the normal to_json(). Instead, I added my own json_data() method, created a Hash, and then called JSON.generate(the_hash). Unfortunately, that gave me the following error:

NoMethodError: undefined method `[]' for #<JSON::Ext::Generator::State:0x1056ba2f0>
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/encoders/hash.rb:45:in `as_json'
	from /Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/encoders/hash.rb:34:in `to_json'
	from /Library/Ruby/Gems/1.8/gems/json-1.2.0/lib/json/common.rb:183:in `generate'

I got the same error regardless of whether I had the json_pure gem or the json gem installed (I tried both). I then happened to notice a mention on the JSON rubyforge page about enforcing a specific JSON variant, which reminded me of something I read elsewhere. So I figured okay, what the hell, and I added require 'json/pure' and…voila! Problem gone. I could see from the error that the default variant (I’m running Rails 2.3.5 right now) was json/ext, which for whatever reason wasn’t working properly. Why that is, I’m not yet sure.

In any case, it works now, and perhaps this will help someone else out.

Small post, just to document a funny bit of code that I needed. Thanks to this blog post I got a start on making use of the Rails tag helpers from inside a controller. I needed to do this because I wanted to change an ActiveScaffold link label to be an image, and in order to do that I had to create the image tag in the controller in order to set the link label. Kind of annoying, but I was curious if it would work.

The basic trick is shown in the above-linked blog post, which is to create a “mini-helper” class inside ApplicationController, and sort of “link” to it via a help method. So I added to my ApplicationController:

  def self.tag_helper

  class TagHelper
    include Singleton
    include ActionView::Helpers::TagHelper
    include ActionView::Helpers::AssetTagHelper

Now I have a method available to my controllers called tag_helper, which uses a Singleton instance of TagHelper, which in turn includes the helpers. This requires including Singleton, in order to have one instance of the helper, and then the ActionView helpers. TagHelper is needed for AssetTagHelper to work, unfortunately.

So now in my controller that extends ApplicationController, I do the ActiveScaffold configuration and set the link labels. The idea here is that ActiveScaffold creates a list of items with “Edit”, “Delete”, and “Show” links. Instead of using those words as the links, I wanted to use some nice icons:

  active_scaffold :mything do |config|
    config.columns = [:name, :number, :active, :description] = tag_helper.image_tag('document_edit.png', :width => "30") = tag_helper.image_tag('document_delete.png', :width => "30") = tag_helper.image_tag('document.png', :width => "30")
    list.sorting = {:name => 'ASC'}

I configure which columns to display, and the sorting order, and I set the labels for the “update”, “delete”, and “show” links. I use the tag_helper() method to get access to the image_tag() method, which then does its thing and returns the properly-formed tag.

Nothing magic, but it took a little while looking at the ActiveScaffold code to decipher the not-quite-right docs for setting the labels, and then some more time to find a nice way to use the tag helper so I didn’t have to hard-code the tags. Someone else might find it useful, so here it is.


Get every new post delivered to your Inbox.