Book review: PHP and MongoDB Web Development January 24, 2012

I recently was provided with an eBook review copy of PHP and MongoDB Web Development from Packt Publishing, by Rubayeet Islam. Being interested in MongoDB, though a bit out of date with my PHP development, I read through the book fairly quickly.

As the title indicates, the author focuses on using MongoDB to provide the storage back-end for PHP web applications. After a short introduction to the basic concepts behind MongoDB, we get a walk-through of installing MongoDB and getting PHP to talk with it, before starting in on building a blog. It’s a safe example, since a blog is a reasonable candidate for a document store like MongoDB. It also provides a way to address one of the big design questions when using MongoDB, which is when to use embedded documents and when to store references. To my mind the question is glossed over a bit too quickly, but it is discussed.

Additional projects like session management and geolocation get a bit off-track, as a lot of time is spent describing the concepts rather than MongoDB, but the meatier sections that get into topics like Map-Reduce (creating a tag cloud) and web analytics are certainly worthwhile. I did feel that the chapter reviewing two MongoDB management tools could have been skipped, since the information will likely be out of date within a couple of months.

Overall, this is a reasonable beginner’s guide, as its subtitle indicates. There’s a great deal of PHP code filling its pages, which will give you a starting point if you need a boost to get going. Reading through it will give you the basics about MongoDB, and a bit more — hints on indexing, optimizing, and Map-Reduce will keep you running. A lot of the information felt cursory, and I would have appreciated more depth, but that’s probably just me wanting more than a beginner’s introduction. Perhaps more relevant were my concerns about the copy-editing and grammar. I didn’t notice any actual errors, but the grammar is quite rough and it made me wonder. It may make me old-fashioned these days, but I still expect my books to be well-edited and grammatically correct. The issue didn’t get directly in the way of the information to be had, but it’s still a pity. Nonetheless, if you’re a PHP developer and you’re looking to get started with MongoDB, you’ll doubtless find this a useful book.

PHP and MongoDB Web Development from Packt Publishing, also available from Amazon.

Little Redis Book January 23, 2012

Just a quick note that Karl Seguin has created a nice mini-Redis book, available for free download. Check his announcement here and grab the book. He’s also posted the source of the book to GitHub.

Devise and OmniAuth for Single Sign On December 31, 2010

I just wanted to put a quick link to a great blog post about using Devise and OmniAuth to create a single-sign on mechanism where multiple apps can share an authentication mechanism. It’s very straightforward; check it out here.

Getting user info into a Radiant page December 31, 2010

This took me a while so I thought I should share the solution — however, see the caveat at the end, because there’s an element I haven’t tested yet.

The first requirement here is integrating Devise into Radiant. For the most part, the information at this page will get you there, though I’ll work on a separate post going through the process in detail. Once you have Devise working, then you have a user object, and naturally you’d like to display a “Logged in as…” element in your Radiant layout, right? Not so easy, it turns out.

In my testing I called the Devise model PortalUser since it has to be differentiated from the User model that Radiant uses. I put the authentication stuff into a custom extension, which we’ll call “my_auth”. So, I end up with my_auth_extension.rb:

class MyAuthExtension < Radiant::Extension
  
  SiteController.class_eval do
    include ContentManagement
    prepend_before_filter {|controller| controller.instance_eval {Thread.current[:current_portal_user] = current_portal_user} }
    prepend_before_filter {|controller| controller.instance_eval {authenticate_portal_user! if radiant_page_request?}}
  end

  # activate() method left out for brevity
end

The filter to call authenticate_portal_user! is needed to get Devise working. The other filter is the important one here, and what it does is get the current_portal_user reference in the controller and place it into the current thread for later access. This is the only way I’ve found (so far) to get something from a controller in Radiant to a tag. I’ve tried various instance variable tricks, all sorts of things, with no luck. If anyone has another solution, please do comment below, because yes, this seems like a hack.

Now we go create a new tag to display the logged-in user’s email address. In our extension we have lib/user_tags.rb:

module UserTags
  include Radiant::Taggable

  desc "Outputs the user email address"
  tag "user_email" do |tag|
    current_user = Thread.current[:current_portal_user]
    @user_email = current_user.email
    parse_template 'usertags/_email_template'
  end

  private

    def parse_template(filename)
      require 'erb'
      template = ''
      File.open("#{MyAuthExtension.root}/app/views/" + filename + '.html.erb', 'r') { |f|
        template = f.read
      }
      ERB.new(template).result(binding)
    end
end

First, let me give credit for the parse_template() method to Chris Parrish in this post. This tag simply gets the user object from the thread, and sets @user_email accordingly, which can then be used by the ERB template. parse_template() grabs the partial using the filename passed in, and renders it, which ends up being output by the tag. The partial, which lives in your extension as app/views/usertags/_email_template.html.erb, is simply:

<%= @user_email %>

So there’s nothing to that, really. If you modify your Radiant layout to include Logged in as: <r:user_email /> then you should be all set.

At the beginning I mentioned a caveat. I have not tested this yet to see what the effects of Radiant’s caching are — I am assuming that the tag contents will not be cached and thus all is well, but we will see. I’ve been bitten by the caching before in unexpected ways.

Anyway, I hope this helps someone out.

ActiveSalesforce gem travails August 26, 2010

I’ve been working with the ActiveSalesforce gem, which is very cool but also frustrating, because there are numerous versions of it and none of them is authoritative. I started with the “basic” one you’ll get if you begin at the RubyForge page. It works, but I encountered problems trying to get relationships to work using belongs_to and so forth.

I looked around a bit, and ended up trying the fork by John Reilly, which fixed that problem, and things seemed to be working well. Then I discovered an odd error, which was causing any fields containing an ampersand (‘&’) to be truncated such that everything up to and including the ampersand was being lost. This meant names like ‘Acme & Associates’ came out as simply ‘ Associates’. Not so cool.

>> my_obj = Salesforce::Account.find('0014000000MNdgAAAA') 
activesalesforce: sql=SELECT * FROM Account WHERE (Account.id = '0014000000MNdgAAAA') 
#<Salesforce::Account:0x1051f7e48> { 
                                      :id => "0014000000MNdgAAAA", 
                                      :name => " Associates" 

Note that the name isn’t ‘Acme & Associates’ — everything up to the ampersand is being lost during the fetch. I added some debugging output in the gem to verify that the result being passed back was indeed truncated. Thanks to the ActiveSalesforce mailing list I was pointed to some other forks, and I grabbed one of them seemed a likely candidate, being forked from the althor880-activerecord-activesalesforce-adapter one, which included a switch to using Hpricot for XML parsing. So I did a git clone of http://github.com/blaines/activerecord-activesalesforce-adapter.git, did gem build activerecord-activesalesforce-adapter.gemspec and sudo gem install activerecord-activesalesforce-adapter-2.3.7.gem and tried it out. Note that in my case, the commit at HEAD that I grabbed had some unfixed merge conflicts that I had to take care of before I could build the gem, which was inconvenient but not a big problem. Most importantly, indeed, it did the trick: the fields are now coming through with the ampersands intact!

Sadly, however, I started getting a new error on testing in my app:

/!\ FAILSAFE /!\  Wed Aug 25 14:21:25 -0700 2010
  Status: 500 Internal Server Error
  User can't be referred

I scratched my head for a bit, and then looking at the log I saw:

(eval):1: warning: already initialized constant User
(eval):1: warning: already initialized constant User
(eval):1: warning: already initialized constant Assets
(eval):1: warning: already initialized constant User
(eval):1: warning: already initialized constant User
(eval):1: warning: already initialized constant User
(eval):1: warning: already initialized constant Note
...etc...

I hadn’t seen this before with the other version of the gem. It implied, though, that on startup it was initializing a bunch of Salesforce-related classes. Weird, but I went ahead and created a new Salesforce::User class in my app:

class Salesforce::User < Salesforce::SalesforceBase
  set_table_name "User"
end

SalesforceBase is simply my small base class that handles setting the connection information so I can set it in one place for all of my Salesforce-related models. And what do you know, once I had this class, it apparently disambiguated things, and the error went away. I’m guessing that there was a namespace collision with the User class needed by Devise, and creating this one to point to Salesforce cleared up the confusion. Not ideal, but I can live with it.

So I now have something that works. Clearly, though, there are overall issues with this gem — although the Rubygems page looks nice, it doesn’t seem to point to a canonical “latest and greatest” version, and there are a half-dozen forks in various different states. This makes it pretty difficult to know exactly what you’re getting when you try to work with ActiveSalesforce, unfortunately.

Rails and forms using accepts_nested_attributes_for July 23, 2010

I recently had to put together a particularly ugly web form, with dynamically-expanding multi-nested elements, and came up against some rather odd behavior from Rails’ nested forms, using Rails 2.3.8. First off, I have to give thanks to Railscasts for saving me a bunch of time creating the dynamic portion of the nested form — see that episode and the following one for a great solution which got me started. Secondly, note that in Rails 2.3.5, some nested forms behavior was simply broken, and when I upgraded to 2.3.8 it fixed a number of small issues.

Unfortunately, for the form I was working on I had to nest two levels deep, which complicated things. The relationship was something like this: the form was created to update an event, which can involve multiple companies. For each involved company, there is additional metadata. So there are three models involved: Event, CompanyEvent, and Company. CompanyEvent is more than just a join model, since it contains metadata about the relationship. In theory, the nested-nested models weren’t a problem, simply by putting the proper directive in each model:

class Event < ActiveRecord::Base

  has_many :company_events
  accepts_nested_attributes_for :company_events, :allow_destroy => true
end

class CompanyEvent < ActiveRecord::Base
  belongs_to :company
  accepts_nested_attributes_for :company
  belongs_to :event
end

class Company < ActiveRecord::Base
  has_many :company_events
  has_many :events, :through => :company_events
end

Thanks to the accepts_nested_attributes_for directives, the form for an event can easily incorporate entries for company events, which in turn incorporate companies. As an example:

<% form_for(@event, :url => event_path(@event)) do |f| %>
  <%= f.text_field :event_title %>
  <% f.fields_for :company_events do |builder| %>
      <%= builder.text_field :company_role %>
      <% builder.fields_for(:company) do |company_form| -%>
          <%= company_form.text_field :name %>
      <% end -%>
  <% end -%>
<% end -%>

The above is a slimmed-down form, of course, but serves to demonstrate the nested form approach. The event form is the outer one, which contains fields_for the company_events model — each current instance of a company_event associated with the event will be rendered with its company_role (for example, what role the company plays at the event in question). Within that nested form, another fields_for is included for the company, which will pull in the company name as a field.

As shown in the Railscast linked above, you can also include a field to mark a nested entry as deleted, such as . Check out the Railscast for a full demonstration, since there’s no need to repeat it here.

The issue I encountered, though, was mysterious: when I tried to change the name of a company in the nested form, I got an error: “Couldn’t find Company with ID=12345 for CompanyEvent with ID=6789″. This didn’t make much sense, since obviously there wouldn’t be a matching entry, because I was changing the company and thus the company id would have also changed! It was a mystery why the code would be trying to find a matching row using both ids. I actually had to go into the code for nested_attributes.rb and look into the assign_nested_attributes_for_one_to_one_association method to see what was going on. The key to it seemed to be the use of the :update_only option on the accepts_nested_attributes_for directive. I added that to the CompanyEvent model:

class CompanyEvent < ActiveRecord::Base
  belongs_to :company
  accepts_nested_attributes_for :company, :update_only => true
  belongs_to :event
end

And suddenly it worked. The slim documentation for the :update_only option wasn’t very helpful (see http://api.rubyonrails.org/classes/ActiveRecord/NestedAttributes/ClassMethods.html), as it says that “an existing record may only be updated” and “A new record may only be created when there is no existing record.” Which seems rather obvious, but almost implies that a record can’t be deleted, since it’s “update only”. Otherwise, why on earth would you not want an existing record to be updated? Perhaps this should be the default behavior, though I haven’t tested to figure out what the alternative really means. I need to look at Rails 3 and see what’s changed about the nested forms behavior, and perhaps this is mapped out more clearly there.

In any case, perhaps this will save someone else some pain, since it took me some time to work out what was going on. And I realize that I’ve skimmed over a lot of details of how to do nested forms, since this isn’t intended to be a how-to but more of a watch-out post. If anyone thinks that a general nested-forms how-to post would be useful, let me know and I can put one together.

Odd Ruby Array Behavior July 7, 2010

Here’s an oddity that we just ran across this afternoon, and it’s a nice bit of Ruby trivia to break the bit of silence here on the blog. It has to do with using an ordinary each on a Ruby array, and what happens if you change the array while you’re iterating. Here’s an example in irb:

irb(main):004:0> s = [1, 2, 3, 4, 5]
[
    [0] 1,
    [1] 2,
    [2] 3,
    [3] 4,
    [4] 5
]
irb(main):005:0> s.each do |i|
irb(main):006:1* puts "> #{i}"
irb(main):007:1> s.delete(i) if (i == 3)
irb(main):008:1> end
> 1
> 2
> 3
> 5
[
    [0] 1,
    [1] 2,
    [2] 4,
    [3] 5
]

As you can see, we first create an array containing the numbers 1 through 5. Then we do a simple iteration, and print out the value of each array element. If it’s element 3, then we delete that element from the array, and continue on.

What you might expect to happen is that it will print all of the entries, 1 through 5, and at the end the array will simply be missing element 3. That would make sense. However, it’s not what happens. As shown above, instead we end up skipping element 4 altogether!

While unexpected, there is some logic to why this happens. While we iterate, Ruby is essentially holding a pointer, or an offset, into the array. When we’re on element 3, the offset is 2 (counting from zero). When we delete element 3, the array shrinks, but the offset clearly is left at 2. When we continue our iteration, Ruby increments the offset to 3, which ends up pointing to the value “5″ because we deleted one element. That means that we end up skipping the value “4″.

We happened to encounter this problem while working in one of our models in a method that needed to delete all the elements in an associated model, and we had code that basically did the following:

other_models.each {|m| other_models.delete(m)}

The test data had three other_model entries, but every time it deleted the first and the third, leaving the second one. As we discovered, that’s because deleting the first one meant that the array shrank, and the each ended up skipping the second element.

It’s hard to say whether this should be considered a Ruby bug or not, but I’m feeling a bit inclined to say it is…

Displaying Ruby and environment details May 29, 2010

I stumbled on this in a random email list archive, and modified it slightly, but I certainly can’t take credit for it. In any case, it’s a really nice way to see what the environment you’re running in looks like so I thought I’d share. For example, I’ll run this in script/console to verify things like the Rails and Ruby version. It can be quite handy:

>> Object.constants.sort.each {|c| cv=Object.const_get(c); print c, "=", cv, "\n" unless Module === cv}; true
ARGF=ARGF
ARGV=
CROSS_COMPILING=nil
ENV=ENV
FALSE=false
NIL=nil
PLATFORM=x86_64-linux
RAILS_CACHE=#<ActiveSupport::Cache::MemoryStore:0x2aaaaf8b71d8>
RAILS_DEFAULT_LOGGER=#<ActiveSupport::BufferedLogger:0x2aaaaf8caee0>
RAILS_ENV=production
RAILS_GEM_VERSION=2.3.5
RAILS_ROOT=/opt/rpx/app/releases/20100525230203
RAILTIES_PATH=/usr/local/lib/ruby/gems/1.8/gems/rails-2.3.5/lib/..
RELATIVE_RAILS_ROOT=/opt/rpx/app/releases/20100525230203/config/..
RELEASE_DATE=2010-01-10
RPM_CONTRIB_LIB=/usr/local/lib/ruby/gems/1.8/gems/rpm_contrib-1.0.10/lib
RUBY_COPYRIGHT=ruby - Copyright (C) 1993-2010 Yukihiro Matsumoto
RUBY_DESCRIPTION=ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]
RUBY_PATCHLEVEL=249
RUBY_PLATFORM=x86_64-linux
RUBY_RELEASE_DATE=2010-01-10
RUBY_VERSION=1.8.7
STDERR=#<IO:0x2aaaaab15ab0>
STDIN=#<IO:0x2aaaaab15b00>
STDOUT=#<IO:0x2aaaaab15ad8>
TOPLEVEL_BINDING=#<Binding:0x2aaaaab08f90>
TRUE=true
VERSION=1.8.7
=> true

I put the “true” at the end just to prevent it from spewing out the object info.

Updated Redis Cheat-sheet April 23, 2010

I have finally had the chance to update my Redis cheat-sheet so that it has the latest commands, including the Hash, Multi/Exec, and Pub/Sub stuff. I’m calling this v2.0 of the cheat-sheet since there are lots of changes and it in theory puts it in-sync with Redis 2.0, though I’m sure there will be small changes to come.

I’ve also done the right thing and created a new GitHub repository for it, with the OmniGraffle file there (though not the fonts, yet, pending looking at them to see if that’s okay). So, enjoy, and let me know if I’ve missed/goofed anything, of course. The repo is here and the direct link to the PDF if that’s all you want is here.

Using redis_logger for application logging April 16, 2010

I just pushed a new project to github, redis_logger. I decided to give this a go and see if it ended up as potentially useful as I thought it might, and I’m pretty pleased with its initial version, limited though it is.

The idea is that by installing the tiny gem you can add logging into Redis to your application, including the ability to group log entries together, and then browse the groups, including intersections between groups.

As an example: let’s say you do like I did, which is to add request logging to your Rails application. I added this to my application_controller.rb:

  before_filter :log_request

  def log_request
    RedisLogger.debug({ "request" => request.url,
                        "remote_ip" => request.remote_ip,
                        "user_id" => current_user.id,
                        "username" => current_user.email
    }, "requests")
  end

This will create a log entry as a Hash in Redis, containing the key/value pairs that were passed in. A “timestamp” value is also automatically added for you. This entry will be added into two groups, “debug” and “requests” — the “requests” group is passed into the call as the optional second parameter.

I could also add a call in my controller to log a warning if a user tries to access a page and is redirected to the signin page, or an error if an exception is passed up. I could then view all of the log entries in the intersection of “error” and “requests” to see only errors logged in the “requests” group, and not the debug or warn messages.

One of the great things about this is that it was so easy to do using Redis, once I worked out the approach. Using Sets, the code stores the entries and then adds them to the log groups as sets. Using Hashes, each entry is stored with its key/value pairs intact. I’m thinking about things l ike adding some more standard keys, like the current “timestamp” key, and enabling some additional functionality like adding tags for additional searchability within the groups.

Having the log entries in Redis is great, but browsing them is the fun part. So, there’s also redis_logger-web, a simple Sinatra app that lists the groups and lets you view the log entries. You can click on a group name to see its entries, most recent to oldest, or you can select multiple groups and view the intersection of their entries. Right now that’s limited to just the most recent 100 entries, until I work out the best way to save temporary sets and clean them up in a cron. In reality, though, the most recent 100 entries is what’s generally useful. Adding export functionality is first on my list, because that will be extremely handy for analysis.

Scaling, testing, and adding the essential multi-threading are next, of course. Sound interesting? Please, grab it, fork it, enhance it, let me know what you think.