gamification in rails

I’d like to add some gamification features to HeavyInk.com.

First, what is “gamification” ?

https://en.wikipedia.org/wiki/Gamification

Gamification is the use of game thinking and game mechanics in non-game contexts to engage users in solving problems. Gamification has been studied and applied in several domains, such as to improve user engagement, physical exercise, return on investment, data quality, timeliness, and learning. A review of research on gamification shows that most studies on gamification find positive effects from gamification.

Engagement is the goal here: I’d like people to engage with HeavyInk, read more pages, buy more graphic novels, etc.

One of the wonders of rails is all the preexisting libraries (“gems”). It’s like the joke about standards: they’re wonderful; there are so many to choose from!

The choices for gamification gems seem to be

Neither ‘Gamify’ nor ‘Honor’ have been updated in over a year, which knocks them both out of the running for me.

How to compare the other two?

The core concepts of gamification are

  • Points
  • Badges
  • Levels
  • Leaderboards / rankings

A perusal of the READMEs show that Merit and Gioco both support all of these.

‘Gioco’ comes up first in Google rankings, which seems like an utterly silly thing to consider, but it does speak to the number of incoming links, which speaks to programmer engagement, which speaks to common practice, likelihood of maintenance, etc.

‘Merit’ on the other hand has had checkins as recently as 4 days ago, so it seems very ‘live’.

Gioco Merit
Google rank high ?
last update 3 months 4 days
contributors 28 7
contributors 207 385

Both look like reasonable choices.

One thing that I’m not in love with about Gioco is that a fair amount of stuff (creating new badges, etc.) seems to take place via rake, e.g.:


	rake gioco:add_badge[BADGE_NAME,DEFAULT]

whereas Merit does this via code:


	Merit::Badge.create!( 	id: 1, 	name: "Yearling", 	description: "Active member for a year", 	custom_fields: { difficulty: :silver } 	)

I prefer the latter approach, as it makes it easier to do things via migrations, which works for my development cycle: write the migration, do the development, then check everything in and deploy (running migrations as a hook).

So far both of these look like good packages, and I need to do more research.

Onward!

Posted in Uncategorized | Leave a comment

using papertrail to find weird problems

The problem

Recently I had a problem where thousands of customer line items in the HeavyInk.com e-commerce app were getting cancelled.

The code that cancelled line items was (a) a mess, and (b) spread all over the code base. The result was that there were several places where the bug could be hiding.

Some refactoring

The first thing I did was cleaned up the code, by replacing code in other modules like this:

  class LineItem < ActiveRecord::Base
    attr_protected       # <-- total access

      PENDING   = 1
      SHIPPED   = 2
      CANCELLED = 3
  end

  class Foo
   def Bar
     self.line_items.each { |li| li.update_attributes(:state => LineItem::CANCELLED) }
   end
 end

with

 class LineItem < ActiveRecord::Base

    attr_protected :state # <-- not as secure as it should be, but better

      PENDING   = 1
      SHIPPED   = 2
      CANCELLED = 3

    private_constant : PENDING
    private_constant :SHIPPED 
    private_constant :CANCELLED 

  def cancel
    update_attribute(:state, USER_CANCELLED)
  end

  end

  class Foo
   def Bar
     self.line_items.each(&:cancel)
   end
 end

this is much more hygenic in three different ways:

1) it stops knowledge of internal constants from leaking out
2) it stops direct manipulation of internal state
3) it gives us a single point of access on LineItem cancellations and allows us to do sanity checking there

Sadly, it was not enough, on its own, to fix the bug.

Enter paper_trail

While watching an unrelated railscast I saw a reference to paper_trail in passing.

Paper trail, for those not in the know, is a phenomenal gem that allows one to instrument an ActiveRecord class to automatically store checkpointed versions just before each edit.

Go read the docs!

Installation is simple. Add this to your Gemfile


gem 'paper_trail', '>= 3.0.0.beta1'

and run a migration.

…but I wanted a bit more. I needed to know what piece of code was disrupting my data.

For this I didn’t just want to see what the line items looked like before they got squashed; I wanted to see who was responsible for squashing them.

I added a migration


class AddMetadataToPapertrail < ActiveRecord::Migration
  def change
    add_column     :versions, :stack, :text, :null => true
    add_column     :versions, :server, :string, :null => true
    add_column     :versions, :uri,    :string, :null => true
  end
end

(why record the server? Because the same code base runs on multiple machines, all sharing one database).

and an initializer


module PaperTrail
  class Version < ActiveRecord::Base

    # TJIC extension of papertrail - I want to store stack traces (at least for now)
    # 27 Sep 2013
    #
    attr_accessible :stack, :uri, :server

    def get_stack() 
      YAML::load(stack)
    end

    def print_stack()
      puts "server = #{server}"
      puts "uri    = #{uri}"

      get_stack.each do |entry|
        puts "  * #{entry}"
      end
    end


  end
end

tweaked application_controller.rb a bit:


  # http://stackoverflow.com/questions/6307138/how-do-i-get-request-uri-in-model-in-rails
  #
  # set a global variable that can be used later, inside models, etc.
  #
  before_filter :set_request_as_global
  def set_request_as_global
     $request = request
  end  

and then instrumented my LineItem class thusly:


class LineItem < ActiveRecord::Base

  #----------
  # papertrail debugging 
  #----------
  has_paper_trail :meta => { :stack => :get_stack,
                             :uri   => :get_uri,
                             :server=> :get_server,
   }
  def get_uri()  
    # $request being set depends on a before_filter in 
    # application_controller.rb
    "#{$request.andand.protocol}#{$request.andand.host_with_port}#{$request.andand.fullpath}"
  end
  
  def get_server()
    # potentially faster than file-system based caching
    #
    $server ||= `hostname`.chomp
  end

  def get_stack() 
    trace = caller(0)
    trace.reject! { |ii| ii.match(/active|action|railties/) }
    YAML::dump(trace)
  end


end

The end result was that I deployed the code to a live environment, then waited for the problem to strike again.

When it did, I went into the console


rails console production

bad_li = LineItem...
bad_li.versions.last.print_stack

which printed out


  server = 'host3'
  uri    = "http://heavyink.com/me/subscription...."
  * /home/tjic/bus/hi/src/hiw/app/models/line_item.rb:202:in `get_stack'
  * /usr/local/rvm/gems/ruby-1.9.3-p429/gems/paper_trail-3.0.0.beta1/lib/paper_trail/has_paper_trail.rb:253:in `block in merge_metadata'
  * /usr/local/rvm/gems/ruby-1.9.3-p429/gems/paper_trail-3.0.0.beta1/lib/paper_trail/has_paper_trail.rb:244:in `each'
  * /usr/local/rvm/gems/ruby-1.9.3-p429/gems/paper_trail-3.0.0.beta1/lib/paper_trail/has_paper_trail.rb:244:in `merge_metadata'
  * /usr/local/rvm/gems/ruby-1.9.3-p429/gems/paper_trail-3.0.0.beta1/lib/paper_trail/has_paper_trail.rb:205:in `record_create'
  * /home/tjic/bus/hi/src/hiw/app/models/order.rb:274:in `block in add_products_to_existing'
  * /home/tjic/bus/hi/src/hiw/app/models/order.rb:270:in `each'
  * /home/tjic/bus/hi/src/hiw/app/models/order.rb:270:in `add_products_to_existing'
  * /home/tjic/bus/hi/src/hiw/app/models/order.rb:207:in `block in create_for_products'
  * /home/tjic/bus/hi/src/hiw/app/models/order.rb:201:in `create_for_products'
  * /home/tjic/bus/hi/src/hiw/app/models/subscription.rb:365:in `update_line_items'
  * /home/tjic/bus/hi/src/hiw/app/models/subscription.rb:626:in `factory_create'
  * /home/tjic/bus/hi/src/hiw/app/models/subscription_list.rb:57:in `add_subscribable'

and BAM – there it is. Line 274 of order.rb. Thousands of lines of unit, functional, and integration tests hadn’t caught it, but there was a weird confluence of almost-right scopes (some using lambdas) chained together in such a way that if a variable that we never expected to be null, we’d do a mammoth select, and then cancel far too many line items.

Problem solved.

…and I never could have done it without paper_trail !

Posted in Uncategorized | Leave a comment

the Unix Way (small tools chained together) used to find the size of a front yard

While house hunting with Google maps and realtor.com a friend asked me “how big a piece of property that’s 20 acres?”

Google maps conveniently has a key in the lower left hand corner, so we can see what 200 or 500 or 1,000 linear feet looks like.

So the question is: “if we assume the 20 acres are in the form of a square, how long is each side?”

The command ‘units’ can tell us the number of square feet in an acre.

    units '20 acres' 'ft^2'

yields

    * 871203.48
    / 1.1478375e-06

Now we want to pull out just that first number.

Piping this through

    head -1

peels off the first line, and then a sequence of calls to ‘cut’ gives us just the number we want

    units '20 acres' 'ft^2' | head -1 | cut -f2 | cut -d' ' -f2 
    => 871203.48

Now we need to feed this into ‘dc’ to take a square root. We know that unix command ‘xargs’ is going to come into play…but how do we get dc to accept things from the command line?

With the flag “-”, thusly:

    echo "arg1" "arg2" | dc - 

We can build the dc command with

    echo <square feet> <square root operator> <print operator> | dc -

Putting it all together we get

    echo  `units '20 acres' 'ft^2' | head -1 | cut -f2 | cut -d' ' -f2 ` "v p" | dc -
    => 933.38

This is a lot of research to do for one just one calculation, but it ties together a bunch of concepts and tools that are useful for a general-purpose hacker:

  • The unix philosophy of small programs and pipes
  • the ‘head’ command
  • the ‘cut’ command
  • the ‘units’ command
  • the ‘dc’ command

Add these tools to your repertoire and you’ll not regret it.

Posted in Uncategorized | Leave a comment

ActiveRecord, DateTime, Time, and time zones

Note: everything below applies to Ruby 1.9.3p429 and Rails 3.2.13.

The problem

The HeavyInk.com auction feature suffered a hiccup yesterday.

The auctions were scheduled to go live at noon and they didn’t – what had gone wrong?

The problem – in more detail

The answer turns out to hinge on ActiveRecord, DateTime, Time, and time zones, so I’ll explore a bit:

First, the auctions are active records and look like this:


create_table "auctions", :force => true do |t|
  t.integer  "product_id",                                                   :null => false
  t.string   "product_type",                                                 :null => false

  t.datetime "time_start",                                                   :null => false
  t.datetime "time_end",                                                     :null => false

end

The controller is trivial:


class AuctionController < ApplicationController
  def index
    @auctions = Auction.current
  end
end


You see the scope “current”, and you see the two fields time_start and time_end in the class definition, so you can already guess what the scope does, and you’re right:


class Auction < ActiveRecord::Base

  scope :current,      lambda { { :conditions => ["(ISNULL(time_start) OR time_start <= ?) AND (ISNULL(time_end) OR time_end >= ?)", DateTime.now, DateTime.now] }}

  def self.generate_for_incentives
    incentive_covers = ...

      time_start = Date.today.following_monday.to_datetime + 12.hours
    time_end   = time_start + 2.hours

    auctions = incentive_covers.each do |inc| 
      Auction.create!(:product => inc, :time_start =>time_start, :time_end =>time_end...)
    end
  end
end
  
  

The problem was that when the auctions should have been live on the website, they weren’t – a query of the form

Auction.current

returned nothing.

Why?

A digression – timezones

Let’s take a quick look at timezones. As you know, the Earth has 24 timezones. The “reference” timezone is Greenwich Mean Time
UTC time. Wikipedia tells us that “For most purposes, UTC is synonymous with GMT”.

Now, an immediate question pops up: how does Rails handle times in active records?

An excellent question!

The answer is that all times are stored as UTC times, and displayed in your local time zone.

If you’re in Greenwich England, your application can generate a timestamp (like 2:13pm) and store it in the database, and the two values are identical. If you’re in The Eastern US (+4 hours), things are different: your application generates a timestamp (e.g. 2:13pm) and ActiveRecord swizzles that into 16:13 before storing it to the database. Later, when you retrieve that activerecord, the auto-magic happens in reverse and Customer.created_at (for example) returns exactly the timestamp you’d expect.

(Note that other applications do similar things: mysql does exactly the same hack for Timestamp values.)

How does ActiveRecord know what offset to use?

Easy! You specify it! (By the way, if you haven’t already done this, now is a good time to update your application.rb files):

module Heavyink
  class Application < Rails::Application

    ...
    config.time_zone = "Eastern Time (US & Canada)"
    ...

  end
end

The solution

Now, back to the problem.

At the core, the issue is that I was generating a time value that did not have time zone information inside it, so when it was stored into the database the magic didn’t happen.

Specifically, I generated a timestamp along lines like this:


Date.today.to_datetime


Note the result


=> Tue, 03 Sep 2013 00:00:00 +0000 


Note the +0000 at the end – no time zone information!

If I had instead done this:


Date.today.to_time


the result would be


=> 2013-09-03 00:00:00 -0400 

…which includes the timezone information (US Eastern Time is four hours off from UTC).

So, I changed this line of code:


time_start = Date.today.following_monday.to_datetime + 12.hours

to


time_start = Date.today.following_monday.to_time + 12.hours

and everything worked.

The moral

Once upon a time Ruby Dates held just the date (with no time information), Ruby Times held just time information (with no dates) and Ruby DateTimes held both.

Since then the Time class has grown to hold dates as well, and ActiveRecord has started to use ActiveSupport::TimeWithZone.

you can convert back and forth between all of these

- But -

Times and DateTimes support .zone() and Dates do not


1.9.3-p429 :008 > Time.now.zone
=> "EDT" 
1.9.3-p429 :009 > DateTime.now.zone
=> "-04:00" 
1.9.3-p429 :010 > Date.today.zone
NoMethodError

So if you start with a Date, your time zone doesn’t get stuck in it, and you’ll end up having problems later.

So if you need timezones (to compare with created_at, updated_at, or anything similar) make sure that you start with Time.

(see also)

Posted in Uncategorized | 1 Comment

Best Twitter Gem for Ruby on Rails

The last time someone answered the question “what’s the best twitter gem for Ruby on Rails?” was three years ago.

The landscape has changed since then.

Twitter4r has gotten stale – over two years since the last updates!

The gem twitter gem is actively developed, and even has a twitter account (@gem, aptly enough).

I’ve looked over the code and it looks clean and good, so I’m moving HeavyInk over to it now.

Posted in Uncategorized | Leave a comment