Minimal Rakefile for RSpec, Cucumber and Jasmine

Feb 28th, 2013 • Permalink

I found I was repeatedly Googling for the Rakefile configuration to run various kinds of tests, so for my own reference, and hopefully someone else’s, here’s an example:

Note that you’ll need the guard-jasmine gem for the Jasmine task.

require 'rspec/core/rake_task'
require 'cucumber/rake/task'
require 'guard/jasmine/task'

RSpec::Core::RakeTask.new
Cucumber::Rake::Task.new
Guard::JasmineTask.new

task :default => [:spec, :cucumber, 'guard:jasmine']

Playing With (RSpec)-Fire

Feb 21st, 2013 • Permalink

I’ve heard rspec-fire mentioned in a few talks by Gary Bernhardt as a way to keep test doubles in sync with their real classes. This prevents the risk of all the isolated tests passing but the actual app being broken, which is a common concern when doing GOOS-style testing where all the collaborators are replaced by test doubles.

Paraphrasing from the rspec-fire README:

rspec-fire only checks that the methods exist if the doubled class has already been loaded. No checking will happen when running the spec in isolation, but when run in the context of the full app (either as a full spec run or by explicitly preloading collaborators on the command line) a failure will be triggered if an invalid method is being stubbed.

It took me a little time to get my head around how rspec-fire works, so I’ve written up a slightly extended example here.

I followed the instructions in the README and found a small problem, but within minutes @myronmarston had fixed it.

Let’s start with the class that we want to test. The example in the rspec-fire README inherits from Struct to keep things concise, but for clarity let’s make it a normal class:

class User
  def initialize(notifier)
    @notifier = notifier
  end

  def suspend!
    @notifier.notify("suspended as")
  end
end

You can check out my code at https://github.com/andyw8/try-rspec-fire.

Here’s what the spec for this class would look like, just using RSpec’s built-in support for test doubles:

describe User, '#suspend!' do
  it 'sends a notification' do
    notifier = double("EmailNotifier")

    notifier.should_receive(:notify).with("suspended as")

    user = User.new(notifier)
    user.suspend!
  end
end

(Keep in mind that in RSpec, mock and stub are simply aliases for double)

Our EmailNotifier would look like this:

class EmailNotifier
  def notify(message)
    # ...
  end
end

Run rspec spec/user_spec.rb and this should pass.

Now let’s say the requirements change. We want to add an option to specify whether the notification should be in plain text or html. The spec becomes:

describe User, '#suspend!' do
  it 'sends a notification in the appropriate format' do
    notifier = double("EmailNotifier")
    format = stub

    notifier.should_receive(:notify).with("suspended as", format)

    user = User.new(notifier)
    user.suspend!(format)
  end
end

To make this spec pass, we change the user class so that #suspend! takes a format parameter:

class User
  def initialize(notifier)
    @notifier = notifier
  end

  def suspend!(format)
    @notifier.notify("suspended as", format)
  end
end

The spec now passes. However, if we tried to use this in a real app it would be broken, because we haven’t updated our EmailNotifier class.

This is the problem that spec-fire solves.

We first need to add a few lines to spec_helper.rb:

require 'rspec/fire'

RSpec.configure do |config|
  config.include(RSpec::Fire)
end

And we need one small change to the spec, changing double("EmailNotifier") to fire_double("EmailNotifier").

We then re-run the spec, but this time we preload the collaborator:

$ rspec -I lib -r email_notifier.rb spec/user_spec.rb

That command looks slightly cryptic. Let’s check what the RSpec docs say:

Usage: rspec [options] [files or directories]

    -I PATH                          Specify PATH to add to $LOAD_PATH (may be used more than once).
    -r, --require PATH               Require a file.

User#suspend!
  sends a notification (FAILED - 1)

We’re telling RSpec to require the real EmailNotifier class. When we call fire_double, it sees that the real class is loaded, uses that instead of the test double, and so triggers a failure:

1) User#suspend! sends a notification in html
   Failure/Error: notifier.should_receive(:notify).with("suspended as", :html)
     Wrong number of arguments for notify. Expected 1, got 2.
   # ./spec/user_spec.rb:9:in `block (2 levels) in <top (required)>'

Let’s fix EmailNotifier:

class EmailNotifier
  def notify(message, format)
    # ...
  end
end

And the specs will now pass when testing against the real EmailNotifier.

So why does this all matter?

When all the unit tests are passing, but the actual app is still broken, a common reaction is to add more integration and system tests as a ‘safety-net’. But these kind of tests tend to be brittle and slow to run.

We need both kinds of tests, but the vast majority of our test suite should be isolated unit tests.

In his talk Fast Test, Slow Test, Gary Bernhardt suggests aiming for a ratio of 90/10 between unit and system tests, approaching 95/5 or even 99/1 as time goes on.

rspec-fire gives you greater confidence that the isolated tests’ boundaries line up with the real system.

Exploring Concerns for Rails 4

Dec 23rd, 2012 • Permalink

A recent episode of Railscasts Pro covered two approaches to splitting up a large Rails model – Concerns and Service Objects.

Concerns are supported by Rails 3 but will become more ‘official’ in Rails 4. There’s some controversy about whether Concerns or Service Objects are the best approach. DHH is pushing Concerns but OO proponents tend to prefer Service Objects. I believe that both have their place and can be used together.

An Example

I began by looking at one of my ActiveRecord model classes to see what it made sense to extract. Initially I created seperate concerns for validations, assocations, and accessors, e.g.:

# app/models/trader.rb
class Trader
  include Accessors
  include Validations
  include Associations
  ...
end

But I then realised this was probably a bad idea:

If I add new behaviour to Trader then I’d probably need to change more than one of these concerns.
It’s unlikely I’d be able to re-use any of those concerns for another model.

So instead, I tried to group the code into ‘topics’ and came up with the following:

# app/models/trader.rb
class Trader
  include BasicInfo
  include FriendlyURL
  include Location
  include OpeningHours
  include Schedule
  include Permissions
end

In general, I believe a concern should be named after a domain concept (which I think is true for all of the above, except perhaps FriendlyURL).

Let’s look at the Schedule concern as an example:

# app/models/concerns/trader/schedule.rb
class Trader
  module Schedule
    extend ActiveSupport::Concern

    included do
      attr_accessible :market_days_ids

      has_many :appearances
      has_many :market_days, :through => :appearances
    end

    def open?
      ...
    end

    def closed?
      ...
    end
  end
end

As shown, it can include a mix of validations, accessibilty, associations, as well as the plain methods. This is a much better way of grouping related functionality.

Sharing Concerns

One thing that wasn’t clear from the documentation and other articles I’ve seen is how a concern can be shared amongst multiple models. Let’s say you have two concerns with the same name:

app/models/concerns/schedule.rb
app/models/concerns/trader/schedule.rb

and the following model:

# app/models/trader.rb
class Trader
  include Schedule
end

In this case, the concern in app/models/concerns/trader/schedule.rb would be used, and the other ignored.

However, if that concern was removed, the model would pick up the other concern. So a model will search in it’s own concerns directory first, and if the concern isn’t found there, it will look in the top-level concerns directory.

If you prefer a more explicit way of indicating a shared concern, you can namespace it:

# app/models/trader.rb
class Trader
  include Shared::Schedule
end

# app/models/concerns/shared/schedule.rb
module Shared::Schedule
  extend ActiveSupport::Concern
  ...
end

I hope you find this useful.

Using Zeus With Cucumber and Guard

Dec 5th, 2012 • Permalink

Zeus is rapidly overtaking Spork as the tool of choice to speed up running Cucumber integration tests against Rails apps.

I wanted to use Zeus alongside Guard to provide a rapid feedback cycle when doing BDD. This ended up being quite fiddly to get working, and it seems not many people are doing this yet, so here I will describe my configuration.

These instructions are for Mac OS X 10.8. On other platforms you’ll probably need to pick different gems for notifications and filesystems events.

Gemfile

Zeus seems to be picky about which groups it loads gems from, depending on what command you’re running. The configuration below works for me, but may not be optimal:

# just an extract

group :development do
  gem 'guard', '~> 1.5.4'
  gem 'guard-cucumber', '~> 1.2.2'
  gem 'guard-rspec', '~> 2.3.0'
  gem 'rb-fsevent', '~> 0.9.2'
end

group :development, :test do
  gem 'rspec', '~> 2.12.0'
  gem 'rspec-rails', '~> 2.12.0'
end

group :test do
  gem 'capybara-webkit', '~> 0.7.1'
  gem 'cucumber-rails', '~> 1.1.0', :require => false
  gem 'terminal-notifier-guard' # Mac OS X 10.8 only
  gem 'zeus', '~> 0.12.0'
end

I’ve specified versions to show which combination worked for me. As some of these gems need to interact closely with each other, you need to be careful when upgrading in order to keep them ‘in sync’.

Note that the Zeus gem is listed in the :test group, even though I’m not running Zeus through Bundler. I’m not entirely sure why it has to be here, but I get an error zeus is not part of the bundle when it’s not present.

Guardfile

My Guardfile is fairly standard. guard-rspec already has support for Zeus, so all I had to do was enable zeus and disable bundler:

guard 'rspec', zeus: true, bundler: false do
  ...
end

guard-cucumber doesn’t have built-in support for Zeus, but the command_prefix setting can be used instead:

guard 'cucumber', command_prefix: 'zeus', bundler: false do
  ...
end

Booting Up

Start Zeus without using Bunder:

$ zeus start

Then start Guard via Bundler:

$ bundle exec guard

Now you can start building!

TDD Triangulation Practice

Aug 22nd, 2012 • Permalink

After attending Jason Goreman’s Intensive TDD workshop, I decided to have some practice at the triangulation aspect of Test-Driven Development. I strictly kept to the TDD mantra of:

Red: Write a failing test
Green: Make the test pass with the simplest implementation possible
Refactor: Remove duplication or improve the design
Repeat

I based my effort on the Scoring Project from RubyKoans. This already includes some tests but I wrote mine from scratch.

To recap the rules:

A set of three ones is 1000 points
A set of three numbers (other than ones) is worth 100 times the number. (e.g. three fives is 500 points).
A one (that is not part of a set of three) is worth 100 points.
A five (that is not part of a set of three) is worth 50 points.
Everything else is worth 0 points.

I started with what seemed like the simplest test possible:

describe "#score" do
  it "is 0 for an empty list" do
    score([]).should == 0
  end
end

Let’s make it green:

def score(dice)
  0
end

So, how do you decide what test to write next? It seems that rules 3 and 4 are dependent on rules 1 and 2, so let’s postpone those for later and write a failing test for rule 1:

it "is 1000 for a set of three ones" do
  score([1,1,1]).should == 1000
end

Sticking the rule of only writing the minimum code needed to pass the test:

def score(dice)
  if dice == [1,1,1]
    1000
  else
    0
  end
end

A confession here: This is how I initially approached the problem, but I found I got stuck at a ‘deadlock’ position where it wasn’t possible to make progress with small, simple refactorings. Looking at the set of rules from a distance, we can see this the end result is an accumulation of the scores from each rule. Therefore, having a series of branches for alternative conditions probably isn’t going to work.

Let’s refactor the code to this to give a better base to build upon:

def score(dice)
  total = 0
  total += 1000 if dice == [1,1,1]
  total
end

Write another failing test for rule 1:

  it "is 1000 for a set including three ones" do
    score([1,2,1,1]).should == 1000
  end

Make it green:

def score(dice)
  total = 0
  total += 1000 if dice == [1,1,1] || dice == [1,2,1,1]
  total
end

At this point we can see duplication starting to appear, so it’s time to refactor and generalise the solution:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) == 3
  total
end

Now let’s write a failing test for rule 2:

it "is 600 for a set of three sixes" do
  score([6,6,6]).should == 600
end

Make it green:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) == 3
  total += 600 if dice == [6,6,6]
  total
end

Write another failing test for rule 2:

it "is 600 for a set including three sixes" do
  score([6,6,6,1]).should = 600
end

Make it green:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) == 3
  total += 600 if dice == [6,6,6] || dice == [6,6,6,1]
  total
end

Refactor to remove duplication:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) == 3
  total += 600 if dice.count(6) == 3
  total
end

Write another failing test for rule 2:

it "is 200 for a set including three twos" do
  score(3,2,2,2,4]).should = 200
end

Make it green:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) == 3
  total += 600 if dice.count(6) == 3
  total += 200 if dice.count(2) == 3
  total
end

We can see some duplication creeping in - we would need five lines to cover three twos up to three sixes, so let’s refactor to generalise:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) == 3
  2.upto(6).each do |i|
    total += i * 100 if dice.count(i) == 3
  end
  total
end

We also need to consider the case of more than three of the same value:

it "is 200 for a set including three twos" do
  score(2,2,2,2]).should = 200
end

A simple change makes this green:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) >= 3
  2.upto(6).each do |i|
    total += i * 100 if dice.count(i) == 3
  end
  total
end

Let’s add a failing test for Rule 3:

it "is 100 for a one (that is not part of a set of three)" do
  score([1]).should = 100
end

Make it green:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) >= 3
  2.upto(6).each do |i|
    total += i * 100 if dice.count(i) == 3
  end
  total += 100 if score == [1]
  total
end

And another failing test for Rule 3:

it "is 200 for two ones (that are not part of a set of three)" do
  score([1,1]).should = 200
end

Make it green:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) >= 3
  2.upto(6).each do |i|
    total += i * 100 if dice.count(i) == 3
  end
  total += 100 if score == [1]
  total += 200 if score == [1,1]
  total
end

Another failing test:

it "is 200 for a set including two ones (that are not part of a set of three)" do
  score([1,3,1]).should = 200
end

Make it green:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) == 3
  2.upto(6).each do |i|
    total += i * 100 if dice.count(i) == 3
  end
  total += 100 if score == [1]
  total += 200 if score == [1,1]
  total += 200 if score == [1,3,1]
  total
end

Refactor and generalise:

def score(dice)
  total = 0
  total += 1000 if dice.count(1) >= 3
  2.upto(6).each do |i|
    total += i * 100 if dice.count(i) >= 3
  end
  total += 100 * dice.count(1)
  total
end

Which can be further improved:

def score(dice)
  total = 0
  dice.uniq.each do |i|
    next if dice.count(i) < 3
    total += i == 1 ? 1000 : i * 100
  end
  total += 100 * dice.count(1)
end

Write a failing test for rule 4:

it "is 100 for a set including two fives (that are not part of a set of three)" do
  score([5,3,5]).should = 100
end

Make it green:

def score(dice)
  total = 0
  dice.uniq.each do |i|
    next if dice.count(i) < 3
    total += i == 1 ? 1000 : i * 100
  end
  total += 100 * dice.count(1)
  total += 50 * dice.count(5)
end

Now let’s consider the slightly tricker case of interdependent rules:

it "is 550 for a set including three fives along with a single five" do
  score([5,5,5,5]).should == 550
end

The current implementation will give an incorrect answer of 700 because it’s counting the fives as a triple and then counting them again as single fives. We need to make sure they aren’t counted twice.

A simple way of doing that is to sort the array and then drop the first 3 values of the array:

def score(dice)
  total = 0
  dice.sort!
  dice.uniq.each do |i|
    next if dice.count(i) < 3
    total += i == 1 ? 1000 : i * 100
    dice = dice.drop(3)
  end
  total += 100 * dice.count(1)
  total += 50 * dice.count(5)
end

I’m happy with that final solution, and it also passes the RubyKoans tests.

What I Learned

Triangulation is a useful technique but doesn’t bypass the need for the analysis and thinking required to solve tricky problems. If you try to do it on ‘autopilot’ you probably won’t succeed. The order you choose to write the tests, and the code you write early on can have a significant impact in how much effort is required discover to the algorithm.

Cucumber, BDD and London 2012

Aug 20th, 2012 • Permalink

I recently finished a contract with BBC Future Media working on the London 2012 Olympics coverage. I was part of the Sport Olympic Service team which published pages for each athlete, country, discipline, event and venue as well as the medal tables.

My role was as a Developer-in-Test, in which I used my coding background to help build quality into the end product using techniques such as Specification by Example, Continuous Integration and Behaviour-Driven Development.

This post describes how we made use of Ruby, Cucumber and some other tools in our approach. (I have simplified some aspects for the sake of readability and conciseness).

Architecture Overview

The BBC Forge platform uses PHP and Zend at the front-end and connects to a number of back-end services over HTTP. For the Olympics coverage, some data would provided by a third-party as a REST API serving XML responses.

A major challenge was that the third-party APIs had not yet been built. We were given examples of what to expect in the responses but our development would have to be done in parallel with the third-party’s own development.

Requirements Analysis

We wrote specifications using the Gherkin format, keeping in mind Mike Cohn’s INVEST guidelines which say stories should be:

Independent
Negotiable
Valuable
Estimable
Small
Testable

As an example, consider the story for the medals table. There are very particular rules for the correct order to list the countries in the medals table. Working with developers and business analysts, we came up with the following acceptance criteria:

Feature: Medals Table

  Countries are ranked by the number of golds, silvers,
  then bronzes, followed by their official IOC code.

  Scenario: No medals
    Given no medals have been awarded
    When I view the medals table
    Then a placeholder message should be present

  Scenario: Different number of gold
    Given the medals:
      | country       | gold |
      | United States | 3    |
      | China         | 1    |     
      | Great Britain | 2    |
    When I view the medals table
    Then the ranking should be United States, Great Britain, China

  Scenario: Same number of gold but different number of silver
    Given the medals:
      | country       | gold | silver |
      | United States | 2    | 3      |
      | China         | 2    | 1      |
      | Great Britain | 2    | 2      |
    When I view the medals table
    Then the ranking should be United States, Great Britain, China

  Scenario: Same number of gold and silver but different number of bronze
    Given the medals:
      | country       | gold | silver | bronze |
      | United States | 2    | 2      | 3      |
      | China         | 2    | 2      | 1      |
      | Great Britain | 2    | 2      | 2      |
    When I view the medals table
    Then the ranking should be United States, Great Britain, China

  Scenario: Same number of gold, silver and bronze
    Given the United States, China and Great Britain have equal gold, silver and bronze 
    When I view the medals table
    Then the ranking should be China, Great Britain, United States

(The IOC code is a three-letter acronym such as USA or GBR)

Our data provider supplied examples of the XML format which looked something like this:

<countries>
  <country name="United States" code="USA" gold="1" silver="2" bronze="3" />
  <country name="Great Britain" code="GBR" gold="1" silver="2" bronze="3" />
</countries>

So in order to test all five scenarios above, a different XML fixture would be needed for each one. Creating these by hand would be time-consuming, and most of the feeds were more complex than this example. Additionally, the XML structure was subject to change, which would also add to the maintenance burden.

So to automate this, I built a template with placeholders which could be populated with the appropriate data for each scenario.

I chose to use the Liquid templating library as the curly braces stand out from the surrounding XML, but there are numerous other viable choices. Converted into a Liquid template, the XML example above would look something like:

<countries> {% for country in countries %} <country name="{{ country.name }}" code="{{ country.code }}" gold="{{ country.gold }}" silver="{{ country.silver }}" bronze="{{ country.bronze }}" /> {% endfor %} </countries>

So then, the required fixtures could be generated using code such as:

Liquid::Template.parse(template).render 'countries' => [
  { name: 'United States',
    code: 'USA',
    gold: 1
    silver: 2,
    bronze: 3 },
  { name: 'Great Britain',
    code: 'GBR',
    gold: 1
    silver: 2,
    bronze: 3 } ]

The next challenge was how to run tests against these fixtures. As the site isn’t built in Ruby, stubbing the HTTP classes wasn’t an option. Instead, I used REST-Assured, a highly-useful testing tool built by another BBC contractor Artem Avetisyan.

REST-Assured has a simple API which defines that a particular query string should give a specified response:

Given /^there no medals have been awarded "([^"])*"$/ do |query|
  xml = Liquid::Template.parse(template).render 'countries' => []
  RestAssured::Double.create fullpath: "/medals", content: xml
end

We ran REST-Assured on Heroku for convenience. For the testing environment, we added a hook so that a the API endpoint configuration could be overridden using a query string parameter, e.g. http://test.bbc.co.uk/olympics/medals?host=bbc382981.herokuapp.com

These Cucumber scenarios formed part of our continous integration builds running on Hudson, helping to make the BBC’s London 2012 coverage a great success.

How to Easily Delete a Large Amazon S3 Bucket

Aug 8th, 2012 • Permalink

I recently had to delete an S3 bucket containing over 200,000 objects. S3 prevents deletion of non-empty buckets, and deleting this many objects is virtually impossible using any GUI. I looked into an API-based approach but support for multi-object delete still seems limited.

If you don’t mind waiting 24 hours for your bucket to be cleared, there’s a very simple solution using S3’s lifecycle feature:

Log into the S3 Management Console
Select the bucket you want to clear
Select Properties from the toolbar
Click the Lifecycle tab in the Properties panel
Add a rule with no prefix and expiration of 1 day
Save the rule

Check back 24 hours later you’ll find all your objects have been deleted.

RubyMotion App Template

May 28th, 2012 • Permalink

I’ve been trying out RubyMotion lately, and I’ve created a simple template project with some sensible defaults to make it quick and easy to start a new app:

Folders for models, views, controllers and vendored frameworks to encourage a Rails-esque layout convention
Spec folder layout to mirror the app folder
Support for TestFlight
Support for CocoaPods
Default values for app version and identifier
Placeholders for configuring icons and device families

You can find it at github.com/andyw8/rubymotion_app_template

First Post With Octopress

Apr 18th, 2012 • Permalink

Hello world!

Blog Archives

Andy Waite

Ruby and iOS Developer in London

Minimal Rakefile for RSpec, Cucumber and Jasmine

Playing With (RSpec)-Fire

Exploring Concerns for Rails 4

An Example

Sharing Concerns

Using Zeus With Cucumber and Guard

Gemfile

Guardfile

Booting Up

TDD Triangulation Practice

What I Learned

Cucumber, BDD and London 2012

Architecture Overview

Requirements Analysis

How to Easily Delete a Large Amazon S3 Bucket

RubyMotion App Template

First Post With Octopress