Andy Waite

Contract Ruby, Rails, JavaScript and iOS Developer

Cucumber, BDD and London 2012

Permalink

I recently finished a contract with BBC Future Media working on the London 2012 Olympics coverage. I was part of the Sport Olympic Service team which published pages for each athlete, country, discipline, event and venue as well as the medal tables.

My role was as a Developer-in-Test, in which I used my coding background to help build quality into the end product using techniques such as Specification by Example, Continuous Integration and Behaviour-Driven Development.

This post describes how we made use of Ruby, Cucumber and some other tools in our approach. (I have simplified some aspects for the sake of readability and conciseness).

Architecture Overview

The BBC Forge platform uses PHP and Zend at the front-end and connects to a number of back-end services over HTTP. For the Olympics coverage, some data would provided by a third-party as a REST API serving XML responses.

A major challenge was that the third-party APIs had not yet been built. We were given examples of what to expect in the responses but our development would have to be done in parallel with the third-party’s own development.

Requirements Analysis

We wrote specifications using the Gherkin format, keeping in mind Mike Cohn’s INVEST guidelines which say stories should be:

  • Independent
  • Negotiable
  • Valuable
  • Estimable
  • Small
  • Testable

As an example, consider the story for the medals table. There are very particular rules for the correct order to list the countries in the medals table. Working with developers and business analysts, we came up with the following acceptance criteria:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
Feature: Medals Table

  Countries are ranked by the number of golds, silvers,
  then bronzes, followed by their official IOC code.

  Scenario: No medals
    Given no medals have been awarded
    When I view the medals table
    Then a placeholder message should be present

  Scenario: Different number of gold
    Given the medals:
      | country       | gold |
      | United States | 3    |
      | China         | 1    |     
      | Great Britain | 2    |
    When I view the medals table
    Then the ranking should be United States, Great Britain, China

  Scenario: Same number of gold but different number of silver
    Given the medals:
      | country       | gold | silver |
      | United States | 2    | 3      |
      | China         | 2    | 1      |
      | Great Britain | 2    | 2      |
    When I view the medals table
    Then the ranking should be United States, Great Britain, China

  Scenario: Same number of gold and silver but different number of bronze
    Given the medals:
      | country       | gold | silver | bronze |
      | United States | 2    | 2      | 3      |
      | China         | 2    | 2      | 1      |
      | Great Britain | 2    | 2      | 2      |
    When I view the medals table
    Then the ranking should be United States, Great Britain, China

  Scenario: Same number of gold, silver and bronze
    Given the United States, China and Great Britain have equal gold, silver and bronze 
    When I view the medals table
    Then the ranking should be China, Great Britain, United States

(The IOC code is a three-letter acronym such as USA or GBR)

Our data provider supplied examples of the XML format which looked something like this:

1
2
3
4
<countries>
  <country name="United States" code="USA" gold="1" silver="2" bronze="3" />
  <country name="Great Britain" code="GBR" gold="1" silver="2" bronze="3" />
</countries>

So in order to test all five scenarios above, a different XML fixture would be needed for each one. Creating these by hand would be time-consuming, and most of the feeds were more complex than this example. Additionally, the XML structure was subject to change, which would also add to the maintenance burden.

So to automate this, I built a template with placeholders which could be populated with the appropriate data for each scenario.

I chose to use the Liquid templating library as the curly braces stand out from the surrounding XML, but there are numerous other viable choices. Converted into a Liquid template, the XML example above would look something like:

So then, the required fixtures could be generated using code such as:

1
2
3
4
5
6
7
8
9
10
11
Liquid::Template.parse(template).render 'countries' => [
  { name: 'United States',
    code: 'USA',
    gold: 1
    silver: 2,
    bronze: 3 },
  { name: 'Great Britain',
    code: 'GBR',
    gold: 1
    silver: 2,
    bronze: 3 } ]

The next challenge was how to run tests against these fixtures. As the site isn’t built in Ruby, stubbing the HTTP classes wasn’t an option. Instead, I used REST-Assured, a highly-useful testing tool built by another BBC contractor Artem Avetisyan.

REST-Assured has a simple API which defines that a particular query string should give a specified response:

1
2
3
4
Given /^there no medals have been awarded "([^"])*"$/ do |query|
  xml = Liquid::Template.parse(template).render 'countries' => []
  RestAssured::Double.create fullpath: "/medals", content: xml
end

We ran REST-Assured on Heroku for convenience. For the testing environment, we added a hook so that a the API endpoint configuration could be overridden using a query string parameter, e.g. http://test.bbc.co.uk/olympics/medals?host=bbc382981.herokuapp.com

These Cucumber scenarios formed part of our continous integration builds running on Hudson, helping to make the BBC’s London 2012 coverage a great success.

Comments