I recently finished a contract with BBC Future Media working on the London 2012 Olympics coverage. I was part of the Sport Olympic Service team which published pages for each athlete, country, discipline, event and venue as well as the medal tables.
My role was as a Developer-in-Test, in which I used my coding background to help build quality into the end product using techniques such as Specification by Example, Continuous Integration and Behaviour-Driven Development.
This post describes how we made use of Ruby, Cucumber and some other tools in our approach. (I have simplified some aspects for the sake of readability and conciseness).
The BBC Forge platform uses PHP and Zend at the front-end and connects to a number of back-end services over HTTP. For the Olympics coverage, some data would provided by a third-party as a REST API serving XML responses.
A major challenge was that the third-party APIs had not yet been built. We were given examples of what to expect in the responses but our development would have to be done in parallel with the third-party’s own development.
We wrote specifications using the Gherkin format, keeping in mind Mike Cohn’s INVEST guidelines which say stories should be:
As an example, consider the story for the medals table. There are very particular rules for the correct order to list the countries in the medals table. Working with developers and business analysts, we came up with the following acceptance criteria:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
(The IOC code is a three-letter acronym such as USA or GBR)
Our data provider supplied examples of the XML format which looked something like this:
1 2 3 4
So in order to test all five scenarios above, a different XML fixture would be needed for each one. Creating these by hand would be time-consuming, and most of the feeds were more complex than this example. Additionally, the XML structure was subject to change, which would also add to the maintenance burden.
So to automate this, I built a template with placeholders which could be populated with the appropriate data for each scenario.
I chose to use the Liquid templating library as the curly braces stand out from the surrounding XML, but there are numerous other viable choices. Converted into a Liquid template, the XML example above would look something like:
So then, the required fixtures could be generated using code such as:
1 2 3 4 5 6 7 8 9 10 11
The next challenge was how to run tests against these fixtures. As the site isn’t built in Ruby, stubbing the HTTP classes wasn’t an option. Instead, I used REST-Assured, a highly-useful testing tool built by another BBC contractor Artem Avetisyan.
REST-Assured has a simple API which defines that a particular query string should give a specified response:
1 2 3 4
We ran REST-Assured on Heroku for convenience. For the testing environment, we added a hook so that a the API endpoint configuration could be overridden using a query string parameter, e.g. http://test.bbc.co.uk/olympics/medals?host=bbc382981.herokuapp.com
These Cucumber scenarios formed part of our continous integration builds running on Hudson, helping to make the BBC’s London 2012 coverage a great success.