I’ve heard rspec-fire mentioned in a few talks by Gary Bernhardt as a way to keep test doubles in sync with their real classes. This prevents the risk of all the isolated tests passing but the actual app being broken, which is a common concern when doing GOOS-style testing where all the collaborators are replaced by test doubles.
Paraphrasing from the rspec-fire README:
rspec-fire only checks that the methods exist if the doubled class has already been loaded. No checking will happen when running the spec in isolation, but when run in the context of the full app (either as a full spec run or by explicitly preloading collaborators on the command line) a failure will be triggered if an invalid method is being stubbed.
It took me a little time to get my head around how rspec-fire works, so I’ve written up a slightly extended example here.
Here’s what the spec for this class would look like, just using RSpec’s built-in support for test doubles:
describeUser,'#suspend!'doit'sends a notification'donotifier=double("EmailNotifier")notifier.should_receive(:notify).with("suspended as")user=User.new(notifier)user.suspend!endend
(Keep in mind that in RSpec, mock and stub are simply aliases for double)
Our EmailNotifier would look like this:
Run rspec spec/user_spec.rb and this should pass.
Now let’s say the requirements change. We want to add an option to specify whether the notification should be in plain text or html. The spec becomes:
describeUser,'#suspend!'doit'sends a notification in the appropriate format'donotifier=double("EmailNotifier")format=stubnotifier.should_receive(:notify).with("suspended as",format)user=User.new(notifier)user.suspend!(format)endend
To make this spec pass, we change the user class so that #suspend! takes a format parameter:
That command looks slightly cryptic. Let’s check what the RSpec docs say:
Usage: rspec [options] [files or directories]
-I PATH Specify PATH to add to $LOAD_PATH (may be used more than once).
-r, --require PATH Require a file.
sends a notification (FAILED - 1)
We’re telling RSpec to require the real EmailNotifier class. When we call fire_double, it sees that the real class is loaded, uses that instead of the test double, and so triggers a failure:
1) User#suspend! sends a notification in html
Failure/Error: notifier.should_receive(:notify).with("suspended as", :html)
Wrong number of arguments for notify. Expected 1, got 2.
# ./spec/user_spec.rb:9:in `block (2 levels) in <top (required)>'
And the specs will now pass when testing against the real EmailNotifier.
So why does this all matter?
When all the unit tests are passing, but the actual app is still broken, a common reaction is to add more integration and system tests as a ‘safety-net’. But these kind of tests tend to be brittle and slow to run.
We need both kinds of tests, but the vast majority of our test suite should be isolated unit tests.
In his talk Fast Test, Slow Test, Gary Bernhardt suggests aiming for a ratio of 90/10 between unit and system tests, approaching 95/5 or even 99/1 as time goes on.
rspec-fire gives you greater confidence that the isolated tests’ boundaries line up with the real system.
A recent episode of Railscasts Pro covered two approaches to splitting up a large Rails model – Concerns and Service Objects.
Concerns are supported by Rails 3 but will become more ‘official’ in Rails 4. There’s some controversy about whether Concerns or Service Objects are the best approach. DHH is pushing Concerns but OO proponents tend to prefer Service Objects. I believe that both have their place and can be used together.
I began by looking at one of my ActiveRecord model classes to see what it made sense to extract. Initially I created seperate concerns for validations, assocations, and accessors, e.g.:
In this case, the concern in app/models/concerns/trader/schedule.rb would be used, and the other ignored.
However, if that concern was removed, the model would pick up the other concern. So a model will search in it’s own concerns directory first, and if the concern isn’t found there, it will look in the top-level concerns directory.
If you prefer a more explicit way of indicating a shared concern, you can namespace it:
Zeus is rapidly overtaking Spork as the tool of choice to speed up running Cucumber integration tests against Rails apps.
I wanted to use Zeus alongside Guard to provide a rapid feedback cycle when doing BDD. This ended up being quite fiddly to get working, and it seems not many people are doing this yet, so here I will describe my configuration.
These instructions are for Mac OS X 10.8. On other platforms you’ll probably need to pick different gems for notifications and filesystems events.
Zeus seems to be picky about which groups it loads gems from, depending on what command you’re running. The configuration below works for me, but may not be optimal:
# just an extractgroup:developmentdogem'guard','~> 1.5.4'gem'guard-cucumber','~> 1.2.2'gem'guard-rspec','~> 2.3.0'gem'rb-fsevent','~> 0.9.2'endgroup:development,:testdogem'rspec','~> 2.12.0'gem'rspec-rails','~> 2.12.0'endgroup:testdogem'capybara-webkit','~> 0.7.1'gem'cucumber-rails','~> 1.1.0',:require=>falsegem'terminal-notifier-guard'# Mac OS X 10.8 onlygem'zeus','~> 0.12.0'end
I’ve specified versions to show which combination worked for me. As some of these gems need to interact closely with each other, you need to be careful when upgrading in order to keep them ‘in sync’.
Note that the Zeus gem is listed in the :test group, even though I’m not running Zeus through Bundler. I’m not entirely sure why it has to be here, but I get an error zeus is not part of the bundle when it’s not present.
My Guardfile is fairly standard. guard-rspec already has support for Zeus, so all I had to do was enable zeus and disable bundler:
guard-cucumber doesn’t have built-in support for Zeus, but the command_prefix setting can be used instead:
Green: Make the test pass with the simplest implementation possible
Refactor: Remove duplication or improve the design
I based my effort on the Scoring Project from RubyKoans. This already includes some tests but I wrote mine from scratch.
To recap the rules:
A set of three ones is 1000 points
A set of three numbers (other than ones) is worth 100 times the number. (e.g. three fives is 500 points).
A one (that is not part of a set of three) is worth 100 points.
A five (that is not part of a set of three) is worth 50 points.
Everything else is worth 0 points.
I started with what seemed like the simplest test possible:
describe"#score"doit"is 0 for an empty list"doscore().should==0endend
Let’s make it green:
So, how do you decide what test to write next? It seems that rules 3 and 4 are dependent on rules 1 and 2, so let’s postpone those for later and write a failing test for rule 1:
it"is 1000 for a set of three ones"doscore([1,1,1]).should==1000end
Sticking the rule of only writing the minimum code needed to pass the test:
A confession here: This is how I initially approached the problem, but I found I got stuck at a ‘deadlock’ position where it wasn’t possible to make progress with small, simple refactorings. Looking at the set of rules from a distance, we can see this the end result is an accumulation of the scores from each rule. Therefore, having a series of branches for alternative conditions probably isn’t going to work.
Let’s refactor the code to this to give a better base to build upon:
Now let’s consider the slightly tricker case of interdependent rules:
it"is 550 for a set including three fives along with a single five"doscore([5,5,5,5]).should==550end
The current implementation will give an incorrect answer of 700 because it’s counting the fives as a triple and then counting them again as single fives. We need to make sure they aren’t counted twice.
A simple way of doing that is to sort the array and then drop the first 3 values of the array:
I’m happy with that final solution, and it also passes the RubyKoans tests.
What I Learned
Triangulation is a useful technique but doesn’t bypass the need for the analysis and thinking required to solve tricky problems. If you try to do it on ‘autopilot’ you probably won’t succeed. The order you choose to write the tests, and the code you write early on can have a significant impact in how much effort is required discover to the algorithm.
My role was as a Developer-in-Test, in which I used my coding background to help build quality into the end product using techniques such as Specification by Example, Continuous Integration and Behaviour-Driven Development.
This post describes how we made use of Ruby, Cucumber and some other tools in our approach. (I have simplified some aspects for the sake of readability and conciseness).
The BBC Forge platform uses PHP and Zend at the front-end and connects to a number of back-end services over HTTP. For the Olympics coverage, some data would provided by a third-party as a REST API serving XML responses.
A major challenge was that the third-party APIs had not yet been built. We were given examples of what to expect in the responses but our development would have to be done in parallel with the third-party’s own development.
As an example, consider the story for the medals table. There are very particular rules for the correct order to list the countries in the medals table. Working with developers and business analysts, we came up with the following acceptance criteria:
Feature: Medals Table Countries are ranked by the number of golds, silvers, then bronzes, followed by their official IOC code.Scenario: No medals Given no medals have been awardedWhen I view the medals tableThen a placeholder message should be presentScenario: Different number of gold Given the medals: | country | gold | | United States | 3 | | China | 1 | | Great Britain | 2 |When I view the medals tableThen the ranking should be United States, Great Britain, ChinaScenario: Same number of gold but different number of silver Given the medals: | country | gold | silver | | United States | 2 | 3 | | China | 2 | 1 | | Great Britain | 2 | 2 |When I view the medals tableThen the ranking should be United States, Great Britain, ChinaScenario: Same number of gold and silver but different number of bronze Given the medals: | country | gold | silver | bronze | | United States | 2 | 2 | 3 | | China | 2 | 2 | 1 | | Great Britain | 2 | 2 | 2 |When I view the medals tableThen the ranking should be United States, Great Britain, ChinaScenario: Same number of gold, silver and bronze Given the United States, China and Great Britain have equal gold, silver and bronze When I view the medals tableThen the ranking should be China, Great Britain, United States
(The IOC code is a three-letter acronym such as USA or GBR)
Our data provider supplied examples of the XML format which looked something like this:
So in order to test all five scenarios above, a different XML fixture would be needed for each one. Creating these by hand would be time-consuming, and most of the feeds were more complex than this example. Additionally, the XML structure was subject to change, which would also add to the maintenance burden.
So to automate this, I built a template with placeholders which could be populated with the appropriate data for each scenario.
I chose to use the Liquid templating library as the curly braces stand out from the surrounding XML, but there are numerous other viable choices. Converted into a Liquid template, the XML example above would look something like:
So then, the required fixtures could be generated using code such as:
The next challenge was how to run tests against these fixtures. As the site isn’t built in Ruby, stubbing the HTTP classes wasn’t an option. Instead, I used REST-Assured, a highly-useful testing tool built by another BBC contractor Artem Avetisyan.
REST-Assured has a simple API which defines that a particular query string should give a specified response:
Given/^there no medals have been awarded "([^"])*"$/do|query|xml=Liquid::Template.parse(template).render'countries'=>RestAssured::Double.createfullpath:"/medals",content:xmlend
We ran REST-Assured on Heroku for convenience. For the testing environment, we added a hook so that a the API endpoint configuration could be overridden using a query string parameter, e.g. http://test.bbc.co.uk/olympics/medals?host=bbc382981.herokuapp.com
These Cucumber scenarios formed part of our continous integration builds running on Hudson, helping to make the BBC’s London 2012 coverage a great success.
I recently had to delete an S3 bucket containing over 200,000 objects. S3 prevents deletion of non-empty buckets, and deleting this many objects is virtually impossible using any GUI. I looked into an API-based approach but support for multi-object delete still seems limited.
If you don’t mind waiting 24 hours for your bucket to be cleared, there’s a very simple solution using S3’s lifecycle feature: