casperfabricius.com

When I recently started development of a new, big Rails project, I spend some time researching and considering the many different testing frameworks that are available in the Ruby community today. I have grown very satisfied with the fail/pass rhythm of test-driven development (TDD) – write a test that fails, then write just enough code to make it pass. I also like RSpec a lot, so some would say that I am fact doing behaviour-driven development (BDD) – to me, the only real difference between TDD and BDD is the syntax, and I tend to find RSpec’s examples slightly more readable TestUnit’s tests. TDD gives me a small gratification – a sense of accomplishment, albeit on a small scale – every time a test passes in my autospec terminal.

Howver, TDD’s basic rule of always writing a failing test before writing any new code, doesn’t always leave me feeling very productive. For this reason I have mostly abandoned testing views in isolation (as made possible by RSpec’s view tests), as well as testing Rails helpers with the exception of complicated helper methods with some business logic in them. I still tests views by, behold, viewing them in the browser, and I haven’t been swept away by Cucumber, Webrat, Selenium and their fellow high-level test frameworks yet – but I am open.

I can skip view-testing without feeling too bad about it (wonder how many potential customers that statement scared away), but although I live and breath by the Skinny Controller – Fat Models notion, I still find functional testing of my controllers to be a must in Rails TDD. Unfortunately, I have never really gotten along with RSpec’s default way of testing controllers. It feels bloated, and the RSpec mocking framework seems to – yes – mock me as I add more and more stubs and expectations to avoid tests from failing. So in my research for testing frameworks, I knew I had to find a better way to feel productive and happy about writing controller code again.

Many people hate test fixtures in a very intense and passionate way. I actually kind of like them, I just think they are a bit too stupid and hard to maintain. I also love all the good stuff coming from the thoughtbot guys, and their alternative to Rails’ fixtures – Factory Girl – is really, really good. Not just because it has a pretty name and a fancy syntax, but because it is really nothing more than Rails’ fixtures done right.

I am getting ahead of myself here. Testing seems to be something people have very different opinions about, and as such I don’t the perfect mix of testing frameworks for everyone exists. RSpec has probably come closest, and I am sticking with in my test setup while adding and replacing parts of it with newer and better solutions. The mix has come like this:

• RSpec as the overall testing framework
• RR (DoubleRuby) for mocking and stubbing
• Shoulda to get sweet test macros
• Factory Girl to create test data
• Stubble for easing my controller spec pains

Do I really need five testing frameworks to be productive with TDD? Not at all, Rails is bundled with all I need to do TDD. But along the way, I have seen the need to make my tests DRY’er, more readable and easier to maintain, and these frameworks help me with that in different ways. I have already touched on the RSpec vs. TestUnit issue: It comes down to personal taste, and RR, Shoulda, Factory Girl and Stubble all works with both of these frameworks – isn’t that nice?

I initially chose to try RR because it seemed to be what the cool kids are using at the moment (and that often turns out to be good), but now that I am getting into it, I am really starting appreciate its compact syntax and advanced features. You could argue that RR’s syntax:

  mock(User).find { @user }

- is much harder to read than, say, RSpec’s mocking syntax:

  User.should_receive(:find).and_return(@user)

- but I’d say that RR’s syntax is actually more “Ruby-ish” than RSpec’s. If you really understand Ruby, RR will most likely appeal to you.

I use Shoulda mainly to get DRY’er tests that are easier to read, than when I write macros myself. Shoulda has defined a set macros that are quickly becoming a standard, and as something quite new, they also exists as RSpec matchers. Statements such as:

  it { should validate_presence_of(:state) }

- simply makes TDD faster and more enjoyable, because it would feel wrong to have to write 3-5 lines of test code to test a single line of Rails code.

I can’t say if Factory Girl is better than, say, Machinist (which is actually heavily inspired by the former, according to the creator), but it is without a about much better than Rails’ fixtures from maintenance point of view, the syntax is pretty and it feels productive to use.

If you already knew about everything else mention in this article, chances are that Stubble will still be news to you. Not even released in a version 0.0.1 yet, this project is a small testing framework aimed at making stubbing and mocking in controller tests much easier. When the author, David Chelimsky – who is also behind RSpec – mentioned this in passing at a presentation at RailsConf 2009, my curiosity was immediately piqued. Could this be solution to functional testing woes? Short answer: I’m not really sure yet. It’s still too early to say. Currently, I can only get Stubble to work in very simple cases like this:

  describe "on GET to :show" do
    def do_get(params = {})
      stubbing(Teaser) do |teaser|
        get :show, {:id => teaser.id}.merge(params)
        yield(teaser)
      end
    end

    it { do_get { |teaser| should assign_to(:teaser).with(teaser) } }
    it { do_get { |teaser| should render_template('show') } }
  end

- where I have even had to wrap the stubbing, which is the core of Stubble, in my own method to get the compact testing style I like. I don’t think I use Stubble in the way it is intended, but I’d like to be able to simply wrap do my entire outer describe in the stubbing statement, this having the magical teaser variable available in all my tests, helper methods and before-statements. Also, I have not yet managed to mock or stub anything on the magical teaser variable, so if I for example want to verify that some method is called on teaser, I can’t use Stubble in that test. Again, to be fair, Stubble is in very early development, and I still need to learn about it, but if these two problems are solved, I think Stubble is going to ease a lot of my controller testing pain.

If you, dear reader, feel that you have a better, or simply different, mix of testing frameworks, I’d love a comment from you about your choices and reasons for going with one thing over the other.