fhwang.net

On fixtures and first-time testers

Over on Josh Susser’s blog, DHH seems to be upset that I called the selling of fixtures to Rails newcomers a lie. Reply below:

Basically, you can lie to someone with the best of intentions, and you can even lie to someone without meaning to. I think this is always a pitfall when you try to simplify, and when you try to sell something.

I’m not disputing the good intentions. DHH and the rest of the Rails-core folks deserve tremendous credit for smoothing the learning curve for web programmers. Over the years I’ve personally seen a number of computer programmers “grow up with Rails”, so to speak, and I think it helped that Rails put a tremendous amount of energy into simplifying the path for newcomers, and then marketing that simplicity effectively.

But there’s a difference between accidental complexity and essential complexity. Some things are hard because people make them hard, but some things are hard because they’re hard. As I’ve argued before—this was a big part of my RubyConf talk last year—I think that testing is an essentially hard task. There’s no silver bullet. We don’t do it because it’s easy. We do it because it’s worth the time, and because we’re professionals.

I don’t want to minimize the difficulty that Rails faces in educating less-experience programmers. I know there are a lot of people coming to Rails that have never done serious OO and never done testing, and if testing is made to sound too scary or difficult, they’ll just keep putting it off.

But what I’ve never understood is why fixtures are actually easier for a noob than just creating the model in Ruby. Why is this:

david:
  id: 1
  login: david

... simpler than this?

@david = User.new :login => 'david'

If anything I’d argue that the 2nd form is far far simpler, because it uses Ruby, a syntax that you’re already using to write the app itself. Fixtures, on the other hand, use YAML, which some percent of Rails newcomers have never used. So, great, one more syntax to learn, as if a web programmer doesn’t have enough on her mind, between SQL and Ruby and Rails and HTML and Javascript and CSS. (In general, I’d say I’m not a big fan of using YAML when you’re also using Ruby, which is already a very elegant way to write a configuration file.)

In fact, I wonder if fixtures are detrimental as a teaching tool because they seem to imply there’s something special about creating or modifying test data. I’ve had a few conversations with people who never tested before Rails, and I say “why don’t you just save the model in your test setup, using Ruby”, and they have this look on their face like that never occurred to them. It’s as if fixtures somehow mystify the process a bit.

Of all the people I know who have been doing Rails development for some time, and who have reasonably well-tested code, almost none of them use fixtures. Those that do use fixtures use them in extremely precise ways, often mixing them with other test setup techniques, usually either RSpec-style mocking, or just straight-up creating records in the test DB. In contrast, more than once I have had a conversation with somebody who’s relatively new to Rails that includes them saying “yeah, once I stopped using fixtures all those problems went away.”

If anything, the people I know who are using fixtures do so for the performance optimization, since fixtures do a good job of hooking into transactions in the underlying DB. But if that’s the primary argument for fixtures, then why are they the default? Isn’t that a premature optimization? Shouldn’t we first encourage Rails noobs to test the testing waters with very simple data creation in Ruby form? And then when they start to be unhappy about how slowly their tests run, we can introduce different optimizations, such as fixtures or RSpec mocking or whatnot. That seems like the right order to me.

blog comments powered by Disqus

« Previous post

Next post »