Fixtures Are Painful
I just got back from RailsConf 2007, it was a brain-expanding conference. I spent a lot of time talking to Rails coders about how they do testing and use fixtures. I saw some patterns emerge. Generally, people agree that fixtures are painful:
- referencing ids in the fixtures is annoying and prone to error
- the lack of a grouping mechanism makes selecting them harder
- fixtures get out of date with model changes
I plan to do a few posts about fixtures in the next coming weeks, to share some of what I have learned. For now, I’ll focus on validation.
The project I’m currently working on has a large number of fixtures (over 100 models). It’s become really hard to manage them all. Over time, some fixtures busted, and it became hard to diagnose random problems in tests.
We are fortunate to be working with “zenspider”:http://blog.zenspider.com/, he’s helping us get our tests, dev process and performance in better shape. I complained a lot about invalid fixtures, and how I longed for a Rake task that could identify the broken ones. He offered up this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Put it in your project Rakefile, and you can then run:
You will get back a list of all the fixtures that are not valid, with the validation messages.
But what about edge cases? What about “bad” form data? DHH has declared that most folks want all fixtures loaded at the start of the tests. The data in fixtures does not have to pass model validation before it is loaded into the db.
After discussing this with a number of folks, I’m starting to believe:
- All fixtures should be valid at all times
- Fixtures should provide stuff required by your app to run tests (your admin user, default options, etc)
- Edge cases should be created in tests, not fixtures
- Test invalid data by loading a good fixture and changing it
rake db:fixtures:validate is helpful for keeping them valid. I plan to use it whenever I do a migration, just to ensure that nothing got broken.