Autotest Sounds with playlists!

Apr 09, 2008

Ken Collins put together an awesome update to the autotest sound plugin. His version supports a playlist directory, so you can easily cycle through different init, red and green sounds. His sounds are hilarious!

http://www.metaskills.net/2008/4/6/autotest-playlist-for-red-green-feedback

I’ve been using it all day :)

Autotest: Now, With Sound Effects! 21 Comments

Jul 28, 2007

Update April 9, 2008: Ken Collins has released a new version of the sound plugin with playlist support!

We’ve all been enjoying autotest, part of the ZenTest gem. If you’ve tricked out your kit, then you have plugins configured, so at minimum you’re red, green and growling. Now, things get really fun.

Watch a screencast of autotest running with sound effects

I’m stoked to announce the sound plugin for autotest. This simple chunk of code will fire off sounds for different events in autotest. I’ve provided a set of custom-made sounds, produced with my trusty Nord Modular synthesizer and fine-tuned for an optimal testing experience. You should be able to use these all day without annoying your neighbors too much.

Here’s what you need to do:

1. Install mgp321

in OS X:
$ sudo port install mpg321
for Linux:
$ sudo apt-get install mpg321

2. Download and extract the plugin

The starter sound fx are in the zip file. Extract it in your home directory, it will create ~/autotest/sound.

autotest-sound-1_2.zip (86k)

3. Configure your ~/.autotest file:

require '~/autotest/sound/sound.rb'
Autotest::Sound.sound_path = "~/autotest/sound/sound_fx/" 

Enjoy TDD with audio feedback!

I’ve been using this setup for several weeks now. I initially wrote it as a gag, but I have since found it to be incredibly useful. It’s nice know what your testing status via audio – you don’t have to switch windows or take your eyes off the code. I’ve even turned off Growl, I don’t need it any more. audio makes testing more fun. :)

If there are any problems or feedback, please post a comment here.

UPDATE:Plugin instructions and zip file updated, now with Windows support. Thanks, John and Jamie.

UPDATE #2:Fixed bad path in instructions and doc fixes in zip file. (thanks, Matt)

Validating Fixtures

May 23, 2007

Fixtures Are Painful

I just got back from RailsConf 2007, it was a brain-expanding conference. I spent a lot of time talking to Rails coders about how they do testing and use fixtures. I saw some patterns emerge. Generally, people agree that fixtures are painful:

  • referencing ids in the fixtures is annoying and prone to error
  • the lack of a grouping mechanism makes selecting them harder
  • fixtures get out of date with model changes

I plan to do a few posts about fixtures in the next coming weeks, to share some of what I have learned. For now, I’ll focus on validation.

Fixtures Break

The project I’m currently working on has a large number of fixtures (over 100 models). It’s become really hard to manage them all. Over time, some fixtures busted, and it became hard to diagnose random problems in tests.

We are fortunate to be working with zenspider, he’s helping us get our tests, dev process and performance in better shape. I complained a lot about invalid fixtures, and how I longed for a Rake task that could identify the broken ones. He offered up this:

namespace :db do
  namespace :fixtures do
    task :validate => :environment do
      name_map = Hash.new { |h,k| h[k] = k }

      Dir.chdir("app/models") do
        map = `grep set_table_name *.rb`.gsub(/[:\'\"]+|set_table_name/, '').split
        Hash[*map].each do |file, name|
          name_map[name] = file.sub(/\.rb$/, '')
        end
      end

      Dir["test/fixtures/*.yml"].each do |fixture|
        fixture = name_map[File.basename(fixture, ".yml")]

        begin
          klass = fixture.classify.constantize
          klass.find(:all).each do |thing|
            unless thing.valid? then
              puts "#{fixture}: id ##{thing.id} is invalid:" 
              thing.errors.full_messages.each do |msg|
                puts "   - #{msg}" 
              end
            end
          end
        rescue => e
          puts "#{fixture}: skipping: #{e.message}" 
        end
      end
    end # validate
  end # fixtures
end # db

Put it in your project Rakefile, and you can then run:

$ rake db:fixtures:validate

You will get back a list of all the fixtures that are not valid, with the validation messages.

But what about edge cases? What about “bad” form data? DHH has declared that most folks want all fixtures loaded at the start of the tests. The data in fixtures does not have to pass model validation before it is loaded into the db.

After discussing this with a number of folks, I’m starting to believe:

  • All fixtures should be valid at all times
  • Fixtures should provide stuff required by your app to run tests (your admin user, default options, etc)
  • Edge cases should be created in tests, not fixtures
  • Test invalid data by loading a good fixture and changing it

Rake db:fixtures:validate is helpful for keeping them valid. I plan to use it whenever I do a migration, just to ensure that nothing got broken.

Rake task to run a custom group of tests

Apr 16, 2007

I have not really been getting into blogging yet, obviously. It’s been several months since the most recent post. I’m in New York at the moment, getting ready to launch a rails web project. More details about that later.

For now, I just had to post something I did recently that was handy: Using rake to run a specific group of files.

There are some great built-in tasks provided by Rake. Let’s look at the testing-related ones (run ‘rake -T’ to see them all):

rake test:functionals               # Run the functional tests in test/functional
rake test:integration               # Run the integration tests in test/integration
rake test:plugins                   # Run the plugin tests in vendor/plugins/*/**/test (or specify with PLUGIN=name)
rake test:recent                    # Test recent changes
rake test:uncommitted               # Test changes since last checkin (only Subversion)
rake test:units                     # Run the unit tests in test/unit

These are all very useful. But on the site I’m currently working on, I wanted to run a set of tests that were related to a section of the web site: the web store. I wanted to be able to run a specific combination of unit, functional, and integration tests.

I looked at the source for the tasks that rails provides for rake. The testing tasks are defined in lib/tasks/testing.rake in the rails distribution. The code for the unit testing task, for example:

  desc "Run the unit tests in test/unit" 
  Rake::TestTask.new(:units => "db:test:prepare") do |t|
    t.libs << "test" 
    t.pattern = 'test/unit/**/*_test.rb'
    t.verbose = true
  end

The files are specified as a glob. But, I did not want to use a file pattern. I consulted the rake API docs and discovered the test_files= setter method of Rake::TestTask. Here’s what I ended up with:

namespace :test do
  tests = []
  tests << "test/integration/store_test.rb" 
  # models
  %w(product product_category product_categorization product_option product_option_value 
    line_item cart order shipment).each do |file|
      tests << "test/unit/#{file}_test.rb" 
  end
  # controllers
  %w(cart checkout store).each do |file|
    tests << "test/functional/#{file}_controller_test.rb" 
  end
  # admin controllers
  %w(products product_option_values product_options orders).each do |file|
    tests << "test/functional/admin/#{file}_controller_test.rb" 
  end
  Rake::TestTask.new(:store => "db:test:prepare") do |t|
    t.libs << "test" 
    t.test_files = tests
    t.verbose = true
  end
  Rake::Task['test:store'].comment = "Run the store-related tests" 
end

I saved this task in lib/tasks/store_tests.rake under my project directory. Now, I can do this:

$ rake test:store
(in ~/rails_project)
/opt/local/bin/ruby -Ilib:test "/opt/local/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake/rake_test_loader.rb" "test/integration/store_test.rb" "test/unit/product_test.rb" "test/unit/product_category_test.rb" "test/unit/product_categorization_test.rb"  [... etc ...]
Loaded suite /opt/local/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake/rake_test_loader
Started
....................................................................
Finished in 10.799886 seconds.

68 tests, 682 assertions, 0 failures, 0 errors

All the store tests are run as a group. This is a nice way to verify that things are working in the money-making part of the site.

However, I have to remember to update the task whenever I refactor or make additions. I’m thinking about symbolic links, now. Perhaps the tests could be grouped together in a directory (like test/groups/store) by making symbolic links to them in there. And, subversion deals with symlinks well…