Reminder Tests

"How are we going to remember to X if/when we Y?"

I ask myself that, or hear a team member ask a question like that, at least once a week.

The first question is: do you need to do X if/when you do Y because there's some sort of duplication? If so, eliminate the duplication. But there are situations where there's not duplication. In that case, write a test. The test either needs to check that X is always true, or it needs to detect if Y happens. Here are some examples in Ruby, but this technique applies to any language.

Using Piston to Install Plugins

"How are we going to remember to use piston when we install plugins?"

Piston is a great tool for installing plugins. It records the repository that the plugin was installed from, along with the remote revision. One reason that this is so important is that plugins are often not released with versions like gems are. They're usually just installed from trunk (or master, for the git users). So in the future, when you want to update the plugin, it helps if you know where you installed it from and which revision it was on when you installed it.

So let's say you tell everybody on your development team: "when you install a plugin, use piston." Everybody sees the benefit and agrees that it's a good idea. But inevitably, somebody will install a plugin and forget. Or you'll get a new team member who doesn't know about piston. You can leave a comment somewhere - maybe a development guidelines wiki page, or a README in the vendor/plugins directory. But it would be much better to have an executable reminder. One that tells you if you forgot. So write a test that checks that all plugins were installed with piston.

test "all plugins are installed with piston" do
  Dir.glob("#{RAILS_ROOT}/vendor/plugins/*").each do |plugin_dir|
    piston_root = `svn propget piston:root #{plugin_dir}`
    if piston_root.blank?
      raise "The #{plugin_dir} plugin was not installed with piston."
      # could leave comments here about how to install a plugin with piston
    end
  end
end

Presto. Now if anybody forgets to use piston, there's a test that reminds him or her.

Updating a Dependency

"How are we going to remember to remove this code when we update to the next version of Rails?"

I recently needed to apply a bug fix from edge rails to my Rails 2.1.0 project. I thought about applying it directly to vendor/rails since the next release of Rails should include the fix. And of course, I would have written a test for it to make sure it did. But team policy was to avoid making changes directly to vendor. So I decided to make an extension to override the buggy Rails method. I didn't want the code to stick around after we updated Rails though. At that point it would become unnecessary, and only increase the chances that the fix might be incompatible with something else in Rails. So I wrote a test that would fail when we updated to the next version of Rails.

test "this is only necessary for rails 2.1.0" do
  assert_equal "2.1.0", Rails::VERSION::STRING
  # we can remove this code once we update to the next version of rails
  # (as long as the tests for the fix still pass)
end

You could use this technique for any reminder that you need to give to somebody updating a dependency.

Database Changes

"How are we going to remember to use the AFTER option when we add a column?"

I usually use the MySQL AFTER option when adding columns to database tables so that they're alphabetized. It makes it easier to run a SELECT * query and then look for a certain column, especially if the table has more than a handful of columns. But it's easy to forget to add a column that way. Hence, a test as a reminder.

test "columns are in alphabetical order" do
  # use add_column .... :after => "existing_column" to insert sorted
  ActiveRecord::Base.connection.tables.each do |table|
    columns = ActiveRecord::Base.connection.columns(table).map(&:name)
    columns -= ["id"]
    assert_equal columns.sort, columns
  end
end

Embedded Images

Here's an example from CarePages' domain. CarePages sends welcome e-mails on the behalf of providers and affiliates to new users. Unfortunately, we noticed that some of the welcome messages (which are stored in the database) had image tags embedded directly in them. This meant that if we ever moved the image file, we would break the welcome message. The best solution would have been to change the messages in a way that would make sure moving an image file wouldn't break the message. But under time constraints, we decided to just write a test that those images were there.

test "embedded images are present" do
  # if any of these files move
  # welcome message that will need to be updated
  assert File.exists?("#{RAILS_ROOT}/public/images/clients/some_image.jpg")
  assert File.exists?("#{RAILS_ROOT}/public/images/clients/another_image.jpg")
end

This way, if anybody moves the image file, they also know to update the welcome message.

Summary

Using tests for reminders is a great way to make sure you don't forget anything. But don't go overboard with it. You shouldn't have reminder tests failing daily - unless you're team is really forgetful about something they need to be doing all the time.