If you haven't been satisfied by any existing continuous integration tools for Ruby, try CruiseControl.rb - I tried it today, and it's great.

Version 1.0 comes with a few ways to receive notification - e-mail, jabber, and rss. However, many developers using OSX prefer Growl notifications. I wrote a plugin that is currently pending as a patch for CC.rb 1.1, but until then:

Download the CruiseControl.rb Growl Notifier Plugin

Installation:
Extract the archive into builder_plugins/installed

Configuration:
In builds/your_project/cruise_config.rb add the hosts to notify:

Project.configure do |project|
  project.growl_notifier.subscribers = %w[
    192.168.0.100
    192.168.0.101
  ]
end

If you want to change any Growl preferences, look for CruiseControl.rb in your applications list in the Growl preferences.

Also, because the plugin uses Eric Hodel's ruby-growl library, notifications can be sent from non-OSX machines.