Testing Fragment Caching

The Risk in Fragment Caching

From The need for speed: Making Basecamp faster:

We've begun using Memcached in a variety of spots. Caching can be tricky with dynamic apps like Basecamp since different people often see different things, but we've implemented it carefully where it could be used to its best advantage.

37signals is right. Fragment caching can be tricky. The risk is that a user sees the wrong content, but that risk can be mitigated with tests.

Testing Fragment Caching

Since it's really easy for a developer or designer to make a change that causes problems with caching, it's a good idea to write a test for any pages that are using fragment caching.

Essentially what you need to test is that page content is exactly the same for a user regardless of whether fragments hit the cache or not. So the basic steps are:
(1) with caching off, capture the page content from user1's perspective
(2) with caching on, view the page using user2
(3) with caching still on, view the page using user1 and make sure the body matches the captured body from step 1.

You should run a test following these steps for a few combinations of user1 and user2. The more complicated the cache key is, the more tests you should have.

Here's an implementation using test/unit with Rails.

def test_fragment_caching_on_some_page
  user1 = create_user :first_name => "Nick"
  user2 = create_user :first_name => "Dan"
  check_fragment_caching(user1, user2) do |user|
    @request.session[:user_id] = user.id
    get :some_page
  end
end

def check_fragment_caching(user1, user2)
  Rails.cache.clear
  ActionController::Base.perform_caching = false
  yield user1
  user_1_not_cached_body = @response.body
  
  ActionController::Base.perform_caching = true
  yield user2
  
  yield user1
  assert_equal user_1_not_cached_body, @response.body
end

Example

Let's look at a simple example of how this works. Let's create a fragment with a static cache key.

<% cache "some_fragment" do -%>
  hello
<% end -%>

And the test will pass.

Started
.
Finished in 0.147882 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

But now a developer comes along and decides to greet users by name.

<% cache "some_fragment" do -%>
  hello <%= current_user.first_name %>
<% end -%>

You can see that this will create a caching problem. If the first user to visit the page is named Dan, the fragment will be cached as "hello Dan." Then the next user, who isn't named Dan, will be greeted with "hello Dan." But the test should catch this.

Started
F
Finished in 0.15005 seconds.

  1) Failure:
test_fragment_caching_on_some_page(SomeControllerControllerTest)
method check_fragment_caching in some_controller_controller_test.rb at line 30
method test_fragment_caching_on_some_page in some_controller_controller_test.rb at line 14
<"  hello Nick\n"> expected but was
<"  hello Dan\n">.

Now that there is user specific information displayed inside the fragment, we'll need to update the cache key.

<% cache ["some_fragment", current_user] do -%>
  hello <%= current_user.first_name %>
<% end -%>

And if we run the test, we're back to green.

Started
.
Finished in 0.149098 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

Changing the cache key won't always be the answer. Sometimes you'll want to re-think the change to the page, add a dynamic element using javascript, or take a different approach altogether. But the important part is catching the caching bug quickly.

Summary

Try to hit the all your pages that use fragment caching with different users. Try with an admin/non-admin, users under different accounts, etc. And hopefully if somebody makes a change that messes up the caching, your test will give fast feedback.