Brittle Tests Are Testing Things You Don't Care About

I was recently testing the pagination of a certain page on my site. I only wanted to show 12 records at a time. The simple test was to spin up 15 records and then assert that only 12 were assigned in the controller.

models_controller_spec.rb
describe 'pagination' do
  it 'only shows 12 records' do
    15.times { FactoryGirl.create(:model) }
    get :index
    expect(assigns(:models).length).to eq 12
  end
end

Eventually I added a static block of 3 “featured” records to the top and therefore only needed to paginate 9.

I changed the controller and hit save, only to watch my tests turn red.

It occurred to me that the number of records per page is a feature that could change quite frequently as I added more features and tweaked for performance. Trying to keep this test synchronized was going to cause me more and more pain, especially as it would distract me from implementing the features I wanted to focus on.

One workaround, I suppose, would be to make the number of records per page a property of the controller.

models_controller_spec.rb
describe 'pagination' do
  it 'paginates based on the controllers models_per_page property' do
    (controller.models_per_page.times + 1).times { FactoryGirl.create(:model) }
    get :index
    expect(assigns(:models).length).to eq controller.models_per_page
  end
end

I kind of dislike this approach because it elevates an implementation detail out to the public interface of the controller. The controller is supposed to handle a request and return a view. Clients of the controller shouldn’t concern themselves with its pagination settings. The models_per_page method is only being exposed to assist in testing.

I had to ask myself why I cared about pagination. I was not truly concerned with how many records are being shown, I really just cared that the entire models table is not loaded into memory on each request.

And since that’s what I cared about, that’s what I should assert.

models_controller_spec.rb
describe 'pagination' do
  it 'paginates' do
    15.times { FactoryGirl.create(:model) } #assumes pagination < 15
    get :index
    expect(assigns(:models).length).to be < Model.count
  end
end

This only asserts that the controller pulls fewer Models than exist in the db. It’s a less brittle test because it only makes assertions about behavior I care about.