fhwang.net

Testing Rails against a running Redis instance (and doing it with Hydra to boot)

Profitably has been running Redis since very early on. And as I’m planning to lean on Redis even more heavily going forward, particularly for Resque, I decided to beef up how Redis is tested in the Rails app. Since not a lot of people are talking about what this takes, here are my notes about my initial setup:

Testing approaches

If your Rails app uses Redis, there are basically three approaches you can take for when the code under test hits Redis.

  1. Mock Redis
  2. Stub Redis
  3. Stop messing around and just use Redis

Mocking is a popular choice in the Rails world, and that looks something like this:

  class UserTest < ActiveSupport::TestCase
    def setup
      @user = User.create!
    end

    test "get my cached info" do
      $redis.expects(:get, "User/info/#{@user.id}").returns(YAML.dump([1,2,3]))
      assert_equal([1,2,3], @user.get_cached_info)
    end
  end

But, as I’ve said at RubyConf, I’m not a fan of mocking. I think Yehuda’s rule of thumb is a good one: Don’t mock anything you own.

Stubbing is the practice of creating a fake class that acts like the dependency, and it’s what I was doing at first. I had a class called RedisStub that I substituted in the test environment for the real redis-rb library. It was a pure Ruby class that supported a handful of Redis methods and simply stored the results in a transient way, and it worked fine for testing at first.
  class RedisStub
    def initialize
      @atoms = {}
    end

    def get(key)
      @atoms[key]
    end

    def set(key, value)
      @atoms[key.to_s] = value.to_s
      'OK'
    end
  end

  class UserTest < ActiveSupport::TestCase
    def setup
      @user = User.create!
      $redis = RedisStub.new
    end

    test "get my cached info" do
      $redis.set("User/info/#{@user.id}", YAML.dump([1,2,3]))
      assert_equal([1,2,3], @user.get_cached_info)
    end
  end

However, it didn’t feel that stable. For one thing, Redis has a pretty decent number of commands, so eventually I was going to clone a lot of stuff. For another, you never know if you introduce a bug into your stub code itself. There are techniques to minimize bugs in your stubs (I talk about some in that Testing Heresies talk I linked to above), but they don’t bring the odds of bugs down to zero. Every line of new code in the world is a chance to introduce a bug, right?

Testing against Redis directly

So I decided to hit a real running instance of Redis, since I’m already developing with it on my laptop. So I happily did a git rm test/helpers/redis_stub.rb and set about running my tests against Redis itself.

The first problem I hit is that I’ve already got Redis running for my development environment, and it would be really annoying if running my tests broke something I was working on in the browser. However, it’s pretty easy to run redis-server more than once on a machine. I don’t know if there’s any reason to do this in production, but it comes in pretty handy for testing.

First, I copied /usr/local/redis-1.2.6/redis.conf to ./test/redis.conf and modified some of the settings there:
  daemonize yes

  port 10000
You run this with:
  $ /usr/local/redis-1.2.6/redis-server ./test/redis.conf
And I added this to the bottom of ./config/environments/test.rb:
  port = 10000
  $redis = Redis.new(:port => port)
  begin
    $redis.get('foo')
  rescue Errno::ECONNREFUSED
    puts "Redis is not listening in port #{port}. Run:" 
    puts "  /usr/local/redis-1.2.6/redis-server ./test/redis.conf" 
    exit
  end

That part at the end tries to use the test redis server before doing anything else, because the next time I restart my laptop, I’m bound to forget to turn on the test redis instance.

This works fairly well, though you’re dealing with an external data store that doesn’t automatically reset its state every time you run a new test case, so you want to reset redis every time you run a test:
  class UserTest < ActiveSupport::TestCase
    def setup
      $redis.flushall
    end

    ...
  end

Extra credit: Making it work with Hydra

This worked fine if I was running ruby test/unit/user_test.rb, or even rake test:units, but not if I ran rake itself. Because, to make it a bit more complex, I’m using Hydra.

Hydra, if you don’t know, is a pretty great library that optimizes your test suite by running it on multiple cores. You can set it up to run distributed across different machines, but I’m just using it on two cores for my MacBook Pro, and even that gives you dramatic speed improvements.

However, when I ran Hydra through Redis, it was causing lots of problems. Which makes sense when you think about it—Hydra forks off one process for each core, and as those processes churn through your test suite, all that setup and teardown logic is going to run through the same Redis keyspace, and there will inevitably be test cases stepping on each other.

Rails solves this problem in the relational database by using transactions: It does all the work of a test case inside a RDBMS transaction, and then rolls back at the end of the test. Years ago when I first heard of this technique I was fairly skeptical, but it ends up being fairly clean in practice, and you can work around the edge cases. Anyway, that’s why Hydra works just fine with two or more processes hitting the same test database in MySQL or Postgres or whatever—each process is in its own transaction, and since that transaction will never be committed, a process can’t change the database that another process sees. (Incidentally, I decided to use Postgres with Profitably, and my only regret is that I didn’t leave behind MySQL sooner. But that’s a subject for another post.)

Anyway, the most recent stable version of Hydra doesn’t have any transactional logic I could use in the same way. It appears that there are some new transactional features on the way, but I decided not to go bleeding-edge here, because I’d rather not run non-stable Redis release in production, or run different versions of Redis in production and development.

Instead, what I did was to run a different Redis for each test process. This would probably be really cumbersome for some setups, but in my simple case of two processes, it works like a charm.

First, instead of having one config file at ./test/redis.conf, I had two: ./test/redis.0.conf and ./test/redis.1.conf. The only way they vary is by having a different port number: redis.0.conf uses port 10000, and redis.1.conf uses port 10001.

Then, I have ./config/environments/test.rb use Process.pid to figure out which of the two ports to use:
  process_increment = Process.pid % 2    # will be either 0 or 1
  port = 10000 + process_increment       # will be either 10000 or 10001
  $redis = Redis.new(:port => port)
  begin
    $redis.get('foo')
  rescue Errno::ECONNREFUSED
    puts "Redis is not listening in port #{port}. Run:" 
    puts "  /usr/local/redis-1.2.6/redis-server ./test/redis.#{process_increment}.conf" 
    exit
  end

Hydra spins off two processes in quick succession, and they’re pretty much guaranteed to be one odd PID and one even PID. (I suppose it’s possible for somebody else to schedule a process in-between the two but I’m not worrying about that edge case today.) So based on the PID, the two processes will be using port 10000 and 10001, though it’s practically random (and insignificant) which process grabs which. I suppose if you were running on some quad-core desktop machine you could just do it with 4 port numbers & configuration files, though this might break down at some point.

So hopefully this will be helpful to other people wondering how to do full-stack testing against Redis in Rails. Comments and improvements are welcome, of course.

blog comments powered by Disqus

« Previous post

Next post »