www.hans-eric.com

We’ve done a lot with testing frameworks over the years, but does the testing concern deserve its own standalone DSL?

This intriguing question was asked by Michael Feathers in his Mock Objects: Leaping out of the Language post. My spontaneous answer is: Absolutely!

I’m a big fan of xUnit frameworks, but when I imagine an alternative unit-testing specific language one special property comes to mind, a feature that would really make a difference. I’d call it unconditional mocking. With a DSL based unit-testing framework one could test really complex objects, even legacy code, since mocking internal objects would require no change to the original programming interface.

For example, this (nonsense) code

class A {
  private B _b;

  // constructor
  this A() {
    _b = new B()
  }
}

unittest {
  // B can not be mocked
  A a = new A();
}

would require refactoring in order for _b to be mockable.

class A {
  private B _b;

  // constructor
  this A(B b) {
    _b = b;
  }
}

unittest {
  // B could be mocked
  B b = new BMock(...);
  A a = new A(b);
}

But in a unit-testing DSL, one should be able to mock any object, in this case B, without changing the source code first. This is handy for dealing with the unit-testing paradox: Refactoring requires a unit-testing harness to make sure no functionality gets broken, but unit-testing requires testable code; So what to do when the code isn’t testable? A unit-testing DSL would make it easier to put up the initial testing harness.

Also, as Michael points out, a unit-testing DSL could be used to mock any kind of construction, not just objects: Functions and methods for instance. Oh man, could I have use for such a feature?

To give us an image of a DSL for unit-testing in a non-object-oriented language like C, Michael provides this example:

function send_port_command with 90, “CMD: 12”
            calls io_mode which returns M_READ
            calls set_mode with M_WRITE
            calls write_byte with 90
            calls write_bytes with “12”
            returns E_OKAY

That would be testing a function like this:

status send_port_command(byte port, const char *message)
{
  if(io_mode() == M_READ)
    set_mode(M_WRITE);
  write_byte(port)
  write_bytes(translate_command(message));
  return E_OKAY;
}

I have a problem with his example though. In my opinion the test-code resembles the target code a little too much, like a bad manager performing low-level supervision. Too detailed testing beats the purpose since it makes changes more difficult. My philosophy is that test-code should test WHAT the code does, and not bother too much on the HOW.

So, my not so thought through proposal, using Michaels example, would be something like this:

TEST send_port_command

MOCK write_byte(port)
EXPECT port == 90

MOCK write_bytes(bytes)
EXPECT bytes == "12"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY

Of course there should be support for more advanced mock features like call counting:

MOCK  write_bytes(bytes)
EXPECT   "12", "13"

CALL send_port_command with 90, "CMD: 12"
EXPECT E_OKAY
CALL send_port_command with 90, "CMD: 13"
EXPECT E_OKAY

or

MOCK  write_bytes(bytes)
EXPECT  2 CALLS

or sequential values

MOCK  io_mode
RETURN  M_READ, M_WRITE

Implementing the DSL would be a hefty task though. But, the problems aside, how would your unit-testing DSL be like? I’d be very interested to hear your opinions.

Cheers!