Tests as specifications

The other day I was doing the yahtzee kata in C in CyberDojo and I got caught (not for the first time) by a classic gotcha...

I wrote my tests very very simply as follows:


...
static void two_pairs_scored_as_one_pair_is_spots_on_highest_pair(void)
{
    assert(...);
}

static void full_house_scored_as_one_pair_is_spots_on_the_pair(void)
{
    assert(...);
}
...

typedef void test(void);

static test * tests[] =
{
    two_pairs_scored_as_one_pair_is_spots_on_highest_pair,
    full_house_scored_as_one_pair_is_spots_on_the_pair,
    ...,
    NULL,
};

int main(void)
{
    for (int at = 0; tests[at] != NULL; at++)
        tests[at]();
    return 0;
}


I wrote another test and hit the Run-Tests button. But the new test didn't get run because I'd forgotten to add the name of the function to the tests array. Ooops. The new test compiled ok, and I'd tripped up thinking the test was passing when it wasn't even being run.

I've practised in CyberDojo enough that I don't get "sucked into the code" as much as I used to and when something like that happens there's now a fair chance I'll have enough awareness to notice it as it happens.

I stopped and thought about what had happened for a moment and I realized something. My problem was that I was writing the test function definition first. I should be writing the name of the test in the tests array first. And only once I've done that should I define it. There is a nice echo of the idea of Test First Design to this.

Of course in a language with reflection you only need to write the test function. Be that as it is, I like the the way the contents of the tests array lists the function names as specifications. In a language with reflection you don't get that.

2 comments:

  1. You don't need reflection. We commonly achieve the same result in C++ using static registry objects.

    I don't think you lose the concept of the specification by doing so. They might not be concentrated in one place in the code but they are still there. Any decent test framework should provide you with the ability to list the available test cases.

    (Here comes the plug:) My C++ unit test framework, Catch, takes this even further. Test names are strings (function names are generated and hidden), and are associated with a free form description field too - so you can express your specification much more naturally. nested SECTIONs within a test case (all with their own name and description) take this further still.
    And Catch provides an easy way to list all available test cases, with descriptions.

    Of course, you can use Catch to test pure C code too (and Objective-C), and with the single-header-only version is pretty lightweight. So except in the most brutally self-contained situations there is no excuse ;-)

    ReplyDelete
  2. Hi Phil,
    yes C++ frameworks do that. But I was using C. In CyberDojo you are brutally self-contained by design! Catch is very cool. Thanks.

    ReplyDelete