Perl already has great unit testing support with modules like Test::Simple and Test::More. But even so, these modules follow the traditional test-case model of testing that requires you to do a lot of work that probably could be delegated to the computer.
Think of it this way. You're looking for needles (errors) in a haystack (your program's behavior). You want to find the needles and get rid of them. Most unit testing systems make you point out to the computer each and every piece of hay you want it to inspect and what to expect at each. Consider this case. OK, now this one, here. And this one, too. Now this one. And, yes, this one.... To build confidence in whether any particular desired property of your software holds, you must create enough test cases to expose the needles in the haystack.
Here's a typical Perl unit test using Test::More:
# AngularDifference.tEach one of the is(...) statements is a particular case to test. The first argument is the output of the thing we're testing. The second is the expected output. The third is the name of the test case, to be used in testing reports. The comments above each block of test cases give you an idea of the needles we're trying to expose with the following cases.
use Test::More tests => 6;
# identical angles should be zero
is( angdiff( 0, 0), 0, "zero at zero" );
is( angdiff( 90, 90), 0, "zero at 90" );
# order of angles shouldn't matter
is( angdiff( 0, 45), 45, "0,45 -> 45" );
is( angdiff( 45, 0), 45, "45,0 -> 45" );
# should return the smallest angle between
is( angdiff( 0,270), 90, "0,270 -> 90" );
# multiples of 360-degrees shouldn't matter
my ($a,$b) = (360 * 2, 360 * 4);
is( angdiff($a,$b+23),23, "$a,$b+23 -> 23" );
In this example, we're testing a function angdiff that computes the difference between two angles a and b (in degrees). The correct result is the smallest angle between a and b. The test cases all attempt to see whether the function faithfully holds to this one simple definition. It seems doubtful that even this many cases will provide enough coverage of our implementation's behavior to give us much confidence. So we may be forced to write even more cases. What a pain.
With QuickCheck and LectroTest, on the other hand, things are different. You make the computer write the test cases for you. All you do is describe the shape of the haystack and tell the computer what a needle looks like, in general. The computer then runs over to the haystack and starts testing pieces of hay randomly and at a blinding pace. After a while it may find a needle. In that case, it emits a counterexample and stops. You can plug the counterexample into your code to figure out what went wrong. (You can also add the counterexample to a list of regression tests to run later.)
Here's the LectroTest property to check whether our implementation of angdiff faithfully holds to the definition:
# AngularDifference.l.tThe first part of the Property gives the shape of the haystack we're searching. It's the part in between the ##[ and ]## delimiters. The second part is the needle-recognizer. It's a bit of code that results in either true (no needle) or false (needle present). Finally, the property has a name, again for human consumption in reports and such.
##[ a <- Int, n <- Int, diff <- Int(range=>[-180,180]) ]##
my $b = $a + 360 * $n + $diff;
angdiff($a, $b) == abs($diff);
}, name => "definition via inverse";
The above is actually a just Perl program. To check whether our property holds, we simply run it:
$ perl AngularDifference.l.tOops! Looks like there's a problem in our implementation of angdiff.
not ok 1 - 'definition via inverse' falsified in 9 attempts
# $a = -1;
# $diff = 1;
# $n = -1;
If you are interested in this kind of thing, check out the slides from the talk I recently gave for the Pittsburgh Perl Mongers: "Free Unit Tests In Perl with LectroTest". In the talk I elaborate on the example above and explain how to build LectroTest properties. I also show how to check more complicated properties, such as whether an RFC-2822-compliant email-address parser meets the spec.
P.S. If you read this far, thank you! I had to write quickly, so this probably isn't easy to read. Since I haven't posted to HuSi in a while, I figured I ought to let you know what I've been up to.
Your pal, Tom.
P.P.S. Did the robot's claws frighten you? (See poll.)
|< I'm giving up. | BBC White season: 'Rivers of Blood' >|