Print Story Docs is hard!
Diary
By tmoertel (Sun Sep 12, 2004 at 08:03:18 PM EST) (all tags)
I spent a good part of the weekend writing and rewriting docs for LectroTest. I'm now up to about 25 pages. Finally, all of the pieces of the system are documented in a non-sucky fashion. (If you're curious or want to help me proofread: LectroTestDocs.pdf. If there is anything that is confusing or awkward, let me know.)

More stuff inside.



I also submitted LectroTest to CPAN, where it lives as Test::LectroTest in the module list.

The CPAN testers are cool. They download distributions from CPAN, build them, run test suites, and report the results. They caught a bug in LectroTest 0.20_02 that manifested itself on Win32 platforms. I had a test case for that very same bug, but it reported a false OK because the bug was coincidentally not triggered for particular instance I checked. But not on Win32. Thanks CPAN testers!

What's next? More docs: Tutorial. Step-by-step examples. Tricksy examples.

Back to the writing pad.

< This is a pebble, number 4 of an uncertain number. | BBC White season: 'Rivers of Blood' >
Docs is hard! | 5 comments (5 topical, 0 hidden) | Trackback
I hate PDFs by BadDoggie (3.00 / 0) #1 Sun Sep 12, 2004 at 08:48:45 PM EST
Please mail me an easily editable copy (text, MSWord, AbiWord, whatever) and I'll do some. Change tracking in MS Word is very good for this sort of thing.

If you need help with the technical content as opposed to just the general document contents and layout, I'll need more time.

woof.

"Eppur si muove." -- Galileo Galilei
"Nevertheless, it moves."

Thx, but no can do on Word format by tmoertel (3.00 / 0) #2 Mon Sep 13, 2004 at 01:50:51 AM EST
Thanks for your offer, but the documentation is embedded in the source code via Perl's "Plain Old Documentation" scheme. Moving it to Word is not a realistic option. (BTW, change tracking is done after-the-fact using GNU diff on the source code.)

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
Doc notes by jacob (6.00 / 1) #3 Mon Sep 13, 2004 at 05:28:17 AM EST
Holy comprehensiveness, Batman!

A couple notes:

  1. It's not clear except in the discussion of scalefn in section 5, a section whose introduction explictly warns you not to read it, how the framework uses generators' size parameters in testing code. I don't know whether anybody else would be bothered by this, but I kept wondering during the generators section how this argument was going to be used.
  2. retries, again in section 5: I'm still mystified as to what this parameter might be doing. Under what conditions would the testing framework retry a test case? Am I supposed to infer that if this were set to 0 the testing framework would never duplicate a test? If so, what happens if I make a testcase where the haystack is just Unit("hi") and tell it to run a thousand tests?

These are just nits, by the way; I think the docs are very solid and gave me a good insight into the way the tool works. Overall I think this is a really cool tool. I did a Scheme implementation myself over the weekend, and it was really cool how effective it was given the very small size of the code that implemented it. (My implementation's not done yet, but I tend to save working versions of code in my public_html directory: http://www.cs.uchicago.edu/~jacobm/testblaster.ss is where you'll find it, but no guarantees it'll work when you check it!)

By the way, doing that implementation made me wonder why you chose to make generators objects rather than one-argument functions: it seems to me that HOFs are a perfect match for the problem. You'll see that my generators and combinators are just one or two lines apiece, and all very straightforward. Any reasons why you chose to use objects instead?

--

THANK YOU! by tmoertel (3.00 / 0) #4 Mon Sep 13, 2004 at 04:58:58 PM EST
Jacob, you are the man! I seriously appreciate your taking the time to read the docs. If you're ever in Pittsburgh, I owe you a beer. Or two, even.

Regarding your question about retries, it's there to allow you to ask for the a trial to be retried using new input. For example, if you were testing a square-root function and wanted to avoid testing negative inputs:

Property {
    ##[ x <- Float ]##
    return $tcon->retry if $x < 0;  # can't take sqrt when $x<0
    my $sx = my_sqrt( $x );
    abs($sx * $sx - x) < 0.000_001;
};
Every negative number $x would result in a retry, which goes against the retry count but not the trial count. The retries configuration parameter determines how many retries to allow before concluding that something is wrong (infinite loop?) and bailing out.

That's kind of a dumb example because you could use the range=> parameter of the generator to ask for positive-only numbers, but in some cases its easer to retry the few instances where you get undesirable inputs than to prevent them from being generated in the first place.

On the objects vs. 1-arg functions question, all of my generators are 1-arg functions under the hood. I wrap them in a simple object to better match the impedance of common Perl culture, where people are more familiar with OOP than with FP. (Take a look at what the Gen function does in Generator.pm.)

P.S. I dig testblaster. Great name, and a tight, concise implementation.

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
Scheme-Check by tmoertel (3.00 / 0) #5 Tue Sep 14, 2004 at 03:16:42 AM EST
I just noticed on the QuickCheck home page that there's a link to a scheme implementation called Scheme-Check.

--
Write Perl code? Check out LectroTest. Write markup-dense XML? Check out PXSL.

[ Parent ]
Docs is hard! | 5 comments (5 topical, 0 hidden) | Trackback