docstring-testing: love→hate→forget→rediscover→repeat

I have a love-hate relationship with using python docstrings for testing. They're simple, easy-to-use, and provide both usage-documentation and actual unit-test functionality. But whenever I start using them in any serious extensive way, I go back to using regular unit-tests in a file or directory -- because of limitations of docstring-tests (which I'm not addressing here). So I haven't used them in a while. But for a informal collaborative experimental project, without testing set up, today I wanted to add a few simple tests to a function to confirm that it's really doing what we expect. I remembered docstrings for testing, and I'm once-again in the love-phase.

Here's a function in some experimental code a few of us are working on -- to which I added a few docstring-tests. The docstring is everything between the initial and closing """. Generally docstrings are just used to provide multi-line comments on the purpose of a function. But they're cool for two reasons:

• tools such as Sphinx can be run on code to auto-generate documentation from docstrings. (I never use that.)

• assertions can be auto-executed as tests.

In this function, here are the doc-test assertions. If the project had a normal unit-test harness, running tests the regular way would not only run the unit-tests, but also automatically test these assertion-statements. This experimental code has no test-harness set up; there are no unit-test imports. But the docstring tests can still be run like this: % python -m doctest ./data_clean.py

Running that command shows no output if the test-assertions pass (you would see lots of output if running with the -v (verbose) flag). But imagine if the part that reads:

    >>> stringify_list( 42 )
    '42'

...instead were:

    >>> stringify_list( 42 )
    'blah'

...then, running the test would yield:

**********************************************************************
File "/path/to/ml_OTA_experiment_stuff/ml_bdr_ota_experiment/./data_clean.py", line 133, in data_clean.stringify_list
Failed example:
    stringify_list( 42 )
Expected:
    'blah'
Got:
    '42'
**********************************************************************
1 items had failures:
   1 of   4 in data_clean.stringify_list
***Test Failed*** 1 failures.

Cool -- simple lightweight documentation and testing; very nice!