Re: ADA Popularity Discussion Request

[Benjamin Ketcham <bketcham@drizzle.com>]

Richard  Riehle <adaworks@earthlink.net> wrote:
> 
> Finally, testing can only test for what we know needs to be tested.   We
> must design for the unexpected, even as we test for the expected.  It is
> impossible to test for the unexpected.

While I agree with the overall point here, I keep seeing this
assertion, here made with especially great conviction: "it is
impossible to test for the unexpected".  As with many absolute
statements, it is not technically correct, although it expresses
a valid general truth.  Consider the method of kernel testing
where system calls are invoked with random arguments; this kind
of test often turns up "unexpected" problems.  I recall that
early in the days of Windows NT, this kind of approach uncovered
bugs that led to complete system hangs (I'd hope they have fixed
such things by now, and that MS themselves make sure they are the
first to find such things by trying such tests in-house before
releasing new kernels; thankfully I don't have to use NT or whatever
it's called now, so I don't know).

Same goes for UIs; some kinds of testing involve sending random
UI events at a program until it crashes or does something incorrect.

Sure, it's generally not possible to test *every* possible state
of large software, but this doesn't mean that testing *many* states,
particularly states that might not be "expected", by feeding it
random data, is not a valid, important, and sometimes fruitful approach.

--Benjamin