From mboxrd@z Thu Jan  1 00:00:00 1970
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham
	autolearn_force=no version=3.4.4
X-Google-Thread: 103376,c406e0c4a6eb74ed
X-Google-Attributes: gid103376,public
X-Google-Language: ENGLISH,ASCII-7-bit
Path: 
 g2news1.google.com!news1.google.com!newsfeed.stanford.edu!sn-xit-02!sn-xit-01!sn-post-01!supernews.com!corp.supernews.com!not-for-mail
From: Benjamin Ketcham <bketcham@drizzle.com>
Newsgroups: comp.lang.ada
Subject: Re: ADA Popularity Discussion Request
Date: Tue, 07 Sep 2004 03:57:04 -0000
Organization: Ye 'Ol Disorganized NNTPCache groupie
Message-ID: <1094529422.982635@yasure>
References: <49dc98cf.0408110556.18ae7df@posting.google.com>
   <slrncj0nmf.hl6.adi@ppc201.mipool.uni-jena.de>
 <cgppvn$t4m$1@a1-hrz.uni-duisburg.de>
 <6F2Yc.848$8d1.621@newsread2.news.pas.earthlink.net>
 <m3isb2ey3w.fsf@rigel.goldenthreadtech.com>
 <hMnYc.1759$w%6.1486@newsread1.news.pas.earthlink.net>
 <m3656vfgmf.fsf@rigel.goldenthreadtech.com>
      <vpblkhspuA9F@eisner.encompasserve.org>
 <e749549b.0409051449.574ec9ce@posting.google.com>
 <vS8%c.69$xA1.4@newsread3.news.pas.earthlink.net>
User-Agent: tin/1.6.2-20030910 ("Pabbay") (UNIX) (Linux/2.4.20-29.8.progeny.8
 (i686))
Cache-Post-Path: yasure!unknown@cascadia.drizzle.com
X-Cache: nntpcache 2.4.0b5 (see http://www.nntpcache.org/)
X-Complaints-To: abuse@supernews.com
Xref: g2news1.google.com comp.lang.ada:3410
Date: 2004-09-07T03:57:04+00:00
List-Id: <comp.lang.ada>

Richard  Riehle <adaworks@earthlink.net> wrote:
> 
> Finally, testing can only test for what we know needs to be tested.   We
> must design for the unexpected, even as we test for the expected.  It is
> impossible to test for the unexpected.

While I agree with the overall point here, I keep seeing this
assertion, here made with especially great conviction: "it is
impossible to test for the unexpected".  As with many absolute
statements, it is not technically correct, although it expresses
a valid general truth.  Consider the method of kernel testing
where system calls are invoked with random arguments; this kind
of test often turns up "unexpected" problems.  I recall that
early in the days of Windows NT, this kind of approach uncovered
bugs that led to complete system hangs (I'd hope they have fixed
such things by now, and that MS themselves make sure they are the
first to find such things by trying such tests in-house before
releasing new kernels; thankfully I don't have to use NT or whatever
it's called now, so I don't know).

Same goes for UIs; some kinds of testing involve sending random
UI events at a program until it crashes or does something incorrect.

Sure, it's generally not possible to test *every* possible state
of large software, but this doesn't mean that testing *many* states,
particularly states that might not be "expected", by feeding it
random data, is not a valid, important, and sometimes fruitful approach.

--Benjamin