Re: Ada and software testing

["G.B." <bauhaus@notmyhomepage.invalid>]

On 12.07.21 02:49, Paul Rubin wrote:

> Well, that went on for longer than I expected.  My questions are
> basically:
> 
>      Q1. Are there good recommendations for Ada testing strategies?  Do
>      the tests resemble the stuff in the Sqlite doc?
> 
>      Q2. Is fuzz testing an important part of Ada testing, and does it
>      tend to find many bugs?

I'd like to add, if I may, a third question, perhaps a a follow-up question,
after having been bitten by a bug that was hidden behind assumptions.

Is there a way of systematically looking for hiding places of bugs
specifically in places external to the program text? And, then, what kind
of mock-ups could establish typical testing patterns? I/O is mentioned
in in the sqlite examples, but what if you do not assume that there
is going to be X .equiv. I/O?

Example: Some external library, of very closed source nature,
exposes an unforeseen behavior. It turns out that a library function
uses a lock, and while waiting for it, the function call times out,
the client program reports failure and terminates normally - with
side effects...

After the fact, after some reading and then some testing, in an adjusted
setup, it all seems plausible. "But, I didn't think of that!".
Educated guesses about what the library might do need to be based
on a vast set of documents, plus the seller of the library
also sells expensive training. Programs need a quick fix, though.

So, what is a proper testing strategy once the programmers have found
that the transitive closure of some call might sometimes incur externally
caused behavior? Such as timeout, or ordering effect due to concurrency?