Re: Ada and software testing

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On 2021-07-12 02:49, Paul Rubin wrote:

> Well, that went on for longer than I expected.  My questions are
> basically:
> 
>      Q1. Are there good recommendations for Ada testing strategies?  Do
>      the tests resemble the stuff in the Sqlite doc?
> 
>      Q2. Is fuzz testing an important part of Ada testing, and does it
>      tend to find many bugs?

I do not think so.

Here is a war story of a bug I fixed recently. A network protocol 
implementation used a callback to send the next portion of data, when 
the transport becomes available.

The callback implementation peeks a portion of data from the outgoing 
queue and *asynchronously* sends it away. *If* initiation of sending is 
successful, the queue is popped.

OK?

No, it is a bug that almost never shows itself because initiation of I/O 
would normally deprive the task of the processor. But if it does not and 
I/O completes without losing the processor, the callback is called 
recursively *before* popping the queue and the *same* portion of data is 
sent again.

Now, nether 100% coverage, nor fuzz, not even 100% black box testing can 
detect this, arguably trivial bug.

[The fix is to make recursive calls void]

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de