From mboxrd@z Thu Jan  1 00:00:00 1970
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	ip-172-31-74-118.ec2.internal
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=3.0 tests=BAYES_00 autolearn=ham
	autolearn_force=no version=3.4.6
Path: eternal-september.org!reader02.eternal-september.org!aioe.org!5WHqCw2XxjHb2npjM9GYbw.user.gioia.aioe.org.POSTED!not-for-mail
From: "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>
Newsgroups: comp.lang.ada
Subject: Re: Ada and software testing
Date: Mon, 12 Jul 2021 10:40:24 +0200
Organization: Aioe.org NNTP Server
Message-ID: <scgv5m$obu$1@gioia.aioe.org>
References: <871r84cq4r.fsf@nightsong.com>
NNTP-Posting-Host: 5WHqCw2XxjHb2npjM9GYbw.user.gioia.aioe.org
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Complaints-To: abuse@aioe.org
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
X-Notice: Filtered by postfilter v. 0.9.2
Content-Language: en-US
Xref: reader02.eternal-september.org comp.lang.ada:62373
List-Id: <comp.lang.ada>

On 2021-07-12 02:49, Paul Rubin wrote:

> Well, that went on for longer than I expected.  My questions are
> basically:
> 
>      Q1. Are there good recommendations for Ada testing strategies?  Do
>      the tests resemble the stuff in the Sqlite doc?
> 
>      Q2. Is fuzz testing an important part of Ada testing, and does it
>      tend to find many bugs?

I do not think so.

Here is a war story of a bug I fixed recently. A network protocol 
implementation used a callback to send the next portion of data, when 
the transport becomes available.

The callback implementation peeks a portion of data from the outgoing 
queue and *asynchronously* sends it away. *If* initiation of sending is 
successful, the queue is popped.

OK?

No, it is a bug that almost never shows itself because initiation of I/O 
would normally deprive the task of the processor. But if it does not and 
I/O completes without losing the processor, the callback is called 
recursively *before* popping the queue and the *same* portion of data is 
sent again.

Now, nether 100% coverage, nor fuzz, not even 100% black box testing can 
detect this, arguably trivial bug.

[The fix is to make recursive calls void]

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de