Re: Ada95 to ANSI_C converter

[Arthur Schwarz <schwarza@gdls.com>]

Robert Dewar wrote:
> 
> Keith says
> 
> <<(I *like* nasty test cases)>>
> 
> The trouble with nasty test cases is that they can sometimes result in
> implementors wasting huge amounts of time getting things right that are
> of no possible concern to users. The old ACVC suite was full of such
> things, and it is still quite often the case that we make a change
> and it affects none of our user programs in our regression suite, but
> it causes an obscure bug in some ACVC test. Equally, it is often the
> case that a change blows away huge numbers of user tests, but the
> ACVC hums merrily on with no errors.
> 
> Anecdotally (we really should collect data on this, we have the raw data),
> we have the impression that the Ada 95 tests tend to track user programs
> more closely. They are certainly much less full of "nasty test cases".


(Anecdotally): One program that I wrote (Fortran IV/66) broke the
compiler.
I was told by my manager that my program 'stressed' it (the compiler).
And
at that time and since I have wondered why! If the language permits a
construct
and the compiler doesn't accept it, or what's worse, the generated code
doesn't
perform the operation correctly, then the compiler is "broke", not the
construct
and not the language. I would then say that a "nasty test case" is just
another
language construct which must be accepted by the compiler. It's only
nasty when
the compiler can not process it correctly. The same goes for 'obscure',
'seldom
used', and of course, 'of no possible use to users'. To my jaundiced eye
this all
seems to be an excuse to direct effort at 'big payoff' items, the ones
that seem
to be most likely of use. The problem is that the devil is in the
details, and it
is just those other, legal, small payoff items which cause great labor
for the 
implementor to debug; transfering the cost of the compiler from
developer to user.

Your other comment that the ACVC test does not catch all errors is valid
but I think
not very meaningful. The complexity of language and compiler and
computer and
operating system and ... conspire to make exhaustive testing too
exhaustive. If
this is accepted as an accurate statement, it would seem unlikely that
any test
suite, except for very small 'languages', could ever be exhaustive
enough to catch
all possible compiler errors.

And this is not to attempt to derate ACT or your own efforts. Only to
say that
IMO the issue is not meaningful in that failing to pass a "nasty test
case" means
that the language feature tested is not useable and the compiler
erroneous.

art

My own opinions are often my own, seldom my companies.