Re: Ada vs. C Comparison Data ?

Re: Ada vs. C Comparison Data ?

[Charles H. Sampson]

In article <9108051956.AA05473@ajpo.sei.cmu.edu> byrne@ARECIBO.AERO.ORG ("DAN J
. BYRNE") writes:
>
>  I have been asked to find data that justifies using Ada as opposed 
>to C (or C++) for a large distributed system using UNIX workstations.
>I need technical reasons, such as maintenance costs, error rates, COTS
>tools and risk.  The data needs to be hard numbers with citable sources.
>Simply saying that Ada has a lower total life-cycle cost won't hack it.

     The Air Force has recently completed a study that might be what you
need.  It was done under the direction of Lloyd Mosemann, Deputy Assistant
Secretary of the Air Force for Communications, Computers, and Logistics.
The point was to compare Ada and C++ to determine when waivers for C++
might be warranted.  Five contractors looked at different facets of the
question.  All five came to the same conclusion: There are no compelling
reasons to waive the Ada requirement to use C++.  I don't recall that they
specifically addressed the issue of UNIX workstations but the results are
probably valid even if they didn't.

     For four of the five contractors, "hard" data were used.  (As hard
as such data can be in an area like this.)  The fifth's facet was inher-
ently more blue sky, so hard data were impossible.

     I don't know what the official channel is to get a copy of this re-
port.  The one I have came from my local Alsys representative and looked
like about a 30th generation Xerox.

                              Charlie

Re: Ada vs. C Comparison Data ?

[cis.ohio-state.edu!magnus.acs.ohio-state.edu!csn!arrayb!wicklund]

In <3219@cod.NOSC.MIL> sampson@cod.NOSC.MIL (Charles H. Sampson) writes:

>     The Air Force has recently completed a study that might be what you
>need.  It was done under the direction of Lloyd Mosemann, Deputy Assistant
>Secretary of the Air Force for Communications, Computers, and Logistics.
>The point was to compare Ada and C++ to determine when waivers for C++
>might be warranted.  Five contractors looked at different facets of the
>question.  All five came to the same conclusion: There are no compelling
>reasons to waive the Ada requirement to use C++.  I don't recall that they
>specifically addressed the issue of UNIX workstations but the results are
>probably valid even if they didn't.

>     For four of the five contractors, "hard" data were used.  (As hard
>as such data can be in an area like this.)  The fifth's facet was inher-
>ently more blue sky, so hard data were impossible.

Of course, this is like sugar companies showing health risks in
artificial sweeteners, tobacco companies showing no health risks for
cigarettes, etc.  The Air Force has basically been ordered to find no
exceptions for ADA use, so it's no surprise that they have.

I'm not saying the results are wrong, but I'd rather see a study by an
unbiased source (if such exists).  I imagine ATT could sponsor a study
and find the opposite result.

Re: Ada vs. C Comparison Data ?

[Jim Showalter]

wicklund@intellistor.com (Tom Wicklund) writes:

>Of course, this is like sugar companies showing health risks in
>artificial sweeteners, tobacco companies showing no health risks for
>cigarettes, etc.  The Air Force has basically been ordered to find no
>exceptions for ADA use, so it's no surprise that they have.

Funny, I don't recall any such wording in the charter for the group
the did the study. Must have been in the fine print somewhere. The
charter I read seemed pretty straightforward: compare the two languages
and see if there is any compelling reason to weaken the Ada mandate in
favor of C++ for at least some classes of applications. Nothing at all
about finding no exceptions to Ada use, whitewashing, or any of the rest
of it.

>I imagine ATT could sponsor a study
>and find the opposite result.

Not if they were honest about it. It ultimately boils down to whether or
not you believe the people conducting the study are scientists or ideologues.
If they're scientists, then they'll do the right thing no matter what. If
they're ideologues, then they should be removed from the study and replaced
with scientists. Since I've never met any of the people who did the Air Force
study, I extend to them the benefit of the doubt.
-- 
*** LIMITLESS SOFTWARE, Inc: Jim Showalter, jls@netcom.com, (408) 243-0630 ****
*Proven solutions to software problems. Consulting and training on all aspects*
*of software development. Management/process/methodology. Architecture/design/*
*reuse. Quality/productivity. Risk reduction. EFFECTIVE OO usage. Ada/C++.    *

Re: Ada vs. C Comparison Data ?

[timothy shimeall]

I have met some of the people who conducted the study, or at least
the NPS substudy (check my Organization line) and let me assure you
that they are scientists and not idealogues.  In fact, one of them
recently won a Presidential Young Investigator award for her work in
Software Prototyping.  While she is currently using Ada in her work,
she is NOT emotionally committed to the language.

I'd suggest that people get and read the full report (and I'm still
trying to get a copy, myself) before issuing blanket accusations of
poor quality or politically-generated bias.
				Tim

Re: Ada vs. C Comparison Data ?

[cis.ohio-state.edu!magnus.acs.ohio-state.edu!csn!arrayb!wicklund]

In <1991Aug08.083832.21573@netcom.COM> jls@netcom.COM (Jim Showalter) writes:

>wicklund@intellistor.com (Tom Wicklund) writes:

>>Of course, this is like sugar companies showing health risks in
>>artificial sweeteners, tobacco companies showing no health risks for
>>cigarettes, etc.  The Air Force has basically been ordered to find no
>>exceptions for ADA use, so it's no surprise that they have.

>Funny, I don't recall any such wording in the charter for the group
>the did the study. Must have been in the fine print somewhere. The
>charter I read seemed pretty straightforward: compare the two languages
>and see if there is any compelling reason to weaken the Ada mandate in
>favor of C++ for at least some classes of applications. Nothing at all
>about finding no exceptions to Ada use, whitewashing, or any of the rest
>of it.

My posting was written before reading the press conference remarks
posted here.  Below are some comments after reading them, some of
which will be pertinent to a non-defense industry comparison of Ada
and C++.  The quotes below are from the press conference remarks
posted here.


>>I imagine ATT could sponsor a study and find the opposite result.

>Not if they were honest about it.

Actually, if ATT sponsored an objective comparison of C and Ada they
would probably come up with the opposite result.  The reasons would be
the same reasons that DOD prefers Ada -- ATT has a strong committment
to C, there are more tools, compilers, etc. for C on ATT machines, ATT
has C training programs but not Ada, etc.  To a large extent, the Air
Force's reasons for Ada are because they are Ada oriented.



Some comments on the summary of the Air Force comparison of Ada and C++:

PLEASE NOTE: I'm not disputing the results of the study in the context
of DOD use of Ada.  However, this subject was brought up in the
context of a non DOD user comparing Ada and C.  Many of the points I
make below are to point out this (necessary) bias in the study which
may not apply for non-DOD users.

After reading the press conference remarks, I agree that this was not
a whitewash.  However, the fact remains that the study was performed
by groups with a vested interest in Ada and DOD.  DOD has a well
documented history of adjusting results to match what they want to
see, whether it's overestimates of Soviet capabilities or test results
of weapons.  A study of this sort must be carefully examined for the
assumptions used and checked for any intentional or unintentional
bias.  A defense contractor is also likely to think twice before
slamming Ada since they've got a lot invested in it.


>A.   Tools, Environments, and Training: IDA Substudy.

>     The Institute for Defense Analyses (IDA) collected and
>analyzed information on the market availability of commercial-
>off-the-shelf products available from U.S. sources for Ada and
>C++ compilers, tools, education, and training.

Read this section carefully -- several justifications for Ada are
because DOD has a strong investment in Ada.  This is a valid reason
for DOD preferring Ada but not necessarily a good reason for a
commercial firm without an existing Ada investment.  Other reasons are
based on C++ being younger, unstandardized, and not currently under
DOD control.  This says nothing about the capabilities of the
language, just that Ada is already in the DOD process.  A good reason
for DOD, maybe not for somebody else.

>     Both languages are supported on PCs and workstations.  Ada
>is also supported on mainframes.  Ada, but not C++, has cross
>compilation systems.

Untrue.  I'm sure cfront is available on mainframes running UNIX.
Cross compilation systems include at least Microtek C++ for the 68K
and Turbo C++ with Paradigm Systems embedded development package.  I'm
sure others exist.  The Turbo C++ add-on has existed for over a year,
so should have been known to IDA if they did a thorough study.  C++
tools aren't oriented to DOD environments, but this is because there's
no market due to the Ada requirement.

>Code generators exist for Ada but none so far for C++.

Untrue -- G++, Zortech C++, Borland C++, Oregon C++, Green Hills C++,
NDP C++, and TopSpeed C++ are listed as native compilers in the C++
products list periodically posted to comp.lang.c++.  A couple might
actually be cfront based, most are not.


>B.   Faceted IBM Language Selection Methodology: SEI Substudy.

To evaluate these results, look at the detailed report.  Different
applications have very different requirements.  Somebody else might
weigh the categories differently.


>C.   Macro Cost Analysis: CTA Substudy.

I don't know enough statistics, but the relatively small number of C++
projects and the relatively small difference in the numbers may make
some of this study's results meaningless.

>                           Productivity    Number of
>                           (SLOC/MM)       Data Points

See comp.software-eng for a discussion of whether measuring
productivity by lines of code per man month is a valid test.  these
results also don't consider the relative functionality of a line of
Ada or C++ (I have no idea what it would be).

>Typically, the Ada developments were performed in accordance with
>military standards and incorporated formal reviews, additional
>documentation, and additional engineering support activities such
>as formal quality assurance (QA) and configuration management
>(CM).  Most C++ projects are commercial and do not extensively
>incorporate such activities.  Additionally, on such projects
>developers are typically intimately involved with users,
>resulting in considerably less requirements engineering effort. 
>Consequently, applications on which C++ is used are inherently
>less costly, so that the reported productivity rates are
>favorably skewed toward C++.

Funny -- Software quality experts will tell you that all the formal
reviews, QA, etc. improve productivity and reduce costs.  Informal
requirements with frequent user interaction are especially expensive
compared to a fixed set of requirements up front.  So if the C++
projects were done more formally they should cost less, right?

>                         Integration    FQT            Number of
>                         Error Rates    Error Rates    Data      
>                         (Errors/KSLOC) (Errors/KSLOC) Points
> *   Norm (all languages) 33             3              543
> *   Average (Ada)        24             1              153
> *   Average (C++)        31             3               23

An apples and oranges comparison if Ada developments used better
design methods, so caught more problems before integration.  The C++
rate should go down with the application of these methods.

Again, this study points out the relative maturity of Ada, the tight
standardization, and DOD's single language strategy.  Important for
some users, not for others.


>D.   Micro Cost Analysis: TRW Substudy.

The summarized results of this study don't give enough information to
be meaningful.  I do note that some Ada advantages appear to be in the
standardized support environment, which one can get with C++ by
choosing the proper add-on product or enforcing some development
guidelines.

However, the primary strengths and weaknesses listed appear accurate.


>E.   Ada Policy Issues: NPS Study.

This summary appears well thought out, definately not a whitewash.
They acknowledge the usefulness of 4GLs and other higher level
software generators while keeping to DOD's Ada commitment.  Again, the
Ada poriton applies more to DOD than other users.

This report does skirt around an interesting issue -- The use of CASE
tools which directly generate code or languages such as Ada versions
of LEX, YACC, and state machine generators are technically violations
of the Ada mandate.  If I write a scanner in ALEX I'm not writing
strict Ada code.  Taking the Ada mandate literally one can't use any
of these tools since your source code isn't Ada.  Hopefully DOD will
find a way to handle tools of this sort without prohibiting them or
getting into a different type of language explosion.

Re: Ada vs. C Comparison Data ?

[Jim Showalter]

wicklund@intellistor.com (Tom Wicklund) writes:

>Other reasons are
>based on C++ being younger, unstandardized, and not currently under
>DOD control.  This says nothing about the capabilities of the
>language, just that Ada is already in the DOD process.  A good reason
>for DOD, maybe not for somebody else.

Immaturity and lack of standardization seem like good reasons to stay
away from a language whether one is DoD or not. As for capabilities,
I find the two languages much more similar to one another than different:
both offer strong typing, data encapsulation, separation of specification
from implementation, etc. Each is missing one major feature the other has:
Ada lacks inheritance/polymorphism and C++ lacks concurrency.

>Funny -- Software quality experts will tell you that all the formal
>reviews, QA, etc. improve productivity and reduce costs.  Informal
>requirements with frequent user interaction are especially expensive
>compared to a fixed set of requirements up front.  So if the C++
>projects were done more formally they should cost less, right?

I'd _like_ to believe this, but yars of experience with DoD sites
leads me to the opposite conclusion. At least in DoD land, most of
the formal stuff is a gigantic time and resource sink with little
or no benefit.

>Again, this study points out the relative maturity of Ada, the tight
>standardization, and DOD's single language strategy.  Important for
>some users, not for others.

Again, I cannot think of a compelling argument for using a language that
is immature and weakly (if at all) standardized. Can you?

>This report does skirt around an interesting issue -- The use of CASE
>tools which directly generate code or languages such as Ada versions
>of LEX, YACC, and state machine generators are technically violations
>of the Ada mandate.  If I write a scanner in ALEX I'm not writing
>strict Ada code.  Taking the Ada mandate literally one can't use any
>of these tools since your source code isn't Ada.

Semantics. The resulting language of implementation is Ada, so why
is this a violation of the mandate? There is almost always some
front-end "language" used that is not Ada, be it bubbles-and-arcs
or some formal specification language, or whatever. The Ada mandate
never says that such design approaches cannot be used--only that
the language used to implement the design (however the design was
arrived at) must be Ada.
-- 
*** LIMITLESS SOFTWARE, Inc: Jim Showalter, jls@netcom.com, (408) 243-0630 ****
*Proven solutions to software problems. Consulting and training on all aspects*
*of software development. Management/process/methodology. Architecture/design/*
*reuse. Quality/productivity. Risk reduction. EFFECTIVE OO usage. Ada/C++.    *