Boeing 737 and 737 MAX software

Boeing 737 and 737 MAX software

[Paul Rubin]

Does anyone know anything about this?  It has been under some criticism
lately.

I have heard that the 777 software was almost entirely in Ada.  It also
sounds as if Boeing's software operation may have slipped in recent
years, not good news for the 737 MAX.

Re: Boeing 737 and 737 MAX software

[Jere]

On Friday, April 5, 2019 at 5:16:22 PM UTC-4, Paul Rubin wrote:
> Does anyone know anything about this?  It has been under some criticism
> lately.
> 
> I have heard that the 777 software was almost entirely in Ada.  It also
> sounds as if Boeing's software operation may have slipped in recent
> years, not good news for the 737 MAX.

I've honestly not read anything that actually indicates a software error,
though I haven't looked in the last few days at it.  Everything I have 
read up to now indicates Engineering and Management no-nos.  They
appeared to improperly designed how some of the hardware was to be installed
then made a software patch to compensate for it.  They then convinced
some governing body that there was no pilot retraining needed for the new
software.  I don't think the software was actually at fault.  Again, 
maybe something different has surfaced in the last few days that I haven't
seen yet.

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

On Fri, 05 Apr 2019 14:16:20 -0700, Paul Rubin <no.email@nospam.invalid>
declaimed the following:

>Does anyone know anything about this?  It has been under some criticism
>lately.
>
>I have heard that the 777 software was almost entirely in Ada.  It also
>sounds as if Boeing's software operation may have slipped in recent
>years, not good news for the 737 MAX.

	Unless things have changed severely -- GE Aviation (formerly Smith's
Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
the processor boxes).

	However, I have the impression (from TV news) the software is
functioning /as designed/. Some reports have indicated that Boeing designed
the hardware (and corresponding software requirements) such that only one
sensor is used for the MCAS subsystem -- and a fault in that sensor results
in MCAS attempting to prevent a (non) stall by pushing the nose down. Some
hints in the news that Boeing is changing the requirements (well, in truth,
the news only says Boeing is changing the software) to have MCAS
cross-reference with other flight parameter data -- and making an optional
bit of hardware (additional sensors) standard.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/ 

Re: Boeing 737 and 737 MAX software

[Niklas Holsti]

On 19-04-06 20:30 , Dennis Lee Bieber wrote:
> On Fri, 05 Apr 2019 14:16:20 -0700, Paul Rubin <no.email@nospam.invalid>
> declaimed the following:
>
>> Does anyone know anything about this?  It has been under some criticism
>> lately.

As I've read more about these accidents than I usually do, I will boldly 
(and perhaps foolhardily) describe how I have understood it. All info is 
from public sources, I have no insider info. I am not a pilot, and 
moreover I write from recollection of my reading and have no references 
to give, so reader beware.

>> I have heard that the 777 software was almost entirely in Ada.  It also
>> sounds as if Boeing's software operation may have slipped in recent
>> years, not good news for the 737 MAX.
>
> 	Unless things have changed severely -- GE Aviation (formerly Smith's
> Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
> the processor boxes).
>
> 	However, I have the impression (from TV news) the software is
> functioning /as designed/.

All info I have seen agrees with that.

> Some reports have indicated that Boeing designed
> the hardware (and corresponding software requirements) such that only one
> sensor is used for the MCAS subsystem

The are two angle-of-attack (AoA) sensors, one on each side of the nose. 
They feed two redundant computers, each able to run MCAS. Normally only 
one MCAS instance is running and it uses only its "own" AoA sensor.

The original design of MCAS gave it rather little control authority, 
which is probably why this single-sensor approach was accepted.

> -- and a fault in that sensor results
> in MCAS attempting to prevent a (non) stall by pushing the nose down.

Yes, but MCAS does not apply a temporary nose-down command -- as if 
pushing the stick forward -- it changes the pitch trim, the overall 
angle of the horizontal stabilizer, giving the plane a permanent 
tendency to dive. This trim change can be overridden by the pilots, but 
only if they notice that it has happened.

In the original MCAS design, one activation of MCAS changed the pitch 
trim by a small amount, at most 0.6 degrees IIRC, and this limit was 
reported in the MCAS design documentation to the authorities. During 
testing, Boeing found that it was not enough, and they increased it 
quite a lot, to over 2 degrees IIRC. One source I read claimed that this 
change was _not_ updated in the documentation shown to the authorities.

Moreover, by design MCAS would repeat this trim change, with a certain 
minimum interval, as long as the AoA sensor reading remained too large 
and indicated a risk of stall. This iteration should converge and stop 
if the sensor is working, but if the sensor fails and is stuck at a high 
AoA (the false value reported in the second accident was around 60 
degrees, IIRC) then MCAS will incrementally and cumulatively keep 
increasing the pitch trim and the diving tendency. If the pilots do not 
understand what is happening, they will find it ever harder to 
counteract the "dive" trim with stick inputs.

> Some
> hints in the news that Boeing is changing the requirements (well, in truth,
> the news only says Boeing is changing the software) to have MCAS
> cross-reference with other flight parameter data -- and making an optional
> bit of hardware (additional sensors) standard.

AIUI the modifed MCAS will read both AoA sensors and will disable itself 
if they disagree, and the disagreement will also be reported by a 
display. This display is the new piece of HW which used to be an option. 
There are no new sensors, AIUI.

I believe Boeing are also changing the minimum interval between MCAS 
activations -- perhaps even allowing only one activation -- so as to 
prevent a cumulatively increasing "dive" trim.

In summary, it seems to me that the criticality of MCAS, and thus the 
need for redundant sensors, was not realized for two reasons: (1) in its 
initial design, MCAS command authority was small, and (2) the 
possibility of multiple repeated commands (due to a stuck sensor) and 
the resulting large cumulative command (large change of pitch trim) was 
not considered.

A kind of "criticality creep".

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Boeing 737 and 737 MAX software

[Paul Rubin]

Jere <jhb.chat@gmail.com> writes:
> On Friday, April 5, 2019 at 5:16:22 PM UTC-4, Paul Rubin wrote:
>> It also sounds as if Boeing's software operation may have slipped in
>> recent years, not good news for the 737 MAX.
> I've honestly not read anything that actually indicates a software error,

I don't think anyone claimed that the 737 MAX issue was a software bug
per se, but I've heard some noises to the effect that when the FAA
looked into Boeing's proposal to update the MCAS software, they found
that the software in general was in worse shape than they had hoped.  So
I wondered if there had been any process changes, and along with that,
any language changes.

Also apparently this type of software is certified using DO-178C
requirements, which describe multiple levels of criticality.  Level A is
for the most critical functions, stuff that can crash the plane, and has
the most stringent requirements.  Boeing apparently submitted the MCAS
software under level C.  In retrospect that looks like a big error.

Some Hacker News threads regarding the 737:

https://news.ycombinator.com/item?id=19565759
https://news.ycombinator.com/item?id=19573893
https://news.ycombinator.com/item?id=19577602

Re: Boeing 737 and 737 MAX software

[tranngocduong]

On Saturday, April 6, 2019 at 4:16:22 AM UTC+7, Paul Rubin wrote:
> Does anyone know anything about this?  It has been under some criticism
> lately.
> 
> I have heard that the 777 software was almost entirely in Ada.  It also
> sounds as if Boeing's software operation may have slipped in recent
> years, not good news for the 737 MAX.

I know nothing about the software. But I don't think it is written in Ada. If it was, programmers must have chosen a wrong subtype.

In the crashed Ethiopian 302 aircraft, ~30 seconds before impact, the nose points 40 deg. toward ground, and the AoA indicates 75 deg. As 40 + 75 = 115 > 90 deg., this would imply negative airspeed.

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

On Fri, 12 Apr 2019 00:46:31 -0700 (PDT), tranngocduong@gmail.com declaimed
the following:

>
>I know nothing about the software. But I don't think it is written in Ada. If it was, programmers must have chosen a wrong subtype.
>
	It's Ada... (In the past, I was doing maintenance on the FMS "BootROM"
code -- which, while not the actual run-time flight software, is
responsible for doing CRC checks of the software and databases, reading new
software from dataloaders, and loading which application is to run based
upon external settings. The FMS software links with the same base "OS".

>In the crashed Ethiopian 302 aircraft, ~30 seconds before impact, the nose points 40 deg. toward ground, and the AoA indicates 75 deg. As 40 + 75 = 115 > 90 deg., this would imply negative airspeed.

	With a big enough tail wind, negative airspeed is possible... Also, AoA
is the relative angle of air flow over the wing, and independent of nose
attitude. An inverse micro-burst (air coming straight up from the ground,
rather than down from above) could register a high AoA even with the nose
pointing down.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com 

Re: Boeing 737 and 737 MAX software

[Maciej Sobczak]

> I know nothing about the software.

Me neither, so we are in a perfect condition to discuss it!

> But I don't think it is written in Ada. If it was, programmers must have chosen a wrong subtype.
> 
> In the crashed Ethiopian 302 aircraft, ~30 seconds before impact, the nose points 40 deg. toward ground, and the AoA indicates 75 deg. As 40 + 75 = 115 > 90 deg., this would imply negative airspeed.

So how would Ada help in this situation?
What if the above addition is never performed? Because, you know, the result of this addition indicates a flight parameter that either nobody cares about or that is already obtained by other (presumably more reliable) means?

In which case Ada would be a perfectly valid language there.

-- 
Maciej Sobczak * http://www.inspirel.com

Re: Boeing 737 and 737 MAX software

[tranngocduong]

On Thursday, April 18, 2019 at 12:27:44 AM UTC+7, Maciej Sobczak wrote:
> > I know nothing about the software.
> 
> Me neither, so we are in a perfect condition to discuss it!
> 
> > But I don't think it is written in Ada. If it was, programmers must have chosen a wrong subtype.
> > 
> > In the crashed Ethiopian 302 aircraft, ~30 seconds before impact, the nose points 40 deg. toward ground, and the AoA indicates 75 deg. As 40 + 75 = 115 > 90 deg., this would imply negative airspeed.
> 
> So how would Ada help in this situation?
> What if the above addition is never performed? Because, you know, the result of this addition indicates a flight parameter that either nobody cares about or that is already obtained by other (presumably more reliable) means?
> 
> In which case Ada would be a perfectly valid language there.
> 
> -- 
> Maciej Sobczak * http://www.inspirel.com

The addition computes airspeed. More specifically it computes the elevation angle, one of the 3 components of the airspeed (as a vector in 3-dimensional space), quite a critical flight parameter.

True, airspeed needs not be computed this way. But it must be somehow computed. All flight parameters must be somehow computed. If not for flying, then at least for monitoring. The software is supposed to run on multiple (redundant) computers, takes input from multiple (redundant) sensors, computes flight parameters using multiple (redundant) algorithms.

True, it is possible that the software was written in Ada. But then, the fact that it didn't raise an exception (or even worse, programmers managed to find a proof that it can never raise one), indicating a failure to detect/handle so "exceptional" situations as AoA implying negative [horizontal] airspeed, is simply unbelievable.

Re: Boeing 737 and 737 MAX software

[Maciej Sobczak]

> True, airspeed needs not be computed this way. But it must be somehow computed.

Note that the airspeed had to be computed (or measured) long before the new sensor was invented, so presumably the method of computing it was not taking the new sensor into account anyway. And I don't expect that method to be changed just because some new sensor is installed.

I don't see any reason for using this addition anywhere in the system.

> True, it is possible that the software was written in Ada. But then, the fact that it didn't raise an exception

If the addition was never performed (because there was no reason to do it), then it is quite reasonable that no exception was raised.

One could imagine a contract that binds several such values in constraints that are motivated at the system-level and this is arguably where Ada could help. But I doubt such novel programming techniques would be even considered.

> indicating a failure to detect/handle so "exceptional" situations as AoA implying negative [horizontal] airspeed, is simply unbelievable.

Why? The new sensor was not installed to detect negative airspeed, but to detect stalls.

This system might have been written in Ada or C (I don't expect anything else to be even considered) with the same results. Which, arguably, is not helping to promote the language (whichever was used).

-- 
Maciej Sobczak * http://www.inspirel.com

Re: Boeing 737 and 737 MAX software

[Simon Wright]

tranngocduong@gmail.com writes:

> In the crashed Ethiopian 302 aircraft, ~30 seconds before impact, the
> nose points 40 deg. toward ground, and the AoA indicates 75 deg. As 40
> + 75 = 115 > 90 deg., this would imply negative airspeed.

I think that if it implied anything it would be that the speed over the
ground was negative and the aircraft was descending extremely
rapidly. Neither of which is impossible.

But, as Maciej hs been saying, what makes you think that this
caluclation was ever performed? 

Re: Boeing 737 and 737 MAX software

[tranngocduong]

On Thursday, April 18, 2019 at 7:44:17 PM UTC+7, Maciej Sobczak wrote:
> > True, airspeed needs not be computed this way. But it must be somehow computed.
> 
> Note that the airspeed had to be computed (or measured) long before the new sensor was invented, so presumably the method of computing it was not taking the new sensor into account anyway. And I don't expect that method to be changed just because some new sensor is installed.
> 
> I don't see any reason for using this addition anywhere in the system.
> 
> > True, it is possible that the software was written in Ada. But then, the fact that it didn't raise an exception
> 
> If the addition was never performed (because there was no reason to do it), then it is quite reasonable that no exception was raised.
> 
> One could imagine a contract that binds several such values in constraints that are motivated at the system-level and this is arguably where Ada could help. But I doubt such novel programming techniques would be even considered.
> 
> > indicating a failure to detect/handle so "exceptional" situations as AoA implying negative [horizontal] airspeed, is simply unbelievable.
> 
> Why? The new sensor was not installed to detect negative airspeed, but to detect stalls.
> 
> This system might have been written in Ada or C (I don't expect anything else to be even considered) with the same results. Which, arguably, is not helping to promote the language (whichever was used).
> 
> -- 
> Maciej Sobczak * http://www.inspirel.com

I'm not sure what is the "new sensor" but I'll try to explain my point in other words.

Generally, flight control softwares are supposed to use multiple, redundant, algorithms (including multiple, redundant, computers and sensors) to derive flight parameters. The derived values, if not used directly to control the aircraft, may be used to monitor the flying computer, the sensors, or the software itself. 

In particular, in the case of the 737, we may assume that sufficiently redundant computation can detect faulty AoA sensor without using any additional AoA sensor. The aforementioned addition is an example of such a (redundant) computation.

Note that this method is not novel. It is just an instance of a more general principle that every input data have to be validated.

Yet the faulty left AoA sensor went undetected. How can it be?

a) Ada was used but programmers have chosen a wrong (too relaxed) subtype, or other language was used and programmers failed to code whatever equivalent to raising and handling a CONSTRAINT_ERROR. Simply: software bug.

b) Contrary to general belief, the software was not programmed with multiple redundant computation. Simply: process failure.

I chose to believe a).

Re: Boeing 737 and 737 MAX software

[tranngocduong]

On Thursday, April 18, 2019 at 8:50:27 PM UTC+7, Simon Wright wrote:
> 
> > In the crashed Ethiopian 302 aircraft, ~30 seconds before impact, the
> > nose points 40 deg. toward ground, and the AoA indicates 75 deg. As 40
> > + 75 = 115 > 90 deg., this would imply negative airspeed.
> 
> I think that if it implied anything it would be that the speed over the
> ground was negative and the aircraft was descending extremely
> rapidly. Neither of which is impossible.

The 115 deg. is the [elevation angle of] airspeed (i.e. speed of the aircraft relative to the air), not groundspeed (ie. relative to the ground). That's because the AoA is the angle between the aircraft and the airspeed. 

It is true that even airspeed can be negative. Negative airspeed is not exceptional for acrobatic and fighter aircraft. For the 737 however, that would be an exceptionally unusual situation. 

Unusual enough to believe that a sensor, most likely the AoA sensor, failed. Enough to disable all AoA sensor-based flight control algorithms, such as the MCAS. Enough to trigger an alternative algorithm, raise an alert, or completely disengage the flight control computer.

> 
> But, as Maciej hs been saying, what makes you think that this
> caluclation was ever performed?

I don't think so. I think that the software must perform many redundant calculations similar to this one. Sensors and flight computers are redundant, just like hydraulic systems and control surfaces. This calculation demonstrates that redundant calculation can help detect possible sensor failure even without a second AoA sensor.

Re: Boeing 737 and 737 MAX software

[Niklas Holsti]

On 19-04-18 16:53 , tranngocduong@gmail.com wrote:

> Generally, flight control softwares are supposed to use multiple,
> redundant, algorithms (including multiple, redundant, computers and
> sensors) to derive flight parameters.

Yes.

   [snip]

> Yet the faulty left AoA sensor went undetected. How can it be?

The descriptions of the MCAS system that I have seen say quite clearly 
that it used only one of the two AoA sensors mounted on the aircraft 
(and that this single-sensor design is unacceptable for a flight-control 
system that ended up with this level of authority and criticality).

I have not seen any statement about other standard SW that would be able 
to flag an AoA sensor as faulty. There was an optional addition that 
could do it, not mounted on the planes that crashed. The whole MCAS 
system was an add-on and perhaps for that reason not well integrated 
with the rest of the flight SW (this is speculation on my part).

> a) Ada was used but programmers have chosen a wrong (too relaxed)
> subtype, or other language was used and programmers failed to code
> whatever equivalent to raising and handling a CONSTRAINT_ERROR.
> Simply: software bug.
>
> b) Contrary to general belief, the software was not programmed with
> multiple redundant computation. Simply: process failure.
>
> I chose to believe a).

 From the descriptions I have read, it is clear (to me) that (b) was the 
case, at least that MCAS used a single AoA sensor and there was no 
cross-check against the other AoA sensor or other sensors or 
computations. Moreover, the descriptions of the planned correction to 
the 737 MAX focus on using both AoA sensors and warning the pilots if 
they disagree, which is coherent with (b) but not with (a).

On the issue of Ada subtypes, it seems to me that if the SW 
specification, design and coding considers sensor faults (as it of 
course should), the normal approach for such critical SW is _not_ to use 
strongly constrained subtypes and rely on Constraint_Error handling. The 
normal approach is to add explicit and specific range checks and 
sensor/computation cross-checks. That would be much easier to specify 
and test, and would also make it much easier to identify the faulty 
sensor(s) and to handle such situations properly.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Boeing 737 and 737 MAX software

[tranngocduong]

On Thursday, April 18, 2019 at 10:13:05 PM UTC+7, Niklas Holsti wrote:

> The descriptions of the MCAS system that I have seen say quite clearly 
> that it used only one of the two AoA sensors mounted on the aircraft 
> (and that this single-sensor design is unacceptable for a flight-control 
> system that ended up with this level of authority and criticality).
> 
> I have not seen any statement about other standard SW that would be able 
> to flag an AoA sensor as faulty. There was an optional addition that 
> could do it, not mounted on the planes that crashed. The whole MCAS 
> system was an add-on and perhaps for that reason not well integrated 
> with the rest of the flight SW (this is speculation on my part).
> 
The optional "AoA disagree" indicator is just a surface issue. I've seen a couple of analysis stating that it is really not very helpful. To my limited knowledge, AoA is a critical parameter that is used by many flight control algorithms, not just the MCAS. The real issue is thus the failure to detect an unreliable sensor. If the failure was a "feature, not bug", the entire flight software (and its certificate) would be questioned.

> > a) Ada was used but programmers have chosen a wrong (too relaxed)
> > subtype, or other language was used and programmers failed to code
> > whatever equivalent to raising and handling a CONSTRAINT_ERROR.
> > Simply: software bug.
> >
> > b) Contrary to general belief, the software was not programmed with
> > multiple redundant computation. Simply: process failure.
> >
> > I chose to believe a).
> 
>  From the descriptions I have read, it is clear (to me) that (b) was the 
> case, at least that MCAS used a single AoA sensor and there was no 
> cross-check against the other AoA sensor or other sensors or 
> computations. Moreover, the descriptions of the planned correction to 
> the 737 MAX focus on using both AoA sensors and warning the pilots if 
> they disagree, which is coherent with (b) but not with (a).
> 
If b) was the case, not only the software, but the entire process would be questioned. A bug would take weeks or months to fix. A software would take years to re-engineer. A process would take decades to develop. Would Boeing as a company risk its very existence by comiting such a big mistake? I don't think so.

> On the issue of Ada subtypes, it seems to me that if the SW 
> specification, design and coding considers sensor faults (as it of 
> course should), the normal approach for such critical SW is _not_ to use 
> strongly constrained subtypes and rely on Constraint_Error handling. The 
> normal approach is to add explicit and specific range checks and 
> sensor/computation cross-checks. That would be much easier to specify 
> and test, and would also make it much easier to identify the faulty 
> sensor(s) and to handle such situations properly.
> 
> -- 
> Niklas Holsti
> Tidorum Ltd

I know. SPARK and ParaSail do not allow exceptions for a reason. But because the MCAS was [mistakenly] classified as a non-critical component, programmers may choose to use less formal/rigorous methodology, which give rise for constraints and exceptions if Ada was selected.

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

On Thu, 18 Apr 2019 06:53:10 -0700 (PDT), tranngocduong@gmail.com declaimed
the following:

>
>a) Ada was used but programmers have chosen a wrong (too relaxed) subtype, or other language was used and programmers failed to code whatever equivalent to raising and handling a CONSTRAINT_ERROR. Simply: software bug.
>

	The common action on any exception is to log it (in flash memory) and
/restart/ the FMS software. Restarting likely includes synchronizing with
the second FMS -- but after such a synchronization, aircraft control would
have been given to the primary FMS; which likely would have almost
immediately produced an exception and.... repeat until the pilots manually
switch control to the second FMS processor.

>b) Contrary to general belief, the software was not programmed with multiple redundant computation. Simply: process failure.
>
>I chose to believe a).

	It is most likely a variant of B. MCAS was supposed to nudge the
aircraft attitude when it sensed a potential stall condition from just AoA
(airflow angle against the wings) with no concern for air speed;
pre-existing air speed computations were not changed by the addition of
MCAS (couldn't have been if MCAS can be manually disabled in flight).
Without the (formerly optional) hardware, this becomes a single sensor
matter -- and one which can not be detected as faulty (while each FMS may
have had its own sensor, during a disagreement, the primary FMS likely
pushes /its/ computed aircraft state to the secondary FMS which is supposed
to start computations from those values; probably diverging again until the
next sync interval -- get enough of these divergences and the secondary
might be the one to shut down; the FMS displays might show "SINGLE FMS"
mode])


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com 

Re: Boeing 737 and 737 MAX software

[Niklas Holsti]

On 19-04-18 19:21 , tranngocduong@gmail.com wrote:
> On Thursday, April 18, 2019 at 10:13:05 PM UTC+7, Niklas Holsti
> wrote:
>
>> The descriptions of the MCAS system that I have seen say quite
>> clearly that it used only one of the two AoA sensors mounted on the
>> aircraft (and that this single-sensor design is unacceptable for a
>> flight-control system that ended up with this level of authority
>> and criticality).
>>
>> I have not seen any statement about other standard SW that would be
>> able to flag an AoA sensor as faulty. There was an optional
>> addition that could do it, not mounted on the planes that crashed.
>> The whole MCAS system was an add-on and perhaps for that reason not
>> well integrated with the rest of the flight SW (this is speculation
>> on my part).
>>
> The optional "AoA disagree" indicator is just a surface issue. I've
> seen a couple of analysis stating that it is really not very helpful.

I didn't claim that it was helpful, but the fact that the "fix" is 
making it standard rather than optional suggests strongly that the MCAS 
originally used only one AoA sensor, confirming other descriptions of 
the issue.

> To my limited knowledge, AoA is a critical parameter that is used by
> many flight control algorithms, not just the MCAS. The real issue is
> thus the failure to detect an unreliable sensor. If the failure was a
> "feature, not bug", the entire flight software (and its certificate)
> would be questioned.

I've read rumours that even if the U.S. FAA lets the fixed 737 MAX fly 
again, other air safety authorities (Europe, for example) might not be 
satisfied, for that very reason -- suspicion that the flight software 
process was at fault.

 From more recent descriptions of the two crashes, it seems that the 
problem also involves complex interaction between MCAS, the enabling or 
disabling of the elevation trim motors, restarts of the control 
computer, and the fact that manual correction of the elevation trim 
becomes impossibly hard when the MCAS-commanded large "dive" trim 
applies large aerodynamic forces to the trim mechanism. Thus the problem 
was not only in the software process, but also in the controllability of 
the aircraft under anomalies -- a chain of failures, as typical for 
accidents.

>>> b) Contrary to general belief, the software was not programmed
>>> with multiple redundant computation. Simply: process failure.

    [snip]

> If b) was the case, not only the software, but the entire process
> would be questioned. A bug would take weeks or months to fix. A
> software would take years to re-engineer. A process would take
> decades to develop. Would Boeing as a company risk its very existence
> by comiting such a big mistake? I don't think so.

The suspicion involves Boeing sliding down two slippery slopes, as I 
understand it:

1) For MCAS in particular, its control authority was greatly increased 
from its first design to the flying version, but this was not propagated 
into a new consideration of its criticality.

2) For the process in general, an increasing complacency ("we know how 
to do it") and increasing delegation of checks from the FAA to Boeing 
(and other airplance builders), combined with specific driving forces 
for 737 MAX (urgency + desire to avoid pilot retraining).

I am reminded of the Space Shuttle O-rings... and perhaps also of the 
scandals with automotive SW hiding emissions, leading to multi-billion 
losses for the guilty European companies...

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Boeing 737 and 737 MAX software

[Paul Rubin]

Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
> On the issue of Ada subtypes, it seems to me that if the SW
> specification, design and coding considers sensor faults (as it of
> course should), the normal approach for such critical SW 

One of the criticisms of the decisions leading to the MCAS software is
that the software is certified only at DO-178B level C, defined as
software whose consequences are (https://en.wikipedia.org/wiki/DO-178B):

    Major – Failure is significant, but has a lesser impact than a
    Hazardous failure (for example, leads to passenger discomfort rather
    than injuries) or significantly increases crew workload (safety
    related)  

This is instead of level A (catastrophic, the whole plane can be lost),
or level B (hazardous, people can be injured).  The rationale was that
at worst MCAS going wrong would change the nose pitch by a few degrees
and then the pilot could fix it.  They didn't consider the possibility
of it activating over and over again, tilting a few more degrees each
time.

Since the software was treated as level C, its development and
certification process was less rigorous than what it would have gotten
at a more critical level.

Certifying and developing this system at level C instead of level A was
itself obviously some kind of process failure.  I believe finding out
how that happened is one of the investigation's objectives.

Re: Boeing 737 and 737 MAX software

[Randy Brukardt]

<tranngocduong@gmail.com> wrote in message 
news:70d7f427-ddce-4ec0-aba3-99edab0780bc@googlegroups.com...
...
> Would Boeing as a company risk its very existence by comiting such a big 
> mistake? I don't think so.

You have a crazy amount of faith in big companies. I think they're far more 
like a barely controllable self-perpetuating organizism. At most, the 
leaders can put in some inputs, but the outputs may not be anything like 
like the leaders intended.

Niklas gave several examples, there are many more (less critical) examples 
out there. Just look up "New Coke"!

                                           Randy.

Re: Boeing 737 and 737 MAX software

[Paul Rubin]

"Randy Brukardt" <randy@rrsoftware.com> writes:
> At most, the leaders can put in some inputs, but the outputs may not
> be anything like like the leaders intended.

Even worse, the outputs might be exactly what the leaders intended.
Boeing may have been a well managed company a few decades ago, but the
stuff coming out recently makes it sound like it's currently run by
Dilbert's pointy-haired boss.

Re: Boeing 737 and 737 MAX software

[Paul Rubin]

FWIW here's an article that just came out in IEEE Spectrum, by a
programmer and private pilot pontificating about the 737 software and
MCAS.  I don't have the impression that it's a good article from an
paviation standpoint but don't know enough about the topic to be sure.

Anyway, here it is, since it has been getting some attention:

https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

FYI: http://www.b737.org.uk/fmc.htm (Note: Smiths Industries sold the
branch to GE Aviation). U13 is the first for the MAX

http://www.b737.org.uk/mcas.htm


Interesting... http://www.b737.org.uk/glareshield.htm#fcc indicates that
Collins is responsible for the component that processes this sensor (Though
I suspect the inputs go through the GE Aviation FMS/FMC -- which likely
means only the primary FMS side is passed on). The FCC is more what might
be considered a "classic" "auto-pilot" [as can be seen -- desired air
speed, heading, altitude, and ascent/descent rates); the FMS handles route
optimization, GPS paths, etc. along with collecting and relaying data to
the display systems..


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com 

Re: Boeing 737 and 737 MAX software

[tranngocduong]

On Friday, April 19, 2019 at 5:04:54 AM UTC+7, Paul Rubin wrote:
> FWIW here's an article that just came out in IEEE Spectrum, by a
> programmer and private pilot pontificating about the 737 software and
> MCAS.  I don't have the impression that it's a good article from an
> paviation standpoint but don't know enough about the topic to be sure.
> 
> Anyway, here it is, since it has been getting some attention:
> 
> https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer

Thank you for introducing the article. 

I, too, am under similar impression. I think that the article may contain oversimplified or otherwise imprecise information. Nevertheless, I find it inspiring: it hepls me to come up with a theory which can explain many conflicting facts and viewpoints. (Example of conflicting facts, the general public believe MCAS is an anti-stall, i.e. critical, system, while Boeing insists it is a control-quality, i.e. non-critical, system.) 

I'm just trying to sort things up in my mind. I'm not intended to blame or to defend anyone.

1. The 737 has two, not only one, autopilots (AP1 and AP2). They work without any coordination, with the only exception that at most one can be active at a time. In particular, neither of them monitors or can override the other.

2. Technically, an autopilot can take input from a set of sensors on its side as well as a second set of sensors on the other side. However, as a design philosophy, to keep them as independent as possible, systematic effort was made to let them not to look at the other side's set of sensors unless it is absolutely necessary.

3. Each autopilot can detect sensor failures on its own side and manage to fly even in case of one or couple of simultaneous faulty sensors. It disengages if the number of faulty sensors go beyond of its ability.

4. For simplicity, the two autopilots are documented as "the autopilot" and there is only one button on either (captain or first-officer) side to engage/disengage one. The simplicity comes at certain inconvenience. For example, the captain can only activate AP1 while the first officer can only activate AP2, and the captain may fail but the first officer may succeed to activate "the autopilot".

5. On the 737 MAX, two anti-stall systems (AS1, AS2) were added, in the same design principle that there is no coordination between the four systems (AP1, AP2, AS1, AS2) except that at most one can be active at a time. Because AS1 and AS2 have authority to move control surfaces (such as the horizontal stabilizer), they're classified safety-critical.

6. Additionally on the MAX, "maneuvering characteristics" (MC), a control-quality tweaker, was added. The very purpose is to keep the control stick feeling exactly as it was on previous 737 generations. It changes the [artificial] control feeling by the classical mechanism: a "force generator" controlled by a "feel computer". It does not touch any control surface. Because of that, it is classified non-critical.

7. In order to save time, the MC and the ASs were submitted to certification, and were eventually certified, as a single system, under the umbrella name MCAS. The name Anti-Stall was changed to Augmentation System. The category was changed from safety-critical to non-critical.

8. Like AP1/AP2, AS1/AS2 is able to detect faulty AoA sensor on its side and terminate itself. However, a bug causes AS1/AS2 fail to do its job in certain cases, such as multiple sensor failures.

9. After the first accident, the FAA ordered to "fix the MCAS". After the second accident, it finds that the name "MCAS" not correct anymore and insists in resolving _two_ problems. Maybe MC and AS. Or AP and AS. And it insists in re-classifying the AS safety-critical. Although generally, that would be very hard to impossible, in this case that's no problem for Boeing: the AS was designed to be (1) safety-critical, and (2) separately of the AP, in the first place. Therefore, about 10 - 14 months (since the first accident) would suffice.

Re: Boeing 737 and 737 MAX software

[tranngocduong]

On Friday, April 19, 2019 at 1:20:21 AM UTC+7, Niklas Holsti wrote:
> ...
> manual correction of the elevation trim 
> becomes impossibly hard when the MCAS-commanded large "dive" trim 
> applies large aerodynamic forces to the trim mechanism. Thus the problem 
> was not only in the software process, but also in the controllability of 
> the aircraft under anomalies
> ...
> Niklas Holsti
> Tidorum Ltd

I know. I've seen a couple of pilots confirming this situtation from simulator experiments.

Every aircraft is designed so that resuming from anormaly, i.e. returning a control surface from heavily deflected position to neutral, is easy thanks to aerodynamic force. 

In particular, manual correction of the horizontal stabilizer becomes impossibly hard because the pilot is deflecting the elevator to the maximum, i.e. he/she is [unknowingly] generating maximal aerodynamic force which goes _against_ the manual correction.

Aerodynamics is friend, not enemy. Pilots are supposed to use it, not to fight it.

In order to return the stabilizer, one have to return the elevator to neutral first.

True, this maneuvre is not so easy as it sounds. It requires training and has became less well-known to pilots in the recent decades. Just like assembly language to programmers. The control stick is like a high-level programming language while the trim wheel is like the assembly language. It is more powerful and it gives more flexibility. But it requires more knowledge and responsibility. Nevertheless, it is there, and perfectly usable.

That said, I don't think manual controlability is a problem of the 737.

Re: Boeing 737 and 737 MAX software

[Norman Worth]

Dennis Lee Bieber wrote:
> On Thu, 18 Apr 2019 06:53:10 -0700 (PDT), tranngocduong@gmail.com declaimed
> the following:
> 
>>
>> a) Ada was used but programmers have chosen a wrong (too relaxed) subtype, or other language was used and programmers failed to code whatever equivalent to raising and handling a CONSTRAINT_ERROR. Simply: software bug.
>>
> 
> 	The common action on any exception is to log it (in flash memory) and
> /restart/ the FMS software. Restarting likely includes synchronizing with
> the second FMS -- but after such a synchronization, aircraft control would
> have been given to the primary FMS; which likely would have almost
> immediately produced an exception and.... repeat until the pilots manually
> switch control to the second FMS processor.
> 
>> b) Contrary to general belief, the software was not programmed with multiple redundant computation. Simply: process failure.
>>
>> I chose to believe a).
> 
> 	It is most likely a variant of B. MCAS was supposed to nudge the
> aircraft attitude when it sensed a potential stall condition from just AoA
> (airflow angle against the wings) with no concern for air speed;
> pre-existing air speed computations were not changed by the addition of
> MCAS (couldn't have been if MCAS can be manually disabled in flight).
> Without the (formerly optional) hardware, this becomes a single sensor
> matter -- and one which can not be detected as faulty (while each FMS may
> have had its own sensor, during a disagreement, the primary FMS likely
> pushes /its/ computed aircraft state to the secondary FMS which is supposed
> to start computations from those values; probably diverging again until the
> next sync interval -- get enough of these divergences and the secondary
> might be the one to shut down; the FMS displays might show "SINGLE FMS"
> mode])
> 
> 
A good programming language will not compensate for a bad system design!

Re: Boeing 737 and 737 MAX software

[russ lyttle]

On 4/22/19 3:36 PM, Norman Worth wrote:
> Dennis Lee Bieber wrote:
>> On Thu, 18 Apr 2019 06:53:10 -0700 (PDT), tranngocduong@gmail.com 
>> declaimed
>> the following:
>>
>>>
>>> a) Ada was used but programmers have chosen a wrong (too relaxed) 
>>> subtype, or other language was used and programmers failed to code 
>>> whatever equivalent to raising and handling a CONSTRAINT_ERROR. 
>>> Simply: software bug.
>>>
>>
>>     The common action on any exception is to log it (in flash memory) and
>> /restart/ the FMS software. Restarting likely includes synchronizing with
>> the second FMS -- but after such a synchronization, aircraft control 
>> would
>> have been given to the primary FMS; which likely would have almost
>> immediately produced an exception and.... repeat until the pilots 
>> manually
>> switch control to the second FMS processor.
>>
>>> b) Contrary to general belief, the software was not programmed with 
>>> multiple redundant computation. Simply: process failure.
>>>
>>> I chose to believe a).
>>
>>     It is most likely a variant of B. MCAS was supposed to nudge the
>> aircraft attitude when it sensed a potential stall condition from just 
>> AoA
>> (airflow angle against the wings) with no concern for air speed;
>> pre-existing air speed computations were not changed by the addition of
>> MCAS (couldn't have been if MCAS can be manually disabled in flight).
>> Without the (formerly optional) hardware, this becomes a single sensor
>> matter -- and one which can not be detected as faulty (while each FMS may
>> have had its own sensor, during a disagreement, the primary FMS likely
>> pushes /its/ computed aircraft state to the secondary FMS which is 
>> supposed
>> to start computations from those values; probably diverging again 
>> until the
>> next sync interval -- get enough of these divergences and the secondary
>> might be the one to shut down; the FMS displays might show "SINGLE FMS"
>> mode])
>>
>>
> A good programming language will not compensate for a bad system design!
Been trying to convince management of that for almost 50 years. No luck.

Re: Boeing 737 and 737 MAX software

[robin.vowels]

On Saturday, April 6, 2019 at 8:16:22 AM UTC+11, Paul Rubin wrote:
> Does anyone know anything about this?  It has been under some criticism
> lately.
> 
> I have heard that the 777 software was almost entirely in Ada.  It also
> sounds as if Boeing's software operation may have slipped in recent
> years, not good news for the 737 MAX.

It seems that computers (and their computer programs) 
are not really suitable to take command of aeroplanes. 

This week appeared a re-run of the (then new) A320 fatal crash on 
a demonstration flight (Air Crash Investigations). 

In that case, the computer overrode the pilot, 
even when the pilot called for full power and climbing. 
Power was increased, but the computer failed to set 
the ailerons to climb, so that the plane continued 
flying horizontally at 30 feet and into trees at the end of the runway. 
The pilot had made a gross error in flying too close to the ground 
(30 feet), and the computer thought that the pilot wanted the 
plane to land. 

Tonight was screened on TV a report on the two new Boeing 737 MAX 
planes that crashed -- one in March this year and another in 
October last year. 

In both cases, a sensor failed; it was the input to the stall 
correction computer.  The computer sensed that the plane was 
about to stall (it wasn't), and put the nose down.  But the 
attitude of the planes was set to 60 degrees, which is an 
awfully steep dive, and not merely a minor correction to help 
the plane to avoid a stall. 

The pilot had five seconds to consult  a "check" list 
(a thick manual) as to what to do.  Even if he had been 
able to correct the problem (by adjusting the ailerons), 
the computer applied the stall correction again for 
10 seconds (which resulted in another steep dive), 
and the pilot had another 5 seconds to correct the 
problem. This 5/10-second cycle repeated ad infinitum. 
In one case, the plane was close to the ground, as it had 
not long before taken off from the runway. Each steep dive 
took the plane closer to the ground ... 

One major point of the TV report was that Boeing had never 
made mention of this software feature in any manual, 
so no pilots had been trained to deal with a situation 
in which software took over flying the plane. 

Three questions: 
1. Whatever happened to the stick shaker? 
2. Why was the dive so ridiculously steep? 
3. Why did not the autopilot save the plane 
   as it careered towards the ground? 

Usually the best operator of a plane is the pilot, 
and he should always be able to take over from 
any computer program (autopilot or stall correction) 
and to fly the plane manually in the event that the 
automatic equipment fails. 

Re: Boeing 737 and 737 MAX software

[robin.vowels]

On Monday, May 6, 2019 at 12:29:13 AM UTC+10, robin...@gmail.com wrote:
> On Saturday, April 6, 2019 at 8:16:22 AM UTC+11, Paul Rubin wrote:
> > Does anyone know anything about this?  It has been under some criticism
> > lately.
> > 
> > I have heard that the 777 software was almost entirely in Ada.  It also
> > sounds as if Boeing's software operation may have slipped in recent
> > years, not good news for the 737 MAX.
> 
> It seems that computers (and their computer programs) 
> are not really suitable to take command of aeroplanes. 
> 
> This week appeared a re-run of the (then new) A320 fatal crash on 
> a demonstration flight (Air Crash Investigations). 
> 
> In that case, the computer overrode the pilot, 
> even when the pilot called for full power and climbing. 
> Power was increased, but the computer failed to set 
> the ailerons to climb, so that the plane continued 
> flying horizontally at 30 feet and into trees at the end of the runway. 
> The pilot had made a gross error in flying too close to the ground 
> (30 feet), and the computer thought that the pilot wanted the 
> plane to land. 
> 
> Tonight was screened on TV a report on the two new Boeing 737 MAX 
> planes that crashed -- one in March this year and another in 
> October last year. 
> 
> In both cases, a sensor failed; it was the input to the stall 
> correction computer.  The computer sensed that the plane was 
> about to stall (it wasn't), and put the nose down.  But the 
> attitude of the planes was set to 60 degrees, which is an 
> awfully steep dive, and not merely a minor correction to help 
> the plane to avoid a stall. 
> 
> The pilot had five seconds to consult  a "check" list 
> (a thick manual) as to what to do.  Even if he had been 
> able to correct the problem (by adjusting the ailerons), 
> the computer applied the stall correction again for 
> 10 seconds (which resulted in another steep dive), 
> and the pilot had another 5 seconds to correct the 
> problem. This 5/10-second cycle repeated ad infinitum. 
> In one case, the plane was close to the ground, as it had 
> not long before taken off from the runway. Each steep dive 
> took the plane closer to the ground ... 
> 
> One major point of the TV report was that Boeing had never 
> made mention of this software feature in any manual, 
> so no pilots had been trained to deal with a situation 
> in which software took over flying the plane. 
> 
> Three questions: 
> 1. Whatever happened to the stick shaker? 
> 2. Why was the dive so ridiculously steep? 
> 3. Why did not the autopilot save the plane 
>    as it careered towards the ground? 
> 
> Usually the best operator of a plane is the pilot, 
> and he should always be able to take over from 
> any computer program (autopilot or stall correction) 
> and to fly the plane manually in the event that the 
> automatic equipment fails.

CNN says today (6 May 2019):

<<In its statement Sunday, Boeing maintained that the software
  issue "did not adversely impact airplane safety or operation." >>

Are they (Boeing) stupid?

TWO of the planes crashed as a direct of the software issue.

CNN also revealed that Boeing knew of the problem months before
the crash.

In offering the MAX to customers, Boeing claimed that no new training
was required.
That was a bare-faced lie.  Boeing concealed the existence of
the stall software, and there was nothing about it in any of
the manuals issued to companies that bought the plane.

Pilots required training in order to be able to deal with the
sudden control of the plane by the stall software.

The insidious problem of the software (iterated above) is that
it doesn't just correct the plane's attitude; it repeatedly
does so at 5-second intervals, thereby preventing the pilots
from making a recovery.

Most disturbing: were the malfunction of the sensor to take place
immediately after take-off (or very soon thereafter),
there is no way that the pilots could recover,
with the software repeatedly putting the nose down.

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

On Mon, 6 May 2019 06:54:41 -0700 (PDT), robin.vowels@gmail.com declaimed
the following:

>In offering the MAX to customers, Boeing claimed that no new training
>was required.
>That was a bare-faced lie.  Boeing concealed the existence of
>the stall software, and there was nothing about it in any of
>the manuals issued to companies that bought the plane.
>

	From a fairly detailed article I'd read, MCAS isn't even considered an
anti-stall system.

	The primary purpose of MCAS was to compensate for the fact that the MAX
engines are mounted further forward and higher than previous 737 series
engines. This change was needed to obtain ground clearance with the larger
engines.

	The position change also has a subtle effect in that they produce added
lift in front of the wings at some angles of attack; this would have
produced a tendency to increase a "nose-up" attitude.



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com 

Re: Boeing 737 and 737 MAX software

[Paul Rubin]

Dennis Lee Bieber <wlfraed@ix.netcom.com> writes:
> 	Unless things have changed severely -- GE Aviation (formerly Smith's
> Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
> the processor boxes).

Don't know if this is the FMS but it sounds like things may have changed:

https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers

It remains the mystery at the heart of Boeing Co.’s 737 Max crisis: how
a company renowned for meticulous design made seemingly basic software
mistakes leading to a pair of deadly crashes. Longtime Boeing engineers
say the effort was complicated by a push to outsource work to lower-paid
contractors.

The Max software -- plagued by issues that could keep the planes
grounded months longer after U.S. regulators this week revealed a new
flaw -- was developed at a time Boeing was laying off experienced
engineers and pressing suppliers to cut costs.

Increasingly, the iconic American planemaker and its subcontractors have
relied on temporary workers making as little as $9 an hour to develop
and test software, often from countries lacking a deep background in
aerospace -- notably India.

...

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

On Fri, 28 Jun 2019 16:45:44 -0700, Paul Rubin <no.email@nospam.invalid>
declaimed the following:

>Dennis Lee Bieber <wlfraed@ix.netcom.com> writes:
>> 	Unless things have changed severely -- GE Aviation (formerly Smith's
>> Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
>> the processor boxes).
>
>Don't know if this is the FMS but it sounds like things may have changed:
>
>https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers
>

	Note that the article emphasizes 

"""
Based on resumes posted on social media, HCL engineers helped develop and
test the Max’s flight-display software, while employees from another Indian
company, Cyient Ltd., handled software for flight-test equipment.
"""

	Flight displays are separate processors that receive "instructions"
from the FMS on what to display (and flight-test equipment, to me, is not
flight software/hardware itself, but instead may be a ground-based console
to emulate flight sensors and inputs). Overly simplifying -- it's like the
ubiquitous 2line x 16character LCDs... the display has to interpret the
drawing instructions sent from the main application processor, but is not
controlling the aircraft itself..

	I'll concede I don't know who actually built the display/keyboard units
used for input to the FMS (and the other displays of the "glass cockpit"
are another matter -- during my short four years the "autopilot" and other
displays were simulated on Windows boxes for testing FMS software and
flight boxes. [I envied those testers -- they got to use a TCP "dataloader"
to the test boxes... I had to test the "bootrom" using the floppy disk
"field" dataloader -- which took hours just to load the FMS software; a
full test took two 5 hour sessions to load FMS, databases, verify it
wouldn't load oversize database on older flight boxes, verify dataload
option disappears from menu system when (simulated) aircraft is in the air
(no load on suspension)...])


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

Re: Boeing 737 and 737 MAX software

[Paul Rubin]

Dennis Lee Bieber <wlfraed@ix.netcom.com> writes:
> Overly simplifying -- it's like the ubiquitous 2line x 16character
> LCDs... the display has to interpret the drawing instructions sent
> from the main application processor, but is not controlling the
> aircraft itself..

Ah, good point, though I wonder if the news article is implying that the
software is connected to the crashes in some way.  Separate question:
would MCAS count as part of the FMS?  Just wondering.

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

On Fri, 28 Jun 2019 20:38:23 -0700, Paul Rubin <no.email@nospam.invalid>
declaimed the following:

>
>Ah, good point, though I wonder if the news article is implying that the
>software is connected to the crashes in some way.  Separate question:
>would MCAS count as part of the FMS?  Just wondering.

	Well, based upon
https://aviation.stackexchange.com/questions/54545/is-the-737s-autopilot-part-of-the-fms-or-is-it-a-separate-entity
"FMS" is the entire system including the displays, while the FMC is the
computer that is in charge of the flight management (waypoints, fuel usage,
etc. and distributing data to the other components) while the "autopilot"
is another component. I'd expect MCAS to be software in the FMC component.

I'm probably a bit sloppy in that I tend to read "FMS" as "Flight
Management SOFTWARE" which resides in the FMC (a 68040 processor -- the
code I worked on assumed 30MHz, but I've read an article indicating the
processor clock itself is twice that -- original FMC models were 20MHz
[40?]).


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

Re: Boeing 737 and 737 MAX software

[robin.vowels]

On Monday, May 6, 2019 at 12:29:13 AM UTC+10, robin...@gmail.com wrote:
> On Saturday, April 6, 2019 at 8:16:22 AM UTC+11, Paul Rubin wrote:
> > Does anyone know anything about this?  It has been under some criticism
> > lately.
> > 
> > I have heard that the 777 software was almost entirely in Ada.  It also
> > sounds as if Boeing's software operation may have slipped in recent
> > years, not good news for the 737 MAX.
> 
> It seems that computers (and their computer programs) 
> are not really suitable to take command of aeroplanes. 
> 
> This week appeared a re-run of the (then new) A320 fatal crash on 
> a demonstration flight (Air Crash Investigations). 
> 
> In that case, the computer overrode the pilot, 
> even when the pilot called for full power and climbing. 
> Power was increased, but the computer failed to set 
> the ailerons to climb, so that the plane continued 
> flying horizontally at 30 feet and into trees at the end of the runway. 
> The pilot had made a gross error in flying too close to the ground 
> (30 feet), and the computer thought that the pilot wanted the 
> plane to land. 
> 
> Tonight was screened on TV a report on the two new Boeing 737 MAX 
> planes that crashed -- one in March this year and another in 
> October last year. 
> 
> In both cases, a sensor failed; it was the input to the stall 
> correction computer.  The computer sensed that the plane was 
> about to stall (it wasn't), and put the nose down.  But the 
> attitude of the planes was set to 60 degrees, which is an 
> awfully steep dive, and not merely a minor correction to help 
> the plane to avoid a stall. 
> 
> The pilot had five seconds to consult  a "check" list 
> (a thick manual) as to what to do.  Even if he had been 
> able to correct the problem (by adjusting the ailerons), 
> the computer applied the stall correction again for 
> 10 seconds (which resulted in another steep dive), 
> and the pilot had another 5 seconds to correct the 
> problem. This 5/10-second cycle repeated ad infinitum. 
> In one case, the plane was close to the ground, as it had 
> not long before taken off from the runway. Each steep dive 
> took the plane closer to the ground ... 
> 
> One major point of the TV report was that Boeing had never 
> made mention of this software feature in any manual, 
> so no pilots had been trained to deal with a situation 
> in which software took over flying the plane. 
> 
> Three questions: 
> 1. Whatever happened to the stick shaker? 
> 2. Why was the dive so ridiculously steep? 
> 3. Why did not the autopilot save the plane 
>    as it careered towards the ground? 
> 
> Usually the best operator of a plane is the pilot, 
> and he should always be able to take over from 
> any computer program (autopilot or stall correction) 
> and to fly the plane manually in the event that the 
> automatic equipment fails.

Recently was screened, for the first time, 
on Aircraft Crash Investigations [probably also in the USA] 
the report on the Airbus A330 Qantas that experienced 
loss of control of the aircraft when it suddenly dived 
at 10 degrees.  The controls failed to right the plane. 
Passengers and crew hit the ceiling, injuring themselves 
and breaking ceiling panels. 

The plane eventually responded to stick control. 

The same thing happened again, with angle of attack 10 degrees. 

Again, after a while, the plane responded to the pilot's control. 

Analysis showed that one of the units sends data (height and 
angle of attack) to the flight computer. 
The data is sent with a tag.  The unit got those tags mixed up: 
it sent one set of the height data with the angle-of-attack tag, 
and vice versa. 

That (apparently) appeared to be a programming error. 
The information was sent to Airbus. 

As to landing the plane, the pilots made an emergency landing, 
side-slipping the plane in order to lose height quickly, 
but also to gain speed in case the same thing happened again, 
which would give them more chance of recovering. The pilots 
realized that, if the plane misbehaved close to ground, 
with a normal slow descent, there would be no chance of recovery.

Re: Boeing 737 and 737 MAX software

[robin.vowels]

On Saturday, June 29, 2019 at 9:45:46 AM UTC+10, Paul Rubin wrote:
> Dennis Lee Bieber <w.......@ix.netcom.com> writes:
> > 	Unless things have changed severely -- GE Aviation (formerly Smith's
> > Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
> > the processor boxes).
> 
> Don't know if this is the FMS but it sounds like things may have changed:
> 
> https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers
> 
> It remains the mystery at the heart of Boeing Co.’s 737 Max crisis: how
> a company renowned for meticulous design made seemingly basic software
> mistakes leading to a pair of deadly crashes. Longtime Boeing engineers
> say the effort was complicated by a push to outsource work to lower-paid
> contractors.
> 
> The Max software -- plagued by issues that could keep the planes
> grounded months longer after U.S. regulators this week revealed a new
> flaw -- was developed at a time Boeing was laying off experienced
> engineers and pressing suppliers to cut costs.
> 
> Increasingly, the iconic American planemaker and its subcontractors have
> relied on temporary workers making as little as $9 an hour to develop
> and test software, often from countries lacking a deep background in
> aerospace -- notably India.

The basic issue behind this problem (and that of the Airbus's 3xx series)
is that the pilot should be able to take control.
In both types of plane, the pilot could do nothing with the controls
when the software malfunctioned.

Re: Boeing 737 and 737 MAX software

[Paul Rubin]

Dennis Lee Bieber <wlfraed@ix.netcom.com> writes:
> 	Unless things have changed severely -- GE Aviation (formerly Smith's
> Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
> the processor boxes).

It looks like Collins Aerospace (formerly Rockwell Collins) is now
getting some heat over the flight deck software:

https://www.nytimes.com/2019/11/07/business/boeing-737-max-collins.html

Re: Boeing 737 and 737 MAX software

[Dennis Lee Bieber]

On Thu, 07 Nov 2019 17:12:53 -0800, Paul Rubin <no.email@nospam.invalid>
declaimed the following:

>Dennis Lee Bieber <wlfraed@ix.netcom.com> writes:
>> 	Unless things have changed severely -- GE Aviation (formerly Smith's
>> Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
>> the processor boxes).
>
>It looks like Collins Aerospace (formerly Rockwell Collins) is now
>getting some heat over the flight deck software:
>
>https://www.nytimes.com/2019/11/07/business/boeing-737-max-collins.html

	Though "the displays" don't really process sensor information. The FMS
sends display instructions to the display boxes, which then format and
render the actual screens. Unless they believe the display boxes should
have taking those display instructions and done local matching (for
example, AoA sensor vs a gyro pitch angle, and then "red-flagged" the
display of those items).

	If Collins is now building the FMS processor boxes, they must be
building them to the same specification as the GE boxes -- since the
"BootROM" which GE provided the older boxes underwent updates for the MAX
model, yet was tested on the same old processor boxes (It won't run on the
very oldest as those had a different memory map -- pre-MAX BootROM used a
memory layout that was common to both generations of processor boxes; the
MAX software moved some stuff into a region that doesn't exist on the
oldest).

	I'm fairly certain that the FMS for MAX was also GE effort -- though
that may have been the waypoint/navigation/routing/fuel management stuff,
with Collins maybe doing the sensor and flight control integration stuff.
{I'd been on the BootROM/"platform" side of the building, FMS was another
room}




-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed@ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

Re: Boeing 737 and 737 MAX software

[robin.vowels]

On Friday, November 8, 2019 at 12:13:07 PM UTC+11, Paul Rubin wrote:
> Dennis Lee Bieber <w.....@ix.netcom.com> writes:
> > 	Unless things have changed severely -- GE Aviation (formerly Smith's
> > Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
> > the processor boxes).
> 
> It looks like Collins Aerospace (formerly Rockwell Collins) is now
> getting some heat over the flight deck software:
> 
> https://www.nytimes.com/2019/11/07/business/boeing-737-max-collins.html

Since Boeing manufactured the plane, it's basically their problem -
no matter who they contract to supply parts.

The MAX software might not be the only problem that Boeing will have
with the Max.
737 NGs are suffering premature fatigue cracks with the pickle forks -
these are what hold the wings to the body of the aircraft.
These fatigue cracks are occurring after some 30,000 flights,
which is about half of the design life of some 60,000 flights.

The MAXes have larger and more powerful engines that may stress the
wings more than the earlier engines.  This may result in fatigue cracks
appearing even earlier than on the NG's.

Re: Boeing 737 and 737 MAX software

[Optikos]

On Monday, November 18, 2019 at 5:16:42 AM UTC-6, robin...@gmail.com wrote:
> On Friday, November 8, 2019 at 12:13:07 PM UTC+11, Paul Rubin wrote:
> > Dennis Lee Bieber <w.....@ix.netcom.com> writes:
> > > 	Unless things have changed severely -- GE Aviation (formerly Smith's
> > > Aerospace, formerly Lear Siegler) produces the 737 FMS software (and also
> > > the processor boxes).
> > 
> > It looks like Collins Aerospace (formerly Rockwell Collins) is now
> > getting some heat over the flight deck software:
> > 
> > https://www.nytimes.com/2019/11/07/business/boeing-737-max-collins.html
> 
> Since Boeing manufactured the plane, it's basically their problem -
> no matter who they contract to supply parts.

Fractional/contributing culpability of suppliers & subcontractors is a key portion of the USA's legal system.  And perhaps more to the point, fractional/contributing culpability of suppliers & subcontractors (and the meticulous tracking/feedback/transparency thereof) is a key tenant of AS9100 that is the mantra (largely driven by Boeing no less) throughout the aerospace industry.

> The MAX software might not be the only problem that Boeing will have
> with the Max.

Well, this is obviously quite true in the MAX since software written in Ada was being utilized to cover up the fundamental aerodynamic problem of excessive lift by over-powered engines.  (Conversely, the software taking input from a lone single-point-of-failure sensor was unwise, when at least 1 sensor was available that could be presented in conflict for human authorization of what to do, and in some configurations 2 sensors that could vote.)

> 737 NGs are suffering premature fatigue cracks with the pickle forks -
> these are what hold the wings to the body of the aircraft.
> These fatigue cracks are occurring after some 30,000 flights,
> which is about half of the design life of some 60,000 flights.

Another potential symptom of excessive force by engines in the pre-MAX variant, as you note below.

> The MAXes have larger and more powerful engines that may stress the
> wings more than the earlier engines.  This may result in fatigue cracks
> appearing even earlier than on the NG's.

A not-quite-fully-stated subtext of this thread (and more so in this most-recent posting) is:

1) How much should Ada software engineers & the software management chain thereof be merely strictly obedient servants that incompletely/pathetically cover up hardware problems with software?  “No null pointer de-references and impeccable type-correctness when the design is viewed in the small, but oopsie it still slaughters people by the hundreds at a time when the design is viewed in the large without any significant amount of compiler enforcement of higher-order assurances of correctness.”

2) How much should Ada software engineers implement software to knowingly read from a singe-point-of-failure sensor when 1 or 2 more sensors are available, just because the specification said to read from “a” sensor?

3) How much should Ada software engineers have a glorious language, then let slipshod topics like this slide outside the language in a not-my-job or above-my-paygrade or contractual-obligation or don't-make-waves attitude?  Or equivalently how much should Ada software engineers allow these things to be human debates with management instead of moving them within a regime of compiler enforcement to beat over the head of management to prove management or system engineers wrong:  hey, look at where your misleadership led, to an otherwise irresolvable compile-time error?

4) Indeed, should the Ada programming language and/or Ada's libraries/frameworks/provers present engineering best-practices to the ultimate app-domain, such as an in-your-face disdain for single points of failure and other obvious higher-order brain-fart mistakes or, worse, intentional do-it-anyway mismanagement from up the chain of command?

5) Couldn't a valid criticism of Ada be:  Ada solves the 1975-era software-quality problem well, but doesn't solve higher-order software-quality (and system-quality) issues that have become prominent, say, post-1980?  Or equivalently, isn't the tragedy of Ada ultimately not its lack of dominance & lack of overwhelming popularity, but rather the most potent tragedy of Ada is that it ceased pursuing ever more-immense/aggressive variants of its originally-chartered mission, remaining stunted in solving primarily software-engineering challenges known in, say, 1975 with just more & more icing on the same old cake?  (I view SPARK as effectively saying this same criticism of Ada.  I view Rust as even effectively saying that Ada rested on its laurels of solving merely even the 1975-era problem too soon.)  I have observed such a false sense of perfection from managers & lead engineers who brag about software written in Ada simply because it was written in Ada and thus is naturally high-quality software by definition.

Be careful how you answer or dismiss those questions above, because your answers might strongly resemble the analogous answers or dismissals that some portions of the C or C++ community might give to those same questions if they were directed toward C or C++ instead of toward Ada.