NASA confirms seriousness of 2nd Starliner software issue


Genesis cover

On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.

 
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit.

 
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
 

"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs." --San Antonio Express-News

At a press conference today, NASA and Boeing officials confirmed the rumors that there was a second software error during Starliner’s unmanned demo mission in December that might have caused a serious failure had it not been caught on time.

[After the first software error], engineers began reviewing other critical software sequences as a precaution and discovered yet another problem. Software used to control thruster firings needed to safely jettison the Starliner’s service module just before re-entry was mis-configured, set for the wrong phase of flight.

Had the problem not been found and corrected, the cylindrical service module’s thrusters could have fired in the wrong sequence, driving it back into the crew module and possibly triggering a tumble or even damaging the ship’s protective heat shield.

While a detailed analysis was not carried out at the time, “nothing good can come from those two spacecraft bumping back into one another,” said Jim Chilton, a senior vice president for Boeing Space and Launch.

That two different software errors were not caught prior to flight has NASA demanding a complete review of Boeing’s quality control systems. And NASA here is correct. Boeing as a company appears to have fundamental quality control issues up and down the line, in all its projects. A complete review appears warranted.

Readers!
 

My July fund-raiser for Behind the Black is now over. The support from my readers was unprecedented, making this July campaign the best ever, twice over. What a marvelous way to celebrate the website's tenth anniversary!
 

Thank you! The number of donations in July, and continuing now at the beginning of August, is too many for me to thank you all personally. Please forgive me by accepting my thank you here, in public, on the website.
 

If you did not donate or subscribe in July and still wish to, note that the tip jar remains available year round.


 

Regular readers can support Behind The Black with a contribution via paypal:

Or with a subscription with regular donations from your Paypal or credit card account:


 

If Paypal doesn't work for you, you can support Behind The Black directly by sending your donation by check, payable to Robert Zimmerman, to
 
Behind The Black
c/o Robert Zimmerman
P.O.Box 1262
Cortaro, AZ 85652

6 comments

  • Scott M.

    Valve timing? They didn’t have the proper valve timing? To paraphrase an originally-NSFW quote from the film ‘A History Of Violence’…HOW DO YOU MESS THAT UP?

    (Takes deep breath)

    Okay. SpaceX has had its share of mess-ups, but those have been real-world, physical things. CRS-7 was due to a not-up-to-spec strut, while AMOS-6 loss was due to an unexpected interaction between carbon fiber and supercooled liquid oxygen.

    The Boeing/Starliner failures are due to things which should have been caught on any competent ground test. The fact that they weren’t casts deep doubts on their entire process. If NASA allows humans on the next Starliner it’ll be (IMHO) just as bad as letting the Challenger launch during cold weather. Sure, it might work out fine but boy are they rolling the dice.

  • tom

    QFA:”we had numerous process escapes”

    I have not followed this closely at all, but I suspect they were following Agile Methodology, which is the latest *management* buzzword fad that has taken I.T. by storm, and has been used to replace other methodologies even where it is not appropriate.

    It has a very short development-test-release life-cycle that all too often is used to force a shortening of the testing cycle, and increases “successful” software release which makes project management look good, on paper.

    While it is appropriate for products with quick turnaround like User Interfaces were flaws are quickly found while in use and fixed, and damage can be minimal, it has also been used where it is not appropriate, like enterprise size projects that formerly would use the Waterfall model, and take weeks or months for the dev-test-release cycle. In projects this size an error can propagate through systems to the point it can shut down mission critical applications, and take days if not weeks to diagnose and repair as the error(s) are traced through multiple connected systems and databases.

    I suspect software for an air/space craft in-flight is closer to the latter, where it cannot be easily fixed when errors are found in use.

  • J Fincannon

    The press conference brought up the issue of possible cell phone tower interference/noise with the orbiting spacecraft space to ground communications system. I had never heard this being a problem in spacecraft before. Has anyone?

  • Steve C

    Boeing has gone past poor quality control to a corporate culture of sloppiness. The company needs a wire brush enema. Fire anyone in a corner office, establish an QC group totally independent from production. On the floor, reward success, punish failure and if the union balks, move the whole mess to a Right -to -Work state. All this is in lu of letting them go broke and selling off the pieces.

  • wayne

    Steve–
    good stuff.

    Michigan is open for business and we have a skilled workforce….

    “Michigan became the 24th “Right to Work” state when Governor Rick Snyder signed into law on December 11, 2012, Public Act No. 348.”

    Biden tells coal workers to learn code for future jobs
    1-1-2020
    https://youtu.be/I-XXrRZlkVo
    3:34

  • Andrew

    I was in the business of software Quality Control back in the 90s. As I was leaving that industry I was already noticing a sever reluctance on the part of computer programmers to submit their precious coding to a thorough QA pass, first by “black box” testers who code actually read code, then serious abusive passes by “White Box” testers who banged on the software based on the most abusive end user actions that they could imagine.

    You can see the results in the gaming field. Games that have memory leaks, glitches, that actually grind your entire operating system and HARDWARE to a halt causing the entire computer to reboot itself. Boeing has, apparently, given all the software issues that have been cropping up in the 737 MAX debacle, and now this mess, has apparently been a leader in the foolish notion that coders write perfect code that does not need testing.

    This crap is inexcusable. Microsoft did such a crappy job when I was in the business that the running joke was, “Do you actually want your autopilot to run on WINDOWS?!?!?!?!?!?!?!?!?” Well guess what. Boeing apparently took that as their particular path to glory.

Leave a Reply

Your email address will not be published. Required fields are marked *