NASA safety panel raises more questions about Boeing and Starliner

In its quarterly meeting yesterday, NASA’s safety panel raised more questions about the software problems during the unmanned demo mission of Boeing’s Starliner manned capsule in December.

NASA’s Aerospace Safety Advisory Panel (ASAP) revealed today that a second software error was discovered during the uncrewed Boeing Starliner flight test in December. Had it gone undetected during the flight, it had the potential to cause “catastrophic spacecraft failure” during reentry. The panel wants a complete review of Boeing’s software verification processes before NASA decides whether a second uncrewed flight test is needed. In an email this evening, Boeing said it appreciates the input and is working on a plan with NASA to address all the issues and decide what comes next.

In that Boeing email it noted that it was “unclear” what the consequences would have been if this second software issue had not been fixed.

The safety panel also called for an overall organizational review of the entire Boeing company, similar to the review done to SpaceX after Elon Musk was videoed taking a toke on a joint during a podcast interview.

The decision on whether Boeing will be required to fly another unmanned demo mission is targeted for before the end of February.

One comment: While there is clear evidence here that Boeing had issues on that demo flight that must be resolved before humans fly on Starliner, we must also recognize that NASA’s safety panel has an unfortunate tendency to overstate risk, demanding margins of safety that are frequently unrealistic for an endeavor pushing the envelope of exploration. That panel has also exhibited an almost corrupt bias against private commercial space, while looking past much more serious safety issues in the NASA-built SLS and Orion programs.

At the same time, the larger corporate issues here with Boeing do appear far more systemic and concerning that those that occurred with SpaceX. A cold independent audit of the company by NASA could actually do Boeing a lot of good.

4 comments

Boeing budgets for extra unmanned Starliner test

Capitalism in space: Boeing has put aside $410 million in its next budget to pay for a possible second unmanned Starliner test, just in case NASA demands it.

The company said in its fourth quarter earnings release Jan. 29 that it was taking the charge “primarily to provision for an additional uncrewed mission for the Commercial Crew program, performance and mix.” It noted that NASA was still reviewing data from the Orbital Flight Test (OFT) mission in December that was cut short, without a docking at the International Space Station, by a timer problem.

“NASA is in the process of reviewing the data from our December 2019 mission,” Greg Smith, chief financial officer at Boeing, said in an earnings call. “NASA’s approval is required to proceed with a flight test with astronauts on board. Given this obligation, we are provisioned for another uncrewed mission.” Neither he nor Boeing’s new chief executive, David Calhoun, elaborated on that during the call, which was devoted primarily to issues related to the company’s 737 MAX airliner.

It might be too early to say, but my instincts are telling me that this decision, made very quickly, is a very good sign for Boeing. It suggests that Calhoun doesn’t fool around, that he takes very seriously the need for Boeing to serve its customers. In the past Boeing would have lobbied NASA, its customer, to pay for a possible additional flight (something NASA is not required to do according to the contract). Now Boeing instead makes it clear that it has accepted the responsibility of that additional flight, right off the bat, something that any good and healthy company should do.

3 comments

Boeing releases video of Starliner’s first orbital demo flight

Capitalism in space: Boeing has released a video showing what it was like to be on its Starliner capsule during its first orbital demo flight on December 20, 2019.

Flying alongside the uncrewed Starliner’s only official passenger — a spacesuit-clad, instrumented dummy (or anthropometric test device) named “Rosie” (after the World War II icon Rosie the Riveter), Snoopy, in plush doll form, served as the vehicle’s “zero-g indicator.” The video shows the doll floating weightless at the end of its “leash” after the Starliner entered Earth orbit.

The video is embedded below the fold. It is relatively boring, which actually is a good thing. The interior of the capsule does not seem much disturbed during each phase of the flight, from launch, separation from launch vehicle, and touchdown.
» Read more

1 comment

Starliner lands safely after failed orbital insertion

Capitalism in space:Boeing’s Starliner capsule successfully landed today in New Mexico, returning to Earth prematurely because of its failure to reach its proper orbit after launch two days ago.

The article quotes extensively from both NASA and Boeing officials touting the many successful achievements of this flight, while trying to minimize the failure that prevented the capsule from docking with ISS properly. And that failure?

The mission elapsed timer issue that cut short Starliner’s planned eight-day mission started before the spacecraft lifted off Friday from Cape Canaveral aboard a United Launch Alliance Atlas 5 rocket, according to Chilton. “Our spacecraft needs to reach down into the Atlas 5 and figure out what time it is, where the Atlas 5 is in its mission profile, and then we set the clock based on that,” Chilton said in a press conference Saturday. “Somehow we reached in there and grabbed the wrong (number). This doesn’t look like an Atlas problem. This looks like we reached in and grabbed the wrong coefficient.”

“As a result of starting the clock at the wrong time, the spacecraft upon reaching space, she thought she was later in the mission, and, being autonomous, started to behave that way,” Chilton said. “And so it wasn’t in the orbit we expected without the burn and it wasn’t in the attitude expected and was, in fact, adjusting that attitude.”

I read this and find myself appalled. While I agree that overall the mission proved the capsule capable of launching humans to ISS (which is why NASA is considering making the next Starliner mission manned despite this failure), this failure suggests a worrisome lack of quality control at Boeing. I can’t even imagine how the Starliner software could be mis-configured to “grab the wrong number.” This explanation makes no sense, and suggests they are spinning the failure to avoid telling us what they really did wrong.

Either way, I suspect that NASA will approve a manned launch for Starliner’s next orbital flight, but will do so only after dwelling on the problem for at least six months.

14 comments

Starliner launch fails, spacecraft to return to Earth

After being successfully placed in a preliminary orbit by ULA’s Atlas 5 rocket early this morning, Boeing’s Starliner capsule failed to reach its required orbit for docking with ISS when its own rocket engines did not fire properly at the right time.

The orbit it is in is stable, and the spacecraft is undamaged. Engineers now plan to bring it back to Earth on Sunday, landing at White Sands, New Mexico.

It appears some software issue had the capsule fire its own rockets either at the wrong time or for too short a time. The spacecraft was then in the wrong orbit, and needed to use too much fuel to correct this issue, making it impossible to dock with ISS.

More information here:

However, for reasons Boeing engineers do not yet understand, Starliner’s Mission Event Timer clock malfunctioned, causing the vehicle to think it was at a different point in the mission and at a different time in its mission that it actually was.

…This resulted in Starliner’s Reaction Control System thinking the Orbit Insertion Burn was underway and executing a series of burns to keep the vehicle oriented in the insertion burn attitude; however, the Orbit Insertion Burn was not actually occurring.

When mission controllers realized the issue, they sent manual commands to Starliner to perform an Orbit Insertion Burn in a backup window that came roughly eight minutes after the planned maneuver. However, a known and brief gap in NASA satellite communications caused a further delay.

By the time Starliner was finally able to burn its engines and get into a stable orbit, it had burned 25% more propellant than anticipated.

Boeing is certainly not having a good year. First it has had to shut down production on its new 737-Max airplane due to several crashes caused by software issues. Next its SLS rocket for NASA has had endless cost overruns and delays. Now Starliner fails during its first launch.

For ULA, however, the Atlas 5 rocket performed exactly as planned, so this launch gets listed as a success. They have now completed 5 launches this year.

31 comments

Inspector general slams NASA’s management for bonus payments to Boeing

In a report [pdf] issued yesterday, NASA’s inspector general blasted the agency’s manned commercial space management for issuing a $287 million bonus payment to Boeing to help it avoid delays in developing its Starliner capsule — which would have caused gaps in future American flights to ISS — even though the cost to use Russian Soyuz capsules would have been far less.

Worse, the agency never even allowed SpaceX to make its own competitive offer.

NASA agreed to pay Boeing Co (BA.N) a $287 million premium for “additional flexibilities” to accelerate production of the company’s Starliner crew vehicle and avoid an 18-month gap in flights to the International Space Station. NASA’s inspector general called it an “unreasonable” boost to Boeing’s fixed-priced $4.2 billion dollar contract.

Instead, the inspector general said the space agency could have saved $144 million by making “simple changes” to Starliner’s planned launch schedule, including buying additional seats from Russia’s space agency, which the United States has been reliant on since the 2011 retirement of its space shuttle program.

…NASA justified the additional funds to avoid a gap in space station operations. But SpaceX, the other provider, “was not provided an opportunity to propose a solution, even though the company previously offered shorter production lead times than Boeing,” the report said. [emphasis mine]

I’ve read the report, and from it the impression is clear that when NASA management discovered that Boeing was facing delays in Starliner and needed extra cash, it decided to funnel that cash to it, irrespective of cost. While it is likely that the agency did so because it did not wish to buy more Russian Soyuz seats, it makes no sense that it didn’t ask SpaceX for its own competitive bid. By not doing so the management’s foolish bias towards Boeing is starkly illustrated

Eric Berger at Ars Technica also notes that the report makes clear how Boeing’s prices for Starliner are 60% higher than SpaceX’s Crew Dragon prices, further illustrating how the agency favors Boeing over SpaceX.

Boeing’s per-seat price already seemed like it would cost more than SpaceX. The company has received a total of $4.82 billion from NASA over the lifetime of the commercial crew program, compared to $3.14 billion for SpaceX. However, for the first time the government has published a per-seat price: $90 million for Starliner and $55 million for Dragon. Each capsule is expected to carry four astronauts to the space station during a nominal mission.

What is notable about Boeing’s price is that it is also higher than what NASA has paid the Russian space corporation, Roscosmos, for Soyuz spacecraft seats to fly US and partner-nation astronauts to the space station. Overall, NASA paid Russia an average cost per seat of $55.4 million for the 70 completed and planned missions from 2006 through 2020. Since 2017, NASA has paid an average of $79.7 million.

I don’t have a problem with NASA favoring Boeing over Russia, considering the national priorities. I can also understand the agency’s willingness to keep buying some Starliner seats in order to guarantee an American launch redundancy. However, giving Boeing even more money to keep its schedule going, when SpaceX is available to fill the gaps, demonstrates the corruption in the agency’s management. They haven’t the slightest understanding of how private enterprise and competition works.

The report is also filled with the same tiresome complaints about the on-going delays to the manned commercial program, focusing greatly on past technical issues (now mostly solved) while hiding in obscure language how it is NASA’s paperwork that is likely to cause all further delays.

34 comments

More detail on pad abort test parachute issue

At a press telecon yesterday Boeing outlined in more detail the cause of the failure of one main parachute to deploy during its November 4 Starliner pad abort test.

In a call with reporters, John Mulholland, vice president and program manager for commercial crew at Boeing, said an investigation after the Nov. 4 test at White Sands Missile Range in New Mexico led the company to conclude that a “lack of secure connection” between a pilot parachute and the main parachute prevented that main parachute, one of three, from deploying.

The pilot parachute is designed to deploy first, and pull out the main parachute. However, Mulholland said that hardware inspections and photographs taken during “closeout” of the vehicle prior to the test showed that a pin that links the pilot and main parachutes was not inserted properly.

“It’s very difficult, when you’re connecting that, to verify visually that it’s secured properly,” he said, in part because that portion of the parachute system is enclosed in a “protective sheath” intended to limit abrasion but which also makes it difficult to visually confirm the pin is in place. “In this particular case that pin wasn’t through the loop, but it wasn’t discovered in initial visual inspections because of that protective sheath.”

Mulholland said Boeing is modifying assembly procedures through what he called “fairly easy steps,” such as pull tests, to ensure those pins are properly installed. Technicians have already confirmed that the same parachute linkages are properly installed on the three parachutes on the Starliner that will launch in December on an orbital flight test to the International Space Station. [emphasis mine]

That a hardware inspection and photos taken before launch revealed this issue and resulted in nothing being done should rise serious questions at Boeing about its quality control processes. Based on the press telecon, however, it does not appear that Boeing is asking those questions. From a different report:

[John Mulholland, Boeing’s Starliner program manager] praised the rigging team, saying “even before we got eyes on the hardware, that team on their own initiative (was) reviewing the close-out photos and the processes, and they identified the potential issue that was subsequently validated by hardware inspection.”

“Most importantly, they raised their hand and and let us know what they believe the problem was,” he said. “It’s really a testament to the transparency of that team. The speak-up culture that we have, that is what we need on this program.”

While it is good that the rigging team was willing to speak up afterward, it is very bad that their procedures allowed the launch to go forward. The company says it has now changed its rigging procedures, but I don’t sense any effort on Boeing’s part to find out why its so-called “speak-up culture” failed to have these engineers speak-up, before launch.

5 comments

New video of Starliner pad abort test

Boeing has released a new video of the Starliner pad abort test on November 4th, showing the full flight.

I have embedded the footage below the fold. The one aspect of this test that I have as yet not seen any explanation for is the red cloud to the left of the capsule’s touch down spot. It surely looks like the kind of smoke one sees from the release of certain toxic fuels. It was also something that the live stream video focused on, suggesting the possibility that its existence was important and needed to be recorded for engineering reasons.

Regardless, the fact that any onboard astronauts would have been safely returned to Earth, based on this test, should mean Boeing’s abort system is functioning properly. They note that they have pinpointed the reason one parachute did not deploy (“attributed to the lack of a secure connection between the pilot chute and one of the main chutes”), a problem that is probably quite simple to fix. Hopefully that one failure will not cause any significant delays in their future flights, including the first manned flight next year.
» Read more

9 comments

Boeing & NASA declare pad abort test a success

According to the NASA press release for yesterday pad abort test of Boeing’s Starliner capsule, the test was a success even though one of three main parachutes did not deploy successfully.

A pitcharound maneuver rotated the spacecraft into position for landing as it neared its peak altitude of approximately 4,500 feet. Two of three Starliner’s main parachutes deployed just under half a minute into the test, and the service module separated from the crew module a few seconds later. Although designed with three parachutes, two opening successfully is acceptable for the test parameters and crew safety. After one minute, the heat shield was released and airbags inflated, and the Starliner eased to the ground beneath its parachutes.

All reports say that this parachute issue will not effect the December 17 planned launch of the first unmanned orbital flight to ISS.

I find NASA’s reaction to this anomaly fascinating. Previously the agency repeatedly made a very big deal about the slightest anomaly by both Boeing and SpaceX on any test or procedure. While the agency’s response to these problems could have been reasonably justified, the caution it sometimes exhibited, often causing significant delays that might have been avoidable, was somewhat disturbing, especially when contrasted with the agency’s willingness to accept far more serious issues in connection with SLS and Orion.

Now however, the agency has no problem with the failure of one parachute to deploy during this test. While I actually agree with this response, the contrast is interesting and suggests to me that politics and deadlines (with the Russian Soyuz contract running out) are finally exerting some influence over NASA’s safety people. I suspect it has been made clear to them that unless something really seriously goes wrong, as long as the tests would have resulted in living astronauts, the safety bureaucrats had better not stand in the way of progress.

If so, this is very good news. It means that, assuming nothing really goes wrong with the remaining tests, the first manned missions are finally going to occur next year, relatively early in the year.

Posted at the Hayabusa-2/OSIRIS-REx asteroid conference in Tucson this week.

8 comments

Boeing completes Starliner pad abort test

Embedded below the fold is the video of today’s Starliner pad abort test, cued up to just before launch. While the capsule landed safely, it appears that one of it’s parachutes deploy improperly. If so, this probably means Boeing will not be able to launch the unmanned demo flight to ISS on December 17.

No one during the podcast mentioned this fact, so it could mean that they considered the landing a success regardless. It is even possible that they planned it with only two chutes. Or it could be the corporate culture at Boeing, similar to the culture in the Soviet Union, to avoid mentioning non-obvious problems to the public in order to make believe all is well. We will have to wait and see.

UPDATE: More information here on the failure of one chute:

Video of the test appeared to show all three chutes deploy, but only two remained attached to Starliner – a significant issue that will have to be investigated and evaluated.

Hat tip to reader Col. Beausabre for the link to the video.
» Read more

16 comments
1 9 10 11 12 13 16