NASA/Boeing: Cause of Starliner thruster failure identified
According to NASA and Boeing officials, ground static fire engine tests have now identified the likely cause of the thruster failures on the Starliner capsule during its docking to ISS in early June, and puts them in a position next week to determine a return date for the capsule and its two astronauts.
It appears the problem is related to teflon seals in the thrusters, detected while engineers did a series of tests on the ground with another Starliner capsule. Based on this information, Boeing thinks it can fix the problem on future capsules, while also insuring there will be no problems returning the astronauts from ISS.
The thrusters in question are all attitude thrusters, where there is a lot of redundancy and the issue has been seen to be well controlled from the start. The larger thrusters used for the undocking and de-orbit burn have been tested as well, and have not shown any similar issues at all.
The ground tests have also identified the cause of the helium leaks within the capsule engine system. Boeing will use this data to fix later capsules as well. These leaks are not a concern for the return to Earth.
The plan now is to do in the next few days one more set of static fire tests with the capsule docked on ISS, doing short bursts with all the attitude thrusters to further confirm what has been learned on the ground. If that goes as expected, a final meeting next week will determine the return date for the capsule and crew.
Readers!
Please consider supporting my work here at Behind the Black. Your support allows me the freedom and ability to analyze objectively the ongoing renaissance in space, as well as the cultural changes -- for good or ill -- that are happening across America. Fourteen years ago I wrote that SLS and Orion were a bad ideas, a waste of money, would be years behind schedule, and better replaced by commercial private enterprise. Only now does it appear that Washington might finally recognize this reality.
In 2020 when the world panicked over COVID I wrote that the panic was unnecessary, that the virus was apparently simply a variation of the flu, that masks were not simply pointless but if worn incorrectly were a health threat, that the lockdowns were a disaster and did nothing to stop the spread of COVID. Only in the past year have some of our so-called experts in the health field have begun to recognize these facts.
Your help allows me to do this kind of intelligent analysis. I take no advertising or sponsors, so my reporting isn't influenced by donations by established space or drug companies. Instead, I rely entirely on donations and subscriptions from my readers, which gives me the freedom to write what I think, unencumbered by outside influences.
You can support me either by giving a one-time contribution or a regular subscription. There are four ways of doing so:
1. Zelle: This is the only internet method that charges no fees. All you have to do is use the Zelle link at your internet bank and give my name and email address (zimmerman at nasw dot org). What you donate is what I get.
2. Patreon: Go to my website there and pick one of five monthly subscription amounts, or by making a one-time donation.
3. A Paypal Donation or subscription:
4. Donate by check, payable to Robert Zimmerman and mailed to
Behind The Black
c/o Robert Zimmerman
P.O.Box 1262
Cortaro, AZ 85652
You can also support me by buying one of my books, as noted in the boxes interspersed throughout the webpage or shown in the menu above.
According to NASA and Boeing officials, ground static fire engine tests have now identified the likely cause of the thruster failures on the Starliner capsule during its docking to ISS in early June, and puts them in a position next week to determine a return date for the capsule and its two astronauts.
It appears the problem is related to teflon seals in the thrusters, detected while engineers did a series of tests on the ground with another Starliner capsule. Based on this information, Boeing thinks it can fix the problem on future capsules, while also insuring there will be no problems returning the astronauts from ISS.
The thrusters in question are all attitude thrusters, where there is a lot of redundancy and the issue has been seen to be well controlled from the start. The larger thrusters used for the undocking and de-orbit burn have been tested as well, and have not shown any similar issues at all.
The ground tests have also identified the cause of the helium leaks within the capsule engine system. Boeing will use this data to fix later capsules as well. These leaks are not a concern for the return to Earth.
The plan now is to do in the next few days one more set of static fire tests with the capsule docked on ISS, doing short bursts with all the attitude thrusters to further confirm what has been learned on the ground. If that goes as expected, a final meeting next week will determine the return date for the capsule and crew.
Readers!
Please consider supporting my work here at Behind the Black. Your support allows me the freedom and ability to analyze objectively the ongoing renaissance in space, as well as the cultural changes -- for good or ill -- that are happening across America. Fourteen years ago I wrote that SLS and Orion were a bad ideas, a waste of money, would be years behind schedule, and better replaced by commercial private enterprise. Only now does it appear that Washington might finally recognize this reality.
In 2020 when the world panicked over COVID I wrote that the panic was unnecessary, that the virus was apparently simply a variation of the flu, that masks were not simply pointless but if worn incorrectly were a health threat, that the lockdowns were a disaster and did nothing to stop the spread of COVID. Only in the past year have some of our so-called experts in the health field have begun to recognize these facts.
Your help allows me to do this kind of intelligent analysis. I take no advertising or sponsors, so my reporting isn't influenced by donations by established space or drug companies. Instead, I rely entirely on donations and subscriptions from my readers, which gives me the freedom to write what I think, unencumbered by outside influences.
You can support me either by giving a one-time contribution or a regular subscription. There are four ways of doing so:
1. Zelle: This is the only internet method that charges no fees. All you have to do is use the Zelle link at your internet bank and give my name and email address (zimmerman at nasw dot org). What you donate is what I get.
2. Patreon: Go to my website there and pick one of five monthly subscription amounts, or by making a one-time donation.
3. A Paypal Donation or subscription:
4. Donate by check, payable to Robert Zimmerman and mailed to
Behind The Black
c/o Robert Zimmerman
P.O.Box 1262
Cortaro, AZ 85652
You can also support me by buying one of my books, as noted in the boxes interspersed throughout the webpage or shown in the menu above.
Didn’t they do the same testing long before the launch?
You would think either the problems would have cropped up then or this is a time thing. The seals can not sit compressed for an extended amount of time and still be expected to work properly in the cold of space.
Behind the scenes, NASA appears to be not quite so confident as they made out in the press conference that they believe it is safe to return Butch and Suni in the Calypso. Eric Berger tweeted this out this afternoon:
“It’s clear NASA does not want to deviate from its base plan of using Starliner to come home, and this remains most likely. But it is not certain. SpaceX and NASA have been quietly studying launching Crew-9 two astronauts. Suits are available for Butch and Suni.”
https://x.com/SciGuySpace/status/1816501562388390335
Some additional exchanges of note under that tweet:
Eric is asked: “Are you sure they’re doing that study? Like – really, really sure. Good sources?” Eric replies: “I would not have written that if I was not sure.”
Eric is asked: “?(with) two astronauts? Also, where are we on SpaceX return to flight w/Falcon 9?” Eric replies with more details: “I don’t know the precise arrangement for coming back, but launching Crew-9 with two would allow NASA to bring back all of Crew-8, Crew-9, and Butch and Suni on two Dragons. SpaceX is ready to return to flight; hot fired last night. Waiting on regulatory approval.”
And then, to followup, Berger slipped a comment into the thread on the SpaceXLounge subreddit, which sounds slightly more ominous: “For a long time I was super-skeptical that any other option was being considered other than Starliner to return Butch and Suni. But in recent days I’ve been hearing from more and more people that Dragon is being actively worked as a backup option. For obvious reasons NASA does not want to do this, because it probably kills the Starliner program and any chance of a second crew transport system. Anyway, at this point I am about 80-20 in favor of the crew coming back on Starliner. But it’s definitely worth watching whereas a couple of weeks ago I would have dismissed the possibility.”
https://www.reddit.com/r/SpaceXLounge/comments/1ec0vwd/comment/lexlkkr/
So it is more likely than not that Butch and Suni will come back on the Starliner. But it is not guaranteed. And NASA and SpaceX have active efforts underway for a contingency to return them on a Crew Dragon – most likely, the Crew Dragon for Crew 9 – while Calypso is remotely piloted uncrewed back to Earth. That would mean that they would officially be added to the next ISS Expedition, and see their stay on ISS last an additional 6 months.
Richard M: All this appears to be good-sense due diligence on NASA’s part. However, based on the technical information released by Boeing, it seems more than 95% to me that the crew should return on that capsule.
If NASA decides otherwise it will as you say, destroy Starliner as a viable alternative to Dragon, and cause it years of other problems. Only yesterday Berger was writing about the problems of only having SpaceX as a provider, and NASA’s concern of that fact. Do you really think it will now act to destroy its one manned alternative to ISS?
Hello Robert,
Oh, to quote Fox News, “I report, you decide.” I know nothing beyond what Berger is reporting – and come to that, I have none of his sources! I thought I would just pass it along, because he *does* have good sources – and until now, he has been, as he says, a skeptic of any notion that Butch and Suni would not return on Starliner.
Again, the greater likelihood is that they *will* return on Starliner. But yes, given that doing otherwise has a good chance of nuking Starliner as part of the Commercial Crew program (and thus any last hope, as you rightly say, of a U.S. crew backup capability for the time being), it would not be a decision NASA would take lightly. That is to say, if they did so, I would have to think that they had developed a serious case that it just was not very safe to bring them back on Calypso.
Stay tuned.
Richard M: Just so there is no misunderstanding, I meant no criticism of you for reporting Berger’s reporting.
In the past if we only had one launch provider no one cared. We didn’t have the ISS to worry about.
We really only cared about having just one launch provider after the Soviets started to work on the ISS also. HASA told everyone that the Russians being on the ISS was a safety multiplier since we could ride home on their ship if need be.(they forgot to say our suits didn’t fit their ships.)
Then the Shuttle failed and all talk of a second launch provider went awy for years and years,
Now we do have an American launch provider and the second launch provider is back. But they forget that Space X has two launch points and can provide multiple launches a week. They also have more than one manned ship.
All we have to do is ask Space X to always have a man rated capsule and trunk ready to launch. They could interrupt their schedule and have it in space inside a week.
You do not need a second launch provider you need a second launch vehicle.
I would like to test this at least once.
pzatchok asked: “Didn’t they do the same testing long before the launch?”
They did testing, but it was more of an acceptance and qualification testing.
The testing they have done recently was to investigate the problem that arose since then, that was not seen during the previous test regimens. These tests are different in that they seek specific knowledge from a known problem, not doing generic testing to see whether any problems develop. As pzatchok noted, conditions are different on orbit than on the ground. These conditions are simulated on the ground as best as possible or practical, but they are not exact.
Once a problem occurs, the test engineers know more about the suspect conditions that may be causing the problem. The testing is to verify whether the engineers understand the phenomenon and the conditions. Once understood, either the design can be changed for future hardware or the methods (e.g. the software or the procedures) can be changed to avoid the specific conditions.
One of the purposes of the thermal vacuum test that a spacecraft goes through is to verify the thermal model and to make any necessary changes. In the linked video, Mark Nappi, from Boeing, gave a review from the previous briefing, a few weeks ago, and gave an update for what is known today as a result of the last few weeks of testing. One of the things he mentioned was the likelihood of modifying their thermal model based upon actual flight experience.
“The seals can not sit compressed for an extended amount of time and still be expected to work properly in the cold of space.”
Teflon can be a tricky sealing material, but it is also very useful. It has a tendency to “mush” under pressure, or to slowly flow away from the pressure. This is useful in your household plumbing to prevent leaks at certain threaded connections, but it makes for useless washers under screw heads; as the screw that you tightened today soon becomes loose. It is clear to me that compromises are made when using teflon in these seals, as they have seen some degradation after long-term exposure to the propellants and they discovered some seal bulging that sounds consistent with that “mushing” tendency.
Material selection is very important, but there are always compromises in the choice. You can get a very good seal with a metal seal, and helium will not leak, but that seal works only the first time, so for valves, metal seals are not quite as good as rubber seals for repeated uses. But rubber isn’t as good at low temperatures, and helium is atomic and permeates the rubber and leaks right through the material. You have to find the right compromise that works for your use of the valve and for the hot thruster on the other side of the valve. When they say it is rocket science, it really is rocket science. Rockets don’t let anything be easy — even small thrusters.
Richard M wrote: “And NASA and SpaceX have active efforts underway for a contingency to return them on a Crew Dragon”
Eirc Berger may have been hearing rumors that were generated from the contingency planning. In the game of telephone, one person’s statement can change and twist into something very different. Last week, I was the victim of such mistranslation when I told one person that I hoped a meeting would happen sooner rather than later, which then was quickly rumored to be set for a specific date, and when no time could be learned through the grapevine, people started coming to me to learn when and where they should go for the meeting. If I had been a news reporter, I would have thought I had multiple reliable sources confirming the meeting and its date, just not the time or place. (By the way: frustratingly, that meeting still has not been set.)
From what I heard in that briefing, I would be shocked if anyone involved currently thinks that the spacecraft could not safely perform its return function. Mark Nappi had mentioned that in the previous briefing they didn’t yet have a root cause of the problem, but they now think they are much closer to understanding their system and how to manage it from now on. Their next thruster tests, in a few days, are intended to help verify their understanding of the system.
Hello Edward,
1. Berger’s tweet has caught a lot of attention – it is up to 100K views now – and some flack, too. Objections seem to fall into two categories: a) “Berger is a pro-SpaceX hack and this is more of the same.” b) Berger is either misunderstanding an earlier study contract done with SpaceX that NASA has already clarified does not have any relevance to a “rescue” mission, or simply exaggerating normal contingency planning. I won’t bother with dealing with (a), but I think Eric is informed enough to not misconstrue source reports in the way suggested by (b). It seems hard to avoid the conclusion that NASA is working with SpaceX on something more than just the usual contingency planning.
Indeed, pressed on this point that his report runs contrary to any sense of what Nappi and Stich were saying in the press conference, Berger replied: “Cool. I do have sources other than the official press conference.”
I still default to an expectation that they will run these final thruster tests, and Butch and Suni will come home on Starliner. (I also hope that this what happens – safely!) But I do think we should no longer think of a Dragon “rescue” as a purely theoretical possibility.
2. It astonishes me that Boeing never did integrated testing of the thruster doghouses. After the disasters with the software on OFT-1, you would think Boeing would have learned its lessons about the risks of failing to do integrated testing.
Not flack, but FLAK (from the German “Flug Abwehr Kanone ” loosely translated as “Antiaircraft Gun”. It replaced the Great War “Archie” as RAF slang for AA fire and US airmen adopted it from them. The term FLAK applied both to the weapons and the branch of the Luftwaffe that operated them). https://nationalinterest.org/blog/reboot/how-nazi-germany-introduced-flak-military-lexicon-184347
Why not let the two STRANDED astronauts decide : Come home on the BOEING capsule or hitch a ride on a SpaceX craft??
I know which one I’d take – save me a seat Mr. Musk!!!
pzatchok wrote: “But they forget that Space X has two launch points and can provide multiple launches a week. They also have more than one manned ship. All we have to do is ask Space X to always have a man rated capsule and trunk ready to launch. They could interrupt their schedule and have it in space inside a week. You do not need a second launch provider you need a second launch vehicle.”
That works OK until the launch vehicle’s upper stage malfunctions and is out of service for a few weeks or months while the investigation and corrective actions take place. America was grounded for a year and a half after each tragedy with Apollo and the Shuttle. NASA was lucky with Apollo 13.
The reason for the second launch provider and second launch vehicle is to cover the times when one is not available, especially for safety concerns. The Air Force desired two different rockets for similar reasons.
Once a third manned spacecraft becomes available, manned Dream Chaser, for instance, we should start to see some real price and service competition.
_____________________
Richard M,
You wrote: “2. It astonishes me that Boeing never did integrated testing of the thruster doghouses. After the disasters with the software on OFT-1, you would think Boeing would have learned its lessons about the risks of failing to do integrated testing.”
There is a long history of companies that should have known better not performing integrated testing. This is what happened to Mars Polar Lander in 1998. The design was for a switch to turn off the landing engine when the legs splayed out on touchdown. Simple enough, but during the test of the legs opening in preparation of touchdown, the rest of the craft was not assembled to the leg assembly, so they never discovered that when the legs snapped open during deployment, before they returned to the normal open position they overextended by an amount that would trigger the cutoff switch. Thus, on landing on Mars, the engine shut down when the legs deployed, too high over the landing spot.
This from a company whose test engineering motto is “test as you fly; fly as you test.” This is grammatically correct, but the correct translation is: ‘test like you fly; fly like you test.’
The Hubble Space Telescope’s optical problem would have been discovered by a similar integrated test, but that cost too much, so they relied instead on a tool that turned out to be used improperly. Robert writes about it in his book The Universe in a Mirror.
Come to think of it, Hubble and Mars Polar Explorer were both made by the same company.
Integrated testing can be expensive and time consuming, but sometimes they find fatal flaws.
Being astonished is a perfectly rational and appropriate reaction. “What were they thinking?” is another good one, because how much could that doghouse test really cost? Instead, they relied upon a model. What were they thinking?
Unless they were lying during the briefing, I still would be shocked if anyone involved currently thinks that the spacecraft could not safely perform its return function. They thought they could use it safely before they understood the problem, because they have redundancy in the attitude control system.