SpaceX announces completion of investigation of July 11 Falcon 9 second stage failure
SpaceX today issued an update outlining completion of its investigation of July 11 Falcon 9 second stage failure, noting that it had located the problem and instituted fixes throughout its fleet.
It turns out the upper stage was not destroyed when it restarted its engine.
During the first burn of Falcon 9’s second stage engine, a liquid oxygen leak developed within the insulation around the upper stage engine. The cause of the leak was identified as a crack in a sense line for a pressure sensor attached to the vehicle’s oxygen system. This line cracked due to fatigue caused by high loading from engine vibration and looseness in the clamp that normally constrains the line. Despite the leak, the second stage engine continued to operate through the duration of its first burn, and completed its engine shutdown, where it entered the coast phase of the mission in the intended elliptical parking orbit.
A second burn of the upper stage engine was planned to circularize the orbit ahead of satellite deployment. However, the liquid oxygen leak on the upper stage led to the excessive cooling of engine components, most importantly those associated with delivery of ignition fluid to the engine. As a result, the engine experienced a hard start rather than a controlled burn, which damaged the engine hardware and caused the upper stage to subsequently lose attitude control. Even so, the second stage continued to operate as designed, deploying the Starlink satellites and successfully completing stage passivation, a process of venting down stored energy on the stage, which occurs at the conclusion of every Falcon mission.
For near term upcoming flights, they have removed the sense line and the sensor, stating that neither is essential and can be covered by other sensors.
This SpaceX update implies that the rocket can now resume launches, but I have yet to see a confirmation from the FAA. Since July 19th SpaceX has scheduled launches to begin the next day, and with each day shifted that schedule back 24 hours. Right now it has a Starlink satellite launch scheduled for tomorrow at 9:21 pm from Florida. We will have to wait to see if it gets an official okay from the FAA before then.
UPDATE: The FAA has approved SpaceX to resume launches. Apparently it accepted SpaceX’s request that because no public safety issues remain, it can allow launches to resume even though the FAA has not yet retyped SpaceX’s report to make it sound as if the FAA has concluded its own investigation.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The print edition can be purchased at Amazon. from any other book seller, or direct from my ebook publisher, ebookit.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
SpaceX today issued an update outlining completion of its investigation of July 11 Falcon 9 second stage failure, noting that it had located the problem and instituted fixes throughout its fleet.
It turns out the upper stage was not destroyed when it restarted its engine.
During the first burn of Falcon 9’s second stage engine, a liquid oxygen leak developed within the insulation around the upper stage engine. The cause of the leak was identified as a crack in a sense line for a pressure sensor attached to the vehicle’s oxygen system. This line cracked due to fatigue caused by high loading from engine vibration and looseness in the clamp that normally constrains the line. Despite the leak, the second stage engine continued to operate through the duration of its first burn, and completed its engine shutdown, where it entered the coast phase of the mission in the intended elliptical parking orbit.
A second burn of the upper stage engine was planned to circularize the orbit ahead of satellite deployment. However, the liquid oxygen leak on the upper stage led to the excessive cooling of engine components, most importantly those associated with delivery of ignition fluid to the engine. As a result, the engine experienced a hard start rather than a controlled burn, which damaged the engine hardware and caused the upper stage to subsequently lose attitude control. Even so, the second stage continued to operate as designed, deploying the Starlink satellites and successfully completing stage passivation, a process of venting down stored energy on the stage, which occurs at the conclusion of every Falcon mission.
For near term upcoming flights, they have removed the sense line and the sensor, stating that neither is essential and can be covered by other sensors.
This SpaceX update implies that the rocket can now resume launches, but I have yet to see a confirmation from the FAA. Since July 19th SpaceX has scheduled launches to begin the next day, and with each day shifted that schedule back 24 hours. Right now it has a Starlink satellite launch scheduled for tomorrow at 9:21 pm from Florida. We will have to wait to see if it gets an official okay from the FAA before then.
UPDATE: The FAA has approved SpaceX to resume launches. Apparently it accepted SpaceX’s request that because no public safety issues remain, it can allow launches to resume even though the FAA has not yet retyped SpaceX’s report to make it sound as if the FAA has concluded its own investigation.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The print edition can be purchased at Amazon. from any other book seller, or direct from my ebook publisher, ebookit. The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
Appears this is from FAA.
https://x.com/spaceflightnow/status/1816623223330549928?s=46
Gary: Thank you. I have amended the post to include this good news.
It seems that this was a problem that developed during assembly, when a clamp was installed looser than is acceptable. Solution: Remove the redundant line. The best part is no part. SpaceX is half a dozen launches behind, because they did not have the best part.
From the SpaceX link:
SpaceX, in its early years, developed a culture of continual improvement of its launch vehicle. Every launch of every vehicle around the world generates an enormous amount of performance data that can be used to improve performance, safety, and reliability. This statement from SpaceX suggests that this culture may still continue at SpaceX.
NASA, however, did not do this in at least two important areas. Rather than correcting a deviation of O-ring performance on the Space Shuttle, they normalized its deviation (“it’s OK as is”) and relied upon Thiokol to make sure the situation remained safe. Rather than correcting chunks of insulation coming off the External Tank during launch, they again decided it was OK as is.
Is SpaceX continually making minor, unsung changes to their hardware before problems develop? Did the company learn from NASA’s mistakes?
SpaceX: after 350 consecutive perfect flights, we had a second stage failure that resulted in partial mission failure.
FAA: OMG! Stop everything, ground the vehicle, we need a full safety review.
SpaceX: While our investigation is ongoing, we request a waiver to resume flights since there was no safety threat involved in this failure.
FAA:
SpaceX: Ok, our investigation is complete, no big deal, easy fix. About that waiver?
FAA: ……. Ok
BTB Audience: If there was no safety hazard as you just acknowledged, why did you order the vehicle grounded, hmmm?
Once again, Starlink shows its utility to SpaceX in a new way: It provides a way for SpaceX to rapidly recover from a failure by launching its own payloads to demonstrate to outside clients that its rocket is safe to fly on once again, without endangering those outside payloads, or having to fly a useless mass simulator.
Richard M: While making money at the same time.
I don’t doubt their explanation of how the failure occurred, but I am baffled by how this was deduced!
Ray Van Dune – “nothing concentrates the mind like the prospect of a hanging in the morning”.
This is one case where the FAA did its job. Something went wrong, FAA demanded an answer, an answer came forth, FAA checked the maths.
David Eastman asked: “If there was no safety hazard as you just acknowledged, why did [the FAA] order the vehicle grounded, hmmm?”
NASASpaceFlight has a video on the FAA’s role in this kind of investigation:
https://www.youtube.com/watch?v=uU-hKxKUrxY (17 minutes: “Why Can’t Falcon 9 Fly?”)
John Galloway lists several reasons why the FAA would get involved and checks-off which reasons don’t apply, a couple that might, and a couple that he thinks do apply to give the FAA its involvement. He also reminds us, at the end, [Spoiler Alert? Maybe?] of what Robert tells us: The FAA has some limited knowledge of spaceflight problems, but they don’t know the rocket, so SpaceX does the investigation on its own rocket and reports its findings to the FAA.
[Real Spoiler]
Galloway believes that these two reasons for why the FAA would get involved apply to the failed launch:
√ Unplanned permanent loss of the vehicle.
√ Failure to complete a launch or reentry as planned.
I don’t always test my code, but when I do I do it in production.
https://x.com/chrismaddern/status/565925512250662912/photo/1