June 2, 2023 Zimmerman/Batchelor podcast
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
“The think Boeing is the cat’s pajamas, and so therefore they look at SpaceX more aggressively.”
This was unquestionably true for a long time. But I think there’s been a decisive shift toward a favoring of SpaceX that’s been unfolding over the last few years. (As you pointed out on the other thread, Kathy Lueders played a big role in this.) The more they’ve worked with SpaceX in launching their payloads, and launching crew and cargo to ISS, the more impressed they’ve been. And, conversely, the more frustrated they’ve become with Boeing. It took longer than it should have . . . but the shift is unmistakable.
And I think this is a major reason why SpaceX secured the Starship HLS contract. Yes, SpaceX had the low bid, but deep down, NASA management had decided they were most confident in SpaceX actually executing on it. At this point, NASA is using SpaceX to put almost everything they need into space now, from astronauts to PSYCHE to CLPS missions to the Moon.
Between SpaceX and one of the private space stations now in development, there is no need for NASA in Artemis!
Furthermore, it is arguable that a lunar gateway is not even required: Starship(s) could function as gateway and lander, or the gateway concept might be dispensed with altogether if we are using a high-capacity vehicle like Starship.
When is it going to occur to someone that NASA could specialize in promoting and co-ordinating advanced scientific space exploration missions, instruments and techniques, and leave the space vehicles and habitats to private industry?
I will go so far as to say that one of the reasons BO, ULA, Lockheed, and Boeing are losing market share and underperforming is that NASA let them continue to operate under the old model of a government sub-contractor, rather than a space prime contractor and entrepreneur!
Today, 4 June 2023 is the 2/3 century anniversary of Sputnik.
Robert,
In the interview, part 1 at 5:44, you said, “How could they possibly get to this point in the program and discover that those links to the parachute cords are too weak?” The reason is that they trusted the data. Now they are reviewing, and thus questioning, pretty much everything, as they should. They are verifying data, designs, and assemblies to make sure that everything is as good as stated, but they found a problem with the parachute connection test and its data recording, that the data was recorded incorrectly. They also found a tape that could burn under certain conditions. This is their job, because they are there specifically to find these errors, if they exist. They are there to verify that the verification testing truly proves that the spacecraft is good to go. They are there to uncover these kinds of problems, and it is because these kinds of problems can happen that these people are there. The darnedest things not only happen during spaceflight (one Mars probe compressed some of its data in a way that could not be properly decompressed back on Earth), but they can also happen during verification testing. Although I don’t want to speculate, as someone may take me seriously, possibilities abound for the cause of the trouble and why no one noticed until now; could they have had the wrong data logger for the load cell that they used?
This does not make it any less disgraceful, disturbing, or puzzling. Whatever caused the improper recording of the data should have been discovered at the time of the test and preferably before the test began. Attention to detail was lost at that part of the verification program.
The question should not be ‘how did it get to this point in the program before the discovery’ but should be ‘how could they not have discovered it at the time of the test?’ It may seem like the same question, but if it isn’t discovered by the time the test data is accepted, then no one will question the data until this point in the program, during a detailed review, when virtually everything is scrutinized. It turned out that buried somewhere in the paperwork was evidence of a problem with data from a parachute test and evidence that a tape was used that was susceptible to combustion under conditions that exist (or could exist) onboard the spacecraft.
The paperwork contains these kinds of details so that if an incident (accident or disaster) happens later, then there is a record of how the spacecraft (or aircraft) was designed, built, and tested so that the cause and root cause of the problem could be determined during the investigation. In this case, these problems were discovered before any incidents occurred. We should be happy that the process includes this level of review before any incidents. The mishaps during the first unmanned test flight could have been avoided had they done this level of review with the software before the flight. Instead, that level of review came afterward, after they discovered an error that harmed the mission.
If this was a customer (NASA) induced review or audit, then we should wonder what would assure that these kinds of reviews would continue to happen when commercial companies do this work independent of NASA, when space vehicles and habitats are left to private industry to design, build, and operate. Would these companies be as cheap as Boeing, or would they take safety as seriously as the airlines did in the 1980s, when they decided to implement safety programs that have kept U.S. airlines from killing more than one passenger in the past two decades? We already have space companies whose customer base is not as much NASA but is more the general public. Blue Origin’s New Shepard, Virgin Galactic’s SpaceShipTwo, and World View’s balloons are examples. Have they been similarly reviewed for unexpected problems, such as flammable tape or understrength parachute connections? This may be one of the downsides to Robert’s recommendations in Capitalism in Space.
Apparently, this level of review did not happen before the unmanned tests but is happening now because lives are on the line during the next flight test. As with Dragon, they will certify flightworthinesss (which appears to not be a word) after the first manned test flight.
Such detailed reviews are why certification takes so long, and why it took so long to certify Crew Dragon. Many people were impatient for its certification, at the time, because they saw a successful Dragon mission and assumed all must be well if the mission succeeded. In this case, the parachute problem would not be revealed during a successful flight but could only be found if a parachute failed, and I am not sure that it is definite that one or more of the links would fail; it may be only a possibility of happening. My recollection is that Starliner underwent a test with one parachute intentionally unopened just to test that configuration. Likewise, without such a detailed review, the tape problem would not be revealed unless a fire broke out aboard the spacecraft.
________________
Ray Van Dune wrote: “[I]t is arguable that a lunar gateway is not even required: Starship(s) could function as gateway and lander, or the gateway concept might be dispensed with altogether if we are using a high-capacity vehicle like Starship.
“When is it going to occur to someone that NASA could specialize in promoting and co-ordinating advanced scientific space exploration missions, instruments and techniques, and leave the space vehicles and habitats to private industry?”
The problem with NASA depending upon private industry is that these companies do not yet have operational hardware to launch to the Moon or land on the Moon, so it is unreasonable for NASA to bank on untried future hardware. Who knows whether their designs will work or whether the companies can stay in business long enough to deliver their services to NASA.
Wait.
NASA didn’t have an operational moon rocket until Orion/SLS flew last December, so until that time NASA was banking on untried future hardware. They are still banking on unbuilt hardware for the Gateway lunar orbital station, and for landing on the Moon they are banking on Blue Origin’s unbuilt lunar lander or SpaceX’s Starship, which cannot even make it to orbit. Maybe it is reasonable, after all, for NASA to trust untried, unflown, unbuilt future hardware. In addition, some of the newer companies (e.g. Orbital Sciences, Rocket Lab, SpaceX) have shown performance capabilities that the heritage companies (e.g. Boeing) don’t have and the eagerness to try new methods, even impossible methods.
Heck, When Kennedy promised us the Moon, six decades ago, we didn’t even have any of the basic capabilities for space maneuvering, much less the designs for the spacecraft that would execute those maneuvers to get to and land on the Moon; methods and materials needed were unknown, and it was unknown whether they could even exist.
Since it is OK to depend so entirely on untried hardware, methods, and techniques, maybe NASA would be better off abandoning the expensive and infrequent SLS in favor of commercial solutions to lunar travel.
NASA has long been known for pushing the boundaries of what is possible. When was it that it lost this can-do attitude? SpaceX took “can-do” as their own, ready to publicly fail in one of their development tests, during which they have exhibited that the impossible is practical after all. Blue Origin and Rocket Lab are working on their own versions of reusable launch vehicles. The reusable single stage to orbit fad of a quarter century ago has been shown to be a fad, but reusability is still the goal.
“I will go so far as to say that one of the reasons BO, ULA, Lockheed, and Boeing are losing market share and underperforming is that NASA let them continue to operate under the old model of a government sub-contractor, rather than a space prime contractor and entrepreneur!”
Excellent point. It appears that Robert’s policy paper, Capitalism in Space, recommends a method of procurement that is vastly superior to the old ways of doing things. Rather than purchase the hardware of rockets, spacecraft, and space stations, government should purchase rides or rent time on each of these.
Robert concentrated on cost savings and a bit on better availability, which would result in service to a wider customer base, not just NASA but other commercial customers and other countries’ companies and space programs. He also emphasized competition encouraging more frequent creation of better innovations.
One thing that Robert may have missed was the possibility of a wider variety of solutions and products. More providers means a greater variety. When all we have is NASA’s SLS, all we get is what NASA has. If we have SpaceX’s Falcons, Relativity’s launch vehicle, and Rocket Lab’s Electron, too, then we have a variety of five launchers, not just one. As more launchers come online (e.g. Vulcan and New Glenn), we have a wider variety from which to choose, and we can choose the one that best meets our needs for each mission.
____________________
pzatchok,
You wrote: “If Boeing was using a single engineer to approve the tape/wiring harness specs then they are idiots. On a project this size you would think it would at least be a two or even three man team who finally approves all material purchases.”
I once had a quality engineer (QE) who annoyingly questioned everything. Drawing checkers do the same thing. It was a pain to remember why we did things a non-standard way or to remember how the design fit into the next assembly in a unique way. I had to justify methods or designs months after a meeting in which the solution seemed so obvious that I sometimes couldn’t even remember that there was a problem to solve — we once needed an electronics box to have a conductive external surface, which is unusual, and it took me a minute to remember the brief conversation in which the other engineer (he could have been a systems engineer, or maybe he was the electrical engineer responsible for the inner workings of the box) explained to me why, but I had responsibility for the drawing, so I had to explain it to the QE. So, yes, several of us got together to figure out how to do or make something, and at least one independent set of eyes would check our work.
Again with the speculation: it is possible that the tape was acceptable when the harnesses were designed but a design change in one or more areas in the capsule created conditions that could be potential problems with that particular tape. Did anyone check the harness before making or approving that change? This is why it is nice to have a systems engineer on a project, as his job is to know and check these details. Did Boeing spring for the cost of a systems engineer or were they too cheap for that? This is why the management team is so important to a project.
Design changes that seem innocuous can end up being a nightmare. If not caught soon enough, the engineers end up having to “correct” the part or harness that was not the cause of the nightmare, because the changed part is too difficult or expensive to redesign yet again and the cable can just be wrapped locally with a protective cover.
Also:
Thanks for the link to the weird planes. It brings back memories.
”In this case, the parachute problem would not be revealed during a successful flight but could only be found if a parachute failed, and I am not sure that it is definite that one or more of the links would fail; it may be only a possibility of happening. My recollection is that Starliner underwent a test with one parachute intentionally unopened just to test that configuration.”
Close, but not exactly correct. During the pad abort test one of the parachutes remained *un*intentionally unopened. The links did not fail, so you are correct that a single parachute failure does not guarantee link failure. It would take other failures in addition to a single parachute failure. Nonetheless it still needs to be fixed before CFT, as I’m sure you’d agree.
But this highlights the limitation of relying on testing to find these kind of errors. Testing is good, but once you’ve eliminated the single-point failures, testing for all of the multi-point failures becomes near impossible. The vehicles are so complex that the number of permutations and combinations of failures you’d have to test to achieve even two 9’s of safety (99% safe) is too high to achieve in one lifetime. You need a detailed review by experienced engineers. If Boeing had relied on the parachute-out pad abort test, they would have missed this particular combination of failures.
My hunch is that the tape issue will be similar. Starliner has flown twice now and has been powered up on the ground many, many times without bursting into flames. So I suspect that this issue takes multiple failures to cause a fire. I suspect that when the details are released (if they ever are), we’ll find that it takes something like one particular short in one particular avionics box that sends high current down a low-current wire bundle in such a way as to not trip the circuit breaker while the vehicle is in one particular thermal state before the tape issue presents itself.
I don’t know that to be the case — maybe it is something forehead-slapping bad like the software problems on OFT-1 — but my hunch is that it will be something more subtle than that. We just have to wait and see.
mkent,
“as I’m sure you’d agree.”
Yes, I do. Factors of safety (or margins of safety, if you prefer) must be assured.
“ But this highlights the limitation of relying on testing to find these kind of errors.”
Verification testing only seeks to verify that the system works as intended. The verification test on the parachute link did not do the job it was supposed to do. It did not successfully assure the factor of safety, because they didn’t realize the data was recorded incorrectly.
“You need a detailed review by experienced engineers.”
As I thought I emphatically emphasized, that is why they were there doing the review.
“ So I suspect that this issue takes multiple failures to cause a fire.”
Which is why they are closing that last hole in the cheese, if we use the airlines’ Swiss cheese model of accident/error prevention.
In the case of the parachute link, one hole in the cheese was designing with the wrong material or using the right material in an incorrect way for the needed strength. Another hole was the error in the data collection, with another hole being not catching the error. The review is where the slice of cheese did not have a lined-up hole, and a possible future disaster was averted now. This part of the safety system is working, catching some errors that made it through design and test. However, we wish that the errors had been caught earlier, or better yet that the errors had not been made in the first place.
The problem is not that Boeing is fixing these. The problem is that they didn’t catch these problems at the proper times, when the fix was a slight change to the drawings.
Back when I was building and testing satellites, I used to say to my technicians, “everyone makes mistakes, and you are allowed one mistake a day. Don’t waste it on something small.” Then someone at another company dropped a satellite onto the floor, and I changed my aphorism.
When is the best time to find a problem? A saying I had when I was in design:
A change to the layout costs $10.
A change to the drawing costs $100
A change to the released drawing costs $1,000
A change to the manufactured part costs $10,000
A change to the part on the assembly costs $100,000
A change to the part installed on the spacecraft costs $1,000,000
A change to the part once the spacecraft is ready for launch costs $10,000,000
A change to the part once the spacecraft is launched costs $100,000,000, the cost of the spacecraft.