https://www.todayifoundout.com/index.php/2020/10/the-german-rocket-fighter-that-dissolved-its-pilots-alive/

The hypergolic fuels are all very corrosive. Far Far more than standard aircraft fuels which range from little more than specially formulated gasoline to kerosene and diesel fuel.

These are good questions to ask, and they are questions that rocket engineers have asked. How does New Shepard prevent its thruster injectors from being clogged with dust on landing, or Dragon prevent anything from being corroded by sea water on splashdown? The Russians don’t have to worry about the dust kicked up on Soyuz landings, because they don’t reuse their spacecraft. The Americans didn’t have to worry about sea water on Mercury, Gemini, or Apollo for the same reason.

(Airplanes don’t fly into dust, because that may end the service life of the engine(s). They usually don’t land in the water, because that ends the life of most planes, and can end the life of a seaplane when the waves are too rough.)

But now our spacecraft have to survive contact with the enemy: the outside environment outside of rocket fairings. This requires changes to designs and operations. For example, operational changes to Curiosity have dramatically reduced the wear on its wheels.

Fortunately, there was some experience with the Space Shuttle, but they didn’t have as much dust or sea water to deal with after landing. We did discover, however, that the Shuttle didn’t do well with the stuff that was kicked up from the runway on landing, no matter how much they FOD walked or swept the runway. (A foreign objects and debris (FOD) walk is a line of people walking along a runway or cleanroom looking for the tiniest debris, such as pebbles or dropped screws, zip ties, and cetera. Runway FOD walks were invented long before the Shuttle needed them.) A bad experience with the Space Shuttle also showed us one way in which it can be tricky to design for reusability.

Aircraft are susceptible to humidity, cold, heat, dust, and other factors, which is one of the reasons pilots do a walk around before entering the cockpit. A century of bad experiences has taught us much about the problems that the environment can cause. It isn’t just the environment that can cause problems with reusable flight hardware, it can be wear, methods, distracted pilots, poor maintenance (such as poorly done or not done often enough), ergonomic design, etc. There is a nonfiction book called Fate Is The Hunter and a fictitious movie by the same name that present examples of “learning experiences.”

When you have a problem while driving your car, you pull right over and call for a tow. If you have a problem while flying an aircraft, you have only minutes in which to find a place to safely land (which is trickier at night). If you have a problem while in a spacecraft, you may have to wait hours (or days or months, on a trip to Mars) before you can get to safety. The risks increase but so do the consequences. Unbelievably, we have made flying safer than driving. Making spaceflight safe in reusable spacecraft could take some doing, but we are willing to do what it takes, and our experience in making flying safe will be critical to our success.

Why build indoors? If you see unpainted airplanes, they are usually a color such as green. This is a coating which protects the otherwise bare aluminum from corrosion. Starship development test units are being built “outdoors” because they are made of Stainless Steel, or more generically and less brand-name: corrosion resistant steel. Although they have to worry about bird droppings, they don’t have to worry as much about rust and other corrosions. Due to the low number of times they expect any test unit to fly, SpaceX probably isn’t terribly worried about storing certain pieces of hardware in the open air. There may not be much time for spots of corrosion to spread into real problems.

I would expect them to build their operational flight units indoors, however.

pzatchok asked: “I wonder how many engineers are between the actual assembly workers and the real decision makers? When a problem crops up how many people have to get involved before a corrective action is taken?”

It depends upon how much of a problem it is. One of my technicians once draped a wire over a small thruster on a communication satellite, and he didn’t realize he had done so until after he tugged on the wire. I wrote a non-compliance report to make sure it was inspected — so that a quality engineer would get involved. The propulsion group sent a couple of engineers to inspect the thruster for damage and alignment, but it was OK. The tech had not tugged hard before realizing the error. Including the propulsion group and the people who track open non-compliance reports, there were probably about a dozen people involved, including the supervisors, such as mine, the quality engineer who handled the report, and the propulsion engineers who inspected the thruster. A tiny error probably cost a couple thousand dollars and added a record to the spacecraft file.

Something like Boeing’s two Starliner problems involves a large number of people at many levels of authority and responsibility, both engineering and management, looking at more than just the problem that occurred but also at related items in order to assure all levels that this was not a systemic problem but is localized to the oxidizer valves only. The fault tree that they talked about in the press conference is where they are looking at the item plus related items, and they are looking at a large range of possible causes or contributing factors to the cause of the problem, meaning that a large number of people are investigating these items and the possible contributing factors. There would also be people who had to keep the customer updated on findings and eventual corrective actions. There would be several internal meetings and some customer meetings, and there may even be a customer representative at the internal meetings. These meetings would discuss the possible contributing factors and explain why they are or are not involved in the problem. Some of these factors may include human factors (were errors made), procedures (were they followed, are they correct), design and manufacturing, performance during previous testing, the propellant loading activities are likely being closely scrutinized, the possibility of effects from the storm, assuring that the ground support equipment truly sent the proper commands, the corrosion found during inspection, and other things I haven’t thought of.

When a problem crops up how many people have to get involved before a corrective action is taken?

Add in accountants if they have closed the budget and paid for that subsection/assembly.

It’s a question of incentives.

During the 1960’s, government was driven by its legitimate foreign-policy and defense concerns, so NASA was resourced and enabled by our government to successfully address those concerns with respect to our presence in space. And the personnel within NASA, who shared those same Cold War concerns and were old enough to have a memory of the greater challenge of WWII, responded successfully to the government’s direction. All involved had a legitimate objective: out-do the significant, totalitarian, foreign competition to prevent their dominance … much the same objective Reagan had in the 1980’s regarding both economic and defense policy.

Once we beat the Russians to the moon and met that objective, NASA gradually became just another government agency and conduit for Congressional pork. It lost the incentive to sustain excellence, IMO, just like almost every other government entity does over time.

I was working for a Boeing contractor in the late ’90’s, and this article pretty well explains what happened:

“The 1997 merger that paved the way for the Boeing 737 Max crisis” on qz.com.

Then corporate moved to Chicago in 2001, putting management 1500 miles from the factory floor. The doom-and-gloom predictions of the time have fairly come true.

How does it go wrong, and how does it go this wrong?

How does it go wrong with any company? There once was a time when, the saying went, no one got fired for buying IBM (either the stock or their products), but when was the last time you heard of IBM outside of historical context? How did they go from being the business machine company to being an unknown (are they still in business)? Did anyone else notice that they disappeared from the Earth just as soon as they introduced their version of the personal computer — the version that the whole world was waiting for so that they could buy it and be guaranteed to not get fired?

Something happened at Boeing, because the company that we thought was so careful is having such a terrible time with quality, and the company that builds its test flight units outdoors in an all-out hurry, thus is not careful at all, has so many successes doing things that no one has ever done before. It is the experienced company that is meticulous vs. the new company that innovates in a rush.

When it came to Commercail Crew Program, the tortoise lost to the hare.

In the super heavy launch arena, the same thing is happening, despite the tortoise having a six year head start. Congress has no urgency to launch SLS, despite the 2024 lunar landing goal, but SpaceX has urgency to get Starship to Mars. The hare has a reason to keep going, but the tortoise took a few years not making much progress, having had year-for-year schedule slips. It was during this time of slipping schedules that Starship was first announced to the world. The hare got into the race while the tortoise slept.

William,
You asked: “I still don’t understand how Falcon 9 first stage can fly 10 times (or 100 times) Do they replace every valve and widget? The Boeing commercial fleet has flown for decades in rain, snow, and dust (not volcanic ash) all without catastrophic failure.”

Robert is correct. Blue Origin, Rocket Lab, and SpaceX are working toward building rockets more like aircraft than the expendable items that rockets have been for seventy-five years. This means that instead of being robust for a few tests and a launch, parts have to be robust for thousands of cycles and possibly thousands of hours of flight time.

This may mean that parts that had been made as light as possible now have to be made somewhat heavier, or other changes may be necessary to reduce the effects of wear. The transition from expendable to reusable could be tricky as everyone learns what works in space for that many cycles. We cannot always use aircraft parts for spacecraft. Some parts are designed to work in air, some require gravity (flying a barrel roll with a 707 was proved possible, but some aircraft don’t do well upside down). The Athena had a failure on first flight partly because they used helicopter gyros for upper stage guidance, but in the thin air the gyros suffered from corona discharge.

Elon Musk may be very sensitive to quality control issues. Early in the Tesla production of its first car, there had been a quality problem for the entire fleet. Fortunately, the fleet was around 200 cars, but the recall was relatively expensive for a company that was just starting up.

pzatchok,
Please keep in mind my tale about the connectors becoming contaminated despite being protected with covers inside a cleanroom. The darnedest things happen. Also, the valves use Teflon gaskets, not the usual rubber-substitutes. Teflon is a tricky substance when pressed, because the stuff “mushes” in ways that rubbers don’t. My experience was to not use Teflon as a washer under a screw, for that reason, as the force on the screw will relax and it won’t be as secure in the threads as it should. Boeing said that they expected a small amount of leakage, and that they had believed this would be acceptable.

V-Man wrote: “everyone knows that Boeing would have gotten another check.”

They may not have. Their high price for the contract should anticipate expensive trouble such as this. Because these kinds of things can happen in development, the US government long ago started allowing “cost plus” contracts, which paid even the unexpected costs of development plus a profit. The problem was that this became standard, and it often resulted in incentives to not try hard to complete the development program. Fixed price is riskier for the vendor (Boeing, Northrup Grumman, Sierra Nevada, and SpaceX), but there is incentive for things to work correctly the first time.

As people have noted, it does not always work correctly the first time.

Jeff Wright wrote: “Marshall’s calling was big rocketry, Robert… Keeping things in house works. Area 5I proves that”

SLS proves that this does not work at Marshall, and several other rocket projects in the past half century proves that it doesn’t work anywhere at NASA, anymore. Congress turned NASA from a can-do organization to something else. Congress squanders the knowledge, skills, and talents of its rocket scientists, engineers, and technicians. We did super heavy launch with SLS, and it was designed by Congress but not to do what we need done. The Space Shuttle and ISS turned into disappointments, but SLS was designed to disappoint.

Even if NASA achieves its stretch goal of one launch per year, that is not enough. NASA will have to choose whether to use that launch to take people to a moon base, send a probe to deep space, or put a major piece of hardware into orbit (e.g. a Mars mission module). How long will it take NASA to send men to Mars if it can only construct the transit spacecraft when it doesn’t send men to the moon base?

Why didn’t Marshall develop reusable booster stages? Why didn’t Marshall develop Starship? Why did Orion, which isn’t that much more capable than Dragon or Starliner, take so long and cost so much to develop? The answer is that Congress is not eager to improve its space hardware, and it isn’t eager to get new hardware in operation quickly.

So, no. In house NASA does not work, because Congress won’t let it.

We had expected so much to happen as a result of the Space Shuttle. Space travel would become commonplace, alternate vehicles would be made, civilians would become astronauts, and commercial manufacturing would bring us benefits directly from space, but government chose to go a different way. With the ISS we still don’t have commercial manufacturing. We aren’t getting most of the benefits from space that we had expected and thought our tax dollars were paying for, and with government in charge we still won’t get these benefits twenty years from now. Once Starliner is operational, we will get these benefits twice as fast as we would with Crew Dragon alone. Eventually we will have manned Dream Chaser, and we will really be flying.

This is why so many people are eager for commercial space competition. The competitors will provide what we want to buy, and they will work to do it for less cost. Government has failed to do either of these things. When we let government be in charge, all we get is what government wants. When we are in charge, we get what we want.

Richard M: NASA had ran out of patience with Boeing back in the spring of 2020, when they chose SpaceX to provide cargo to Lunar Gateway. In assessing the four bids, Boeing ranked last, and was given this horrible assessment by NASA:

Of the four contenders, [Boeing] had the lowest overall technical and mission suitability scores. In addition, Boeing’s proposal was characterized as “inaccurate” and possessing no “significant strengths.” Boeing also was cited with a “significant weakness” in its proposal for pushing back on providing its software source code.

Due to its high price and ill-suited proposal for the lunar cargo contract, NASA didn’t even consider the proposal among the final bidders. In his assessment late last year, NASA’s acting chief of human spaceflight, Ken Bowersox, wrote, “Since Boeing’s proposal was the highest priced and the lowest rated under the Mission Suitability factor, while additionally providing a conditional fixed price, I have decided to eliminate Boeing from further award consideration.” [emphasis mine]

The Starliner contract had been awarded several years earlier. I suspect if that contract was offered now, Boeing would not get it.

1) Again, the CCtCap Commercial Crew contract Boeing has with NASA is a fixed cost contract, and it only pays out as milestones and deliverables are met. A fair chunk of that $5.1 billion remains unpaid until Boeing successfully does its flight tests, and then the operational missions stipulated. So: They really do have plenty of incentive to get this thing operational.

2) The more we learn about the valve problem, the more perplexing it becomes – and, probably, in ways that are not as disastrous as the valve problem SpaceX had with Dragon in April 2019. There were no issues with the valves in the failed December 2019 test flight; and a test was done in final prep last month without incident. And yet, somehow, when another check was done the day before launch, 13 of 24 NTO valves would not open. Why? Moisture penetration, says, Boeing; but not from the storm. Odd.

John Bolmer, Boeing VP and Starliner Program Manager said this in the press conference:
“There’s 24 of the oxidizer valves, there’s 24 fuel valves, and then there are 16 helium valves that comprise the system. They are all of the exact same design. The only valves that we saw issues with were with the oxidizer. And that’s due to the phenomena that I described with the permeation of the NTO through that. So we had no issues with any of the fuel valves — the less toxic, corrosive substance. And of course the helium is more inert and we didn’t have any issues with any of the helium valves. Once again, all of the same design. It’s a factor that goes with the oxidizer, and it’s limited to just those oxidizer valves.”

It’s hard to think that some kind of *design* fix isn’t going to be needed to rectify it (or mitigate it sufficiently), once they complete the root cause analysis. Which may be just as well since it’s not apparent it can’t get a launch window until November anyway.

It’s a bad look (and a bad hit to the bottom line, as this rollback and de-stacking will cost the company, not NASA) for Boeing, but perhaps more because they’re *already* so far behind the 8-ball. Were it not for the valve leak problem in the 2018 testing round and the software integration failures that busted the 2019 OFT-1 mission, Starliner would have been flying crew for two years now. This issue may be less culpable when all is said and done, but clearly people are running out of patience with Boeing; even Kathy Lueders was unable to fully hide her frustration.

And the same two questions about those gaskets.
If they knew nitric oxide would form when they were exposed to oxidizer why use them? And exactly how does water get into a sealed closed system inside the rocket and inside a building?

Absolutely, and they did work at the time. They failed a couple of days (weeks?) later. Now, that subsequent failure could have occurred in orbit (even worse, docked at the ISS), hence my question.

(Which no one has answered because we don’t need to – everyone knows that Boeing would have gotten another check. “We need two options! — and never mind that Dreamchaser over there…”)

Jul 15, 2019 https://spaceflightnow.com › spacex…

All check valves of this type were made of titanium and the explosion was a shock to NASA. I believe they are now made with stainless steel. And NASA is requiring all to use it now thanks to SpaceX.

This has all been looked at as Old Space vs New Space…yet both together have been matched by Russia/China…who you might say have together moved into “deep parity.” I don’t know that old or new space can have a victory over the other without it being pyrrhic. I think many here and elsewhere see Musk beating the world, and old space dying off. Or vice versa in Gary Church’s case.

But if I were a chi-com, what would I want to see?

Reagan redistributed more wealth to the third world than any liberal who wanted a modest but honest increase in foreign aid…all by watering down the border and allowing jobs to go overseas…and daring to question others patriotism.

So I want old space and new space to bloody each other…and I’ll let Musk take down my competitor Boeing for me…maybe with a plant of my own to damage valves and make them look even worse. Then…then, when old aerospace jobs are dead I will target Elon. Texas is going purple, and a second coming of Proxmire by way of Austin means “mostly peaceful protests” are headed to Boca Chica. Oh, not now…just wait. I want to see Elon die of a heart attack on the Tesla floor…after we sell enough masks to Americans to make up for the Trump shortfalls…he the only President we didn’t own.

People who want American Old and New Space to work together? They must be silenced! Thankfully the libertarians dominating space websites are doing that for us. They are our best assets and don’t even know it. Sun Rand did more damage to the USA that our little red book ever could. Only dirty hippies who used every four letter word but “work” and “soap” read that. But the Randians get off putting people out of work even more than our Green plants do. We must have the doctors come up with another disease for us. Talk radio didn’t push back as early and as hard on medical restrictions as on environmental ones. Maybe another “Patriot Act” with Fox pushing it after another “Muslim” attack. That always works.

———-signed Xi Jinping

I once again wonder if Boeing has any quality control systems at all. For such a serious problem — the failure of 13 valves out of 24 — to suddenly pop up just hours before launch, when they have been developing this capsule for years, and even had an extra year and a half to check the capsule out after the failures during the first unmanned demo flight in December 2019, is somewhat astonishing, and very disturbing.

My understanding is that the valves worked during spacecraft fueling a few weeks before the launch attempt. Whatever caused them to fail happened since then.

Others will argue that problems like this can always appear unexpectedly in space hardware. I say hogwash. Boeing is not inventing something new with Starliner. This is a capsule, using heritage engineering first invented in the late 1950s. It should not be so hard to get this right.

You’re right. Boeing is not inventing new valve technology for the Starliner. They’re using the same type of valves used in every other manned spacecraft out there, including Dragon and Orion. And like Dragon and Orion, Starliner has environmental seals protecting the propulsion system from humidity before launch. It is currently thought that those seals were damaged by a thunderstorm the day before the launch attempt, though the jury is still out on that. I expect more data will be forthcoming in the coming weeks.

Boeing’s profit motive isn’t the issue, it’s their business plan for getting the profits. Since NASA will continue to pay them while they are “developing” Starliner, they have no real incentive (beyond PR) to actually put anything in space. So they can take a year to replace a power system, or months to check the valves and be sure they won’t lose any money on the deal.

1) What power system are you talking about? 2) They’ve already lost a great deal of money on the deal — $410 million — because NASA is not continuing to pay them. The Commercial Crew contract is firm, fixed-price.

The Boeing commercial fleet has flown for decades in rain, snow, and dust (not volcanic ash) all without catastrophic failure. Military planes and helicopters are also flown in extreme adverse environments.

1) Starliner has not had a catastrophic failure. 2) The Boeing commercial airplanes, military planes, and helicopters don’t use nitrogen tetroxide as oxidizer.

I am curious about the need for environmentally controlled cleanrooms to build reusable space ships. What other transportation system is so fragile, not aircraft, ships, submarines?

Aircraft are built in similar environmentally controlled facilities as Starliner.

The root cause is using “other people’s money” and cost-plus contracts. Boeing Space is a corporate-welfare recipient performing exactly as you would expect a welfare recipient to perform.

Most of Boeing’s major government contracts are firm, fixed-price. They are the only major prime contractor that signs significant firm, fixed-price development contracts.

The rd-180 stockpile is VERY small. As in a handful of flights.

If by “handful” you mean 30, you’re right. If you mean less than 30, you’re wrong, because there are 30 Atlas V flights left on the manifest.

And is we had blue prints for them wouldn’t we already be making our own?

Pratt & Whitney already has the blueprints for the RD-180 and a license to manufacture them through 2022.

Why is not a single company in America trying to buy the manufacturing license for those engines?

What would be the point? They can’t be used for national security launches, so every launch provider who could use them would need an alternate engine anyway. Much easier to just use that alternate engine for everything.

And the Boeing chief engineer repeatedly said they would have to take the craft “back to the factory” to take it apart! Finally a correspondent rescued them by asking “Where is the factory?” Turns out he meant the old Shuttle processing facility at the cape! These people are so isolated from reality they can’t communicate with normal human beings.

Huh? What’s your point? The former Orbiter Processing Facility *is* the Starliner factory.

Can someone reassure me, that the weapons Boeing manufactures for our military, will actually work, when the War starts?

Considering that the F-15 Eagle (with a combat kill ratio of 100 to 0), F/A-18 Hornet, AV-8B Harrier, AH-64 Apache, C-17 Globemaster, CH-47 Chinook, A-10 Warthog, AGM-84 Harpoon, SLAM, JDAM, SDB, and FLM are all combat proven and considered the best in their class worldwide, whether the weapons work is not what you should worry about. You should be worried about whether the missions they’re used on are appropriate and, if so, what the rules of engagement are.

Say the Nauka launch and docking had been flawless. Starliner was ready to go at the time, weather okay, so a launch was probable. Then the valves get stuck *in orbit* and the capsule is lost.

Highly unlikely. Having the valves open is one of the launch commit criteria.

I bet the gasket just naturally broke down after sitting for YEARS on the shelf and or installed. I never trust a rubber product after 10 years.

I don’t think rubber would react well to nitrogen tetroxide. My understanding is that the seal material is teflon.

OK, that’s enough for one comment.

Willam: Ditto as for my expectations for the future, c1969.

The mistake our generation made was buying into the Soviet-style government “space program” approach. There are signs that this is now finally being abandoned, and thus we have real achievement in space.

No more “programs!” Let’s have a chaotic, free, space industry competing to provide space services to anyone who wants it, however they want it.

Several of my guns are nitrided to stop the steel the oxidizing. It works great.

Those valves are INSIDE the rocket. They are protected from outside air changes by that first.
Second the rocket is inside a climate controlled assembly building. This is to help keep moister from forming from rapid temperature changes.
Third the gaskets are INSIDE the valves. Which is an sealed and protected system.

Now was there oxidizer loaded into the system at anytime since the system was installed. If so why was the system not flushed and cleaned after that test filling?

I bet the gasket just naturally broke down after sitting for YEARS on the shelf and or installed. I never trust a rubber product after 10 years.
Especially a valve gasket. After a few years of sitting in one place they tend to take a set shape and when finally used and moved do not reshape to make a good seal. By the way O-rings are the worst.

Willam: Y’know, there are valves I’m sure in the fuel lines on every airplane ever flown, as well as in all cars. They don’t leak or get corroded easily.

What can damage them is non-use, which might be what happened with Starliner. Starliner sat for years waiting for this flight. Sitting without much exercise the nitric acide they theorized caused the corrosion had time to work its evil. Using the valves regularly would wash it out.

SpaceX solves this problem by treating its stages like airplanes. They only make money on them when they are in the air, so they want to keep them flying.

The company also appears to have a very robust quality control system. If something fails, they fix it, but they fix it so the problem never reappears again.

Think back to early 2020. SpaceX had a string of launch aborts. Elon Musk called a meeting demanding that this be fixed. Whatever they did, they have been launching practically weekly since with no abort.

Say the Nauka launch and docking had been flawless. Starliner was ready to go at the time, weather okay, so a launch was probable. Then the valves get stuck *in orbit* and the capsule is lost.

What happens then? Boeing has to do a third qualification flight? Or Starliner is cancelled altogether? Or, more probably, Boeing cries to Congress (“space is hard!”) and gets another cash envelope?

Somebody else needs to start thinking big-and-fully-reusable. Elon needs competition, needs other competent and creative thinkers to try other ideas, or he’s going to wind up with a de facto monopoly, and all of the temptations that come with one. His focus on “unprofitable” Martian colonization might kick that can down the road for a time, but eventually it would come back to bite us as his successors turn their focus more towards maximizing short-term profits.

It was about the time that Paul Spudis wrote in his book, The Value of the Moon, “Regrettably, strategic confusion currently abounds in the American civil space program.” Eeric Burger noted that NASA had been set adrift:
http://www.houstonchronicle.com/nasa/adrift/1/

Obama and Congress turned NASA into a cluster.