GAO: Cost and scheduling problems with many big NASA projects
A new Government Accountability Office audit [pdf] that reviewed 22 major NASA projects, including Orion and SLS, has found that many of them have significant scheduling and cost problems.
Let’s just go through them all:
- SLS: “The SLS program’s schedule is deteriorating and it is at increased risk of exceeding its cost baseline and missing its November 2018 launch readiness date.”
- Orion: “The Orion program is increasingly at risk of missing the November 2018 launch date for its first uncrewed exploration mission.”
- Mars 2020: “The Mars 2020 project has not met key best practices for reducing product development risk.”
- Asteroid Redirect Robot Mission (ARRM): “In August 2016, the ARRM project entered the preliminary design and technology completion phase with a higher cost and longer schedule than previously estimated.”
- Europa Clipper: “At the project’s most recent decision review, its independent review board stated that it was at risk of exceeding its preliminary cost and schedule ranges unless its scope or complexity was reduced.”
- Ground Systems (EGS) upgrade: “The EGS program’s schedule is deteriorating and it is at increased risk of exceeding its cost baseline and missing its November 2018 launch readiness date.”
- ICESat-2: “The ICESat-2 project has encountered problems with the flight lasers in its sole instrument—the Advanced Topographic Laser Altimeter System (ATLAS)—that will likely cause it to miss its committed launch date and could cause it to exceed its current cost baseline.”
- InSight: “The InSight project missed its committed launch date of March 2016 and exceeded its cost baseline due to technical issues with its primary science payload—the Seismic Experiment for Interior Structure (SEIS) instrument—which is contributed by the French space agency (CNES).”
- Ionospheric Connection Explorer (ICE): “The ICON project has experienced technical issues and delays in system integration and testing, but it still on track to launch in July 2017—3 months earlier than its committed launch date.”
- James Webb Space Telescope: “In December 2016, we found that the primary threat to the JWST project continues to be the ability of the observatory development and integration contractor, Northrop Grumman, to control its costs.”
- Radiation Budget Instrument (RBI): “NASA’s joint cost and schedule confidence level analysis indicated that the likelihood of the project meeting the date is low and the project’s independent review board described the schedule as optimistic when compared to similar instruments. … The RBI project’s prime contractor Harris continues to experience cost overruns.”
- Space Network Ground Segment Sustainment (SGSS): “The SGSS project has exceeded the new cost and schedule baseline NASA set for it in June 2015 and further cost and schedule growth is likely.”
Not all the projects audited were a disaster. GRACE-FO, Landsat-9, NISA, Solar Probe Plus, SWOT, TESS, and WFIRST have few significant problems, though even with these there have been delays with each project still facing significant cost and scheduling risks.
As for Commercial Crew, the audit notes delays and problems, but these appear to be mostly linked to the bureaucratic and somewhat unjustified demands by NASA for increased safety, such as the agency’s refusal to accept the use of the Atlas 5 with a Russian first stage engine and its concerns about SpaceX’s plans to fuel the rocket with astronauts on-board (even though astronauts have been aboard fueled rockets with every other manned launch for the entire history of space exploration).
Overall, this audit does not speak well of either NASA’s management or the contractors with whom the agency has routinely worked. Space engineering is hard, but many of these problems seem more related to either incompetence or a willingness of NASA to forgive bad work too often. The number of contractors or government agencies listed here who have failed entirely at their jobs is appalling.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
It could’ve been worse.
> SpaceX’s plans to fuel the rocket with astronauts on-board (something that has been done with every other manned rocket for the entire history of manned space exploration).
I think prior practice for Mercury/Gemini/Apollo/Shuttle was to have the rocket fueled while the astronauts were away. Then to have the astros arrive and hop in.
SpaceX’s proposed scheme is to have the astros arrive to a non-fueled rocket, strap in, ground crew leaves, then load the fuel. The astronauts would have the capsule escape system as a last resort
It is arguable if one or the other of these is safe for the astronauts and/or the ground crew. Both have risks, and both have advantages.
fred k: I suppose my phrasing is poor. NASA somehow implies that there is more danger with putting the fuel in the tanks while the astronauts are on board versus having the astronauts board while the fuel is already in the tanks. To me, this is empty sophistry, created for the sole purpose by NASA of slowing SpaceX down so that they might get SLS off first.
I have revised the language of my post to make it more accurate.
Actually, the AMOS-6 explosion occurred during pressurization of the second stage helium tanks in preparation for a static fire. It is a more hazardous time when propellants and pressurants are in a transient state than when they are either not present, or present in a final equilibrium state.
Surprising NASA is having cost and schedule problems. This is my business. Project Management. They are supposed to have project controls in place. It’s the law. Something fundamentally wrong with how they do contracting probably. And they may not have qualified PMPs involved. Maybe NASA should get out the business of building space projects and running programs. Private industry is probably the solution to the problem.
“…such as the agency’s refusal to accept the use of the Atlas 5 with a Russian first stage engine”
In fairness, this is being driven pretty much by congressional pressure, for political reasons we’re all aware of.
But other than that: NASA cannot blame Congress for much of the rest of these problems.
NASA is just another example of ‘too big to fail.’ It needs to be downsized (or get a goal worthy of the resources spent.)
Problems this blatant are easy for adults to fix. Government is not run by adults.
NASA was a success in the space race of the 1960s because they were a new organization that hired people from the private industry. SpaceX is so successful because they too hire people who know how to make things happen. Is that a fair assessment?
I painfully listened to an interview of the (now fortunately) ex-NASA director of science, by the Planetary society a few months ago. She only talked about sex and gender and race. Science doesn’t seem to be on her mind at all. And that in an interview with a space nerd crowd. Disgusting. Sad.
(I used to prefer Shakespearean words like “horrible”, but that’s kind of out of fashion it seems. Sad.)
“NASA was a success in the space race of the 1960s because”
Because they had access to the Nation’s checkbook and the mantra was waste anything but time.
Their success was open ended funding.
Vladislaw–
good stuff.
Not to diminish any of the accomplishments, but basically a cold-war version of the Manhattan Project. It just costs huge amounts of resources.
tangentially, ran across an excellent James Burke program, while on my way to re-watch all the Ascent of Man episodes referenced in a different thread
James Burke
“The other side of the moon”
1979
https://youtu.be/puWbQ1b-ljU
(57:19)
Vladislaw
“NASA was a success in the space race of the 1960s because”
Because they had access to the Nation’s checkbook and the mantra was waste anything but time.
Their success was open ended funding.
Zubrin pointed out that NASA gets just as much money today as they did then, when adjusted for inflation.
NASA didn’t have the institutional barriers then that it does now. It also had a clear purpose. Today? NASA is often its own worst enemy, followed closely by congress.
Vladislaw
“Because they had access to the Nation’s checkbook and the mantra was waste anything but time.”
No, throwing more money in the black hole of socialism doesn’t help at all. SLS/Orion is an example of that. Especially if you contrast it with SpaceX. I mean, they could’ve launched 3,000 astronauts in 3000 Dragons instead of un-developing the Shuttle into SLS/Orion. We are talking not one, but two orders of magnitude difference here.
In the 1960s, in many Western countries, bureaucracy grew enormously. And when you start a new infrastructure building organization from scratch by recruiting people from the private sector, you can do good for a while. But since socialism means lack of leadership and accountability, it doesn’t take many years until the organization gets totally corrupted and improductive by special interests, not least the middle management which males sure to protect its privileges. After the foundation, NASA has hired people from school. They go from one planned economy to another. They don’t have grown up experiences of making things happen. They study management theory, but they never manage anything in real life.
The problem with socialism is that there exists no feedback from results to the doer. Actions have no consequences for the person acting, and by evolution we then economically ignore what we do and focus on something which does have consequences. People are not allowed to use their brains and make evaluations and choices. It doesn’t matter to the NASA employees whether anything ever flies. NASA is too small to affect how people vote, so it is safe to fail completely. Which they of course do. It is much easier that way. Nothing to lose. No incentives.
2,000 astronauts in 300 Dragons, to correct my typo above. 43 / 0.133 billion dollar for SLS-Orion versus what NASA will pay SpaceX for the first crewed Dragon launch. Heck, it would’ve been cheaper and safer to keep the Shuttle!
And I must say that the academics, the astrophysicists, are doing a fantastic job even in the government, even in NASA and other space agencies. I think they are too removed from the political and business world for the difference between a so called planned economy and a market economy to matter much. Politicians and money grabbing middle management are a bit afraid of them. Scientists are bit like a magic priesthood to them, they just want them to say the PC thing about global warming. The academics have their own system of peer reviews and selections. There’s no market price on discovering the unknown anyway.
Rene Borbon wrote: “[NASA is] supposed to have project controls in place. It’s the law. Something fundamentally wrong with how they do contracting probably.”
NASA has a unique problem: it pushes the technology envelope. When they talk about risk factors, one of the meanings is the risk that the development of a new, high-tech instrument or device will have an unexpected problem that causes a delay. Technology development just comes with that risk. If they were to use off the shelf hardware, then they would not get the best information about their probes’ targets. For the expense of the launch, we certainly want to get back the best data.
One problem that SLS is having results from doing spin welding in a way that is untried at the thickness that they are doing it. An unexpected problem developed, but their risk assessment should have taken into account such unexpected problems. NASA has learned to expect the unexpected. They quite often come across, as someone in government put it a few months ago, the unknown unknowns.
With the development of the technology, the rest of us get use of that technology in our own daily products. Several people that I have worked with went on to jobs where they incorporated their new expertise into every day products, such as HP ink-jet printers.
We tend to not see cost overruns and schedule slips in the development of other products, because NASA is a high profile organization with budgets, schedules, and missions that matter to all of us. If a new gizmo on next year’s car model is delayed by a year, most of us wouldn’t know about the delay and wouldn’t care, either.
Another problem is more self inflicted. I have complained before that NASA sometimes changes requirements during mid-development. Such changes invariable change the cost and delivery schedule. That is most definitely a management problem, but usually comes with the hope of getting the best data possible.
MfK wrote: “Actually, the AMOS-6 explosion occurred during pressurization of the second stage helium tanks in preparation for a static fire. It is a more hazardous time when propellants and pressurants are in a transient state than when they are either not present, or present in a final equilibrium state.”
Actually, it demonstrates that a fully or partially fueled rocket is more dangerous to be around, just as the Soviet Union’s Nedelin Catastrophe demonstrated or the Brazilian 2003 pad explosion demonstrated. At least with the astronauts aboard during fueling, their escape system can take them safely away from the explosion, as the Soyuz T-10 did in 1983, and there are no spacecraft close-out crews at the launch pads that risk their lives in the case of such an explosion.
Edward your points well received. In project management theory, one must take into account risk. These risks should be factored into cost and schedule, use of management reserves, and schedule reserves. It is well known that the federal government budget is under pressure and as a result cost overruns are less acceptable these days when a blank checkbook is not provided. Under promise and over deliver. Manage expectations.
I would agree there seems to be a management problem and a scope management problem even considering the nature of the risks in their programs. The Space Shuttle and other NASA disasters are evidence of this. There is no reason why our brave astronauts should be put at risk for schedule and political needs.
It is time for NASA and the country to recognize NASA’s days of papering over risk, cost, and schedule are no longer acceptable. I wish them all the success possible.
Rene Borbon,
Robert Zimmerman pointed out yet another problem that NASA management is stuck with: Congressional micromanagement. Congress holds the purse-strings and pulls on those strings with impunity. Then NASA and its management get blamed when things get expensive or unwieldy.
http://behindtheblack.com/behind-the-black/points-of-information/nasa-inspector-general-blasts-agency-construction-of-sls-test-stands/
I do not think that NASA purposefully puts its astronauts in unnecessary danger, but as the Apollo 1 episode of “From the Earth to the Moon” points out, sometimes there is a failure of imagination as to what can go wrong. After Columbia, it was not until the leading edge of the wing was found that anyone imagined that the loss was not due to damage to the underside tiles.
Challenger happened because Thiokol did a very poor job of communicating to NASA’s engineers. Using the same view foils (Power Point charts), Thiokol said one thing to NASA in March of 1985 (it is OK to fly at low temperatures) and another thing in January of 1986 (it is not OK to fly at low temperatures). Thiokol even made the suggestion that the Shuttle not fly when the temperature is below a Spring day’s temperature. Both of these miscommunications left NASA’s engineers confused and asking questions that were later used by the press against NASA. It seems that Rene Borbon is still using these questions against NASA, because the press did a very poor job of communicating the truth to us (e.g. a waiver does the exact opposite of what the press said it does).
Many of us were astonished that NASA would even consider flying astronauts on the first SLS test flight and relieved when they chose not to. Once again, such consideration seems to have been politically motivated, not NASA motivated.
Just as with aircraft, learning the dangers and the solutions to spaceflight will take time and will cost lives that will seem needlessly lost:
https://www.youtube.com/watch?v=GXbdJ3kyVyU (7 minutes — “Bill Whittle: The Deal”)
More than 15 years, now, and counting.
In addition, even when we know how to do it right, it does not always happen right:
https://www.youtube.com/watch?v=jCBbosq9-RI (10 minutes)
On further reflection, NASA is stuck obeying two masters. It is responsible directly to the White House, but it must please Congress, too. When either or both of these entities tells NASA to do something stupid, then smart people at NASA are trapped into doing stupid things.
This goes beyond internal empire building at NASA or decisions to go to Mars the expensive way rather than the inexpensive way because the inexpensive way leaves too many NASA departments out of the project (thus there is no projects for those departments to be part of). Both of these problems are also problems resulting from poor management.
So, Rene Borbon, while I have pointed out several non-management problems, you are right that there are still many management problems, even internal to NASA, not just (mis)management from greedy or stupid politicians.