Questions raised about NEOWISE asteroid data analysis
A computer entrepreneur has raised questions about the data analysis used by the scientists in charge of NASA’s NEOWISE space telescope (formerly called the Wide-field Infrared Space Telescope, or WISE).
Myhrvold, a former chief technologist for Microsoft, founded the patent-buying firm Intellectual Ventures in Bellevue, Washington, in 2000; on the side, he pursues interests ranging from modernist cuisine to palaeontology. A few years ago, he began exploring ways to detect dangerous space rocks. He soon argued3 that the Large Synoptic Survey Telescope, a ground-based telescope being built in Chile, would have the capacity to find nearly all the same asteroids as NASA’s proposed successor to NEOWISE, called NEOCam.
That turned his attention to how asteroids could be studied in space, and to the NEOWISE data. “I thought, this will be great, maybe we’ll be able to find some new and interesting things in here,” he says. But Myhrvold soon became frustrated with the quality and analysis of the data. He posted a critical preprint on arXiv in May 2016, and the peer-review game was on.
His first peer-reviewed critique was published in Icarus in March4. In it, he explored the mathematics of how asteroids radiate heat, and said that the NEOWISE team should have accounted for such effects more thoroughly in its work.
The latest paper1 holds the bulk of the NEOWISE critique. Among other things, Myhrvold argues that the NEOWISE team applied many different modelling techniques to many different combinations of data to achieve its final results. He also criticizes the choice to include previously published data on the diameter of certain asteroids in the data set, rather than using NEOWISE measurements — which, though less precise, are at least consistent with the rest of the database. Such choices undermine the statistical rigour of the database, he says.
Alan Harris, a planetary scientist with the consulting firm MoreData! in La Cañada Flintridge, California, was one of the paper’s reviewers. “In my opinion, it has important things to say,” he says. “It is my hope that the scientific community will read the paper and pay attention to the analysis Myhrvold has presented, as he has raised a number of significant issues.”
The disagreement involves the NEOWISE team’s estimate of asteroid sizes, based on the infrared data. Myhrvoid questions their estimates.
More details about the clashes between Myhrvoid and the NEOWISE science team over the past two years can be found here. The NASA scientists do not come off well. They appear to be very defensive, acting to stonewall any review of their work. Repeatedly they attempted to defy Myhrvoid’s FOIA requests (only made when they refused to release their raw data), including redacting significant information for no justifiable reason.
I have really only one question: Does the behavior of these NASA planetary scientists sound familiar? To me it does, and what it reminds me of speaks very badly for the science being done in the NEOWISE mission at NASA.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
A computer entrepreneur has raised questions about the data analysis used by the scientists in charge of NASA’s NEOWISE space telescope (formerly called the Wide-field Infrared Space Telescope, or WISE).
Myhrvold, a former chief technologist for Microsoft, founded the patent-buying firm Intellectual Ventures in Bellevue, Washington, in 2000; on the side, he pursues interests ranging from modernist cuisine to palaeontology. A few years ago, he began exploring ways to detect dangerous space rocks. He soon argued3 that the Large Synoptic Survey Telescope, a ground-based telescope being built in Chile, would have the capacity to find nearly all the same asteroids as NASA’s proposed successor to NEOWISE, called NEOCam.
That turned his attention to how asteroids could be studied in space, and to the NEOWISE data. “I thought, this will be great, maybe we’ll be able to find some new and interesting things in here,” he says. But Myhrvold soon became frustrated with the quality and analysis of the data. He posted a critical preprint on arXiv in May 2016, and the peer-review game was on.
His first peer-reviewed critique was published in Icarus in March4. In it, he explored the mathematics of how asteroids radiate heat, and said that the NEOWISE team should have accounted for such effects more thoroughly in its work.
The latest paper1 holds the bulk of the NEOWISE critique. Among other things, Myhrvold argues that the NEOWISE team applied many different modelling techniques to many different combinations of data to achieve its final results. He also criticizes the choice to include previously published data on the diameter of certain asteroids in the data set, rather than using NEOWISE measurements — which, though less precise, are at least consistent with the rest of the database. Such choices undermine the statistical rigour of the database, he says.
Alan Harris, a planetary scientist with the consulting firm MoreData! in La Cañada Flintridge, California, was one of the paper’s reviewers. “In my opinion, it has important things to say,” he says. “It is my hope that the scientific community will read the paper and pay attention to the analysis Myhrvold has presented, as he has raised a number of significant issues.”
The disagreement involves the NEOWISE team’s estimate of asteroid sizes, based on the infrared data. Myhrvoid questions their estimates.
More details about the clashes between Myhrvoid and the NEOWISE science team over the past two years can be found here. The NASA scientists do not come off well. They appear to be very defensive, acting to stonewall any review of their work. Repeatedly they attempted to defy Myhrvoid’s FOIA requests (only made when they refused to release their raw data), including redacting significant information for no justifiable reason.
I have really only one question: Does the behavior of these NASA planetary scientists sound familiar? To me it does, and what it reminds me of speaks very badly for the science being done in the NEOWISE mission at NASA.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
“…Does the behavior of these NASA planetary scientists sound familiar? To me it does, and what it reminds me of speaks very badly for the science being done in the NEOWISE mission at NASA.”
This seems to be an artifact of the continuing rise in the importance of NEO analysis alongside the importance of that analysis to future budgets for NEO missions. Yes, that is similar to ground-based data for climate and its problems with analysis. However, unless anyone can see an institutional connection, between NEOWISE and the GISS, for instance, I am doubtful it is that direct. This is a problem that science has ever more frequently as it gets more and more of its funding from politicians (institutional or elected) that *need* prominence to justify spending.
The cure is more people like Myhrvold, who will pay for analysis *outside* the political funding hierarchies, by the millions of such people. That will require further exposure of the innards of the sausage factory that produces so much of our science funding today. It will also require an expanding and confident economy for people to be willing to make such investments.
At least the NASA managment is not part of the defensive behavior. From the second link:
Unfortunately, this means that it is the scientists themselves who are unable or unwilling to explain their interpretation of the data. Good science requires the ability and willingness to present these explanations along with the data.
I don’t see NASA’s management in as favorable a way as Edward.
Sounds to me like NASA’s higher ups were perfectly happy with the data being hidden away but being closer to the political arena they realized a fight with Myhrvold and his powerful legal team would be one they were likely to lose. So they made a wise political decision.
Notice how even after the FOIA was granted, the info was released in bits and pieces. Even after NASA management “came down largely on his side”. Myhrvold had to file twice more.
It’s scandalous that a citizen has to file a FOIA to see scientific data that belongs to the people.
The only gov’t info that should be withheld in any way is info that truly impacts national security or sensitive personal info.
When Myhrvold worked for Microsoft, the tech he developed on their premises with their equipment belonged to Microsoft. If he (or any such person at any private company) tried to with hold company info from the company bosses they would be out the door.
It’s time gov’t workers understand that their boss is the people of the US.
Without serious consequences nothing will change.
Try telling an IRS auditor that they’ll have to file a FOIA request before you start showing them your financial data.
Better still, which of us lowly citizens could successfully negotiate that you’ll only allow a private firm to check your server out, and not hand it over the the FBI for analysis.
Klystron–
nail, on the head!
pivoting tangentially….
The L-128 Klystron Tube
“How a Klystron Tube Works”
(Inventor Rudy Dehn explains how the high powered microwave device works.)
https://youtu.be/TsBTI3tO5-8
4:48
Mitch S. wrote: “It’s scandalous that a citizen has to file a FOIA to see scientific data that belongs to the people.”
My understanding of the article and the essay is that the data was made available to the public, which is why Myrhvold has his own analysis of the data, but the “various unpublished techniques that they [JPL scientists] had used in a series of highly cited papers” were not made available. This is the method used to interpret the data, which is what the JPL scientists are reluctant or unable to explain, even to other scientists.
Myrhvold’s interpretation of events does not leave me thinking that NASA management was being political. I agree that NASA needs to maintain control over its scientists and make sure that they are open and honest with We the People, otherwise NASA’s reputation will eventually be seriously harmed. However, it is up to Myrhvold, not NASA management, to determine whether he is satisfied with the information from JPL. We can see that he is not.
NASA management may not consider this to be a big deal, because the differences between the asteroid size calculations from NEOWISE and other recent asteroid studies may not seem terribly significant. However, Myrhvold seems to be concerned and curious about what he thinks are inconsistencies. “Such choices undermine the statistical rigour of the database, he says.”
Last month (I think) we commenters at the BtB site briefly discussed the contributions of amateur civilian astronomers. If this group of amateurs is to be of any help to the professionals, then the professionals must also be willing to accept critique of their work and be able to defend their own techniques and conclusions. This is not a new concept in science; in order to receive a PhD, the candidate must go through a process known as defending his dissertation. If he can defend his work to a critical committee of experts, then surely he can defend it to the public. The advantage over the dissertation defense is that time can be spent reviewing the techniques and writing a well thought out, detailed explanation of them, rather than thinking on the spot when orally defending a dissertation.
Myrhvold seems to be willing to discuss and defend his work and techniques. The reluctance of the JPL scientists to defend their work makes me wonder how careful they were when they did it, or whether they now are second guessing themselves on their efficacy.
Jordan B Peterson:
12 conservative principles in 12 minutes
https://youtu.be/_MyduTaCh18
12:29