...Why scientific programming does not compute
25 Oct 2010 5:43:30 UTC
Topic 195404
(moderation:
14 Oct 2010 | VOL 467 | 775
“There are terrifying statistics showing that almost all of what scientists know about coding is self-taught,†says Wilson. “They just don’t know how bad they are.â€
Wilson developed the week-long course to introduce
science graduate students to tools that have been software-industry standards for 30 years — such as ‘version control’, which allows multiple programmers to make changes to the same code, while keeping track of all changes.
The bottom line message is scientists are mostly self-taught programmers and not very good at it. Worse programs that work wrong have caused papers to be withdrawn. This is very much worth the read.
=====
Worst to me is even Microsoft would not tolerate the effectively non-existent validation standards. The "test" of a program is usually if it gives the expected results and nothing more.
If for example the CRU expects the world to be getting warmer then the software does not in the least confirm it is getting warmer only that the output says it is getting warmer. That says nothing about the software working right nor the validity of the results. The results could be built into the software without regard to the input data as the "hockey stick" was deconstructed a few years ago.
...Why scientific programming does not compute
)
I think, or hope, that the CRU example is an outlier - probably more a result of lack of scientific attitude rather than failed technique per se. The likely conclusion ( most probable objectively ) is not so much whether the programming was good or bad, but that even good programming could not save bad science. The key revelation in that example is that deliberate non-random filtering of data sets automatically kills the assumptions on which statistical analysis is based. If you skew inputs then you trash the entire basic logic of the tools in that area of maths - which would still be a problem with an abacus. The programming aspect then simply becomes a layer of obfuscation upon good old fashioned fraud ( eg Piltdown Man ). There's an old adage that a false premise can imply anything ( true or false ). I think the computer involvement is peripheral to the core of the CRU debacle. Why the CRU matter got alot of attention is that, much like Watergate, the trail led right to the top - specifically the IPCC and ultimately it's heavy reliance on the CRU work. Alas this aspect seems to have been poorly understood ( and perhaps deliberately so ) in the blizzard of spin that surrounds this topic. But everyone can learn from this .... whatever your personal spin flavor is. We still need proper objective understanding of our climate regardless.
In my view a better question is - how do you define 'scientific' behaviours? What deserves the valid use of that term as a description of areas of inquiry? Certainly climatology generally has taken a hammering in this regard. The CRU response to a divergence in the trends between dendritic ( tree ring ) and direct temperature data ought to have been to investigate the underlying physical reasons for the disparity. NOT to have chosen one set in preference to the other without good basis. By analogy if you had a particle accelerator and found that half the particles went one way while the other half the other, then you wouldn't choose one set and suppress the other data, but sort out why you appear to have two particle types. The first ( suppressive ) technique makes the data follow the conclusion, whereas the second is more honest - specifically you claim no conclusions until some can be made ( if at all ). Further data collection lights the way that your thinking should go. Software testing can still validate a bad design, giving a false sense of security. That is : you can find out that you built an excellent aeroplane, even though a submarine was intended. :-)
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
I don't need any software but
)
I don't need any software but only my eyes. All the glaciers I climbed in my youth in the Alpi Giulie are disappearing. This is very bad.
Tullio
RE: I don't need any
)
The Scientists say that at one point in time waaay back when the Earth was a ball of ice with even the oceans being being covered by 50 foot thick ice, the Scientists also say that at another point in time the Earth was covered by a bunch of volcanoes making it a very hot place. I think the question to your observation is 'is this a normal thing' or 'is this the Earth responding to the influences put on it by its inhabitants'. Until we get enough info the answer is 'we just don't know enough right now' to answer the question properly. And to be honest we may never know the answer, it may just take too long for the answer to be proved.
And the question about data collection was to me, best answered by the show CSI:Las Vegas a couple of years ago, they said, and I am paraphrasing, 'it is not my job to prove guilt or innocence, it is my job to collect the facts, the evidence speaks for itself'. That is the problem with some Scientists today, they want the evidence to speak for them and their agendas instead of whatever the facts are. Way back when Sgt. Joe Friday said 'just the facts mam, I just want the facts'. I think in this day and age of results oriented agendas and 'I want it NOW' thinking it can be very hard to stay objective!
Well, I am running climate
)
Well, I am running climate models at climateprediction.net just to answer your doubts. I agree we need more facts but they all go in the same direction: the Earth is warming. If this is due to man or the Sun we don't know yet. But I think that reducing gaseous emissions and bettering the efficiency of all industrial processes is the way to go even if global warming is not caused by man.
Tullio
Another ploy, if you like,
)
Another ploy, if you like, has been the argument or advocacy by 'referral to authority'. Roughly speaking this goes 'so and so said it, so it must be true'. Now it's not unreasonable to claim ignorance of specifics and arcana thus, similiar to specialist evidence in law courts, we need to rely upon devoted intellectual resources. However one can go too far, and also like a law trial one ought to examine the veracity of any claim to specialist knowledge.
An excellent example DownUnda in the climate debate has been the CSIRO, the federal government funded science research body, which has been proposed by some sources as being authoritative. So effectively the planet is warming because the CSIRO says it is. What I find amusing about that line of reasoning isn't so much whether the CSIRO is right or wrong, but that those who posit the authority of CSIRO have self evidently done so very little homework, or have neglected to reveal any they have done. One ought to do some background checks if you're going run that line of logic ....
You see this is the very same CSIRO that was dragged, kicking and screaming, to a Royal commission in the 1970's for their scientifically fraudulent behaviour in the 1950's ( partly under a previous institutional title ) regarding the then ( British ) nuclear testing in outback Australia. To cut to the chase : they buffed reports that stated that cracking off nukes was not only not bad for the indigenous inhabitants but a positive boon. Back then they ( aborigines ) were viewed by 'civilised' white anglo-saxons as being on a par with native animals - kangaroos et al. So the range safety efforts only targeted whites. Literally. It is actually unknown how many were killed on the day or later on because they were out in the field when the button was pushed. This beggars belief today but that's the record. So science covers criminal activity? All this was a product of doing the work you are paid to do. In this case the paymasters were the political institutions of the day. [ Google say with 'CSIRO' and 'Maralinga' or try this book for instance. ]
OK, you might say, that was then and this is now. So time heals and improves us all, right? You'd think that an institution with that sort of not-to-distant history under it's belt might be real keen to get us to relax about how they go about things nowadays. Well, you'd be wrong. When the Emissions Trading Scheme went to parliament last year various proposals were based on CSIRO work in recent times. When queried the CSIRO essentially slammed the door saying - and get this - it's true because we say it is. Not even partial disclosure of their methods, data and thinking, but a patently silly self-referential statement that any abrasive three year old could come up with.
Note the issue now isn't whether the CSIRO is right or wrong, because at present that is an un-examinable hypothesis ( the CSIRO response has ensured that ). The issue is their active prevention of any testing of their claim to authority. Interestingly some of their computer modeling on climate change was questioned by a direct request to reveal the source code. Which has yet to be honoured, and was rejected with some blather about confidentiality etc ( this from a group of people supported by public funds ..... ). Shades of CRU perhaps? Do we need yet another CSIRO focused judicial investigation?
[ And of course one could ask why the current paymasters aren't forcing that issue, which is very much in their power. Also I am personally confused by such a heavy reliance on GW advocacy. There was already an avalanche of good reasons to reduce our hydrocarbon dependency prior to that hypothesis. I remember the good old days of Rachel Carson .... ]
Cheers, Mike.
( edit ) No doubt I'd be labeled as a 'climate skeptic' or some such, which distorts my real intent - but much like cold fusion claims I'm skeptical of those who attempt to use the rubric of 'science' as a cover for personal opinion, preference and gain ( see Al Gore ). You can't really use science to answer sociological questions ..... but you can use emotion to divert objective measurement and thought.
( edit ) To be more exact a humble field officer in the CSIRO, biochemist Hedley Marsden, had the guts to blow the whistle on higher management who were very compliant to political influence. One bizarre 'scientific finding' was that atom bomb radiation exposure was to be preferred, for safety, over a medical chest X-ray! Now the bulk of frequencies from nuclear processes is in the 'gamma ray' range, much more energetic, and so a comparison at X-ray wavelengths alone could give that favourable answer. However that's like saying I'll have less chance of a 60 mph collision if I travel in a car going 90 mph!!
( edit ) More Maralinga detail here, which I'd point out page 18 of 48 ( by pdf count ) :
while no especial hard link was found ( no emails then, nor memo trail alas ) to firmly prove collusion b/w upper CSIRO and Federal Government committees, the only logical remaining consistent and simplest deduction is exactly that. Occam.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
RE: But I think that
)
Hear that!
If you sit in a flooded boat you don't point at the waves and say "it's not my fault".
You man the water pumps.
RE: The bottom line message
)
Well, that's why the AEI employs software engineers and computer scientists like me.
And for CPDN at least there were simulations ran outside of the expected parameter space to validate the simulation and parameter space regions they were searching in. Turned out to be difficult to get funding for such experiments / searches. This was mentioned in some length at the Pangalactic BOINC Workshop / Citizen Cyberscience Summit in London in September, though I can't find the talk at first glance.
BM
BM
RE: RE: But I think that
)
My point is simple: use prudent avoidance.
Tullio
RE: Well, I am running
)
Is the software you are running open source? If so, where can I download it?
If it is not open source then one factually does not know a thing about the nature of the results.
Getting back to the theoretical question, if the input data is not the raw data and taken "as is" then the methods of statistics we have today do not apply. What would then be interesting is to see the same analysis of the raw and "massaged" data and compare the results. If the results are different then one can gauge the effect of the massaging.
RE: I don't need any
)
This isn't quite a number crunching issue but there is no basis for a good or bad decision regarding temperature.
This is similar to the "ben franklin" closing technique in real estate. The salesman suggests making a list of the good and bad points buying the house. He suggests some good reasons and has them written down. If the buyers come with bad reasons the salesman quickly suggests another good reason and has that written down. In a short period of time there is a long list of good reasons and one or two bad reasons.
The West seems to like apocalypses so all we hear are bad things that might happen. Of course 'might' morphs into 'will' and 'in the future' morphs into 'no later than noon tomorrow.' So the no longer global warming but now climate change ben franklin is a long list of negatives.
So for the anecdotal glacier we have a longer growing season, uncovered land for more plants to grow on both of which consume more CO2. In the process wildlife increases living on this increase in plant life. If this is indeed global then growing seasons for food production become longer and extend further north which is exactly in the right direction for the expected doubling of the world population in the next four decades.
Such statements rapidly devolve to dataless argumentation which is political not scientific. And that is the best we have today permeating the entire discussion.