I worked briefly in the 1980s for a doctor who was a pioneer in arthroscopic knee surgery to repair ACL tears. He came up with a theory that by doing a certain procedure he invented that he could help people who had worn out the cartilage that separates the two leg bones and keeps them from grinding each other. Once worn away (are you marathon runners listening?), knee cartilage will never grow back and heal. His operation was designed to see if it was possible to regrow cartilage.
So he performed probably 100 of this experimental operation – his definition of “success” was that the space between the two bones measured on an xray increased. At some point, he decided to do a survey of those patients to see if the operation had actually HELPED the patient in a meaningful way – my guess is this was to prove to insurance companies that the operation was medically useful so insurance would pay for the operation. In most cases, insurance will not pay for experimental procedures.
So a questionnaire was created asking things like “Do you have less pain than before the operation?”, “Have you had additional knee operations?”, “Are you able to walk easier or further than before the operation?”… but before the questionnaires were sent out, the doctor went through the list and pulled out the names of those patients who he remembered to have negative outcomes or were potential lawsuits just by being reminded of their operation.
Correlation does not prove causation – but if you cherry pick the population to exclude those data points that do not support your hypothesis, you aren’t even proving correlation. Drug companies get caught doing this all the time (but not enough)…. they’ll exclude outcomes that don’t support the hypothesis on some rational basis, but not exclude their successes on the same basis. Drugs will not be tested on women who might become pregnant for the obvious reason, but then generalize the effectiveness of the drug for people who didn’t take it during the clinical trials.
(Even with the cherry picking of data, the survey found no positive benefit to the operation and many reports of negative outcomes)
The global warming folks have been accused of this same thing – of picking only those temperature monitoring stations which had results that supported the hypothesis and excluding the data from those that didn’t. NASA had deployed a network of ocean temperature monitoring stations and were stunned when they reported no change in ocean temperatures so launched an effort to “fix” the monitors so they would support the desired conclusion. This is not science, it is fraud. More often than that, when you find scientific fraud, there is a pile of money involved.
http://www.npr.org/templates/story/story.php?storyId=88520025
“Environmental Science” is an oxymoron. Like lazy reporters, most of these people have their minds completely made up about the facts before they conducted any experiments. Thus any datum points that don’t fit their hypothesis are tossed out as being spurious. That way they ensure their grant money keeps rolling in.
Remember when Al Gore said Global Warming, er…, Global Climate Change (that’s better) was “settled science?” Ole Al forgot the definitions of Theory vs. Laws as taught in 5th grad science (are you smarter than a fifth grader, Al?).
Remember when some dingbat scientist stated that Mars did have water and that Global Warming had killed all the life on Mars? Same kind of science that concluded rotten meat spontaneously generated maggots and flies. Wasn’t that sometime in the 1600’s?
So I will now go on record by stating my Theory of Global Climate Change:
Global Climate Change is a natural phenomena, not a man-made one (except for the data points used in the fraudulent papers published to date).
I await my Nobel prize.
“Murder your darlings,” is the soul of science. One can have nearly complete certitude respecting this or that pet theory, and still be a scientist simply by designing a study, formulating the hypothesis and stating it in the null, doing the measurements and observations, taking in ALL data, running the data through statistical analysis and publishing the results, whether or not they conform to preconceived notions. Either way, learning occurs. But when one hides data that suggest weaknesses or outright erroneousness of the hypothesis, then one no longer is a scientist but is an advocate at best, a shill perhaps, or an outright demagogue and liar.