More About Fraudulent Psychological Research
Back in September, 2015 we published a report about fraudulent psychological research (“Many Psychology Findings Not as Strong as Claimed“). The Weekly Standard elaborated on the story in their October 19 edition, “Making It All Up – The behavioral sciences scandal.”
We thought our supporters might appreciate a few choice quotes from The Weekly Standard magazine.
“Over 270 researchers, working as the Reproducibility Project, had gathered 100 studies from three of the most prestigious journals in the field of social psychology. Then they set about to redo the experiments and see if they could get the same results. Mostly they used the materials and methods the original researchers had used. Direct replications are seldom attempted in the social sciences, even though the ability to repeat an experiment and get the same findings is supposed to be a cornerstone of scientific knowledge. It’s the way to separate real information from flukes and anomalies.”
“The researchers, [Shankar] Vedantam glumly told his NPR audience, ‘found something very disappointing. Nearly two-thirds of the experiments did not replicate, meaning that scientists repeated these studies but could not obtain the results that were found by the original research team.'”
“Statistical significance is the holy grail of social science research, the sign that an effect in an experiment is real and not an accident. It has its uses. It is indispensable in opinion polling, where a randomly selected sample of people can be statistically enhanced and then assumed to represent a much larger population.
“But the participants in behavioral science experiments are almost never randomly selected, and the samples are often quite small. Even the wizardry of statistical significance cannot show them to be representative of any people other than themselves.”
“Publication bias, compounded with statistical weakness, makes a floodtide of false positives. ‘Much of the scientific literature, perhaps half, may simply be untrue,’ wrote the editor of the medical journal Lancet not long ago. Following the Reproducibility Project, we now know his guess was probably too low, at least in the behavioral sciences. The literature, continued the editor, is ‘afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance.'”
“The defenders of behavioral science like to say it is the study of ‘real people in real-life situations.’ In fact, for the most part, it is the study of American college kids sitting in psych labs.”
“A week after the Reproducibility Project set off its cluster bomb, President Obama’s Social and Behavioral Sciences Team issued its first annual report. … Evidently impressed with all this science, President Obama issued an executive order directing federal agencies ‘to use behavioral insights to better serve the American people.’ Agency heads and personnel directors were instructed to ‘recruit behavioral science experts to join the Federal government as necessary to achieve the goals of this directive.’ We should have known! After all the bogus claims and hyped findings and preening researchers, after the tortured data and dazed psych students, this is the final product of the mammoth efforts of behavioral science: a federal jobs program for behavioral scientists.”