Research as usual

Andrew Gelman just posted a follow-up on one of the big psychological research “scandals” of 2011: Daryl Bem's “Feeling the future” paper. Bem relates a series of experiments showing evidence of paranormal “precognition” (for example predicting on which side of the screen erotic pictures are going to appear or remembering some words better before learning them). Given the nature of the paper, the editors of the journal that published it decided to explain their decision and invite a methodological critique.

The study has been heavily commented on the web. Since then, another methodological critique and failures to replicate Bem's results have appeared (but it turns out to be more difficult to publish them in the JPSP than a paper arguing for psychic powers). One of the main point on which all commenters agree is that the original studies are actually quite banal methodologically speaking, as far as social psychology experiments go. Tal Yarkoni details all the little flaws that make it possible to find such spurious results but none of them seem very big and all are pretty common in the psychological literature.

All of this reminded me of another little scandal that unfolded last year: Satoshi Kanazawa blog post proclaiming that “black women are less attractive” (more on the content and methodological flaw in the analysis). It was not the first time that Kanazawa posted stupid and offensive stuff on his blog but this time, it started a storm of controversy, complete with calls to sack him, an official investigation, letters of support and Psychology Today finally caving in to the pressure and removing the text. True, it was not a peer-reviewed article but the sad thing is that his usual output is not much better and still his supporters are perfectly right when they stress that he has published many articles that were judged sound by reviewers.

The two scandals were quite different but in both cases boil down to offensive and ludicrous findings that still meet the current methodological standards within psychology. This should perhaps tell us something about those standards…

PS: A new post from Andrew Gelman with some thought on how to improve the situation just came in as I was writing this entry.