Saturday, January 8, 2011

Men who stare at -porn-? OH! I get it!

So, let's summarize the study first. Daryl Bem would have his test subjects look at a screen, where a pornographic image would appear in a moment, either on the left or on the right. More than half the time, people guessed correctly on which side it would appear. Since people would guess before the computer made the random decision, this was a test of precognition. So, evidence for precognition. Champagne time.

Of course there's controversy. There's a quote in the linked article of, “[Bem’s results] indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data.” This gem of backward thinking (the evidence of your bad experimentation is your results) was attributed to Eric-Jan Wagenmakers.

So what are these amazing results? Well, Daryl Bem found that people guessed correctly an average 53.1% of the time.

Oh.

No, no. You don't have to tell me, it's statistically significant. That's great. But where's that kapow? Where are the big numbers that would get me all titillated? 70%, 85%, 120%. Not fifty-three. Especially when fifty is the average. It's a wonder that there's been no scientific revolution with these kinds of numbers.

Maybe because I'm not a scientist, but I'm not impressed (impressed being an emotion) by most statistically significant numbers. Scientists throw around one percents and one-point-five percents so much that I wonder what kind of returns they expect out of their stock portfolios.

Statistically significant is not the same as emotionally significant.

Let me speak assuming the psi is real, which is a pretty rare assumption for me. I think a scientist needs to do one of these studies, maybe ten trials of 100 people each, with 100 guesses per person, keeping it simple. And then, ignore the average. Forget about it, whatever it was, and, instead, scour your database for the 100 best performers in that group of 1,000. Put these mini Uri Gellers together in the same study, and get some results that really shine. Put together the first and the second round as a part of the required protocol, so that it's still repeatable, let your results sweep the world. This goes for the ganzfield folk, and the pornographic precog folk and everyone else that's trying to impress us with their one-digit percentages. So that I don't need a statistician to tell me it's significant. I can just see the significance with my eyeballs, and feel it in my gut.

And, if you put together your star pupils, and your results still aren't that impressive, then don't make excuses, and don't say shoulda/woulda/coulda. Set aside a moment for some soul-searching, making very very sure that you're not barking up the wrong tree with all of this psi research, and act accordingly. Be a scientist.

Thanks for reading.

PS: This is a public service announcement, regarding the book, that was adapted to a movie, the name of which was used in the article I linked to, the name of which I used in my blog post, here.

You may have seen the movie, The Men Who Stare at Goats. Whatever you think of that movie (I didn't care for it, myself) please check out the book that it was adapted from (of course, the current edition has the movie cover). The author, Jon Ronson, investigates the government's experiment with creating psychic soldiers by talking to the people involved. Painfully entertaining and interesting, the book is a documentary, which makes me wonder where someone got the idea of making a feature film out of it (as opposed to a film documentary.) It's very very interesting, and the film, in my mind, doesn't do it justice at all.