Gaël Laurans

Election season and statistics

2012-01-28T20:00:00.001+01:00

The US primary season is now in full swing and it's as good an occasion as any to play with some data. Some interesting stuff from a statistical angle:

Mapping the results from Iowa at Offensive politics (via Revolution Analytics). Some other interesting resources about maps in R: “R” you ready?, Baptiste Coulmont.
Matt Asher runs a simulation to look at the difference between the first results reported on TV and the final count.
Andrew Gelman has a series of posts on possible irregularities in the Russian elections. In fact, using statistical techniques to look for signs of fraud in election data seems to be a growing field, see e.g. Walter Mebane and The Forensics of Election Fraud.

Research as usual

2012-01-26T20:00:00.003+01:00

Andrew Gelman just posted a follow-up on one of the big psychological research “scandals” of 2011: Daryl Bem's “Feeling the future” paper. Bem relates a series of experiments showing evidence of paranormal “precognition” (for example predicting on which side of the screen erotic pictures are going to appear or remembering some words better before learning them). Given the nature of the paper, the editors of the journal that published it decided to explain their decision and invite a methodological critique.

The study has been heavily commented on the web. Since then, another methodological critique and failures to replicate Bem's results have appeared (but it turns out to be more difficult to publish them in the JPSP than a paper arguing for psychic powers). One of the main point on which all commenters agree is that the original studies are actually quite banal methodologically speaking, as far as social psychology experiments go. Tal Yarkoni details all the little flaws that make it possible to find such spurious results but none of them seem very big and all are pretty common in the psychological literature.

All of this reminded me of another little scandal that unfolded last year: Satoshi Kanazawa blog post proclaiming that “black women are less attractive” (more on the content and methodological flaw in the analysis). It was not the first time that Kanazawa posted stupid and offensive stuff on his blog but this time, it started a storm of controversy, complete with calls to sack him, an official investigation, letters of support and Psychology Today finally caving in to the pressure and removing the text. True, it was not a peer-reviewed article but the sad thing is that his usual output is not much better and still his supporters are perfectly right when they stress that he has published many articles that were judged sound by reviewers.

The two scandals were quite different but in both cases boil down to offensive and ludicrous findings that still meet the current methodological standards within psychology. This should perhaps tell us something about those standards…

PS: A new post from Andrew Gelman with some thought on how to improve the situation just came in as I was writing this entry.

Our programme will resume shortly

2012-01-13T19:16:00.004+01:00

It's now over two years since the last time I have posted anything on this blog. Not that I was intensively blogging before, but I had a pretty good excuse. Now that my PhD is out (it should be available soon on the TU Delft repository), I intend to return to blogging. Let's see how it goes…

DPPI 2009

2009-11-06T21:47:00.005+01:00

Jeroen warned me: the second week of blogging is the most dangerous one and he was right. In fact, I haven't posted anything in more than two month.

Beside some computer problems, one of the thing keeping me busy during that time was DPPI 2009. It turned out to be an enjoyable conference, with great food, and surprisingly well organized given the way these things are usually run in France. I have also met some nice people and had some interesting discussion but unfortunately did not find any presentation that would be directly relevant to my work.

One fascinating thing with design-oriented conferences is that organizers always try to do something a little bit « different », for example with the traditional conference bag or the badge given to each participant. DPPI added an interesting twist to the conference dinner: it was an assigned seated dinner and the seating plan was to be created automatically based on social contacts during the conference. Everyone had to get a ring with a RFID chip and the time spent together during the coffee breaks would be used to form groups. The result was displayed all the time on a big LCD screen in the hallway. A couple of conference attendees in fact used this screen for a clever hack, writing their name on tape and pasting it directly on a table.

At the end of the day, it did not work out very well, though. First, many people did not get a ring or even refuse to use the system, which was to be expected but is still a bit disappointing. Then, many technical constraints made the system a lot less cool than it might have been. In fact, the system would only notice a ring if it was placed in the middle of one of three tables in the hallway. This, combined with an apparently straightforward matching algorithm (the more you spent time with someone, the more likely you were to be seated together), made sure that no serendipitous discovery happened. It felt much more like telling the system whom you want to seat with rather than providing occasions to meet new people or reflect on social encounters during the conference.

But the last blow was dealt by the organizers themselves: instead of using numbers or little name signs to reflect the plan produced by the computer, they decided to print it on a poster, remove a seat from each table and let the tables unmarked. The idea was ostensibly to force people to mingle instead of sticking to the seating plan. The results was highly predictable: people sat together with acquaintances, ignoring the seating plan altogether. It was not all that important however and the diner turned out to be very enjoyable!

Computing a confidence interval for ρ

2009-09-20T22:42:00.000+02:00

Curiously, neither R nor SPSS seem to offer a simple way to compute a confidence interval for Pearson's correlation coefficient based on r and the sample size. R base package includes the cor.test function which does provide a confidence interval based on Fisher z transformation but it takes the full data set as input. Even then, the confidence interval depends only on the sample correlation and on the sample size so the extra information is not really needed, except to compute the sample correlation coefficient in the first place. The confidence interval can therefore just as well be computed from published correlation coefficients, without going back to the original data set.

The formula is relatively simple and can be found in any statistics textbook but tracking it down and computing it by hand every time can be somewhat cumbersome. Here is a short R function to do it easily.

r.cint <- function(r,n,level=.95) {
    z <- 0.5*log((1+r)/(1-r))
    zse <- 1/sqrt(n-3)
    zmin <- z - zse * qnorm((1-level)/2,lower.tail=FALSE)
    zmax <- z + zse * qnorm((1-level)/2,lower.tail=FALSE)  
    return(c((exp(2*zmin)-1)/(exp(2*zmin)+1),(exp(2*zmax)-1)/(exp(2*zmax)+1)))
}

The result can also be used as an hypothesis test by checking if the confidence interval includes 0 or not. The conclusion is very similar but not identical to the tests reported by SPSS CORRELATIONS procedure or R cor.test, because these p values are based on another test statistic (and on the t distribution).

What's the point? As is plain to see from the formulas, the standard error of the z-transformed correlation depends only on the sample size (that's the point of the transformation). At a given level, the width of the confidence interval therefore also depends solely on the sample size. With a very small sample size, the point estimate is going to be very imprecise and even an impressive r can hide a modest correlation, whereas a moderate observed correlation could reflect anything from a small correlation in the other direction to a strong correlation in the same direction. If nothing else, the confidence interval makes this imprecision visible and helps to interpret results based on experiments with a very small number of participants.

Bootstrapping techniques can also be used to construct a confidence interval for a correlation coefficient but they require access to the original data set and cannot be computed based only on typical research articles.

Salmon and voodoo

2009-09-17T22:08:00.000+02:00

If you follow any neuroscience/psychology blog, chances are you already heard of the « puzzingly high correlations in fMRI studies of emotion, personality, and social cognition » paper (ex-voodoo correlation). If you haven't, start with this summary of the debate by the neurocritic and follow the links. On one level, the discussion revolves around the statistical analyses used in some fMRI studies in social neuroscience. It's pretty technical but there is a lot to learn even if you are not into neuroimaging. On another level, the diffusion of the paper on the web and the ensuing discussion on various blogs sparked a debate on peer review and scientific debate. Plenty of food for thought there as well.

Anyway, I just discovered (via the neuroskeptic), that Craig Bennett (of Prefrontal.org) recently presented a poster illustrating how improper statistical analysis can lead to spurious detection of BOLD changes in a dead salmon. This really drives the point home cunningly.

Emotion slider

2009-09-15T18:11:00.000+02:00

Beside some rather unsuccessful attempts at psychophysiological measurement, the best part of the last two years have been occupied by a series of experiments with a device (the “emotion slider”) I developed together with Pieter Desmet (one of my PhD advisors and also the person responsible for the sketches in the paper), Rob Luxen (who never stopped improving the electronics) and Hannah Ottens (who actually built the thing).

I presented a first article about it at the Design Research Society conference in Sheffield last year and I never uploaded it to my website, but I recently noticed that it is now available online. The paper is titled Designing a research tool and describes the development of the device from a design angle.

Since then, I have also written a more classical experimental psychology-type of paper, which I finally presented at ACII last week. Apparently it's not online yet but you can always contact me for more info. In the meantime, you can also download the slides from my presentation.

In a few words the conclusion of all this is that there is evidence of compatibility effects between movements on the slider and affective state (for example how good or bad you feel or how you evaluate a picture). Participants in the experiment were quicker to push to evaluate positive pictures than negative pictures and also quicker to pull to evaluate negative pictures than positive pictures. This difference in response time shows the one set of movements is more intuitive or easier than the other one.

However, unlike what some earlier reports suggested, this effect is very sensitive to the context (how the slider is positioned, what the instructions are, if there is some form of feedback, etc.) With "neutral" instructions (in my case I asked the participants to « push » and « pull » without any other precision) and a slider positioned between the screen and the user, the more natural mapping seems to be pushing for « positive » and pulling for « negative ».

Using the « wrong » movements also seems to have a small but noticeable effect on the number of errors people make but the evidence is not very strong.

Back from ACII 2009

2009-09-13T12:06:00.000+02:00

Since thursday, I have been attending the second « Affective Computing & Intelligent Interaction » in Amsterdam. Generally speaking I was quite impressed by the quality of the research, even if the conclusion often went along the lines of « things are complex, it's difficult ». On a lighter note, the venue was great and everything went smoothly, even if the food and timing were less impressive (keynotes at 8 something in the morning and a conference finishing late on Saturday are tough!)

As far as I am concerned, the last session (« Guidelines for Affective Signal Processing: From Lab to Life ») was the most interesting but many other papers are well worth checking out. A few things that caught my attention are Rana el Kaliouby's emotion recognition system for children with autism spectrum disorders, Dimitry Tseterukou's affective haptics, Elisabeth Eichhorn's Recording Inner Life prototype, Jennifer Robison's paper on the consequences of affective feedback (she got a well-deserved best paper award).

A few of my colleagues from Delft also presented their work: Valentijn Visch had a paper on attribution of emotion by observers based on basic movement parameters and Miguel Bruns Alonso presented the last prototype that came out of his work on tangible interaction and stress reduction.

I don't know if the proceedings are online yet but in the mean time you can check out the conference website and contact the authors directly, most of them are really happy to send out copies of their article when asked.

My brand new blog!

2009-09-13T11:39:00.001+02:00

It's been some time that I first considered opening a blog and I have been thinking from time to time about potential content and cool gadgets, even drawing a few wireframes of the layout I wanted it to have. It never got past this stage but while attending the ACII conference in Amsterdam last week, I really felt that it would be great to have some place to post some comments or present the many interesting things I saw. That's the reason why I put all the plans for a carefully designed website aside and finally decided to simply sign up on blogger, pick a template and click on “create a blog”. It does not even have a real title but that's the way blogging was supposed to be, right?