Try and picture the scene: you’re in a narrow tube in almost complete darkness, there’s a loud thumping noise surrounding you and you’re watching episodes of the 90s sitcom, ‘Home Improvement’, with Tim The Tool Man Taylor and his family. There’s a panic button in case you feel claustrophobic, but it’s all over in less than an hour. It sounds a little surreal, but that’s what it would have been like to be a subject whose functional magnetic resonance imaging (fMRI) brain data was used in last year’s Pittsburgh Brain Analysis Competition.

After you’ve watched three episodes, kindly folk in glasses and white coats would take you out of the scanner bore, give you a glass of water and then over the next few days, they’d ask you to watch those same three episodes again over and over. On the second viewing, they’d ask you ‘How amused are you?’ every couple of seconds. On the third viewing, they’d keep wanting to know how aroused you are on a moment-by-moment basis. Then, ‘Can you see anyone’s face on the screen?’, ‘Is there music playing?’, ‘Are people speaking?’ and so on, until you’ve watched every moment of every episode thirteen times, each time being asked something different about your experience.

Our job, as a team entering the competition, was to try and understand the mapping between your brain data and the subjective experiences you reported. For two of the episodes, we were given your brain data along with the thirteen numbers for every corresponding moment that described your arousal, amusement, whether there were faces on the screen, music playing, people speaking etc. Our team, comprising psychologists, neuroscientists, physicists and engineers, put together a pipeline of algorithms and techniques to whittle down your brain to just the areas we needed and smooth away as much of the noise and complexity as possible. Think of these first two episodes as the ‘training’ data. Then, we were given only the brain data for the third episode, the ‘test’ episode, from which we had to predict the reported experience ratings.

Our predictions were then correlated with the subjects’ actual reports, and we were given a score. We ended up coming second in the whole competition, and we’re hoping for the top spot in 2007. Much of this effort has had a direct payoff for our day-to-day research. We now routinely incorporate a lot of these machine learning techniques when trying to understand the representations used by different neural systems, and how they relate to behavior.

Members of the team: David Blei, Eugene Brevdo, Ronald Bryan, Melissa Carroll, Denis Chigirev, Greg Detre, Andrew Engell, Shannon Hughes, Christopher Moore, Ehren Newman, Ken Norman, Vaidehi Natu, Susan Robison, Greg Stephens, Matt Weber, and David Weiss