Breaking the seal

I’ve wanted to try daily writing for an impossibly long time, but the first words didn’t want to be dragged out.

In my case, they were unstoppered by a day in London. I pinballed from train platforms to coffee shops, oblivious bustle all around me, far away from the furrowed-browed finger-pecking at Memrise HQ. That context-shift provided a firebreak from the quotidian, and I was finally in the mood to mentally roll up my sleeves and rub my hands together.

Writing’s like peeing – once you break the seal, the words just spill forth all evening.

I was able to decant a dozen half-thoughts that I queued up like toy soldiers, to be birthed one by one over the following week. It’s now rather fun to receive a blog post from my previous self every day.

A letter to a prospective grad student

Preface: I wrote this to a friend asking me for advice about whether to embark on a science PhD. At the time of writing I still had more than a year to go – so I could see the summit in the distance, but I was feeling grim about the steep icewall I had to climb to get there.

In retrospect, I think a lot about Jeff Bezos’ advice: don’t be proud of your talents – be proud of the things you really worked hard to achieve. For this reason, I’m more proud of (and glad about) my PhD than anything else I’ve yet done.

—-

It’s hard for me to summarize my thoughts on grad school, perhaps because it varied so much in so many ways. Grad school was wonderful when I was excited about it – for the first few years, there was literally nothing I wanted to do more than talk and think and write and program lab stuff. Every week was filled with new ideas, a sense of progress and discovery, and I bounded into the lab every morning.

I don’t know what changed exactly, but at some point, I started to really lose enthusiasm. I’m perenially stymied by an inability to understand the source of my own motivations, and to make sense of my own emotions. So I don’t really feel like I understand why the joy started to fade. Perhaps because I worked for years on ambitious experiments that didn’t work out. Because I’d been in one place for years. Because I’m a little flighty. Because I thrive in a more competitive or fast-moving jobs. Because really I love AI and computers a little more than brains. Because I wanted to be my own boss. Because I lost confidence. Because I need to feel part of a team working towards a common goal. Because I needed more inter-personal contact with a range of different people. Because I’m not temperamentally suited to be a scientist. Because I need to be in a city. Because I felt obliged to finish it, after investing so much into it, long after I would have left a normal job. Because the specialization necessary can come to seem like a straitjacket. Because I got obsessed with new ideas. I don’t know.

It seems to me that a PhD is the right move if one loves what one’s doing, and one wants to be an academic. Of course, you can’t know for sure in advance that both of those are true. But if you think they might be, then go for it! While I think we have some things in common, I don’t expect the idiosyncracies of my experiences to apply closely to anyone else, so don’t look too closely for parallels to yourself in my issues above.

Right now, starting a company feels like the job I’ve been looking for my whole life, but I wouldn’t have the wherewithal to do it unless I’d been through the last few years.

I don’t know where your path will lead. Like me, I think you get excited about a lot of things, and could happily set off in many different directions, including becoming a great and happy scientist.

This email doesn’t really answer any of your questions. I’m sorry about that – I just don’t want to give advice one way or the other, because I think you’ll make the right choices without my advice, and because you’ll make whatever choices you make into the right ones. You are a lucky guy, in this (technical) sense – http://gregdetre.blogspot.com/2009/10/i-dont-believe-in-luck.html

🙂

Keep me posted.

When I am famous, I will decline interviews

Reading the 5-page staged and glossy magazine interview in a hotel room with a famous actor has always filled me with a peculiar kind of existential dread. There’s something a little horrifying about an hour of conversation in cold type, bereft of the intonation, expression, context and rapport that make anything one says out loud bearable. And at the end of it all, to be distilled, distorted, interpreted and weighed by the pen of a stranger… Who could have the strength of character to read about but not become their own caricature?

In contrast, the last page of the Sunday Times magazine features ‘a life in the day of’ a happy array of personalities and professions. I like the concreteness of a single day as a window into someone else’s micro challenges and achievements. I realize that these days are probably fictionalized composites – but fiction makes for a sweet, concentrated and memorable pill. And at the end of it, there is no distillation, no weighing – just the reality of a daily rhythm.

When I am famous, I will decline interviews.

P.S. That said, I still remember being stopped in my tracks when a fashion photographer relative asked me sweetly ‘what did you today?’ in the midst of my PhD. My day had consisted of:

  • 2 hours debugging a misplaced comma
  • so that I could finish the 3-day long project of rearchitecting my non-parametric statistics to work across-subjects
  • in order to get a better sense of whether results from the latest in a long line of experiments were actually better than chance
  • so that we could tell whether reminding people and distracting them at the same time was causing them to forget
  • to test our computational theory that half-remembering a memory actually weakens it
  • which would have deep implications for our understanding how the brain learns and self-organizes

But really, I’d been comma-hunting, and it seemed hard to fit that into a the kind of response usually expected from ‘what did you do today?’.

The muses are deaf, so speak up

Good thoughts tend to shy away from short walks with a destination. They’re kept at bay by the neuroses and instant replays that circle endlessly like tethered carrion.

Do you want to know the only way I’ve found to think while walking? Talk out loud. Loudly proudly aloud. Feel free to gesticulate. Close your eyes if traffic conditions permit. Tell yourself a story. Don’t use your normal voice.

Why would talking out loud make such a colossal difference? Perhaps because repetition feels explicitly boring out loud, so we avoid re-treading the same paths. Perhaps because full sentences flush and flesh out our half-thoughts? Perhaps because serializing our massively parallel murmur squeezes the thoughts out one at a time with greater velocity, like putting your thumb on a hose.

The effect is so striking that I’ve wondered about potential neuroscientific explanations. It could be that different neural pathways are being activated – perhaps it is only by vocalizing that we recruit speech production areas, or only by hearing our own voice we recruit speech comprehension areas. Or just that there’s less neural juice sluicing down the byways of my mind during my inner monologue, and the extra oomph required to speak gives the thoughts extra vivacity.

The explanation I favour? If I’m going to have to listen to myself, I want to be entertained.

P.S. For best results, wear a hat and learn to talk like Tom Waits.

How can your iPhone make you even more entertaining and interesting than you already are?

Have you ever had a conversation with smart friends that got hung up on some disputed point of fact, or tip of the tongue memory failure? Don’t you just wish someone would step in with the answer to unclog the free flow of ideas and happy banter? Disputes about facts and tip-of-the-tongue feelings *should* be a relic of the 20th century. So there are two things that are remarkable here:

– Through smartphones and search engines, we can marshal thousands of machines to produce the answer in the blink of your mind’s eye.

– But we have to perform that instantaneous incantation with pudgy fingers and a 0.3G internet connection. I challenge anyone to find the name of an actor in under 2 minutes with an iPhone with crappy reception. While those 120 seconds creep past, you’re coldly ignoring your friends, and the conversation is gasping on the table like a naked baby on a spacewalk.

Here’s one technological solution to this social problem:

– At the beginning of the conversation, we all put our iPhones on the table, and fire up the Inforager app.

– Inforager is listening to us, uploading the audio of our conversation to voice-recognizing clouds.

– It runs dozens of google searches continually in the background, displaying result-snippet-bubbles that float past, driven by the whorls and eddies of our conversation. While we’re talking about the beardy guy with the Greek name in The Hangover, a bubble for ‘Zach Galifianakis – IMDb’ looms large, only to be nudged offscreen as we move to debating whether the ‘candied sunchokes’ on the restaurant menu are likely to taste more like sunflowers or artichokes, while the other half of the table engages in a dialog on the nature of catnip.

In other words, the answers to questions we have are being provided in real time in response to our conversation. This frees us up to talk about what matters.

—-

Technical notes:

– If multiple people at the same table were calling Inforager, it would use the multiple sound sources to do a better job of distinguishing voices and improving audio quality.

– Is it possible to use the phone (rather than the 3G connection) to upload the audio data? That would drain the battery much less.

– I made up the name Inforager.

A wiki for spaces. A town anyone can edit. School architecture founded on mnemonic principles

When we think of wikis, we think of text, like the Wikipedia. But this notion of content that anyone can view and anyone can edit has barely unfurled its wings. What if we were to apply it to space?

For instance, imagine growing a World of Warcraft town as a community. Each person could design and improve upon the buildings, fill the walls with graffiti, neighborhoods would define themselves… the ease and pace of iteration might even generate new ideas about town planning.

Alternatively, let’s build on Ed Cooke’s fantastic plan for school architecture in the future [cached]:

Children, well known to be compulsive absorbers of information, crucially learn what they are interested in. Like all animals, they are interested in spaces.

I’d like to see schools’ spatial layout reflect the history of Western culture, and thereby implicitly teach it. A snake-like line of school buildings could begin at one end in Ancient times and run on, in temporally organized fashion, up to the computer science blocks of the present day. Key themes and figures from each epoch could provide the names for classrooms, which could also reflect some of the architecture, customs and furniture of the day.

Because in five years of school, everyone learns every detail of the spatial organisation of the buildings, and because memories always attach to the spaces in which they were first formed, merely attending such a school would give one a wonderfully detailed sense of the history and structure of Western civilisation. And it wouldn’t need to be prescriptive, for one could take advantage of the second source of childrens’ interest – things they have a role in – to redouble the effect. Each year-group could, over the course of five years, reconsider, re-design and re-build one of the twelve epochs/buildings.

Convincing someone to build a school organized on mnemonic principles is going to be tricky. But in the meantime, perhaps schools’ online presence might take the form of a spatial wiki. Students could make changes ranging from decor to naming to overall organization, shaping their online school to their memories and vice versa. We love to deeply inhabit our environment by shaping it – what could be better than exercising our rich faculty for spatial navigation imaginatively?

Master Turkers

Amazon’s Mechanical Turk is an amazing service where one can create a simple task that can be micro-out-sourced to many people over the web, each of whom performs a small parcel of it. For instance, if you wanted 1000 people to highlight faces in photographs, think of synonyms for words, or provide from-the-hip feedback on your website, Mechanical Turk is ideal.

For reasons that are unclear to me, people seem to be willing to work for far below a minimum wage performing pretty dull tasks. As an experimental psychologist, I’m torn between feelings of data lust at the number of participants I could thus thriftily recruit, and concern about the quality of their data. What kind of person is willing to engage in dull tasks that must feel meaningless from a worm’s eye view? Where’s the incentive to do a good job?

It seems to me that there might be a market for tasks that require more effort, skill or thought, for which one would like to be able to cherry pick the participants. For this to work, you’d need a rich reputation scheme to Mechanical Turk, to pick out the Master Turks.

I’m picturing myself in holidays as an undergraduate. If someone was willing to pay more £10/hour (roughly what I was earning as a medical secretary), I (or my more talented peers) would have happily:

  • researched historical facts for a novel
  • proofread a doctoral thesis
  • helped with market research for a business plan
  • written a catchy jingle
  • filmed a youtube video using your product
  • written a program to generate verbal reasoning or arithmetic questions for an exam
  • provided summaries of white papers

You could imagine non-fixed-rate payment schemes, e.g.

  •   a competition where the best submissions divide the spoils
  •   an auction, so that more enjoyable tasks would be bid lower

And, deliciously, you could create a meta peer-review system where other Master Turkers’ task is to rate the submissions you’ve received.

Stack Overflow is going to transform the programming job market by making answering people’s questions satisfying, and then providing a metric of someone’s expertise that will help them land a job.

There have been many precedents of this kind of idea, but it seems strange that none of them have taken off. This feels like a way to demonstrate one’s abilities on potentially interesting tasks that would provide a portfolio of work to supplement a job application.

Brain orchestras and fMRI analyses

[With help from David Weiss]

I spent much of my PhD working on algorithms for making sense of gigabytes of brain data from fMRI scanners, especially on a fairly new approach called Multi-variate Pattern Analysis (MVPA). I want to show you how the MVPA approach is useful for tackling certain kinds of questions.

Think of the brain as a kind of orchestra. You have lots of separate instruments playing at the same time, and you can subdivide them in lots of different ways, e.g.

  • You can subdivide the orchestra into parts by location – the 1st violins, the brass, the percussion etc.
  • Or you could organize them by what they’re doing. Say the 2nd violins, the oboes and the trumpets have the melody, while the clarinets and the tubas have the harmony. [The harps are doing their own thing and the bassoonist is drunk.]

Likewise, there are all kinds of things going on at once in the brain.

  • You can subdivide the brain by location – frontal, temporal, parietal, occipital lobes.
  • Or you could organize the sub-parts by what they’re doing – vision, language, executive control, motor etc.

Let’s go back to thinking about how the multivariate approach differs in the kinds of questions it can address.

Standard univariate analysis is useful if you want to tell which instruments are involved in one case rather than another, e.g.

  • violins are more active in Beethoven than Mozart, but for trumpets it’s the other way around

  vs

  • one part of the brain is more active when looking at houses than faces, but for another part it’s the other way around

In contrast, a multivariate analysis might be useful if you want to know:

  • is this Mozart or Beethoven?
  • is this the brain of someone looking at faces or houses?

Now, let’s introduce one more concept: dimensionality reduction is an attempt to boil down many instruments (or brain regions) into a few key themes/groups:

Take the famous da-da-da-dum of Beethoven’s Fifth, where the entire orchestra is one voice – one could more or less describe the entire orchestra’s activity in terms of just one theme/process. In contrast, for Bach or something more complex and interwoven, it might be very hard to summarize what’s going in with less than 10 themes.

Likewise, maybe it’s straightforward to summarize the brain’s activity with just one or two processes when you’re doing a very simple task like looking at faces vs houses, but if you’re doing something more complicated (like watching a movie) then multiple processes are interacting in complex ways.

David Weiss‘s PACA algorithm boils down the brain’s activity over time into just a few themes. Once you’ve summarized the 50,000 readouts we get from fMRI every few seconds into 50, it’s much more feasible to try and compare different cognitive processes – just as it’s much easier to compare Mozart and Beethoven by looking at the scores of a few key instruments than looking at the full orchestral scores.

PACA was inspired by a bunch of existing dimensionality reduction algorithms that could equally be applied to problems like voice, face or handwriting recognition.

But its magic involves adding a few constraints that are particularly relevant to the brain. Here’s one example of a constraint: it doesn’t allow its estimate of a theme’s presence at a given moment to go below zero. Think of it like this – when was the last time you heard an anti-violin? Or had an anti-thought? In other words, PACA breaks the manifold streams of activity in the brain down to just a few that are all present to a greater or lesser degree at each moment.

P.S. If you hated this, you might also hate How to beat an fMRI lie detector.

Auto-links

Most wikis require you to perform one of two contortions to create a link:

  • Use CamelCase. Much like a camel, this is robust, but tiring to finger.
  • Wrap things in [“symbols that are hard to type”].

In both cases, you need to know in advance that you plan to create a link, and be enough of a disciplined philistine to overcome the effort and overlook the ugliness.

Auto-links are the solution [1] – here’s how they work. Say you create a page called ‘Camel case’. Now, type Camel case anywhere else, and that ‘Camel case’ text will be turned into an auto-link as you go. In other words, the wiki notices that you’ve typed the name of an existing page in the midst of your text, and automatically creates a link for you. If you go back and edit the text, the link goes away. [2]

Links between pages become evident to readers without any extra effort on the part of the writer. If I type ‘MySQL’ and an auto-link appears, it’s easy to see that a relevant page about it already exists.

Having used such a system for a long time, I have come to appreciate the tiny flash of satisfaction at seeing a link appear with no extra effort, confirming that the page does indeed exist [3], and making navigation while editing a breeze. Pages that I wrote years ago are now festooned with links to pages that were created long afterwards. Indeed, the most satisfying feeling of all is when an auto-link pops up to a page I’d forgotten I wrote. Lazy serendipity!

[1] see Per Sederberg‘s implementation in Emacs Freex mode, though we called them ‘implicit links’ back then

[2] To do this the way God intended requires running a regex containing all the pagetitles in your wiki over what you type on every keystroke – this is very nearly instantaneous for even 10k documents.

[3] For extra points, allow pages to have multiple aliases, so that (for instance) ‘database’, ‘databases’ and ‘MySQL’ all point to the same page.