“We object to the claim that brain games offer consumers a scientifically grounded avenue to reduce or reverse cognitive decline when there is no compelling scientific evidence to date that they do. The promise of a magic bullet detracts from the best evidence to date, which is that cognitive health in old age reflects the long-term effects of healthy, engaged lifestyles. In the judgment of the signatories below, exaggerated and misleading claims exploit the anxieties of older adults about impending cognitive decline. We encourage continued careful research and validation in this field.”

Couldn’t have said it better myself.

Read the full statement here.
The complete list of signatories can be found on the press release.
Don’t forget the prefrontal.org post from last month on brain training.

Dr. Merzenich gave a great 80-minute talk to a crowded room. His talk was pretty similar to others that I have watched on YouTube, so if you are curious about his research you might head over there. Here is a link to his Google TechTalk, which was quite good: http://www.youtube.com/watch?v=UyPrL0cmJRs.

I had a whole series of points ready to bring up to try and foster some lively discussion during the panel. Due to time issues the panel was only about 20-25 minutes long. This didn’t leave much room for lively discussion after introductions were done. Good material for a blog post though…

What follows are some elaborations on the points that I had prepared for the response panel. They are all my opinion, occasionally backed by specific citations and research. They are deliberately intended to be controversial, which will hopefully make for interesting reading.

* Our brain function and structure certainly reflect their environment

Developmental psychologists and neuroscientists have known for some time that the brain was plastic and responds to its environment. After all, we research a period of life that is highly plastic, with a four-year-old brain having 10x as many connections as it will later in life. Changes in the brain of a child are sculpted by a mixture of genetic predisposition and environmental context. That is, some changes are hard-wired and will happen no matter what, while other changes are in response to the environment.

Even later in life it is recognized that the brain can change its structure and function and adapt to its surroundings. Among others, milestone neuroimaging studies investigating structural differences in the brains of London taxi drivers (Maguire et al., 2000) and changes following learning to juggle (Boyke et al., 2008) validated everything that we already knew: the brain is not a static organ. In that context brain training isn’t a revolution in our understanding of the brain, but a new method to mold that plasticity in the direction of our choosing.

* Everything, even your breakfast, changes your brain

The vast majority of functional neuroimaging is done with a defined cognitive task. The investigator will look at the relative difference in activity between different conditions of the task to determine indices of activity. A more recent investigative method, resting state imaging, is different. The subject is given no instructions, other than to sit quietly in the scanner and clear their head to the best of their ability. From the data collected during this scan you can begin to look at which brain areas have correlated activity, indicating that they tend to work together. By doing scans before and after a specific intervention,you can then learn about which functional networks are affected by treatment. So, what have we learned from these studies?

Everything changes the brain. Everything.

The Neuroskeptic had a great argument in a recent blog post, which is that even the act of opening your eyes can have a dramatic influence on the state of the brain. This can be seen even in commodity EEG equipment as a change in power in the alpha band. He was discussing a recent study that found a single dose of an antidepressant can have a significant effect on functional connectivity in the brain (Schaefer et al., 2014).

The challenge to neuroscientists is to separate the wheat from the chaff. If everything changes the brain, then which changes are actually relevant? Much of the brain training research does a good job of this, looking for measurable changes in behavior following training. As for the rest of neuroimaging, well, I have a feeling that we are going to see a lot more “crossing the street changes the brain” type of articles…

* Cognitive capacity is a mix of genetic and environmental factors

There seems to be an approximate 50/50 split in terms of what determines your intelligence. About half comes from your parents in the form of your DNA. The other half comes from the sum total of all experiences you have ever had in your life, including brain training. This is great, because it means that we have a lot of leeway to push our cognitive abilities around. Make positive changes and you will see improvements, make negative changes and you will see degradation. A second point should be made here though, which is that each of us carries genetic predispositions that will be difficult or impossible to change through training.

One extreme example of genetic factors at play is in psychopathy. There is evidence that it is difficult or impossible to rehabilitate true clinical psychopaths. I never got a chance to ask Dr. Merzenich about this counterexample. I have to believe that he would say that even clinical psychopaths could benefit from brain training. I have a different view, which is that a certain percentage of our cognition is hard-wired, and may or may not be able to be changed. Some of us have burdens that cannot be trained away.

* Individual differences can create very different training outcomes

We typically look at the results of an fMRI study, see the blobs, and immediately assume that the entire group of research subjects must have been using those areas of the brain. This is somewhat true in terms of the statistics, but there is a richer story to be understood. First, some subjects do not show increases in activity, and some even show reductions in activity! Further, the patterns of activity across the rest of the brain will often be wildly different from one individual to another. My postdoc advisor, Michael Miller, has done a lot of research on this, going so far as to call each person’s pattern of activity a “neural fingerprint”.

What does this mean for brain training? Well, if everyone is using a highly varied mix of functional networks as they navigate a cognitive task then we would likely expect to see a highly varied response to brain training as a consequence. For example, if you leverage executive working memory to a greater degree than I do while completing a task and we then complete a working memory brain training program then you would likely demonstrate a greater benefit than I would.

One issue I had with Dr. Merzenich’s talk was his assertion that every ability could be improved at any age. This is a very strong argument. He cited an animal study which showed that cognitive training in rats led to a range of behavioral and physical improvements, right down to improved immune function. Knowing that each individual human could have a different response to each form of brain training, I would put forward that most cognitive abilities could be improved using brain training. I would further argue that there will be a varied generalization of brain training to other tasks that remain untrained.

* The relevant networks for training might be counter-intuitive

It will not always be immediately apparent which functional networks in the brain should be targeted with brain training to derive a practical cognitive benefit. This will be particularly true as you try to train cognitive abilities that lie higher and higher in the hierarchy of function. For example, how would you train an adolescent to better handle risky situations? The common refrain here would be to train inhibitory control, so that the teen could better inhibit a prepotent response. That may have some benefit, but evidence is accumulating that one representation of social risk is modeled by an ‘as-if’ loop, which attempts to predict body and emotional state following a decision (Preuschoff et al., 2008). It may be that something like mindfulness meditation may be more beneficial here relative to brain training (but I am only guessing).

* Why aren’t more people using brain training?

Dr. Merzenich suggested that there were immediate benefits following brain training that lasted for years, transferred to other cognitive abilities, and had a significant effect size. He argued that brain training was the logical path to enhanced cognitive capacity and cognitive rehabilitation. Sounds persuasive, yes? If all of that is true, then why isn’t every hospital and school clamoring for access to the training programs? Insurance companies aren’t fools, contrary to the opinion of most Americans. If they can obtain the same benefit from $1k of brain training instead of $10k of traditional treatment, do you not think that they would be pushing doctors to prescribe it? What’s the hangup?

* Social interaction as a highly complex cognitive task

I thought that one subject that didn’t get a lot of play in Dr. Merzenich’s talk was the cognitive requirements of social interaction. Social interaction is not frequently viewed as a highly complex cognitive task. That is largely because we are so damned good at it. You can model the current internal state of multiple individuals while simultaneously predicting how that internal state will change based on some tidbit of information that you are about to communicate. That is simply incredible. There is a reason that it takes us over two decades to become ‘adult’ in terms of our behavior, and a large part of that is having the processing capacity and social experience necessary to interact with others as an adult. It takes way more than 10,000 hours…

I bring this up mostly because I think that social interaction is a critical part of our daily lives, and I see it as critical during the rehabilitation process as well. Which patient has a better overall outcome: the one who does brain training alone at home twice a week or the one who sees a therapist for training twice a week? My money is on the latter. I say this not only for the obvious emotional benefit of social interaction, but because navigating the social landscape is one of the most cognitively difficult tasks that we, as humans, accomplish.

* The need to define a ‘challenging’ cognitive task

I think that much of the benefit that comes from the brain training has relatively little to do with the tasks that are presented and more to do with the progressive difficulty of the software that drives them. One major point in Dr. Merzenich’s talk that deserved additional attention was that there was very little benefit to be had with brain training if the participant is not adequately challenged. That means that the task cannot be too difficult, causing frustration, and cannot be not too easy, meaning no benefit. Instead, you want the participant to get the majority of trials correct (~80%) and still have room to grow.

When I was in grad school my roommate dropped me in an MRI scanner for 2.5 hours to complete a mildly complex task. To answer your questions, yes, I did that task for over two hours and, no, it was not fun at all. What the data showed at the end was fairly amazing though. At first, when the task was difficult, I had to utilize a range of frontal and parietal resources to complete the task. It took effort. Two hours later, when I didn’t even have to think about the task anymore, the majority of activity was subcortical. It had become automatic.

We may not get a lot of cognitive benefit out of social interaction because we are already experts in social interaction. Likewise, when you first start doing crosswords everyday you will likely see a cognitive bump, but as you get better at crossword puzzles the marginal increase you see will diminish. This is the secret sauce of brain training. If you had a set of crossword puzzles that increased in difficulty as your skills improved I think that you would continue to observe improvements over time.

* Brain training seems to have mixed results

First, I haven’t read the entire body of literature on brain training, so I am limited to the handful of papers that I have reviewed. One consistent thread that I have observed is that the training can offer improvements that confer long-term benefits on a scale of many years. Still, the effect size of the studies I reviewed isn’t always huge.

A recent study by Wolinsky et al. (2013), showed an effect size of around 0.25 for most of their training, as indexed by the Cohen’s d statistic. This was across several experimental conditions one year after the initial training.

Another, larger study from the NIH found that cognitive benefits can stay around for over a decade. The Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE) study followed 2,832 participants for ten years following an initial six-week brain training program (Rebok et al., 2014). After ten years approximately 70% of subjects in the training group were above their initial performance baseline. This is relative to 50-60% in the control group. The effect sizes varied depending on the condition, but ranged from approximately 0.20 to 0.50.

Head on over to the Cohen’s d effect size interpreter at http://rpsychologist.com/d3/cohend/ and look at the overlap of the treatment and control group distributions for a d of 0.2. Yeah, there is a benefit, but there is not a dramatic shift. Effect sizes of 0.5, which were observed in the ACTIVE study, are better. Still, you are not jumping several standard deviations up the curve.

The authors of both studies discus that the practical effect of brain training is a 2-7 year delay in certain aspects of cognitive decline. Again, not super huge. You are not turning a 70-year-old brain into that of a 30-year-old. Still, when I am 70 I will take any improvement I can get.

* Exercise and nutrition as global cognitive benefits

It is no secret that I am a huge fan of exercise and mindfulness meditation as ways to improve your brain. Many of the points in my impromptu post on running still ring true, and there is an increasing body of evidence that exercise, nutrition, meditation, sleep, and medication can improve the function of the brain. Dr. Merzenich accepted that these had positive influences, and mentioned them early in his talk. Still, he was somewhat dismissive of their ability to significantly improve cognition or aid in rehabilitation, saying that they formed part of a “complete package” of rehabilitation.

I took strong issue with his stance on this issue. He stated that exercise and nutrition provide “essentially no benefit” relative to brain training on the tasks that they had investigated. I have a hard time completely believing that, especially given the range of benefits that have been published in scientific literature.

* Warning against false prophets of brain training

Brain training has a huge buzz right now, and a number of startups have been founded provide brain training tools. PositScience is not alone in a market that includes Lumosity, Cogmed, CogniFit, LearningRx, and many others. Which ones really work?

How many programs are just peddling mildly difficult games with an advertisement that it improves the brain? I sleep better at night knowing that Dr. Merzenich is the Chief Scientific Officer of PositScience, but how many companies do not have a solid scientist at the helm, or the research to back up their claims?

* Realtime neuroimaging techniques could play a role in the future

Brain training certainly has some effect. Further, there is very little downside to using brain training tools. The only possible negative I could see is the opportunity cost. We all have a finite amount of time during the day, and if you are in your back office engaged in brain training instead of exercising or interacting socially then you may be missing out on other ways to improve your cognitive function.

I am most excited about the market for realtime feedback on cognitive performance based on neuroimaging data. Christopher deCharms has been doing a bit of this using realtime fMRI to help control pain in patients. He founded a startup, Omneuron, to investigate this technique and capitalize on it. What I would like to see is a consumer-grade EEG system (like the Emotiv headgear) that integrates realtime brain state with brain training. I think that this is really the Holy Grail, because you are no longer basing the progressive difficulty of a task on the behavior of the trainee, but instead you could potentially increase difficulty based on the actual cognitive load.

Perhaps it is time for me to start up a company of my own…

I tested the prefrontal.org server and found that it was indeed vulnerable. I took this as an opportunity to start fresh, and updated to a new version of Debian Linux on a new server. Let me know if anything remains broken, but we should be good to go at this point.

If you are anywhere close to Santa Barbara on Monday, September 22 I would highly encourage you to swing by and see Dr. Merzenich’s presentation. He is a world-renowned expert on brain plasticity, particularly in the context of cognitive rehabilitation. I have the honor of serving on a response panel following the lecture, which I am particularly looking forward to. It should be a really fun evening.

See the below web pages for more information on the event:
http://www.sbch.org/
http://www.eventbrite.com/

Hope to see you there!

Well, between you and me, there is a perfectly good reason why that is the case.

That was about the time I completed the paperwork to end my neuroscience career and begin a new adventure in private industry as a software engineer.

Wait, what?

The short version is that my wife and I have decided to make Santa Barbara, California our permanent home. Given that the postdoc is a temporary position usually lasting less than five years, the time was coming soon to move on up to being a professor or move on out of academia. After much deliberation, I have chosen the latter.

I opted not to announce the transition publicly for some time in case I changed my mind, or if the new position didn’t work out. I figured I could always head back to research as long as nobody noticed I had temporarily left. Now, here we are, almost two years later and I have neglected to let you, my closest internet-friends, in on the big news. How terrible!

Some important factors in the decision:

(a) Funding. There were plans afoot for me to make the transition from postdoctoral researcher to research faculty. This would have potentially kept the lights on in the Bennett household for many years to come. I easily could have enjoyed a long career focused uniquely on research in cognitive neuroscience. The only risk? Funding. Funding rates have been dropping year over year for decades. The average age at which a new investigator finally gets their first R01 grant has risen to 43+. The percent of funded research project grants has fallen from 32% in 1999 to 24% in 2007. The success rate for funding on your first grant submission is now 12%. [see Broken-Pipeline.pdf]

It’s not that I can’t handle the continuous funding chase, but it is a bitter struggle. The situation isn’t getting better either. Would you want to place a bet on whether success rates are going to fall below 20% or rise back to 30% as we move forward? Further, if a grant falls through, and there happens to be a gap in funding coverage, then my paycheck would simply cease to exist. Working for a private employer may be ‘at will’ employment, but the chances of a negative outcome may be far lower than rolling the dice with the NIH, NSF, or other funding agency every few years.

The lab that I was working in experienced just such a shortfall this last year as the federal sequester hit the campus.  The lab was funded primarily by funds that, filtered through other agencies, came from the U.S. Army.  When the sequester arrived, lab funding took a double-digit drop.  My postdoc advisor was hard pressed to pay people’s salaries.  Data acquisition stopped and everyone just had to hope that they had enough quality data to publish/graduate.  That is a scary proposition.

(c) Compensation. The postdoctoral researcher is seen as a training position where you are learning important new skills and scientific perspectives from the principal investigator that you work for. Based on the salary data I collected as part of my job search, postdocs are typically getting paid a (literal) fraction of their value in private industry. This, again, tends to get better when you become an assistant professor, but it can take a while to climb the ladder.  I would have received a bit of a bump if I would have gone into a research faculty position, but it paled in comparison to the, um, significant increase I negotiated as part of my new software development position.

(b) Lifestyle. Part and parcel of academia is the fact that everyone is constantly moving around. Your best friend today may or may not be in the same town with you next year. While this stabilizes a bit when/if you finally land that coveted assistant professor position, the time leading up to that point is defined by flittering around the country. First you head off to grad school, then off to a postdoc, then to who knows where. I know some individuals who are on their fourth city in ten years, with another move on the way. For some this is exhilarating, as you are essentially getting paid to see the world. For me it has been harder, constantly making incredible friends and then having to say goodbye. I now have friends stretching from Washington, to Maine, to Florida, and back to California.  It was finally time to stop flying around and put down some roots.

The other lifestyle point is that my wife and I have really fallen in love with Santa Barbara, California. The weather is amazing. We are able to live two blocks from the beach and an eight minute drive to hiking in the mountains. I am able to commute by bike almost every day of the year. These geographical and climate features are fantastic, but beyond that is the fact that we have met some amazing people in Santa Barbara, and it just feels like home now. I don’t want to say goodbye, and now I don’t have to.

Things aren’t all bad as a postdoc, however. I would be remiss if I didn’t highlight some incredible benefits of the old job:

(a) Schedule. Being able to set your own schedule is an incredible benefit of the postdoc. I am a nightowl by nature, so working 10am-6pm with another hour or two of reading in the late evening suited me perfectly. Working an 8-5pm job is not really my cup of tea. The flexibility to take time away from work was also fantastic as a postdoc.  I had some genuinely awesome travel and volunteer experiences that I might have never experienced in a regular job, including trips to Hawaii and several excursions to Europe. Now I have to hoard my ATO days like they are bars of gold.

(b) People. I have met some of the most genuinely awesome people in my years as a researcher. There is something truly magical about a group of people who have similar interests and come together for a common cause. I found that special kinship in graduate school, my postdoc, and also with the larger community of cognitive neuroscientists. Even though I am leaving neuroscience professionally, I will always identify with this cadre of individuals who brand themselves ‘neuroscience geeks’. While there is some of this kinship in my new position, there is an overriding sense that many folks are just punching the clock.

(c) Academic Freedom. You are at the mercy of your postdoctoral advisor to a point, but at the end of the day you have an awesome job where, for all intents and purposes, you are getting paid to think. How absolutely freaking incredible is that? You know that one thing that you are most curious about? Here is some money to learn everything you can about that one thing and then here is some more money to find out the answers to questions we don’t even know to ask yet. I won’t lie to you – I miss it all. Frequently.

So, what am I doing now?

Well, I am working for a small scientific instrumentation company in Goleta, California that goes by the name [redacted]. Well, I could tell you their name, but I would need to obtain prior approval from my corporate overlords (no joke). I work as a firmware engineer in the software group, helping to maintain the embedded applications that drive the instruments. As a neuroscientist, the best part of my week was spending an entire afternoon writing a gnarly MATLAB script. Now, I get to do that every day. I could not have hoped for a better place to end up after falling out of the Ivory Tower.

I have had a really, really great run as a professional neuroscience researcher. Great friends. Great research. Amazing conferences. Deep debates on the nature of of the brain and cognition. An Ig Nobel award! I credit any success I might have had to the wonderful mentors who took me under their wing, and to the long list of quality people I have encountered on my academic journey. I love cognitive neuroscience and scientific exploration more than I can adequately put into words. Still, I was not content to let the sunk costs of a graduate education dictate my future actions, especially when other options are now a better fit for my personal and professional goals.

~Craig [Prefrontal]

“He will be mourned by his families and friends; he will be mourned by his nation; he will be mourned by the people of the world; he will be mourned by a Mother Earth that dared send one of her sons into the unknown.

In their exploration, they stirred the people of the world to feel as one; in his death, he binds more tightly the brotherhood of man.

In ancient days, men looked at stars and saw their heroes in the constellations. In modern times, we do much the same, but our heroes are epic men of flesh and blood.

Others will follow, and surely find their way home. Man’s search will not be denied. But he was the first, and he will remain the foremost in our hearts.

For every human being who looks up at the moon in the nights to come will know that there is some corner of another world that is forever mankind.”

– Nixon, in a speech prepared if Armstrong/Aldrin were stranded on the Moon’s surface. It has been paraphrased to be singular instead of plural.

The original text: http://www.lettersofnote.com/2010/11/in-event-of-moon-disaster.html

The best humor always has a touch of reality. The comic is from the site Saturday Morning Breakfast Cereal – check it out!

Introduction:

The fMRI Data Center (fMRIDC) was founded as a large-scale repository for functional neuroimaging datasets from around the world. Since its inception the archive has grown to hold 122 fMRI datasets from a diverse array of cognitive domains. For years these datasets have been made available at no cost to any interested party. Within the last 12 months there have been 543 requests for 725 datasets coming from a mix of 60% domestic and 40% international sources. The goal of this project was to investigate data quality across the entire fMRIDC archive by holding each study up to the same stringent examination criteria. We hoped to determine what percent of studies could adequately be reused in a larger meta-analysis of functional imaging data.

Methods:

We examined each of the 122 datasets contained in the fMRIDC archive. Initial criteria for inclusion required a dataset to contain functional MRI data in normal human volunteers. This eliminated all studies with only anatomical data, nonhuman data, and all clinical datasets. Further criteria for inclusion required datasets to have whole-brain coverage, no anomalous signal dropouts, no severe MR artifacts, and a minimum group size of 8 subjects. This eliminated all studies with gross data quality problems. It should be noted that only studies with data problems across all subjects were excluded on this basis. A single subject with bad data would not lead to disqualification.

Results:

Across all datasets we found that 48% of studies in the fMRIDC archive had issues that prevented their reanalysis. The most likely reason for exclusion was missing fMRI data (19 studies), with the second most likely reason being missing study metadata (11 studies). These issues have nothing to do with the data themselves, but center around problems related to acquisition of the data in a complete set. Other issues we found that would prevent the reuse of data included incomplete brain coverage (9 studies), corrupt/blank data (6 studies), data with severe visible artifact (6 studies), experiments with less than 8 subjects (5 studies), nonhuman data (2 studies), and experiments with only anatomical data (1 study).

Conclusions:

This project represents the first step in understanding how data quality varies across a large sample of fMRI studies. From this analysis we can conclude that only about half of the studies met our criteria for further reanalysis. Still, the figure of 48% should not be taken as an indicator of quality across all fMRI experiments in the literature. The vast majority of issues had to do with the challenge of acquiring datasets and study metadata from the original authors.

Steve Jobs died today. I found out while I was on the bus as I came home from work.

He was a technology pioneer to be sure. Certainly one of the most effective CEOs to come around since the title was invented. Through his leadership a stream of amazing and beautiful devices were released to the public, turning Apple from a company on the verge of bankruptcy to one of the most profitable in the United States. From his early work on the Apple I to the wildly successful iPhone 4, he revolutionized the daily life of billions of people around the world.

I felt a strong feeling of loss when I heard that he had died. It came from the untimely departure of a man who I had never met, but nevertheless saw fit to draw personal inspiration from.

Why was I drawn to Steve Jobs? It was his idea that all details matter, even down to the individual pixel. It was the notion that even the intangible minutiae will impact our perception of an object, like the exact radius of a corner or the amount of friction on a piece of glass. It was the mandate that you aren’t finished until you pour a piece of your soul into your creation.

So many of my own greatest accomplishments have been done using tools that once existed only in his mind. Steve Jobs made me want to be a better creator, and a better person.

Before I heard the news I had spent the afternoon working through a book on Objective-C, the programming language used in the creation of Mac, iPhone, and iPad applications. I got an itch to do some OS X programming the night before, but I needed a refresher on the syntax of the language to get going again. In hindsight, I can think of no better tribute to the man than spending the day becoming a better programmer, honing my skills to one day create something insanely great.

While I was on the bus I downloaded his Stanford Commencement address and listened to it again with new perspective. One passage struck me in particular:

Remembering that I'll be dead soon is the most important tool I've ever encountered to help me make the big choices in life. Because almost everything — all external expectations, all pride, all fear of embarrassment or failure - these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked. There is no reason not to follow your heart.- Steve Jobs, Stanford Commencement, 2005

Stay Hungry. Stay Foolish. Thanks Steve.

Above Image: My first computer, an Apple //c. I am pretty sure that Steve didn’t have a hand in how it looked.

Full text of the Stanford Commencement:
http://news.stanford.edu/news/2005/june15/jobs-061505.html

Read stories on the creation of the Macintosh:
http://folklore.org/StoryView.py?project=Macintosh&story=More_Like_A_Porsche.txt
http://folklore.org/

Andy Ihnatko’s remembrance:
http://ihnatko.com/2011/10/05/steve-jobs/

Walt Mossberg’s remembrance:
http://allthingsd.com/20111005/the-steve-jobs-i-knew/?mod=tweet
 

Summary:

A branding consultant (Martin Lindstrom) commissions a neuromarketing company (MindSign) to do a neuroimaging study. Sixteen subjects underwent fMRI data acquisition while being shown audio and video of ringing iPhones. Visual and auditory cortex was active across all conditions. There was also activity in the insula. The authors interpret the sensory cortex activity as a kind of cross-modal synesthesia experience. The authors further interpret the insula activity as the subjects experiencing feelings of love and compassion. Headlines around the web ring loudly with headlines “YOU LOVE YOUR iPHONE”.

Web points of interest:

1) Read the original op-ed piece by Martin Lindstrom to give yourself some context regarding what was said and the arguments that were made. It will probably make your skin crawl with tales of babies wanting cell phones to be iPhones and terrible definitions of synesthesia. Stick with it anyway.
http://www.nytimes.com/2011/10/01/opinion/you-love-your-iphone-literally.html

2) Start at Russ Poldrack’s weblog and read his first post on the topic. He called it crap, and he was being direct and truthful.
http://www.russpoldrack.org/2011/10/nyt-editorial-fmri-complete-crap.html

3) Now read Tal Yarkoni’s excellent in-depth discussion of the problem. If you read nothing else today, go and check this one out.
http://www.talyarkoni.org/blog/2011/10/01/the-new-york-times-blows-it-big-time-on-brain-imaging/

4) Next, read the post by Vaughan Bell at Mind Hacks, which is also a nice follow-up. Double points for using the term “facepalm jamboree”.
http://mindhacks.com/2011/10/02/the-new-york-times-wees-itself-in-public/

5) Finally, see the list of people who support Poldrack’s position on the Lindstrom article. Many of the best minds in neuroscience are agreed that the Op-Ed piece is not representative of good science:
http://www.russpoldrack.org/2011/10/signers-of-letter-to-editor-of-new-york.html

To be honest, I don’t have a whole lot to add to the conversation. On the topic of reverse inference you really can’t do better than Russ Poldrack and Tal Yarkoni. The Yarkoni blog post is particularly good, effectively nuking the Lindstrom piece from orbit. It is, in a way, poetic since Poldrack and Yarkoni are working on the databases and methods that will enable probabilities to be put on arguments such as Lindstrom’s. That is, if insula activation is observed how likely is it that the emotion of ‘love’ is being experienced. To give their technology a try surf on over to http://neurosynth.org/ and check it out.

One aspect of the debate that I am particularly interested in is the purported role of the insula in the experience of love and affection. Unfortunately, Lindstrom provided very little detail in terms of the spatial location of their insula activity, effectively preventing anyone from criticizing the work on that basis. But, for the sake of argument, let’s put the insular question forward. Does it matter where in the insula that the activity was observed? The short answer is: absolutely.

There is an excellent paper by A. D. “Bud” Craig entitled “Forebrain emotional asymmetry: a neuroanatomical basis?” that details how the left and right insula have a different pattern of connectivity to the homeostatic afferents that provide information on our current body state. Craig describes how the right insula is preferentially involved in sympathetic nervous system activity geared toward engaging with the environment, energy use, and even “fight or flight” responses. Conversely, the left insula is preferentially involved in parasympathetic activity geared toward contentment, energy conservation, and “rest and digest” responses.

In our evolution, humans seem to have bolted-on social components to this underlying insular emotional asymmetry. The right insula seems to be associated with the experience of social disgust and social avoidance. This has been seen in work such as the original Philips et al. (1997) paper, showing prominent right anterior insula activity during disgust. The left insula seems to be associated with the experience of social compassion and social approach. There is less evidence for this, but meta-analyses such as Ortigue et al. (2010) have reported this pattern.

In short, leaving out which hemisphere the results occurred in is a huge faux pax on the part of Lindstrom. It is not the greatest sin of the piece, and probably not even the greatest sin of the insula argument. Still, it is certainly a prominent FAIL from the perspective of a researcher with an interest in the insula.

One final point of discussion I would like to raise is with regard to an earlier prefrontal.org post on the Seven Sins of Neuromarketing. Let’s see which ones are most prominent in the current discussion:

1) The curtain of proprietary analysis methods limits our knowledge of how effective neuromarketing can be.

We have no idea what methods Lindstrom and his colleagues used to arrive at their findings. It could be the best study in the history of ever, or it could be riddled with common statistical flaws. We have no idea because the work isn’t peer-reviewed. As before, we don’t even know where in the insula the results were located!

3) Most people’s introduction to neuromarketing is through press releases, not peer-reviewed studies.

Let’s just establish this as a rule: the New York Times editorial page is not the right place to introduce the world to your cutting-edge, unproven fMRI methods. Period. In fact, we should come up with a verb for what always happens afterward: you get Poldrack’d.

4) Neuromarketing methods are not immune to subjectivity and bias.

In a way, scientific claims are guilty until proven innocent by empirical evidence. Honestly, can I trust a man who has written books with titles like Buyology, Brandwashed, and Brand Sense to be objective with regard to a neuromarketing study with a sensational headline? If this was work was peer-reviewed then we could evaluate his evidence in a balanced manner, but an Op-Ed piece does not allow for this luxury and leaves the question of bias open.

6) People are rushing the field to make a quick buck, and not everyone is trustworthy.

I think that this represents the case in point.

Ortigue S, Bianchi-Demicheli F, Patel N, Frum C, Lewis JW. (2010). Neuroimaging of love: fMRI meta-analysis evidence toward new perspectives in sexual medicine. J Sex Med. 7(11): 3541-3552.

Phillips ML, Young AW, Senior C, Brammer M, Andrew C, Calder AJ, Bullmore ET, Perrett DI, Rowland D, Williams SC, Gray JA, David AS. (1997). A specific neural substrate for perceiving facial expressions of disgust. Nature. 389(6650): 495-498.
 

There is a great article by Nieuwenhuis, Forstmann, and Wagenmakers in this month’s edition of Nature Neuroscience. Entitled “Erroneous analyses of interactions in neuroscience: a problem of significance”, the paper discusses the problem of how to gauge when two effects differ in neuroscience. It turns out that many papers misjudge the difference between effects by basing their judgement on significance values, even though they should know better.

The crux of the issue is that it is improper to judge the difference between two effects by looking at their relative significance. The perceived difference between a significant effect ( i.e. p < 0.05) and non-significant effect (i.e. p > 0.05) does not necessarily mean that the two effects are themselves significantly different. You have to explicitly test for that.

In fMRI, this could mean relating one brain area that is significant to another brain area that is not significant. The temptation is to discuss the significant region as being more active than the nonsignificant region based on the fact that the latter region was below the significance threshold. This actually may or may not be the case.

Andrew Gelman and Hal Stern wrote a similar article on the problem a few years ago. The focus of their piece was simply to draw attention to the issue through the use of several theoretical and real life examples. While they were able to say that the problem existed, they were unable to say how prevalent the problem was across any particular scientific discipline. The power of the Nieuwenhuis, Forstmann, and Wagenmakers paper is that it extends the Gelman & Stern work through an analysis of the existing literature to put concrete numbers on how widespread the problem is in neuroscience.

The authors conducted a survey of 513 articles in major neuroscience journals. They identified 157 papers containing an analysis where the authors would be tempted to make an inferential error by focusing on significance. They found that in 78 out of 157 cases (50%) the authors did indeed make an error. That is far higher that I would have guessed, and one of the reasons I felt compelled to write about it today. I mean, come on, fifty percent? Really?

In the next to last paragraph the authors specifically state the the error of comparing significance levels is particularly acute in neuroimaging. From my perspective we are almost setup for failure in this regard, as significant regions are visualized as a range of attention-grabbing colors while regions that are not significant are visualized as completely blank.

I could rail on a bit longer, but that is time you could be using to go and read this article. There is a lot of good information in the text – it is short, punchy, and well worth your time.

Some additional discussion on the topic:
http://andrewgelman.com/2011/09/the-difference-between-significant-and-not-significant/

Gelman A and Stern H. (2006). The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant. The American Statistician 60(4), 328-331.

Nieuwenhuis S, Forstmann BU, and Wagenmakers EJ. (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nature Neuroscience 14, 1105-1107.

Research Participants Wanted

The Human Memory and Neuroimaging Lab in the Department of Psychological and Brain Sciences at UCSB is seeking research participants for a functional magnetic resonance imaging (fMRI) study investigating the relationship between various personality and cognitive factors and memory. The study will take place on two separate days and will last about two hours each day. Participants will respond to questionnaires, complete cognitive tests, and have their brain activity measured using fMRI. Participants will be compensated with $20/hour and will receive an image of their brain.

To be eligible, participants must:
• Be between the ages of 25 and 35
• Be native English-speakers
• Not be pregnant
• Not have any metal in their bodies that cannot be removed
• Not be claustrophobic

Please email ucsbmemorylab@gmail.com or call (805) 283-9603 for more info.

Click here to read a PDF of our recruiting flyer.

The Stanford Interdisciplinary Group on Neuroscience and Society (SIGNS) is hosting the debate, which is focused on neuroscience in the marketplace. Jim Sullivan, the CEO of NeuroSky, Uma Karmarkar from the Stanford Graduate School of Business, and myself will all weigh in on the topic of whether neuroscience is being used to manipulate consumers.

I think you might already know where I stand based on my ‘Seven Sins’ neuromarketing post, but the event promises to be a lively affair with a diverse array of perspectives. Come check it out if you are in the bay area next week!

Grab some details on the event, or check out the event poster for more information.

http://www.forbes.com/forbes/2009/1116/marketing-hyundai-neurofocus-brain-waves-battle-for-the-brain.html

In the online version of the article there was a user comment from a neuromarketing company CEO defending the honor of his business and the field in which they operate. He went so far as to compare the launch of neuromarketing with the initial steps of market research in the early 20th century. He further argued that neuromarketing would bring about the next revolution in understanding consumer behavior.

I have to admit, my gut reaction on first reading this statement was one of mild disgust. This got me thinking about why neuromarketing hangs in a cloud of disdain among many scientists. Below are some of the ‘sins’ which I feel currently plague the field of neuromarketing. This is all just my opinion of course, but I do think that it raises some interesting points for discussion.

1) The curtain of proprietary analysis methods limits our knowledge of how effective neuromarketing can be.

Neuromarketing seems to be primarily driven by the private industry, not academia. This is not to say that research into consumer behavior has not occurred at the university level. There has been a lot of good neuroeconomics research in the last several years. Still, it is mostly companies in private industry that are driving the application of these findings to practical consumer behaviors. Because these companies are in competition with each other they are reluctant to give others the recipe to their secret analysis sauce. From the outside this means that the analysis pipeline of all neuromarketing companies is that of a black box, with data going in one end and the results-you-need coming out the other.

My colleagues and I have the position that fMRI research utilizing incorrect statistics can generate a large number of false positives. That is, many of the results will be there simply because of noise. Because so much of the current neuromarketing data is hidden behind the veil of proprietary analysis methods it is impossible to judge how successful their methods actually are, and to what degree their findings are false positives.

2) There is little peer-reviewed literature that is specific to neuromarketing.

Neuromarketing is an emerging discipline that will, in time, give us new insight into human behavior. Unfortunately, little peer-reviewed research has currently been published in this area. Search for ‘neuromarketing‘ in the PubMed database of abstracts (www.pubmed.com) and you will find all of ten publications. This must change for neuromarketing to mature.

Again, without peer-reviewed results on the effectiveness of neuromarketing experiments all we have to rely on are self-reports from the neuromarketing firms themselves. An issue similar to the file-drawer problem then exists. The file-drawer problem is when only positive results get published in journals while negative results sit unpublished in the file drawer. Neuromarketing companies will be likely to report positive results while negative results sit undistributed. Either way, the end result is a biased understanding.

3) Most people’s introduction to neuromarketing is through press releases, not peer-reviewed studies.

In 2006 there was an “instant-science” article released online by Marco Iacoboni et al. revealing their analysis of fMRI date obtain while subjects were watching Super Bowl advertisements. The much-discussed post, entitled “Who Really Won the Super Bowl?”, tried to determine the most effective commercial by judging which one activated regions involved in reward and empathy to the greatest degree. They determined that a commercial from Disney fared the best when evaluated by these measures. Many neuroscientists shook their heads and moved on.

In 2009 the same group published an op-ed in the New York Times detailing the results of scanning 20 individuals while looking at pictures and videos of leading political candidates. They drew conclusions on candidate evaluations by examining activity in areas like the amygdala and anterior cingulate. For example, they concluded that amygdala activity indicated a state of anxiety and cingulate activity indicated cognitive conflict. These oversimplifications were so well publicized and widely distributed that a number of leading neuroscientists were compelled to publish a letter in the New York Times calling the Iacoboni results into question.

Let’s put it this way, when many of the top minds in neuroimaging feel compelled to assemble a letter to the New York times regarding your non-peer-reviewed neuromarketing/neuropolitics results then the field has a problem.

There are a handful of peer-reviewed neuromarketing papers that do deliver. One recent paper by Michael Schaefer was a very interesting investigation into the representation of brand associations. However, these type of studies are typically rare, and it remains that the signal-to-noise ratio of information in the press is very low.

4) Neuromarketing methods are not immune to subjectivity and bias.

One of the most highly touted aspects of neuromarketing methods is that they are free from subjectivity and bias on the part of the participant. For example, asking a subject what they thought of a particular brand introduces the muddying waters of conscious consideration. The person’s response will be colored by a complex web of tangential cognitive factors and contextual biases. The promise of neuromarketing is that you can bypass these confounding factors to get at the heart of the matter – the real representation of the brand. While this is true to a degree, an entirely new set of confounding factors is introduced during the analysis of neuromarketing data.

While many neuromarketing measures are indeed more objective than verbal reports, I must disagree with the observation that they are unfiltered, true reports of the underlying representation. While the signals are not filtered by the consciousness of the research subject, a great deal of manipulation and filtering of the data is done by the researcher. This does introduce the potential for bias, simply by a different avenue.

Small changes in processing pipelines can have a huge impact on the power of fMRI to detect relevant signals. Some excellent papers by Stephen Strother come to mind with regard to this point. With no knowledge of what is going on we have no idea how objective the analyses by these companies can be.

5) The value per dollar of neuromarketing methods has yet to be determined.

Neuromarketing studies are expensive. The Forbes article says that an average EEG or fMRI marketing study costs in the neighborhood of $50,000. Immediately this number can trigger a ‘more expensive = better’ response, especially if you have a large budget to support such studies. What rarely gets discussed is what kind of value you obtain in return for the huge amount of money that is spent.

The key question in neuromarketing is what information can you get with EEG / fMRI / eye tracking / biometrics that you cannot obtain using other methods. If I can spend $1000 to do a traditional market study that gets me 85% of what a $50,000 fMRI study does then the return on my neuromarketing investment is not great. Thinking about it another way, how much less or more could I get across 50 traditional studies relative to the value of one neuromarketing study.

Many companies are not limited by the extreme cost of neuromarketing studies, and a significant fraction of them are not afraid to take the risk to try something new. Perhaps part of the motivation is also the fear of being left behind – that a competitor will take the risk and gain a competitive advantage in consumer understanding. Whatever the motivation, there will always be a market for neuromarketing methods. Still, we must still acknowledge that the value of neuromarketing is an open question.

6) People are rushing the field to make a quick buck, and not everyone is trustworthy.

The emergence of neuromarketing represents a modern day gold rush in terms of buzz and promises. Brilliant researchers will be attracted to this opportunity and will significantly advance the field of neuromarketing. Morally questionable individuals will also be drawn to the opportunity, and will end up giving the field a black eye. Reputations will build up over time and trustworthy companies will emerge from the fray, but the current situation is more akin to the wild west than a civilized exchange.

7) The true value of neuromarketing is obscured by the above-mentioned problems.

I thought I would end on a high note. There is certainly significant value to using neuromarketing methods in consumer research. Why else would companies like Nielsen Holdings be investing in neuromarketing firms like NeuroFocus? One of the biggest problems is that the true value of these methods is obscured by those who treat it as a gimmick and have the loudest voice. The next ten years will represent a true shakedown of the neuromarketing industry. Companies that are able to provide real value to their customers will live on while those who simply seek to make pretty pictures will fall by the wayside. It will be a fascinating time to be an observer of the business and politics in this emerging field.

Conclusions.

The above points ignore many other issues facing neuromarketing. I have completely bypassed a discussion of the ethics of neuromarketing. Many people worry that technologies like fMRI will help marketers find the ‘buy button’ in the brain, stripping away people’s free will in product choice. I am not terribly worried about that discussion, perhaps because I am ignoring the problem or perhaps because I know too much about brain function or neuroimaging methods. Regardless, there are other issues and hurdles that neuromarketing must address to grow as a field.

In the end I do wish neuromarketing great success. I simply fear that those individuals who are seeking to profit on the popularity will tarnish the reputation of neuromarketing before it is able to legitimize itself.