Will "4.37 Degrees of Separation" play at the multiplex? Awhile back I wrote about recent research to test the Six Degrees of Separation theory. New evidence suggests that people are separated by an average of 4.74 degrees (only 4.37 in the U.S.). Doesn't really roll off the tongue, and I wouldn't expect Will Smith to star in a sequel. But this latest research applies only to people on Facebook; a New York Times piece reminds us "the cohort was a self-selected group, in this case people with online access who use a particular Web site".

Desperately seeking scientific integrity. Early on, President Obama launched an effort to ensure that the public can "trust the science and scientific process informing public policy decisions". His March 9, 2009 memorandum made agencies responsible for "the highest level of integrity in all aspects of the executive branch's involvement". Great idea to de-politicize science, though it's much easier said than done -- as evidenced by the nearly-two-year delay in providing "guidelines" for agencies issuing this policy. Really. Those guidelines were published December 17, 2010 [pdf here].

Oh, and meanwhile the Obama Administration was widely criticized for using sloppy science to support its moratorium on offshore drilling after the BP disaster on the Gulf Coast. It's never simple: In real life it requires weighing risks, factoring in economic impacts, and making political tradeoffs (consider this analysis of the economic impact of that ban - [pdf here]). A scientific integrity rule won't give you all the answers you need when making such complex decisions: It's surely not as simple as determining whether scientific findings were manipulated for political purposes.

As explained in a recent National Public Radio story, now we're seeing the first challenge to federal evidence-gathering under this new regime: It's directed at the Bureau of Land Management (a branch of the Department of the Interior). More about that in a moment.

Why is this so hard? It's been rough going for agencies issuing scientific integrity policies. The basics are straightforward enough: Preventing people from twisting or quashing scientific evidence. But here are some reasons why the process is so problematic.

1. Scope. There's no bright line identifying what "science" should be included in an assessment, and therefore subjected to integrity requirements. This complicates things enough when you're working with "objective" evidence. It's even trickier when you bring in fuzzier stuff, like the dismal science (economics), or risk assessment. How can we say for sure what should be considered? Our assumptions about what's important determine what evidence we recognize - whether consciously or subconsciously. Our choices are influenced by our values - but we may not be fully aware of our values, or we may not want to articulate them in a transparent way.
2. Dissemination. Each agency's integrity policy is supposed to provide for open communication, and guide how evidence is presented to the public (see the 2010 guidelines mentioned earlier). It wouldn't serve anyone to have a free-for-all; consistent, controlled dissemination can improve usefulness and understanding. But not everyone agrees on the rights and responsibilities of scientists who want to discuss their findings with the public.
3. Whistle blowing. Among the substantial hurdles is the handling of whistle blowers. (I suppose if such a policy is going to have teeth, the people who want to blow whistles need to feel they can do so without losing their heads.)
4. Transparency. Scientific groups - such as the Union of Concerned Scientists - say they still want to see external accountability under these policies. So far, investigations of misconduct are internal.

12291 all over again? Thirty years ago, President Reagan signed Executive Order 12291, requiring cost-benefit analysis for 'major' federal regulations (those expected to impact the U.S. economy by $100 million or more). Clinton issued a similar order in 1993. In theory, this should have de-politicized some agency decision-making processes. The results (or lack thereof) were the subject of my doctoral dissertation.

As with the new mandate for scientific integrity, a requirement to weigh regulatory costs against benefits leaves lots of room for interpretation, and requires value judgments. When EPA issues a rule under the Clean Air Act, it's difficult enough to estimate how many hospital visits or early deaths are caused by a particular type of airborne particulate matter. But figuring out the social costs is harder still: Requiring businesses & governments to reduce those emissions can lead to job cuts and economic loss, which themselves cause poverty and negative health impacts.

Challenging BLM's process. Citing the new scientific integrity policy, a group called Public Employees for Environmental Responsibility (PEER) has filed a complaint against the BLM, saying "The U.S. Bureau of Land Management is carrying out an ambitious plan to map ecological trends throughout the Western U.S. but has directed scientists to exclude livestock grazing as a possible factor in changing landscapes....  [O]ne of the biggest scientific studies ever undertaken by BLM was fatally skewed from its inception by political pressure.... As a result, the assessments do not consider massive grazing impacts even though trivial disturbance factors such as rock hounding are included, [and they] limit consideration of grazing-related information only when combined in an undifferentiated lump with other native and introduced ungulates (such as deer, elk, wild horses and feral donkeys)." I didn't know we had a feral ungulate problem. But I digress.

This is a good example of how choices about collecting evidence can strongly influence the results. NPR explains that the Dept. of Interior has a scientific integrity officer who is responsible for investigating allegations of political interference. I wish him Godspeed.

You can join in. Social scientists at Yahoo! and Facebook have launched the Small World Experiment, "designed to test the hypothesis that anyone in the world can get a message to anyone else in just 'six degrees of separation' by passing it from friend to friend. Sociologists have tried to prove (or disprove) this claim for decades, but it is still unresolved.

"Now, using Facebook we finally have the technology to put the hypothesis to a proper scientific test. By participating in this experiment, you'll not only get to see how you're connected to people you might never otherwise encounter, you will also be helping to advance the science of social networks."

Photo credit: Film-Buff Movie Reviews, where they play Six Degrees of Kevin Bacon. I chose this picture of young Kevin because this weekend I saw a trailer for the Footloose remake. Some things should probably stay un-remade. (I was there to see Crazy, Stupid, Love, which I highly recommend. Kevin's in that one, too.)

Writing in New York Magazine, Kathryn Schulz takes Tina Rosenberg to task for touting a "social cure", formerly known as peer pressure, "a panacea so powerful that it can help you quit smoking, lose weight, escape poverty, ace calculus, combat Islamic terrorism, overthrow a dictator". (Thanks to Elizabeth Lusk, @gestaltKT, for the link. Here's her Evidence Soup interview.)

Group Think praises Rosenberg's earlier work (she is a MacArthur Foundation grant recipient). But Schulz has a bone to pick with her latest book, Join the Club: How Peer Pressure Can Transform the World. [Photo courtesy of New York Magazine.] She explains:

Big Idea books have been around for a long time; see The Communist Manifesto. But the Big Idea Book Club (I mean “club” as Rosenberg defines it: an identifiable in-group with enough status to influence the behavior of others) is a recent phenomenon. Its accidental founder and president in apparent perpetuity is Malcolm Gladwell. Its membership, like the membership of most powerful groups, is largely male. Its combined sales are stratospheric; whatever these books are hawking, we can’t stop buying it.

As for the books themselves, I’ll generalize (as, often, do they). Big Idea tomes typically pull promiscuously from behavioral economics, cognitive science, and evolutionary psychology. They coin phrases the way Zimbabwe prints bills. They relish upending conventional wisdom: Not thinking becomes thinking, everything bad turns out to be good, and the world is—go figure—flat.

(With Gladwell’s Blink, this mania for the counterintuitive runs top-speed into a wall, crumples to the ground, and stares dizzily at the little birds circling overhead. This is, let me remind you, a best-selling book about the counterintuitive importance of thinking intuitively.) What troubles me about the Big Idea Book Club is the way ideas often slide toward ideologies - grand unifying theories of culture, cognition, happiness, talent, the Internet, the future, you name it.

Well said, Ms. Schulz.

Last week's Science Friday was recorded in San Antonio, Texas. The city's mayor, Julian Castro, and several others spoke about efforts to modernize the energy grid there and across the state. Turns out, their peak load happens closer to 10:30pm. This discovery was made after they began collecting very granular meter data during a pilot project.

Aha moment in Texas. Brewster McCracken, executive director of the Pecan Street Project in Austin, explained that "We kind of almost by accident have ended up with the world's deepest database on how people use energy and now gas and water. It's in 15-second increments.... The most granular data before that was 15 minutes."

"... that's obviously going to be pretty impractical and actually not necessary for utilities, but it is very important for product development to understand, you know, if you're going to try to create a product that is of value to a customer, it's really important to understand what the customer wants. And so one of the ways to do that is to find out their data. And we're finding out, for instance, in the summertime, surprisingly, that the peak time of electricity usage in terms of draw on the grid is 10:30 at night. It's not 5:00 in the afternoon, which was a huge surprise."

McCracken continued: "... here's why, actually. And it doesn't show up in 15-minute data. It only shows up when you take it down to a finer level. We do this in my family, a lot of folks do. You turn down your AC when you go to bed at night to make it a little bit cooler. And the AC has such a huge influence on your home energy usage that it's about - the draw, peak draw can be up to 20 percent higher at like 10:30 at night, 10:00 at night, as it is at 5:00 in the afternoon."

Anecdotal evidence from Colorado. It's 10:16pm as I finish this post. Next thing I'm doing? Cranking up the AC and turning in for the night.

I first wrote about the kerfuffle in 2007. It all began when the University of Washington announced a study finding that for each hour-per-day spent watching baby DVDs/videos, infants understood on average six to eight fewer words than those who didn't watch, and recommended that parents limit their use. "This analysis reveals a large negative association between viewing of baby DVDs/videos and vocabulary acquisition," they claimed. (There's a recap of events on Wikipedia. The mainstream press piled on in a big way.)

Much ado about nothing? Initially, I thought this seemed to be a case of a researcher making alarmist statements to the press, not because the evidence was significant, but to promote themselves. And that's still how it seems. I asked back then "Even if this is true, where is the evidence that 6-8 fewer infant words will matter in the long run?" Using this limited research to claim that Baby Einstein (and other) videos are harmful seemed over the top. Another reason I doubted the importance of the study: "Only 17 percent of 384 babies in the survey were put in front of videos for an hour or more each day. The average baby watched only about 9 minutes a day." One has to wonder what factors influenced the children's development during the, um, other 23 hours and 51 minutes of their day.

Show me all the evidence. The creators of Baby Einstein have been fighting back ever since. Bill Clark and Julie Aigner-Clark have obtained internal documents related to the study, and say they've succeeded in having $175,000 of their legal bills paid by the University of Washington. They also say the documents "confirm what they always suspected: The study was deeply flawed and unfairly characterized Baby Einstein products."

According to today's story, the Clarks found "correspondence from one researcher concerned about how certain results were analyzed. While children 8 months to 16 months who watched baby videos fell behind in vocabulary, the study also found that in children 17 months to 24 months, vocabulary increased and the negative effects evaporated." (Apparently that vocabulary rebound was part of the published report but was downplayed in news releases.)

No follow-up. An internal email, between a reviewer and one of the researchers, asked  "What's the notion about how (we're) reconciling the fact that there was an effect on the young kids but it washed out by the time they were 17-24, and we now will be wanting to follow the young kids when they're older?" Apparently there was supposed to be follow-up research with the same parents, but that was scrapped due to cost.

What a waste of resources. Disney eventually was pressured to offer refunds to Baby Einstein customers. The U.S. Federal Trade Commission (FTC) even wasted time on this. Eventually, as explained in the Washington Post, a separate group of researchers found that although "an hour a day of television viewing won't make your kid a genius, it won't harm his development either.... At first glance, the study's authors found decreased language and visual motor skills in kids who watched more television. But then, the researchers adjusted the results to account for the mother's age, income, education, marital status and vocabulary. And you know what? They found that mom's education level and vocabulary rather than the television greatly impacted baby's."

Physicians can adhere to the principles of professionalism by practicing high-quality, evidence-based care and advocating for just and cost-effective distribution of finite clinical resources. To promote these principles, the National Physicians Alliance (NPA) initiated a project titled "Promoting Good Stewardship in Clinical Practice" that aimed to develop a list of the top 5 activities in family medicine, internal medicine, and pediatrics where the quality of care could be improved.

NPA members sought to identify "ideal" medical activities, meaning they would be common in primary care practice, would be strongly supported by evidence, would lead to significant health benefits, and would reduce risks, harms, and costs. A number of primary care activites were field tested in family medicine, internal medicine, and pediatrics: Participants identified common clinical activities that could lead to higher quality care and better use of finite clinical resources. Based on the findings, the Good Stewardship group is recommending that its "Top 5" lists of activities be implemented in primary care practice across the United States.

Medscape recapped the findings in Most Wasteful Treatments, Procedures in Primary Care (free registration required). "A study commissioned by a reform-minded medical society has come up with specialty-specific lists of procedures in primary care that do little if anything to improve outcomes but excel at wasting limited healthcare dollars."

Medscape concludes that many of the Top 5 recommendations consist of "don'ts." For instance: "A prime example is routinely ordering diagnostic imaging for patients with low back pain — but with no warning flags, such as severe or progressive neurologic deficits — within the first 6 weeks, according to the study."

To put the findings into practice, NPA plans to "produce training videos to help physicians discuss the guidelines with patients and gain their understanding and trust, lest patients think they are being denied needed care (a common scenario when they ask for antibiotics to treat a viral infection)."

1. John D. Cook is into programming, research, math, and statistics. He tweets via an impressive list of accounts (entire list here), including @AnalysisFact and @StatFact.

Responding to the common criticism of evidence that's not grounded in classic RCT (randomized, controlled trial) methodology, he writes: "As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence-based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence-based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute."

2. Carnival of Math (oh, how I love that name). By day he goes by Mike Croucher, making insightful and witty observations, such as this one about the 95% confidence interval: "Last year around 27,000 psychology papers used a 95% confidence interval. So we can say with 95% confidence that at least 1,292 of them were wrong."

Mr. Carnival also writes technical stuff at Walking Randomly "because it's more fun than getting there in a straight line."

3. William M. Briggs bills himself as the Statistician to the Stars! He consults on statistical problems, and is an adjunct professor of statistical science at Cornell University (Ithaca, New York, USA). Awhile back, Briggs wrote an excellent piece Classical statistics is like a criminal trial: "Some thing has happened, and a hypothesis is put forth which, based on probative background information, the investigator thinks likely caused the thing. Evidence -- mostly in the form of “data”, but non-quantitative information too -- for this hypothesis is put forth.... The jury consists of mathematical formulae whose duty is to report solely on the likelihood of the evidence given the hypothesis is false. And since the evidence nearly always appears rare assuming the hypothesis is false, most trials result in a conviction -- meaning, the investigator’s viewpoint is confirmed."

Briggs explains how he thinks we should look at things, using global warming as an example: "Trying to fit carbon dioxide into the frame as the culprit for a historically observed increase in temperature is vastly less interesting and useful than in being able to accurately predict what will happen. It is easy enough to find enough evidence to convict our poor gas classically. But if it were part of a gang of gases, the others would go free. Meanwhile, in celebrating our conviction, we would remain ignorant of what will happen, or will issue poor predictions because we were so intent on assigning blame."

SMS/TCPA lawsuit against Twitter. Some folks unsubscribed from Twitter SMS messages. So Twitter sent them SMS messages confirming that they were unsubscribed. Now these folks are suing (possibly in a class action), claiming violation of privacy under the Telephone Consumer Protection Act. Details on TechCrunch.

Supreme Court allows cheap legal shot at Chipotle. My admiration for Chipotle's human resources policy is well documented. So it disturbs me that someone who's basically a litigious, gold-digging whiner is being allowed to use up their resources in court. Here's the complaint: People in wheel chairs can't see the food being displayed at Chipotle because the counters are a bit too high. Evidently, the individual bringing the suit has sued dozens of establishments for access violations, and taken cash settlements. Yet the U.S. Supreme Court is allowing the case to go on.

Wacky Warning Label competition. Sadly, U.S. companies often feel the need to place extreme warning labels on consumer products. So much so that there's a contest to determine the silliest cautionary statement. NPR interviewed Bob Dorigo Jones, one of the competition organizers, who recently announced finalists for 2011 (the contest is in its 14th year). They were particularly astounded by a brochure accompanying a child's bicycle. "There are pictures of little children riding bikes, most of them on training wheels, but the warning label says: 'The action depicted in this brochure is potentially dangerous. The riders seen are experienced experts or professionals.'" *sigh*

Robert Siegel of NPR reminds us that "If you turn up your radio dial, the volume will get louder."

Okay. About that other evidence. You may have heard that God's Elect are being called home on May 21. I was curious what evidence they have: Carl Sagan reminded us that "Extraordinary claims demand extraordinary evidence". I happened across an interview with Harold Camping, who calculated the doomsday date. In the NPR story Divining Doomsday,  Camping says "the Bible is written in a code, and for those who are able to decipher it, it's clear as daylight. With all the proofs that God has given us, and all the signs, I am absolutely certain [that Judgment Day will arrive on May 21]. It is going to happen. There is no Plan B."

NPR reminds us "even Jesus said he didn't know when Judgment Day would come. But Camping is not bothered by that, nor by the fact that he wrongly predicted Judgment Day once before, in 1994. 'It was based on incomplete research and I was quite aware that the research was incomplete,' he says."

Camping is confident he's correctly calculated the date this time. Soon we'll all know whether or not his evidence was valid.

Each resident spends one month in the department, and is expected to write, compose, and produce several songs during that time. This is a serious effort. For example, this month's resident songwriter, Kay Stanton, aspires to become the Jane Goodall of music. (*According to the guidelines established by David Allan Coe, she'll need to work in some stuff about Mom, prison, and trucks.)

Move over, Bill Nye. This may sound silly to some, but I think otherwise: Cross-pollination and interdisciplinary learning are what we need more of. The process of explaining their research - and seeing it in a new light - can substantially benefit the scientists. And a layman's explanation of complicated evidence is always welcome.

NPR did a story this week on the NIMBioS program. Here's a video and interview with Jay Clark, one of the songwriters. His song Sexual Selection explains that "we all like to copulate". There may be a Top 40 hit in his future.

Brynjolfsson (@erikbryn) and two co-authors just published Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? Their findings suggest that large companies who adopt data-driven decision-making can achieve 5-6% higher productivity -- beyond what can be explained by factors such as technology investment.

Lohr explains that in this research study "data-driven decision making was defined not only by collecting data, but also by how it is used — or not — in making crucial decisions, like whether to create a new product or service. The central distinction, according to Mr. Brynjolfsson, is between decisions based mainly on 'data and analysis' and on the traditional management arts of 'experience and intuition.'"  

Big data = Big dollars. Even without these findings, companies have already recognized the value of data analytics. Lohr says that the MISO vendors — Microsoft, IBM, SAP and Oracle — have collectively spent more than $25 billion buying up companies specializing in data and analytics: "IBM alone says it has spent $14 billion on 25 companies that focus on data analytics. That business now employs 8,000 consultants and 200 mathematicians. I.B.M. said last week that it expected its analytics business to grow to $16 billion by 2015."

From the research abstract: "We examine whether performance is higher in firms that emphasize decisionmaking based on data and business analytics (which we term a data-driven decisionmaking approach or DDD). Using detailed survey data on the business practices and information technology investments of 179 large publicly traded firms, we find that firms that adopt DDD have output and productivity that is 5-6% higher than what would be expected given their other investments and information technology usage. Using instrumental variables methods, we find evidence that these effects do not appear to be due to reverse causality. Furthermore, the relationship between DDD and performance also appears in other performance measures such as asset utilization, return on equity and market value. Our results provide some of the first large scale data on the direct connection between data-driven decisionmaking and firm performance."

Stay tuned. I downloaded a free pdf of this research from SSRN here. I'll be exploring the details of the methodology and findings later this week. I'm particularly interested in their measures.