Think how differently you would have to go about filling your needs if you lived on a desert island rather than in a society of men and women.

“[The] thirst for objective knowledge is one of the most neglected aspects of the thought of people we call ‘primitive,’” observed anthropologist Claude Lévi-Strauss.  Living in a state of nature with your survival under constant threat is a powerful incentive to learn the cause-and-effect relationships of the physical world.  Primitive people have to be part meteorologist, part horticulturalist, and part engineer, as did our early ancestors.

Thanks to civilization, each of us personally no longer needs to acquire that kind of knowledge.  We still live in the physical world, and we still have the same basic needs for nourishment, shelter and health maintenance.  Somebody in our social system needs to have the technical knowledge to grow food, fabricate things and cure what ails us, but we don’t.

That doesn’t relieve us of the need to manipulate the environment to fill our needs, though; it just adds a sequence to the causal chain that concludes with their fulfillment.  If we want to make a hole in the ground, we can pick up a shovel and dig one.  Or we can use a different kind of tool: another human.

You need to know how to use a tool in order to get it to do what you want.  Whoever actually digs the hole will need to have the physical ability and knowledge of basic mechanics to use the shovel, or some other implement capable of doing the job.  We’ll call that implement the first-order tool.

If you want to have a hole dug and are unable or unwilling to do it yourself, you could use a second-order tool, another person, to operate the first-order tool.  Just as the eventual shovel-wielder needs to know how to make that tool do its job, you need to know how to make the second-order tool do its job – pick up the shovel and dig the hole.

The significance of being able to use people as tools – of human agency – to bring about the physical effects that support and enhance our lives is hard to overstate.  It’s almost like traveling to a new universe that replaces our laws of physics with its own cause-and-effect rules.

The instructions for our second-order tool might look like this:

We have to work with people’s heads to get them to perform actions that benefit us.  It’s impossible to physically force someone to pick up a shovel and dig a hole, as you would use a hammer or lever.  In order to use humans as tools, we have to get their brains to send a signal to their muscles to move in such a way as to cause the physical effect we seek.

To do that, we need to understand cognition.  We build mental representations of the world that encode our assumptions about its cause-and-effect relationships, and allow us to “sandbox” actions we’re considering – try them out and see what’s likely to happen.

The informal word for mental representation is “belief.”  Whether you believe that Bigfoot is real, that cell phones cause cancer, or that Google is making us stupid, those propositions are slices of the mental models you build to represent the world and the rules by which it operates.  When someone asks (or orders) you to do something, you consult your mental models and calculate what’s likely to happen if you do it, and also if you don’t.

We steer cars by turning a wheel.  We steer people by appealing to and manipulating their beliefs.  To make our human tools work for us, we leverage the entries in their mental lookup tables about what causes lead to what effects.

We could direct their attention to the consequences of failing to do what we want them to do.  That can include undesirable effects that we personally promise to cause (I’ll punch you; I’ll sue you; I’ll withhold sex; I won’t recommend you for promotion).  Or we might invoke other supposed cause-and-effect relationships to get them to do our bidding (Congolese orphans will starve if you don’t; God will hold it against you).

For the purpose of making our second-order tool do the work we’ve assigned it, what matters is not what really will happen as a result of their choice (our threat to punish them could be a bluff; God may not exist, or may not care), but what they believe will happen – what causal rules are coded into their mental models.  If the rule we seek to leverage is already there, so much the better.  If not, we can try to program it in – to convince them that A leads to B.

Most of the time, we operate our human tools using a carrot rather than a stick.  In return for manipulating the environment to cause an effect that fills a need of ours, we offer to do the same for an equivalent need of theirs, either directly, or using a substitute for value, money.  You scratch my back, I’ll scratch yours.

But the equivalence of that exchange depends on how accurately our information processing systems link causes with effects.  For example, we’re fairly confident that putting a liquid made from the fossilized bodies of ancient plants and animals into our cars makes them move, so we generously reward those with the knowledge and ability to extract it.  But we also spend a lot on additives that claim to improve gas mileage, despite repeated scientific tests that find virtually no benefit to them.

Then there’s the herbal and dietary supplements industry, to which we trade tens of billions of dollars’ value of our work annually for health benefits that are either non-existent or so small that they’re overwhelmed by other effects.  The fact is, misattribution and the placebo effect account for much of our evaluation of the effectiveness of the goods for which we swap our labor and skills.  There’s nothing in Coke’s formula that makes us “open happiness” by snapping off the cap; “scrubbing bubbles” is just a marketer’s metaphor to boost sales of a bathroom cleaner whose actual performance is no better than its competitors’.  The word “organic” on product labels may well account for the greatest amount of work in history traded for benefits perceived above and beyond those objectively caused.

It’s not just to make products that we use people as tools, but also to cause other effects we desire.  We rent CEO tools to turn struggling companies around and politician tools to create jobs and defuse world tensions.  Here, causality can get really complicated.  For one thing, we’re using them as n-order tools in a causal chain: they act on the people who report to them, who in turn manipulate their staffs, and so on, until, if all goes as planned, the effect we seek (solvency, peace) pops out at the end.

And this kind of tool often operates inside a large system in which a variety of forces (some controllable, some not) interact in complex ways to determine the outcome, such that it can be impossible to reliably calculate one person’s contribution.  When causation is complicated, it’s easier for plausible but false explanations to “pass.”

All this doesn’t mean that we can rip each other off at will.  Even though the saying, “The truth will out,” is false as an absolute, we can jigger belief only within certain limits.  Our fondness for citing “The Emperor’s New Clothes,” though, shows that the wiggle zone can be pretty wide, and the fact remains that all we have to do to make our human tools work for us is get them to believe we can create an effect they desire, even if all we’re really providing is a placebo.

Sum all of these causal attributions across a population, embed them into their institutions and practices, and you’ve got a new kind of reality: social reality.  Status, power, the distribution of social rewards and sanctions – it can be argued that all are ultimately based on collective inferences about causality that are only loosely correlated with what really causes what.

That disconnect isn’t going away anytime soon, because it’s a consequence of the cognitive biases, errors and capacity limitations that are our evolutionary legacy.  What’s a second-order tool who doesn’t want to be taken advantage of – or a user of second-order tools who wants to get the most from them – to do?

How well an organism thrives depends on how well it masters its environment’s causal rules.  We live simultaneously in two different worlds, one physical, one social, each with its own rulebook.  Many, if not most, of us are better at knowing and manipulating one of those causal systems than the other.

So we go with our strengths.  If you’re good at knowing the objective (physical) world, you can get the best value for your work in occupations with low tolerances for error in working with its causal rules, such as astronaut, farmer, or computer programmer.  People with good social and persuasive skills can maximize the return on their labor in careers that depend more on – and allow more wiggle room in – the construction and manipulation of beliefs.  Examples include salesperson, coach and elected official.

We’ve come a long way from the African savanna.  Civilization didn’t overturn the laws of physics, but it made their direct effects less important to the quality of most of our lives than the effects that flow from each other’s interpretation of them.  Human agency is a game-changer.

The psychology world is abuzz over a recent paper by Hugo Mercier and Dan Sperber suggesting that reason evolved not to know what’s true, but to help us win arguments. “Reasoning doesn’t have this function of helping us to get better beliefs and make better decisions,” says Mercier. “It was a purely social phenomenon. It evolved to help us convince others and to be careful when others try to convince us.”

I think this idea, which they’re calling the argumentative theory of reasoning, is partially correct – but only partially. The rationales for our actions are the causal rules coded into our mental models of the world. The accuracy of those models was honed, through natural selection, by the feedback loop that links causes with effects. For most of our evolution, this process was driven almost entirely by the laws of physics. We noticed that we got sick after eating a certain kind of berry, so we stopped eating it. Argumentative skill would not have contributed to adaptiveness in a world in which most of the effects having implications for human survival did not have social causes.

But we’ve been social animals for a long time, too. Long enough for evolution to have made us better at leveraging the natural biases and capacity limitations of human cognition to get other people to create physical effects for us (wealth, status) that contribute to our survival. The development of argumentative skills could be seen as a refinement of our ability to manipulate one particular kind of tool (other people) whose influence in determining our fate was elevated by civilization.

It’s all about the cause-and-effect feedback loop. Effects – be their agents the wind, germs, or other people – promote behaviors that maximize salutary outcomes and minimize disadvantageous ones.

Its proponents were right all along – behavior drives psychology.

In the distant past, dinosaurs roamed the earth. Almost as long ago – or so it seems – the prevailing approach to psychology was behaviorism, which studied the conditioning process through which organisms learn to respond to stimuli in the environment. Because thought processes can’t be directly observed and measured like behavior can, consciousness and the mind were pretty much left out of the picture.

By the 1980s, researchers had figured out lots of ways to make mental processes accessible to scientific methods, reducing the opacity of the black box that cognition had been. Mentalistic terms like purpose, importance and mood, banned from the behaviorist lexicon, became legitimate constructs for psychological theories, allowing researchers to better understand what makes us tick.

We don’t talk much about stimulus and response any more, perhaps as a reaction against psychology’s long obsession with them. But we still live in a world of causes and effects in which our life outcomes depend to a large degree on how we choose to behave (or not to behave) in response to the environmental situations we encounter. The mind’s main job is to choose the best responses to stimuli.

Cognition is not an end in itself, but a means to the end of action. The most important questions confronting organisms are not about environmental states (what is that thing, what are its properties, what relationships does it have with other things), but what we should or shouldn’t be doing right now. Answers to the former kinds of questions are useful only to the extent that they inform the latter (excepting the case of effects that are caused directly by thought, and do not require behavior to cause them).

Cognition, in other words, is the handmaiden of behavior. We think (primarily) in order to do. The task for which our information processing system evolved is the derivation, from a set of environmental inputs, of the behavioral response that maximizes the probability of the occurrence of an effect that contributes to our survival (or minimizes the probability of one that threatens it).

It’s in our interest to be good at casual attribution – knowing our environment’s cause-and-effect relationships, particularly those with implications for our lives. When the causal relationship is simple and unambiguous, the mental processing required to map a stimulus to a response is minimal and quick, and the probability that the desired effect will result is high, as when we jerk our hand from a hot stove. (We have a term for such a low-level behavioral decision: reflex.)

But the world is a complex place, and causality is often messy, with many factors interacting in complicated ways to cause outcomes. Some problems are beyond our computational power, and we have limited access to the data needed to fully understand (i.e., mentally model accurately and in full detail) some causal systems, leaving many of our solutions underdetermined. We don’t have the option of passing on life problems that are too hard for us to solve with a high degree of accuracy, because failure to act is itself a decision. We have to go with our best hunch.

Starting before we’re even born, we move through a continuous stream of rapidly changing stimulus fields that require us to dynamically adjust our behavior to maximize the good and minimize the bad. To make those adjustments, we monitor the events and states that follow our actions and those of other people and things and make inferences about what caused them. Our lives are one long operant conditioning experiment.

Whether and in what direction behavior is reinforced by what follows it depends on our perception of effects and on the causal attribution made by cognition. Suppose, for example, two homeowners with lawns of exactly the same type and condition buy Turf Titan lawn thickener and apply it the same way with the same (objectively measured) result – their lawns get a little thicker. Homeowner A notices the difference and is positively reinforced by it: he becomes a regular customer. But Homeowner B has one of those annoying neighbors with an immaculate lawn, and against its perfection, he doesn’t notice his own lawn’s modest improvement. “Doesn’t work,” he mutters as he tosses the half-used bag in the trash.

Hold on, you might protest; it’s not a fair comparison – if this were an experiment, you would have controlled for the adjacent lawn to prevent a contrast effect from biasing one subject. But that’s the point – we behave in the natural world where parameters vary freely, their camouflage providing many places for causal relationships to hide from cognition.

If we misattribute or fail to detect many environmental effects, though, we must get some of them right. Our species wouldn’t have survived if we weren’t at least moderately good at detecting the signal of nature’s cause-and-effect relationships against the noise of competing confounds. Objectively existing forces – in both the natural world and the social one – impose effects on us whether we understand them or not. Those forces shape our behavior, but only to the degree that cognition’s causal attributions are accurate.

We might call this the “The truth will out” threshold (though it’s really more of a gradient, because our mental models correspond to their objective referents in degrees). If the signal is strong enough and/or the noise level is low enough, we learn the causal relationship, and behavior is positively or negatively reinforced appropriately. Example: feeling violently ill moments after eating poison berries, leading to the (probably rapid) extinction of that behavior.

Suppose, though, that the berries are only mildly toxic, and it takes many hours for symptoms to develop. Result: the berries still cause their negative health effects when consumed, but the truth of their toxicity doesn’t “out” – we don’t learn it, and don’t modify our behavior accordingly. So it is with many of the environmental effects that impact us. Their causes go un- or misattributed by cognition.

The degree to which the truth doesn’t out is the latitude that we have to construct social realities that are robust to correction by objective forces. As far as we’re concerned, the mildly poisonous berries are a valuable part of our diet. We might even be able to manipulate others’ mental models of the world’s causal relationships and convince them that not only do the berries lack negative effects, they cause a positive one: enhanced sexual function. Coincidentally, we may have some available for purchase.

I’ve been speaking of cognition modeling the world’s cause-and-effect relationships more or less literally, as if we carry a list of rules and rationales in a mental look-up table. E.g., “Don’t touch hot things because they hurt.” Some of our behavior selection might work something like that, but it’s important to note that what we typically think of as comprehension or understanding is not required to learn a behavior that causally contributes to a desired outcome, only that we produce an efficacious response when presented with a particular environmental situation. As far as objective causation is concerned, doing the right thing for the wrong reasons is just as good as doing it for the right ones, because it’s the behavior, and the behavior alone, that does the causing (or the contributing to a cause). How we chose it is irrelevant. All a neural network (and that may be all our brains are) “knows” is that Input Array X maps to Output Array Y – and that’s enough.

The behaviorists were right after all that behavior and reinforcement lie at the core of psychology. They just left the interpreter (cognition) out of the feedback loop that coordinates responses with stimuli.

A Review of Wrong: Why Experts Keep Failing Us – And How to Know When Not to Trust Them, by David H. Freedman, New York: Little, Brown and Company, 2010. 304 pps. $25.99. ISBN-13: 978-0316023788

The market for books about how ordinary people make thinking mistakes being fairly saturated (Predictably Irrational, Sway, Nudge), it makes sense that someone would turn the spotlight on a group that’s supposed to mess up less than the rest: experts. Journalist David Freedman walks us through an impressive list of false and conflicting claims made by experts in a variety of fields that really drives home the dubiousness of much – if not most – of what passes for expert wisdom. The book is worth this carefully assembled and annotated collection of dueling truth claims alone.

There’s plenty of blame for our expert misinformation to go around, says Freedman. From respected scientists to financial wizards to self-proclaimed relationship gurus, people whom we credit with specialized knowledge conduct sloppy research, suppress disconfirming data and leap to unwarranted conclusions. Journalists oversimplify and misrepresent study findings. Bad advice thrives in part because the public demands easy fixes that are “resonant, provocative and colorful.”

We would expect Wrong to cover famous cases of expert fraud like the South Korean human embryo cloning scandal, and it does. Most of the expert errors documented here, however, are not intentional, but originate in the cognitive biases to which everyone is prone. Like the rest of us, experts have sharper eyes for data that supports their hypotheses, claim to have started out looking for what they eventually found, and play to their employers’ metrics (research funding agencies). Nor are peer review and other forms of self-regulation much of a remedy: Thomas Kuhn showed some fifty years ago how the practices of scientific communities reinforce and perpetuate prevailing paradigms.

It would be unremarkable to learn that experts sometimes make mistakes, but if Wrong is right, the magnitude of the problem is much greater than most people suspect. The findings of two out of every three published medical studies fail to hold up. When you consider that the truth claims of less highly-educated and credentialed experts like the inventor of the latest diet or management fad are on average even less reliable, you realize we’re awash in untrustworthy advice.

Books written to hammer a single point are vulnerable to overstating their case, and Freedman’s expert targets are sometimes damned if they do and damned if they don’t. For example, he criticizes mainstream scientific research for careless procedures and small, unrepresentative samples but later defends junk science because, even though its procedural rigor and signal to noise ratio are typically even lower, it occasionally stumbles upon a nugget missed by the pros. That’s having it both ways. Similarly, he fails to take his own advice against retrospective sense-making when he says the warning signs in a famous case of scientific fraud were “glaringly obvious” when looked back upon.

Freedman knows his task leaves him open to the charge of begging the question: the same lack of certainty he says accompanies expert judgments must also apply to his own assumptions that particular expert claims are wrong. I agree with his defense that, though any individual claim may be mistaken, accumulating and pooling evidence allows us to converge on the truth. A historical progression of geographic maps provides an example: early maps of the world are wildly discrepant, but the shapes of the continents and details in their coastlines gradually converge and stabilize across cartographers over time.

So common are the serious errors catalogued in Wrong by even the most eminent researchers and institutions, though, and so influential are false claims in directing the flow of dollars and in propping up whole industries and reputations, that by the time you reach the simple guidelines at the end of the book for knowing when to suspect that an expert opinion might be wrong, it feels the equivalent of being advised to move a foot inland from the beach to protect against tsunamis.

Wrong’s abundant examples of how experts fail us demonstrate how complex the world is. The contingencies of cause-and-effect relationships can be many and difficult to trace, and good advice in one situation can lead to disastrous results in another.

Unfortunately, directing experts (or the middlemen who report their work) to qualify their truth claims with all of the ways they could be wrong, and publicizing negative findings as well as positive ones, are no solution. That just adds to the pile of stuff we have to sift through, and makes it harder for our brains to fulfill what is perhaps their most important function, and is the main reason we rely on experts in the first place: information reduction; sorting through the noise for the signal. The best we can hope to do (whether expert or layman) is narrow the confidence intervals of our predictions a little, and be wrong a little less often.

On the surface, Wrong is about the untrustworthiness of expert advice, but it has much deeper implications. As William James observed a century ago:

Truth lives, in fact, for the most part on a credit system. Our thoughts and beliefs “pass,” so long as nothing challenges them, just as bank-notes pass so long as nobody refuses them.

Even though experts might eventually converge on the truth about specific clearly defined causal relationships, across the multitude of main effects and complex interactions that each of us experiences in a typical lifetime, the saying, “the truth will out,” is false. At any given time, a substantial number of our individual and shared beliefs about the causes of those effects are simply wrong. We’re guaranteed to take to our graves false conclusions about why this diet did or didn’t work for us, why that relationship went sour, or whether our grown child wouldn’t have committed spousal abuse if only we’d spared the rod.

The degree to which the truth doesn’t “out” is the latitude that experts – and anyone, for that matter – have to construct social realities with impunity. For the objectively false claims described in Wrong and others we hear every day to have the power they do to launch movements, sell products, determine government policy and distribute social rewards, being right doesn’t matter as much as being accepted.

That, I think, is the real lesson of Wrong.

This review appears in the July 14, 2010 eSkeptic newsletter, and will be published in Skeptic magazine Vol. 16, No. 2.

You may have heard about Google Wave, the Next Big Thing in online collaborative tools, if the Internet giant has anything to say about it.  It’s an ambitious and somewhat complicated way to coordinate and record multimedia collaborations.  When Slate columnist and author Farhad Manjoo took Wave for a test drive, he was put off by the fact that others could see his individual keystrokes as he typed in it.  Unlike other text-based applications (like instant messaging), which don’t send anything until you hit Enter.

Few of us type perfect and final sentences every time.  Sometimes we hit the wrong key; sometimes we back up and change words.  If you’re IM-ing and type, “That guy is a nut case unusual,” your recipient will see only the post-editing version, but if you’re typing in Google Wave, other participants will see the whole thing as you type.

Manjoo felt like the application was allowing others to look into his head while he thought.  We picture ourselves having little editor-censors that allow us to think whatever we want, but get to approve the final, expressed version.  In fact, our processing of information is continuous and reiterative, such that the pronouncement of the contents of an information stream at any point as “final” is arbitrary and misleading.  The figure below plots different kinds of cognitive output by the amount of processing that typically goes into each.

The process (or more properly, one cycle of it) begins in thought, where we assemble fragments of ideas into a coherent expression.  Once we’ve chosen the initial words, our fingers start to type, although we’re still building the expression and could change our minds, which is what makes watching a Google Wave comment under construction feel like peering into the skull.  Speech is only a little less granular: e.g., there are nine letters in the word “objective” but only three phonemes.  Multiply the cognitive processing we put into an instant message several-fold and you have the editorial control we typically exercise over blog posts, which are longer and have more elements to coordinate.  As with IM, we don’t hit Enter and publish them until we’re satisfied with the whole thing.

The two rightmost examples go extra-cranial and pool our processing power with others’.  An email thread between a pair of correspondents trying to figure something out has the benefit of both repeated iterations and two different perspectives to refine its conclusion.  An even greater number of authors, and more deliberation and verification, are often (but not always) involved with the creation of a Wikipedia article. 

The output of cognitive processing is typically a representation of objective reality; a truth claim about that reality.  That’s mostly what we think, talk and write about.  On the surface, it might seem that the more time and effort we put into building that representation, and the more heads we can combine to gain the benefit of additional brainpower, the more likely we are to “get it right” – to end up with an expression that faithfully mirrors the objective fact it represents.

But it often doesn’t work out that way.  Additional processing might allow us to crunch more data and catch and correct some cognitive errors and biases, but it can reinforce and magnify others.  If we’re communicating with other people, our goal may not be to collectively discover what’s objectively true, but to manipulate belief.

Even when a thinker with a particularly keen intellect labors over and eventually wins acclaim for solving a problem, the last word may have yet to be writ.  Einstein, Freud and Wittgenstein all repudiated significant portions of their earlier work later in life. Which doesn’t necessarily mean they weren’t right the first time.

Our beliefs and communications hover around the objective truth, but there’s no necessary relationship between how much cognitive work we put into them and how veridical they are.  Additional processing may bring us closer to that truth, cause us to overshoot it, or allow us to steer around it.