Clay Shirky on the use of evidence in society (his response to the 2007 Edge question)

Clay Shirky's response to last year's Edge question "What are you optimistic about?". Very good I think.

Evidence. 

As schoolchildren, we learn that different weights fall at the same speed. This simple and readily tested observation, first published by Galileo, refuted Aristotle, who claimed that heavy things fall faster. As Galileo put it in Two New Sciences "I greatly doubt that Aristotle ever tested by experiment whether it be true..." We are left to wonder how people could have believed what they were told, and for two millennia at that, without ever checking? Surely the power of evidence over authority is obvious. 

Except it isn't. Even today, evidence has barely begun to upend authority; the world is still more in thrall to Aristotle than Galileo. As a simple example, the time-honored advice for those suffering from bad backs has been bed rest. Only recently, though, have we discovered bed rest isn't the best treatment, and isn't even particularly good compared to moderate activity. How did this discovery come about? A researcher in the field of Evidence-based Medicine surveyed multiple databases of trials and results for patients with back pain. (It tells us something about medicine's current form that we even need a term like Evidence-based Medicine.) And why did it take so long to look at the evidence? Same reason it took so long to question Aristotle: some doctor in the distant past reasoned that bed rest would be a good idea, and it became the authoritative and little-questioned view. 

In school, the embrace of evidence is often taught as if it were a one-time revolution that has now been internalized by society. It hasn't. The idea of evidence is consistently radical: Take nothing on faith. No authority is infallible. If you figure out a good way to test something, you can contradict hallowed figures with impunity. 

Evidence will continue to improve society, but slowly — this is long-view optimism. The use of evidence dragged the curious mind from the confusion of alchemy into the precision of chemistry in the historical blink of an eye, but its progress past the hard sciences has been considerably slower. Even accepting that evidence should shape our views is inconsistent with much human behavior. Everything from the belief in supernatural beings to deference to elders pushes against the idea that a single person, if he or she comes to understand the evidence, should be allowed to upend a millennium of cherished human belief. 

It is only in the last hundred years that evidence has even begun spreading from the hard sciences into other parts of human life. Previous platitudes about the unpredictability or universal plasticity of human behavior are giving way to areas of inquiry with names like Sociobiology, Evolutionary Psychology, and Behavioral Economics. (That we need a label like Behavioral Economics says as much about economics as Evidence-based Medicine does about medicine.) As reliance on evidence spreads, it takes with it an understanding of how it works. Apologists for religion often bolster their claims by noting that it is impossible to disprove the existence of supernatural beings. This argument assumes that their listeners don't understand how evidence works — it makes sense to believe in things for which there is evidence, and no sense to believe in things for which there is none. As evidence moves out of the lab and into everywhere else, rhetorical tricks like that are going to be progressively less effective. There will still be fundamentalists, of course — probably more of them, as improved evidence requires a heightened ability to shield the mind — but the oxymoronic middle ground of 'religious but reasonable' will become progressively harder to occupy. 

This isn't just about religion, though. Most of the really important parts of our lives ·who we love and how, how we live and why, why we lie and when — have yet to yield their secrets to real evidence. We will see a gradual spread of things like evidence-based politics and law — what is the evidence that this expenditure, or that proposed bill, will have the predicted result? The expectation that evidence can answer questions about the structure of society will discomfit every form of government that relies on sacrosanct beliefs. Theocracy and communism are different in many ways, but they share the same central bug — they are based on some set of assertions that must remain beyond question. 

Social science is expanding because we are better about gathering data and about understanding it. We have gone from a drought to a flood of data about personal and social behavior in the last generation. We will learn more about the human condition in the next two decades years than we did in the last two millennia, and we will then begin to apply what we learn, everywhere. Evidence-based treaties. Evidence-based teaching. Evidence-based industrial design. Evidence-based parenting. 

There will always be some questions we can't answer, but they will be closer in spirit to "Who put the bomp in the bomp-bah-bomp-bah-bomp?" than to "Why do fools fall in love?" There is an astonishing amount of work going on on that latter question right now, and there's a reasonable chance we'll have a really good answer, to it and to thousands of other questions once thought to be beyond study or explanation, in the coming years.