The value of basic research

I believe that basic research has far more value to society than is generally recognised. I think it is severely undervalued, across pretty much all segments of society, including many people who work at research institutions such as universities. And I believe this is because people misunderstand its nature, and have the wrong framework for thinking about its value.

In this post, I want to do some initial 'gathering of my thoughts' on this topic.

Basic or applied research is intended to uncover some new knowledge or develop some new technology (so what I'm talking about also includes basic research as part of developing devices and products). Central to the misunderstanding of basic research is the idea that you should evaluate the potential value of some research by looking at what problems it would solve.

So obviously when it comes to applied research this is how we evaluate any proposed research. Basic research, on the other hand, is by definition not about solving specific real-world problems. This doesn't seem to stop people from wanting justifications for the research on the basis of what sorts of real-world problems its results could be applied to. And because basic research isn't focused on solving problems it is typically seen as being about "satisfying curiosity".

So basic research doesn't seem very valuable. It's hard to think of the specific real-world problems it's going to solve, and framing it as "satisfying curiosity" makes it seem of little real-world use, and certainly not something deserving of significant amounts of precious governmental funding.

Seeing the value of potential research in terms of solving real-world problems sounds good. Who wouldn't want to solve real world problems? Isn't anything else being self-indulgent? But, I argue, while this sounds good in principle, it is a lot more problematic than it seems in actuality.


Before I start talking about this framing of basic research, I want to argue that, even for applied research, "what real-world problems could this solve" isn't actually a particularly good way to evaluate potential research. For starters, remember that we're talking about potential value here. It's all well and good to make claims about what sorts of problems the research is going to be potentially useful for. But we have to remember we're talking about a claim, not the actual reality. We're talking about a prediction, not what pans out.

It's very easy to claim some research is going to be useful for some problem, but the real, important question is, does it actually turn out to be useful for it? People are notoriously poor at making predictions. The world is very complex, and we are working with very limited information and aren't very good at making predictions with it. I would bet that a high percentage of research that is claimed to be for solving some particular problem doesn't turn out to be useful for it.

You might respond, but that's just the nature of research, what better alternative is there? My answer to that it is better to evaluate the potential of the researcher rather than the claims of what problems the research is going to address. There's been calls for this in science funding. And this is a lesson that's been learned in the (roughly) post-2000 start-up world (e.g. how Y Combinator evaluates applications made to it), which I think is applicable to research (starting a startup is a kind of research task, basically in figuring out what customers want).


Back to evaluating the value of potential basic research. The key to the perspective I'm arguing for is to understand how research (whether basic or applied) provides value. I am arguing that "what problems it solves" is too narrow a perspective, and that the value it provides comes from what it enables.

To see what I mean by "enables", consider any historical advance that was made through some research, and consider what, if that advance was deleted from history, would not have been practically possible. If calculus had never been invented, how much of the modern world would not have been able to exist? What if Newton's laws of physics was deleted from history? Darwin's theory of evolution? Quantum mechanics? The wheel? The ability to make things from iron? The printing press? And so on. I think we would find that the answer would amount to "a hell of a lot". The research that developed these ideas weren't developed for creating any of the myriad of ways they enable the fabric of modern society, but they were all essential components that enabled those elements of that fabric.

The positive effects of research spans far beyond any problems they were created to solve. In fact, I would argue that the positive effects they have by enabling other developments far outweigh whatever they may have been explicitly created to solve. This is because when we talk about solving problems, we're usually talking about one specific kind of application, but when we talk about what some development enables, we are talking about systemic benefits it has, and systemic benefits are deeper, more widespread, and cumulative.

(Hindsight can obscure this. In hindsight, we can talk of the problem that something like the printing press solves, because we are so deeply familiar with the world that the printing press (eventually) enabled. But at the time things are never this clear, and typically, these things that are so valuable in hindsight because of the systemic changes they enable, are seen by most people as pretty useless).


It will help us if we flesh out this picture of new ideas and technology enabling other developments. There is first-order enabling, where an idea or technology enables some other idea or technology to be developed. There is second-order enabling, where an idea or technology enables some other idea or technology to be developed, which in turn enables yet other ideas or technology to be developed. And in this fashion, there is third-, forth-, fifth- (etc) order enabling. What we are acknowledging here is how complex the web is of the growth of our knowledge and technology. Ideas and technology build on other ideas and technology. The greater the number of previous ideas and technology you have, the more you have to build on, and the more the frontiers of knowledge and technology expand.

If you look at at some technology or knowledge that is vital to the modern world, the web of technologies and knowledge that led up to it (and enabled it) go back a very long way. If you examine that web you'll see that those necessary components come from all over the place. Different ideas or technologies, which were developed at quite different times from each other, by people working in different domains, who developed the knowledge or ideas for quite different purposes. Typically they were seen as having little value when they were first developed.

(The documentary series Connections, by James Burke (Amazon link), provides a pretty good illustration of this process).


The processes of developing new technologies or knowledge are combinative, in the sense that they build on combinations of existing technologies and pieces of knowledge. Basic research provides new building blocks for this combinative process to work with, providing the potential to enable new nth-order developments down the line.

It's not possible to predict where and how a particular piece of knowledge or technology may play a combinative role in enabling some new technology, and thus how useful it will turn out to be. It is, rather, a statistical matter. You can know the overall properties of the process. We can know that, creating new such pieces will overall lead to important developments. We can know that there is value in developing new pieces of knowledge or technology because they increase the number of building blocks that provide potential combinative enablers to useful developments.

Research funding for basic research should be seen more like investing in the stock market. People who invest know that it's a statistical matter: you can't be sure that any individual stock is going to pay well. You know that you need to diversify, and make a number of smaller bets. That overall, the stock market tends to lead to returns. We should be looking at basic research more as an overall endeavour.


The way that basic research is currently funded is analogous to central planning in an economy. It assumes that people can effectively evaluate what is good research to do. And of course, my argument is that, just like with economies, this can't be done. Central planning sounds good on paper, but leads to worse results.

We might ask, how are we meant to choose what basic research to fund, if it is all potentially useful? One part of the answer is one we mentioned earlier: to fund the researcher rather than the research project. That is, to find ways to evaluate the potential of the researcher. Again, this is something that has already been proposed.

Another part of the answer ties in with a concern people have about the perceived lack of utility of basic research, which we might call the "number of angels dancing on the head of a pin" concern. This concern is that basic research will lead to utterly useless kinds of investigation that clearly can have no real-world value.

I think that a lot of this concern comes less from research done in science and more from certain sorts of research done in philosophy. The philosopher Daniel C. Dennett has written a good (and readable) paper how to distinguish between useful and less-useful kinds of basic research (in philosophy), called Higher-order truths about chmess [PDF] (Google search).

As a general rule, I believe that we can avoid the generally-useless kinds of basic research if we focus on trying to understand something new about what exists in reality, or try to develop something new in our ability to manipulate what exists in reality. This rule does not provide any guarantees, but remember that the overall process of basic research is where the value lies. We should see any such input to that process that fits this rule as having intrinsic value as something to look into.


To summarise, instead of seeing basic research in terms of the framing of "what problems does this solve" or as merely "satisfying curiosity", we should see basic research as providing new inputs that can help enable new developments in the statistical, overall process of knowledge/technological development. In this role, history clearly shows it is of tremendous value to society.

.

Update: the ideas in this post are an instance of Considering broader details than specific cases in isolation.