Fact checking and linking to supporting evidence

With all the talk about 'fake news' and the spreading of lies and misinformation on social media, the idea of fact checking has been thrown about a bit.

In this post I'd like to try to situate fact checking within a broader context. I've spoken before about "Improving the spread of truth in the information ecosystem". The basic idea is that we want to find ways to help the truth spread, and to hinder the spread of falsehoods. In that post I gave some fact checking sites as examples of initiatives in this area.

To me, what fact checking is about is trying to enable people to create explicit connections -- links -- between claims and evidence. Fact checking is where you're doing so to check someone else's claim, to see whether it holds up. But connecting claims and evidence also includes where you're using evidence to substantiate your own claims.

The ideal situation, in my view, would be where whenever someone makes a claim, they also try and link to sources that substantiate that claim. Where the default is to always try to show evidence.

This, of course, would not be easy to achieve. But it's worthwhile considering how we can make it as easy as possible to for this to be done.

What I'm suggesting might sound like the way that scientific papers cite sources. I actually think we can do better than this. 1) usually whole papers are cited, and these are often too granular. You have to dig around in the paper to find the bit specifically related to the claim being made, and do a bit of reverse engineering to see why/how that claim was being arrived at in the paper. 2) papers aren't laid out with the intention of clearly showing the evidence supporting the specific claim.

I'm thinking of a setup where a person could link to a web-page that is specifically devoted to the evidence for a particular claim, which might link to other such pages for sub-claims.

The end-goal would be having substantial numbers of people -- volunteers and those supported by institutions -- focused on creating such pages. And where there was a norm whereby people would always try to link to substantiating evidence whenever they make a non-trivial claim (where you'll at least feel uncomfortable when you don't do this).

Of course the big question is, how could we get to such an end-goal? I don't think it'd be easy. I think if it were to happen, it would be drawn-out process, involving small, piece-meal steps forwards.

Momentum would need to grow. I can imagine more and more datasets and statistics being put online, and then people utilising these to create pages showing the evidence for specific claims based upon these (ideally, and whenever possible, they would show their claims as being the results of queries on the datasets, queries that could easily be independently verified). The more that such a thing is done, the more the idea of doing such a thing has the potential to spread.

Sources of such information would need to establish their integrity. So they are known as being a trustworthy source of evidence on claims. Ideally they would be non-partisan, though that obviously couldn't practically be done for all claims.

In addition to making the information available that shows the support for particular claims, we'd also need to make it much easier to find and make use of this information. To lower the friction involved in linking to it.

This is a large topic in itself, but here is an example of what I'm thinking of: auto-suggesting evidence sources and particular claims, through the use of machine learning. Think of a particular forum where people discuss a topic, such as politics. It might be a sub-reddit on politics. A machine learning tool could analyze the text of all the discussions on the forum to get a sense of the kinds of topics discussed, and to get an idea of what evidence sources are relevant to the forum. Whenever a forum member goes to write a comment, it could use the context of the comment they are replying to (and what it is replying to, and so on), plus the comment-text the member has written so far, to suggest possible claims (and evidence pages for them) that the member might be making.

I have no idea how far away we are from a sufficiently-effective tool of this sort -- it's just meant as an illustration of how we might be able to lower the friction for linking to supporting evidence.