Many rationalist blogs and communities online have been using an epistemic confidence marker on posts, to indicate the level of certainty that the author feels about the content. There are typically 5 or so levels, ranging from the most confident to the least. I don’t think that these epistemic confidence levels have been formally standardized, but you can find one version of the description here.

Of course, most things in the world are not labeled with such markers. And so, as a relatively skeptical person, I mentally assign an epistemic confidence status to everything I read, based on the various factors that I can perceive, including context, origin, author, my own level of expertise on the subject, quality of content, etc… I noticed however, that in my best effort to avoid being a man of one study, I don’t dole out my epistemic judgement equally. There are certain subject areas that I have a much higher level of skepticism for. In other words, the default epistemic confidence that I assign to works in these areas are much lower than normal. I’m not sure whether these discriminations are fair, but I’ll list a few of them below, along with some thoughts.

Health science. I think there are some things in health science that we know with relative certainty, but almost everything in the media today fall outside of this boundary. For example, I think we know, with a good degree of epistemic confidence, the deficiency and toxicity levels of each of the vitamins. But for what we really want to know, i.e. the “optimal” amount of vitamins for a healthy life, I think we know pretty much nothing about. The focus of attention in health science these days (e.g. nutrition) is precisely where we have near-zero epistemic confidence in. As a result, I treat almost everything in health science as fiction these days, otherwise it is difficult to reconcile how everything causes and prevents cancer simultaneously.

Cognitive science. This is a bit of a mild one. My negative bias against cognitive science is mostly reserved for the theories that people invent to explain how the brain and consciousness works. I don’t have a lower than usual epistemic confidence in the data that cognitive scientists collect (unlike health science). The frameworks that people form in order to explain the data though, is much more dubious in my opinion. I think that our understanding of the brain is still in its infancy, and as a result, there are many different yet equally plausible theories to explain the things that we observe. I think of most theories in cognitive science as just lenses to look at things with, some of which are interesting and sometimes useful. Even the most popular and accepted frameworks, such as system 1 vs. system 2 thinking, suffer an epistemic confidence reduction in my mind.

High-level analysis of broad, complex issues. This includes most things in economics, politics, history, and sociology. Almost nothing in a complex system is explained by a single cause, or even a few simple causes. A proper analysis of such systems would most probably have to be equally complex, or vague and inconclusive. You would need people who have expertise in a broad range of subjects, and have deeply integrated their knowledge across all the disciplines, in order to have a chance at a fair analysis. This is arguably impossible today, and instead the people who are trying to analyze these issues are only specialized in some specific niche areas, which are not enough to form the complete, accurate picture. Not only that, but their analysis in the niche is usually so detailed and precise, that it actually can give us the illusion of elevated overall correctness, despite being completely off. I typically look at these complex issues with a meta-analysis approach¹, which should be less prone to these issues. This naturally means viewing each piece of the meta-analysis (i.e. the individual analyses) with a more skeptical mindset.

Archeology and paleontology. My reservations here are similar to the ones for cognitive science. For the most part I don’t hold a negative bias against the specific findings of archaeology and paleontology. But when people try to generalize these findings to the period that they are from, my skepticism alarm bells begin to ring. Since it is so difficult for things from the long past to be preserved, the data that we have to study (e.g. fossils, archaeological sites, even written records) is but a tiny glimpse into the past periods. On top of that, it is a heavily biased glimpse. It’s a bit like peering into a house via a tiny hole in the door, where we can only see a specific corner of the room, and trying to infer what the rest of the house interior is like.

Here I must admit to my own biases and ignorance. I’m sure that my epistemic confidence judgement system is flawed. I am most likely being unfairly distrustful of certain things, while being oblivious to the lack of epistemic foundations in others.

That being said, looking at the areas that I apply the epistemic confidence discount on today, some patterns seem to emerge. I am generally more skeptical of things that are too complex for us to analyze adequately, and things for which we are simply missing (good) data on. I also think that one should especially beware when these areas intersect with areas of mass popularity (nutrition science is a good example in recent years), because there will be much more opportunities to be misled.