A recent Farnam Street post talks about Chesterton’s Fence, a very popular concept in the rationalist community. The author talks about second order effects as the key takeaway: changes often have unintended second order consequences, so we need to understand the original state of affairs (i.e. why the fence was placed there) before making our change (i.e. removing the fence). Although this is the literal interpretation of Chesterton’s example, I think it’s missing a key point, that of epistemic humility.

Yes, Chesterton’s Fence is about the dangers of rash, inconsiderate actions by hubristic actors. However, saying that we need to understand the original intent and be aware of second order effects, then ending the story there, only shifts the location where hubris begins. It does not change our thinking qualitatively. It is a perspective that is still filled with a sense of epistemic confidence, that we can truly understand things if we just took into consideration these additional data points.

But reality isn’t so simple, and this isn’t enough. Once a fence has been placed within a complex environment, it will begin to accrue lots of effects that not even its original creator would know about, so understanding the intention is not enough. Effects also don’t stop at the second degree. Second order effects themselves trigger additional higher levels of effects and so on, which makes predicting things impossible, even when you are aware of the possibility of such effects.

Yet we need to act, to make changes, to take down fences, lest the world would stagnate and slowly decay into darkness. The more fundamental idea behind the Fence is that there will always be unknowable higher order effects (good and bad) for any change we make, and that our decision should always take into account both the effects that we know about (i.e. can reasonably predict), and the possibility of the potential unknowable effects. It’s not fooling ourselves into thinking that we can anticipate Black Swans, but to act with consideration of their inevitable future appearance. Practically this means that for a change to be justified in an existing functioning system, the foreseeable benefits often need to be a bit larger than seem reasonable, in order to offset whatever unknowable harm the change may bring.