James's Blog

Sharing random thoughts, stories and ideas.

Progress

Posted: Sep 18, 2020
◷ 14 minute read

I.

The great narrative of the development of the modern human civilization tells of a miraculous story.

Throughout most of history, it goes, human societies mostly engaged in zero-sum games. Because the primary scarcity was food, and its method of production - agriculture - relied on the usage of land, the only way to grow was to acquire more land. Physical geography is both fixed and finite, so in order for one village or kingdom to be better off, it has to take it from another village or kingdom. Sure, in the earlier days, people did discover new land over time, but there were usually already people there (which led to wars), and eventually we ran out of desirable land to discover altogether. The proverbial size of the pie was fixed, and people fought each other for bigger slices.

Then came the Enlightenment, the Industrial Revolution, and the unprecedented amount of knowledge discoveries in the past couple of centuries. Food is no longer the primary scarce resource for most societies. We can produce more than we could ever eat using a very reasonable amount of physical space. We became better off from the massive expansions of trade networks, as well as the numerous scientific and technological innovations that improved our efficiency. Land, and by extension raw physical resources in general, gradually stopped being the bottleneck for growth. Instead, knowledge and ideas became the new primary scarcity, and luckily for us, they are not limited to some finite amount by physical reality. Now the game became positive-sum. The pie that we all share can get bigger if we all just worked together, and everyone can keep the same slice as it will grow over time.

This shift from zero-sum to positive-sum games, as cliché as it may be, is fundamental to how we view the modern society today. It transformed a mostly conflict-driven mode of interaction between groups into a much more cooperation-driven mode. The two World Wars in the first half of the 20th century were really the last hurrah of the older, more rivalrous world view. Most people don’t want war today not because we’ve grown inherently more civilized, but rather because conflict hurts everyone involved in positive-sum games. We all still desire others’ slices of the pie, but why risk potentially losing our own slice to fight others for theirs, when everyone’s slices will get bigger if we cooperated?

However, despite this transformative shift in our world view, two things remain unchanged. One is that we still desire growth. This probably has its roots in our evolutionary history, after all, every parent wants their children to be better off than they were. Two is that we are still the same humans as before, in the zero-sum games era. The same baser instincts of in-group bias and the blood thirsty potential for violence are still ingrained in us, just missing the opportunities to show their claws most of the time.

So what if the modern day positive-sum games cannot last forever? What if it’s already over? The fundamental force that makes positive-sum games possible is progress. If the whole pie does not grow over time from scientific or social progress, the game reverts back to zero-sum, and we will begin vying for each others’ slices once more. Unfortunately, on the subject of progress, things don’t look very good. In recent years, more individuals have become interested in the question of the speed of progress, and their assessment, which I tend to agree with, is mostly negative: progress is slowing down.

II.

There are many who disagree with the claim that the speed of our progress is slowing down, and I think this disagreement mostly comes from the confusion over the different kinds of progress. The classic Technological Revolutions and Financial Capital by Carlota Perez contains a closely related idea that will help illustrate this. In Perez’s theory of technological revolution, there is the concept of the S-Curve. According to her, every major technological revolution goes through 4 phases, over a period of roughly half a century:

  1. Irruption: the initial emergence of a new paradigm. Early new products and industries begin to form; growth is explosive and speed of innovation is rapid
  2. Frenzy: full constellation of new industries, technology systems, and their supporting infrastructures take form
  3. Synergy: complete expansion of innovation and market potential within the newly formed industries and systems
  4. Maturity: last of the new products are created, with earlier ones approaching maturity and market saturation. Growth and innovation plateau

Phases 1 and 2 together make up what Perez calls the installation period (phase 1 is essentially the “zero to one” idea from Peter Thiel), while phases 3 and 4 together are known as the deployment period (the scaling from “one to many”). Sustained progress, as some argue, relies on the repeated layering of successive S-Curves over time. As the last innovation S-Curve hits its stagnant 4th phase, a new paradigm shift must be discovered which will carry us forward over the next few decades (see this diagram for a clear illustration).

Extending this idea more broadly, we can divide progress into two types, which I will call true and ancillary. True progress consists of inventions, discoveries, innovations, or social structural changes that directly and significantly improves the rate of progress itself. All other forms of progress are ancillary. A few things of note:

  • Not all inventions or innovations fall into these two categories, as some are not “progressive” at all. To be considered as contributing to progress, they must have a net positive affect on the world, i.e. making people better off. The Middle Ages had tons of innovations in torturing devices for example, but I would not consider them any type of progress. In fact, there are inventions and innovations that are arguably regressive, contributing negatively to progress
  • The true vs. ancillary progress distinction is not quite the same as the S-Curve phase division, or the “zero to one” and “one to many” division. There are many technological innovations that spawn new products, markets, and industries, that do not significantly impact the rate of progress itself. These would be considered going from “zero to one”, yet I would consider them ancillary
  • Like most categories, these types are not strictly binary. Instead, all inventions and innovations fall somewhere along the spectrum of “more true” vs. “more ancillary”, depending on their net effects on the speed of progress itself

True progress enables a positive feedback loop, since it directly impacts the speed of progress itself. It has a second-order effect: not only does it make everyone better off (the first-order effect), it also makes it easier to create or discover other things that will make everyone better off. The invention of writing is a good example of true progress, and the Internet may be another1. However precisely because of this compounding effect, true progress cannot be sustained frequently and continuously over long periods of time. Anything that compounds gives rise to exponential growth, which is impossible to continue forever in the finite universe that we exist in. In practice, this is not at all something we have to worry about, as the sheer difficulty of generating sources of true progress is so high that our ability is a far more critical bottleneck than the fundamental constraints of reality. Whatever the limitation, the point is that true progress cannot occur continuously, and will forever happen only in infrequent, revolutionary occasions, similar to the successive layers of S-Curves of innovation over time.

But at least we still have all the ancillary progress, right? One of the common hypotheses to explain why the apparent speed of progress has slowed down is the idea of the low-hanging fruit, that all the easy inventions and discoveries have already been made, and so to make new ones is much harder and therefore slower. It is questionable whether this is a major contributor to the slowdown of progress as seen in recent times, but the general idea that it describes seems practically true: progress becomes increasingly more difficult as we move forward (see this paper for some concrete data, for example the graphs on page 1111). True progress however, is able to offset this, with its own positive compounding effect. But in the absence of any significant true progress over a prolonged period of time, the difficulty in making new progressive discoveries and innovations may become so high that even ancillary progress may appear to have flatlined. After all, even an infinite series of positive values could sum to only a small, finite amount. This is why I’ve given “true progress” its name like so, instead of some other more descriptive term (such as “compounding progress”): because in the long run, it may be the only real source of progress.

I suspect that those who don’t think the speed of progress has slowed down in recent years may simply be blinded by the amount of ancillary progress being made. They may look impressive and bedazzling from up close, but in the grand scheme of things (i.e. zoom out, think in decades and centuries instead of mere years), they are not moving the needle by much and matter very little.

III.

If true progress can only happen on infrequent, rare occasions, and in between, ancillary progress may become so slow as to plateau, what does it mean for the way we see the world? It seems that the idea that we have permanently progressed from playing zero-sum games to positive-sum games may be too good to be true after all. Since our positive-sum game relies on progress, if the speed of our progress flattens, then the world may revert back into a zero-sum state. Once the pie itself stops growing, the only way to quench our thirst for growth is to start taking slices from others.

In fact, it may be that zero-sum is the default and equilibrium state of our world, and that we can only ever temporarily escape it during periods of transition to a higher equilibrium state. Such transitions are triggered by the creation of some significant amount of true progress, and as a result, the world for a short period becomes positive-sum, as we are treated to the surge of new possibilities and large amounts of ancillary progress. But since true progress is not continuously sustainable, neither is this positive-sum world. As the initial excitement dies out and progress inevitably slows down again, we will have settled on another new equilibrium state, one level higher but nonetheless zero-sum.

Ancillary progress can help bridge the gap between successive times when true progress is made, to make it less stagnant or at least appear to be less stagnant. But they cannot do this forever, the low-hanging fruits in the new garden created by the last source of true progress will eventually be picked clean. If we don’t reach the next equilibrium state via a transition, this illusion will be dispelled sooner or later and we may descend into the conflict-driven world of zero-sum games once more.

I fear that we may be getting close to this point today, though I sincerely hope I’m wrong. The ancillary progress that has been masking the lack of true progress in recent decades is running out of steam. Having only lived in the most peaceful and prosperous positive-sum game period of human history, the thought of the world returning, even partially, to some zero-sum arrangement is quite alarming. Major shifts in people’s world views like this do not happen suddenly, they develop gradually over years and decades, which means that some of these trends can already be observed in recent years.

Given that sources of true progress are not predictable, one thing that is somewhat within out control is our expectations. If we could somehow adjust our expectations and stop our obsession with growth, or at least be okay with mere incremental growth, then a zero-sum world, even if it comes to be, may not be so bad. We can simply be content with the slice of pie that we’ve each got already, and live on until the next pie-growing event occurs. But this is a terrible strategy. First, the expectation of constant growth is not only deeply entrenched in our minds, but has also shaped our modern social structures, including the entirety of our current financial system. These structures will likely stop functioning properly if the desire for growth stops, which could lead to a disaster of its own if we do not find appropriate ways to revise them. Second, although a population-wide lack of desire for real progress may curtail the conflicts from zero-sum games, it may also perpetuate a self-fulfilling prophecy, forever impeding true progress.

So our best bet is to push forward, and try our hardest to discover the next wave of true progress before the world devolves into some zero-sum configuration from the stagnation. Zero-sum may be the default equilibrium state of the world, but if we can manage to do this successfully and repeatedly, we may just be able to perpetuate the positive-sum game as a quasi-permanent state (using on ancillary progress to bridge the years in between the occurrence of true progress).

IV.

Modern day rationalists, despite their interests in studying progress as a subject, are perhaps ill-suited to be the drivers of true progress. From how to give away your money to charity to the study of certain hypothetical future threats to humanity, hyper-rational individuals can’t seem to live without logically sound justifications and ostensibly comprehensive analyses. Progress, on the other hand, almost never works out like this. They often start out as crazy “that would never work” ideas about the future, and need the continued efforts of groups of equally crazy “delusional” individuals to have a chance of being realized. This is not to say that these people don’t think rationally - they absolutely do - but simply that the case for why their ideas will work out can never be justified to the degree that would satisfy a diehard rationalist.

A kind of irrational optimism (or dare I say, a certain faith about the future) is often required to make true progress. Maybe the increased emphasis on having strong rational arguments for decisions in certain circles of the intellectual community is costing us some non-negligible amount of speed of progress. Even the popular phrase about taking risks says that we should “take calculated risks”, implying that the risk-to-reward ratio should be more or less quantified, and must make sense before we proceed. For a better shot at true progress, maybe what we need are Bigger, Hairier, more Audacious Goals, and more smart people to rally behind them despite the lack of concrete evidence for success.

Failing to accomplish that ourselves, we can also be “forced” into it, when faced with a credible, tangible existential threat. For this to work, the threat must be overwhelming yet not totally impossible to overcome. This is why World War II worked wonders to accelerate progress, as did the Cold War - even with Mutual Assured Destruction, people did believe that we could eventually triumph2. The threat of climate change is the modern day threat for us, but I feel that it’s simply not visceral or material enough of an enemy to drive us. It’s a bit depressing to think that we may be complacent by nature, and that great impending calamities are needed to force us to push ourselves forward.

Money seems to be a good alternative to the threat of an apocalypse, but it doesn’t work well everywhere. Most importantly, I don’t think it works well where it matters the most: in areas of potential true progress. For money to be a driver of progress, the inventors, researchers, and investors must be able to capture the gains produced, in order to generate the returns that justify the capital investment. This is easier to do in the upper layers of the “innovation stack”, but a lot harder in the more foundational areas. Generally, the closer something is to people (or “users”, as the modern tech industry calls them) directly, the easier it is to capture value, because it is less fungible. Yet the more fundamental inventions or discoveries, often containing the most valuable knowledge about the nature of reality and are the sources of true progress, are generic and hidden from people’s eyes, making it nearly impossible for their creators to capture the immense value they generate.

We see this throughout the history innovation. The Internet, for example, which has spawned trillions of dollars of value to society, yielded nearly no direct payout for its inventors. On the other hand, Instagram, which is nowhere as transformative, and wouldn’t have been possible at all without the Internet, made its creators rich beyond belief in a short few years. Pure capitalism and true progress may lack symbiosis precisely because of this asymmetry in the ability to capture value. The work that gets funded easily tend to result in ancillary progress, while the things that could result in true progress struggle to attract capital. The blindspots of profit-driven investors may be exactly where the next source of true progress lays hidden.

So here’s a potential recipe for finding more sources of true progress. Get more people to have some clear, crazy visions for transformatively better futures, unbounded by whether they will profit from it (of course, if possible, all the better). And with a healthy amount of irrational or blind optimism about the likelihood of realizing that vision, work hard towards it. Bootstrap the endeavor if possible, so they can’t be forced into making suboptimal strategic decisions by some capital-return-obsessed third party. Or die (in the degenerative zero-sum world) trying.


  1. There are certainly parts of the Internet that I would consider as sources of true progress. But there are also large parts of it that is not, or even contributing negative progress (e.g. many, but not all, social networks). The overall net effect is hard to determine, and I think even today in 2020, it is too early to say whether or not the Internet has been a “true progress” type invention. The potential is definitely there, but the current state of the Internet leaves a lot to be desired. ↩︎

  2. Some hold the opinion that in the later years of the Cold War, MAD became so overwhelming that it pushed many people into various forms of escapism, and that it was not positive for progress. But I think a lot of progress was made in the humanities and art during this period, including a lot of significant progress in social structures. They may not be direct scientific progress, but true progress often do take the form of social changes. ↩︎