Words, especially when wielded by a skilled dialectician, can be a very powerful tool to affect change. History seems to show that under the right circumstances, anyone can be convinced of anything with words. And the circumstances, or contexts, are critical here, perhaps equally so to the words. After all, me telling you to “hand over your wallet” is a lot more convincing when I have a loaded gun pointed at your face. But I have been thinking about the power of words in a more abstract, theoretical sense, devoid of the context. Suppose a person is in a natural, resting state, and is presented with some words, written or spoken. Does there exist a set of words, tailored to the individual, that would convince the person of, or to do, almost anything?

Of course, there is no easy, formal way to investigate this question (it isn’t even very well defined). But my hunch is that the answer is yes, that for any individual (assuming they are literate and somewhat intelligent), there exists some set of words that can convince them of anything. Finding that effective set of words for the an individual and a given idea to convince them of may be exceedingly difficult (in fact I think it is almost certainly reducible to the halting problem, and thus not computable), but it still exists.

If this is true, then there exists a set of words, perhaps not even very long, that when shown to an individual, can quickly lead to a much larger scale event. The initial individual can be convinced to create other sets of words to convince others of things, in a chain reaction that expands and eventually spreads to the whole world. This could lead to some global catastrophic event, or something positive. This is nothing new, some crude form of this happens all the time, and has been happening throughout human history. Demagogues, priests, scholars routinely do this to the people, settings into motion wars and periods of prosperity alike. But here I’m talking about the theoretical existence of some optimal set of words that can do anything to anyone, and is maximally effective, which is quite a bit scarier.

This theoretical question may not be as pointless as it seems. Currently we see tremendous progress in the world of generative AI, particular in text generation. The recently released GPT-3 is able to output some very impressive (and hilarious) long form texts based on a very short prompt, that are very coherent and intelligible. If the answer to the question posed above is even a soft yes (i.e. words can convince people of many important, but not all, things), then the potential power wielded by these text generation AIs could be great. This is the main reason OpenAI has been very hesitant to release full models of the GPT-n series of work. The “evil AI that will take over the world” trope may come to be in the form of some future iteration of GPT, seemingly benign and only generating text.