James's Blog

Sharing random thoughts, stories and ideas.

Malformed Questions

Posted: Jun 7, 2022
◷ 3 minute read

I like the idea of having no stupid questions. I enjoy both asking and answering them, as these are often situations where lots of learning take place. But questions can be bad or malformed, which is different from “stupid”. Here, by malformed I’m referring to questions which are framed - intentionally or unintentionally - in ways that actually make it more difficult to get to the desired answers. Sometimes they are bad in such non-obvious ways that trick even the answerer.

A fairly well-known type of malformed questions is the leading question, where the framing contains within it a preferred way of thinking. We are perhaps more familiar with these questions in the legal context, where law enforcement and lawyers exploit them to get people to answer in a particular way. But it shows up everywhere. For example, asking “what should we do about problem X” can be leading, because it implicitly carries the assumption that we should do something about X, and that the only thing to determine now is what to do.

Another way that questions can be malformed is through what I’d characterize as “forced dimensionality reduction”. The simplest version of this is asking binary-answer questions about a complex topic. “Is personality based on nature or nurture” is a quintessential example of this, as is the recently popular practice of “fact checking” (the correctness of statements is not necessarily binary). But there are also many less obvious ways that this type of question can appear.

Take the question of “why did the Industrial Revolution happen when and where it did”. At a glance, this seems like a perfectly good way to ask the question. Yes, the topic is very complex, and any responsible answer will surely be appropriately nuanced, taking into account many factors of consideration. Yet I still think the question is somewhat malformed, and the trouble is in the word “why”. Most causal explanations that we encounter, as demanded by “why” questions, tend to be low dimensional, and have exclusive or unique answers. They are things like “gravity causes the apple to fall from the tree”, or “John had to borrow money from his sister because he overspent his salary on an expensive car”. But some causal explanations are not unique. Say, after extensive research, a group of scholars conclude that the Industrial Revolution happened when and where it did due to 100 different factors. If 20 of these factors were changed or removed, it may still have happened, with only minor differences, because the remaining factors are enough. Instead of asking why, the more fitting way is to ask about factors of contribution and their relative strengths.

As a side note, forced dimensionality reduction is also why trolley problems and many utilitarian arguments don’t really work. Answers that are completely correct in the simpler (i.e. lower dimension, forcefully reduced) version of the problem may be entirely inappropriate in more complex real life situations. We tend to just believe in the power of extrapolation in preserving logical coherency, but there is no reason why it always works, and often it doesn’t.

The last type of malformed questions I’ll talk about here is also the most difficult to identify. First we need to observe that the range of answers for the domain of questions is not closed within the realm of words. Put another way, not all questions that can be formulated as words have answers in words. The easiest to understand example is the question “what is the melody of your favorite song”. The most appropriate answer is to hum or sing the song vocally, and not a verbal description of the melody. Of course, I would not consider this question malformed, since it does not restrict the answer to be in only words, and so it doesn’t make it harder to get the desired answer. But sometimes we do put this restriction on the answers we expect, often implicitly, and that’s when things go wrong.

The best example in my mind of this type of error is the question “what is the meaning of life”. It is almost always asked with the expectation of receiving a proposition (or a set of propositions) as an answer. But it may very well be that the appropriate answer to such a deep question at least in part lies in the non-verbal domain. Maybe it is partly verbal and partly experiential, which cannot be described adequately. And so whenever I am asked this question, I am tempted to say “sorry, but there is no propositional answer that I can give which would convey my answer properly”.