Paradox of Causality from Arrow’s Impossibility Theorem

cfrank

Here is a proof of Arrow's Impossibility Theorem:
Youtube Video

I can’t shake the feeling that there is something wrong with this proof, and that there is some kind of subtle logical fallacy. This isn’t because I feel like IIA and Pareto Efficiency are so important, in fact I think they are both somewhat peculiar to begin with, as is the requirement of an output that is a strict total ordering, and I’m not trying to “save” them per se. I have not read Arrow’s own formal proof (yet) or other formal proofs of his result on paper, but if the result follows the same logic as the video, then I have some doubts, and I’m trying to articulate what those doubts are about here.

As explained on the Wikipedia page on Kenneth Arrow's paper "Social Choice and Individual Values," where his impossibility theorem was proved, "Arrow himself expresses hope at the end of his Nobel prize lecture that, though the philosophical and distributive implications of the paradox of social choice were 'still not clear,' others would 'take this paradox as a challenge rather than as a discouraging barrier.'"

In line with this observation, I believe actually that there may be a fundamental issue with the establishment of the identity of the dictator as formally proved by Arrow. If my reasoning is correct, then the formal impossibility theorem has been misinterpreted, and whether a voting system is a "dictatorship" or "non-dictatorship” may actually be inconsequential. In particular, the so-called “dictator” as formally defined by Arrow may very well exist, but this does not actually imply that the “dictator” is in control of the election. Rather, it might just as well be that the election is in control of the “dictator.” Here is a narrative to illustrate the paradox:

Suppose that four children named Alice, Bob, Charlie and Dale have a game night every Thursday at Alice’s house, where they play one game each of Chess, Yahtzee, and Poker. They use a deterministic ranked-order voting system to decide the ordering of the games, and vote at the beginning of each game night, since each week, each of them is more eager to play their more preferred games. They all have agreed on a particular voting system, and are pleased to know for a fact that the system they have chosen to use satisfies the independence of irrelevant alternatives and Pareto efficiency criteria, and will return a strict ordering of the games. They use this system many times, and accumulate an extensive record of their votes and the outcomes of each election. The way they do this is in a specific order: firstly (1) they submit their ballots; secondly (2) they observe the result of the system; and then thirdly (3) they record the result and the corresponding set of ballots in their record book.

On the day following one game night, another child named Kenny moves into the neighborhood. He is invited to Alice’s house that day and introduced to the other children, who mention their voting system and their record book. They also invite Kenny to join them on the game night of the following week. Kenny is intrigued by their voting system and the record they have kept of it, and asks to borrow the record for the rest of the week.

At home, he examines the record. However, he notices that, peculiarly, the other children have not indicated whose ballot is whose! Rather, the ballots are simply ordered from left to right. Tacitly, he assumes that the ballots have been arranged in alphabetical order, and using the exact logic in the proof in the video of Arrow's Impossibility Theorem and the record of the ballots, he concludes that Charlie is a dictator—that is, his conclusion is that their voting system simply ignores every ballot except for Charlie’s.

Before voting on the following game night, Kenny remarks that he thinks the children's "voting system" joke was very funny, since he has concluded that their "democracy" is in fact just a dictatorship controlled by Charlie. The children seem puzzled though, and thinking they are stringing him along, Kenny takes out the record book and demonstrates his proof that Charlie is a dictator. Charlie still seems puzzled, however, and states,

"But Kenny, we actually didn't put the ballots in alphabetical order. In fact, since the order we put the ballots in doesn't change the result of the election, we agreed to record our ballots in a totally random order from left to right each week!"

Kenny is now puzzled. "But... then what have I shown? Is it that the voter who is placed third from the left in this record is a dictator?"

Alice answers, "Well, I think that would be rather absurd, wouldn't it? The 'third from the left' voter does not even exist until after we have already seen the result of our system, and then the ‘third from the left’ voter will be chosen from among us totally at random. Are you suggesting that, somehow, our voting system can predict the future?”

Do you agree that it seems like something is wrong somewhere? That “something” could just be me making some kind of logical error, but I’m not having an easy time finding it if it’s there, and in any case I’m not having an easy time reconciling this narrative. What I mean to illustrate by this narrative is that there seems perhaps to be a causal misinterpretation of the formalism of Arrow’s theorem. Each time the ballots change, it is actually not clear that the voter identities “behind” each ballot have not changed as well. It is therefore not necessarily possible to “fix” the “dictator’s” ballot while simultaneously altering every other ballot, since the identity of the dictatorial ballot is potentially contingent on all of the ballots. If that’s true, then the specific identity of the dictator may not be predetermined by the system, but rather may be contingent on the election results.

I understand that the formality would have us associate the “first” ballot with “voter 1,” and the “second” ballot with “voter 2,” but again, voters 1 and 2 are indistinguishable except for their ballots and the arbitrary numbering. What is stopping us from applying the theorem to conclude that the “third from the left” voter is a dictator?

Just to be clear, I am quite confident that I fully understand all of the logic in the proof of the theorem. It is a theorem and the proof is correct. My skepticism is about how that formal theorem is being interpreted.

Toby Pereira

Well, aside from that, I think Arrow's Theorem is overstated in terms of its importance anyway. Pretty much all ranked-ballot methods fail IIA and if they don't they are unreasonable in some other manner. This has been known for ages anyway as it is a logical consequence of the Condorcet paradox.

So Arrow's Theorem was no great paradigm shift in our understanding. It's a non-event if you ask me.

rob

@toby-pereira said in Paradox of Identity in the Proof of Arrow's Impossibility Theorem: the existence of a “Dictator” is Non-constructive:

I think Arrow's Theorem is overstated in terms of its importance anyway.

Yup. To me it's "importance" is in slowing down progress and adoption of better methods....

I compared it in another thread to the proof that you can't express Pi as a decimal. It's technically true, but given that there is no limit to how close you can come ( https://techxplore.com/news/2019-03-pi-trillion-decimals-google-day.html ), it would be stupid to use it to argue that any calculations that rely on a decimal representation of Pi are unreliable.

Same thing here. Arrow (and Gibbard) are red herrings, when it comes to practical mechanisms for resolving group decisions.

robla

Kenneth Arrow absolutely deserved his Nobel Prize, because Arrow's Theorem was (and is) a big deal. The exact choice of criteria was beside the point; what Arrow did was describe a few "common sense" criteria and then showed them to be mutually exclusive. Later work (e.g. Gibbard-Satterthwaite) had more compelling criteria, but his description (and proof) of how these criteria can be mutually exclusive keeps us from wasting time when evaluating methods.

The metaphor I frequently use: there's no such thing as a perfect vehicle (infinitely fast, spacious cargo capacity, fits in a backpack, completely safe, high fuel efficiency), However, new vehicles come out every year that are purportedly better than the prior year's vehicles, and there are many old vehicles that don't live up to today's safety and performance standards. First-past-the-post barely lived up to 18th century performance standards (and quickly led to the two-party system that George Washington hoped to avoid by creating a norm against parties. We see how that worked out.

Arrow's Theorem has been turned into an excuse by people touting inferior voting methods ("no system is perfect"), but we should be able to explain why the status quo sucks without trashing Arrow and his work.

rob

@robla Yeah I'm not trashing Arrow so much as the people who do what you describe (tout inferior methods and justify it with Arrow and Gibbard).

I do think Arrow and Gibbard have far less relevance to many discussions than many people give them. Note that the "counting criteria" approach is also symptom of the same thing. Basically, it's black and white thinking, which distracts from the non-black and white issues at hand.

Like with your car analogy.... no one complains about a car not being "completely safe", we all recognize that there are circumstances that you can die in a car crash, and that some cars are better than others but none are perfect. If anyone treated that as a binary, most of us would think they were rather clueless.

But when you see this sort of thing: alt text
there is a strong implication that the more criteria a method meets, the better, without regard to the degree that a method handles the issue referenced by the criterion.

Toby Pereira

@robla said in Paradox of Causality from Arrow’s Impossibility Theorem:

Kenneth Arrow absolutely deserved his Nobel Prize, because Arrow's Theorem was (and is) a big deal. The exact choice of criteria was beside the point; what Arrow did was describe a few "common sense" criteria and then showed them to be mutually exclusive.

But my point is that we already knew that. Amongst Arrow's criteria is IIA. And that's the one that always fails in reasonable methods. It's not like some methds fail this, others fail that. How many methods are there that are just dictatorships, for example? It's IIA all the way. Arrow's Theorem can be reworded in plain English as "With a few reasonable background assumptions, any reasonable ranked-ballot voting method fails IIA." And that has been known for centuries. Arrow just dressed it up differently.

@toby-pereira said in Paradox of Causality from Arrow’s Impossibility Theorem:

And that's the one that always fails in reasonable methods

Approval passes IIA but fails universal domain

Toby Pereira

@andy-dienes said in Paradox of Causality from Arrow’s Impossibility Theorem:

@toby-pereira said in Paradox of Causality from Arrow’s Impossibility Theorem:

And that's the one that always fails in reasonable methods

Approval passes IIA but fails universal domain

How exactly did Arrow define universal domain? On the Wikipedia, unrestricted domain is defined as "a property of social welfare functions in which all preferences of all voters (but no other considerations) are allowed", which is a bit vague. What does it mean by all preferences?

Does score fail? What about score that allows any number between the min and max score so that a full preference order can be inferred?

cfrank

@toby-pereira "universal domain" means that the social welfare function is well-defined for all possible (finite) sets of ranked-order ballots, and will always return a total, strict ranking of all of the candidates as its output. Score does not meet the universal domain criterion, since it does not return a strict ranking of all candidates. In particular, it is possible for two candidates to have the same sum of scores, so the score system itself cannot decide on the rank order between those candidates without an additional tie-breaking procedure.

This relaxation of the output structure from strict ranking to a weak ranking is one reason why score procedures are not subject to Arrow's theorem, the other being that the ballots also are not strict rankings. I believe Arrow eventually came to the opinion that social welfare functions are too restrictive and that score ballots without strict rankings would be preferable, possibly since they might be able to satisfy some score-analogs of IIA and Pareto efficiency without becoming dictatorial and probably among other reasons too.

rob

@andy-dienes said in Paradox of Causality from Arrow’s Impossibility Theorem:

Approval passes IIA

But only under some (completely nuts) assumptions. For instance, if there are 3 candidates, and I approve Alice and Bob, but not Chris, for it to be independent of irrelevant candidate Chris, they have to assume that I would still approve both Alice and Bob if they were the only candidates. That makes no sense.

@rob I'm just going off of the definitions, no assumptions needed

Agreed that voters' ballots will likely change depending on the candidates in the race in Approval more with a higher probability than in most other voting rules.

@Toby-Pereira to answer your question I'm looking into it. Arrow's Theorem actually has quite a few (slightly different) formalizations, and it looks like what I said is technically not true for the version defined on Wikipedia since that one only allows (strict) linear orders, but I feel quite sure I saw a formalization where the domain was all (weak) linear orders. I will try to find it.