The search for the "holy grail" and non-deterministic methods
-
As many of you will be aware, the search for a "holy grail" of score/approval PR methods has been ongoing for quite some time, but no-one (as far as I'm aware) has managed to get all the necessary criteria together in one method.
Different people will obviously have different ideas on which criteria are necessary, but my own vision of it was always that is was not about practicalities or immunity to voting strategy, but a "pure" form of PR that could work off voters' utility scores, analogous to the plain score voting single-winner method.
Anyway, some of the criteria I felt it should pass are strong PR (or ULC), monotonicity, IIA, IIB. But while, as far as I'm aware, no-one has proved these criteria are incompatible, it does seem very likely at this point.
However, it does seem that we can get basically everything we want as long as we are prepared to sacrifice either determinism or the requirement that each elected representative has equal power.
My general feeling now is that the most "perfect" form of PR is what I outlined here.
"For weighted power within parliament, I would advocate the following (that I've posted elsewhere before). This is the approval voting version:
Work out the probability of each candidate being picked given the following algorithm:
Pick a ballot at random and list the candidates approved on this ballot. Pick another ballot at random, and strike off from the list all candidates not also approved on this ballot. Continue until one candidate is left. If the number of candidates goes from >1 to 0 in one go, ignore that ballot and continue. If any tie cannot be broken, then elect the remaining candidates with equal probability.
The probabilities would be the weight in parliament.
As for score voting, I would always advocate using the KP transformation."
Not that I would recommend giving representatives unequal power in this way, but I think that this is the most mathematically accurate form of PR (unless there's a better score converter than the KP-transformation).
But instead of weighted power, you could apply the method non-deterministically by selecting candidates according to the above randomised algorithm.
I actually think there are advantages to having non-deterministic PR methods. The main advantage is that you can get a better level of PR and you don't sacrifice proportionality by having small regions. If an ideology or party has 5% of the support across the country, then it should get about 5% of the representation. If you have regions with e.g. 5 representatives and it's done deterministically, it might get none. It also simplifies the process. PR methods can get very complex with all the calculating of quotas etc., whereas just picking ballots at random cleans this up. It also means that for politicians there are no "safe seats". There is always a chance of being ejected, so they have to appeal to as many people as possible, not just their usual fan base.
But in terms of a practical non-deterministic method, I don't think I would use the above anyway. Instead you could do this:
For an e.g. 5-seat region, voters simply rank their top 5 candidates. A ballot is picked at random and you elect the top-ranked candidate that has not yet been elected. Continue until 5 are elected. Clean and simple. And I don't see any major problems with it (unless you count its lack of determinism).
Edit - I might also start a wiki page on the "holy grail" method.
-
Lack of determinism seems to be a no go.... especially in our new world of questioning the outcome of every election. What do you do if there is a recount?
That said, I generally avoid worrying about PR because it is impractical in the US (would require too many changes), as well as making parties a part of the system rather than a (mostly) unfortunate side effect.
-
With random ballot methods, it doesn't have to be done by literally picking a ballot at random. The ballots can all be counted and totalled, and then recounted if necessary. Then all the lottery probabilities can be calculated and agreed upon from that. The actual lottery itself cannot be redone and arguably that is the weak link in the chain, but I'm sure a secure method can be found and agreed upon. After all, many countries have prize draw lotteries for large amounts of money.
PR doesn't have to mean parties are part of the system. It depends on the PR method, but what I described above doesn't mention parties, and even STV, which is fairly mainstream, doesn't have parties built into it.
-
@Toby-Pereira "The ballots can all be counted and totalled, and then recounted if necessary. Then all the lottery probabilities can be calculated and agreed upon from that. "
Does that mean the lottery is not done until any recount are completed? Currently, results are available immediately after most elections (often while the voting continues), but with the caveat that votes could still come in and the numbers could be adjusted (and, in rare case, a different winner chosen).
But this system, by my understanding, can't even do that because until the lottery happens, you can't know much of anything. So you have to choose a time to do the lottery that balances two competing factors, 1) the desire to let people know the results as quickly as possible (with minimal unknowns), and 2) the desire to get every last vote accounted for.
What if you do the lottery two weeks after the election (already, a long wait that is going to annoy a lot of people), and then, a week after that, someone presents credible evidence that some ballots were not counted. Currently, that's already messy but you can go ahead and have a recount, and only in the extremely unlikely event that the outcome is changed, it is a fairly minor issue for the public. Here, though, you run into the additional messiness of having to decide if you need to do the lottery again. In theory, the new lottery could change the outcome, when the additional ballots themselves weren't enough to do it.
This is very different from choosing actual lottery winners, because in that case there is no voting and tabulating at all, so you don't have to synchronize it with another process.
So yeah, not to throw a wet blanket on your idea, but to me anything that would qualify as a holy grail would not have any explicit randomness involved, especially if the random event happens after the actual voting and counting.
-
I think there would always be people who are against lottery methods in principle regardless of practicalities. But aside from that, I don't really see the complications you raise as fatal to the system. I don't imagine there would be too many people who would be for lottery methods in principle but would be turned off the idea by the prospect of dealing with recounts.
"Does that mean the lottery is not done until any recount are completed?" - It probably does mean that, yes.
"What if you do the lottery two weeks after the election (already, a long wait that is going to annoy a lot of people), and then, a week after that, someone presents credible evidence that some ballots were not counted." - Well, looking at what happens here in the UK with parliamentary elections, the results generally come in during the night after the voting has been completed over a period of, say, 12 hours. If there is a recount somewhere, then it would delay the result in that constituency, but not normally outside of the normal bounds. The result is then announced and it's official.
It would be most unsatisfactory, lottery or not, if new evidence then came to light two weeks later and an MP then potentially had to be thrown out of their job. I've never heard of it happening, and I don't see why it would be any more likely, or more unsatisfactory, if we switched to a lottery system. Something would have to be decided if new evidence came to light two weeks later, and potentially the lottery redone, but such a rare event I do not see as a deal-breaker.
And finally, in terms of calling such a method a "holy grail", I've always used that term purely with regards to criteria rather than practicalities. Maybe determinism should be considered such a criterion anyway but I see that as a separate conversation.
-
@Toby-Pereira said in The search for the "holy grail" and non-deterministic methods:
It would be most unsatisfactory, lottery or not, if new evidence then came to light two weeks later and an MP then potentially had to be thrown out of their job. I've never heard of it happening,
You haven't heard of recounts? Or only specific to MPs? I'm talking about in the time period between the election and the person taking office. In the US, lots of times there are mail in votes that don't arrive for a while. If the election isn't particularly close, its no big deal. Those aren't recounts per se, but in a system where you are waiting to initiate some outcome-affecting critical step, it all gets weird because you have so much more of a demand to get all the votes counted by a particular time.
and I don't see why it would be any more likely, or more unsatisfactory, if we switched to a lottery system.
It's not more likely, it's just a much bigger problem when it does happen.
Something would have to be decided if new evidence came to light two weeks later, and potentially the lottery redone, but such a rare event I do not see as a deal-breaker.
The trick is deciding when to redo the lottery. You could have people trying to get recounts, not because they think there are enough votes to swing it the other direction, but simply so they can trigger another lottery which itself could swing it.
Especially with what has been going on recently in the US, introducing this extra layer of ambiguity in the process -- another thing for people to fight over when it doesn't go their way -- sounds like a tough sell.
-
OK - I think the main problem is that recounting the votes, or adding in a few late arrivals, in a deterministic election might change the result but generally only if it's close anyway and people will see it as reasonable that a small change can swing it.
Whereas in a lottery method, a small change in the vote count might trigger a new lottery, but because it's a lottery it might change the result quite drastically.
It might be more common in the US, but in the UK I don't think it's normal for a whole load of new votes to come in after the result has been declared. Obviously things could be changed to make that less likely to happen. However, let's say it does happen. I think in cases like this, or other recount cases, it might be acceptable to "amend" the lottery rather than simply rerun it. This would result in a less drastic change, if any change at all.
For example, the lottery is done from five random ballots (or in a mathematically equivalent manner after a full count) and five MPs are elected. Then some new ballots arrive, and these form 1% of the total. Then a new lottery is done but to decide whether any of these ballots should oust any of those already used in the original lottery. Originally there might have been 990 ballots, and these could have been assigned numbers 1 to 990 in terms of their selection order for the lottery. Then 10 new ballots arrive (1% of the total), and they are randomly slotted in the order with the other 990 staying in the same relative order.
Unless any of the new ballots make the top 5 (or slightly more than 5 if some ballots don't give a full top 5), then no change is made. It's extremely unlikely more than one MP would ever change in such a recount, and it would normally be none, unless the count was extremely wrong to begin with.
-
@Toby-Pereira Yes your first two sentences sum it up very concisely. Keeping in mind what we are seeing now with people trying to overturn elections when they see any hint of ambiguity they can exploit. The decision over when to re-do a lottery would be ambiguous.
For what it's worth, I have long thought that intelligently applied randomness in a Condorcet tabulation (to break cycles) would be awesome and address places where strategy can creep in. But I long ago concluded they were a no-go for the reasons above.
-
@Toby-Pereira said in The search for the "holy grail" and non-deterministic methods:
Edit - I might also start a wiki page on the "holy grail" method.
On this, I was thinking that for the weighted candidate method, it could be called COWPEA, which could stand for something like Candidates Optimally Weighted in Proportional Election using Approval voting. And the other method could just be the COWPEA Lottery.
-
I've set up pages for COWPEA and COWPEA Lottery.
-
Further to what I've said already, I do think that COWPEA is the approval (and score with KP) PR "Holy Grail". Obviously it elects candidates in different weights (or can be used for a party list election to determine the proportion of seats that each party should win), so isn't useful for a an equal-weight candidate-based election, but by Holy Grail, we mean the objectively most proportional method rather than the most practical one, but obviously practical ones could be based on it.
From the wiki page:
"The weight each candidate gets in parliament is the same as the probability that they would be elected in the following lottery:
Pick a ballot at random and list the candidates approved on this ballot. Pick another ballot at random, and strike off from the list all candidates not also approved on this ballot. Continue until one candidate is left and elect this candidate. If the number of candidates goes from >1 to 0 in one go, ignore that ballot and continue. If any tie cannot be broken, then elect the tied candidates with equal probability."
The philosophy behind it is that by picking ballots at random, in the long run every voter's ballot will be the "top ballot" an equal number of times, which means that every voter is guaranteed at least 1/v of the representation for v voters. Then if a voter has approved more than one candidate, by using the same random ballot process, it guarantees that among the candidates they approved there will be proportionality among the other voters too, and so on. So it uses the same philosophy all the way down so has a nice symmetry to it. It guarantees IIB because ballots that have approved all or no candidates are essentially ignored.
The process is not really random of course. Another way of looking at it is that we put the voters' ballots in every possible one of the v! orderings.
Parker Friedland (who I'm not sure has signed up on this forum) seems to agree as he said so in this thread - or here for the archived version without the pictures.
"And finally, this is what I speculate (and apparently what Toby Pereira also speculates) what the ideal proportions of the legislature C M and Y should control are if the legislature had a number of seats that approached infinity."
"Because we both came up with this independently makes me think that it is truly the ideal (at-least with approval ballots) even more so."
The annoying problem it has is that it's a process rather than a measure. You don't find a potential slate of candidates and measure how good it is. You just do the process and say "These are the optimum candidate ratios." And that means you can't just use this to find the best set of, say 5, candidates if you want a normal equal-weight parliament. But obviously the non-deterministic COWPEA Lottery method exists.
Obviously if there is a way of doing this deterministically while keeping the criteria (e.g. monotonicity, IIB, ULC) then that would be great.
-
I discussed in this thread how COWPEA can fail a multiwinner version of Pareto efficiency, which you could define as follows:
If every voter has approved at least as many candidates in set X as set Y, and at least one voter has approved more candidates in set X than set Y, then set Y should not be the winning set.
But I then countered in this thread that it might not actually be such a desirable property.
However, it does leave me wondering how optimal candidate Thiele voting would behave. Thiele's biggest failing is that it fails ULC, but with different candidate weighting allowed, a universally liked candidate would get all the power, so the problem might go away. But there might be residual problems when a candidate isn't universally liked, but is by certain factions, or if factions mix in a certain way. So I'm not sure if the problem would remain. Thiele would automatically pass the Pareto criterion above, though as said, it's debatable how desirable it is.
I think in general it would lead to more majoritarian but less purely proportional results than COWPEA.
