Collaborative Coding for Simulation

If anybody would feel inclined to make their tabulation API 'black boxed' so you can just hand me a .csv file and I will hand you a list of winners, I would be happy to implement a great number of voting methods, including some of the most complicated like Stable Voting or Meek-STV.

I do want to contribute, but I'm just much more comfortable iterating on local scripts in Julia or Python than I am with JS. Does JS have the ability to invoke command line scripts?

rob

@andy-dienes JS can be run on the command line via node.js, which is quick to download and install and generally "just works". Of course it also allows you to run web servers and the like, but you can also just say "node myprogram.js" to run things from the command line on mac, windows and any flavor of linux.

I prefer things like Codepen, though, because it is so easily sharable. You can just point people to the Codepen url, and they can paste whatever data in they want. They can also fork it if they want to edit the code. I did a bunch of work to parse ballot formats that are easy to use in forums. (comma separated aren't terrible, but that seems a bit more optimized for the machine rather than for human eyes, and I prefer optimizing for humans)

See https://codepen.io/karmatics/pen/ExKZVjM for example. You can paste in data or use the default. (just click "tabulate" in the bottom right)

But yeah if you prefer work at the command line, but do it in javascript, I'd be glad to convert them to Codepens (or set up a simple template so you can do it yourself), so you can just refer people to a url rather than having them send you csv files, which doesn't lend itself to experimentation and such.

@rob Oops, let me clarify. Does JS have the ability to easily call other scripts from command line? Something like Python's subprocess.call()

rob

@andy-dienes I went back and thought about what you asked, I think the answer is "yes":

https://nodejs.org/api/child_process.html

However I first thought you were asking how to include one JS file in another, or spawn a thread, or whatever... I'll just leave what I wrote about all that.....

Here I'm using modules.exports and require in the normal way to include one js file in another (and call a function in the included module from the original file):

main.js

console.log("hello there");

var otherModule = require("./included.js");
otherModule.sayHi();

included.js

module.exports = {
    sayHi: () => {
        console.log("hi!");
    }
};

then run it:

> node main
hello there
hi!

I'm not sure what you're trying to do.... spawn a thread? Javascript is single-threaded typically but uses callbacks (and the more modern async and promise stuff) when you want to start something that runs separately. I mostly do that for animations or various UI stuff, but that tends to be overkill for just, say, running a calculation that runs in a few milliseconds.

There is also WebWorkers which are essentially background threads. (yes they work in non-browser environments like node.js) I've never had the need for them.

BTW, if you are interested in checking out Codepen, I made a super simple starter app that allows you to do really basic stuff that might be relevant to us (such as process text input and produce text output), with a easy to use UI. Anyone should be able to follow it.

https://codepen.io/karmatics/pen/oNqmGoa

@rob said in Collaborative Coding for Simulation:

I'm not sure what you're trying to do.... spawn a thread?

I'm wondering if there is some way to run

js.run_shell_command("julia my_awesome_and_hard_to_implement_voting_method.jl --election=ballot_data.csv --seats=3 --output=winners.csv")

Some of these algorithms are quite pleasant to implement given tools & libraries present in Julia / Python, but I wouldn't even know where to look for the same in JS. Maybe just for mental context, pretend I'm talking about implementing Schulze-STV which involves solving a linear optimization program for every single pair of possible sets of winners.

rob

@andy-dienes
Probably exec() is what you want, although spawn() is a bit more powerful I think for long running programs.

Here's exec. Most of the bulk below is just showing how to handle all the stderr etc.

const { exec } = require('child_process');

const newProcess = exec('julia my_awesome_and_hard_to_implement_voting_method.jl --election=ballot_data.csv --seats=3 --output=winners.csv', function (error, stdout, stderr) {
  if (error) {
    console.log(error.stack);
    console.log('error code: ' + error.code);
    console.log('signal: ' + error.signal);
  }
  console.log('stdout: ' + stdout);
  console.log('stderr: ' + stderr);
});

newProcess.on('exit', function (code) {
  console.log('exited with exit code ' + code);
});

I don't know what to suggest as far as Schulze-STV, that method makes my brain hurt. But I'd hope you can implement it with just some for loops or basic recursion.... right?

@rob said in Collaborative Coding for Simulation:

But I'd hope you can implement it with just some for loops or basic recursion.... right?

Alas, no. It requires solving a linear program (aka an optimization program that looks like this) for every single pair of possible committees, of which there are exponentially many... I have implemented the algorithm using JuMP for which there are a number of convex solvers (e.g. COSMO, Clarabel, or the more canonical CLP), and in Python there is stuff like Pyomo, cvxopt, or even just scipy, but the corresponding JS libraries seem less pleasant to use.

There are other methods too for which some serious complexity is needed like the above. More examples are the strategy-proof randomized Condorcet methods (1) and (2). Also, there are methods which technically can be implemented with simple algorithms, but really benefit from just having the language be natively fast, e.g. Stable Voting or PAV; similarly if some kind of Monte Carlo simulations are desired, one can run a whole lot more of these in a whole lot less time using Julia / Numpy than I imagine one could ever do in JS.

I am not criticizing the choice to use JS and there seems to be a good context to use it here---I'm just trying to give reasons why it might be useful to 'black box' the actual winner selection into separately-executable scripts.

rob

@andy-dienes Gotcha. I tend to be a fan of simple methods.

All the methods I've done are very few lines of code and could process millions of ballots in milliseconds.

I always found it odd that, for instance, the equal vote people had never bothered even providing a reference implementation of STAR in JS. (which meant it was a huge ordeal when we tried to do a STAR election in this community with only a dozen candidates and about the same number of voters a couple years ago)

Jack Waugh

@rob said in Collaborative Coding for Simulation:

exec

Of course, this technique would preclude running the election in the front end. It would have to send a request to the server. But that's not a blocking issue. It may still be practical, even if the researcher is rapidly tweaking the problem definition. The Internet is faster than the old phone lines were.

rob

@jack-waugh said in Collaborative Coding for Simulation:

But that's not a blocking issue.

I think there are huge advantages to making stuff that runs on the front end, if you want others in the community to be able to run it. Especially if you don't have web hosting. But mostly if you want others to be able to fork your code and play with it or build on top of it, such as to build it into a voting widget, to build a simulator, etc.

But I can see why on really hard methods like Schulze (which I'm convinced was invented as a parody of improved voting methods.... "let's see if we can make something so complicated none of us even understand it" ), it makes some sense to do it in whatever language makes it easier.