« home

DataGen Workers Optimisation

I released DataGen v0.0.9 during lunch break yesterday. This version includes the support to limit how many workers can run concurrently, which is something that I’ve always wanted to add since day one. I finally got the time to do it last weekend, and it turned out to be an easy task thanks to Rod Vagg’s worker-farm module. Why is this necessary? The problem with previous versions of DataGen was that when you want to generate 20 data files, then 20 worker processes will be created and run concurrently.…

more »