Load balancing for distributed interferometric image reconstruction

Abstract

We present a new algorithm to perform wide-field radio interferometric image reconstruction, with exact non-coplanar correction, that scales to big-data. This algorithm allows us to image 2 billion visibilities on 50 nodes of a computing cluster for a 25 by 25 degree field of view, in a little over an hour. We build on the recently developed distributed $w$-stacking $w$-projection hybrid algorithm, extending it to include a new distributed degridding algorithm that balances the computational load of the $w$-projection gridding kernels. The implementation of our algorithm is made publicly available in the PURIFY software package. Wide-field image reconstruction for data sets of this size cannot be performed effectively using the allocated computational resources without computational load balancing, demonstrating that our algorithms are critical for next-generation wide-field radio interferometers.