Dp-Signsgd: When Efficiency Meets Privacy And Robustness

Federated learning (FL) has emerged as a promising collab- oration paradigm by enabling a multitude of parties to con- struct a joint model without exposing their private training data. Three main challenges in FL are efficiency, privacy, and robustness. The recently proposed SIGNSGD with majority vote shows a promising direction to deal with efficiency and Byzantine robustness. However, there is no guarantee that SIGNSGD is privacy-preserving. In this paper, we bridge this gap by presenting an improved method called DP-SIGNSGD, which can enjoy all the aforementioned properties. We also present an error-feedback variant of the proposed DP-SIGNSGD which further improves the learning performance in FL. We experimentally demonstrate the effectiveness of our proposed methods with extensive experiments on the image datasets.