Balancing Privacy and Performance in Federated Learning: a Systematic Literature Review on Methods and Metrics

Federated Learning (FL) has emerged as a novel paradigm in the area of Artificial Intelligence (AI), emphasizing decentralized data utilization and bringing learning to the edge or directly on-device. While this approach eliminates the need for data centralization, ensuring enhanced privacy and protection of sensitive information, it is not without challenges. Particularly during the training phase and the exchange of model update parameters between servers and clients, new privacy challenges have arisen. While several privacy-preserving FL solutions have been developed to mitigate potential breaches in FL architectures, their integration poses its own set of challenges. Incorporating these privacy-preserving mechanisms into FL at the edge computing level can increase both communication and computational overheads, which may, in turn, compromise data utility and learning performance metrics. This paper provides a systematic literature review on essential methods and metrics to support the most appropriate trade-offs between FL privacy and other performance-related application requirements such as accuracy, loss, convergence time, utility, communication, and computation overhead. We aim to provide an extensive overview of recent privacy-preserving mechanisms in FL used across various applications, placing a particular focus on quantitative privacy assessment approaches in FL and the necessity of achieving a balance between privacy and the other requirements of real-world FL applications. This review collects, classifies, and discusses relevant papers in a structured manner, emphasizing challenges, open issues, and promising research directions.