Abstract/Summary

Clustered Federated Learning (FL) is an extension of FL that organizes participating devices into clusters or groups,aiming to train heterogeneous models by grouping devices withdiverse data distributions, thus fostering local collaboration within clusters for improved model performance. The increased model diversity introduces challenges stemming from the inherent heterogeneity in functions being modeled (e.g. for targeted advertising in platforms like Facebook), leading to a more natural prevalence of non-independent, non-identically distributed (non-IID) data in clustered FL environments. Moreover, another notable security challenge is that malicious agents can not only change their model updates to the aggregator but also misreport their cluster associations/identities in the clustered FL process. This paper introduces an algorithm called Coordinate-wise Median Clustered (CMC) to address the two challenges of heterogeneous model generation across agents and being robust against malicious agents. This is an algorithm that combines clustering and robust techniques, taking ideas from co-ordinate wise median attack-robust to adapt them to clustered FL. We prove that Fed Avg per cluster does not converge under adversarial attacks while our CMC algorithm converges quickly to achieve high accuracy.