On Assessing Heterogeneity Management Solutions in Federated Learning Systems

Federated machine learning, often referred to as Federated Learning (FL), is a convenient way to distribute machine learning tasks at different nodes. FL helps keep privacy and may save bandwidth, but comes with some peculiarities. One key problem is the management of heterogeneity to obtain robust and convergent models. Among the many dimensions, this paper tackles heterogeneity by considering node selection and workload optimization and assesses some alternative solutions to better understand the pros and cons. We implemented them in Flower, a flexible user-friendly FL framework, and evaluated their performance compared to baseline techniques on an MLP model and using the MNIST dataset with different degrees of heterogeneity in data distribution. The results obtained provide interesting insights on their effectiveness, convergence speed, and stability. The goal is also to encourage the community to extend the experiments and play with different strategies, features, and characteristics.

  @inproceedings{baresi2024assessing,
    title={On Assessing Heterogeneity Management Solutions in Federated Learning Systems},
    author={Baresi, Luciano and Dolci, Tommaso and Wehbe, Iyad},
    booktitle={IEEE/ACM 17th International Conference on Utility and Cloud Computing},
    pages={517--522},
    year={2024},
    organization={IEEE/ACM},
    doi={10.1109/UCC63386.2024.00080}
  }