Cover
Vol. 1 No. 1 (2025): ETES

Published: December 31, 2025

Pages: 37-44

Research Article

A Comparative Analysis of Federated Learning on Non-IID Data for an Intrusion Detection System

Rusul Tareq Khudhair* 1 Goh Chin Hock 1 Asmidar Bte Abu-Bakar 1
1College of Computing & Informatics (CCI) Universiti Tenaga Nasional, Kajang, Malaysia
History
  • Received: November 8, 2025
  • Revised: December 27, 2025
  • Accepted: December 28, 2025
  • Available Online: December 31, 2025
DOI

https://doi.org/10.65505/ETES-25-0011

Abstract

Federated learning (FL) offers a robust and privacy-preserving approach for developing collaborative intrusion detection systems (IDS). However, statistical variance severely hinders its practical application. Although privacy-preserving federated learning models have been used to develop intrusion detection systems for cyberattacks, problems arise when statistical variance is present. In practice, the performance of the FedAvg algorithm is significantly affected by the heterogeneous distribution of customer data in a real-world network. This distribution causes skewness among customer data, resulting in poor detection accuracy, delayed convergence, and model instability. In this paper, presents conduct a comprehensive comparison of the Scaffold algorithm with the FedAvg baseline using the CICIDS2017 datasets. Because the Scaffold algorithm addresses the client skew problem using control variables, it is considered a state-of-the-art federated optimization technique under the heterogeneous partitioning approach. This paper documents the importance of using the Scaffold algorithm as a reliable and essential tool for building high-performance detection systems in a variety of scientific settings. Therefore, our results demonstrate that Scaffold achieved more stable convergence and outperformed FedAvg, with a 15.1% increase in F1-score and a 13.6% higher overall accuracy under highly skewed data distributions. The present evaluation process operates through simulation testing, but physical testbed implementation remains essential for future work to evaluate real-world deployment challenges.

Keywords

References

  1. Xu, Binbin, et al. “Cross-Component Transferable Transformer Pipeline Obeying Dynamic Seesaw for Rotating Machinery with Imbalanced Data.” Sensors, vol. 23, no. 17, 25 Aug. 2023, pp. 7431–7431, https://doi.org/10.3390/s23177431.
  2. Busra Buyuktanir, et al. “Federated Learning in Intrusion Detection: Advancements, Applications, and Future Directions.” Cluster Computing, vol. 28, no. 7, 4 Aug. 2025, https://doi.org/10.1007/s10586-025-05325-w.
  3. R. Lazzarini, H. Tianfield, and V. Charissis, “Federated Learning for IoT Intrusion Detection,” AI, vol. 4, no. 3, pp. 509–530, Sep. 2023,
  4. https://doi.org/10.3390/ai4030028
  5. Li, Tian, et al. “Federated Learning: Challenges, Methods, and Future Directions.” IEEE Signal Processing Magazine, vol. 37, no. 3, May 2020, pp. 50–60.https://doi.org/10.1109/msp.2020.2975749.
  6. Zeng, Yan, et al. “Adaptive Federated Learning with Non-IID Data.” The Computer Journal, vol. 10.1093/comjnl/bxac118, no. 11, November 2023, Pages 2758–2772, 30 Sept. 2022, https://doi.org/10.1093/comjnl/bxac118.
  7. Peng, Haonan, et al. “FD-IDS: Federated Learning with Knowledge Distillation for Intrusion Detection in Non-IID IoT Environments.” Sensors, vol. 25, no. 14, 10 July 2025, p. 4309, https://doi.org/10.3390/s25144309.
  8. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” proceedings.mlr.press, Apr. 10, 2017. https://proceedings.mlr.press/v54/mcmahan17a?ref=https://githubhelp.com
  9. Karimireddy, Sai Praneeth, et al. “SCAFFOLD: Stochastic Controlled Averaging for Federated Learning.” ArXiv:1910.06378 [Cs, Math, Stat], vol. 1910.06378, no. Sai Praneeth Karimireddy, 9 Apr. 2021, arxiv.org/abs/1910.06378.
  10. E. Gad, Zubair Md Fadlullah, and M. M. Fouda, “A Robust Federated Learning Approach for Combating Attacks Against IoT Systems Under Non-IID Challenges,” arXiv (Cornell University), pp. 1–6, May 2024, https://doi.org/10.1109/smartnets61466.2024.10577749.
  11. Xu, Haoran, et al. “Federated Learning with Sample-Level Client Drift Mitigation.” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 20, 11 Apr. 2025, pp. 21752–21760, ojs.aaai.org/index.php/AAAI/article/view/35480, https://doi.org/10.1609/aaai.v39i20.35480
  12. Jose, Jinsi, and Deepa V. Jose. “Deep Learning Algorithms for Intrusion Detection Systems in Internet of Things Using CIC-IDS 2017 Dataset.” International Journal of Electrical and Computer Engineering (IJECE), vol. 13, no. 1, 1 Feb. 2023, p. 1134, https://doi.org/10.11591/ijece.v13i1.pp1134-1141.
  13. Kurniabudi, et al. “CICIDS-2017 Dataset Feature Analysis with Information Gain for Anomaly Detection.” IEEE Access, vol. 8, no. 10.1016/j.ins.2021.06.039, 2020, pp. 132911–132921, ieeexplore.ieee.org/document/9142219, https://doi.org/10.1109/ACCESS.2020.3009843.
  14. Antonio Pietrabissa, and Danilo Menegatti. Decentralised Learning for Intelligent Control Systems. 2 Jan. 2022.
  15. S. Khan et al., “Bilevel Hyperparameter Optimization and Neural Architecture Search for Enhanced Breast Cancer Detection in Smart Hospitals Interconnected with Decentralized Federated Learning Environment,” IEEE Access, vol. 12, pp. 63618–63628, 2024, doi: https://doi.org/10.1109/access.2024.3392572.
  16. Kazi Fatema, et al. “Federated XAI IDS: An Explainable and Safeguarding Privacy Approach to Detect Intrusion Combining Federated Learning and SHAP.” Future Internet, vol. 17, no. 6, 26 May 2025, pp. 234–234, www.mdpi.com/1999-5903/17/6/234, https://doi.org/10.3390/fi17060234.
  17. T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification,” arXiv:1909.06335 [cs, stat], Sep. 2019, Available: https://arxiv.org/abs/1909.06335.
  18. Swets, J. “Measuring the Accuracy of Diagnostic Systems.” Science, vol. 240, no. 4857, 3 June 1988, pp. 1285–1293, https://doi.org/10.1126/science.3287615.
  19. Ashish Mohare. “Accuracy.” Scribd, 2025, www.scribd.com/document/428946227/Accuracy. Accessed 26 Dec. 2025.
  20. Fedorchenko, Elena, et al. “Comparative Review of the Intrusion Detection Systems Based on Federated Learning: Advantages and Open Challenges.” Algorithms, vol. 15, no. 7, 15 July 2022, p. 247, https://doi.org/10.3390/a15070247.
  21. Humphrey, A, et al. “Machine-Learning Classification of Astronomical Sources: Estimating F1-Score in the Absence of Ground Truth.” Royal Astronomical Society, vol. 517, no. 1, 16 Oct. 2022, pp. L116–L120, https://doi.org/10.1093/mnrasl/slac120.