Deep Q learning algorithm for detecting DDoS attacks on IoT devices

Abstract

The rapid expansion of internet of things (IoT) networks has heightened security risks, particularly regarding distributed denial of service (DDoS) attacks against devices with limited computing capacity. High detection accuracy is crucial for these resource-constrained environments, where false positives can disrupt legitimate traffic and false negatives allow attacks to persist. However, modern reinforcement learning (RL) and machine learning (ML) intrusion detection solutions often exhibit poor generalization due to static state representations. To address this, this paper proposes a deep Q-learning (DQL) framework that integrates K-means clustering directly into the RL action space. Unlike prior RL-based IDS models, our approach dynamically integrates clustering into the learning process, enabling adaptive state representation and improved generalization to unseen traffic patterns. The system is formulated as a Markov decision process where the agent optimizes a composite reward function based on accuracy, precision, recall, and F1-score. Evaluated on the N-BaIoT dataset using 10-fold cross-validation, the proposed method achieves a classification accuracy of 98.95% and a weighted F1-score of 98.73%, significantly outperforming traditional ML and RL baselines. These results demonstrate the framework's effectiveness as a scalable, adaptive solution for intelligent IoT DDoS detection.