Enhancing Intrusion Detection on Imbalanced Network Traffic Using SMOTETomek with Machine Learning and Deep Learning Classifiers
Keywords:
Machine Learning, Intrusion Detection System, Imbalanced and Overlapping Dataset, SMOTETomek LinkAbstract
The growing sophistication of cyberattacks required intrusion detection systems (IDS) capable of accurately identifying malicious network traffic, even under highly imbalanced and overlapping data conditions. Traditional machine learning models often struggled in such scenarios due to biased learning toward majority classes. This study presented a comparative evaluation of traditional machine learning and deep learning classifiers, including Decision Tree, Random Forest, K-Nearest Neighbors, XGBoost, Convolutional Neural Network, and Bidirectional Long Short-Term Memory (Bi-LSTM), for binary intrusion detection. The NF-UNSW-NB15-V2 dataset was used for training, and NF-UNSW-NB15-V3 was used for testing to ensure unbiased cross-version validation mitigate data leakage associated with random partitioning. To address class imbalance and overlapping decision boundaries, the SMOTETomek resampling technique was applied and compared against the baseline setting. Performance was assessed using standard classification metrics. Experimental results revealed that SMOTETomek-enhanced Bi-LSTM and XGBoost models achieve superior Recall and F1-score. With SMOTETomek, XGBoost achieves a Recall of 92.18% and an F1-score of 90.87%, improving by 12.87% and 2.42%, respectively. Similarly, Bi-LSTM reached a Recall of 96.87% and an F1-score of 96.65%, reflecting gains of 2.65% and 1.15%. Additionally, SMOTETomek notably improved most models, highlighting its importance in cybersecurity classification tasks. This study provided guidance on model selection for IDS under imbalanced data and demonstrated that combining deep learning with data-level balancing improved intrusion detection robustness in real-world networks.
References
A. L. Buczak and E. Guven, “A survey of data mining and machine learning methods for cyber security intrusion detection,” IEEE Commun. Surv. Tut., vol. 18, no. 2, pp. 1153–1176, 2016, doi: 10.1109/COMST.2015.2494502.
H. Liu and B. Lang, “Machine learning and deep learning methods for intrusion detection systems: A survey,” Appl. Sci., vol. 9, no. 20, Oct. 2019, Art. no. 4396, doi: 10.3390/app9204396.
H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263–1284, Sep. 2009, doi: 10.1109/TKDE.2008.239.
B. Krawczyk, “Learning from imbalanced data: Open challenges and future directions,” Prog. Artif. Intell., vol. 5, no. 4, pp. 221–232, Apr. 2016, doi: 10.1007/s13748-016-0094-0.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” J. Artif. Intell. Res., vol. 16, no. 16, pp. 321–357, Jun. 2002, doi: 10.1613/jair.953.
A. Fernández, S. García, M. Galar, R. C. Prati, B. Krawczyk, and F. Herrera, Learning from Imbalanced Data Sets. Cham, Switzerland: Springer International Publishing, 2018, doi: 10. 1007/978-3-319-98074-4.
G. E. A. P. A. Batista, R. C. Prati, and M. C. Monard, “A study of the behavior of several methods for balancing machine learning training data,” ACM SIGKDD Explor. Newslett., vol. 6, no. 1, pp. 20–29, Jun. 2004, doi: 10.1145/1007730.1007735.
T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining – KDD, 16, vol. 1, no. 1, pp. 785–794, Aug. 2016, doi: 10.1145/2939672.2939785.
L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, Oct. 2001, doi: 10.1023/A:1010933404324.
Y. Kim, “Convolutional neural networks for sentence classification,” 2014, arXiv: 1408.5882, doi: 10.48550/arxiv.1408.5882.
S. Wang, X. Wang, S. Wang, and D. Wang, “Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting,” Int. J. Elect. Power Energy Syst., vol. 109, pp. 470–479, Jul. 2019, doi: 10.1016/j.ijepes.2019.02.022.
A. Alshammari and A. Aldribi, “Apply machine learning techniques to detect malicious network traffic in cloud computing,” J. Big Data, vol. 8, Jun. 2021, Art. no. 90, doi: 10.1186/s40537-021-00475-1.
M. Kumar, M. Hanumanthappa, and T. V. S. Kumar, “Intrusion detection system using decision tree algorithm,” in IEEE 14th International Conference on Communication Technology, Chengdu, China, 2012, pp. 629–634, doi: 10.1109/ICCT.2012.6511281.
M. Dixit, R. Sharma, S. Shaikh, and K. Muley, “Internet traffic detection using Naïve Bayes and K-Nearest Neighbors (KNN) algorithm,” in International Conference on Intelligent Computing and Control Systems (ICCS), Madurai, India, 2019, pp. 1153–1157, doi: 10.1109/ICCS45141.2019.9065655.
N. Farnaaz and M. A. Jabbar, “Random forest modeling for network intrusion detection system,” Procedia Comput. Sci., vol. 89, pp. 213–217, Jan. 2016, doi: 10.1016/j.procs.2016.06.047.
I. L. Cherif and A. Kortebi, “On using eXtreme Gradient Boosting (XGBoost) machine learning algorithm for home network traffic classification,” in Wireless Days (WD), Manchester, UK, 2019, pp. 1–6, doi: 10.1109/WD.2019.8734193.
A. Deshmukh and K. Ravulakollu, “An efficient CNN-based intrusion detection system for IoT: Use case towards cybersecurity,” Technologies, vol. 12, no. 10, Oct. 2024, Art. no. 203, doi: 10.3390/technologies12100203.
I. Eweoya et al., “A CNN-based network profiling system for intrusion detection: Addressing evolving cyber threats,” Asian J. Comput. Sci. Technol., vol. 14, no. 1, pp. 20–27, Mar. 2025, doi: 10.70112/ajcst-2025.14.1.4332.
R. M. Al-Khatib, L. Heilat, W. Qudah, S. Alhatamleh, and A. Al-Khateeb, “A novel improved deep learning model based on Bi-LSTM algorithm for intrusion detection in WSN,” Networks and Heterogeneous Media, vol. 20, no. 2, pp. 532–565, 2025, doi: 10.3934/nhm. 2025024.
N. Moustafa and J. Slay, “UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set),” in 2015 Military Communications and Information Systems Conference (MilCIS), Canberra, ACT, Australia, 2015, pp. 1–6, doi: 10.1109/MilCIS.2015. 7348942.
N. Moustafa and J. Slay, “The evaluation of network anomaly detection systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 dataset,” Inf. Secur. J. Glob. Perspect., vol. 25, no. 1–3, pp. 18–31, Jan. 2016, doi: 10.1080/19393555.2015.1125974.
T. Wu, H. Fan, H. Zhu, C. You, H. Zhou, and X. Huang, “Intrusion detection system combined enhanced random forest with SMOTE algorithm,” EURASIP J. Adv. Signal Process., vol. 2022, May 2022, Art. No. 39, doi: 10.1186/s13634-022-00871-6.
R. Ahsan, W. Shi, and J. Corriveau, “Network intrusion detection using machine learning approaches: Addressing data imbalance,” IET Cyber-Phys. Syst. Theor. Appl., vol. 7, no. 1, Jun. 2021, doi: 10.1049/cps2.12013.
M. A. Talukder, S. Sharmin, M. A. Uddin, M. M. Islam, and S. Aryal, “MLSTL-WSN: Machine learning-based intrusion detection using SMOTETomek in WSNs,” Int. J. Inf. Secur., vol. 23, pp. 2139–2158, Mar. 2024, doi: 10.1007/s10207-024-00833-z.
L. V and S. Rajkumar, “Hybrid ensemble federated learning using SMOTE-Tomek for efficient DDoS detection on constrained edge devices over 5G networks,” Results Eng., vol. 28, Dec. 2025, Art. no. 107601, doi: 10.1016/j.rineng.2025.107601.
K. Omari and A. Oukhatar, “Advanced phishing website detection with SMOTETomek-XGB: Addressing class imbalance for optimal results,” Procedia Comput. Sci., vol. 252, pp. 289–295, 2025, doi: 10.1016/j.procs.2024.12.031.
D. A. Otchere, T. O. A. Ganat, J. O. Ojero, B. N. Tackie-Otoo, and M. Y. Taki, “Application of gradient boosting regression model for the evaluation of feature selection techniques in improving reservoir characterisation predictions,” J. Pet. Sci. Eng., vol. 208, Jan. 2022, Art. no. 109244, doi: 10.1016/j.petrol.2021.109244.
L. Mohammadpour, T. C. Ling, C. S. Liew, and A. Aryanfar, “A survey of CNN-based network intrusion detection,” Appl. Sci., vol. 12, no. 16, Aug. 2022, Art. no. 8162, doi: 10.3390/app12168162.
S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recognition: A convolutional neural-network approach,” IEEE Trans. Neural Netw., vol. 8, no. 1, pp. 98–113, Jan. 1997, doi: 10.1109/72.554195.
L. K. Suresh Kumar, “An efficient network intrusion detection model combining CNN and BiLSTM,” CIBG, vol. 27, no. 6, pp. 1782–1801, Dec. 2021. [Online]. Available: https://cibgp.com/index.php/1323-6903/article/view/2273/2233
J. Gao, “Network intrusion detection method combining CNN and BiLSTM in cloud computing environment,” Comput. Intell. Neurosci., vol. 2022, Apr. 2022, Art. no. 7272479, doi: 10.1155/2022/7272479.
J. J. A. Armenteros, C. K. Sønderby, S. K. Sønderby, H. Nielsen, and O. Winther, “DeepLoc: Prediction of protein subcellular localization using deep learning,” Bioinformatics, vol. 33, no. 21, pp. 3387–3395, Nov. 2017, doi: 10.1093/bioinformatics/btx431.
J. Zhang, X. Zhang, Z. Liu, F. Fu, Y. Jiao, and F. Xu, “A network intrusion detection model based on BiLSTM with multi-head attention mechanism,” Electronics, vol. 12, no. 19, Oct. 2023, Art. no. 4170, doi: 10.3390/electronics12194170.
E. F. Swana, W. Doorsamy, and P. Bokoro, “Tomek link and SMOTE approaches for machine fault classification with an imbalanced dataset,” Sensors, vol. 22, no. 9, Jan. 2022, Art. no. 3246, doi: 10.3390/s22093246.
V. Jyothsna and K. M. Prasad, “Anomaly-based intrusion detection system,” in Computer and Network Security, J. Sen, Ed., IntechOpen, Jun. 2019, doi: 10.5772/intechopen.82287..
G. Velarde, A. Sudhir, S. Deshmane, A. Deshmunkh, K. Sharma, and V. Joshi, “Evaluating XGBoost for balanced and imbalanced data: Application to Fraud detection,” 2023, arXiv: 2303.15218, doi: 10.48550/arxiv.2303.15218.
Downloads
Published
How to Cite
License
Copyright (c) 2026 NKRAFA JOURNAL OF SCIENCE AND TECHNOLOGY
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
- Content and information in articles published in NKRAFA Journal of Science and Technology are comment and responsibility of authors of articles directly. Journal editorial do no need to agree or share any responsibility.
- NKRAFA Journal of Science and Technology Articles holds the copyright of the content, pictures, images etc. which published in it. If any person or agency require to reuse all or some part of articles, the permission must be obtained from the NKRAFA Journal of Science and Technology.
