Real-Time Anomaly Detection of Cloud File Access Events Using Streaming Machine Learning Techniques
Keywords:
detection, anomaly, cloud, file access events, streaming machine learningAbstract
This study aimed to develop a real-time anomaly detection approach for cloud file access events using streaming machine learning techniques. The research was organized into five phases: baseline model construction, model stability analysis, feature importance ranking, reduced feature-set comparison, and the main experiment on a schema-adapted data stream transformed from HDFS logs to mimic cloud file access events. The evaluation utilized a streaming dataset comprising a total of 250,000 events (consisting of 225,000 normal events and 25,000 anomalous events). The models examined in this study were Half-Space Trees (HST), Logistic Regression (LR), and Adaptive Random Forest (ARF), and their performance was evaluated using the prequential approach, which is suitable for continuously arriving data. The results showed that ARF achieved the highest numerical performance when the core feature set was combined with 11 additional engineered features, with an Accuracy of 0.849965 and an F1-score of 0.839121. However, when the number of features and system complexity were also considered, the core feature set combined with 5 additional engineered features was more appropriate for practical use, while still maintaining an Accuracy of 0.803863 and an F1-score of 0.782305. These findings indicate that ARF is the most suitable primary model for anomaly detection in cloud file-system streams, and that the selected feature configuration can serve as a basis for developing a risk detection and alerting system in real-world cloud environments.
References
Attou H, Guezzaz A, Benkirane S, Azrour M, Farhaoui Y. Cloud-based intrusion detection approach using machine learning techniques. Big Data Min Anal 2023;6(3):311-20.
Al-Ghuwairi AR, Sharrab Y, Al-Fraihat D, AlElaimat M, Alsarhan A, Algarni A. Intrusion detection in cloud computing based on time series anomalies utilizing machine learning. J Cloud Comput 2023;12(1):127.
Nassif AB, Talib MA, Nasir Q, Albadani H, Dakalbab FM. Machine learning for cloud security: a systematic review. IEEE Access 2021;9:20717-35.
Auxsorn T. A performance measurement framework for IaaS logging system with threat mitigation. Ph.D. Thesis, Naresuan University. Phitsanulok; 2022.
Padhiar S, Patel R. Outside the Closed World: On Using Machine Learning for Network Intrusion. in: Proceedings of Information and Communication Technology for Intelligent Systems (ICTIS), September 8, 2023; Singapore; 2023. p. 265-70.
Lu T, Wang L, Zhao X. Review of anomaly detection algorithms for data streams. Appl Sci 2023;13(10):6353.
Cao Y, Ma Y, Zhu Y, Ting KM. Revisiting streaming anomaly detection: Benchmark and evaluation. Artif Intell Rev 2025;58(1):8.
Suarez-Cetrulo AL, Quintana D, Cervantes A. A survey on machine learning for recurring concept drifting data streams. Expert Syst Appl 2023;213:118934.
Quickwit. Index a logging dataset locally. Quickwit Documentation [Internet]. 2026 [cited 2026 May 13]. Available from: https://quickwit.io/docs/0.7.1/get-started/tutorials/tutorial-hdfs-logs
MITRE. ATT&CK Matrix for Enterprise. The MITRE Corporation [Internet]. 2026 [cited 2026 May 22]. Available from: https://attack.mitre.org/
Al-Sada B, Sadighian A, Oligeri G. MITRE ATT&CK: State of the art and way forward. ACM Comput Surv 2024;57(1):1-37.
Liu JJ, Cassales GW, Liu FT, Pfahringer B, Bifet A. Adaptive Isolation Forest. In: International Conference on Discovery Science, September 22, 2025; Slovenia; 2025. p. 363-78.
AlQabbany AO, Azmi AM. Measuring the effectiveness of adaptive random forest for handling concept drift in big data streams. Entropy 2021;23(7):859.
Cano A, Krawczyk B. ROSE: robust online self-adjusting ensemble for continual learning on imbalanced drifting data streams. Mach Learn 2022;111(7):2561-99.
Garcia-Mendez S, de Arriba-Perez F, Leal F, Veloso B, Malheiro B, Burguillo-Rial JC. An explainable machine learning framework for railway predictive maintenance using data streams from the metro operator of Portugal. Sci Rep 2025;15(1):27495.
Xu J, Lin C, Liu F, Wang Y, Xiong W, Li Z, et al. StreamAD: A cloud platform metrics-oriented benchmark for unsupervised online anomaly detection. BenchCouncil Trans Benchmarks Stand Eval 2023;3(2):100121.
Dhal P, Azad C. A fine-tuning deep learning with multi-objective-based feature selection approach for the classification of text. Neural Comput Appl 2024;36(7):3525–53.
Gama J, Zliobaite I, Bifet A, Pechenizkiy M, Bouchachia A. A survey on concept drift adaptation. ACM Comput Surv 2014;46(4):1-37.
Souiden I, Omri MN, Brahmi Z. A survey of outlier detection in high dimensional data streams. Comput Sci Rev 2022;44:100463.
Saiyed AM. Experimental Evaluation of Real-Time Data Streaming Analytics Using Reinforcement Learning. J Integr Des Res 2024;5(4):51-61.
Bifet A, Gavalda R, Holmes G, Pfahringer B. Machine learning for data streams: with practical examples in MOA. Massachusetts: The MIT Press; 2017.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Huachiew Chalermprakiet Science and Technology Journal
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
บทความทุกบทความที่ได้รับการตีพิมพ์ถือเป็นลิขสิทธิ์ของ คณะวิทยาศาสตร์แฟละเทคโนโลยี มหาวิทยาลัยหัวเฉียวเฉลิมพระเกียรติ
