Transformer-Based Email Classification for Workflow Automation in Small and Medium-Sized Enterprises

Takorn  Prexawanprasut

Authors

Takorn Prexawanprasut School of Science and Technology, Sukhothai Thammathirat Open University, Nonthaburi, Thailand

Keywords:

Email classification, natural language processing (NLP), transformer models, BERT, XLM-R, machine learning, workflow management system (WMS), small and medium-sized enterprises (SMEs)

Abstract

Email remains a primary medium of business communication, yet small and medium-sized enterprises (SMEs) often lack the capacity to adopt enterprise-level solutions, resulting in inefficiencies in handling large volumes of unstructured messages. This study evaluates advanced natural language processing (NLP) techniques for automating email classification and integrating structured outputs into workflow management systems (WMS). A dataset of 12,500 emails collected between January 2021 and December 2024 was categorized into four operational domains—sales, shipping, billing, and transportation—and used to compare three approaches: a keyword-based rule system, classical machine learning classifiers (naïve Bayes, logistic regression, support vector machines, random forest), and transformer-based architectures (BERT, DistilBERT, XLM-R). Performance was assessed using accuracy, precision, recall, and F1-score, with statistical tests applied to confirm significance. Results show that while the rule-based baseline achieved limited performance, and classical models offered moderate improvements, transformer-based methods achieved the highest overall accuracy, with XLM-R surpassing 92%. Importantly, integration of the best-performing model into a prototype WMS demonstrated practical value by enabling real-time classification and extraction of structured information such as invoice numbers, shipment codes, and client identifiers. These findings highlight the potential of transformer-based models to deliver scalable, cost-effective workflow automation for SMEs, reducing manual workload, enhancing efficiency, and improving responsiveness in dynamic business environments.

References

Alsmadi I, Alhami I. Clustering and classification of email contents. J King Saud Univ Comput Inf Sci. 2015; 27(1): 46-57.

Ayodele T, Khusainov R, Ndzi D. Email classification and summarization: A machine learning approach. In Proceedings of the IET Conference on Wireless, Mobile and Sensor Networks, Shanghai, China. 2007; 805-808.

Ayodele T, Zhou S, Khusainov R. Email grouping and summarization: An unsupervised learning technique. In Proceedings of the 2009 WRI World Congress on Computer Science and Information Engineering, Los Angeles, CA: IEEE. 2009; 5: 575-579.

Cavus M, Biecek P. Investigating the impact of balancing, filtering, and complexity on predictive multiplicity: A data-centric perspective. Inf Fusion. 2025; 123: 103243. https://doi.org/10.1016/j.inffus.2025.103243

Conneau A, Khandelwal K, Goyal N, Chaudhary V, Wenzek G, Guzmán F, Stoyanov V. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, Washington, USA. 2020; 8440-8451. https://doi.org/10.18653/v1/2020.acl-main.747

Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota. 2019; 4171-4186.

Howard J, Ruder S. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia. 2018; 328-339.

Huang Y. Situation awareness and information fusion in sales and customer engagement: A paradigm shift. In Proceedings of the 2020 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), Victoria, BC, Canada. 2020; 113-121.

Iqbal K, Khan MS. Email classification analysis using machine learning techniques. Appl Comput Inform. 2022; 21(11): 390-402.

Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations (ICLR) Workshop Track, Scottsdale, AZ. 2013. https://arxiv.org/abs/1301.3781

Prexawanprasut T, Chaipornkaew P. Email classification model for workflow management systems. Walailak J Sci Technol. 2017; 14(10): 783-790.

Sanh V, Debut L, Chaumond J, Wolf T. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. In Proceedings of the NeurIPS EMC² Workshop, Vancouver, Canada. 2019. Available from: https://arxiv.org/abs/1910.01108

Schuff D, Turetken O, D’Arcy JD, Croson D. Managing E-mail overload: Solutions and future challenges. Computer. 2007; 40(2): 31-36.

Stańdo A, Cavus M, Biecek P. The effect of balancing methods on model behavior in imbalanced classification problems. In Proceedings of Machine Learning Research, Vienna, Austria. 2024; 241: 16-30. Available from: https://proceedings.mlr.press/v241/stando24a/stando24a.pdf

Suryawanshi V, Shaikh M, Gondole M, Khunteta M, Wani M. Phishing email detection using natural language processing techniques. Int J Res Anal Rev. 2025; 12(2): 48-55.

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Polosukhin I. Attention is all you need. In Advances in Neural Information Processing Systems 30 (NeurIPS 2017), Long Beach, CA. 2017; 5998-6008. Available from: https://arxiv.org/abs/1706.03762

Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le QV. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019), Vancouver, Canada. 2019; 5753-5763. Available from: https://arxiv.org/abs/1906.08237

Zhang X, Zhao J, LeCun Y. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems 28 (NeurIPS 2015), Montreal, Canada. 2015; 649-657. Available from: https://arxiv.org/abs/1509.01626

Zhou C, Sun C, Liu Z, Lau F. A C-LSTM neural network for text classification. In Proceedings of the 26th International Conference on Computational Linguistics (COLING 2016), Osaka, Japan. 2016; 525-534.