A Semantic Vector-Based Retrieval Framework for University Project Repositories Using Transformer Embeddings and Approximate Nearest Neighbor Search
Article Sidebar
Main Article Content
Abstract
This research aimed to: 1) develop a vector-based semantic search framework for a university project repository; 2) study the results of implementing a semantic search system integrating Sentence Transformer models with Approximate Nearest Neighbor (ANN) algorithms; and 3) examine the results of evaluating the system’s performance and semantic matching quality in the context of university project data. The research process consisted of collecting data from 937 projects, designing the data structure, developing the model, and testing the system performance. The research instruments included a database system, data processing software, and a semantic search quality evaluation framework used to analyze the system performance.
The research findings revealed that: 1) the developed semantic search framework could effectively retrieve contextually relevant projects, with response times suitable for interactive search systems, and could clearly distinguish highly relevant projects from less relevant ones; 2) the results of implementing the semantic search system showed that the use of Sentence Transformer models together with an ANN index (Annoy) improved the accuracy of retrieval results compared with keyword-based search, particularly for Thai-language data with lexical diversity and incomplete metadata; and 3) the system performance evaluation showed that the Response Level metric was at a good level, reflecting the system’s ability to respond to user intent and demonstrating the potential of applying semantic search to support knowledge management and data-driven decision-making in higher education institutions.
Article Details
References
Uttaradit Rajabhat University, “Engagement Information System: Semantic Search System,” 2025. [Online]. Available: https://info.uru.ac.th/engagement/semantic-search.
P. Netisopakul, G. Wohlgenannt, A. Pulich, and Z. Z. Hlaing, “Improving the state-of-the-art in Thai semantic similarity using distributional semantics and ontological information,” PLOS ONE, vol. 16,
C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge, UK: Cambridge University Press, 2008.
P. D. Turney and P. Pantel, “From frequency to meaning: Vector space models of semantics,” Journal of Artificial Intelligence Research, vol. 37, pp. 141–188, 2010, doi: 10.1613/jair.2834.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Minneapolis, MN, USA, 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 2019, pp. 3982–3992, doi: 10.18653/v1/D19-1410.
J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with GPUs,” IEEE Transactions on Big Data, vol. 7, no. 3, pp. 535–547, 2019, doi: 10.1109/TBDATA.2019.2921572.
E. Bernhardsson, “Annoy: Approximate nearest neighbors in C++/Python,” 2018. [Online]. Available: https://github.com/spotify/annoy.
T. Junlabuddee and K. Tuamsuk, “Semantic search system for research data in information science,” Journal of Information Science (Thailand), vol. 39, no. 3, pp. 43–61, 2021. (in Thai)
N. Kaewboonma, J. Panawong, E. Pianhanuruk, and M. Buranarach, “Development of intelligent semantic search system for rubber research data in Thailand,” in AIP Conference Proceedings, vol. 1891, no. 1, p. 020073, 2017, doi: 10.1063/1.5005333.
K. Jearanaitanakij et al., “Thai question-answering system using similarity search and LLM,” ECTI Transactions on Computer and Information Technology (ECTI-CIT), vol. 18, no. 3, pp. 406–416, 2024, doi: 10.37936/ecti-cit.2024183.255313.
J. Guo, Y. Fan, Q. Ai, and W. B. Croft, “A deep relevance matching model for ad-hoc retrieval,” in Proceedings of the 25th ACM International Conference on Information and Knowledge Management, Indianapolis, IN, USA, 2016, pp. 55–64, doi: 10.1145/2983323.2983769.
V. Karpukhin et al., “Dense passage retrieval for open-domain question answering,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 6769–6781, doi: 10.18653/v1/2020.emnlp-main.550.
S. Zhang, L. Yao, A. Sun, and Y. Tay, “Deep learning based recommender system: A survey and new perspectives,” ACM Computing Surveys, vol. 52, no. 1, pp. 1–38, 2019, doi: 10.1145/3285029.
P. Indyk and R. Motwani, “Approximate nearest neighbors: Toward removing the curse of dimensionality,” in Proceedings of the 30th Annual ACM Symposium on Theory of Computing, Dallas, TX, USA, 1998, pp. 604–613, doi: 10.1145/276694.276876.
R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval: The Concepts and Technology Behind Search, 2nd ed. Boston, MA, USA: Addison-Wesley, 2011.
T. Wolf et al., “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 38–45, doi: 10.18653/v1/2020.emnlp-demos.6.
Y. A. Malkov and D. A. Yashunin, “Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 4, pp. 824–836, 2020, doi: 10.1109/TPAMI.2018.2889473.
