The role of artificial intelligence in advancing the performance of information retrieval

Abstract

The motivation behind applying artificial intelligence (AI) in information retrieval (IR) is that the current methodologies include algorithms designed by researchers, leaving space for the applicability of genetic AI algorithms in IR. While different algorithms designed by developers rely on the originality or performance of the algorithm, precise results are achieved through integrating AI algorithms with traditional algorithms. The proposed methodology introduces document structure weighting with optimized performance. It is enabled by employing genetic algorithm and genetic programming for learning optimal weights in ranking document components. The Croft probabilistic ranking, vector space inner product models, and the BM25 standard were compared with each other after AI integration. Genetic algorithm and genetic programming were applied in the stemming and thesaurus forming processes of these models. Inducing genetic algorithm and genetic programming into the specified models increased the mean average precision of the Croft model and the vector space method by approximately 5% while there were no observable result improvements in BM25. It was found that applying genetic algorithm and genetic programming in learning synonyms and stemming rules, respectively, increased the overall performance of IR models, emphasizing the need for AI in IR.