LLM Powered Search on your Laptop

Retrieval augmented Large Language models have helped in multiple NLP applications of answering factual questions, summarization etc. In this study, we introduce a compact yet powerful search engine, optimized for deployment on the laptop, that encompasses all ARXIV articles, based on fine tuned LLM for ranking and a vector database for retrieving. The main idea is to use a pre-trained language model to encode the titles and abstracts of the ARXIV articles into dense vectors, and store them in a vector database that supports fast similarity search. Then a Large Language model trained for multiple tasks of summarization and learning to rank ranks the articles. On a custom query set, we see the LLM powered Search engine running on a laptop beats the arxiv search engine in terms of NDCG. The application can be extended to private documents that are not indexed in web, hence will be useful for broader AI for Good and academicians.