Can 2 LLM calls boost your RAG’s performance?

Building a real-world Retrieval Augmented Generation (RAG) system for handling company reports presents unique challenges and solutions. Initially struggling with generating accurate responses from unstructured data, the author experimented with different models and retrieval methods. Ultimately, using a smaller in-house LLM, Mistral 7B, for both generating metadata and crafting responses, outperformed even a powerful LLM like GPT-4. The key takeaway is the effective use of metadata filters and strategic application of smaller LLMs for enhanced performance.