Daily Dose of Data Science | Avi Chawla | Substack·1y[Hands-on] Tool calling in LLMs
Tool calling allows language models to perform specific tasks by invoking external tools or APIs. The process involves recognizing when an external tool is needed, invoking the tool, and integrating its output into the model's response. This enhances the flexibility and capability of LLMs. A demo is provided to build a stock price retrieval assistant using the yfinance library.
Towards Dev·1yMastering Chunking for Effective RAG: Beyond Basics with Qdrant and Reranking
Chunking is essential in Retrieval-Augmented Generation (RAG) workflows, breaking large documents into manageable pieces to optimize data ingestion. Different chunking strategies, such as semantic chunking and topic node parsing, enhance the effectiveness of RAG pipelines when combined with Qdrant’s hybrid vector search and reranking methods. An evaluation framework assesses the quality of RAG pipelines through metrics like faithfulness, answer relevancy, and answer correctness, providing insights into which combinations perform best.
NVIDIA Developer·1yAn Easy Introduction to Multimodal Retrieval-Augmented Generation for Video and Audio
Building a multimodal retrieval augmented generation (RAG) system for video and audio involves capturing and indexing information across different modalities like text, images, audio, and video. There are three primary approaches: using a common embedding space, building parallel retrieval pipelines, or grounding information in a common modality like text. When dealing with videos, it’s crucial to manage computational costs, extract meaningful information from frames, and preserve actions across frames. The process includes audio and video ingestion, blending information from both, setting up a retriever, and generating answers using a large language model.