Towards Data Science is a community-powered publication that showcases work in data science, machine learning and artificial intelligence. Every day newcomers, seasoned researchers and industry practitioners publish tutorials, research notes and real-world case studies that help the field move forward.

Towards Data Science

A deep dive into RAG chunking strategies based on real production failures. Starting from a compliance incident caused by a split paragraph, the author walks through fixed-size chunking, sentence windows, hierarchical chunking, and semantic chunking — explaining when each works and when it fails. Special attention is given to enterprise document types: PDFs with complex layouts, tables (converted to natural-language sentences for better retrieval), and slide decks with diagrams. A routing framework selects the right parser per document type. RAGAS metrics (context recall, faithfulness, context precision) are used throughout to quantify improvements, with context recall jumping from 0.72 to 0.88 after switching to sentence windows for narrative documents.

Your Chunks Failed Your RAG in Production

What Chunking Is and Why Most Engineers Underestimate It

When Your Documents Have Structure: Hierarchical Chunking

The Problem Nobody Talks About: PDFs, Tables, and Slides