ZFS Fast Dedupe: How It Works and When Not to Use It

ZFS Fast Dedupe improves deduplication performance through dedicated special VDEVs and optimized lookup tables, but still carries significant performance costs as tables grow. While ideal for workloads with truly duplicated data (like cloning VMs or development environments), most workloads don't benefit from it. The feature works by storing lookup tables that compare data blocks, which starts fast but slows over time as tables expand. Special VDEV drives can accelerate lookups, but once added to RAID-Z pools, they cannot be removed and their failure means pool loss. Encrypted data and unique media files see minimal benefit due to high entropy.