LLM2Vec: A Simple AI Approach to Transform Any Decoder-Only LLM into a Text Encoder Achieving SOTA Performance on MTEB in the Unsupervised and Supervised Category

LLM2Vec is a simple AI approach that transforms any decoder-only LLM into a text encoder, achieving state-of-the-art performance on the Massive Text Embeddings Benchmark (MTEB) in the unsupervised and supervised category. The method uses bidirectional attention, masked next token prediction, and unsupervised contrastive learning to develop robust representations.