Large Language Models, MirrorBERT — Transforming Models into Universal Lexical and Sentence Encoders

MirrorBERT is a model that uses self-supervision and data augmentation techniques to achieve comparable performance to fine-tuned models like BERT and RoBERTa on semantic similarity tasks. It can transform pretrained models into universal lexical encoders with high efficiency.