Meet RedPajama: An AI Project to Create Fully Open-Source Large Language Models Beginning with the Release of a 1.2 Trillion Token Dataset

The RedPajama project aims to create leading, fully open-source models. The first step of the project, reproducing the LLaMA training dataset, has been completed. The project is being supported by the Oak Ridge Leadership Computing Facility through the INCITE program. The team is excited to instruct and tune the models.