Note: in the above we only target two tensors q_proj and v_proj, but you can add several other lora targets: For 16 bit model, the token generation speed is ~3 tokens/sec on a Linux machine w/o GPU…

DevGenius serves as a platform for sharing insights and experiences in software development, offering articles and interviews with industry experts to gain  insights into various aspects of software engineering. By exploring DevGenius's curated content, developers can learn from real-world experiences, best practices, and lessons learned from seasoned professionals across different domains of software development. Whether you're looking for career advice, technical insights, or inspiration for your next project, DevGenius provides a  knowledge and expertise to help you succeed in your software development journey.

Dev Genius

Learn how to fine-tune the Mixtral-8x7B-Instruct model on your own data. Set up the environment, prepare the data, and run the fine-tuning script. The trained model will be saved in the 'mixtral' folder.

How to fine-tune Mixtral-8x7B-Instruct on your own data?