Note: in the above we only target two tensors q_proj and v_proj, but you can add several other lora targets: For 16 bit model, the token generation speed is ~3 tokens/sec on a Linux machine w/o GPU…

GOOpenAI is a blog or publication that focuses on exploring and discussing advancements, research, and applications related to artificial intelligence (AI) and machine learning (ML). Through articles, tutorials, and analysis, GOOpenAI provides insights into  AI technologies, research breakthroughs, and their potential impact on various industries and domains. Developers and AI enthusiasts can learn about the latest developments in AI, gain practical knowledge, and stay updated with trends in the field.

GoPenAI

Learn how to fine-tune the Mixtral-8x7B-Instruct model on your own data using three steps: setting up the environment, putting your data in the specified format, and running the fine-tuning script. The script will start fine-tuning the model with LoRA and save the model to a 'mixtral' folder.

How to fine-tune Mixtral-8x7B-Instruct on your own data?