People ask me what they should attempt newly if they are training Indic LLMs since there are already a lot of Llama2/3 and Gemma based models. DORA goes one step further and adds additional…

GOOpenAI is a blog or publication that focuses on exploring and discussing advancements, research, and applications related to artificial intelligence (AI) and machine learning (ML). Through articles, tutorials, and analysis, GOOpenAI provides insights into  AI technologies, research breakthroughs, and their potential impact on various industries and domains. Developers and AI enthusiasts can learn about the latest developments in AI, gain practical knowledge, and stay updated with trends in the field.

GoPenAI

Two techniques to improve existing Indic LLM finetuned models: train with DORA (not plain LORA) and train with ORPO.

Improving Existing Indic LLM Finetuned Models

Answer.AI - Efficient finetuning of Llama 3 with FSDP QDoRA

Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch