MobiLlama is a compact Small Language Model (SLM) with 0.5 billion parameters, designed for resource-constrained devices. It focuses on maintaining high performance while being energy-efficient, ensuring privacy, and reducing computational costs. Inspired by TinyLlama and Llama-2, MobiLlama integrates parameter sharing to optimize pre-training and deployment. It balances computational efficiency and the capability to understand complex language patterns, providing an efficient alternative to larger models like ChatGPT and Falcon.
Sort: