It’s limited in some ways like you cannot have conversation history. Let’s try to load and use one of these models using Python. Download quantized model. In my case it was full model…

GOOpenAI is a blog or publication that focuses on exploring and discussing advancements, research, and applications related to artificial intelligence (AI) and machine learning (ML). Through articles, tutorials, and analysis, GOOpenAI provides insights into  AI technologies, research breakthroughs, and their potential impact on various industries and domains. Developers and AI enthusiasts can learn about the latest developments in AI, gain practical knowledge, and stay updated with trends in the field.

GoPenAI

Learn how to load and use quantized models with Python/llama_cpp_python. Get the prerequisites for installation and download the necessary tools. Explore an example of using WxPython for GUI application.

Tiny LLM hacks: Loading quantized model using Python/llama_cpp_python.