How I run LLMs locally

Running LLMs locally can be achieved with various open-source tools on a powerful computer with a core i9 CPU, 4090 GPU, and 96 GB RAM. LLMs performance varies based on model size and hardware specifications. Tools like Ollama, Open WebUI, and llamafile are used for running models, while AUTOMATIC1111 and Fooocus are preferred Show more