Community Picks·2yHow to Run an LLM Locally with Pieces
The post provides information on running Local Large Language Models (LLLMs) locally within Pieces for Developers. It discusses the demand for secure and efficient machine learning solutions, hardware requirements for running LLMs, the difference between GPU and CPU, the best GPUs for local LLMs, troubleshooting common issues, and future-proofing the setup.
Firebase Developers·2yGemma2 Tutorial: Local LLM Development with Genkit
Google's open-source LLM, Gemma2, can be run locally using Ollama. This step-by-step guide explains how to install Gemma2, create a Genkit project, and integrate it with Ollama Gemma2. The process involves installing appropriate packages, configuring Genkit, and writing a sample flow to handle requests. The setup is efficient enough to be accomplished in around 30 lines of code.
YouTube·2yRun ALL Your AI Locally in Minutes (LLMs, RAG, and more)
Learn how to set up a comprehensive local AI infrastructure using a package from the n8n team, featuring components like olama for LLMs, Quadrant for the vector database, and PostgreSQL for the SQL database. The setup process involves configuring environment variables, Docker Compose files, and integrating various tools to create a fully local RAG AI agent. The guide offers detailed steps on customizing and extending the package, troubleshooting, and future enhancement plans.
Qt·2yExamples of local LLM usage
The post outlines how to use local large language models (LLMs) on a MacBook Pro M1 Max for generating images, extracting text from audio, and summarizing text. The author describes the use of stable-diffusion.cpp for image generation, whisper.cpp for audio transcription, and llama.cpp for text summarization. Detailed scripts and time performance metrics are provided for each use case.
Tech With Tim·2yCheck Out This INSANE AI Web Scraper
An individual has built an AI web scraper that can collect property information from websites and organize it into tables without manually inspecting the site's content. The scraper uses a local LLM to parse through the DOM content and generate the desired results. A full tutorial on how to build this scraper is available.
Sam Witteveen·2yGemma 2 - Local RAG with Ollama and LangChain
Gemma 2 has been released for multiple formats including Keras, PyTorch, and Hugging Face transformers. This post details the author's experience using the 9B and 27B models in Ollama, highlighting the better performance of the 9B model for real-time responses. A straightforward script is provided to create a fully local Retrieval-Augmented Generation (RAG) system using Gemma 2, Nomic embeddings, and ChromaDB, all executed within VSCode. The steps involve setting up an indexer, embedding transcripts from Alex Hormozi's YouTube channel, and handling text splitting methods. Debugging tips and additional add-ons for the RAG system are also discussed.