Artificial Intelligence is no doubt the future of not just software development but the whole world. And I'm on a mission to master it - focusing first on mastering AI Agents.

In this video I show you an INCREDIBLE packaged local AI solution with Ollama for the LLMs, Qdrant for RAG, Postgres for the SQL database, and n8n for the no code workflow automations. This package is SO easy to set up and use! I walk you through the setup quick and then even show you how to use it to build a RAG AI Agent that runs entirely locally!

One really important thing to keep in mind is that while I created a no code AI agent with n8n in this video, this whole local AI setup could be used to actually code your AI agents! It doesn't have to be "no code" n8n for the actual agent. A custom coded RAG AI agent with n8n for the accompanying workflow automations (agent tools!) is another fantastic use of this local AI package.

Hardware: I am actually using a laptop for this video - with a 3070 laptop GPU and an i7-12700H CPU with 14 cores and 20 threads. So nothing crazy at all! It takes up to 5 minutes for me to get a response from Llama 3.1 8b including RAG. Definitely not fast enough for a chatbot though those kind of speeds could be okay for some agentic workloads. Typically though you will want a GPU with at least 8GB of VRAM for an 8b model, and at least 16GB of VRAM for a 70b model like Llama 3.1 70b.

00:00 - 01:17 - Unveiling the Local AI Package
01:18 - 06:38 - Installing the Local AI Package
06:39 - 07:45 - Exploring the Docker Containers
07:46 - 18:07 - Creating a Local RAG AI Agent
18:08 - 19:18 - Testing the Local RAG AI Agent
19:19 - 20:19 - Outro + Future Ideas

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

My version of the local AI starter kit with my improvements, as well as the template for the n8n RAG AI Agent workflow I created in the video, can be found here:

https://github.com/coleam00/ai-agents-masterclass/tree/main/local-ai-packaged

Link to the Local AI Starter Kit by n8n:

https://github.com/n8n-io/self-hosted-ai-starter-kit

I recommended GitHub Desktop and Docker Desktop in the video - here are the links for both:

https://desktop.github.com/download/
https://www.docker.com/products/docker-desktop/

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Join me as I push the limits of what is possible with AI. I'll be uploading videos twice a week - Sundays and Wednesdays at 7:00 PM CDT!

YouTube

Learn how to set up a comprehensive local AI infrastructure using a package from the n8n team, featuring components like olama for LLMs, Quadrant for the vector database, and PostgreSQL for the SQL database. The setup process involves configuring environment variables, Docker Compose files, and integrating various tools to create a fully local RAG AI agent. The guide offers detailed steps on customizing and extending the package, troubleshooting, and future enhancement plans.

Run ALL Your AI Locally in Minutes (LLMs, RAG, and more)