Best of PyTorch — July 2024

1. 1
   Article

   Machine Learning Mastery·2y

   5 Tips for Getting Started with Deep Learning

   Deep learning, a subset of machine learning inspired by the human brain, has become essential in areas like computer vision, speech recognition, and text generation. To get started, focus on understanding machine learning basics, select a comfortable deep-learning framework (such as TensorFlow, PyTorch, or Keras), learn neural network architectures, start with simple projects, and practice regularly while engaging with the community for feedback and guidance.

   60

   1
2. 2
   Article

   Towards AI·2y

   A Practical Guide to Building GPT-2 with PyTorch (Part 1)

   Learn how to build and train a GPT-2 language model from scratch using PyTorch. This guide outlines steps to create a custom tokenizer, data loader, and a simple language model, demonstrating the process with Taylor Swift and Ed Sheeran song data. Follow along with the code provided to understand and implement each part of the model.

   45
3. 3
   Article

   Hacker News·2y

   SylphAI-Inc/LightRAG: The "PyTorch" library for LLM applications.

   LightRAG is a PyTorch library designed to assist developers with building and optimizing Retriever-Agent-Generator (RAG) pipelines for large language model (LLM) applications. It emphasizes a light, modular, and robust codebase that is 100% readable. LightRAG caters to diverse LLM use cases, from general AI applications like chatbots and summarization to traditional NLP tasks. With a clean, customizable setup, developers can trust and effectively implement it in production.

   25
4. 4
   Article

   Hacker News·2y

   KwaiVGI/LivePortrait: Make one portrait alive!

   LivePortrait is a GitHub repository containing the official PyTorch implementation of the LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control paper. The repository includes the initial version of the inference code and models, with continuous updates. Users can clone the repo, set up the environment using conda, install necessary dependencies, download pretrained weights, and run various scripts to animate portraits. The post also offers performance evaluation results on an RTX 4090 GPU and provides a Gradio interface for enhanced usability.

   19
5. 5
   Article

   PyTorch·2y

   Quantization-Aware Training for Large Language Models with PyTorch

   The post describes an end-to-end Quantization-Aware Training (QAT) process in PyTorch for large language models. It highlights how QAT can significantly improve accuracy and reduce perplexity degradation compared to post-training quantization (PTQ). Users can leverage QAT APIs in torchao for fine-tuning models in torchtune. Experimental results demonstrate substantial improvements in model performance when QAT is applied, particularly for the Llama3 model. The post also discusses future directions such as mixed-precision quantization, hyperparameter tuning, and extending QAT to other layers and more complex data types.

   13
6. 6
   Article

   KDnuggets·2y

   A Beginner’s Guide to PyTorch

   PyTorch, an open-source deep learning package developed by Meta AI, offers flexible model architecture, native CUDA support, and Python-based lower-level controls. The post explains the basics of using PyTorch, including installation, creating and manipulating Tensors, and training a simple neural network using the `nn.Module` class. It also covers evaluating the trained model using sample data.

   11
7. 7
   Article

   GoPenAI·2y

   Adversarial Attacks in Graph Neural Networks

   Adversarial attacks pose significant security threats to machine learning models, including graph neural networks (GNNs), by making small perturbations to input data that cause incorrect predictions. This tutorial covers implementing four types of adversarial attacks (FGSM, PGD, Carlini & Wagner, DeepFool) on GNNs using the PyTorch-Geometric library and the Cora dataset. It demonstrates the impact of these attacks on model accuracy and suggests mixed training with clean and perturbed data as a defense strategy.

   10