Best of PyTorch — August 2024

1. 1
   Article

   Hacker News·2y

   PyTorch is dead. Long live JAX.

   The post critiques PyTorch's effectiveness in industrial-scale scientific computing, arguing it wasn't designed for large-scale, distributed systems. In contrast, JAX, developed by DeepMind, offers a compiler-centered approach with better scalability and performance, making it more suitable for large-scale AI research. JAX's commitment to functional programming and reproducibility further enhances its utility, while PyTorch's attempts to integrate multiple backends lead to fragmentation and inefficiency. The post urges the adoption of JAX for improved research productivity and reliability.

   50

   2
2. 2
   Article

   ITNEXT·2y

   Mini PyTorch from Scratch — Module 6 (part 2)

   Introduces the 2d upsampling operation using nearest neighbor interpolation, an alternative to transposed convolution to avoid checkerboard artifacts. The post details the implementation of the Upsample2d class, which includes methods for resizing with nearest neighbor interpolation. This lays the groundwork for building complex image generation networks like UNet and GANs.

   17
3. 3
   Article

   Collections·2y

   Introducing Torchchat: Seamlessly Run PyTorch LLMs on Any Device

   Torchchat is a versatile framework designed by the PyTorch team for running large language models (LLMs) across various devices, including desktops, servers, and mobile platforms. It supports multiple interaction modes such as command line interfaces, browser-hosted chats, and a REST API, and offers robust performance optimizations like Ahead-Of-Time (AOT) Inductor and ExecuTorch. The framework is optimized for leading LLMs like Llama 3 and 3.1, supporting export, quantization, and evaluation to ensure efficient and accurate model inference.

   15
4. 4
   Article

   ITNEXT·2y

   Mini PyTorch from Scratch — Module 5 (part 6)

   This post details the implementation of a facial landmark detection sample using custom image and key-point augmentations, leveraging a ResNet18 model. It outlines how to handle datasets, apply corresponding transformations, and execute a training loop with PyTorch, ensuring synchronization between image and key-point augmentations. The entire sample code is accessible on Github.

   11
5. 5
   Article

   Daily Dose of Data Science | Avi Chawla | Substack·2y

   DropBlock vs. Dropout for Regularizing CNNs

   Dropout, while effective for fully connected layers, falls short in convolutional layers because of the spatial correlation between nearby pixels. DropBlock addresses this by dropping contiguous blocks of pixels rather than individual ones, thus improving model robustness. On ImageNet data, DropBlock outperforms Dropout with a notable accuracy gain. It is easily implementable in PyTorch with adjustable parameters like block size and drop rate.

   10