DigitalOcean Community·2yPyTorch 101: Understanding Hooks
Learn how to use hooks in PyTorch for debugging and visualization during the training process. This tutorial explains the concept and functionality of hooks, including both forward and backward hooks, and provides code examples to demonstrate their usage. It also discusses the intricacies of using hooks with tensors and nn.Module objects, cautioning about potential complications in complex networks.
Daily Dose of Data Science | Avi Chawla | Substack·2yA Subtle Trick to Optimize Neural Network Training
Discover a subtle optimization trick for neural network training that involves normalizing data after transferring it to the GPU. This simple rearrangement can significantly reduce data transfer time, especially in tasks like image classification where pixel values are initially 8-bit integers. While the technique may not apply to all use cases, such as NLP, it can offer noticeable performance gains in applicable scenarios.
Daily Dose of Data Science | Avi Chawla | Substack·2yDeep Learning Models Can Learn Non-Existing Patterns
Deep learning models can sometimes learn non-existing patterns, especially when data is not properly shuffled during training. This post illustrates an example where a classification neural network failed to converge due to label-ordered data but performed well when the data was shuffled. Shuffling helps in mini-batch gradient descent by ensuring that each mini-batch contains a balanced representation of classes. Be mindful of this and other potential pitfalls to improve model generalization and performance.
Daily Dose of Data Science | Avi Chawla | Substack·2yImplementing a Siamese Network
This guide provides a beginner-friendly implementation of a Siamese network using the MNIST dataset. It covers dataset creation, defining the network, applying contrastive loss, and training the model. The post includes detailed steps for creating image pairs, defining a custom dataset class, and testing the model's performance. Key takeaway: Siamese networks require labeled data, but methods to handle unlabeled data are in discussion.
Daily Dose of Data Science | Avi Chawla | Substack·2yA Counterintuitive Behaviour of PyTorch DataLoader
PyTorch DataLoader applies transformations on the fly during iteration, leading to redundant computations and potentially longer training times. The suggested solution is to transform datasets beforehand using libraries like NumPy and create the DataLoader using the pre-transformed dataset. This approach circumvents the inefficiency of applying transformations during each epoch.