Best of Performance — July 2024

1. 1
   Article

   Substack·2y

   System Design: How to Scale a Database

   Scaling a database is essential as an application grows to maintain optimal performance. Strategies include: vertical scaling, adding resources to one server; indexing, creating indexes on frequently queried columns; sharding, splitting data across different servers; vertical partitioning, separating columns into smaller tables; caching, storing frequently accessed data in a faster storage layer; replication, creating copies of the database in different regions; materialized views, pre-computing and storing complex query results; and data denormalization, introducing redundancy to optimize reads by combining tables. Each method has trade-offs and can be combined based on application needs.

   1.3K

   26
2. 2
   Article

   System Design Codex·2y

   8 Strategies for Reducing Latency

   High latency can render an application unusable, frustrating users and negatively impacting business outcomes. Developers need to understand low-latency strategies such as caching, using Content Delivery Networks (CDNs), load balancing, asynchronous processing, database indexing, data compression, pre-caching, and utilizing keep-alive connections to mitigate these issues and improve performance.

   805

   21
3. 3
   Article

   DEV·2y

   Image Lazy Loading

   The post introduces image lazy loading as a technique to improve React application performance. It highlights advantages such as faster initial load times, reduced bandwidth usage, smoother scrolling, and improved search engine ranking. It provides an example using both native lazy loading attributes and the `react-lazy-load-image-component` npm package. The author advises caution for certain use cases like carousels and above-the-fold images.

   437

   13
4. 4
   Article

   Community Picks·2y

   Typesense

   Typesense is a privacy-friendly, open-source search engine optimized for high performance and ease-of-use. It employs advanced search algorithms, corrects typos, offers flexible ranking, and supports a variety of search features such as synonym recognition, filtering, and geographic searches. Typesense can handle multiple users' data, sort records dynamically, and provide resilient, production-grade search services with simple setup steps.

   330

   10
5. 5
   Article

   swizec.com·2y

   90% of performance is data access patterns

   Optimizing your data access patterns can significantly enhance application performance. A real-world example illustrates how identifying and removing an outdated session middleware fetching patient appointment data reduced CPU usage by 66% and halved the latency of the slowest API requests. This case underscores the evolving nature of systems and the critical role of observability in catching inefficiencies.

   214

   2
6. 6
   Article

   ITNEXT·2y

   Performance Benchmarking: gRPC+Protobuf vs. HTTP+JSON

   A performance benchmarking experiment compared gRPC with Protocol Buffers to HTTP with JSON for microservices communication in Go. The tests, run on a local machine, showed that HTTP/2 was slower than gRPC, which in turn was slower than HTTP/1. Despite these results, Protocol Buffers over gRPC remain an effective choice for inter-service communication due to its structured format.

   184

   2
7. 7
   Article

   freeCodeCamp·2y

   How to Use Python Generators – Explained With Code Examples

   Python generators are a powerful tool for lazy iteration through sequences, optimizing memory usage and enhancing performance for large datasets. They produce items one at a time only when needed, making them ideal for handling streams of data, iterative algorithms, and real-time simulations. Generators help in writing cleaner and more readable code by managing iteration state automatically and efficiently.

   169

   8
8. 8
   Article

   System Design Codex·2y

   7 Techniques for Database Performance & Scaling

   The performance and scalability of databases are crucial for enhancing user experience. Important factors affecting database performance include item size, item type, dataset size, and throughput requirements. Seven effective techniques for optimizing database performance include indexing, materialized views, denormalization, vertical scaling, caching, replication, and sharding. Each technique offers unique benefits and trade-offs. Indexing improves query speed but uses additional disk space, while materialized views reduce query time but require extra storage. Denormalization enhances read performance at the cost of data redundancy. Vertical scaling boosts performance but has hardware limits. Caching decreases database load and read time but may involve data staleness. Replication enhances read performance and availability yet introduces replication lag. Finally, sharding enables horizontal scaling and cost reductions but adds complexity in data management.

   153

   1
9. 9
   Article

   KDnuggets·2y

   5 Tips for Improving SQL Query Performance

   Strong SQL skills are crucial in data roles, where optimizing query performance can significantly impact application efficiency. Key tips include avoiding SELECT * by specifying columns, using GROUP BY instead of SELECT DISTINCT, limiting query results, and employing indexes with caution. Balancing these techniques can improve query performance and ensure efficient database operations.

   149

   3
10. 10
    Article

    Hussein Nasser·2y

    The Art of database systems

    Understanding database systems revolves around key principles: data must be read from the disk to memory and then to CPU cache to be useful, with each step significantly faster than the last. Disk space is limited, memory is scarce, and CPU caches are even scarcer, so it's crucial to fully utilize chunks of data at each stage. These insights stress the importance of efficient SQL queries and database configurations to maximize performance.

    140

    3
11. 11
    Article

    Awesome Go·2y

    How I sent 500 million HTTP requests to 2.5 million hosts

    An individual leveraged Go's simplicity and concurrency to send 500 million HTTP/1.1 requests to 2.5 million hosts for an ethical hacking use case. By utilizing Kubernetes for horizontal scaling and optimizing both code and libraries, they achieved this massive scale efficiently. Key optimizations included pre-resolving DNS, hand-crafting HTTP requests, and using the fasthttp library.

    137

    1
12. 12
    Article

    Community Picks·2y

    Efficient DOM Manipulation with the Virtual DOM and Refs

    Efficient DOM manipulation is key to fast and responsive websites. Traditional methods of handling the DOM can slow down complex web pages. React's Virtual DOM offers a solution by minimizing updates to only necessary parts, significantly improving performance. React's Diffing Algorithm identifies changes between versions of the Virtual DOM, ensuring efficient updates. For specific tasks like focusing on an input or scrolling, React provides the useRef Hook for direct DOM manipulation. By leveraging these techniques, developers can build smoother and faster applications.

    126

    4
13. 13
    Article

    Community Picks·2y

    Can Postgres replace Redis as a cache?

    The post discusses the possibility of using Postgres as a cache instead of Redis, highlighting that while Postgres can simplify the technology stack and reduce costs, it lacks the performance and specialized features of Redis. Unlogged tables in Postgres improve write performance but not read performance. Moreover, Redis offers built-in features like expiration and eviction policies that are complex to implement in Postgres. Ultimately, Redis is recommended for its superior performance and simplicity in use as a caching solution.

    121

    2
14. 14
    Article

    stitcher.io·2y

    Improved lazy loading

    The post discusses the design and implementation of an ORM named Tempest, focusing on improved lazy loading mechanisms. The key approach includes using typed properties for static analysis, making minimal configuration necessary by inferring relations directly from property types and docblocks. It contrasts Tempest's strict lazy loading with Laravel's default approach which allows lazy loading, thereby preventing n+1 query problems and optimizing performance. The post also explores PHP's technical details, including handling uninitialized properties and triggering property accessors for lazy loading.

    105

    1
15. 15
    Video

    Community Picks·2y

    7 Must-know Strategies to Scale Your Database

    Understanding when and why to scale your database is essential to maintain optimal performance as your application grows. Key strategies include indexing for quick data retrieval, using materialized views for pre-computed snapshots of data, and implementing denormalization to simplify complex queries. Vertical scaling, adding resources to a single server, and caching frequently accessed data in a fast storage layer can enhance responsiveness. Replication bolsters availability and fault tolerance by creating database copies on multiple servers. Sharding, which involves splitting a database into smaller sections, enables horizontal scaling and manages large data loads efficiently.

    89
16. 16
    Article

    Community Picks·2y

    One Million Checkboxes

    The creator of 'One Million Checkboxes' shares insights into the rapid development and scaling of his website, utilizing bit arrays, Redis, WebSockets, and React-window for performance. A peculiar bug related to bit ordering is discussed, highlighting the challenges of efficient checkbox state management in a collaborative setting.

    85

    4
17. 17
    Article

    DEV·2y

    Don't ride a donkey when you can drive a brand-new sports car

    GraphQL offers a modern approach to API development, providing type safety, self-documentation, and better performance. It allows clients to retrieve exactly the data they need and supports multiple programming languages, making it versatile for various use cases. Tools like GraphQL Playground, GraphiQL, and GraphQL codegen enhance the development experience by simplifying querying and reducing boilerplate code. Communities such as 'The Guild' offer support, making GraphQL a robust choice over traditional REST APIs.

    83

    6
18. 18
    Article

    asayer·2y

    Optimizing Angular Performance with HttpInterceptor Caching

    Learn how to enhance your Angular application's performance by implementing HTTP caching with HttpInterceptor. This guide covers the basics of HttpInterceptor, creating a cache service, and integrating the caching mechanism into an Angular app. Benefits include centralized caching logic, improved performance, consistent caching strategy, and offline support. Step-by-step instructions walk you through setting up the Angular project, creating caching logic, and developing a simple HttpInterceptor. The result is a more efficient and user-friendly application with faster load times and reduced server load.

    81

    4
19. 19
    Article

    Hacker News·2y

    35% Faster Than The Filesystem

    SQLite can read and write small blobs roughly 35% faster than directly accessing individual files on disk. SQLite also uses about 20% less disk space due to its more efficient storage format. The performance gains come from reduced system call overhead and efficient data packing. Tests have shown SQLite's competitive latency with direct file I/O, with some systems exhibiting higher SQLite performance. Variations in performance are observed across different operating systems and hardware.

    80

    1
20. 20
    Article

    habr·2y

    Spans in C#: Your Best Friend for Efficient Coding

    Spans in C# provide a powerful way to handle memory efficiently, reducing allocations and improving performance. They allow for direct manipulation of memory regions, support safe code practices, and are stack-allocated. This guide covers their use, differences between Span<T> and ReadOnlySpan<T>, and practical examples, including JSON parsing, converting arrays, and strings to spans. Ensuring proper scope and compatibility with development environments is crucial for optimal use.

    78
21. 21
    Article

    iO tech_hub·2y

    Making Web Workers enjoyable

    Web Workers allow developers to offload intensive tasks in a single-threaded JavaScript environment to maintain a responsive main thread. Comlink, a library from Google, simplifies the process by adopting an RPC style for message transmission, making it easier to work with Web Workers. This results in enhanced developer experience and more efficient web applications.

    61

    2
22. 22
    Article

    Code Maze·2y

    Local Functions vs Lambda Expressions in C#

    Local functions and lambda expressions in C# serve different purposes and have distinct performance characteristics. Local functions are methods nested within other methods and excel in performance since they can avoid heap allocation and use struct instead of class captures. Lambda expressions, while concise and useful for passing as delegates, generally require heap allocation and object instantiation. Local functions can be generic, recursive, and used as iterators, whereas lambda expressions struggle with these features. Thus, local functions are often more versatile and performant, especially when not using APIs that demand delegates.

    60
23. 23
    Article

    Netflix TechBlog·2y

    Java 21 Virtual Threads - Dude, Where’s My Lock?

    Netflix engineers encountered issues with virtual threads in Java 21 causing timeouts and hung instances in their microservices running on SpringBoot 3 and embedded Tomcat. Detailed diagnostics revealed that virtual threads were waiting for reentrant locks, with OS threads pinned and unable to release, leading to a variation of deadlock. The investigation and resolution process underscores the challenges and potential of adopting virtual threads for improved performance.

    59
24. 24
    Article

    asayer·2y

    Forever Functional: Workers and Promises

    JavaScript allows concurrency through async functions, but to optimize performance further, web workers and worker threads can be used. This guide demonstrates how to use workers functionally to improve performance, starting with a simple Fibonacci calculation. By handling workers with promises, parallel processing is achieved, but creating new workers each time can cause delays. A worker pool solution is introduced to reuse workers, enhancing efficiency without compromising parallelism.

    59

    1
25. 25
    Article

    Community Picks·2y

    Performance Benchmarks: Comparing Query Latency across TypeScript ORMs & Databases

    Performance benchmarks compare query latencies of three TypeScript ORMs (Prisma, TypeORM, Drizzle) across PostgreSQL databases on AWS RDS, Supabase, and Neon. Benchmarking methodology includes 14 queries, executed 500 times on an EC2 instance to measure query latencies. Results show that performance varies based on the specific query, dataset, schema, and infrastructure. Most queries perform similarly across different ORMs, with some exceptions like 'Nested find all' queries. Factors such as network latency and limited API features impact the results. Prisma Optimize offers insights and recommendations for better query performance.

    56

    4