Community Picks·2yHow to Structure Your Backend Code in Node.js (Express.js)
In Node.js applications using Express.js, effective codebase structuring is vital for maintainability, scalability, and easier collaboration. This guide outlines a common directory structure, explaining the purpose of each folder and file, from 'app.js' to 'node_modules', to help manage complexity and build robust applications.
ByteByteGo·2yHow Stripe Scaled to 5 Million Database Queries Per Second
Stripe processes $1 trillion in payments and supports 5 million database queries per second with 99.999% uptime. This scalability is achieved through their internal database-as-a-service (DBaaS) called DocDB, which extends MongoDB's community edition. DocDB includes features like horizontal scaling, multi-tenancy, strong security, and a Data Movement Platform that facilitates zero-downtime data migrations across database shards.
ITNEXT·2yPerformance Benchmarking: gRPC+Protobuf vs. HTTP+JSON
A performance benchmarking experiment compared gRPC with Protocol Buffers to HTTP with JSON for microservices communication in Go. The tests, run on a local machine, showed that HTTP/2 was slower than gRPC, which in turn was slower than HTTP/1. Despite these results, Protocol Buffers over gRPC remain an effective choice for inter-service communication due to its structured format.
Hussein Nasser·2yThe Art of database systems
Understanding database systems revolves around key principles: data must be read from the disk to memory and then to CPU cache to be useful, with each step significantly faster than the last. Disk space is limited, memory is scarce, and CPU caches are even scarcer, so it's crucial to fully utilize chunks of data at each stage. These insights stress the importance of efficient SQL queries and database configurations to maximize performance.
Substack·2yA Primer on Databases
Databases have been fundamental to software development for decades. The post discusses their history, from the invention of SQL databases to the rise of unstructured and cloud-based databases. It highlights the current database landscape, including the significance of transaction processing and analytics. The piece also touches on emerging technologies like vector databases, which are crucial for AI development. The author emphasizes that while the core technology of databases is not extremely complex, distribution and platformization will be key factors for future success in the database market.
asayer·2yDocument your API with Swagger
Swagger simplifies API documentation by providing clear, interactive tools that enhance the API development process. It allows developers to define endpoints, parameters, and responses efficiently. Built on the OpenAPI Specification, Swagger supports multiple programming languages and frameworks, offering both online and local installation options. The tool's advanced features include code generation, testing, and customization, making it a powerful asset for creating comprehensive and user-friendly API documentation.
swizec.com·2yWhy SQL is Forever
SQL and relational databases remain fundamental for transactional data, despite the advances and popularity of NoSQL technologies over the past decades. Many NoSQL systems have either been removed, adapted to include SQL/natively support transactions, or are mainly used for caching and analytics. This demonstrates the enduring flexibility and utility of SQL, including new features like JSON support and vector databases, which relational databases have successfully integrated while maintaining ACID properties.
Community Picks·2yCan Postgres replace Redis as a cache?
The post discusses the possibility of using Postgres as a cache instead of Redis, highlighting that while Postgres can simplify the technology stack and reduce costs, it lacks the performance and specialized features of Redis. Unlogged tables in Postgres improve write performance but not read performance. Moreover, Redis offers built-in features like expiration and eviction policies that are complex to implement in Postgres. Ultimately, Redis is recommended for its superior performance and simplicity in use as a caching solution.
Community Picks·2yThe Performance Impact of Writing Bad SQL Queries
Poorly written SQL queries can severely degrade database performance, leading to slow response times and inefficient resource utilization. Common mistakes include using 'SELECT *', ignoring execution plans, and inefficient joins. SQL’s simplicity can lead to writing slow queries, especially without proper knowledge or under tight deadlines. Sometimes, systems can tolerate inefficient queries in non-critical applications or low-concurrency environments. However, these bad queries can cause hidden bottlenecks and increased resource consumption. Using tools like execution plans and IDE plugins can help optimize SQL queries, ensuring better system efficiency and scalability.
Community Picks·2yThe ultimate multi-model database
SurrealDB is a versatile multi-model database designed for modern applications. It integrates the database layer, querying layer, and API/authentication into one platform. It supports SurrealQL, GraphQL, REST, and JSON-RPC for data access and modification. SurrealDB can handle advanced querying without complex JOINs and offers real-time data syncing. It can operate in-memory or as a distributed cluster, suitable for various environments. Additionally, it supports embedded JavaScript functions and ES2020 standards for advanced data manipulation.
Web Tools Weekly·2yCSS Tools, JSON, Databases, Vue
This post highlights various tools spanning CSS, HTML, JSON, databases, and Vue. It includes advanced CSS and HTML browser extensions, useful JavaScript libraries, and tools to simplify database and JSON management. Additionally, it features a collection of Vue tools for creating maps, handling props dynamically, and scaffolding new projects with ease.
Hacker News·2yNode.js is Here to Stay
Node.js has become an essential element for modern web development since its release in 2009. With over 6.3 million websites powered by it and its ubiquity among Fortune 500 companies, Node.js remains highly relevant. The platform boasts a lightweight, event-driven architecture suitable for scalability and high-concurrency. It continues to evolve with new features like ECMAScript Modules (ESM), worker threads, built-in Fetch API, and improved security measures. Node.js ensures up-to-date security through an active response team and coordination with the Open Source Security Foundation. Despite rumors of its decline, metrics indicate that Node.js is thriving and will continue to be a robust choice for developers.
Community Picks·2ygRPC fundamentals with Go
gRPC is a powerful tool for establishing robust and scalable communication between server applications. This post explores using gRPC with Go through a practical example of a coffee shop, demonstrating how to define messages using Protocol Buffers, generate code, implement a server, and create a client. Key advantages of gRPC over REST APIs include type safety, performance optimization, and efficient handling of large data transfers through features like streaming RPCs.
Hacker News·2y35% Faster Than The Filesystem
SQLite can read and write small blobs roughly 35% faster than directly accessing individual files on disk. SQLite also uses about 20% less disk space due to its more efficient storage format. The performance gains come from reduced system call overhead and efficient data packing. Tests have shown SQLite's competitive latency with direct file I/O, with some systems exhibiting higher SQLite performance. Variations in performance are observed across different operating systems and hardware.
Tinybird·2yBest practices for timestamps and time zones in databases
The post provides best practices for managing timestamps and time zones in databases, emphasizing the importance of using UTC for storing historical timestamps. It discusses avoiding unnecessary complexity, ensuring unambiguous time representations, using appropriate data types, understanding time zone relationships, and leveraging system-provided functions for time conversions. The guide underscores the need for careful data transformation and thorough testing to avoid errors in time-based analytics.
Community Picks·2yHow to copy large Postgres databases in seconds
Neon introduces a revolutionary way to copy large Postgres databases instantly using database branching. This method uses a copy-on-write technique, making it both fast and resource-efficient. Traditional methods like physical file copies, logical backups, and logical replication are less efficient for large databases. With Neon branching, creating and managing separate database instances for development, testing, and migrations becomes much easier without significant additional storage requirements.
Community Picks·2yDealing with Race Conditions: A Practical Example
The post describes a practical example of dealing with race conditions in an application managing on-call shifts for doctors. It explains how naive API implementations can lead to race conditions and demonstrates two PostgreSQL-based solutions—serializable transaction isolation and advisory locks—to handle these issues. The article includes SQL snippets and code examples for implementing these solutions and discusses the importance of addressing race conditions in various real-life scenarios.
Baeldung·2yProtobuf vs. gRPC
Microservices architecture relies on efficient communication methods. Protobuf, a language-neutral mechanism for serializing structured data, offers compact and efficient data transfer but lacks human readability. gRPC, an open-source RPC framework built on HTTP/2 and using Protobuf for defining services, enables high-performance communication with features like streaming and multiplexing. While Protobuf excels in data serialization, gRPC provides a robust RPC framework suitable for real-time data exchange. The choice between them depends on specific needs, balancing efficiency, readability, and ease of configuration.
DEV·2yPOST only? Let's end this absurd API design debate
Addressing the debate on 'POST only' APIs, the post explains the misconceptions about POST-only designs, distinguishing between HTTP methods and API design styles. It highlights the principles of effective API design and compares RESTful and RPC API styles, emphasizing the appropriate scenarios for each. REST focuses on resources and standard operations, making it ideal for web services; RPC simplifies remote calls to resemble local function calls, suitable for internal services or complex operations. The ultimate goal should be clear, consistent, and efficient API design tailored to specific project needs.
ElixirStatus·2yLearning Elixir as a frontend developer
A frontend developer shares their journey of learning Elixir and Phoenix to expand their backend skills. The post highlights the initial learning curve, challenges encountered, and the satisfaction gained from mastering new concepts. It discusses both the strengths and weaknesses of Elixir, along with the overall positive experience, while emphasizing the importance of stepping out of one's comfort zone to grow professionally.
Community Picks·2yBridging Backend and Data Engineering: Communicating Through Events
In modern software development, seamless communication between backend services and data engineering pipelines is crucial. Traditional methods like REST APIs and batch processing often fall short for real-time demands. An event-driven architecture (EDA) offers a solution by using asynchronous event communication, enabling integration of diverse systems. A practical approach is setting up a Pub/Sub system where services broadcast and consume events via standardized formats. This method allows for selective event subscription and facilitates efficient asynchronous communication without overhauling the infrastructure.
Community Picks·2yHow to scale Laravel: beyond the basics (Advanced Laravel Scaling)
This post provides five advanced tips for scaling Laravel applications to handle millions of users. Key strategies include separating read and write database connections, using batch inserts and updates to reduce database overhead, implementing full-page caching to improve performance, utilizing pre-computed JSON files on a CDN to lessen server load, and avoiding synchronous report generation by queuing reports and using read replicas.
Hacker News·2yOwez/yark: YouTube archiving made simple.
Yark simplifies YouTube archiving by allowing users to easily download and manage video and metadata. After installing Python and FFmpeg, users can create an archive, download content with the refresh command, and view everything via a locally hosted offline website. The archive is organized into a readable directory structure. Light and dark modes are supported, and users are encouraged to suggest features through the issues tab of the repository.
PlanetScale·2ySharding strategies: directory-based, range-based, and hash-based
Discover the different types of sharding strategies—directory-based, range-based, and hash-based—along with their pros and cons. Understand how solutions like Vitess and PlanetScale are making sharding more approachable, even though it remains a complex task. Learn how to choose the right sharding strategy based on your database needs while considering the potential challenges like uneven data distribution and added query complexity.
