Best of Cloud — October 2024

  1. 1
    Article
    Avatar of medium_jsMedium·2y

    10 Best Resources to Learn Software Architecture in 2025

    Mastering software architecture is crucial for handling complex systems and transitioning from a developer role to an architect role. Essential resources include books like 'Designing Data-Intensive Applications' and courses such as 'The Complete Microservices and Event-Driven Architecture' on Udemy. Additionally, whitepapers and engineering blogs provide valuable insights. These resources cover various architectural styles, principles, and real-world challenges, helping you design scalable, maintainable, and high-performing systems.

  2. 2
    Article
    Avatar of devtoDEV·2y

    Dockerfile Best Practices: How to Create Efficient Containers

    In the era of microservices and cloud computing, optimizing Dockerfiles is essential for building lightweight, fast, and secure containers. Key practices include using minimal base images like Alpine or Scratch, reducing layers by combining commands, optimizing layer caching, installing dependencies wisely, managing secrets carefully, using a '.dockerignore' file, employing multi-stage builds, avoiding running containers as the root user, scanning for vulnerabilities, and directing logs to standard output. Regular updates and using specific version pins also enhance security and reproducibility.

  3. 3
    Article
    Avatar of bytebytegoByteByteGo·2y

    The Trillion Message Kafka Setup at Walmart

    Walmart's Apache Kafka setup processes trillions of messages daily with a 99.99% availability rate, supporting critical data movement, event-driven microservices, and streaming analytics. The team addressed challenges like consumer rebalancing, poison pill messages, and costs by designing a Message Proxy Service (MPS). This service decouples Kafka message consumption from its partition-based model, allowing consumer applications to scale independently and handling consumer failures effectively.

  4. 4
    Video
    Avatar of fireshipFireship·2y

    Appwrite in 100 Seconds

    Appwrite is a free, open-source backend platform designed to streamline the development of web and mobile applications. It offers essential backend services like server hosting, relational databases, file storage, secure authentication, and real-time notifications. With SDKs for multiple platforms, including major JavaScript frameworks, React Native, Flutter, Android, and iOS, developers can quickly implement features such as user authentication with minimal code. Appwrite also provides a robust admin dashboard and supports running server-side code for custom backend functionality.

  5. 5
    Article
    Avatar of lobstersLobsters·2y

    Talos Linux

    Talos Linux is a minimal, secure, and immutable operating system designed specifically for Kubernetes environments. It supports various platforms including cloud, bare metal, and virtualization. The system is managed entirely via an API, eliminating the need for SSH, shell, or console access. Talos Linux enhances security through mutual TLS authentication, immutable infrastructure, and adherence to best practices set by NIST and CIS. It also ensures predictability and evolvability by providing atomic updates and the latest stable versions of Kubernetes and Linux.

  6. 6
    Article
    Avatar of collectionsCollections·2y

    Understanding Backends for Frontends (BFF) Pattern

    Backends for Frontends (BFF) is an architectural pattern that assigns specific backends to different client types, optimizing API design for enhanced user experience. This approach improves performance, scalability, security, and simplifies frontend development by acting as a middleware layer. It uses gateways and orchestrators to manage diverse workflows and access modes, though it introduces complexity in sharing common functionalities and maintenance.

  7. 7
    Article
    Avatar of itnextITNEXT·2y

    Ambiguous architectural patterns

    The post discusses ambiguous architectural patterns such as Monolith, Microkernel, Domain Services, Cells, and Nanoservices. It highlights how these terms have evolved and sometimes been misinterpreted or redefined over time, leading to confusion. It stresses the importance of understanding these differences to avoid miscommunication and inefficiency in software development.

  8. 8
    Video
    Avatar of techworldwithnanaTechWorld with Nana·2y

    Full Golang Tutorial - Learn Go by Building a TodoList App

    Learn the essentials of Go (Golang) by building a to-do list application. This crash course covers the motivation behind Go's development, setting up the development environment, and understanding basic syntax and core concepts such as data types, variables, loops, and functions. Additionally, it demonstrates how Go is particularly suited for multi-threaded, concurrent applications, making it ideal for cloud-based deployments.

  9. 9
    Article
    Avatar of phProduct Hunt·2y

    Rabona Tunnel - Expose Your Localhost Securely

    Rabona Tunnel allows developers to securely expose their localhost to the internet. It's a developer tool launched on October 7th, 2024. Ideal for tasks requiring remote access to local development environments.

  10. 10
    Video
    Avatar of communityCommunity Picks·2y

    Apache Kafka vs RabbitMQ Performance (Latency - Throughput - Saturation)

    A detailed comparison between Apache Kafka and RabbitMQ, focusing on latency, throughput, CPU and memory usage, and disk operations. The test reveals that RabbitMQ offers lower latency, while Kafka provides higher throughput and resilience. The comparison extends to RabbitMQ Streams, where Kafka again shows superior performance. The post provides a comprehensive analysis suited for selecting the right messaging broker based on specific needs.

  11. 11
    Article
    Avatar of systemdesigncodexSystem Design Codex·2y

    Kafka Load Balancing at Agoda for Terabytes of Data

    Agoda uses Kafka to manage hundreds of terabytes of data across various supply systems, including hotels and restaurants, ensuring real-time price updates. They faced challenges with the traditional round-robin partitioning and consumer assignment due to heterogeneous hardware and uneven workloads, resulting in over-provisioning. Agoda addressed these issues by implementing dynamic, lag-aware strategies, including a lag-aware producer and consumer, to optimize message distribution and reduce latency.

  12. 12
    Article
    Avatar of softwaretestingmagazineSoftware Testing Magazine·2y

    Top 10 Software Testing Trends for 2024

    The software testing industry is set to grow at 7% CAGR from 2024-2032, driven by advancements in testing methodologies and tools. Challenges such as scarce talent, poor documentation, and time constraints are common. Emerging trends include scriptless test automation, IoT testing, AI and machine learning in testing, cloud-based testing, and shift-left testing. Technologies like blockchain and big data testing are also becoming crucial. Embracing these trends helps improve efficiency, speed up release cycles, and ensure high-quality software delivery.

  13. 13
    Article
    Avatar of itnextITNEXT·2y

    Self-Hosting a Container Registry

    Self-hosting a container registry allows developers to maintain control over their container images, ensuring greater security and compliance with regulations. The guide provides step-by-step instructions to set up a registry server using Docker and Docker Compose, covering the configuration of Docker registry, Nginx for SSL/TLS handling, and domain setup. Alternatives like using Kubernetes or managed services like Harbor are also discussed.

  14. 14
    Article
    Avatar of hnHacker News·2y

    [2408.13296] The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities

    The report provides a comprehensive examination of fine-tuning Large Language Models (LLMs) by integrating theoretical insights with practical applications. It covers the historical evolution of LLMs, fine-tuning methodologies, and introduces a seven-stage pipeline for fine-tuning. Key topics include dealing with imbalanced datasets, optimization techniques, parameter-efficient methods like LoRA, and advanced techniques such as Mixture of Experts (MoE) and Proximal Policy Optimization (PPO). The report also addresses validation frameworks, post-deployment monitoring, inference optimization, and challenges related to scalability, privacy, and accountability, offering actionable insights for navigating LLM fine-tuning.

  15. 15
    Article
    Avatar of devtronDevtron·2y

    Best 5 Alternatives to Kubernetes Dashboard

    Managing Kubernetes at scale can be complex, particularly with multiple clusters spread across regions. The default Kubernetes dashboard offers basic functionality but struggles with multi-cluster management, advanced analytics, and fine-grained RBAC. Alternatives like Devtron, OpenLens, Skooner, Headlamp, and Octant provide enhanced features such as comprehensive visibility, advanced troubleshooting, and unified control over multiple clusters. These tools can significantly improve operational efficiency and cluster management for organizations.

  16. 16
    Article
    Avatar of infoqInfoQ·2y

    Building a Global Caching System at Netflix: A Deep Dive to Global Replication

    Netflix uses a global replication strategy with EVCache, a distributed key-value store, to ensure data availability across four regions. EVCache handles 30 million global replication events and 400 million operations per second, leveraging 200 Memcached clusters and 22,000 servers. Features include client-initiated replication, topology-aware clients, and batch compression, which reduce network costs and enhance performance. The replication process involves client-initiated data mutations, Kafka for metadata handling, and SQS for robust error handling.

  17. 17
    Video
    Avatar of communityCommunity Picks·2y

    MySQL vs PostgreSQL (Round 2) Performance Benchmark (Latency - Throughput - Saturation)

    Performance comparison between PostgreSQL and MySQL focusing on latency, throughput, and system resource usage. The benchmark simulates an e-commerce website managing customer and order data, with tests for INSERT, UPDATE, DELETE, and SELECT operations. Improvements include optimized settings for both databases and utilizing i3en EC2 instances. PostgreSQL outperforms MySQL in write-heavy operations, while read operations show smaller differences.

  18. 18
    Article
    Avatar of communityCommunity Picks·2y

    Choosing Between Docker Swarm and Kubernetes for Container Management

    Docker Swarm and Kubernetes are essential tools for container orchestration in distributed environments. Docker Swarm is simpler to manage and integrates well with Docker, making it suitable for small-scale operations. Kubernetes, developed by Google, offers advanced features like auto-scaling, self-healing, and extensive tool integrations, making it ideal for complex, large-scale deployments. Understanding their key differences in networking, storage solutions, security, integration, and community support can help you choose the right tool for your needs.

  19. 19
    Article
    Avatar of unzipUnzip·2y

    🛡️Auth as a Service

    Auth as a Service (AaaS) simplifies the implementation of authentication by allowing developers to add some wrapper code and manage setups via environment variables and redirect URLs. It offers features like passwordless logins, user management, and billing integration. While AaaS can accelerate time to market and aid in compliance, concerns about pricing and vendor lock-in are significant. Various providers, such as Auth0, Stytch, and Firebase, compete in this space, each offering unique capabilities and user experiences.

  20. 20
    Article
    Avatar of communityCommunity Picks·2y

    The greatest resume I've ever seen

    Daniel Singletary transitioned from being a plumber to a cloud DevOps engineer through hard work and completing the Cloud Resume Challenge. Despite having no professional IT experience, his resume showcased his strong troubleshooting, collaboration, and testing skills honed in plumbing. Daniel's story demonstrates how non-traditional backgrounds can lead to successful tech careers through perseverance and skill-building.

  21. 21
    Article
    Avatar of selfhstselfh.st·2y

    This Week in Self-Hosted (4 October 2024)

    The latest updates in self-hosted software cover significant advancements like Borg's new rclone support for cloud backups, Mozilla's beta release of Thunderbird for Android, and various updates across numerous self-hosted applications like Home Assistant, Keycloak, and qBittorrent. New software releases were also highlighted, including apps like CheckHomeIP for email notifications and PSWD for secure password generation. Additionally, Streamyfin, a user-friendly Jellyfin client for mobile devices, was spotlighted for its modern features.

  22. 22
    Article
    Avatar of cerbosCerbos·2y

    What is authorization? Examples and definitions

    Authorization is a fundamental component of secure systems that controls access to resources based on rules, contextual details, and checks. Various paradigms like Role-Based Access Control (RBAC), Attribute-Based Access Control (ABAC), Discretionary Access Control (DAC), Mandatory Access Control (MAC), and Relationship-Based Access Control (ReBAC) offer different approaches to managing permissions. Key practices include adhering to the principle of least privilege, conducting regular audits, ensuring scalability, balancing flexibility and security, and monitoring performance. Effective authorization balances ease of use with security, protecting sensitive information while remaining user-friendly.

  23. 23
    Article
    Avatar of dhhDavid Heinemeier Hansson·2y

    Kamal 2: Thou need not PaaS

    Kamal 2 is a significant upgrade to the tool that simplifies deploying containerized applications on your own hardware, eliminating the need for complex solutions like Kubernetes. It supports deploying multiple applications on a single server and auto-provisions SSL certificates via Let's Encrypt. Kamal 2 is versatile, working with dedicated servers, cloud VMs, and inexpensive VPS setups, allowing seamless transitions without hefty migration costs. A demo showcases deploying a Go and a Rails application within 30 minutes.

  24. 24
    Article
    Avatar of evolvedevevolvedev·2y

    The Ultimate Guide for Deploying Go Applications to Production

    Learn to deploy Go applications and services in the cloud. The guide covers building binaries, deploying on VPS, using Docker, and more.

  25. 25
    Article
    Avatar of dhhDavid Heinemeier Hansson·2y

    Our cloud-exit savings will now top ten million over five years

    37signals has successfully transitioned several cloud applications, including Basecamp and HEY, from AWS to their own hardware. This move resulted in estimated savings of over $10 million over five years. They achieved this by utilizing existing data center infrastructure and efficiently managing new Dell hardware and Pure Storage setups. Although the cloud offers benefits in early stages or for fluctuating loads, significant cost savings can be realized by analyzing and transitioning substantial workloads to owned hardware.

See all Cloud archives