Best of FinOps — 2024
- 1
- 2
Hacker News·2y
Tracking supermarket prices with playwright
In Dec 2022, a website was created to track price changes in Greece's largest supermarkets using Playwright for web scraping. The main challenges included handling JavaScript-based sites, automating the scraping process, and avoiding IP restrictions. After initial attempts with an old laptop failed, a decision was made to use Hetzner for its cost-efficiency. The setup integrated Tailscale to tackle IP restrictions and used a CI server to manage daily scraping tasks. Optimizations focused on improving scrape speed and cost-efficiency, like upgrading server specs and reducing data fetched.
- 3
Community Picks·2y
How We Reduced a $1,000/month Imgix Bill to $1 using Google Cloud - Hardcover
Adam Fortuna shares how his team reduced their Imgix bill from $1,000/month to $1 using Google Cloud. By migrating to Google Cloud Run and utilizing Imaginary, a Go-based image processing server, they managed to downscale their costs significantly. The new setup involves using a CDN, load balancer, and custom caching configurations to achieve faster response times and lower expenses.
- 4
Semaphore·2y
10 Open-Source Tools for Optimizing Cloud Expenses
Discover 10 open-source tools that can help organizations optimize their cloud expenses. These tools provide cost efficiency, budget management, and increased ROI. The tools include Kubernetes, Terraform, Grafana, Prometheus, Apache Kafka, Elasticsearch, Hadoop, OpenStack, Docker, and Apache Spark.
- 5
Medium·2y
How We Decreased Our Monthly AWS Costs from $10,000 to $1,500
Managing cloud infrastructure can become costly if not optimized. This guide shares steps to significantly reduce AWS costs from $10,000 to $1,500 per month. Key actions include cleaning up ECR images, optimizing ECS Fargate services, managing S3 storage, converting VPN usage to SSH tunnels, right-sizing RDS and DocumentDB instances, purchasing ECS savings plans, consolidating load balancers, removing redundant IPv4 addresses and environments, and right-sizing remaining resources.
- 6
David Heinemeier Hansson·2y
Our cloud-exit savings will now top ten million over five years
37signals has successfully transitioned several cloud applications, including Basecamp and HEY, from AWS to their own hardware. This move resulted in estimated savings of over $10 million over five years. They achieved this by utilizing existing data center infrastructure and efficiently managing new Dell hardware and Pure Storage setups. Although the cloud offers benefits in early stages or for fluctuating loads, significant cost savings can be realized by analyzing and transitioning substantial workloads to owned hardware.
- 7
How we reduced our cloud spending by 20%
Duolingo successfully reduced its cloud spending by 20% through various methods such as improving observability, removing unnecessary resources, leveraging AWS optimization strategies, and reducing cloud traffic. The key steps included using third-party tools for cost tracking, deleting unused data and services, optimizing compute resource allocation, and refining RDS and EC2 usage strategies. These initiatives not only saved money but also improved code health and engineering practices.
- 8
Quastor Daily·2y
How Canva Collects 25 Billion Events Per Day
Canva collects and processes 25 billion events per day using AWS Kinesis for real-time streaming data. Key highlights include their use of Kinesis Data Streams to ingest data, routing it to Snowflake for processing, and techniques such as event compression to minimize costs, which saved them $600k annually. By switching from AWS SQS to Kinesis, Canva reduced their costs by 85% and ensured low latency and high uptime for their data pipeline.
- 9
finout·2y
AWS Cost Optimization: 6 Free Tools & 10 Hacks to Cut AWS Bills
AWS cost optimization helps manage and reduce cloud expenditures without compromising performance. Strategies include selecting appropriate instance types, utilizing discounted pricing models like reserved and spot instances, and leveraging free optimization tools such as AWS Cost Explorer and AWS Budgets. Key practices involve right-sizing, increasing elasticity, optimizing storage, and continuous monitoring. Additionally, adopting hacks like exploring AWS bills, choosing the right region, and using spot instances can significantly cut costs.
- 10
databricks·2y
Building a Cost-Optimized Chatbot with Semantic Caching
Chatbots are useful tools for businesses, enhancing efficiency and supporting employees by providing informed responses. However, high-performing models can be expensive to query at scale. A cost-saving strategy, semantic caching, reuses responses for similar questions, reducing redundant computations. Databricks offers an optimal platform for implementing this approach, providing necessary components like Vector Search and MLflow. While semantic caching can reduce costs and latency, slight declines in response quality must be weighed against these benefits. Databricks Mosaic AI efficiently supports these implementations with robust governance and model evaluation tools.
- 11
Cast AI·2y
Kubernetes Cost Optimization: Reduce Your Cloud Bill
Running applications on Kubernetes can lead to high costs if not managed properly. Common cost traps include overprovisioning, improper scaling, selecting wrong cloud instances, and cost tracking issues. Monitoring key metrics such as daily spend, resource utilization, and historical cost allocation can help control expenses. Additionally, automation tools can significantly improve cost efficiency and allow teams to focus on higher-value tasks.
- 12
DevOps Life·2y
Migrated Infrastructure from EC2 to Kubernetes & Docker
A recent project at Zupee involved migrating infrastructure from EC2 instances to a containerized environment using Docker and Kubernetes. This move brought significant cost savings and maintained zero downtime while coping with increased traffic. The process highlighted the importance of FinOps in bridging engineering and finance for sound infrastructure decisions. The successful transition also emphasized the role of DevOps practices in ensuring scalability and performance.
- 13
How to reduce Postgres compute costs as you scale
Optimizing cloud costs is essential, and compute resources often consume the larger portion of your Postgres bill compared to storage. In managed services like Amazon RDS, compute costs are influenced by instance class, on-demand vs reserved pricing, and single-AZ vs multi-AZ deployments. Strategies for optimization include choosing the right database strategy, appropriately sizing instances, manually downsizing or pausing instances, and optimizing database queries. Neon offers an enhanced approach with autoscaling and autosuspend features, which dynamically adjust compute resources based on load, significantly reducing costs and management overhead.
- 14
Stack Overflow Blog·2y
Best practices for cost-efficient Kafka clusters
Kafka facilitates real-time data processing across distributed systems, but managing costs while maintaining performance requires careful planning. Key cost drivers include computing infrastructure, data transfer, and storage. Different deployment types (serverless, hosted, and self-hosted) impact costs uniquely. Cost-efficiency involves continuous optimization, such as removing inactive resources, enabling client-level compression, avoiding default settings, and adopting dynamic sizing to match workloads. Following these best practices can help keep Kafka clusters cost-efficient.
- 15
- 16
MongoDB·2y
Building Gen AI with MongoDB & AI Partners
Generative AI is being increasingly adopted, and MongoDB, in collaboration with several AI and tech partners, is working to provide the right tools and education for companies to build genAI applications. Their initiatives include webinars and video content that cover both broad and specific AI topics. New partnerships with companies like Astronomer, CloudZero, ObjectBox, and Rasa aim to enhance data orchestration, cloud cost optimization, mobile data management, and conversational AI capabilities.
- 17
The New Stack·2y
Cloud vs. On-Prem: Comparing Long-Term Costs
The post compares the long-term costs of cloud vs. on-premises infrastructure, emphasizing that while the cloud offers convenience and flexibility, it may not be cost-effective for sustained use. It discusses scenarios where moving out of the cloud can save significant costs, especially for large-scale operations. The European Data Act’s impact on free data transfer from the cloud and various cost-saving strategies for on-prem setups are highlighted. It also touches on the advancements in hardware and open source software that make maintaining on-prem infrastructure easier.
- 18
Lobsters·2yMigrating from AWS to Self-Hosting ⚡ Zig Programming Language
Ziglang.org transitioned from AWS S3 + CloudFront to a more cost-effective self-hosting solution using a Hetzner instance. The motivation behind this move was to reduce high AWS costs, highlighting that 99% uptime is sufficient for their needs. By doing this, they aim to cut expenses and allocate more funds to pay contributors. Future plans include providing torrent files for releases to further optimize resource use.
- 19
AWS Budgets vs. Cost Explorer: Features, Use Cases, Pros & Cons
AWS Budgets and AWS Cost Explorer are key financial management tools for AWS users. AWS Budgets helps set and manage spending limits with real-time alerts, focusing on budget control and prevention of cost overruns. AWS Cost Explorer provides detailed cost and usage reports, historical data analysis, and visualizations to optimize spending. Businesses can choose based on their need for either proactive budget management or in-depth cost analysis.
- 20
Understanding the FinOps Lifecycle: Inform, Optimize, Operate
FinOps (Finance+DevOps) provides a framework for efficient cloud cost management, emphasizing real-time adjustments and collaboration among engineering, finance, and business teams. The FinOps lifecycle consists of three key phases: Inform, Optimize, and Operate, each aimed at enhancing cloud efficiency and reducing costs. The Inform phase focuses on data collection and visibility, the Optimize phase on improving efficiency and reducing wastage, and the Operate phase on implementing strategies with a focus on automation and accountability. Tools like Finout can help in effectively implementing FinOps by providing visibility, optimizing resources, and encouraging collaboration.
- 21
GoPenAI·2y
Make the OpenAI Function Calling Work Better and Cheaper with a Two-Step Function Call 🚀
Learn how to make OpenAI function calling work better and cheaper with a two-step function call. By only sending necessary function details, you can lower the cost per message and improve AI performance.
- 22
How we’ve saved 98% in cloud costs by writing our own database
The author explains how they saved 98% in cloud costs by writing their own database. They highlight the challenges of using existing databases for their specific use case and describe the features and trade-offs of their custom solution.
- 23
How To Migrate Stateful Workloads On Kubernetes With Zero Downtime
Kubernetes was designed for ephemeral, stateless workloads but falls short for stateful applications requiring persistent data and long-running processes. This post discusses challenges such as interruptions causing downtime, low resource utilization, and balancing cost with performance. CAST AI's Container Live Migration addresses these by enabling seamless migration of stateful workloads, optimizing resource utilization, and reducing costs through efficient bin-packing and Spot instance usage.
- 24
Amazon Frugal Architecture Explained 💰
This post explains Amazon's frugal architecture principles through a hypothetical case study, highlighting the importance of considering costs in system design. Key practices include using the 80-20 rule to avoid feature creep, implementing autoscaling to manage infrastructure costs, setting up monitoring for resource usage, understanding tradeoffs in architectural decisions, and optimizing the system periodically.
- 25
