Code Like A Girl·2yHow I created an AWS System to crush the Architect exam
The post shares insights on building a notification system using AWS services to help revise for the AWS Solutions Architect Certification. The system sends PowerPoint slides as WhatsApp messages using Lambda, S3, EventBridge, and Twilio, assisting in timely and structured revision. The architecture is scalable and can be used for any revision needs.
Community Picks·1ycloudlena/s3manager: A Web GUI for your S3 buckets
s3manager is a Web GUI written in Go for managing S3 buckets from any provider. It allows users to list buckets, create new ones, upload and download objects, and delete objects. The application is configurable via environment variables and can be run locally, with Docker, or deployed to a Kubernetes cluster. It supports SSL, various signature types, IAM roles, and server-side encryption.
Substack·2yS3 Is Showing Its Age
S3 is falling behind its competitors in terms of features such as compare-and-swap operation, multi-region buckets, and object appends. These missing features are becoming a bigger problem as new infrastructure relies on object storage as the primary persistence layer. The decision to abandon or embrace S3 depends on the system's use cases and design goals. Preconditions on S3 and a unified S3Express One Zone API would make this decision easier.
Be A Better Dev·1yI Wish I Knew This One Thing Before Learning AWS
Understanding the core services in AWS, such as EC2, S3, SQS, SNS, and DynamoDB, is essential before diving into other services. These foundational services form the basis upon which other, more advanced services are built. Having a strong understanding of these basics will facilitate learning and using more complex AWS offerings.
Theo - t3․gg·2yWe stopped using serverless. The results are insane.
Upload Thing V7 drastically improves file upload speeds, making uploads five times faster. Major changes include seamless pausing and resuming of uploads, integration support for Remix, and the transition from relying on direct S3 uploads to a custom upload server, simplifying the process and enhancing control. This update reduces the number of steps and requests needed, enhances security, and unlocks new features like bring-your-own-bucket and better file validation.
Community Picks·2yHow to upload to S3 in Next.js and save references in Postgres
Learn how to add a feature to a Next.js app that allows users to upload files to Amazon S3 using presigned URLs, and save references to these files in a Postgres database. The guide covers creating a Neon project, setting up an S3 bucket, creating IAM user access keys, and setting up a Next.js application with necessary API routes to handle file uploads and database operations.
Materialized View·1yS3 Is the New SFTP
Fintech companies handle diverse data processing tasks, including shuffling files between vendors and partners, often using SFTP. Transitioning to modern data lakehouses using S3, Apache Iceberg, and Apache Parquet can centralize and streamline this process. This new method allows ease of access and management while maintaining advantages such as fast transfers and central access control. Although challenges like schema evolution remain, adopting data lakehouses can benefit companies seeking efficient and scalable data solutions. The trend is supported by rising customer demand and the involvement of startups providing innovative data export platforms.
Lobsters·1yBuilding a distributed log using S3 (under 150 lines of Go)
The post describes how to implement a durable, distributed, and highly available log using AWS S3 in less than 150 lines of Go. Key highlights include the structure of the log interface, the implementation of the Append and Read operations, handling concurrent writes with S3 conditional writes, and failover/crash recovery mechanisms. The open-source project includes code and tests, with several open issues for further improvements.
Medium·1yAWS S3 Deep Dive
Amazon S3 (Simple Storage Service) is a scalable object storage service offering high availability, security, and performance for various use cases like data lakes, backups, and big data analytics. It distinguishes itself from block storage and file storage, providing a flat storage structure through objects and buckets. Key features include data encryption, access control via IAM policies, and different bucket types optimized for various tasks such as general purpose, directory-specific, or table-based analytics. S3 also enhances data integrity and upload management via multipart uploads, versioning, and checksum validation. Additional services include S3 Object Lambda for real-time data transformation and S3 Batch Operations for large-scale data management tasks.
Medium·2yHow We Decreased Our Monthly AWS Costs from $10,000 to $1,500
Managing cloud infrastructure can become costly if not optimized. This guide shares steps to significantly reduce AWS costs from $10,000 to $1,500 per month. Key actions include cleaning up ECR images, optimizing ECS Fargate services, managing S3 storage, converting VPN usage to SSH tunnels, right-sizing RDS and DocumentDB instances, purchasing ECS savings plans, consolidating load balancers, removing redundant IPv4 addresses and environments, and right-sizing remaining resources.
ThePrimeTime·2yPrivate EMPTY S3 Bucket COST ME $1300
Discover how an empty private S3 bucket can cause a skyrocketing AWS bill, due to unauthorized requests from third parties and misconfigurations of an open-source tool. Learn about the potential risks and security measures to prevent such incidents.
AWS Tip·2yNode.js CreateReadStream and AWS MultipartUpload: Unlocking Memory and Network Efficient File Uploading Strategy.
Learn how to optimize file uploading in Node.js using fs.createReadStream and AWS MultipartUpload. Avoid high memory usage and improve performance when dealing with large files. AWS recommends using MultipartUpload for files over 100 MB.
Community Picks·2yThe Last Amazon S3 Video You'll Need To Watch (Presigned URLs, Multipart Upload)
Learn best practices for working with Amazon S3 from a .NET application. This guide covers using the S3 SDK, working with pre-signed URLs, and handling multi-part uploads for large files. Practical tips include navigating the AWS console, configuring S3 buckets, and implementing efficient file upload and download functionalities. Explore examples on configuring CORS and using client-side applications for direct file interactions with S3.
Neon·2yHow to create an AWS S3 bucket for Postgres backups
Learn how to set up an AWS S3 bucket to store Postgres backups, a useful step for managing backups in a multi-tenant architecture. This guide includes setting up AWS Providers and Roles, creating an OIDC Identity Provider and Role, and configuring an S3 bucket with an updated policy. The process involves using GitHub Actions to automate transfers, ideal for handling numerous projects efficiently.
freeCodeCamp·2yHow to Upload Large Files Efficiently with AWS S3 Multipart Upload
Efficiently handle large file uploads using Amazon S3 multipart upload. This approach splits a large file into smaller parts, uploads them independently, and combines them to create the final object. The guide covers setting up AWS S3, configuring a backend with Node.js, and building a React frontend for the upload process. This method enhances upload performance, adds fault tolerance, and allows pausing and resuming of uploads, making it suitable for large files and unstable networks.
freeCodeCamp·2yHow to Use Time To Live in Event-Driven Architecture in AWS
Time to Live (TTL) is a concept used to determine the lifespan of data in a distributed system. It is beneficial for ephemeral data like cached content, analytic metrics, and short-lived social media content. However, it is less useful for permanent data like media for streaming platforms or bank transactions. AWS services like SQS, S3, and DynamoDB support TTL, allowing developers to automate data expiration and efficiently manage resources. TTL can also be integrated with event-driven architectures to execute actions upon data expiration.
Snyk·2yHow to secure an S3 bucket on AWS?
Amazon S3 provides scalable, high-performance cloud storage, but securing S3 buckets is crucial to prevent data breaches. Key vulnerabilities include misconfigured permissions, unencrypted data, and lack of monitoring. Best practices for S3 bucket security include safeguarding permissions using IAM roles, enforcing encryption, and implementing secure transport policies like HTTPS. Regular auditing and using tools like Snyk for automated detection and remediation of cloud misconfigurations can significantly bolster security.
Faun·2yBuilding Infrastructure to Serve Millions of Users on AWS: Preparation
This post provides step-by-step instructions on building infrastructure to serve millions of users on AWS. It covers topics such as creating virtual machines, installing WordPress on a virtual machine, creating a database on AWS, and connecting WordPress to an S3 bucket.
AWS in Plain English·2yAWS S3 and Endpoints Benefits
AWS S3 is a highly available object storage service with security and encryption features. There are different ways to access S3 bucket, including using VPC Gateway Endpoint or VPC Interface Endpoint. Consider using interface endpoint for secure access across regions or from on-premises.
Hacker News·2yHacking misconfigured AWS S3 buckets: A complete guide
AWS S3 buckets are widely used for storing both public and sensitive data, but misconfigurations can lead to serious security issues. The guide covers common security misconfigurations in AWS S3, ways to enumerate and test S3 buckets for vulnerabilities, and useful tools for automating these checks. It also provides practical commands and tips for validating S3 bucket permissions, ensuring access controls, and maintaining security practices such as S3 versioning.
