Best of ELK — October 2024

1. 1
   Article

   Trendyol Tech·2y

   Turning Millions of Kafka Events Into Meaningful Reports for Sellers

   Trendyol’s Export Center developed the Seller Reporting API to transform Kafka event data into insightful reports for sellers. They used Elasticsearch for data storage and effective Date Histogram Aggregation to handle time-based data. The implementation involved creating a system to index order events and querying the indexed data to create detailed reports. These reports cater to sellers' needs for data over various periods, comparing sales across different regions and currencies.

   32

   1
2. 2
   Article

   Netflix TechBlog·2y

   Introducing Netflix TimeSeries Data Abstraction Layer

   Netflix has introduced a TimeSeries Data Abstraction Layer designed to handle vast amounts of temporal data with millisecond access latency. Key features include efficient data partitioning, flexible storage integration (using Apache Cassandra and Elasticsearch), and scalability to manage high-throughput, immutable temporal event data. This abstraction layer optimizes storage and query efficiency, addressing issues like global read/write operations, tunable configurations, bursty traffic management, and cost efficiency. It plays a vital role in various Netflix services like user interaction tracking, feature rollout analysis, and asset impression tracking.

   29
3. 3
   Article

   Baeldung·2y

   Logstash vs. Kafka

   Logstash and Kafka are powerful tools for managing real-time data streams, with Logstash specializing in data processing and Kafka excelling in distributed event streaming. Logstash is ideal for transforming log data and forwarding it to various outputs, while Kafka is designed for high-throughput, fault-tolerant message delivery. This post provides an in-depth comparison of their components, command-line examples, and discusses how they can work together to build robust data pipelines.

   16
4. 4
   Article

   BigData Boutique blog·2y

   Elasticsearch Performance and Cost Efficiency on Elastic Cloud and On-Prem

   Discover essential strategies to optimize Elasticsearch performance and cost efficiency for both Elastic Cloud and on-premises deployments. Key tactics include scaling up vs. scaling out, data tiering, continuous monitoring of critical metrics, efficient shard distribution, and advanced query optimization techniques. Participants in a recent webinar hosted by BigData Boutique and Elastic learned how to enhance their Elasticsearch setups for optimal performance and cost-effectiveness.

   12