Best of ELK — November 2024

1. 1
   Article

   The New Stack·1y

   Elasticsearch Was Great, But Vector Databases Are the Future

   Keyword matching, represented by Elasticsearch, has been the standard for information retrieval systems. However, as AI-powered semantic search technology advances, vector databases are becoming central to a new era of search. Combining both approaches, hybrid search uses a mix of vector and traditional search methods, balancing semantic relevance with exact keyword matching. Milvus is highlighted as a vector database offering efficiencies and performance improvements over Elasticsearch, particularly in handling dense and sparse vectors. This unified approach simplifies infrastructure and enhances search capabilities, making vector databases a promising solution for future advanced search needs.

   62
2. 2
   Article

   InfoSec Write-ups·1y

   Building an Integrated Threat Intelligence Platform Using Python and Kibana

   The post discusses the creation of a comprehensive Threat Intelligence Platform (TIP) using Python, Elasticsearch, and Kibana. Key features include breach monitoring, subdomain enumeration, phishing domain detection, GitHub leak searches, IOCs integration, dark web monitoring, and HTTP header analysis. The system uses Python scripts for data collection, Elasticsearch for data storage, and Kibana for visualization. The post emphasizes ethical considerations, including data privacy, legality, and secure coding practices.

   62
3. 3
   Article

   BigData Boutique blog·2y

   OpenSearch Data Migration from Elasticsearch - The Guide

   Learn how to migrate from Elasticsearch to OpenSearch with minimal downtime and no data loss. This guide covers upgrading your Elasticsearch version, setting up your OpenSearch cluster, checking plugin compatibility, backing up data, and planning the transition period. It also discusses methods for data migration, ensuring data integrity, and post-migration tasks including verifying data accuracy and updating applications to work with OpenSearch.

   16
4. 4
   Article

   BigData Boutique blog·2y

   Elasticsearch and OpenSearch Query Limits

   Elasticsearch and OpenSearch are robust search and analytics engines with several query limits to ensure performance and resource efficiency. They include limits on result size, max clause count, field data, query throughput, and complex joins. Understanding and configuring these limits is essential to maintain optimal performance and avoid errors.

   15