Best of Data EngineeringFebruary 2024

  1. 1
    Article
    Avatar of tinybirdTinybird·2y

    How to choose the right type of database

    Understanding the different types of databases, factors to consider when choosing a database, and the implications of the CAP theorem on database selection.

  2. 2
    Article
    Avatar of taroTaro·2y

    The Fundamentals of Data Engineering - Preface + Chapter 1: Data Engineering Described

    This post provides an overview of the book 'Fundamentals of Data Engineering', discussing the motivations behind the book and the importance of data engineering in relation to data science and machine learning. It covers the Data Science Hierarchy of Needs, the Data Engineering Lifecycle, and the skills and activities of a Data Engineer. The post also discusses the stages of Data Maturity and the different types of Data Engineers. The next blog post will cover Chapter 2 of the book.

  3. 3
    Article
    Avatar of inPlainEngHQPython in Plain English·2y

    Setting up PostgreSQL and PgAdmin4 using Docker: A Simplified Approach.

    This post provides a simplified approach to setting up PostgreSQL and PgAdmin4 using Docker. It explains the benefits of containerization and guides readers through the process of installing Docker, running the Hello World container, and setting up PostgreSQL and PgAdmin4 containers. The post emphasizes the ease and efficiency of managing databases with containerization.

  4. 4
    Article
    Avatar of kdnuggetsKDnuggets·2y

    Free Data Engineering Course for Beginners

    Learn essential data engineering skills through a free course covering Docker, SQL, data pipeline creation, dbt, cron jobs, Airflow, and Airbyte.

  5. 5
    Article
    Avatar of kdnuggetsKDnuggets·2y

    Collection of Free Courses to Learn Data Science, Data Engineering, Machine Learning, MLOps, and LLMOps

    A collection of free courses for learning data science, data engineering, machine learning, MLOps, and LLMOps is provided. The courses are self-paced and community-based, offering valuable resources for beginners and experienced professionals.

  6. 6
    Article
    Avatar of motherduckMotherDuck·2y

    DuckDB & Python : end-to-end data engineering project

    This post discusses an end-to-end data engineering project using DuckDB and Python. It explores the architecture, ingestion pipeline, and sinking data using DuckDB. The post also provides instructions on how to fetch data from PyPi using Python and DuckDB.

  7. 7
    Article
    Avatar of medium_jsMedium·2y

    What we learned after running Airflow on Kubernetes for 2 years

    Learnings and insights from running Airflow on Kubernetes for 2 years, including executor choices, decoupling and dynamic DAG generation, fine-tuning configurations, and notifications/alerting/observability.