Best of Data Engineering — April 2024

  1. 1
    Article
    Avatar of kdnuggetsKDnuggets·2y

    7 Python Libraries Every Data Engineer Should Know

    Discover some essential Python libraries for data engineers, including Requests for API data extraction, BeautifulSoup for web scraping, Pandas for data manipulation, SQLAlchemy for database work, Airflow for workflow orchestration, PySpark for big data processing, and Kafka-Python for real-time data processing.

  2. 2
    Article
    Avatar of kdnuggetsKDnuggets·2y

    7 Steps to Mastering Data Engineering

    Learn the 7 steps to mastering data engineering, including containerization, workflow orchestration, data warehousing, analytics engineering, batch processing, streaming data processing, and building an end-to-end data pipeline.

  3. 3
    Article
    Avatar of substackSubstack·2y

    Data Pipeline - Incremental vs Full Load

    Learn about the pros, cons, and use cases of incremental and full load data pipeline design patterns in this article.

  4. 4
    Article
    Avatar of substackSubstack·2y

    The 2024 breaking into data engineering roadmap

    Learn about the necessary skills and steps to break into data engineering in 2024. Topics covered include learning SQL, Python or Scala, distributed compute, data modeling, building a portfolio project, and building a brand for interviews.

  5. 5
    Article
    Avatar of substackSubstack·2y

    SQL vs Python Data Pipelines

    The post discusses the decision of when to use Python and when to use SQL in data pipelines. It emphasizes the importance of striking a balance between the two and making good technical decisions. It also highlights the pitfalls to be careful about in both SQL and Python pipelines.

  6. 6
    Article
    Avatar of medium_jsMedium·2y

    Mastering the Art of Data: Python Code Snippets to Elevate Your Skills

    Master the art of data with Python and enhance your data skills using powerful Python code snippets. From data engineering and data science to data visualization and machine learning, this article covers a wide range of data-related tasks. Explore essential Python code snippets, tips, and tricks to streamline workflows, uncover insights, and deliver high-quality data solutions. Whether you're a beginner or an experienced practitioner, this article serves as a gateway to mastering data with Python.

See all Data Engineering archives