7 Projects to Master Data Engineering
KDnuggets
DECEMBER 4, 2024
Learn to build, run, and manage data engineering pipelines both locally and in the cloud using popular tools.
Data Engineer Related Topics 
7 Steps to Mastering Data Engineering
KDnuggets
APRIL 12, 2024
The only data engineering roadmap you need for an introduction to concepts, tools, and techniques to collect, store, transform, analyze, and model data.
Data Engineer 
382
382 Join 17,000+
professionals
Sign Up for our Newsletter
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Free Data Engineering Course for Beginners
KDnuggets
FEBRUARY 12, 2024
Interested in data engineering but don't know where to start? Get up to speed in data engineering fundamentals with this free course.
Data Engineer 353
353 
Landing a Data Engineer Role: Free Courses and Certifications
KDnuggets
JULY 15, 2024
Is it possible to learn data engineering for free? I claim it is and present the evidence for that in the form of 10 free data engineering courses.
Data Engineer 385
385 
Project Ideas to Master Data Engineering
KDnuggets
AUGUST 30, 2024
Data engineering is best learned by doing projects. Here are six projects focusing on different data engineering skills to ensure you have it all covered. But which ones?
Data Engineer 349
349 
7 Python Libraries Every Data Engineer Should Know
KDnuggets
APRIL 25, 2024
Interested in switching to data engineering? Here’s a list of Python libraries you’ll find super helpful.
Data Engineer 400
400 
10 GitHub Repositories to Master Data Engineering
KDnuggets
MAY 21, 2024
Learn data engineering through free courses, tutorials, books, tools, guides, roadmaps, practice exercises, projects, and other resources.
Data Engineer 361
361 
Data Engineering – A Journal with Pragmatic Blueprint
Analytics Vidhya
JUNE 23, 2022
This article was published as a part of the Data Science Blogathon. Introduction to Data Engineering In recent days the consignment of data produced from innumerable sources is drastically increasing day-to-day. So, processing and storing of these data has also become highly strenuous.
Data Engineer 378
378 
9 Skills You Need to Become a Data Engineer
KDnuggets
NOVEMBER 2, 2022
A data engineer is a fast-growing profession with amazing challenges and rewards. Which skills do you need to become a data engineer? In this post, we’ll take a look at both hard and soft skills.
Data Engineer 400
400 
Introduction to SQL for Data Engineering
Analytics Vidhya
APRIL 23, 2022
The post Introduction to SQL for Data Engineering appeared first on Analytics Vidhya. So this time I’ll be answering some of the factual questions about SQL which every beginner needs to know before getting […].
SQL 400
400 
7 Data Engineering Tools for Beginners
KDnuggets
OCTOBER 3, 2024
Learn the data engineering tools for data orchestration, database management, batch processing, ETL (Extract, Transform, Load), data transformation, data visualization, and data streaming.
Data Engineer 364
364 
10 Built-In Python Modules Every Data Engineer Should Know
KDnuggets
SEPTEMBER 2, 2024
Interested in data engineering? Check out this round-up of built-in Python modules that'll come in handy for data engineering tasks.
Data Engineer 382
382 
The Only Free Course You Need To Become a Professional Data Engineer
KDnuggets
JANUARY 26, 2024
Data Engineering ZoomCamp offers free access to reading materials, video tutorials, assignments, homeworks, projects, and workshops.
Data Engineer 391
391 
Top 9 Python Libraries for Data Engineers
Analytics Vidhya
MAY 30, 2024
Introduction Python is the favorite language for most data engineers due to its adaptability and abundance of libraries for various tasks such as manipulation, machine learning, and data visualization. This post looks at the top 9 Python libraries necessary for data engineers to have successful careers.
Data Engineer 317
317 
What Does A Data Engineer Do?
Adrian Bridgwater for Forbes
SEPTEMBER 20, 2024
What Is A Data Engineer? It’s a moving definition really, because the role of the data engineer itself is changing.
Data Engineer 317
317 
Top 20 Data Engineering Project Ideas [With Source Code]
Analytics Vidhya
SEPTEMBER 20, 2023
Data engineering plays a pivotal role in the vast data ecosystem by collecting, transforming, and delivering data essential for analytics, reporting, and machine learning. Aspiring data engineers often seek real-world projects to gain hands-on experience and showcase their expertise.
Data Engineer 353
353 
QlikView for Data Engineers Explained with Architecture
Analytics Vidhya
AUGUST 6, 2022
With QlikView, you can analyze and visualize data and their relationships and use these analyzes to make decisions. It Supports various data sources, including […]. The post QlikView for Data Engineers Explained with Architecture appeared first on Analytics Vidhya.
Data Engineer 383
383 
What Data Engineers Really Do?
Analytics Vidhya
JUNE 25, 2023
In a data-driven world, behind-the-scenes heroes like data engineers play a crucial role in ensuring smooth data flow. A data engineer investigates the issue, identifies a glitch in the e-commerce platform’s data funnel, and swiftly implements seamless data pipelines.
Data Engineer 306
306 
5 SQL Visualization Tools for Data Engineers
KDnuggets
FEBRUARY 24, 2023
This article will discuss SQL visualization, its role in augmenting the modern-day data engineer, and five categories of SQL visualization tools.
Data Engineer 344
344 
Web Scrapping- Tool for Data Engineering
Analytics Vidhya
SEPTEMBER 26, 2022
The post Web Scrapping- Tool for Data Engineering appeared first on Analytics Vidhya. The usefulness of the topic is one that easily helps other disciplines. Web content could be required in a way that makes it less effective to visit and use a website […].
Data Engineer 364
364 
Largest Data Engineering Survey Reports on Adoption of Modern Data Stack Tools
insideBIGDATA
JUNE 14, 2023
Airbyte, creators of a fast-growing open-source data integration platform, made available results of the biggest data engineering survey in the market which provides insights into the latest trends, tools, and practices in data engineering – especially adoption of tools in the modern data stack.
Data Engineer 374
374 
Understand the ACID and BASE in Morden Data Engineering
Analytics Vidhya
DECEMBER 12, 2022
Introduction Dear Data Engineers, this article is a very interesting topic. Let me give some flashback; a few years ago, Mr.Someone in the discussion coined the new word how ACID and BASE properties of DATA. The post Understand the ACID and BASE in Morden Data Engineering appeared first on Analytics Vidhya.
Data Engineer 353
353 
Most Essential 2023 Interview Questions on Data Engineering
Analytics Vidhya
FEBRUARY 7, 2023
Introduction Data engineering is the field of study that deals with the design, construction, deployment, and maintenance of data processing systems. The goal of this domain is to collect, store, and process data efficiently and efficiently so that it can be used to support business decisions and power data-driven applications.
Data Engineer 318
318 
Data Engineering for Streaming Data on GCP
Analytics Vidhya
APRIL 3, 2023
Real-time dashboards such as GCP provide strong data visualization and actionable information for decision-makers. Nevertheless, setting up a streaming data pipeline to power such dashboards may […] The post Data Engineering for Streaming Data on GCP appeared first on Analytics Vidhya.
Data Engineer 319
319 
Neo4j vs. Amazon Neptune: Graph Databases in Data Engineering
Analytics Vidhya
AUGUST 4, 2024
These powerful tools are designed to manage and query intricate data relationships effortlessly. This article discusses […] The post Neo4j vs. Amazon Neptune: Graph Databases in Data Engineering appeared first on Analytics Vidhya.
Data Engineer 305
305 
Data Abstraction for Data Engineering with its Different Levels
Analytics Vidhya
OCTOBER 10, 2022
Introduction A data model is an abstraction of real-world events that we use to create, capture, and store data in a database that user applications require, omitting unnecessary details. The post Data Abstraction for Data Engineering with its Different Levels appeared first on Analytics Vidhya.
Data Engineer 349
349 
Modern Data Engineering with MAGE: Empowering Efficient Data Processing
Analytics Vidhya
JUNE 20, 2023
Introduction In today’s data-driven world, organizations across industries are dealing with massive volumes of data, complex pipelines, and the need for efficient data processing.
Data Engineer 305
305 
Step-by-Step Roadmap to Become a Data Engineer in 2023
Analytics Vidhya
JANUARY 2, 2023
While not all of us are tech enthusiasts, we all have a fair knowledge of how Data Science works in our day-to-day lives. All of this is based on Data Science which is […]. The post Step-by-Step Roadmap to Become a Data Engineer in 2023 appeared first on Analytics Vidhya.
Data Engineer 329
329 
KDnuggets News, December 6: GitHub Repositories to Master Machine Learning • 5 Free Courses to Master Data Engineering
KDnuggets
DECEMBER 6, 2023
This week on KDnuggets: Discover GitHub repositories from machine learning courses, bootcamps, books, tools, interview questions, cheat sheets, MLOps platforms, and more to master ML and secure your dream job • Data engineers must prepare and manage the infrastructure and tools necessary for the whole data workflow in a data-driven company • And much, (..)
Machine Learning 351
351 
Databricks Assistant Tips & Tricks for Data Engineers
databricks
MAY 1, 2024
The generative AI revolution is transforming the way that teams work, and Databricks Assistant leverages the best of these advancements. It allows you.
Data Engineer 331
331 
8 Data Engineering Jobs You Need to Choose from in 2024
KDnuggets
JULY 9, 2024
Choosing a career as a data engineer is not just a good decision, it might just be the perfect one.
Data Engineer 246
246 
Data Scientist, Data Engineer & Other Data Careers, Explained
KDnuggets
APRIL 27, 2022
In this article, we will have a look at five distinct data careers, and hopefully provide some advice on how to get one's feet wet in this convoluted field.
Data Scientist 400
400 
Introducing Databricks LakeFlow: A unified, intelligent solution for data engineering
databricks
JUNE 13, 2024
Today, we are excited to announce Databricks LakeFlow, a new solution that contains everything you need to build and operate production data pipelines.
Data Engineer 361
361 
We Don’t Need Data Scientists, We Need Data Engineers
KDnuggets
DECEMBER 9, 2022
As more people are entering the field of Data Science and more companies are hiring for data-centric roles, what type of jobs are currently in highest demand?
Data Scientist 400
400 
How Do Data Scientists and Data Engineers Work Together?
KDnuggets
AUGUST 18, 2022
If you’re considering a career in data science, it’s important to understand how these two fields differ, and which one might be more appropriate for someone with your skills and interests.
Data Scientist 399
399 
Collection of Free Courses to Learn Data Science, Data Engineering, Machine Learning, MLOps, and LLMOps
KDnuggets
FEBRUARY 28, 2024
Begin your data professional journey from the basics of statistics to building a production-grade AI application.
Machine Learning 400
400 
Data Engineering Vault: A 1000 Node Second Brain for DE Knowledge
Hacker News
SEPTEMBER 15, 2024
Welcome to the Data Engineering Vault an integral part of my Second Brain. It’s a curated network of data engineering knowledge, designed to facilitate exploration and discovery.
Data Engineer 181
181 
5 Free Online Courses to Learn Data Engineering Fundamentals
KDnuggets
JULY 31, 2024
Kickstart a new career in one of the most popular tech careers where you can earn a 6 figure salary.
Data Engineer 284
284 
Intelligent Data Engineering for Enterprise AI with Databricks and Informatica
databricks
OCTOBER 22, 2024
Generative AI holds tremendous promise for how organizations unlock value from their data. However, it also comes with a litany of challenges around.
Data Engineer 278
278 
Ultimate Collection of 50 Free Courses for Mastering Data Science
KDnuggets
APRIL 19, 2024
The collection includes free courses on Python, SQL, Data Analytics, Business Intelligence, Data Engineering, Machine Learning, Deep Learning, Generative AI, and MLOps.
Data Science 388
388 
Data Engineering and GenAI: The Tools Practitioners Need
databricks
AUGUST 30, 2024
As organizations leverage their proprietary data for models, many encounter the hard truth: The best GenAI models in the world will not succeed without good data.
Data Engineer 235
235 
A Complete Collection of Data Science Free Courses – Part 2
KDnuggets
MARCH 29, 2023
The second part covers the list of Machine Learning, Deep Learning, Computer Vision, Natural Language Processing, Data Engineering, and MLOps.
The Complete Collection of Data Science Books – Part 1
KDnuggets
MAY 19, 2022
Read the best books on Programming, Statistics, Data Engineering, Web Scraping, Data Analytics, Business Intelligence, Data Applications, Data Management, Big Data, and Cloud Architecture.
Data Science 400
400 
5 Tips for Building Scalable Data Pipelines
KDnuggets
NOVEMBER 25, 2024
Building data pipelines is a very important skill that you should learn as a data engineer. A data pipeline is just a series of procedures that transport data from one location to another, frequently changing it along the way.
Data Pipeline 328
328 
Let's personalize your content