Dataconomy

FEBRUARY 23, 2023

Data engineers play a crucial role in managing and processing big data. They are responsible for designing, building, and maintaining the infrastructure and tools needed to manage and process large volumes of data effectively. What is data engineering?

Big Data 91

Big Data Big Data Data Engineering Data Engineering 91

The MLOps Blog

JUNE 27, 2023

Alignment to other tools in the organization’s tech stack Consider how well the MLOps tool integrates with your existing tools and workflows, such as data sources, data engineering platforms, code repositories, CI/CD pipelines, monitoring systems, etc. Dolt Dolt is an open-source relational database system built on Git.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

IBM Journey to AI blog

JANUARY 5, 2023

The first generation of data architectures represented by enterprise data warehouse and business intelligence platforms were characterized by thousands of ETL jobs, tables, and reports that only a small group of specialized data engineers understood, resulting in an under-realized positive impact on the business.

Data Quality 52

Data Quality Data Lakes Data Warehouse Big Data 52

Dataconomy

JULY 28, 2023

It is known for its ability to connect to almost any database and offers features like reusable data flows, automating repetitive work. Trifacta Trifacta is a data profiling and wrangling tool that stands out with its rich features and ease of use.

Power BI 91

Power BI Data Preparation Exploratory Data Analysis Machine Learning 91

Alation

MAY 24, 2022

Prime examples of this in the data catalog include: Trust Flags — Allow the data community to endorse, warn, and deprecate data to signal whether data can or can’t be used. Data Profiling — Statistics such as min, max, mean, and null can be applied to certain columns to understand its shape.

Data Quality 98

Data Quality Data Governance ETL Data Observability 98

Iguazio

FEBRUARY 17, 2024

Global Financial Data (GDF) An extensive database of current and historical financial data, providing updated information alongside data from hundreds of years ago. The database covers topics like market indicators, exchange rates, commodities, incomes and more. Nasdaq Data Link is considered to be very reliable.

Machine Learning 40

Machine Learning Machine Learning ML ML 40

The MLOps Blog

MAY 17, 2023

This article explores the importance of ETL pipelines in machine learning, a hands-on example of building ETL pipelines with a popular tool, and suggests the best ways for data engineers to enhance and sustain their pipelines. We also need data profiling i.e. data discovery, to understand if the data is appropriate for ETL.

ETL 59

ETL Data Pipeline ML ML 59