The MLOps Blog

JUNE 27, 2023

These practices are vital for maintaining data integrity, enabling collaboration, facilitating reproducibility, and supporting reliable and accurate machine learning model development and deployment. You can define expectations about data quality, track data drift, and monitor changes in data distributions over time.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

Alation

MAY 24, 2022

Prime examples of this in the data catalog include: Trust Flags — Allow the data community to endorse, warn, and deprecate data to signal whether data can or can’t be used. Data Profiling — Statistics such as min, max, mean, and null can be applied to certain columns to understand its shape.

Data Quality 98

Data Quality Data Governance ETL Data Observability 98

Precisely

JUNE 26, 2023

As organizations embark on data quality improvement initiatives, they need to develop a clear definition of the metrics and standards suited to their specific needs and objectives. Do the takeaways we’ve covered resonate with your own data integrity needs and challenges?

Data Quality 52

Data Quality Data Silos Analytics Analytics 52

Iguazio

FEBRUARY 17, 2024

World Bank Open Data The World Bank provides access to open global development data across 5,437 datasets. Open Finances” includes data about loans, financial reporting, procurement, projects and more. The data is intended to be easy to download, filter and slice and dice, so it can be easily consumed.

Machine Learning 40

Machine Learning Machine Learning ML ML 40

The MLOps Blog

MAY 17, 2023

ETL data pipeline architecture | Source: Author Data Discovery: Data can be sourced from various types of systems, such as databases, file systems, APIs, or streaming sources. We also need data profiling i.e. data discovery, to understand if the data is appropriate for ETL.

ETL 59

ETL Data Pipeline ML ML 59

The MLOps Blog

MARCH 15, 2023

This is a difficult decision at the onset, as the volume of data is a factor of time and keeps varying with time, but an initial estimate can be quickly gauged by analyzing this aspect by running a pilot. Also, the industry best practices suggest performing a quick data profiling to understand the data growth.

Data Pipeline 59

Data Pipeline ETL SQL Data Quality 59