Alation

SEPTEMBER 7, 2021

In addition, Alation provides a quick preview and sample of the data to help data scientists and analysts with greater data quality insights. Alation’s deep data profiling helps data scientists and analysts get important data profiling insights.

Data Governance 59

Data Governance Data Scientist Data Quality Data Profiling 59

Alation

MAY 24, 2022

Prime examples of this in the data catalog include: Trust Flags — Allow the data community to endorse, warn, and deprecate data to signal whether data can or can’t be used. Data Profiling — Statistics such as min, max, mean, and null can be applied to certain columns to understand its shape.

Data Quality 98

Data Quality Data Governance ETL Data Observability 98

DagsHub

AUGUST 23, 2024

A data quality standard might specify that when storing client information, we must always include email addresses and phone numbers as part of the contact details. If any of these is missing, the client data is considered incomplete. Data Profiling Data profiling involves analyzing and summarizing data (e.g.

Data Quality 52

Data Quality Data Governance Machine Learning Machine Learning 52

Pickl AI

OCTOBER 11, 2023

While they provide various data-related tools, they may also offer features related to Data Observability within their platform. Informatica might enable organizations to monitor data flows and ensure data quality as part of their data management processes.

Data Observability 52

Data Observability Data Quality Data Pipeline Data Governance 52

Alation

JANUARY 13, 2022

Data mesh forgoes technology edicts and instead argues for “decentralized data ownership” and the need to treat “data as a product”. Gartner on Data Fabric. Moreover, data catalogs play a central role in both data fabric and data mesh. Let’s turn our attention now to data mesh.

Data Lakes 59

Data Lakes Data Governance Data Quality Data Warehouse 59

Iguazio

FEBRUARY 17, 2024

Nasdaq Data Link is considered to be very reliable. They promise to only share datasets that have passed their curation and quality process and have gone through their own data engineering system. UK Data Service Datasets covering the UK’s economy, population and social research. Get the datasets here 8.

Machine Learning 40

Machine Learning Machine Learning ML ML 40

The MLOps Blog

MAY 17, 2023

This article explores the importance of ETL pipelines in machine learning, a hands-on example of building ETL pipelines with a popular tool, and suggests the best ways for data engineers to enhance and sustain their pipelines. We also need data profiling i.e. data discovery, to understand if the data is appropriate for ETL.

ETL 59

ETL Data Pipeline ML ML 59

Snorkel AI

MAY 12, 2023

One of these is a library that we open-sourced a little while back called the Data Profiler. The Data Profiler is a library that is really designed for understanding your data and understanding changes in the data and the schema over time. It is essentially a Python library. You can pip install it.

Machine Learning 52

Machine Learning Machine Learning ML ML 52

Snorkel AI

MAY 12, 2023

One of these is a library that we open-sourced a little while back called the Data Profiler. The Data Profiler is a library that is really designed for understanding your data and understanding changes in the data and the schema over time. It is essentially a Python library. You can pip install it.

Machine Learning 52

Machine Learning Machine Learning ML ML 52

phData

AUGUST 17, 2023

In the rapidly evolving landscape of data engineering, Snowflake Data Cloud has emerged as a leading cloud-based data warehousing solution, providing powerful capabilities for storing, processing, and analyzing vast amounts of data. Include tasks to ensure data integrity, accuracy, and consistency.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52