Hacker News

JULY 4, 2023

The second section will delve more deeply into the various approaches that can be used to handle recursive schema definitions. The following Go Arrow schema definition provides an example of such a schema, instrumented with a collection of annotations. The depth of this definition cannot be predetermined.

Data Models 123

Data Models Data Modeling Data Observability 123

Alation

JANUARY 20, 2022

This starts by determining the critical data elements for the enterprise. These items become in scope for the data quality program. Step 2: Data Definitions. Here each critical data element is described so there are no inconsistencies between users or data stakeholders. Step 4: Data Sources.

Data Quality 94

Data Quality Data Governance Data Profiling Clean Data 94

Precisely

NOVEMBER 13, 2024

The Importance of a System of Record MDM’s role in your data landscape is closely tied to the concept of a system of record: a centralized repository where critical business data is stored and managed. It can also link with most commonly-used systems like your CRM, ERP, and marketing platforms.

Data Silos 64

Data Silos Data Observability Data Governance Data Quality 64

Alation

MAY 24, 2022

This has created many different data quality tools and offerings in the market today and we’re thrilled to see the innovation. People will need high-quality data to trust information and make decisions. A business glossary is critical to aligning an organization around the definition of business terms.

Data Quality 98

Data Quality Data Governance ETL Data Observability 98

The MLOps Blog

AUGUST 3, 2023

And then, we’re trying to boot out features of the platform and the open-source to be able to take Hamilton data flow definitions and help you auto-generate the Airflow tasks. To a junior data scientist, it doesn’t matter if you’re using Airflow, Prefect , Dexter. I term it as a feature definition store.

ML 52

ML ML Data Scientist Machine Learning 52

Pickl AI

JULY 8, 2024

These pipelines automate collecting, transforming, and delivering data, crucial for informed decision-making and operational efficiency across industries. Robust validation and monitoring frameworks enhance pipeline reliability and trustworthiness, safeguarding against data-driven decision-making risks.

Data Pipeline 52

Data Pipeline Data Quality Database Apache Kafka 52

Pickl AI

JUNE 7, 2024

It is a process for moving and managing data from various sources to a central data warehouse. This process ensures that data is accurate, consistent, and usable for analysis and reporting. Definition and Explanation of the ETL Process ETL is a data integration method that combines data from multiple sources.

ETL 40

ETL Data Quality Data Pipeline Data Warehouse 40