IBM Data Science in Practice

NOVEMBER 28, 2022

IBM Multicloud Data Integration helps organizations connect data from disparate sources, build data pipelines, remediate data issues, enrich data, and deliver integrated data to multicloud platforms where it can easily accessed by data consumers or built into a data product.

Data Pipeline 130

Data Pipeline Data Quality Data Preparation Data Governance 130

Smart Data Collective

JULY 15, 2021

You can think of a data catalog as an enhanced Access database or library card catalog system. It helps you locate and discover data that fit your search criteria. With data catalogs, you won’t have to waste time looking for information you think you have. What Does a Data Catalog Do?

Data Quality 133

Data Quality Database Data Pipeline Data Observability 133

The MLOps Blog

JUNE 27, 2023

Data quality control: Robust dataset labeling and annotation tools incorporate quality control mechanisms such as inter-annotator agreement analysis, review workflows, and data validation checks to ensure the accuracy and reliability of annotations. Dolt Dolt is an open-source relational database system built on Git.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

ODSC - Open Data Science

FEBRUARY 28, 2023

Making Data Observable Bigeye The quality of the data powering your machine learning algorithms should not be a mystery. Bigeye’s data observability platform helps data science teams “measure, improve, and communicate data quality at any scale.”

Data Science 96

Data Science Machine Learning Machine Learning AI 96

IBM Journey to AI blog

OCTOBER 22, 2024

The ability to effectively deploy AI into production rests upon the strength of an organization’s data strategy because AI is only as strong as the data that underpins it. IBM Databand underpins this set of capabilities with data observability for pipeline monitoring and issue remediation.

Data Silos 102

Data Silos Data Pipeline DataOps Business Intelligence 102

IBM Journey to AI blog

JULY 3, 2024

Modernizing your data infrastructure to hybrid cloud for applications, analytics and gen AI Adopting multicloud and hybrid strategies is becoming mandatory, requiring databases that support flexible deployments across the hybrid cloud. This ensures you have a data foundation that grows with your data needs, wherever your data resides.

AI 45

AI Data Quality AI Database 45

IBM Journey to AI blog

DECEMBER 16, 2022

An enterprise data catalog does all that a library inventory system does – namely streamlining data discovery and access across data sources – and a lot more. For example, data catalogs have evolved to deliver governance capabilities like managing data quality and data privacy and compliance.

Data Quality 52

Data Quality Data Governance Data Scientist Data Wrangling 52

phData

AUGUST 10, 2023

The implementation of a data vault architecture requires the integration of multiple technologies to effectively support the design principles and meet the organization’s requirements. Implement business rules and validations: Data Vault models often involve enforcing business rules and performing data quality checks.

SQL 52

SQL Data Observability Data Quality Data Pipeline 52

Pickl AI

JULY 8, 2024

Organisations leverage diverse methods to gather data, including: Direct Data Capture: Real-time collection from sensors, devices, or web services. Database Extraction: Retrieval from structured databases using query languages like SQL. Data Warehouses : Centralised repositories optimised for analytics and reporting.

Data Pipeline 52

Data Pipeline Data Quality Database Apache Kafka 52

Pickl AI

JUNE 7, 2024

Top contenders like Apache Airflow and AWS Glue offer unique features, empowering businesses with efficient workflows, high data quality, and informed decision-making capabilities. Introduction In today’s business landscape, data integration is vital. Transformation Transformation is the second step in the ETL process.

ETL 40

ETL Data Pipeline Data Quality Data Warehouse 40

Dataversity

JUNE 19, 2023

In part one of this article, we discussed how data testing can specifically test a data object (e.g., table, column, metadata) at one particular point in the data pipeline.

Data Pipeline 52

Data Pipeline Database Data Modeling Data Models 52

Precisely

FEBRUARY 2, 2023

Pipelines must have robust data integration capabilities that integrate data from multiple data silos, including the extensive list of applications used throughout the organization, databases and even mainframes. Changes to one database must also be reflected in any other database in real time.

Data Pipeline 52

Data Pipeline Data Silos Database Data Observability 52

Precisely

OCTOBER 5, 2023

When we think about the big picture of data integrity – that’s data with maximum accuracy, consistency, and context – it becomes abundantly clear why data enrichment is one of its six key pillars (along with data integration, data observability, data quality, data governance, and location intelligence).

Data Quality 69

Data Quality Analytics Analytics Database 69

Precisely

JULY 18, 2024

Here are four aspects of a data management approach that you should consider to increase the success of an architecture: Break down data silos by automating the integration of essential data – from legacy mainframes and midrange systems, databases, apps, and more – into your logical data warehouse or data lake.

Data Lakes 52

Data Lakes Data Warehouse Data Governance Machine Learning 52

phData

FEBRUARY 22, 2023

Data Quality Good testing is an essential part of ensuring the integrity and reliability of data. Without testing, it is difficult to know whether the data is accurate, complete, and free of errors. Below, we will walk through some baseline tests every team could and should run to ensure data quality.

SQL 52

SQL Data Quality Database ETL 52

The MLOps Blog

AUGUST 3, 2023

One of the features that Hamilton has is that it has a really lightweight data quality runtime check. If you’re using tabular data, there’s Pandera. Like they didn’t have to think about, you know, data observability, but look, if you provided those data, we captured things about it.

ML 52

ML ML Data Scientist Machine Learning 52

Precisely

FEBRUARY 9, 2024

For instance, a data breach or violation of privacy standards can lead to liability, expensive fines, and a slew of negative publicity that’s a hit to brand reputation and trustworthiness. If inconsistencies and inaccuracies in the customer database can be fixed, the organization’s data analytics initiatives can presumably proceed.

Data Quality 69

Data Quality DataOps Data Silos Artificial Intelligence 69

Precisely

OCTOBER 12, 2023

Monitoring and improving business quality rules and technical quality rules to define what “good” looks like. Creating data observability routines to inform key users of any changes or exceptions that crop up within the data, enabling a more proactive approach to compliance. Privacy requirements. Intelligence.

Data Quality 64

Data Quality Data Governance ML ML 64

Pickl AI

MARCH 19, 2025

Without data engineering , companies would struggle to analyse information and make informed decisions. What Does a Data Engineer Do? A data engineer creates and manages the pipelines that transfer data from different sources to databases or cloud storage. How is Data Engineering Different from Data Science?

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52