ODSC - Open Data Science

FEBRUARY 28, 2023

Making Data Observable Bigeye The quality of the data powering your machine learning algorithms should not be a mystery. Bigeye’s data observability platform helps data science teams “measure, improve, and communicate data quality at any scale.”

Data Science 96

Data Science Machine Learning Machine Learning AI 96

Precisely

JULY 12, 2024

However, simply having high-quality data does not, of itself, ensure that an organization will find it useful. That is where data integrity comes into play.

Data Quality 105

Data Quality Data Governance Database Analytics 105

IBM Journey to AI blog

OCTOBER 22, 2024

IBM’s data integration portfolio includes tools such as IBM DataStage for ETL/ELT processing, IBM StreamSets for real-time streaming data pipelines, and IBM Data Replication for low-latency, near real-time data synchronization.

Data Silos 102

Data Silos Data Pipeline DataOps Business Intelligence 102

IBM Journey to AI blog

JULY 3, 2024

Modernizing your data infrastructure to hybrid cloud for applications, analytics and gen AI Adopting multicloud and hybrid strategies is becoming mandatory, requiring databases that support flexible deployments across the hybrid cloud. This ensures you have a data foundation that grows with your data needs, wherever your data resides.

Data Quality 45

Data Quality AI AI Database 45

Dataversity

JUNE 19, 2023

In part one of this article, we discussed how data testing can specifically test a data object (e.g., table, column, metadata) at one particular point in the data pipeline.

Data Pipeline 52

Data Pipeline Database Data Models Data Modeling 52

Alation

MAY 24, 2022

This has created many different data quality tools and offerings in the market today and we’re thrilled to see the innovation. People will need high-quality data to trust information and make decisions. Lineage helps everyone to see relationships between data, from source to target, and diagnose critical problems.

Data Quality 98

Data Quality Data Governance ETL Data Observability 98

The MLOps Blog

JUNE 27, 2023

It integrates with Git and provides a Git-like interface for data versioning, allowing you to track changes, manage branches, and collaborate with data teams effectively. Dolt Dolt is an open-source relational database system built on Git. It could help you detect and prevent data pipeline failures, data drift, and anomalies.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

Precisely

FEBRUARY 2, 2023

Pipelines must have robust data integration capabilities that integrate data from multiple data silos, including the extensive list of applications used throughout the organization, databases and even mainframes. Changes to one database must also be reflected in any other database in real time.

Data Pipeline 52

Data Pipeline Data Silos Database Data Observability 52

phData

AUGUST 10, 2023

The implementation of a data vault architecture requires the integration of multiple technologies to effectively support the design principles and meet the organization’s requirements. Let’s explore one of the methods for implementing near real-time (NRT) data vaults using Snowflake Continuous Data Pipelines.

SQL 52

SQL Data Observability Data Quality Data Pipeline 52

Precisely

AUGUST 26, 2024

Here’s what our panelists had to say about how critical this collaboration is for customers: “Last year we launched two solutions that integrate with Precisely CDC database replication technology – that’s AWS Mainframe Modernization Data Replication for both mainframe and IBM I. What does my backlog look like?

AWS 64

AWS Data Observability Database Analytics 64

Pickl AI

JULY 8, 2024

Organisations leverage diverse methods to gather data, including: Direct Data Capture: Real-time collection from sensors, devices, or web services. Database Extraction: Retrieval from structured databases using query languages like SQL. Data Warehouses : Centralised repositories optimised for analytics and reporting.

Data Pipeline 52

Data Pipeline Data Quality Database Apache Kafka 52

Precisely

MARCH 14, 2023

Simply design data pipelines, point them to the cloud environment, and execute. Enjoy a much more sophisticated computing environment than what’s available in a standard relational database – with data architectures based on tools like Databricks and Snowflake. What does all this mean for your business? Bigger, better results.

Data Quality 52

Data Quality Cloud Data Data Pipeline Data Observability 52

Precisely

JULY 18, 2024

Here are four aspects of a data management approach that you should consider to increase the success of an architecture: Break down data silos by automating the integration of essential data – from legacy mainframes and midrange systems, databases, apps, and more – into your logical data warehouse or data lake.

Data Lakes 52

Data Lakes Data Warehouse Data Governance Machine Learning 52

IBM Journey to AI blog

DECEMBER 16, 2022

Because Alex can use a data catalog to search all data assets across the company, she has access to the most relevant and up-to-date information. She can search structured or unstructured data, visualizations and dashboards, machine learning models, and database connections.

Data Quality 52

Data Quality Data Governance Data Wrangling Data Scientist 52

Precisely

OCTOBER 5, 2023

When we think about the big picture of data integrity – that’s data with maximum accuracy, consistency, and context – it becomes abundantly clear why data enrichment is one of its six key pillars (along with data integration, data observability, data quality, data governance, and location intelligence).

Data Quality 69

Data Quality Analytics Analytics Database 69

Pickl AI

JUNE 7, 2024

It involves retrieving data from various sources, such as databases, spreadsheets, or even cloud storage. The goal is to collect relevant data without affecting the source system’s performance. Compatibility with Existing Systems and Data Sources Compatibility is critical. How to drop a database in SQL server?

ETL 40

ETL Data Quality Data Pipeline Data Warehouse 40

phData

FEBRUARY 22, 2023

For example, at phData, we recommend breaking your data into three main areas: Raw, Staging, and Transformed (these layers may be called different things, but ultimately exist in some fashion). You might want three different databases or schemas depending on your use case. Having a test that finds these duplicates is important.

SQL 52

SQL Data Quality Database ETL 52

The MLOps Blog

AUGUST 3, 2023

Like they didn’t have to think about, you know, data observability, but look, if you provided those data, we captured things about it. I mean pretty basic, you could say S3, so we store them in a structured manner on S3, but you know, we paired that with a database which had the actual metadata and pointer.

ML 52

ML ML Data Scientist Machine Learning 52

Precisely

FEBRUARY 9, 2024

For instance, a data breach or violation of privacy standards can lead to liability, expensive fines, and a slew of negative publicity that’s a hit to brand reputation and trustworthiness. If inconsistencies and inaccuracies in the customer database can be fixed, the organization’s data analytics initiatives can presumably proceed.

Data Quality 69

Data Quality DataOps Data Silos Artificial Intelligence 69

Precisely

OCTOBER 12, 2023

This includes understanding the impact of change within one data element on the various other data elements and compliance requirements throughout the organization. Creating data observability routines to inform key users of any changes or exceptions that crop up within the data, enabling a more proactive approach to compliance.

Data Quality 64

Data Quality Data Governance ML ML 64

Pickl AI

MARCH 19, 2025

Without data engineering , companies would struggle to analyse information and make informed decisions. What Does a Data Engineer Do? A data engineer creates and manages the pipelines that transfer data from different sources to databases or cloud storage. How is Data Engineering Different from Data Science?

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52