Data Science Blog

JULY 23, 2023

Databricks is an ideal tool for realizing a Data Mesh due to its unified data platform, scalability, and performance. It enables data collaboration and sharing, supports Delta Lake for data quality, and ensures robust data governance and security.

Data Science 246

Data Science Azure Power BI Business Intelligence 246

Alation

FEBRUARY 14, 2023

And a data breach poses more than just a PR risk — by violating regulations like GDPR , a data leak can impact your bottom line, too. This is where successful data governance programs can act as a savior to many organizations. This begs the question: What makes data governance successful? Where do you start?

Data Governance 52

Data Governance Data Quality Data Analyst Data Engineer 52

Dataversity

AUGUST 14, 2024

The secret is to combine smart analytics with a strong data engineering strategy. As we continue into 2024, data engineering trends and insights will continue to be critical for businesses hoping to prosper in this cutthroat industry.

Data Engineering 59

Data Engineering Data Engineering Data Engineer Data Engineering 59

Alation

FEBRUARY 17, 2023

This past week, I had the pleasure of hosting Data Governance for Dummies author Jonathan Reichental for a fireside chat , along with Denise Swanson , Data Governance lead at Alation. Can you have proper data management without establishing a formal data governance program?

Data Governance 52

Data Governance Data Quality Data Analyst Data Pipeline 52

ODSC - Open Data Science

JANUARY 18, 2024

Data engineering is a rapidly growing field, and there is a high demand for skilled data engineers. If you are a data scientist, you may be wondering if you can transition into data engineering. In this blog post, we will discuss how you can become a data engineer if you are a data scientist.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

IBM Journey to AI blog

FEBRUARY 26, 2024

The financial services industry has been in the process of modernizing its data governance for more than a decade. But as we inch closer to global economic downturn, the need for top-notch governance has become increasingly urgent. Trust and data governance Data governance isn’t new, especially in the financial world.

Data Pipeline 112

Data Pipeline Data Engineering Data Engineer Data Engineering 112

ODSC - Open Data Science

MARCH 22, 2024

Be sure to check out his talk, “ Building Data Contracts with Open Source Tools ,” there! Data engineering is a critical function in all industries. However, data engineering grows exponentially as the company grows, acquires, or merges with others. Why is that?

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

Data Science Blog

NOVEMBER 15, 2023

The creation of this data model requires the data connection to the source system (e.g. SAP ERP), the extraction of the data and, above all, the data modeling for the event log. They offer consistency and standardization across data structures, improving data accuracy and integrity.

Data Modeling 245

Data Modeling Data Models Business Intelligence Business Intelligence 245

Pickl AI

JULY 25, 2023

Unfolding the difference between data engineer, data scientist, and data analyst. Data engineers are essential professionals responsible for designing, constructing, and maintaining an organization’s data infrastructure. Read more to know.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

Alation

FEBRUARY 18, 2021

Alation increases search relevancy with data domains, adds new data governance capabilities, and speeds up time-to-insight with an Open Connector Framework SDK. Categorize data by domain. As a data consumer, sometimes you just want data in a single category. Data quality is essential to data governance.

Data Governance 98

Data Governance Data Quality Data Engineering Data Engineer 98

Dataversity

AUGUST 10, 2023

To get to the bottom of these questions and more, we conducted a survey of 100 survey respondents, at least 63 […] The post Which Data Quality Issues Are Plaguing Data Engineers Today? appeared first on DATAVERSITY.

Data Quality 52

Data Quality Data Engineering Data Engineering Data Engineer 52

Alation

APRIL 6, 2023

Data governance is traditionally applied to structured data assets that are most often found in databases and information systems. This blog focuses on governing spreadsheets that contain data, information, and metadata, and must themselves be governed.

Data Governance 52

Data Governance Database Data Lakes Data Warehouse 52

Dataversity

AUGUST 5, 2022

Data-driven culture cannot exist without the democratization of the data. Data democratization certainly does not mean unrestricted access to all […]. The post How a Modern Data Engineering Stack Can Help Create a Data-Driven Culture appeared first on DATAVERSITY.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

IBM Journey to AI blog

APRIL 20, 2023

This trust depends on an understanding of the data that inform risk models: where does it come from, where is it being used, and what are the ripple effects of a change? Moreover, banks must stay in compliance with industry regulations like BCBS 239, which focus on improving banks’ risk data aggregation and risk reporting capabilities.

Database 86

Database Data Engineering Data Engineer Data Engineering 86

AWS Machine Learning Blog

FEBRUARY 25, 2025

This blog post is co-written with Gene Arnold from Alation. To build a generative AI -based conversational application integrated with relevant data sources, an enterprise needs to invest time, money, and people. First, you would need build connectors to the data sources.

AWS 105

AWS AI AI Natural Language Processing 105

IBM Journey to AI blog

APRIL 25, 2023

In a prior blog , we pointed out that warehouses, known for high-performance data processing for business intelligence, can quickly become expensive for new data and evolving workloads. To do so, Presto and Spark need to readily work with existing and modern data warehouse infrastructures.

Data Warehouse 58

Data Warehouse Data Engineering Data Engineer Data Engineering 58

AWS Machine Learning Blog

OCTOBER 20, 2023

Data and governance foundations – This function uses a data mesh architecture for setting up and operating the data lake, central feature store, and data governance foundations to enable fine-grained data access. About the authors Ram Vittal is a Principal ML Solutions Architect at AWS.

ML 134

ML ML AWS Data Lakes 134

IBM Journey to AI blog

JUNE 8, 2023

Data is the lifeblood of successful organizations. Beyond the traditional data roles—data engineers, analysts, architects—decision-makers across an organization need flexible, self-service access to data-driven insights accelerated by artificial intelligence (AI).

Business Intelligence 78

Business Intelligence Business Intelligence AI AI 78

AWS Machine Learning Blog

APRIL 26, 2023

Data engineering – Identifies the data sources, sets up data ingestion and pipelines, and prepares data using Data Wrangler. Data science – The heart of ML EBA and focuses on feature engineering, model training, hyperparameter tuning, and model validation. Monitoring setup (model, data drift).

ML 116

ML ML AWS Machine Learning 116

Alation

AUGUST 30, 2022

we are introducing Alation Anywhere, extending data intelligence directly to the tools in your modern data stack, starting with Tableau. We continue to make deep investments in governance, including new capabilities in the Stewardship Workbench, a core part of the Data Governance App. Data governance at scale.

Data Governance 52

Data Governance Tableau Data Quality Data Analyst 52

Dataversity

SEPTEMBER 28, 2022

Data engineering is a fascinating and fulfilling career – you are at the helm of every business operation that requires data, and as long as users generate data, businesses will always need data engineers. The journey to becoming a successful data engineer […].

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

phData

FEBRUARY 16, 2023

An ACE is a dedicated team or unit within an organization that is responsible for managing and optimizing the use of data and analytics. They will be responsible for leading data-driven projects and initiatives–and for communicating the insights and recommendations derived from data analysis to stakeholders.

Analytics 52

Analytics Analytics Data Analyst Data Governance 52

Alation

DECEMBER 23, 2021

The elf teams used data engineering to improve gift matching and deployed big data to scale the naughty and nice list long ago , before either approach was even considered within our warmer climes. The best data was discovered, experts were identified, and conversations were starting. Make data governance an asset.

Data Governance 59

Data Governance Data Pipeline Tableau Big Data 59

AWS Machine Learning Blog

FEBRUARY 13, 2024

Let’s demystify this using the following personas and a real-world analogy: Data and ML engineers (owners and producers) – They lay the groundwork by feeding data into the feature store Data scientists (consumers) – They extract and utilize this data to craft their models Data engineers serve as architects sketching the initial blueprint.

AWS 129

AWS ML ML Machine Learning 129

IBM Journey to AI blog

AUGUST 12, 2024

Consequently, AIOps is designed to harness data and insight generation capabilities to help organizations manage increasingly complex IT stacks. Explore IBM Turbonomic The post AIOps vs. MLOps: Harnessing big data for “smarter” ITOPs appeared first on IBM Blog.

Big Data 106

Big Data Big Data ML ML 106

phData

MAY 28, 2024

With Cortex, business analysts, data engineers, and developers can easily incorporate Predictive and Generative AI into their workflows using simple SQL commands and intuitive interfaces. As an example, check out our blog post on implementing a data engineering pipeline for Sentiment Analysis with dbt.

Machine Learning 52

Machine Learning Machine Learning Data Engineering Data Engineer 52

IBM Journey to AI blog

DECEMBER 16, 2022

And because data assets within the catalog have quality scores and social recommendations, Alex has greater trust and confidence in the data she’s using for her decision-making recommendations. This is especially helpful when handling massive amounts of big data. Protected and compliant data.

Data Quality 52

Data Quality Data Governance Data Scientist Data Wrangling 52

Alation

AUGUST 16, 2022

This week, IDC released its second IDC MarketScape for Data Catalogs report, and we’re excited to share that Alation was recognized as a leader for the second consecutive time. These include data analysts, stewards, business users , and data engineers. Leader in Forrester Wave: Data Governance Solutions.

Data Quality 52

Data Quality Data Governance Cloud Data Data Engineering 52

Alation

JANUARY 13, 2022

Data mesh forgoes technology edicts and instead argues for “decentralized data ownership” and the need to treat “data as a product”. Gartner on Data Fabric. Moreover, data catalogs play a central role in both data fabric and data mesh. Let’s turn our attention now to data mesh.

Data Lakes 59

Data Lakes Data Governance Data Quality Data Warehouse 59

Alation

APRIL 18, 2023

For any data user in an enterprise today, data profiling is a key tool for resolving data quality issues and building new data solutions. In this blog, we’ll cover the definition of data profiling, top use cases, and share important techniques and best practices for data profiling today.

Data Profiling 52

Data Profiling Data Quality Data Governance Data Pipeline 52

Pickl AI

OCTOBER 11, 2023

Data Observability and Data Quality are two key aspects of data management. The focus of this blog is going to be on Data Observability tools and their key framework. The growing landscape of technology has motivated organizations to adopt newer ways to harness the power of data.

Data Observability 52

Data Observability Data Quality Data Pipeline Data Governance 52

phData

FEBRUARY 14, 2023

Insurance companies often face challenges with data silos and inconsistencies among their legacy systems. To address these issues, they need a centralized and integrated data platform that serves as a single source of truth, preferably with strong data governance capabilities.

Data Governance 52

Data Governance Data Silos Predictive Analytics Data Scientist 52

IBM Journey to AI blog

JANUARY 5, 2023

The first generation of data architectures represented by enterprise data warehouse and business intelligence platforms were characterized by thousands of ETL jobs, tables, and reports that only a small group of specialized data engineers understood, resulting in an under-realized positive impact on the business.

Data Quality 52

Data Quality Data Lakes Data Warehouse Big Data 52

Alation

DECEMBER 13, 2022

Today a modern catalog hosts a wide range of users (like business leaders, data scientists and engineers) and supports an even wider set of use cases (like data governance , self-service , and cloud migration ). So feckless buyers may resort to buying separate data catalogs for use cases like…. Data governance.

DataOps 52

DataOps Data Governance Data Silos Data Scientist 52

phData

NOVEMBER 9, 2023

This blog was originally written by Keith Smith and updated for 2024 by Justin Delisi. Snowflake’s Data Cloud has emerged as a leader in cloud data warehousing. As they grow in both their complexity and data production/consumption, a data governance strategy needs to be designed as part of your information architecture.

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98

phData

MARCH 22, 2023

Snowpark , an innovative technology from the Snowflake Data Cloud , promises to meet this demand by allowing data scientists to develop complex data transformation logic using familiar programming languages such as Java, Scala, and Python. Checkout these blogs and reach out to our Data Science and ML team today!

ML 52

ML ML Python Machine Learning 52

Alation

OCTOBER 12, 2021

One may define enterprise data analytics as the ability to find, understand, analyze, and trust data to drive strategy and decision-making. Enterprise data analytics integrates data, business, and analytics disciplines, including: Data management. Data engineering. Subscribe to Alation's Blog.

Analytics 52

Analytics Analytics Data Governance DataOps 52