AWS Machine Learning Blog

APRIL 3, 2025

About the Authors Emrah Kaya is Data Engineering Manager at Omron Europe and Platform Lead for ODAP Project. With his extensive background on Cloud & Data Architecture, Emrah leads key OMRONs technological advancement initiatives, including artificial intelligence, machine learning, or data science.

AWS 79

AWS Data Governance Data Silos SQL 79

Data Science Blog

JULY 23, 2023

Using data sharing (in Databricks: Delta Sharing) data products or single datasets can be shared through applications and owners. The post Data Mesh Architecture on Cloud for BI, Data Science and Process Mining appeared first on Data Science Blog.

Data Science 246

Data Science Azure Power BI Business Intelligence 246

phData

MARCH 8, 2023

Fivetran, a cloud-based automated data integration platform, has emerged as a leading choice among businesses looking for an easy and cost-effective way to unify their data from various sources. It allows organizations to easily connect their disparate data sources without having to manage any infrastructure.

Data Warehouse 96

Data Warehouse Data Engineering Data Engineer Data Engineering 96

AWS Machine Learning Blog

NOVEMBER 15, 2024

She has extensive experience in data and analytics, application development, infrastructure engineering, and DevSecOps. Joel Elscott is a Senior Data Engineer on the Principal AI Enablement team. Joel lives in Des Moines, Iowa, with his wife and five children, and is also a group fitness instructor.

AWS 113

AWS AI AI Machine Learning 113

phData

NOVEMBER 9, 2023

This blog was originally written by Keith Smith and updated for 2024 by Justin Delisi. Snowflake’s Data Cloud has emerged as a leader in cloud data warehousing. In 2022, the term data mesh has started to become increasingly popular among Snowflake and the broader industry. What is a Cloud Data Warehouse?

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98

phData

NOVEMBER 8, 2024

Open Table Format (OTF) architecture now provides a solution for efficient data storage, management, and processing while ensuring compatibility across different platforms. In this blog, we will discuss: What is the Open Table format (OTF)? Why should we use it? A Brief History of OTF A comparative study between the major OTFs.

Data Lakes 52

Data Lakes Data Warehouse Database Azure 52

Data Science Dojo

JULY 3, 2024

In this blog, we will explore the arena of data science bootcamps and lay down a guide for you to choose the best data science bootcamp. What do Data Science Bootcamps Offer? Big Data Technologies : Handling and processing large datasets using tools like Hadoop, Spark, and cloud platforms such as AWS and Google Cloud.

Data Science 150

Data Science Machine Learning Machine Learning Data Visualization 150

Dataversity

SEPTEMBER 28, 2022

Data engineering is a fascinating and fulfilling career – you are at the helm of every business operation that requires data, and as long as users generate data, businesses will always need data engineers. The journey to becoming a successful data engineer […].

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

phData

JULY 17, 2023

Over the past few decades, the corporate data landscape has changed significantly. The shift from on-premise databases and spreadsheets to the modern era of cloud data warehouses and AI/ LLMs has transformed what businesses can do with data. What is the Modern Data Stack? Data modeling, data cleanup, etc.

Data Warehouse 52

Data Warehouse Data Pipeline Cloud Data ETL 52

phData

AUGUST 6, 2024

Organizations must ensure their data pipelines are well designed and implemented to achieve this, especially as their engagement with cloud data platforms such as the Snowflake Data Cloud grows. For customers in Snowflake, Snowpark is a powerful tool for building these effective and scalable data pipelines.

Data Pipeline 52

Data Pipeline Python Data Engineer Data Engineering 52

IBM Journey to AI blog

MAY 15, 2024

There are several styles of data integration. Data engineers build data pipelines, which are called data integration tasks or jobs, as incremental steps to perform data operations and orchestrate these data pipelines in an overall workflow.

Data Pipeline 85

Data Pipeline ETL SQL Database 85

phData

MARCH 27, 2024

While learning Snowflake presents its challenges, the benefits for any data professional are immense. In this blog, I’ll guide you towards success in your Snowflake learning journey. Snowflake’s SnowPro Advanced Certifications assess advanced Snowflake knowledge and skills relating to five data science roles.

Data Analyst 52

Data Analyst Data Engineer Data Engineering Data Engineering 52

Data Science Blog

JUNE 27, 2023

Its focus on unique, ongoing events allows for effective and responsive data processing. The post Big Data – Lambda or Kappa Architecture? appeared first on Data Science Blog.

Big Data 130

Big Data Big Data Apache Kafka Database 130

phData

JULY 24, 2024

In this blog, we’ll explore some marketing questions you may ask yourself every day that can be answered by the experts at phData through data. Marketing Questions phData Can Answer with Data How Do We Create Personalized Marketing Campaigns? Are we getting a return on our investment in the campaigns we deploy? Why phData?

Machine Learning 52

Machine Learning Machine Learning Data Engineering Data Engineering 52

Alation

OCTOBER 18, 2022

Data analysts and engineers use dbt to transform, test, and document data in the cloud data warehouse. Making this data visible in the data catalog will let data teams share their work, support re-use, and empower everyone to better understand and trust data. Subscribe to Alation's Blog.

Data Analyst 52

Data Analyst Data Engineer Data Engineering Data Engineering 52

phData

JUNE 26, 2024

That’s why companies have turned to the experts at phData to be able to answer these questions and more through the use of data-driven facts and predictions. In this blog, we’ll discuss some of the questions you and many other retail and CPG businesses ask daily and how phData can answer them using data.

Machine Learning 52

Machine Learning Machine Learning Data Engineer Data Engineering 52

Alation

APRIL 4, 2023

This expanded connector to Databricks Unity Catalog does just that, delivering to joint customers a comprehensive view of all cloud data. New Connectivity for dbt Modern data engineers confront complex, challenging data environments and need to empower data users for self-service. Now with this new 2023.1

DataOps 52

DataOps Data Engineer Data Engineering Data Engineering 52

phData

FEBRUARY 15, 2024

With the advent of cloud data warehouses and the ability to (seemingly) infinitely scale analytics on an organization’s data, centralizing and using that data to discover what drives customer engagement has become a top priority for executives across all industries and verticals. What Are the Challenges With Fan 360?

Data Warehouse 52

Data Warehouse Cloud Data Data Silos Analytics 52

phData

SEPTEMBER 27, 2024

Modern business operations rely heavily on data engineering and transformation processes to turn raw data into valuable insights. Matillion, a robust ELT (Extract, Load, Transform) platform, simplifies data integration and transformation complexities with a no-code or high-code experience. What is a Matillion Job?

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

AWS Machine Learning Blog

OCTOBER 18, 2023

He is passionate about helping customers to build scalable and modern data analytics solutions to gain insights from the data. Tayo Olajide is a seasoned Cloud Data Engineering generalist with over a decade of experience in architecting and implementing data solutions in cloud environments.

AWS 124

AWS ML ML Machine Learning 124

Alation

AUGUST 16, 2022

This week, IDC released its second IDC MarketScape for Data Catalogs report, and we’re excited to share that Alation was recognized as a leader for the second consecutive time. These include data analysts, stewards, business users , and data engineers. Alation launched Alation Cloud Service (ACS) in April, 2021.

Data Quality 52

Data Quality Data Governance Cloud Data Data Engineer 52

phData

FEBRUARY 13, 2024

Fivetran has announced a new orchestration integration with dbt Cloud that allows you to seamlessly connect your dbt transformation pipelines with your Fivetran ingestion pipelines. In this blog, we’ll explore why this news is such a big deal. What is Fivetran and dbt Cloud?

Data Pipeline 52

Data Pipeline Cloud Data Data Engineer Data Engineering 52

phData

SEPTEMBER 2, 2024

Best practices are a pivotal part of any software development, and data engineering is no exception. This ensures the data pipelines we create are robust, durable, and secure, providing the desired data to the organization effectively and consistently. What Are Matillion Jobs and Why Do They Matter?

ETL 52

ETL Data Warehouse SQL Database 52

Alation

SEPTEMBER 7, 2021

This is the last of the 4-part blog series. In the previous blog , we discussed how Alation provides a platform for data scientists and analysts to complete projects and analysis at speed. In this blog we will discuss how Alation helps minimize risk with active data governance. Subscribe to Alation's Blog.

Data Governance 59

Data Governance Data Scientist Data Quality Data Profiling 59

Alation

AUGUST 30, 2022

These range from data sources , including SaaS applications like Salesforce; ELT like Fivetran; cloud data warehouses like Snowflake; and data science and BI tools like Tableau. This expansive map of tools constitutes today’s modern data stack. Read Q&A blog with Raj. Subscribe to Alation's Blog.

Data Governance 52

Data Governance Tableau Data Quality Data Analyst 52

Alation

AUGUST 11, 2022

Data mesh says architectures should be decentralized because there are inherent problems with centralized architectures. For example, when we centralize, all the focus goes on the data engineers. But there are only so many data engineers available in the market today; there’s a big skills shortage.

Data Warehouse 52

Data Warehouse Data Engineer Data Engineering Data Engineering 52

IBM Journey to AI blog

JULY 17, 2023

In this blog, I will cover: What is watsonx.ai? sales conversation summaries, insurance coverage, meeting transcripts, contract information) Generate: Generate text content for a specific purpose, such as marketing campaigns, job descriptions, blogs or articles, and email drafting support. What capabilities are included in watsonx.ai?

AI 74

AI AI Machine Learning Machine Learning 74

phData

JULY 18, 2024

That’s why businesses like yours have turned to the experts at phData to answer questions like these through data-driven facts and insights. In this blog, we’ll discuss some of the top questions you and other manufacturers may ask yourself that can be answered by phData through data. Why phData?

Machine Learning 52

Machine Learning Machine Learning Natural Language Processing Data Engineer 52

phData

OCTOBER 17, 2023

In our previous blog , we discussed how Fivetran and dbt scale for any data volume and workload, both small and large. Now, you might be wondering what these tools can do for your data team and the efficiency of your organization as a whole. Can these tools help reduce the time our data engineers spend fixing things?

ETL 52

ETL Data Pipeline Data Engineer Data Engineering 52

phData

JANUARY 4, 2023

The Snowflake Data Cloud is a modern data warehouse that allows companies to take advantage of its cloud-based architecture to improve efficiencies while at the same time reducing costs. In this blog post, we will explore the reasons why many organizations are choosing to migrate from Netezza to Snowflake.

Data Warehouse 52

Data Warehouse SQL Database ETL 52

Alation

MARCH 29, 2021

Finding that data is often half the battle. This is why the ability to quickly search and discover data across the enterprise is the first step towards data-driven decision making. In this blog, we will discuss how data catalogs accelerate search & discovery. Subscribe to Alation's Blog.

Machine Learning 59

Machine Learning Machine Learning Data Lakes Hadoop 59

phData

JUNE 14, 2023

In recent years, data engineering teams working with the Snowflake Data Cloud platform have embraced the continuous integration/continuous delivery (CI/CD) software development process to develop data products and manage ETL/ELT workloads more efficiently. What Are the Benefits of CI/CD Pipeline For Snowflake?

Data Pipeline 52

Data Pipeline Database SQL Data Engineer 52

AWS Machine Learning Blog

DECEMBER 1, 2023

The following diagram shows the SageMaker Canvas data flow after adding visual transformations. You have completed the entire data processing and feature engineering step using visual workflows in SageMaker Canvas. Kasi Muthu is a senior partner solutions architect focusing on data and AI/ML at AWS based out of Houston, TX.

Machine Learning 115

Machine Learning Machine Learning Data Preparation ML 115

Alation

MARCH 1, 2022

Through Impact Analysis, users can determine if a problem occurred with data upstream, and locate the impacted data downstream. With robust data lineage, data engineers can find and fix issues fast and prevent them from recurring. Similarly, analysts gain a clear view of how data is created.

Data Warehouse 52

Data Warehouse Data Lakes Cloud Data Database 52

phData

MARCH 29, 2024

Founded in 2014 by three leading cloud engineers, phData focuses on solving real-world data engineering, operations, and advanced analytics problems with the best cloud platforms and products. Over the years, one of our primary focuses became Snowflake and migrating customers to this leading cloud data platform.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

Alation

MARCH 22, 2022

Accenture calls it the Intelligent Data Foundation (IDF), and it’s used by dozens of enterprises with very complex data landscapes and analytic requirements. Simply put, IDF standardizes data engineering processes. They can better understand data transformations, checks, and normalization.

DataOps 52

DataOps Data Pipeline Data Engineer Data Engineering 52

phData

SEPTEMBER 20, 2023

It’s all about breaking down data silos, empowering domain teams to take ownership of their data, and fostering a culture of data collaboration. And when it comes to implementing a data mesh, one platform has been making waves in the data world: the Snowflake Data Cloud.

Data Silos 52

Data Silos Database Data Quality Data Engineer 52