IBM Journey to AI blog

JULY 5, 2023

Data Lakes have been around for well over a decade now, supporting the analytic operations of some of the largest world corporations. Such data volumes are not easy to move, migrate or modernize. The challenges of a monolithic data lake architecture Data lakes are, at a high level, single repositories of data at scale.

Data Lakes 78

Data Lakes Data Warehouse Data Governance Analytics 78

Dataconomy

JUNE 1, 2023

Unified data storage : Fabric’s centralized data lake, Microsoft OneLake, eliminates data silos and provides a unified storage system, simplifying data access and retrieval. OneLake is designed to store a single copy of data in a unified location, leveraging the open-source Apache Parquet format.

Power BI 194

Power BI Data Lakes Azure Data Silos 194

AWS Machine Learning Blog

JANUARY 4, 2024

Generative AI has opened up a lot of potential in the field of AI. One such area that is evolving is using natural language processing (NLP) to unlock new opportunities for accessing data through intuitive SQL queries. The primary goal is to automatically generate SQL queries from natural language text.

SQL 140

SQL Database AI AI 140

AWS Machine Learning Blog

DECEMBER 16, 2024

We spoke with Dr. Swami Sivasubramanian, Vice President of Data and AI, shortly after AWS re:Invent 2024 to hear his impressionsand to get insights on how the latest AWS innovations help meet the real-world needs of customers as they build and scale transformative generative AI applications. Canva uses AWS to power 1.2

AWS 95

AWS AI AI Data Warehouse 95

AWS Machine Learning Blog

APRIL 3, 2025

At the heart of this transformation is the OMRON Data & Analytics Platform (ODAP), an innovative initiative designed to revolutionize how the company harnesses its data assets. The robust security features provided by Amazon S3, including encryption and durability, were used to provide data protection.

AWS 81

AWS Data Governance Data Silos SQL 81

AWS Machine Learning Blog

DECEMBER 11, 2024

Earlier this year, we published the first in a series of posts about how AWS is transforming our seller and customer journeys using generative AI. Using our AI assistant built on Amazon Q, team members are saving hours of time each week. This time adds up individually, but also collectively at the team and organizational level.

AWS 105

AWS Database AI AI 105

Tableau

JUNE 8, 2021

Domain experts, for example, feel they are still overly reliant on core IT to access the data assets they need to make effective business decisions. In all of these conversations there is a sense of inertia: Data warehouses and data lakes feel cumbersome and data pipelines just aren't agile enough.

Tableau 102

Tableau Data Lakes Data Warehouse SQL 102

Flipboard

DECEMBER 6, 2023

These datasets are often a mix of numerical and text data, at times structured, unstructured, or semi-structured. needed to address some of these challenges in one of their many AI use cases built on AWS. RAG with semantic search – Conventional RAG with semantic search was the last step before moving to SQL generation.

SQL 168

SQL Database AWS Machine Learning 168

Flipboard

DECEMBER 18, 2023

Amazon Redshift powers data-driven decisions for tens of thousands of customers every day with a fully managed, AI-powered cloud data warehouse, delivering the best price-performance for your analytics workloads. Discover how you can use Amazon Redshift to build a data mesh architecture to analyze your data.

AWS 139

AWS Data Warehouse ETL SQL 139

phData

SEPTEMBER 19, 2023

With the amount of data companies are using growing to unprecedented levels, organizations are grappling with the challenge of efficiently managing and deriving insights from these vast volumes of structured and unstructured data. What is a Data Lake? Consistency of data throughout the data lake.

Data Lakes 52

Data Lakes Data Modeling Data Models Data Warehouse 52

Pickl AI

NOVEMBER 15, 2023

Discover the nuanced dissimilarities between Data Lakes and Data Warehouses. Data management in the digital age has become a crucial aspect of businesses, and two prominent concepts in this realm are Data Lakes and Data Warehouses. It acts as a repository for storing all the data.

Data Lakes 52

Data Lakes Data Warehouse Database ETL 52

ODSC - Open Data Science

SEPTEMBER 29, 2023

Data management problems can also lead to data silos; disparate collections of databases that don’t communicate with each other, leading to flawed analysis based on incomplete or incorrect datasets. One way to address this is to implement a data lake: a large and complex database of diverse datasets all stored in their original format.

Data Lakes 52

Data Lakes Clustering Big Data Big Data 52

AWS Machine Learning Blog

JUNE 13, 2023

Organizations can maximize the value of their modern data architecture with generative AI solutions while innovating continuously. The natural language capabilities allow non-technical users to query data through conversational English rather than complex SQL. The following diagram illustrates the architecture.

Database 97

Database SQL AWS AI 97

AWS Machine Learning Blog

FEBRUARY 21, 2025

With the current housing shortage and affordability concerns, Rocket simplifies the homeownership process through an intuitive and AI-driven experience. Data exploration and model development were conducted using well-known machine learning (ML) tools such as Jupyter or Apache Zeppelin notebooks.

Data Science 75

Data Science AWS Hadoop Data Scientist 75

ODSC - Open Data Science

SEPTEMBER 12, 2023

Although setting up a database to run your analyses may seem like an arduous task, modern open-source time series databases can provide significant benefits to any scientist running time series analysis on a large data set — and with much less effort than you might imagine. Interested in attending an ODSC event?

Data Scientist 97

Data Scientist Database Data Lakes Data Science 97

IBM Journey to AI blog

APRIL 24, 2023

By running reports on historical data, a data warehouse can clarify what systems and processes are working and what methods need improvement. Data warehouse is the base architecture for artificial intelligence and machine learning (AI/ML) solutions as well.

Data Warehouse 112

Data Warehouse Data Lakes Database Big Data 112

DagsHub

DECEMBER 11, 2023

Dolt Created in 2019, Dolt is an open-source tool for managing SQL databases that uses version control similar to Git. It versions tables instead of files and has a SQL query interface for those tables. However, these tools have functional gaps for more advanced data workflows. It provides both community and enterprise editions.

Machine Learning 98

Machine Learning Machine Learning Data Lakes Database 98

AWS Machine Learning Blog

SEPTEMBER 27, 2024

Specifically, we cover the computer vision and artificial intelligence (AI) techniques used to combine datasets into a list of prioritized tasks for field teams to investigate and mitigate. SageMaker JumpStart provided deployable models that could be trained for object detection use cases with minimal data science knowledge and overhead.

AWS 120

AWS Data Lakes ML ML 120

IBM Journey to AI blog

SEPTEMBER 25, 2023

Every day, millions of riders use the Uber app, unwittingly contributing to a complex web of data-driven decisions. This blog takes you on a journey into the world of Uber’s analytics and the critical role that Presto, the open source SQL query engine, plays in driving their success. What is Presto?

Data Lakes 102

Data Lakes Analytics Analytics Clustering 102

ODSC - Open Data Science

JULY 27, 2023

Adopting & Scaling AI, a Beginner’s Guide to Prompt Engineering, and Pretraining Large Language Models 5 Concerns Surrounding the Scaling and Adoption of AI From privacy and data security to job displacement and more, these are five concerns that people have about AI right now. What Can AI Teach Us About Data Centers?

Data Lakes 52

Data Lakes SQL AI AI 52

IBM Journey to AI blog

AUGUST 4, 2023

It’s distributed both in the cloud and on-premises, allowing extensive use and movement across clouds, apps and networks, as well as stores of data at rest. An architecture designed for data democratization aims to be flexible, integrated, agile and secure to enable the use of data and artificial intelligence (AI) at scale.

Data Lakes 73

Data Lakes AI AI Data Governance 73

Tableau

JUNE 8, 2021

Domain experts, for example, feel they are still overly reliant on core IT to access the data assets they need to make effective business decisions. In all of these conversations there is a sense of inertia: Data warehouses and data lakes feel cumbersome and data pipelines just aren't agile enough.

Tableau 52

Tableau Data Lakes Data Warehouse SQL 52

Snorkel AI

MAY 26, 2023

[link] Ahmad Khan, head of artificial intelligence and machine learning strategy at Snowflake gave a presentation entitled “Scalable SQL + Python ML Pipelines in the Cloud” about his company’s Snowpark service at Snorkel AI’s Future of Data-Centric AI virtual conference in August 2022. Welcome everybody.

SQL 52

SQL ML ML Python 52

Snorkel AI

MAY 26, 2023

[link] Ahmad Khan, head of artificial intelligence and machine learning strategy at Snowflake gave a presentation entitled “Scalable SQL + Python ML Pipelines in the Cloud” about his company’s Snowpark service at Snorkel AI’s Future of Data-Centric AI virtual conference in August 2022. Welcome everybody.

SQL 52

SQL ML ML Python 52

Flipboard

AUGUST 17, 2023

Amazon Redshift uses SQL to analyze structured and semi-structured data across data warehouses, operational databases, and data lakes, using AWS-designed hardware and ML to deliver the best price-performance at any scale. You can use query_string to filter your dataset by SQL and unload it to Amazon S3.

ML 123

ML ML AWS Data Warehouse 123

AWS Machine Learning Blog

AUGUST 21, 2023

You can streamline the process of feature engineering and data preparation with SageMaker Data Wrangler and finish each stage of the data preparation workflow (including data selection, purification, exploration, visualization, and processing at scale) within a single visual interface.

AWS 98

AWS Data Lakes Clustering Data Preparation 98

ODSC - Open Data Science

FEBRUARY 24, 2023

Cleanlab GitHub | Website Cleanlab is focused on data-centric AI (DCAI), providing algorithms/interfaces to help companies (across all industries) improve the quality of their datasets and diagnose/fix various issues in them. With Great Expectations , data teams can express what they “expect” from their data using simple assertions.

Exploratory Data Analysis 126

Exploratory Data Analysis Data Visualization Data Analysis Data Analysis 126

IBM Journey to AI blog

MARCH 14, 2024

Businesses face significant hurdles when preparing data for artificial intelligence (AI) applications. The existence of data silos and duplication, alongside apprehensions regarding data quality, presents a multifaceted environment for organizations to manage.

AWS 93

AWS Database ETL AI 93

IBM Journey to AI blog

JULY 17, 2023

is our enterprise-ready next-generation studio for AI builders, bringing together traditional machine learning (ML) and new generative AI capabilities powered by foundation models. With watsonx.ai, businesses can effectively train, validate, tune and deploy AI models with confidence and at scale across their enterprise.

AI 74

AI AI Machine Learning Machine Learning 74

How to Learn Machine Learning

MAY 2, 2025

If you’re curious about leveraging cutting-edge AI capabilities without the headache of managing complex infrastructure, you’ve come to the right place! This is where Azure Machine Learning shines by democratizing access to advanced AI capabilities. Learn more from the Responsible AI dashboard documentation.

Azure 52

Azure Machine Learning Machine Learning Data Science 52

ODSC - Open Data Science

FEBRUARY 2, 2023

Computer Science and Computer Engineering Similar to knowing statistics and math, a data scientist should know the fundamentals of computer science as well. While knowing Python, R, and SQL are expected, you’ll need to go beyond that. Big Data As datasets become larger and more complex, knowing how to work with them will be key.

Data Science 98

Data Science Data Scientist Computer Science Computer Science 98

AWS Machine Learning Blog

NOVEMBER 27, 2023

This post is co-authored by Daryl Martis, Director of Product, Salesforce Einstein AI. This is the third post in a series discussing the integration of Salesforce Data Cloud and Amazon SageMaker. Einstein Studio is a gateway to AI tools on Salesforce Data Cloud. Manage Data Cloud profile data ( Data Cloud_profile_api ).

ML 97

ML ML AWS SQL 97

Women in Big Data

NOVEMBER 27, 2024

Its PostgreSQL foundation ensures compatibility with most SQL clients. While it shares similarities with PostgreSQL, there are key differences that must be considered during application development. Strengths : High performance with SQL support, easy integration with other AWS services, and strong security features.

Data Warehouse 59

Data Warehouse Big Data Big Data Azure 59

Pickl AI

APRIL 21, 2025

Key Skills for Data Science: A data scientist typically needs a blend of skills: Mathematics and Statistics: To understand the theoretical underpinnings of models. Programming: Often in languages like Python or R, using libraries for data manipulation, analysis, and machine learning.

Big Data 52

Big Data Big Data Data Science Machine Learning 52

DataRobot Blog

MARCH 7, 2023

Organizations that want to prove the value of AI by developing, deploying, and managing machine learning models at scale can now do so quickly using the DataRobot AI Platform on Microsoft Azure. This generates reliable business insights and sustains AI-driven value across the enterprise.

Azure 59

Azure Machine Learning Machine Learning AI 59

IBM Journey to AI blog

SEPTEMBER 19, 2023

Overview: Data science vs data analytics Think of data science as the overarching umbrella that covers a wide range of tasks performed to find patterns in large datasets, structure data for use, train machine learning models and develop artificial intelligence (AI) applications.

Data Science 94

Data Science Analytics Analytics Data Scientist 94

SAS Software

NOVEMBER 29, 2023

Azure Data Lake Storage (ADLS) Gen2のストレージアカウントの作成 3-2.ストレージアカウントのデータストレージコンテナの作成 Azure SynapseのSQLデータベースをSASライブラリとして定義 4-3.Azure Bulkload機能について 3.BULKLOAD機能を利用するためのAzure側で必要なサービスの作成 BULKLOAD機能を利用するためのAzure側で必要なサービスの作成 3-1.Azure ストレージアカウントのデータストレージコンテナの作成 3-3.ストレージアカウントの利用ユーザー権限の設定 ストレージアカウントの利用ユーザー権限の設定 3-4.データ書き込み用のSASコードの実行

Azure 59

Azure Data Lakes SQL Analytics 59