ODSC - Open Data Science

SEPTEMBER 27, 2023

In the ever-evolving world of big data, managing vast amounts of information efficiently has become a critical challenge for businesses across the globe. As data lakes gain prominence as a preferred solution for storing and processing enormous datasets, the need for effective data version control mechanisms becomes increasingly evident.

Data Lakes 98

Data Lakes Data Warehouse Database Big Data 98

Dataconomy

MARCH 17, 2025

MinIO, a provider of high-performance object storage for AI, announced several upcoming enhancements to its AIStor product at NVIDIA GTC. These updates are designed to deepen MinIO’s support for the NVIDIA AI ecosystem and improve the efficiency and utilization of AI infrastructure. It will increase CPU efficiency.

Data Lakes 91

Data Lakes AI AI ML 91

Data Science Blog

FEBRUARY 7, 2023

Auch bei Process Mining tut sich gerade viel, Machine Learning hält Einzug ins Process Mining, Prozesse können immer granularer analysiert werden, auch unstrukturierte Daten können unter Einsatz von AI mit in die Analyse einbezogen werden usw. Was gerade zum Trend wird, ist der Aufbau eines Data Lakehouses.

Business Intelligence 286

Business Intelligence Business Intelligence Data Warehouse Data Lakes 286

Dataversity

MARCH 24, 2021

Enterprises migrating on-prem data environments to the cloud in pursuit of more robust, flexible, and integrated analytics and AI/ML capabilities are fueling a surge in cloud data lake implementations. The post How to Ensure Your New Cloud Data Lake Is Secure appeared first on DATAVERSITY.

Data Lakes 59

Data Lakes Cloud Data ML ML 59

Dataversity

JANUARY 28, 2021

It has been ten years since Pentaho Chief Technology Officer James Dixon coined the term “data lake.” While data warehouse (DWH) systems have had longer existence and recognition, the data industry has embraced the more […]. The post A Bridge Between Data Lakes and Data Warehouses appeared first on DATAVERSITY.

Data Lakes 59

Data Lakes Data Warehouse Data Quality Data Governance 59

Dataconomy

JUNE 1, 2023

Unified data storage : Fabric’s centralized data lake, Microsoft OneLake, eliminates data silos and provides a unified storage system, simplifying data access and retrieval. OneLake is designed to store a single copy of data in a unified location, leveraging the open-source Apache Parquet format.

Power BI 194

Power BI Data Lakes Azure Data Silos 194

Alation

OCTOBER 21, 2021

Data Swamp vs Data Lake. When you imagine a lake, it’s likely an idyllic image of a tree-ringed body of reflective water amid singing birds and dabbling ducks. I’ll take the lake, thank you very much. Many organizations have built a data lake to solve their data storage, access, and utilization challenges.

Data Lakes 52

Data Lakes Data Governance Data Warehouse Business Intelligence 52

Flipboard

DECEMBER 11, 2024

Many of these applications are complex to build because they require collaboration across teams and the integration of data, tools, and services. Data engineers use data warehouses, data lakes, and analytics tools to load, transform, clean, and aggregate data.

SQL 160

SQL AWS Data Lakes AI 160

ODSC - Open Data Science

JUNE 7, 2023

To make your data management processes easier, here’s a primer on data lakes, and our picks for a few data lake vendors worth considering. What is a data lake? First, a data lake is a centralized repository that allows users or an organization to store and analyze large volumes of data.

Data Lakes 52

Data Lakes Azure Data Warehouse Hadoop 52

phData

SEPTEMBER 19, 2023

With the amount of data companies are using growing to unprecedented levels, organizations are grappling with the challenge of efficiently managing and deriving insights from these vast volumes of structured and unstructured data. What is a Data Lake? Consistency of data throughout the data lake.

Data Lakes 52

Data Lakes Data Models Data Modeling Data Warehouse 52

Pickl AI

NOVEMBER 15, 2023

Discover the nuanced dissimilarities between Data Lakes and Data Warehouses. Data management in the digital age has become a crucial aspect of businesses, and two prominent concepts in this realm are Data Lakes and Data Warehouses. It acts as a repository for storing all the data.

Data Lakes 52

Data Lakes Data Warehouse Database ETL 52

KDnuggets

DECEMBER 14, 2021

We have solicited insights from experts at industry-leading companies, asking: "What were the main AI, Data Science, Machine Learning Developments in 2021 and what key trends do you expect in 2022?" Read their opinions here.

Data Science 399

Data Science Machine Learning Machine Learning Analytics 399

Dataversity

OCTOBER 14, 2022

The data being talked about is useful for businesses to draw insights, formulate strategies, and understand trends and customer behavior, among others. […]. The post Maximize the ROI of Your Enterprise Data Lake appeared first on DATAVERSITY.

Data Lakes 52

Data Lakes Analytics Analytics Artificial Intelligence 52

ODSC - Open Data Science

SEPTEMBER 29, 2023

Data management problems can also lead to data silos; disparate collections of databases that don’t communicate with each other, leading to flawed analysis based on incomplete or incorrect datasets. One way to address this is to implement a data lake: a large and complex database of diverse datasets all stored in their original format.

Data Lakes 52

Data Lakes Clustering Big Data Big Data 52

ODSC - Open Data Science

JUNE 1, 2023

Real-Time ML with Spark and SBERT, AI Coding Assistants, Data Lake Vendors, and ODSC East Highlights Getting Up to Speed on Real-Time Machine Learning with Spark and SBERT Learn more about real-time machine learning by using this approach that uses Apache Spark and SBERT. Is an AI Coding Assistant Right For You?

Data Lakes 52

Data Lakes ML ML Citizen Data Scientist 52

AWS Machine Learning Blog

JANUARY 29, 2025

Generative AI can revolutionize organizations by enabling the creation of innovative applications that offer enhanced customer and employee experiences. In this post, we evaluate different generative AI operating model architectures that could be adopted.

AWS 86

AWS AI AI Database 86

AWS Machine Learning Blog

AUGUST 8, 2024

As one of the largest AWS customers, Twilio engages with data, artificial intelligence (AI), and machine learning (ML) services to run their daily workloads. Data is the foundational layer for all generative AI and ML applications. The following diagram illustrates the solution architecture.

SQL 122

SQL Data Lakes Data Analyst AWS 122

Towards AI

MARCH 28, 2024

Author(s): Towards AI Editorial Team Originally published on Towards AI. Good morning, AI enthusiasts! This week, I’m super excited to announce that we are finally releasing our book, ‘Building AI for Production; Enhancing LLM Abilities and Reliability with Fine-Tuning and RAG,’ where we gathered all our learnings.

Data Lakes 97

Data Lakes AI AI Azure 97

AWS Machine Learning Blog

NOVEMBER 12, 2024

He is focused on Big Data, Data Lakes, Streaming and batch Analytics services and generative AI technologies. He works with strategic customers who are using AI/ML to solve complex business problems. Varun Mehta is a Sr. Solutions Architect at AWS. Outside of work, he loves to spend time with his wife and kids

SQL 106

SQL AWS AI AI 106

Tableau

JUNE 8, 2021

Domain experts, for example, feel they are still overly reliant on core IT to access the data assets they need to make effective business decisions. In all of these conversations there is a sense of inertia: Data warehouses and data lakes feel cumbersome and data pipelines just aren't agile enough.

Tableau 102

Tableau Data Lakes Data Warehouse SQL 102

ODSC - Open Data Science

AUGUST 31, 2023

Visualization for Clustering Methods Clustering methods are a big part of data science, and here’s a primer on how you can visualize them. Lemley on Generative AI and the Law Here’s what Mark A. Lemley, law Professor at Stanford, thinks about legal issues that arise from generative AI, the memorization problem, and more.

Clustering 98

Clustering Data Lakes Data Science Artificial Intelligence 98

AWS Machine Learning Blog

NOVEMBER 6, 2024

The rise of large language models (LLMs) and foundation models (FMs) has revolutionized the field of natural language processing (NLP) and artificial intelligence (AI). These powerful models, trained on vast amounts of data, can generate human-like text, answer questions, and even engage in creative writing tasks.

AWS 116

AWS Python Machine Learning Machine Learning 116

Flipboard

OCTOBER 25, 2023

By some estimates, unstructured data can make up to 80–90% of all new enterprise data and is growing many times faster than structured data. After decades of digitizing everything in your enterprise, you may have an enormous amount of data, but with dormant value. These services write the output to a data lake.

AWS 167

AWS ML ML Analytics 167

IBM Journey to AI blog

APRIL 15, 2024

An inaccurate AI prediction in a marketing campaign is a minor nuisance, but an inaccurate AI prediction on a manufacturing shopfloor can be fatal. Or we create a data lake, which quickly degenerates to a data swamp. Summarization Summarization remains the top use case for generative AI (gen AI) technology.

AI 116

AI AI Data Lakes Analytics 116

ODSC - Open Data Science

MAY 24, 2023

Building a Business with a Real-Time Analytics Stack, Streaming ML Without a Data Lake, and Google’s PaLM 2 Building a Pizza Delivery Service with a Real-Time Analytics Stack The best businesses react quickly and with informed decisions. Here’s a use case of how you can use a real-time analytics stack to build a pizza delivery service.

Data Lakes 52

Data Lakes ML ML Analytics 52

IBM Journey to AI blog

OCTOBER 4, 2023

Artificial intelligence (AI) is now at the forefront of how enterprises work with data to help reinvent operations, improve customer experiences, and maintain a competitive advantage. It’s no longer a nice-to-have, but an integral part of a successful data strategy. Why does AI need an open data lakehouse architecture?

Data Lakes 95

Data Lakes Data Warehouse AI AI 95

Data Science Connect

JULY 28, 2023

These platforms provide data engineers with the flexibility to develop and deploy IoT applications efficiently. Data Lakes for Centralized Storage Data lakes serve as centralized repositories for storing raw and processed IoT data.

Internet of Things 130

Internet of Things Data Engineering Data Engineer Data Engineering 130

AWS Machine Learning Blog

DECEMBER 11, 2024

Earlier this year, we published the first in a series of posts about how AWS is transforming our seller and customer journeys using generative AI. Using our AI assistant built on Amazon Q, team members are saving hours of time each week. This time adds up individually, but also collectively at the team and organizational level.

AWS 99

AWS Database AI AI 99

IBM Journey to AI blog

MAY 9, 2023

We stand on the frontier of an AI revolution. Over the past decade, deep learning arose from a seismic collision of data availability and sheer compute power, enabling a host of impressive AI capabilities. It sounds like a joke, but it’s not, as anyone who has tried to solve business problems with AI may know.

AI 139

AI AI Data Quality Data Lakes 139

Dataconomy

FEBRUARY 14, 2025

AI-driven revenue optimization The new system enables hoteliers to manage pricing dynamically , making data-driven adjustments across rooms, event spaces, and F&B outlets. The AI-powered automation provides forecasting, strategic planning assistance, and customizable rate management to improve overall profitability.

Data Lakes 91

Data Lakes AI AI Deep Learning 91

AWS Machine Learning Blog

OCTOBER 20, 2023

Recent developments in generative AI models have further sped up the need of ML adoption across industries. However, implementing security, data privacy, and governance controls are still key challenges faced by customers when implementing ML workloads at scale.

ML 128

ML ML AWS Data Lakes 128

ODSC - Open Data Science

MARCH 21, 2025

In todays fast-paced data-driven world, open-source solutions are transforming industries by providing flexible, scalable, and community-driven innovations. Whether youre a data scientist, engineer, or AI researcher, tapping into open-source technologies can accelerate your work while fostering collaboration.

Data Scientist 52

Data Scientist Data Visualization Data Science Data Lakes 52

AWS Machine Learning Blog

JUNE 21, 2024

To accomplish this, eSentire built AI Investigator, a natural language query tool for their customers to access security platform data by using AWS generative artificial intelligence (AI) capabilities. This helps customers quickly and seamlessly explore their security data and accelerate internal investigations.

AWS 123

AWS AI AI Natural Language Processing 123

Dataconomy

FEBRUARY 20, 2024

Every tech evangelist and their grandma was gushing about generative AI, hailing it as the dawn of a new creative epoch. The “wow” factor has waned, replaced by a nagging question: Is AI creative? Is AI creative? As AI matures, its ability to process information, adapt, and learn will exponentially increase.

AI 113

AI AI Algorithm Data Lakes 113

AWS Machine Learning Blog

JUNE 18, 2024

This post presents a solution that uses a generative artificial intelligence (AI) to standardize air quality data from low-cost sensors in Africa, specifically addressing the air quality data integration problem of low-cost sensors. Qiong (Jo) Zhang , PhD, is a Senior Partner Solutions Architect at AWS, specializing in AI/ML.

AWS 126

AWS Python AI AI 126

Flipboard

NOVEMBER 24, 2023

Data scientists and ML engineers require capable tooling and sufficient compute for their work. To pave the way for the growth of AI, BMW Group needed to make a leap regarding scalability and elasticity while reducing operational overhead, software licensing, and hardware management.

ML 154

ML ML AWS AI 154

IBM Data Science in Practice

JANUARY 9, 2024

One groundbreaking technology that has emerged as a game-changer is asset performance management (APM) artificial intelligence (AI). However, embarking on the journey of implementing artificial intelligence (AI) in your asset performance management strategy can be both exciting and daunting.

ML 130

ML ML AI AI 130