ODSC - Open Data Science

SEPTEMBER 27, 2023

In the ever-evolving world of big data, managing vast amounts of information efficiently has become a critical challenge for businesses across the globe. As data lakes gain prominence as a preferred solution for storing and processing enormous datasets, the need for effective data version control mechanisms becomes increasingly evident.

Data Lakes 98

Data Lakes Data Warehouse Database Big Data 98

Data Science 101

NOVEMBER 7, 2019

Microsoft just held one of its largest conferences of the year, and a few major announcements were made which pertain to the cloud data science world. Azure Synapse Analytics can be seen as a merge of Azure SQL Data Warehouse and Azure Data Lake. Those are the big data science announcements of the week.

Data Science 106

Data Science Azure SQL Machine Learning 106

Data Science 101

DECEMBER 16, 2019

Even though Amazon is taking a break from announcements (probably focusing on Christmas shoppers), there are still some updates in the cloud data science world. Azure Database for MySQL now supports MySQL 8.0 Wow, the last two weeks were taken over by the flurry of announcements from Amazon. Here they are. Thanks for reading.

Cloud Data 81

Cloud Data Data Science Azure Natural Language Processing 81

IBM Journey to AI blog

SEPTEMBER 19, 2023

Though you may encounter the terms “data science” and “data analytics” being used interchangeably in conversations or online, they refer to two distinctly different concepts. Meanwhile, data analytics is the act of examining datasets to extract value and find answers to specific questions.

Data Science 94

Data Science Analytics Analytics Data Scientist 94

Dataconomy

JUNE 1, 2023

Unified data storage : Fabric’s centralized data lake, Microsoft OneLake, eliminates data silos and provides a unified storage system, simplifying data access and retrieval. OneLake is designed to store a single copy of data in a unified location, leveraging the open-source Apache Parquet format.

Power BI 194

Power BI Data Lakes Azure Data Silos 194

ODSC - Open Data Science

MARCH 21, 2025

AiriaAI-Powered Insights Airia is leveraging AI to provide open-source tools that help organizations gain deeper insights into their data. Their contributions to AI and data science communities make it easier to integrate cutting-edge analytics into business strategies.

Data Scientist 52

Data Scientist Data Visualization Data Science Data Lakes 52

Dataconomy

MARCH 4, 2025

Data mining is a fascinating field that blends statistical techniques, machine learning, and database systems to reveal insights hidden within vast amounts of data. Businesses across various sectors are leveraging data mining to gain a competitive edge, improve decision-making, and optimize operations.

Data Mining 91

Data Mining Data Mining Data Mining Decision Trees 91

Pickl AI

NOVEMBER 15, 2023

Discover the nuanced dissimilarities between Data Lakes and Data Warehouses. Data management in the digital age has become a crucial aspect of businesses, and two prominent concepts in this realm are Data Lakes and Data Warehouses. It acts as a repository for storing all the data.

Data Warehouse 52

Data Warehouse Data Lakes Database ETL 52

ODSC - Open Data Science

SEPTEMBER 12, 2023

Be sure to check out his talk, “ What is a Time-series Database and Why do I Need One? Most data scientists are familiar with the concept of time series data and work with it often. The time series database (TSDB) , however, is still an underutilized tool in the data science community. at ODSC West 2023.

Data Scientist 97

Data Scientist Database Data Lakes Data Science 97

phData

SEPTEMBER 19, 2023

With the amount of data companies are using growing to unprecedented levels, organizations are grappling with the challenge of efficiently managing and deriving insights from these vast volumes of structured and unstructured data. What is a Data Lake? Consistency of data throughout the data lake.

Data Lakes 52

Data Lakes Data Models Data Modeling Data Warehouse 52

ODSC - Open Data Science

SEPTEMBER 29, 2023

Data management problems can also lead to data silos; disparate collections of databases that don’t communicate with each other, leading to flawed analysis based on incomplete or incorrect datasets. The data lake can then refine, enrich, index, and analyze that data. and various countries in Europe.

Data Lakes 52

Data Lakes Clustering Big Data Big Data 52

ODSC - Open Data Science

MARCH 23, 2023

Each month, ODSC has a few insightful webinars that touch on a range of issues that are important in the data science world, from use cases of machine learning models, to new techniques/frameworks, and more. These systems are based on real-time machine learning with a feature store, a vector database, and a model serving platform.

Data Science 52

Data Science Data Lakes Machine Learning Machine Learning 52

ODSC - Open Data Science

OCTOBER 13, 2024

Be sure to check out her talk, “ Don’t Go Over the Deep End: Building an Effective OSS Management Layer for Your Data Lake ,” there! Managing a data lake can often feel like being lost at sea — especially when dealing with both structured and unstructured data.

Data Lakes 40

Data Lakes Database Data Pipeline SQL 40

DagsHub

DECEMBER 11, 2023

The following points illustrates some of the main reasons why data versioning is crucial to the success of any data science and machine learning project: Storage space One of the reasons of versioning data is to be able to keep track of multiple versions of the same data which obviously need to be stored as well.

Machine Learning 98

Machine Learning Machine Learning Data Lakes Database 98

ODSC - Open Data Science

MAY 16, 2023

Comprehensive data privacy laws in at least four states are going into effect this year, and more than a dozen states have similar legislation in the works. Database management may become increasingly complex as organizations must account for more of these laws.

Database 98

Database Data Science Data Lakes Data Observability 98

ODSC - Open Data Science

FEBRUARY 24, 2023

There are many well-known libraries and platforms for data analysis such as Pandas and Tableau, in addition to analytical databases like ClickHouse, MariaDB, Apache Druid, Apache Pinot, Google BigQuery, Amazon RedShift, etc. With Great Expectations , data teams can express what they “expect” from their data using simple assertions.

Exploratory Data Analysis 126

Exploratory Data Analysis Data Visualization Data Analysis Data Analysis 126

AWS Machine Learning Blog

AUGUST 8, 2024

Managing and retrieving the right information can be complex, especially for data analysts working with large data lakes and complex SQL queries. This post highlights how Twilio enabled natural language-driven data exploration of business intelligence (BI) data with RAG and Amazon Bedrock.

SQL 110

SQL Data Lakes Data Analyst AWS 110

Data Science Connect

JULY 28, 2023

Data Collection and Integration Data engineers are responsible for designing robust data collection systems that gather information from various IoT devices and sensors. This data is then integrated into centralized databases for further processing and analysis.

Internet of Things 130

Internet of Things Data Engineer Data Engineering Data Engineering 130

ODSC - Open Data Science

FEBRUARY 15, 2024

The Future of the Single Source of Truth is an Open Data Lake Organizations that strive for high-performance data systems are increasingly turning towards the ELT (Extract, Load, Transform) model using an open data lake. To DIY you need to: host an API, build a UI, and run or rent a database.

Data Lakes 52

Data Lakes Data Visualization Machine Learning Machine Learning 52

AWS Machine Learning Blog

JUNE 20, 2024

Our goal was to improve the user experience of an existing application used to explore the counters and insights data. The data is stored in a data lake and retrieved by SQL using Amazon Athena. The problem Making data accessible to users through applications has always been a challenge.

SQL 96

SQL Database AWS Machine Learning 96

ODSC - Open Data Science

JANUARY 18, 2024

These professionals will work with their colleagues to ensure that data is accessible, with proper access. So let’s go through each step one by one, and help you build a roadmap toward becoming a data engineer. Identify your existing data science strengths. Stay on top of data engineering trends. Get more training!

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

phData

NOVEMBER 9, 2023

A cloud data warehouse is designed to combine a concept that every organization knows, namely a data warehouse, and optimizes the components of it, for the cloud. What is a Data Lake? A Data Lake is a location to store raw data that is in any format that an organization may produce or collect.

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98

AWS Machine Learning Blog

JUNE 5, 2023

Many teams are turning to Athena to enable interactive querying and analyze their data in the respective data stores without creating multiple data copies. Athena allows applications to use standard SQL to query massive amounts of data on an S3 data lake. Create a data lake with Lake Formation.

Machine Learning 79

Machine Learning Machine Learning AWS Data Lakes 79

Flipboard

DECEMBER 6, 2023

Considering the nature of the time series dataset, Q4 also realized that it would have to continuously perform incremental pre-training as new data came in. This would have required a dedicated cross-disciplinary team with expertise in data science, machine learning, and domain knowledge.

SQL 168

SQL Database AWS Machine Learning 168

Flipboard

OCTOBER 25, 2023

Why it’s challenging to process and manage unstructured data Unstructured data makes up a large proportion of the data in the enterprise that can’t be stored in a traditional relational database management systems (RDBMS). These services write the output to a data lake.

AWS 167

AWS ML ML Analytics 167

IBM Journey to AI blog

SEPTEMBER 25, 2023

Because of its distributed nature, Presto scales for petabytes and exabytes of data. The evolution of Presto at Uber Beginning of a data analytics journey Uber began their analytical journey with a traditional analytical database platform at the core of their analytics.

Data Lakes 102

Data Lakes Analytics Analytics Clustering 102

The MLOps Blog

JUNE 27, 2023

With built-in components and integration with Google Cloud services, Vertex AI simplifies the end-to-end machine learning process, making it easier for data science teams to build and deploy models at scale. Metaflow Metaflow helps data scientists and machine learning engineers build, manage, and deploy data science projects.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

ODSC - Open Data Science

NOVEMBER 19, 2024

ODSC West 2024 showcased a wide range of talks and workshops from leading data science, AI, and machine learning experts. This blog highlights some of the most impactful AI slides from the world’s best data science instructors, focusing on cutting-edge advancements in AI, data modeling, and deployment strategies.

Deep Learning 52

Deep Learning Deep Learning Data Science AI 52

AWS Machine Learning Blog

AUGUST 21, 2023

You can streamline the process of feature engineering and data preparation with SageMaker Data Wrangler and finish each stage of the data preparation workflow (including data selection, purification, exploration, visualization, and processing at scale) within a single visual interface.

AWS 89

AWS Data Lakes Clustering Data Preparation 89

Flipboard

NOVEMBER 24, 2023

The service streamlines ML development and production workflows (MLOps) across BMW by providing a cost-efficient and scalable development environment that facilitates seamless collaboration between data science and engineering teams worldwide. This results in faster experimentation and shorter idea validation cycles.

ML 153

ML ML AWS Data Scientist 153

Alation

AUGUST 4, 2022

And where data was available, the ability to access and interpret it proved problematic. Big data can grow too big fast. Left unchecked, data lakes became data swamps. Some data lake implementations required expensive ‘cleansing pumps’ to make them navigable again. Where Should Big Data Go from Here?

Big Data 95

Big Data Big Data Apache Kafka Data Lakes 95

AWS Machine Learning Blog

FEBRUARY 28, 2024

Structured Query Language (SQL) is a complex language that requires an understanding of databases and metadata. The solution in this post aims to bring enterprise analytics operations to the next level by shortening the path to your data using natural language. This table is used for finding the correct table, database, and attributes.

SQL 122

SQL AWS Database ML 122

IBM Journey to AI blog

DECEMBER 7, 2023

Online analytical processing (OLAP) database systems and artificial intelligence (AI) complement each other and can help enhance data analysis and decision-making when used in tandem. Defining OLAP today OLAP database systems have significantly evolved since their inception in the early 1990s.

Data Preparation 72

Data Preparation Database Data Analysis Data Analysis 72

O'Reilly Media

OCTOBER 19, 2021

Similarly, it would be pointless to pretend that a data-intensive application resembles a run-off-the-mill microservice which can be built with the usual software toolchain consisting of, say, GitHub, Docker, and Kubernetes. Adapted from the book Effective Data Science Infrastructure. Data Science Layers.

ML 145

ML ML Data Scientist AWS 145

ODSC - Open Data Science

JANUARY 30, 2024

This is a pretty important job as once the data has been integrated, it can be used for a variety of purposes, such as: Reporting and analytics Business intelligence Machine learning Data mining All of this provides stakeholders and even their own teams with the data they need when they need it.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

ODSC - Open Data Science

APRIL 3, 2023

As the sibling of data science, data analytics is still a hot field that garners significant interest. Companies have plenty of data at their disposal and are looking for people who can make sense of it and make deductions quickly and efficiently.

Analytics 52

Analytics Analytics Data Analyst Data Science 52

AWS Machine Learning Blog

SEPTEMBER 1, 2023

These teams are as follows: Advanced analytics team (data lake and data mesh) – Data engineers are responsible for preparing and ingesting data from multiple sources, building ETL (extract, transform, and load) pipelines to curate and catalog the data, and prepare the necessary historical data for the ML use cases.

AI 118

AI AI ML ML 118