ODSC - Open Data Science

JULY 14, 2023

Below you’ll find just a few of the many expert-led sessions at ODSC Europe 2023 that attendees loved — and you can view them for yourself here ! And don’t miss the chance to join us for our upcoming free virtual Generative AI Summit on July 20th and ODSC West 2023 in San Francisco (October 31st-November 3rd). What’s next?

Machine Learning 52

Machine Learning Machine Learning Apache Kafka Data Science 52

AWS Machine Learning Blog

JUNE 21, 2024

In 2023, eSentire was looking for ways to deliver differentiated customer experiences by continuing to improve the quality of its security investigations and customer communications. eSentire has over 2 TB of signal data stored in their Amazon Simple Storage Service (Amazon S3) data lake.

AWS 114

AWS AI AI Natural Language Processing 114

phData

NOVEMBER 8, 2024

Note : Cloud Data warehouses like Snowflake and Big Query already have a default time travel feature. However, this feature becomes an absolute must-have if you are operating your analytics on top of your data lake or lakehouse. It can also be integrated into major data platforms like Snowflake.

Data Lakes 52

Data Lakes Data Warehouse Database Azure 52

phData

DECEMBER 4, 2023

Thirty seconds is a good default for human users; if you find that queries are regularly queueing, consider making your warehouse a multi-cluster that scales on-demand. Cluster Count If your warehouse has to serve many concurrent requests, you may need to increase the cluster count to meet demand.

Clustering 98

Clustering Database SQL Data Pipeline 98

ODSC - Open Data Science

JULY 11, 2023

What are the similarities and differences between data centers, data lake houses, and data lakes? Data centers, data lake houses, and data lakes are all related to data storage and management, but they have some key differences. Not a cloud computer?

Data Lakes 40

Data Lakes AI AI Cloud Computing 40

ODSC - Open Data Science

AUGUST 7, 2023

Databricks Databricks is the developer of Delta Lake, an open-source project that brings reliability to data lakes for machine learning and other cases. Their platform was developed for working with Spark and provides automated cluster management and Python-style notebooks.

Artificial Intelligence 52

Artificial Intelligence Artificial Intelligence Machine Learning Machine Learning 52

Pickl AI

NOVEMBER 4, 2024

Role of Data Engineers in the Data Ecosystem Data Engineers play a crucial role in the data ecosystem by bridging the gap between raw data and actionable insights. They are responsible for building and maintaining data architectures, which include databases, data warehouses, and data lakes.

Data Engineering 52

Data Engineering Data Engineering Data Engineering Data Engineer 52

phData

SEPTEMBER 20, 2023

A data mesh is a conceptual architectural approach for managing data in large organizations. Traditional data management approaches often involve centralizing data in a data warehouse or data lake, leading to challenges like data silos, data ownership issues, and data access and processing bottlenecks.

Data Silos 52

Data Silos Database Data Quality Data Engineer 52

Mlearning.ai

FEBRUARY 16, 2023

Thus, the solution allows for scaling data workloads independently from one another and seamlessly handling data warehousing, data lakes , data sharing, and engineering. Entrust your project to our experts, and we’ll identify your unique path to advanced and profitable data operations.

Data Warehouse 52

Data Warehouse Business Intelligence Business Intelligence Database 52

Snorkel AI

MAY 26, 2023

And so data scientists might be leveraging one compute service and might be leveraging an extracted CSV for their experimentation. And then the production teams might be leveraging a totally different single source of truth or data warehouse or data lake and totally different compute infrastructure for deploying models into production.

SQL 52

SQL ML ML Python 52

Snorkel AI

MAY 26, 2023

And so data scientists might be leveraging one compute service and might be leveraging an extracted CSV for their experimentation. And then the production teams might be leveraging a totally different single source of truth or data warehouse or data lake and totally different compute infrastructure for deploying models into production.

SQL 52

SQL ML ML Python 52

The MLOps Blog

MAY 9, 2023

The pipelines are interoperable to build a working system: Data (input) pipeline (data acquisition and feature management steps) This pipeline transports raw data from one location to another. Model/training pipeline This pipeline trains one or more models on the training data with preset hyperparameters.

ML 98

ML ML Machine Learning Machine Learning 98

Flipboard

DECEMBER 4, 2024

The use of separate data warehouses and lakes has created data silos, leading to problems such as lack of interoperability, duplicate governance efforts, complex architectures, and slower time to value. You can use Amazon SageMaker Lakehouse to achieve unified access to data in both data warehouses and data lakes.

Data Lakes 139

Data Lakes Data Warehouse AWS Database 139

IBM Journey to AI blog

SEPTEMBER 25, 2023

When a query is constructed, it passes through a cost-based optimizer, then data is accessed through connectors, cached for performance and analyzed across a series of servers in a cluster. Because of its distributed nature, Presto scales for petabytes and exabytes of data.

Data Lakes 102

Data Lakes Analytics Analytics Clustering 102

phData

SEPTEMBER 20, 2023

This blog was originally written by Keith Smith and updated for 2023 by Nick Goble & Dominick Rocco. You’ve probably heard of the Snowflake Data Cloud , but did you know that Snowflake also offers a revolutionary set of libraries and runtimes called Snowpark?

SQL 98

SQL Python Data Lakes Machine Learning 98

The MLOps Blog

MARCH 21, 2023

Orchestrators are concerned with lower-level abstractions like machines, instances, clusters, service-level grouping, replication, and so on. Along with the schedulers, they are integral to managing the regular workflows your data scientists run and how the tasks in those workflows communicate with the ML platform.

Machine Learning 69

Machine Learning Machine Learning Data Scientist ML 69