Data Science 101

NOVEMBER 7, 2019

Microsoft just held one of its largest conferences of the year, and a few major announcements were made which pertain to the cloud data science world. Azure Synapse Analytics can be seen as a merge of Azure SQL Data Warehouse and Azure Data Lake. Here they are in my order of importance (based upon my opinion).

Data Science 106

Data Science Azure SQL Machine Learning 106

phData

NOVEMBER 4, 2024

Snowflake’s cloud-agnosticism, separation of storage and compute resources, and ability to handle semi-structured data have exemplified Snowflake as the best-in-class cloud data warehousing solution. Snowflake supports data sharing and collaboration across organizations without the need for complex data pipelines.

Machine Learning 69

Machine Learning Machine Learning Data Science ML 69

IBM Journey to AI blog

MAY 9, 2023

At IBM, we believe it is time to place the power of AI in the hands of all kinds of “AI builders” — from data scientists to developers to everyday users who have never written a single line of code. With watsonx.data , businesses can quickly connect to data, get trusted insights and reduce data warehouse costs.

AI 110

AI AI Data Warehouse Machine Learning 110

Dataconomy

SEPTEMBER 4, 2023

Define data ownership, access controls, and data management processes to maintain the integrity and confidentiality of your data. Data integration: Integrate data from various sources into a centralized cloud data warehouse or data lake.

Analytics 203

Analytics Analytics Big Data Analytics Big Data Analytics 203

Alation

JANUARY 17, 2023

The modern data stack is a combination of various software tools used to collect, process, and store data on a well-integrated cloud-based data platform. It is known to have benefits in handling data due to its robustness, speed, and scalability. Data ingestion/integration services. Data orchestration tools.

Data Warehouse 52

Data Warehouse ETL Tableau Cloud Data 52

phData

NOVEMBER 8, 2024

Each snapshot has a separate manifest file that keeps track of the data files associated with that snapshot and hence can be restored/queries whenever needed. Versioning also ensures a safer experimentation environment, where data scientists can test new models or hypotheses on historical data snapshots without impacting live data.

Data Lakes 52

Data Lakes Data Warehouse Database Azure 52

Flipboard

AUGUST 17, 2023

Amazon Redshift is the most popular cloud data warehouse that is used by tens of thousands of customers to analyze exabytes of data every day. Conclusion In this post, we demonstrated an end-to-end data and ML flow from a Redshift data warehouse to SageMaker.

ML 123

ML ML AWS Data Warehouse 123

IBM Journey to AI blog

JULY 24, 2024

By providing access to a wider pool of trusted data, it enhances the relevance and precision of AI models, accelerating innovation in these areas. Optimizing performance with fit-for-purpose query engines In the realm of data management, the diverse nature of data workloads demands a flexible approach to query processing.

Data Governance 69

Data Governance Data Warehouse Data Preparation Analytics 69

Alation

SEPTEMBER 7, 2021

In the previous blog , we discussed how Alation provides a platform for data scientists and analysts to complete projects and analysis at speed. In this blog we will discuss how Alation helps minimize risk with active data governance. But governance is a time-consuming process (for users and data stewards alike).

Data Governance 59

Data Governance Data Scientist Data Quality Data Profiling 59

IBM Journey to AI blog

JULY 11, 2023

Data warehouses are a critical component of any organization’s technology ecosystem. The next generation of IBM Db2 Warehouse brings a host of new capabilities that add cloud object storage support with advanced caching to deliver 4x faster query performance than previously, while cutting storage costs by 34x 1.

Data Warehouse 60

Data Warehouse Database Cloud Data Big Data 60

IBM Journey to AI blog

SEPTEMBER 11, 2023

Db2 Warehouse SaaS, on the other hand, is a fully managed elastic cloud data warehouse with our columnar technology. watsonx.data integration At Think, IBM announced watsonx.data as a new open, hybrid and governed data store optimized for all data, analytics, and AI workloads.

Database 101

Database SQL Data Warehouse Machine Learning 101

Mlearning.ai

FEBRUARY 16, 2023

The demand for information repositories enabling business intelligence and analytics is growing exponentially, giving birth to cloud solutions. The ultimate need for vast storage spaces manifests in data warehouses: specialized systems that aggregate data coming from numerous sources for centralized management and consistency.

Data Warehouse 52

Data Warehouse Business Intelligence Business Intelligence Database 52

Smart Data Collective

OCTOBER 17, 2022

If you haven’t already, moving to the cloud can be a realistic alternative. Cloud data warehouses provide various advantages, including the ability to be more scalable and elastic than conventional warehouses. Can’t get to the data. You can’t afford to waste their time on a few reports.

Big Data 119

Big Data Big Data Data Engineering Data Engineering 119

IBM Journey to AI blog

JULY 17, 2023

Within watsonx.ai, users can take advantage of open-source frameworks like PyTorch, TensorFlow and scikit-learn alongside IBM’s entire machine learning and data science toolkit and its ecosystem tools for code-based and visual data science capabilities.

AI 74

AI AI Machine Learning Machine Learning 74

phData

JULY 18, 2023

With the birth of cloud data warehouses, data applications, and generative AI , processing large volumes of data faster and cheaper is more approachable and desired than ever. First up, let’s dive into the foundation of every Modern Data Stack, a cloud-based data warehouse.

Data Warehouse 52

Data Warehouse Analytics Analytics Cloud Data 52

The MLOps Blog

MAY 17, 2023

ETL pipeline | Source: Author These activities involve extracting data from one system, transforming it, and then processing it into another target system where it can be stored and managed. ML heavily relies on ETL pipelines as the accuracy and effectiveness of a model are directly impacted by the quality of the training data.

ETL 59

ETL Data Pipeline ML ML 59

Alation

AUGUST 9, 2021

Once people have found the data they are looking for, they need to be able to immediately begin their analysis. Pologruto solved this problem by giving people the option to analyze data in the tool of their choice: Jupyter Notebooks for data scientists and more technical teams, and Sigma for business teams.

Analytics 98

Analytics Analytics Data Warehouse Data Scientist 98

Snorkel AI

JANUARY 11, 2023

Time to label training data for ML solution was prohibitively slow given the reliance on external data labeling services that required multiple iterations. Constrained collaboration due to the limited amount of time domain experts and data scientists had to solve for ambiguous labels, which blocked their ability to iterate quickly.

Data Warehouse 52

Data Warehouse Natural Language Processing Data Scientist Cloud Data 52

Alation

JUNE 7, 2022

Fifth Third faced a number of pain points borne of a large data landscape. The Problem: The Data Challenges. The data challenges at Fifth Third will sound familiar to anyone working in an enterprise data landscape. To meet that growing demand, they decided to make everyone a data citizen.

Data Governance 52

Data Governance Data Scientist Data Engineering Data Engineering 52

Precisely

APRIL 3, 2023

It simply wasn’t practical to adopt an approach in which all of an organization’s data would be made available in one central location, for all-purpose business analytics. To speed analytics, data scientists implemented pre-processing functions to aggregate, sort, and manage the most important elements of the data.

ETL 52

ETL Analytics Analytics Database 52

Alation

APRIL 14, 2022

This two-part series will explore how data discovery, fragmented data governance , ongoing data drift, and the need for ML explainability can all be overcome with a data catalog for accurate data and metadata record keeping. The Cloud Data Migration Challenge. Data Governance and Data Security.

Data Governance 59

Data Governance ML ML Cloud Data 59

Alation

OCTOBER 27, 2022

Few actors in the modern data stack have inspired the enthusiasm and fervent support as dbt. This data transformation tool enables data analysts and engineers to transform, test and document data in the cloud data warehouse. Jason: I’m curious to learn about your modern data stack.

Data Analyst 52

Data Analyst Data Scientist Analytics Analytics 52

Alation

JUNE 9, 2021

We have an explosion, not only in the raw amount of data, but in the types of database systems for storing it ( db-engines.com ranks over 340) and architectures for managing it (from operational datastores to data lakes to cloud data warehouses). Organizations are drowning in a deluge of data.

Data Governance 59

Data Governance Database Cloud Data Machine Learning 59

Alation

OCTOBER 18, 2022

Alation is pleased to be named a dbt Metrics Partner and to announce the start of a partnership with dbt, which will bring dbt data into the Alation data catalog. In the modern data stack, dbt is a key tool to make data ready for analysis. Data engineers, analysts, and data scientists often collaborate to transform data.

Data Analyst 52

Data Analyst Data Engineering Data Engineering Data Engineering 52

Snorkel AI

MAY 26, 2023

And one of the biggest challenges that we see is taking an idea, an experiment, or an ML experiment that data scientists might be running in their notebooks and putting that into production. And it might be that these are two totally separate data environments and a lot of times they’re separate for compute processing as well.

SQL 52

SQL ML ML Python 52

Snorkel AI

MAY 26, 2023

And one of the biggest challenges that we see is taking an idea, an experiment, or an ML experiment that data scientists might be running in their notebooks and putting that into production. And it might be that these are two totally separate data environments and a lot of times they’re separate for compute processing as well.

SQL 52

SQL ML ML Python 52

DataRobot Blog

FEBRUARY 21, 2022

With growing pressure on data scientists, every organization needs to ensure that their teams are empowered with the right tools. Data science notebooks have become a crucial part of the data science practice. Cloud-to-Cloud Data Performance 10 3 to 10 6 Faster. This is not an imaginary issue.

Data Science 52

Data Science Python Data Scientist Machine Learning 52

Alation

DECEMBER 15, 2022

That’s your full suite of BI tools , databases and data sources, cloud data warehouses , connectors, and other beloved tools in the modern data stack. It must: 1) connect to everything and 2) be engaged and adopted by everyone. What do we mean by everything ? What do we mean by everyone ?

Data Governance 52

Data Governance Data Quality Data Warehouse Data Scientist 52

Mlearning.ai

JULY 10, 2023

In my 7 years of Data Science journey, I’ve been exposed to a number of different databases including but not limited to Oracle Database, MS SQL, MySQL, EDW, and Apache Hadoop. A lot of you who are already in the data science field must be familiar with BigQuery and its advantages.

SQL 52

SQL Database Apache Hadoop Data Science 52

ODSC - Open Data Science

JULY 11, 2023

If you are a data scientist, manager, or executive with limited time and funds, wondering whether/how to invest in data centers and what the pros, cons, and costs would be, chances are you will start from a similar place as I — having some knowledge then looking for more, be that from humans, machines, or both.

Data Lakes 40

Data Lakes AI AI Cloud Computing 40

Alation

APRIL 14, 2023

Last week, the Alation team had the privilege of joining IT professionals, business leaders, and data analysts and scientists for the Modern Data Stack Conference in San Francisco. In “The modern data stack is dead, long live the modern data stack!” Cloud costs are growing prohibitive. Let’s dive in!

Data Pipeline 52

Data Pipeline Data Warehouse Cloud Data ETL 52

phData

SEPTEMBER 27, 2024

Here’s how a composable CDP might incorporate the modeling approaches we’ve discussed: Data Storage and Processing : This is your foundation. You might choose a cloud data warehouse like the Snowflake AI Data Cloud or BigQuery. These changes are streamed into Iceberg tables in your data lake.

Data Models 52

Data Models Data Modeling Apache Kafka Data Lakes 52

AWS Machine Learning Blog

OCTOBER 28, 2024

The workflow includes the following steps: Within the SageMaker Canvas interface, the user composes a SQL query to run against the GCP BigQuery data warehouse. Athena returns the queried data from BigQuery to SageMaker Canvas, where you can use it for ML model training and development purposes within the no-code interface.

Machine Learning 106

Machine Learning Machine Learning ML ML 106