Data Science Blog

FEBRUARY 4, 2023

If the data sources are additionally expanded to include the machines of production and logistics, much more in-depth analyses for error detection and prevention as well as for optimizing the factory in its dynamic environment become possible.

Cloud Data 147

Cloud Data Data Science Business Intelligence Business Intelligence 147

The MLOps Blog

JUNE 27, 2023

Microsoft Azure ML Platform The Azure Machine Learning platform provides a collaborative workspace that supports various programming languages and frameworks. Google Cloud Vertex AI Google Cloud Vertex AI provides a unified environment for both automated model development with AutoML and custom model training using popular frameworks.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

ODSC - Open Data Science

DECEMBER 5, 2023

We had bigger sessions on getting started with machine learning or SQL, up to advanced topics in NLP, and of course, plenty related to large language models and generative AI.

Data Science 98

Data Science Artificial Intelligence Artificial Intelligence Machine Learning 98

Women in Big Data

NOVEMBER 27, 2024

Snowflake Snowflake is a cloud-based data warehousing platform that offers a highly scalable and efficient architecture designed for performance and ease of use. It features Synapse Studio, a collaborative workspace for data integration, exploration, and analysis, allowing users to manage data pipelines seamlessly.

Data Warehouse 59

Data Warehouse Big Data Big Data Azure 59

IBM Journey to AI blog

AUGUST 12, 2024

AIOPs refers to the application of artificial intelligence (AI) and machine learning (ML) techniques to enhance and automate various aspects of IT operations (ITOps). However, they differ fundamentally in their purpose and level of specialization in AI and ML environments.

Big Data 106

Big Data Big Data ML ML 106

phData

OCTOBER 11, 2024

From credit card processing and insurance underwriting to retail banking, data is reshaping the way these organizations operate. By implementing AI applications effectively, financial services companies can navigate strict regulations while achieving meaningful, value-driven outcomes. This is where AI truly shines.

AI 52

AI AI Data Pipeline ML 52

Pickl AI

NOVEMBER 4, 2024

Effective data governance enhances quality and security throughout the data lifecycle. What is Data Engineering? Data Engineering is designing, constructing, and managing systems that enable data collection, storage, and analysis. They are crucial in ensuring data is readily available for analysis and reporting.

Data Engineering 52

Data Engineering Data Engineering Data Engineering Data Engineer 52

ODSC - Open Data Science

FEBRUARY 2, 2023

In the realm of data science, this entails becoming familiar with new frameworks and tools, seeing what’s trending in AI, and being able to adapt to changing business requirements. Cloud Services The only two to make multiple lists were Amazon Web Services (AWS) and Microsoft Azure.

Data Science 98

Data Science Data Scientist Computer Science Computer Science 98

ODSC - Open Data Science

JANUARY 18, 2024

This individual is responsible for building and maintaining the infrastructure that stores and processes data; the kinds of data can be diverse, but most commonly it will be structured and unstructured data. They’ll also work with software engineers to ensure that the data infrastructure is scalable and reliable.

Data Engineering 52

Data Engineering Data Engineering Data Engineering Data Engineer 52

phData

JULY 5, 2024

The release of ChatGPT in late 2022 introduced generative artificial intelligence to the general public and triggered a new wave of AI-oriented companies, products, and open-source projects that provide tools and frameworks to enable enterprise AI.

AI 52

AI AI Database AWS 52

Pickl AI

JULY 25, 2023

Data engineers are essential professionals responsible for designing, constructing, and maintaining an organization’s data infrastructure. They create data pipelines, ETL processes, and databases to facilitate smooth data flow and storage. Big Data Processing: Apache Hadoop, Apache Spark, etc.

Data Engineering 52

Data Engineering Data Engineering Data Engineering Data Engineer 52

DagsHub

OCTOBER 23, 2024

This article will discuss managing unstructured data for AI and ML projects. You will learn the following: Why unstructured data management is necessary for AI and ML projects. How to properly manage unstructured data. The different tools used in unstructured data management. What is Unstructured Data?

Machine Learning 52

Machine Learning Machine Learning Data Lakes AI 52

Data Science Dojo

JULY 3, 2024

Data Engineering : Building and maintaining data pipelines, ETL (Extract, Transform, Load) processes, and data warehousing. Artificial Intelligence : Concepts of AI include neural networks, natural language processing (NLP), and reinforcement learning.

Data Science 150

Data Science Machine Learning Machine Learning Data Visualization 150

ODSC - Open Data Science

FEBRUARY 6, 2024

Apache Kafka For data engineers dealing with real-time data, Apache Kafka is a game-changer. This open-source streaming platform enables the handling of high-throughput data feeds, ensuring that data pipelines are efficient, reliable, and capable of handling massive volumes of data in real-time.

Data Engineering 52

Data Engineering Data Engineering Data Engineering Data Engineer 52

DrivenData Labs

MAY 21, 2024

Hello from our new, friendly, welcoming, definitely not an AI overlord cookie logo! Data storage ¶ V1 was designed to encourage data scientists to (1) separate their data from their codebase and (2) store their data on the cloud. We have now added support for Azure and GCS as well.

Data Science 233

Data Science Python Data Scientist Data Warehouse 233

phData

NOVEMBER 28, 2023

In July 2023, Matillion launched their fully SaaS platform called Data Productivity Cloud, aiming to create a future-ready, everyone-ready, and AI-ready environment that companies can easily adopt and start automating their data pipelines coding, low-coding, or even no-coding at all.

Data Warehouse 52

Data Warehouse Data Pipeline ETL Azure 52

Pickl AI

JUNE 7, 2024

IBM Infosphere DataStage IBM Infosphere DataStage is an enterprise-level ETL tool that enables users to design, develop, and run data pipelines. Key Features: Graphical Framework: Allows users to design data pipelines with ease using a graphical user interface. Read Further: Azure Data Engineer Jobs.

ETL 40

ETL Data Quality Data Pipeline Data Warehouse 40

ODSC - Open Data Science

JANUARY 3, 2025

The AI Builders Summit is a four-week journey into the cutting-edge advancements in AI, designed to equip participants with practical skills and insights across four pivotal areas: Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), AI Agents , and the art of building comprehensive AIsystems.

AI 40

AI AI ML ML 40

phData

JANUARY 19, 2024

Matillion’s Data Productivity Cloud is a versatile platform designed to increase the productivity of data teams. It provides a unified platform for creating and managing data pipelines that are effective for both coders and non-coders.

Data Pipeline 52

Data Pipeline Data Warehouse ETL Azure 52

Pickl AI

OCTOBER 15, 2024

It supports both batch and real-time data processing , making it highly versatile. Its ability to integrate with cloud platforms like AWS and Azure makes it an excellent choice for businesses moving to the cloud. Apache Nifi Apache Nifi is an open-source ETL tool that automates data flow between systems.

ETL 52

ETL Azure AWS Data Governance 52

IBM Journey to AI blog

AUGUST 6, 2024

Today, all leading CSPs, including Amazon Web Services (AWS Lambda), Microsoft Azure (Azure Functions) and IBM (IBM Cloud Code Engine) offer serverless platforms. Specifically, serverless helps enable something called event-driven AI, where a constant flow of intelligence informs real-time decision-making capabilities.

Cloud Computing 45

Cloud Computing Internet of Things Big Data Big Data 45

phData

JULY 2, 2024

Snowpark, offered by the Snowflake AI Data Cloud , consists of libraries and runtimes that enable secure deployment and processing of non-SQL code, such as Python, Java, and Scala. Developers can seamlessly build data pipelines, ML models, and data applications with User-Defined Functions and Stored Procedures.

Python 52

Python SQL Data Pipeline ML 52

Mlearning.ai

JULY 11, 2023

This includes important stages such as feature engineering, model development, data pipeline construction, and data deployment. For example, when it comes to deploying projects on cloud platforms, different companies may utilize different providers like AWS, GCP, or Azure.

Data Science 40

Data Science Exploratory Data Analysis Azure Power BI 40

The MLOps Blog

FEBRUARY 5, 2023

This pipeline facilitates the smooth, automated flow of information, preventing many problems that enterprises face, such as data corruption, conflict, and duplication of data entries. A streaming data pipeline is an enhanced version which is able to handle millions of events in real-time at scale. Happy Learning!

Machine Learning 72

Machine Learning Machine Learning Data Pipeline Apache Kafka 72

Pickl AI

OCTOBER 17, 2024

Summary: This article explores the significance of ETL Data in Data Management. It highlights key components of the ETL process, best practices for efficiency, and future trends like AI integration and real-time processing, ensuring organisations can leverage their data effectively for strategic decision-making.

ETL 52

ETL Data Warehouse Data Quality Data Governance 52

phData

SEPTEMBER 2, 2024

Best practices are a pivotal part of any software development, and data engineering is no exception. This ensures the data pipelines we create are robust, durable, and secure, providing the desired data to the organization effectively and consistently.

ETL 52

ETL Data Warehouse SQL Database 52

The MLOps Blog

FEBRUARY 27, 2025

Picture this: youve spent months fine-tuning an AI-powered chatbot to provide mental health support. Now think about your work as an AI professional. The challenge for AI researchers and engineers lies in separating desirable biases from harmful algorithmic biases that perpetuate social biases or inequity.

AI 64

AI Machine Learning AI Machine Learning 64

Precisely

MAY 23, 2024

Whatever your approach may be, enterprise data integration has taken on strategic importance. Artificial intelligence (AI) algorithms are trained to detect anomalies. Today’s enterprises need real-time or near-real-time performance, depending on the specific application. Timing matters.

Data Governance 64

Data Governance ETL Data Pipeline Azure 64

DagsHub

DECEMBER 11, 2023

It does not support the ‘dvc repro’ command to reproduce its data pipeline. DVC Released in 2017, Data Version Control ( DVC for short) is an open-source tool created by iterative. However, these tools have functional gaps for more advanced data workflows. It provides both community and enterprise editions.

Machine Learning 98

Machine Learning Machine Learning Data Lakes Database 98

IBM Journey to AI blog

SEPTEMBER 4, 2024

Kafka helps simplify the communication between customers and businesses, using its data pipeline to accurately record events and keep records of orders and cancellations—alerting all relevant parties in real-time. Developers using Apache can speed app development with support for whatever requirements their organization has.

Apache Kafka 58

Apache Kafka Internet of Things Data Pipeline Clustering 58

phData

FEBRUARY 21, 2024

Matillion’s Data Productivity Cloud is a versatile platform designed to increase the productivity of data teams. It provides a unified platform for creating and managing data pipelines that are effective for both coders and non-coders. Please contact our team for assistance in accomplishing this goal.

Data Pipeline 52

Data Pipeline Azure Cloud Data Data Quality 52

DagsHub

APRIL 21, 2024

Especially in the current time when LLM models are making their way for several industry-based generative AI projects. PyTorch Developed by Facebook’s AI Research Lab (FAIR), PyTorch is a popular machine-learning framework that offers a flexible and dynamic approach to building and training neural networks.

Machine Learning 59

Machine Learning Machine Learning ML ML 59

Alation

APRIL 14, 2022

Data discovery is also critical for data governance , which, when ineffective, can actually hinder organizational growth. And, as organizations progress and grow, “data drift” starts to impact data usage, models, and your business. The Cloud Data Migration Challenge. Data pipeline orchestration.

Data Governance 59

Data Governance ML ML Cloud Data 59

Precisely

MAY 23, 2024

Whatever your approach may be, enterprise data integration has taken on strategic importance. Artificial intelligence (AI) algorithms are trained to detect anomalies. Today’s enterprises need real-time or near-real-time performance, depending on the specific application. Timing matters.

Data Governance 52

Data Governance ETL Data Pipeline Azure 52

The MLOps Blog

MARCH 20, 2023

However, in scenarios where dataset versioning solutions are leveraged, there can still be various challenges experienced by ML/AI/Data teams. Data aggregation: Data sources could increase as more data points are required to train ML models. Read more How to Version and Compare Datasets in neptune.ai

ML 59

ML ML Machine Learning Machine Learning 59

ODSC - Open Data Science

JANUARY 7, 2025

Deep Learning Deep learning is a cornerstone of modern AI, and its applications are expanding rapidly. Data Engineering Data engineering remains integral to many data science roles, with workflow pipelines being a key focus. Being familiar with the more popular tools, such as those offered by OpenAI, are expected.

Data Scientist 40

Data Scientist Data Science Machine Learning Machine Learning 40

phData

JANUARY 31, 2024

With Snowflake’s acquisition of Streamlit in 2022, Streamlit applications are now able to be hosted within your Snowflake environment, eliminating the need for extensive knowledge of Docker, Kubernetes, cloud platforms like AWS, GCP, or Azure, authentication and authorization patterns, etc.,

Machine Learning 52

Machine Learning Machine Learning Python Data Scientist 52