The MLOps Blog

JUNE 27, 2023

As you delve into the landscape of MLOps in 2023, you will find a plethora of tools and platforms that have gained traction and are shaping the way models are developed, deployed, and monitored. Open-source tools have gained significant traction due to their flexibility, community support, and adaptability to various workflows.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

Precisely

SEPTEMBER 12, 2023

The Precisely team is excited to be part of Confluent’s Current 2023 conference, September 26 & 27. As a proud member of the Connect with Confluent program , we help organizations going through digital transformation and IT infrastructure modernization break down data silos and power their streaming data pipelines with trusted data.

Data Silos 52

Data Silos Apache Kafka Data Pipeline Data Quality 52

ODSC - Open Data Science

DECEMBER 5, 2023

A new event to ODSC West 2023 were the Lightning talks, which saw a small group of victims (speakers) describe slides picked at random. While we may be done with events for 2023, 2024 is looking to be packed full of conferences, meetups, and virtual events. What’s next?

Data Science 98

Data Science Artificial Intelligence Artificial Intelligence Machine Learning 98

Precisely

JANUARY 9, 2023

So how should companies ensure they are able to make agile, and more confident, decisions in 2023 and beyond? The answer lies in fueling strategic business decisions with trusted data – leveraging high-integrity data that is consistent, accurate, and contextual.

Data Governance 81

Data Governance Data Quality Data Observability Data Pipeline 81

Precisely

JANUARY 19, 2023

In 2023 and beyond, we expect the open source trend to continue, with steady growth in the adoption of tools like Feilong, Tessla, Consolez, and Zowe. In 2023, expect to see broader adoption of streaming data pipelines that bring mainframe data to the cloud, offering a powerful tool for “modernizing in place.”

AWS 52

AWS Cloud Computing Data Pipeline Big Data 52

phData

JANUARY 10, 2024

Please spend a few minutes browsing the apps and tools available in the phData Toolkit today to set yourself up for success in 2023. Explore the phData Toolkit The post phData Toolkit December 2023 Update appeared first on phData. Be sure to follow this series for more updates on the phData Toolkit tools and features.

Data Warehouse 52

Data Warehouse Data Profiling Data Pipeline Database 52

Alation

APRIL 14, 2023

So, how can a data catalog support the critical project of building data pipelines? In his talk, Mitesh revealed that Alation delivers useful information about data via metadata, and explored why context is key to building reliable data pipelines.

Data Pipeline 52

Data Pipeline Data Warehouse Cloud Data ETL 52

ODSC - Open Data Science

FEBRUARY 2, 2023

The role of a data scientist is in demand and 2023 will be no exception. To get a better grip on those changes we reviewed over 25,000 data scientist job descriptions from that past year to find out what employers are looking for in 2023. However, each year the skills and certainly the platforms change somewhat.

Data Science 98

Data Science Data Scientist Computer Science Computer Science 98

phData

SEPTEMBER 7, 2023

We encourage you to spend a few minutes browsing the apps and tools available in the phData Toolkit today to set yourself up for success in 2023. Explore the phData Toolkit The post phData Toolkit August 2023 Update appeared first on phData. Be sure to follow this series for more updates on the phData Toolkit tools and features.

SQL 52

SQL Data Profiling Data Pipeline Database 52

phData

MARCH 1, 2023

This allows you to perform tasks such as ensuring data quality against data sources (once or over time), compare data metrics and metadata across environments, and create/manage data pipelines for all your tables and views. We look forward to hearing from all of you and to what 2023 brings!

SQL 52

SQL Data Pipeline Data Quality Database 52

phData

MARCH 31, 2023

Explore phData Toolkit The post phData Toolkit March 2023 Update appeared first on phData. We encourage you to spend a few minutes browsing the apps and tools available in the phData Toolkit today to set yourself up for success in 2022. Be sure to follow: this series for more updates on the phData Toolkit tools and features.

SQL 52

SQL Data Profiling Data Pipeline Database 52

phData

JULY 29, 2023

Operational Risks identify operational risks such as data loss or failures in the event of an unforeseen outage or disaster. Performance Optimization identify and fix bottlenecks in your data pipelines so that you can get the most out of your Snowflake investment.

SQL 52

SQL Database Data Pipeline 52

phData

JUNE 26, 2023

We encourage you to spend a few minutes browsing the apps and tools available in the phData Toolkit today to set yourself up for success in 2023. Explore phData Toolkit The post phData Toolkit June 2023 Update appeared first on phData. Be sure to follow: this series for more updates on the phData Toolkit tools and features.

SQL 52

SQL Data Profiling Data Pipeline Data Governance 52

ODSC - Open Data Science

FEBRUARY 24, 2023

Great Expectations provides support for different data backends such as flat file formats, SQL databases, Pandas dataframes and Sparks, and comes with built-in notification and data documentation functionality. At ODSC East 2023, we have a number of sessions related to data visualization and data exploration tools.

Exploratory Data Analysis 126

Exploratory Data Analysis Data Visualization Data Analysis Data Analysis 126

Alation

MAY 31, 2023

This year’s DGIQ West will host tutorials, workshops, seminars, general conference sessions, and case studies for global data leaders. DGIQ is June 5-9, 2023, at the Catamaran Resort Hotel and Spa in San Diego, just steps away from the Mission Bay beach. You can learn more about the event and register here.

Data Governance 52

Data Governance DataOps Data Pipeline Business Intelligence 52

Pickl AI

NOVEMBER 4, 2024

Effective data governance enhances quality and security throughout the data lifecycle. What is Data Engineering? Data Engineering is designing, constructing, and managing systems that enable data collection, storage, and analysis. The global data warehouse as a service market was valued at USD 9.06

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

DagsHub

SEPTEMBER 10, 2023

Intermediate Data Pipeline : Build data pipelines using DVC for automation and versioning of Open Source Machine Learning projects. For that, DagsHub added Audio capabilities, enabling you to see its spectrogram, wave, and even listen to it!

Machine Learning 52

Machine Learning Machine Learning Data Pipeline AI 52

Towards AI

OCTOBER 31, 2024

In marketing, for example, AI helps organizations extract actionable insights from vast data sets, leading to targeted campaigns and better customer engagement. Hype Cycle for Emerging Technologies 2023 (source: Gartner) Despite AI’s potential, the quality of input data remains crucial.

Data Quality 104

Data Quality Analytics Analytics Clean Data 104

Towards AI

MARCH 21, 2023

Last Updated on March 21, 2023 by Editorial Team Author(s): Data Science meets Cyber Security Originally published on Towards AI. Navigating the World of Data Engineering: A Beginner’s Guide. A GLIMPSE OF DATA ENGINEERING ❤ IMAGE SOURCE: BY AUTHOR Data or data? What are ETL and data pipelines?

Data Engineering 79

Data Engineering Data Engineering Data Engineering Data Engineer 79

Data Science Dojo

SEPTEMBER 14, 2023

Provide connectors for data sources: Orchestration frameworks typically provide connectors for a variety of data sources, such as databases, cloud storage, and APIs. This makes it easy to connect your data pipeline to the data sources that you need. It is known for its extensibility and modularity.

Data Pipeline 195

Data Pipeline Python Database AI 195

ODSC - Open Data Science

AUGUST 11, 2023

Orchestrating Data Assets instead of Tasks, with Dagster Sandy Ryza | Lead Engineer — Dagster Project | Elementl Asset-based orchestration works well with modern data stack tools like dbt, Meltano, Airbyte, and Fivetran, because those tools already think in terms of assets.

Data Science 52

Data Science Machine Learning Machine Learning ML 52

phData

MAY 28, 2024

Let’s explore more about what this all means to the data community. Coalesce is a technology platform that simplifies creating and maintaining data pipelines with an easy-to-use visual interface and customizable templates, reducing the need for extensive coding. What Does Coalesce Do?

Data Pipeline 52

Data Pipeline SQL Machine Learning Machine Learning 52

ODSC - Open Data Science

DECEMBER 14, 2023

Data Engineering vs Machine Learning Pipelines This tutorial explores the differences between how machine learning and data pipelines work, as well as what is required for each. Here are 7 AI trends that we think will define the landscape over the next year.

K-nearest Neighbors 97

K-nearest Neighbors AI AI Machine Learning 97

phData

NOVEMBER 28, 2023

In July 2023, Matillion launched their fully SaaS platform called Data Productivity Cloud, aiming to create a future-ready, everyone-ready, and AI-ready environment that companies can easily adopt and start automating their data pipelines coding, low-coding, or even no-coding at all.

Data Warehouse 52

Data Warehouse Data Pipeline ETL Azure 52

Flipboard

SEPTEMBER 25, 2023

Y Combinator Photo) Seattle-area startups that just graduated from Y Combinator’s summer 2023 batch are tackling a wide range of problems — with plenty of help from artificial intelligence. Neum AI at its core is an enabler for generative AI applications by helping connect data into vector databases and making it accessible for RAG.

Data Pipeline 169

Data Pipeline AI AI Natural Language Processing 169

phData

NOVEMBER 2, 2023

Managing data pipelines efficiently is paramount for any organization. The Snowflake Data Cloud has introduced a groundbreaking feature that promises to simplify and supercharge this process: Snowflake Dynamic Tables. If you’re looking to leverage the full power of the Snowflake Data Cloud, let phData be your guide.

Data Pipeline 52

Data Pipeline SQL Data Warehouse Data Engineering 52

Precisely

SEPTEMBER 18, 2023

Data observability is a key element of data operations (DataOps). It enables a big-picture understanding of the health of your organization’s data through continuous AI/ML-enabled monitoring – detecting anomalies throughout the data pipeline and preventing data downtime.

Data Observability 52

Data Observability Data Quality Data Pipeline DataOps 52

phData

MARCH 18, 2024

product_id product_name category price update_date 1 MacBook Air (15-inch, M2, 2023) Computers & Accessories $999 June 13, 2023 In the product table, we have a product, MacBook Air (15-inch, M2, 2023), and the price is $999 as of June 13, 2023. SCD Type 1 In this type, changes overwrite the existing data.

Data Warehouse 52

Data Warehouse SQL Data Pipeline Analytics 52

Towards AI

APRIL 4, 2023

Last Updated on April 4, 2023 by Editorial Team Introducing a Python SDK that allows enterprises to effortlessly optimize their ML models for edge devices. We sketch out ideas in notebooks, build data pipelines and training scripts, and integrate with a vibrant ecosystem of Python tools.

ML 96

ML ML Python Machine Learning 96

phData

SEPTEMBER 13, 2023

Third-Party Tools Third-party tools like Matillion or Fivetran can help streamline the process of ingesting Salesforce data into Snowflake. With these tools, businesses can quickly set up data pipelines that automatically extract data from Salesforce and load it into Snowflake. Need help ingesting data into Snowflake?

Tableau 52

Tableau Data Pipeline Data Silos Analytics 52

phData

FEBRUARY 8, 2024

Data Modeling, dbt has gradually emerged as a powerful tool that largely simplifies the process of building and handling data pipelines. dbt is an open-source command-line tool that allows data engineers to transform, test, and document the data into one single hub which follows the best practices of software engineering.

Data Models 52

Data Models Data Modeling Data Warehouse Database 52

Ocean Protocol

DECEMBER 27, 2022

Congratulations to all the winners and kudos to all the participants for the entries — We appreciate your submissions and look forward to seeing many more entries in 2023. Introducing the winners of the ETH price prediction Data Challenge: Edition 2!

Data Silos 52

Data Silos Data Scientist Data Pipeline Algorithm 52

Iguazio

JANUARY 22, 2024

You can watch the full talk this blog post is based on, which took place at ODSC West 2023, here. Feedback - ​Collect production data, metadata, and metrics to tune the model and application further, and to enable governance and explainability. The importance of data pipelines lies in the fact that data pipelines improve quality.

Data Pipeline 52

Data Pipeline ML ML Data Warehouse 52

PyImageSearch

MARCH 20, 2023

Jump Right To The Downloads Section Training and Making Predictions with Siamese Networks and Triplet Loss In the second part of this series, we developed the modules required to build the data pipeline for our face recognition application. Figure 1: Overview of our Face Recognition Pipeline (source: image by the author).

Deep Learning 109

Deep Learning Deep Learning Data Pipeline Python 109

Flipboard

FEBRUARY 13, 2023

Project Structure Creating Our Configuration File Creating Our Data Pipeline Preprocessing Faces: Detection and Cropping Summary Citation Information Building a Dataset for Triplet Loss with Keras and TensorFlow In today’s tutorial, we will take the first step toward building our real-time face recognition application. The dataset.py

Data Pipeline 145

Data Pipeline Deep Learning Deep Learning Python 145

ODSC - Open Data Science

FEBRUARY 14, 2023

Getting into Data Engineering Joe Reis | CEO | Ternary Data Perfect for those interested in getting into data engineering, this talk covers the key questions you should answer when embarking on a data engineering career. Joe Reis addresses the current economic climate in 2023 in particular.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

Ocean Protocol

JANUARY 17, 2024

We launched Predictoor and its Data Farming incentives in September & November 2023, respectively. Flows We released pdr-backend when we launched Predictoor in September 2023, and have been continually improving it since then: fixing bugs, reducing onboarding friction, and adding more capabilities (eg simulation flow).

Data Pipeline 52

Data Pipeline AI AI Analytics 52