AWS Machine Learning Blog

APRIL 18, 2025

Solution overview: Build a generative AI stock price analyzer with RAG For this post, we implement a RAG architecture with Amazon Bedrock Knowledge Bases using a custom connector and topics built with Amazon Managed Streaming for Apache Kafka (Amazon MSK) for a user who may be interested to understand stock price trends.

Apache Kafka 94

Apache Kafka AWS Clustering Database 94

Towards AI

FEBRUARY 29, 2024

Within this article, we will explore the significance of these pipelines and utilise robust tools such as Apache Kafka and Spark to manage vast streams of data efficiently. Apache Kafka Apache Kafka is a distributed event streaming platform used for building real-time data pipelines and streaming applications.

Apache Kafka 140

Apache Kafka SQL Clustering Data Pipeline 140

AWS Machine Learning Blog

FEBRUARY 7, 2025

It is backed by Amazon Managed Streaming for Apache Kafka (Amazon MSK) (8). The transformer gets a CloudEvent with the reference of the image Amazon S3 path, downloads it, and performs model inference over HTTP. The resources in the Kubernetes cluster are deployed in a private subnet.

Analytics 73

Analytics Analytics AWS Clustering 73

phData

MAY 3, 2024

How Snowflake Helps Achieve Real-Time Analytics Snowflake is the ideal platform to achieve real-time analytics for several reasons, but two of the biggest are its ability to manage concurrency due to the multi-cluster architecture of Snowflake and its robust connections to 3rd party tools like Kafka. Looking for additional help?

Apache Kafka 52

Apache Kafka Analytics Analytics ETL 52

AWS Machine Learning Blog

OCTOBER 9, 2024

Customers can use the CloudFormation template to bring up an application stack that receives time-series data from an Amazon Managed Streaming for Apache Kafka (Amazon MSK) streaming source and performs near-real-time anomaly detection in the streaming data. How can I export anomalies data before deleting the resources?

AWS 99

AWS ML ML Data Quality 99

Pickl AI

JULY 29, 2024

Download and extract the Apache Hadoop distribution on all nodes. The open-source software is also free to download and use. Although tools like Apache Kafka and Apache Spark can integrate with Hadoop for real-time processing, managing these additional components can add complexity to the architecture.

Hadoop 52

Hadoop Clustering Big Data Big Data 52

The MLOps Blog

FEBRUARY 5, 2023

For example, before any video streaming services, users had to wait for videos or audio to get downloaded. There are a number of tools that can help with streaming data collection and processing, some popular ones include: Apache Kafka : An open-source, distributed event streaming platform that can handle millions of events per second.

Machine Learning 72

Machine Learning Machine Learning Data Pipeline Apache Kafka 72