Hacker News

DECEMBER 18, 2023

multi-protocol, event-native proxy. Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs. A multi-protocol, event-native proxy. Securely interface web apps, IoT clients, and microservices to Apache Kafka® via declaratively defined, stateless APIs.

Apache Kafka 111

Apache Kafka 111

IBM Journey to AI blog

MAY 9, 2023

There are thousands of business events flowing through every part of your organization every day. These events can provide a wealth of information about what’s actually happening across your business at any moment in time. But operationalizing business events is challenging.

Apache Kafka 110

Apache Kafka Business Intelligence Business Intelligence 110

IBM Journey to AI blog

SEPTEMBER 4, 2024

Apache Kafka is an open-source , distributed streaming platform that allows developers to build real-time, event-driven applications. With Apache Kafka, developers can build applications that continuously use streaming data records and deliver real-time experiences to users. How does Apache Kafka work?

Apache Kafka 58

Apache Kafka Internet of Things Data Pipeline Clustering 58

IBM Journey to AI blog

APRIL 2, 2024

A messaging queue technology is essential for businesses to stay afloat, but building out event-driven architecture fueled by messaging might just be your x-factor. However, IBM MQ and Apache Kafka can sometimes be viewed as competitors, taking each other on in terms of speed, availability, cost and skills.

Apache Kafka 75

Apache Kafka Clustering SQL AI 75

Smart Data Collective

AUGUST 17, 2022

You can safely use an Apache Kafka cluster for seamless data movement from the on-premise hardware solution to the data lake using various cloud services like Amazon’s S3 and others. 5 Key Comparisons in Different Apache Kafka Architectures. 5 Key Comparisons in Different Apache Kafka Architectures.

Apache Kafka 122

Apache Kafka ETL Data Lakes Clustering 122

ODSC - Open Data Science

MAY 31, 2023

Be sure to check out his talk, “ Apache Kafka for Real-Time Machine Learning Without a Data Lake ,” there! The combination of data streaming and machine learning (ML) enables you to build one scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem.

Data Lakes 131

Data Lakes Machine Learning Machine Learning Apache Kafka 131

IBM Journey to AI blog

JANUARY 8, 2024

In modern enterprises, where operations leave a massive digital footprint, business events allow companies to become more adaptable and able to recognize and respond to opportunities or threats as they occur. Teams want more visibility and access to events so they can reuse and innovate on the work of others.

EDA 92

EDA Apache Kafka Clustering Data Governance 92

IBM Journey to AI blog

NOVEMBER 29, 2023

By leveraging AI for real-time event processing, businesses can connect the dots between disparate events to detect and respond to new trends, threats and opportunities. AI and event processing: a two-way street An event-driven architecture is essential for accelerating the speed of business.

Artificial Intelligence 105

Artificial Intelligence Artificial Intelligence Apache Kafka AI 105

Towards AI

FEBRUARY 29, 2024

Within this article, we will explore the significance of these pipelines and utilise robust tools such as Apache Kafka and Spark to manage vast streams of data efficiently. Apache Kafka Apache Kafka is a distributed event streaming platform used for building real-time data pipelines and streaming applications.

Apache Kafka 142

Apache Kafka SQL Clustering Data Pipeline 142

Dataconomy

JULY 29, 2024

Hosted at one of Mindspace’s coworking locations, the event was a convergence of insightful talks and professional networking. Mindspace , a global coworking and flexible office provider with over 45 locations worldwide, including 13 in Germany, offered a conducive environment for this knowledge-sharing event.

Apache Kafka 91

Apache Kafka Data Pipeline Data Warehouse ETL 91

Data Science Dojo

MARCH 9, 2023

Challenges for individuals Traditional messaging brokers, such as Apache Kafka, RabbitMQ, and ActiveMQ, have been widely used to enable communication between applications and services. Additionally, there is a risk of message loss due to crashes, lack of retransmits, and poor monitoring.

Apache Kafka 213

Apache Kafka Azure Data Science Data Pipeline 213

IBM Journey to AI blog

NOVEMBER 21, 2023

Apache Kafka is a well-known open-source event store and stream processing platform and has grown to become the de facto standard for data streaming. Apache Kafka transfers data without validating the information in the messages. Event Streams on IBM Cloud provides a Schema Registry as part of its Enterprise plan.

Apache Kafka 116

Apache Kafka Clustering Data Quality Data Governance 116

Twilio Segment

SEPTEMBER 7, 2021

Event streaming platforms such as Apache Kafka are gaining in importance across all industries. In this article we'll discuss the benefits Apache Kafka implementations can gain from pairing it with a CDP.

Apache Kafka 40

Apache Kafka 40

Data Science Blog

JUNE 27, 2023

In this representation, there is a separate store for events within the speed layer and another store for data loaded during batch processing. It is important to note that in the Lambda architecture, the serving layer can be omitted, allowing batch processing and event streaming to remain separate entities.

Big Data 130

Big Data Big Data Apache Kafka Database 130

AWS Machine Learning Blog

MARCH 10, 2023

We show how you can apply this approach to various data streaming and event-driven architectures, depending on the desired outcome and actions to take to prevent fraud (such as alert the user about the fraud or flag the transaction for additional review). The user can then choose to take action to prevent further abuse.

Machine Learning 138

Machine Learning Machine Learning AWS Apache Kafka 138

IBM Journey to AI blog

MARCH 14, 2024

Apache Kafka stands as a widely recognized open source event store and stream processing platform. One key advantage of opting for managed Kafka services is the delegation of responsibility for broker and operational metrics, allowing users to focus solely on metrics specific to applications. Take the next step.

Apache Kafka 106

Apache Kafka Data Pipeline 106

IBM Journey to AI blog

AUGUST 29, 2024

Event-driven businesses across all industries thrive on real-time data, enabling companies to act on events as they happen rather than after the fact. This is where Apache Flink shines, offering a powerful solution to harness the full potential of an event-driven business model through efficient computing and processing capabilities.

Apache Kafka 45

Apache Kafka Hadoop ETL Data Pipeline 45

ODSC - Open Data Science

JUNE 1, 2023

To understand what it means, we should start by thinking of the world in terms of events, where an event is a thing that happens. And we are going to take those events, become aware of them, and understand them. Stores events in a durable manner so that downstream components can process them.

Analytics 98

Analytics Analytics Apache Kafka Data Science 98

IBM Journey to AI blog

JULY 12, 2024

IBM® Event Automation’s event endpoint management capability makes it easy to describe and document your Kafka topics (event sources) according to the open source AsyncAPI Specification. Why is this important? Our AsycnAPI applicability is broadening in our implementation.

Apache Kafka 40

Apache Kafka Clustering 40

Precisely

DECEMBER 28, 2023

Streaming data pipelines, by extension, offer an architecture capable of handling large volumes of data, accommodating millions of events in near real time. One very popular platform is Apache Kafka , a powerful open-source tool used by thousands of companies. How do streaming data pipelines work?

Data Pipeline 91

Data Pipeline Apache Kafka Big Data Big Data 91

Data Science Dojo

JULY 24, 2023

Unlike traditional batch processing, where data is processed in fixed intervals, stream processing enables organizations to gain insights and respond to events as they happen in real-time.

Big Data 195

Big Data Big Data Data Engineering Data Engineer 195

IBM Journey to AI blog

NOVEMBER 9, 2023

Apache Kafka is a high-performance, highly scalable event streaming platform. To unlock Kafka’s full potential, you need to carefully consider the design of your application. It’s all too easy to write Kafka applications that perform poorly or eventually hit a scalability brick wall.

Apache Kafka 78

Apache Kafka Algorithm Clustering 78

phData

MAY 3, 2024

How Snowflake Helps Achieve Real-Time Analytics Snowflake is the ideal platform to achieve real-time analytics for several reasons, but two of the biggest are its ability to manage concurrency due to the multi-cluster architecture of Snowflake and its robust connections to 3rd party tools like Kafka. Looking for additional help?

Apache Kafka 52

Apache Kafka Analytics Analytics ETL 52

ODSC - Open Data Science

FEBRUARY 6, 2024

Apache Kafka For data engineers dealing with real-time data, Apache Kafka is a game-changer. Interested in attending an ODSC event? Learn more about our upcoming events here. Each platform offers unique features and benefits, making it vital for data engineers to understand their differences.

Data Engineer 52

Data Engineer Data Engineering Data Engineering Data Engineering 52

AWS Machine Learning Blog

NOVEMBER 3, 2023

This process comprises two key components: event data and optical tracking data. Event data collection entails gathering the fundamental building blocks of the game. For the precision needed in shot speed calculations, we must ensure that the ball’s position aligns precisely with the moment of the event.

AWS 127

AWS Apache Kafka Data Scientist Data Science 127

AWS Machine Learning Blog

APRIL 19, 2023

Streaming ingestion – An Amazon Kinesis Data Analytics for Apache Flink application backed by Apache Kafka topics in Amazon Managed Streaming for Apache Kafka (MSK) (Amazon MSK) calculates aggregated features from a transaction stream, and an AWS Lambda function updates the online feature store.

ML 96

ML ML Apache Kafka SQL 96

AWS Machine Learning Blog

MARCH 30, 2023

How Keeper Efficiency is implemented This Bundesliga Match Fact consumes both event and positional data. Event data consists of hand-labelled event descriptions with useful attributes, such as shot on target. This frame is used to synchronize the event data with the positional data.

Machine Learning 100

Machine Learning Machine Learning AWS Apache Kafka 100

AWS Machine Learning Blog

SEPTEMBER 11, 2024

It initially sources input time series data from Amazon Managed Streaming for Apache Kafka (Amazon MSK) using this live stream for model training. He is a clean code advocate and a subject-matter expert on server-less and event-driven architecture. You can follow him on linkedin, syedfurqhan Nirmal Kumar is Sr.

AWS 119

AWS ML ML Apache Kafka 119

AWS Machine Learning Blog

JANUARY 6, 2023

As the users are interacting with TR’s applications, they generate clickstream events, which are published into Amazon Kinesis Data Streams. Then the events are ingested into TR’s centralized streaming platform, which is built on top of Amazon Managed Streaming for Kafka (Amazon MSK).

AWS 92

AWS Data Warehouse ML ML 92

ODSC - Open Data Science

JULY 22, 2023

Expo Hall ODSC events are more than just data science training and networking events. Thank you to everyone who attended for making this event possible, and showing once again why we do what we do — connecting the greater data science community together to push the industry forward. What’s next?

Apache Kafka 52

Apache Kafka Machine Learning Machine Learning Data Science 52

Precisely

SEPTEMBER 12, 2023

Precisely data integrity solutions fuel your Confluent and Apache Kafka streaming data pipelines with trusted data that has maximum accuracy, consistency, and context and we’re ready to share more with you at the upcoming Current 2023. Let’s cover some additional information to know before attending.

Data Silos 52

Data Silos Apache Kafka Data Pipeline Data Quality 52

Pickl AI

SEPTEMBER 18, 2024

It utilises Amazon Web Services (AWS) as its main data lake, processing over 550 billion events daily—equivalent to approximately 1.3 Data in Motion Technologies like Apache Kafka facilitate real-time processing of events and data, allowing Netflix to respond swiftly to user interactions and operational needs.

Big Data 52

Big Data Big Data Apache Kafka Big Data Analytics 52

ODSC - Open Data Science

MAY 24, 2023

Streaming Machine Learning Without a Data Lake The combination of data streaming and ML enables you to build one scalable, reliable, but also simple infrastructure for all machine learning tasks using the Apache Kafka ecosystem. Here’s what you can expect from ODSC Europe. Final ODSC Europe 2023 Schedule Released!

Data Lakes 52

Data Lakes ML ML Analytics 52

ODSC - Open Data Science

JULY 14, 2023

The session participants will learn the theory behind compound sparsity, state-of-the-art techniques, and how to apply it in practice using the Neural Magic platform.

Machine Learning 52

Machine Learning Machine Learning Apache Kafka Data Science 52

The MLOps Blog

FEBRUARY 5, 2023

Streaming data is a continuous flow of information and a foundation of event-driven architecture software model” – RedHat Enterprises around the world are becoming dependent on data more than ever. A streaming data pipeline is an enhanced version which is able to handle millions of events in real-time at scale.

Machine Learning 72

Machine Learning Machine Learning Data Pipeline Apache Kafka 72

Precisely

MARCH 16, 2023

Tools like Splunk, Elastic, and Apache Kafka play a central role in IT operations analytics (ITOA). Alongside a leading ITOA platform like Splunk, Elastic, or Kafka, Ironstream helps your organization improve security, lower costs, and pass compliance audits with flying colors.

Apache Kafka 52

Apache Kafka Analytics Analytics 52

Pickl AI

JULY 30, 2024

Guaranteed Delivery : NiFi ensures that data delivered reliably, even in the event of failures. It maintains a write-ahead log to ensure that the state of FlowFiles preserved, even in the event of a failure. Provenance Repository : This repository records all provenance events related to FlowFiles. Is Apache NiFi Easy to Use?

ETL 52

ETL Data Lakes Big Data Big Data 52