Apache Kafka Primary Use Case

Harsha Ravnikar - PeerSpot reviewer
Senior Solutions Architect at Sysmex America, Inc.

Our primary use cases allow software developers, and application developers, the option to not have to code in their own logic for the retry mechanism. A lot of software, and applications, have this feature of retry built in some way or the other, but they all have some kind of a pre-alpha version of Kafka, up to a certain extent.

View full review »
AT
Group Manager at a media company with 201-500 employees

We have different use cases for different clients. For example, a banking sector client wants to use Apache Kafka for fraud detection and messaging back to the client. For instance, if there's a fraudulent swipe of a credit or debit card, we stream near real-time data (usually three to five minutes old) into the platform. Then, we use a look-up model to relay the data to the messaging queue to the customers. This is just one use case.

We also have data science engineers who use Kafka to feed on the data (usually within the last five to seven minutes of transactions) to detect any fraudulent transactions for internal consumption of the bank. This is not for relaying back to the customer. This client is based in the Middle East, and they have some interesting use cases.

View full review »
Amit Laddha - PeerSpot reviewer
Vice President (Information and Product Management) at Tradebulls Securities (P) Limited

My company uses Apache Kafka to keep some intermediate data in the workflow.

View full review »
Buyer's Guide
Apache Kafka
April 2024
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,415 professionals have used our research since 2012.
AbhishekGupta - PeerSpot reviewer
Engineering Leader at Walmart

Apache Kafka can be deployed on the cloud and on-premise.

We use Apache Kafka internally to build a service on a cluster. Additionally, we use the intermediate persistence layer for events. There are many teams who leverage it as a message queue and further their microservice connections.

View full review »
Reza Sadeghi - PeerSpot reviewer
Software Development Team Lead at asa com

We use Kafka daily for our messaging queue to reduce costs because we have a lot of consumers, producers, and repeat messages. Our company has only one system built on Apache Kafka because it's based on microservices, so all of the applications can communicate using it.

View full review »
Jhon Rico - PeerSpot reviewer
Senior Solutions Architect at BVC

I have previous professional experience using Kafka to implement a system related to gathering software events in one centralized location. 

View full review »
Pratul Shukla - PeerSpot reviewer
Software Engineer at a financial services firm with 10,001+ employees

We have multiple use cases for our Kafka system. One is Kafka Connect, which is used to facilitate communication between different regions with Grid Deal. Another is to distribute events and projects to multiple downstream. We publish all the messages to Kafka and other listeners subscribe and write them to different MQs. Lastly, Kafka Connect is used especially for inter-application communication.

View full review »
MB
Architect at Agence Française de Développement

We use Kafka for Elastic Stack and Kafka SCRAM login.

I have many users of Apache Kafka. It's like a subject to study in enterprises. However, we have not decided if the systems should generalize Apache Kafka for every application and every IT system.

View full review »
Nor EL MALKI - PeerSpot reviewer
Project Manager at Leyton & Associés, SAS

We have a scalable architecture where we need multiple workers to handle some processing. To make it possible, the backend catches the request and puts it in a common medium, which is the queue of Apache Kafka. The workers then can share and process it.

View full review »
SP
CEO at a comms service provider with 11-50 employees

We deploy it for our customers. The main use case is related to log management and metrics because we are a partner of Elastic Stack, and we usually collect information through Kafka.

View full review »
GT
Lead Architect at a financial services firm with 1,001-5,000 employees

We use it extensively in our data pushing, for analytics and all of this type of data that is pushed, rather than on a real-time and payment basis. However, we are using it for offline messages, pushing it for processing, and for heavy, heavy usage, rather than extensively using it for financial data.

View full review »
Mukulit Bhati - PeerSpot reviewer
CTO at InsightGeeks Solutions Pvt.

We use Apache Kafka for patching real-time data that we receive over a data transport layer and for putting the data into Apache Kafka. From Apache Kafka, we use several applications to subscribe to topics from different applications that we serve directly to browsers. Additionally, we use these applications inside our solution and have Apache Kafka Stream, which is connected to MongoDB.

Since we receive data in real-time consisting of IoT devices, running vehicles, their locations, their states, and their VNs, the solution is helpful.

View full review »
Stuart-Cook - PeerSpot reviewer
CEO & Founder at a tech consulting company with 11-50 employees

We used Kafka as a central message bus, transporting data from SNMP through to a database. Some of the processing in between was handled by other components.

View full review »
Rémy NOLLET - PeerSpot reviewer
Data Exchange Architect MQSeries at Decathlon International

We utilize Apache Kafka in several areas, including financials, logistics, and client management to name a few.

View full review »
Silvio Lucas Pereira Filho - PeerSpot reviewer
Senior Tech Lead at RecargaPay

We are using Apache Kafka to extract data from a Portuguese data source, utilizing an open-source project for data capture. The connector for this project is linked to both Kafka and Confluence platforms. We then transform the extracted data and store it in Elasticsearch.

View full review »
AX
Senior Developer at a financial services firm with 10,001+ employees

I primarily use Kafka in the investment banking sector to update prices and inform clients of updates.

View full review »
Joaquin Marques - PeerSpot reviewer
CEO - Founder / Principal Data Scientist / Principal AI Architect at Kanayma LLC

My primary use case for Apache Kafka is replacing ETL and doing data transformations.

View full review »
TD
Head of Technology - Money Movement Platform at a financial services firm with 10,001+ employees

We use Apache Kafka primarily to queue the transactions or total the transactions.

View full review »
AV
Technical Director at Metrofibre Networx

We have a camera monitoring security system, in which we post messages onto the queue, which involves various steps in processing the message, like checking for the number of clients, running it against the police data, etc. So Apache Kafka is a security application with many types of consumers. We set up a workflow system with different sites, which works well.

View full review »
RG
Senior Technology Architect at a tech services company with 10,001+ employees

We use Apache Kafka for financial purposes. Every time one of our subscribed customers is due for an insurance payment, Apache Kafka sends an automated notification to the customer to let them know that their bill is due.

View full review »
Felipe Lopes - PeerSpot reviewer
Engineering Manager at Alice

The primary use case of the solution is for asset communication through our microservices.

View full review »
NK
Director at Tibco

We have got this product, which is meant for integration. So our use cases are essentially integrating with other systems, using any messaging stack. We use these products in Dev and QA and we have connectors for various different messaging applications. Apache Kafka just happens to be one of the messaging applications that we connect with. We also have our own messaging, it's called Enterprise Messaging Server and Rendezvous, we connect to those also. Our product is essentially used for integration. So we connect to almost all messaging applications.

View full review »
JA
Technical Lead at Interface Fintech Ltd

We use an open-source version of this solution, and we have two deployments of it. One is on-prem, and the other is in the cloud. We use the on-prem version to aggregate our logs. We use the cloud version to manage queues for financial services. 

View full review »
SK
Barista Brewing Espresso at Linkedln

Our primary use case of this solution is for data integration and for real-time data consumption. I'm a senior staff engineer for data and infrastructure and we are customers of Apache. 

View full review »
Salvatore Campana - PeerSpot reviewer
CEO & Founder at XAUTOMATA TECHNOLOGY GmbH

We use Apache Kafka to ingest a lot of data in real time that Apache Spark processes, and the result is used for a tech decision in real time – in the IT environment, infrastructure environment, and IOT environment, like for a  manufacturing plant.

This is an open-source framework. We also sell professional services on this solution and specifically create a business application for customers. 

The application is called Sherlogic. We have two kinds of customers. We have end-user customers that use the Sherlogic solution, and maybe customers don't know that there is Spark and Kafka in Sherlogic. But we have another kind of customer that uses professional services by Xautomata to create tailor-made applications in analytics and the automation process.

We use Apache Kafka for our digital cloud.

View full review »
ME
CTO at Estrada & Consultores

We primarily use the solution for upstreaming messages with different payload for our applications ranging from iOT, Food delivery and patient monitoring. 

For example for one solution we have a real-time location finding, whereby a customer for the food delivery solution wants to know, where his or her order is on a map. The delivery person's mobile phone would start publishing its location to Kafka, and then Kafka processes it, and then publishes it to subscribers, or, in this case, the customer. It allows them to see information in real-time almost instantly.

View full review »
DZ
Enterprise Architect at Smals vzw

Apache Kafka is used for more than only a messaging bus but also served as a database to store information. It functioned as a streamer, similar to ETL, to manipulate and transform events before migrating them to other systems for use. The database could also act as a cache. Apache Kafka is used as a database broker, streamer, and source of truth for multiple systems due to its ability to maintain events for at least 10 days. It provided both synchronous and asynchronous communication, making it a complex system that would be easier to understand through diagrams or sketches.

We use reactive frameworks.

View full review »
Ravi Kuppusamy - PeerSpot reviewer
CEO and Founder at BAssure Solutions

We are building solutions on Apache Kafka for four customers. The customers we have are in various sectors, such as healthcare and architecture.

View full review »
KS
Solution Architect at a manufacturing company with 10,001+ employees

I am a solution architect and I used Apache Kafka in this role.

View full review »
Paul Adams - PeerSpot reviewer
Consultant Solution Architect at a tech services company with 51-200 employees

We had an application stack consisting of Salesforce frontend and a Commander VPN position management system and used Apache Kafka to decouple the microservices. Additionally, we planned to use Kafka for stream processing and to use event sourcing to pull data from legacy systems and reference data to form a compacted topic that the microservices could consume.

The usage of Kafka is a combination of deploying on a personal Kubernetes cluster or using a managed service such as MSK. However, most people who use Kafka are using a managed service provided by Confluent. It can be deployed on the cloud or on-premise.

View full review »
RP
Assistant Professor at CHAROTAR UNIVERSITY OF SCIENCE AND TECHNOLOGY

We are in the early stages of testing this solution in our lab as a demo. It is in development and we are not in production at this point.

We are using this solution to relay events when they happen to multiple receivers at once to allow better functionality.

View full review »
JB
Software Support & Development Engineer at a computer software company with 501-1,000 employees

Apache Kafka is used for connecting components between each other in the same application. The use is quite limited, but I was curious about its filtering capability of it.

View full review »
DP
Sr Technical Consultant at a tech services company with 1,001-5,000 employees

One of our clients needed to take events out of SAP to stream them through Apache Kafka while applying data enrichment before reaching the consumers.

View full review »
LP
Owner at Binarylogicworks.com.au

I am a solution architect and this is one of the products that I implement for my customers.

Kafka works well when subscribes want to stream data for specific topics.

View full review »
TM
Building Event-centric Data processing Architectures at a tech services company with 51-200 employees

We use the solution for analytics for streaming. We also use it for fraud detection.

View full review »
TM
Assistant Student at a retailer with 5,001-10,000 employees

I have a lot of messages, and we need to process those messages from many clients. Each client takes those messages and processes them.

I'm using the brokerage partner. I'm not storing or maintaining the application on servers. I'm just a client for the Apache Kafka server.

The solution is deployed on-prem.

View full review »
Abdul-Samad - PeerSpot reviewer
Software Engineer at a tech services company with 201-500 employees

I use Kafka to send network packets from different sources to my cluster. We have around 10 users at my company.

View full review »
Guirino Ciliberti - PeerSpot reviewer
Data Governance & Lineage Product Manager at Primeur

Our primary use case for this solution is streaming.

View full review »
DS
Principal Technology Architect at a computer software company with 5,001-10,000 employees

It's a combination of an on-premise and cloud deployment. We use AWS, and we have our offshore deployment that's on-premise for OpenShift, Red Hat, and Kafka. Red Hat provides managed services and everything. We use Kafka and a specific deployment where we deploy on our basic VMs and consume Kafka as well.

We publish or stream all our business events as well as some of the technical events. You stream it out to Kafka, and multiple consumers develop a different set of solutions. It could be reporting, analytics, or even some data persistence. Later, we used it to build a data lake solution. They all would be consuming the data or events we are streaming into Kafka.

View full review »
MN
Project Engineer at Wipro Limited

We primarily use the solution for big data. We often get a million messages per second, and with such a high output we use Kafka to help us handle it. 

View full review »
JL
Technical Consultant at KPMG

It's convenient and flexible for almost all kinds of data producers. We integrated it with Kafka Streams, which can perform some easy data processing, like summary, count, group, etc

View full review »
ShoaibKhan - PeerSpot reviewer
Technical Specialist at APIZone

This is a system for email and other small devices. There has been a relay of transactions continuously over the last two years it has been in production.

View full review »
YL
Vice President at a consultancy with 51-200 employees

We use Apache Kafka for our messaging. 

We publish a message and ask the subscriber to listen to it. We use it to save events generated by integration with external systems. There are external events, that are first published to our Kafka queue, and then to a topic, and then we save it to our own data storage system.

View full review »
RABBAHMahmoud - PeerSpot reviewer
Senior Technical Architect at RABBAH SOFT

I'm a software architect. The use case will depend my customers. They usually use it for data transfer from static files to a legacy system.

View full review »
Sreekar Nethagani - PeerSpot reviewer
Co-Founder at Attaika

We are a service implementer and we supply this solution to our customers. I'm a company co-founder and we are customers of Apache. 

View full review »
NC
Chief Technology Officer at a tech services company with 1-10 employees

Our primary use case is based on the writing microservices, event architecture and using Kafka as an event bus. We work on distribution - enterprise-grade - and we design, develop and deploy in a confluent environment. We are customers of Kafka and I'm the chief technology officer.

View full review »
JJ
Technology Lead at a tech services company with 10,001+ employees

Our company provides services and we use Apache Kafka as part of the solution that we provide to clients.

One of the use cases is to collect all of the data from multiple endpoints and provide it to the users. Our application integrates with Kafka as a consumer using the API, and then sends information to the users who connect. 

View full review »
Andrea Castorino - PeerSpot reviewer
Program Manager at SirfinPA

We deployed this solution in a project for one of our customers to synchronize the different applications; to transport information from one application to another. I'm a program manager and we are customers of Apache.

View full review »
AM
IBMi/MIMIX Administrator at Arab Bank

We use Kafka for event monitoring.

View full review »
OT
Senior Big Data Developer | Cloudera at Dilisim

We are currently using this solution on our cloud-based clusters.

View full review »
MS
Senior Consultant at a tech services company with 51-200 employees

Apache Kafka is used for stream processing, metric and log aggregations, and as a message queue for connecting different microservices.

View full review »
RH
freelance at a tech services company with 11-50 employees

I am a user, as well as an integrator for our clients. This is one of the products that we implement for others.

View full review »
it_user998961 - PeerSpot reviewer
Enterprice Architect

I am an enterprise architect involved in Big Data and integration projects using Apache Kafa. We use it for integrating our different management systems.

View full review »
WG
Solutions Architect at a tech services company with 201-500 employees

We are a solution provider and Apache Kafka is being used in our client's company.

View full review »
Buyer's Guide
Apache Kafka
April 2024
Learn what your peers think about Apache Kafka. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,415 professionals have used our research since 2012.