Kafka vs RabbitMQ: How to know which one to use
By Evan King
•
When one service needs to talk to another, the simplest option is a direct call. Your order service makes an HTTP request to your inventory service, waits for a response, and moves on. This works great until that inventory service is slow, or down, or getting hammered with traffic. Your order service ends up stuck waiting, timing out, or dropping requests on the floor because it has nowhere to put them.
Message queues solve this by adding a buffer between your services. Instead of calling the inventory service directly, your order service drops a message into a queue and returns immediately. The inventory service picks it up whenever it's ready. If there's a massive spike during a flash sale, the queue absorbs all that traffic and lets downstream services process it at whatever pace they can handle.
This is the foundation of event-driven architecture. Kafka and RabbitMQ both give you this decoupling: producers, consumers, broker in the middle. But how they actually work under the hood is completely different, and that's what should drive your decision.
What Is RabbitMQ
RabbitMQ is a traditional message broker, and it follows a mental model most engineers find intuitive if you've ever used a queue data structure.
A producer sends a message to the broker. The broker looks at routing rules you've configured to figure out which queue that message belongs in. A consumer pulls the message off the queue, processes it, and sends back an acknowledgment. Once acknowledged, RabbitMQ deletes the message.
The broker handles a lot for you. It routes messages to the right queues, tracks which messages have been delivered, and retries failed deliveries. When a message fails repeatedly, RabbitMQ moves it to a dead-letter queue automatically so you can debug it later. You configure the rules; the broker enforces them.
From your perspective as a consumer, this is simple. You connect to a queue, messages show up, you process them, you acknowledge. All the complexity of figuring out where messages came from and where they need to go is the broker's problem, not yours.
This model maps well to task-oriented workloads. Sending emails, processing payments, resizing images. You've got a unit of work that needs to happen, you drop it on a queue, something picks it up and does it.
What Is Kafka
Kafka works differently.
Instead of routing messages to queues and deleting them when they're consumed, Kafka is a distributed append-only log. When a producer sends a message, Kafka appends it to a topic. That message doesn't disappear when someone reads it. It sits there in the log, and depending on how you've configured retention, it might stay there for hours, days, weeks, or indefinitely.
Consumers read from this log, but they're responsible for tracking their own position. Kafka calls this an offset. When your consumer reads message number 500, it remembers that it's at position 500. If it crashes and restarts, it looks up where it left off and picks up from there. Need to reread messages from an hour ago? Rewind the offset.
RabbitMQ is a smart broker with simple consumers. It handles routing, tracks deliveries, manages retries. Kafka is a simple broker with smart consumers. It stores messages in order; consumers decide what to read and when.
The benefit is that messages become durable and replayable. Multiple consumer groups can read the same stream independently. Your analytics team can process the same events as your real-time notification system. A new service that comes online six months from now can read the entire history from day one.
The Core Difference
RabbitMQ is a message broker. Messages flow through it. Kafka is a log. Messages live in it.
In RabbitMQ, once a message is consumed, it's gone. In Kafka, messages persist and any number of consumers can read them, at any time, from any point in history. This one difference drives almost every other tradeoff between the two.
Because they solve different problems, many teams use both: Kafka as the durable event stream that multiple systems read from, RabbitMQ as the task queue that processes the work those events trigger.
Key Technical Differences
Message Retention
In RabbitMQ, messages are deleted after processing. Once consumed, they're gone. This is fine for task queues where you don't need history.
In Kafka, messages persist for a configurable retention period. Days, weeks, or indefinitely. This means you can replay events, rebuild state from scratch, and debug production issues by reprocessing the same data. If your order processing pipeline has a bug, you fix it and replay the affected events. You can add a new consumer that reads the entire history from the beginning.
Ordering Guarantees
Both systems preserve order, but differently.
RabbitMQ queues are strictly ordered. Messages come out in the order they went in. If you have a single consumer, you get perfect ordering. With multiple consumers pulling from the same queue, they process in parallel, so you trade ordering for throughput.
Kafka splits each topic into partitions. Order is guaranteed within a partition, but not across partitions. You control which partition a message goes to by its key. All orders for customer 12345 go to the same partition and get processed in order. Customer 67890's orders go to a different partition. Each customer's orders stay in sequence, but there's no global ordering across partitions.
The tradeoff: RabbitMQ gives you global ordering with a single consumer. Kafka gives you per-entity ordering with parallelism.
Throughput and Latency
RabbitMQ handles around 4,000 to 10,000 messages per second. Latency sits around 1-5 milliseconds for low-volume workloads because the broker pushes messages to consumers immediately. But the broker is doing a lot of work per message: tracking delivery state, managing acknowledgments, handling routing decisions. All that overhead adds up. As throughput increases, latency degrades.
Kafka handles over a million messages per second, roughly 100x more. Latency is higher at baseline, typically 5-50 milliseconds, because consumers pull messages in batches rather than receiving them immediately. The broker does far less work per message. It appends to a sequential log and lets consumers track their own position. Under heavy load, that simplicity pays off. Latency stays consistent even as volume grows.
Routing
Routing is first-class in RabbitMQ. Kafka leaves you on your own.
Say you have order events and different services care about different subsets. Fulfillment wants all orders. Fraud detection only wants orders over $500. Regional warehouses only want orders shipping to their region.
In RabbitMQ, you configure this in the broker. You set up routing rules, and the broker sends each message to the right queues based on its content. The producer just publishes; the broker decides where it goes.
In Kafka, you handle this yourself. Either you create separate topics for each destination and have producers write to all of them, or you have one topic and each consumer filters for what it cares about. More flexibility, but more work.
Delivery Guarantees
What happens when something fails mid-delivery? You have two basic options: lose the message or deliver it again and risk duplicates.
At-most-once means you might lose messages. The broker sends once and doesn't retry. Fast, but if something goes wrong, the message is gone.
At-least-once means you might get duplicates. The broker retries until it gets an acknowledgment. No data loss, but your consumer might see the same message twice.
Both RabbitMQ and Kafka support at-least-once, which is what most applications need.
Kafka also offers exactly-once semantics, but it's more limited than it sounds. It only works when both input and output are Kafka topics, within the same Kafka cluster, using Kafka transactions. The moment you write to a database, call an external API, or cross cluster boundaries, you're back to at-least-once. In practice, most applications still need idempotent consumers regardless.
Don't pick Kafka just because you heard "exactly-once" and thought it solves all your problems. In the real world, your consumers almost certainly write to something outside Kafka, which means you still need idempotency.
Operational Complexity
RabbitMQ is simpler to run. Single binary, straightforward clustering, built-in management UI. For a small team running a few queues, it's approachable.
Kafka is harder. Historically you needed ZooKeeper as a separate coordination service. Newer versions use KRaft mode which eliminates ZooKeeper, but you still need to manage partition rebalancing, broker failures, topic configuration, and consumer group coordination. There's more to learn and more that can go wrong.
Managed services change this calculus significantly. Confluent Cloud, Amazon MSK, or Azure Event Hubs handle most of the Kafka complexity for you. If you're choosing Kafka, strongly consider a managed service unless you have dedicated infrastructure expertise.
When to Use Each
Choose RabbitMQ When You Need
- Task queues and background jobs. Sending emails, processing payments, resizing images. Work goes in, gets done, message disappears.
- Smart routing. The broker decides which worker gets which message based on content.
- Low latency at moderate scale. Sub-5ms delivery when you're not pushing millions of messages.
- Simpler operations. Easier to set up, easier to reason about.
Instagram uses RabbitMQ to process photo uploads. When you post a picture, resizing and filtering happen via background workers pulling from RabbitMQ queues. Reddit uses it to build comment threads and calculate karma scores. These are classic task queue patterns.
Choose Kafka When You Need
- Multiple systems reading the same events. Analytics, fraud detection, billing, and audit logging all consuming one stream independently.
- Replay capability. Reprocess historical data to debug issues or rebuild state.
- Massive scale. Millions of events per second with consistent latency.
- Durable event history. A permanent record of everything that happened.
Netflix processes petabytes of data daily through Kafka for recommendations and billing. Uber uses it for real-time pricing and fraud detection across millions of rides. LinkedIn invented Kafka and still uses it to power their feeds and messaging. These systems need every event stored, replayable, and consumed by dozens of different services.
The Simple Heuristic
If you have a well-defined, constrained use case that fits RabbitMQ's model, like a task queue for background jobs, go with RabbitMQ. It's simpler to operate and the model is intuitive.
For everything else, lean toward Kafka. You get replay, scale, explicit consumer control, and a durable event history. The operational complexity is real, but managed services take most of that away. And once you're on Kafka, you won't run into a ceiling where you suddenly need features it doesn't have.
Or use both. Kafka as the event backbone, RabbitMQ for the background jobs those events trigger.
Mark as read
About The Author
Evan, Co-founder of Hello Interview and former Staff engineer at Meta, possesses a unique vantage point having been on both sides of the tech hiring process. With a track record of conducting hundreds of interviews and securing offers from top tech companies himself, he is now on a mission to help others do the same.
Your account is free and you can post anonymously if you choose.