Event-driven architecture connects producers and consumers through messages describing changes. It can decouple workloads and absorb bursts, but shifts complexity into contracts, consistency and recovery. Apache Kafka fits retained streams and independent consumers; Azure Service Bus fits brokered business work; Azure Event Grid fits reactive event routing. Consider Azure Event Hubs for Azure-managed streaming with Kafka-protocol access. Choose delivery and ownership semantics before comparing product names.
What is event-driven architecture?
A producer records something that happened, messaging infrastructure transports or retains it, and a consumer decides how to respond. An event such as OrderAccepted states a fact. A command such as ReserveInventory asks an identified capability to do work. A message is the transport envelope for either. Calling everything an event can hide who is responsible for accepting or rejecting a request.
A synchronous checkout response answers whether the order was accepted. Asynchronous consumers can subsequently arrange fulfillment, analytics or notifications. Do not return “payment completed” merely because a payment command entered a queue. The user-facing state must distinguish accepted, pending, completed and failed outcomes. A durable broker acknowledgment is evidence of one handoff, not completion of the entire business transaction.
How the event lifecycle works
Define an envelope with event ID, event type, schema version, source, occurrence time, correlation ID and a minimal payload or protected reference. Event time describes the business occurrence; processing time describes when infrastructure saw it. Consumers may observe older events after newer ones. Include an entity version where ordering matters and decide how a consumer handles a gap or stale update.
Loose coupling does not mean no dependency. Consumers depend on the meaning and stability of the contract. Make additive changes where possible, version incompatible semantics and test older consumers. Eventual consistency requires a visible pending state, a reconciliation mechanism and a recovery owner. A dashboard that shows “queued” forever is not a complete workflow.
Why enterprises choose it—and what they take on
| Desired outcome | Design mechanism | Countervailing cost |
|---|---|---|
| Absorb bursts | Durable queue and bounded consumers | Backlog and recovery capacity must be monitored |
| Independent consumers | Subscriptions or consumer groups | Contract evolution and access governance |
| Extend a business process | Add a new subscriber | Duplicate or conflicting side effects |
| Replay analytics | Retain stream and checkpoints | Retention cost and replay-safe destinations |
| Isolate transient failure | Retry with backoff and exception handling | Stale work and operational triage |
Use asynchronous communication when the business tolerates delayed completion or when independent processing is useful. A small request that needs an immediate answer can remain synchronous. Moving it through several brokers can add delay without reducing meaningful risk. Trace a real failure scenario before introducing infrastructure: which dependency can fail, how long can work wait, and who can safely resubmit it?
Apache Kafka explained
Kafka organizes records into partitioned topics. Producers write records; consumers track their progress using offsets. Independent consumer groups can read the same retained stream at their own pace. Within a group, partition assignment controls parallelism; choosing a record key influences which records share an ordering boundary. Retention and compaction policies determine what remains available for later reads. Reference: Apache Kafka design.
Ordering is partition-scoped, not a total order across the business. A hot key can become a bottleneck even when other partitions are idle. Adding partitions can affect key distribution, so treat partitioning as a contract decision. Replay requires both retained records and a destination that can tolerate reprocessing; resetting an offset does not undo external writes.
Kafka producer idempotence and transactions have a defined scope. Exactly-once stream-processing behavior does not automatically cover an external ERP, email service or payment gateway. For those destinations, use business idempotency, atomic writes where available and reconciliation. Managed Kafka can reduce cluster work, but the team still owns topic policies, client settings, schema changes, lag and incident response.
Azure Service Bus explained
Service Bus provides queues for brokered work and topics/subscriptions for publish-subscribe delivery. Receivers settle messages after processing under the selected receive mode. Sessions can coordinate related messages when supported and configured; duplicate detection uses its configured identity/window. Dead-letter queues hold messages needing a separate operational path. Transactions are bounded by documented service behavior, not arbitrary distributed application state. See Service Bus overview.
For order processing, acknowledge only after the intended durable work is recorded. If an external charge succeeded but message completion failed, delivery can repeat: the payment operation needs a stable business key. Monitor lock expiry and long-running handlers. Do not turn a dead-letter queue into permanent storage; record the reason, correct the underlying issue, and resubmit through an authorized, observable recovery procedure.
Azure Event Grid explained
Event Grid routes matching events to interested subscribers. For classic push delivery, events can arrive more than once and out of order; retries and optional dead-lettering follow documented policy. Its dead-letter destination is not the same operational abstraction as a Service Bus dead-letter queue. Reference: Event Grid delivery and retry.
Choose the Event Grid resource and delivery model explicitly. Namespace HTTP capabilities, pull delivery and MQTT workloads have different configuration and support boundaries from classic system/custom topic push delivery. Do not take a feature from one model and assume it applies to all Event Grid deployments. A storage-created notification is a useful trigger, but downstream code must still validate the object, permissions and content before processing.
Kafka vs Service Bus vs Event Grid: detailed comparison
| Dimension | Kafka | Service Bus | Event Grid |
|---|---|---|---|
| Purpose | Retained event streams | Enterprise brokered messaging | Reactive event distribution |
| Model | Partitioned log | Queues/topics/subscriptions | Routing and subscription matching |
| Delivery | Consumer fetch and offsets | Receiver/settlement model | Selected push/pull model |
| Scale | Partitions, brokers and consumer assignments | Tier/capacity and entity design | Resource/delivery-model limits |
| Order | Within a partition | Sessions where applicable | Classic push not ordered |
| Retention | Configured topic retention/compaction | Message lifetime and settlement | Delivery retention/policy, not a Kafka log |
| Replay | Offsets within retained history | Design explicit recovery/republication | Do not assume log-style arbitrary replay |
| Semantics | Client/transaction configuration dependent | Duplicate-safe processing still needed | Duplicate/out-of-order handling needed |
| Failure | Retry topics/consumer recovery as designed | Dead-letter entity and operator process | Configured dead-letter destination/policy |
| Transactions | Kafka-scoped transaction semantics | Documented broker transaction scope | Not arbitrary business transactions |
| Consumers | Independent groups and checkpoints | Competing receivers/subscriptions | Filtered subscribers |
| Azure integration | Connectors/clients or managed deployment | Native Azure SDKs and integrations | Azure system and supported custom events |
| Operations | Cluster/service plus stream governance | Managed service plus entities and handlers | Managed routing plus delivery endpoints |
| Security | Authentication, ACLs and transport settings | Identity, roles and access scope | Publisher/subscriber authorization |
| Cost | Capacity, storage, transfer, operations | Tier, operations, network and support | Operations, delivery and dependencies |
| Best fit | Telemetry/replay/multiple analytic readers | Orders, commands and reliable business work | Resource events and reactive automation |
Throughput is not a comparable single number without payload, partitioning, tier, retention, consumer count and downstream limits. Benchmark an accepted business workload, including recovery. Price the same availability and data-retention requirements. Microsoft’s messaging-service comparison is a starting map, not evidence that one service replaces every other.
What about Azure Event Hubs?
Event Hubs provides managed event ingestion/streaming and a Kafka-protocol endpoint for supported clients. That can reduce cluster administration when Azure-native streaming fits the workload. Protocol compatibility does not imply every Kafka feature, administrative API, extension or configuration behaves identically. Verify your client version, authentication, consumer behavior, transactions and required APIs against the chosen service tier. See Event Hubs Kafka support.
A retained telemetry stream has different semantics from an order-work queue. Event Hubs can serve independent analytic consumers while Service Bus coordinates transactional work. Avoid choosing between them by asking only which is “faster.” Ask whether the primary requirement is retaining and replaying an observation stream or ensuring owned work reaches an accountable processor.
Which platform should you choose?
- Need retained history, independent consumers and replay? Evaluate Kafka or Event Hubs.
- Need commands, competing workers and business-message settlement? Evaluate Service Bus.
- Need filtered reactions to resource or application changes? Evaluate the appropriate Event Grid model.
- Need immediate answer with little independent work? A synchronous API may be simpler.
- Need multiple patterns? Assign one responsibility to each service and price the extra boundary.
Three illustrative enterprise reference architectures
E-commerce orders: Service Bus
An order API validates the request and commits order state plus an outbox record. A relay publishes accepted work. A payment worker uses a payment idempotency key; inventory and fulfillment consume the relevant accepted events. Preserve an explicit order state machine so a delayed consumer does not create contradictory status. Dead-letter reasons and reconciliation identify work that needs intervention. Change the architecture if a synchronous reservation is essential to accepting the order, rather than pretending eventual consistency meets that contract.
Industrial telemetry: Kafka or Event Hubs
Gateways buffer timestamped readings and publish with an asset-based partition key. Separate consumer groups support alerting, history and analytics. Track event-time lateness, sensor quality, lag and retention coverage. A replay used to rebuild analytics must not trigger real-world maintenance actions again. High event volume alone does not justify a cluster: assess managed streaming compatibility and the operating team before committing.
Storage automation: Event Grid and Functions
A storage event triggers a handler that validates the object reference, deduplicates the event, scans the file and records processing status. Quarantine invalid content and route exhausted failures through a monitored recovery path. Do not trust an event payload as authorization or assume notification order equals object version order. Move expensive or long work to a suitable queue/job contract when the handler should acknowledge quickly.
Hybrid architecture and production best practices
A domain service can publish accepted business events to a work broker and separately project a governed subset into a stream for analytics. Event Grid may route infrastructure notifications that feed a processing pipeline. Keep identifiers and semantic versioning consistent across the bridge. Avoid building three copies of authoritative state, and make the bridge’s lag and failure behavior visible.
- Commit database state and outbox together where the persistence model allows; relay with deduplication.
- Use stable event IDs and business operation keys; distinguish a repeated message from a legitimate second order.
- Bound retries with backoff and jitter; separate transient, validation and authorization failures.
- Test schema compatibility, missing fields, poison messages and consumers on older versions.
- Propagate trace/correlation identifiers while minimizing personal or sensitive payloads.
- Define retention, recovery point, replay procedure, permissions and deletion responsibilities.
- Monitor lag, backlog age, terminal failures, successful outcomes and recovery workload.
Common mistakes and incremental .NET modernization
| Mistake | Correction |
|---|---|
| Kafka for one simple job | Evaluate an owned queue consumer first |
| Global-order assumption | State partition/session/entity boundaries |
| Unlimited retry | Bound attempts and provide triage |
| Exactly-once slogan | Scope guarantees and protect external writes |
| No correlation | Connect producer state, message and outcome |
| Events everywhere | Keep synchronous interactions that fit the business |
For an existing .NET Framework or ASP.NET Core system, start at one stable business boundary. Introduce an outbox or durable handoff, build one duplicate-safe consumer and observe it in shadow or controlled production scope. Preserve the existing system of record during transition. Test a crash after commit but before publish, and after the side effect but before acknowledgment. These two gaps reveal more than a happy-path demo. Do not require a full rewrite or microservices migration to adopt one useful asynchronous capability.
Final recommendation
Select the smallest messaging design that meets delivery, retention, identity and recovery requirements. A queue, log and event router solve overlapping but different problems. Document the business completion contract and recovery owner before implementation; then measure accepted throughput and total operating cost rather than a vendor headline. Related reading: microservices and deployment boundaries.
Frequently asked questions
What is event-driven architecture?
It is a design in which producers publish changes and independent consumers react through messaging infrastructure. Its value comes from explicit asynchronous boundaries, not simply installing a broker. Define event meaning, delayed consistency, delivery behavior and accountable recovery so the business knows when work is accepted and when it is actually complete.
Is Kafka an event-driven architecture?
Kafka is infrastructure that can support an event-driven design. The architecture also includes producer transactions, event contracts, consumer behavior, identity, observability and recovery. A Kafka topic alone does not establish meaningful service boundaries, correct ordering across entities or safe replay into external business systems.
How does Kafka differ from Service Bus?
Kafka emphasizes retained partitioned streams and independent consumer progress. Service Bus emphasizes brokered work, queues and subscriptions with settlement and enterprise messaging features. Choose by retention/replay, ownership and transaction needs, and validate the actual configuration rather than assuming either platform supplies universal end-to-end exactly-once processing.
When should I use Event Grid instead of Service Bus?
Evaluate Event Grid for filtered notifications and reactive processing of resource or application changes. Evaluate Service Bus when the contract is owned work with brokered settlement and an operational dead-letter path. Confirm the selected Event Grid delivery model because classic push, namespace HTTP and MQTT capabilities are not interchangeable.
Is Event Hubs an alternative to Kafka?
It can be when Azure-managed streaming and the supported Kafka protocol/API surface meet your needs. Test clients, authentication, required features and operations against the selected tier. Kafka compatibility does not mean unrestricted Kafka administration or complete equivalence with a self-managed or third-party managed Kafka deployment.
Does an event-driven design guarantee delivery?
Guarantees are scoped to particular systems and configurations. A producer can fail before publishing, a consumer can repeat an external write, and retention or policy can expire work. Use durable handoffs, idempotency, monitoring and reconciliation. Broker acknowledgment should not be described as proof that the final business action succeeded.
Is it suitable for microservices?
It can connect independently owned capabilities and reduce synchronous dependencies, but it also adds consistency and operating concerns. Use it where asynchronous completion and independent consumers are meaningful. A modular monolith can use messaging too; microservices are not a prerequisite and events do not automatically create good service boundaries.
What are the disadvantages?
Delayed state, duplicate/out-of-order delivery, contract evolution and distributed debugging require deliberate work. Retention, lag and exception handling add cost and operational ownership. These disadvantages can outweigh the benefit for a simple interaction that needs an immediate answer, so compare against a synchronous or simpler queued design first.
How do I choose an enterprise event broker?
Start with message purpose, ordering scope, replay horizon, retention, consumer independence, security and recovery objectives. Eliminate candidates that fail mandatory controls, then test a representative workload and its failure paths. Include capacity, network, storage and people costs; raw throughput and per-message price are insufficient selection criteria.
Can legacy .NET applications adopt it?
Yes, through an incremental boundary such as an outbox, queue-backed worker or explicit event projection. Preserve supported runtime/security requirements and authoritative data ownership. Validate crash recovery, duplicates and operational handover before expanding. Introducing messaging does not require replacing the entire application or adopting microservices at the same time.
Technical references
- Apache Kafka design
- Service Bus overview
- Event Grid delivery and retry
- messaging-service comparison
- Event Hubs Kafka support
Technical source review: 1 November 2026. Illustrative architectures, assumptions and examples are not client implementations or measured production results. Deployment-specific verification is required.