Azure Event Hubs in C#: A Complete Guide with Example

In modern applications, collecting and processing large amounts of data in real time is essential. Applications such as e-commerce platforms, banking systems, IoT devices, and monitoring systems continuously generate events that need to be captured and processed efficiently. Traditional approaches may struggle to handle high volumes of incoming data, especially when multiple applications produce events simultaneously. This is where Azure Event Hubs comes into the picture.

In this article, we will explore how Azure Event Hubs works, understand its key components and architecture, and learn how to integrate it into a C# application. We will also walk through a practical example of creating an Event Hub in the Azure portal, sending events from a C# application, and receiving and processing those events using the Azure SDK for .NET.

By the end of this article, you will have a clear understanding of Azure Event Hubs and how to use it to build scalable, event-driven applications with C# and .NET.

What is Azure Event Hubs

Azure Event Hubs is a fully managed, cloud-based event streaming platform provided by Microsoft Azure. It can collect, store, and process millions of events per second from multiple sources. It is designed for high-throughput data ingestion and real-time event streaming, making it suitable for scenarios such as application log collection, IoT telemetry, website activity tracking, and real-time analytics.

How Azure Event Hubs Works

Azure Event Hubs works as a central event streaming service that collects events from different applications or devices and makes those events available to consumers for processing.

Think of it as a large, managed pipeline: multiple applications send data to Azure Event Hubs, and other applications read that data and process it independently.

Real-world example

Imagine you are building an e-commerce application. Whenever a customer places an order, views a product, or makes a payment, your application generates an event.

For example:

• OrderPlaced
• ProductViewed
• PaymentCompleted

Instead of sending this data directly to several different systems, your application can publish events to Azure Event Hubs. Separate consumer applications can then process those events for analytics, monitoring, or reporting.

How Azure Event Hubs Processes Events

• Event Hubs Namespace

  A namespace is the management container that hosts one or more Event Hubs. It provides the endpoint and configuration for connecting to the service.

• Event Hub

  An Event Hub is the individual event stream to which producers send data and from which consumers read it.

• Producers and Consumers

  Producers publish events, while consumers read and process them. These applications can run independently and scale according to their workload.

• Partitions and Consumer Groups

  Partitions organize the event stream and support parallel processing. Consumer groups provide independent views of the stream, allowing different applications to read the same events for different purposes.

How Azure Event Hubs Processes Events

1. An application generates events

   A producer is an application or device that creates data. For example, a C# Web API generates an OrderPlaced event when a customer places an order.

2. he producer sends events to Azure Event Hubs

   The application connects to an Event Hub using the Azure SDK for .NET and sends events to the Event Hub endpoint. Events can contain JSON, text, or other serialized data.

3. Event Hubs stores events in partitions

   Azure Event Hubs distributes events across partitions. Each partition is an ordered sequence of events. Partitioning allows the service to handle high volumes of data in parallel.

4. Consumer applications read the events

   A consumer is an application that reads events from Event Hubs. For example, an analytics application can read order events and calculate sales statistics.

5. Consumers process events and track their progress

   Consumers process the events and can save their progress using checkpoints. If a consumer restarts, it can resume reading from its last saved checkpoint rather than starting from the beginning.

Send and Receive Azure Events from Azure Event Hubs Using C#

Let's create a simple C# console application that sends and receies an event to Azure Event Hubs. We'll use the Azure.Messaging.EventHubs library to send events and Azure.Messaging.EventHubs.Processor to receive and process events.

For this example, we'll create a simple console application that:

1. Connects to Azure Event Hubs.
2. Sends an event containing an order notification.
3. Receives events from the Event Hub.
4. Prints the received event data to the console.

Required Azure resources and libraries

You need an Azure Event Hubs namespace and an Event Hub created inside it.

1. Azure Storage Account
2. Azure Event Hubs
3. Azure.Messaging.EventHubs
4. Azure.Messaging.EventHubs.Processor
5. Azure.Storage.Blobs

Install the required NuGet packages
Run the following commands in your .NET console project:

</> powershell
dotnet add package Azure.Messaging.EventHubs
dotnet add package Azure.Messaging.EventHubs.Processor
dotnet add package Azure.Storage.Blobs

What each package does:

NuGet package	Purpose
Azure.Messaging.EventHubs	Sends events and provides low-level event receiving APIs.
Azure.Messaging.EventHubs.Processor	Receives events through a processor that manages partitions and checkpoints.
Azure.Storage.Blobs	Stores checkpoints in Azure Blob Storage.

Configure Azure Event Hubs
Before running the code, create the following resources in the Azure portal:

1. Create an Event Hubs namespace.
2. Create an Event Hub named orders-eventhub.
3. Create a shared access policy with Send and Listen permissions, or configure equivalent Azure role-based access.
4. Copy the connection string and note the Event Hub name.

Send and Receive Azure Events Example in C#

These examples use connection strings for simplicity. For production applications, consider Microsoft Entra ID authentication with managed identities.

Example: Send Microsoft Azure Events using C#

</> C#
using System;
using System.Text;
using System.Threading.Tasks;
using Azure.Messaging.EventHubs;
using Azure.Messaging.EventHubs.Producer;
class Program
{
  private const string EventHubConnectionString =
  "<EVENT_HUB_CONNECTION_STRING>";
  private const string EventHubName = "orders-eventhub";
  static async Task Main(string[] args)
  {
    await using var producer = new EventHubProducerClient(
    EventHubConnectionString,
    EventHubName);
    // Create an event batch
    using EventDataBatch eventBatch =
    await producer.CreateBatchAsync();
    string message =
    "{\"OrderId\":101,\"Product\":\"Laptop\",\"Amount\":75000}";
    // Convert the message into an EventData object
    var eventData = new EventData(
    Encoding.UTF8.GetBytes(message));
    eventBatch.TryAdd(eventData);
    // Send the event to Azure Event Hubs
    await producer.SendAsync(eventBatch);
    Console.WriteLine("Event sent successfully!");
    Console.WriteLine(message);
  }
}

The EventHubProducerClient class is responsible for publishing events to an Event Hub.

How does this code work?

1. EventHubProducerClient connects to your Event Hub.
2. CreateBatchAsync() creates a batch for events.
3. EventData represents an event containing the order information.
4. Encoding.UTF8.GetBytes() converts the JSON message into bytes.
5. TryAdd() adds the event to the batch.
6. SendAsync() publishes the batch to Azure Event Hubs.

Important: For production code, check the Boolean result of TryAdd(). If it returns false, the event did not fit in the batch and must be handled rather than silently ignored.

Example: Receive Microsoft Azure Events using C#

</> C#
using System;
using System.Text;
using System.Threading.Tasks;
using Azure.Messaging.EventHubs;
using Azure.Messaging.EventHubs.Consumer;
using Azure.Messaging.EventHubs.Processor;
using Azure.Storage.Blobs;
class Program
{
  private const string EventHubConnectionString =
  "<EVENT_HUB_CONNECTION_STRING>";
  private const string EventHubName = "orders-eventhub";
  private const string StorageConnectionString =
  "<AZURE_STORAGE_CONNECTION_STRING>";
  private const string ContainerName = "eventhub-checkpoints";
  static async Task Main(string[] args)
  {
    var storageClient = new BlobContainerClient(
    StorageConnectionString,
    ContainerName);
    var processor = new EventProcessorClient(
    storageClient,
    EventHubConsumerClient.DefaultConsumerGroupName,
    EventHubConnectionString,
    EventHubName);
    processor.ProcessEventAsync += ProcessEventHandler;
    processor.ProcessErrorAsync += ProcessErrorHandler;
    Console.WriteLine("Listening for events...");
    Console.WriteLine("Press Enter to stop.");
    await processor.StartProcessingAsync();
    Console.ReadLine();
    await processor.StopProcessingAsync();
  }
  private static async Task ProcessEventHandler(
  ProcessEventArgs args)
  {
    if (args.Data == null)
    return;
    string message = Encoding.UTF8.GetString(
    args.Data.Body.ToArray());
    Console.WriteLine("Event received:");
    Console.WriteLine(message);
    // Save progress after processing the event.
    await args.UpdateCheckpointAsync();
  }
  private static Task ProcessErrorHandler(
  ProcessErrorEventArgs args)
  {
    Console.WriteLine(
    $"Error: {args.Exception.Message}");
    Console.WriteLine(
    $"Operation: {args.Operation}");
    return Task.CompletedTask;
  }
}

How does this code work?

1. BlobContainerClient connects to the storage container used for checkpoints.
2. EventProcessorClient connects to the Event Hub using a consumer group.
3. ProcessEventAsync runs when an event is received.
4. The event body is converted from bytes into a readable string.
5. UpdateCheckpointAsync() saves the processing position so the processor can resume from a stored checkpoint.
6. ProcessErrorAsync handles processing errors.

The checkpoint should be updated after your application has successfully processed the event. Checkpointing does not guarantee exactly-once processing; design your handler to tolerate duplicate events.

Important points to remember

• Run the receiver first, then run the sender to test the complete flow.
• Events are retained according to your Event Hub's retention settings; the receiver does not automatically receive only events sent after it starts.
• Use separate consumer groups when independent applications need to read the same event stream independently.
• Store connection strings securely in configuration or a secret store; do not hardcode them in production.
• For production applications, consider Microsoft Entra ID authentication and managed identities instead of connection strings.
• Use checkpointing and idempotent event processing to recover safely from interruptions.

Remember: Azure Event Hubs is particularly useful for high-throughput event ingestion and streaming analytics. If your main requirement is distributing individual tasks to workers, with queue-style message settlement, Azure Service Bus may be a better fit.

Summary

Azure Event Hubs is a powerful event-streaming service that enables applications to handle large volumes of real-time data efficiently. In this article, we explored how Azure Event Hubs works and how to integrate it into C# applications using the Azure.Messaging.EventHubs and Azure.Messaging.EventHubs.Processor libraries.

We also learned how to send events to an Event Hub, receive and process events, and use Azure Blob Storage to maintain checkpoints. These capabilities make Azure EventHubs useful for scenarios such as application logging, IoT telemetry, order processing, and real-time data analytics.

By integrating Azure Event Hubs into your C# applications, you can build scalable and reliable event-driven solutions. For production applications, remember to implement proper error handling, secure authentication, and checkpoint management to ensure efficient and dependable event processing.

Thanks