How to use IAsyncEnumerable in C#

IAsyncEnumerable is a powerful feature introduced in C# 8.0 that allows us to work with a sequence of data asynchronously. As the name suggests, IAsyncEnumerable is the asynchronous counterpart of IEnumerable, the interface that allows us to easily iterate through the elements of a collection.

Interestingly, the IAsyncEnumerable interface works on a pull-based approach, where the next item will either be created or retrieved when requested by the consumer. Unlike IEnumerable, which waits for the next element to be created, IAsyncEnumerable returns an awaitable that can be resumed later.

In this article, we will examine how we can use the IAsyncEnumerable interface in C# to work with asynchronous streams of data. 

Create a console application project in Visual Studio

First off, let’s create a .NET Core console application project in Visual Studio. Assuming Visual Studio 2022 is installed in your system, follow the steps outlined below to create a new .NET Core console application project in Visual Studio.

1. Launch the Visual Studio IDE.
2. Click on “Create new project.”
3. In the “Create new project” window, select “Console App (.NET Core)” from the list of templates displayed.
4. Click Next.
5. In the “Configure your new project” window, specify the name and location for the new project.
6. Click Next.
7. In the “Additional information” window shown next, choose “.NET 7.0 (Standard Term Support)” as the Framework version you would like to use.
8. Click Create.

We’ll use this .NET 7 console application project to work with the code examples shown in the subsequent sections of this article.

IAsyncEnumerable benefits

The key benefits of IAsyncEnumerable include the following:

• Support for asynchronous streaming: A traditional collection like a list or IEnumerable requires that all elements be available in advance. IAsyncEnumerable, on the other hand, streams items as they become available. Using IAsyncEnumerable is especially beneficial when dealing with large data sets or real-time data streams where all data may not be available in advance. With IAsyncEnumerable, you can start processing items immediately without having to wait for the data set to be loaded in its entirety. When working with real-time data feeds such as stock quotes or social media updates, where new information is constantly being generated and needs to be processed as soon as it is available, this flexibility is beneficial.
• Efficient use of available resources: IAsyncEnumerable allows you to work with large sequences of data in an asynchronous manner, which ensures that valuable computing resources are not wasted.
• Enhanced performance: IAsyncEnumerable can improve your application’s performance by eliminating the need to load all the data at one go. Consequently, the system can use less memory and free up resources. 
• Improved responsiveness: With IAsyncEnumerable, you can easily write code that handles large streams of data responsively. You can also use resources more efficiently and improve overall application performance, so that your application remains responsive even when working with large data sets. 
• Simpler code: IAsyncEnumerable simplifies code by eliminating complex synchronization mechanisms such as locks and semaphores thus reducing the likelihood of deadlocks and other synchronization-related issues in your application.

In the following section we’ll examine a few advanced operations you can perform on asynchronous sequences of data using IAsyncEnumerable.

Filtering with IAsyncEnumerable

The following code snippet illustrate how you can filter an asynchronous sequence of data to retrieve only the even numbers.

public async IAsyncEnumerable<int> GetEvenNumbersAsync(IEnumerable<int> integers)
{
    foreach (var n in integers)
    {
        await Task.Delay(500);
        if (n % 2 == 0)
        {
            yield return n;
        }
    }
}

Aggregating with IAsyncEnumerable

The following code snippet illustrates how you can take advantage of IAsyncEnumerable to calculate the sum of integers in an asynchronous sequence of numbers.

public async Task<int> GetSumAsync(IEnumerable<int> integers)
{
    int sum = 0;
        await foreach (var n in GetNumbersAsync(integers))
    {
        sum += n;
    }
        return sum;
}

Asynchronous projection with IAsyncEnumerable

The following code snippet shows how you can use projections with IAsyncEnumerable.

public async IAsyncEnumerable<int> GetNumbersUsingProjectionAsync(IEnumerable<int> integers)
{
    foreach (var n in integers)
    {
        await Task.Delay(500);
        yield return await Task.Run(() => n * 10);
    }
}

Transforming a sequence with IAsyncEnumerable

The following code snippet shows how you can take advantage of IAsyncEnumerable to transform a sequence of numbers and yield the transformed values.

public async IAsyncEnumerable<string> TransformNumbersAsync(IEnumerable<int> integers)
{
    foreach (var n in integers)
    {
        await Task.Delay(500);
        yield return n.ToString();
    }
}

Batch processing with IAsyncEnumerable

Apart from fetching items one at a time, you can also process items in batches when working with IAsyncEnumerable. This is shown in the code snippet given below.

public static async IAsyncEnumerable<IEnumerable<T>> ProcessBatch<T>(IAsyncEnumerable<T> source, int size,
    CancellationToken cancellationToken = default)
{
    var batch = new List<T>();
    await foreach (var item in source)
    {
        if (cancellationToken.IsCancellationRequested)
            yield break;
        batch.Add(item);
        if (batch.Count >= size)
        {
            yield return batch;
            batch = new List<T>();
        }
    }
    if (batch.Count > 0)
    {
        yield return batch;
    }
}

Buffering with IAsyncEnumerable

You can also take advantage of buffering for better performance in processing items asynchronously.

public static async IAsyncEnumerable<T> ProcessBuffer<T>(this IAsyncEnumerable<T> source, int size,
    CancellationToken cancellationToken = default)
    {
        var queue = new Queue<T>();
        await foreach (var item in source)
        {
            if (cancellationToken.IsCancellationRequested)
                yield break;
            queue.Enqueue(item);
            while (queue.Count >= size)
            {
                yield return queue.Dequeue();
            }
        }
        while (queue.Count > 0)
        {
            yield return queue.Dequeue();
        }
    }

Conclusion

As we’ve seen, IAsyncEnumеrablе providеs sеvеral powеrful fеaturеs that can bе appliеd to asynchronous sеquеncеs. Additionally, you can use IAsyncEnumerable with LINQ operators to perform various operations that include but are not limited to filtering, transforming, and aggregating large data sets. You can also handle runtime errors that might occur when you’re working with IAsyncEnumerable, and you can take advantage of retries and implement logging to handle such errors gracefully.