How to use Parallel.For and Parallel.ForEach in C#

Parallelism is the ability to have parallel execution of tasks on systems that have multiple cores. Support for parallel programming in .NET was introduced in .NET Framework 4. Parallel programming in .NET allows us to use system resources more efficiently and with better programmatic control. This article talks about how we can work with parallelism in .NET Core applications.

To work with the code examples provided in this article, you should have Visual Studio 2019 installed in your system. If you don’t already have a copy, you can download Visual Studio 2019 here.

Create a .NET Core console application project in Visual Studio

First off, let’s create a .NET Core console application project in Visual Studio. Assuming Visual Studio 2019 is installed in your system, follow the steps outlined below to create a new .NET Core console application project in Visual Studio.

1. Launch the Visual Studio IDE.
2. Click on “Create new project.”
3. In the “Create new project” window, select “Console App (.NET Core)” from the list of templates displayed.
4. Click Next.
5. In the “Configure your new project” window, specify the name and location for the new project.
6. Click Create.

We’ll use this project to illustrate parallel programming in .NET Core in the subsequent sections of this article.

Concurrency and parallelism in .NET Core

Concurrency and parallelism are two critical concepts in .NET and .NET Core. Although they appear to be the same, there are subtle differences between them.

Consider two tasks, T1 and T2, that have to be executed by an application. These two tasks are in concurrent execution if one is in an execution state while the other is waiting for its turn. As a result, one of the tasks completes ahead of the other. By contrast, the two tasks are in parallel execution if both execute simultaneously. To achieve task parallelism, the program must run on a CPU with multiple cores.

Parallel.For and Parallel.ForEach in .NET Core

The Parallel.For loop executes iterations that may run in parallel. You can monitor and even manipulate the state of the loop. The Parallel.For loop is just like the for loop except it allows the iterations to run in parallel across multiple threads.

The Parallel.ForEach method splits the work to be done into multiple tasks, one for each item in the collection. Parallel.ForEach is like the foreach loop in C#, except the foreach loop runs on a single thread and processing take place sequentially, while the Parallel.ForEach loop runs on multiple threads and the processing takes place in a parallel manner.

Parallel.ForEach vs. foreach in C#

Consider the following method that accepts an integer as parameter and returns true if it is a prime number.

static bool IsPrime(int integer)
        {
            if (integer <= 1) return false;
            if (integer == 2) return true;
            var limit = Math.Ceiling(Math.Sqrt(integer));
            for (int i = 2; i <= limit; ++i)
                if (integer % i == 0)
                    return false;
            return true;
        }

We’ll now take advantage of ConcurrentDictionary to store the prime numbers and managed thread Ids. Since the prime numbers between two ranges are unique, we can use them as keys and the managed thread Ids as values.

Concurrent collections in .NET are contained inside the System.Collections.Concurrent namespace and provide lock-free and thread-safe implementations of the collection classes. The ConcurrentDictionary class is contained inside the System.Collections.Concurrent namespace and represents a thread-safe dictionary.

The following two methods both use the IsPrime method to check if an integer is a prime number, store the prime numbers and managed thread Ids in an instance of a ConcurrentDictionary, and then return the instance. The first method uses concurrency and the second method uses parallelism.

        private static ConcurrentDictionary<int, int>
        GetPrimeNumbersConcurrent(IList<int> numbers)
        {
            var primes = new ConcurrentDictionary<int, int>();
            foreach (var number in numbers)
            {               
                if(IsPrime(number))
                {
                    primes.TryAdd(number,
                    Thread.CurrentThread.ManagedThreadId);
                }
            }
            return primes;
        }
        private static ConcurrentDictionary<int, int>
        GetPrimeNumbersParallel(IList<int> numbers)
        {
            var primes = new ConcurrentDictionary<int, int>();
            Parallel.ForEach(numbers, number =>
            {
                if (IsPrime(number))
                {
                    primes.TryAdd(number,
                    Thread.CurrentThread.ManagedThreadId);
                }
            });
            return primes;
        }

Concurrent vs. parallel example in C#

The following code snippet illustrates how you can invoke the GetPrimeNumbersConcurrent method to retrieve all prime numbers between 1 to 100 as well as the managed thread Ids.

static void Main(string[] args)
        {
            var numbers = Enumerable.Range(0, 100).ToList();
            var result = GetPrimeNumbersConcurrent(numbers);
            foreach(var number in result)
            {
                Console.WriteLine($"Prime Number:
                {string.Format("{0:0000}",number.Key)},
                Managed Thread Id: {number.Value}");
            }
            Console.Read();
        }

When you execute the above program, you should see the output as shown in Figure 1:

IDG

As you can see, the managed thread Id is the same in each case since we used concurrency in this example. Let’s now see what the output would look like when using thread parallelism. The following code snippet illustrates how you can retrieve prime numbers between 1 to 100 using parallelism.

        static void Main(string[] args)
        {
            var numbers = Enumerable.Range(0, 100).ToList();
            var result = GetPrimeNumbersParallel(numbers);
            foreach(var number in result)
            {
                Console.WriteLine($"Prime Number:
                {string.Format("{0:0000}",number.Key)},
                Managed Thread Id: {number.Value}");
            }
            Console.Read();
        }

When you execute the above program, the output should look something like that shown in Figure 2:

IDG`

As you can see here, because we’ve used Parallel.ForEach, multiple threads have been created and hence the managed thread Ids are dissimilar.

Limit the degree of parallelism in C#

The degree of parallelism is an unsigned integer number that denotes the maximum number of processors that your query should take advantage of while it is in execution. In other words, the degree of parallelism is an integer that denotes the maximum number of tasks that will be executed at the same point in time to process a query.

By default, the Parallel.For and Parallel.ForEach methods have no limit on the number of spawned tasks. So, in the GetPrimeNumbersParallel method shown above, the program attempts to use all of the available threads in the system.

You can take advantage of the MaxDegreeOfParallelism property to limit the number of spawned tasks (per ParallelOptions instance of the Parallel class). If MaxDegreeOfParallelism is set to -1, then there is no limit on the number of concurrently running tasks.

The following code snippet shows how you can set MaxDegreeOfParallelism to use a maximum of 75% resources of the system.

new ParallelOptions
{
    MaxDegreeOfParallelism = Convert.ToInt32(Math.Ceiling((Environment.ProcessorCount * 0.75) * 2.0))
};

Note that in the above snippet we’ve multiplied the processor count by two because each processor contains two cores. Here is the complete updated code of the GetPrimeNumbersParallel method for your reference:

        private static ConcurrentDictionary<int, int> GetPrimeNumbersParallel(IList<int> numbers)
        {
            var primes = new ConcurrentDictionary<int, int>();
            Parallel.ForEach(numbers, number =>
            {
                new ParallelOptions
                {
                    MaxDegreeOfParallelism = Convert.ToInt32(Math.Ceiling((Environment.ProcessorCount * 0.75) * 2.0))
                };
                if (IsPrime(number))
                {
                    primes.TryAdd(number,
                    Thread.CurrentThread.ManagedThreadId);
                }
            });
            return primes;
        }

Determine if a parallel loop is complete in C#

Note that both Parallel.For and Parallel.ForEach return an instance of ParallelLoopResult, which can be used to determine if a parallel loop has completed execution. The following code snippet shows how ParallelLoopResult can be used.

ParallelLoopResult parallelLoopResult = Parallel.ForEach(numbers, number =>
 {
    new ParallelOptions
    {
          MaxDegreeOfParallelism = Convert.ToInt32(Math.Ceiling(
          (Environment.ProcessorCount * 0.75) * 2.0))
    };
    if (IsPrime(number))
    {
          primes.TryAdd(number, Thread.CurrentThread.ManagedThreadId);
    }
 });
Console.WriteLine("IsCompleted: {0}", parallelLoopResult.IsCompleted);

To use Parallel.ForEach in a non-generic collection, you should take advantage of the Enumerable.Cast extension method to convert the collection to a generic collection as illustrated in the code snippet given below.

Parallel.ForEach(nonGenericCollection.Cast<object>(),
    currentElement =>
    {
    });

On a final note, don’t assume that the iterations of Parallel.For or Parallel.ForEach will always execute in parallel. You should also be aware of thread affinity issues. You can read about these and other potential pitfalls in task parallelism in Microsoft’s online documentation here.

How to do more in C#:

• How to avoid GC pressure in C# and .NET
• How to use target typing and covariant returns in C# 9
• How to use top-level programs in C# 9
• How to use pattern matching in C#
• How to work with read-only collections in C#
• How to work with static anonymous functions in C# 9
• How to work with record types in C#
• How to use implicit and explicit operators in C#
• Singleton vs. static classes in C#
• How to log data to the Windows Event Log in C#
• How to use ArrayPool and MemoryPool in C#
• How to use the Buffer class in C#
• How to use HashSet in C#
• How to use named and optional parameters in C#
• How to benchmark C# code using BenchmarkDotNet