Optimizations when Environment.ProcessorCount > 1 in Kubernetes environment

[tylerohlsen]

I've noticed that our pods on a Kubernetes/Docker cluster with millicpu allocations all report Environment.ProcessorCount = 1.

Does the .Net Core CLR have any optimizations when Environment.ProcessorCount > 1 that we are missing out on or is anything is disabled or degraded when Environment.ProcessorCount = 1?

[benaadams]

Threadpool min thread count and GC heaps (in server mode) are based on processor count. This may be desirable for GC as it will also use less memory (depending if the pod with a millicpu allocation also has lower assigned memory).

Threadpool min levels can increased with ThreadPool.SetMinThreads(Int32, Int32) if the processor count defaults aren't working for you.

Other aspect I'm aware of is some concurrent datastructures/locks won't do optimistic spins if there is only one processor; but move straight to yielding/sleeping when blocked.

[tylerohlsen]

FYI to anyone that stumbles across this. Environment.ProcessorCount will equal the integer upper limit of resources you give a pod.

So this spec will report Environment.ProcessorCount = 4

        resources:
          requests:
            cpu: "500m"
          limits:
            cpu: "4000m"

And this spec will report Environment.ProcessorCount = 2

        resources:
          requests:
            cpu: "500m"
          limits:
            cpu: "2500m"

[tylerohlsen]

Is @benaadams list exhaustive of the main optimizations?
Is there a recommendation for ensuring ProcessorCount is > 1 in heavy load environments?

[stephentoub]

Is @benaadams list exhaustive of the main optimizations?

No, there are lots of things that pay attention to Environment.ProcessorCount, or to the equivalent in the runtime's native code. For example, various types create their data structures / layouts with input from ProcessorCount in order to try to minimize contention, e.g. Timer, ConcurrentDictionary. Other types create object pools sized with input from processor count, e.g. HttpClient, SqlClient, System.Reflection.Metadata. Other types that focus on parallelism control the default number of workers used based on processor count, e.g. Parallel, PLINQ. And so on.

Is there a recommendation for ensuring ProcessorCount is > 1 in heavy load environments?

No, the recommendation is that processor count accurately reflect the number of processors available, such that the heuristics that incorporate that information aren't misled.

Of course, if you're in an environment with a heavy load, generally the more cores you have the better off you are (within reason), but certainly a single core for a heavy-load environment is far from ideal.

[benaadams]

Is there a recommendation for ensuring ProcessorCount is > 1 in heavy load environments?

Only if you have ProcessorCount > 1 😄

For example spinning on a lock with one processor isn't good; because what is holding the lock will not make progress because the CPU is being used checking the lock, so its better that it goes straight to sleep releasing the CPU to whatever has the lock.

Whereas if you actually have more than one processor, it can optimistically spin a little in case its a fine grained, short held lock; prior to descheduling the thread without preventing the thread that holds the lock from making progress towards releasing the lock.

[tylerohlsen]

No, the recommendation is that processor count accurately reflect the number of processors available, such that the heuristics that incorporate that information aren't misled.

I don't think it's that straightforward in a containerized environment. The physical machine in one of our environments has 16 cores available. But that is utilized across multiple pods and virtualized by docker. The number of processors available to each pod? That depends on how many pods I stuff onto that machine. And even then, each pod is allowed burst beyond their requested cpu allocation. And even if I set the requested cpu to 1, multiple threads in my pod can each be running on a different physical core (that's up to docker to figure out where they execute).

I think my question is answered unless anyone else has some additional insights they want to share. My question was if there are optimizations I would lose if .Net Core thinks the processor count is 1. And the answer is Yes.

I can make .Net Core think there are as many processors I want via the cpu limit kubernetes parameters. And for me, since I'm running on a physical machine with many cores, I will set that limit to > 1 to make sure I get those optimizations.

It would be nice if it was documented somewhere.

[RussKeldorph]

Closing per above. Feel free to put out a PR improving the documentation.