How to (not) use the large object heap in .Net

When working with .Net, it is important to understand how the garbage collector works. The .Net CLR manages two different heaps, the small object heap (SOH) and the large object heap (LOH). This article will focus on how the runtime manages the large object heap, and why the large object heap is prone to memory fragmentation that can impact the performance of your application. It will also discuss the best practices you can use to keep LOH memory fragmentation to a minimum.

The garbage collector is quite adept at cleaning up resources that are no longer being used by your application. However, the small object heap and the large object heap work differently. Let’s start with the small object heap.

When a small object is created (less than 85KB in size), it is stored in the small object heap. The CLR organizes the small object heap into three generations – generation 0, generation 1, and generation 2 – that are collected at different intervals. Small objects are generally allocated to generation 0, and if they survive a GC cycle, they are promoted to generation 1. If they survive the next GC cycle, they are promoted to generation 2.

Further, garbage collection of the small object heap includes compaction, meaning that as unused objects are collected, the GC moves living objects into the gaps to eliminate fragmentation and make sure that the available memory is contiguous. Of course, compaction involves overhead – both CPU cycles and additional memory for copying objects. Because the benefits of compaction outweigh the costs for small objects, compaction is performed automatically on the small object heap.

Memory holes in the large object heap

However, the cost of compaction is too high for large objects (greater than 85KB in size). Copying and moving large objects not only would involve significant overhead for the garbage collector – the GC would need twice as much memory for garbage collection – but moving large objects would be very time-consuming as well. Therefore, unlike the small object heap, the large object heap is not compacted during garbage collection.

So, how is memory in the large object heap reclaimed? Well, the GC never moves large objects – all it does is remove them when they are no longer needed. In doing so, memory holes are created in the large object heap. This is what is known as memory fragmentation.

One point to note here is that although the GC doesn’t compact the large object heap, it does combine adjacent free blocks in the heap to make larger blocks available. So, if you have adjacent free blocks in the large object heap, the GC combines them to create a larger free block and adds it to the free list as an optimization strategy.

Also keep in mind that the GC collects unused objects from the large object heap only during generation 2 collections. In other words, the GC tries to reclaim memory residing in the small object heap before it tries to reclaim memory from the large object heap. Thus the large object heap is not only subject to memory fragmentation, but large objects live longer, taking up space even when they aren’t being used.

So, the reason large objects are stored in a separate heap is that the cost of compacting an object is directly proportional to the object’s size. Because this cost is substantial and can degrade performance, large objects are stored in a separate heap that is not compacted.

Getting rid of large object heap fragmentation

Now that we know about the pitfalls of large object heap fragmentation, let’s take a quick tour of the best practices that we can adopt to avoid it. A recommended strategy is to identify the large objects in your application and then split them into smaller objects – perhaps using some wrapper class. You can also redesign your application to ensure that you avoid using large objects. Another approach is to recycle the application pool periodically.

Although the GC doesn’t compact the large object heap for you, you can still programmatically compact it. The following code snippet illustrates how this can be achieved.

GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;
GC.Collect();

Note also that there have been a number of significant improvements in the way the large object heap is managed in .Net Framework 4.5. The first is that the runtime now manages the large object heap in a more optimized way by repeatedly checking a free list of available memory blocks and using them whenever possible. In prior versions of the .Net Framework, once a memory block was rejected as a candidate for allocation, it was not considered for subsequent allocations. 

The second improvement is that, if you are using the Server GC mode, the runtime balances the large object heap allocations across all of the logical processors used by the application. Prior to .Net Framework 4.5, only the small object heap allocations were balanced across processors.

These changes significantly boost the performance of large object heap memory.

In summary, while the runtime garbage collector compacts the small object heap as part of an optimization strategy to eliminate memory holes, the runtime never compacts the large object heap for performance reasons. Hence if you are running a program that uses many large objects in an x86 system, you might encounter OutOfMemory exceptions. If you are running that program in an x64 system, you might have a fragmented heap.

By understanding the internals of garbage collection and the intricacies of the large object heap, we can employ strategies that avoid memory fragmentation issues and help our applications run smoothly.