Working with memory mapped files in .Net

File access is a resource-intensive operation. Accessing a file from the disk for an application is a time-consuming operation, and accessing data from the primary memory is always faster. So, what if the disk files your application needs to read from or write to were residing in memory? This is exactly where the concept of memory mapped files fits in. In this article, we will explore on how we can work with memory mapped files in .Net.

Introducing memory mapped files

A memory mapped file is a kernel object that is used to map a file in your disk to a region in the primary memory. Memory mapped files can have major performance gains compared to direct disk access when working with large amount of data or large images. Memory mapped files have been a part of the Win32 API, but until recently, you were restricted to using C++ or PInvoke to write code that leverages memory mapped files in your application. However with .Net Framework 4 you can now work with memory mapped files directly from your .Net applications — the runtime now provides you a managed wrapper with all the necessary wrapper classes to call the Win32 API. The MSDN states: “A memory mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory.”

Why do you need memory mapped files?

Memory mapped files are a good choice when you need to work with a large amount of data and you would like to avoid the cost associated with marshaling and un-marshaling while sharing data across process boundaries. Memory mapped files are great at processing a large file – reading a large file is a resource extensive operation. With memory mapped files, you can map a specific portion of your file into memory and perform I/O operations with that block to speed up access.

A memory mapped file enables you to reserve a range of memory address and use a disk file as the physical storage for the reserved address. In other words, it allows you to reserve a space in the memory and then commit physical storage to that region. This enables you to access data on the disk without the need of performing file I/O operation. Memory mapped files also enable you to share data across multiple processes. The Operating System takes care of managing memory for memory mapped files – you need not bother how the file is partitioned into pages and managed. You can also apply security in your memory mapped file by using the MemoryMappedFileAccess enumeration as a parameter when creating the memory mapped file. 

Persistent and Non-persistent memory mapped files

There are essentially two types of memory mapped files. These are:

Persistent: Persistent memory mapped files are those that are associated with a source file in the disk in your system. When you work with these types of memory mapped files, data is persisted in the disk after the last process working on the file finishes its activity.

Non-persistent: Non-persistent memory mapped files are those that are not associated with a disk file. When you work with these type of memory mapped files, data is not persisted after the last process working on the file has finished its work. Non-persistent memory mapped files are great in sharing memory for inter process communications.

Creating persistent memory mapped files

To create a persistent memory mapped file, you need to make use of the CreateFromFile method of the MemoryMappedFile class. The MemorymappedFile class is present in the System.IO.MemoryMappedFiles namespace.

The following code snippet makes use of the CreateFromFile method to create a memory-mapped file. It next creates a memory-mapped view to a portion of the file.

static long offset = 0x10000000; // 256 megabytes

static long length = 0x20000000; // 512 megabytes

        static void Main()

        {

            using (var memoryMappedFile = MemoryMappedFile.CreateFromFile("F:ImageData.png", FileMode.Open, "PartitionA"))

            {

                using (var accessor = memoryMappedFile.CreateViewAccessor(offset, length))

                {

                    //Other code

                }

            }

        } 

The code snippet given next shows how you can read data from a memory mapped file.

using (MemoryMappedFile memoryMappedFile = MemoryMappedFile.CreateFromFile("F:LargeData.dat"))

            {

                using (MemoryMappedViewStream memoryMappedViewStream = memoryMappedFile.CreateViewStream(0, 1204, MemoryMappedFileAccess.Read))

                {

                    var contentArray = new byte[1024];

                    memoryMappedViewStream.Read(contentArray, 0, contentArray.Length);

                    string content = Encoding.UTF8.GetString(contentArray);

                }

            }

Creating non-persistent memory mapped files

To create non-persistent memory mapped files, i.e., files that are not mapped to an existing file on the disk, you need to leverage the CreateNew and CreateOrOpen methods.

The following code snippet illustrates how a non-persistent memory mapped file can be created.

using(MemoryMappedFile memoryMappedFile = MemoryMappedFile.CreateNew("idg.txt", 5))

            {

                using(MemoryMappedViewAccessor memoryMappedViewAccessor = memoryMappedFile.CreateViewAccessor())

                {

                var data = new[] { (byte)'I', (byte)'D', (byte)'G'};

                for (int i = 0; i < data.Length; i++)

                    memoryMappedViewAccessor.Write(i, data[i]);

                memoryMappedViewAccessor.Dispose();

                memoryMappedFile.Dispose();

                }

            }

You can know more on memory mapped files from this MSDN article.