.NET Matters
ThreadPoolPriority, and MethodImplAttribute
Stephen Toub
Code download available at:NETMatters0411.exe(148 KB)
Q I'm writing an app that queues thousands of operations to the ThreadPool during a run. Some operations are more important than others, and I want them to have a higher priority. In other words, if there are any high-priority operations waiting to be run, I want the ThreadPool to pick up and run those first, regardless of when they were queued. How can I accomplish this?
A The heart of your problem lies in the fact that, while amazingly useful, the ThreadPool is a fairly inextensible class. There is no way to change or influence the internal data structures used to store the queued WaitCallbacks and associated state (the ThreadPool internally uses a queue implemented as a linked list, a fact you can verify by examining the source for ThreadPool in the Shared Source Common Language Infrastructure (SSCLI)). Thus, one way to accomplish your request is to maintain a data structure yourself and force the ThreadPool to ask your data structure for the next WaitCallback to execute.
In the October 2004 edition of the .NET Matters column, I demonstrated how to create a ThreadPoolWait class that provided a mechanism for synchronizing multiple threads from the ThreadPool. To do so, ThreadPoolWait exposed a QueueUserWorkItem method that relegated to ThreadPool.QueueUserWorkItem. But instead of passing along the user-supplied WaitCallback as the callback parameter to ThreadPool.QueueUserWorkItem, it created a new WaitCallback that wrapped its own private handler and allowed ThreadPoolWait to run custom code as soon as the user-supplied callback had completed. You can use a similar kind of mechanism to influence the order in which operations are handled by the ThreadPool.
Figure 1** ThreadPoolPriority Interacting with ThreadPool **
A diagram for a solution is shown in Figure 1 with the corresponding code shown in Figure 2. Just as with ThreadPoolWait, ThreadPoolPriority has a static QueueUserWorkItem method. However, in addition to taking as parameters the WaitCallback to be executed and the associated object state, it also takes a Boolean value that signals whether this callback should be treated as a priority. Rather than immediately passing the WaitCallback along to ThreadPool.QueueUserWorkItem, the method captures the callback and the associated state object into a temporary state class and then appends that new object to an internal queue. As you'll notice though, there are multiple queues. If the priority Boolean is set to true, the object is enqueued onto the priority queue, whereas if it's false it's enqueued onto the standard queue. Then, instead of calling ThreadPool.QueueUserWorkItem with the original WaitCallback, a new WaitCallback is created that wraps the HandleWorkItem static method, and that WaitCallback is passed. Note, too, that no state is explicitly associated with this callback.
Figure 2 ThreadPoolPriority
public class ThreadPoolPriority { private ThreadPoolPriority(){} private static Queue _priorityQueue = new Queue(); private static Queue _standardQueue = new Queue(); public static void QueueUserWorkItem( WaitCallback callback, object state, bool isPriority) { if (callback == null) throw new ArgumentNullException("callback"); new PermissionSet(PermissionState.Unrestricted).Demand(); QueuedCallback qc = new QueuedCallback(); qc.Callback = callback; qc.State = state; lock(_priorityQueue) { (isPriority ? _priorityQueue : _standardQueue).Enqueue(qc); } ThreadPool.UnsafeQueueUserWorkItem( new WaitCallback(HandleWorkItem), null); } private static void HandleWorkItem(object ignored) { QueuedCallback qc; lock(_priorityQueue) { qc = (QueuedCallback)(_priorityQueue.Count > 0 ? _priorityQueue : _standardQueue).Dequeue(); } qc.Callback(qc.State); } private class QueuedCallback { public WaitCallback Callback; public object State; } }
When invoked, HandleWorkItem checks to see whether any callbacks are queued on the priority queue. If so, the first is removed and that one is invoked. Otherwise, an object from the standard queue is dequeued and invoked. Unless something goes drastically wrong, there should always be an object in one of the queues. All access to both queues is protected by a lock in order to prevent race conditions and other multithreading perils.
Of course, my use of two Queues as the data structure is just an example. You can use any data structure that suits your situation. For example, if you want last in first out (LIFO) semantics, you can substitute a single Stack. Or if you wanted more levels of priority, you can implement an actual priority queue data structure (often based on a binary heap). Also, keep in mind that just because a work item begins executing first, there's no guarantee it'll finish first. As an enhancement, you could modify HandleWorkItem to change the current thread's scheduling priority (available through Thread.CurrentThread.Priority) according to the priority of the queued work item.
One more thing you might question is the use of the security demand in the QueueUserWorkItem method:
new PermissionSet(PermissionState.Unrestricted).Demand();
This has the unfortunate consequence of limiting the usability of the solution to those callers with full trust, but I'll explain why it's necessary. When a method calls ThreadPool.QueueUserWorkItem, information about the current call stack (in the form of a CompressedStack) is stored in the ThreadPool's queue along with the WaitCallback and its associated state. When the WaitCallback is executed on one of the ThreadPool's threads, that CompressedStack is pushed onto the executing Thread. Since Code Access Security (CAS) bases its permission checks on the permissions of all the callers on the stack, without this process in place to transfer information about the stack, you could accidentally (or maliciously) open up a security hole. The ThreadPool exposes another public member, UnsafeQueueUserWorkItem, that does not perform this call stack transfer.
So why is my code using UnsafeQueueUserWorkItem? By not invoking the queued WaitCallbacks in the order in which they were queued, I'm inadvertently disassociating the user-supplied WaitCallback from its CompressedStack. Thus, the permissions for the callers associated with the executing thread may have nothing to do with the callback being invoked. The best solution in my class would be to manually store the CompressedStack with the user-supplied WaitCallback and then set this CompressedStack on the ThreadPool's worker thread used to execute the callback.
While the Thread class exposes two methods, GetCompressedStack and SetCompressedStack, that could be used for this very purpose, their use is restricted to libraries signed with the Microsoft strong name, and thus are unavailable. As one of the few valid solutions left to me, my QueueUserWorkItem method demands that any caller attempting to queue a work item must already have full trust. That way, no direct caller could maliciously take advantage of this shortcoming. Given that full trust is required and that the CompressedStack generated for the executed method could be erroneous, I opted to avoid any correlated issues by then using ThreadPool.UnsafeQueueUserWorkItem (which should also be slightly faster given that it doesn't have to capture and store the stack).
The situation changes a bit with the Microsoft® .NET Framework 2.0, currently at Beta 1. In the .NET Framework 1.x, the current call stack is transferred as a CompressedStack to threads used from the ThreadPool and to new threads when they are spun up. However, other important context such as the Thread.CurrentPrincipal is not transferred. In the .NET Framework 2.0, not only is the CAS-context flowed across asynchronous points, but additional execution context is flowed as well, including impersonation information, call context, and HTTP context. As such, with the solution outlined in Figure 2 running on the .NET Framework 2.0, the disconnect between the queued delegate and the context is associated with the work item is worsened.
Fortunately, the .NET Framework team has provided an elegant solution for these types of situations. The new System.Threading.ExecutionContext class contains tons of contextual information about a particular thread and environment, including but not limited to the call stack, the security context, the call context, and the synchronization context. It provides a static Capture method that can be used as a factory to create an ExecutionContext for the current thread. The ExecutionContext.Run static method can then be used with a captured ExecutionContext to execute a ContextCallback delegate under that execution context. Using this, I can rewrite the solution in Figure 2 to better target the .NET Framework 2.0. This new solution is shown in Figure 3.
Figure 3 ThreadPoolPriority for the .NET Framework 2.0
public static class ThreadPoolPriority { private static Queue<QueuedCallback> _priorityQueue = new Queue<QueuedCallback>(), _standardQueue = new Queue<QueuedCallback>(); public static void QueueUserWorkItem( WaitCallback callback, object state, bool isPriority) { if (callback == null) throw new ArgumentNullException("callback"); QueuedCallback qc = new QueuedCallback(); qc.Callback = new ContextCallback(callback); qc.State = state; qc.ExecutionContext = ExecutionContext.Capture(); lock(_priorityQueue) { (isPriority ? _priorityQueue : _standardQueue).Enqueue(qc); } ThreadPool.UnsafeQueueUserWorkItem(HandleWorkItem, null); } private static void HandleWorkItem(object ignored) { QueuedCallback qc; lock(_priorityQueue) { qc = (_priorityQueue.Count > 0 ? _priorityQueue : _standardQueue).Dequeue(); } ExecutionContext.Run(qc.ExecutionContext, qc.Callback, qc.State); } private struct QueuedCallback { public ContextCallback Callback; public object State; public ExecutionContext ExecutionContext; } }
There are a few important points to discuss about this new version. First, in the previous version of ThreadPoolPriority, QueuedCallback's Callback field stored a WaitCallback. In the new version, Callback is a ContextCallback since that is the type ExecutionContext.Run expects as its first argument. However, the public QueueUserWorkItem signature has not changed and still accepts a WaitCallback. Coercing a WaitCallback into a ContextCallback requires a bit of indirection:
qc.Callback = new ContextCallback(callback);
ContextCallback has the same delegate signature as WaitCallback, and thus I can create a ContextCallback that targets the WaitCallback's Invoke method. A call to qc.Callback.Invoke will then result in the execution of callback.Invoke, which is exactly what I want. Of course, in C# a delegate's Invoke method is implied when using this syntax. In other languages, I might have to explicitly specify which method is the target for the ContextCallback delegate. For example, in Visual Basic® the same code would look like:
qc.Callback = New ContextCallback(AddressOf callback.Invoke)
While I'm a big fan of C#, I actually prefer the explicitness of this Visual Basic statement to the previously shown C#.
In addition to the change from WaitCallback to ContextCallback, you'll notice that in a few places I've taken advantage of some of the new C# 2.0 features. Rather than adding a private constructor to the ThreadPoolPriority class in order to prevent external code from instantiating it, I've modified its signature to include the static keyword. This results in the class being compiled to Microsoft intermediate language (MSIL) as:
.class public abstract auto ansi sealed beforefieldinit ThreadPoolPriority extends [mscorlib]System.Object
Since the class is abstract, it can't be instantiated, and as it's sealed, it can't be derived from. Thus, it's impossible to ever have an instance of this class created and can only be used for its static members.
Also worth noting is that I'm taking advantage of generics. Rather than using System.Collections.Queue as I did in Figure 2, I'm using System.Collections.Generics.Queue<QueuedCallback>. This strongly types the Queue so that it can only be used with QueuedCallback structures, improving performance and eliminating the need for me to cast when using Dequeue. I'm also taking advantage of the C# compiler's ability to infer delegate types, so rather than having to write
ThreadPool.UnsafeQueueUserWorkItem( new WaitCallback(HandleWorkItem), null);
I can simply write the following and the compiler generates the equivalent MSIL:
ThreadPool.UnsafeQueueUserWorkItem(HandleWorkItem, null);
Had I wanted to be even fancier, I could have implemented HandleWorkItem inline with QueueUserWorkItem as an anonymous delegate. For more information on the new features in C# 2.0, see Juval Lowy's article in the May 2004 issue of MSDN®Magazine, available at Create Elegant Code with Anonymous Methods, Iterators, and Partial Classes.
Q I've been trying to determine how Guid.NewGuid generates the new GUID. Looking at the code for Guid in the SSCLI and then later using Lutz Roeder's .NET Reflector, I came across a method called by Guid's constructor that looks like the following:
[MethodImpl(MethodImplOptions.InternalCall)] private extern bool CompleteGuid();
The method body is missing, and this odd attribute is applied to the method signature. What does that mean, and where can I find the actual implementation of CompleteGuid?
A There are actually a few questions here, namely what is the MethodImplAttribute, what does it mean for something to be an InternalCall, as a result where can I find the implementation of CompleteGuid, and how does CompleteGuid work? To answer the first question, MethodImplAttribute is what's known as a pseudo-attribute. Unlike custom attributes, pseudo-attributes are not stored as attributes in the metadata; rather, they're used to set bits or fields within the metadata tables. In other words, rather than extending the metadata for a type and being saved along with that type, they actually modify the metadata for a type and are then discarded. For example, the SerializableAttribute is actually a pseudo-attribute that, when applied to a class, sets the CorTypeAttr.tdSerializable metadata bit, while the ComImportAttribute sets the CorTypeAttr.tdImport metadata bit. Similarly, rather than retrieving these attributes from a class using Type.GetCustomAttributes, they're retrieved using Type.Attributes.Serializable and Type.Attributes.Import, respectively. MethodImplAttribute sets the CorMethodImpl metadata bits according to the options specified in its constructor and can be retrieved using MethodBase.GetMethodImplementationFlags.
MethodImplAttribute uses the MethodImplOptions to configure details about the corresponding method implementation. For example, if there is a method that for some reason you want to ensure is never inlined by the just-in-time (JIT) compiler, you can attribute that method with [MethodImpl(MethodImplOptions.NoInlining)]. If you want to specify that the entire method should be single threaded, meaning that only one thread will be able to run that method at a time, you can attribute it with [MethodImpl(MethodImplOptions.Synchronized)]. In fact, with the current implementation of the JIT, on an instance method this is equivalent to surrounding the contents of the method with a lock(this) block. This implies that if multiple instance methods in the same class are attributed with this implementation option, all of those methods will be synchronized around the same sync block (a sync block can be associated with any object and is lazily created for that object the first time a lock is taken on it). As a result, you should shy away from using MethodImplOptions.Synchronized for the same reasons you wouldn't use lock(this).
The extern modifier in C# informs the C# compiler that a method is implemented externally to the class, though it doesn't tell it where it's implemented. You'll often see this keyword used in conjunction with the DllImport attribute, which informs the runtime that the implementation is actually part of an unmanaged DLL. In contrast, MethodImplOptions.InternalCall is used in conjunction with extern to tell the runtime that the method is implemented internally within the system itself. This is done for many of the core .NET Framework methods that are better served by being implemented in unmanaged code. For example, many of the methods on the String, GC, and Math classes are marked as InternalCall. As you've noticed, Guid.CompleteGuid is also an InternalCall.
So how do we find the implementation? The ecall.cpp C++ file (which is located at \clr\vm\ecall.cpp in the SSCLI) contains a table that maps managed internal call methods to unmanaged C++ implementations. Search ecall.cpp for "CompleteGuid" and you'll find the following code:
static ECFunc gGuidFuncs[] = { {FCFuncElement("CompleteGuid", NULL, (LPVOID)GuidNative::CompleteGuid)}, {NULL, NULL, NULL} };
This tells the runtime that the implementation for the managed Guid.CompleteGuid method is actually the native C++ GuidNative::CompleteGuid function. And where is GuidNative::CompleteGuid implemented? It can be found in the comutilnative.cpp file, also located in the \clr\vm\ folder. A cursory examination of this method shows that it simply uses CoCreateGuid to create an unmanaged GUID and then uses memcpy to blit the unmanaged GUID into the managed Guid.
So in the end, Guid.NewGuid relies on CoCreateGuid (and, in turn, on UuidCreate) to generate the globally unique 128-bit identifiers we all know and love. Simplicity at its best.
Send your questions and comments to netqa@microsoft.com.
Stephen Toub is the Technical Editor for MSDN Magazine.