Can't Commit?

Volatile Resource Managers in .NET Bring Transactions to the Common Type

Juval Lowy

Code download available at:Transactions.exe(148 KB)

Contents

Transaction Problem Space
Transaction Properties: ACID
Transactions in .NET Framework 2.0
Volatile Resource Managers
Transaction-Based Locks
Serialization and Cloning
Transactional<T> Walkthrough
Transactional Collections
Conclusion

Transactional programming has traditionally been restricted to database-centric applications. Other types of applications did not see the same benefits from this superior programming model. The Microsoft® .NET Framework 2.0 introduces rudimentary support for volatile resource managers, enabling you to enlist transaction support against memory-based resources, such as class member variables. I will begin this article by briefly discussing the problem space that transactions address and the motivation for using transactions, as well as defining some basic terms such as "ACID." I will then introduce the concept of a resource manager, and explain how and to what extent volatile resource managers are supported in the .NET Framework 2.0.

Next, I'll present the required building blocks for properly utilizing volatile resources, and explain how these building blocks can be used to wrap existing types, such as the collections from System.Collections.Generic. Beyond making extensive use of Visual C# ®2.0, the article walks through some advanced .NET Framework 2.0 programming techniques, including type constraining, type cloning, transaction events, and resource enlistment.

Transaction Problem Space

Proper error handling and recovery are the most vulnerable points of many applications. Once an application fails to perform a particular operation, it should recover and restore the system to a consistent state—usually the state the system was in before the operation that caused the error took place. Typically, any operation that can fail is comprised of multiple smaller steps. Some of those steps can fail after others have succeeded.

The problem with recovery is the sheer number of partial success and partial failure permutations that you have to code against. Trying to handcraft recovery code in a decent-size application is often a futile attempt, resulting in fragile code that is susceptible to any change in the application execution or the business use case, incurring both productivity and performance penalties. In addition, because recovery is all about restoring the system to a consistent state (typically the state before the failed operation), you will need to undo all the steps of the operation that have succeeded. But how do you undo a step such as deleting a row from a table, a node from a linked list, or an item for a collection? Not only that, but what if before the operation failed, some other party accessed your applications and acted upon the system state that you are going to roll back during the recovery? That other party is now acting on inconsistent information and, by definition, is in error. Moreover, your operation may be just a step in some other, much wider operation that spans multiple components from multiple vendors on multiple machines. How would you recover the system as a whole in such a case?

Figure 1** Ensuring Consistency Using Transactions **

The best (and perhaps only) way of maintaining system consistency and dealing properly with the error recovery challenge is to use transactions. A transaction is a set of potentiality complex operations in which the failure of any single operation causes the entire set to fail as one atomic operation. As illustrated in Figure 1, while the transaction is in progress, the system is allowed to be in a temporarily inconsistent state. But once the transaction is complete, the system is guaranteed to be in a consistent state: either a new consistent state (State B in the diagram) or the original consistent state the system was at before the transaction started (State A in the diagram). If the transaction executes successfully, and manages to transfer the system from consistent State A to consistent State B, it is called a committed transaction. If the transaction encounters any error during execution and rolls back all the steps that have already succeeded, it is called an aborted transaction. If the transaction fails to either commit or abort, it is called an in-doubt transaction— these usually require administrator or user assistance to resolve.

Transactional programming requires working with a resource such as a database or a message queue that is capable of participating in a transaction, and able to commit or roll back the changes made during the transaction. Such resources have been around in one form or another for decades. Typically, you inform a resource that you want to perform transactional work against it. This is called enlisting the resource in the transaction. You then perform work against the resource, and if no error occurs, you ask the resource to commit the changes made to its state. If you encounter any error, you ask it to roll back the changes. During a transaction, it is vital that you do not access any non-transactional resources (such as the non-transactional file system), because changes made to those resources will not roll back if the transaction is aborted.

Since systems typically contain in-memory state, objects must also manage their state proactively if it's accessed during a transaction. One way of doing this is to store all the state in a transactional resource, such as a database, and have changes to the state commit or roll back as part of the transaction. Consequently, even though transactions offer a superior, robust, productivity-oriented programming model, they are utilized today only by applications that actually do use resources such as transactional databases. Other types of applications generally provide lip service to recovery functionality by manually crafting solutions for the limited set of error cases that the developers know how to deal with, and as a result they suffer the productivity and quality consequences.

Transaction Properties: ACID

The resource's transactions must abide by four core properties: atomic, consistent, isolated, and durable. These are known collectively as the ACID properties.

Atomic means that when a transaction completes, all the changes it made to the resource state must be made as if they were all one indivisible operation. The changes made to the resource are made as if everything else in the universe stops, the changes are made, and then everything resumes. A transaction should not leave tasks to do in the background once it is finished, as this would violate atomicity. Every operation resulting from the transaction must be included in the transaction itself.

Because transactions are atomic, a client application becomes a lot easier to develop. The client does not have to manage partial failure of its requests or have complex recovery logic. The client knows whether the transaction either succeeds or fails as a whole. In case of failure, the client can choose to issue a new request (start a new transaction), or it can choose to do something else, such as alert the user. The important thing is that the client does not have to recover the system.

Consistent means the transaction must leave the system in a logical state. Note that consistency is different from atomicity. Even if all the changes are committed as one atomic operation, the transaction is required to guarantee that all those changes make sense within the context of the system. Usually, it is up to the developer to ensure the semantics of the operations are consistent. All the transaction is required to do is to transfer the system from one consistent state to another.

Isolated means no other entity (transactional or not) is able to see the intermediate state of the resource during the transaction, because it may be inconsistent (in fact, even if the resource's state is consistent, the transaction could still abort and the changes could be rolled back). Isolation is crucial to overall system consistency. Suppose transaction A allows transaction B access to its intermediate state. Transaction A then aborts, and transaction B decides to commit. The problem is that transaction B based its execution on system state that was rolled back and, therefore, transaction B is left unknowingly inconsistent.

Managing isolation is not trivial. The resources participating in a transaction must lock the data accessed by the transaction from all other parties, and must unlock access to that data when the transaction commits or aborts. In theory, various degrees of transaction isolation are possible. In general, the more isolated the transaction is, the more consistent its results will be.

Transactions in the .NET Framework 2.0 by default use the highest degree of isolation, called serialized. This means the results obtained from a set of concurrent transactions are identical to the results obtained by running each transaction serially. To achieve serialization, all the resources a transaction touches are locked from any other transaction. If other transactions try to access those resources, they are blocked and cannot continue executing until the original transaction commits or aborts.

Durable means the results of a successful transaction are persisted in the system. At any moment, the application could crash, and the memory it was using could be erased. If the changes to the system state were in-memory changes, they would be lost, and the system would be in an inconsistent state. However, even if you store the information in the file system, while it can withstand an application crash, the changes will be lost if the disk crashes. And while you can compensate for disk crashes using redundant drives, it will be of no use in case of a fire in the server room. To handle that, you might utilize multiple mirror sites for your resources, putting them in places with no earthquakes or floods.

As you can see, durability is really a range of options. How resilient to such catastrophes the resource should be is an open question that depends on the nature and sensitivity of the data, your budget, available time and available system administration staff, and so on. If durability is a range that actually means various degrees of persistence, then you could also consider one of the extremes on the spectrum: volatile, in-memory resources. The advantage of volatile resources is that they offer better performance than durable resources, and more importantly, as you will see in this article, they allow you to approximate much better conventional programming models, while using transaction support for error recovery.

Transactions in .NET Framework 2.0

The .NET Framework 2.0 automates the act of enlisting and managing a transaction against transactional resources. The System.Transactions namespace offers a common infrastructure for transaction classes, common behaviors, and helper classes. The .NET Framework defines a resource manager as a resource that can automatically enlist in a transaction managed by System.Transactions. The resource has to detect that it is being accessed by a transaction and enlist in that transaction. The System.Transactions namespace supports a concept called an ambient transaction. This is the transaction your code executes in (and it is stored in thread-local storage). To obtain a reference to the ambient transaction, you call the static Current property of the Transaction class:

Transaction ambientTransaction = Transaction.Current;

If there is no ambient transaction, Current will return null. Figure 2 outlines the most relevant portions of the Transaction class. To perform transactional work against a resource, use the TransactionScope class, defined as follows:

public class TransactionScope : IDisposable { public void Complete(); public void Dispose(); public TransactionScope(); ... // Additional constructors }

As the name implies, the TransactionScope class is used to scope a code section with a transaction:

using(TransactionScope scope = new TransactionScope()) { ... // Perform transactional work here // No errors - commit transaction scope.Complete(); }

In its constructor, the TransactionScope object creates a transaction object and assigns it as the ambient transaction by setting the static Current property of the Transaction class. TransactionScope is a disposable object—the transaction will end once the Dispose method is called (the end of the using block).

Figure 2 Transaction Class

[Serializable] public class Transaction : IDisposable, ISerializable { public event TransactionCompletedEventHandler TransactionCompleted; public Enlistment EnlistDurable(Guid resourceManagerIdentifier, IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions); public Enlistment EnlistDurable(Guid resourceManagerIdentifier, ISinglePhaseNotification singlePhaseNotification, EnlistmentOptions enlistmentOptions); public Enlistment EnlistVolatile( IEnlistmentNotification enlistmentNotification, EnlistmentOptions enlistmentOptions); public Enlistment EnlistVolatile( ISinglePhaseNotification singlePhaseNotification, EnlistmentOptions enlistmentOptions); public void Rollback(); public static Transaction Current{ get; set; } ... // Additional members }

The TransactionScope object has no way of knowing whether the transaction should commit or abort. To address this, every TransactionScope object has a consistency bit, which is set to false by default. You can set the consistency bit to true by calling the Complete method (typically, as the last line in the scope, so that any exceptions thrown up to that point will skip the call). If the transaction ends and the consistency bit is set to false, the transaction will abort. For example, the following scope object will roll back its transaction, because the consistency bit is never changed from its default value:

using(TransactionScope scope = new TransactionScope()) { }

On the other hand, if you do call Complete and the transaction ends with the consistency bit set to true, the transaction will try to commit.

System.Transactions is nice to use because you rarely need to interact with the transaction object directly or explicitly enlist resources, nor must you care if you are part of a local or a distributed transaction. By the virtue of being a System.Transactions transactional resource manager, when it accesses the underlying durable resource, it will use Transaction.Current to obtain a reference to the ambient transaction. It will then call one of the EnlistDurable methods, passing an implementation of IEnlistmentNotification, as shown in Figure 3.

Figure 3 IEnlistmentNotification Interface

public interface IEnlistmentNotification { void Commit(Enlistment enlistment); void InDoubt(Enlistment enlistment); void Prepare(PreparingEnlistment preparingEnlistment); void Rollback(Enlistment enlistment); } public class Enlistment { public void Done(); } public class PreparingEnlistment : Enlistment { public void ForceRollback(); public void Prepared(); ... // Additional members }

The System.Transactions transaction manager uses IEnlistmentNotification to tell the resource manager the outcome of the transaction, asking it to commit or abort. In addition, IEnlistmentNotification is used by System.Transactions to manage the two-phase commit protocol. If all of the participating objects in the transaction vote to commit it, the transaction manager calls the Prepare method on each resource:

void Prepare(PreparingEnlistment preparingEnlistment);

The Prepare essentially asks each resource the following question: if you were asked to commit the changes, would you? This is the first phase of the two-phase commit protocol. If the resource wants to abort the transaction, it calls the ForceRollback method of the supplied PreparingEnlistment object. If the resource is capable of committing the changes, it calls the Prepared method.

In the second phase of the protocol, if all of the resource managers have voted to commit the transaction, the transaction manager calls the Commit method of the IEnlistmentNotification interface. If any one of the resource managers have voted to abort the transaction, then the transaction manager calls the Rollback method. This is how System.Transactions is able to maintain both atomicity and consistency across multiple resources.

Since the two-phase commit protocol is overkill when just a single resource is involved, System.Transactions also offers the ISinglePhaseNotification interface:

public class SinglePhaseEnlistment : Enlistment { public void Aborted(); public void Committed(); ... // Additional methods } public interface ISinglePhaseNotification : IEnlistmentNotification { void SinglePhaseCommit(SinglePhaseEnlistment singlePhaseEnlistment); }

The resource manager can choose to implement ISinglePhaseNotification, and enlist using one of the enlisting methods of Transaction that take an ISinglePhaseNotification. In such a case, if that resource is the single resource in the transaction, System.Transactions will only call the SinglePhaseCommit method and avoid conducting the two-phase commit protocol. It is then up to the resource to tell the transaction manager the outcome of the commit attempt, using the SinglePhaseEnlistment parameter.

Volatile Resource Managers

System.Transactions also enables volatile resource managers, which store their state in memory. While their state is not durable, volatile resource managers do often cater to a much wider set of applications than durable resources. If your class member variables (or method local variables) are volatile resources, you can access them from within the transaction, thus significantly simplifying transactional programming.

As far as System.Transactions is concerned, all that a volatile resource needs to do is implement IEnlistmentNotification, detect that it is being accessed by a transaction, and auto-enlist in the transaction by calling the VolatileEnlist method of the transaction object.

There are a number of hurdles in trying to implement IEnlistmentNotification. Ideally, I would like all my data types to be transactional. I like a transactional integer, transactional string, transactional customer object, and so on:

public class TransactionalInteger : IEnlistmentNotification {...} public class TransactionalString : IEnlistmentNotification {...} public class TransactionalCustomer : IEnlistmentNotification {...}

But done this way, I would have to implement IEnlistmentNotification by each type I want to enlist in a transaction. It is therefore much better to use generics and provide a generic implementation of IEnlistmentNotification. To accommodate that, I wrote a Transactional<T> class:

public class Transactional<T> : IEnlistmentNotification { public Transactional(T value); public Transactional(); public T Value { get; set; } public static implicit operator T(Transactional<T> transactional); ... // More members }

Transactional<T> is a full-blown volatile resource manager that automatically enlists in an ambient transaction and commits or rolls back any changes that are made to its state according to the transaction's outcome.

You can use the Value property of Transactional<T> to access the underlying type and call any method or operator it supports, as shown in Figure 4. The values of the class member variable m_Number and the local variable city rolled back to their state before the transaction.

Figure 4 Using Transactional<T>

public class MyClass { Transactional<int> m_Number = new Transactional<int>(3); public void MyMethod() { Transactional<string> city = new Transactional<string>("New York"); using(TransactionScope scope = new TransactionScope()) { city.Value = "London"; m_Number.Value = 4; m_Number.Value++; Debug.Assert(m_Number.Value == 5); //No call to scope.Complete(), transaction will abort } Debug.Assert(m_Number.Value == 3); Debug.Assert(city.Value == "New York"); Debug.Assert(m_Number == 3);//Uses the == operator int number = m_Number;//Uses the casting operator } }

There are a few challenging issues when implementing Transactional<T> or any other volatile resource manager. The first problem is isolation. The volatile resource must adhere to the isolation property of ACID, lock the underlying resource it manages, and prevent access by multiple transactions. However, .NET only offers thread-based locks, which prevent access only by concurrent threads, not concurrent transactions. The second problem is state management. The resource manager needs to enable the transaction to modify the state of the resource and yet must be able to roll back any such changes if the transaction is aborted.

Transaction-Based Locks

Because the .NET Framework 2.0 does not offer a transaction-based lock, I wrote one myself. I called it TransactionalLock, and gave it a simple interface:

public class TransactionalLock { public void Lock(); public void Unlock(); public bool Locked { get; } }

TransactionalLock provides exclusive locking, thus supporting serializable-level isolation. Only one transaction at a time is allowed to own the lock. You acquire the lock by calling the Lock method, and you release it by calling the Unlock method. If another transaction tries to acquire the lock, it is blocked. Once the lock is acquired, if the same transaction calls Lock multiple times it has no further effect. If there are multiple pending transactions trying to acquire the lock then they are all blocked, they are placed in a queue and allowed ownership of the lock in order. If a transaction owns the lock, TransactionalLock will block even non-transactional calls and place those callers in the queue as well. If the transaction is completed (usually aborted or timed out) while it is waiting to acquire the lock, that transaction is unblocked.

It is important to note that TransactionalLock provides transaction-based synchronized access, not thread-based access. The .NET Framework 2.0 allows multiple threads to execute in the scope of the same transaction, so multiple threads can own the lock as long as they are in the same transaction. In addition, if one thread unlocks TransactionalLock, it unlocks it on behalf of all other threads in the transaction, regardless of how many times Lock was called. This is by design, to facilitate automatic unlocking based on transaction completion. If a multithreaded application interacts directly with a TransactionalLock, then the threads need to coordinate among themselves when it is permissible to call Unlock.

Figure 5 shows the implementation of TransactionalLock with some of the code removed for conciseness. Figure 6 and Figure 7 are UML activity diagrams depicting the Lock and Unlock methods, respectively.

Figure 5 TransactionalLock Class

public class TransactionalLock { LinkedList<KeyValuePair<Transaction,ManualResetEvent>> m_PendingTransactions = new LinkedList<KeyValuePair<Transaction,ManualResetEvent>>(); Transaction m_OwningTransaction; // Property provides thread-safe access to m_OwningTransaction Transaction OwningTransaction { get { ... } set { ... } } public bool Locked { get { return OwningTransaction != null; } } public void Lock() { Lock(Transaction.Current); } void Lock(Transaction transaction) { Monitor.Enter(this); if(OwningTransaction == null) { //Acquire the transaction lock if(transaction != null) OwningTransaction = transaction; Monitor.Exit(this); return; } else //Some transaction owns the lock { //We're done if it's the same one as the method parameter if(OwningTransaction == transaction) { Monitor.Exit(this); return; } //Otherwise, need to acquire the transaction lock else { ManualResetEvent manualEvent = new ManualResetEvent(false); KeyValuePair<Transaction,ManualResetEvent> pair = new KeyValuePair<Transaction,ManualResetEvent>( transaction,manualEvent); m_PendingTransactions.AddLast(pair); if(transaction != null) { transaction.TransactionCompleted += delegate { lock(this) { //Pair may have already been removed if unlocked m_PendingTransactions.Remove(pair); } lock(manualEvent) { if(!manualEvent.SafeWaitHandle.IsClosed) { manualEvent.Set(); } } }; } Monitor.Exit(this); //Block the transaction or the calling thread manualEvent.WaitOne(); lock(manualEvent) manualEvent.Close(); } } } public void Unlock() { Debug.Assert(Locked); lock(this) { OwningTransaction = null; LinkedListNode<KeyValuePair<Transaction,ManualResetEvent>> node = null; if(m_PendingTransactions.Count > 0) { node = m_PendingTransactions.First; m_PendingTransactions.RemoveFirst(); } if(node != null) { Transaction transaction = node.Value.Key; ManualResetEvent manualEvent = node.Value.Value; Lock(transaction); lock(manualEvent) { if(!manualEvent.SafeWaitHandle.IsClosed) { manualEvent.Set(); } } } } } }

Figure 6** Operation of the Lock Method **

Let's walk through the Lock method first. TransactionalLock has the m_OwningTransaction member variable of the type Transaction, and the OwningTransaction property provides thread-safe access to it. As long as OwningTransaction is null, the lock is considered unlocked. When the Lock method is called, if OwningTransaction is null, TransactionalLock simply sets OwningTransaction to the current transaction.

Figure 7** Unlock **

If OwningTransaction is not null, TransactionalLock checks whether the incoming transaction (obtained by calling Transaction.Current) is the same as the owning transaction and, if so, the lock does nothing. If, on the other hand, the incoming transaction is different from the owning transaction, TransactionalLock adds that transaction to a private queue it maintains, m_PendingTransactions, which is merely a generic linked list that stores pairs of key and value. The key for each pair is the transaction that tried to acquire the already-owned lock, and the value is a manual reset event. The idea is that the calling transaction will be blocked waiting for that event to be signaled, and the Unlock method will signal it. The problem now is that the blocked transaction can be aborted (perhaps by another thread in the transaction), or it can simply time out and then abort. However, if the calling transaction thread is blocked, how would it know it was aborted?

Fortunately, the Transaction class provides the TransactionCompleted event (see Figure 2) that you can subscribe to. That event is raised on a thread managed by the transaction manager. The Lock method uses an anonymous method to subscribe to that event. If the transaction is still in the queue, the anonymous method removes the transaction from the queue and signals the manual reset event. The Unlock method is even simpler: first it resets the lock by setting OwningTransaction to null. Unlock then removes from the head of the m_PendingTransactions queue the next pair of transaction and matching. It acquires the lock on behalf of the transaction that was at the head of the queue and then unblocks it by signaling the manual reset event.

Serialization and Cloning

The second problem in implementing Transactional<T> is to support rolling back the state of the underlying resource in case a transaction is aborted. In addition, Transactional<T> must also be able to commit the changes as one atomic operation. I've chosen to make a temporary copy of the state of the resource and let the transaction work on that temporary copy. If the transaction aborts, Transactional<T> simply discards the temporary copy. If the transaction commits, Transactional<T> uses the temporary copy as the new state of the resource.

The question then is how do you make a copy of the state of a resource? When dealing with reference types, merely using the assignment operator is not good enough because it only copies the reference. Constraining the type to support ICloneable is not sufficient because there is no telling whether the returned clone is a deep clone or a shallow copy of the reference (it also is not type safe, as IClonable.Clone returns type object). The best solution is to use serialization, because serialization makes a deep copy of the object and all its members. You can serialize an object to a stream and deserialize a deep copy of the object from that stream. This sequence is encapsulated by the static generic Clone method of the ResourceManager static helper class, shown in Figure 8 with some of the code removed for conciseness. Clone takes a parameter called source of the type parameter T, uses the binary formatter to serialize and deserialize it to and from a memory stream, and returns the cloned object.

Figure 8 ResourceManager Helper Class

public static class ResourceManager { public static T Clone<T>(T source) { IFormatter formatter = new BinaryFormatter(); using(Stream stream = new MemoryStream()) { formatter.Serialize(stream, source); stream.Seek(0, SeekOrigin.Begin); return (T)formatter.Deserialize(stream); } } public static void ConstrainType(Type type) { if(!type.IsSerializable) { string message = "The type " + type + " is not serializable"; throw new InvalidOperationException(message); } } }

Choosing serialization as the cloning mechanism has an important consequence: Transactional<T> only works with serializable types. Sadly, C# 2.0 does not support constraining a type parameter to be a serializable type. To compensate, Transactional<T> will check the type parameter T in its static constructor using the ConstrainType method of ResourceManager (see Figure 9). The static constructor will be called before anything else is done with Transactional<T>, and it is a common technique for enforcing constraints that have no support at compile time. ConstrainType verifies that a given type is serializable by accessing the IsSerializable property of the type.

Figure 9 Implementing Transactional<T>

public class Transactional<T> : IEnlistmentNotification { T m_Value; T m_TemporaryValue; Transaction m_CurrentTransaction; TransactionalLock m_Lock; public Transactional(T value) { m_Lock = new TransactionalLock(); m_Value = value; } public Transactional() : this(default(T)) {} static Transactional() { ResourceManager.ConstrainType(typeof(T)); } void IEnlistmentNotification.Commit(Enlistment enlistment) { IDisposable disposable = m_Value as IDisposable; if(disposable != null) disposable.Dispose(); m_Value = m_TemporaryValue; m_CurrentTransaction = null; m_TemporaryValue= default(T); m_Lock.Unlock(); enlistment.Done(); } void IEnlistmentNotification.InDoubt(Enlistment enlistment) { // Bad for a volatile resource, but not // much that can be done about it m_Lock.Unlock(); enlistment.Done(); } void IEnlistmentNotification.Prepare( PreparingEnlistment preparingEnlistment) { preparingEnlistment.Prepared(); } void IEnlistmentNotification.Rollback(Enlistment enlistment) { m_CurrentTransaction = null; IDisposable disposable = m_TemporaryValue as IDisposable; if(disposable != null) disposable.Dispose(); m_TemporaryValue = default(T); m_Lock.Unlock(); enlistment.Done(); } void Enlist(T t) { Debug.Assert(m_CurrentTransaction == null); m_CurrentTransaction = Transaction.Current; m_CurrentTransaction.EnlistVolatile(this,EnlistmentOptions.None); m_TemporaryValue = ResourceManager.Clone(t); } void SetValue(T t) { m_Lock.Lock(); if(m_CurrentTransaction == null) { if(Transaction.Current == null) { m_Value = t; return; } else { Enlist(t); return; } } else m_TemporaryValue = t; } T GetValue() { m_Lock.Lock(); if(m_CurrentTransaction == null) { if(Transaction.Current == null) return m_Value; else Enlist(m_Value); } return m_TemporaryValue; } public T Value { get { return GetValue(); } set { SetValue(value); } } }

Transactional<T> Walkthrough

With the key elements of transactional locking and resource state cloning in place, implementing Transactional<T> is possible. Figure 9 shows the implementation with some of the code removed for clarity.

Transactional<T> has two member variables of type T. m_Value represents the actual consistent state, and m_TemporaryValue is the temporary copy given to a transaction to work on. Transactional<T> also has the m_Lock member of the type TransactionalLock, used to isolate m_Value and allow only serialized access to it. Finally, Transactional<T> maintains a reference to the current transaction in the m_CurrentTransaction member variable. If m_CurrentTransaction is not null, then Transactional<T> is considered to be enlisted in a transaction.

The Value property delegates to the GetValue and SetValue methods. Figure 10 illustrates the operation of the SetValue method. The first thing SetValue does is lock the transactional lock m_Lock. If the volatile resource is owned by another transaction, then this will block SetValue until its turn comes to access the resource. Once the lock is acquired, SetValue checks whether it is already enlisted in a transaction. If m_CurrentTransaction is null and the call has no transaction, SetValue sets the actual value m_Value, since no transaction support is required. If, on the other hand, Transactional<T> is not enlisted, but the call has a transaction, SetValue enlists in the transaction by calling the Enlist helper method. GetValue functions in a similar manner, except it reads rather than writes to the resource.

Figure 10** Operation of SetValue **

The Enlist method sets m_CurrentTransaction to the incoming current transaction and calls the EnlistVolatile method of the current transaction, thus hooking itself up to the transaction manager of System.Transactions. Finally, Enlist uses ResourceManager.Clone to make a deep copy of the resource and then assigns that to m_TemporaryValue. Note that EnlistVolatile accepts an implementation of IEnlistmentNotification, which Transactional<T> implements explicitly. IEnlistmentNotification is used by the transaction manager to ask Transactional<T> to vote during the two-phase commit protocol and to notify Transactional<T> about the outcome of the transactions (instructing it to commit or abort). Since Transactional<T> will always commit if asked to do so, Prepare simply calls the Prepared method of the preparing element.

Implementing IEnlistmentNotification.Commit and IEnlistmentNotification.Rollback is now straightforward. Commit disposes of m_Value (if it offers an IDisposable), copies the reference to the temporary value in m_TemporaryValue to the actual value m_Value, thus committing the transaction, and unlocks the lock, allowing another transaction to set or get the value of the volatile resource. IEnlistmentNotification.Rollback simply disposes of m_TemporaryValue and then sets m_Value back to its default value, thus discarding the changes, and unlocks the lock.

Transactional Collections

The generic type parameter T for Transactional<T> can be any serializable type. However, you are most likely to use it with two main categories of data structures or resources. The first category includes primitive types like integers and strings or custom objects like a customer or an order. Because there is an unlimited number of such types, what Transactional<T> offers is good enough. Whatever the type may be, as long as it is serializable, you can treat it as a volatile resource manager and access it through Value.

The second category is a collection of individual items such as arrays, linked lists, queues, and so on. You can specify such a collection as the type parameter and access it through the Value property, as shown in the following:

Transactional<int[]> numbers = new Transactional<int[]>(new int[3]); numbers.Value[0] = 1; numbers.Value[1] = 2; numbers.Value[2] = 3; using(TransactionScope scope = new TransactionScope()) { numbers.Value[0] = 11; numbers.Value[1] = 22; numbers.Value[2] = 33; scope.Complete(); } Debug.Assert(numbers.Value[2] == 33);

However, such use of the type parameter leads to somewhat cumbersome programming. I would like to use a transactional array as if it were a normal array and a transactional linked list as if it were a normal linked list, without the need to always dereference it through Value. Since there are only a handful of such useful collections, I defined all the collections in System.Collections.Generic as transactional collections. As an example, TransactionalQueue is shown in Figure 11.

Figure 11 Transactional Collections

public abstract class TransactionalCollection<C,T> : Transactional<C>, IEnumerable<T> where C : IEnumerable<T> { public TransactionalCollection(C collection) { Value = collection; } IEnumerator<T> IEnumerable<T>.GetEnumerator() { return Value.GetEnumerator(); } IEnumerator IEnumerable.GetEnumerator() { return ((IEnumerable<T>)this).GetEnumerator(); } } public class TransactionalQueue<T> : TransactionalCollection<Queue<T>,T>, ICollection { public TransactionalQueue(int capacity) : base(new Queue<T>(capacity)) {} public void Enqueue(T item) { Value.Enqueue(item); } public T Dequeue() { return Value.Dequeue(); } } ...//additional collections

These collections are completely polymorphic with the collections available in System.Collections.Generic. They offer the same methods, and they implement implicitly or explicitly the same interfaces as their respective non-transactional cousins. As a result, they are used in the same way. Here's an example:

TransactionalArray<int> numbers = new TransactionalArray<int>(3); numbers[0] = 1; numbers[1] = 2; numbers[2] = 3; using(TransactionScope scope = new TransactionScope()) { numbers[0] = 11; numbers[1] = 22; numbers[2] = 33; } Debug.Assert(numbers[2] == 3);

Figure 12 shows an example of using TransactionalQueue as a message queue. Any message added to the queue is rejected if the enqueuing transaction is aborted. Any message removed from the queue is added back if the dequeuing transaction is aborted. This enables programming models similar to System.Messaging and Windows Communication Foundation MSMQ binding, where you have automatic canceling (enqueuing is rolled back), guaranteed delivery, and an auto-retry mechanism (processing the message fails, so the dequeuing rolls back).

Figure 12 TransactionalQueue as a Message Queue

[Serializable] class MyMessage { ... } TransactionalQueue<MyMessage> messageQueue = new TransactionalQueue<MyMessage>(); messageQueue.Enqueue(new MyMessage()); using(TransactionScope scope = new TransactionScope()) { messageQueue.Enqueue(new MyMessage()); messageQueue.Enqueue(new MyMessage()); messageQueue.Enqueue(new MyMessage()); Debug.Assert(messageQueue.Count == 4); } Debug.Assert(messageQueue.Count == 1);

To automate parts of the implementation (all collections need to support IEnumerable<T>), the transactional collections derive from the abstract class TransactionalCol-lection<C,T>, which, in turn, derives from Transactional<C>, passing C as the type parameter to it. C is constrained to support IEnumerable<T>, and Value is assigned a value of the type C in the constructor. As a result, TransactionalCollection<C,T> can delegate to Value the implementation of IEnumerable<T>. In each of the concrete transactional collections, implementing the various methods and properties is done by delegating the implementation to the methods and properties now exposed by Value. For each concrete transactional collection, Value exposes the methods it requires because each concrete transactional collection specifies the collection it wraps as the type parameter C when deriving from TransactionalCollection<C,T>:

public class TransactionalList<T> : TransactionalCollection<List<T>,T>, IList<T>,ICollection<T> {...}

You can also use TransactionalCollection<C,T> as the base class for your own custom transactional collections.

Conclusion

I believe that transactional programming, from durable to volatile resources will be the predominant programming model of the future. Volatile resource managers allow you to unleash the fundamental benefits of transactions to everyday objects and common data structures—from integers to dictionaries. Adding this transactional functionality requires minimal changes to the conventional programming model. By eliminating the need to handcraft error handling and recovery, you gain productivity, quality, and faster time to market. This also lowers the cost of long-term maintenance and conforming to new requirements and scenarios.

Juval Lowy is a software architect providing .NET architecture consultation and advanced training. He is the Microsoft Regional Director for the Silicon Valley. His latest book is Programming .NET Components, 2nd Edition (O'Reilly, 2005). Contact Juval at www.idesign.net.