MVP-Submitted: The Power of Custom Workflow Activities (Part 1)

Article
07/03/2013

Maurice de Beijer, www.WindowsWorkflowFoundation.eu, www.TheProblemSolver.nl

Workflow Foundation (WF) is part of the .NET 3.0 Framework alongside Windows Communication Foundation (WCF) and Windows Presentation Foundation (WPF). Where WCF and WPF enable new and improved ways of doing things we have been doing for a long time, the same can’t be said for Workflow Foundation. Using Workflow Foundation enables a developer to create a whole new style of application, the long running and event driven application.

Developing event driven applications isn’t something new, in fact we have been doing that for many years, but typically these event driven applications only run for a relatively short period of time. The principal is simple; the user starts an application, the application activates an event handler of some sort waiting for an event and continues waiting for this event until the user terminates the application. The next time the user starts the application it is a completely separate instance and has no knowledge of the previous state. Naturally there were applications that somehow saved the fact that they were waiting for an event of some sort and could resume the next time they were started but this was something the developer had to program from scratch.

Now, using Workflow Foundation, all this cumbersome saving and restoring state in our own private system is no longer necessary. Workflow Foundation enables applications to persist their state in a standard way. No longer does the application need to check if an event has happened. Now it can inform the workflow runtime that it is interested in an event and this runtime in turn will reactivate our application when this event occurs. This can happen months in the future and even on a completely different machine if needed. These kinds of applications are called workflows.

The workflow runtime is the part of Workflow Foundation that makes sure everything works together. The workflow runtime consists of a number of services, some required and some optional, performing services for the user and the workflows. The workflow runtime itself can be considered a central hub that ties everything together and knows little more than where to send requests. A workflow itself is a unit of work that is required for certain business activity, say processing a purchase order or a loan application. The workflow runtime, or to be more precise the WorkflowLoaderService, is used to create an instance of this workflow. The workflow itself consists of a number of workflow activities that do the actual work. These workflow activities are the basic building blocks of a workflow and can be best compared to reusable classes and functions inside of a regular application. Workflow Foundation includes a number of these activities, performing all sorts of generic tasks. Besides these standard activities a developer can create and use his own activities, something that will often be needed for application or organization of specific tasks. Runtime services, like the WorkflowLoaderService mentioned above, perform all sorts of support activities for executing workflows and its activities. It is also possible, and in fact often required, to create additional custom runtime services.

Why Do We Need Custom Workflow Activities?

Out of the box Workflow Foundation includes a whole range of activities we can use when creating workflows. If we already have this collection of activities why should we need to create additional ones? After all, if we want to execute some code inside of a workflow all we need to do is add a CodeActivity, double click it and write some code in the ExecuteCode event handler. The code we write here may not be very reusable but by placing our code in a separate class we would only need two lines of code to call it. At first glance this might look like a good solution, however, it turns out that this approach has a fundamental drawback due to restrictions of what we can do in the ExecuteCode event.

Take a good look at listing 1. In this very simple example I use a Console.ReadLine() inside of an ExecuteCode activity. At first glance that might seem like no problem but this has some serious implications for our workflow and the workflow runtime.

Private Sub codeActivity1_ExecuteCode( _
ByVal sender As Object, _
ByVal e As EventArgs)

Console.WriteLine("Please enter the input:")
Dim input As String = Console.ReadLine()End Sub

Listing 1. Not a good move, this activity has a blocking statement in its execute

The first major problem is that of scalability. By default the workflow runtime, or more accurately the WorkflowSchedulingService, will start a limited number of concurrent activities. The number depends on your hardware, for a single processor machine this is 5 activities and for a multi processor machine the number is 4 activities per processor. This number is configurable but is always limited to the number of threads in the thread pool. When we start a single workflow with this activity it will appear to work just as expected. However, because the activity blocks the execution, the workflow runtime will assume the activity is busy and not realize that it is in fact waiting on some external input. If we were to start 6 or more workflows with this same activity only the first five workflow activities (assuming a single processor machine) would ask for input, the others would be queued and never start to execute at all even though the first five are really not doing anything.

While that may be a problem we can assume that the requested input will arrive sooner or later and the execution of all workflows will be done. There is however a larger problem and that is state management in long running workflows. The reason workflows can be long running and even survive a machine restart is because the workflow runtime is in control and knows how to persist the workflow state to disk. The state that gets persisted is actually done using the standard .NET binary serializer. This binary serializer saves the state of all fields but does not save a runtime call stack or execution pointer. That means that for the runtime to be able to save the state of a workflow it needs to be completely outside of the call stack. The only way this can happen is if all calls from the workflow runtime into a workflow activity have returned and the workflow runtime is back in control. In the sample code above this was not the case because the ExecuteCode event was blocking, resulting in a runtime that is unable to save the current state and shutdown.

Key takeaway: A workflow activity may never block the execution in the current thread!

Given that sooner or later our workflow will have to wait for some external source, we need a way around this dilemma or else the workflow will not be of much use and certainly not long running. One possible solution is to use a combination of an ExternalDataExchangeService with a CallExternalMethodActivity and a HandleExternalEventActivity pair of activities.

Figure 1. A workflow with a CallExternalMethodActivity used to call an ExternalDataExchangeService

The problem of the blocking Console.ReadLine() call in the first CodeActivity can now be solved using a background thread in the runtime service to ask for the users input. If this sample was started 6 times, assuming the same single processor machine, it would really start all six workflow instances and ask for all six pieces of input. Please note that this implementation is overly simplistic as I am still using the ThreadPool. This means that I still cannot start more than 25 workflows as that would exhaust the ThreadPool. Another problem is that the runtime service keeps no record of the questions asked so it is unable to survive a restart.

Imports SystemImports System.Workflow.ActivitiesImports System.Threading<ExternalDataExchange()> _Public Interface IReadLineService
    Sub ReadLine(ByVal prompt As String)
    Event LineRead As EventHandler(Of LineReadEventArgs)End InterfacePublic Class ReadLineService
    Implements IReadLineService
    Public Event LineRead(ByVal sender AsObject, _
                         ByVal e As LineReadEventArgs) _
        Implements IReadLineService.LineRead

Public Sub ReadLine(ByVal prompt AsString) _
Implements IReadLineService.ReadLine

        Dim instanceId As Guid = WorkflowEnvironment.WorkflowInstanceId
        Dim args As New ReadLineArgs(instanceId, prompt)
        ThreadPool.QueueUserWorkItem(AddressOf ReadLineInternal, args)
    End Sub``    Private Sub ReadLineInternal(ByVal state AsObject)
        Dim args As ReadLineArgs = CType(state, ReadLineArgs)
        Console.WriteLine(args.Prompt)
        Dim input As String = Console.ReadLine()

        Dim e As New LineReadEventArgs(args.InstanceId, input)
        RaiseEvent LineRead(Nothing, e)
    End Sub``    Private Class ReadLineArgs
        Private _instanceId As Guid
        Private _prompt As String``        PublicSub New(ByVal instanceId As Guid, _
            ByVal prompt As String)

            _instanceId = instanceId
            _prompt = prompt
        End Sub``        Public ReadOnly Property InstanceId() As Guid
            Get``                Return _instanceId
            End Get``        End Property``        Public ReadOnly Property Prompt() As String``            Get``                Return _prompt
            End Get``        End Property``    End ClassEnd Class``<Serializable()> _Public Class LineReadEventArgs
    Inherits ExternalDataEventArgs

    Private _input As String``    Public ReadOnly Property Input() As String``        Get``            Return _input
        End Get``    End Property``    Public Sub New(ByVal instanceId As Guid, _
        ByVal input As String)
        MyBase.New(instanceId)
        _input = input
    End SubEnd Class

Listing 2. Requesting input via a ExternalDataExchangeService

While this approach works, and would be reliable with a proper implementation of the ReadLineService, it still leaves something to be desired. There is the fact that we always need a combination of two activities on each workflow, in itself this is not a big problem but it does result in overhead. And the runtime service is quite complicated requiring the use of a class, an interface and an ExternalDataEventArgs subclass. Together these are good reasons why it is often beneficial to create a custom activity.

One question that might arise is why not use a background thread from the CodeActivity itself. There are a few reasons for that. The first is very simple. As soon as the ExecuteCode event returns the workflow runtime marks the activity as completed and starts the next activity, there is no way of telling the workflow that the CodeActivity is still waiting for some external event. Another problem is that even if the workflow runtime did wait there is no way to communicate the fact that some data was entered to the activity and have it continue executing. However, both of these problems are easy to solve with a custom activity.

Creating Custom Activities

The basics of creating your own custom activities isn´t very hard, in fact it is deceptively simple. The real problem is creating activities that will adhere to all the rules of Workflow Foundation and that will run well under all possible circumstances. One complicating factor is that the principals governing Workflow Foundation are new to the average .NET developer. Normally developers are used to creating an object and calling some kind of function on this object, all very straightforward with the developer in control. When developing for Workflow Foundation this is no longer the case as the workflow runtime needs to stay in control. This means you as a developer have to relinquish that control. We don´t create a workflow instance and execute an activity. In this case we ask the workflow runtime to create the workflow instance for us, and what we get back is not an instance of the workflow we specified but a wrapper object of type WorkflowInstance that contains, in a hidden way, the workflow definition. Next we don´t execute the first activity in the workflow but we ask the workflow runtime to schedule this execution. We don´t get any guarantees when it will actually start executing either, it could be the next millisecond or it could be next week, it all depends on the workflow runtime or more accurately on the WorkflowSchedulerService and the current schedule. Only when we adhere to these principals will we create workflow activities that can be dehydrated and rehydrated, as workflow persistence is called, whenever the workflow runtime wants to do so. Let’s start by creating a custom activity that reads the single line from the console.

Imports System.Workflow.ComponentModelImports SystemPublic Class ReadLineActivity
Inherits Activity

    ProtectedOverrides Function Execute( _
        ByVal executionContext As ActivityExecutionContext) _
        As ActivityExecutionStatus

        Dim queueName As String = "MyPrivateQueueName"``        Dim wqs As WorkflowQueuingService
        wqs = executionContext.GetService(Of WorkflowQueuingService)()
        Dim queue As WorkflowQueue = wqs.CreateWorkflowQueue(queueName, True)
        AddHandler queue.QueueItemAvailable, _
            AddressOf queue_QueueItemAvailable

        Dim rls As ReadLineService
        rls = executionContext.GetService(Of ReadLineService)()
        rls.ReadLine("Please enter the input:", queueName)

        Return ActivityExecutionStatus.Executing
    End Function``    Private Sub queue_QueueItemAvailable(ByVal sender AsObject, _
        ByVal e As QueueEventArgs)

Dim executionContext As ActivityExecutionContext
executionContext = CType(sender, ActivityExecutionContext)

        Dim wqs As WorkflowQueuingService
        wqs = executionContext.GetService(Of WorkflowQueuingService)()
        Dim queue As WorkflowQueue = wqs.GetWorkflowQueue(e.QueueName)

Dim data As Object = queue.Dequeue()
wqs.DeleteWorkflowQueue(e.QueueName)

executionContext.CloseActivity()
End SubEnd Class

Listing 3. The ReadLineActivity

In listing 3 we can see how this simple workflow activity is implemented. Even though this is only a very simple example quite a few new concepts appear. First of all there is the ActivityExecutionContext that is passed to the Execute() function as a parameter. This ActivityExecutionContext object is one of the most important objects when it comes to developing custom activities and we will be seeing a lot more of it during the rest of this article. Next we are using the WorkflowQueuingService to create a WorkflowQueue and adding an event handler to this WorkflowQueue. Next the activity uses the ActivityExecutionContext to get a reference to the ReadlineService and start the external request. After this the Execute() function ends by returning the ActivityExecutionStatus.Executing state to the workflow runtime to indicate that the activity isn’t done yet even though the Execute() function itself finished.

Key takeaway: All communications from the workflow runtime to activities use workflow queues.

So when is this activity finished and is the workflow runtime able to schedule the next activity? That occurs in the queue_QueueItemAvailable() function that is called when an item is available in the queue. This function also uses the ActivityExecutionContext, this time passed as the sender, to retrieve the WorkflowQueuingService and through that the WorkflowQueue. After reading the data from the queue the workflow runtime is signaled that it is done by calling the CloseActivity() function on the ActivityExecutionContext. All this code is comparable to what the previous CallExternalMethodActivity and HandleExternalEventActivity combination did as they use the same workflow queuing mechanism behind the scenes.

Imports SystemImports System.ThreadingImports System.Workflow.Runtime.HostingPublic Class ReadLineService
Inherits WorkflowRuntimeService

Public Sub ReadLine(ByVal prompt As String, _
ByVal queueName As String)

        Dim instanceId As Guid = WorkflowEnvironment.WorkflowInstanceId
        Dim args As New ReadLineArgs(instanceId, prompt, queueName)
        ThreadPool.QueueUserWorkItem(AddressOf ReadLineInternal, args)

    End Sub``    Private Sub ReadLineInternal(ByVal state AsObject)
        Dim args As ReadLineArgs = CType(state, ReadLineArgs)
        Console.WriteLine(args.Prompt)
        Dim input As String = Console.ReadLine()

        Dim workflowInstance As WorkflowInstance
        workflowInstance = Runtime.GetWorkflow(args.InstanceId)
        workflowInstance.EnqueueItem(args.QueueName, input, Nothing, Nothing)
    End Sub``    Private Class ReadLineArgs
        Private _instanceId As Guid
        Private _prompt As String``        Private _queueName As String``        Public Sub New(ByVal instanceId As Guid, _
            ByVal prompt As String, _
            ByVal queueName As String)
            _instanceId = instanceId
            _prompt = prompt
            _queueName = queueName
        End Sub``        Public ReadOnly Property InstanceId() As Guid
            Get``                Return _instanceId
            End Get``        End Property``        Public ReadOnly Property Prompt() As String``            Get``                Return _prompt
            End Get``        End Property``        Public ReadOnly Property QueueName() As String``            Get``                Return _queueName
            End Get``        End Property``    End ClassEnd Class

Listing 4. The new ReadLineService

Listing 4 shows the accompanying runtime ReadLineService. This service is much simpler as compared to the previous example and only consists of a single class. This does raise the question about why we still need this runtime service at all. After all, we have full control over the activity execution so couldn’t we just create a background thread in the activity itself? It would simplify things as we no longer need to add the service to the workflow runtime whenever we want to use the activity. Unfortunately it turns out this is impossible due to a number of reasons. The first problem is that we need a reference to the workflow runtime in order to get a reference to the WorkflowInstance that wraps our workflow and activity. This reference is needed to queue the response and from within the activity there is just no way to get a reference to either of these objects. Of course there are ways to get around this, like creating a shared member somewhere and saving the workflow runtime reference there but the designers of Workflow Foundation went through a lot of trouble ensuring we cannot change the runtime from within an activity and it is best to stick to this rule.

Key takeaway: Never change anything on the workflow runtime from within a workflow or activity.

So how about just passing the ActivityExecutionContext to the background thread and using that since we are only interested in closing the activity once we have received the data. Again this won’t work as the ActivityExecutionContext object that is passed as a parameter is disposed as soon as the Execute() call ends and is never reused. While this doesn’t prevent us from hanging on to a reference using it fails with an ObjectDisposedException.

Key takeaway: TheActivityExecutionContext cannot be cached for later use.

How about passing the WorkflowQueue to the background thread? After all, we only need to add the data to the queue and the runtime will take care of the rest. Again this won’t work as trying to add some data to the queue while in another thread will result in an InvalidOperationException with “This method can only be called on a runtime thread.” as its description. It appears there is no way to avoid the extra runtime service but maybe there is a way we can avoid the extra configuration required of the workflow runtime service at runtime.

How about simplifying the work for the workflow developer and adding the runtime service to the workflow runtime from within our custom activity? This will not work because, as previously stated, an activity is not permitted to change the workflow runtime. This is also indicated by the fact that the ActivityExecutionContext does contain a GetService() function, just like the runtime, but there is no matching AddService() function, something the runtime does have.

workflowRuntime.AddService(New ReadLineService())

Listing 5. Adding the ReadLineService to the workflow runtime

The Activity Lifecycle

Before we continue it is important to take a closer look at the lifecycle of an activity. This lifecycle also contains some surprises we need to keep in mind as we develop custom activities. Picture 2 shows this lifecycle. The start of any activity lifecycle is the call to the Initialize() function after which the activity is in the Initialized state. This first Initialize() call gets a parameter of type IServiceProvider but in fact it turns out that it really is an object of the already familiar type ActivityExecutionContext. All status changes in an activity must follow the prescribed path so the only possible next step is the Executing state. Suppose we tried something different, like calling the CloseActivity() from the Initialize () function, we would get an InvalidOperationException with a message saying “Activity status not suitable for closing”. Clearly the workflow runtime enforces its rules!

Figure 2. The activity lifecycle

If we take a close look at picture 2 we see that each state only permits a limited number of state changes. As noted previously a runtime exception is the result of deviating from this path. Each state also has an accompanying overridable function that allows, or sometimes even requires, us to do additional work when that state is reached. These functions are:

Initialize() As soon as the workflow starts.
Execute() When an activity has to do its thing.
OnClosed() When an activity is done. This can be reached via separate routes and may happen more than once.
Cancel() When an activity is canceled. There are multiple ways this can happen.
HandleFault() When an unhandled exception occurs inside of this activity.
Uninitialize() When the workflow is finished. This is the counterpart of the Initialize(). There is no related state as the workflow is done after this event.
Compensate() The Compensate() function can only be called on activities that implement the ICompensatableActivity interface.

Besides the functions in the list above there are two more important overridable functions in the life cycle of an activity. These are the OnActivityExecutionContextLoad() and the OnActivityExecutionContextUnload() and they are called every time an activity is loaded and unloaded. This might appear to be the same as the Initialize() and Uninitialize() functions respectively but there is a difference when we look at workflow persistence. It turns out that the OnActivityExecutionContextLoad() and the OnActivityExecutionContextUnload() functions are also called whenever an activity is loaded from or saved to the persistence store, something that can be useful when an event handler needs to be reinitialized. One function that might be used quite often in normal classes but which is not very useful in Workflow Foundation is the constructor. The main reason is that activities tend to get serialized/deserialized quite a bit, both for persistence and activity cloning, and the constructor doesn’t fire in these cases.

Spawned Context

When we take a good look at the lifecycle diagram above we can see that an activity cannot get to the executing state twice. However in Workflow Foundation there are activities like the WhileActivity and, as the name suggest, it repeats all the contained activities. So how can this happen if an activity can only be executed once? The solution goes by the name of the Spawned Context.

Key takeaway: An activity can only be executed once!

A Spawned Context is really a complicated name for a very simple concept but unfortunately it is something that causes quite a bit of confusion. The basic rule to remember is this: Whenever an activity appears to be executed multiple times the workflow runtime will use the original activity as a template. Every time the activity needs to be executed the workflow runtime will first make a copy, or clone, of the original template activity and execute this copy instead. The result is that any changes to the activities internal fields made during the execution will be done inside of the clones, not the original template, activity. Normally this is no problem but when you write code that expects to find some previous execution result it will not be able to find this.

As a matter of fact it isn’t just the activity to be executed that is cloned but also all nested child activities as well as the ActivityExecutionContext used.

Dim childActivity As Activity = EnabledActivities(0)Dim manager As ActivityExecutionContextManagerDim childContext As ActivityExecutionContextmanager = executionContext.ExecutionContextManagerchildContext = manager.CreateExecutionContext(childActivity)
childContext.ExecuteActivity(childActivity)

Listing 6. Creating a spawned context

The code in listing 6 shows how to go about creating a spawned context. Fortunately this is something we rarely need to do. In fact it is only required when we want to execute our child activity multiple times.

Adding Properties

Sooner or later we also need to add some form of state in the form of properties to our custom activity. And again we are better off doing things a bit different than when developing regular classes. The good thing however is that the change is not Workflow Foundation specific but also applies to Windows Presentation Foundation. While we can work with regular .NET properties inside of our custom activity it is often better to use dependency properties instead. The reason is that dependency properties enable a number of beneficial scenarios.

Public Property Question() As String``    Get``        ReturnCType(GetValue(QuestionProperty), String)
    End Get``    Set(ByVal value As String)
        SetValue(QuestionProperty, value)
    End SetEnd PropertyPublic Shared ReadOnly QuestionProperty As DependencyProperty = _
                       DependencyProperty.Register("Question", _
                       GetType(String), _
                       GetType(ReadLineActivity))

Listing 7. A dependency property inside of a custom activity

One of the most important and visible advantages of using dependency properties is the ability to use activity binding. When using activity binding a property is not set to a specific value but to an expression that redirects retrieving or storing the property value. This expression is actually implemented using an ActivityBind object and assigning in the property sheet with a syntax that looks like: “Activity=ActivityName, Path=PropertyName”. The result is that every time the property is read it really reads the value from the property named PropertyName from the activity named ActivityName. The same holds true when storing a new value. This property in turn can also be a bound property thus creating a whole chain of ActivityBind objects. The final target of an ActivityBind can also be a regular property of field.

Key takeaway: Always use a DependencyProperty instead of a regular property.

A second advantage of a dependency property is the fact that it can be marked as containing metadata. This means that the property can only be set in the designer before the workflow is compiled and not changed at runtime. The Enabled property is an example of a metadata property. While it looks and feels just like the Enabled property of a Windows Forms control the big difference is that a Windows Forms control Enabled status can be changed at runtime while inside of a workflow activity this cannot change at runtime. Therefore it would be better to compare changing the workflow activity Enabled property to changing some code into a comment, something that also requires a recompilation to change. One of the side benefits of this system is that there is no need to save the property value when the workflow is persisted.

A third benefit occurs whenever a workflow needs to be serialized to disk. This serialization is done using a binary format and this could potentially cause errors when de-serializing a workflow if a different version of the assembly is used. This serialization problem could rear its ugly head during long running workflows and if not all assemblies are versioned correctly. When dependency properties are used instead of regular properties all storage is taken care of for us and the likelihood of a de-serialization problem is greatly decreased. To give the developer even more control over the serialization format there is actually an option, using the DependencyPropertyOptions enum, to completely skip some data during serialization.

And that’s not all. There is a fourth benefit to using dependency properties as they allow a developer to create attached properties. An attached property is a property that appears to exist on an activity in the designer but in fact the activity in question doesn’t know about it at all. This is a very powerful feature for adding properties that only make sense in a certain context. An example of this is the WhenCondition property that gets added to nested activities of a ConditionedActivityGroup. This ConditionedActivityGroup then uses the added WhenCondition to check if a certain branch is permitted to be executed, something that only makes sense if the child activity is contained inside of a ConditionedActivityGroup.

Conclusion

Workflow Foundation is a powerful addition to a software developer’s toolkit. Using workflow foundation often requires building your own custom activity classes, something that is not hard to do as long as the basic rules are followed. In this article I described the basics of developing Workflow Foundation activities, in the next article I will drill deeper into activity development and take a look at event driven activities as well as how to deal with long running workflows and designer integration. Although, as the length of this article demonstrates, there are quite a few of these rules to follow, fortunately they aren’t very complex to implement.

About Maurice de Beijer

Maurice de Beijer is an independent software consultant. He has been awarded the yearly Microsoft Most Valuable Professional award since 2005. Besides developing software Maurice also runs the Visual Basic section of the Software Developer Network, the largest Dutch .NET user group. Maurice can be reached through his websites, either www.WindowsWorkflowFoundation.eu or www.TheProblemSolver.nl, or by emailing him at Maurice@TheProblemSolver.nl.