Adding Custom Functions to XPath

Article
06/30/2006

Prajakta Joshi
Microsoft Corporation

October 8, 2002

Summary: Guest author Prajakta Joshi discusses how to create custom functions for XPath using System.Xml APIs from the .NET Framework SDK. Topics include adding extension functions to XPath 1.0, a look forward to XPath 2.0, and using extension functions in XSLT. (14 printed pages)

Requests for extension functions are a commonly discussed topic on the XML and XSL public newsgroups. The motivation to write this article arose from observing a large number of user posts on this topic.

An XPath expression in XPath 1.0 can return one of four basic XPath data types:

String
Number
Boolean
Node-set

XSLT variables introduce an additional type into the expression language, result tree fragment.

The core function library in XPath and a few XSLT specific additional functions offers basic facilities for manipulating XPath data types. A quick glance at these functions reveals that this is not an exhaustive set for all user needs.

XPath Type	Functions
Node set	last(), position(), count(), id(), local-name(), namespace-uri(), name()
String	string(), concat(), starts-with(), contains(), substring-before(), substring-after(), substring(), string-length(), normalize-space(), translate()
Boolean	boolean(), not(), true(), false(), lang()
Number	number(), sum(), floor(), ceiling(), round()
XSLT 1.0 additions	document(), key(), format-number(), current(), unparsed-entity-uri(), generate-id(), system-property()

An XML developer that needs to manipulate a non-XPath data type (such as date) or perform powerful/custom data manipulation with XPath data types often needs additional functions. The purpose of this article is to give an overview of how to implement custom functions for XPath using the System.Xml APIs in the Microsoft .NET Framework SDK.

Comparison of Two Strings

When I was writing my first XPath query, I needed to perform a case insensitive comparison of two strings. My books.xml looked like the following:

<bookstore xmlns:my="urn:http//mycompany.com/">
   <book style="young adult">
      <title>Harry Potter and the Goblet of Fire</title>
      <author>
         <first-name>Mary</first-name>
         <last-name>Gradpre</last-name>
      </author>
      <my:price>8.99</my:price>
   </book>
   <book style="young fiction">
      <title>Lord of the Rings</title>
      <author>
         <first-name>J.</first-name>
         <last-name>Tolkien</last-name>
      </author>
      <my:price>22.50</my:price>
   </book>
</bookstore>

I looked for a function similar to String.Compare() in XPath 1.0 string functions. The closest function I could use in my solution was translate(). I solved my problem with the following piece of code using XPath classes in the System.Xml.XPath namespace:

using System;
using System.Xml;
using System.Xml.XPath;
public class sample
{
   public static void Main(string []args)
   {
      // Load source XML into XPathDocument.
      XPathDocument xd = new XPathDocument(args[0], XmlSpace.Preserve);
      
      // Create XPathNavigator from XPathDocument.
      XPathNavigator nav = xd.CreateNavigator();

      XPathExpression expr;
      expr = 
nav.Compile("/bookstore/book/title[translate(.,'abcdefghijklmnopqrstuvwxyz',
'ABCDEFGHIJKLMNOPQRSTUVWXYZ') = 'HARRY POTTER AND THE GOBLET OF FIRE']");

      XPathNodeIterator iterator = nav.Select(expr);
      // Iterate through selected nodes.
      while (iterator.MoveNext())
      {
         Console.WriteLine("Book title: {0}", iterator.Current.Value);
      }
   }
}

This code produced the following output:

Book title: Harry Potter and the Goblet of Fire

The solution is bit lengthy, but I solved my problem. While writing an XPath query a few days later, I wanted to split a string into an array of substrings at the positions defined by a regular expression match.

<?xml version="1.0" encoding="utf-8" ?> 
<Books>
   <Book>
      <Title>Stephen Hawking's Universe: The Cosmos Explained</Title>
      <Authors>David Filkin, Stephen Hawking</Authors>
   </Book>
   <Book>
      <Title>Writing Secure Code</Title>
      <Authors>Michael Howard, David LeBlanc</Authors>
   </Book>
</Books>

I wanted to find out the n^th author's name from the comma-separated list of <Authors> element—a little complex string manipulation issue. I thought this was a good time to learn how to implement custom extension functions for XPath.

Implementing Extension Functions in XPath

I figured out that the XPath 1.0 recommendation does not define a mechanism for extension functions. However, the good news was that I could provide a custom execution context to the XPath processor to resolve user defined functions and variables in XPath expressions.

The following diagram explains the role of the XsltContext class, the IXsltContextFunction interface, and the IXsltContextVariable interface from the System.Xml.Xsl namespace in my solution.

Figure 1. Role of XsltContext

Key Steps in the Solution

XPathExpression.SetContext(CustomContext) provides the XPath processor (XPathNavigator) with the custom context for resolution of user defined functions and variables. CustomContext, which derives from abstract class XsltContext, implements two key methods, ResolveFunction() and ResolveVariable().
When XPathNavigator sees a user defined function in the XPathExpression, it invokes the ResolveFunction() method on the custom context. ResolveFunction() returns the appropriate custom function, which derives from IXsltContextFunction.
XPathNavigator calls the Invoke() method on this custom function at runtime with the provided arguments.
When XPathNavigator detects a user defined variable in the XPathExpression, it invokes the ResolveVariable() method on the custom context. ResolveVariable() returns the appropriate custom variable that derives from IXsltContextVariable.
XPathNavigator calls the Evaluate() method on this custom variable at runtime.

I decided to write a custom XPath function, Split(), which behaves similar to the RegEx.Split() method in the .NET SDK. Here is how I put all the pieces together.

Role of XsltContext Class

First, I implemented my custom XsltContext in order to provide XPath processor with the necessary information on resolving user-defined functions. ResolveFunction and ResolveVariable are the two key methods of the XsltContext class that one must override to implement custom resolution. These are called at runtime by the XPathNavigator to resolve references to user-defined functions and variables in XPath query expressions.

Notice that I encapsulated an XsltArgumentList object in my CustomContext class. This object is a container for the variables in my XPath expression.

public class CustomContext : XsltContext
{
   // XsltArgumentList to store my user defined variables
   private XsltArgumentList m_ArgList;

   // Constructors 
   public CustomContext()
   {}

   public CustomContext(NameTable nt) : base(nt)
   {
   }

   public CustomContext(NameTable nt, XsltArgumentList argList) : base(nt)
   {
      m_ArgList = argList;
   }

   // Returns the XsltArgumentList that contains custom variable definitions.
   public XsltArgumentList ArgList
   {
      get
      { 
         return m_ArgList;
      }
   }

   // Function to resolve references to my custom functions.
   public override IXsltContextFunction ResolveFunction(string prefix, 
string name, XPathResultType[] ArgTypes)
   {
      XPathRegExExtensionFunction func = null;

      // Create an instance of appropriate extension function class.
      switch (name)
      {
         case "Split":
            // Usage 
            // myFunctions:Split(string source, string Regex_pattern, int n) returns string
            func = new XPathRegExExtensionFunction("Split", 3, 3, new 
XPathResultType[] {XPathResultType.String, XPathResultType.String, 
XPathResultType.Number}, XPathResultType.String);
            break;
         case "Replace":
            // Usage
            // myFunctions:Replace(string source, string Regex_pattern, 
string replacement_string) returns string
            func = new XPathRegExExtensionFunction("Replace", 3, 3, new 
XPathResultType[] {XPathResultType.String, XPathResultType.String, 
XPathResultType.String}, XPathResultType.String);
            break;
      }
      return func;
   }

   // Function to resolve references to my custom variables.
   public override IXsltContextVariable ResolveVariable(string prefix, string name)
   {
      // Create an instance of an XPathExtensionVariable.
      XPathExtensionVariable Var;
      Var = new XPathExtensionVariable(name);

      return Var;
   }

   public override int CompareDocument(string baseUri, string nextbaseUri)
   {
      return 0;
   }

   public override bool PreserveWhitespace(XPathNavigator node)
   {
      return true;
   }

   public override bool Whitespace
   {
      get
      {
         return true;
      }
   }
}

Role of IXsltContextFunction Interface

The next step was to implement the IXsltContextFunction interface for use by my CustomContext class. The Invoke() method on this object is called at runtime by the XPathNavigator with the provided arguments.

public class XPathRegExExtensionFunction : IXsltContextFunction
{
   private XPathResultType[] m_ArgTypes;
   private XPathResultType m_ReturnType;
   private string m_FunctionName;
   private int m_MinArgs;
   private int m_MaxArgs;

   // Methods to access the private fields.
   public int Minargs
   {
      get
      {
         return m_MinArgs;
      }
   }

   public int Maxargs
   {
      get
      {
         return m_MaxArgs;
      }
   }

   public XPathResultType[] ArgTypes
   {
      get
      {
         return m_ArgTypes;
      }
   }

   public XPathResultType ReturnType
   {
      get
      {
         return m_ReturnType;
      }
   }

   // Constructor
   public XPathRegExExtensionFunction(string name, int minArgs, int 
maxArgs, XPathResultType[] argTypes, XPathResultType returnType)
   {
      m_FunctionName = name;
      m_MinArgs = minArgs;
      m_MaxArgs = maxArgs;
      m_ArgTypes = argTypes;
      m_ReturnType = returnType;
   }

   // This method is invoked at run time to execute the user defined function.
   public object Invoke(XsltContext xsltContext, object[] args, 
XPathNavigator docContext)
   {
      Regex r;
      string str = null;

      // The two custom XPath extension functions
      switch (m_FunctionName)
      {
         case "Split":
                r = new Regex(args[1].ToString());
            string [] s1 = r.Split(args[0].ToString());
            int n = Convert.ToInt32(args[2]);
            if (s1.Length < n)
               str = "";
            else
               str = s1[n - 1];
            break;
         case "Replace":
            r = new Regex(args[1].ToString());
            string s2 = r.Replace(args[0].ToString(), args[2].ToString());
            str = s2;
            break;
      }
      return (object) str;
   } 
}

Role of IXsltContextVariable Interface

An XPath expression can contain a user-defined variable reference such as:

XPathExpression expr1 = nav.Compile("myFunctions:Split(string(.), ',', $var)");

To enable runtime resolution of the variable $var, I needed to implement the IXsltContextVariable interface and override the Evaluate() method, which is called at runtime by the XPathNavigator.

public class XPathExtensionVariable : IXsltContextVariable
{
   // The name of the user-defined variable to resolve
   private string m_VarName;

   public XPathExtensionVariable(string VarName)
   {
      m_VarName = VarName;
   }

   // This method is invoked at run time to find the value of the user defined variable.
   public object Evaluate(XsltContext xsltContext)
   {
      XsltArgumentList vars = ((CustomContext) xsltContext).ArgList;
      return vars.GetParam(m_VarName, null);
   }

   public bool IsLocal
   {
      get
      {
         return false;
      }
   }

   public bool IsParam
   {
      get
      {
         return false;
      }
   }

   public XPathResultType VariableType
   {
      get
      {
         return XPathResultType.Any;
      }
   }
}

Putting It All Together

Finally, I used the XPathExpression.SetContext() method, passing it my custom context object. Figure (1 summarizes all the steps in the solution. Notice that the XsltContext class inherits from XmlNamespaceManager, and by using AddNamespace() I added my custom namespace to the collection.

using System;
using System.Xml;
using System.Xml.Xsl;
using System.Xml.XPath;
using System.Text.RegularExpressions;

public class sample
{
   public static void Main(string []argc)
   {
      // Load source XML into XPathDocument.
      XPathDocument doc = new XPathDocument("books.xml", XmlSpace.Preserve);

      // Create XPathNavigator from XPathDocument.
      XPathNavigator nav = doc.CreateNavigator();

      // Add user-defined variable to the XsltArgumentList.
      XsltArgumentList varList = new XsltArgumentList();
      varList.AddParam("var", "", 2);

      // Compile the XPathExpression.
      // Note that the compilation step only checks the query expression
      // for correct XPath syntax.
      // User defined functions and variables are not resolved.
      XPathExpression expr1 = nav.Compile("myFunctions:Split(string(.), ',', $var)");
   
      // Create an instance of a custom XsltContext object.
      CustomContext cntxt = new CustomContext(new NameTable(), varList);

      // Add a namespace definition for myFunctions prefix.
      cntxt.AddNamespace("myFunctions", "http://myXPathExtensionFunctions");

      // Associate the custom context with the XPathExpression object.
      expr1.SetContext(cntxt);

      XPathNodeIterator it = nav.Select("/Books/Book/Authors");
      while (it.MoveNext())
      {
         Console.WriteLine("Authors: {0}", it.Current.Value);
         Console.WriteLine("Second author: {0}", it.Current.Evaluate(expr1));
        }
   }
}

Running this code produced the following output:

Authors: David Filkin, Stephen Hawking
Second author:  Stephen Hawking
Authors: Michael Howard, David LeBlanc
Second author:  David LeBlanc

Other Uses of Extension Functions

Once I discovered the mechanism to use extension functions, I used them in many other data manipulation situations while programming with XPath. Some of the other situations include:

Manipulating dates: Comparing two strings as dates is a common operation. I used the System.DateTime.Compare() method for this purpose.
Manipulating numbers: Math class offers an exhaustive set of methods for common math operations. I used the Abs() method as a custom function.
Manipulating strings: The String class offers an exhaustive set of methods for common string operations. I found the ToUpper() and ToLower() methods very useful as custom functions.

This list is endless indeed. Depending on your particular situation, you can use any of the .NET classes in you custom function.

Future Directions: XPath 2.0

As W3C XML Schema data types become more integrated with XPath 2.0, the Xquery 1.0 and XPath 2.0 functions and operators will offer a richer function library to XML developers than what currently exists in XPath 1.0. This does not completely eliminate the need for user-defined functions in XPath 2.0. Mechanisms for extensions will be indispensable for powerful and extensible query languages for XML.

Extension Functions in XSLT

XslTransform implementation of XSLT 1.0 uses the namespace urn:schemas-microsoft-com:xslt as an extension namespace. It has built-in support for the <msxsl:node-set> extension function and <msxsl:script> extension element.

There are two ways to implement user-defined functions in XSLT:

Style sheet script
Adding an extension object to XsltArgumentList

Acknowledgement

I would like to thank Dare Obasanjo for his help in reviewing this article.

Prajakta Joshi is a member of the Microsoft WebData XML team and works on the XSLT implementation in the System.Xml.Xsl namespace of the .NET Framework.

Feel free to post any questions or comments about this article on the Extreme XML message board on GotDotNet.