Security Briefs

Active Directory Cache Dependencies

Keith Brown

Code download available at:  SecurityBriefs2007_07.exe(182 KB)

Contents

LDAP and SQL Queries
Caching Properties and Objects
SqlDependency and Smarter Caching
The DirSync Control
System.DirectoryServices.Protocols
ADDependency Usage Model
Drilling Deeper
For More Information

If you are in the process of building an application that can be used anywhere in an enterprise, chances are you could benefit by integrating with Active Directory®. There’s a wealth of information right there for the taking if you’re willing to invest just a little time to learn how to program against one of the many APIs that Active Directory supports.

Suppose, for example, your application already uses integrated Windows® authentication to validate a client’s domain identity. You’ve already taken the first—and arguably the biggest—step toward integrating with Active Directory. You are already making use of the user’s existing Active Directory credentials, rather than creating another private user database. This is a great choice because you are making it easy for your application to run in the enterprise without requiring the user to have special credentials for your application.

You may even be taking advantage of techniques such as role-based security, which uses groups in Active Directory to determine the user’s privilege level. You can test for the presence of a particular group by calling the IsInRole method on the WindowsPrincipal class. This is very common.

Take it one step further, though, and things start to get really interesting. If your application is running under a domain account (even a low-privileged one), you should have no problem making queries to Active Directory to learn more about the user. You may be unaware of this, but even if your application is running as Network Service on a domain machine in the lab, that application’s process is represented by the domain credentials of that machine when you use integrated authentication to make secure calls on the network, including calls to Active Directory.

So, alongside that call to IsInRole, you might consider adding code such as that in Figure 1 to retrieve your client’s e-mail address or phone number in an application. Any managed application, from an ASP.NET application to a Windows Communication Foundation service to a Windows Forms desktop app, can benefit from Active Directory integration.

Figure 1 From WindowsIdentity to E-Mail

string email = LookupEmail((WindowsIdentity)User.Identity)

...

public string LookupEmail(WindowsIdentity id) 
{
    string path = string.Format(“LDAP://<SID={0}>”, id.User);
    using (DirectoryEntry user = new DirectoryEntry(
        path, null, null, SECURE_BINDING)) 
    {
        return (string)user.Properties[“mail”].Value;
    }
}

const AuthenticationTypes SECURE_BINDING =
    AuthenticationTypes.Secure | AuthenticationTypes.Signing |
    AuthenticationTypes.Sealing;

There are many useful attributes, such as mail, that you can use to make your application work more seamlessly inside an enterprise. Want to send a friendly e-mail to someone and have it addressed with the user’s first name? The attribute you want is givenName. Want to get the e-mail address of the user’s personal assistant? Just bind to the assistant object for your user and look up his mail attribute.

LDAP and SQL Queries

My code sample in Figure 1 uses a namespace called System.DirectoryServices, which is one of the easiest ways to start integrating with Active Directory. Under the covers, this managed library uses a COM library called Active Directory Services Interface, or ADSI. Under that is a native C library called the LDAP API, where LDAP stands for Lightweight Directory Access Protocol. One of the reasons it’s called lightweight is because it runs over TCP/IP, but that’s a story for another day. Figure 2 illustrates this stack. (I’ll refer back to this figure later in the column.)

Figure 2** Directory Services Stack **

When you communicate with Active Directory, you’re ultimately using LDAP, which is both a network protocol and a programming model with its own query language. Simple queries such as the one shown in Figure 1 are quite trivial to implement.

That’s not necessarily the case when you try to do something such as build an org chart, which is entirely possible since Active Directory allows organizations to track relationships among users. Take for example the manager, directReports, department, and title attributes on a user object in Active Directory. Now, if this sort of data were stored in SQL Server™, it would be possible to generate an org chart with very few round-trips to the database. Using the SQL JOIN operator, you can ask the database to build a report and ship it back to you.

LDAP doesn’t have a JOIN operator. And Active Directory doesn’t have anything akin to stored procedures that you can write to put together custom reports like this on the server side. So when your LDAP queries start to get more complex, you may find yourself doing manual joins on the client to put together the data you need. This can result in many round-trips to the domain controller—if you’re not careful, your application might end up wasting network bandwidth while the user waits impatiently at the console for a response.

Caching Properties and Objects

So what’s the solution? Given that directories are designed for read-often, write-seldom operations, a temporary client-side cache is a logical way to help reduce the overhead of complex LDAP queries. In fact, ADSI already implements a property cache for this reason. When the code in Figure 1 asks for the LDAP attribute called mail, the DirectoryEntry class actually downloads all the attributes on the user object making them available locally (in case you’re wondering, you can control this using the RefreshCache method on DirectoryEntry to select exactly which attributes you care about).

The property cache is useful for caching the attributes of a single object. However, when you’re dealing with many objects, such as in the org chart example, you need to cache those objects, not just their properties. And then the question comes up, "Now that I have this cache of objects, how long can I keep it around?" It seems wasteful to just throw this information away if there is a possibility that you will soon need the data again.

ASP.NET has a caching infrastructure that you can use to help manage a local cache of directory objects. It’s pretty easy to drop something in the cache, specifying a time in the future when the cache should become invalid. If you can’t find what you’re looking for in the cache, you know it has expired and you can then go back to the directory and refresh the cache. Here’s the basic pattern:

List<DirectoryEntry> users = Cache[“users”];
if (null == users) 
{
    // refresh cache from Active Directory
    users = loadUserListFromAD();
    Cache.Add(“users”, users, null, DateTime.Now.AddMinutes(5),
        Cache.NoSlidingExpiration, CacheItemPriority.Normal, null);
}
displayOrgChart(users);

If you’re getting a lot of requests for this org chart, a cache such as this can be a lifesaver.

SqlDependency and Smarter Caching

The release of ADO.NET 2.0 and SQL Server 2005 brought a new class to help manage local data caches. SqlDependency, in the System.Data.SqlClient namespace, uses a feature of SQL Server 2005 called query notifications. In managed code, you can create an instance of SqlDependency and give it a SqlCommand with a query. It will then notify you if the data represented by that query has changed. To limit the burden on the database, the query you watch has some limitations; but keep in mind that the query you watch does not necessarily have to be exactly the same as the query you use to retrieve the data. The query you use as a dependency could be a superset of the query you actually execute, and you can still reap the benefits of this feature.

This got me thinking. If only there was a corresponding ADDependency class! I’ve done some research in the past trying to figure out how to get notifications from Active Directory when its data changes because local caching of LDAP data is very common. I never did have much luck finding a solution. But I recently chatted with Joe Kaplan (Microsoft MVP and Directory Services guru) and he mentioned the DirSync control. A day later, I had built and started testing the ADDependency class I had always wanted.

The DirSync Control

LDAP version 3.0 includes controls that allow a directory service to expose extended features. For most programmers who use the Microsoft® .NET Framework, the term LDAP control probably conjures up thoughts of Windows Forms controls or ASP.NET server-side controls. But all an LDAP control really consists of is a way to send extra metadata along with an LDAP command to control how it works.

Active Directory exposes several proprietary LDAP controls. The one I am going to use here is called DirSync and, as its name implies, it’s all about synchronizing other data stores with data in Active Directory.

If you think about the way Active Directory works, it’s naturally designed for synchronizing data. One of the most important features of the directory is its ability to replicate data among domain controllers (DCs) in an efficient manner. Objects and properties in Active Directory are stamped with 64-bit Update Sequence Numbers (USNs). I’m dramatically simplifying the story, but imagine that a DC maintains a current USN and increments it anytime anything changes, stamping the object and attributes that changed with the current USN. Each time a sibling requests replication from another DC, the DC need only send the last USN it received from that partner, and the receiver will be able to find the objects and attributes that have changed since the last replication to that particular partner.

One major goal of the Active Directory implementation is to avoid swamping the network with replication traffic. This is why Active Directory pays so much attention to tracking data version numbers. It’s pretty obvious how it works once you learn a little about Active Directory replication, but you can also find some documentation at msdn2.microsoft.com/ms677626.aspx about polling for changes using DirSync.

You use the DirSync control in conjunction with a search request. When you make your request, you also send along this extra metadata (the DirSync control data), which includes a synchronization cookie. On your very first request to Active Directory with a DirSync control, you send a blank cookie and Active Directory responds with all the results of your search, along with a cookie value, which is a byte array. When you want to find out if any changes have taken place since you made your last query, you just send that same cookie back via the DirSync control metadata in your next search request.

While the cookie is opaque, it’s fairly obvious what’s in it. Among other things, you would imagine there being some sort of version number, perhaps even a USN as used in replication. So what do you get back in your second request? Only the objects and attributes that have changed since your first request! It turns out that by using the DirSync control, you can actually get more effective caching than you get with SqlDependency because the directory will send you the deltas, which you can use to keep your cached version up-to-date with the least possible impact on the network. This is a very good thing if you are synchronizing or, as in my case, caching lots of data.

System.DirectoryServices.Protocols

It turns out that there are actually two mainstream techniques you can use to program Active Directory using the .NET Framework 2.0 or later. You can use code similar to what I used in Figure 1, in which case you’re using System.DirectoryServices and the ADSI stack shown in Figure 2. But version 2.0 of the Framework came along with a new API for talking to LDAP directories. This new interface is entirely managed and completely divorced from ADSI, providing a breath of fresh air in many ways. ADSI adds its own set of abstractions on top of LDAP, such as the property cache and implicit connection management, which, while convenient, can be a source of confusion and sometimes makes it a bit unclear when an actual LDAP network request is going to take place.

The newer System.DirectoryServices.Protocols namespace carries none of this ADSI baggage. It’s a lower-level interface that sits right on top of the native LDAP API in WLDAP32.DLL. What I like about this interface is its explicit nature. You know when a TCP/IP connection for LDAP is open because you’re the one who opened it by calling LdapConnection.Bind. You know exactly when network traffic is going to occur because you call LdapConnection.SendRequest to make all of your requests, simply sending different classes of messages depending on what you want to accomplish. And if you want to go asynchronous, there’s the expected BeginSendRequest and EndSendRequest.

Because there are very few samples out there highlighting this new namespace, I decided to dive in head first and build ADDependency using this model. I was surprised at how easy and natural it was to use. But let me warn you up front: The documentation for this new API is rather scarce as of this writing. If you’re new to Active Directory, you may find it quite a bit easier to get started using the traditional ADSI model through System.DirectoryServices and then checking out this new API once you’ve gotten your head around the basics.

ADDependency Usage Model

I modeled ADDependency after SqlDependency. If you’ve used one, it won’t be terribly difficult to use the other. Figure 3 shows the public interface to ADDependency. You start by constructing an instance and specifying at least a search root and LDAP filter, such as:

Figure 3 Public Interface for ADDependency

public ADDependency(string dnSearchRoot, string ldapFilter,
    params string[] attrsToWatch) : 
        this(“localhost”, dnSearchRoot, ldapFilter,
            DirectorySynchronizationOptions.ObjectSecurity, attrsToWatch)
{
}

public ADDependency(string adServer, string dnSearchRoot, 
    string ldapFilter, DirectorySynchronizationOptions dirSyncOptions,
    params string[] attrsToWatch) 
{
    this.adServer = adServer;
    this.dnSearchRoot = dnSearchRoot;
    this.ldapFilter = ldapFilter;
    this.attrsToWatch = attrsToWatch;
    this.dirSyncOptions = dirSyncOptions;
}

public void Start() 
{
    if (null != conn) throw new InvalidOperationException(
        “You may only call Start one time.”);
    conn = new LdapConnection(new LdapDirectoryIdentifier(adServer),
        null, AuthType.Negotiate);
    conn.Bind();

    timer = new Timer(timerCallback, null, 
        TimeSpan.FromSeconds(0), pollingInterval);
}

public void Stop() 
{
    timer.Dispose();
    conn.Dispose();
}

public event ChangedEventHander Changed;

public delegate void ChangedEventHander(object sender,
    ChangedEventArgs args);

public class ChangedEventArgs : EventArgs 
{
    public ChangedEventArgs(SearchResponse response) 
    {
        this.Response = response;
    }
    public readonly SearchResponse Response;
}

ADDependency dep = new ADDependency(
    “dc=contoso,dc=com”, “(objectclass=user)”);

Once you have an instance of the dependency, you need to keep it around for the duration of whatever data you’re synchronizing or caching—ADDependency will manage the cookie and an underlying LDAP connection to the server. Now you need to wire up to the Changed event to receive change notifications, like so:

dep.Changed += OnChange;
...
void OnChange(object sender, ChangedEventArgs args) 
{
    foreach (SearchResultEntry e in args.Response.Entries) 
    {
        Console.WriteLine(e.DistinguishedName);
    }
}

The simple handler shown here just prints out the names of all objects that changed recently. In a real implementation, you’d want to update the data in your cache with the attributes on each object in the collection or invalidate a cache and eventually come around and repopulate it by rereading data from the directory.

When you’re ready to open the connection and start the query, simply call start:

dep.Start();

This will start a System.Threading.Timer instance that will, by default, poll Active Directory every N seconds looking to see if anything has changed. N is a constant that you should adjust to balance polling activity with cache coherency. Once running, as long as nothing has changed, this request isn’t very expensive; it’s just a round-trip to the server to find that nothing has changed. But note that the very first time this fires it’s going to return a list of all the objects so that you can initialize your cache.

When you’re finished and want to close the connection, call the Stop method. The current design doesn’t allow you to restart the instance once it’s been stopped (you’ll see an exception if you attempt to do this).

When comparing ADDependency and SqlDependency, you may have noticed a major difference in how they operate. ADDependency polls the directory every N seconds and receives deltas with each request so the user can incrementally update the local cache. SqlDependency, on the other hand, does no polling—it receives notifications pushed from SQL Server 2005 and the caller must then go back to the directory to retrieve the actual data, as opposed to retrieving deltas.

Is polling a bad thing? It can be if you do too much of it. But given that directory data doesn’t change very much, your polling interval shouldn’t be very frequent. As with many decisions, it’s best to measure and decide if this strategy makes sense for you given the data access pattern your application uses.

Drilling Deeper

If you look closely at Figure 3, you’ll see that there’s more to the constructor on ADDependency than I’ve shown so far. For example, you can (and should) pass a list of LDAP attributes that you care about, such as mail, title, manager, and so on. This will limit the size of the initial data you receive and any subsequent deltas. It will also reduce the load on the DC.

On the second, more interesting, constructor, you must specify a particular DC. (The constructor in Figure 1 assumes you’re running on a DC and connects to localhost, which is fine in a lab setting but isn’t likely in an enterprise.) Specifying a particular DC and sticking with it is important. This is because at any given instance in time, two DCs may not necessarily agree on what the truth is about a particular object. Remember, although updates in Active Directory may be relatively infrequent when compared to a SQL database, they do happen occasionally and replication takes a finite amount of time. Besides, it’s not documented whether a DirSync cookie issued by one DC would even make any sense if given to a different DC! However, you should also make the DC configurable rather than hardcoding its name into your app. This could be done via Group Policy, via an application configuration file, or from a method such as System.DirectoryServices.ActiveDirectory.Domain.GetCurrentDomain, which is used to retrieve current domain information for the user.

Just like the queries used with SqlDependency, there are some limitations on the queries you can perform with ADDependency. The search root must be the root of a partition. Note in my example how I set the search root to dc=contoso,dc=com. This is the root of the application partition for the directory hosted at contoso.com. But if I had instead tried to use cn=users,dc=contoso, dc=com to narrow my search, the Start method would have thrown an exception. This is simply a limitation of the DirSync control. You’ll need to use the ldapFilter and attrsToWatch arguments to throttle the data you watch instead.

The magic that makes all this work comes from the DirSyncRequestControl and DirSyncResponseControl classes (see Figure 4). This is how the synchronization cookie (as well as a set of synchronization options) gets passed back and forth to the server. The most important option here is ObjectSecurity, which is only supported by Windows Server® 2003. Without this, you must be highly privileged to use the DirSync control. With this option, you can use a low-privilege account and you’ll only be able to see changes to objects that you would normally be allowed to read, which is probably what you want.

Figure 4 ADDependency Implementation

private void timerCallback(object o) 
{
    beginPollDirectory();
}

private void beginPollDirectory() 
{
    SearchRequest request = new SearchRequest(dnSearchRoot,
        ldapFilter, SearchScope.Subtree, attrsToWatch);
    request.Controls.Add(
        new DirSyncRequestControl(cookie, dirSyncOptions));
    conn.BeginSendRequest(request, requestTimeout,
        PartialResultProcessing.NoPartialResultSupport,
        endPollDirectory, null);
}

private void endPollDirectory(IAsyncResult asyncResult) 
{
    // harvest results
    SearchResponse response = 
        SearchResponse)conn.EndSendRequest(asyncResult);

    // grab the current search cookie
    foreach (DirectoryControl control in response.Controls) 
    {
        DirSyncResponseControl dsrc = control as DirSyncResponseControl;
        if (null != dsrc) cookie = dsrc.Cookie;
    }

    // if anything changed, notify any listeners
    if (response.Entries.Count > 0 && null != Changed)
    {
        Changed(this, new ChangedEventArgs(response));
    }
}

Note the sequence of events. First, the timer fires, initiating a call to timerCallback. This method leads to an asynchronous request being sent to the DC via LdapConnection.BeginSendRequest. The result is collected in the endPollDirectory method of ADDependency, which then fires off its Changed event, which packages the search response in its event arguments.

For the full implementation of ADDependency, download the sample code associated with this article. It includes a few examples: a console application that monitors user objects in the directory and prints out when they change, an ASP.NET application that implements a custom CacheDependency called ADCacheDependency, and a console application that acts as a random agent of change, having users randomly trade job titles every N seconds. (Ever worked at a place like that?) ADCacheDependency invalidates an ASP.NET cache item when a directory change is detected, similar to the way SqlCacheDependency works.

Before I finish, I want to mention the DirectoryNotification LDAP control, which is an area that deserves future exploration. I’ve not yet seen a single example that uses it, but it appears to allow you to configure an asynchronous search and receive results when the search would return something interesting (changed). It may turn out to be an even lighter-weight technique for detecting changes if you plan on rereading all cached objects from the directory, which is exactly how SqlCacheDependency works in ASP.NET.

For More Information

If this column has piqued your interest in directory service programming, let me give you a tip: Pick up a copy of The .NET Developer’s Guide to Directory Service Programming, by Joe Kaplan and Ryan Dunn. There aren’t many books out there that cover programming Active Directory from managed code, and this is the one you’ll find on my desk whenever I’m doing this type of work.

Now what are you waiting for? Go write some code to find out what treasures await in your local directory!

Send your questions and comments for Keith to  briefs@microsoft.com.

Keith Brown is a cofounder of Pluralsight, a premier Microsoft .NET training provider. Keith is the author of Pluralsight’s Applied .NET Security course as well as several books, including The .NET Developer’s Guide to Windows Security, which is available both in print and on the Web.