Handling Language Interoperability with the Microsoft .NET Framework

  • Article

 

Damien Watkins
Monash University

October 2000

Summary: This article outlines the interoperability capabilities of the Microsoft .NET Framework. (17 printed pages)

Contents

Introduction Common Language Runtime
Assignment Compatibility
Metadata Common Language Specification Conclusion

Introduction

With the increasing use of distributed systems, interoperability is a major issue to system developers. The problems of interoperability have been around for many years, and a number of standards and architectures have been developed to address some of these issues, with varying degrees of success. For example:

  • Representation standards, such as External Data Representation (XDR) and Network Data Representation (NDR), address the issue of passing data types between different machines. These standards compensate for such matters as big-endian and little-endian issues, and different word sizes.
  • Architecture standards, such as the Distributed Computing Environment's (DCE) Remote Procedure Call (RPC), the Object Management Group's (OMG) Common Object Request Broker Architecture (CORBA), and Microsoft's Component Object Model (COM), address the issue of calling methods across language, process, and machine boundaries.
  • Language standards such as ANSI C allow the distribution of source code across compilers and machines.
  • Execution environments, such as those provided by the virtual machines (VM) of SmallTalk and Java, allow code to execute on different physical machines by providing a standardized environment for execution.

All of these approaches provide significant benefits to the application developer, but none have solved or even addressed all of the problems associated with a distributed computing environment. One problem that still deserves more attention is the issue of language interoperability. Language interoperability does not just refer to a standardized calling model, such as COM and CORBA, but a schema that allows classes and objects in one language to be used as first class citizens in another language. Ideally, it should be possible for Python code to instantiate a C++ object that inherits from an Eiffel class. This article examines one such architecture, the Microsoft .NET Framework.

Common Language Runtime

A core component of the .NET Framework architecture is the common language runtime. The runtime consists of three main components:

  1. A type system, which supports many of the types and operations found in modern programming languages.
  2. A metadata system, which allows metadata to be persisted with types at compile time and then interrogated by the execution system at run time.
  3. An execution system, which executes .NET Framework programs, utilizing the metadata system information to perform services such as memory management.

On one level, the runtime may be considered as the union of the features of many programming languages. It defines standard types, permitting developers to define their own types and specify function prototypes and implementations for their types. The runtime readily supports an object-oriented programming style, with features such as classification, inheritance, and polymorphism. The runtime can also support non-object-oriented languages.

Figure 1. Elements of the common language runtime

Figure 1 shows one view of the relationship between elements of the runtime. A source file at the top of diagram may hold a definition of a new type in a number of languages, such as C++. This new type will inherit from a type in the .NET Framework libraries, such as Object. When this file is compiled by a .NET Framework C++ compiler, the resulting Microsoft Intermediate Language (MSIL) is persisted in a file along with the new type's metadata. The metadata format used is independent of the programming language in which the type was defined. Once the MSIL for this new type exists, other source files, possibly written in other languages, such as C#, Eiffel, Python, or Visual Basic®, can import the file. The C++ type can then be used within a Python source code file just as if it were a Python type. The process of importing types may be repeated numerous times between different languages. This is represented by the arrow from the MSIL returning to a source file. At run time, the execution system compiles, loads, and starts executing an MSIL file. References to a type defined in a different MSIL file cause that file to be loaded, its metadata is read, and then instances of the new type can be exposed to the runtime.

CLR Type System

When striving for language interoperability, the basic issues remain. First, some form of agreement about the representation of data types must be adopted. In the CORBA world, the Object Management Architecture defines the concepts of object and type, and the CORBA specification quantifies these concepts. In the .NET Framework, the common language runtime type system describes the types within the system. A simplified view of that type system is shown in Figure 2.

Figure 2. Common language runtime type system

The common language runtime type system is divided into two subsystems:

  1. Value types
  2. Reference types

A major distinction between value types and reference types is that value types have no concept of identity. A value type is a sequence of bits in memory. One example of a value type is a 32-bit signed integer. Any two 32-bit signed integers compare as equal if they hold the same number. That is, the sequence of bits is identical. Reference types are the combination of a location, its identity, and a sequence of bits. Reference types can be considered similar to conventional C++ classes, where two objects of the same class may contain the same data values (the same sequence of bits), but they may not be considered equal since they refer to two different objects.

In .NET Framework terminology, an instance of a value type or reference type is known as a value. Every value has one exact type, and this type defines the methods that can be called on that value. At run time, it is not always possible to determine the type of a value by inspection; for example, value types can never be inspected for their exact type.

Value types

Although many inbuilt data types are value types, value types are not limited to inbuilt data types. Value types are often allocated on the run time stack; value types can therefore be local variables, parameters, and return values from functions. For these reasons, user-defined value types should be lightweight, like the inbuilt value types, generally not more than 12 to 16 bytes in size. User-defined structures and classes can be value types and can contain:

  • Methods (both class and instance)
  • Fields (both class and instance)
  • Properties
  • Events

At a fundamental level, properties and events both equate to methods. Properties are a syntactic shortcut; they represent Set and Get methods on logical fields of a class. Events are used to expose asynchronous changes in an observed object. Clients listen for events and when an event is raised, a method call is invoked on the client.

It is not possible to have a value type inherit from another a value type; that is, in .NET Framework terminology, a value type is sealed. Although a value type is often allocated on the stack, a value type can be allocated on the heap. For example, if it is a data member of an object type.

For all value types, there exists a corresponding object type, known as its boxed type. Values of any value type can be boxed and unboxed. Boxing a value type copies the data from the value into an object with a corresponding boxed type, allocated on the garbage-collected heap. Unboxing a value type copies the data from the boxed object into a value. Since a box type is an object type, it may support interface types and thereby provide additional functionality to its unboxed representation. Object types, reference types, and interface types are described more fully in the Reference Types section.

The following code demonstrates the definition and use of a value type in managed C++. The value type is a class with two integers as its only data members. As already stated, since value types are allocated on the stack and passed by value, they should generally be lightweight. The code also demonstrates how a value type cannot be allocated on the heap; the attempt to allocate a VTPoint on the heap causes an error. Casting is used in the example to unbox the value off the heap.

#using <mscorlib.dll>

using namespace System;

__value public class VTPoint
{
public:
  int m_x, m_y;
};

int main(void)
{
  VTPoint a;                          // on stack
  VTPoint * ptr = new VTPoint();      // illegal
  __box VTPoint *VTptr = __box(a);              // box
  VTptr->m_x = 42;
  VTPoint b = *dynamic_cast<VTPoint*>(VTptr);   // unbox
  b.m_y = 42;
  Console::Write("Values are: b.m_x should be 42 is ");
  Console::WriteLine(b.m_x);
  Console::Write("Values are: b.m_y should be 42 is ");
  Console::WriteLine(b.m_y);
  return 0;
}

Reference types

References types are the combination of a location (also known as identity) and a sequence of bits. A location designates an area in memory where values can be stored and the type of values that can be stored there. Locations are therefore type-safe in that only assignment-compatible types can be stored in a location. See the Assignment Compatibility section for an example of assignment compatibility.

The previous code shows that an object type is always a reference type, but not all reference types are object types. An example of an object type is the inbuilt reference type "String." The common language runtime uses the term object to refer to values of an object type. As String is an object type, all instances of the String type are objects. The set of all exact types for all objects is known as the object types. Object types are always allocated on the garbage-collected heap.

The following code demonstrates the definition and use in C++ of an object type called "Point." This definition uses properties, which allow clients to access a logical data member as if it were a public field of the class. The sample code also demonstrates how a reference type cannot be allocated on the stack; the attempt to allocate a Point on the stack will not compile. The code also shows what appears to be direct access to the properties of the object as if they were public attributes. This is syntactic sugar, however; the compiler inserts calls to the get_ and set_ methods defined for the class as needed.

#using <mscorlib.dll>

using namespace System;

__gc class Point
{
  private:
    int m_x, m_y;
  public:
    Point()
    {
      m_x = m_y = 0;
    }  
    __property int get_X()
   {
      return m_x; 
   }
    __property void set_X(int value) 
   { 
      m_x = value;
    };
    __property int get_Y()
   {
      return m_y; 
   }
    __property void set_Y(int value) 
   { 
      m_y = value;
    };
};

int main(void)
{
   Point a;                        // illegal
   Point* ptr = new Point();
   ptr->X = 42;
   ptr->Y = 42;
   Console::Write("ptr->X is ");
   Console::WriteLine(ptr->X);
   Console::Write("ptr->Y is ");
   Console::WriteLine(ptr->Y);
   return 0;
}

Object types

All object types inherit, either directly or indirectly, from the Object class. Although not explicitly stated in the previous code, class Point directly inherits from Object. This class provides a number of methods that can therefore be called on all objects. The methods are:

  • Equals. Returns true if the two objects are equal. Subtypes may override this method to provide either identity or equality comparison.
  • Finalize. Is invoked by the garbage collector before an object's memory is reclaimed.
  • GetHashCode. Returns the hash code for an object.
  • GetType. Returns the type object for this object. Permits access to the metadata for the object.
  • MemberwiseClone. Returns a shallow copy of the object.
  • ToString. Returns a string representation of the object.

Most of the methods defined on Object are public. However, MemberwiseClone and Finalize have protected access as they can only be accessed by subtypes.

The following code shows the public methods that can be called on an object of type Point. The output was generated by a simple program called "Reflector," which is supplied in the .NET Framework SDK. Reflector retrieves the Point class's Type object and then displays its public method's prototypes. Apart from the property access methods, the four public methods that are inherited from Object are shown. The two protected methods are not shown. The output also shows the use of inbuilt types, such as Boolean, Int32, and String.

C:\Program Files\NGWSSDK\Samples\Reflector\VC>Reflector -m -v Point
Class: Point
Methods (8)
        Int32 GetHashCode ()
        Boolean Equals (System.Object)
        System.String ToString ()
        Int32 get_X ()
        Void set_X (Int32)
        Int32 get_Y ()
        Void set_Y (Int32)
        System.Type GetType ()

Interface types

Programming with interfaces types is an extremely powerful concept. Object-oriented programmers are familiar with the concept of substituting a derived type for a base type. However, sometimes two classes are not related by inheritance, but they do share common functionality. For example, many classes may contain methods for saving their state to and from permanent storage. For this purpose, classes not related by inheritance may support common interfaces, allowing programmers to code for the classes' shared behavior based on their shared interface type and not their exact types.

An interface type is a partial specification of a type. It is a contract that binds implementers to provide implementations of the methods contained in the interface. Object types may support many interface types, and many different object types would normally support an interface type. By definition, an interface type can never be an object type or an exact type. Interface types may extend other interface types.

An interface type may contain:

  • Methods (both class and instance)
  • Static fields
  • Properties
  • Events

A major difference between an interface type and an object type (or a value type class) is that an interface type cannot contain instance fields.

The following code demonstrates a user-defined interface type, "IPoint." This interface type declares two attributes (requiring four accessor methods in any implementor). By inheriting from IPoint, the class Point agrees to implement these four abstract methods. The remainder of the definition of the class Point remains unchanged from that in the code in the Reference Types section.

#using <mscorlib.dll>

using namespace System;

__gc __interface IPoint
{
   __property int get_X();
   __property void set_X(int value);
   __property int get_Y();
   __property void set_Y(int value);
};

__gc class Point: public IPoint
{
  ...
};

The common language runtime provides a number of interface types. The following code demonstrates the use of the IEnumerator interface supported by array objects. The array of Point*s allows clients to enumerate over the array by requesting an IEnumerator interface. The IEnumerator interface supports three methods:

  • Current. Returns the current object.
  • MoveNext. Moves the enumerator on to the next object.
  • Reset. Resets the enumerator to its initial position.
int main(void)
{
  Point *points[] = new Point*[5];
  for(int i = 0, length = points->Count; i < length; i++)
  {
    points[i] = new Point(i, i);
  }
  Collections::IEnumerator *e = points->GetEnumerator();
  while(e->MoveNext())
  {
    Object *o = e->Current;
   Point *p = dynamic_cast<Point*>(o);
   Console::WriteLine(p->X);
  }
  return 0;
}

Array objects support many other useful methods, such as Clear, GetLength, and Sort. Array objects also provide support for synchronization. Synchronization is provided by methods such as IsSynchronized and SyncRoot.

Pointer types

Pointer types provide a means of specifying the location of either code or a value. Function pointers refer to code; managed pointers can point at objects located on the garbage-collected heap and unmanaged pointers can address local variables and parameters. The semantics for these different types vary greatly. For example, pointers to managed objects must be registered with the garbage collector so that as objects are moved on the heap, the pointers can be updated. Pointers to local variables have different lifetime issues.

Pointer types are not covered here in detail for two reasons. First, a solid understanding of the .NET Framework architecture is needed to understand the semantic issues of pointers. Second, most programming languages will abstract the existence pointers to such a degree that they will be invisible to programmers.

Assignment Compatibility

Since we've explained the concepts of object types and interface types, albeit very simply, we can now address the question of assignment compatibility. A simple description of assignment compatibility for reference types is as follows:

  • A location of type T, where T may either be an object type or interface type, can hold any object with an exact type of T, or is a subtype of T, or supports the interface T.

The following code demonstrates aspects of assignment compatibility. As Points and Strings are both object types, they inherit from Object. Therefore, both are assignment-compatible with Object. However, Points and Strings are not assignment-compatible with each other. The String class also implements the IComparable interface, so Strings are assignment-compatible with an IComparable, while Points are not.

int main(void)
{
  Point* p = new Point();
  String* s = S"Damien";
  Object* o;
  o = p;          // legal, p is an Object
  o = s;          // legal, s is an Object
  p = s;          // illegal, s is not a Point 
  s = p;          // illegal, p is not a String
  IComparable *i;
  i = s;          // legal, Strings support IComparable
  i = p;          // illegal, Points do not support IComparable
  return 0;
}

Built-in Types

There are built-in value and reference types. The built-in types often have special instructions in the intermediate language, so that they can be handled efficiently by the execution engine. Examples of built-in types are:

  • Bool
  • Char (16-bit Unicode)
  • Integers, both signed and unsigned (8-, 16-, 32-, and 64-bit)
  • Floating point numbers (32- and 64-bit)
  • Object
  • String
  • Machine-dependent integers, both signed and unsigned
  • Machine-dependent floating point

Object and String are an inbuilt references type, while all the other inbuilt types are value types. A machine-dependent signed integer can hold the largest value that an array index can support on a machine. A machine-dependent unsigned integer can hold the largest address value a machine can support.

Metadata

Metadata is the essential link that bridges the runtime type system and the execution engine. Currently, two significant problems exist with many component-based programming systems:

  1. Information about the component (its metadata) is often not stored with the component. Rather, metadata is often stored in auxiliary files, such as interface definition language (IDL) files, type libraries, interface repositories, implementation repositories, and the Registry. The .NET common language runtime stores metadata with types to avoid this issue.
  2. Metadata facilities are primitive. Most facilities only allow developers to specify the syntax of an interface, but not its semantics. Proof of this problem lies with the number of "IDL" extensions that can be found in many systems, such as TINAC's ODL. The .NET Framework common language runtime addresses this problem by providing a standardized metadata extension system known as custom attributes.

Compilers targeting the .NET Framework describe the types they produce with metadata for two reasons:

  1. Metadata permits types defined in one language to be usable in another language. This facility ensures language interoperability in the common language runtime.
  2. The execution engine requires metadata to manage objects. Managing objects includes requirements such as memory management.

Metadata is stored in files in a binary format. This format is not specified, so developers should not rely on its binary layout. Instead, there are a number of methods for reading and writing metadata.

Metadata Extensibility

The .NET Framework's metadata facility is extensible through custom attributes. The following C# code demonstrates the use of custom attributes, as well as language interoperability. The first class defined in the following code, Author, is a custom attribute, and is designated by its inheritance specification. This class contains one field, a String to represent the author's name. It contains two methods, including a constructor, and it overrides its ToString method.

using System.Reflection;

public class Author: Attribute
{
  using System;
  public readonly string name;
  public Author(string name) 
  {
   this.name = name;
  }
  public override String ToString() 
  {
    return String.Format("Author: {0}", name);
  }
}

[Author("Damien Watkins")]
public class CSPoint: Point
{
  public static void Main()
  {
    MemberInfo info = typeof(CSPoint); 
    object[] attributes = info.GetCustomAttributes();
    Console.WriteLine("Custom Attributes are:");
    for (int i = 0; i < attributes.Length; i++) 
    {
      System.Console.WriteLine("Attribute " 
         + i + ": is " + attributes[i].ToString());
    }
  }
}

The next class defined in this figure is a C# class that inherits from our C++ Point class. The definition of the class is preceded by the specification that this class has a custom attribute of type Author. The Main method in the CSPoint class uses the GetCustomAttributes method to access an array of custom attributes for this class and then displays them on the screen.

Assemblies and Manifests

An assembly is a collection of common language runtime types and code. An assembly is a means of grouping related components and, as a side effect, placing them in a namespace. Just as .NET Framework types are self-describing, assemblies are also self-describing. Information about an assembly is contained in its manifest. A manifest contains:

  • Name and version information
  • A list of types and their location in the assembly
  • Dependency information
#using <mscorlib.dll>

using namespace System;
using namespace System::Reflection;

int main(void)
{
   Type *t= Type::GetType("System.String");
   Assembly *a = Assembly::GetAssembly(t);
   AssemblyName *an = a->GetName(false);

   Console::WriteLine("AssemblyName->Name: {0}", an->Name);
   Console::WriteLine("AssemblyName->Codebase:\n\t {0}", an->CodeBase);

   String *rn[] = a->GetManifestResourceNames();
   Console::WriteLine("Resource Name(s):");
   for(int i = 0; i < rn->Count; i++)
      Console::WriteLine(S"\t {0}", rn[i]);
   
   Module *m[] = a->GetModules();
   Console::WriteLine("Module Name(s):");
   for(int i = 0; i < m->Count; i++)
      Console::WriteLine(S"\t {0}", m[i]->FullyQualifiedName);

   return 0;
}

The preceding code demonstrates some aspects of assemblies. The program first retrieves the String class's type object, and then uses the type object to access its assembly information. The following code shows the output generated from this program.

C:\Project7\Exercises\assembly>StringAssembly
AssemblyName->Name: mscorlib
AssemblyName->Codebase:
         file://C:/WINNT/ComPlus/v2000.14.1809/mscorlib.dll
Resource Name(s):
         mscorlib.resources
         prcp.nlp
         sortkey.nlp
         ctype.nlp
         sorttbls.nlp
         culture.nlp
         l_except.nlp
         bopomofo.nlp
         region.nlp
         xjis.nlp
         big5.nlp
         l_intl.nlp
         ksc.nlp
         charinfo.nlp
         prc.nlp
Module Name(s):
         C:\WINNT\ComPlus\v2000.14.1809\mscorlib.dll

Execution System

The common language runtime execution engine is responsible for ensuring that code is executed as required. The execution engine provides facilities for MSIL code such as:

  • Code loading and verification
  • Exception management
  • Just-in-time compilation
  • Memory management
  • Security

Intermediate language

The .NET Framework-compliant compilers translate source code into MSIL instructions and metadata. MSIL instructions resemble assembly language instructions but this code is not specific to a single physical CPU. Rather, MSIL undergoes a secondary compilation to native code. As MSIL is machine independent, it can be moved from machine to machine. Translation from MSIL to native code can be accomplished at any stage up until execution of the code.

The execution engine supplies a number of just-in-time (JIT) compilers. MSIL can be compiled into native code:

  • When it is first installed on a machine.

    Or

  • After it is loaded into memory and before it is executed.

The following code shows sample MSIL code for our Point class. The MSIL is intuitive and it is possible to find:

  • The name of the class (Point) and its inheritance from System.Object.
  • The definition of the class's fields (m_x and m_y).
  • The definition of its constructor (.ctor) and other methods (get_X and set_X).
...
.class auto ansi Point extends ['mscorlib']System.Object
{
  .field private int32 m_x
  .field private int32 m_y
  .method public specialname rtspecialname 
          instance void .ctor() il managed
  {
    // Code size       21 (0x15)
    .maxstack  2
    IL_0000:  ldarg.0
    IL_0001:  call       instance void ['mscorlib']System.Object::.ctor()
    IL_0006:  ldarg.0
    IL_0007:  ldc.i4.0
    IL_0008:  stfld      int32 Point::m_y
    IL_000d:  ldarg.0
    IL_000e:  ldc.i4.0
    IL_000f:  stfld      int32 Point::m_x
    IL_0014:  ret
  } // end of method 'Point::.ctor'
  .method public instance int32 get_X() il managed
  ...
  .method public instance void  set_X(int32 'value') il managed
    .property int32 Y()
  {
    .set instance void set_Y(int32)
    .get instance int32 get_Y()
  } // end of property 'Point::Y'
  ...
...

Managed Versus Unmanaged

The common language runtime uses the terms managed and unmanaged to qualify code, data, and pointers. Managed or unmanaged identify the amount of control that the runtime has over aspects of a program. Anything that is managed is tightly controlled by the common language runtime.

Managed code provides the execution engine with sufficient information (metadata) to allow the execution engine to manage its execution safely. Safe execution includes such aspects of program execution as debugging, inter-language interoperability, memory management, and security. Unmanaged code does not provide such information to the execution engine. The execution engine cannot provide these services for unmanaged code. The majority of code running on Windows platforms today is unmanaged.

The fact that managed and unmanaged code may exist in the same program is not necessarily a problem. However, the interaction between managed and unmanaged code requires close attention. For example, the garbage collector can only run when all threads are suspended. When a thread executes unmanaged code, the execution engine cannot suspend the thread, and garbage collection cannot occur. Another area of concern is exception handling. Managed code will throw a .NET Framework common language runtime exception while native code may throw a WIN32 exception. When exceptions propagate between managed and unmanaged code, the exception needs to change to fit the expected model.

Managed data describes values that are allocated on the garbage-collected heap by the common language runtime. Managed data is only reachable by managed code, but unmanaged data is available to both managed and unmanaged code.

Common Language Specification

If the common language runtime can be described as the union of many programming language features, the common language specification (CLS) is a subset of these features. The subset is not the set of all features common to all languages. Rather it is a set of features common to many programming languages.

Why do we need a CLS? The main reason is that most programming languages may not want to support all of the runtime features. The CLS represents a compliance level that most languages should be able to attain if compiler writers desire interoperability. This functional subset is also the level that many .NET Framework-compliant tools will aim to support, such as Active Server Pages+ (ASP+).

The size of the CLS when compared to the common language runtime is an important concern. By definition, the CLS cannot be bigger than the runtime. However, if the CLS is too small, it will not provide enough functionality to do anything useful. Currently the CLS appears to support more of the runtime than it eliminates. From this perspective, it is more appropriate to discuss the CLS in terms of what it eliminates from the runtime.

Conclusion

This article has provided an overview of the core components of the .NET Framework. Language integration is a central theme. The common language runtime facilitates program language integration to such a degree that it allows programmers to use and define types and libraries in one language and then have these types seamlessly used in other languages.

Acknowledgments

This article is based on an upcoming book that Mark Hammond and I are writing on Microsoft .NET. We must acknowledge many people who helped us along the way. Thanks to James Plamondon who offered us early access to the technology; to Brad Abrams and Jim Miller for numerous discussions and clarifications about the .NET Framework; to Jon Nicponski and Steve Christianson who diligently attend to all the small details; and to all the other Project 7 members whose conversations at meetings truly opened up ideas. Finally, thanks to everyone who designed and built the architecture, compilers, and online documentation through which we were finally able to comprehend the .NET Framework. We hope this article allows other developers to bootstrap into the .NET Framework.