Nullable Types

Sophia Salim — SDET, Visual Studio Managed Languages
Microsoft Corporation

Summary

"Nullable type" is a new feature introduced in Visual Studio 2008. It aims to introduce consistency in how null or nothing is represented in value and reference types. Simply put, value types can now contain the literal "nothing" if they are declared in a special way. In this whitepaper I will discuss how value types can be declared nullable, the use of nullable declared variables, and the interaction of nullable types with other language features.

Introduction

This whitepaper is divided into two broad sections:

• Beginner: This sections provides an introduction to the use, syntax and members of nullable types
• Expert: This section describes how nullable types can be used in conjunction with other language constructs. It also provides details on how operator lifting works for nullable types, and how to use nullables with the new if operator

Beginner

• Defining the Nullable Type
• Syntax
• Simple Conversions and Operations
• Member Access

Expert

Conversions

• Between T and T?
• With Other Types
• Boxing and Nullable Values
• User-defined conversions

Operators

• Pre-Defined Operators
• Concat Operator
• Boolean Expressions
• User-Defined Operators

Lifting

• Lifted Conversions
• Lifted Operators

Nullables in Conjunction with If Operator

Appendix 1

About the Author

Beginner

The null value, represented by the literal Nothing, is a special value in the type system that represents the absence of a value. Not all types can be assigned the null value; a type whose value domain contains the null value is called a nullable type.

All reference types are by definition nullable types. For value types, Orcas will now contain two flavors:

• Nullable value types
• Non-nullable value types

The non-nullable value types are all the value types that we know of today for VB. In Orcas, for each of these value types we have introduced a nullable counterpart. E.g. the value type Integer now has a nullable counterpart "Integer?". Integer can contain values: -2147483647 to 2147483648 and "Integer?" can contain in addition to the values: -2147483647 to 2147483648, the null value "Nothing".

Both the built in and user defined value types (structures and enumerations) can be declared nullable. For an exhaustive list of built in types which can be defined as nullable, please see Appendix 1.

Following code shows different ways in which you can define nullable types:

    'Built in nullable value types
    Dim  i As Integer?
    Dim j? As Integer
    Dim k As Nullable(Of Integer)

Similarly for user defined value types, the same syntax is used:

    Public Structure s1
        Dim mem As Integer
    End Structure
    Sub Main()
        'User defined nullable value types
    Dim i As s1?
    Dim j? As s1
    Dim k As Nullable(Of s1)
    End Sub

Arrays of nullable types can also be declared as follows:

    Dim i As Integer?()
    Dim j?() As Integer
    Dim k As Nullable(Of Integer)()

When the null value is converted to a non-nullable value type, it is implicitly converted to the zero value of the value type. The zero value of a type is the value in its value domain that represents the zeroed out state of a variable of that type. For example, the value Nothing converted to Integer represents the number 0 instead of no value.

All operations on nullable types are null-propagating except concatenation (See Concat Operator and Nullable Types for details). Null propagating operations result in the null value if one of the operands is the null value. Non-null-propagating operations convert operands that are the null value to the zero value of the type of the operation and use those values instead.

The following code illustrates the null propagating operation, + on nullable integers:

    Dim i As Integer?
        i = i + 1 'result is nothing, since the operation is nullpropogating

Each nullable type T? has three default members that can be used to read and test its value:

• Value: Gets the value of the nullable variable as the underlying type T
• GetValueOrDefault: Retrieves the value of the nullable variable. If nothing, it returns the default value of the underlying type T
• HasValue: Returns true or false based on if the variable contains a value or contains nothing

Following code demonstrates the three members:

    Dim i As Integer?
        Console.WriteLine(i.HasValue)
        Console.WriteLine(i.GetValueOrDefault)
        Try
            Console.WriteLine(i.Value)
        Catch ex As InvalidOperationException
            Console.WriteLine("Exception: Cannot access value if nullable variable is nothing")
        End Try
        i = 1
        Console.WriteLine(i.HasValue)
        Console.WriteLine(i.GetValueOrDefault)
        Console.WriteLine(i.Value)

Expert

Consider the value type T, and its nullable counterpart T?. The following rules apply to conversions between T and T?

• T has a predefined widening conversion to T?
• T? has a predefined narrowing conversion to T
• Conversion from T? to T throws a System.InvalidOperationException exception if the value being converted is Nothing

Following is a sample of the conversions:

        Dim i1 As Integer = 1
        Dim i2? As Integer
        Try
            i1 = i2

        Catch ex As InvalidOperationException
            'Expected, since nullable variable contains nothing
        End Try
        'Widening conversion
        i2 = i1 'Now i2 will contain a value 1
        'Narrowing conversion. Compile error when option strict is on
        i1 = i2 'Valid since nullable variable now contains a value

Consider two types T and S, where T has a predefined (widening or narrowing) conversion to type S. The following rules apply to conversions between T, S and their nullable counterparts:

• A conversion of the same type (narrowing or widening) from T? to S?.
• A conversion of the same type (narrowing or widening) from T to S?.
• A narrowing conversion from S? to T.
• Conversion from T? to S? when T? is nothing, results in nothing
• Conversion from S? to T will result in a System.InvalidOperationException exception if S? is nothing

Following is a sample of the conversions between integer and long:

  'Integer(T) has a predefined widening conversion to long(S)
      Dim i1 As Integer
      Dim i2 As Long
      Dim null_i1? As Integer
      Dim null_i2? As Long
        null_i2 = null_i1 'Valid widening conversion. Value of null_i2
                          'is nothing and no exception is thrown
        null_i2 = i1 'Valid widening conversion
        i1 = null_i2 'Narrowing conversion
        null_i2 = Nothing
        Try
            i1 = null_i2
        Catch ex As InvalidOperationException
            'Exception expected because null_i2 is nothing
        End Try

A nullable value type T? when boxed, i.e. converted to Object, Object?, System.ValueType, or System.ValueType?, results in a boxed value of type T rather than T?. This is because reference types are inherently nullable and do not require the nullable type. As a result, when T? is unboxed the value is "unwrapped" to either an instance of the boxed value type T or Nothing. Conversely, when unboxing to a nullable value type T?, the value will be "wrapped" by Nullable(Of T) and Nothing will be unboxed to a null value of type T?.

For example:

  'This function returns the type of nullable variables even
  'when the underlying value is nothing
  
  Public Function CheckType(Of T)(ByVal arg As T) As System.Type
      Return GetType(T)
  End Function
  Sub Main()
      Dim i1? As Integer = Nothing
      Dim o1 As Object = Nothing
      
      Console.WriteLine(CheckType(i1))  ' Will print System.Nullable<System.Int32>
      Console.WriteLine(o1 Is Nothing)   ' Will print True
      i1 = 1
      o1 = i1 'Since calling gettype on an uninitialized object throws an exception
      Console.WriteLine(o1.GetType().ToString())  ' Will print System.Int32
      Dim i2 = CType(o1, Integer?)
      Console.WriteLine(CheckType(i2).ToString())  ' Will print 10
  End Sub

A side effect of this behavior is that a nullable value type T? appears to implement all of the interfaces of T, because converting a value type to an interface requires the type to be boxed. As a result, T? is convertible to all the interfaces that T is convertible to. It is important to note, however, that a nullable value type T? does not actually implement the interfaces of T for the purposes of generic constraint checking or reflection.

Types are allowed to declare user defined conversions between their nullable counterparts and other types. For example the following is valid:  

    Structure T
        ...
    End Structure

    Structure S
        Public Shared Widening Operator CType(ByVal v As S?) As T
            ...
        End Operator
    End Structure

However, for validity purposes all "?" modifiers are first dropped from the types involved in the declaration of the conversion, and then traditional type checking for user defined conversions is applied. The type checking is done against the following rule: for a given type, conversions are only allowed between itself and another type. For example, the following declaration is not legal, because structure S cannot define a conversion from S to S (itself):

    Structure S
        Public Shared Widening Operator CType(ByVal v As S) As S?
            ...
        End Operator
    End Structure

Given any type T with the a pre-defined operator "op", where "op" is not concatenation, following rules apply to T?

• "op" is defined for T?
• If the T? operand is not nothing, result for "op" with T? will be the same as for T
• If the T? operand is nothing, result for "op" with T? will be nothing
• If any operand is nullable, type of the result will be nullable

For example:

  Dim v1? As Integer = 10
        Dim v2 As Long = 20
        Console.WriteLine(v1 + v2)  ' Type of operation will be Long?

The concat operator is special cased for nullables to be non-null propagating. Just like other reference types, a value of nothing inside a nullable variable is translated as the empty string while performing concatenation. Following is a code snippet illustrating this behavior:

  Dim t As Integer?
        Console.WriteLine("This string is concatenated with a null Integer?" & t)
        t = 1
        Console.WriteLine("The value of nullable variable is: " & t)

This behavior ensures a consistency between other reference types and nullables. It also ensures that a null nullable variable does not throw exceptions while users are trying to look at its value using message boxes or the console.

The pre-defined operator set includes the pseudo-operators IsTrue and IsFalse, which are extended to operate on Boolean?. This means that Boolean? may be used in Boolean expressions – if the value is Nothing, then the Boolean expression is both not True and not False. For example:

        Dim x? As Boolean
        x = Nothing
        If x Then
            ' Will not execute
        End If

Note that this means that the short-circuiting logical operators AndAlso and OrElse will evaluate their second operand if the first operand is a null Boolean? value. For example:

        Dim x?, y? As Boolean
        x = Nothing
        y = False

        ' Both x and y will be evaluated
        If x AndAlso y Then
            ' Will not execute
        End If

The definitions of the logical operators And and Or for Boolean? are extended to encompass this three-valued Boolean logic as such:

• And evaluates to True if both operands are True; False if one of the operands is False; Nothing otherwise.
• Or evaluates to True if either operand is True; False if both operands are False; Nothing otherwise.

For example:

        Dim x?, y? As Boolean

        x = Nothing
        y = True

        If x Or y Then
            ' Will execute
        End If

As with conversions, types are allowed to declare user defined operators using their nullable counterparts. For example the following is valid:  

    Structure T
        Dim mem As Integer
    End Structure

    Structure S
        Dim mem As Integer
        Public Shared Operator +(ByVal v1 As S?, ByVal v2 As T) As T
        End Operator
    End Structure

However, for validity purposes all "?" modifiers are first dropped from the types involved in the declaration of the operator, and then traditional type checking for user defined conversions is applied. Note that this rule does not apply to types which are specifically required by an operator:

• The shift operators' second parameter must still be Integer, not Integer?,
• IsTrue and IsFalse must still return Boolean, not Boolean?.

A nullable type has no members of its own. Its members can be lifted from the underlying type. This process of a nullable type adopting members of its underlying type and substituting nullable types for the non- nullable types in the adopted members is called "Lifting".

While performing a conversion from user defined nullable type T? to any other type say S or its nullable counterpart S?, following are the steps used:

1. If a user-defined conversion is present for T?, it is preferred over all other candidate conversions
2. If the value being converted is nothing, no conversion is called and the return value is nothing
3. All user-defined conversions for T are used as candidates while resolving conversions for T?
4. A conversion from T to S is lifted to be a conversion from T? to S?. Arguments of type T? are first converted to type T
5. The user-defined conversion operator is evaluated and we now have the result of type S
6. The result is then converted to the type S?

The last 5 steps define the process of lifting. Following code demonstrates these steps:

  Module Test
      Public result As String
      Structure S
          Dim i As Integer
      End Structure

      Structure T
          Dim i As Integer
          Public Shared Widening Operator CType(ByVal v As T) As S
              result = "Conversion was called"
              Return New S With {.i = v.i}
          End Operator
      End Structure
      Sub Main()
          Dim x As T?
          Dim y As S?

          result = Nothing
          y = x               'Step 2: Returns nothing. Conversion not called
          Console.WriteLine(If(result, "Conversion was not called"))

          result = Nothing
          x = New T With {.i = 1}
          y = x               'Steps 3-6
          Console.WriteLine(If(result, "Conversion was not called"))
          Console.WriteLine("Was coversion successful? Answer: " & (y.Value.i = x.Value.i)) 
      End Sub
   End Module
 

While performing an operation for a user defined nullable type T?, following are the steps used:

1. If a user-defined operator is present for T?, it is preferred over all other candidate operators
2. All user-defined operators for T involving non-nullable operands are also used as candidates while resolving operations for T?
3. If any of the operands is nothing, the result is nothing and the type of the result is the nullable version of the expected result type.
4. If an operator involving non-nullable operands is chosen, it is lifted, i.e. all the operands and result types are converted to their nullable counterparts.
5. Note: Any operand or result that is required to be of a specific type (e.g. second parameter of shift operator or return type of IsTrue or IsFalse operator) is not converted to its nullable counterpart
6. Lifted operator is evaluated by converting the operands to their non-nullable versions
7. The operations defined inside the operator are now performed
8. The result is converted to the nullable version of the result type

Following code demonstrates this process:

  Module Test
      Public result As String
      Structure S
          Dim i As Integer
      End Structure
      
      Structure T Dim i As Integer
          Public Shared Operator +(ByVal v1 As T, ByVal v2 As S) As S
              result = "Operator was called"
              Return New S With {.i = v1.i + v2.i}
          End Operator
      End Structure

      Sub Main()
          Dim x As T?
          Dim y As S?

          result = Nothing
          y = x + y
          Console.WriteLine(If(result, "Operator was not called"))
          Console.WriteLine("Was operation successful? Answer: " & (y Is Nothing))

          result = Nothing
          x = New T With {.i = 1}
          y = New S With {.i = 1}
          y = x + y
          Console.WriteLine(If(result, "Operator was not called"))
          Console.WriteLine("Was operation successful? Answer: " & (y.Value.i = 2))
      End Sub
  End Module

A binary flavor of the If operator has been introduced to simplify working with nullable and reference types. The operator allows for two operands, the first of which must be either a nullable or a reference type. A widening conversion must exist between the two operands, and the wider of the two operand types is the type of the result. If the second operand is non-nullable, the ? is removed from the type of the first operand when determining the result type of the expression.

Following code demonstrates the binary if operator in conjunction with nullables:

        Dim i As Integer?
        Dim j = If(i Is Nothing, 10, i) 'The ternary operator expansion 
                                        'simulating the binary operator
        Dim k = If(i, 10) 'The binary operator in action. Note how this removes
                          'much of the redundant code of the ternary operator.

Following code demonstrates how result types are determined for nullable operands:

    Dim a As Integer?
    Dim b As Long?
    
    Dim i = If(a, 10) '? are dropped from "a" while determining the 
                      'result type. Type of i is Integer
    
    Dim j = If(a, b) '? are retained while determining the result type. 
                     'Type of j is Long? (the wider of the two types)

Built in value types which can be defined as nullable are as follows:

        Byte
        SByte
        Short
        Integer
        Long
        UShort
        UInteger
        ULong
        Single
        Double
        Boolean
        Char
        Date

Sophia Salim is an SDET with Microsoft's Visual Studio Managed Languages group.