Saying Goodbye to an Old Friend

Article
06/30/2006

Michael Howard
Microsoft Security Engineering

March 30, 2004

Summary: Michael Howard documents the work going on to make the C runtime libraries more robust in the face of malicious data. The changes, available in Visual Studio 2005 affect the C runtime and the C++ standard template libraries. (7 printed pages)

Note This material is based on pre-release software. Some of the information contained in the document may change when the product is released.

Every once in a while, we all need to do some serious spring-cleaning, whether it's around the house or in our code. And invariably, when we do start the clean-up effort, we wonder where some of the moldy old crud came from, and why we never noticed it in the past. Some things we keep, and some things we toss out. And if you're anything like me, you replace some of things you throw away with shinier, newer versions.

Let's face it, the C Runtime library is in dire need of a good scrub, and I don't mean a tidy-up, I mean getting in there with steel wool and bleach!

Think about it for a moment. When were functions like strcpy and strcat invented? They were designed in happier times when the C language was first developed by Kernighan and Ritchie long ago, when threats were by no means as serious as they are today, and when networking wasn't as pervasive as it is today. Now don't get me wrong, you can write secure code with functions like strcpy. It's the data that's the culprit, after all. But functions like strcpy don't help you write secure code, and an error in such a function can be catastrophic. Of course, gets is just plain evil! 'Nuff said.

So, what are we doing about it? You may have heard about strsafe.h. It's a set of consistent, safer, string-manipulation functions developed during the Windows Security Push in 2002. It's available in Visual Studio® .NET 2003 and the Platform SDK. Read more about strsafe in Strsafe.h: Safer String Handling in C.

Other functions, such as strlcpy and strlcat are available in many open source operating systems. You can read about these two functions in David Wheeler's Secure Programming for Linux and Unix HOWTO.

While strl* and strsafe are both steps in the right direction, we need to build more robust functionality into the core C runtime library, and that's where the updated CRT comes in to play. The Microsoft Visual C++® libraries team decided to take along, hard look at every single function in the CRT to determine security weaknesses and possible remedies. As you can imagine, many functions, about 400 in fact, were rewritten to be much more secure, as well as help you write more secure code.

What's New in the New CRT?

First, the new CRT functions outlined in this paper will appear in Visual Studio 2005 and may change between now and final release. Second, I want to point out that new libraries don't magically turn insecure code into secure code with a simple flick of a compiler switch, but they sure can help.

The safer alternatives do not replace the old functionality. In other words, strcpy is still strcpy. The safer version has a new name, strcpy_s. However, if you compile with the new library, the older functions are deprecated. So be warned, you will get compiler warnings almost immediately. That said, it's easier to fix a compiler warning than to fix a security bug. Take my word for it!

Some functions, such as a calloc, simply do more parameter checking, but the functionality is the same, so there is no calloc_s. More on calloc in a moment.

One of my favorite changes is in the replacement for strncat, strncat_s. A problem with strncat is that the last argument is not the total size of the destination buffer, rather it is minimum of what's left in the destination and how much to copy. This can lead to all sorts of off-by-one errors, or worse, off-by-lots errors. Take a look at the following example:

if (szURL != NULL) { 
  char   szTmp[MAX_PATH]; 
  char  *szExtSrc, *szExtDst; 

  strncpy(szTmp, szURL, MAX_PATH); 

  szExtSrc = strchr(szURL, '.'); 
  szExtDst = strchr(szTmp, '.'); 

  if(szExtDst) { 
    szExtDst[0] = 0; 

    if (fValid)  
      strncat(szTmp, szExtSrc, MAX_PATH);  
  }
}

The call to strncat is wrong—dangerously wrong. In fact, it's a buffer overrun waiting to happen. You can't copy MAX_PATH characters into szTemp safely, because the call to strncpy already added szURL to the string, essentially reducing the space left in szTmp.

Or, a simpler example:

char szTarget[12];
char *s = "Hello, World";

strncpy(szTarget,s,sizeof(szTarget));
strncat(szTarget,s,sizeof(szTarget));

If you compile this in Visual C++ 2003, you'll get an error indicating the data around szTarget got clobbered. This is the /GS compiler flag at work. It detected a stack-based buffer overrun, and stopped the application.

One way to fix this is to use code like this:

char szTarget[12];
char *s = "Hello, World";

strncpy(szTarget,s,sizeof(szTarget));
strncat(szTarget,s,strlen(szTarget) – strlen(s));

But there's a truly nasty bug lurking in here too. If the length of the destination buffer is exactly the same length as the source buffer, many n-functions do not null-terminate the destination buffer, so the call to strlen(szTarget) could return a length greater than the length of the target because there is no trailing '\0' character. Things just got a whole lot messier!

A more resilient program using the new runtime library looks like this:

char szTarget[12];
char *s = "Hello, World";

size_t cSource = strlen_s(s,20);
strncpy_s(temp,sizeof(szTarget),s,cSource);
strncat_s(temp,sizeof(szTarget),s,cSource);

The two new functions, strncpy_s and strncat_s share a similar function signature:

They both return an error code (errno_t) rather than a pointer.
The destination buffer (char *).
The total character count of destination buffer (size_t).
The source buffer (const char *).
The total character count of the source buffer (size_t).

Note the two buffer counts, one for each buffer. There is no need to keep an ongoing tally of the destination buffer count, which certainly makes like a little easier. There are a couple of other cool features. Both functions always null-terminate the string, but here's the bit I really like. Take a look at the code example from my previous Spot the Security Flaw section:

void noOverflow(char *str)
{
  char buffer[10];
  strncpy(buffer,str,(sizeof(buffer)-1));
  buffer[(sizeof(buffer)-1)]=0;
  /* Avoiding buffer overflow with the above two lines */
}

I found this in a document published in December 2003, from a large multi-national software company (no, not Microsoft) showing developers the virtues of secure coding. The problem is that there's a bad security flaw in this code. If *str points to NULL, strncpy fails as it tries to copy a NULL-pointer—Oops! strlcat used in various open source software has the same issue, but not strncat_s.

The reason strncat_s doesn't fail is because all of the updated runtime functions perform much more stringent input parameter checking. Here's the parameter validation block from strncat_s:

/* validation section */
_VALIDATE_RETURN_ERRCODE(front != NULL, EINVAL);
_VALIDATE_RETURN_ERRCODE(sizeInTChars > 0, EINVAL);
_VALIDATE_RETURN_ERRCODE(back != NULL || count == 0, EINVAL);

The validation macro is:

#define _VALIDATE_RETURN_ERRCODE( expr, errorcode \    
{                                                 \                         
      _ASSERTE( ( expr ) );                       \     
      if ( !( expr ) )                            \
      {                                            \
        errno = errorcode;                        \   
        _INVALID_PARAMETER(expr);                 \          
        return ( errorcode );                     \           
      }                                           \ 
}

_INVALID_PARAMETER gives file information in a debug build on failure to help people debug the code.

Weren't we always taught in school to check function arguments? Well, finally, the CRT does. It only took about twenty years.

You should be aware that strsafe functions, such as StringCchCopy and StringCchCat behave differently than strncpy_s and strncat_s. If strncat_s detects an error, the function will set the string to NULL. However, by default, strsafe will fill the destination with as much data as possible, and then NULL-terminate the string. You can mimic the same behavior in strsafe with code like this:

StringCchCatEx(dst,sizeof(dst)/sizeof(dst[0]),src,NULL,NULL,STRSAFE_NULL_ON_FAILURE)

Other buffer manipulation functions getting the same royal treatment include the various printf and scanf functions, mbstowcs, strerror, _strdate and _strtime, asctime, as well as ctime. There are functions other than buffer manipulation functions updated too, including _makepath, _splitpath, getenv, rand and many, many more.

The calloc function is another interesting function. It's susceptible to an integer overflow bug if size * num wraps beyond 2^32. To fix this, the updated calloc verifies that the calculation does not overflow:

/* ensure that (size * num) does not overflow */
if (num > 0 && (_HEAP_MAXREQ / num) <= size)
{
   errno=ENOMEM;
   return NULL;
}

What About C++?

The Visual C++ team didn't stop with the C runtime libraries. There are known issues in the Standard Template Library (STL). Did you know that you can get a buffer overrun using iterators? Many of the security risks associated with the misuse of iterators can be eliminated by using iterators that terminate (or throw) when advanced beyond their valid range. Here's an example.

#include <vector>

vector<int> v(10);      // vector with size == 10
v[20] = 10;             // yields buffer overrun
vector<int>::iterator it = v.end();

// advancing past the end causes a buffer overrun
++it;

By simply compiling this code with #define _SECURE_SCL (1), all iterators are range-checked.

You can also use the new functionality without using the new #define. For example, the following version of the code above does not cause an overflow:

vector<int> v(10);      // vector with size == 10
stdext::checked_iterator<vector<int> > ck_it(v, v.end());

// advancing past the end will terminate the program
++ck_it;

Other upgraded classes and methods include operator[] (vector, string, deque, bitset classes, and so on), front and back (vector, queue, list classes, and so on) Various algorithms are updated too, including copy, copy_backward, *_copy, transform, fill, set_*, and more.

What About Standards?

So, you're probably thinking, what about standards? Aren't C and C++ governed by standards? Yes they are standardized, and Microsoft has presented its latest submission to the standards committee. The draft can be found at https://std.dkuug.dk/jtc1/sc22/wg14/www/docs/n1031.pdf.

Summary

I'm really excited about the new libraries. While they will not make your code magically more secure, they are one more weapon against buffer overruns. Before I'm done, I'd like to thank the Visual C++ library team for doing this excellent work, especially Martyn Lovell for his help in putting this document together.

"The CRT is dead, long live the new CRT!"

Spot the Security Bug

A number of people worked out my last bug. The answer is earlier in this article. The so-called "safe" strncpy does not check for the either argument being NULL, and can cause your application to dump core or access violate.

Okay, on to this months defect. What's up with this code?

void ReadDataFromFile(char *szFilename,
LPOVERLAPPED_COMPLETION_ROUTINE func) {

   HANDLE hFile = CreateFile(szFilename,
                  FILE_ALL_ACCESS,
                  FILE_SHARE_READ,
                  NULL,
                  OPEN_EXISTING,
                  FILE_ATTRIBUTE_NORMAL |         
                  FILE_FLAG_OVERLAPPED,
                  NULL); 
   OVERLAPPED io;
   memset(&io,0,sizeof OVERLAPPED);
   DWORD dwWritten=0, dwRes=0;

   // Issue read operation
   const size_t cBuff = 1024;
   char buff[cBuff];
   if (!ReadFileEx(hFile,buff,cBuff,&io,func)){
      // oops! Bail!
   }

   // rest of code

}

Michael Howard is a Senior Security Program Manager in the Secure Engineering group at Microsoft and is the coauthor of Writing Secure Code, now in its second edition, and the main author of Designing Secure Web-based Applications for Windows 2000. His main focus in life is making sure people design, build, test, and document nothing short of a secure system. His favorite line is "One person's feature is another's exploit."