How to Use the Encrypting File System

 

Network Associates Laboratories

September 2002

Applies to:
    Microsoft® Windows Server® 2003
    Microsoft® Windows® XP Professional

Summary: Learn how to use the Encrypting File System (EFS), a transparent file encryption service, as it exists in Windows XP Professional and Windows Server 2003. We will first discuss the architecture of EFS including operation, data recovery, and the EFS components. We will follow with a description of how to use EFS, both from the shell and programmatically. Finally, we will finish with some discussion on EFS security.

Contents

Introduction
Architecture
Using EFS
Programmatic Use
Security
Contributors
References

Introduction

The Encrypting File System (EFS) is a transparent file encryption service provided by Windows Server 2003. EFS provides file confidentiality but does not provide any integrity or authentication protection. EFS offers an optional data recovery capability that allows access to the files if the user keys are lost or destroyed. EFS also provides for multiple users to share access to an encrypted file by using their own access credentials.

Architecture

Keys and Certificates

EFS uses standard x.509 certificates for all access credentials. Each protected file is encrypted with a randomly generated File Encryption Key (FEK) by using a symmetric encryption algorithm. EFS "wraps" the FEK by encrypting it with the public keys from one or more EFS certificates. To access an encrypted file, you must have the private key that corresponds to one of the public keys used to "wrap" the FEK. Any user that has access to one of the private keys can get access to a file by first decrypting the wrapped FEK with the private key and then decrypting the file with the recovered FEK.

Operation

EFS has four main operations: opening, reading, writing, and converting files. Because EFS is designed to be transparent, opening, reading, and writing encrypted files is no different than for normal files; applications use normal Windows APIs. Converting files is the process of either encrypting a plaintext file or decrypting an encrypted file. Conversion requires a specific interface.

Open

An application uses Windows API CreateFile() or OpenFile() to open encrypted files. When an encrypted file is opened, EFS unwraps the file encryption key and creates the necessary context to allow for decryption or encryption of data.

Read

An application uses the Windows APIs ReadFile(), ReadFileEx(), and ReadFileScatter() to read encrypted files. When an encrypted file is read, the data is first read into memory and then EFS decrypts the data in place. The decrypted data is then returned to the application. When an application requests that an encrypted file be mapped to memory, the data is decrypted immediately before it is mapped.

Write

An application uses the Windows APIs WriteFile(), WriteFileEx(), and WriteFileScatter() to write encrypted files. When an encrypted files is written, EFS encrypts the data in place, and then the encrypted data is written to disk. When an application requests that a memory-mapped file be unmapped, the mapped data is encrypted before it is written back to disk.

Convert

One of the overriding concerns with the conversion process is that data must never be lost, even when a power outage or other catastrophic failure occurs. To this end, EFS keeps backups of unencrypted data until the entire conversion process is complete.

When EFS receives a request to convert a file, a number of steps happen:

  • EFS starts by performing a number of verification checks. These checks include whether the file can be encrypted and whether there is enough disk space to encrypt the file. If the file is marked as a system file or resides in the %systemroot% directory, then the file cannot be encrypted.
  • EFS then generates a file encryption key and wraps it with the current user's public key. If a recovery policy (discussed later) is defined, the FEK is also wrapped in any required recovery agent public keys.
  • After the file encryption key has been generated and wrapped, EFS metadata is created. EFS metadata contains the Data Decryption Field (DDF), which is the FEK wrapped in all the public keys of users that have been given access to the file. It also contains the Data Recovery Field (DRF), which is the FEK wrapped in all recovery agent public keys. EFS also stores other information, such as the EFS version and encryption algorithm in the metadata.
  • Next, EFS creates a temporary file in the current directory. Each data stream in the source file is copied to the temporary file for backup purposes. Each data stream in the original file is then truncated to a length of zero. Its length is then set back to its original size. This essentially deletes all the data in the stream. Next, EFS writes the metadata to the original file. At this point, EFS has the plaintext data in a temporary file and an empty source file that is marked encrypted because of the presence of the EFS metadata.
  • EFS reads the streams from the temporary file and writes them to the original file. Due to EFS' transparent nature, the writes encrypt the data before actually writing it to disk.
  • After all of the data has been written back to the original file, EFS verifies that the file is encrypted and then deletes the temporary file. If the conversion process fails for some reason, EFS will restore the file to its original state.

Conversion to plaintext follows a similar process, in that a temporary file is created and the source file is written to it. The source file is then truncated and the data is read from the temporary file and written back to the source file in the clear.

Notes   A side effect of this process is that when the temporary file is deleted, plaintext remnants of the file may remain in unused space on the disk. The command-line tool Cipher.exe has a switch (/W) to wipe all of the marked free space on a partition. This operation can clear the plaintext remnants of all files that do not fit entirely within a Master File Table (MFT) record left over from encryption. The MFT record size depends on the disk cluster size but is usually 1024 bytes. The file size that fits within the MFT record depends on how much space is left in the MFT record after accounting for the file metadata. The MFT can contain other data, for example, file attributes or streams. Additionally, a file may have more data than the MFT record. For more information about the MFT, see the Master File Table and MFT Zone entry in the Windows Development documentation in the MSDN Library (https://msdn.microsoft.com/library/).

To completely prevent the possibility of having plaintext remnants on a disk, never convert files from a plaintext source. Instead, create a directory and encrypt it. When a file is created in an encrypted directory, the file is immediately written to the disk encrypted, and no temporary file is created.

For existing files that need encryption, convert all existing files that need protection and then use Cipher.exe to wipe all free space on the partition. From then on, all files should be created in encrypted directories.

Data Recovery

EFS provides a data recovery capability. Data recovery is used to recover encrypted files in the event that user keys are lost or destroyed. It also allows organizations to define and enforce a policy to access data stored on company systems. The capability is defined by a recovery policy that is automatically enabled on domain machines and disabled on stand-alone machines. In a domain, the recovery policy is distributed by using group policy from the active directory.

When a recovery policy is in place, all FEKs are additionally wrapped with the recovery agent public keys. These wrapped FEKs make up the Data Recovery Field (DRF), which EFS rewrites after every operation to ensure that the most current recovery policy is used.

Protecting recovery agent private keys is critical. Every encrypted file in a domain has its FEK wrapped by a recovery public key. If an attacker gains access to the corresponding recovery private key, the attacker could access any encrypted file in the domain.

Recovery policy

Domain machines

The default recovery policy in a domain consists of the domain administrator. When the domain administrator logs on to the domain controller for the first time, an EFS recovery certificate is generated and stored in the local profile. That certificate is added to the recovery policy. Any domain administrator can then create recovery agents in the domain and generate EFS certificates for those recovery agents. Those certificates can be added to the recovery policy.

When a domain machine is booting, the group policy, including the recovery policy, is retrieved from the Active Directory. The machine then overwrites whatever recovery policy is configured with the one obtained from group policy. Periodically, the machine will query the Active Directory for an updated group policy, including the recovery policy.

The recovery policy obtained from the Active Directory is cached locally on the domain-joined machine. If the machine is unable to contact the Active Directory and download the updated policy, it will use the cached policy.

A domain administrator can also configure an empty recovery policy for the domain. An empty policy disables EFS for Windows 2000 clients but not for Windows XP or Windows Server 2003 clients. An empty recovery policy is different from not defining a policy because an administrator must create an empty policy but not add any recovery agents to the policy. If an administrator removes the policy, then no recovery policy is defined.

Group policy objects can be created at multiple levels in an Active Directory tree. A local recovery policy can also be configured on a domain machine. It will, however, never be used because the domain recovery policy takes precedence. For more information about Group Policy propagation and precedence, see Windows Server 2003 Help.

Stand-alone machines

No recovery policy initially exists on a stand-alone machine. Administrators on stand-alone machines can modify the EFS recovery policy. Recovery certificates can be created and added to the policy.

Components

For EFS to work transparently, its components must exist at many layers in the operating system. These components are broken out into two groups: user-space components and kernel-space components. Figure 1 shows the components involved when files are accessed. The boxes with "(EFS)" in them contain only EFS code, while the boxes with EFS in a sub-box, contain some EFS code.

ms995356.efs-fig-01(en-us,MSDN.10).gif

Figure 1. Operating System Components and Relationships

User-space components

There are five components in user space that have EFS-specific code in them. They are: Local Security Authority (LSA), Feclient, Efsadu, Shell, and Winlogon.

Local Security Authority

The Local Security Authority (LSA) performs the majority of the user-space EFS operations. The LSA responds to requests from the kernel to create an FEK and wrap it in the proper user and recovery agent public keys. The FEK is returned to the kernel and used to encrypt a file. The LSA also performs the reverse operation of unwrapping a FEK and returning it to the kernel to decrypt a file.

During the wrapping process, the recovery policy is checked to determine whether the FEK should be wrapped by recovery agent public keys. During unwrapping, the recovery policy is checked again to determine whether additional or new recovery certificates exist, which would indicate that the FEK should be rewrapped. The LSA also checks to determine whether the current user's EFS certificate has changed, which would indicate that the FEK should be rewrapped.

The LSA exports a number of interfaces through RPC that enable communication with Feclient. The LSA also exports a number of interfaces through LPC that enable communication with Ksecdd.

To improve performance, the LSA maintains a cache of handles to private keys used for wrapping and unwrapping FEKs. These handles point to private keys that remain in the CSP and are, therefore, in a safe location.

Feclient

The Windows APIs pass all EFS-related calls to the Feclient DLL. Feclient is then responsible for calling the EFS RPC interfaces in the LSA. The reason for this two-stage process of calling the EFS routines in LSA is that EFS supports remote storage of encrypted files on other servers running Windows Server 2003 or servers running Windows 2000. The Feclient is responsible for determining which server the file resides on and contacting the LSA on that server.

Efsadu

The Efsadu DLL provides the EFS-specific user interface. It is accessed when the Advanced dialog box of the Properties dialog box is opened, and the Details button, next to Encrypt file, is clicked. Clicking this button opens the Encryption Details dialog box, discussed later.

Shell

The shell can display the file names of encrypted files in explorer windows in a green color. The shell determines whether a file is encrypted by calling an EFS file status function.

Winlogon

Winlogon behaves differently on domain controllers and local machines. Winlogon creates a default recovery policy when a domain administrator logs on to a domain controller for the first time. On a local machine, Winlogon calls the security configuration engine to process the EFS recovery policy obtained from group policy.

Kernel-space components

There are three components in kernel space that have EFS-specific code in them: NTFS, WebDAV Redirector, and Ksecdd. Their relationships are shown earlier in Figure 1.

NTFS

The NTFS driver contains the majority of the EFS code in the kernel. This driver responds to file system controls (FSCTLs) from the LSA to encrypt or decrypt a file. When a file is opened, the driver passes the wrapped FEK to Ksecdd to be unwrapped by the LSA. The driver then decrypts the data when it is read from the file and encrypts the data when it is written to the file.

The EFS API in the NTFS driver also provides a set of functions for reading and writing encrypted files in a raw mode. Raw mode allows direct reading and writing of the ciphertext. This allows the Windows file backup API to read and write the encrypted data for backup purposes while still maintaining confidentiality. This API is intended for use by backup software applications only.

The NTFS driver maintains a cache of unwrapped FEKs that it receives from the LSA to improve performance for files opened multiple times in quick succession. By default, FEKs are cached for five seconds. The cache time is configurable from two to thirty seconds.

WebDAV Redirector

The WebDAV client redirector is responsible for handling encrypted files on WebDAV shares. The redirector works by retrieving the file off the server and then caching it locally while edits are performed. After the file is closed, the redirector stores the file on the server. While the file is cached locally, the redirector calls the NTFS driver to read and write the file.

When the file is stored on the server, the client redirector performs a BackupRead() on the locally cached file and takes the data and stores it on the server. BackupRead() returns the encrypted contents of the file while the file remains encrypted on the system. When a file is retrieved from the server, the redirector performs a BackupWrite() to the cached file which takes the encrypted data and restores it to a normal, but encrypted, NTFS file.

Ksecdd

Ksecdd is a very thin component that NTFS calls to communicate with the LSA. Ksecdd is responsible for setting up the LPC communications to the LSA.

Using EFS

EFS is designed to be transparent to both users and applications; however, users will have to interact with EFS directly to perform some actions. These actions consist of converting files (encryption and decryption); adding users to files; and removing users from files. All of these actions are performed through the Advanced dialog box of the Properties dialog box, which is accessed by right-clicking a file in Windows Explorer. This dialog box is shown in Figure 2.

ms995356.efs-fig-02(en-us,MSDN.10).gif

Figure 2. Advanced dialog box of the Properties dialog box

Encryption

To encrypt a file, check the Encrypt contents to secure data checkbox in the Advanced dialog box. EFS then converts this file as discussed earlier.

To encrypt a directory, the same steps are followed as for a file. If the directory is empty, an encrypted flag is set in the directory header telling EFS that any file created in this directory is to be created encrypted. If the directory is not empty, another dialog box is displayed (shown in Figure 3), that prompts you for whether to apply the encryption to this folder only or to this folder, subfolders and files.

ms995356.efs-fig-03(en-us,MSDN.10).gif

Figure 3. Confirm encryption dialog box

If the Apply changes to this folder only option is selected, then the directory is treated as if it were empty, and the encrypted flag is set in the directory header. If, however, the Apply changes to this folder, subfolders and files option is selected, then every file in this directory, and all subdirectories, is encrypted as discussed earlier. The encrypted flag is then set for every subdirectory, including the current directory, in the directory headers.

Decryption

Decryption is the opposite of encryption and the process is the same as encrypting a file. To decrypt a file, uncheck the Encrypt contents to secure data checkbox in the Advanced dialog box. EFS then decrypts this file as discussed earlier.

To decrypt a directory, the same steps are followed as for a file. If the directory is empty, the encrypted flag is unset in the directory header. If the directory is nonempty, another dialog box is displayed (shown in Figure 4), that prompts you for whether to apply the decryption to this folder only or to this folder, subfolders and files.

ms995356.efs-fig-04(en-us,MSDN.10).gif

Figure 4. Confirm decryption dialog box

If the Apply changes to this folder only option is selected, then the directory is treated as if it were empty, and the encrypted flag is unset in the directory header. If, however, the Apply changes to this folder, subfolders and files option is selected, then every file in this directory, and all subdirectories, is decrypted as discussed earlier. The encrypted flag is then unset for every subdirectory, including the current directory, in the directory headers.

Adding Users

Adding users to a file gives those users cryptographic access to that file. Cryptographic access means the users are able to decrypt and encrypt the file, as well as add and remove other users. Having cryptographic access, however, does not imply the users have file-system access. File-system access is controlled through NTFS file access control lists (ACLs). For a user to have full access to a protected file, the ACLs must be set to allow a user to access the file in addition to adding the user being given cryptographic access.

Click the Details button in the Advanced dialog box to bring up the Encryption Details dialog box, shown in Figure 5.

ms995356.efs-fig-05(en-us,MSDN.10).gif

Figure 5. Encryption Details dialog box

To add users to a file, click the Add button. Any user that has an EFS certificate in the current user's Trusted People or Other People certificate store can be added. The active directory can also be searched for users.

Removing Users

Removing a user from a file removes the users' cryptographic access to that file. The process is the same as adding users. Click the Details button in the Advanced dialog box to bring up the Encryption Details dialog box, shown in Figure 5.

To remove users from a file, click the Remove button. Removing users only removes cryptographic access to the file. If the file-system ACLs are not adjusted, then the removed users still have file-system access to the file.

Recovery

To recover a file, the recovery agent must be logged on to the system where the file exists and must have access to the recovery certificate private key.

A recovery agent could also open the file and save it to a new file outside of an encrypted directory hierarchy, thereby saving the file unencrypted.

To avoid having the recovery agent import the recovery certificate private key onto a user workstation, a recovery station could be set up. Users could then bring the files needing recovery to the recovery station and the recovery agent could log in to the station and recover the files.

Programmatic Use

Interfaces

EFS provides a number of public Windows interfaces for interacting with encrypted files. These interfaces are available through the Windows SDK.

  • EncryptFile

  • DecryptFile

  • AddUsersToEncryptedFile

  • RemoveUsersFromEncryptedFile

  • DuplicateEncryptionInfoFile

  • QueryUsersOnEncryptedFile

  • QueryRecoveryAgentsOnEncryptedFile

  • FileEncryptionStatus

  • SetUserFileEncryptionKey

  • EncryptionDisable

Data Structures

The EFS interfaces described earlier use a number of data structures, which are also available through the Windows SDK.

  • EFS_CERTIFICATE_BLOB

  • ENCRYPTION_CERTIFICATE

  • ENCRYPTION_CERTIFICATE_LIST

  • EFS_HASH_BLOB

  • ENCRYPTION_CERTIFICATE_HASH

  • ENCRYPTION_CERTIFICATE_HASH_LIST

  • SECURITY_ATTRIBUTES

Security

Overall, EFS makes a reasonable effort at providing file confidentiality. The EFS components are well-designed and implemented. The EFS code makes good attempts to clean up resources when finished with them and to recover from system failures while performing operations. The EFS components also reuse existing code through system APIs for key and certificate management.

It is important to note that file integrity or authentication protection are not services EFS provides.

The design of EFS has made some conscious tradeoffs between absolute security and convenience. These tradeoff decisions result in some edge-case scenarios, which we describe later. None of these edge-case scenarios are bugs; they are results of the design decisions and were known from the start. Additionally, the vast majority of EFS users will never encounter these edge-case scenarios in normal use. We discuss them here simply to make users aware that using EFS does require a heightened awareness of file handling.

  • First, it is possible after certain actions for two files to share an FEK. If two files share an FEK, a user with cryptographic access to either of the files can use the FEK from the file they can access to decrypt the other file. The user must also have file-system access to the other file.

    For example, Bob creates an encrypted file. He then copies the file, by using Windows Explorer or some other method that uses the Windows CopyFile() interface, to a new file. The result is that the new file has the same FEK as the original file. If Amy can get the FEK from either file, she can then access both files.

    A recommended way to prevent this from occurring is for a user to open the original encrypted file in an editor and perform a Save As and save the data to a new file. This creates a completely new file with a new FEK.

  • Second, when a user is removed from an encrypted file, there is no absolute guarantee that cryptographic access has been fully revoked. If the newly-removed user has a backup of the original file which grants them access, they can decrypt the FEK from the backup file and access the new file. Again, the user must have file-system access to the new file.

    For example, Alice creates an encrypted file and then grants access rights to Bob. After some time, Alice removes Bob from the file. If Bob had saved the FEK from the file while he had access to it, or if he had a backup of the original file, he could then decrypt the new file after Alice had removed him. To accomplish this, Bob would have to write custom code to decrypt the file and have physical offline access to the file.

    No recommended procedure exists to prevent this scenario because after a user has access to the plaintext data of a file, the user can copy the data to another medium physically separate from the original encrypted file.

  • Third, EFS does not provide file integrity protection or file authentication. If a user can gain file-system access to an encrypted file, for exmple, by bypassing the normal file access controls, then that user can replace the original encrypted file with a different file without any notification to the file owner. Of course, this is no worse than not using EFS, and because EFS does provide confidentiality, that user will not know the contents of the original encrypted file.

    For example, Bob creates an encrypted file. Amy, in some manner, is able to gain file-system access to the file. She cannot read the contents of the file, but she can replace the encrypted file with another file, without any notification to Bob.

    A recommended way to prevent this from occurring is to maintain tight access controls and physical controls of files to prevent spoofing attacks.

Contributors

The contributors of this document are Wesley Griffin, Michael Heyman, Richard Clayton, Michael St. Johns, and David Carman, members of the Cryptographic Technologies Group in Network Associates Laboratories. Network Associates Laboratories is a multiple-discipline research organization with world-renowned expertise in the areas of network security, cryptographic technologies, security infrastructure components, secure execution environments, adaptive network defense, distributed systems security, and security architecture modeling.

The contributors would like to acknowledge the help of Mike Lai, David Cross, Robert Gu, and Drew Cooper, all at Microsoft, for providing information about EFS. We would also like to thank Doug Bayer and Dave Thompson for contributing Microsoft resources.

References

Encrypting File System in Windows XP and Windows Server 2003

MSDN Library

Windows NT File System Internals: A Developer's Guide, Rajeev Nagar, O'Reilly, September 1997.