Working with Pictures, Video, and Cameras in Windows Mobile 5.0

  • Article

 

Christian Forsberg
business anyplace

February 2006

Applies to:
   Windows Mobile version 5.0
   Microsoft Visual Studio .NET 2005
   Microsoft .NET Compact Framework version 2.0

Summary: Learn about the new multimedia features available in Windows Mobile 5.0 and the .NET Compact Framework 2.0, and learn how to use these features in your enterprise applications. This article demonstrates the new functionality by using a download code sample and additional code examples that are written in C#. (43 printed pages)

Download Working with Multimedia.msi from the Microsoft Download Center.

Contents

Introduction
Inspection Business Process
Application Design
Sample Application Walkthrough
Code Walkthrough
Media Player Control
DirectShow
Conclusion

Introduction

Windows Mobile version 5.0 provides excellent multimedia support with integrated camera application programming interfaces (APIs) that enable developers to embed camera, picture, and video functionality directly into their applications. It also includes a number of new multimedia APIs. Microsoft DirectShow provides developers with greater control and flexibility in capture and playback of video and audio streams. Microsoft Windows Media Player 10 Mobile enables developers to integrate media player functionality into their applications, including library management and playback. Microsoft DirectDraw provides developers with greater control and flexibility in rendering high-speed two-dimensional graphics. Microsoft Direct3D enables developers to deliver rich three-dimensional gaming environments, and this API is also available from managed code.

In this article, you will learn how to use the integrated camera APIs for capturing photos and videos by a practical sample application that is based on a business scenario—an inspection. The sample shows how the new high-level constructs, such as the picture selection and camera capture dialogs, can be included easily in an enterprise application. You will see how to use the media player control in your applications by using both native code and the new managed Component Object Model (COM) interoperability features with some excellent community add-in code. You will also learn about how you can use the more advanced features available in the DirectShow API, if you require more control in the interaction of media and a camera. Sample DirectShow code is included to show playback and capture of both video and audio.

This article starts by describing the basic task of creating some multimedia support for a common business scenario.

Inspection Business Process

Although the task of inspection exists in many different industries, it is usually performed in the field. Inspectors are typically mobile workers, and many parts of their work are of a data-collection nature. The challenge is that as much information as possible needs to be gathered to minimize the risk of differences in perception of the inspection made and also to maximize the trust between the parties involved. For example, a car inspector makes an inspection before a car is about to change owners, and the accuracy of the inspection is probably in the best interest of both the seller and the buyer. However, the fact that the work is done in the field, along with the requirement to gather a lot of information, creates a very tough situation.

Using traditional solutions, like handwriting on paper, might look efficient during the inspection, but not recording the information digitally creates many problems after the inspection. The hand-written notes need to be entered into an information system, and most often, someone other than the inspector enters the information. This situation often results in poor information quality and sometimes even loss of information.

Because text might not capture all information, inspectors have used photos for a long time. But traditional photos might also create problems after the inspection because the quality of the photos might not be sufficient, and the quality is discovered only after the photos return from development.

With all these problems, the concept of using a mobile device equipped with a camera to capture notes, photos, and even video sounds like a perfect solution. But because capturing information in the form of text on a mobile device like a Pocket PC is still somewhat challenging, the role of multimedia becomes more important. The right mix of capturing text, photos, dictation, and video with audio in an integrated way might be the recipe to success. This article focuses on making such a vision come to life, and it starts by formalizing the requirements. A generic business process for inspections can be defined as follows:

  1. Check out the necessary inspections from the back office (server).
  2. Record inspection with notes.
  3. Collect exhibits like photos and videos.
  4. Check inspections back in to the back office (server).

The business process can be illustrated as shown in Figure 1.

Figure 1. Inspection business process

As shown in Figure 1, the main steps in the business process involve a number of activities that work as main requirements for the new solution. Even if the second (record inspection) and third (collect exhibits) steps are put in sequence, the work of doing these tasks is probably iterative. To find more specific requirements, you can further detail the business process by expanding each process step into its own subprocess flow.

After you have satisfactorily defined the new process, the next step is to look at the design of the solution.

Application Design

The article, Northwind Pocket Service: Field Service for Windows Mobile-based Pocket PCs, provides a good introduction to the architectural work in a mobile solution. The article, Northwind Pocket Inventory: Logistics for Windows Mobile 2003-based PocketPCs, includes a description of the most important deliverables (artifacts) in the application design, and the design really begins with the definition of the use cases. Because this article has a multimedia focus, Figure 2 shows some of the most interesting use cases implemented in the sample application.

Figure 2. Use case model for collecting exhibits

Other very important artifacts to create early in the design process are the dialog model and sample dialogs. Figure 3 illustrates the dialog model for the sample application.

Figure 3. Dialog model

The dialog model gives an overview of the dialogs (forms) included in the application, and it also gives an overview of the navigation between these dialogs. Note that the dialogs with dotted borders are not implemented in this article's sample.

Figure 4 shows some sample dialogs.

Figure 4. Sample dialogs

The dialog samples are drawn in a general drawing tool (in this case, Microsoft PowerPoint), and someone knowledgeable in user interface design should be involved in creating dialogs like these. The early visualization of the application dialogs gives the users and other stakeholders an opportunity to understand how the application will look (and work), and changes are very easy to make at this stage.

The download code sample provides all of the preceding figures as a PowerPoint presentation that you can reuse when creating your own diagrams.

Sample Application Walkthrough

The example client scenario is a Pocket PC application written with Microsoft Visual Studio .NET 2005 in C# and targets the Microsoft .NET Compact Framework version 2.0.

The application shows how to support the inspection business process by using a Pocket PC. To make the sample more authentic, it uses a specific inspection scenario—a car inspector. The user of the application performs inspections for different purposes. For example, the user can perform an inspection just after an accident has occurred or after the car is repaired. Other examples are general inspections performed before a new insurance policy is signed or when the car is about to change owners. This walkthrough covers the general type of inspection, or "check" as the user interface sometimes refers to it. The scenario is that inspections are created in the back office, and when the inspector synchronizes with a back-office server, new inspections are downloaded to the device.

This article will comment on some of the design choices during the walkthrough of the application. Also note that this article explores parts of the code after it describes the application's user interface design.

Main Screen

When you start the application, the first screen is the main screen that you can use to search for and select an inspection, as shown in Figure 5.

Figure 5. Main screen (Inspections)

First, you can search for a name and a type of inspection. Then, when you press the left soft key (Find), the application searches for available inspections. Note the available types of inspections that can be searched for.

The right soft key (Menu) provides some of the general application functionality, like application options and synchronization with the server. This functionality is not implemented in this sample because the focus of this article is on multimedia, but there are numerous articles on MSDN covering these subjects (an example is Northwind Pocket Sales: Field Sales for Windows Mobile 2003 Software for Pocket PCs).

When the application makes a search, it adds the search results to the list of inspections. Tapping and holding an inspection in the list brings up a shortcut menu with the command (Inspect) to perform the selected inspection, as shown in Figure 6.

Figure 6. Search results

Inspection

When you select the Inspect command (shown earlier in Figure 6), the application brings up the Inspection screen, shown in Figure 7.

Figure 7. Inspection screen

On the Inspection screen, you can edit the information about the inspection. You can modify the name and type (for example, if the name was misspelled), and then you can enter the notes of the inspection as free text. The notes entered in Figure 7 make references to different parts of the vehicle, and it would be very valuable to complement these notes with actual photos. That is what the Exhibits section of the screen (on the lower part of Figure 7) is for. Figure 7 also shows that if you tap and hold in the Exhibits section, a shortcut menu appears so you can add (New), edit (Edit), or remove (Delete) exhibits attached to the inspection.

Exhibits

If you select the New command in the shortcut menu of the Exhibits section, the Exhibit screen appears. Here, you can enter a name for the exhibit. Then, when you press the left soft key (New), a shortcut menu appears where you can select to add an exhibit from a file (From File), by taking a new photo (Photo), or by making a new video (Video) with a built-in camera (if one is available on the device), as shown in Figure 8.

Figure 8. Exhibit screen

Selecting the From File command brings up a screen to select a media file, as shown in Figure 9.

Figure 9. File selection screen. Click the thumbnail for a larger image.

You can navigate through the list of media files by using the hardware navigation pad (up, down, left, right), and you can select a file (such as a photo or video) by pressing the Action key (usually in the middle of the hardware navigation pad). When you select a photo, you return to the Exhibit screen with the selected photo visible. When you then press the Menu soft key, a shortcut menu appears and contains a View command, as shown in Figure 10.

Figure 10. Exhibit screen with selected photo. Click the thumbnail for a larger image.

When you select the View command, the picture viewer application starts. In this application, you can examine the picture in more detail, and you can even manipulate the picture. The commands shown in Figure 11 that are available by tapping the Menu soft key demonstrate that you can send the picture (Beam picture) to another device (for example, Pocket PC, Smartphone, or desktop computer), and you can Edit it in a limited way (for example, rotate).

Figure 11. Picture viewer. Click the thumbnail for a larger image.

When you tap ok, you return to the Exhibit screen.

On the file selection screen (shown earlier in Figure 9), if you select to add a video file, the Exhibit screen appears. If you then tap Menu, a shortcut menu appears, as shown in Figure 12.

Figure 12. Exhibit screen with video selected. Click the thumbnail for a larger image.

When you select the View command for a video exhibit, the media player starts and plays the video, as shown in Figure 13.

Figure 13. Video shown in the media player. Click the thumbnail for a larger image.

Note that because the exhibit identity (a value of type uniqueidentifier) is used as the name of the file, this is the name shown in the media player.

On the Exhibit screen (shown earlier in Figure 8), if you select the Photo command to add a new photo, the application displays a screen where you can capture a new photo, as shown in Figure 14.

Figure 14. Photo capture screen. Click the thumbnail for a larger image.

When you are recording a new video, the capture screen looks very similar to that shown in Figure 14 (the symbol of a film roll is then replaced with a symbol of a movie camera). Note, however, that Figure 14 is a concept screenshot, and this screen might look slightly different on an actual Windows Mobile 5.0–based Pocket PCs.

This concludes the walkthrough of the client application, and now it's about time to look at some of the source code.

Code Walkthrough

This section walks you through some of the source code for the example client scenario. You'll see how to use the select picture and camera capture dialogs from both managed and native code. However, the native code samples are not included in this article's accompanying source code. Then, you'll see how to use other applications with your media and how to store the media in a database.

Select Picture (Managed Code)

The most important functionality when you're integrating multimedia with your enterprise applications is probably the ability to use pictures (including photos) and videos that are already stored in the file system. If the device is not equipped with a built-in camera, you can still use this method to integrate multimedia. For example, you might use a separate digital camera to take the photos or capture the videos, and then transfer the photos or videos to the Pocket PC by using the infrared port or Bluetooth. To meet this need, a ready-made dialog has been included with the Windows Mobile 5.0 platform, and it is available from managed (.NET Compact Framework) code as the SelectPictureDialog class in the "Microsoft.WindowsMobile.Forms" namespace.

To create the file selection screen shown earlier in Figure 9, you can use the following code example.

private void fromFileMenuItem_Click(object sender, EventArgs e)
{
  SelectPictureDialog selectPictureDialog = new SelectPictureDialog();
  selectPictureDialog.Owner = this;
  selectPictureDialog.Title = "Select Exhibit Photo or Video";
  selectPictureDialog.CameraAccess = false;
  selectPictureDialog.Filter = "All files|*.*";
  if(selectPictureDialog.ShowDialog() == DialogResult.OK &&
    selectPictureDialog.FileName.Length > 0)
  {
    fileExtension = Path.GetExtension(selectPictureDialog.FileName);
    File.Copy(selectPictureDialog.FileName, fileName());
    if(fileExtension.ToLower() == ".jpg")
      pictureBox.Image = new Bitmap(fileName());
    else
    {
      ComponentResourceManager resources =
        new ComponentResourceManager(typeof(ExhibitForm));
      pictureBox.Image = ((System.Drawing.Image)
        (resources.GetObject("pictureBox.Image")));
    }
  }
}

In the preceding code, you see that the owner of the dialog is set to the current window along with the title. The CameraAccess property indicates whether the camera should be available from the dialog window. If this property is set to true and a camera is present, a camera symbol is visible in the list of files. When you select the camera symbol, a CameraCaptureDialog dialog appears so you can take a photo or make a video recording. Note that because the SelectPictureDialog dialog is able to link to the CameraCaptureDialog dialog (though not shown in this sample), this is really the minimum integration you need to do to enable the inclusion of media from the file system and a built-in camera.

The Filter property defines the search filter to apply when the application shows the list of files. The default value of this property is "Image Files(*.BMP;*.JPG;*.GIF)|*.BMP;*.JPG;*.GIF".

The next thing that happens in the preceding code is the actual presentation of the dialog. This presentation happens by means of the ShowDialog method, which halts the execution until the dialog is closed. As usual, the ShowDialog method returns a value indicating how the dialog was closed. If the user pressed the Action key on the device or tapped ok, the dialog returns DialogResult.OK. If a picture was selected, the FileName property includes the name of the selected file. If both these conditions are true, the extension of the file selected is saved, and the file is copied to the sample's media folder. If the selected file is a photo (in this sample, only JPEG files are used for photos), the picture box (pictureBox) on the Exhibit screen is updated with the selected picture. If the selected file is a video, the default image (the enlarged media player document symbol shown in Figure 12) is loaded from the application resources.

The code for the fileName method is as follows.

private string fileName()
{
  return Common.Values.MediaPath + Path.DirectorySeparatorChar +
      exhibitID.ToString() + fileExtension;
}

The preceding code creates the exhibit's full file name, which is a combination of the exhibit identity (a unique identifier) and the file name extension of the selected file. All media files are stored in a common folder (Common.Values.MediaPath).

Figure 15 shows the complete definition of the SelectPictureDialog class.

Figure 15. SelectPictureDialog class

Because the Filter property can include more than one filter, you can use the FilterIndex property to set the default filter index. For example, if the Filter property is set to "Bitmap Files|*.bmp|JPEG Files|*.jpg|GIF Files|*.gif", and the FilterIndex is set to 1, the dialog will search for JPEG files.

In addition to the properties already mentioned, you can select an initial folder with the InitialDirectory property to search for pictures. You can prevent the user from changing the folder by setting the LockDirectory property to true. You can use the SortOrder property to set the initial sort order of the files found, and you can set this property to sort on date, name, or size in both ascending and descending order. You can use the ShowDrmContent property to indicate whether Digital Rights Management (DRM) protected files should appear in the dialog. If the DRM–protected files are protected such that they cannot be forwarded, they can appear in the dialog only if the ShowForwardLockedContent property is set to true.

Select Picture (Native Code)

To use the same dialog described in the previous section from native code, you use the GetOpenFileNameEx function with the OPENFILENAMEEX structure as follows.

TCHAR szFile[MAX_PATH];
OPENFILENAMEEX ofnex = {0};

ofnex.lStructSize     = sizeof(ofnex);
ofnex.hwndOwner       = g_hWnd;
ofnex.lpstrFile       = szFile;
ofnex.nMaxFile        = sizeof(szFile) / sizeof(szFile[0]);
ofnex.lpstrFilter     = TEXT("All Files (*.*)\0*.*\0");
ofnex.lpstrTitle      = TEXT("Select Exhibit Photo or Video");
ofnex.ExFlags         = OFN_EXFLAG_THUMBNAILVIEW;
ofnex.lpstrInitialDir = NULL;

if(GetOpenFileNameEx(&ofnex))
{
  // The selected file name is in szFile
}

The owner window in the preceding code is the main application window (defined globally as g_hWnd), and note that the filter parts are separated with NULL characters. You use OFN_EXFLAG_THUMBNAILVIEW to display the ListView control in thumbnail format, and the ExFlags member can include the flag OFN_EXFLAG_DETAILSVIEW to display the ListView in details format. Other options are OFN_EXFLAG_HIDEDRMPROTECTED to exclude the display of DRM-protected files and OFN_EXFLAG_HIDEDRMFORWARDLOCKED to exclude DRM-protected files that cannot be forwarded.

Capture Photo and Video (Managed Code)

Even though many mobile devices have been equipped with built-in cameras for some time, there was initially a lack of developer support for using these cameras. Almost no manufacturers made the cameras available from managed code, and even native APIs were hard to find. The only viable option for enterprise developers who wanted to integrate media into their applications was to do file-level integration (as described previously in the discussion of the picture selection dialog).

With the Windows Mobile 5.0 software, all of that has changed. A generic camera API is now defined that device manufacturers will support. Also, to meet the need of enterprise developers who are building applications by using the .NET Compact Framework, Windows Mobile 5.0 software includes a ready-made dialog. The dialog is named CameraCaptureDialog, and you can find it in the "Microsoft.WindowsMobile.Forms" namespace.

To create the photo capture screen shown earlier in Figure 14, you can use the following code example.

private void photoMenuItem_Click(object sender, EventArgs e)
{
  CameraCaptureDialog cameraCaptureDialog = new CameraCaptureDialog();
  cameraCaptureDialog.Owner = this;
  cameraCaptureDialog.Title = "Take Exhibit Photo";
  cameraCaptureDialog.Mode = CameraCaptureMode.Still;
  if(cameraCaptureDialog.ShowDialog() == DialogResult.OK &&
    cameraCaptureDialog.FileName.Length > 0)
  {
    fileExtension = Path.GetExtension(cameraCaptureDialog.FileName);
    File.Copy(cameraCaptureDialog.FileName, fileName());
    pictureBox.Image = new Bitmap(fileName());
  }
}

The CameraCaptureDialog class works very similarly to the SelectPictureDialog class discussed previously. In the preceding code, the owner of the dialog is set to the current window, along with the title. The Mode property indicates whether the camera capture dialog should be used to take a photo or to record a video. In this case, the user wants a photo (CameraCaptureMode.Still). The dialog is displayed by means of the ShowDialog method, and that method returns a value indicating how the dialog was closed. If the user tapped ok, the dialog returns DialogResult.OK. If a picture was taken, the FileName property includes the name of the newly created picture (photo) file. If both these conditions are true, the extension of the selected file is saved, the file is copied to the sample's media folder, and the picture box (pictureBox) on the Exhibit screen is updated with the selected picture.

To use the same dialog to capture a video, you use the following code example.

private void videoMenuItem_Click(object sender, EventArgs e)
{
  CameraCaptureDialog cameraCaptureDialog = new CameraCaptureDialog();
  cameraCaptureDialog.Owner = this;
  cameraCaptureDialog.Title = "Take Exhibit Video";
  cameraCaptureDialog.Mode = CameraCaptureMode.VideoWithAudio;
  if(cameraCaptureDialog.ShowDialog() == DialogResult.OK &&
    cameraCaptureDialog.FileName.Length > 0)
  {
    fileExtension = Path.GetExtension(cameraCaptureDialog.FileName);
    File.Copy(cameraCaptureDialog.FileName, fileName());
    ComponentResourceManager resources =
      new ComponentResourceManager(typeof(ExhibitForm));
    pictureBox.Image = ((System.Drawing.Image)
      (resources.GetObject("pictureBox.Image"))); ;
  }
}

Note how the preceding code is very similar to the code for capturing a photo, but the Mode property is instead set to capture a video recording with sound (CameraCaptureMode.VideoWithAudio). The Mode property can also be set to capture video without sound (CameraCaptureMode.VideoOnly). Also, the default image (the enlarged media player document symbol shown in Figure 12) is loaded from the application's resources into the picture box.

Figure 16 shows the complete definition of the CameraCaptureDialog class.

Figure 16. CameraCaptureDialog class

In addition to the properties already mentioned, if you set the DefaultFileName property before you use the ShowDialog method to show the dialog, that name will be used as the file name for the new photo or video. You can use the InitialDirectory property to specify where the captured file (photo or video) will be stored. To request a resolution of the captured photo or video, set the Resolution property to a Size instance as follows.

cameraCaptureDialog.Resolution = new Size(320, 240);

You use the StillQuality property to set the compression level of the photo. In the CameraCaptureStillQuality enumeration, the High option means that a low compression rate is used to keep the high quality of the photo. The Low option is the opposite (high compression with lower quality as a result). The Normal option creates a result with a quality between the High and Low options.

Use the VideoTimeLimit property to set a maximum recording time for the new video. The default value is zero, which means that there's no time limit.

The final property is VideoTypes, which you can use to select what type of video you want to capture. Basically, on a Windows Mobile 5.0–based device, two different video types are available—Multimedia Messaging Service (MMS) and Windows Media Video (WMV). The CameraCaptureVideoTypes enumeration provides the possible values. Use Messaging for recording an MMS video, and use Standard to record a WMV video. There is also an All enumeration value to make the Resolution property determine which type of video to record. If you use the All option and the Resolution height and width equal zero, the last used resolution is used again. You also need to set the Resolution property for the Messaging and Standard enumeration values.

Capture Photo and Video (Native Code)

If you want to use the same dialog (as described previously) from native code, you use the SHCameraCapture function with the SHCAMERACAPTURE structure as follows.

HRESULT hResult;
SHCAMERACAPTURE shcc;
ZeroMemory(&shcc, sizeof(shcc));

shcc.cbSize     = sizeof(shcc);
shcc.hwndOwner  = g_hWnd;
shcc.pszTitle   = TEXT("Take Exhibit Video");
shcc.Mode       = CAMERACAPTURE_MODE_VIDEOWITHAUDIO;
shcc.VideoTypes = CAMERACAPTURE_VIDEOTYPE_ALL;

if(S_OK == SHCameraCapture(&shcc))
{
  // The selected file name is in shcc.szFile
}

The owner window in the preceding code is the main application window (defined globally as g_hWnd). The other values for the Mode member are CAMERACAPTURE_MODE_STILL for photos (this is the default) and CAMERACAPTURE_MODE_VIDEOONLY for video without sound. The other values for the VideoTypes member are CAMERACAPTURE_VIDEOTYPE_STANDARD for WMV videos and CAMERACAPTURE_VIDEOTYPE_MESSAGING for MMS videos. When you use one of these two latter values, neither of the nResolutionWidth or nResolutionHeight members can be zero.

View Photo and Video

To open the screens shown earlier in Figure 11 (picture viewer) and Figure 13 (media player), you can use the following code.

private void viewMenuItem_Click(object sender, EventArgs e)
{
  Process process = new Process();
  process.StartInfo.FileName = fileName();
  process.StartInfo.Verb = "Open";
  process.StartInfo.UseShellExecute = true;
  process.Start();
}

You use the Process class of the "System.Diagnostics" namespace to create a new process for a specific file. Note that when the UseShellExecute property is set to true, the file name can be any file type associated with an executable file that has a default open action. On a Windows Mobile 5.0–based Pocket PCs, the .jpg extension is associated with the picture and video viewer executable file (pimg.exe), and the .wmv extension is associated with the media player executable file (wmplayer.exe).

Save Media to Database

In the previous sample, the media is saved in a file that is copied to a specific (media) folder for later synchronization with the server. The media (file) information can also be stored in the database by means of code similar to the following code example.

FileStream fs = File.Open(fileName(), FileMode.Open);
byte[] imageData = new byte[fs.Length];
fs.Read(imageData, 0, imageData.Length);
fs.Close();

SqlCeCommand cmd = cn.CreateCommand();
cmd.CommandText = "UPDATE Exhibit SET Media=?" +
  " WHERE ExhibitID='" + exhibitID + "'";
SqlCeParameter param = cmd.Parameters.Add("p0", SqlDbType.Image);
param.Value = imageData;
cmd.ExecuteNonQuery();

The file is read into a byte array, and that array is then used to update the correct exhibit row in the database. The database column holding the media data (Media) in the preceding code is assumed to be of type image. Note that to play back the media by using the external programs (pimg.exe and wmplayer.exe), the media needs to be written to a file again before the external program is called.

The choice of whether to store the media files in the file system or in the database probably depends on the requirements of the application. Both techniques have advantages and disadvantages. Storing media files in the file system makes recording, playback, and manipulation more straightforward, and it makes separating the storage of data and media easier. For example, the database can be located in the faster memory of the device to increase performance while the media is stored on a storage card with the possibility to store more data but with slower access. However, when these files need to be synchronized with a server, some custom solution (like HTTP upload and attachments to XML Web services) is needed. With the media stored in the database, the media needs to be extracted to a file whenever playback or manipulation is needed, but the synchronization is easier because both database synchronization (like Remote Data Access and Merge Replication in Microsoft SQL Server Mobile Edition) and XML Web service synchronization (DataSet over SOAP) support this data transfer. In any case, you should consider compressing the media both locally on the device and when it is transferred between the device and server.

Highlights of Code Beyond Multimedia

Now that you have walked through the multimedia functionality implemented in the sample application, you can walk through some of the other code because it is also interesting from an enterprise application perspective. For example, you use the Registry class in the .NET Compact Framework 2.0 namespace "Microsoft.Win32" as the following code example shows to get registry settings.

string registryKey = @"SOFTWARE\Microsoft\MsdnSamples\Inspection";
RegistryKey key = Registry.LocalMachine.CreateSubKey(registryKey);
string userName = key.GetValue("UserName", "<default>").ToString();

To save the same registry settings, you use the following code.

RegistryKey key = Registry.LocalMachine.OpenSubKey(registryKey, true);
key.SetValue("UserName", userName);

Another very interesting part of the code example is the data access. Starting in the user interface implementation, the following is the code to fill the Type combo box (shown earlier in Figure 5).

try
{
  DataTable dt;
  using(InspectionHandler inspectionHandler = new InspectionHandler())
    dt = inspectionHandler.GetAllTypes().Tables[0];
  DataRow dr = dt.NewRow();
  dr["TypeName"] = "<All types>";
  dr["InspectionTypeID"] = Guid.Empty;
  dt.Rows.InsertAt(dr, 0);
  typeComboBox.DisplayMember = "TypeName";
  typeComboBox.ValueMember = "InspectionTypeID";
  typeComboBox.DataSource = dt;
  typeComboBox.SelectedIndex = 0;
}
catch(Exception)
{
  MessageBox.Show("Could not load inspection types!", this.Text);
}

The data table (dt) with all types is retrieved by means of the inspection handler class instance (inspectionHandler). A first row is added to the data table before it is added as a data source to the combo box. The implementation of the method in the inspection handler (GetAllTypes) looks like the following.

public DataSet GetAllTypes()
{
  return SqlHelper.ExecuteDataset(cn, CommandType.Text,
    "SELECT * FROM InspectionType", "InspectionType");
}

The SqlHelper class is actually a somewhat enhanced version of the Data Access Application Block included in Application Blocks 1.0 from OpenNETCF. Application Blocks 1.0 is a port of the desktop computer application blocks. The method (ExecuteDataSet) takes the following parameters: database connection (connection of type SqlCeConnection), command type (commandType), statement (commandText), and name of returned table (tableName). The overload used here looks like the following code example.

public static DataSet ExecuteDataset(SqlCeConnection connection,
  CommandType commandType, string commandText, string tableName)
{
  DataSet ds = ExecuteDataset(connection, commandType, commandText,
   (SqlCeParameter[])null);
  ds.Tables[0].TableName = tableName;
  return ds;
}

The other overload that is called is implemented as shown in the following code example.

public static DataSet ExecuteDataset(SqlCeConnection connection,
  CommandType commandType, string commandText,
  params SqlCeParameter[] commandParameters)
{
  SqlCeCommand cmd = new SqlCeCommand();
  bool mustCloseConnection = false;
  PrepareCommand(cmd, connection, (SqlCeTransaction)null, commandType,
    commandText, commandParameters, out mustCloseConnection );
          
  SqlCeDataAdapter da = new SqlCeDataAdapter(cmd);
  DataSet ds = new DataSet();
  ds.Locale = CultureInfo.InvariantCulture;
  da.Fill(ds, 0, 100, "Table");
      
  cmd.Parameters.Clear();

  if(mustCloseConnection)
    connection.Close();

  return ds;
}

A SQL Mobile command (SqlCeCommand) instance (cmd) is created and prepared for execution. Then, a data adapter (da) and a data set (ds) are created, and the data adapter is used to fill the data set with data. Note how you can specify that you want to return only the first 100 rows. A limit like this is probably a good idea; a large list of results is seldom wanted because of the affect on performance and also because browsing large data sets is not very efficient on a mobile device. Any parameters are detached from the command object, so they can be used again. Finally, the created data set is returned.

You prepare the command as shown in the following code example.

private static void PrepareCommand(SqlCeCommand command,
  SqlCeConnection connection, SqlCeTransaction transaction,
  CommandType commandType, string commandText, SqlCeParameter[]
  commandParameters, out bool mustCloseConnection )
{
  if(connection.State != ConnectionState.Open)
  {
    mustCloseConnection = true;
    connection.Open();
  }
  else
    mustCloseConnection = false;

  command.Connection = connection;
  command.CommandText = commandText;
  command.CommandType = commandType;

  if(transaction != null)
    command.Transaction = transaction;

  if(commandParameters != null)
    AttachParameters(command, commandParameters);

  return;
}

The connection is opened if it is not already open, and the command is configured according to the parameters. For more details about the implementation of SqlHelper, see this article's download code sample and the Data Access Application Block documentation.

For any enterprise developer, the much improved support for data access in both the .NET Compact Framework 2.0 and Visual Studio .NET 2005 is more than welcome. The fact that you can work with the database directly from within the development tools is a huge productivity enhancer. The typical development process is as follows: You create the SQL Server Mobile Edition database file (sdf extension) in SQL Server Management Studio. You can also use that tool to add all of the tables and data. However, you can also connect to the SQL Server Mobile Edition database from the Server Explorer window in SQL Server Management Studio, where you can add tables and data, test queries, and similar items. When it's time for debugging, you can deploy the same database file along with the application. There is a large amount of other new data access functionality that you also should explore.

Media Player Control

Starting the standard media player application as a separate process (as shown earlier) will probably meet the business needs in many scenarios, but if you want more control over the video playback, you can use the media player control. The media player control (version 10) is included with the Windows Mobile 5.0 software, and it enables developers to use the media player as a custom control within their own applications.

Host Media Player Control (Native Code)

The media player control is a regular ActiveX control (.ocx file), and its use should not present many problems for native developers knowledgeable in COM. To get started, you should view the Windows Media Player Mobile code samples, which include code to host the media player control. However, because those samples were written for the previous generation of tools for native smart device development, Microsoft eMbedded Visual C++, this article's download code sample includes the results of an attempt to convert the media player sample (CEWMPHostML) by means of the eMbedded Visual C++ Upgrade Wizard for Visual Studio 2005 Beta 2.

The creation of the host window and the control is as shown in the following code example.

CAxWindow                 m_wndView;
CComPtr<IWMPPlayer>       m_spWMPPlayer;
CComPtr<IConnectionPoint> m_spConnectionPoint;
DWORD                      m_dwAdviseCookie;

RECT rcClient;
GetClientRect(&rcClient);
m_wndView.Create(m_hWnd, rcClient, NULL, 
  WS_CHILD | WS_VISIBLE | WS_CLIPCHILDREN | WS_CLIPSIBLINGS);
if(NULL == m_wndView.m_hWnd)
  goto FAILURE;

CComPtr<IAxWinHostWindow> spHost;
HRESULT hr = m_wndView.QueryHost(&spHost);
if(FAILMSG(hr))
  goto FAILURE;

hr = spHost->CreateControl(CComBSTR(
  _T("{6BF52A52-394A-11d3-B153-00C04F79FAA6}")), m_wndView, 0);
if(FAILMSG(hr))
  goto FAILURE;

hr = m_wndView.QueryControl(&m_spWMPPlayer);
if(FAILMSG(hr))
  goto FAILURE;

CComWMPEventDispatch *pEventListener = NULL;
hr = CComWMPEventDispatch::CreateInstance(&pEventListener);
CComPtr<IWMPEvents> spEventListener = pEventListener;
if(FAILMSG(hr))
  goto FAILURE;

CComPtr<IConnectionPointContainer> spConnectionContainer;
hr = m_spWMPPlayer->QueryInterface(__uuidof(IConnectionPointContainer),
  (void**)&spConnectionContainer);
if(FAILMSG(hr))
  goto FAILURE;

hr = spConnectionContainer->FindConnectionPoint(__uuidof(IWMPEvents),
  &m_spConnectionPoint);
if(FAILMSG(hr))
  goto FAILURE;

m_dwAdviseCookie = 0; 
hr = m_spConnectionPoint->Advise(spEventListener, &m_dwAdviseCookie);
if(FAILMSG(hr))
  goto FAILURE;

After the global declarations (first four rows), the control host window (m_wndView) is created as a child (and with the same size as the client area) of the main application window (m_hWnd). Then, the host window's interface reference (spHost) is retrieved and used to create the media player control. A GUID (that corresponds to the version-independent ProgID WMPlayer.OCX) is used to create the control, and the actual interface reference (m_spWMPPlayer) to the control is retrieved from the host window. Finally, the event handling is set up with the event dispatcher (pEventHandler) listening for all kinds of events. Note the extensive error handling on each call, which you will appreciate during both debugging and testing.

The next main thing that happens is that the user selects to open a file by using the menu, as shown in the following code example.

CFileOpenDlg dlgOpen;
if(dlgOpen.DoModal(m_hWnd) == IDOK)
{
  HRESULT hr = m_spWMPPlayer->put_URL(_T(dlgOpen.m_bstrName);
  if(FAILMSG(hr))
    return 0;
}
return 0;

A dialog (dlgOpen) appears where the user can select the file name. If the user closes this dialog by tapping OK, the media player's URL property is set to the name entered in the dialog box. The media player control then plays the file.

The event handling is implemented as follows.

HRESULT CWMPEventDispatch::Invoke(DISPID dispIdMember, REFIID riid,
  LCID lcid, WORD wFlags, DISPPARAMS FAR* pDispParams,
  VARIANT FAR* pVarResult, EXCEPINFO FAR* pExcepInfo,
  unsigned int FAR*  puArgErr)
{
  if (!pDispParams)
    return E_POINTER;

  if (pDispParams->cNamedArgs != 0)
    return DISP_E_NONAMEDARGS;

  HRESULT hr = DISP_E_MEMBERNOTFOUND;

  switch(dispIdMember)
  {
    case DISPID_WMPCOREEVENT_OPENSTATECHANGE:
      OpenStateChange(pDispParams->rgvarg[0].lVal);
      break;

    case DISPID_WMPCOREEVENT_PLAYSTATECHANGE:
      PlayStateChange(pDispParams->rgvarg[0].lVal);
      break;

    case DISPID_WMPCOREEVENT_AUDIOLANGUAGECHANGE:
      AudioLanguageChange(pDispParams->rgvarg[0].lVal);
      break;

    case DISPID_WMPCOREEVENT_STATUSCHANGE:
      StatusChange();
      break;

    // Here comes another 40 events

    case DISPID_WMPOCXEVENT_MOUSEUP:
      MouseUp(pDispParams->rgvarg[3].iVal,
        pDispParams->rgvarg[2].iVal, pDispParams->rgvarg[1].lVal,
        pDispParams->rgvarg[0].lVal);
      break;
  }
  return( hr);
}

For each event dispatch identity (dispIdMember), a separate method is called to act on the event. As you can see in the preceding code, another 40 events are implemented in the download code sample. The extensive set of properties, methods, and events that the media player control publishes makes this control an interesting choice for media playback.

Host Media Player Control (Managed Code)

Although the .NET Compact Framework 2.0 includes support for COM interoperability, there's no built-in support for ActiveX controls. But thanks to fellow MVP, Alex Feinman, there's a way to use ActiveX controls from managed code, and in his article, Hosting ActiveX Controls in Compact Framework 2.0 Applications you will find a way to use ActiveX controls from managed code. By using this technology and building on the previous sample, Figure 17 shows how the Exhibit screen with integrated video playback may look in a Smartphone version of the sample application.

Figure 17. Integrated playback by means of the media player control. Click the thumbnail for a larger image.

Just as in the Pocket PC sample (shown in Figure 10), the exhibit name can be edited, but here the playback can be initiated from the same screen.

The code to create this form starts with the following designer code.

private AxWMPLib.AxWindowsMediaPlayer windowsMediaPlayer;
this.windowsMediaPlayer = new AxWMPLib.AxWindowsMediaPlayer();

this.windowsMediaPlayer.Location = new System.Drawing.Point(0, 28);
this.windowsMediaPlayer.Name = "windowsMediaPlayer";
this.windowsMediaPlayer.Size = new System.Drawing.Size(176, 152);
this.windowsMediaPlayer.TabIndex = 0;
this.windowsMediaPlayer.Text = "windowsMediaPlayer";

As you can see, after the wrapping of the ActiveX control is in place, the control can be handled just like any managed control. Building on the preceding code example, the media player control can be loaded with the exhibit video as follows.

windowsMediaPlayer.URL = fileName();

The URL attribute can be a URL pointing to a streaming source or a remote file, or it can be a file in the local file system. The code behind the Menu commands (Play, Pause, and Stop) looks like the following code example.

private void playMenuItem_Click(object sender, EventArgs e)
{
  windowsMediaPlayer.Ctlcontrols.play();
}

private void pauseMenuItem_Click(object sender, EventArgs e)
{
  windowsMediaPlayer.Ctlcontrols.pause();
}

private void stopMenuItem_Click(object sender, EventArgs e)
{
  windowsMediaPlayer.Ctlcontrols.stop();
}

This basic interaction with the media player control is not very difficult, and the way you set up event handlers is not much more difficult. Just as with any other managed control, you add an event handler as follows.

this.windowsMediaPlayer.StatusChange +=
  new System.EventHandler(windowsMediaPlayer_StatusChange);

Then, the event handler is implemented as follows.

void windowsMediaPlayer_StatusChange(object sender, System.EventArgs e)
{
  string status = windowsMediaPlayer.status;
  // Do something with status string
}

Just as mentioned in the discussion of the use of the media player control from native code, this control has many possibilities. Almost everything that you can do with the stand-alone media player is available in the control. Examples include working with play lists, connections, remote media, player state, and many events (such as playing and end of stream).

DirectShow

Now that you have covered the basics, it's time to look at the more advanced uses of the multimedia support in the Windows Mobile 5.0 software. One of the most interesting developments is the inclusion of the DirectShow API in Windows Mobile 5.0 software. The main reason for looking at this API is that you need more control over the multimedia used in your applications. If the high-level support described previously is not sufficient for meeting the requirements of your application, DirectShow provides more control and greater flexibility. DirectShow is a native API and is therefore available through the native toolset (Visual C++), as shown in the code examples later in this article.

If you want to access DirectShow from managed code, one solution is to package the use of DirectShow in a native DLL that your managed code calls by using platform invoke. This is probably a good solution for functionality, like media capture, that does not depend on user interface components (like media playback). Another solution is to wrap the DirectShow COM components by using the COM interoperability included in the .NET Compact Framework 2.0. There is an interesting project called DirectShowNET Library that provides a managed wrapper for DirectShow on the desktop computer, but there is currently no version available for use with the .NET Compact Framework.

A good general approach is to implement the user interface, business logic, database access, and basic camera interaction by using managed code. Then, as more control and flexibility is required, you can use the media player control for media playback and DirectShow for media capture (with the functionality wrapped in a native DLL that is called through platform invoke from the managed code).

DirectShow was known internally as Quartz during its development, and that is still the name of the main DLL. It was first released as Microsoft ActiveMovie version 1.0 in July 1996, when it provided an ActiveX control for media playback that supported playback of Motion Picture Experts Group (MPEG) 1, Audio-Video Interleaved (AVI), and QuickTime videos, in addition to audio files. After being part of the Microsoft DirectX SDK for some years, the DirectShow SDK is now included with the Platform SDK, and it provides tools and information for developing DirectShow filters and applications.

DirectShow for Windows Mobile 5.0 software is an architecture for streaming media that provides both high-quality playback and capture of multimedia streams. It supports many formats, such as waveform (WAV), MP3 (MPEG Audio Layer-3), AVI, Advanced Streaming Format (ASF), and MPEG.

DirectShow is integrated with the other DirectX technologies that Windows Mobile 5.0 supports—DirectDraw and Direct3D. DirectShow uses any available video and audio acceleration hardware, but it also supports systems that do not use acceleration hardware.

DirectShow simplifies media playback and format conversion, but for applications that need custom solutions, it also provides access to the underlying stream control architecture. For example, you can create your own components to support new media formats or custom effects. Examples of applications you can write by using DirectShow are AVI and MP3 players, an AVI–to–ASF converter, and audio/video capture and editing applications. DirectShow is based on COM and provides a number of COM components. To extend DirectShow, you need to implement your own COM components.

Filters and Filter Graphs

The main building block of DirectShow is a component called a filter. A filter is a software (actually a COM) component that performs an operation on a multimedia stream. For example, filters can read files, get video from a video capture device, decode various stream formats, and pass data to the graphics or sound card.

Filters receive input and produce output, and the information is passed between filters though the filter pins. A pin is a filter port, and it can be either an input port or an output port. If a filter decodes WMV video, the input is the WMV–encoded stream and the output is a series of uncompressed video frames. In DirectShow, an application performs any task by connecting chains of filters together, so the output from one filter becomes the input for another. A set of connected filters is called a filter graph, and Figure 18 shows a filter graph for playing a video file with sound.

Figure 18. Filter graph of a typical video file

A filter graph must follow certain guidelines, and the first guideline is that you need a source filter. This is the initial source of your data, whether it's a file, a URL for streaming media, or a device such as a built-in camera. The output of the source filter is then run through any number of transform filters. Transform filters are any intermediate filters that take a certain type of input data, modify the data coming in, and then pass the modified data to its output. The final piece of a graph is a renderer filter. Renderer filters are the final destination of any data handled in a filter graph. Renderers can represent things such as a window for displaying video on the screen, a sound card for emitting sound, or a file writer for storing data to disk.

In Figure 18, the filters are as follows:

  • The file source filter reads the video file from the file system.
  • The splitter filter parses the file content into two streams, a compressed video stream and an audio stream.
  • The video decoder filter decodes the video frames.
  • The video renderer filter draws the frames to the display by using DirectDraw or graphics device interface (GDI).
  • The Sound Device filter plays the audio stream by using DirectSound.

Note that the small squares on the edge of the filters in Figure 18 represent the pins of each filter.

Included with the DirectShow SDK is a tool for working with filter graphs called GraphEdit. Figure 19 shows a WMV video file that the GraphEdit tool has rendered.

Figure 19. Filter Graph of a .wmv file in the GraphEdit tool. Click the thumbnail for a larger image.

You can use the GraphEdit tool to render files, build custom graphs, test custom filters, step (frame by frame) through a graph, and conduct similar tasks. Using this tool will save you a lot of development time if the graphs you are using in your application are already confirmed to run in GraphEdit. (It is important to remember that if the filter graph does not work in GraphEdit, it's not going to work in your application either.) GraphEdit can even save a complete filter graph to a file (.grf extension), and your application can later load that file.

DirectShow Application Structure

The first thing that a DirectShow application does is to use the most important COM component, the Filter Graph Manager. To save the application developer from the complex task of managing the filters and their interaction, this component is a high-level construct that simplifies the control of the filter graph and its filters. You can use the Filter Graph Manager to build the filter graph by connecting the filters together, and then your application can make simple calls like Run, Pause, and Stop to control the flow of data through a filter graph. During the processing of the filter graph, the Filter Graph Manager also passes event notifications to the application. For more control over the streaming process, you can also access the filters directly through their COM interfaces. In any case, a good general knowledge of COM is an asset when you're using DirectShow.

As a quick recapitulation, the typical steps of a DirectShow application are:

  1. Create a Filter Graph Manager instance.
  2. Use the Filter Graph Manager instance to build a filter graph by using the rendering functionality or filters directly.
  3. Control the media flow by making high-level calls to the Filter Graph Manager instance, and respond to events that the Filter Graph Manager instance raises.

When processing is complete, release the Filter Graph Manager instance and all filters used.

One important thing to realize when you're using DirectShow for media playback is that it uses separate threads to run the filter graph. During the execution of a DirectShow filter graph, you can see that several threads are created and run because DirectShow creates one thread for the Filter Graph Manager, and then creates a separate thread for each filter in the filter graph. Your application will therefore continue to run while DirectShow plays back the media file, and that is a good thing for user interface responsiveness in most applications. However, you need to allow enough time for the DirectShow threads to run. For example, if your application's main thread does a lot of processing while playing a media file, the media will be choppy due to the lower priority of those threads.

Video Playback

A simple task, such as showing a video file, and some basic console application code can look like the following.

#include <dshow.h>
void __cdecl main(void)
{
  IGraphBuilder *pGraphBuilder;
  IMediaControl *pMediaControl;
  CoInitialize(NULL);
    
  CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC, 
    IID_IGraphBuilder, (void **)&pGraphBuilder);
  pGraphBuilder->QueryInterface(IID_IMediaControl,
    (void **)&pMediaControl);

  pGraphBuilder->RenderFile(L"\\test.wmv", NULL);

  pMediaControl->Run();

  MessageBox(NULL, "Click OK to end playback.", "DirectShow", MB_OK);

  pMediaControl->Release();
  pGraphBuilder->Release();
  CoUninitialize();
}

The previous code example starts by declaring and initializing the COM library, and then that COM library is used to create the Filter Graph Manager instance with references to both the graph builder (IGraphBuilder) and media control (IMediaControl) interfaces. The graph builder creates the filter graph by rendering (using the RenderFile method) a video file (test.wmv), and then the media control interface (pMediaControl) starts (with the Run method) the filter graph processing. A message box appears to prevent the application from closing, but that does not affect the rendering of the video because the filter graph runs on a separate thread. When the user taps OK in the message box, the interface references are released along with the COM library.

A more sophisticated solution is to listen for events from the Filter Graph Manager instance by using the following code.

#include <dshow.h>
void __cdecl main(void)
{
  IGraphBuilder *pGraphBuilder;
  IMediaControl *pMediaControl;
  IMediaEvent   *pMediaEvent;
  CoInitialize(NULL);
    
  CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC, 
    IID_IGraphBuilder, (void **)&pGraphBuilder);
  pGraphBuilder->QueryInterface(IID_IMediaControl,
    (void **)&pMediaControl);
  pGraphBuilder->QueryInterface(IID_IMediaEvent,
    (void **)&pMediaEvent);

  pGraphBuilder->RenderFile(L"\\test.wmv", NULL);

  pMediaControl->Run();

  long eventCode;
  pMediaEvent->WaitForCompletion(INFINITE, &eventCode);

  pMediaControl->Release();
  pGraphBuilder->Release();
  CoUninitialize();
}

Note that the preceding code is the almost the same as the basic code example before it, with the additions in bold. Here you create a reference to the event (IMediaEvent) interface, and that reference is used to wait for the completion of the filter graph processing. In a real-world application, however, you should avoid the use of INFINITE because it can cause the application to block indefinitely.

If you don't specify otherwise, the playback occurs in a separate pop-up window. But in many cases, you probably want to make the playback window a child window in your application. To specify the owner, style, and position of the playback window, you can use the following code (modified from the previous code example).

#include <dshow.h>
void __cdecl main(void)
{
  IGraphBuilder *pGraphBuilder;
  IMediaControl *pMediaControl;
  IMediaEvent   *pMediaEvent;
  IVideoWindow  *pVideoWindow;
  CoInitialize(NULL);
    
  CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC, 
    IID_IGraphBuilder, (void **)&pGraphBuilder);
  pGraphBuilder->QueryInterface(IID_IMediaControl,
    (void **)&pMediaControl);
  pGraphBuilder->QueryInterface(IID_IMediaEvent,
    (void **)&pMediaEvent);
  pGraphBuilder->QueryInterface(IID_IVideoWindow,
    (void **)&pVideoWindow);

  pGraphBuilder->RenderFile(L"\\test.wmv", NULL);

  pVideoWindow->put_Owner((OAHWND)g_hwnd);
  pVideoWindow->put_WindowStyle(WS_CHILD | WS_CLIPSIBLINGS);

  RECT rect;
  GetClientRect(g_hWnd, &rect);
  pVideoWindow->SetWindowPosition(0, 0, rect.right, rect.bottom);

  pMediaControl->Run();

  long eventCode;
  pMediaEvent->WaitForCompletion(INFINITE, &eventCode);

  pVideoWindow->Release();
  pMediaControl->Release();
  pGraphBuilder->Release();
  CoUninitialize();
}

Again, the changes are in bold. First, the owner and the style for a typical child window are set. Then, the size of the video playback window is set to the same size and position as the client area (loaded in rect) of the main application window (g_hWnd).

Note that in the previous code samples, the error handling is excluded to make them more easily read. For completeness, the same code with error handling included is as follows.

#include <dshow.h>
void __cdecl main(void)
{
  IGraphBuilder *pGraphBuilder;
  IMediaControl *pMediaControl;
  IMediaEvent   *pMediaEvent;
  IVideoWindow  *pVideoWindow;
  HRESULT hr = CoInitialize(NULL);
  if(FAILED(hr))
  {
    printf("ERROR: Couldn't initialize COM library!");
    return;
  }

  hr = CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC, 
    IID_IGraphBuilder, (void **)&pGraphBuilder);
  if(FAILED(hr))
  {
    printf("ERROR: Couldn't create Filter Graph Manager instance!");
    return;
  }

  pGraphBuilder->QueryInterface(IID_IMediaControl,
    (void **)&pMediaControl);
  pGraphBuilder->QueryInterface(IID_IMediaEvent,
    (void **)&pMediaEvent);
  pGraphBuilder->QueryInterface(IID_IVideoWindow,
    (void **)&pVideoWindow);

  hr = pGraphBuilder->RenderFile(L"\\test.wmv", NULL);
  if(SUCCEEDED(hr))
  {
    pVideoWindow->put_Owner((OAHWND)g_hwnd);
    pVideoWindow->put_WindowStyle(WS_CHILD | WS_CLIPSIBLINGS);

    RECT rect;
    GetClientRect(g_hWnd, &rect);
    pVideoWindow->SetWindowPosition(0, 0, rect.right, rect.bottom);

    hr = pMediaControl->Run();
    if(SUCCEEDED(hr))
    {
      long eventCode;
      pMediaEvent->WaitForCompletion(INFINITE, &eventCode);
    }
    else
      printf("ERROR: Couldn't run filter graph!");
  }
  else
    printf("ERROR: Couldn't render video file!");

  pVideoWindow->Release();
  pMediaControl->Release();
  pGraphBuilder->Release();
  CoUninitialize();
}

To receive events that the Filter Graph Manager raises, you need the following two global declarations.

#define WM_FILTERGRAPHNOTIFY WM_APP + 1
IMediaEventEx *g_pMediaEventEx = NULL;

Note that the constant WM_FILTERGRAPHNOTIFY could be set to any value, and WM_APP + 1 is just an example. Just before playing the stream (running the filter graph), use the following code.

g_pGraphBuilder->QueryInterface(IID_IMediaEventEx,
  (void **)&g_pMediaEventEx);
g_pMediaEventEx->SetNotifyWindow((OAHWND)g_hWnd,
  WM_FILTERGRAPHNOTIFY, 0);

The preceding code instructs the Filter Graph Manager to send events to your main application window (g_hWnd) with the message identity of the second parameter (WM_FILTERGRAPHNOTIFY). Note that the third parameter of the SetNotifyWindow call will come back to the application as the lParam parameter of the window message (WM_FILTERGRAPHNOTIFY). In the previous example code, this parameter is not used, and it is therefore set to zero. However, this parameter can be used to pass instance data along with the events.

Now, you can add the following code to the message loop of your application (usually in the WndProc function).

case WM_FILTERGRAPHNOTIFY:
  HandleFilterGraphEvent();
  break;

Then, you can use the following code for the event handling.

void HandleFilterGraphEvent()
{
  if (g_pMediaEventEx == NULL)
    return;

  long eventCode;
  LONG_PTR param1, param2;
  HRESULT hr;
  while(SUCCEEDED(g_pMediaEventEx->GetEvent(&eventCode,
    &param1, &param2, 0)))
  {
    g_pMediaEventEx->FreeEventParams(eventCode, param1, param2);
    switch(eventCode)
    {
      case EC_COMPLETE:
      case EC_USERABORT:
      case EC_ERRORABORT:
        g_pMediaControl->Stop();
        long eventCode;
        g_pMediaEvent->WaitForCompletion(INFINITE, &eventCode);
        g_pMediaEventEx->SetNotifyWindow(NULL, 0, 0);
        g_pMediaEventEx->Release();
        g_pMediaEventEx = NULL;
        // Do other clean-up (releases, etc.)
        PostQuitMessage(0);
        return;
    }
  } 
}

The handling is exited if the event pointer is not set, and then all of the events on the queue are retrieved. The fourth parameter of the GetEvent method is the time (in milliseconds) to wait for an event. Because the events from the Filter Graph Manager are already on the queue, this parameter can be set to zero, which means not to wait. Note that the EC_COMPLETE event does not automatically stop the filter graph from processing, so it is good practice to stop the filter graph when this event is received.

This introduction should get you started implementing media playback in your own applications, so now this article will address the more complex task of capturing video and sound.

Video Capture

Creating a filter graph for video and audio capture is more complicated than creating a filter graph for playback. Figure 20 shows a typical filter graph for video capture with sound.

Figure 20. Filter graph for video capture with sound

To help you create and control capture filter graphs, DirectShow provides a component called Capture Graph Builder. Just as for the playback filter graphs, you start by creating a Filter Graph Manager instance. Then, you create the Capture Graph Builder instance and connect the two. The native code looks like the following.

IGraphBuilder *pGraphBuilder;
ICaptureGraphBuilder2 *pCaptureGraphBuilder;

CoCreateInstance(CLSID_FilterGraph, 0, CLSCTX_INPROC_SERVER,
  IID_IGraphBuilder, (void**)&pGraphBuilder);

CoCreateInstance(CLSID_CaptureGraphBuilder, NULL,
  CLSCTX_INPROC_SERVER, IID_ICaptureGraphBuilder2,
  (void**)&pCaptureGraphBuilder);

pCaptureGraphBuilder->SetFiltergraph(pGraphBuilder);

Because it will save you some coding (especially regarding the COM-related code), you can use the following Active Template Library (ATL) code instead.

CComPtr<IGraphBuilder>         pGraphBuilder;
CComPtr<ICaptureGraphBuilder2> pCaptureGraphBuilder;

pCaptureGraphBuilder.CoCreateInstance(CLSID_CaptureGraphBuilder);
pGraphBuilder.CoCreateInstance(CLSID_FilterGraph);
pCaptureGraphBuilder->SetFiltergraph(pGraphBuilder);

Continuing with ATL, the next step is to initialize the video capture filter (shown earlier in Figure 20) by using the following code.

CComPtr<IBaseFilter>         pVideoCapture;
CComPtr<IPersistPropertyBag> pPropertyBag;
DEVMGR_DEVICE_INFORMATION    di;
CPropertyBag                 PropBag;
CComVariant                  varCamName;
GUID guidCamera = { 0xCB998A05, 0x122C, 0x4166, 0x84, 0x6A,
                    0x93, 0x3E, 0x4D, 0x7E, 0x3C, 0x86 };

di.dwSize = sizeof(di);
HANDLE handle = FindFirstDevice(DeviceSearchByGuid, &guidCamera, &di);
FindClose(handle);
pVideoCapture.CoCreateInstance(CLSID_VideoCapture)); 
pVideoCapture.QueryInterface(&pPropertyBag));
varCamName = di.szLegacyName;
PropBag.Write(L"VCapName", &varCamName);   
pPropertyBag->Load(&PropBag, NULL);
pPropertyBag.Release();
pGraphBuilder->AddFilter(pVideoCapture, L"Video capture source");

The first camera capture device is retrieved by means of the FindFirstDevice function with the second parameter set to DEVCLASS_CAMERA_GUID (which corresponds to the GUID [CB998A05-122C-4166-846A-933E4D7E3C86] that is hard-coded in the preceding code), which is the most reliable way of finding a capture device. A property bag instance (PropBag is an instance of a custom class [CPropertyBag] that implements the IPropertyBag interface) is used to pass the capture device name information to the capture filter, and then the video capture filter is added to the filter graph.

The next step is to initialize the audio capture filter, and to do that, you can use the following code.

CComPtr<IBaseFilter> pAudioCaptureFilter;

pAudioCaptureFilter.CoCreateInstance(CLSID_AudioCapture);
pAudioCaptureFilter.QueryInterface(&pPropertyBag);
pPropertyBag->Load(NULL, NULL);
pGraphBuilder->AddFilter(pAudioCaptureFilter, L"Audio Capture Filter");

The audio capture filter is created and added to the filter graph. Now it's time to initialize the video encoder and add it to the filter graph. You have the option of using DirectX Media Object (DMO) instances inside a filter graph with the help of the DMO Wrapper filter. To encode the video with the WMV 9 DMO by using the DMO Wrapper filter, you can use the following code.

CComPtr<IBaseFilter>       pVideoEncoder;
CComPtr<IDMOWrapperFilter> pVideoWrapperFilter;

pVideoEncoder.CoCreateInstance(CLSID_DMOWrapperFilter);
pVideoEncoder.QueryInterface(&pVideoWrapperFilter);
pVideoWrapperFilter->Init(CLSID_CWMV9EncMediaObject,
  DMOCATEGORY_VIDEO_ENCODER);
pGraphBuilder->AddFilter(pVideoEncoder, L"WMV9 DMO Encoder");

After the WMV 9 encoder is loaded into the DMO Wrapper filter, it is time to load the ASF multiplexer and set the file name of the multiplexer (by using a reference to the multiplexer's file sink interface). The code to do that looks like the following.

CComPtr<IBaseFilter>     pAsfWriter;
CComPtr<IFileSinkFilter> pFileSink;

pAsfWriter.CoCreateInstance(CLSID_ASFWriter);
pAsfWriter->QueryInterface(IID_IFileSinkFilter, (void**) &pFileSink);
pFileSink->SetFileName(L"\\My Documents\\test.asf", NULL);

You have now created all of the necessary filters in the graph. The next step is to connect the pins of the filters together by using the following code.

pCaptureGraphBuilder->RenderStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Video, pVideoCapture, pVideoEncoder, pAsfWriter );

pCaptureGraphBuilder->RenderStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Audio, pAudioCaptureFilter, NULL, pAsfWriter );

pCaptureGraphBuilder->RenderStream(&PIN_CATEGORY_PREVIEW,
  &MEDIATYPE_Video, pVideoCapture, NULL, NULL );

The video capture is connected to the multiplexer through the video encoder, and then the audio capture is also connected to the multiplexer. Last, the preview pin of the video capture filter is connected to the video renderer. The video renderer does not need to be specified (as the last parameter) because it is the default.

Now that you have initialized the filters, added the filters to the filter graph, and connected all of the pins, you are ready to capture data by using the following code.

CComPtr<IMediaControl> pMediaControl;
CComPtr<IMediaEvent>   pMediaEvent;
CComPtr<IMediaSeeking> pMediaSeeking;

pCaptureGraphBuilder->ControlStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Video, pVideoCapture, 0, 0 , 0, 0);
pCaptureGraphBuilder->ControlStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Audio, pAudioCaptureFilter, 0, 0, 0, 0);

pGraphBuilder.QueryInterface(&pMediaControl);
pMediaControl->Run();
Sleep(1000);

LONGLONG dwStart = 0;
LONGLONG dwEnd = MAXLONGLONG;
OutputDebugString(L"Starting to capture the first file" );
pCaptureGraphBuilder->ControlStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Video, pVideoCapture, &dwStart, &dwEnd, 0, 0);
pCaptureGraphBuilder->ControlStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Audio, pAudioCaptureFilter, &dwStart, &dwEnd, 0, 0);
Sleep(5000);

OutputDebugString(L"Stopping the capture");
pGraphBuilder.QueryInterface(&pMediaSeeking);
pMediaSeeking->GetCurrentPosition(&dwEnd);
pCaptureGraphBuilder->ControlStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Video, pVideoCapture, &dwStart, &dwEnd, 1, 2);
pCaptureGraphBuilder->ControlStream(&PIN_CATEGORY_CAPTURE,
  &MEDIATYPE_Audio, pAudioCaptureFilter, &dwStart, &dwEnd, 1, 2);

OutputDebugString(L"Wating for the control stream events");
pGraphBuilder.QueryInterface(&pMediaEvent);
long lEventCode;
LONG_PTR lParam1, lParam2;
do
{
  pMediaEvent->GetEvent(&lEventCode, &lParam1, &lParam2, INFINITE);
  pMediaEvent->FreeEventParams(lEventCode, lParam1, lParam2);

  if(lEventCode == EC_STREAM_CONTROL_STOPPED)
  {
    OutputDebugString(L"Received a control stream stop event");
    count++;
  }
} while(count < 2);

OutputDebugString(L"The file has been captured");

The capture graph is blocked by means of the video and audio control streams, and then it is allowed to run for a second prior to actually capturing data. This delay gives the capture graph time to make sure that all of its buffers are allocated and all processes are synchronized. A capture is made for five seconds, and then the capture is stopped. The control streams for video and audio are used to stop the stream, and finally a loop waits for an event that signals that the stream is stopped.

A more deterministic way (than waiting for a second) of knowing when the capture graph is running would be to use code similar to the following.

OAFilterState state = State_Stopped;
pMediaControl->Run();  
while(state != State_Running)  
  pMediaControl->GetState(100, &state);

The first parameter of the GetState method is the timeout in milliseconds; therefore, this code will try every tenth of a second to see if the capture graph is up and running.

For more details about video capture, see the CameraCapture sample that comes with the Windows Mobile 5.0 Pocket PC SDK.

Custom Filters

As mentioned earlier, there are basically three types of filters: source filters, transform filters, and renderer filters. Source filters provide raw multimedia data from a source like a file, a URL, or a live source like a camera. Source filters can just pass the raw data on to a parser or splitter filter, or they can do the parsing or splitting themselves. Renderer filters accept fully processed data and present it on a monitor or speaker, and they include filters that write files. All filters between the source filters and the renderer filters are transform filters. Transform filters use raw or already partially processed data and process it more before passing it on to the next filter. There are many different types of transform filters; some parse byte streams into samples or frames, whereas others do compression or decompression or even format conversions.

Even though DirectShow includes a number of ready-made filters for playing, converting, and capturing many different media formats, developers can build their own custom filters for handling custom or standard data formats. When a custom filter is implemented, it is likely to be a transform filter. It may be a filter to add effects to a video stream, like fade-in or fade-out.

The DirectShow SDK includes a number of custom filters, and the SDK documentation provides a good introduction to writing custom filters. The following extract from the SDK documentation provides the basics steps of creating a transform filter:

  1. Determine whether the filter must copy media samples or can handle them in place. The fewer copies in the media stream, the better. However, some filters require a copy operation; this requirement influences the choice of base classes.
  2. Determine which base classes to use and derive the filter class (and pin classes, if necessary) from the base classes. In this step, you create the header or headers for your filter. In many cases, you can use the transform base classes, derive your class from the correct transform filter class, and override a few member functions. In other cases, you can use the more generic base classes. These classes implement most of the connection and negotiation mechanism; but the classes also allow flexibility at the cost of overriding more member functions.
  3. Add the code necessary to instantiate the filter. This step requires adding a static CreateInstance member function to your derived class and also a global array that contains the name of the filter, a CLSID, and a pointer to that member function.
  4. Call the NonDelegatingQueryInterface function to distribute any unique interfaces in your filter. This step addresses the COM aspects of implementing interfaces, other than those in the base classes.
  5. Override the base class member functions. This step includes writing the transform function that is unique to your filter and overriding a few member functions that are necessary for the connection process, such as setting the allocator size or providing media types.

For more details, see the SDK documentation.

Conclusion

The extended support for multimedia in Windows Mobile 5.0 software can empower both your managed and native applications, and the high-level constructs like the picture selection and camera capture dialogs are easily integrated into the applications. For many advanced media playback scenarios, the media player control can be a valid option. To get even more control and flexibility, the DirectShow API provides the low-level functionality to capture, decode, render, and convert video and audio streams. With these resources, you can fulfill the multimedia requirements that your users request.