Integrating Rich Client Communications with the Microsoft Real-Time Communications API

Click here to download sample - RTCSampleCode.exe.

Duane Burton
Sr. Technical Marketing Engineer
Intel Corporation

Jim Huang
Sr. Technical Marketing Engineer
Intel Corporation

June 2002

Applies to:
     Microsoft® Windows® XP

Summary: Learn the basics of how to create or integrate features of the Real-time Communications (RTC) application programming interface (API) to enable audio and video conferencing, application sharing, whiteboard sharing, a simple point-to-point chat session, and interfacing with the audio and video tuning wizard. The RTC API provides remarkable PC-based communication innovations for any Microsoft Windows XP-based application. (13 printed pages)

Download RTCSampleCode.zip.

Introduction
The RTC Client Interface
Initializing a Session
Handling RTC Events
Creating a Communication Session
Processing Real-Time Streaming Session Events
Closing the Session
Platform Performance
Conclusion
Resources

The Microsoft Real-time Communications (RTC) application programming interface (API) provides remarkable PC-based communication innovations—instant messaging, voice and video conferencing, and application sharing/collaboration—for any Microsoft® Windows® XP-based application.

Using rich communications with the RTC API is a straightforward process.

1. The enhanced client application determines the capabilities of the client communication platform.
2. The application selects the preferred audio and video devices to use during the communication session.
3. The application initiates a communication session.
4. The RTC layer coordinates the capture, compression, and transmission of data, thus removing this task from the application. The audio and video codecs used is based on the connection quality between both participants.
5. The session participant's application receives, decompresses, and replays the transmitted data.

Figure 1. Sample UI for the Audio/Video conferencing

This article describes how to add PC-to-PC RTC based capabilities to an application; we assume that you are familiar with developing Windows applications using COM objects. The source code discussed throughout this article is available in its entirety through the link provided at the beginning of this document. We'll discuss PC-to-Phone, presence, and XML profiles in a future article.

The sample code demonstrates the basics of how to create or integrate features of the Real-time Communications API to enable audio and video conferencing, application sharing, whiteboard sharing, a simple point-to-point chat session, and interfacing with the audio and video tuning wizard. Other features which are available through RTC but will not be discussed in this document are Acoustic Echo Cancellation (AEC), Forward Error Correction (FEC), bandwidth estimation, Dynamic Jitter Buffer management, Automatic Gain Control (AGC), and Quality Control (QC) Algorithms. Media Support in the Microsoft Windows Real-Time Communications Client describes the above features.

Required Header File: rtccore.h

The application you're enhancing acquires the RTC client interface with CoCreateInstance() using CLSID_RTCClient (GUID = {7a42ea29-a2b7-40c4-b091-f6f024aa89be}). Once the interface is available, Initialize() the COM object to determine the communication session capabilities of the platform.

   // Initialize the RTC COM object
   hr = CoCreateInstance (CLSID_RTCClient, NULL,
      CLSCTX_INPROC_SERVER, IID_IRTCClient,
      (LPVOID *)&m_pClient);      

      // Initialize the client interface
hr = m_pClient->Initialize();

The next step is to select the preferred types of communication and associated devices (camera, microphone). The default is to enable all communication types. If the participants in the communication session can share applications, transmit instant messages, audio, and video, those features are all enabled automatically. If a participant does not support a particular communication type, it is disabled for all session participants.

   m_pClient->SetPreferredMediaTypes ( RTCMT_ALL, VARIANT_TRUE );

The platform capabilities of the session participants and the available bandwidth determine which audio and video codecs to use.

• Video. The Windows RTC client supports the H.261 and H.263 codecs at QCIF (176×144) resolution. These variable bit rate codecs send video data between the rates of 6 and 125 Kbps. It is possible to influence the targeted spatial and temporal resolution of the video transition using the IRTCClient interface methods put_MaxBitRate and put_TemporalSpatialTradeOff.
• Audio. The Windows RTC client supports many audio codecs. The audio codec is based on the quality of the connection between both end-points. The following table lists the supported audio codecs.
  Expand table

  CODEC	Sampling Rate (kHz)	Bit Rate (Kbps)	Frame Size (msec)
  G.711	8	64	20
  G.722.1	16	24	20
  G.723	8	6.4	30, 60, or 90
  GSM	8	13	20
  DVI4	8	32	20
  SIREN	16	16	20, or 40

Adjust the communication devices after the preferred communication types and associated devices are selected. The RTC API provides a wizard to help fine tune the camera and microphone. Use the method InvokeTuningWizard() of the RTCClient interface to adjust their settings.

Figure 2. Camera Tuning Wizard

Figure 3. Microphone Tuning Wizard

Before the application can connect to other participants, it has to be able to process the events that RTC fires off during the session. In PC-to-PC communications, the application traps events for instant messaging, volume intensity, media, client messages, and session state changes. The code below demonstrates how to only create an event filter to trap for specific types of RTC events.

The lEventMask sets a set of events that the application is interested in. (For a complete list of events available, search for RTC_EVENT on the MSDN Web site for more information of each event.) The CRTCEvents class dispatches the events for the attached client. The RTCEvents object creates an interface between the application and the IRTCEventNotification interface. All RTC events will be processed by the RTCEvents class.

    // Set the event filter to listen for RTC events
    // Using RTCEF_ALL will listen for all events
    // For the sample application, we will demonstrate how to set the 
    // event listener for a limited set of events.
    long lEventMask = RTCEF_SESSION_STATE_CHANGE |
                 RTCEF_MESSAGING |
                 RTCEF_MEDIA |
                 RTCEF_INTENSITY |
                 RTCEF_CLIENT;

    hr = m_pClient->put_EventFilter( lEventMask );

    // Create the event sink object
    m_pEvents = new CRTCEvents;

    // initialize the event handler
    hr = m_pEvents->Advise( m_pClient, m_hWnd );

    // Set the listen mode for RTC client
    // RTCLM_BOTH opens the standard SIP port 5060, as well as
    // a dynamic port.
    hr = m_pClient->put_ListenForIncomingSessions(RTCLM_BOTH);

The audio and video media types can be added and deleted during a session, so the client must listen for these types of events. See the Processing Real-Time Streaming Session Events section for more on state changes and event processing.

Once the event handler has been registered with the IRTCEventNotification sink, receiving and handling RTC events is fairly simple. When the RTC events are received by the sample application, the application's event handler posts a message to the application's message handler. The OnRTCEvent() function processes all the different types of events received by the application.

OnRTCEvent(UINT message, WPARAM wParam, LPARAM lParam)
{

    // Based on the RTC_EVENT type, query for the 
    // appropriate event interface and call a helper
    // method to handle the event
    switch ( wParam )
    {
       ….
                ….
       ….
        case RTCE_MEDIA:
            {
                IRTCMediaEvent * pEvent = NULL;

                hr = pDisp->QueryInterface( IID_IRTCMediaEvent,
                                            (void **)&pEvent );

                if (SUCCEEDED(hr))
                {
                    OnRTCMediaEvent(pEvent);
                    SAFE_RELEASE(pEvent);
                }
            }
            break;
                ….
                ….
       ….
}

Before you can place a call with RTC, you must create and initialize a communication session. Then you can place the call by entering the IP address of the participant. It is also possible to activate a communication session by entering an e-mail address or a telephone number. However, this functionality requires a SIP registrar server, which is outside the scope of this article. Visit MSDN for more information on SIP registrar server.

RTC does not currently support multi-party video conferencing sessions, so the application must first verify that a video conferencing session is not already underway before it initiates a new session. In this first release, the Windows RTC client supports only multi-party Phone-to-Phone communication sessions, not multi-party audio and video and audio only conferencing.

To make a call to another PC, identify the RTC session type and create a session of that type using the IRTCSession interface. The following code illustrates how to create the session.

HRESULT CAVDConfDlg::MakeCall(RTC_SESSION_TYPE enType, BSTR bstrURI)
{
    ...
 
    // Create the session
    IRTCSession * pSession = NULL;

    hr = m_pClient->CreateSession(enType, NULL, NULL, 0, &pSession);

    // Add the participant to the session
    hr = pSession->AddParticipant(bstrURI, NULL, &m_Participant);

    ...
    return S_OK;
}

Depending on the session type, there can be RTC media events, audio intensity events, instant messaging events, and session state changes.

Handling media events requires getting the media type, then getting the event type and reason, and then delivering that message to the session window. Applications can use the get_MediaType() to receive messages for audio, video, T120, and Real-time Transport Protocol (RTP) events. The sample application shows how to retrieve media events and pass it to the media dialog box to be process.

void CAVDConfDlg::OnRTCMediaEvent(IRTCMediaEvent *pEvent)
{
    ...

    hr = pEvent->get_MediaType(&lMediaType);

    hr = pEvent->get_EventType(&enType);           
    
    hr = pEvent->get_EventReason(&enReason);

    if ((m_AVDlg) && (m_AVDlg.GetState () != RTCSS_IDLE))
    {
        // Deliver the media state to the session window
        m_AVDlg.DeliverMedia(lMediaType, enType, enReason);
    }
}

Intensity events are events that occur when the intensity level changes on either the speaker or microphone devices. The application can use the get_Direction() function to acquire the audio device that has changed. Once the device has been determined, the application can get the current properties of that device and process the changes. The application can show changes to the volume through a control slider or show an audio level meter to the user.

void CAVDConfDlg::OnRTCIntensityEvent(IRTCIntensityEvent *pEvent)
{
    ...

    hr = pEvent->get_Direction(&enDevice);

    hr = pEvent->get_Level(&lLevel);

    hr = pEvent->get_Min(&lMin);

    hr = pEvent->get_Max(&lMax);

    if (m_AVDlg.GetState () != RTCSS_IDLE)
    {
        // Deliver the intensity state to the session window
        m_AVDlg.DeliverIntensity(enDevice, lLevel);
    }
}

Instant messaging information is passed between session participants through the IRTCMessagingEvent interface. When a messaging event occurs, the application must acquire the session and event type and get the participant associated with the session so that the messages can be delivered to the appropriate party. The event handler also processes any changes in the participant session state.

HRESULT CAVDConfDlg::OnRTCMessagingEvent(IRTCMessagingEvent *pEvent)
{
    ...

    hr = pEvent->get_Session(&pSession);

    hr = pEvent->get_EventType(&enType);

    hr = pEvent->get_Participant(&pParticipant);

    if (enType == RTCMSET_MESSAGE)
    {
        hr = pEvent->get_MessageHeader(&bstrContentType);

        hr = pEvent->get_Message(&bstrMessage);

        // Deliver the message to the session window
        if (m_cMessageDlg)
      m_cMessageDlg.DeliverMessage(pParticipant, bstrContentType, 
         bstrMessage);

    }
    else if (enType == RTCMSET_STATUS)
    {
        hr = pEvent->get_UserStatus(&enStatus);

        // Deliver the user status to the session window
        m_cMessageDlg.DeliverUserStatus(pParticipant, enStatus);
    }
   return S_OK;
}

The session state changes follow the same process as the other RTC events. Session state changes can include setting up a new audio/video session or notifying the client of an incoming instant message. The following example shows what happens when a request for a communication session is received; the client is notified of the request by a ring, the request is answered, and a communication session is started.

Void CAVDConfDlg::OnRTCSessionStateChangeEvent(IRTCSessionStateChangeEvent 
   *pEvent)
{
    ...

    hr = pEvent->get_State(&enState);

    hr = pEvent->get_Session(&pSession);

    switch ( enState )
   {
   case RTCSS_INCOMING:
        {
      ... 

            // This event is called when an incoming call occurs
            RTC_SESSION_TYPE enType;

            hr = pSession->get_Type(&enType); 

            // Ring the bell
            m_pClient->PlayRing(RTCRT_PHONE, VARIANT_TRUE);

            // Accept the session
            hr = pSession->Answer();
         }
     }
    ...
}

Starting T120 application sharing is as easy as invoking the StartT120Applet method associated with the IRTCClient interface.

hr = m_pClient->StartT120Applet ( RTCTA_APPSHARING );

To add whiteboard support to the application, invoke the StartT120Applet method using the RTCTA_WHITEBOARD enumeration.

hr = m_pClient->StartT120Applet ( RTCTA_WHITEBOARD );

To close a communication session, all running T120 applications must be shutdown. Then the RTC Client interface calls ShutDown() and completes the process of closing the communication session.

Using the rich client communication features of RTC requires a processor with modest performance. In the following example, a Pentium® III processor running at 1 GHz and a Pentium 4 processor at 2.2 GHz was used to determine processor utilization while running RTC features. The following table describes the percent of processor utilization during the use of RTC features outlined in this article.

1 Pentium 4 processor platform configuration: Intel® Desktop Board D850MV; 256MB PC800 RDRAM, on-board audio, nVidia* GeForce*2 Ultra; Windows XP Professional

2 Pentium III processor platform configuration: Intel Desktop Board VC820; 256MB PC133 SDRAM, nVidia* GeForce*2 Ultra, Creative* Sound Blaster* Live*, Windows XP Professional

Developing communication tools under Windows XP has been simplified using the Real-time Communication client API. Developers can rapidly design, prototype, and develop their communication application. Existing audio and video conferencing applications can benefit from RTC through the addition of its rich communication features. Developing applications with the RTC API also benefits from a unified communication protocol. This increases your application's interoperability with other text messaging and audio/video conferencing applications. Coupling the RTC API with the power of Intel processors and Microsoft Windows XP, you deliver an innovative end-user communication experience.

Intel Developer Services

Media Support in the Microsoft Windows Real-time Communications Client

Microsoft Platform SDK: Real-Time Communications Client