Building Ink Chat

Article
03/02/2007

Sam George
Microsoft Corporation

April 2002

Applies to:
Microsoft® Windows® XP Tablet PC Edition

Summary: Get architecture and code samples for extending Instant Messaging to include the Ink data type introduced in the Windows XP Tablet PC Edition operating system. Learn how Ink interoperates on other Microsoft operating systems. (22 printed pages)

Overview
Three Levels of Ink Support
Detecting the Level of Ink Support
InkEdit and Rich Edit Controls
Architectural Overview
Editing Ink on the Tablet PC OS and a Redist OS
Sending Interoperable Ink Data
Loading Interoperable Ink Data
Manipulating Ink in the Chat Entry
Scrolling Ink in the Chat Entry
OLE Control Implementation Details
Conclusion

Overview

One of the key advances in the Microsoft® Windows® XP Tablet PC Edition operating system is the ability to easily allow developers to add support for pen based input to applications. This input is referred to as Ink and the rich API's included in the Windows XP Tablet PC Edition operating system make it simple to collect, persist, load, manipulate and transform this new data type. This article will cover the following:

The Tablet PC Ink object model used to collect Ink.
The different levels of support that operating systems other than Windows XP Tablet PC Edition have for Ink.
How to display collected Ink on other operating systems.

This article covers Ink interoperability between various operating systems by describing an architecture for building Ink support into Instant Messaging clients, so that Ink can be collected by the Windows XP Tablet PC Edition operating system, broadcast to other operating systems, and rendered on them.

Three Levels of Ink Support

The Windows XP Tablet PC Edition operating system includes support for the Ink data type, but it may not be obvious how other operating systems can work with this new data type.

Windows XP Tablet PC Edition includes a redistributable merge module (MSM) that gives other operating systems the ability to render Ink collected by a Tablet PC. Merge modules include all the information necessary for a Microsoft Windows installer to install a shared component. This merge module can be installed on the following operating systems:

Windows XP Professional
Windows XP Professional Service Pack 1
Windows 2000 Professional Service Pack 2

The redistributable merge module installs the Windows XP Tablet PC Edition Ink COM and .NET libraries, as well as InkPicture and InkEdit controls. It also provides the ability to render and collect Ink from the COM and .NET object models, but provides only the ability to render Ink in the InkPicture and InkEdit controls.

There are three levels of Ink support available to an application that runs on multiple operating systems:

Collection, rendering and recognition support. Available only on the Windows XP Tablet PC Edition.
Partial collection and full rendering support. Available on any supported operating system with the redistributable merge module installed.
No support. Available on any operating system without the redistributable merge module installed (either because it is not supported, or has not been installed).

Detecting the Level of Ink Support

Given the three levels of Ink support, developers who create setups for applications that are installed on non-Tablet PC operating systems can choose whether or not to install the merge module. This choice is, of course, entirely up to the application developer. For example, web-based applications with client side ActiveX code that is installed over the Internet might opt to not install it, while most Win32 applications with a dependency on Ink will probably have it installed.

There are different ways to check for the level of Ink support on a given operating system, based on the parts of the platform an application is using. For example, if an application is using an InkEdit control, it will create an instance of the control and then check the InkEdit's InkMode property.

In the architecture described in this document, we will not be using the InkEdit control but will instead be using the InkOverlay COM object. InkOverlay can collect Ink on both a non-Tablet PC operating system with the merge module installed as well as on the Windows XP Tablet PC Edition. The following code shows how to detect if InkOverlay is present. If InkOverlay is present on the operating system, it can collect and render Ink. Note, however, that the Ink recognition engines are not installed with the merge module.

Detecting the level of Ink support at runtime in C++

// Headers for Tablet PC Automation interfaces
#include <atlbase.h>
#include <msinkaut.h>
#include <msinkaut_i.c>

BOOL               defaultRecoInstalled;
BOOL               InkEnabledOS;
CComPtr<IInkOverlay>      spIInkOverlay;
CComPtr<IInkRecognizerContext> spIInkRecognizerContext;

InkEnabledOS = FALSE;

// Create an InkOverlay object.
HRESULT hr = spIInkOverlay.CoCreateInstance(CLSID_InkOverlay);
if (SUCCEEDED(hr))
{
      // InkOverlay object is present. We can collect and render Ink
   InkEnabledOS = TRUE;
}
   
defaultRecoInstalled = FALSE;

// Create the default recognition context
hr = spIInkRecognizerContext.CoCreateInstance
(CLSID_InkRecognizerContext);
if (SUCCEEDED(hr))
{
      // The default recognizer is installed.  We can reco Ink
   defaultRecoInstalled = TRUE;
}

Detecting the level of Ink support at runtime in C#

using System;
using System.Reflection;
using System.IO;

bool   InkEnabledOS = false;
bool   defaultRecoInstalled = false;
try
{
   // load the Microsoft.Ink assembly using 
   Reflection so the .NET Loader
      // won't throw an exception before this code is reached
  
   // NOTE: you must change the version number to
    match the installed version
   Assembly asm = 
      Assembly.Load("Microsoft.Ink, Version=1.0.XXXX.X, 
Culture=neutral, PublicKeyToken=35ac7114bc3930a2");
   
   // use reflection to instance an InkOverlay
   Type InkOverlayType = asm.GetType("Microsoft.Ink.InkOverlay");
   ConstructorInfo InkOverlayConstructor = 
      InkOverlayType.GetConstructor(new Type[0]);

   // this will throw if the InkOverlay is not installed
   object InkOverlay = InkOverlayConstructor.Invoke(new object[0]);
   
   InkEnabledOS = true;

   // use reflection to instance an RecognizerContext
   Type InkRecoContextType = 
      asm.GetType("Microsoft.Ink.RecognizerContext");

   ConstructorInfo InkRecoContextConstructor = 
      InkRecoContextType.GetConstructor(new Type[0]);

   // this will throw if the RecognizerContext is not installed
   object InkRecoCOntext = 
   InkRecoContextConstructor.Invoke(new object[0]);

   defaultRecoInstalled = true;

}   
catch(FileNotFoundException)
{
   // we're running on a non-tablet os without the redist
   // Microsoft.Ink.dll was not found
}
catch(Exception)
{
   // The InkOverlay or the RecognizerContext is not present
   // check the boolean values to determine which
}

Note that an application can also call ::GetSystemMetrics(SM_TABLETPC) to determine if it is running on the Windows XP Tablet PC Edition operating system; however, a non-zero return value does not guarantee that the InkOverlay or RecognizerContext is installed. The Windows XP Tablet PC Edition SDK states:

"Use the Windows GetSystemMetrics API and pass in SM_TABLETPC as the value of the index (SM_TABLETPC is defined in Winuser.h). The method returns true or nonzero if the Microsoft Windows XP Tablet PC Edition operating system is running; FALSE or zero otherwise."

"Applications should not rely on a true or nonzero value to mean all Tablet PC components are installed and working. Applications should instance the controls needed to determine if they are available."

InkEdit and Rich Edit Controls

All Microsoft operating systems from Windows 95 to Windows XP XP include a version of the Rich Edit control that can be used to display RTF formatted text and embedded objects — a GIF for example. The Windows XP Tablet PC Edition operating system introduces the InkEdit control which is a superset of the Rich Edit 3.0 control. It adds many new features to the Rich Edit 3.0 control, including converting Ink to text and optionally displaying Ink inline with RTF text.

For the first version of the Windows XP Tablet PC Edition operating system, InkEdit control, programmatically adding Ink containers that do not resize with ambient text changes is not supported. Additionally, the InkEdit control does not provide the additional EditingMode support that InkOverlay provides to select and delete Ink. For these two reasons, this architecture encourages the use of Rich Edit control with an InkOverlay attached to it for the chat entry, and Rich Edit control for the chat history.

This introduces one additional level of complexity in that building of a successful chat history that can render both text and Ink (or a GIF that represents the Ink on a non-Tablet OS) also requires building of a simple OLE container. This control will be described in detail later in the article. For now, let us examine the architecture at a high level.

Architectural Overview

Now that we know how to determine the level of Ink support the operating system provides and have a rudimentary idea of the controls and Ink objects needed, let's look at the architecture for extending Instant Messaging to include the Ink data type.

Figure 1. Ink collection and broadcast scenario

Figure 1 shows Ink being collected on the Windows XP Tablet PC Edition operating system and then being broadcast to other clients running on operating systems that have the three different levels of Ink support.

The Windows XP Tablet PC Edition operating system. This operating system is labeled as Tablet PC OS.
A non-Windows XP Tablet PC Edition operating system that supports the Windows XP Tablet PC Edition redistributable and that has it installed. This operating system is labeled as Redist OS.
A non-Windows XP Tablet PC Edition operating system that does not have the Windows XP Tablet PC Edition redistributable installed. This operating system is labeled as Non-Redist OS.

Starting with the first step in the diagram:

Ink is collected in an Instant Messaging client running on the Windows XP Tablet PC Edition operating system. The Instant Messaging client is using a Rich Edit control with an InkOverlay attached to it to collect the Ink, and it saves the Ink for broadcast by specifying the PersistenceFormat.Gif option in the Ink object's Save method. This creates data that can be used to either construct a GIF, or be loaded to load it into an Ink object.
The GIF data with the embedded Ink data is broadcast to the three other clients. How this data is transferred (whether by TCP/IP, HTTP or another means) is outside the scope of this paper.
The GIF data with the embedded Ink data is received by each of the clients and now it is up to them to render it. The client on the far left (labeled Tablet PC OS) is running on a Windows XP Tablet PC Edition operating system and is using a Rich Edit control for the chat history window. It inserts the Ink into an OLE object and then inserts the OLE object into the Rich Edit control. The OLE control owns the decision to display the data as Ink or as a GIF based on the level of Ink support detected. On this first client, the control renders the Ink as Ink.

The client in the middle (labeled Redist OS) is not running on a Windows XP Tablet PC Edition operating system but it does have the redistributable merge module installed, so it too uses the OLE control inserted into a Rich Edit control to render the Ink data.
The GIF data with the embedded Ink data is broadcast to the three other clients. How this data is transferred, whether by TCP/IP, HTTP or another means is outside the scope of this paper.
The GIF data with the embedded Ink data is received by each of the clients and now it is up to the client to render it. The client on the far left (labeled Tablet PC OS) is running on a Windows XP Tablet PC Edition operating system and is using a Rich Edit control for the chat history window. It inserts the Ink into an OLE object and then inserts the OLE object into the Rich Edit control. The OLE control owns the decision to display the data as Ink or as a GIF based on the level of Ink support detected. On this first client, the control renders the Ink as Ink.

The client in the middle (labeled Redist OS) is not running on a Windows XP Tablet PC Edition operating system but it does have the redistributable Merge Module installed, so it too uses the OLE control inserted into a Rich Edit control to render the Ink data.

The client on the far right (labeled Non-Redist OS) is not running on a Windows XP Tablet PC Edition operating system and it does not have the redistributable merge module installed, so it uses the OLE control and inserts it into the Rich Edit control, but the OLE control creates a GIF from the Ink data.

Editing Ink on the Tablet PC OS and a Redist OS

If you notice in Figure 1, the Ink that reads 'Hello' in the chat history windows is labeled 'Ink – Editable' on the Tablet PC OS and Redist OS clients. This means that the Ink has been preserved as Ink and can be annotated upon by dragging it down into the Rich Edit entry control. At a high level, this works by implementing drag and drop in the Rich Edit chat entry control and when an OLE container with Ink is dropped, you copy the Ink the container has into the InkOverlay bound to the Rich Edit chat entry control. This allows an Ink chat user to annotate any former Ink chat from the chat history. This is not possible on the Non-Redist OS, because it has no InkOverlay bound to the Rich Edit chat entry control to copy to.

Sending Interoperable Ink Data

Now that we have looked at a high level overview of the Ink chat architecture, let's examine the details that make it all work. We will start with how to persist Ink in a way that can be reconstructed into either a GIF, or used by the Ink object's Load method to restore the Ink with full fidelity and instance a Strokes collection.

The Windows XP Tablet PC Edition COM and .NET API's include a method that allows Ink to be saved in several formats. For the purpose of interoperable Ink chat, we will be using the GIF format. The GIF format includes an extensibility section in which metadata that is not rendered can be stored. The Windows XP Tablet PC Edition COM and .NET API's leverage this to store the Ink data along with its GIF representation. This is shown in the following code example.

Saving Ink in the interoperable GIF format in C++

// Headers for Tablet PC Automation interfaces
#include <atlbase.h>
#include <msInkaut.h>
#include <msInkaut_i.c>

CComPtr<IInkOverlay>      spIInkOverlay;
CComPtr<IInkDisp>         spIInkDisp;

// Assume the InkOverlay has Ink in it already.
// Get a pointer to the Ink object
spIInkOverlay->get_Ink(&spIInkDisp);

// Call save specifying the 'fortified GIF format'
CComVariant svarInkData;
spIInkDisp->Save(IPF_GIF, IPCM_Default, &svarInkData);

// svarInkData now has a safearray of VT_UI1 (bytes)

Saving Ink in the interoperable GIF format in C#

using Microsoft.Ink;

Microsoft.Ink.InkOverlay InkOverlay;

// Call save specifying the fortified GIF format
byte[] InkData = 
     InkOverlay.Ink.Save(Microsoft.Ink.PersistenceFormat.Gif);
// InkData is now initialized as an array of 
bytes containing the Ink data

Once we have the Ink in this format, we can broadcast it to the listening clients in a single format that can be converted to Ink on clients capable of rendering Ink, and to GIFs on clients that cannot render Ink.

Loading Interoperable Ink Data

Once the data has been transmitted over the wire to the listening clients and turned back in to a VARIANT array of BYTEs in C++ or a byte[] array in C# implementations, it is time to render it. Assuming the client code has already determined the level of Ink support present on the operating system, the following code can be used to render either a GIF or Ink. For this architecture, this code would reside in the OLE control (in C++) that gets inserted into the Rich Edit history control. In C#, a control that inherits from System.Windows.Forms.Panel (or some other lightweight control) and uses an InkOverlay can be used.

Loading Ink in C++

//
// Loads the internal Ink object with fortified GIF 
// data and displays Ink if we're running on a machine 
// capable of displaying it.
//
STDMETHODIMP SampleOleControl::LoadInkGifData(VARIANT Data)
{
   if(Data.vt != (VT_ARRAY | VT_UI1))
   {
      //this isn't the right kind of data
      return E_INVALIDARG;
   }

   if(InkEnabledOS)
   {
            // 
            // Get a pointer to the Ink object
            //
            HRESULT hr = spIInkOverlay->get_Ink(&spIInkDisp);
            if(FAILED(hr))
      {
         return hr;
      }

      //
      // load the Ink data
      //
      hr = spIInkDisp->Load(Data);
      if(FAILED(hr))
      {
         return hr;
      }
   }
   else
   {
      
      //
      // create and load a GIF
      //
      CImage* pImage = new CImage();

      //
      // get an IStream inteface on the Ink gif data
      //
      CComPtr<IStream> spIStream;
      HRESULT hr = CreateStreamFromSafeArray(&Data, &spIStream);
      if(FAILED(hr))
      {
         return hr;
      }
      
      //
      // create a GIF
      //
      hr = pImage ->Load(spIStream);
      if(FAILED(hr))
      {
         return hr;
      }
   }
   return S_OK;
}

//
// Takes a VARIANT array of bytes and 
// returns an IStream interface to it
//
HRESULT SampleOleControl::CreateStreamFromSafeArray
(
     VARIANT *pvSafeArray, IStream **ppIStream
)
{
    HRESULT hr = S_OK;

    *ppIStream = NULL;

    SAFEARRAY *psa = (SAFEARRAY *)(pvSafeArray->parray);
        
    ULONG ulSize = psa->rgsabound[0].cElements;
    if (ulSize == 0)
        return E_FAIL;

    HGLOBAL hMem = ::GlobalAlloc(GHND, ulSize);
    if (hMem == NULL)
        return E_OUTOFMEMORY;

    byte *pb;
    hr = SafeArrayAccessData(psa, (void **)&pb);
    if (FAILED(hr))
        return hr;

    if (! pb)
        return E_FAIL;


    // Copy Ink data into global mem block
    byte *pbStm = (byte *)::GlobalLock(hMem);
    ::memcpy(pbStm, pb, ulSize);
    ::GlobalUnlock(hMem);
    ::SafeArrayUnaccessData(psa);
    
    hr = ::CreateStreamOnHGlobal(hMem, TRUE, ppIStream);
    
    return hr;
}

Detecting the level of Ink support at runtime in C#

private LoadInkGifData(byte[] InkData)
{
   if(InkEnabledOS)
   {
      //
      // just load the Ink into the Ink overlay
      //
      InkOverlay.Ink.Load(InkData);
   }
   else
   {
      //
      // get a memory stream 
      //
      MemoryStream stream = new MemoryStream(InkData);
      stream.Position = 0;

      //
      // create an image with the data
      //
      Image image = Bitmap.FromStream(stream);
      stream.Close();
   }
}

Manipulating Ink in the Chat Entry

Now that we have Ink being transmitted across the wire to listening clients, and have clients that interpret that data as Ink or a GIF depending on the OS Ink support, let's look at what it takes to modify the Ink in the Rich Edit chat entry control. Users will find this useful to delete Ink that they had added, or to select it and move it around. They can also customize the color, width and transparency of the Ink they are chatting with. We will examine how to:

Select Ink using InkOverlay's built in support for selection
Delete Ink using InkOverlay's built in support for deletion
Change the appearance of Ink

Thanks to the factoring of the Tablet PC COM and .NET object model, doing this is very simple. The following code demonstrates how to change the InkOverlay to selection or deletion mode:

Moving the InkOverlay to Selection or Deletion mode in C++

// Headers for Tablet PC Automation interfaces
#include <atlbase.h>
#include <msInkaut.h>
#include <msInkaut_i.c>

//
// disable the overlay and change to delete mode
//
spIInkOverlay->put_Enabled(VARIANT_FALSE);

spIInkOverlay->put_EditingMode(IOEM_Delete);
// NOTE: to change to selection, just comment out the line
// above and use this:
// spIInkOverlay->put_EditingMode(IOEM_Select);

//
// specify stroke deleting
//
spIInkOverlay->put_EraserMode(IOERM_StrokeErase);

//
// re-enable the overlay, we're done!
//
spIInkOverlay->put_Enabled(VARIANT_TRUE);

Moving the InkOverlay to Selection or Deletion mode in C#

using Microsoft.Ink;

//
// disable the overlay and change to delete mode
//      
InkOverlay.Enabled = false;
InkOverlay.EditingMode = InkOverlayEditingMode.Delete;

//
// specify stroke deleting
//
InkOverlay.EraserMode = InkOverlayEraserMode.StrokeErase;
// NOTE: to change to selection, just comment out the line
// above and use this:
// InkOverlay.EditingMode = InkOverlayEditingMode.Select;

//
// re-enable the overlay, we're done!
//
InkOverlay.Enabled = true;

Next, we will examine how to change from opaque, narrow Ink to thick, translucent Ink:

Changing Ink DrawingAttributes in C++

// Headers for Tablet PC Automation interfaces
#include <atlbase.h>
#include <msInkaut.h>
#include <msInkaut_i.c>

//
// Modify default drawing attributes
//
CComPtr<IInkDrawingAttributes> spIInkDrawAttrs = NULL;
hr = spIInkOverlay->get_DefaultDrawingAttributes(&spIInkDrawAttrs);
if (SUCCEEDED(hr))
{
    //
    // Modify the width and transparency
    //
    spIInkDrawAttrs->put_Width((float)200);
    spIInkDrawAttrs->put_Transparency(120);
    
    //
    // Set the new drawing attributes
    //    
    spIInkOverlay->putref_DefaultDrawingAttributes(spIInkDrawAttrs);
}

Changing Ink DrawingAttributes in C#

using Microsoft.Ink;

//
// Set the new drawing attributes
//    

InkOverlay.DefaultDrawingAttributes.Width = 200F;
InkOverlay.DefaultDrawingAttributes.Transparency = 120;

Scrolling Ink in the Chat Entry

To round out the text entry window as a functional editor of Ink data, let's take a look at what is involved in scrolling the InkOverlay to allow creating and editing Ink that is larger than the window. When scrolling any window, you must ensure that any painting code takes into account the scrollbar position. The InkOverlay handles all the repainting, so our task is to update the ViewTransform associated with the InkOverlay's renderer every time the scrollbar position changes. One thing to note when calculating how much to move the Ink is that the PixelToInkSpace function takes into account any transforms currently applied to the renderer. To cancel this out, we'll translate (0,0) at the same time and take the difference.

Scrolling an InkOverlay in C++

// window procedure
case WM_HSCROLL:
   // update ScrollInfo struct and SetScrollInfo()
   // ...

   //  Retrieve the scroll info
   GetScrollInfo(hWnd, SB_HORZ, &si);

   //   Convert to ink space.  Since we are updating the view 
   //   transform of the renderer, the origin of the ink 
coordinates 
   //   may have moved relative to the screen.   
   xScroll = -si.nPos;

   g_pIInkRenderer->PixelToInkSpace(hdc, &xScroll, &yScroll);
   g_pIInkRenderer->PixelToInkSpac(hdc, &xOrigin, &yOrigin);

   //   Apply a translation transform to the ink
   g_pIInkTransform->put_eDx(xScroll-xOrigin);
   g_pIInkRenderer->SetViewTransform(g_pIInkTransform);

   // We've moved the ink.  Make sure it gets repainted.
   InvalidateRgn(hWnd, NULL, true);
   UpdateWindow(hWnd);
   break;

Scrolling an InkOverlay in C#

void hScrollBar1_OnScroll(object sender, ScrollEventArgs args)
{
   xCurrentScroll = args.NewValue;
   
   //   Convert to ink space.  Since we are updating the view 
   //   transform of the renderer, the origin of the ink 
coordinates 
   //   may have moved relative to the screen.   
   Point scrollAmount = new Point(-xCurrentScroll, -yCurrentScroll);
   Point origin = new Point(0, 0);

   Graphics g = CreateGraphics();
   inkOverlay.Renderer.PixelToInkSpace(g, ref scrollAmount);
   inkOverlay.Renderer.PixelToInkSpace(g, ref origin);

   //   Create a translation transform based on the current x and y 
   //   scroll positions
   Matrix m = new Matrix();
   m.Translate(scrollAmount.X-origin.X, scrollAmount.Y - origin.Y);
   inkOverlay.Renderer.SetViewTransform(m);

   // We've moved the ink.  Make sure it gets repainted.
   Invalidate();
}

OLE Control Implementation Details

The Rich Edit control can easily support everything we need for the implementation of the chat history window. Let's take a closer look at what is needed to get Ink strokes to render in the Rich Edit control. For the purposes of this article, the only functionality we really need is the ability to append objects to the chat history window and to copy them from the history window into the entry window for editing. The ATL "Minimal Control" wizard will allow us to create a fully functional OLE control with next to no effort, so for simplicity we will start from there.

Rendering Ink in an OLE Control

Since a basic control does not have a window handle, we will forgo the InkOverlay used in the chat entry and use the InkRenderer object's Draw method to draw the strokes in the control's OnDraw method. One thing to be mindful of is that windowless OLE controls are often asked to draw themselves on their parent control's device context. To ensure that the InkRenderer paints the strokes in the correct position, we may need to move all the strokes so they show up in the right place. The InkRenderer's ViewTransform object provides the desired functionality.

if(m_inkEnabledOS)
{
   //   The rect we have here will not necessarily be relative to 
the
   //   control itself.  It is instead the rect of our control in 
the
   //   containing control's coordinate space.  We need to apply a 
   //   view translation to the ink to get it to render within this
   //   rectangle.
   CComPtr<IInkStrokes> spIStrokes;   
   // grab the top left of our bounding rectangle
   long x, y;      
      
   x = di.prcBounds->left;
   y = di.prcBounds->top;

   m_spIInkDisp->get_Strokes(&spIStrokes);
            
   // convert to HIMETRIC
   m_spIInkRenderer->PixelToInkSpace((long)di.hdcDraw, &x, &y);      

   // grab the existing transform
   m_spIInkRenderer->GetViewTransform(m_spIInkTransform);

   // set the translation
   m_spIInkTransform->Translate((float)x, (float)y);

   // apply the transform
   m_spIInkRenderer->SetViewTransform(m_spIInkTransform);
      
   m_spIInkRenderer->Draw((long)(di.hdcDraw), spIStrokes);
}

Supporting Drag/Drop in an OLE Control

When you drag or copy an OLE object from the Rich Edit control, the GetClipboardData() method of our object is called to retrieve the clipboard representation of the data. Since the only data in this object is the ink data, we let the Ink object do all the work with the Ink object's ClipboardCopy method.

STDMETHODIMP SampleOLEControl::GetClipboardData(DWORD dwReserved,
                    IDataObject **ppDataObject)
{
   HRESULT hr;
   // Let the Ink object do all the work.
   hr = m_spIInkDisp->ClipboardCopy(NULL, ICF_Default, 
                     ICB_ExtractOnly, ppDataObject);

   return hr;
}

Using Your OLE Control with Rich Edit in C++

Now that we have the control, we need to be able to insert it into the chat history window and to drag it from the chat history to the chat entry for editing. Microsoft Knowledge Base article 141549 provides general sample code for using OLE controls in the Rich Edit control.

Dropping/Pasting the OLE Object into Rich Edit in C++

For the chat entry window, we want to intercept the pasting of Ink data into the Rich Edit control and instead paste it into the attached InkOverlay. All extensions to the default behavior of the Rich Edit control are done by implementing a class which inherits the IRichEditOleCallback interface. For this topic, we are only interested in the QueryAcceptData() callback to control paste operations, so we'll just return E_NOTIMPL for everything else.

//
//  FUNCTION:   CRichEditOleCallback::QueryAcceptData
//
//  PURPOSE:    Called to ask whether the data that is being pasted or
//          dragged should be accepted.  We'll intercept anything
//          Ink related.
//
HRESULT CRichEditOleCallback::QueryAcceptData(LPDATAOBJECT  pDataObj ,
                       CLIPFORMAT* pcfFormat,
                       DWORD /* reco */,
                       BOOL  fReally ,
                       HGLOBAL /* hMetaPict */)
{
   HRESULT hr;
   CComPtr<IInkStrokes> spIStrokes;

   VARIANT_BOOL canPaste;
   // check whether the ink overlay's ink object 
   can handle the pasting
   m_spIInkDisp->CanPaste(pDataObj, &canPaste);
   if (canPaste == VARIANT_FALSE)
      // tell RichEdit we don't want to paste this
      return E_FAIL;
   
   if (!fReally)
      // Tell the RichEdit that we CAN paste it, but didn't
      return S_FALSE;

   // try to let the ink overlay handle the pasting
   hr = m_spIInkDisp->ClipboardPaste(0, 0, pDataObject, 
&spIStrokes);
   if(SUCCEEDED(hr))
      // tell RichEdit that we handled the paste
      return S_FALSE;

   // nothing we can do, let RichEdit handle the paste
   return S_OK;
   
}

One thing to note is that the Ink copy and paste operations used above do not preserve the location of the strokes relative to the ink surface. Any ink dragged from the chat history to the chat entry window would appear in the upper left corner of the window. If this is not the desired behavior, you can attach the relative position to the ink object by means of an extended property before performing the copy operation. In the paste code above, you could then retrieve the coordinates and move the pasted strokes accordingly.

Conclusion

The Tablet PC API provides a rich interface for editing, persisting, and transforming pen input in an application. The redistributable merge module allows the collection and rendering of ink on a machine that is not Tablet PC, and the ability to persist as an enhanced gif means that Ink can be displayed on any machine without losing the ability to edit it in the future.

Share via

Building Ink Chat

Contents:

Overview

Three Levels of Ink Support

Detecting the Level of Ink Support

Detecting the level of Ink support at runtime in C++

Detecting the level of Ink support at runtime in C#

InkEdit and Rich Edit Controls

Architectural Overview

Editing Ink on the Tablet PC OS and a Redist OS

Sending Interoperable Ink Data

Saving Ink in the interoperable GIF format in C++

Saving Ink in the interoperable GIF format in C#

Loading Interoperable Ink Data

Loading Ink in C++

Detecting the level of Ink support at runtime in C#

Manipulating Ink in the Chat Entry

Moving the InkOverlay to Selection or Deletion mode in C++

Moving the InkOverlay to Selection or Deletion mode in C#

Changing Ink DrawingAttributes in C++

Changing Ink DrawingAttributes in C#

Scrolling Ink in the Chat Entry

Scrolling an InkOverlay in C++

Scrolling an InkOverlay in C#

OLE Control Implementation Details

Rendering Ink in an OLE Control

Supporting Drag/Drop in an OLE Control

Using Your OLE Control with Rich Edit in C++

Dropping/Pasting the OLE Object into Rich Edit in C++

Conclusion

Additional resources