Creating Tablet PC Applications with Microsoft Visual FoxPro 

 

Mike Stewart
Visual FoxPro Team
Microsoft Corporation

March 2004

Summary:   Learn how to use the Microsoft® Tablet PC Platform SDK to capture digital ink, recognize handwriting, handle events raised by Tablet PC controls, and persist ink to files in Microsoft Visual FoxPro® applications. (14 pages)

Contents

Introduction
System Requirements
Design Considerations
Collecting Ink
Selecting and Deleting Ink
Persisting and Loading Ink
Conclusion

Introduction

The controls and APIs necessary to utilize many of the ink features of the Microsoft Windows® XP Tablet PC Edition are included in the Tablet PC Platform SDK. The Tablet PC Platform SDK can be downloaded at https://www.microsoft.com/windowsxp/tabletpc/developers/default.asp. The Tablet PC Platform SDK exposes ink objects in two ways: through Component Object Model (COM) Automation and the Microsoft .NET Framework common language runtime (CLR) via managed APIs. Because Visual FoxPro (VFP) does not compile to the CLR, this paper will focus on COM Automation APIs.

The Tablet PC looks at ink in a way different from previous devices that capture ink. With previous devices, such as the Apple Newton or Microsoft Windows CE/Pocket PC devices, the emphasis was on recognizing ink as handwriting. With the Tablet PC, the philosophy changes to looking at digital ink as data, with a de-emphasis on converting it to text. For instance, you can take notes using the Windows Journal application included with the Tablet PC and Microsoft's OneNote™, and you do not need to convert those notes to text. That being said, the recognition capabilities of the Tablet PC are very good, with a high rate of recognition for those times when you need to convert ink to text.

The objects of interest to VFP developers are the InkEdit, InkPicture, and InkOverlay objects. The InkEdit control is similar to a VFP EditBox, the difference being that the InkEdit control can capture and recognize digital ink. The InkPicture control is exactly what you think it would be: a picture that can be placed in the control, and users can ink on top of the picture. The InkOverlay control invisibly "overlays" a window, and can render, capture, and recognize ink.

System Requirements

• Microsoft Tablet PC Platform SDK
• Microsoft Windows XP   You can use the Tablet PC Platform SDK on Microsoft Windows XP, but Microsoft Windows XP Tablet PC Edition is highly recommended.
• Microsoft Visual FoxPro 8.0 or above   It is possible to utilize nearly all of the functionality of the Tablet PC Platform SDK with earlier versions of VFP. However, several bugs related to Microsoft Windows XP Tablet PC Edition were fixed in VFP 8.0.
• **Digitizer   **USB digitizers are available from several manufacturers, such as Wacom (http://www.wacom.com), or applications can be developed using the built-in digitizer of a Tablet PC.

Only on a Tablet PC?

The Tablet PC Platform SDK requires only Microsoft Windows XP. However, the functionality of controls is significantly enhanced when using Windows XP Tablet PC Edition. There is no recognition engine and some controls do not capture ink if the SDK is not installed on a Tablet PC. Therefore, it is recommended that you have access to at least one Tablet PC for testing and debugging.

Ideally, you want to code and debug your applications on a Tablet PC. But who wants to write code with a pen? Even the convertible models sport keyboards that are not full-sized. Several options are available to work around this. One solution is to use a USB keyboard and mouse and connect a monitor to the Tablet PC's VGA port (or use a docking station if one is available for your Tablet PC). Another solution is to use Windows XP's Remote Desktop feature to access the Tablet PC with a desktop computer.

Design Considerations

A frequently asked question is, "How do I know if my application is running on a Tablet PC?" This is easily done by calling the Microsoft Win32® GetSystemMetrics function with a parameter of SM_TABLETPC (86). A non-zero return value indicates that the Windows XP Tablet PC Edition is running; a return value of zero indicates it is not.

#define SM_TABLETPC 86
Declare Integer GetSystemMetrics in Win32API Integer
retVal = GetSystemMetrics(SM_TABLETPC)

If retVal <> 0
    Wait Window "Running on a Tablet PC"
Else
    Wait Window "*Not* running on a Tablet PC"
Endif

A non-zero return value does not indicate whether all Tablet PC components installed and working, however. To determine if a component is installed, attempt to create an instance of the component and check for errors during creation.

For example:

Try
    oInkEdit = Newobject("InkEd.InkEdit.2")
Catch To oException
    Wait Window "InkEdit control is not installed!"
Endtry

A Tablet PC can have two modes of screen display: portrait and landscape. Landscape is used most often when the Tablet PC is docked or used as a standard laptop, and portrait is used when using it as a tablet.

Figure 1. Landscape mode

Figure 2. Portrait mode

Your application may need to determine whether the user's Tablet PC is in portrait or landscape mode so that screen elements can be resized appropriately. Using the VFP SYSMETRIC() function with parameters of 1 or 2, you can determine the width and height of the display. If the width is less than height, then the Tablet PC is in portrait mode.

There are other considerations to take into account when designing your application. Refer to the Tablet PC Platform SDK Help file under "Planning Your Application" for more information.

Collecting Ink

The simplest way to capture pen input is to use the InkCollector object provided by the Tablet PC Platform SDK. It's extremely easy to use, requiring just three simple steps to set up and activate. The InkCollector is attached to a form via the form's window handle (hWnd), and it allows pen input on the entire form.

To enable ink collection on a form

1. Create a new object using the InkCollector.
2. Attach the InkCollector to a form using the form's window handles.
3. Activate the InkCollector by setting its Enabled property to true (.T.)

Run the following code and start inking!

Local oInkCollector as MSINKAUT.inkcollector.1
Local oForm1 as Form
oForm1 = NewObject("form")

*-- Create a new InkCollector, and set the 
*-- window handle to be the form on which
*-- we want to work with the InkCollector
oInkCollector = NewObject("msinkaut.inkcollector.1")
oInkCollector.hWnd = oForm1.HWnd

*-- That's it, we're done setting it up. Enable
*-- the InkCollector so we can start using it.
oInkCollector.Enabled = .t.

*-- InkCollector is turned on, show the form
oForm1.Show
Read Events

Note that the Tablet PC-specific part of the code is here:

oInkCollector = NewObject("msinkaut.inkcollector.1")
oInkCollector.hWnd = oForm1.HWnd

If you've used the Windows Journal, you probably expect a Tablet PC application to allow you to select, edit, and delete ink. Using the InkCollector object, you can draw on your form, recognize what was written, save and load ink. However, you cannot use the InkCollector method in your VFP application to select, edit, or deleted ink. To select, edit, or delete ink, use the InkOverlay class. The InkOverlay class is a superset of the InkCollector, so it will have all of the functionality of the InkCollector.

The InkOverlay Class

Using the InkOverlay class is just as easy as using the InkCollector. But, because of its added functionality, there's a lot more to cover with this class. Let's start with a simple form that implements the InkOverlay and adds the ability to recognize what the user writes. Note that to click the Recognize button, you'll need to press Alt-R. We'll get to why in a minute.

*-------------------------------------
*--    Program:   InkOl1.prg
*--
*--    Author:    Mike Stewart
*--    Comments:  Simple InkOverlay demo using an
*--               InkOverlay class with a recognizer
*--               button.
*-------------------------------------
PUBLIC oform1

oform1=NEWOBJECT("InkOverlayDemo1")
oform1.Show
RETURN

DEFINE CLASS InkOverlayDemo1 AS form

    Top = 0
    Left = 0
    Height = 416
    Width = 659
    DoCreate = .T.
    Caption = "Form1"
    Name = "Form1"

    ADD OBJECT inkRecognize AS Commandbutton WITH ;
        Top = 368, ;
        Left = 20, ;
        Caption = "\<Recognize ", ;
        Name = "Inkbutton1"

    ADD OBJECT edtrecognized AS editbox WITH ;
        Height = 121, ;
        Left = 20, ;
        Top = 236, ;
        Width = 589, ;
        Name = "edtRecognized"

    PROCEDURE Init
        PUBLIC oink As msinkaut.inkoverlay.1
        oInk = NEWOBJECT("msinkaut.inkoverlay.1")

        WITH oInk 
            *-- Point it to the window for which you
            *-- want to capture ink. Because VFP does
            *-- not have hWnds for individual controls,
            *-- this can only be done at the form level.
            .hwnd = thisform.HWnd

            *-- The Attachmode property
            *-- determines whether the 
            *-- InkOverlay sits in front
            *-- or behind the controls on
            *-- the form.
            .AttachMode = 0  && IOAM_Behind 

            *-- Set everything before enabling,
            *-- or else error occurs.
            .enabled = 1
            
*-- Set to collect both ink and gestures
        .CollectionMode = 2  && ICM_InkAndGesture 
        ENDWITH
    ENDPROC

    PROCEDURE inkRecognize.Click
    thisform.edtRecognized.value = ;
    oInk.ink.strokes.tostring
    ENDPROC
ENDDEFINE

Figure 3. InkOverlay form after clicking Recognize button

With only 38 lines of code, we now have a working form that captures ink and has the ability to recognize it and put it into a VFP control that can be bound to a field in a table.

If you run the code above, you might notice that it was hard to click the Recognize button using the pen or the mouse. That's because when you attach the InkOverlay object to a form, it covers the entire form, just like the InkCollector class. Obviously, that is not going to work for most applications because forms will most likely have textboxes and other controls to collect user input for binding to tables.

There's a simple solution for this problem, and that is to define a rectangle for the InkOverlay. The Tablet PC Platform SDK provides an InkRectangle object and the Inkoverlay.SetWindowInputRectangle method for just such a task. The changes to previous code are minor. First, create a SetRectangle method with the following code.

Procedure SetRectangle
LOCAL oInkRectangle as MSINKAUT.inkrectangle
oInkrectangle = NEWOBJECT("msinkaut.inkrectangle")

*-- Set the bottom of the rectangle
*-- to be the top of the editbox.
*-- Doing this means the InkOverlay
*-- will not overlap our controls.
oInkRectangle.Bottom = thisform.edtRecognized.Top

*-- The rest of the dimensions of the
*-- rectangle will match those of the 
*-- form.
oinkrectangle.Top = thisform.Top
oinkrectangle.Left = thisform.Left
oinkrectangle.Right = thisform.Width - thisform.Left

*-- Call the method by passing our InkRectangle
*-- and we're done.
ThisForm.InkOverlay.SetWindowInputRectangle(oInkRectangle)

Next, call the method from the form's Init method:

.AttachMode = 1  && IOAM_InFront 
this.setrectangle

Run the code, and you'll see that the pen input area is now restricted to the area above the editbox.

Another solution is to set the InkOverlay.AttachMode property to zero (IOAM_Behind). This places the InkOverlay behind the controls on the form so you can use the controls as you would normally. One problem with this approach is that the user can ink between the controls, but not on them. A well-defined inking surface via the SetInputWindowRectangle method might be a better approach for your users.

Selecting and Deleting Ink

The following code shows how to add a combobox to select between three cursors:

• Ink cursor – used for drawing and writing.
• Selection lasso – used to select ink.
• Eraser – used to delete ink.

First, add a combobox to the class definition:

ADD OBJECT inkPenType AS combobox WITH ;
    RowSourceType = 1, ;
    RowSource = "Ink,Select,Delete", ;
    Height = 24, ;
    Left = 36, ;
    Top = 428, ;
    Width = 168, ;
    Name = "inkPenType"

Now add the code for the combobox's InteractiveChange event to set the EditingMode property based on the user's selection:

PROCEDURE inkPenType.InteractiveChange
    DO Case
    CASE this.Value = "Ink"
        thisform.inkoverlay.EditingMode = 0  && IOEM_Ink 
    CASE this.Value = "Select"
        thisform.inkoverlay.EditingMode = 2  && IOEM_Select 
    CASE this.Value = "Delete"
        thisform.inkoverlay.EditingMode = 1  && IOEM_Delete
        *-- Lastly, set the EraserMode based on 
        *-- whether entire strokes should be erased 
        *-- when the pen touches the screen, 
        *-- or if pixels should be erased:
        *-- Set EraserMode to 0 to erase strokes,
        *--                or 1 to erase pixels
        *-- thisform.inkoverlay.EraserMode = 0
        thisform.Inkoverlay.EraserMode = 1
    ENDCASE
ENDPROC

By selecting the appropriate option in the combobox, the user can now change how the pen behaves.

Persisting and Loading Ink

In some circumstances, a user may want to save ink and load it at a later time.

First, let's add a couple of command buttons to the form's class definition, one to save ink and one to load ink onto the form:

ADD OBJECT cmdSave AS commandbutton WITH ;
    Top = 360, ;
    Left = 357, ;
    Height = 27, ;
    Width = 84, ;
    Caption = "\<Save", ;
    Name = "cmdSave"

ADD OBJECT cmdLoad AS commandbutton WITH ;
    Top = 360, ;
    Left = 456, ;
    Height = 27, ;
    Width = 84, ;
    Caption = "L\<oad", ;
    Name = "cmdLoad"

Now add code to the Save button's click event:

PROCEDURE cmdSave.Click
    lsInk = ThisForm.inkoverlay.Ink.Save()
    STRTOFILE(lsInk, GETFILE("isf"))
ENDPROC

The InkOverlay's Ink object uses its Save method to return a byte array representing the ink on the InkOverlay. Using FoxPro's STRTOFILE() function, this byte array (which FoxPro handles as a string) can then be saved to disk.

To retrieve the saved ink object using the Load button, use the Ink.Load method:

PROCEDURE cmdLoad.Click
    lsInk = CREATEBINARY(FILETOSTR(GETFILE("isf")))
    thisform.inkoverlay.Enabled = 0
    thisform.inkoverlay.ink.Load(lsInk)
    thisform.inkoverlay.Enabled = 1

    *--Tell control to redraw, or changes will not be made visible.
    *-- Use the InkRectangle object we 
    *-- created during form Init.
    ThisForm.inkoverlay.Draw(ThisForm.InkRectangle)
ENDPROC

Note the use of the CREATEBINARY() function when pulling the ink file into a FoxPro variable. This is necessary to get it in a format that the InkOverlay object will accept. Also note that the InkOverlay must first be disabled before loading the ink, otherwise an error occurs.

Try this: Save some ink to a file, exit the form, rerun it and load the ink you created. Now click the Recognize button. The InkOverlay recognizes the ink! That's because the InkOverlay saves not just the position of the pixels so it can redraw it later, it also saves the metadata needed for the recognizer to recognize it.

What does this metadata contain? The file's metadata contains information such as a date and time stamp for each stroke, as well as a unique identifier for each stroke.

InkEdit Control

The InkEdit control is a superset of the RichText control. The InkEdit control is designed to provide an easy way to capture, display, and recognize ink. To implement an InkEdit control on a VFP form, drop an ActiveX Control from the Form Controls toolbar. When prompted, select the InkEdit control.

Figure 4. Adding the Microsoft InkEdit control to a form

If you now run the form, the InkEdit control will accept ink and recognize it after the default of 2000 milliseconds. You can change the recognition delay with the RecognitionTimeout property of the InkEdit control. The InkEdit control will also already have the default Text property of "Olecontrol1", but you can easily change that later.

Figure 5. Before recognition

Figure 6. After recognition

Because the InkEdit control is based on the RichText control, users can type in the InkEdit control as they would with a standard textbox or editbox. Using the InkEdit control, you can create applications that give the user the option of typing data or inking it in.

InkPicture Control

The InkPicture control combines most of the attributes of the InkOverlay control with the ability to put ink over image files. The following code demonstrates how to create a form with an InkPicture control, and lets the user choose an image to load. The InkPicture control can also be dropped onto a form in the same way as the InkEdit control. In Figure 4, it is the control just below the InkEdit control.

PUBLIC ofrminkpicture

ofrminkpicture=NEWOBJECT("frminkpicture")
ofrminkpicture.Show
RETURN

DEFINE CLASS frminkpicture AS form
    Top = 0
    Left = 0
    Height = 460
    Width = 469
    DoCreate = .T.
    Caption = "InkPicture Form"
    AllowOutput = .F.
    Name = "frmInkPicture"

    ADD OBJECT olecontrol1 AS olecontrol WITH ;
        Top = 12, ;
        Left = 7, ;
        Height = 444, ;
        Width = 360, ;
        OleClass = "msinkaut.inkpicture.1"
        OleLCID = "1033"
        Name = "Olecontrol1"

    ADD OBJECT cmdload AS commandbutton WITH ;
        Top = 24, ;
        Left = 374, ;
        Height = 27, ;
        Width = 84, ;
        Caption = "\<Load Image", ;
        Name = "cmdLoad"

    ADD OBJECT cmdcolor AS commandbutton WITH ;
        Top = 60, ;
        Left = 374, ;
        Height = 27, ;
        Width = 84, ;
        Caption = "\<Color", ;
        Name = "cmdColor"


    PROCEDURE cmdLoad.Click
        WITH thisform.olecontrol1
            Try
                .Picture =; 
LoadPicture;
(GetFile("bmp;jpg;jpeg;gif",;
"Graphics files"))
            Catch To oException
                MessageBox("No file selected")
            Endtry
        ENDWITH 
    ENDPROC

    PROCEDURE cmdcolor.Click
        thisform.olecontrol1.DefaultDrawingAttributes.Color =; 
GetColor()
    ENDPROC
ENDDEFINE

The OleControl is defined in this piece of code:

OleClass = "msinkaut.inkpicture.1"
OleLCID = "1033"

Once the control is defined, the form gives the user the ability to load a graphics file into the Load button's Click method using only one line of code. Of course, you'll want to include error handling code as well.

PROCEDURE cmdLoad.Click
WITH thisform.olecontrol1
Try
        .Picture =; 
LoadPicture;
(GetFile("bmp;jpg;jpeg;gif",;
"Graphics files"))
    Catch To oException
        MessageBox("No file selected")
    Endtry
ENDWITH 
ENDPROC

You can also add a Color button to allow users to change the color of the ink. This capability is also available in the InkOverlay control.

PROCEDURE cmdcolor.Click
thisform.olecontrol1.DefaultDrawingAttributes.Color =; 
GetColor()
ENDPROC

The DefaultDrawingAttributes collection offers a variety of ways to change the way that ink is drawn. The one member of the collection in this example is Color, and the code simply calls FoxPro's GETCOLOR() function to obtain the user's choice of ink color.

Like the InkOverlay control, the InkPicture control can recognize ink, save it, and load it. The code for doing so with the InkOverlay control will port directly over to the InkPicture control.

Conclusion

These are only some of the things you can do with the Tablet PC SDK and Visual FoxPro on the Tablet PC, designed to get you on your way to writing Tablet PC applications. Refer to the Tablet PC SDK Help file for a complete list of the properties, events, and methods for the controls discussed here.

© Microsoft Corporation. All rights reserved.