Using the InkPicture Stroke and NewInAirPackets and NewPackets Events

Leszynski Group Inc.

July 2003

Applies to:
   Microsoft® Windows® XP Tablet PC Edition

Summary: This article discusses using ink controls for the Microsoft Windows XP Tablet PC Edition operating system. We will discuss the InkPicture control's Stroke event as well as the NewInAirPackets and NewPackets events, all of which are included as part of the Tablet PC SDK. Examples of these controls include counting user strokes by means of the Stroke event and tracking pen placement over the screen by means of the NewInAirPackets event.

This article is intended for developers creating ink-based applications. Code samples were created using Microsoft Visual Basic .NET and C#, and the Microsoft Tablet PC Platform SDK (Beta) Build 2149. (18 printed pages)

Contents

Using the InkPicture Stroke Event
Using the NewInAirPackets and NewPackets Events
Conclusion

Using the InkPicture Stroke Event

The InkPicture control provides a rich event model that developers can use to enhance and process ink. The InkPicture control is derived from the PictureBox base class and provides the ability to capture ink as well as perform handwriting recognition, although there is no text display area within an InkPicture control.

In the user interface, the InkPicture control offers a space in which the user can annotate. This control is useful for capturing signatures, drawings, markups, or other annotation that need not be recognized as text. The control can also contain an image over which the user can draw, which is useful in cases where a form may be the underlying image. The accepted image file formats for InkPicture include the following: .jpg, .bmp, .png, or .gif.

Adding InkPicture to Visual Studio

To use the InkPicture control, add it to your Visual Studio® .NET Toolbox.

1. Right-click on the Toolbox.
2. Select Customize Toolbox… from the popup menu.
3. Click on the .NET Framework Components tab.
4. Check InkPicture and then click OK.

You can also add this control by referencing and creating it in code. However, adding it to your Toolbox is the quickest and most efficient means.

Add InkPicture to your Form

Add an instance of the InkPicture to your form by selecting the InkPicture control in your Toolbox. Then draw a rectangle region on your form.

Note   The ink region is contained within the bounds of the control. If the control is too small, the user may not have enough room to write and recognition may not work effectively. We recommend that you make the control large enough for users to write easily several words inside the control boundaries.

You do not need to place a picture within the control. To create a region where users can draw pictures or annotate, you can leave the picture property empty.

Enabling InkPicture

By default, the InkPicture control will capture and display Ink. You do not have to set any properties to activate this control at run time.

Employing the Stroke Event

The Stroke event handler has two parameters: (i) Sender, which specifies the InkPicture object that fired the Stroke event; and (ii) an InkCollectorStrokeEventArgs object, which has several useful properties, including Cursor, Stroke and Cancel.

For the purposes of this paper, we will limit the discussion to the Stroke and Cancel properties in the context of inserting ink. However, it is important to note that the Stroke event is also fired during the erase and the select editing modes. Therefore, be sure you know which editing mode is set when interpreting the Stroke event.

A stroke is defined as a pen down, pen movement, pen up sequence. During a stroke, packets containing points are collected in the Stroke object. Once a stroke is complete (i.e. the pen is up), the Stroke event is fired.

One drawback of using the Stroke event is that it is fired upon completion of a stroke. In cases when you need to know when a stroke has begun, you can use instead the NewPackets event, which is discussed later in this paper.

Stroke

The Stroke event provides a means for an application to detect when the user begins and ends a stroke. For example, the beginning and end points of a stroke would be critical in an ink clock control. This type of control might draw the hands of the clock based on the pen down and pen up positions and disregards the intermediary pen movement. The stroke is then redrawn as a straight line between these two points.

Tracking the end point of a stroke can also help identify when the user has drawn out of bounds. In such cases, the ink can be rendered in a different color or erased to indicate to the user they have drawn outside a predefined region.

You can use the Stroke event for counting strokes, changing the ink attributes of a stroke or for smoothing a stroke once it's completed. An example of anti-aliasing and curving a stroke is available in the Using InkEdit, InkPicture, and Enhancing the Appearance of Ink whitepaper.

The Stroke event can also be used to place strokes into Strokes or CustomStrokes collections. Grouping strokes allows you to manipulate the ink attributes by collection instead of on a stroke-by-stroke basis.

Cancel

The Cancel property allows you to cancel, or eliminate, a stroke upon completion. This property may be useful for dealing with strokes drawn in an area of the control where you do not want the user to write, or where the persistence of ink on the screen is not necessary.

The following code is a sample definition for the Stroke event and its arguments. The sender argument is the InkPicture control that received the stroke.

In C#

private void inkPicture1_Stroke(object sender,
Microsoft.Ink.InkCollectorStrokeEventArgs e)
{
        
}

In VB .NET

Private Sub InkPicture1_Stroke(ByVal sender As Object, ByVal e As _
Microsoft.Ink.InkCollectorStrokeEventArgs) Handles InkPicture1.Stroke
Sender

If you want to cancel the stroke, you can set this property to true:

In C#

e.Cancel = true;

In VB .NET

e.Cancel = True

Example: Counting Strokes

Bill is writing an application that will count strokes. On the fifth stroke, Bill wants to display a message to the user that says "You drew 5 strokes!"

To create this functionality, he writes the following code into a form.

In C#

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;

namespace CodeSampleWhitePapers1
{
    /// <summary>
    /// Summary description for Form1.
    /// </summary>
    public class Form1 : System.Windows.Forms.Form
    {
        private Microsoft.Ink.InkPicture inkPicture1;
        int mStrokeCount = 0;
        /// <summary>
        /// Required designer variable.
        /// </summary>
        private System.ComponentModel.Container components = null;

        public Form1()
        {
            //
            // Required for Windows Form Designer support
            //
            InitializeComponent();

            //
            // TODO: Add any constructor code after 
            // InitializeComponent call
            //
        }

        /// <summary>
        /// Clean up any resources being used.
        /// </summary>
        protected override void Dispose( bool disposing )
        {
            if( disposing )
            {
                if (components != null) 
                {
                    components.Dispose();
                }
            }
            base.Dispose( disposing );
        }

        #region Windows Form Designer generated code
        /// <summary>
        /// Required method for Designer support - do not modify
        /// the contents of this method with the code editor.
        /// </summary>
        private void InitializeComponent()
        {
            this.inkPicture1 = new Microsoft.Ink.InkPicture();
            this.SuspendLayout();
            // 
            // inkPicture1
            // 
            this.inkPicture1.Location = new System.Drawing.Point(8, 8);
            this.inkPicture1.MarginX = -2147483648;
            this.inkPicture1.MarginY = -2147483648;
            this.inkPicture1.Name = "inkPicture1";
            this.inkPicture1.Size = new System.Drawing.Size(272, 208);
            this.inkPicture1.TabIndex = 0;
            this.inkPicture1.Stroke += new 
Microsoft.Ink.InkCollectorStrokeEventHandler(this.inkPicture1_Stroke);
            // 
            // Form1
            // 
            this.AutoScaleBaseSize = new System.Drawing.Size(5, 13);
            this.ClientSize = new System.Drawing.Size(292, 266);
            this.Controls.AddRange(new System.Windows.Forms.Control[] {
                                               this.inkPicture1});
            this.Name = "Form1";
            this.Text = "Form1";
            this.Load += new System.EventHandler(this.Form1_Load);
            this.ResumeLayout(false);

        }
        #endregion

        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        [STAThread]
        static void Main() 
        {
            Application.Run(new Form1());
        }

        private void inkPicture1_Stroke(object sender, 
Microsoft.Ink.InkCollectorStrokeEventArgs e)
        {
            mStrokeCount += 1;
            if (mStrokeCount == 5)
            {
            MessageBox.Show("You drew 5 strokes!");
            mStrokeCount = 0;
            }
        }
    }
}

In VB .NET

Public Class Form1
    Inherits System.Windows.Forms.Form

#Region " Windows Form Designer generated code "

    Public Sub New()
        MyBase.New()

        'This call is required by the Windows Form Designer.
        InitializeComponent()

        'Add any initialization after the InitializeComponent() call

    End Sub

    'Form overrides dispose to clean up the component list.
    Protected Overloads Overrides Sub Dispose(ByVal disposing 
As Boolean)
        If disposing Then
            If Not (components Is Nothing) Then
                components.Dispose()
            End If
        End If
        MyBase.Dispose(disposing)
    End Sub

    'Required by the Windows Form Designer
    Private components As System.ComponentModel.IContainer

    'NOTE: The following procedure is required by the 
Windows Form Designer
    'It can be modified using the Windows Form Designer.  
    'Do not modify it using the code editor.
    Friend WithEvents InkPicture1 As Microsoft.Ink.InkPicture
    <System.Diagnostics.DebuggerStepThrough()> 
Private Sub InitializeComponent()
        Me.InkPicture1 = New Microsoft.Ink.InkPicture()
        Me.SuspendLayout()
        '
        'InkPicture1
        '
        Me.InkPicture1.Dock = System.Windows.Forms.DockStyle.Fill
        Me.InkPicture1.MarginX = -2147483648
        Me.InkPicture1.MarginY = -2147483648
        Me.InkPicture1.Name = "InkPicture1"
        Me.InkPicture1.Size = New System.Drawing.Size(292, 266)
        Me.InkPicture1.TabIndex = 0
        '
        'Form1
        '
        Me.AutoScaleBaseSize = New System.Drawing.Size(5, 13)
        Me.ClientSize = New System.Drawing.Size(292, 266)
        Me.Controls.AddRange(New System.Windows.Forms.Control() 
{Me.InkPicture1})
        Me.Name = "Form1"
        Me.Text = "Form1"
        Me.ResumeLayout(False)

    End Sub

#End Region

    Private mStrokeCount As Integer = 0
    Private Sub InkPicture1_Stroke(ByVal sender As Object, 
ByVal e As Microsoft.Ink.InkCollectorStrokeEventArgs) 
Handles InkPicture1.Stroke
        mStrokeCount += 1
        If mStrokeCount = 5 Then
            MsgBox("You drew 5 strokes!")
            mStrokeCount = 0
        End If
    End Sub

End Class

Figure 1 shows how the program will look when five strokes are drawn:

Figure 1. Example of counting strokes

Using the NewInAirPackets and NewPackets Events

While the Stroke event offers information about a stroke upon its completion, the NewInAirPackets and NewPackets events allow you to gather information before and when you begin a stroke, allowing more options for tracking pen and ink behavior.

Using NewInAirPackets

Using the NewInAirPackets event allows you to detect the presence and location of the pen when it is over an InkPicture control. The pen is detected before it touches the screen. This event is fired many times per second and has a very high resolution rate.

Knowing when the user is hovering over a control can be useful in cases where you want to alter the user interface to match the pen position. For example, you can highlight a control to indicate to the user that the control is active, or you can black out a control that is inactive or for some reason cannot receive ink in a designated region. You can also highlight text, or change other ink attributes, as the user hovers over it. When content exists on the Clipboard, you can change the cursor to indicate whether the user can paste the content into the control.

Using NewPackets

Using the NewPackets event enables you to detect Stroke packet data prior to the Stroke event firing. The first time this event fires indicates the starting of a stroke. The NewPackets event will not fire unless the pen is down and touching the screen.

Knowing when the pen is touching the screen is useful in cases when you want to change the ink attributes. For example, you can change the ink to highlight mode when the user hovers over an InkPicture control that contains a text file as the background image.

Preserving Performance

The NewInAirPackets event fires when the pen is hovering over but not touching the digitizer screen above the InkPicture control, while the NewPackets event fires when the pen touches the screen. These events do not fire at the same time.

Packets can be generated hundreds of times per second, resulting in the NewInAirPackets and the NewPackets events firing hundreds of times per second. You should ensure that any code placed into the NewInAirPackets or NewPackets events is short and not processor-intensive. Placing too much code or complex code into these events can cause your application to slow down, reducing application performance, or, in the worst case, can cause your application to lock up.

Setting NewInAirPackets Event Parameters

The following code sample offers a definition of the NewInAirPackets event. Note that this event only fires when the pen is not touching the digitizer tablet. The sender parameter is the InkPicture object that received the packets. The e parameter is the InkCollectorNewInAirPacketsEventArgs object. There are a number of properties in this object, most notably the PacketData property which returns an array. This array contains information about the packets that are arriving. In the Tracking Pen Placement example, we will show you how to extract information from this array in a useful way.

In C#

private void InkPicture1_NewInAirPackets(object sender, 
Microsoft.Ink.InkCollectorNewInAirPacketsEventArgs e)

In VB .NET

Private Sub InkPicture1_NewInAirPackets(ByVal sender As Object, _
ByVal e As Microsoft.Ink.InkCollectorNewInAirPacketsEventArgs) _
Handles InkPicture1.NewInAirPackets

Setting NewPackets Event Parameters

The parameters for this event are similar to those for the NewInAirPackets event shown in the previous code sample. Note that these packets are generated when the pen is touching the digitizer.

Example: Tracking Pen Placement

Bill wants to track the pen. When the pen enters a small region in the upper-left corner of his InkPicture control, he wants to display a message that says, "Hi there!" Bill only wants the message to appear if the pen is not touching the screen.

To implement this behavior, he writes the following code into a form.

In C#

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using Microsoft.Ink;

namespace CodeSampleWhitePapers1
{
    /// <summary>
    /// Summary description for Form1.
    /// </summary>
    public class Form1 : System.Windows.Forms.Form
    {
        private Microsoft.Ink.InkPicture inkPicture1;
        
        /// <summary>
        /// Required designer variable.
        /// </summary>
        private System.ComponentModel.Container components = null;
        Rectangle mHotRectangle = new Rectangle(0, 0, 600, 600);
        int indexX=0;
        int indexY=0;

        public Form1()
        {
            //
            // Required for Windows Form Designer support
            //
            InitializeComponent();

            //
            // TODO: Add any constructor code after 
            // InitializeComponent call
            //
            GetXYIndexes(ref indexX, ref indexY);
        }

        /// <summary>
        /// Clean up any resources being used.
        /// </summary>
        protected override void Dispose( bool disposing )
        {
            if( disposing )
            {
                if (components != null) 
                {
                    components.Dispose();
                }
            }
            base.Dispose( disposing );
        }

        #region Windows Form Designer generated code
        /// <summary>
        /// Required method for Designer support - do not modify
        /// the contents of this method with the code editor.
        /// </summary>
        private void InitializeComponent()
        {
            this.inkPicture1 = new Microsoft.Ink.InkPicture();
            this.SuspendLayout();
            // 
            // inkPicture1
            // 
            this.inkPicture1.Location = new System.Drawing.Point(8, 8);
            this.inkPicture1.MarginX = -2147483648;
            this.inkPicture1.MarginY = -2147483648;
            this.inkPicture1.Name = "inkPicture1";
            this.inkPicture1.Size = new System.Drawing.Size(272, 208);
            this.inkPicture1.TabIndex = 0;
            this.inkPicture1.NewInAirPackets += new 
Microsoft.Ink.InkCollectorNewInAirPacketsEventHandler
(this.inkPicture1_NewInAirPackets);
            
            // 
            // Form1
            // 
            this.AutoScaleBaseSize = new System.Drawing.Size(5, 13);
            this.ClientSize = new System.Drawing.Size(292, 266);
            this.Controls.AddRange(new System.Windows.Forms.Control[] {
                                                  this.inkPicture1});
            this.Name = "Form1";
            this.Text = "Form1";
            this.Load += new System.EventHandler(this.Form1_Load);
            this.ResumeLayout(false);
        }
        #endregion

        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        [STAThread]
        static void Main() 
        {
            Application.Run(new Form1());
        }

        private void GetXYIndexes(ref int theXIndex, ref int theYIndex)
        {
            Guid[] theGuids = inkPicture1.DesiredPacketDescription;

            for (int i=0; i<theGuids.Length -1; i++)
            {
                if (theGuids[i].Equals(PacketProperty.X))
                    theXIndex = i;

                if (theGuids[i].Equals(PacketProperty.Y))
                    theYIndex = i;
            }
        }

        private void inkPicture1_NewInAirPackets(object sender, 
Microsoft.Ink.InkCollectorNewInAirPacketsEventArgs e)
        {
            if (mHotRectangle.Contains(e.PacketData[indexX], 
e.PacketData[indexY]))  
            {
                MessageBox.Show("Hi there!");
            }
        }
    }
}

In VB .NET

Imports Microsoft.Ink
Public Class Form1
    Inherits System.Windows.Forms.Form

#Region " Windows Form Designer generated code "
    Public Sub New()
        MyBase.New()

        'This call is required by the Windows Form Designer.
        InitializeComponent()

        'Initialize the target rectangle
        'Note that this is in InkSpace not pixels
        mHotRectangle = New Rectangle(0, 0, 600, 600)

        'Save the X and Y data locations within the packet data.
        GetXYIndexes(indexX, indexY)
    End Sub


    'Form overrides dispose to clean up the component list.
    Protected Overloads Overrides Sub Dispose(ByVal _
disposing As Boolean)
        If disposing Then
            If Not (components Is Nothing) Then
                components.Dispose()
            End If
        End If
        MyBase.Dispose(disposing)
    End Sub

    'Required by the Windows Form Designer
    Private components As System.ComponentModel.IContainer

    'NOTE: The following procedure is required _
by the Windows Form Designer
    'It can be modified using the Windows Form Designer.  
    'Do not modify it using the code editor.
    Friend WithEvents InkPicture1 As Microsoft.Ink.InkPicture
    Friend WithEvents Label1 As System.Windows.Forms.Label
    <System.Diagnostics.DebuggerStepThrough()> _
Private Sub InitializeComponent()
        Me.InkPicture1 = New Microsoft.Ink.InkPicture()
        Me.Label1 = New System.Windows.Forms.Label()
        Me.SuspendLayout()
        '
        'InkPicture1
        '
        Me.InkPicture1.BorderStyle = _
System.Windows.Forms.BorderStyle.FixedSingle
        Me.InkPicture1.Location = New System.Drawing.Point(43, 49)
        Me.InkPicture1.MarginX = -2147483648
        Me.InkPicture1.MarginY = -2147483648
        Me.InkPicture1.Name = "InkPicture1"
        Me.InkPicture1.Size = New System.Drawing.Size(221, 188)
        Me.InkPicture1.TabIndex = 0
        '
        'Label1
        '
        Me.Label1.AutoSize = True
        Me.Label1.Location = New System.Drawing.Point(20, 32)
        Me.Label1.Name = "Label1"
        Me.Label1.Size = New System.Drawing.Size(84, 13)
        Me.Label1.TabIndex = 1
        Me.Label1.Text = "Hot Spot below:"
        '
        'Form1
        '
        Me.AutoScaleBaseSize = New System.Drawing.Size(5, 13)
        Me.ClientSize = New System.Drawing.Size(292, 266)
        Me.Controls.AddRange(New System.Windows.Forms.Control() _
{Me.Label1, Me.InkPicture1})
        Me.Name = "Form1"
        Me.Text = "Form2"
        Me.ResumeLayout(False)
    End Sub

#End Region
    Dim indexX, indexY As Integer
    Dim mHotRectangle As Rectangle

    Private Sub GetXYIndexes(ByRef theXIndex As Integer, _
        ByRef theYIndex As Integer)
        ' Get the indexes of the X and Y data within the raw
        ' packet data array. 
        Dim theGuids() As Guid = InkPicture1.DesiredPacketDescription
        Dim i As Integer
        For i = 0 To theGuids.Length - 1
            If theGuids(i).Equals(PacketProperty.X) Then
                theXIndex = i
            End If
            If theGuids(i).Equals(PacketProperty.Y) Then
                theYIndex = i
            End If
        Next
    End Sub

    Private Sub NewInAirPackets_Event(ByVal sender As Object, _
        ByVal e As InkCollectorNewInAirPacketsEventArgs) _
Handles InkPicture1.NewInAirPackets
        'The event may return with data for multiple packets.
        'We'll just look at the first packet to make things easier
        If mHotRectangle.Contains( _
            e.PacketData(indexX), e.PacketData(indexY)) Then
            MsgBox("Hi There!")
        End If
    End Sub
End Class

Figure 2 shows the dialog displayed when the pen moves over the upper-left corner region of the InkPicture control.

Figure 2. Example of tracking pen placement

Conclusion

InkPicture events are an excellent avenue for interfacing with the pen and for detecting what the user is doing with the pen. These events allow you the flexibility to detect pen placement above a control or on the page, or to track strokes upon completion. Tracking pen activity this closely can be very helpful in educational applications, medical forms, database input applications, and even software development applications.

About Leszynski Group Inc.

Leszynski Group develops line-of-business applications and retail software built around databases, XML, and ink, using Microsoft .NET architecture. Leszynski Group was the first solution provider to develop and ship ink-based applications for the Tablet PC.