Adding Semantic Information

This topic is the fifth of seven tutorial topics covering tools in the Microsoft Speech Application SDK Version 1.1 (SASDK). This section of the tutorial uses Speech Control Editor, which is the all-purpose tool for assembling a working speech-enabled Web application in Microsoft Visual Studio .NET 2003. Speech Control Editor manipulates ASP.NET Speech Controls that produce Speech Application Language Tags (SALT) and JScript on the client. The developer uses Speech Control Editor to specify prompts to speak to the user, to recognize answers using grammars, and to confirm the answers.

For reference information on Speech Control Editor, see Creating Speech-Enabled Web Pages for an introduction to the topic, and Recognizing and Confirming Answers for specifics on QA controls. In addition, Designing Dialogue Flow contains guidelines for designing voice-only applications. Handling Data contains more information on the SemanticMap control and SemanticItems.

This sequence of tutorials demonstrates how to build a simple voice-only ASP.NET Web application using the SASDK. Specifically, the tutorials demonstrate building a Start page of an imaginary pizza ordering service, for use by telephony Speech Application Language Tags (SALT) clients.

The procedures for creating the pizza ordering service application build on each other. Performing the procedures in sequence is therefore important.

The application needs to gather information from the caller's responses to the prompts that each control plays. It also needs to retain that information for use throughout the application. Each rule in the grammar specifies semantic information in Semantic Markup Language (SML) returned by the Speech Recognition (SR) engine after it analyzes the caller's responses. The SemanticMap control is the repository for this information. It consists of a collection of SemanticItem controls, which contain information about semantic state and binding and postback properties. These SemanticItem controls contain the SML markup. To get the semantic information, it is necessary to:

1. Specify what each rule in the grammar returns.
2. Add a SemanticItem for semantic information returned by each rule.
3. Specify in each control how to associate the semantic information returned by the grammar with a SemanticItem.

Getting Semantic Information from Rules

Up to this point, the rules in the grammar created in the first section of the tutorial, Creating a Grammar, do nothing. Each rule in the grammar, however, needs to return semantic information. This section of the tutorial demonstrates how to get semantic information from the grammar, using script tags.

To get semantic information from the Size rule

1. In Solution Explorer, double-click PizzaOrder.grxml to open it in Grammar Explorer.

2. Double-click the Size rule to open it in the Rule Editor window.

3. From the Grammar tab on the Toolbox, drag a Script Tag element onto the canvas and drop it in the List element with the sizes (small, medium, and large), just to the right of the Phrase element with the text "small." Similarly, drag Script Tags to the right of the Phrase elements with "medium" and "large."

   The rule now looks like the following illustration.

1. Double-click the Script Tag element to the right of the Phrase element with "small" and then click the ellipsis button (...) to open the Semantic Script Editor.
2. On the Assignment tab, select Return result in a sub-property of this Rule: and enter Size in the Name of property edit box. In the Source of semantic result panel, Constant is selected by default, and "small" appears by default in the Enter value field.
   The Semantic Script Editor now looks like the following illustration.

1. Click OK.
2. Right-click the PizzaOrder.grxml - Size tab, and select Save Grammars\PizzaOrder.grxml to save the grammar file.
3. To check the SML returned, enter "small" in the Recognition string text box, and click Check.
   The SML returned looks like the values in the following illustration.

1. This semantic script returns the specified value, tagged as Size, in the SML output. A confirmation prompt will later use this text, and a production application would display or print this text for the pizza maker and the cashier to see.
2. Repeat steps 4 through 7 for the Phrase elements with "medium" and "large."

To get semantic information from the PhoneNumber rule

1. In Solution Explorer, double-click PizzaOrder.grxml to open it in Grammar Explorer.
2. Double-click the PhoneNumber rule to open it in the Rule Editor window.
3. On the Toolbox, click the Grammar tab, drag a Script Tag element onto the canvas and drop it just to the right of the RuleRef element.
   The rule now looks like the following illustration.

1. Double-click the Script Tag element and then click the ellipsis button (...) to open the Semantic Script Editor.
   In the Source of semantic result panel, click Rule variable and drop the list in the drop-down list box. This list has values that you can return in SML from this rule. In this case, it is necessary to insert some script to return three values in the SML:
   • The text of the spoken phone number, to use later in a confirmation prompt.
   • The area code and the local phone number as digits that a production application would display or print for the pizza maker and the cashier to see.
2. Click the Script tab to display the Script editor.
3. Insert the following script into the script pane, overwriting the default script:

   $.PhoneNumberText = $USPhoneNumber.AreaCode._attributes.text + " " + $USPhoneNumber.LocalNumber._attributes.text;
   $.AreaCodeDigits =  $USPhoneNumber.AreaCode;
   $.LocalNumberDigits = $USPhoneNumber.LocalNumber;

   The Semantic Script Editor now looks like the following illustration.

1. Click OK.
2. Right-click the PizzaOrder.grxml - PhoneNumber tab, and select Save Grammars\PizzaOrder.grxml to save the grammar file.
3. To check the returned SML, type four two five five five five oh eight zero zero in the Recognition string text box, and click Check.
   The SML returned looks like the values in the following illustration.

1. This semantic script returns the recognized text as a value, tagged as PhoneNumberText, and the area code and the local number as digits, tagged as AreaCodeDigits and LocalPhoneDigits, in the SML output.

For more information on variable assignments, see Creating Simple Variable Assignments and Creating Complex Operations in Adding Semantic Interpretation Information.

To get semantic information from the Confirm rule

1. Double-click the Confirm rule to open it in the Rule Editor window.
2. On the Toolbox, click the Grammar tab, drag a Script Tag element onto the canvas and drop it just to the right of the RuleRef element with the Yes rule.
3. Drag a Script Tag element onto the canvas and drop it just to the right of the RuleRef element with the No rule.
   The rule now looks like the following illustration.

1. Double-click the top Script Tag element and then click the ellipsis button (...) to open the Semantic Script Editor.
2. On the Assignment tab, select Return result in a sub-property of this Rule: and enter Yes in the Name of property edit box.
3. In the Source of semantic result panel, click Rule variable, select - Last Rule ($$) - from the drop-down list box.
4. Select Select property, and choose _value from the drop-down list.
   The Semantic Script Editor now looks like the following illustration.

1. Click OK.
2. Follow the previous steps with the bottom script tag element, except enter No in the Name of property edit box.

This semantic script returns "Yes" or "No" as a value, tagged as either Yes or No, depending on which phrase the SR engine recognizes in the SML output. If the caller says Nope or I don't think so, for example, the rule returns the value "No." This confirmation will later determine the flow of the dialogue.

To get semantic information from the Cancel rule

1. Double-click the Cancel rule to open it in the Rule Editor window.
2. On the Toolbox, click the Grammar tab, drag a Script Tag element onto the canvas and drop it just to the right of the RuleRef element.
3. Double-click the Script Tag element and then click the ellipsis button (...) to open the Semantic Script Editor.
4. On the Assignment tab, select Return result in a sub-property of this Rule: and enter Cancel in the Name of property edit box.
5. In the Source of semantic result panel, click Rule variable and select - Recognized Text - from the drop-down list box.
6. Click OK.
7. Right-click the PizzaOrder.grxml - Cancel tab and select Save Grammars\PizzaOrder.grxml from the context menu to save the grammar file.

This semantic script returns the recognized text as a value in the SML output.

Specifying Semantic Information in a SemanticMap Control

The SemanticMap control specifies semantic items to be used throughout the application to contain the semantic information returned in SML from the Speech Recognition (SR) engine. This section of the tutorial creates those semantic items. The Speech Web Application Project Wizard added a SemanticMap control by default when it created the basic application framework.

To add Semantic Items

1. In Default.aspx, on the design canvas right-click the SemanticMap control, and select Property Builder.

2. On SemanticMap Properties, click Add to add a semantic item.

3. In the ID edit box, replace the default value with siSize.

4. In the TargetElement drop-down list box, select PizzaPanel.

5. In the TargetAttribute drop-down list box, select value.

6. Select BindOnChanged.

7. Click Apply.

8. Repeat steps 2 through 7 to create the following SemanticItems:

   siPhoneNumberText
   siPhoneNumberAreaCodeDigits
   siPhoneNumberLocalDigits
   siYes
   siNo
   siCancel

   SemanticMap Properties now looks like the following illustration.

1. Click OK to close SemanticMap Properties.
2. Right-click the Default.aspx tab, and from the context menu, select Save Default.aspx.

Gathering Semantic Information from the QA Controls

To this point, the QA controls for Size and PhoneNumber have merely played back prompts. This section specifies recognition rules from PizzaOrder.grxml for the controls, and specifies which recognition results get bound to which SemanticItems.

To add a grammar rule to SizeQA

1. Right-click SizeQA, and select Property Builder.
2. In SizeQA Properties, in the tree view pane on the left, select Input.
   This property sheet specifies the grammar rules to use, and the answers to extract from the recognition results.
3. In the right pane, click Add Existing Grammar.
4. In Add Existing Grammar, click Browse next to the Grammar Path edit box.
5. In Grammar URL browser, click Grammars in the Projects: pane, and then double-click PizzaOrder.grxml in the Contents of 'Grammars' pane.
6. In Add Existing Grammar, click Browse in the Active Rule section.
7. In Rule Browser, select Size from the list of rules, and click Select Active Rule.
8. In Add Existing Grammar, click OK.
   SizeQA Properties now looks like the following illustration.

To add semantic values to SizeQA

This section uses the XPathTrigger Sample Sentence Tool to input a phrase recognized by the selected rule, and display the SML returned by the SR engine. The SML also appears as a drop-down list under XPathTrigger on the Answers tab. The XPathTrigger value will bind to a SemanticItem on the Answers tab.

1. In SizeQA Properties, click XPathTrigger Sample Sentence Tool.
2. In the Sample speech input edit box, type large, and click Go.
   SizeQA Properties looks like the following illustration, with the returned SML displayed.

1. On the Answers tab, double-click the first cell under SemanticItem to display the drop-down list box, and select siSize.
2. Click the XPathTrigger drop-down list box, and select SML/Size.
   That binds the semantic information in SML/Size to the siSize SemanticItem in SemanticMap.
   The Answer tab now looks like the following illustration.

1. Click OK.

To add a grammar rule to PhoneQA

Using the earlier procedure To add a grammar rule to SizeQA as a guide, connect to the grammar file PizzaOrder.grxml, and set PhoneNumber as the active rule.

To add semantic values to PhoneQA

1. In PhoneQA Properties, click XPathTrigger Sample Sentence Tool.
2. In the Sample speech input edit box, type four two five five five five zero nine oh oh, and click Go.
   The Sample speech input section looks like the following illustration, with the returned SML displayed.

Using the earlier procedure To add semantic values to SizeQA as a guide, add the settings shown in the following illustration.

1. Click OK.
2. Right-click the Default.aspx tab, and from the context menu, select Save Default.aspx.