Authors:

    Jezz Santos (Microsoft)
    Edward Bakker (LogicaCMG)

Contributors:

    Rene Schrieken (LogicaCMG)

Reviewers:

    Attila Hajdrik (Microsoft)
    Gareth Jones (Microsoft)
    Pablo Galiano (Clarius Consulting)

October 2007

Applies To:

    .NET Framework 2.0
    Visual Studio 2005
    Visual Studio SDK 4.0
    Guidance Automation Extensions/Toolkit 1.2

Summary: One of the most challenging parts of creating an installer for a software factory solution is knowing exactly what dependencies, components, and configuration need to be installed to a target machine. For the types of package project types (VSX, GAT and DSL) involved in software factory solutions today, this information is neither obvious to derive nor trivial to implement.

Get the reference implementation.

Download the Reference Implementation for this article. You can use this as an example in your own organization. Please read the EULA before using this reference.

Contents

Introduction
Packaging Software Factories
GAT Packages
DSL Packages
Combining Components into Single Package
Creating a Combined Software Factory Installer
Summary
About the Authors

Introduction

Welcome to part 1 of a short series on packaging Software Factories solutions with Visual Studio 2005 and implementing a team build for Software Factories using Team Foundation Server.

Creating software factory solutions today, on the Microsoft development platform, typically involves the combination of one or more Guidance Automation Toolkit (GAT) guidance packages, and one or more Domain Specific Language (DSL) packages. Often these packages are supplemented with other components and customizations of Visual Studio Extensibility (VSX) in a VSX package, including: tool-windows, editors, commands, services etc.

Ideally, the deployment of any software factory solution would be within a single installer (MSI) containing all GAT guidance packages, DSL’s and VSX packages containing the extensibility code and components. Although each authoring toolkit (such as: GAT and DSL) offers a means of creating a setup project exclusively for that individual package, there is no easy and integrated way of combining multiple GAT and DSL packages into one combined setup project, containing the required information to deploy these combined packages as a single software factory release.

GAT and DSL package projects types both provide source artifacts (recipe manifests and definition models) that require intermediate parsing/compilation/registration steps before design-time development changes take effect in the executables they produce at runtime. In the case of GAT, a special registration process parses a package XML manifest file, and compiles and registers the recipes and templates to be used by the Guidance Automation eXtensions (GAX) runtime. In the case of a DSL, the DSL model definition requires transformation of its text templates into source code files which are then compiled into an assembly and registered with Visual Studio for runtime execution. Both these quite unique intermediate ‘pre-compilation’ steps require deliberate and manual intervention steps from the developers creating these package types. This is acceptable for individually managed development projects. But neither pre-compilation step is enforced as part of a check-in process or as part of the build process. In managed team development scenarios, these pre-compilation steps will need to be explicitly invoked to verify and ensure that resulting compiled executables reflect the current design-time changes in the intermediate artifacts, so that they don’t break builds, and that can be deployed.

This series provides information and guidance, for those implementing software factory solutions today, that addresses both packaging software factories into MSI installers, and implementation of team build environments for these types of solutions. Part 1 in this series, this part, addresses the creation of installers using Visual Studio 2005. The second part in the series addresses the implementation of team build of these solutions using Team Foundation Server.

Packaging Software Factories

The recommended way to package and deploy a software factory solution, like any other solution, is via an MSI installer. MSI is the preferred way to install, maintain and remove software from a windows computer.

Standard VS Setup Projects

With Visual Studio, MSI installers to meet the needs of most solutions can be created using standard Visual Studio setup projects. Although other MSI building products are readily available on the market for catering to more advanced installer needs (e.g. InstallShield and Wise etc.), standard VS setup projects provide most, if not all, basic capabilities for packaging most software factory solutions.

Creating a setup project to define an installer in Visual Studio is simply a matter of adding a standard VS setup project to your solution and then configuring the installer using its various editors.

This project type provides a friendly graphical user interface for configuring the installer, with various editors to configure the basic options. The capabilities of this setup project are limited to a basic set of capabilities of MSI for ease of use.

However, configuration of the editors and features of the setup project cannot be easily automated or extended, and the file format which the setup project uses is in a very difficult format to edit and modify with tools. Furthermore, as we will see in the second part of this series, this type of setup project cannot be built in a team build scenario using Team Foundation Server (TFS) because the project file is not MSBUILD compatible, and will be ignored by MSBUILD.

WIX Setup Projects

An alternative technology which can be used to create installer projects is Windows Installer XML (WIX). WIX is a free Microsoft open-source technology for building MSI’s just from XML files. The XML files used by WIX define the contents of the installer package.  The WIX toolset provides a WIX compiler (candle.exe and light.exe) to create your MSI from these files. WIX provides a number of tools in its toolset to compile the WIX XML files into an MSI. The WIX toolset also provides additional tools (such as: dark.exe) to reverse engineer existing MSI’s into WIX XML.

WIX files can be authored using any XML editor. WIX setup projects for Visual Studio are provided by 3^rd party tools such as WIX Votive.

At the time of writing, the WIX toolset and WIX Votive are available in version 2.0 and version 3.0.

·         WIX Votive 2.0 uses non-MSBUILD compatible setup project files which cannot be built in an automated build environment such as Team Foundation Server.

·         WIX Votive 3.0 does use MSBUILD compatible setup project files suitable for automated build processes, but is considered still under development and unstable.

To work around these issues, an alternative project type is defined later in the document that uses WIX 2.0 with MSBUILD compatible project files to leverage the WIX toolset.

The advantage of using WIX over using the standard VS setup project are that: you have a much finer granularity of control over the created installer because WIX accommodates more features of MSI, and you can automate and script the creation and modification of WIX XML files with other tools. However, creating and editing these XML files by hand is very tedious, and there is little graphical user interface provided by tools like Votive to assist you to do more easily and more reliably.

WixEdit.exe is a 3^rd party graphical WIX editing tool, which improves the manual editing of WIX XML files. However at the time of writing, this tool was still under development.

Orca.exe, an MSI editing tool, available as part of the Microsoft Platform SDK, used for customizing and validating existing compiled MSI’s.

The net result, whether using standard VS setup projects or WIX projects, is that either of these tools and their technologies provides a means to author a setup project to create and installer for your software factory solution. Furthermore, either of these tools can be used to package individual project types or combine several project types together as one installer.

MSI Installers

Before we delve into what the various software factory package types are that we are building installers for, and how we build those installers, let’s have a quick look at how we define an installer and its base components used in the setup projects.

An MSI setup project defines an installer in terms of the following primary components:

·         Registry Searches

·         Launch Conditions

·         Files

·         Registry Settings

·         Custom Actions

An MSI setup project also defines other components such as a user interface wizard and component features. But for the purposes of this article we are going to only address the above primary components, as these are the ones that vary the most between any deployed installer for a software factory.

Registry Searches

A ‘registry search’ is executed at the start of the installer to gather information from the target machine’s registry. This information is then assigned to property variables which are made available to other components in the installer for defining things like: files and folder names, parameters to custom actions etc.

A registry search is defined by specifying: a registry hive, key path, and value, and this value is then assigned to a named property variable. For example:

If a registry search does not find the specified registry value on the target machine, its result is invalid, and this can be tested for by other MSI components.

Launch Conditions

A ‘launch condition’ is executed at the start of the installer to verify certain pre-conditions of the installer’s execution. Typically, launch conditions are used to ensure certain pre-requisite components are already installed on the target machine prior to execution of this installer. If a launch condition fails, the installer is halted, the user is informed of which condition was not met, and then the installer is terminated.

Launch conditions typically test for valid values of registry searches.

Launch conditions can also test ‘File Searches’. File searches are not described in this article.

A launch condition is defined by specifying a registry or file search condition to test, and a message to display to the user if that test fails. For example:

Typically, the message displayed will declare that a pre-requisite component does not exist on the target machine and that that component needs to be installed before this installer can be run. Typically, this message will also include a URL to where the pre-requisite component can be downloaded or acquired from.

Files & Folders

Files and folders represent one of the basic units of physical assets that are installed to a target machine by the installer (registry settings being the other). An installer typically defines the location of the files to be installed, and folders are created as required. Files include assemblies, configuration files, resource files, shortcuts etc. Folders include: the user’s or machines programs menu, program files folders, application and machine data folders and the global assembly cache.

A file is defined by specifying the destination folder, filename and other file attributes. Folders are assigned a property variable for easy referencing by files. For example:

Folder locations and file names can use property variables assigned by registry searches or calculated and provided by the installer from the target machines environment.

For a list of common property variables see the online MSI documentation.

Registry Settings

Registry settings define the other physical asset type that is typically installed to a target machine by the installer. An installer defines the registry hive, key and value to assign.

A registry setting is defined by specifying the registry hive, key, value type and value. For example:

Registry key name and values can use property variables assigned by registry searches or calculated and provided by the installer from the target machines environment.

For a list of common property variables see the online MSI documentation.

Custom Actions

Custom Actions define actions or tasks that need to be executed in addition to executing launch conditions, installing files, folders and registry settings. Custom actions can be executed at various stages throughout the installation and un-installation processes. Typically, custom actions are used to execute scripts that run at the end of the installer. (e.g. launch a readme file, invoke an application, execute additional configuration or tools etc.)

For a list of common custom actions see the online MSI documentation.

Custom actions are typically implemented using scripts. The standard VS setup project however defines a mechanism that allows the installer developer to provide custom actions in an Installer Class as part of the solution they are deploying. This mechanism then provides the necessary hooks into the MSI custom actions engine to execute the installer class code at the right stage.

Installer classes do require that the code is installed on the target machine, and that it can be executed at install time by the installer.

Other setup project technologies, such as WIX, are able to implement custom actions using scripting mechanisms to avoid the trappings of installer classes.

A custom action is defined by specifying an installer stage to execute the action, and a script or installer class to execute. For example:

Scripts can use property variables assigned by registry searches or calculated and provided by the installer from the target machines environment. When using installer classes, these property values are passes using ‘Custom Action Data’.

Factory Package Types

Building software factories today, commonly involves the creation of templates and recipes defined in GAT guidance packages as a starting point. This is clearly evident in the early generation of the ‘1GAXPAK’ (single guidance package) types of software factories available from Microsoft and other vendors. It is becoming increasingly more common to combine and integrate the templates and recipes of these factories with one or more domain specific languages; each defined in their own DSL packages. Some factories desire a richer integration and user experience, and achieve this by adding additional Visual Studio eXtensibility (VSX) components, such as: services, tool-windows, editors, commands, menus etc. For these types of factories, the authors extend and enhance the capabilities of the software factory by creating a VSX project type package.

In the following sections, we explain what these relevant package project types are, how they work, and what the relevant MSI components are that you need to configure to build an installer package.

As far as what setup project technology or tool to use to create your installer, we leave the technical details of that to the final chapter ‘Creating a Combined Software Factory Installer’, where we put all this knowledge together and describe what you need to do to create a combined software factory installer that includes one or more of each of these package project types.

VSX Packages

Note. A VSX package was formerly referred to as a Visual Studio Integration (VSIP) Package.

A common means to extend Visual Studio today is by creating a Visual Studio Extensibility (VSX) package project, to contain the various extensibility components and customizations you desire (i.e. the services, tool-windows, editors, commands, menus etc.).

Visual Studio ‘Add-ins’ are also another means of providing Visual Studio integration, but typically the level of integration required by software factories demands the implementation of VSX packages. VS Add-ins are not considered in this article.

Once developed, VSX packages are compiled and registered with Visual Studio before being loaded and used by Visual Studio at runtime. Registration is achieved by adding various registry entries to the Visual Studio registry hive representing each extensibility component. In managed .NET development of these packages, various attributes of the Managed Package Framework (MPF) are used to declare registration information for the various components in the VSX package. At compile time this registration information is collected and added to the registry using the package registration tool ‘regpkg.exe‘. This tool reflects over the MPF attributes of the compiled assembly and updates the registration information in the registry for Visual Studio. This enabled debugging of the package at development time.

You can obtain these registry settings at design time by exporting them from your package using regpkg.exe to a Registry file (‘*.reg’) or to a WIX file (‘*.wxi’), using the appropriate command line switches.

These registry settings along with your VSX package assembly are then included in your setup project and to build an installer to deploy your VSX package.

The creation and development of many extensibility components can be significantly accelerated by using the ‘VSSDK Assist’ toolkit, which encapsulates many of the best practices for developing Visual Studio extensibility components into a set of guidance automation recipes you can execute on your VSX package project. ‘VSSDK Assist’ is a community supported development tool.

At development time, to perform this registration automatically for the developer to enable debugging, a VSX package project file (‘*.csproj’) contains some VSX specific MSBUILD targets that invokes regpkg.exe. This ensures that any changes to package are updated in the registry for debugging.

<Project>
  <PropertyGroup>
    <TargetRegistryRoot>Software\Microsoft\VisualStudio\8.0Exp</TargetRegistryRoot>
    <RegisterOutputPackage>true</RegisterOutputPackage>
    <RegisterWithCodebase>true</RegisterWithCodebase>
  </PropertyGroup>
  <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
  <Import Project="$(ProgramFiles)\Visual Studio 2005 SDK\2007.02\VisualStudioIntegration\ _
		Tools\Build\Microsoft.VsSDK.targets" />
...
</Project>

By default, for debugging, VSX packages are intended to be deployed privately and not to the Global Assembly Cache (GAC). (<RegisterWithCodebase>true</RegisterWithCodebase>)

However, for deployment of VSX packages, the best practice is to deploy and execute them from the GAC.

Creating an Installer

At present in Visual Studio there are no project types that create setup projects specifically for deployment of VSX packages. Setup projects must be manually created and configured by the VSX package developer. This process can be tedious and error prone because of its manual nature.

An example of a standard VS setup project and a WIX setup project for a VSX package can be found in the sample with this article.

Creating a VSX Installer Manually

The following sections details the MSI specific components (such as: registry searches, launch conditions, files, registry and custom actions) that are most important for creating an MSI compliant installer for this package project type.

The following section details what the relevant MSI components are, but does not detail how to implement them in any type of setup project technology. That information is deferred to the last section of the document where the details for implementing an installer, for both a standard setup project and a WIX setup project, are given in the context of creating a combined installer.

The following steps detail the process and MSI components for building a VSX installer.

1.     Ensure your VSX package project compiles.

2.     Add a setup project to your package solution.

3.     In the ‘bin\Debug’ directory of your VSX package project, execute the following command line:

"C:\Program Files\Visual Studio 2005
SDK\2007.02\VisualStudioIntegration\Tools\Bin\regpkg.exe" /regfile:"vsxpackage.reg"
"MyVsPackage.dll"

"C:\Program Files\Visual Studio 2005
SDK\2007.02\VisualStudioIntegration\Tools\Bin\regpkg.exe" /wixfile:"vsxpackage.wxi"
"MyVsPackage.dll"

Note. This script can be enhanced and used in a ‘Post Build’ event on the project.

Registry Searches

Launch Conditions

File System

Registry

Import all registry entries from the ‘vsxpackage.reg’ file created earlier in step 3.

Ensure that any imported registry settings containing absolute file paths are replaced with relative paths prepended with [TARGETDIR] so they refer to absolute paths on target machine determined at install time.

Custom Actions

In the case of all VSX packages, after the package assembly has been installed, and after uninstall, Visual Studio needs to be instructed to reset its various resource caches; such as the toolbox, commands, templates etc. To do this, a custom action is required to execute the following command line:

devenv.exe /setup 

This command is only required to be executed once for all newly installed VSX packages.

Further Information

The process for completing a VSX installer is documented in detail on MSDN.

The Microsoft VSX team maintains the VSX Team Blog to provide information and the VSX MSDN Forum provides assistance for building and supporting Visual Studio extensibility packages.

GAT Packages

The Guidance Automation Toolkit (GAT) is the authoring tool for creating guidance packages (GP’s) which are loaded at runtime by the Guidance Automation eXtensions (GAX) runtime. Although GAX itself is a VSX package, the package projects which GAT creates are not VSX packages. These GAT projects are merely class library assemblies defining: custom references, actions, value providers and converters etc. These assemblies also contain project, file and solution templates and the recipes which are associated to the other classes in the assembly.

Registration and runtime integration with Visual Studio of the recipes, commands and templates of a custom GAT package is not done in the standard way normal VSX packages work. Instead, the Guidance Automation eXtensions (GAX) provides a runtime platform that loads guidance packages, and their recipes, commands and templates into Visual Studio as needed (i.e. when you open a solution associated to a GAT package).

All the design-time recipe information for a GAT package is contained with a single XML manifest file which defines various pieces of information about the package (including the package identity and name). This package manifest file is processed by various GAT and GAX components in order to register and load the package at runtime.

It is possible to split the recipes of this manifest into separate files which are included in the manifest using XInclude statements.

Unlike normal VSX Packages, the Visual Studio registration of GAT packages is performed by a separate tool. This tool is in fact a recipe that the authoring environment of GAT provides, which is associated to your guidance package solution. This recipe parses the package manifest file, and registers basic information about the GAT package: templates, commands and other information into the registry and file system. This information is then used for loading the guidance package at runtime by GAX.

To facilitate this registration process, and to allow setup projects to leverage this registration mechanism, a guidance package is created with an associated installer project containing an installer class to install that guidance package.

Notice that GAT utilizes a standard VS setup project to deploy the guidance package.

The association between the GAT package project and the installer class project is achieved by referencing the installer class project from the GAT package project (Usually a project reference within same solution).

When you register the GAT package (using the registration recipe) the installer class is invoked and registration tasks are performed on the GAT package project.

Creating an Installer

For creating an MSI installer for your guidance package, GAT automatically creates a standard VS setup project along with the installer class project and your guidance package project. This setup project includes the package assembly, and all the content files of the GAT package project (including the package manifest file and the various code templates etc.) and is configured automatically for the GAT package.

A ‘Custom Action’ is included in the setup project which invokes the installer class (contained in installer class project), passing the path of the package XML manifest file to the installer class. When the setup project is executed on the target machine, this custom action invokes the installer class to do the package registration on the installed content files.

Fortunately this setup project and installer mechanism are provided to you automatically when you create a custom guidance package solution, and no further configuration is required by the GAT package developer. However, one caveat with this mechanism is that you require a single dedicated installer class (one per assembly) for each GAT package assembly, and the assembly containing that installer class must be deployed to the ‘Public Assemblies’ folder of Visual Studio (by default: ‘C:\Program Files\Visual Studio 8\Common7\IDE\PublicAssemblies’), for both installation and un-installation of the GAT package. This is a requirement of where GAX is deployed by default.

It is entirely feasible to replace the provided standard setup project with a similar WIX setup project of your own, although you will have to manually create and configure the WIX setup project.

An example of a WIX setup project for a GAT package can be found in the sample with this article.

Creating a GAT Installer Manually

The following sections details the MSI specific components (such as: registry searches, launch conditions, files, registry and custom actions) that are most important for creating an MSI compliant installer for this package project type.

The following section details what the relevant MSI components are, but does not detail how to implement them in any type of setup project technology. That information is deferred to the last section of the document where the details for implementing an installer, for both a standard setup project and a WIX setup project, are given in the context of creating a combined installer.

The following steps detail the process and MSI components for building a GAT installer.

1.      Ensure your GAT package project compiles.

2.      Register your GAT package using the ‘Register Guidance Package’ recipe provided by GAX.

Registry Searches

Launch Conditions

File System

Registry

No components.

Custom Actions

The custom actions required by GAT only exist in the form of installer classes that are included in the GAT installer project.

Where: ‘MyGuidancePackage.xml’ is the filename of your GAT manifest file in the GAT package project.

GAT Setup Projects

By default GAT creates and uses a standard VS setup project for deployment. This has a number of disadvantages when it comes to automating the build of the GAT solution related to the fact that a standard VS setup project cannot be built on a build server such as Team Foundation Server. This is because the standard VS setup project is not MSBUILD compliant.

WIX offers an alternative means to create a GAT setup project reusing the assets already provided by default from GAT such as the installer class, and WIX setup projects can be automated in a build process.

An example of a WIX setup project for a GAT package can be found in the sample with this article.

Further Information

The GAT registration and setup process is not currently documented in detail on MSDN. However the GAT MSDN Forum provides assistance in building and supporting custom GAT packages.

DSL Packages

The Domain-Specific Language Tools (DSL Tools) defines its extensibility components (i.e. designers, menu commands, tool-windows, etc.) in an enhanced VSX package project (the ‘DslPackage’ project in a DSL solution). This VSX package behaves and is registered and loaded in the same way as normal VSX packages.

All the Visual Studio extensibility components defined by a DSL definition are generated by text templates as source files in the package project. These components also utilize MPF attributes in the same way as VSX package components. At compile time, the compiled package assembly is registered with Visual Studio automatically in the same way as VSX packages using special MSBUILD commands in the project file (*.csproj). In addition, the DSL Tools adds the assembly to the GAC (gacutil.exe) each time the assembly is rebuilt.

<Project>
  <PropertyGroup>
    ...
    <TargetRegistryRoot>Software\Microsoft\VisualStudio\8.0Exp</TargetRegistryRoot>
    <RegisterOutputPackage>true</RegisterOutputPackage>
    <RegisterWithCodebase>false</RegisterWithCodebase>
    <GacTargetOutput>true</GacTargetOutput>
  </PropertyGroup>
  <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
  <Import Project="$(ProgramFiles)\Visual Studio 2005 SDK\2007.02\VisualStudioIntegration\ _
		Tools\Build\Microsoft.VsSDK.targets" />
  <Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\DSLTools\v2.0\ _
		Microsoft.DSLTools.targets" />
...
</Project>

Notice that by default, for debugging, DSL packages are deployed to the GAC and not privately (<RegisterWithCodebase>false</RegisterWithCodebase>), unlike the default for VSX packages.

For deployment, like VSX packages, DSL package assemblies should also be deployed to the GAC.

Creating an Installer

For creating an installer for a DSL, the DSL Tools also provides a separate DSL setup project that you need to add this to your DSL solution manually.

The DSL Tools utilizes WIX (version 2.0) and a special setup project with text templates to generate the installer from the DSL definition and package project.

The DSL Tools don’t use WIX Votive 2.0 setup projects, instead the WIX files are contained within a standard class library project, and compiled using tools from the WIX 2.0 toolset and custom MSBUILD targets.

There are a number of reasons this is a WIX setup project instead of standard Visual Studio setup project. Primarily, as mentioned before, it is very hard to automate modifications to a standard setup project to synchronize with any changes to the DSL solution (i.e. resources, editors, menu commands, tool-windows, file extensions, etc.). In a standard setup project these would have to be manually configured and there would be no automation interface to automatically synchronize changes from DSL definition to setup project. Furthermore, is anticipated that DSL developers will customize a DSL project to some degree and potentially add other ext