CCPAPI.DLL Basics

  • Article

 

Dennis Crain, Program Manager
Windows HPC team

May 2006

Contents

Overview
Interfaces
Job-Task Relationship
Using CCPAPI
Writing the Code

Overview

CCPAPI.DLL provides access to the Windows Compute Cluster Server 2003 (CCS) scheduler. This DLL provides COM interfaces to numerous objects within the CCS scheduler. By writing applications or scripts using these interfaces you may connect to a cluster and manage jobs, job resources, tasks, nodes, environment variables, extended job terms and much more. This article will describe the basic functionality of CCPAPI.DLL and provide guidance in writing your first serial job scheduling application using the DLL and C#.

Interfaces

Eleven interfaces are provided by CCPAPI including:

  • ICluster
  • IClusterCounter
  • IClusterEnumerable
  • IExecutionResult
  • IJob
  • IJboCounter
  • INameValue
  • INameValueCollection
  • INode
  • IResourceUsage
  • ITask

As you write scheduler applications and/or scripts you will find that four of these interfaces are primary to the task (no pun intended) at hand including ICluster, ITask, Ijob, and INode. The other seven support these primary interfaces.

Before proceeding it is important to thoroughly understand the relationship of tasks to jobs. The following description of jobs and tasks is excerpted from the CCS SDK documentation.

Job-Task Relationship

A task represents the execution of a single process or multiple processes on a compute node. A collection of tasks that is used to perform a computation is known as a job. Jobs are used to reserve the resources required by tasks. The job may be thought of as the parent of a task.

Compute Jobs

The following illustrates the job life cycle:

Job Life Cycle

Figure 1. Job Life Cycle

AddJob() sends the job to the server but the job state, expressed as part of the enumeration JobStatus, remains NotSubmitted. You can add tasks to the job and then submit the job using SubmitJob(). If the job is successfully submitted, the job state transitions to Queued.

When resources are allocated to a job, the job state transitions to Running. When a job is completed, its allocated resources are released and its job state changes to Finished, Failed, or Cancelled.

Compute Tasks

The following illustrates the task life cycle super-imposed over the parent job life cycle:

Job-Task Relationship

Figure 2: Job-Task Relationship

All tasks that are associated with a job are queued or submitted when the parent job is queued or submitted. When resources are allocated to a task, its task state, expressed as part of the enumeration TaskStatus, changes to Running. When a task is completed, its allocated resources are released and its task state changes to Finished, Failed, or Cancelled.

Using CCPAPI

Using CCPAPI, in its simplest terms, is a five-step process.

  • Connect to the cluster.
  • Create a job.
  • Create a task.
  • Add the task to the job.
  • Submit the job/task for execution.

To connect to the cluster use the ICluster::Connect method. Create a job using the ICluster::CreateJob method. To create a task, use the ICluster::CreateTask method. To add a child task to a job, use the ICluster::AddTask method. Finally, submit the job using ICluster::SubmitJob().

Writing the Code

This sample, written in C#, demonstrates the five steps described above. The code submits a serial job. There are other types of jobs such as workflow, parameter sweeps, and parallel. Future articles will describe these uses of CCPAPI.

If you would like to code this sample you must first install Visual Studio 2005 and the Microsoft Compute Cluster Pack SDK on a 32- or 64-bit computer. If you wish to execute the program you must have Microsoft Windows Compute Cluster Server 2003 installed on at least one 64-bit computer or a cluster of 64-bit machines. If you are using one 64-bit machine, configure it as a head and compute node during the installation of CCS.

Project Description

This code illustrates a console application that, after having connected to the cluster, creates and submits a job with the task cmd.exe /c helloworld.

References and Namespace

You must add CCPAPI.DLL as a reference to your application. In Visual Studio, this is done by choosing "Add Reference" from the Project menu. Then add the using Microsoft.ComputeCluster statement to the code as follows:

using Microsoft.ComputeCluster;

Declaring CCS Variables

In this sample, the code requires the ICluster, Ijob, and ITask interfaces. If you needed other CCS interfaces you would declare then in a similar manner. The last two variables are for convenience only.

static ICluster myCluster;
static IJob myJob;
static ITask myTask;
static string clusterName = "localhost";
static string commandLine = "cmd.exe /c helloworld";

Main

The following Main function follows the five-step process for using CCPAPI. Connect to the cluster, create a task, create a job, add the task to the job, and submit the job.

static void Main(string[] args)
{
    try
    {
        //initialize the cluster
        myCluster = InitCluster(clusterName);
        //initialize the task
        myTask = InitTask(commandLine, myEnvVars);
        //initialize the job
        myJob = InitJob(myTask);
        //add the job/task to the cluster               
        myCluster.AddJob(myJob);
        //submit the job for execution
        myCluster.SubmitJob(myJob.Id, null, null, true, 0);
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

Initialize the Cluster

Initializing the cluster, in this example, is really nothing more than connecting to the cluster. The cluster name is the machine name of the headnode of your cluster. If you are executing the application on the headnode then using "localhost" is sufficient.

private static ICluster InitCluster(string myClustername)
 {
     ICluster c = null;
     c = new Cluster();
     c.Connect(myClustername);
     return (c);
 }

Initialize the Task

Recall that the task is a process or collection of processes that which you wish to execute on a compute node. In the case of this sample, the process is cmd.exe /c helloworld. This process is assigned to the CommandLine property in the task object.

private static ITask InitTask(string commandLine)
{
    ITask t = null;
    t = new Task();
    t.CommandLine = commandLine;
    return (t);
}

Initialize the Job

This function takes the task object as a parameter. After the job is created the task is added to the job. Upon returning from this function the job is ready to be submitted for execution.

private static IJob InitJob(ITask myTask)
{
    IJob j = null;
    j = new Job();
    j.AddTask(myTask);
    return (j);
}

Complete Source Code

using System;
using Microsoft.ComputeCluster;

namespace CCSJobSubmissionConsoleApplication
{
    class Program
    {
        static ICluster myCluster = null;
        static IJob myJob = null;
        static ITask myTask = null;
        static string clusterName = "localhost";
        static string commandLine = "cmd.exe /c helloworld";

        static void Main(string[] args)
        {
            try
            {
                //initialize the cluster
                myCluster = InitCluster(clusterName);

                //initialize the task
                myTask = InitTask(commandLine, myEnvVars);

                //initialize the job
                myJob = InitJob(myTask);

                //add the job to the cluster               
                myCluster.AddJob(myJob);

                //submit the job for execution
                myCluster.SubmitJob(myJob.Id, null, null, true, 0);
            }
            catch (Exception)
            {
                throw;
            }
        }

        private static ICluster InitCluster(string myClustername)
        {
            ICluster c = null;
            c = new Cluster();
            c.Connect(myClustername);
            return (c);
        }

        private static ITask InitTask(string commandLine)
        {
            ITask t = null;
            t = new Task();
            t.CommandLine = commandLine;
            return (t);
        }

        private static IJob InitJob(ITask myTask)
        {
            IJob j = null;
            j = new Job();
            j.AddTask(myTask);
            j.AddTask(myTask);
            return (j);
        }
    }
}