CCPAPI.DLL Basics
Dennis Crain, Program Manager
Windows HPC team
May 2006
Contents
Overview
Interfaces
Job-Task Relationship
Using CCPAPI
Writing the Code
CCPAPI.DLL provides access to the Windows Compute Cluster Server 2003 (CCS) scheduler. This DLL provides COM interfaces to numerous objects within the CCS scheduler. By writing applications or scripts using these interfaces you may connect to a cluster and manage jobs, job resources, tasks, nodes, environment variables, extended job terms and much more. This article will describe the basic functionality of CCPAPI.DLL and provide guidance in writing your first serial job scheduling application using the DLL and C#.
Eleven interfaces are provided by CCPAPI including:
- ICluster
- IClusterCounter
- IClusterEnumerable
- IExecutionResult
- IJob
- IJboCounter
- INameValue
- INameValueCollection
- INode
- IResourceUsage
- ITask
As you write scheduler applications and/or scripts you will find that four of these interfaces are primary to the task (no pun intended) at hand including ICluster, ITask, Ijob, and INode. The other seven support these primary interfaces.
Before proceeding it is important to thoroughly understand the relationship of tasks to jobs. The following description of jobs and tasks is excerpted from the CCS SDK documentation.
A task represents the execution of a single process or multiple processes on a compute node. A collection of tasks that is used to perform a computation is known as a job. Jobs are used to reserve the resources required by tasks. The job may be thought of as the parent of a task.
The following illustrates the job life cycle:
Figure 1. Job Life Cycle
AddJob()
sends the job to the server but the job state, expressed as part of the enumeration JobStatus,
remains NotSubmitted
. You can add tasks to the job and then submit the job using SubmitJob()
. If the job is successfully submitted, the job state transitions to Queued
.
When resources are allocated to a job, the job state transitions to Running
. When a job is completed, its allocated resources are released and its job state changes to Finished
, Failed
, or Cancelled
.
The following illustrates the task life cycle super-imposed over the parent job life cycle:
Figure 2: Job-Task Relationship
All tasks that are associated with a job are queued or submitted when the parent job is queued or submitted. When resources are allocated to a task, its task state, expressed as part of the enumeration TaskStatus
, changes to Running. When a task is completed, its allocated resources are released and its task state changes to Finished
, Failed
, or Cancelled
.
Using CCPAPI, in its simplest terms, is a five-step process.
- Connect to the cluster.
- Create a job.
- Create a task.
- Add the task to the job.
- Submit the job/task for execution.
To connect to the cluster use the ICluster::Connect
method. Create a job using the ICluster::CreateJob
method. To create a task, use the ICluster::CreateTask
method. To add a child task to a job, use the ICluster::AddTask
method. Finally, submit the job using ICluster::SubmitJob()
.
This sample, written in C#, demonstrates the five steps described above. The code submits a serial job. There are other types of jobs such as workflow, parameter sweeps, and parallel. Future articles will describe these uses of CCPAPI.
If you would like to code this sample you must first install Visual Studio 2005 and the Microsoft Compute Cluster Pack SDK on a 32- or 64-bit computer. If you wish to execute the program you must have Microsoft Windows Compute Cluster Server 2003 installed on at least one 64-bit computer or a cluster of 64-bit machines. If you are using one 64-bit machine, configure it as a head and compute node during the installation of CCS.
This code illustrates a console application that, after having connected to the cluster, creates and submits a job with the task cmd.exe /c helloworld
.
You must add CCPAPI.DLL as a reference to your application. In Visual Studio, this is done by choosing "Add Reference" from the Project menu. Then add the using Microsoft.ComputeCluster
statement to the code as follows:
using Microsoft.ComputeCluster;
In this sample, the code requires the ICluster
, Ijob,
and ITask
interfaces. If you needed other CCS interfaces you would declare then in a similar manner. The last two variables are for convenience only.
static ICluster myCluster;
static IJob myJob;
static ITask myTask;
static string clusterName = "localhost";
static string commandLine = "cmd.exe /c helloworld";
The following Main function follows the five-step process for using CCPAPI. Connect to the cluster, create a task, create a job, add the task to the job, and submit the job.
static void Main(string[] args)
{
try
{
//initialize the cluster
myCluster = InitCluster(clusterName);
//initialize the task
myTask = InitTask(commandLine, myEnvVars);
//initialize the job
myJob = InitJob(myTask);
//add the job/task to the cluster
myCluster.AddJob(myJob);
//submit the job for execution
myCluster.SubmitJob(myJob.Id, null, null, true, 0);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
Initializing the cluster, in this example, is really nothing more than connecting to the cluster. The cluster name is the machine name of the headnode of your cluster. If you are executing the application on the headnode then using "localhost" is sufficient.
private static ICluster InitCluster(string myClustername)
{
ICluster c = null;
c = new Cluster();
c.Connect(myClustername);
return (c);
}
Recall that the task is a process or collection of processes that which you wish to execute on a compute node. In the case of this sample, the process is cmd.exe /c helloworld
. This process is assigned to the CommandLine
property in the task object.
private static ITask InitTask(string commandLine)
{
ITask t = null;
t = new Task();
t.CommandLine = commandLine;
return (t);
}
This function takes the task object as a parameter. After the job is created the task is added to the job. Upon returning from this function the job is ready to be submitted for execution.
private static IJob InitJob(ITask myTask)
{
IJob j = null;
j = new Job();
j.AddTask(myTask);
return (j);
}
using System;
using Microsoft.ComputeCluster;
namespace CCSJobSubmissionConsoleApplication
{
class Program
{
static ICluster myCluster = null;
static IJob myJob = null;
static ITask myTask = null;
static string clusterName = "localhost";
static string commandLine = "cmd.exe /c helloworld";
static void Main(string[] args)
{
try
{
//initialize the cluster
myCluster = InitCluster(clusterName);
//initialize the task
myTask = InitTask(commandLine, myEnvVars);
//initialize the job
myJob = InitJob(myTask);
//add the job to the cluster
myCluster.AddJob(myJob);
//submit the job for execution
myCluster.SubmitJob(myJob.Id, null, null, true, 0);
}
catch (Exception)
{
throw;
}
}
private static ICluster InitCluster(string myClustername)
{
ICluster c = null;
c = new Cluster();
c.Connect(myClustername);
return (c);
}
private static ITask InitTask(string commandLine)
{
ITask t = null;
t = new Task();
t.CommandLine = commandLine;
return (t);
}
private static IJob InitJob(ITask myTask)
{
IJob j = null;
j = new Job();
j.AddTask(myTask);
j.AddTask(myTask);
return (j);
}
}
}