Globus Toolkit 3.9.2 Development Documentation: GRAM

GRAM Web Services Interfaces

This page provides information about the web services interfaces in WS GRAM:

Introduction

The key concepts for the GRAM component have not changed. Its purpose is still to provide the mechanisms to execute remote applications for a user. Given an RSL (Resource Specification Language) job description, GRAM submits the job to a scheduling system such as PBS or Condor, or to a simple fork-based way of spawning processes, and monitors it until completion. More details can be found here:

http://www-unix.globus.org/toolkit/docs/3.2/gram/key

WSRF GRAM Web Services

The GRAM services in GT4 are WSRF compliant. One of the key concepts in the WSRF specification is the decoupling of a service with the public "state" of the service in the interface via the implied resource pattern. Following this concept, the data of GT4 GRAM jobs is published as part of WSRF resources, while there is only one service to start jobs or query and monitor their state. This is different from the OGSI model of GT3 where each job was represented as a separate service. There still is a job factory service that can be called in order to create job instances (represented as WSRF resources). Each scheduling system that GRAM is interfaced with is represented as a separate factory resource. By making a call to the factory service while associating the call to the appropriate factory resource, the job submitting actor can create a job resource mapping to a job in the chosen scheduling system.

The WSRF GRAM service deployment therefore consists of two deployed services:

ManagedJobFactoryService (MJFS)

A single MJFS is used to create all jobs for all users. For each local resource manager, a dedicated Managed Job Factory Resource (MJFR) enables the MJFS to publish information about the characteristics of the compute resource, for example:

  • host information
  • GridFTP URL (for file staging and streaming)
  • compute cluster size and configuration, and so on...

The ManagedJobFactoryPortType GRAM-specific operation is:

createManagedJob

The input of this operation consists of a job description and an optional initial termination time for the MJR. This operation creates a MJR and replies with an endpoint reference (EPR) that:

  • is qualified with the identifier to the newly created MJR
  • points to the MJS.

The returned EPR enables the submitting user to obtain a resource-qualified reference to the MJS and start the job (see ManagedJobService (MJS))

The ManagedJobFactoryPortType also has all the operations and publishes all the resource properties (via the MJFR) defined in the following WS-ResourceProperties port types:

  • GetResourceProperty
  • GetMultipleResourceProperties
  • QueryResourceProperties

For more details, see the WSDL definition:

http://www-unix.mcs.anl.gov/~smartin/gt4/managed_job_factory_port_type_compact.wsdl

ManagedJobService (MJS)

A single MJS is used to manage all jobs for all users. Each Managed Job Resource (MJR) enables the MJS to publish information about the individual job the MJR represents. This information can be accessed by querying the MJS for the resource properties of a given MJR, such as the:

  • current job state
  • stdout location
  • stderr location
  • exit code, and so on.

The ManagedJobPortType GRAM-specific operation is:

start

which takes no parameters and submits the job to the back-end scheduling system.

The ManagedJobPortType also has all the operations and publishes all the resource properties (via the MJFR) defined in the following port types:

WS-ResourceProperties port types:

  • GetResourceProperty
  • GetMultipleResourceProperties
  • QueryResourceProperties

WS-ResourceLifetime port types:

  • ScheduledResourceTermination
  • ImmediateResourceTermination

WS-BaseNotification port type:

  • NotificationProducer

For more details, see the WSDL definition:

Interacting with the GRAM services

The following diagram describes a typical interaction with the GRAM service:

The following table describes the steps in the above process:

1 The job submitter obtains information about the host and the scheduling system by querying the corresponding ManagedJobFactoryResource (in this example, the "Fork" resource) for resource properties.
2 The job submitter asks the factory service to create a new job resource according to a supplied job description.
3 The job submitter subscribes for notifications on changes to the job state.
4 The job submitter starts the job.
5 The job resource notifies its remote listener(s) of state changes.
6

The job submitter destroys the job resource when it no longer needs it.

Resource Specification Language (RSL)

The RSL is an XML schema defined language used to describe a GRAM job to be executed by the GRAM service. It's a general means to describe almost any job. The GRAM client reads an RSL file and unmarshalls it into a job description that it then sends to the MJFS.

The RSL-written job description is translated by the GRAM service implementation into commands understood by the local resource manager or batch scheduling system specified by the client.

The biggest change to the RSL schema from version 3.2 of the Globus Toolkit is that the substitution definitions and references were removed. This was changed in order to use stronger typing for the job attributes and enable standard XML parsing and serialization tools to validate the values against the schema, if needed. The purpose of the RSL substitutions was for the client to be able to create a generic RSL that could be used to submit jobs to different GRAM services at different sites, and the GRAM service would substitute in the site unique values. For example, a user's home directory is not always the same at different sites; the GRAM service RSL substitution variable HOME made it so the user does did not have to specify a hard-coded path specific to each targetted system. The functionality previously provided by RSL substitutions will be a new requirement for GRAM clients to perform, except for the HOME substitution, which is still supported as the variable ${GLOBUS_USER_HOME}. GRAM service defined substitution values will be published as MJFS Resource Properties, but a GRAM client will need to query for these values and edit the RSL accordingly before submitting the job.

For more information about the RSL schema, go to:

http://www-unix.mcs.anl.gov/~smartin/gt4/rsl_schema.html

RSL Examples

Please see the Testing page for RSL examples.

GRAM Command-line Client

This command, managed-job-globusrun, is used to submit jobs to Globus resources. The job startup is done using the GRAM services. Also, the GASS service can be used to provide access to remote files and for redirecting standard output streams. For more information, see Command line tool: managed-job-globusrun.

GRAM Client API

When the supplied client is not enough to satisfy partcular needs, it is possible to write a dedicated client using the GRAM client API. This API adds an abstraction layer on top of the service proxies generated by the tooling from the WSDL schemas. The main class of the client API is org.globus.exec.client.GramJob, which represents a remote job.

Classes of the GRAM client API:

http://www-unix.globus.org/api/javadoc-3.9.2-gram/client/

Stubs generated by the tooling from the WSDL and XML schemas:

http://www-unix.globus.org/api/javadoc-3.9.2-gram/common/

Related Documents