GT 3.9.4 Development Release Notes for WS GRAM

Component Overview

Web Services Grid Resource Allocation and Management (WS GRAM) component comprises a set of WSRF-compliant Web services to locate, submit, monitor, and cancel jobs on Grid computing resources. WS_GRAM is not a job scheduler, but rather a set of services and clients for communicating with a range of different local job schedulers using a common protocol. WS GRAM is meant to address a range of jobs where reliable operation, stateful monitoring, credential management, and file staging are important.

Feature Summary

Features new in release 3.9.4

  • Improved service performance:
    • Job concurrency
    • Throughput
    • Latency
  • Improved service reliability/recovery
  • Support for mpich-g2 jobs:
    • multi-job submission capabilites
    • ability to coordinate processes in a job
    • ability to coordinate subjobs in a multi-job
  • Publishing of the job's exit code
  • The ability to select the account under which the remote job will be run. If a user's grid credential is mapped to multiple accounts, then the user can specify, in the RSL, under which account the job should be run.

Other Supported Features

  • Remote job execution and management
  • Uniform and flexible interface to batch scheduling systems
  • File staging before and after job execution

Deprecated Features

  • Service managed data streaming of job's stdout/err during execution.
  • File staging using the GASS protocol
  • File caching of stages files, e.g. GASS Cache

Bug Fixes

  • Bugzilla url here
  • ...
  • Bugzilla url here

Known Problems

Technology Dependencies

GRAM depends on the following GT components:

  • Java WS Core
  • Transport-Level Security
  • Delegation Service
  • RFT
  • GridFTP
  • MDS - internal libraries

GRAM depends on the following 3rd party software. The dependency exists only for the batch schedulers configured, thus making job submissions possible to the batch scheduling service:

  • PBS
  • Condor
  • LSF
  • other batch schedulers... (where the GRAM scheduler interface has been implemented)

Supported Platforms

Tested platforms for GRAM:

  • Linux

Backward Compatibility Summary

Protocol changes since GT version 3.2:

  • The protocol has been changed to be WSRF compliant. There is no backward compatibility between this version and any previous versions.

API changes since GT version 3.2:

  • The MJFS create() operation has become createManagedJob() and, now provides the option to send a uuid. A client can use this uuid to recover a job handle in the event that the reply message is not received. Given this new method, the start() operation can be removed. This operation also allows a notification subscription to specified. Without the start() operation, this is the only way to reliably get all job state notifications.
  • The MJS start() operation has been removed. Its purpose was to ensure that the client had recieved the job handle prior to the job being submitted (and thus consuming resources), and is redundant with the uuid functionality.

Exception changes since GT version 3.2:

  • list the new/changed exceptions here...

Schema changes since GT version 3.2. See the 3.9.4 User's Guide for more information about the new RSL syntax:

  • Executable is now a single local file path. Remote URLs are no longer allowed. If executable staging is desired, it should be added to the fileStageIn directive.
  • stdin is now a single local file path. Remote URLs are no longer allowed. If stdin staging is desired, it should be added to the fileStageIn directive.
  • stdout is now a single local file path, instead of a list of remote URLs. If stdout staging is desired, it should be added to the fileStageOut directive.
  • stderr is now a single local file path, instead of a list of remote URLs. If stderr staging is desired, it should be added to the fileStageOut directive.
  • scratch_directory has been removed.
  • GramMyJobType has been removed. "Collective" functionality is always available if a job chooses to use it.
  • File Staging related RSL attributes have been replaced with RFT file stransfer attributes/syntax.
  • RSL substitution definitions and substitution references have been removed in order to be able to use standard XML parsing/serialization tools.
  • RSL variables have been added. These are keywords denoted in the form of ${variable name} that can be found in certain RSL attributes.
  • Explicit credential references have added, which, along with use of the new DelegationFactory service, replace the old implicit delegation model.

For More Information

Click here for more information about this component.