GT 3.9.5 Component Guide to Public Interfaces: RFT

Semantics and syntax of APIs

Programming Model Overview

The Reliable Transfer Service (RFT) is a WSRF based service that provides interfaces for controlling and monitoring third party file transfers using GridFTP servers. The client controlling the transfers (in this case RFT )  is hosted inside of a Grid service so it can be managed using the soft state model. It is essentially a reliable and recoverable version of the GT2 globus-url-copy tool and more. In 3.9.5 RFT can also perform file deletion, recursive directory deletion operations. It is also used by GRAM to perform all the staging operations and cleanup operations.

Component API

Some relevant API :

Semantics and syntax of the WSDL

Protocol overview

RFT Service implementation in 3.9.5 uses standard SOAP messages over HTTP to submit and manage a set of 3rd party GridFTP transfers and to delete files using GridFTP. The user creates a RFT resource by submitting a list of URL pairs of  files that need to be transferred/deleted  to RFT Factory service. The user also specifies the time to live for the resource the user is creating to a GT 3.9.5 Container in which RFT is deployed and configured. The resource is created after the user is properly authorized and authenticated. RFT service implementation exposes operations to control and manage the transfers (the resource). The operations exposed by both RFT factory and RFT service are briefly described below. The resource the user created also exposes the state of the transfer as a resource property to which the user can either subscribe for changes or poll for the changes in state periodically using standard command line clients.

Operations

RFT Factory Service

Used to create a Reliable File Transfer resource. The operations exposed by the factory are as follows:

  • createReliableFileTransfer: Creates a Reliable File Transfer Resource.
    • Input Parameters: Initial Termination time , Transfer Request or Delete Request
    • Output parameters:  Termination time, Current time, Endpoint reference of the Resource created. ( This should be stored by the user as it is needed to query the status of the resource and to perform any further operations on the resource.
    • Fault: createReliableFileTransferFault:

RFT Service

Used to manage the Resource created using the RFT Factory Service. The operations exposed by the service are as follows:

  • start: Starts executing the transfers/deletes
    • Input  Parameters: None
    • Output Parameters: None
    • Fault: RepeatedlyStartedFault:
  • getStatus: To get the status of a particular file.
    • Input Parameters:  A source URL of the file that is part of the request.
    • Output Parameters: Transfer Status Type
    • Fault: RFTDatabaseFault
  • getStatusSet : To get the status of a set of files in a request
    • Input Parameters:  int  from ( the relative position of the transfer in the request)  and int offset ( Number of files queried)
    • Output Parameters:  An array of TransferStatusType
    • Fault: RFTDatabaseFault
  • cancel:  To cancel a transfer that is part of a resource.
    • Input Parameters : int from ( the relative position of the transfer in the request ) int to
    • Output Parameters: None
    • Fault: RFTDatabaseFault

Resource properties

RFT Factory Service

  • ActiveResourceInstances: A dynamic resource property of total number of active rft resources in the container at a given point of time.
  • TotalNumberOfTransfers: A dynamic resource property of total number of transfers/deletes performed since the RFT service is deployed in this container
  • TotalNumberOfActiveTransfers: A dynamic resource property of number of active transfers across all rft resources in a container at a given point of time.
  • TotalNumberOfBytesTransferred: A dynamic resource property of total number of bytes transferred by all RFT resources created since the deployment of the service.
  • RFTFactoryStartTime: Time when the service was deployed in the container. Used to calculate uptime.
  • DelegationServiceEPR: The end point reference of the Delegation resource that holds the delegated credential used in executing the resource.

RFT Service

  • RequestStatusProperty : represents the current state of the resource ( Active, Pending, Failed, Finished). It also includes the last fault message encountered while executing the request.
  • OverallStatusProperty:  provides current state of the transfer by providing number of Transfers done, pending, active, failed, cancelled and retrying. It also contains the fault message, if any, raised during a transfer.
  • TotalBytes: provides the total number of bytes transferred by the resource
  • TotalTime: provides the total time taken to transfer the above mentioned total bytes.

Faults

RFT Factory Service

  • createReliableFileTransferFault: All the errors encountered during the creation of the RFT resource are mapped to this fault. Any security related errors are caught even before the factory and are thrown to the user/client.

RFT Service

  • RepeatedlyStartedFault: This is raised if a client calls start more than once on a resource.
  • RFTDatabaseFault: Thrown when the service is unable to find the resource the user/client is querying for.

WSDL and Schema Definition

You can find links to all the RFT schemas here.

Command-line tools

Command line tool: rft for RFT transfers

Tool description

Submits a transfer to the Reliable File Transfer Service and prints out the status of the transfer on the console.

Command syntax and options

rft  [-h <host-ip of the container defaults to localhost> 
-port <port, defaults to 8080>
-l <lifetime for the resource default 60mins> 
-m <security mechanism. 'msg' for secure message or 'conv' for 
 secure conversation and 'trans' for transport. Defaults to 
   secure transport.>
-p <protection type, 'sig' signature and 'enc' encryption, 
 defaults to signature >
-z <authorization mechanism can be self or host. default self> 
-file <file to write EPR of created Reliable  File Transfer Resource]> 
-f <path to the file that contains list of transfers>

This is a sample transfer file that the command-line client will be able to parse. It can also be found in $GLOBUS_LOCATION/share/globus_wsrf_rft_client/ along with other samples for directory transfers and deletes (lines starting with # are comments): 

#true=binary false=ascii
true
#Block size in bytes
16000
#TCP Buffer size in bytes
16000
#Notpt (No thirdPartyTransfer)
false
#Number of parallel streams
1
#Data Channel Authentication (DCAU)
true
# Concurrency of the request
1
#Grid Subject name of the source gridftp server
/DC=org/DC=doegrids/OU=People/CN=Ravi Madduri 134710
#Grid Subject name of the destination gridftp server
/DC=org/DC=doegrids/OU=People/CN=Ravi Madduri 134710
#Transfer all or none of the transfers
false
#Maximum number of retries
10
#Source/Dest URL Pairs
gsiftp://localhost:5678/tmp/rftTest.tmp
gsiftp://localhost:5678/tmp/rftTest_Done.tmp

Limitations

This command line client is very dumb and simple and does not do any intelligent parsing of various command line options and the options in the sample transfer file. It works fine if used in the way documented here.

Command-line tool: rft-delete for RFT deletes

Tool description

This command-line tool is used to submit a list of files to be deleted.

Command and options 

rft-delete [-h <host-ip of the container default localhost> 
-port <port, defaults to 8080>
-l <lifetime for the resource default 60mins> 
-m <security mechanism. 'msg' for secure message or 'conv' for 
 secure conversation and 'trans' for transport. Defaults to 
   secure transport.>
-p <protection type, 'sig' signature and 'enc' encryption, 
 defaults to signature >
-z <authorization mechanism can be self or host. default self> 
-file <file to write EPR of created Reliable  File Transfer Resource]> 
-f <path to the file that contains list of transfers>

This is a sample file that the command line client will be able to parse, it can also be found in $GLOBUS_LOCATION/share/globus_wsrf_rft_client/ along with other samples for directory transfers and deletes (lines starting with # are comments): 

# Subject name (defaults to host subject)
  /DC=org/DC=doegrids/OU=People/CN=Ravi Madduri 134710
  gsiftp://localhost:5678/tmp/rftTest_Done.tmp
  gsiftp://localhost:5678/tmp/rftTest_Done1.tmp

Limitations

No limitations with this commandline tool.

Overview of Graphical User Interface

There is no GUI for RFT service in this release.

Semantics and syntax of domain-specific interface

Please look here for information on how RFT schemas look like.

Configuration interface

Configuration overview

RFT has the following prerequisites:

  • Java WS Core - this is built and installed in a default GT 3.9.5 installation.
  • a host certificate (see Required Configuration.)
  • GridFTP Server - GridFTP perfoms the actual file transfer and is built and installed in a default GT 3.9.5 installation.
  • PostgreSQL - PostgreSQL is used to store the state of the transfer to allow for restart after failures. The interface to PostgreSQL is JDBC so any DBMS that supports JDBC can be used, although no other has been tested. For instructions on configuring the PostgreSQL database for RFT, click here.

RFT can be registered to an MDS index service to facilitiate monitoring and discovery. The MDS documentation contains a note on registering RFT to an Index Service.

Syntax of the interface

The security of the service can be configured by modifying the security descriptor. It allows for configuring the credentials that will be used by the service, type of authentication and authorization that needs to be enforced. By default, the following security configuration is installed:

  • Credentials set for use by container is used. If that is not specified, default credentials are used.
  • GSI Secure conversation authentication is enforced for all methods.

Note: Changing required authentication and authorization method will require suitable changes to the clients that contact this service.

To alter security descriptor configuration, refer to Security Descriptors. The file to be altered is $GLOBUS_LOCATION/etc/globus_wsrf_rft/security-config.xml

Environment variable interface

The only Env variable that needs to be set for RFT is GLOBUS_LOCATION in order to run the command line clients, which should be set to globus installation.