GT 3.9.5 RFT : Developer's Guide

Introduction

RFT Service implementation in GT 3.9.5 uses standard SOAP messages over HTTP to submit and manage a set of 3rd party GridFTP transfers and to delete files using GridFTP. The user creates a RFT resource by submitting a list of URL pairs of  files that need to be transferred/deleted  to RFT Factory service. The user also specifies the time to live for the resource the user is creating to a GT 3.9.5 container in which RFT is deployed and configured. The resource is created after the user is properly authorized and authenticated. RFT service implementation exposes operations to control and manage the transfers (the resource). The operations exposed by both RFT factory and RFT service are briefly described below. The resource the user created also exposes the state of the transfer as a resource property to which the user can either subscribe for changes or poll for the changes in state periodically using standard command line clients.

Architecture and design overview

A design doc can be found here.

Public interface

The semantics and syntax of the APIs and WSDL for the component, along with descriptions of domain-specific structured interface data, can be found in the public interface guide.

Usage scenarios

  • Scenario 1: You have a large dataset and you want to make sure all the files are transferred. RFT is used to reliably transfer the data.
  • Scenario 2: RFT is also used to perform staging operations in GRAM and to remove files and directories on a gridftp server.

Tutorials

There are no tutorials available at this point.

Feature summary

Features new in release 3.9.5

  • Delete files : Delete a set of files/directories on a GridFTP server.
  • Exponential Backoff: Configurable exponential back off  before a failed transfer is retried
  • Transfer All or None: If this option is set and one of the transfers in the request fails RFT will stop transferring the remainder of the request and delete the files that were already transferred successfully.
  • Transfer Permissions :  File permissions are restored at the destination once the file is transffered successfully. This can be configured to throw a fatal error or a transient error.
  • Configurable number of concurrent transfers per container and per request.
  • Better Error reporting and Faults.
  • Database purge of the request and transfers after life time expiration.
  • Cumulative (aggregate ) Resource Properties on the factory provide some statistical information.
  • One status Resource Property for the entire transfer.

Other Supported Features

  • Recursive directory transfers and deletes.
  • Parallel streams
  • TCP Buffer Size
  • third-party transfers
  • Data channel authentication (DCAU)

Deprecated Features

  • None

Tested platforms

Tested platforms for RFT:

  • Linux

RFT is currently being tested on:

  • Windows
  • Mac OS X

Backward compatibility summary

Protocol changes since GT version 3.2

  • Added All or None option, maximum attempts, finishBy to transfer request
  • Not backwards compatible with OGSI version

API changes since GT version 3.2

  • None

Exception changes since GT version 3.2

  • None

Schema changes since GT version 3.2

  • WSDL changes to work with new Java WS Core

Technology dependencies

RFT depends on the following GT components:

  • Java WS Core
  • WS Authentication and Authorization
  • Delegation Service
  • Service Groups
  • MDS useful RP

RFT depends on the following 3rd party software:

  • PostgreSQL 7.1 version or later. Not tested with 8.0 yet.

Security considerations

Permissions of service configuration files

The service configuration files such as jndi-config.xml or server-config.wsdd (located under etc/<gar>/ directory) contains private information such as database passwords and username. Ensure that these configuration files are only readable by the user that is running the container.

The deployment process automatically sets the permissions of jndi-config.xml and server-config.wsdd files as user readable only. However, this might not work correctly on all platforms and this does not apply to any other configuration files.

Access of information stored in the database

RFT stores the transfer request in a database. Proper security measures need to be taken to protect the access of the data by granting/revoking appropriate permissions on tables that are created for RFT use and other steps that are appropriate and consistent with site specific security measures.

Permissions of persistent data

RFT uses subscription persistence API from GT4 core to store all of its subscription data under the ~/.globus/pe rsisted directory. Ensure that the entire ~/.globus/persisted directory is only readable by the user running the container.

Permissions of user's delegated proxy credential file

User's delegated proxy is stored in a file in /tmp directory ( on the host running the container ) and appropriate permissions are set on it so it can be accessed only by user running the container. This file is removed by the container after the expiration of RFT resource time to live. If it is removed manually before the expiration of resource lifetime all the transfers will fail. The transfers will also fail to start if there is not enough free space in /tmp dir to store the delegated credential.

Debugging

A standard way to debug RFT is to make container print out more verbose error messages. You can do that by doing the following steps:

Edit $GLOBUS_LOCATION/container-log4j.properties and add following line to it: log4j.category.org.globus.transfer=DEBUG . For more verbosity add log4j.category.org.globus.ftp=DEBUG which will print out Gridftp messages too.

Troubleshooting

Database configuration is the most complicated and important part of RFT setup. You can find more instructions on troubleshooting it here.

Related Documentation