The Reliable Transfer Service (RFT) is a WSRF based service that provides interfaces for controlling and monitoring third party file transfers using GridFTP servers. The client controlling the transfers (in this case RFT) is hosted inside of a Grid service so it can be managed using the soft state model. It is essentially a reliable and recoverable version of the GT2 globus-url-copy tool and more. In GT 4.1.0, RFT can also perform file deletion and recursive directory deletion operations. It is also used by GRAM to perform all the staging operations and cleanup operations.

1.2. Component API

Some relevant APIs:

2. Semantics and syntax of the WSDL

2.1. Protocol overview

The RFT service implementation in GT 4.1.0 uses standard SOAP messages over HTTP to submit and manage a set of 3rd party GridFTP transfers and to delete files using GridFTP. The user creates an RFT resource by submitting a list of URL pairs of files that need to be transferred/deleted to the RFT Factory service. The user also specifies the time to live for the resource the user is creating to the GT 4.1.0 container in which RFT is deployed and configured. The resource is created after the user is properly authorized and authenticated. RFT service implementation exposes operations to control and manages the transfers (the resource). The operations exposed by both the RFT factory and the RFT service are briefly described below. The resource the user created also exposes the state of the transfer as a resource property to which the user can either subscribe for changes or poll for the changes in state periodically using standard command line clients.

2.2. Operations

Please find below operations of both RFT Factory and RFT Service Implementation.

2.2.1. RFT Factory Service

Used to create a Reliable File Transfer resource. The operations exposed by the factory are as follows:

createReliableFileTransfer: Creates a Reliable File Transfer resource.
- Input Parameters: Initial Termination time, Transfer Request or Delete Request.
- Output parameters: Termination time, Current time, Endpoint reference of the Resource created. This should be stored by the user, as it is needed to query the status of the resource and to perform any further operations on the resource.
- Fault: createReliableFileTransferFault.

2.2.2. RFT Service

Used to manage the Resource created using the RFT Factory Service. The operations exposed by the service are as follows:

start:Starts executing the transfers/deletes.
- Input Parameters: None
- Output Parameters: None
- Fault: RepeatedlyStartedFault
getStatus:To get the status of a particular file.
- Input Parameters: A source URL of the file that is part of the request.
- Output Parameters: Transfer Status Type
- Fault: RFTDatabaseFault
getStatusSet: To get the status of a set of files in a request.
- Input Parameters: int from (the relative position of the transfer in the request) and int offset (the number of files queried).
- Output Parameters: An array of TransferStatusType.
- Fault: RFTDatabaseFault
cancel: To cancel a transfer that is part of a resource.
- Input Parameters: int from (the relative position of the transfer in the request) and int to.
- Output Parameters: None
- Fault: RFTDatabaseFault

2.3. Resource Properties

The resource properties of RFT Factory (which acts both as a resource and a service at the same time) and RFT Resource are found below:

2.3.1. RFT Factory Resource Properties

ActiveResourceInstances: A dynamic resource property of the total number of active RFT resources in the container at a given point of time.
TotalNumberOfTransfers: A dynamic resource property of the total number of transfers/deletes performed since the RFT service was deployed in this container.
TotalNumberOfActiveTransfers: A dynamic resource property of the number of active transfers across all rft resources in a container at a given point of time.
TotalNumberOfBytesTransferred: A dynamic resource property of the total number of bytes transferred by all RFT resources created since the deployment of the service.
RFTFactoryStartTime: Time when the service was deployed in the container. Used to calculate uptime.
DelegationServiceEPR: The end point reference of the Delegation resource that holds the delegated credential used in executing the resource.

2.3.2. RFT Resource Properties

OverallStatus: This is a complex type providing the overall status of an RFT resource by providing the number of transfers pending, active, finished, retrying, failed, and cancelled. Each of these values can be obtained by invoking getTransfers(Finished/Active/Failed/Restarted/Pending/Cancelled) on OverallStatus Resource Property. Note that this Resource Property gets updated every time one of the transfers changes state, so there can be and will be more than one update in the life time of a RFT resource if you subscribe to this RP. This Resource Property also includes the last fault (if thrown) from a transfer and can be accessed by invoking getFault on OverallStatus. This will indicate why a transfer has failed.
RequestStatus: This is a complex type resource property providing the status of an RFT resource in the form of Pending/Active/Done/Failed. The status can be obtained from RequestStatusType by invoking getRequestStatus(). This will result in one of four status strings (Pending/Active/Done/Failed/Cancelled). This RP also contains a fault that denotes the last fault in a RFT resource and can be accessed by invoking getFault(). If a client is subscribed to this RP, there will be only be 2 updates in the life time of an RFT resource (Pending->Active->Done, Pending->Active->Failed, Pending->Active->Cancelled, and Pending->Cancelled).
TotalBytes: This provides the total number of bytes transferred by the resource.
TotalTime: This provides the total time taken to transfer the above-mentioned total bytes.

2.4. Faults

Faults from the RFT Factory Service and RFT Service can be found below:

2.4.1. RFT Factory Service

createReliableFileTransferFault: All the errors encountered during the creation of the RFT resource are mapped to this fault. Any security related errors are caught before the factory and are thrown to the user/client.

2.4.2. RFT Service

RepeatedlyStartedFault: This is raised if a client calls start more than once on a resource.
RFTDatabaseFault: This is thrown when the service is unable to find the resource the user/client is querying for.

2.5. WSDL and Schema Definition

You can find links to all the RFT schemas here.

3. Semantics and syntax of non-WSDL protocols

[describe other protocols. if none, state so.]

4. Command-line tools

Please see the RFT Command Reference.

5. Overview of Graphical User Interface

There is no GUI for the RFT service in this release.

6. Semantics and syntax of domain-specific interface

6.1. Request Schema

Please go here to view the entire RFT transfer request schema documentation.

6.2. Request Options

6.2.1. General Options

These options are set in the transferRequest and deleteRequest elements and apply similarly for each.

concurrency
This denotes number of files in the request that needs to be transferred at one time.
maxAttempts
Maximum number of attempts after transient errors to execute the transfer or deletion before giving up and raising an error.
finishBy
(Not Implemented) In future versions of RFT this will be used to enforce time constraints on a transfer.

6.2.2. Transfer Options

These options are set in the rftOptions element (see RFTOptionsType for more details) and are specific to file transfers. They can be specified as defaults for all transfers under the transferRequest element, and/or individually under each transfer element (see TransferType for more details):

        <transferRequest>
            <transfer>...</transfer>
            <rftOptions>
                <-- option elements here -->
            </rftOptions>
        </transferRequest>

AND/OR

        <transferRequest>
            <transfer>
                ...
                <sourceUrl>
                <destinationUrl>
                ...
                <rftOptions>
                    <-- option elements here -->
                </rftOptions>
            </transfer>
        </transferRequest>

binary
Transfer as a binary file. Default is "true".
blockSize
Specifies the size of the data blocks to use in the transfer.
tcpBufferSize
Specifies the TCP buffer size used for the transfer.
notpt
If set to "true", third-party transfer mode will not be use. Instead, a client thread will be started that will GET data from the source server and and PUT data to the destination server. Default is "false".
parallelStreams
Specifies the number of parallel streams to use during the transfer. Default is 1.
dcau
Specifies whether or not to use data channel authentication. Default is true.
subjectName
Specifies the credential subject to use for authenticating both the source and destination servers.
destinationSubjectName
Specifies the credential subject to use for authenticating the destination server.
sourceSubjectName
Specifies the credential subject to use for authenticating the source server.
userName
Specifies the username to be used to perform the transfer which sometimes may not be the same as transfer requester.

6.2.3. Deletion Options

These options are set in the deleteOptions element (see DeleteOptionsType for more details), and are specific to file deletions. They can be specified as defaults for all deletions under the deleteRequest element, and/or individually under each deletion element (see DeleteType for more details):

        <deleteRequest>
            <deletion>...</deletion>
            <deleteOptions>
                <-- option elements here -->
            </deleteOptions>
        </deleteRequest>

AND/OR

        <deleteRequest>
            <deletion>
                ...
                <file>
                <deleteOptions>
                    <-- option elements here -->
                </deleteOptions>
            </deletion>
        </deleteRequest>

subjectName
Specifies the credential subject to use for authenticating the target server.
userName
Specifies the username to be used to perform the deletion.

7. Configuration interface

7.1. Configuration overview

RFT has the following prerequisites:

Java WS Core - This is built and installed in a GT 4.1.0 System Administrator's Guide.
A host certificate (see GT 4.1.0 System Administrator's Guide).
GridFTP - GridFTP performs the actual file transfer and is built and installed in a GT 4.1.0 System Administrator's Guide.
PostgreSQL - PostgreSQL is used to store the state of the transfer to allow for restart after failures. The interface to PostgreSQL is JDBC, so any DBMS that supports JDBC can be used, although no others have been tested. For instructions on configuring the PostgreSQL database for RFT, see below. .

7.2. Syntax of the interface

The security of the service can be configured by modifying the security descriptor. It allows for configuring the credentials that will be used by the service, type of authentication and authorization that needs to be enforced. By default, the following security configuration is installed:

Credentials set for use by the container are used. If they arenot specified, default credentials are used.
GSI Secure conversation authentication is enforced for all methods.

Note: Changing the required authentication and authorization method will require suitable changes to the clients that contact this service.

To alter the security descriptor configuration, refer to security descriptor. The file to be altered is $GLOBUS_LOCATION/etc/globus_wsrf_rft/security-config.xml.

7.3. Required configuration: configuring the PostgreSQL database

PostgreSQL (version 7.1 or greater) needs to be installed and configured for RFT to work. You can either use the packages which came with your operating system (RPMs, DEBs, ...) or build from source. We used PostgreSQL version 7.3.2 for our testing and the following instructions are good for the same.

Install PostgreSQL. Instructions on how to install/configure PostgreSQL can be found here.
Configure the postmaster daemon so that it accepts TCP connections. This can be done by adding the "-o -i" switch to the postmaster script (This is either the init.d script found in /etc/init.d/postgresql or /var/lib/, depending on how you installed PostgreSQL). Follow the instructions here to start the postmaster with the -i option.
You will now need to create a PostgreSQL user that will connect to the database. This is usually the account under which the container is running. You can create a PostgreSQL user by running the following command: su postgres; createuser globus. If you get the following error: psql: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"? this generally means that either your postmaster is not started with the -i option or you didn't restart the postmaster after the above mentioned step.
Now you need to set security on the database you are about to create. You can do it by following the steps below:
sudo vi /var/lib/pgsql/data/pg_hba.conf and append the following line to the file:
host rftDatabase "username" "host-ip" 255.255.255.255 md5 Note: use crypt instead of md5 if you are using PostgreSQL 7.3 or earlier.
sudo /etc/init.d/postgresql restart
To create the database that is used for RFT run (as user globus): createdb rftDatabase.
To populate the RFT database with the appropriate schemas run: psql -d rftDatabase -f $GLOBUS_LOCATION/share/globus_wsrf_rft/rft_schema.sql. Now that you have created a database to store RFT's state, the following steps configure RFT to find the database:
Open $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml.
Find the dbConfiguration section under the ReliableFileTransferService <service> section.
Change the connectionString to point to the machine on which you installed PostgreSQL and to the name of the database you used in step 2. If you installed PostgreSQL on the same machine as your Globus install, the default should work fine for you.
Change the userName to the name of the user who owns/created the database and do the same for the password (it also depends on how you configured your database).
Don't worry about the other parameters in the section. The defaults should work fine for now.
Edit the configuration section under ReliableFileTransferService. There are two values that can be edited in this section:
- backOff: Time in seconds you want RFT to backoff before a failed transfer is retried by RFT. The default should work fine for now.
- maxActiveAllowed: This is the number of transfers the container can do at given point. The default should be fine for now.

7.4. RFT auto-registration with default WS MDS Index Service

With a default GT 4.1.0 installation, the RFT service is automatically registered with the default WS MDS Index Service running in the same container for monitoring and discovery purposes.

There is a jndi resource defined in $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml as follows :

 
  <resource name="mdsConfiguration" 
   
    type="org.globus.wsrf.impl.servicegroup.client.MDSConfiguration">
     <resourceParams>
       <parameter> 
         <name>reg</name>
         <value>true</value>
       </parameter>
       <parameter> 
         <name>factory</name>
         <value>org.globus.wsrf.jndi.BeanFactory</value>
       </parameter>
     </resourceParams>
   </resource>

To configure the automatic registration of RFT to the default WS MDS Index Service, change the value of the parameter <reg> as follows:

true turns on auto-registration; this is the default in GT 4.1.0.
false turns off auto-registration.

7.4.1. Configuring resource properties

By default, the following resource properties (from the RFT Factory Resource) are sent to the default Index Service:

ActiveResourceInstances: A dynamic resource property of the total number of active RFT resources in the container at a given point of time.
TotalNumberOfTransfers: A dynamic resource property of the total number of transfers/deletes performed since the RFT service was deployed in this container.
TotalNumberOfActiveTransfers: A dynamic resource property of the number of active transfers across all rft resources in a container at a given point of time.
TotalNumberOfBytesTransferred: A dynamic resource property of the total number of bytes transferred by all RFT resources created since the deployment of the service.
RFTFactoryStartTime: Time when the service was deployed in the container. Used to calculate uptime.
DelegationServiceEPR: The end point reference of the Delegation resource that holds the delegated credential used in executing the resource.

You can configure which resource properties are sent in RFT's registration.xml file, $GLOBUS_LOCATION/etc/globus_wsrf_rft/registration.xml. The following is the relevant section of the file:

      <Content xsi:type="agg:AggregatorContent"
        xmlns:agg="http://mds.globus.org/aggregator/types">
        
      <agg:AggregatorConfig xsi:type="agg:AggregatorConfig">
              
        <agg:GetMultipleResourcePropertiesPollType
         xmlns:rft="http://www.globus.org/namespaces/2004/10/rft">
        <!-- Specifies that the index should refresh information
            every 60000 milliseconds (once per minute) -->
          <agg:PollIntervalMillis>60000</agg:PollIntervalMillis>
                            
          <!-- specifies that all Resource Properties should be
              collected from the RFT factory -->
                            
          <agg:ResourcePropertyNames>rft:TotalNumberOfBytesTransferred</agg:ResourcePropertyNames>
          <agg:ResourcePropertyNames>rft:TotalNumberOfActiveTransfers</agg:ResourcePropertyNames>
          <agg:ResourcePropertyNames>rft:RFTFactoryStartTime</agg:ResourcePropertyNames>
          <agg:ResourcePropertyNames>rft:ActiveResourceInstances</agg:ResourcePropertyNames>
                            
          <agg:ResourcePropertyNames>rft:TotalNumberOfTransfers</agg:ResourcePropertyNames>
                      
        </agg:GetMultipleResourcePropertiesPollType>
      </agg:AggregatorConfig> 
      <agg:AggregatorData/>
      </Content>

7.5. Registering RFT manually with default WS MDS Index Service

If a third party needs to register an RFT service manually, see Registering with mds-servicegroup-add in the WS MDS Aggregator Framework documentation.

8. Environment variable interface

The only environment variable that needs to be set for RFT is GLOBUS_LOCATION, in order to run the command line clients, which should be set to the location of the globus installation.

Prev	Up	Next
GT 4.1.0 Reliable File Transfer (RFT) Service: Developer's Guide	Home	GT 4.1.0 RFT: Quality Profile