GT 5.0.1 User's Guide

The grid-cert-diagnostics program checks prints diagnostics about the user's certificates, and host security environment.

% grid-cert-diagnostics -p

openssl verify -CApath /etc/grid-security/certificates
  -purpose sslclient ~/.globus/usercert.pem

openssl s_client -ssl3 -cert ~/.globus/usercert.pem -key 
  ~/.globus/userkey.pem -CApath /etc/grid-security/certificates 
  -connect <host:port>

Here <host:port> denotes the server and port you connect to.

If it prints an error and puts you back at the command prompt, then it typically means that the server has closed the connection, i.e. that the server was not happy with the client's certificate and verification. Check the SSL log on the server.

If the command "hangs" then it has actually opened a telnet style (but secure) socket, and you can "talk" to the server.

You should be able to scroll up and see the subject names of the server's verification chain:

depth=2 /DC=net/DC=ES/O=ESnet/OU=Certificate Authorities/CN=ESnet Root CA 1
verify return:1
depth=1 /DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1
verify return:1
depth=0 /DC=org/DC=doegrids/OU=Services/CN=wiggum.mcs.anl.gov
verify return:1
    

In this case, there were no errors. Errors would give you an extra line next to the subject name of the certificate that caused the error.

Requires root login on server:

    openssl verify -CApath /etc/grid-security/certificates -purpose sslserver 
     /etc/grid-security/hostcert.pem

One of the most basic tasks in GridFTP is to "put" files, i.e., moving a file from your file system to the server. So for example, if you want to move the file /tmp/foo from a file system accessible to the host on which you are running your client to a file name /tmp/bar on a host named remote.machine.my.edu running a GridFTP server, you would use this command:

globus-url-copy -vb -tcp-bs 2097152 -p 4 file:///tmp/foo gsiftp://remote.machine.my.edu/tmp/bar

	Note
	In theory, remote.machine.my.edu could be the same host as the one on which you are running your client, but that is normally only done in testing situations.

A get, i.e, moving a file from a server to your file system, would just reverse the source and destination URLs:

	Tip
	Remember file: always refers to your file system.

globus-url-copy -vb -tcp-bs 2097152 -p 4 gsiftp://remote.machine.my.edu/tmp/bar file:///tmp/foo

Finally, if you want to move a file between two GridFTP servers (a third party transfer), both URLs would use gsiftp: as the protocol:

globus-url-copy -vb -tcp-bs 2097152 -p 4 gsiftp://other.machine.my.edu/tmp/foo gsiftp://remote.machine.my.edu/tmp/bar

If you want more information and details on URLs and the command line options, the Key Concepts gives basic definitions and an overview of the GridFTP protocol as well as our implementation of it.

This example shows how to check that an RSL document contains a syntactically correct job description. Note that this mode does not do semantic validation of the RSL, so an RSL document that passes this test may not work when submitted to a GRAM5 service.

% globusrun -p "&(executable=a.out)"

RSL Parsed Successfully...

% globusrun -p "&/executable=a.out)"

ERROR: cannot parse RSL &/executable=/adfadf/adf /adf /adf)

Syntax: globusrun [-help] [-f RSL file] [-s][-b][-d][...] [-r RM] [RSL]


Use -help to display full usage

This example shows how to check that a globus-gatekeeper is running at a particular contact and that the client and service have mutually-trusted credentials.

Example 4.11. GRAM Authentication test

% globusrun -a -r grid.example.org:2119/jobmanager-pbs
GRAM Authentication test successful
% globusrun -a -r grid.example.org:2119/jobmanager-lsf
GRAM Authentication test failure: the gatekeeper failed to find the requested service
% globusrun -a -r grid.example.org:2119/jobmanager-pbs:host@not.example.org
GRAM Authentication test failure: an authorization operation failed
globus_xio_gsi: gss_init_sec_context failed.
GSS Major Status: Unexpected Gatekeeper or Service Name
globus_gsi_gssapi: Authorization denied: The name of the remote host
(host@not.example.org), and the expected name for the remote host
(grid.example.org) do not match. This happens when the name in the host
certificate does not match the information obtained from DNS and is often a DNS
configuration problem.
                

	Note
	The DNS configuration problem was a common issue in GRAM2, but GRAM5 will not depend on DNS to resolve names for mutual authentication.

This example shows how to determine what software version of GRAM5 is deployed at a particular service contact.

Example 4.12. GRAM version check

% globusrun -j -r grid.example.org:2119/jobmanager-pbs:host@not.example.org
Toolkit version: 4.3.0-HEAD
Job Manager version: 10.5 (1256257907-0)
                

	Note
	This example shows the version number for an unreleased development version of GRAM5. The actual numbers returned will be different.

	Note
	This feature is new in GRAM5. When contacting a GRAM2 service, globusrun will display the following error message: GRAM version check failed : an incoming HTTP message did not contain the expected information

This example shows how to submit interactive job with globusrun. When the -s is used, the output of the job command is returned to the client and displayed as if the command ran locally. This is similar to the behavior of the globus-job-run program described above.

% globusrun -s -r example.grid.org/jobmanager-pbs "&(executable=/bin/hostname)(count=5)"
node03.grid.example.org
node01.grid.example.org
node02.grid.example.urg
node05.grid.example.org
node04.grid.example.org

This example shows how to submit, monitor, and cancel a batch job using globusrun. This method is useful for the case where the job may run for a long time, the job may be queued for a long time, or when there are network reliability issues between the client and service.

% globusrun -b -r grid.example.org:2119/jobmanager-pbs "&(executable=/bin/sleep)(arguments=500)"
globus_gram_client_callback_allow successful
GRAM Job submission successful
https://grid.example.org:38824/16001608125017717261/5295612977486019989/
GLOBUS_GRAM_PROTOCOL_JOB_STATE_PENDING
% globusrun -status https://grid.example.org:38824/16001608125017717261/5295612977486019989/
PENDING
% globusrun -k https://grid.example.org:38824/16001608125017717261/5295612977486019989/
% 

The following example shows how to refresh the credential used by a job manager and a job.

Example 4.15. Refreshing a Credential

% globusrun -refresh-proxy https://grid.example.org:38824/16001608125017717261/5295612977486019989/
% echo $?
0

	Note
	In GT 5.0.1, globusrun does not print any diagnostics when given the -refresh-proxy command-line option. Therefore, check the exit code as above to ensure that the refresh is successful.

When the Job Manager's credential is about to expire, it sends a message to all clients registered for GLOBUS_GRAM_PROTOCOL_JOB_STATE_FAILED notifications that the job manager is terminating and that the job will continue to run without the job manager.

Any client which receives such a message can (if necessary) generate a new proxy as described above and then submit a restart request to start a job manager with a new credential. This job manager will resume monitoring the jobs which were started prior to proxy expiration.

In this example, the globusrun displays an error message when the job manager's proxy is about to expire. The user creates a new proxy and resumes monitoring the job with globusrun.

% globusrun -r grid.example.org "&(executable=a.out)"
globus_gram_client_callback_allow successful
GRAM Job submission successful
GLOBUS_GRAM_PROTOCOL_JOB_STATE_ACTIVE
GLOBUS_GRAM_PROTOCOL_JOB_STATE_FAILED
GRAM Job failed because the user proxy expired (job is still running) (error code 131)
% grid-proxy-init
Your identity: /DC=org/DC=example/OU=grid/CN=Joe User
Enter GRID pass phrase for this identity:
Creating proxy ........................................................................... Done
Your proxy is valid until: Tue Nov 10 04:25:03 2009
% globusrun -r grid.example.org "&(restart="https://grid.example.org:1997/16001700477575114131/5295612977486005428/)"
globus_gram_client_callback_allow successful
GRAM Job submission successful
GLOBUS_GRAM_PROTOCOL_JOB_STATE_ACTIVE
GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE

In addition to the standard output and error stream output done by globusrun, GRAM5 can do basic file management tasks to stage files to the GRAM5 service node before submitting a job and to stage files from the GRAM5 service node to a file service after the job completes.

GRAM5 file staging supports four URL schemes: ftp, gsiftp, http, and https. Note, that for the https scheme, GRAM expects the file server to be running with the same identity as the client.

General file staging is controlled by three RSL attributes: file_stage_in, file_stage_in_shared, and file_stage_out. In addition, the files named by the RSL attributes executable, stdin may be staged in and the files named by the RSL attributes stdout and stderr may be staged out.

The file_stage_in_shared RSL attribute instructs GRAM to store a local copy of the resource named by the URL in the GASS cache. This is useful if multiple concurrent jobs will be accessing one or more common files. The GASS cache will manage a reference count for files in the cache and remove them when all jobs that refer to them complete.

The following example shows how to stage a few files from a GridFTP server to the GRAM node. It uses the rsl_substitution mechanism to define a subsitution variable to reduce the amount of redundancy in the job description.

% globusrun -s -r grid.example.org:2119/jobmanager-pbs \
    "&(rsl_substitution = (GRIDFTP_SERVER gsiftp://gridftp.example.org)) \
      (executable=/bin/ls)
      (arguments=/tmp/staged_file)
      (file_stage_in = ($(GRIDFTP_SERVER)/staged_file /tmp/staged_file))"
/tmp/staged_file

The next example uses the file_stage_in_shared RSL attribute to stage a file into the cache. The file is transferred from the client using the GASS https server embedded in the globusrun program when the -s option is used.

% globusrun -s -r grid.example.org:2119/jobmanager-pbs \
    "&(executable=/bin/ls) \
      (arguments = -l /tmp/staged_file_link1 /tmp/staged_file_link1) \
      (file_stage_in_shared = \
          (\$(GLOBUSRUN_GASS_URL)/staged_file1 /tmp/staged_file_link1))"
lrwxr-xr-x  1 juser   juser  120 Nov 11 20:37 /tmp/staged_file1 -> /home/juser/.globus/.gass_cache/local/md5/ff/771bded8a2c7dacc1a1c0fecafa0ce/md5/39/13ab3db7fc002ed54012083ae6ed1c/data

The final staging example uses the file_stage_out RSL attribute to transfer a file from the GRAM service to an FTP server using anonymous FTP

% globusrun -r grid.example.org:2119/jobmanager-pbs \
    "&(executable=a.out) \
      (file_stage_out = (results.txt ftp://anonymous:nopass@ftp.example.org/incoming/results.txt))"
% 

	Note
	In all of the above cases, multiple files may be staged using any combination of the supported URL schemes.

GRAM5 supports creating a per-job scratch directory which can be used as a place to store files that will be automatically removed by GRAM when the job completes. It also supports an explicit list of files to remove when the job completes.

This example shows how to stage files into a scratch directory. It again uses the embedded GASS https server, stages to the GRAM service, then runs /bin/ls in the temporary directory. After the job completes, the contents of $(SCRATCH_DIRECTORY) and the directory itself are removed.

% globusrun -s grid.example.org:2119/jobmanager-pbs \
    "&(scratch_dir = \$(HOME)) \
      (directory = \$(SCRATCH_DIRECTORY))
      (file_stage_in = \
          (\$(GLOBUSRUN_GASS_URL)/inputfile $(SCRATCH_DIRECTORY)/inputfile)) \
      (executable = /bin/ls)"
inputfile

This example shows how to explicitly remove a file that was created by the job.

% globusrun -s grid.example.org:2119/jobmanager-pbs \
    "&(executable = /bin/touch) \
      (arguments = temporary_file) \
      (file_clean_up = temporary_file)"
% 

The globusrun command supports a two-phase commit protocol to ensure that the client knows the contact of the job which has been created so that it can be monitored or canceled in the case of a client or service error. The two-phase commit affects both job submission and termination.

The two-phase protocol is enabled by using the two_phase RSL attribute, as in the next example. When this is enabled, job submission will fail with the error GLOBUS_GRAM_PROTOCOL_ERROR_WAITING_FOR_COMMIT. The client must respond to this signal with either the GLOBUS_GRAM_PROTOCOL_JOB_SIGNAL_COMMIT_REQUEST or GLOBUS_GRAM_PROTOCOL_JOB_SIGNAL_COMMIT_EXTEND signals to either commit the job to execution or delay the commit timeout. One of these signals must be sent prior to the two phase commit timeout, or the job will be discarded by the GRAM service.

A two phase protocol is also used at job termination if the save_state RSL attribute is used along with the two_phase attribute. When the job manager sends a callback with the job state set to GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE or GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE it will wait to clean up the job until the two phase commit occurs. The client must reply with the GLOBUS_GRAM_PROTOCOL_JOB_SIGNAL_COMMIT_END signal to cause the job to be cleaned. Otherwise, the job will be unloaded from memory until a client restarts the job and sends the signal.

% globusrun -r grid.example.org:2119/jobmanager-pbs \
    "&(two_phase = 30) \
      (save_state = yes) \
      (executable = a.out)"

globus_gram_client_callback_allow successful
GRAM Job submission successful
GLOBUS_GRAM_PROTOCOL_JOB_STATE_PENDING
GLOBUS_GRAM_PROTOCOL_JOB_STATE_ACTIVE
GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE
% 

If a job manager or client exits before a job has completed, the job will continue to run. The client can reconnect to a job manager and receive job state notifications and output using the restart RSL attribute.

% globus-job-submit grid.example.org:2119/jobmanager-pbs /bin/sleep 90
https://grid.example.org:38824/16001746665595486521/5295612977486005662/
% globusrun -r grid.example.org:2119/jobmanager-pbs \
    "&(restart = https://grid.example.org:38824/16001746665595486521/5295612977486005662/)"
globus_gram_client_callback_allow successful
GRAM Job submission successful
GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE
% 

To submit a job that runs a java program, the client must ensure that the job can find the Java interpreter and its classes. This example sets the default PATH and CLASSPATH environment variables and uses the shell to locate the path to the java program.

% globusrun -r grid.example.org:2119/jobmanager-pbs \
    "&(environment = (PATH '/usr/bin:/bin') (CLASSPATH \$(SCRATCH_DIRECTORY)))
      (scratch_dir = \$(HOME)) 
      (directory = \$(SCRATCH_DIRECTORY))
      (rsl_substitution = (JAVA_SERVER http://java.example.org))
      (file_stage_in = 
          (\$(JAVA_SERVER)/example.jar \$(SCRATCH_DIRECTORY)/example.jar) 
          (\$(JAVA_SERVER)/support.jar \$(SCRATCH_DIRECTORY)/support.jar))
      (executable=/bin/sh)
      (arguments=-c 'java -jar example.jar')"
globus_gram_client_callback_allow successful
GRAM Job submission successful
GLOBUS_GRAM_PROTOCOL_JOB_STATE_PENDING
GLOBUS_GRAM_PROTOCOL_JOB_STATE_ACTIVE
GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE
%