Globus Toolkit 3.2: Installation Guide
Configuring RLS - Basic
This procedure includes the steps required to set up an RLS server. Post setup configuration (tuning the server parameters etc) are not included in this document.
This procedure includes the following sections:
If using PostgreSQL:
If using MySQL:
- Installing the RLS Server
- Configuring the RLS Database
- Configuring the RLS Server
- Starting the RLS Server
- Configuring RLS for WS Index Service
- Testing RLS Installation
- Installing the RLS Client
- Redhat 9 Incompatibility
You need to download and install the following software (follow the links to download):
The following environment variables can be used to override the default locations. These should be set prior to installing the RLS server.
The location of iODBC and the odbc.ini file must be specified before installing the RLS server. Also if you're using MySQL its top level installation directory must be specified.By default these are assumed to be in $GLOBUS_LOCATION.
In addition if you're building from source and wish to build the client Java API (included in the server bundles) you need to set the path to the Java Development Toolkit (JDK) version 1.4 or later.
You can use the following commands to set these variables. You only need to set these variables for RLS installation, they are not used when running RLS. This document assumes you are using the csh shell or one of its variants, if you're using sh or something similar (eg bash ) you should change the setenv commands to export variable=value .
The following commands were used during RLS development to install iODBC version 3.0.5.
$IODBCSRC is the directory where you untarred the iODBC sources
$ODBCINIDIR is the directory where you plan to install the
The contents should include the path to where you intend to install
the ODBC driver for your RDBMS (such as
following is an example that should work with psqlODBC. It assumes
you will name your LRC and RLI databases
[ODBC Data Sources]
Note: You do not need an RLI database if you plan to use Bloom filters for LRC to RLI updates (Bloom filters are kept in memory), in this case you can omit the RLI entries below.
Bug: psqlODBC will not find a Data Source Name (DSN) in the system odbc.ini file $ODBCINIDIR/odbc.ini . It will find DSNs in the user's odbc.ini file if it exists <$HOME/.odbc.ini).
One work around is to copy or symlink the system odbc.ini file
to each user's home directory. psqlODBC does find system DSNs in a
file called odbcinst.ini , which is looked for in the etc subdirectory
where iODBC was installed $GLOBUS_IODBC_PATH/etc/odbcinst.ini .
So another option besides creating user .odbc.ini files is
to copy or symlink the system odbc.ini file to $GLOBUS_IODBC_PATH/etc/odbcinst.ini .
Someone who understands this better may have a better answer.
If you're using MySQL and changed how how MySQL clients
connect to the MySQL server in my.cnf (eg the port number or
socket name) then you should set option to
The commands used to install Postgres 7.2.3 on the RLS development system were as follows.
$POSTGRESSRC is the directory where the PostgreSQL source was untarred.
Initialize PostgreSQL and start the server by running:
initdb -D /path/to/postgres-datadir
Create the database user (in our example, called
createuser -P dbuser
Important: Be sure to do periodic
The following commands were used to install psqlODBC 7.2.5.
Install psqlODBC by running the following commands
$PSQLODBCSRC is the directory where you untarred the psqlODBC source:
Note: The configure script that comes with psqlODBC supports
Once you've installed and configured MySQL you must start the database server and create the database user/password that RLS will use to connect to the database.
Start the database server by running:
mysqld_safe [--defaults-file path to your my.cnf file ]
To create the database user and password that RLS will use you must run the MySQL command line tool mysql , and specify the following commands.
mysql> use mysql;
These commands assume the username you will create for RLS is
Creation of the LRC and/or RLI databases is covered below in the section RLS Server Database Configuration
Recommended Version: 3.51.03 (why is this recommended if it's buggy??).
It also assumes that iODBC was installed in $GLOBUS_LOCATION, this may be changed by changing the --with-iodbc-includes and --with-iodbc-libs options.
Install MyODBC in $GLOBUS_LOCATION (you may choose a different directory if you wish, by changing the --prefix option to configure below.)
$MYODBCSRC is the directory where you untarred the MyODBC sources.
$ODBCINIDIR is the directory where you created the odbc.ini file.
Bug: There is a bug in MyODBC version 3.51.05 and earlier.
The debug code is not thread safe, and the RLS server will get a segmentation
violation and die if this code is enabled. In versions 3.51.05 and later
the debug code can be disabled with the configure option
setenv CFLAGS -DBUG_OFF
Download the appropriate bundle. This installs RLS client, server and setup packages as well as other globus packages RLS depends on.
The download page for Globus Toolkit 3.2 is at:
RLS Source Bundle
The RLS source bundle is included in the "All Services" and "WS Base Installers" bundles.
Note: When using these bundles, RLS will not be built by the installer script unless the environment variable GLOBUS_IODBC_PATH is set.
If you prefer, you can find the source bundle at:
RLS Binary Bundle
The RLS binary bundle is included in the "All Services" and "WS Base Installers" bundles linked against libiodbc-3.0.5.
Install the source bundle by running:
gpt-build globus-rls-server-3.2-src_bundle.tar.gz gcc32dbgpthr gpt-postinstall
|RLS server configuration is specified in
Create a database user that the RLS server will use to connect to the DBMS.
The database user and password you pick must be specified in the RLS server configuration file, as well as the name of the database(s) you will create (see below).
Decide which database(s) the RLS server will use (and that you will create in step ?):
If the RLS server is a Local Replica Catalog (LRC) server you will need to create the LRC database.
If the server is a Replica Location Index (RLI) server, you may need to create a RLI database.
An RLI server can receive updates from LRC servers in one of two forms, as LFN lists (in which case the RLI database must be created), or highly compressed Bloom filters. Since Bloom filters are so small, they are kept in memory and no database is required. An RLS server can be configured as both an LRC and RLI server.
Configure the schema file(s) for the database(s) you will create.
GT3.2 installed the schema files for the LRC and RLI databases in $GLOBUS_LOCATION/setup/globus.
There are separate files for PostgreSQL and MySQL:
Edit these files to set the name of the database user
you created for RLS, and the names of the databases configured in
By default the database user is dbuser , the LRC database name is lrc1000 and the RLI database name is rli1000.
Create the database(s) with the following commands (note once again that you do not need to create an RLI database if you are configuring an RLI server updated by Bloom filters):
For PostgreSQL, run:
createdb -O dbuser -U dbuser -W lrc1000
For MySQL, run:
mysql -p -u dbuser < $GLOBUS_LOCATION/setup/globus/globus-rls-lrc-mysql.sql
Review the server configuration file
A minimal configuration file for both an LRC and RLI server would be:
# Configure the database connection info
The server uses a host certificate to identify itself to clients. By default this certificate is located in the files /etc/grid-security/hostcert.pem and /etc/grid-security/hostkey.pem . Host certificates have a distinguished name of the form /CN=host/ FQDN . If the host you plan to run the RLS server is on does not have a host certificate you must obtain one from your Certificate Authority. The RLS server must be run as the same user who owns the host certificate files (typically root). The location of the host certificate files may be specified in $GLOBUS_LOCATION/etc/globus-rls-server.conf :
rlskeyfile path-to-key-file # default /etc/grid-security/hostkey.pem
It is possible to run the RLS server without authentication, by starting it with the -N option, and using URL's of the form rlsn://server to connect to it. If authentication is enabled RLI servers must include acl configuration options that match the identities of LRC servers that update it, that grant the rli_update permission to the LRCs.
Start the RLS Server by running:
Similarly, you can stop the RLS Server by running:
Important: See Notes on Initializing RLS.
|The server package includes a program called
To enable Index Service reporting add the contents of
If necessary, set your virtual organization (VO) name
The default value is
You must restart your MDS (GRIS) server after modifying
You can use the programs
Start the server in debug mode with the command:
$GLOBUS_LOCATION/bin/globus-rls-server -d [-N]
Ping the server using
$GLOBUS_LOCATION/bin/globus-rls-admin -p rls://serverhost
If you disabled authentication (by starting the server with the
$GLOBUS_LOCATION/bin/globus-rls-admin -p rlsn://serverhost
The RLS source bundle includes a test subdirectory (find
1) edit the config file
2) Run the script :
The script does the following:
3) You may need to grant access to the test databases for the database user using grant commands similar to what you used to create your production database(s). See MySQL Installation .
The RLS Server bundles (source and binary) include the RLS client bundles.
The RLS client bundles include a Java version of the RLS API (implemented as JNI wrappers to the C API). If you're building from source and wish to build the Java API you will need to set the environment variable JAVA_HOME to the directory where the Java Development Toolkit (JDK) version 1.4 or later is installed on your system before building the RLS client.
Build the RLS client source bundle by running:
gpt-build globus-rls-client-1.1-src_bundle.tar.gz gcc32dbgpthr
Install the RLS client binary bundle do running:
This note applies to Redhat 9 but could also apply to other linux distributions.
There have been occurences of RLS servers hanging on Redhat 9 systems.
The external symptoms are:
- The server does not accept new connections from clients, with an error message similar to:connect(rls://XXXXX): globus_rls_client: IO timeout: globus_io_tcp_register_connect() timed out after 30 seconds
- Often, the server continues to receive and send updates as configured and respond to signals. You can check this by querying other servers that interact with the one that's hung.
All the server threads are waiting to be signaled on a condition variable. Sometimes, this is in
globus_iofunctions, particularly in
This seems to be due to a problem in the new kernel and thread libraries of Redhat 9. A problem in
pthread_cond_wait()causes threads not to wake up correctly.
This problem has been seen with the following kernels and glibc packages:
The problems don't seem to arise when RLS is linked with older pthread libraries. This can be done as by adding a couple of lines to the RLS startup script in
$GLOBUS_LOCATION/sbin/SXXrls, as shown:<--- START ---> #!/bin/sh GLOBUS_LOCATION=/opt/gt3.2 MYSQL=/opt/mysql IODBC=/opt/iodbc export GLOBUS_LOCATION #Redhat 9 workaround LD_ASSUME_KERNEL=2.4.1 export LD_ASSUME_KERNEL <--- END --->
On i586 systems, set LD_ASSUME_KERNEL=2.2.5
Notes on RLS Initialization
Please be advised (and advise other users responsible for bringing up the RLS) that the startup initialization may take a few minutes before the RLS may be accessible. The initialization involves two key operations that may consume significant resources causing the server to appear temporarily unresponsive. Users of RLS may mistakenly assume that RLS failed to startup and may kill the server and start over. Some users may fall into this in a repeated cycle, believing that the RLS is unable to startup properly.
If the RLS is configured to send compressed updates (Bloom filters) to other RLIs, the RLS startup will involve initialization of the Bloom filter representing the current contents of the local replica catalog (LRC). This step is a prerequisite before any additional operations may be allowed, therefore no client connections are permitted until the initialization is complete. In our test environment, we have seen over 30 seconds delay due to creation of the Bloom filter corresponding to 1 million LFN names on a system with Dual 1 GHz CPU and 1.5 GB RAM. You may experience greater delays at larger scales and/or when running RLS with more limited system resources.
If the RLS is configured to send uncompressed udpates (LFN lists) to other RLIs, the RLS startup will not involve any additional initialization delay, however, the RLS will spawn an initial full catalog update to all RLIs it updates. Though these updates will take place on separate threads of execution after the initialization of the system, they will consume a great amount of processor activity. Depending on the volume of the local replica catalog (LRC), this processor activity may initially interfere with a client operation. In our test environment, we have seen our initial "globus-rls-admin ping..." operation may suffer a delay and timeout in 30 seconds, the second "ping" may delay for a few seconds but will successfully return, and the third and every subsequent "ping" operation will successfully return immediately throughout the duration of the update. The system exhibits the same behavior for any other client operation, such as a "globus-rls-cli query..." operation.
For more information, see the RLS System Administrator's Guide.