GT 3.9.5 RLS : System Administrator's Guide

You need to download and install the following software (follow the links to download):

• Installation of GT 3.9.5
  
• a Relation Database Server (RDBMS) that supports ODBC. We provide instructions for PostgreSQL and MySQL.
  
• If you use PostgreSQL, you'll also need psqlODBC (the ODBC driver for PostgreSQL)
  
• If you use MySQL, you'll also need the MyODBC (Connector/ODBC) packages. MySQL's top level installation directory must be specified. By default these are assumed to be in $GLOBUS_LOCATION.
  
• the iODBC package is used to interface to the ODBC layer of the RDBMS. The location of iODBC and the odbc.ini file must be specified before installing the RLS server.

The following environment variables can be used to override the default locations. These should be set prior to installing the RLS server.

The location of iODBC and the odbc.ini file must be specified before installing the RLS server. Also if you're using MySQL its top level installation directory must be specified.By default these are assumed to be in $GLOBUS_LOCATION.

In addition if you're building from source and wish to build the client Java API (included in the server bundles) you need to set the path to the Java Development Toolkit (JDK) version 1.4 or later.

You can use the following commands to set these variables. You only need to set these variables for RLS installation, they are not used when running RLS. This document assumes you are using the csh shell or one of its variants, if you're using sh or something similar (eg bash ) you should change the setenv commands to export variable=value .

• setenv GLOBUS_IODBC_PATH $GLOBUS_LOCATION
• setenv ODBCINI $GLOBUS_LOCATION/var/odbc.ini
• setenv JAVA_HOME /usr/jdk/1.4
• setenv GLOBUS_MYSQL_PATH $GLOBUS_LOCATION # if using MySQL

The following commands were used during RLS development to install iODBC version 3.0.5.

cd $IODBCSRC
./configure --prefix=$GLOBUS_IODBC_PATH --disable-gtktest --with-pthreads --disable-gui --with-iodbc-inidir=$ODBCINIDIR
gmake
gmake install

where:

$IODBCSRC is the directory where you untarred the iODBC sources

$ODBCINIDIR is the directory where you plan to install the odbc.ini file (which you will create in the next step.)

Create the odbc.ini file in $ODBCINIDIR:

The contents should include the path to where you intend to install the ODBC driver for your RDBMS (such as psqlodbc.so or libmyodbc3.so).

The following is an example that should work with psqlODBC. It assumes you will name your LRC and RLI databases lrc1000 and rli1000:

[ODBC Data Sources]
lrc1000=lrc database
rli1000=rli database

[lrc1000]
Description=LRC database
DSN=lrc1000
Servertype=postgres
Servername=localhost
Database=lrc1000
ReadOnly=no

[rli1000]
Description=RLI database
DSN=rli1000
Servertype=postgres
Servername=localhost
Database=rli1000
ReadOnly=no

[Default]
Driver=/path/to/psqlodbc.so
Port=5432

Note: You do not need an RLI database if you plan to use Bloom filters for LRC to RLI updates (Bloom filters are kept in memory), in this case you can omit the RLI entries below.

Bug: psqlODBC will not find a Data Source Name (DSN) in the system odbc.ini file $ODBCINIDIR/odbc.ini . It will find DSNs in the user's odbc.ini file if it exists <$HOME/.odbc.ini).

One work around is to copy or symlink the system odbc.ini file to each user's home directory. psqlODBC does find system DSNs in a file called odbcinst.ini , which is looked for in the etc subdirectory where iODBC was installed $GLOBUS_IODBC_PATH/etc/odbcinst.ini . So another option besides creating user .odbc.ini files is to copy or symlink the system odbc.ini file to $GLOBUS_IODBC_PATH/etc/odbcinst.ini . Someone who understands this better may have a better answer.

If you're using MySQL and changed how how MySQL clients connect to the MySQL server in my.cnf (eg the port number or socket name) then you should set option to 65536 in odbc.ini for each database. This tells MyODBC to read the client section of my.cnf to find the changed connection parameters.

[lrc1000]
option = 65536

[rli1000]
option = 65536

The commands used to install Postgres 7.2.3 on the RLS development system were as follows.

cd $POSTGRESSRC
./configure --prefix=$GLOBUS_LOCATION
gmake
gmake install

$POSTGRESSRC is the directory where the PostgreSQL source was untarred.

Initialize PostgreSQL and start the server by running:

initdb -D /path/to/postgres-datadir
postmaster -D /path/to/postgres-datadir -i -o -F

The -o -F flags to postmaster disable fsync() calls after transactions (which, although it improves performance, raises the risk of DB corruption.)

Create the database user (in our example, called dbuser) and password that RLS will use:

createuser -P dbuser

Important: Be sure to do periodic vacuum and analyze commands on all your Postgres databases. The Postgres documentation recommends doing this daily from cron. Failure to do this can seriously degrade performance, to the point where routine RLS operations (such as LRC to RLI softstate updates) timeout and fail. Please see the Postgres documentation for further details.

The following commands were used to install psqlODBC 7.2.5.

Install psqlODBC by running the following commands

cd $PSQLODBCSRC
setenv CPPFLAGS -I$(IODBC_INSTALLDIR)/include
./configure --prefix=$GLOBUS_LOCATION --enable-pthreads
gmake
gmake install

$PSQLODBCSRC is the directory where you untarred the psqlODBC source:

Note: The configure script that comes with psqlODBC supports a --with-iodbc optionl. However when the RLS developers used this it resulted in RLS servers with corrupt memory that would dump core while opening the database connection. It seems to work fine (with iODBC) without this option.

Once you've installed and configured MySQL you must start the database server and create the database user/password that RLS will use to connect to the database.

Start the database server by running:

mysqld_safe [--defaults-file path to your my.cnf file ]

To create the database user and password that RLS will use you must run the MySQL command line tool mysql , and specify the following commands.

mysql> use mysql;
mysql> grant all on lrc1000.* to dbuser@localhost identified by 'dbpassword';
mysql> grant all on rli1000.* to dbuser@localhost identified by 'dbpassword';

These commands assume the username you will create for RLS is dbuser with password dbpassword , and the database(s) you will create for your LRC and/or RLI server are lrc1000 and rli1000.

Creation of the LRC and/or RLI databases is covered below in the section RLS Server Database Configuration

Recommended Version: 3.51.06

It also assumes that iODBC was installed in $GLOBUS_LOCATION, this may be changed by changing the --with-iodbc-includes and --with-iodbc-libs options.

Install MyODBC in $GLOBUS_LOCATION (you may choose a different directory if you wish, by changing the --prefix option to configure below.)

cd $MYODBCSRC
./configure --prefix=$GLOBUS_LOCATION --with-mysql-libs=$GLOBUS_MYSQL_PATH/lib/mysql --with-mysql-includes=$GLOBUS_MYSQL_PATH/include/mysql --with-iodbc-includes=$GLOBUS_LOCATION/include --with-iodbc-libs=$GLOBUS_LOCATION/lib --with-odbc-ini=$ODBCINIDIR
gmake
gmake install

where:

$MYODBCSRC is the directory where you untarred the MyODBC sources.

$ODBCINIDIR is the directory where you created the odbc.ini file.

Bug: There is a bug in MyODBC version 3.51.05 and earlier. The debug code is not thread safe, and the RLS server will get a segmentation violation and die if this code is enabled. In versions 3.51.05 and later the debug code can be disabled with the configure option --without-debug. In earlier versions it is disabled by defining DBUG_OFF, as in the following example:

setenv CFLAGS -DBUG_OFF

Download the appropriate bundle. RLS is included as part of the Globus Toolkit bundle. See the Globus Toolkit Development Downloads for a listing of available software.

Note: When using these bundles, RLS will not be built by the installer script unless the environment variable GLOBUS_IODBC_PATH is set.

RLS is installed as a part of the standard install. For basic installation instructions, see the Installation Guide

Create a database user that the RLS server will use to connect to the DBMS.

The database user and password you pick must be specified in the RLS server configuration file, as well as the name of the database(s) you will create (see below).

db_user dbuser
db_pwd dbpassword
lrc_dbname lrc1000 # optional (if LRC server)
rli_dbname rli1000 # optional (if RLI server)

Decide which database(s) the RLS server will use (and that you will create in step ?):

If the RLS server is a Local Replica Catalog (LRC) server you will need to create the LRC database.

If the server is a Replica Location Index (RLI) server, you may need to create a RLI database.

An RLI server can receive updates from LRC servers in one of two forms, as LFN lists (in which case the RLI database must be created), or highly compressed Bloom filters. Since Bloom filters are so small, they are kept in memory and no database is required. An RLS server can be configured as both an LRC and RLI server.

Configure the schema file(s) for the database(s) you will create.

GT3.2 installed the schema files for the LRC and RLI databases in $GLOBUS_LOCATION/setup/globus.

There are separate files for PostgreSQL and MySQL:

• globus-rls-lrc-postgres.sql
• globus-rls-rli-postgres.sql
• globus-rls-lrc-mysql.sql
• globus-rls-rli-mysql.sql

Edit these files to set the name of the database user you created for RLS, and the names of the databases configured in $GLOBUS_LOCATION/etc/globus-rls-server.conf.

By default the database user is dbuser , the LRC database name is lrc1000 and the RLI database name is rli1000.

Create the database(s) with the following commands (note once again that you do not need to create an RLI database if you are configuring an RLI server updated by Bloom filters):

For PostgreSQL, run:

createdb -O dbuser -U dbuser -W lrc1000
createdb -O dbuser -U dbuser -W rli1000
psql -W -U dbuser -d lrc1000 -f $GLOBUS_LOCATION/setup/globus/globus-rls-lrc-postgres.sql
psql -W -U dbuser -d rli1000 -f $GLOBUS_LOCATION/setup/globus/globus-rls-rli-postgres.sql

For MySQL, run:

mysql -p -u dbuser < $GLOBUS_LOCATION/setup/globus/globus-rls-lrc-mysql.sql
mysql -p -u dbuser < $GLOBUS_LOCATION/setup/globus/globus-rls-rli-mysql.sql

Review the server configuration file $GLOBUS_LOCATION/etc/globus-rls-server.conf and change any options you want. The server man page globus-rls-server(8) has complete details on all the options.

A minimal configuration file for both an LRC and RLI server would be:

# Configure the database connection info
db_user dbuser
db_pwd dbpassword

# If the server is an LRC server
lrc_server true
lrc_dbname lrc1000

# If the server is an RLI server
rli_server true
rli_dbname rli1000 # Not needed if updated by Bloom filters

# Configure who can make requests of the server
acl .*: all # RE matching grid-mapfile users or DNs from x509 certs

The server uses a host certificate to identify itself to clients. By default this certificate is located in the files /etc/grid-security/hostcert.pem and /etc/grid-security/hostkey.pem . Host certificates have a distinguished name of the form /CN=host/ FQDN . If the host you plan to run the RLS server is on does not have a host certificate you must obtain one from your Certificate Authority. The RLS server must be run as the same user who owns the host certificate files (typically root). The location of the host certificate files may be specified in $GLOBUS_LOCATION/etc/globus-rls-server.conf :

rlscertfile path-to-cert-file # default /etc/grid-security/hostcert.pem
rlskeyfile path-to-key-file # default /etc/grid-security/hostkey.pem

It is possible to run the RLS server without authentication, by starting it with the -N option, and using URL's of the form rlsn://server to connect to it. If authentication is enabled RLI servers must include acl configuration options that match the identities of LRC servers that update it, that grant the rli_update permission to the LRCs.

Start the RLS Server by running:

$GLOBUS_LOCATION/sbin/SXXrls start

Similarly, you can stop the RLS Server by running:

$GLOBUS_LOCATION/sbin/SXXrls stop 

Important: See Notes on Initializing RLS.

To enable Index Service reporting add the contents of the file $GLOBUS_LOCATION/setup/globus/rls-ldif.conf to the GT? Index Service configuration file $GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf

If necessary, set your virtual organization (VO) name in $GLOBUS_LOCATION/setup/globus/rls-ldif.conf

The default value is local . The VO name is referenced twice, on the lines beginning dn: and args:.

22

You must restart your MDS (GRIS) server after modifying $GLOBUS_LOCATION/etc/grid-info-resoruce-ldif.conf , you can use the following commands to do so:

$GLOBUS_LOCATION/sbin/SXXgris stop
$GLOBUS_LOCATION/sbin/SXXgris start

acl entries may be a combination of DNs and local usernames. If a DN is not found in the gridmap file then it is used to search the acl list.

A gridmap file may also be used to map DNs to local usernames, which in turn are matched against the regular expressions in the acl list to determine the user's permissions.

user is a regular expression matching distinguished names (or local usernames if a gridmap file is used) of users allowed to make calls to the server.

There may be multiple acl entries, the first match found is used to determine a user's privileges.

[permission] is one or more of the following values:

Enable or disable GSI authentication.

The default value is true.

If authentication is enabled (true), clients should use the URL schema rls: to connect to the server.

If authentication is not enabled (false), clients should use the URL schema rlsn:.

Password to use to connect to MYSQL server.

The default value is changethis.

Username to use to connect to MYSQL server.

The default value is dbperson.

Seconds after which idle connections close.

The default value is 900.

Number of hash functions to use in Bloom filters.

The default value is 3.

Possible values are 1 through 8.

This value, in conjunction with lrc_bloomfilter_ratio, will determine the number of false positives that may be expected when querying an RLI that is updated via Bloom filters.

Note: The default values of 3 and 10 give a false positive rate of approximately 1%.

Sets ratio of bloom filter size (in bits) to number of LFNs in the LRC catalog (in other words, size of the Bloom filter as a multiple of the number of LFNs in the LRC database.) Only meaningful if Bloom filters are used to update an RLI. Too small a value will generate too many false positives, too large wastes memory and network bandwidth.

The default value is 10.

Note: The default values of 3 and 10 give a false positive rate of approximately 1%.

LRC to RLI updates are buffered until either the buffer is full or this much time in seconds has elapsed since the last update.

The default value is 30.

Name of LRC database.

The default value is lrcdb.

If LRC server, the value should be true.

The default value is false.

Interval in seconds between LRC to RLI updates when the RLI is updated by Bloom filters. In other words, how often an LRC server does a Bloom filter softstate update.

This can be much smaller than the interval between updates without using Bloom filters (lrc_update_ll).

The default value is 300.

Turns LRC to RLI immediate mode updates on (true) or off (false).

The default value is false.

Number of seconds before an LRC server does a LFN list softstate update.

The default value is 86400.

Seconds to wait before an LRC server will retry to connect to an RLI server that it needs to update.

The default value is 300.

Maximum seconds to wait before re-trying listen in the event of an I/O error.

The default value is 300.

Maximum number of idle threads, excess threads are killed.

The default value is 5.

Maximum number of simultaneous connections.

The default value is 100.

Maximum number of threads running at one time.

The default value is 30.

URL of server.

The default value is rls://<hostname>:port

Sets environment variable ODBCINI.

If not specified, and ODBCINI is not already set, then the default value is $GLOBUS_LOCATION/var/odbc.ini.

Directory where pid file should be written.

The default value is /var/run.

Port the server listens on.

The default value is 39281.

Sets the maximum number of results returned by a query.

The default value is 0 (zero), which means no limit.

If a query request includes a limit greater than this value, an error (GLOBUS_RLS_BADARG) is returned.

If the query request has no limit specified, then at most result_limit records are returned by a query.

RLI servers must have this set to accept Bloom filter updates.

If true, then only Bloom filter updates are accepted from LRCs.

If false, full LFN lists are accepted.

Note: If Bloom filters are enabled, then the RLI does not support wildcarded queries.

If an RLI is configured to accept bloom filters (rli_bloomfilter true), then Bloom filters may be saved to this directory after updates.

This directory is scanned when an RLI server starts up and is used to initialize Bloom filters for each LRC that updated the RLI.

This option is useful when you want the RLI to recover its data immediately after a restart rather than wait for LRCs to send another update.

If the LRCs are updating frequently, this option is unnecessary, and may be wasteful in that each Bloom filter is written to disk after each update.

Bloom filters are not saved to disk.

This is the default.

Bloom filters are saved to the default directory:

• $GLOBUS_LOCATION/var/rls-bloomfilters if GLOBUS_LOCATION is set
  
• else, /tmp/rls-bloomfilters

Bloom filters are saved to the named directory.

Any other string is used as the directory name unchanged.

The Bloom filter files in this directory have the name of the URL of the LRC that sent the Bloom filter, with slashes (/) changed to percent signs (%), and ".bf" appended.

Name of RLI database.

The default value is rlidb.

Interval (in seconds) between RLI expirations of stale entries. In other words, how often an RLI server will check for stale entries in its database.

The default value is 28800.

Interval (in seconds) after which entries in the RLI database are considered stale (presumably because they were deleted in the LRC.)

The default value is 86400.

This value should be no smaller than lrc_update_ll.

Stale RLI entries are not returned in queries.

Note: If the LRC server is responding, this value is not used. Instead the value of lrc_update_ll or lrc_update_bf is retrieved from the LRC server, multiplied by 1.2, and used as the value for this option.

If RLI server, the value should be true.

The default value is false.

Name of X.509 certificate file identifying server.

This value is set by setting environment variable X509_USER_CERT.

Name of X.509 key file for server.

This value is set by setting environment variable X509_USER_KEY.

Number of threads to start initially.

The default value is 3.

$GLOBUS_LOCATION/bin/globus-rls-server -d [-N]

The -N option is helpful: if you do not have a host certificate for the server host, or a user certificate for yourself, it disables authentication.

$GLOBUS_LOCATION/bin/globus-rls-admin -p rls://serverhost

If you disabled authentication (by starting the server with the -N option), then use this command:

$GLOBUS_LOCATION/bin/globus-rls-admin -p rlsn://serverhost