GT 3.9.5 Component Guide to Public Interfaces: RLS

Semantics and syntax of APIs

Programming Model Overview

The RLS provides a Client API for C and Java based clients. The RLS Client C API is provided in the form of a library (e.g., .so file). Any installation of RLS will include the shared library as part of the $GLOBUS_LOCATION/include and $GLOBUS_LOCATION/lib directories. The RLS Client Java API depends on the shared library, which it links to via the Java Native Interface (JNI).

Component API

Semantics and syntax of the WSDL

There is no support for this type of interface for RLS.

Command-line tools

Command line tools include the Server, Admin Tool, and Client Tool.

RLS Server (globus-rls-server)

The RLS server (globus-rls-server) can be configured as either or both of the following:

  • Location Replica Catalog (LRC) server, which manages Logical FileName (LFN) to Physical FileName (PFN) mappings in a database.

    Note:     If globus-rls-server is configured as an LRC server, the RLI servers that it sends updates to should be added to the database using globus-rls-admin.
  • Replica Location Index (RLI) server, which manages mappings of LFNs to LRC servers.

Clients wishing to locate one or more physical filenames associated with a logical filename should first contact an RLI server, which will return a list of LRCs that may know about the LFN. The LRC servers are then contacted in turn to find the physical filenames.

Note: RLI information may be out of date, so clients should be prepared to get a negative response when contacting an LRC (or no response at all if the LRC server is unavailable.)

This page contains the following topics:

Synopsis

[ -B lrc_update_bf ] [ -b maxbackoff ] [ -C rlscertfile ] [ -c conffile ] [ -d ] [ -e rli_expire_int ] [ -F lrc_update_factor ] [ -f maxfreethreads ] [ -I true|false [ -i idletimeout ] [ -K rlskeyfile ] [ -L loglevel ] [ -l true|false ] [ -M maxconnections ] [ -m maxthreads ] [ -N ] [ -o lrc_buffer_time ] [ -p pidfiledir ] [ -r true|false ] [ -S rli_expire_stale ] [ -s startthreads ] [ -t timeout ] [ -U myurl ] [ -u lrc_update_ll ] [ -v ]

LRC to RLI Updates

Two methods exist for LRC servers to inform RLI servers of their LFNs.

  • By default, the LFNs are sent from the LRC to the RLI.

    This can be time consuming if the number of LFNs is large, but does give the RLI an exact list of the LFNs known to the LRC, and it allows wildcard searching of the RLI.
  • Alternatively, Bloom filters may be sent, which are highly compressed summaries of the LFNs; however they do not allow wildcard searching, and they will generate more "false positives" when querying an RLI.

Please see below for more on Bloom filters.

globus-rls-admin can be used to manage the list of RLIs that an LRC server updates. This includes partitioning LFNs among multiple RLI servers.

A softstate algorithm is used in both update modes: periodically the LRC server sends its state (LFN information) to the RLI servers it updates.

The RLI servers add these LFNs to their index, or update a timestamp if the LFNs were already known. RLI servers expire information about LFN,LRC mappings if they haven't been updated for a period longer than the softstate update interval.

The following options in the Configuration file control the softstate algorithm when an LRC updates an RLI by sending LFNs:

Updates to an LRC (new LFNs or deleted LFNs) normally don't propagate to RLI servers until the next softstate update (controlled by options lrc_update_ll and lrc_update_bf.

However, by enabling "immediate update" mode (set lrc_update_immediate to true), an LRC will send updates to an RLI within lrc_buffer_time seconds.

If updates are done with LFN lists, then only the LFNs that have been added or deleted to the LRC are sent. If Bloom filters are used, then the entire Bloom filter is sent.

When immediate updates are enabled, the interval between softstate updates is multiplied by lrc_update_factor as long as no updates have failed (LRC and RLI are considered to be in sync). This can greatly reduce the number of softstate updates an LRC needs to send to an RLI.

Incremental updates are buffered by the LRC server until either 200 updates have accumulated (when LFN lists are used), or lrc_buffer_time seconds have passed since the last update.

Bloom filter updates

A Bloom filter is an array of bits. Each LFN is hashed multiple times and the corresponding bits in the Bloom filter are set.

Querying an RLI to verify if an LFN exists is done by performing the same hashes, and checking if the bits in the filter are on. If not, then the LFN is known not to exist. If they're all on, then all that's known is that the LFN probably exists.

The size of the Bloom filter (as a multiple of the number of LFNs) and the number of hash functions, control the false positive rate. The default values of 10 and 3 give a false positive rate of approximately 1%.

The advantage of Bloom filters is their efficiency. For example, if the LRC has 1,000,000 LFNs in its database, with an average length of 20 bytes, then 20,000,000 bytes must be sent to an RLI during a softstate update (assuming no partitioning). The RLI server must perform 1,000,000 updates to its database to create new LFN,LRC mappings, or update timestamps on existing entries. With Bloom filters, only 1,250,000 bytes are sent (10 x 1,000,000 bits / 8), and there are no database operations on the RLI (Bloom filters are maintained entirely in memory). A comparison of the time to perform a 1,000,000 LFN update took 20 minutes sending all the LFNs, and less than 1 second using a Bloom filter. However as noted before, Bloom filters do not support wild card searches of an RLI.

Note: An LRC server can update some RLIs with Bloom filters, and others with LFNs. However an RLI server can only be updated using one method.

The following options in the Configuration file control Bloom filter updates:

Log messages

globus-rls-server uses syslog to log errors and other information (facility LOG_DAEMON) when it's running in normal (daemon) mode.

If the -d option (debug) is specified, then log messages are written to stdout.

Signals

The server will reread its configuration file if it receives a HUP signal. It will wait for all current requests to complete and shut down cleanly if sent any of the following signals: INT, QUIT or TERM.

Options (globus-rls-server)

The following table describes the command line options available for globus-rls-server:

-b maxbackoff Maximum time (in seconds) that globus-rls-server will attempt to reopen the socket it listens on after an I/O error.
-C rlscertfile Name of X.509 certificate file that identifies the server; sets environment variable X509_USER_CERT.
-c conffile

Name of configuration file for server.

The default is $GLOBUS_LOCATION/etc/globus-rls-server.conf if the environment variable GLOBUS_LOCATION is set; else , /usr/local/etc/globus-rls-server.conf.
-d

Enable debugging.

Server will not detach from controlling terminal and log messages will be written to stdout rather than syslog. For additional logging verbosity set loglevel (see -L option) to higher values.
-e rli_expire_int Interval (seconds) at which an RLI server should expire stale entries.
-F lrc_update_factor If lrc_update_immediate mode is on, and the LRC server is in sync with an RLI server (an LRC and RLI are synced if there have been no failed updates since the last full softstate update), then the interval between RLI updates for this server ( lrc_update_ll ) is multipled by lrc_update_factor.
-f maxfreethreads Maximum number of idle threads server will leave running. Excess threads are terminated.
-I true|false

Turns LRC to RLI immediate update mode on (true) or off (false).

The default value is false.

-i idletimeout Seconds after which idle client connections are timed out.
-K rlskeyfile Name of X.509 key file. Sets environment variable X509_USER_KEY.
-L loglevel Sets log level. By default this is 0, which means only errors will be logged. Higher values mean more verbose logging.
-l true|false

Configure whether server is an LRC server.

Default is false.
-M maxconnections

Maximum number of active connections. Should be small enough to prevent server from running out of open file descriptors.

The default value is 100.
-m maxthreads Maximum number of threads server will start up to support simultaneous requests.
-N

Disable authentication checking.

This option is intended for debugging. Clients should use the URL RLSN://host to disable authentication on the client side.
-o lrc_buffer_time

LRC to RLI updates are buffered until either the buffer is full or this much time (in seconds) has elapsed since the last update.

The default value is 30.
-p pidfiledir Directory where PID files should be written.
-r

Configure whether server is an RLI server.

The default value is false.
-S rli_expire_stale

Interval (in seconds) after which entries in the RLI database are considered stale (presumably because they were deleted in the LRC.)

Stale entries are not returned in queries.
-s startthreads Number of threads to start up initially.
-t timeout

Timeout (in seconds) for calls to other RLS servers (in other words, for LRC calls to send an update to an RLI.)

A value of 0 disables timeouts.

The default value is 30.
-U myurl URL for this server.
-u lrc_update_ll Interval (in seconds) between lfn-list LRC to RLI updates.
-v Show version and exits.

Admin Tool (globus-rls-admin) for RLS

The RLS Administration Tool (globus-rls-admin) performs administrative operations on a RLS server

Synopsis

-A|-a|-C option value|-c option|-d|-e|-p|-q|-s|-t timeout|-u|-v [ rli ] [ pattern ] [ server ]

Options

-A

Adds rli to list of RLI servers updated by an LRC server using Bloom filters.

Note: Partitions are not supported with Bloom filters, the LRC server maintains one Bloom filter for all LFNs in its database, which is sent to all RLI servers configured to receive Bloom filter updates with this option.
-a

Adds rli and optionally pattern to the list of RLI servers that the LRC server sends updates to (using a list of LFNs).

If pattern is specified, then only LFNs matching it will be sent to rli.

If rli is added with no patterns, then it is sent all updates. Pattern matching is done using standard Unix file globbing.
-C option value

Sets server option to value.

Important: This does not update the configuration file. Tthe next time the server is restarted, the configuration change will be lost.
-c option

Retrieves configuration value for specified option from server.

If option is set to all, then all options are retrieved.
-d

Removes rli and pattern from list of RLI servers that the LRC server sends updates to.

If pattern is not specified, then all entries for rli are removed.

Note: If all patterns are removed separately, then rli is sent all updates. To stop any updates from being sent to rli, do not specify pattern

-e Clears LRC database. Removes all lfn,pfn mappings.
-p Verifies that the server is responding.
-q Causes RLS server to exit.
-S

Shows statistics and other information gathered by RLS server.

Intended to be input into GRIS.
-s

Shows list of RLI servers and patterns being sent updates by the LRC server.

If rli or pattern are not specified, they are considered wildcards.
-t timeout

Sets timeout (in seconds) for RLS server requests.

The default value is 30.
-u Causes LRC server to immediately start full softstate updates to any RLI servers previously added with the -a option.
-v Shows version and exit.

Client Tool (globus-rls-cli) for RLS

The RLS Client Tool (globus-rls-cli) provides a command line interface to some of the functions supported by RLS. It also supports an interactive interface (if command is not specified). In interactive move double quotes may be used to encode an argument that contains white space.

Synopsis

command [ -c ] [ -h ] [ -l reslimit ] [ -s ] [ -t timeout ] [ -u ] [ command ] rls-server

Options

The client command tool uses getopt for command line parsing.

Note: Some versions will continue scanning for options (works that begin with a hyphen) for the entire command, line, which makes it impossible to specify negative integer or floating point value for an attribute. The workaround for this problem is to tell getopt() that there are no more options, by including 2 hyphens. For example, to specify the value -2 you must enter -- -2.

-c Sets "clearvalues" flag when deleting an attribute (will remove any attribute value records when an attribute is deleted).
-h Shows usage.
-l reslimit

Sets limit on number of results returned by wildcard query at a time.

Zero means no limit.
-s Uses SQL style wildcards (% and _).
-t timeout Sets timeout (in seconds) for RLS server requests. Default is 30 seconds.
-u Uses Unix style wildcards ( and ?).
-v Shows version.

Commands

add <lfn> <pfn> Adds pfn to mappings of lfn in an LRC catalog.
attribute add <object> <attr> <obj-type> <attr-type> Adds an attribute to an object, object should be the lfn or pfn name. obj-type should be one of lfn or pfn. attr-type should be one of date, float int, or string. If <value> is of type date then it should be in the form "YYYY-MM-DD HH:MM:DD".
attribute bulk add <object> <attr> <obj-type> Bulk adds attribute values.
attribute bulk delete <object> <attr> <obj-type> Bulk deletes attributes.
attribute bulk query <attr> <obj-type> <object> Bulk queries attributes.
attribute define <attr> <obj-type> <attr-type> Defines new attribute.
attribute delete <object> <attr> <obj-type> Removes attribute from object.
attribute modify <object> <attr> <obj-type> <attr-type> Modifies the value of an attribute.
attribute query <object> <attr> <obj-type> Retrieves value of specified attribute for object.
attribute search <attr> <obj-type> <operator> <attr-type> Searches for objects which have the specified attribute matching operator and value. operator should be one of =, !=, >, >=, <, <=, like.
attribute show <attr> <obj-type> Shows attribute definition. If attr is a hypen (-) then all attributes are shown.
attribute undefine <attr> <obj-type> Deletes an attribute definition. Will return an error if any objects possess this attribute.
bulk add <lfn> <pfn> [<lfn> <pfn> Bulk adds lfn, pfn mappings.
bulk create <lfn> <pfn> [<lfn> <pfn> Bulk creates lfn, pfn mappings.
bulk delete <lfn> <pfn> [<lfn> <pfn> Bulk deletes lfn, pfn mappings.
bulk query lrc lfn [<lfn> ...] Bulk queries lrc for lfns.
bulk query lrc pfn [<pfn> ...] Bulk queries lrc for pfns.
bulk query rli lfn [<lfn> ...] Bulk queries rli for lfns.
create <lfn> <pfn> Creates a new lfn, pfn mapping in an LRC catalog.
delete <lfn> <pfn> Deletes a lfn, pfn mapping from an LRC catalog.
exit Exits interactive session.
help Prints help message.
query lrc lfn <lfn> Queries an LRC server for mappings of lfn.
query lrc pfn <pfn> Queries an LRC server for mappings to pfn.
query rli lfn <lfn> Queries an RLI server for mappings of lfn.
query wildcard lrc lfn <lfn-pattern> Performs a wildcarded query of an LRC server for mappings of lfn-pattern. Patterns use the standard Unix wildcard characters, an asterisk () matches 0 or more characters, and a question mark (?) matches any single character.
query wildcard lrc pfn <pfn-pattern> Queries an LRC server for mappings to pfn-pattern. Patterns use the standard Unix wildcard characters, an asterisk () matches 0 or more characters, and a question mark (?) matches any single character.
query wildcard rli lfn <lfn-pattern> Queries an RLI server for mappings of lfn-pattern. Patterns use the standard Unix wildcard characters, an asterisk () matches 0 or more characters, and a question mark (?) matches any single character.
set reslimit <limit> Sets limit on number of results returned at a time by a wildcard query. Zero means no limit.
set timeout <timeout>

Sets timeout (in seconds) on calls to the RLS server.

The default value is 30.
version Shows version and exit.

Overview of Graphical User Interface

There is no support for this type of interface for RLS.

Semantics and syntax of domain-specific interface

There is no support for this type of interface for RLS.

Configuration interface

Configuration overview

The globus-rls-server.conf file configures the RLS server.

If the configuration file is not specified on the command line (see the -c option) then it is looked for in both:

  • $GLOBUS_LOCATION/etc/globus-rls-server.conf
  • /usr/local/etc/globus-rls-server.conf if GLOBUS_LOCATION is not set

Most command line options for globus-rls-server may also be set in the configuration file, however command line options always override items found in the configuration file.

The configuration file is a sequence of lines consisting of a keyword, whitespace, and a value. Comments begin with # and end with a newline.

Syntax of the interface

acl user: permission [permission]

acl entries may be a combination of DNs and local usernames. If a DN is not found in the gridmap file then it is used to search the acl list.

A gridmap file may also be used to map DNs to local usernames, which in turn are matched against the regular expressions in the acl list to determine the user's permissions.

user is a regular expression matching distinguished names (or local usernames if a gridmap file is used) of users allowed to make calls to the server.

There may be multiple acl entries, the first match found is used to determine a user's privileges.

[permission] is one or more of the following values:

lrc_read Allows client to read an LRC.
lrc_update Allows client to update an LRC.
rli_read Allows client to read an RLI.
rli_update Allows client to update an RLI.
admin Allows client to update an LRC's list of RLIs to send updates to.
stats Allows client to read performance statistics.
all Allows client to do all of the above.
authentication true|false

Enable or disable GSI authentication.

The default value is true.

If authentication is enabled (true), clients should use the URL schema rls: to connect to the server.

If authentication is not enabled (false), clients should use the URL schema rlsn:.
db_pwd password

Password to use to connect to MYSQL server.

The default value is changethis.
db_user databaseuser

Username to use to connect to MYSQL server.

The default value is dbperson.
idletimeout seconds

Seconds after which idle connections close.

The default value is 900.
loglevel N Sets loglevel to N (default is 0). Higher levels mean more verbosity.
lrc_bloomfilter_numhash N

Number of hash functions to use in Bloom filters.

The default value is 3.

Possible values are 1 through 8.

This value, in conjunction with lrc_bloomfilter_ratio, will determine the number of false positives that may be expected when querying an RLI that is updated via Bloom filters.

Note: The default values of 3 and 10 give a false positive rate of approximately 1%.

lrc_bloomfilter_ratio N

Sets ratio of bloom filter size (in bits) to number of LFNs in the LRC catalog (in other words, size of the Bloom filter as a multiple of the number of LFNs in the LRC database.) Only meaningful if Bloom filters are used to update an RLI. Too small a value will generate too many false positives, too large wastes memory and network bandwidth.

The default value is 10.

Note: The default values of 3 and 10 give a false positive rate of approximately 1%.
lrc_buffer_time N

LRC to RLI updates are buffered until either the buffer is full or this much time in seconds has elapsed since the last update.

The default value is 30.
lrc_dbname

Name of LRC database.

The default value is lrcdb.
lrc_server true|false

If LRC server, the value should be true.

The default value is false.
lrc_update_bf seconds

Interval in seconds between LRC to RLI updates when the RLI is updated by Bloom filters. In other words, how often an LRC server does a Bloom filter softstate update.

This can be much smaller than the interval between updates without using Bloom filters (lrc_update_ll).

The default value is 300.
lrc_update_factor N If lrc_update_immediate mode is on, and the LRC server is in sync with an RLI server (an LRC and RLI are synced if there have been no failed updates since the last full softstate update), then the interval between RLI updates for this server ( lrc_update_ll ) is multiplied by the value of this option.
lrc_update_immediate true|false

Turns LRC to RLI immediate mode updates on (true) or off (false).

The default value is false.
lrc_update_ll seconds

Number of seconds before an LRC server does a LFN list softstate update.

The default value is 86400.
lrc_update_retry seconds

Seconds to wait before an LRC server will retry to connect to an RLI server that it needs to update.

The default value is 300.
maxbackoff seconds

Maximum seconds to wait before re-trying listen in the event of an I/O error.

The default value is 300.
maxfreethreads N

Maximum number of idle threads, excess threads are killed.

The default value is 5.
maxconnections N

Maximum number of simultaneous connections.

The default value is 100.
maxthreads N

Maximum number of threads running at one time.

The default value is 30.
myurl URL

URL of server.

The default value is rls://<hostname>:port

odbcini filename

Sets environment variable ODBCINI.

If not specified, and ODBCINI is not already set, then the default value is $GLOBUS_LOCATION/var/odbc.ini.

pidfiledir directory

Directory where pid file should be written.

The default value is /var/run.
port N

Port the server listens on.

The default value is 39281.
result_limit limit

Sets the maximum number of results returned by a query.

The default value is 0 (zero), which means no limit.

If a query request includes a limit greater than this value, an error (GLOBUS_RLS_BADARG) is returned.

If the query request has no limit specified, then at most result_limit records are returned by a query.

 
rli_bloomfilter true|false

RLI servers must have this set to accept Bloom filter updates.

If true, then only Bloom filter updates are accepted from LRCs.

If false, full LFN lists are accepted.

Note: If Bloom filters are enabled, then the RLI does not support wildcarded queries.
rli_bloomfilter_dir none|default|pathname

If an RLI is configured to accept bloom filters (rli_bloomfilter true), then Bloom filters may be saved to this directory after updates.

This directory is scanned when an RLI server starts up and is used to initialize Bloom filters for each LRC that updated the RLI.

This option is useful when you want the RLI to recover its data immediately after a restart rather than wait for LRCs to send another update.

If the LRCs are updating frequently, this option is unnecessary, and may be wasteful in that each Bloom filter is written to disk after each update.
none

Bloom filters are not saved to disk.

This is the default.

default

Bloom filters are saved to the default directory:

  • $GLOBUS_LOCATION/var/rls-bloomfilters if GLOBUS_LOCATION is set
  • else, /tmp/rls-bloomfilters
pathname

Bloom filters are saved to the named directory.

Any other string is used as the directory name unchanged.

The Bloom filter files in this directory have the name of the URL of the LRC that sent the Bloom filter, with slashes (/) changed to percent signs (%), and ".bf" appended.
rli_dbname database

Name of RLI database.

The default value is rlidb.
rli_expire_int seconds

Interval (in seconds) between RLI expirations of stale entries. In other words, how often an RLI server will check for stale entries in its database.

The default value is 28800.
rli_expire_stale seconds

Interval (in seconds) after which entries in the RLI database are considered stale (presumably because they were deleted in the LRC.)

The default value is 86400.

This value should be no smaller than lrc_update_ll.

Stale RLI entries are not returned in queries.

Note: If the LRC server is responding, this value is not used. Instead the value of lrc_update_ll or lrc_update_bf is retrieved from the LRC server, multiplied by 1.2, and used as the value for this option.

rli_server true|false

If RLI server, the value should be true.

The default value is false.
rlscertfile filename

Name of X.509 certificate file identifying server.

This value is set by setting environment variable X509_USER_CERT.
rlskeyfile

Name of X.509 key file for server.

This value is set by setting environment variable X509_USER_KEY.
startthreads N

Number of threads to start initially.

The default value is 3.
timeout seconds Timeout (in seconds) for calls to other RLS servers (eg for LRC calls to send an update to an RLI).

Environment variable interface

There is no support for this type of interface for RLS.