GT 3.9.5 Component Guide to Public Interfaces: WS MDS Index Service

Semantics and syntax of APIs

Programming Model Overview

Index service queries are performed using resource property requests; consult the Java WS Core documentation for details.

The contents of an index are maintained using the aggregator framework programming model, and can receive data from any aggregator source. Information about how to configure existing aggregator sources (such as the aggregator sources distributed with the Globus Toolkit, which include one that polls for resource property information, one that collects resource property information through subscription/notification, and one that collects information by executing an executable program) is found in the Configuration section of the MDS Index Public Interface Guide; information about how to create new aggregator sources can be found in the aggregator framework documentation.

Semantics and syntax of the WSDL

The index service inherits its WSDL interface from the aggregator framework module:

Protocol overview

The aggregator framework builds on the WS-ServiceGroup and WS-ResourceLifetime specifications. Those specifications should be consulted for details on the syntax of each operation.

Each aggregator framework is represented as a WS-ServiceGroup (specifically, an AggregatorServiceGroup).

Resources may be registered to an AggregatorServiceGroup using the AggregatorServiceGroup Add operation. Each registration will be represented as a ServiceGroupEntry resource. Resources may be registered to an AggregatorServiceGroup using the service group add operation, which will cause an entry to be added to the service group.

The entry will include configuration parameters for the aggregator source; when the registration is made, the following will happen:

  1. The appropriate aggregation source and sinks will be informed,
  2. the aggregator source will begin collecting data and inserting it into the corresponding service group entry,
  3. and the aggregator sink will begin processing the information in the service group entries.

The method of collection by source and processing by the sink is dependent on the particular instantiation of the aggregator framework (see per-source documentation for source information and per-service documentation for sink information.)

Operations

AggregatorServiceGroup

  • add: This operation is used to register a specified resource with the aggregator framework. In addition to the requirements made by the WS-ServiceGroup specification, the Content element of each registration must be an AggregatorContent type, with the AggregatorConfig element containing configuration information specific to each source and sink (documented elsewhere).
    If you are implementing the trigger service, the AggregatorConfig element must contain an element of xsi:type TriggerRuleType, defining the conditions upon which this trigger registration will fire. This element is documented further in the Trigger Service Admin Guide.

AggregatorServiceGroupEntry

  • setTerminationTime: This operation can be used to set the termination time of the registration, as detailed in WS-ResourceLifetime.

Resource properties

AggregatorServiceGroup

  • Entry: This resource property publishes details of each registered resource, including both an EPR to the resource, the aggregator framework configuration information, and data from the sink.
  • RegistrationCount: This resource property publishes registration load information (the total number of registrations since service startup and decaying averages)

In addition to the above resource properties, the trigger service exposes the following:

  • ActionsFiredCount: This resource property counts the total number and average rate of rule firings.

Faults

[list and briefly describe each fault]

WSDL and Schema Definition

Other relevant source files are the:

Note: If you are implementing the trigger service:

  • The TriggerRuleType is defined in trigger-types.xsd in trigger/source
  • The ActionsFiredCount resource property is an accumulator resource property, which is defined in the usefulrp/schema module.

Command-line tools

End-user command line tools for WS MDS Index Service

The index service exposes information via service groups and is accessed using the same command-line tools used to query other WSRF services for information. These tools are documented in detail in the Java WS Core User's Guide.

Administrative command-line tools for WS MDS Index Service

mds-servicegroup-add

mds-servicegroup-add creates a set of registrations to a WS-ServiceGroup and periodically renews those registrations. It is intended primarily for registering grid resources to MDS services such as the index and trigger services.

Registrations are defined in an XML configuration file, which is documented in the Aggregator Framework System Administrator's Guide.

Command syntax

mds-servicegroup-add -s http://foo [options] config.xml

Options and Arguments

-s http://foo This dummy option is required but ignored. All end point references used by mds-servicegroup-add are read from the configuration file, not the command line.
-a By default, mds-servicegroup-add will attempt to make an authenticated connection to each service group. This option is used to specify anonymous connections (and to prevent mds-servicegroup-add from failing if you don't have a valid Grid credential).
-z auth_type Specify an authorization type: self, to fail if the server's identity is different from the user's identity, host, to fail if the server does not have a valid server certificate, or none to continue regardless of the server's identity.
config.xml the configuration file

The syntax of the configuration file used by mds-servicegroup-add is described in detail in the MDS Index Public Interface Guide. In addition, the Globus Toolkit distribution includes an example configuration file $GLOBUS_LOCATION/etc/globus_wsrf_mds_aggregator/example-aggregator-registration.xml.

The simplest way to register services to an index is to edit the example file, replacing the EPRs in that file with the EPRs for your resources, and then to run mds-servicegroup-add to perform the registrations specified in that file.

Limitations

It is necessary for the tool to continue to run in order for the registrations that it maintains to be kept alive, as registrations will otherwise time out.

Overview of Graphical User Interface

There is no GUI specifically for the Index Service. The release contains a tech preview of WebMDS which can be used to display monitoring information collected in an Index Service in a normal web browser.

Semantics and syntax of domain-specific interface

Interface introduction

There is no domain-specific interface specific to the Index Service, however, the ExecutionAggregatorSource, which may be used by the Index Service, does have a domain-specific interface (specifically, the inputs provided to and outputs expected from the executable program).

Syntax of the interface

The syntax of the execution source's domain-specific interface is described in the Aggregator Framework Developer's Guide.

Configuration interface

Configuration overview

For a basic installation, the index service itself does not need any configuration changes from default.

In order for information to appear in the index, the source of that information must be registered to the information service. Information sources are registered using tools like mds-servicegroup-add. Each registration has a limited lifetime; mds-servicegroup-add should be left running in the background so that it can continue to refresh registrations. Depending on administration preference, it may be run on the same host as the index, on the same host as a member resource, or on any other host(s).

The Index service is built on the WS MDS Aggregator Framework and can use any aggregator source to collect information. In the most common case, the index service uses the QueryAggregatorSource to gather resource property values from the registered resource using one of the three WS-Resource Properties operations to poll for information; the polling method used depends on the configuration element supplied in the registration content.

Two other aggregator sources are supplied with the distribution: the SubscriptionAggregatorSource, which gathers resource property values through subscription/notification, and the ExecutionAggregatorSource, which executes an external program to gather information.

Syntax of the interface

Specifying the Aggregator Source

The aggregation source used to collect data can be changed from default by editing the aggregatorSource parameter in the index configuration in $GLOBUS_LOCATION/etc/globus_wsrf_mds_index/jndi-config.xml: 

  <resource name="configuration"
               type="org.globus.mds.index.impl.IndexConfiguration">
    <resourceParams>
      <parameter>
        <name> factory</name>
        <value>org.globus.wsrf.jndi.BeanFactory</value>
      </parameter>
      <parameter>
        <name>aggregatorSource</name>
        <value>org.globus.mds.aggregator.impl.QueryAggregatorSource</value>
      </parameter>
    </resourceParams>

This parameter specifies a java class that will be used to collect data for the index. By default it is set to the QueryAggregatorSource. It can be changed to one of the other sources supplied with the toolkit, or to one installed later. Details of the supplied sources are in the Aggregator Framework Developers Guide.

Configuring the Aggregator Source

Configuration options are specified by creating a configuration file and running mds-servicegroup-add to perform the registrations specified in that configuration file. The syntax of that file is: 
<?xml version="1.0" encoding="UTF-8" ?>
<ServiceGroupRegistrations
  xmlns="http://mds.globus.org/servicegroup/client" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xmlns:agg="http://mds.globus.org/aggregator/types">

   <defaultServiceGroupEPR>
      Default service group EPR
   </defaultServiceGroupEPR>

   <defaultRegistrantEPR>
      Default registrant EPR
   </defaultRegistrantEPR>

   <defaultSecurityDescriptorFile>
      Path name of security descriptor file
   </defaultSecurityDescriptorFile>

   One or more of the following:
   <ServiceGroupRegistrationParameters>
      <ServiceGroupEPR>
         EPR of the service group to register to
      </ServiceGroupEPR>
      <RegistrantEPR>
         EPR of the entity to be monitored.
      </RegistrantEPR>
      <InitialTerminationTime>
         Initial termination time
      </InitialTerminationTime>
      <RefreshIntervalSecs>
         Refresh interval, in seconds
      </RefreshIntervalSecs>
      <Content>
         Aggregator-source-specific configuration parameters
      </Content>
   </ServiceGroupRegistrationParameters>

</ServiceGroupRegistrations>
Each ServiceGroupRegistrationParameters block specifies the parameters used to register a resource to a service group. The parameters specified in this block are:
ServiceGroupEPR The EPR of the service group to register to. This parameter may be omitted if a defaultServiceGroupEPR block is specified; in this case, the value of defaultServiceGroupEPR will be used instead.
RegistrantEPR The EPR of the resource to register. This parameter may be omitted if a defaultRegistrantEPR block is specified; in this case, the value of defaultRegistrantEPR will be used instead.
InitialTerminationTime The initial termination time of this registration (this may be omitted).
RefreshIntervalSecs The refresh interval, in seconds.
Content Aggregator-source-specific registration parameters. The content blocks for the various aggregator sources are described in detail in the following sections.

The defaultServiceGroupEPR block provides a convenient way to register a number of resources to a single service group -- for example, if you wish to register several resources to your default VO index, you can specify that index as the default service group and omit the ServiceGroupEPR blocks from each ServiceGroupRegistrationParameters block.

The defaultRegistrantEPR block provides a convenient way to register a single resource to several service groups -- for example, if you wish to register your local GRAM server to several index servers, you can specify your GRAM server as the default registrant and omit the RegistrantEPR blocks from each ServiceGroupRegistrationParameters block.

ServiceGroupRegistration Content Blocks for QueryAggregatorSource

The QueryAggregatorSource can use one of the following three configuration blocks.
GetResourcePropertyPollType
If a GetResourcePropertyPollType block is used, QueryAggregatorSource will request a single resource property. The block has this form: 
   <Content xsi:type="agg:AggregatorContent"
      xmlns:agg="http://mds.globus.org/aggregator/types">
      <agg:AggregatorConfig xsi:type="agg:AggregatorConfig">
         <agg:GetResourcePropertyPollType>
            <agg:PollIntervalMillis>interval_in_ms</agg:PollIntervalMillis>
            <agg:ResourcePropertyName>rp_namespace:rp_localname</agg:ResourcePropertyName>
         </agg:GetResourcePropertyPollType>
      </agg:AggregatorConfig>
      <agg:AggregatorData/>
   </Content>
The PollIntervalMillis parameter is the poll refresh period in milliseconds; the ResourcePropertyName parameter is the QName of the resource property to poll for.
GetMultipleResourcePropertiesPollType
If a GetMultipleResourcePropertiesPollType block is used, QueryAggregatorSource will request one or more resource properties. The block has this form: 
   <Content
        xmlns:agg="http://mds.globus.org/aggregator/types"
        xsi:type="agg:AggregatorContent">
      <agg:AggregatorConfig xsi:type="agg:AggregatorConfig">
         <agg:GetMultipleResourcePropertiesPollType>
            <agg:PollIntervalMillis>interval_in_ms</agg:PollIntervalMillis>
            <agg:ResourcePropertyNames>rp1_namespace:rp1_localname</agg:ResourcePropertyNames>
            <agg:ResourcePropertyNames>rp2_namespace:rp3_localname</agg:ResourcePropertyNames>
            <agg:ResourcePropertyNames>rp3_namespace:rp3_localname</agg:ResourcePropertyNames>
         </agg:GetMultipleResourcePropertiesPollType>
      </agg:AggregatorConfig>
      <agg:AggregatorData/>
   </Content>
The PollIntervalMillis parameter is the poll refresh period in milliseconds; the ResourcePropertyNames parameters are the QNames of the resource properties to poll for. There is no limit on the number of ResourcePropertyNames that may be specified.
QueryResourcePropertiesPollType
If a QueryResourcePropertiesPollType block is used, QueryAggregatorSource will request that a query be executed against the Resource Property Set of the remote resource. In the GT 3.9.5 implementation of core, the only query language that is supported is XPath. The block has this form: 
   <Content
        xmlns:agg="http://mds.globus.org/aggregator/types"
        xsi:type="agg:AggregatorContent">
      <agg:AggregatorConfig xsi:type="agg:AggregatorConfig">
         <agg:QueryResourcePropertiesPollType>
            <agg:PollIntervalMillis>interval_in_ms</agg:PollIntervalMillis>
            <agg:QueryExpression Dialect="dialect">
               Query Expression
            </agg:QueryExpression>
         </agg:QueryResourcePropertiesPollType>
      </agg:AggregatorConfig>
      <agg:AggregatorData/>
   </Content>
The PollIntervalMillis parameter is the poll refresh period in milliseconds. The QueryExpression is an xsd:any element; the Dialect attribute specifies the dialect of the query expression.

ServiceGroupRegistration Content Blocks for SubscriptionAggregatorSource

The SubscriptionAggregatorSource gathers resource property values from the registered resource using WS-Notification subscriptions. The configuration block for SubscriptionAggregatorSource looks like this: 

   <Content
        xmlns:agg="http://mds.globus.org/aggregator/types"
        xsi:type="agg:AggregatorContent">
      <agg:AggregatorConfig xsi:type="agg:AggregatorConfig">
         <agg:AggregatorSubscriptionType>
             <TopicExpression Dialect="dialect">
                Topic Expression
             </TopicExpression>
             <Precondition Dialect="dialect">
                Precondition
             </Precondition>
             <Selector Dialect="dialect">
                Selector
             </Selector>
             <SubscriptionPolicy>
                Subscription Policy
             </SubscriptionPolicy>
             <InitialTerminationTime>time</InitialTerminationTime>
         </agg:AggregatorSubscriptionType>
      </agg:AggregatorConfig>
      <agg:AggregatorData/>
   </Content>
The only required parameter is the TopicExpression, which specifies the topic expression to use in the subscription request. [TODO: link to generic notification/subscription docs].

ServiceGroupRegistration Content Blocks for ExecutionAggregatorSource

The ExecutionAggregatorSource gathers arbitrary XML information about a registered resource by executing an external script and passing registration as parameters. The configuration block for ExecutionAggregatorSource looks like this: 
   <Content xsi:type="agg:AggregatorContent"
      xmlns:agg="http://mds.globus.org/aggregator/types">
      <agg:AggregatorConfig xsi:type="agg:AggregatorConfig">
         <agg:ExecutionPollType>
            <agg:PollIntervalMillis>interval_in_ms</agg:PollIntervalMillis>
            <agg:ProbeName>dummy_namespace:filename</agg:ProbeName>
         </agg:ExecutionPollType>
      </agg:AggregatorConfig>
      <agg:AggregatorData/>
   </Content>
The PollIntervalMillis parameter is the poll refresh period in milliseconds. The ProbeName parameter specifies the path name to the executable file, relative to the $GLOBUS_LOCATION/libexec/aggrexec directory. The path name should be specified as the local name part of this QName; the namespace part is ignored.

Environment variable interface

There are no Index Service specific environment variables.