Software Links
Getting Started
- Doc Structure
- A Globus Primer
- Globus Is Modular!
- Quickstart
- Installing GT
- Platform Notes
- Migrating from GT2
- Migrating from GT3
Reference
- PDF version
- Best Practices
- Coding Guidelines
- API docs
- Public Interfaces
- Resource Properties
- Samples
- Glossary
- Performance Studies
Common Runtime
Security
Data Mgt
Information Svcs
Execution Mgt
Table of Contents
The MDS4 Archive Service stores information about grid resources and allows that information to be queried by client tools. Information can be added to the archive via a number of different mechanisms: since the Archive Service is implemented using the Aggregator Framework, any aggregator source can be used to provide information for the archive.
This document describes the programmatic interfaces to the Archive Service. See also general Globus Toolkit coding guidelines and GT 4.1.2 best practices.
Features new in release 4.1.2
Based on WSRF rather than OGSI
- Aggregated data is published through WS-Resource Properties mechanisms
- The Aggregator Framework module (basis of the archive service) collects data from monitored resources using WS-Resource Properties collection mechanisms, including the base WS-Resource Properties poll operations and WS-Notification.
- Persistent configuration of aggregations has been refactored. The service no longer has its own config file for specifying aggregations. Instead a separate client is used to make appropriate registrations. This client may be started alongside the container for the equivalent effect. The client may also be deployed elsewhere to support resource-side configuration (providing similar functionality to the GT3.2 RegistryPublishProvider) or at a third location.
Other Supported Features
- The archive appears as a ServiceGroup (defined in WSRF) which lists registered resources alongside dynamically collected information from those resources. This information can be examined using (for example) XPath queries to discover resources that match desired constraints.
Deprecated Features
- None.
Tested Platforms for MDS4 Archive Service:
- Linux on i386
- Linux on PPC
- Windows XP
Tested containers for MDS4 Archive Service:
- Java WS Core container
- Tomcat 5.0.28
Protocol changes since GT version 4.0
- Generally incompatible with the GT 3.x archive service as the service has been remodelled to use WSRF instead of OGSI.
API changes since GT version 4.0
- The Aggregator Framework API used internally has retained the general flavor of the GT 3.2 Aggregator Framework API, but is not directly compatible with it.
Schema changes since GT version 4.0
- Schemas used for configuration and publishing are conceptually similar but are not compatible, primarily due to WSRF remodelling.
The Archive Service depends on the following GT components:
The Archive Service depends on the following 3rd party software:
- None
The security considerations for the Aggregator Framework also apply to the Archive Service:
By default, the aggregator sources do not use authentication credentials -- they retrieve information using anonymous SSL authentication or no authentication at all, and thus retrieve only publicly-available information. If a user or administrator changes that configuration so that a service's aggregator source uses credentials to acquire non-privileged data, then that user or administrator must configure the service's aggregator sink to limit access to authorized users.
There are essentially two interfaces to the Archive Service -- one for getting information into the archive, and one for retrieving information from the archive.
Information is stored into the Archive Service as service group entries using the standard MDS4 Core APIs for resource property queries or subscription/notification.
Because the Archive is implemented as a MDS4 Aggregator Framework, the programmatic interface for getting information into the archive is to create an aggregator source. The Aggregator Framework's architecture is described in the next section.
The WS MDS Aggregator Framework is the software framework on which WS MDS services are built. The Aggregator Framework collects data from an aggregator source and sends that data to an aggregator sink for processing. Aggregator sources distributed with the Globus Toolkit include modules that query resource properties, acquire data through subscription/notification, and execute programs to generate data. Another way of describing the Aggregator Framework is that it is designed to facilitate the collecting of information from or about WS-Resources via plugin aggregator sources and the feeding of that information to plugin aggregator sinks, which can then perform actions such as re-publishing, logging, or archiving the information.
Aggregators work on a type of service group called an
AggregatorServiceGroupRP. Resources may be
registered to an AggregatorServiceGroupRP using the service
group add operation, which will cause an entry to be
added to the service group. The entry will include configuration
parameters for the aggregator source; when the registration is
made, the appropriate aggregation source and sinks will be
informed; the aggregator source will begin collecting data and
inserting it into the corresponding service group entry, and the
aggregator sink will begin processing the information in the
service group entries.
The method of collection by source and processing by the sink is dependent on the particular instantiation of the aggregator framework.
The aggregator sinks distributed with the toolkit
(org.globus.mds.aggregator.impl.ServiceGroupEntryAggregatorSink
and org.globus.mds.trigger.impl.TriggerResource) are
described in the following table.
Table 1. Standard aggregator sinks
| Aggregator Sink | Service Implemented | Description |
|---|---|---|
ServiceGroupEntryAggregatorSink | Index Service | The servicegroup sink (used by the Index Service) publishes received data as content in the AggregatingServiceGroup entry used to manage the registration. This data can therefore be retrieved by querying the index for its 'entries' resource property. |
TriggerResource | Trigger Service | The Trigger Service provides an aggregator sink which receives data, applies tests to that data, and if the tests match, runs a specified executable. See the WS MDS Trigger Service documentation for more information. |
The aggregator sources supplied with the toolkit collect information using resource property queries (query sources), subscription/notification (subscription sources), and execution of external programs (execution sources).
The aggregator sources supplied with the Globus Toolkit are listed in the following table.
![[Note]](/docbook-images/note.gif) | Note |
|---|---|
All aggregator sources listed in this
table are in the |
Table 2. Standard aggregator sources
| Aggregator Source | Description |
|---|---|
| QueryAggregatorSource | The query source collects information from a registered resource by using WS-Resource Properties polling mechanisms:
Polls are made periodically, with both the period and target Resource Properties specified in the registration message. |
| SubscriptionAggregatorSource | The subscription source collects information from a registered resource using WS-Notification mechanisms. Data is delivered when property values change, rather than periodically. |
| ExecutionAggregatorSource | The execution source collects information about (not necessarily from) a registered resource by execution of a local executable, which is passed as input the identity of the registered resource. Details of the interface between the execution source and local executables are in Execution Aggregator Sources Reference. |
Archive Service queries are performed using resource property requests; consult Java WS Core for details.
The contents of an Archive are stored in a Xindice database and can be easily retrieved using the aggregator framework programming model (which can receive data from any aggregator source). Information about how to configure existing aggregator sources (such as the aggregator sources distributed with the Globus Toolkit, which include one that polls for resource property information, one that collects resource property information through subscription/notification, and one that collects information by executing an executable program) is found in the Aggregator Sources Reference; information about how to create new aggregator sources can be found in Aggregator Developer's Guide.
The Archive service inherits its WSDL interface from the Aggregator Framework module, included below:
The Aggregator Framework builds on the WS-ServiceGroup and WS-ResourceLifetime specifications. Those specifications should be consulted for details on the syntax of each operation.
Each Aggregator Framework is represented as a WS-ServiceGroup (specifically, an AggregatorServiceGroup).
Resources may be registered to an AggregatorServiceGroup using
the AggregatorServiceGroup Add operation. Each registration will be
represented as a ServiceGroupEntry resource. Resources may be
registered to an AggregatorServiceGroup using the service
group add operation, which will cause an entry to be
added to the service group.
The entry will include configuration parameters for the aggregator source; when the registration is made, the following will happen:
- The appropriate aggregation source and sinks will be informed,
- the aggregator source will begin collecting data and inserting it into the corresponding service group entry,
- and the aggregator sink will begin processing the information in the service group entries.
The method of collection by source and processing by the sink is dependent on the particular instantiation of the aggregator framework (see per-source documentation for source information and per-service documentation for sink information - for example the Index Service and the Trigger Service.)
add:This operation is used to register a specified resource with the Aggregator Framework. In addition to the requirements made by the WS-ServiceGroup specification, the Content element of each registration must be an AggregatorContent type, with the AggregatorConfig element containing configuration information specific to each source and sink (documented in the Aggregator System Administrator's Guide).
Entry:This resource property publishes details of each registered resource, including both an EPR to the resource, the Aggregator Framework configuration information, and data from the sink.RegistrationCount:This resource property publishes registration load information (the total number of registrations since service startup and decaying averages)
The Aggregator Framework throws standard WS-ServiceGroup, WS-ResourceLifetime, and WS-ResourceProperties faults and does not define any new faults of its own.
Other relevant source files are the:
- WSRF service group schema
- WSRF resource lifetime schema
- MDS Usefulrp schema.
The mds-archive-add(1) command is used to add a specific document to the Archive Service.
The mds-archive-get(1) command is used to retrieve a specific document from the Archive Service.
The mds-archive-query(1) command is used to retrieve a single or a set of documents from the Archive Service from a starting time period to an end time period that matches a specified XPath query, given a set of namespaces.
There is no GUI specifically for the Archive Service at this time.
There is no domain-specific interface specific to the Archive Service, however, the ExecutionAggregatorSource, which may be used by the Archive Service, does have a domain-specific interface (specifically, the inputs provided to and outputs expected from the executable program).
The syntax of the execution source's domain-specific interface is described in Execution Aggregator Sources Reference.
Information is retrieved from the archive using mds-archive-get and mds-archive-query command line tools:
GetDocumentto request a single Document by type, andQueryDocumentto perform an XPath query on an Archive to retrieve documents
Information is added to an archive by way of an aggregator source. The Globus Toolkit distribution includes several standard aggregator sources (see the Aggregator Sources Reference for more details). To create your own custom information source, see the Aggregator Developer's Guide.
The mds-archive-add tool can also be used
Use of the archive service is covered in the Build a Grid Service Tutorial (GlobusWORLD 2005).
See Debug section of the Java WS Core Developer's Guide for general information on logging, including which files to edit to set logging properties.
To turn on debug logging for the Archive Service, add the line:
log4j.category.org.globus.mds.archive=DEBUG
to the appropriate properties file. Since the Archive Service is implemented using the Aggregator Framework, you may also want to turn on aggregator debugging by adding this line:
log4j.category.org.globus.mds.aggregator=DEBUG
General troubleshooting information can be found in the Section 8, “Troubleshooting”.
Specifications for resource properties, service groups, and subscription/notification are available at http://www.globus.org/wsrf/.