(Draft) A Software Installation Information Provider for MDS 2.x
by Neill Miller under the guidance of Jennifer Schopf.
Warning: This information is only a draft and is currently unofficial. The Schema, the code, and all of the information may change in the future.
Quick Guide to Installing and Using this Software Information Provider
Introduction
The Schema
The Configuration File
The Code
Enabling The Provider
An Example Query Output
Screenshots
Introduction
The MDS may benefit from having a core information provider which publishes information about the local software installed on each GRIS host of a Globus based grid environment. Users or administrators of the installation may be interested in knowing if a particular tool or compiler is available on a number of systems, as well as the version. For example, does gcc exist on host "foo"? Which version of automake (if any) is present on host "bar"? Does host "baz" meet the requirements for compiling and running my application, given that I need python, guile, and a fortran compiler?
What is explained here is a basic draft of a minimum amount of information required to be useful, packaged into an information provider for MDS 2.x. The schema, the code, and some screenshots are used to illustrate the implementation details.
The Schema
First things first, we need a schema to represent the information that we'd like to publish. I've chosen the following, which may need some improvements. These additions have been appended to the $GLOBUS_LOCATION/etc/grid-info-resource.schema file.
attributetype ( 1.3.6.1.4.1.3536.2.6.3536.10.1.109
NAME 'Mds-Application-Group-config'
DESC 'The location of this application group configuration file'
EQUALITY caseIgnoreMatch
ORDERING caseIgnoreOrderingMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.44
SINGLE-VALUE
)
attributetype ( 1.3.6.1.4.1.3536.2.6.3536.10.1.110
NAME 'Mds-Application-name'
DESC 'The name of the application being presented'
EQUALITY caseIgnoreMatch
ORDERING caseIgnoreOrderingMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.44
SINGLE-VALUE
)
attributetype ( 1.3.6.1.4.1.3536.2.6.3536.10.1.111
NAME 'Mds-Application-location'
DESC 'The path to the installed application'
EQUALITY caseIgnoreMatch
ORDERING caseIgnoreOrderingMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.44
SINGLE-VALUE
)
attributetype ( 1.3.6.1.4.1.3536.2.6.3536.10.1.112
NAME 'Mds-Application-version'
DESC 'The version of the installed application'
EQUALITY caseIgnoreMatch
ORDERING caseIgnoreOrderingMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.44
SINGLE-VALUE
)
attributetype ( 1.3.6.1.4.1.3536.2.6.3536.10.1.113
NAME 'Mds-Application-info'
DESC 'Additional information about the installed application'
EQUALITY caseIgnoreMatch
ORDERING caseIgnoreOrderingMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.44
SINGLE-VALUE
)
attributetype ( 1.3.6.1.4.1.3536.2.6.3536.10.1.114
NAME 'Mds-Application-Group-name'
DESC 'A group name of software applications'
EQUALITY caseIgnoreMatch
ORDERING caseIgnoreOrderingMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.44
SINGLE-VALUE
)
objectclass ( 1.3.6.1.4.1.3536.2.6.3536.10.1.115
NAME 'MdsApplicationGroup'
DESC 'Information about a Group of Software Applications'
SUP 'Mds'
STRUCTURAL
MUST ( Mds-Application-Group-name $ Mds-Application-Group-config )
MAY ( Mds-Application-name )
)
objectclass ( 1.3.6.1.4.1.3536.2.6.3536.10.1.116
NAME 'MdsApplication'
DESC 'Information about a particular Software Applications'
SUP 'Mds'
STRUCTURAL
MUST ( Mds-Application-name $ Mds-Application-version )
MAY ( Mds-Application-location $ Mds-Application-info )
)
NOTE: The OID values have been cleared for testing by Jennifer Schopf,
but of course are subject to change if required.
The schema shows that we have two basic components (objectclasses). The MdsApplicationGroup is a logical group of applications. For this information provider, that logical grouping would be "probed software" since the point of the provider is to determine whether or not a particular GRIS/GIIS host has the software that we're looking for. Since the software may not exist on the individual host, the grouping of "software" is too vague. A name is required for each logical group of published application groups, for convenience. Other required information that should be published is the location of the configuration file, should the administrator be curious about the versioning of this configuration.
The MdsApplication objectclass describes a particular application. An application consists of a required name and version. Optionally an application may contain the path to the installed application and some miscellaneous information about the application. The path should not be required, as it may be pose a potential security risk, depending on the application of course.
The Configuration File
A simple configuration file is required for several reasons. First, it dictates which software this information provider should look for. Secondly, it controls (to some degree) the information that will be output about the applications. I've chosen a simple plain-text format for configuration for ease of use and administration. The configuration file has been tentatively placed at $GLOBUS_LOCATION/etc/grid-info-soft.conf. An example configuration file looks like this:
# The grid-info-soft-* information provider will probe for all software # listed in here with a properly formatted line. VERSION_STR specifies # a command line switch used for determining the application version. # VERSION_LIMIT limits the number of lines of the application version # output that will be shown in the final information output. Software: java PATH=yes VERSION_STR=-version VERSION_LIMIT=1 MISC=yes Software: gcc PATH=yes VERSION_STR=--version VERSION_LIMIT=1 MISC=yes Software: python PATH=yes VERSION_STR=-V VERSION_LIMIT=1 MISC=yes Software: gdb PATH=yes VERSION_STR=--version VERSION_LIMIT=1 MISC=yes Software: foobar PATH=yes VERSION_STR=--version VERSION_LIMIT=1 MISC=yes Software: vi PATH=yes VERSION_STR=--version VERSION_LIMIT=2 MISC=yes Software: XFree86 PATH=yes VERSION_STR=-version VERSION_LIMIT=2 MISC=yes Software: emacs PATH=yes VERSION_STR=--version VERSION_LIMIT=1 MISC=yes Software: guile PATH=yes VERSION_STR=--version VERSION_LIMIT=1 MISC=yesAs the comments at the top of the example configuration file specifies, each line has a specific format that must be followed in order for the information provider to be able to properly parse it and return the requested information.
Each line must start with the token "Software: ". Note that the trailing space must be included. (This format was arbitrarily chosen, and while it has small spacing nuances, it remains quite flexible). After the Software listing, an application should be specified as it would appear on invocation from a local system. This means that the application listing is case sensitive, since most UNIX-styled file systems are case sensitive and "Emacs" is not considered the same as "emacs".
Following the application name, a "PATH=" line must appear. This line dictates whether or not the full path to the application will appear in the LDIF output (i.e. published in the MDS). Valid values for this are "yes" and "no".
Following the PATH line, a version string must be present. The version string is required so that the information provider knows how to determine the version of the particular application, should it be present on the system. Many programs share the same syntax for determining the version information, however some applications require special command line switches. For this reason, the version string must be present. Also, many applications generate verbose output in their version information. To limit the output length (and more importantly, to not mess up the outputted LDIF records), the version limit was found to be useful. The version limit contrains the number of lines from the version output to be published in LDIF format to the MDS. Most applications will likely have this value set to 1.
For flexibility, an additional information field about a particular piece of software can also be published. The last field in the example configuration file is a "MISC=" field. The valid values for this are "yes" and "no". This option determines if additional information about the application should be published, should there be any. (NOTE: Currently, the output of the "whereis application-name" is used as additional information).
It should also be noted that no spaces should exist between any of the required tokens and the equal ("=") operator. Similarly, no spaces should exist between the equal ("=") operator and the specified value. The above example configuration is syntactically correct.
The Code
Here you can sample the current source code for this draft implementation. It is written as a bash shell script (like the other core information providers included with MDS 2.x). It may have some issues with code placement, as not all platforms have been tested (or even implemented).
grid-info-soft-posix
grid-info-soft-common
Enabling The Provider
Now that we have the information provider written, the last step is to enable it so that we can test it. I've added the following entry into the $GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf file:
# generate probed software information every 1 day dn: Mds-Host-hn=glob, Mds-Vo-name=local, o=grid objectclass: GlobusTop objectclass: GlobusActiveObject objectclass: GlobusActiveSearch type: exec path: /opt/globus/libexec base: grid-info-soft-posix args: -devclassobj -devobjs -dn Mds-Host-hn=glob,Mds-Vo-name=local,o=grid -validto-secs 8640 0 -keepto-secs 86400 cachetime: 86400 timelimit: 100 sizelimit: 100NOTE: You will need to change the dn line (at the top, and also in the args line a little further down) that lists the hostname for your machine. My host name is "glob", so replace where it says "glob" with the hostname of your machine.
This information is pretty straightforward. It tells the MDS how to call the newly written information provider, as well as for how long to cache the output data. One day was selected, although that can be adjusted by the MDS administrator for a more appropriate setting specific to the environment. The output is also arbitrarily limited to 100 records, and this may need adjustment depending on the needs of the administrator.
An Example Query Output
The provided code, using the above example configuration will produce output similar to the following when run on the command line:
neillm@glob libexec $ ./grid-info-soft-posix dn: objectclass: MdsApplicationGroup Mds-Application-Group-config: /opt/globus/etc/grid-info-soft.conf Mds-Application-name: XFree86 Mds-Application-name: emacs Mds-Application-name: foobar Mds-Application-name: gcc Mds-Application-name: gdb Mds-Application-name: guile Mds-Application-name: java Mds-Application-name: python Mds-Application-name: vi dn: Mds-Application-Group-name=probed software, objectclass: MdsApplicationGroup Mds-Application-Group-config: /opt/globus/etc/grid-info-soft.conf Mds-Application-Group-name: probed software Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z Mds-Application-name: XFree86 Mds-Application-name: emacs Mds-Application-name: foobar Mds-Application-name: gcc Mds-Application-name: gdb Mds-Application-name: guile Mds-Application-name: java Mds-Application-name: python Mds-Application-name: vi dn: Mds-Application-name=java, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: java Mds-Application-location: /usr/local/j2sdk1.4.1/bin/java Mds-Application-version: java version "1.4.1-beta" Mds-Application-info: java: /opt/blackdown-jre-1.3.1/bin/java /opt/blackdown-jdk -1.3.1/bin/java Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=gcc, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: gcc Mds-Application-location: /usr/bin/gcc Mds-Application-version: 2.95.3 Mds-Application-info: gcc: /usr/bin/gcc /usr/man/man1/gcc.1.gz /usr/share/man/ma n1/gcc.1.gz Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=python, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: python Mds-Application-location: /usr/bin/python Mds-Application-version: Python 2.2.1 Mds-Application-info: python: /usr/bin/python2.2 /usr/bin/python /usr/lib/python 2.2 /usr/include/python2.2 /usr/man/man1/python.1.gz /usr/share/man/man1/python. 1.gz Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=gdb, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: gdb Mds-Application-location: /usr/bin/gdb Mds-Application-version: GNU gdb 5.1.1 Mds-Application-info: gdb: /usr/bin/gdb /usr/man/man1/gdb.1.gz /usr/share/man/ma n1/gdb.1.gz Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=foobar, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: foobar Mds-Application-location: "foobar" location not available Mds-Application-version: "foobar" version not available Mds-Application-info: "foobar" info not available Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=vi, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: vi Mds-Application-location: /usr/bin/vi Mds-Application-version: VIM - Vi IMproved 6.1 (2002 Mar 24, compiled Jul 9 200 2 04-10-04) Included patches- 1, 3-15, 17-18, 20-22, 24-34, 36-42, 45-47, 50-64, 67-72, 74 Mds-Application-info: vi: /usr/bin/vi Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=XFree86, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: XFree86 Mds-Application-location: /usr/X11R6/bin/XFree86 Mds-Application-version: XFree86 Version 4.2.0 / X Window System Mds-Application-info: XFree86: /usr/X11R6/bin/XFree86 /usr/bin/X11/XFree86 Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=emacs, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: emacs Mds-Application-location: /usr/bin/emacs Mds-Application-version: GNU Emacs 21.2.1 Mds-Application-info: emacs: /usr/bin/emacs /usr/lib/emacs /usr/share/emacs /usr /man/man1/emacs.1.gz /usr/share/man/man1/emacs.1.gz Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z dn: Mds-Application-name=guile, Mds-Application-Group-name=probed software, objectclass: MdsApplication Mds-Application-name: guile Mds-Application-location: /usr/bin/guile Mds-Application-version: Guile 1.4 Mds-Application-info: guile: /usr/bin/guile /usr/include/guile /usr/share/guile Mds-validfrom: 20020814020432Z Mds-validto: 20020814020432Z Mds-keepto: 20020814020432Z
On a real query on a running MDS installation, the following output is emitted.
neillm@glob libexec $ ../sbin/SXXgris start Starting up Openldap 2.0 SLAPD server for the GRIS neillm@glob libexec $ grid-info-search -x "(objectclass=MdsApplication)" -L version: 1 # # filter: (objectclass=MdsApplication) # requesting: ALL # # java, probed software, glob, local, grid dn: Mds-Application-name=java, Mds-Application-Group-name=probed software, Mds -Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: java Mds-Application-location: /usr/local/j2sdk1.4.1/bin/java Mds-Application-version: java version "1.4.1-beta" Mds-Application-info: java: /opt/blackdown-jre-1.3.1/bin/java /opt/blackdown-j dk-1.3.1/bin/java Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # gcc, probed software, glob, local, grid dn: Mds-Application-name=gcc, Mds-Application-Group-name=probed software, Mds- Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: gcc Mds-Application-location: /usr/bin/gcc Mds-Application-version: 2.95.3 Mds-Application-info: gcc: /usr/bin/gcc /usr/man/man1/gcc.1.gz /usr/share/man/ man1/gcc.1.gz Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # python, probed software, glob, local, grid dn: Mds-Application-name=python, Mds-Application-Group-name=probed software, M ds-Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: python Mds-Application-location: /usr/bin/python Mds-Application-version: Python 2.2.1 Mds-Application-info: python: /usr/bin/python2.2 /usr/bin/python /usr/lib/pyth on2.2 /usr/include/python2.2 /usr/man/man1/python.1.gz /usr/share/man/man1/py thon.1.gz Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # gdb, probed software, glob, local, grid dn: Mds-Application-name=gdb, Mds-Application-Group-name=probed software, Mds- Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: gdb Mds-Application-location: /usr/bin/gdb Mds-Application-version: GNU gdb 5.1.1 Mds-Application-info: gdb: /usr/bin/gdb /usr/man/man1/gdb.1.gz /usr/share/man/ man1/gdb.1.gz Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # foobar, probed software, glob, local, grid dn: Mds-Application-name=foobar, Mds-Application-Group-name=probed software, M ds-Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: foobar Mds-Application-location: "foobar" location not available Mds-Application-version: "foobar" version not available Mds-Application-info: "foobar" info not available Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # vi, probed software, glob, local, grid dn: Mds-Application-name=vi, Mds-Application-Group-name=probed software, Mds-H ost-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: vi Mds-Application-location: /usr/bin/vi Mds-Application-version: VIM - Vi IMproved 6.1 (2002 Mar 24, compiled Jul 9 2 002 04-10-04) Mds-Application-info: vi: /usr/bin/vi Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # XFree86, probed software, glob, local, grid dn: Mds-Application-name=XFree86, Mds-Application-Group-name=probed software, Mds-Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: XFree86 Mds-Application-location: /usr/X11R6/bin/XFree86 Mds-Application-version: Mds-Application-info: XFree86: /usr/X11R6/bin/XFree86 /usr/bin/X11/XFree86 Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # emacs, probed software, glob, local, grid dn: Mds-Application-name=emacs, Mds-Application-Group-name=probed software, Md s-Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: emacs Mds-Application-location: /usr/bin/emacs Mds-Application-version: GNU Emacs 21.2.1 Mds-Application-info: emacs: /usr/bin/emacs /usr/lib/emacs /usr/share/emacs /u sr/man/man1/emacs.1.gz /usr/share/man/man1/emacs.1.gz Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # guile, probed software, glob, local, grid dn: Mds-Application-name=guile, Mds-Application-Group-name=probed software, Md s-Host-hn=glob,Mds-Vo-name=local,o=grid objectClass: MdsApplication Mds-Application-name: guile Mds-Application-location: /usr/bin/guile Mds-Application-version: Guile 1.4 Mds-Application-info: guile: /usr/bin/guile /usr/include/guile /usr/share/guil e Mds-validfrom: 20020814020636Z Mds-validto: 20020815020636Z Mds-keepto: 20020815020636Z # search result # numResponses: 10 # numEntries: 9
Screenshots!
Finally, some screenshots have been provided to further illustrate the usefulness of such an information provider once integrated into the MDS. The following pictures are screenshots captured of a java based LDAP browser on a GNU/Linux system running the MDS locally. Each snapshot shows off slightly different viewings of the data in the same application browser. Enjoy!
