GT 4.0.2 Incremental Release Notes

Introduction

The Globus Toolkit development team is pleased to provide a new incremental release of the Globus Toolkit 4.0. Users who wish to receive the latest bug fixes are encouraged to install this release. Highlights of 4.0.2 include:

  • Several important bug fixes in all components
  • Support for Tomcat 5.5.x
  • Improved WSDL parsing and serialization of complex types in C WS Core
  • Improved performance and a new programmer's tutorial for Python WS Core
  • Beta-quality integration between the SweGrid Accounting System (SGAS) tech preview and WS GRAM
  • Improved ODBC support in the Replica Location Service (RLS)
  • Improved resource management in OGSA-DAI WSRF
  • Scalability improvements to the Data Replication Service (DRS)
  • Job throughput improvements in WS GRAM

This page describes only the changes since the 4.0.1 release. For details regarding previous 4.0 releases, see:

Obtaining and Installing Software

To download this release, go to the 4.0.2 Download page.

For information on installing this release, go to the 4.0 Installation Guide.

Documentation

Use the existing documentation for 4.0, which starts at http://www.globus.org/toolkit/docs/4.0/. For more information about how our new documentation is structured, please see the new Documentation Overview.

Support

An overview on GT support can be found here.

Licensing

The Globus Toolkit is distributed and licensed for use under the terms of the Apache License, Version 2.0.

To view the licenses of the 3rd party software used by the developers of the Globus Toolkit, click here.

Usage Statistics

For full information about usage statistics collected by GT 4.0.2 and how to opt-out, see Usage Statistics Collection by the Globus Alliance.

Summary of Changes Since 4.0.1

Common Runtime Components

The following changes have occurred for Java WS Core:

  • Added Tomcat 5.5.x support.
  • Added TEST.pl for unit tests to integrate with Globus testing framework.
  • Fixed additional cases where WS-Addressing headers were not added to the SOAP Fault messages.

The following changes have occurred for C WS Core.

  • Improved WSDL parsing and bindings package generation
  • Improved serialization of complex types

The following changes have occurred for Python WS Core since 4.0.1:

  • Various performance modifications, approximately halving the time for a single rpc operation.
  • Added python "proxy" tools for creating and managing a RFC 3820 compliant proxy.
  • Complete separation between the pyGridWare installation and your site.
  • New drop-in services based on twisted "rpy" scripts.
  • New tool "wsdl2web.py" that creates a new web/grid services site from a WSDL definition, so one can move quickly from WSDL authoring to deployment.
  • Additional "layer" on top of soap stack, making it easier to use ZSI python objects.
  • Additional support for M2Crypto, which is replacing pyGlobus.security as the crypto package for message layer security.
  • GRAM client "pyGridWare/bin/ManagedJobClient.py"
  • New Tutorial at "http://dsd.lbl.gov/gtg/projects/pyGridWare/doc/tutorial/html/index.html".

Other than minor bug fixes, there were no changes to this release of XIO.

No changes have occurred for C Common Libraries.

Security

No changes, other than bug fixes have been made to Community Authorization Service (CAS).

Other than listed bug fixes, synchronization in delegation resource was improved.

Other than bug fixes, no changes have occurred for Message/Transport-level security.

No changes have occurred for WS Authorization Framework.

Other than bug fixes, no changes have occurred for Pre-Web Services Authorization & Authentication.

The following changes have occurred for MyProxy:

  • Updated GPT package from 2.3 to 3.5.
  • API changes: new trusted_retrievers member for struct myproxy_creds and struct myproxy_request_t
  • Added ability for myproxy-server to act as a CA by setting certificate_issuer options in myproxy-server.config.
  • Added support for CA username to DN resolution via LDAP using myproxy-server.config ca_ldap options.
  • Dropped support for certificate_issuer option in myproxy-server.config; use certificate_issuer_cert instead.
  • Added support for certificate-only authentication to the myproxy-server via myproxy-init -Z and myproxy-server.config trusted_retrievers options.
  • Added myproxy-server authentication support for Pubcookie (http://www.pubcookie.org/) granting cookie via pubcookie_granting_cert and pubcookie_app_server_key myproxy-server.config options.
  • Added myproxy-init --local_proxy option to create a local proxy credential after storing a credential on the myproxy-server.
  • Added myproxy-init --keyfile/--certfile options.
  • Install example myproxy-server.config to $GLOBUS_LOCATION/share/myproxy instead of potentially overwriting existing version in $GLOBUS_LOCATION/etc.
  • In myproxy-logon, added support for writing credential to standard output via '-o -' option; also added --quiet option.
  • Added myproxy-logon --no_credentials option to authenticate without retrieving credentials.
  • Added certificate_mapapp call-out for mapping usernames to certificate subject distinguished names for the CA module.
  • added certificate_extfile and certificate_extapp (call-out) for setting extensions in certificates issued by CA module.

Other than bug fixes, no changes have been made for SimpleCA since 4.0.1.

The following changes have occurred for GSI-OpenSSH:

  • Updated GPT package from 3.5 to 3.7.
  • Moved to OpenSSH 4.2p1.
  • Incorporated openssh-4.2p1-gsskex-20050926-2.patch from Simon Wilkinson adding support for gssapi-keyex authentication method (http://www.sxw.org.uk/computing/patches/openssh.html).
  • Incorporated High Performance Networking (HPN) patch (openssh-4.2p1-hpn11-none.diff) from Pittsburgh Supercomputing Center (http://www.psc.edu/networking/projects/hpn-ssh/) for improved network throughput over long and high bandwidth links.
  • Enabled PAM password authentication by default, unless PAM library and headers are unusable.
  • Added server-side GSIAllowLimitedProxy option to enable acceptance of limited proxy credentials.

Instructions on how to integrate the SweGrid Accounting System (SGAS) contribution with GT4 WS-GRAM now exist in the latest snapshot release at http://www.sgas.se/source/. It has been tested with the Fork and PBS adapters and can be considered beta- quality.

Data Management

No significant changes occured in RFT from last version

Aside from bug fixes, no changes have been made for GridFTP since 4.0.1. However, please remember that a protocol change was made in 4.0.1 to match a wuftpd extension regarding listings and resolve an issue with a function in the Java Cog Kit. This breaks technical protocol compatibility with RFC959 (FTP), but matches the behavior of 2.4.x servers. See Bug 3507 for details.

Other than bug fixes, no changes have occurred for RLS. The bug fixes introduced in this release improve RLS support for ODBC 3.0 drivers, add support for the popular unixODBC manager, and enable RLS to work on 64-bit platforms.

This section includes information for versions 2.0 and 2.1 of OGSA-DAI WSRF which is distributed with GT 4.0.2 (the GT4.0.1 release contained version 1.0 of the OGSA-DAI WSRF distribution).

  • The documentation bundled with the OGSA-DAI distribution explicitly mentions GT4.0.1. Everything will work fine with GT4.0.2 too.
  • The canonical documentation for the OGSA-DAI 2.1 release is contained in the release documentation so please use this for installation and usage.

Release 2.1

OGSA-DAI WSRF 2.1 is a WSRF-compliant version of OGSA-DAI and runs upon Globus Toolkit 4.0.2. It is not compatible with Globus Toolkit 3. The source release should still be compatible with Globus Toolkit 4.0. The main changes between this and the previous release are as follows:

OGSA-DAI WSRF 2.1 is almost identical to OGSA-DAI WSRF 2.0 which was released just one week earlier. This release was motivated by:

  • A need to allow data resource accessors to be notified whenever a session is terminated, so that session-related resource properties can be removed.
  • A bug fix to ensure that a single block of data can flow through a pipeline containing an inputStream activity and an outputStream activity.

If you have OGSA-DAI WSRF 2.0 and were not dependent on or concerned by these then there is no requirement for you to upgrade.

The additions / changes were as follows:

  • In core/src/java/:
    • Changed uk.org.ogsadai.activity.ActivityRequest
    • Changed uk.org.ogsadai.activity.CallThroughExternalPipe
    • Changed uk.org.ogsadai.activity.CallThroughPipe
    • Changed uk.org.ogsadai.service.resource.DataServiceResource
    • Changed uk.org.ogsadai.sessions.SessionManagerFactory
    • Changed uk.org.ogsadai.sessions.SessionConfigurationLoader
    • Changed uk.org.ogsadai.sessions.impl.SessionConfiguration
    • Changed uk.org.ogsadai.sessions.impl.TransientSessionManager
    • Changed uk.org.ogsadai.sessions.impl.TransientSessionManagerFactory
    • Added uk.org.ogsadai.dataresource.DataResourceListener
    • Added uk.org.ogsadai.dataresource.DataResourceEvent
    • Added uk.org.ogsadai.dataresource.DataResourceEventDispatcher
    • Added uk.org.ogsadai.dataresource.TestDataResource
    • Added uk.org.ogsadai.sessions.impl.TransientSessionConfiguration
  • In doc:
    • Changed tutorials/dataresource/howTo.html
    • Changed clients/clienttoolkit/ObtainingMetaData.html
    • Changed misc/HowToUpgrade.html
    • Changed misc/KnownProblems.html
    • Changed clients/clienttoolkit/index.html
    • Changed clients/clienttoolkit/SecureServices.html

Release 2.0

OGSA-DAI WSRF 2.0 is a WSRF-compliant version of OGSA-DAI and runs upon Globus Toolkit 4.0.1. It is not compatible with Globus Toolkit 3. The source release should still be compatible with Globus Toolkit 4.0.

The major features of this release are as follows:

  • OGSA-DAI WSRF 2.0 is compatible with Globus Toolkit 4.0.2 and the should still work with Globus Toolkit 4.0 (if you have the OGSA-DAI WSRF 2.0 binary distribution you'll need to get the source distribution and compile against Globus Toolkit 4.0).
  • Data service resources support concurrent request execution and queueing of requests.
  • All data service resources exposed by a specific service share the same settings for the number of concurrent requests that can be executed and the number that can be queued. These are specified at service deployment time.
  • Sessions are now supported. Multiple perform documents can be submitted which operate within the context of the same session. State can be stored in session attributes. Data transport operations now operate within the scope of a session.
  • Data service accessors have been provided. These manage access to a data resource on behalf of a data service resource. They each have their own specific configuration files.
  • Configurable data services provide an undeploy operation so that a service can be instructed to no longer expose a specific data service resource. This is reflected in the client toolkit.
  • There have been numerous performance improvements most notably in the refactoring of our java.sql.ResultSet to WebRowSet converters.
  • Client and server-side now support transport-level security between clients and services.
  • Client and server-side now support message-level security between clients and services.
  • Services can be configured to register themselves with a Globus MDS Index service running in the same container.
  • The OGSA-DAI data browser is now compatible with both OGSA-DAI WSRF 2.0 and OGSA-DAI WSI 2.0 services.

The main changes between this and the previous release are as follows:

  • The date in all OGSA-DAI XML Schema namespaces have been updated i.e.: http://ogsadai.org.uk/namespaces/2005/03/... has been updated to: http://ogsadai.org.uk/namespaces/2005/10/...
  • Data service resources now have a:
    • data resource class configuration file which specifies the data resource accessor to use.
    • session configuration file which specifies the maximum and default timeouts for sessions.
  • Files data service resources no longer have associated role map documents.
  • Data resource configuration files dataResource element no longer has the implementation attribute.
  • Data resource configuration files for file data resources now have a new XML Schema. This is in schema/ogsadai/xsd/data_resource_config_files.xsd. The main changes are:
    • The rolemap is no longer needed.
    • The productInfo is no longer needed.
  • Service interfaces have been completely refactored.
    • A terminate operation has been added. This is a placeholder and currently does nothing. It will be used to support request termination in future releases.
    • The updateRoleMaps operation for configurable data services no longer exists.
    • All OGSA-DAI-specific service operations throw one of five new fault types. The client toolkit has been completely refactored to reflect these fault types as Java exceptions.
  • SQL activities no longer implicitly convert java.sql.ResultSets to WebRowSet. A new activity - sqlResultsToXML - has been provided to make this explicit. Whereas in previous releases you would use a sqlQueryStatement activity, for example, you now use a sqlQueryStatement activity connected to a sqlResultsToXML activity.
  • Encrypted role map files are no longer supported and ogsadai-startup-tomcat.jar is no longer provided.
  • OGSA-DAI tutorial examples are compiled as part of a source distribution build and are distributed within ogsadai-examples.jar.
  • The Client Toolkit APIs have been modified to support sessions and concurrency. Some methods such as the poll method are now parameterised with a SessionID object. A user may attach session requirements to an ActivityRequest before performing it.
  • The request status resource property available in previous versions is no longer available. Instead a session request status resource property is created dynamically whenever a new session is created. This indicates the processing status of the request that is currently joined to the session.

As indicated in the release notes for the 4.0.1 release of the DRS, we have stabilized the interfaces to the service. The WSDL remains unchanged from the 4.0.1 release. The bug fixes for this release significantly improve the scalability of the service. We have tested concurrent operations and scalability and resolved problems in the DRS code as well as in its dependent components (Java WS Core and RFT) which affected DRS performance.

Information Services

Except for bug fixes, no changes have been made to the WS MDS Aggregator component since 4.0.1.

No changes have occurred for WS MDS Index service.

Other than a build bug fix, no changes have occurred for Pre-WS MDS since 4.0.1.

No changes have occurred for WS MDS Trigger service.

Since 4.0.1, some new example XSL transform files have been added to the WebMDS distribution.

Execution Management

Overall, WS GRAM in 4.0.2 is much improved over 4.0.1.

A focused (and ongoing) effort for large job loads to complete reliably has helped to identify a number of improvements. See Campaign 4197 for the complete details and note the 6 bugs marked as dependencies to this bug. One of the 6 improvements made (Enh 4330) was implementing a better algorithm for processing jobs from the internal job run queue. This improved responsiveness significantly. During a large throughput run to a 4.0.1 WS GRAM service, a separate simple /bin/date Fork job took > 10 minutes to return. In 4.0.2, the same Fork job was processed in ~2 minutes.

The WS GRAM testing infrastructure has been improved with automated throughput testing - http://skynet-login.isi.edu/gram-testing/. Nightly, the WS GRAM code is checked out of various CVS branches, built, and throughput tests are run. This has helped identify bugs more quickly and ease the effort of resolving bugs that can be difficult to reproduce. For a good example, see Bug 4235. The WS GRAM service is not released until these tests are passing consistently from the CVS release branch. This has helped us provide an overall better WS GRAM service.

There were 38 bug fixes since 4.0.1; here are a few worth highlighting:

  • Improvements were made to help identify service installation/configuration problems between WS GRAM and the local Resource Manager by improving the SEG reporting on fatal errors. Bug 4229
  • The variable "SCRATCH_DIRECTORY" was not being set in the job's environment. Bug 4192
  • WS GRAM was not failing the job on some fatal error conditions. Bug 4247 Bug 4279 Bug 3631
  • Job error reporting was improved. Bug 4273 Bug 4241
  • For the (default) INFO logging, WS GRAM will produce entry and exit logging for a job. Bug 3742

Other than a bug fix, no change has occurred for Pre-WS GRAM.

There are no changes for the Community Scheduler Framework (CSF) since the 4.0.1 release.

There's been a bug fix for Globus Teleoperations Control Protocol (GTCP) since 4.0.1.

The following changes have occurred for the Workspace Management Service (WMS) since 4.0.1 (the Technology Preview 5 release is included):

  • Added a new database persistence store.
  • Added a database based dynamic account implementation called LeasedAccountsResource.

Cross-component Tools

The following changes have occurred for JGlobus module of Java CoG Kit.

  • Added support for HTTP POST operation for the https/httpg URL connection handler.
  • The proxy refresh code was updated to reload the proxy file automatically whenever the file changes in any way.
  • The GridFTP client library was optimized to use one less thread during data transfer and to use buffered I/O.
  • Fixed the getDefault() methods in TrustedCertificates and CertificationRevocationList classes to differentiate scenarios when the default certificate locations are used vs. when non-default locations are used.
  • Updated GssSocket to automatically close itself if getOutputStream() or getInputStream() fails with an error.

The following changes have occurred for pyGlobus: internal refactoring, improved documentation, and maintenence on the myProxy module.

All Fixed Bugs Since 4.0.1

Common Runtime Components

The following bugs were fixed for Java WS Core:

  • Bug 3651: Misspelled Deserialization in exception message
  • Bug 3737: Support for socket timeouts on the server side
  • Bug 3750: Allow for setting the location where the persistence files are stored
  • Bug 3259: Generated Unix scripts cannot handle certain variables with spaces
  • Bug 3776: globus-start-container requires user to delete orphaned pid file
  • Bug 3899: globus-stop-container-detached does not give the container the chance to finish normally
  • Bug 3813: Race condition in container creation/shutdown
  • Bug 3814: Problem shutting down the container from within one of its threads
  • Bug 3863: ResourcePropertyTopic sendOldValue is ignored
  • Bug 3932: Problems with white space when passing options to Windows versions of command wrappers
  • Bug 3889: Newline missing when using core serialization utility
  • Bug 4107: Notification deadlock
  • Bug 4188: Container fails on expired proxy and cannot recover
  • Bug 4325: Exceptions of type "expectedType" could be more useful

The following bugs were fixed for C WS Core:

  • Bug 3641: C WS Security package dependency error.
  • Bug 3727: Invalid mutex unlock in client bindings.
  • Bug 3805: Bindings generator does not handle wsdl:documentation nodes
  • Bug 3904: Generated packages are not relocatable.
  • Bug 4086: Attributes of xsd:schema elements are not handled correctly when a schema file is included.
  • Bug 4168: Trailing slash in libdir paths causes build error on AIX
  • Bug 4171: Date serialization errors in WS C tests
  • Bug 4252: xsi:type attributes missing on complex types

Many bugs have been fixed for Python WS Core in the ZSI SOAP layer since 4.0.1. Check out the ZSI bug tracker on sourceforge.

The following bug was fixed for XIO:

  • Bug 3733: Potential buffer overflow in http driver

The following bugs were fixed in the C Common Libraries:

  • Bug 3392: globusrun-ws Bus error
  • Bug 3427: Issues with permission of file testcred.pem located in gl...
  • Bug 3624: Linux on sparc64 hw support
  • Bug 3713: i18n does not build on HP-UX/IA-64 platform
  • Bug 3936: Nightly build failure on AIX in trunk
  • Bug 3946: don't set $SASL_PATH unnecessarily
  • Bug 4240: compilation error when building globus_debug.c against MP...
  • Bug 4309: HP-UX IA64 broken due to not hardcoding libpaths

Security

The following bugs were fixed for CAS:

  • Bug 3259: Parsing errors in CAS Clients when environment varaibles are used.

The following bugs were fixed for Delegation Service:

  • Bug 3864: Fixes to prevent deadlock when notification listeners are not removed upon resource destroy.

The following bugs were fixed for Message/Transport-level security:

  • Bug 3891: Public credentials of client is not populated in PEER_SUBJECT object when secure transport is used.
  • Bug 4188: Container does not recover from expired proxy.

No bugs were fixed for WS Authorization Framework.

The following bugs were fixed for Pre-WS Authentication and Authorization since 4.0.1:

The following bugs were fixed for MyProxy:

  • Fixed memory errors found with myproxy-test -valgrind.
  • Fixed segmentation fault on reverse lookup in client.
  • Fixed SASL build problems on Darwin.
  • Fixed client-side memory leak in GSI_SOCKET_authentication_init().
  • Fixed compilation problem on platforms without setenv().
  • Fixed static builds using 'gpt-build -static'.
  • Fixes for SASL support.

The following bug fixes have been made to SimpleCA since 4.0.1:

  • Bug 3702: SimpleCA build fails on HP-UX/IA-64 platform
  • Bug 3958: bash-ism in SimpleCA
  • Bug 4164: Simple CA installation fails on RH 4 on IBM zSeries.

The following bugs were fixed for GSI-OpenSSH:

  • Bug 300: Only delegate credentials if GSI authentication is used.

SGAS is in technical preview and as such it is subject to rapid development and testing. During the course of this effort, several bugs and problems have been found and corrected.

Data Management

The following bugs were fixed for RFT:

  • Bug 2419: Need for RFT Package
  • Bug 3326: Remove globus_common dependency from RFT setup
  • Bug 3719: RFT Client doesn't retries after a transient transfer error
  • Bug 3685: RFT loosing connection with MySQL
  • Bug 3633: Directory removal not workinoth MySQL schema for RFT
  • Bug 3634: Problem with RFT IDs when using MySQL database
  • Bug 3640: RFT Unit Tests - System.err - Badly formatted numbers in ...
  • Bug 3644: MySQL Schema - duplicate primary key constraint name
  • Bug 3646: RFT Test - change default authzVal to "host" from "self
  • Bug 3653: patch to RFTTest.java - add log4j logging
  • Bug 3661: rft comman - default port should be 8443 instead of 8080
  • Bug 3662: rft command - fails with NegativeArraySizeException
  • Bug 3717: RFT client hangs up if all the transfers fail
  • Bug 3745: AllOrNone Notifications
  • Bug 3833: Problems with rft-delete
  • Bug 3925: Misconfigured database + Index service registration == me...
  • Bug 3943: RFT -help text and on-line doc are out of sync and need i....
  • Bug 3944: RFT messages on missing input file
  • Bug 3959: MT issues in RFT?
  • Bug 4225: error deleting a directory
  • Bug 3864: RFT errors impact GRAM stability

The following bugs were fixed for GridFTP:

  • Bug 3618: GridFTP fails to transfer a file if I've only group permissions
  • Bug 3632: CDUP/CWD dir parsing problems
  • Bug 3769: -cd wont create more than one level of directories
  • Bug 3902: 500 Command failed. : globus_xio: An end of file occurred
  • Bug 3919: GT 4.0.1 g-u-c dumps core in batch mode with restartability
  • Bug 3956: Globus-url-copy does not create multilevel directories.
  • Bug 4149: env GLOBUS_HOSTNAME=invalid globus-gass-server abort
  • Bug 4185: Recursive directory transfer failure in globus-url-copy
  • Bug 4208: gridftp server segfaults if it can't open its logfile
  • Bug 4280: GridFtp server could not take long home directory name

The following bugs were fixed for RLS:

  • Bug 3119: Support for building against unixODBC
  • Bug 3403: ODBC 3.0 return code (SQL_NO_DATA) not supported for searched update/delete SQL calls
  • Bug 3846: Workaround needed for bugs in ODBC, iodbc, MySQL ODBC, and Postgres ODBC for 64-bit platform support

No OGSA-DAI WSRF-specific bug fixes have been made for this release.

  • Bug 4202: Replication requests fail or perpetually pending due to erroneous RFT status code
  • Bug 4203: Internal replication operations performed synchronously due to WorkManager bug
  • Bug 4224: DRS misinterprets RFT failure status
  • Bug 4230: DRS deadlocks in thread scheduling

Information Services

The following bugs were fixed for WS MDS ServiceGroup:

  • Bug 3673: IndexOutOfBoundsException in EntryResourceProperty

The following bugs were fixed for WS MDS Aggregator:

  • Bug 4326: ExecutionAggregatorSource input stream bugs

No new bugs were fixed for the WS MDS Index Service

The following bug has been fixed for Pre-WS MDS since 4.0.1:

  • Bug 4167: SASL build failure in trunk and 4.0 branch

No new bugs were fixed for the WS MDS Trigger Service

No bugs have been fixed for WebMDS since 4.0.1.

Execution Management

The following bugs were fixed for WS GRAM:

  • Bug #3190: Staging jobs are not recoverable
  • Bug #3631: Delegation fetching errors should cause the job to fail i...
  • Bug #3642: Incorrect canceling of condor jobs
  • Bug #3738: self auth is applied to globusrun-ws, but host auth is st...
  • Bug #3698: user not allowed to override GLOBUS_LOCATION
  • Bug #3002: sudo path isn't set correctly in libexec/globus-sh-tools-...
  • Bug #3812: Absolute Windows paths not registering as absolute paths.
  • Bug #3772: Problem parsing certain PBS scheduler logs
  • Bug #3777: PBS SEG module can stop prematurely on PBS restart
  • Bug #3458: globusrun-ws isn't automatically releasing holds
  • Bug #3966: PBS SEG not refreshing
  • Bug #3931: Failed to retrieve Resource Properties from ManagedJobSer...
  • Bug #3935: GRAM setup scripts must be run from $GLOBUS_LOCATION/setu...
  • Bug #4187: AuthorizationHelper broken on container restart
  • Bug #3185: pbs job manager exit code
  • Bug #4229: Improve SEG fatal Error logging
  • Bug #2266: SEG needs to communicate errors better
  • Bug #3757: Remote exit status 0 ambiguity
  • Bug #4247: JSM registration error not failing job
  • Bug #4192: Missing SCRATCH_DIRECTORY env var.
  • Bug #4279: File mapping errors should fail the job.
  • Bug #4273: globusrun-ws error reports are ugly
  • Bug #4253: unable to monitor job for state changes
  • Bug #4297: PBS SEG module does not follow torque log rotation
  • Bug #4198: Bad reg-ex in condor poll script
  • Bug #4327: Script crashing with "Terminated"
  • Bug #4329: Missing context upon job recovery.
  • Bug #4330: Job Run Queue Processing Algorithm Inefficient
  • Bug #4331: Jobs disappear from state machine under heavy loads.
  • Bug #2397: optimize rft request if running in same container
  • Bug #4241: Allow for multi-line error from scheduler commands
  • Bug #4170: LoadLeveler 3.3.1 includes a GT4 GRAM Scheduler Adapter -...
  • Bug #3770: Incorrect GPT metadata breaks VDT builds
  • Bug #3687: %ENV mispelled %env in globus-job-manager-script.pl
  • Bug #3699: pbs scheduler event generator must be started after logs ...
  • Bug #1039: jobmanager-condor doesn't support the java universe
  • Bug #4159: Improve container error message when SEG execution fails
  • Bug #4111: locking / synch problems
  • Bug #3742: GRAM should log entry/exit of jobs to INFO level
  • Bug #4339: Condor adapter's poll subroutine not working for count > 1

The following bug was fixed for Pre-WS GRAM:

  • Bug 3355: $JAVA_HOME/bin/java vs [[PATH]] java
  • Bug 3411: Condor-G and pre-WS GT4 problems
  • Bug 4112: Add Solaris 5.9 to find-condor-tools utility
  • Bug 3798: Massive memory leaks in GRAM client library
  • Bug 3942: globus_gram_client_job_request() is MT-unsafe

No bug was fixed for the Community Scheduler Framework (CSF).

The following bugs were fixed for Message/Transport-level security:

  • Bug 3891: Public credentials of client is not populated in PEER_SUBJECT object when secure transport is used.
  • Bug 4188: Container does not recover from expired proxy.

The following bug was fixed for Globus Teleoperations Control Protocol (GTCP) since 4.0.1:

  • 4322 Compile errors with Java 1.5

No bugs were fixed for Workspace Management Service (WMS).

Cross-component Tools

The following bugs were fixed for JGlobus module of the Java CoG Kit:

  • Bug 3623: org.globus.gsi.CertUtil functionality/comment problem
  • Bug 3669: org/globus/util/I18n.java classloader issues
  • Bug 3819: DN parsing error
  • Bug 3963: Tabs in the grid-mapfile cause errors for secure services
  • Bug 4188: Container fails on expired proxy and cannot recover

The myProxy module was fixed to work with the later versions of myProxy.

Known Problems As of 4.0.2

Common Runtime Components

The following problems and limitations are known to exist for Java WS Core at the time of the 4.0.2 release:

  • Limitations

    • WS-Notification support:

      • Only the Simple topic dialect is supported (others can be added)
      • Only flat topic spaces are supported (architecture does allow for more advanced structures)
      • Actions on the precondition, selector and policy fields in a subscription are not supported
      • When a resource is removed its subscriptions are not removed automatically
    • Only XPath resource property queries are supported (others can be added)
    • A resource might not get destroyed at the exact time as indicated by the scheduled termination time. A sweeper thread that removes expired resources runs periodically (every 1 minute by default) so an expired resource might not get removed until the next time the sweeper thread runs.
    • SOAP messages with attachments are not supported. In fact, the Axis version distributed with GT was compiled without attachment support.
    • In certain cases, the "dialect" attribute of TopicExpression or QueryExpression is not serialized properly as defined in the schema. An "org.globus.dialect.attr.qualified" Java system property was added to control how the serialization of the dialect attribute. Please see the Bug 3513 for details.
  • Known Bugs

    • Bug 2471: Message security signature verification issues
    • Bug 2445: Same input and output messages in WSDL confuse Axis
    • Bug 2921: Support for TerminationTimeChangeRejectedFault
    • Bug 2926: Local transport does not work without a current MessageContext
    • Bug 3113: Processing by the WSDLPreprocessor produces output different depending on the JVM
    • Bug 3482: wsa:From is not set correctly when service calls another service
    • Bug 3483: xsd:anyType not serialized correctly

The following problems are known to exist for C WS Core at the time of the 4.0.2 release:

  • Multiple schemas which use the same namespace prefixes can confuse the WSDL parser.
  • Nillable elements are not serialized or deserialized correctly if the element does not contain the minOccurs="0" attribute
  • The service engine and clients are not thread-safe

Message layer security in Python WS Core is in a state of flux, the release of 4.0.2 came as a surprise and we were in the middle of a move. The client- side is working with globus-4.0.2rc1.

The following problems are known to exist for XIO at the time of the 4.0.2 release:

  • Bug 3082: Under certain circumstances there is a small memory leak.

The following problems are known to exist for C Common Libraries at the time of the 4.0.2 release:

  • Bug 484: Mem leak on deactivate_all
  • Bug 686: globus_mutex_lock() behaves differently on Unix -vs- Win32
  • Bug 930: inttypes.h included without testing if it's present
  • Bug 1141: Misleading error messages using globus-makefile-header
  • Bug 2315: Globus common fails to compile if there is no ipv6 support
  • Bug 2791: Command globus-domainname needs g11n
  • Bug 2793: Command globus-generalized-time needs g11n
  • Bug 2794: Command globus-hostname needs g11n
  • Bug 2801: Command globus-version needs g11n
  • Bug 3081: Small leak in libltldl
  • Bug 3107: Fatal error: tcp_init(): globus_io_tcp_create_listener() ...
  • Bug 3171: add RFC 2253 principal name to JAAS subject?
  • Bug 3436: globus_uuid MAC address retrieval doesn't work on solaris
  • Bug 3509: GLOBUS_COMMON_MODULE->GLOBUS_CALLBACK_MODULE deactivation...
  • Bug 3581: gpt/globus_core improvements
  • Bug 3610: globus-makefile-header returns bad field
  • Bug 4035: MacOS threading issues
  • Bug 4097: Signal handling probrem on MacOS X
  • Bug 4318: Pkgdata needs to make source dir explicit

Security

The following problem is known to exist for CAS at the time of the 4.0.2 release:

  • Bug 3371: cas-group-admin delete fails if grant all rights is given to the same group that was created.

No new problems are known to exist for the Delegation Service.

No new problems are known to exist for Message/Transport-level security.

No new problems are known to exist for WS Authorization Framework.

No problems are known to exist for Pre-WS Authentication & Authorization at the time of the 4.0.2 release.

The following problem is known to exist for MyProxy at the time of the 4.0.2 release:

  • Bug 2709: The MyProxy package isn't internationalized.

There are no new known problems for SimpleCA.

No problems are known to exist for GSI-OpenSSH at the time of the 4.0.2 release.

Besides the limited number of tested GRAM adapter backends, we are not aware of any outstanding issues at this time. That said, SGAS is a technical preview and to be regarded as pre-production quality.

Data Management

The following problems are known to exist for RFT at the time of 4.0.2 release:

  • Bug 3121: Bug in enforcing maximum active transfers in a container

The following problems are known to exist for GridFTP at the time of the 4.0.2 release:

  • Bug 3324: gridftpd croaking on startup - globus_l_io_tcp_contact_string failed
  • Bug 3214: Backend of GT4 GridFTPd should ignore ~/.globus/certificates
  • Bug 3387: client library doesn't respect GLOBUS_HOSTNAME
  • Bug 3622: gridftp server improperly handles detach option in config file
  • Bug 3761: globus_ftp_control library hangs at various places (depending on os)
  • Bug 3807: "not a plain file" could be "file not found"
  • Bug 3845: data channel authentication failure
  • Bug 4119: Malformed UTF-8 character problem in globus-ftp-client tests
  • Bug 4274: client library has no default striping layout.
  • Bug 4344: Globus-url-copy client does not reuse the connection when run in 3rd party mode

The following problems are known to exist for RLS at the time of the 4.0.2 release:

  • Threading/Libc Problems: set LD_ASSUME_KERNEL=2.2.5 in your environment and see Platform Notes for more information.
  • Bug 2601: Missing 'sigset' linking error
  • Bug 2821: GT3.9.5: jni.h not found when compiling RLS on MacOS X
  • Bug 3215: RLS client portability improvements
  • Bug 3253: GT4 build fails on Mac OS X 10.4 Tiger
  • Bug 3656: ACLs cannot be modified dynamically
  • Bug 3828: RLSClient crashes when a null parameter is passed to a method
  • Bug 3830: RLS searches for RLI tables in LRC database
  • Bug 4114: Java apis fail on 64 bit machine
  • Bug 4141: regexec call in auth.c's auth_getperms
  • Bug 4142: globus-rls-admin -s always indicates RLI does not exist

The following problems are known to exist for OGSA-DAI at the time of the 4.0.2 release and will be addressed in the future. If you have any others that you feel should be added to this list then please let us know.

You may also want to consult the platform-specific FAQ and general FAQ. Additional information may be posted in the advisories page which report any problems since the release, or the OGSA-DAI bugzilla which may also have information on any problems with the current release.

  • Performance:
    • Accessing the databaseSchema property for DB2/Cloudscape can sometimes retrieve unrelated meta-data and also cause OGSA-DAI to crash.
    • Accessing the databaseSchema property for Oracle causes a server-side java.lang.OutOfMemoryError.
    • Attempting to return a very large data set in a Response document can cause a server-side java.lang.OutOfMemoryError. This can be avoided by submitting an asynchronous request (one that uses an outputStream) and retrieving the results using a data service's data transport operations.
    • Queries of tables for millions of small rows can eventually cause a server-side java.lang.OutOfMemoryError even if using stream activities and data transport operations.
    • The fileWritingActivity attempts to read the whole file in a oner and so could cause a server-side java.lang.OutOfMemoryError for very large files.
    • Some databases, such as dBASE IV database on Microsoft Windows 2000, allow unusual characters, such as 0xC to be contained in certain field types. When field values containing this character, OGSA-DAI encodes them into invalid XML that subsequently causes Xerces-dependent components to raise an org.xml.sax.SAXParseException. Activities that have two outputs can sometimes cause java.lang.OutOfMemoryErrors to occur server-side if manipulating large amounts of data. This can arise when using the GZIP activities for example.
      • Such activities are driven by one of the two connected activities and the other activity can accumulate data without processing it. So, for example for the GZIP activities OGSA-DAI focuses on the GZIP meta-data output steam and data only flows from this stream when a file has been completely unzipped. Therefore for large files the whole contents of the file flows into the activity and is buffered without being sent on.
      • OGSA-DAI's GZIP activities will warn if memory usage is close to maximum and throw an OGSA-DAI exception server-side so that the web services container does not crash.
  • Error and exception handling:
    • The failure of a single activity in a request can cause the status of all activities to be set to ERROR, even those that are not connected and succeeded.
    • The data transport activities can gulp exceptions. For example if an sqlQueryStatement with a incorrectly formed SQL query statment is connected to an outputStream activity then attempting to pull data from the outputStream using the data transport operations results in no data - no indication of an error is given to the client.
    • The uk.org.ogsadai.exception.DAILogger methods can sometimes log the wrong line numbers. Searching the logs will usually reveal where the problem actually arose.
  • Security:
    • Encrypted role map files are currently unsupported.
    • If a client does not provide a certificate then deliverFromGridFTP and deliverToGridFTP in a request will fail.
  • File activities:
    • The fileAccessActivity encodes text line-by-line if Base 64 encoding is being used. This may give different results from encoding a whole file in a oner.
    • In fileWritingActivity, EOF means "end of file" when perLine has value false but "end of line" when perLine has value true.
    • In fileWritingActivity only strings from its input stream are handled. Other block types e.g. bytes are ignored.
  • General:
    • Note that when deploying new data service resources dynamically via configurable data service this only works if the JARs required by the data service resource (e.g. database driver JARs) are already within the web service container's library directories. If this is not the case then the container has to be restarted.
    • The data service terminate operation currently does nothing. It is intended as a placeholder for future development.
    • Meta-data from a database, returned in the databaseSchema property, can be case-sensitive depending upon the database. For example MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
    • The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang UnsupportedMethodException.
    • Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.
    • deliverFromGridFTP and deliverToGridFTP do not allow the setting of certain GridFTP parameters.
  • There is no support in the client toolkit for certain activities. These include:
    • directoryAccessActivity
    • fileAccessActivity
    • fileManipulationActivity
    • deliverFromFTP
    • deliverToFTP
    • gzipDecompression
  • The deliverToStream activity only works if services are deployed in Tomcat.

The following problems are known to exist for DRS at the time of the 4.0.2 release:

  • See Java WS Core buglist concerning container freeze, which may affect DRS performance: Bug 3502: Container freezes (CPU spin) when making remote call between ws-resources. See record for WORKAROUND.
  • Bug 4231: DRS does not implement subscription/notification

Information Services

There are no known problems with the WS MDS Aggregator API at the time of the 4.0.2 release.

No known problems exist for the WS MDS Index service at the time of the 4.0.2 release.

No known problems exist for Pre-WS MDS at the time of the 4.0.2 release.

No problems are known to exist for the WS-MDS Trigger service at the time of the 4.0.2 release.

The following problems are known to exist for WebMDS at the time of the 4.0.2 release:

Execution Management

The following problems are known to exist for WS GRAM at the time of the 4.0.1 release:

  • Bug #1562:Condor jobs fail when running globus-sh-exec job
  • Bug #2049:Batch providers need a namespace
  • Bug #3571:ant not found during install
  • Bug #3575:SEG dependent on GLOBUS_LOCATION env var
  • Bug #3579:takes over 4 seconds to start a single job with 4.0.1 bra...
  • Bug #3778:WS-GRAM dependant on GT2 job manager
  • Bug #2527:Add a failure case for non-existant queue to scheduler te...
  • Bug #2286:internationalization
  • Bug #3495:queue information, job count not reported to MDS
  • Bug #3726:GlobusRun error message typo
  • Bug #3844:Bad GPT metadata for globus_wsrf_gram_client_tools-1.0
  • Bug #3803:Default scratchDirectory doesn't exist
  • Bug #3384:Inconsistent jobType/count parameter semantics
  • Bug #3892:Out of date performance data?
  • Bug #3672:Streaming with PBS fails
  • Bug #3897:Must modify Globus in order to use authorization callouts
  • Bug #3911:Bug in GRAM documentation for Condor log file
  • Bug #3746:bad link on scheduler tutorial
  • Bug #4116:globus_module_activate( GLOBUS_GRAM_CLIENT_MODULE ) behav...
  • Bug #3675:wsrf gram client file streaming error
  • Bug #4153:gram scheduler test failing - submit202
  • Bug #4161:Managed Job Types, holding, state, userSubject and exitCo...
  • Bug #4162:fill in LINK TO SEG API Doc
  • Bug #4181:Allow File Staging To/From globusrun-ws application witho...
  • Bug #4182:Improve Condor/Fork Job Monitoring for reliability and se...
  • Bug #4178:no job output when streaming with globusrun-ws
  • Bug #4191:globusrun-ws job submission hangs
  • Bug #3910:Bad permissions on condor log file prevents job submissions
  • Bug #3529:setup/postinstall fatal errors should be warnings
  • Bug #4216:Empty submit_test/submitxxx.err causes FAILure in Local t...
  • Bug #4275:globus-gram-local-proxy-tool fails on Solaris
  • Bug #4388:globusrun-ws handles -self differently than java clients
  • Bug #3748:WS-GRAM Plugable Resource Manager Backend
  • Bug #3751:convert persistence data store from files to a database
  • Bug #4207:Enabling dynamic job description variables using softenv
  • Bug #4319:globus_scheduler_event_generator doesn't build with -stat...
  • Bug #3948:Service must release all of its resources on deactivation
  • Bug #4321:globusrun-ws freezes with "-job-delegate" option

The following problems are known to exist for Pre-WS GRAM at the time of the 4.0.2 release:

  • Bug #1037: globus-gram-reporter does not publish additional LSF job ...
  • Bug #1551: Race condition in job manager
  • Bug #1550: Fixes for race condition in job manager
  • Bug #1868: globus_gram_reporter is missing from GT 3.2.1 PreWS distr...
  • Bug #1934: Gatekeeper's syslog output cannot be controlled
  • Bug #2235: GT3.2.1 globus_gram_client_tools-4.3 patch broken
  • Bug #3373: globus removes the temporary job directory before pbs wri...
  • Bug #3425: personal gatekeeper does not work on AIX
  • Bug #3428: globus-personal-gatekeeper -list failing
  • Bug #2739: Gatekeeper AuthZ/Gridmap Callout result logging
  • Bug #2741: catching SIGSEGV if dynamic loading of authorization modu...
  • Bug #4150: do not use break in Perl, use last
  • Bug #4235: globus-job-manager doesn't exit if the job fails.
  • Bug #4335: Incorrectly terminated message in globus-gram-job-manager...

There are no known problems for the Community Scheduler Framework (CSF) at the time of the 4.0.2 release.

No problems are known to exist for Globus Teleoperations Control Protocol (GTCP) at the time of the 4.0.2 release.

As of 4.0.2, the LeasedAccountsResource implementation will quarantine an account if the expiration script fails for any reason; but currently we provide no user friendly mechanism for manually un-quarantining the account when the situation is cleared by an administrator. We are planning to release an update shortly.

Cross-component Tools

The following problems are known to exist for JGlobus module of the Java CoG Kit at the time of the 4.0.2 release:

  • Bug 1056: FTP client in extended block mode after receiving EODC does not wait for EOD on the same channel
  • Bug 2277: GridFTPClient bug Parallel, Passive, EBlockImageDCReader does not receive EOF
  • Bug 2345: Allow for a way to set cert and key location using environment variables
  • Bug 2413: A rare decrypt error
  • Bug 3299: Extended Key Usage certificate extension not supported

There are no known problems to exist for pyGlobus at the time of the 4.0.2 release.