XIO: Developer's Guide
Architecture
This page explains the external view of the Globus XIO architecture.
Graphical Overview
Globus XIO is broken down into two main components, framework and drivers. The following picture illustrates the architecture:
Framework
The Globus XIO framework manages IO operation requests that an application makes via the user API. The framework does not work to deliver the data in an IO operation nor does it manipulate the data. All of that work is done by the drivers. The framework's job is to manage requests and map them to the drivers' interface. It is the drivers themselves that are responsible for manipulating and transporting the data.
Drivers
A driver is the component of Globus XIO that is responsible for manipulating and transporting the user's data. There are two types of drivers:
- Transform drivers manipulate the data buffers passed to it via the user API and the XIO framework.
- Transport drivers are capable of sending the data over a wire.
Drivers are grouped into stacks, that is, one driver on top of
another. When an IO operation is requested by the Globus XIO, the framework passes
the operation request to every driver in the order they are in the stack.
When the bottom level driver (the transport driver) finishes shipping the
data, it passes the data request back to the XIO framework. Globus XIO
then delivers the request back up the stack in this manner until it
reaches the top, at which point the application is notified that the
request is complete.
In a driver stack there can only be one transport driver.
The reason for this is simple. The transport driver is the one responsible
for sending or receiving the data. Once this type of operation is performed,
it makes no sense to pass the request down the stack, the data has just been
transfered. It is now time to pass the operation back up the stack.
There can be many transform drivers in any given driver stack. A transform
driver can manipulate the requested operations as they pass.
Some good examples of transport drivers are security wrappers and compression.
However a transport driver can also add additional protocols.
For example, a stack could consist of a TCP transport driver and an HTTP
transform driver. The HTTP driver would be responsible for marshalling the
HTTP protocol and the TCP driver would be responsible for shipping that
protocol over the wire.
Example
The following picture illustrates a user application using Globus XIO to speak the GridFTP protocol across a TCP connection:
The user has built a stack consisting of one transform driver and one transport driver. TCP is the transport driver in this stack, and as all transport modules must be, it is at the bottom of the stack. Above TCP is the GSI transform driver which performs necessary messaging to authenticate a user and the integrity of the data.
The first thing the user application does after building the stack is call
the XIO user API function globus_xio_open().
The Globus XIO framework creates internal data structures for tracking this operation and then passes the operation request to the GSI driver.
The GSI driver has nothing to do before the underlying stack has opened a handle so it simply passes the request down the stack.
The request is then passed to the TCP driver.
The TCP driver then executes the socket level transport code contained within to establish a connection to the given contact string.
Once the TCP connection has been established the TCP driver will notify the XIO framework that it has completed its request and thereby the GSI driver will be notified that the open operation it had previously passed down the stack has now completed.
At this point the GSI driver will start the authentication processes (note
that at this point the user does not yet have an open handle). The GSI driver
has an open handle and upon it several sends and receives are performed to
authenticate the connection.
If the GSI driver is not satisfied with the authentication
process it closes the handle it has to the stack below it and tells the XIO
framework that it has completed the open request with an error.
If it is satisfied
it simply tells the XIO framework that it has completed the open operation.
The user is now notified that the open operation completed, and if it was successful they now have an open handle.
Other operations work in much the same way:
- A user posts a read.
- The read request is first delivered to the GSI driver.
- The GSI driver wraps the buffer and passes the modified buffer down the stack.
- The framework delivers the write request with the newly modified buffer to the TCP driver.
- The TCP driver writes the data across the socket mapped to this handle. When it finishes it notifies the framework.
- The framework notifies the GSI driver.
- The GSI driver has nothing more to do so it notifies the framework that it is complete.
- Framework then notifies the user.
Note: The above example is simplified for the purposes of understanding. There are optimizations built into Globus XIO which alter the course of events outlined above. However, conceptually the above is accurate.
Driver Interface
There is a well-defined interface to a driver. Drivers are modular components with specific tasks. The purpose of drivers in the Globus XIO library is extensibility. As more and more protocols are developed, more and more drivers can be written to implement these protocols. As new drivers are written they can be added to Globus XIO as either statically linked libraries or dynamically loaded libraries. In the case of dynamic loading, it is not even necessary to recompile existing source code. Each driver has a unique name according to the Globus XIO driver naming convention. A program simply needs to be aware of this name (this can obviously be passed in via the command line) and the Globus XIO framework is responsible for loading that driver.