README

ARC Middleware
==============

The NorduGrid middleware (or the Advanced Resource Connector, ARC) is
an open source software solution enabling production quality
computational and data grids.

Since the first release (May 2002) the middleware is deployed and
being used in production environments. Emphasis is put on scalability,
stability, reliability and performance of the middleware. A growing
number of grid projects, like Swegrid, DCGC, NDGF chose ARC as their
middleware. The largest production grid of the world is running on ARC
too.

This release (October 2007) is a technology preview version of the
next generation of ARC. It is based on a service container - the
Hosting Environment Daemon (HED) - and different grid capabilities are
implemented as Web Services residing in HED. Currently, an OGSA BES
compliant execution service - ARC Resource-coupled EXecution service
(A-REX) - and an echo service (for testing purposes) are included, but
the set of services is rapidly growing.


Dependencies
============

The middleware is written in C/C++ and some smaller part in Perl. ARC
builds upon standard Open Source libraries and solutions. Building the
software from source or installing a precompiled binary requires
different external packages, furthermore the client and server
packages have different dependencies too. The detailed compile and
runtime external dependencies are given in the INSTALL file, below a
merged list of the requirements (build, install, server, client) is
shown:

  Mandatory (on client as well as server side):
    o gnu make, autotools (autoconf>=2.56)
    o C++ compiler and library
    o libtool
    o pkg-config
    o gthread-2.0 version 2.4.7 or later
    o glibmm-2.4 version 2.4.7 or later
    o libxml-2.0 version 2.6.0 or later
    o openssl version 0.9.7a or later
    o gsoap++ version 2.7 or later
      (http://www.cs.fsu.edu/~engelen/soap.html)
    o e2fsprogs-devel
    o doxygen

  Optional (mainly aplicable on server side):
    o swig version 1.3.28 or later
    o java sdk 1.4 or later for Java bindings
    o python-devel for Python bindings
    o Grid Packaging Tools (GPT) (http://www.gridpackagingtools.org/)
    o Globus Toolkit 4 (http://www.globus.org/) which contains
      - Globus RLS client
      - Globus FTP client
      - Globus RSL


Getting the software
====================

The middleware is free to deploy anywhere by anybody. Pre-built binary
releases for a dozen of Linux platforms can be downloaded from
ftp.nordugrid.org.

The software is released under the GNU General Public License (GPL)
(see the LICENSE file).

The NorduGrid repository hosts the source code, and provides all the
required external software which are not part of a standard Linux
distribution.


Building & Installation
=======================

The ARC middleware can be built from the openly available source code
provided as a tarball. The build is using the GNU autotools. This
means that a standard installation into /usr/local can be made with:

  ./configure
  make
  make install

If you wish to install into <directory>, use the following instead:

  ./configure --prefix=<directory>
  make
  make install


Grid Certificates
=================

In order to use ARC, a grid certificate is necessary. More information
about grid certificates and how to get them can be found on
http://www.nordugrid.org/documents/certificate_howto.html

So far, only the certificates without password can be used for MCCTLS (both 
the client side and server side).
Certificate with password (for client side) will be supported when certificate 
delegation is available.


ARC Server Setup & Configuration
================================

The configuration of the ARC server is specified in an XML file, the
location of which is specified as a command line argument with the -c
option. An example of a configuration file that sets up an A-REX
execution service is shown below:

<?xml version="1.0"?>
<ArcConfig 
  xmlns="http://www.nordugrid.org/schemas/ArcConfig/2007"
  xmlns:tcp="http://www.nordugrid.org/schemas/ArcMCCTCP/2007"
  xmlns:arex="http://www.nordugrid.org/schemas/a-rex/Config"
>
  <Server>
    <Foreground/>
    <Pidfile>./arcserver.pid</Pidfile>
    <Logger level="WARNING">./logfile</Logger>
  </Server>
  <ModuleManager>
    <Path>/usr/local/lib/arc/</Path>
  </ModuleManager>
  <Plugins><Name>mcctcp</Name></Plugins>
  <Plugins><Name>mcctls</Name></Plugins>
  <Plugins><Name>mcchttp</Name></Plugins>
  <Plugins><Name>mccsoap</Name></Plugins>
  <Plugins><Name>arex</Name></Plugins>
  <Chain>
    <Component name="tcp.service" id="tcp">
      <next id="tls"/> 
      <tcp:Listen><tcp:Port>60000</tcp:Port></tcp:Listen>
    </Component>
    <Component name="tls.service" id="tls">
      <next id="http"/>
      <KeyPath>/etc/grid-security/key.pem</KeyPath>
      <CertificatePath>/etc/grid-security/cert.pem</CertificatePath>
      <CACertificatePath>/etc/grid-security/ca.pem</CACertificatePath>
    </Component>
    <Component name="http.service" id="http">
      <next id="soap">POST</next>
    </Component>
    <Component name="soap.service" id="soap">
      <next id="plexer"/>
    </Component>
    <Plexer name="plexer.service" id="plexer">
      <next id="a-rex">/arex</next>
    </Plexer>
    <Service name="a-rex" id="a-rex">
      <arex:endpoint>https://localhost:60000/arex</arex:endpoint>
      <arex:username>nobody</arex:username>
      <arex:gmconfig>/etc/arc.conf</arex:gmconfig>
    </Service>
  </Chain>
</ArcConfig>


Grid Manager
============

In order to use the A-REX service, you need to run a Grid-Manager as
well. There is not yet a Grid-Manager available in the new generation
of version of ARC, but the Grid-Manager from ARC 0.6.x can be used
meanwhile. How to install and set up that Grid-Manager is described in
detail on http://www.nordugrid.org/documents/ng-server-install.html
and in NORDUGRID-TECH-14 "The ARC Computational Job Management Module
- A-REX: Description and Administrator's Manual".


Starting the Server
===================

To start the ARC server, type the following on the command line:

  /usr/local/sbin/arcserver -c /etc/arcserv.xml &


The A-REX Client
================

The configuration of the ARC client is specified in an XML file. The
location of the configuration file is specified by the environment
variable ARC_AREX_CONFIG. If there is no such environment variable,
the configuration file is assumed to be arex_client.xml in the current
working directory. An example configuration file is shown below:

<ArcConfig
  xmlns="http://www.nordugrid.org/schemas/ArcConfig/2007"
  xmlns:tcp="http://www.nordugrid.org/schemas/ArcMCCTCP/2007">
  <ModuleManager>
    <Path>/usr/local/lib/arc/</Path>
  </ModuleManager>
  <Plugins><Name>mcctcp</Name></Plugins>
  <Plugins><Name>mcctls</Name></Plugins>
  <Plugins><Name>mcchttp</Name></Plugins>
  <Plugins><Name>mccsoap</Name></Plugins>
  <Chain>
    <Component name='tcp.client' id='tcp'>
      <tcp:Connect>
        <tcp:Host>127.0.0.1</tcp:Host>
        <tcp:Port>60000</tcp:Port>
      </tcp:Connect>
    </Component>
    <Component name='tls.client' id='tls'>
      <next id='tcp'/>
      <KeyPath>/etc/grid-security/key.pem</KeyPath>
      <CertificatePath>/etc/grid-security/cert.pem</CertificatePath>
      <CACertificatePath>/etc/grid-security/ca.pem</CACertificatePath>
    </Component>
    <Component name='http.client' id='http'>
      <next id='tls'/>
      <Method>POST</Method>
      <Endpoint>/arex</Endpoint>
    </Component>
    <Component name='soap.client' id='soap' entry='soap'>
      <next id='http'/>
    </Component>
  </Chain>
</ArcConfig>


Using the Client
================

To submit a job, type

  /usr/local/bin/apsub <JSDL-file> <JobID-file>

where <JSDL-file> is the location of a file that contains the job
specification in JSDL format and <JobID-file> the location of a file
in which the Job ID will be stored.

To query the status of a job, type

  /usr/local/bin/apstat <JobID-file>

where <JobID-file> the location of the file in which the Job ID is
stored (the file created when the job was submitted).

To terminate a job, type

  /usr/local/bin/apkill <JobID-file>

where <JobID-file> the location of the file in which the Job ID is
stored (the file created when the job was submitted).

More detailed about how to use the client tools are explained in
NORDUGRID-TECH-15 "WS-Based ARC Clients".


Contributing
============

The open source development of the ARC middleware is coordinated by
the NorduGrid Collaboration. Currently, the main contributor is the
KnowARC project (www.knowarc.eu), but the Collaboration is open to new
members. Contributions from the community to the software and the
documentation is welcomed. Sources can be downloaded from the Web site
or Subversion.

The developers maintain a Task List which describes outstanding issues
that have to be addressed in the framework of the NorduGrid
development. Bugs marked as 'feature requests' in the NorduGrid
bugzilla also contain open development areas. For a more detailed
description, write access to Subversion and further questions, write
to the nordugrid-discuss mailing list. Ongoing and completed Grid
Research projects and student assignments related to the middleware
are listed on the Web site as well.


Support, documentation, mailing lists, contact
==============================================

User support and site installation assistance is provided via the
request tracking system available at nordugrid-support@nordugrid.org.
In addition, NorduGrid runs a couple of mailing lists, among which the
nordugrid-discuss mailing list is a general forum for all kind of
issues related to the ARC middleware.

NorduGrid deploys the Bugzilla problem tracking system. Feature and
enhancement requests, as well as discovered problems, should be
reported there.

Research papers, overview talks, reference manuals, user guides,
installation instructions, conference presentations, FAQ and even
tutorial materials can be fetched from the documentation section of
www.nordugrid.org

Contact information is kept updated on the www.nordugrid.org web site.