User documentation for CASTOR at the RAL Tier-1
CASTOR (CERN Advanced STORage manager) is a software technology used to manage a large tape-based storage system. It was developed at CERN, but it has been customised for the environment at RAL. This page has a brief explanation of the system and some guidance for uses, with pointers to more detailed information. The information is especially aimed at non-LHC VOs.
CASTOR components
CASTOR is a complex system with a number of more-or-less separate components. General information about CASTOR can be found on the CERN web site:
CASTOR home page at CERN CASTOR User GuideHowever, the configuration at RAL is somewhat different, so this page explains some of the RAL-specific details, as illustrated in this schematic diagram.
The tape storage system
The Tier-1 has a large robotic tape store with a potential capacity of around 5 Pb, and a total of ?? tape drives. This is shared between all the user communities, but some of the drives are reserved to particular VOs to prevent one VO starving another of resources. The mounting of tape cartridges to and from tape drives is managed by a piece of software called the Volume Drive Queue Manager (VDQM), but this is normally transparent to users.
As files are deleted from tapes, gaps are left. A piece of software called Repack is used to move files around on tape to recover this space. In addition there is a possiblility to group related files together on tape in file families, but this is somewhat complex and is not described here.
The Name Server
The CASTOR name server maintains a single namespace for all files stored in the system at RAL, using an underlying Oracle database. This is a unix-like namespace with a root of /castor/ads.rl.ac.uk/. The namserver host is castorns.ads.rl.ac.uk; usually this will be used by default, but if necessary it can be defined using the CNS_HOST environment variable.
Nameserver commands are prefixed with ns. These are generally rather low-level commands which are unlikely to be needed by most users, but the nsls command may be useful to list files in a similar way to the unix ls. The environment variable CASTOR_HOME can be defined as a prefix used with relative path names.
RFIO
RFIO (Remote File I/O) is a software protocol which provides Unix-like access to files in the CASTOR namespace. Note that this is not a true Unix filesystem, but a library which mimics the standard posix i/o functions and a set of command-line tools similar to the standard Unix tools. Particularly useful commands are rfdir (similar to nsls described above) and rfcp (similar to the Unix cp).
The current version of RFIO is not Grid-aware, it maps users according to their local Unix uid/gid, which limits its usefulness in a Grid environment, especially at RAL where users are in general no longer given local accounts. A Grid-aware version of RFIO has been developed and is expected to appear with the next major upgrade to CASTOR, but this may not be until the end of 2009. However, the latter is already in use with the DPM disk storage system used at many Grid sites, and the clients are therefore distributed with the standard grid User Interface software. Unfortunately these have the same names but are not interoperable with the CASTOR RFIO tools. The former are generally stored in /opt/lcg/bin and the latter in /usr/bin with the latter usually coming first in the PATH, so it may be necessary to ensure that you refer to them using the full path name.
The Stagers
A Stager is a software system which manages files on a pool of disk servers, and transfers (stages) those files to and from the tape system. In general CASTOR expects to use the disk servers as a cache, from which files can be deleted if space is needed as they can be subsequently recalled from tape. However, CASTOR has also been recently enhanced to support the use of disk-only files with no tape copy.
At RAL each of the major LHC experiments (ATLAS, CMS and LHCb) have their own stager to avoid contention. The other experiments all share a single stager. Requests in the stagers are scheduled using the LSF batch scheduler. The disk pools managed by a stager are divided into service classes which have a set of defined properties, for example whether the files are backed up to tape or not.
There are again some command-line tools to interact with the stager. The most useful of these is stager_qry -M <filename>, which gives some information about the status of the given file. These commands need to know the stager host name, which can be set using the STAGE_HOST environment variable. The current names are catlasstager.ads.rl.ac.uk, ccmsstager.ads.rl.ac.uk, clhcbstager.ads.rl.ac.uk and genstager.ads.rl.ac.uk.
The SRM
The information system
High-level client tools
Last modified Wed 10 December 2008
Switch to HTTPS . Website Help . Print View . Built with GridSite 1.4.3