NAME

Queue - interface to queuing systems.


SYNOPSIS

There will be a proper instructions later.


DESCRIPTION


VERSIONS

$Id: Queue.pm.html,v 1.1.1.1 2005/08/18 13:18:25 hunter Exp $

Copyright (c) European Bioinformatics Institute 2002


AUTHORS / ACKNOWLEDGEMENTS

Ville Silventoinen <vsi@ebi.ac.uk>


CONSTRUCTORS

new

 Description: Constructor allocates an anonymous hash, which
              is tied to the class.
 Arguments:   -
 Returns:     $self object


PUBLIC METHODS

submit

 Description: Submits a new job to the queue.
              Tool configuration MUST define following values:
              'queue'   => batch queue system name
              'jobid'   => unique job id
              'cmdline' => tool command line to be executed
              If the queue configuration contains tags that are specific
              to the tool, they must be set in the tool configuration.
              For example, 'lsf42' uses 'resource' and 'jobname' tags,
              which are set in the tool configuration. When queue
              configuration is created, the tool configuration is used
              to override all key-value pairs in queue configuration.
              This is why you should be familiar with the keys in the
              queue configuration to avoid unwanted conflicts.
 Arguments:   $tool    Tool name or Dispatcher::Config object reference.
                       When tool name is given, this method creates
                       Dispatcher::Config object for the tool and parses
                       the tool configuration.
              $input   Reference to an array (optional). Each array element
                       can be a string (path to the file), a FileHandle
                       reference or a FileHandle glob. Please read
                       "perldoc File::Copy" for additional information.
              $sync    0: asynchronous submission (default).
                       1: synchronous submission.
 Returns:     1, job ID on success
              0, msg    on failure

submitWithArgs

 Description: Submits a new job to the queue. This method takes the
              arguments for the command line and creates the Config
              object by using the tool name. If the arguments contain
              paths to files, it is the responsibility of the caller
              to ensure that the submitting host and executing host
              can access the files in a shared directory.
              Tool configuration MUST define following values:
              'binary' => binary path
              'queue'  => batch queue system name
              'jobid'  => unique job identifier
              If the queue configuration contains tags that are specific
              to the tool, they must be set in the tool configuration.
              For example, 'lsf42' uses 'resource' and 'jobname' tags,
              which are set in the tool configuration. When queue
              configuration is created, the tool configuration is used
              to override all key-value pairs in queue configuration.
              This is why you should be familiar with the keys in the
              queue configuration to avoid unwanted conflicts.
 Arguments:   $tname    Tool name.
              $args     Reference to an array containing command
                        line arguments (optional).
              $sync     0: asynchronous submission (default).
                        1: synchronous submission.
              $std      Reference to an array or hash. When an array is
                        passed, it defines paths to the redirection files
                        for stdin, stdout, stderr and exit code in this
                        order. When a hash is passed, the files are
                        defined in a following way:
                        stdin    => '/path/to/my.stdin.file'
                        stdout   => '/path/to/my.stdout.file'
                        stderr   => '/path/to/my.stderr.file'
                        exitcode => '/path/to/my.exitcode.file'
                        Any one of the redirection files can be undefined,
                        in which case the redirection is not done. For example,
                        if the application does not write to stdout, the caller
                        must use undef value (or empty string) in the array or
                        not define the 'stdout' key in the hash. Caller is
                        responsible for using unique file names, any existing
                        files with same names will be overwritten.
              $env      Reference to an array or hash. When an array
                        is passed, it defines which environment
                        variables are set in the tool command template
                        using the current environment values (in the
                        submitting host). When a hash is passed, the
                        environment variable values will be set from
                        the hash.
              $jobid    Job identifier (optional). Submit will be done using
                        this job ID. This is useful in cases when the caller
                        wants to do a synhronous submission and check/kill
                        the job before it is finished. See also method
                        createJobID.
 Returns:     1, job ID on success
              0, msg    on failure

kill

 Description: Terminates a job.
 Arguments:   $tool     Tool name or Dispatcher::Config object reference.
                        When tool name is given, this method creates
                        Dispatcher::Config object for the tool and parses
                        the tool configuration.
                        Tool configuration MUST define following values:
                        'queue' => batch queue system name
              $jobid    Job ID returned by submit.
 Returns:     1, ''  on success
              0, msg on failure

check

 Description: Returns job status.
 Arguments:   $tool     Tool name or Dispatcher::Config object reference.
                        When tool name is given, this method creates
                        Dispatcher::Config object for the tool and parses
                        the tool configuration.
                        Tool configuration MUST define following values:
                        'queue' => batch queue system name
              $jobid    Job ID returned by submit.
 Returns:     JOB_NOT_FOUND, ''  when job is not found
              JOB_PENDING,   ''  when job is pending
              JOB_RUNNING,   ''  when job is running
              JOB_DONE,      ''  when job finished ok
              JOB_FAILED,    ''  when job failed
              0,               msg on failure

createJobName

 Description: Creates job name.
              DEPRECATED Use createJobID method instead.
              Job name is same as job ID. Dispatcher uses solely job ID
              to identify the job. Job name is an LSF concept.
 Arguments:   $tname   Tool name.
 Returns:     1, job name on success
              0, msg      on failure

createJobID

 Description: Creates job identifier.
 Arguments:   $tname    Tool name.
 Returns:     1, job ID on success
              0, msg    on failure

cleanup

 Description: Removes any temporary files created during job submission.
              You should call this method AFTER the job has finished to
              remove temporary files created by Queue.
 Arguments:   $tname    Tool name.
              $jobid    Job ID.
 Returns:     1, number of deleted files on success
              0, msg on failure


PRIVATE METHODS

getToolSpecificValue

 Description: Returns value from configuration that depends on tool or
              tool group.
 Arguments:   $queue     Dispatcher::Config object for queue.
              $key       Configuration key.
              $tname     Tool name.
              $tgroup    Tool group.
 Returns:     value as string

getCmdSpecificValue

 Description: Returns value from configuration that depends on command.
 Arguments:   $queue    Dispatcher::Config object for queue.
              $key      Configuration key.
              $cmd      Command.
 Returns:     value as string

getQueueJobID

 Description: Returns queue specific jobid matching Dispatcher jobid.
 Arguments:   $jobid    Dispatcher jobid.
              $qname    Queue name.
              $queue    Dispatcher::Config object for queue.
              $dir      Directory where queue jobid is stored.
 Returns:     $qjobid

createCmdFile

 Description: Creates a command file.
 Arguments:   $cmdfile     Path to command file to be created.
              $template    Path to command file template.
              $tags        Reference to a hash containing tags as
                           key-value pairs. For example, key 'cmd'
                           will match tag [%cmd] in the template
                           and be replaced with 'cmd' value in hash.
 Returns:     1, ''  on success
              0, msg on error

createToolCmdFile

 Description: Creates a tool command file.
 Arguments:   $cmdfile     Path to command file to be created.
              $template    Path to command file template.
              $binary      Path to tool binary.
              $args        Reference to an array containing command line
                           arguments (optional).
              $std         Reference to an array or hash containing
                           redirection files (optional).
              $env         Reference to an array or hash containing
                           environment variables to be set (optional).
 Returns:     1, ''  on success
              0, msg on error

getUniqueID

 Description: Generates unique identifier.
 Arguments:   -
 Returns:     id string

copyInputFiles

 Description: Copies local input files to configured (shared) directory.
 Arguments:   $tool    Tool name or Dispatcher::Config object reference.
                       When tool name is given, this method creates
                       Dispatcher::Config object for the tool and parses
                       the tool configuration.
                       When tool accepts only one input file, use 'toolinput'
                       to define the path:
                       'toolinput' => path to input file
                       When tool accepts multiple input files, use
                       'toolinput.N' to define the paths:
                       'toolinput.1' => path to first input file
                       'toolinput.2' => path to second input file ...
              $input   Reference to an array. Each array element can be
                       a string (path to the file), a FileHandle reference or
                       a FileHandle glob. Please read "perldoc File::Copy" for
                       additional information.
 Returns:     1, ''  on success
              0, msg on failure

exec

 Description: Executes a command file and returns stdout and stderr.
              FIXME: IO::Select is not used because can_read() does not
              always return the file handles.
 Arguments:   $cmdfile    Command file to execute.
              $outref     Scalar reference for output (stdout).
              $errref     Scalar reference for errors (stderr).
 Returns:     1, ''  on success
              0, msg on error

_checkDirectory

 Description: Checks if given directory exists and tries to create it.
 Arguments:   $dir     Directory.
              $mode    Numeric mode (see DEFAULT_DIR_MODE).
 Returns:     1, ''  on success
              0, msg on failure

_createConfig

 Description: Creates Dispatcher::Config object.
 Arguments:   $tool     Tool name or
                        Tool object or
                        Dispatcher::Config object
              $jobid    Job identifier (optional). Used to set defaults.
 Returns:     1, reference to Dispatcher::Config object on success
              0, msg on failure

_init

 Description: Initializes the object.
 Arguments:   -
 Returns:     -