Queue (dirq 1.8 API)

All Superinterfaces:

Iterable<String>

All Known Implementing Classes:

QueueNull, QueueSimple
```
public interface Queue
extends Iterable<String>
```
Queue - object oriented interface to a directory based queue.

Description
The goal of this module is to offer a queue system using the underlying filesystem for storage, security and to prevent race conditions via atomic operations. It focuses on simplicity, robustness and scalability.
This module allows multiple concurrent readers and writers to interact with the same queue.
Different implementations are available so readers and writers can be written in different programming languages:
- A Perl implementation of the same algorithms is available at http://search.cpan.org/dist/Directory-Queue/
- A Python implementation of the same algorithms is available at https://github.com/cern-mig/python-dirq
There is no knowledge of priority within a queue. If multiple priorities are needed, multiple queues should be used.

Terminology
An element is something that contains one or more pieces of data. With QueueSimple queues, an element can only contain one binary string.
A queue is a "best effort" FIFO (First In - First Out) collection of elements.
It is very hard to guarantee pure FIFO behavior with multiple writers using the same queue. Consider for instance:
- Writer1: calls the add() method
- Writer2: calls the add() method
- Writer2: the add() method returns
- Writer1: the add() method returns
Who should be first in the queue, Writer1 or Writer2?
For simplicity, this implementation provides only "best effort" FIFO, i.e. there is a very high probability that elements are processed in FIFO order but this is not guaranteed. This is achieved by using a high-resolution timer and having elements sorted by the time their final directory gets created.

Locking
Adding an element is not a problem because the add() method is atomic.
In order to support multiple reader processes interacting with the same queue, advisory locking is used. Processes should first lock an element before working with it. In fact, the get() and remove() methods report a fatal error if they are called on unlocked elements.
If the process that created the lock dies without unlocking the element, we end up with a staled lock. The purge() method can be used to remove these staled locks.
An element can basically be in only one of two states: locked or unlocked.
A newly created element is unlocked as a writer usually does not need to do anything more with it.
Iterators return all the elements, regardless of their states.
There is no method to get an element state as this information is usually useless since it may change at any time. Instead, programs should directly try to lock elements to make sure they are indeed locked.

Security
There are no specific security mechanisms in this module.
The elements are stored as plain files and directories. The filesystem security features (owner, group, permissions, ACLs...) should be used to adequately protect the data.
By default, the process' umask is respected. See the class constructor documentation if you want an other behavior.
If multiple readers and writers with different uids are expected, the easiest solution is to have all the files and directories inside the toplevel directory world-writable (i.e. umask=0). Then, the permissions of the toplevel directory itself (e.g. group-writable) are enough to control who can access the queue.
Author:

Lionel Cons <lionel.cons@cern.ch>, Massimo Paladin <massimo.paladin@gmail.com> Copyright (C) CERN 2012-2016

Method Summary

All Methods Instance Methods Abstract Methods
Modifier and Type	Method and Description
`String`	`add(byte[] data)` Add byte array data to the queue.
`String`	`add(String data)` Add String data to the queue.
`String`	`addPath(String path)` Add the given file (identified by its path) to the queue and return the corresponding element name, the file must be on the same filesystem and will be moved to the queue.
`int`	`count()` Return the number of elements in the queue.
`String`	`get(String name)` Get the given locked element as String data.
`byte[]`	`getAsByteArray(String name)` Get the given locked element as byte array data.
`String`	`getId()` Return a unique identifier for the queue.
`String`	`getPath(String name)` Get the path of the given locked element.
`String`	`getQueuePath()` Return the path of the queue.
`boolean`	`lock(String name)` Lock an element in permissive mode.
`boolean`	`lock(String name, boolean permissive)` Lock an element.
`void`	`purge()` Purge the queue by removing unused intermediate directories, removing too old temporary elements and unlocking too old locked elements (aka staled locks); note: this can take a long time on queues with many elements.
`void`	`purge(int maxLock)` Purge the queue by removing unused intermediate directories, removing too old temporary elements and unlocking too old locked elements (aka staled locks); note: this can take a long time on queues with many elements.
`void`	`purge(int maxLock, int maxTemp)` Purge the queue by removing unused intermediate directories, removing too old temporary elements and unlocking too old locked elements (aka staled locks); note: this can take a long time on queues with many elements.
`void`	`remove(String name)` Remove a locked element from the queue.
`boolean`	`unlock(String name)` Unlock an element in non-permissive mode.
`boolean`	`unlock(String name, boolean permissive)` Unlock an element.

Methods inherited from interface java.lang.Iterable
forEach, iterator, spliterator

- Method Detail
  - getQueuePath
```
String getQueuePath()
```
    Return the path of the queue.
    
    Returns:
    
    queue path
  - getId
```
String getId()
```
    Return a unique identifier for the queue.
    
    Returns:
    
    unique queue identifier
  - add
```
String add(String data)
    throws IOException
```
    Add String data to the queue.
    
    Parameters:
    
    data - data to be added
    
    Returns:
    
    element name (as directory_name/file_name)
    
    Throws:
    
    IOException - if any file operation fails
  - add
```
String add(byte[] data)
    throws IOException
```
    Add byte array data to the queue.
    
    Parameters:
    
    data - data to be added
    
    Returns:
    
    element name (as directory_name/file_name)
    
    Throws:
    
    IOException - if any file operation fails
  - addPath
```
String addPath(String path)
        throws IOException
```
    Add the given file (identified by its path) to the queue and return the corresponding element name, the file must be on the same filesystem and will be moved to the queue.
    
    Parameters:
    
    path - path of the file to be added
    
    Returns:
    
    element name (as directory_name/file_name)
    
    Throws:
    
    IOException - if any file operation fails
  - get
```
String get(String name)
    throws IOException
```
    Get the given locked element as String data.
    
    Parameters:
    
    name - name of the element to be retrieved
    
    Returns:
    
    data associated with the given element
    
    Throws:
    
    IOException
  - getAsByteArray
```
byte[] getAsByteArray(String name)
               throws IOException
```
    Get the given locked element as byte array data.
    
    Parameters:
    
    name - name of the element to be retrieved
    
    Returns:
    
    data associated with the given element
    
    Throws:
    
    IOException
  - getPath
```
String getPath(String name)
```
    Get the path of the given locked element.
    This pathFile can be read but not removed, you must use the remove() method for this purpose.
    
    Parameters:
    
    name - name of the element
    
    Returns:
    
    path of the element
  - lock
```
boolean lock(String name)
      throws IOException
```
    Lock an element in permissive mode.
    
    Parameters:
    
    name - name of the element to be locked
    
    Returns:
    
    true on success, false if the element could not be locked
    
    Throws:
    
    IOException - if any file operation fails
  - lock
```
boolean lock(String name,
             boolean permissive)
      throws IOException
```
    Lock an element.
    
    Parameters:
    
    name - name of the element to be locked
    
    permissive - work in permissive mode
    
    Returns:
    
    true on success, false if the element could not be locked
    
    Throws:
    
    IOException - if any file operation fails
  - unlock
```
boolean unlock(String name)
        throws IOException
```
    Unlock an element in non-permissive mode.
    
    Parameters:
    
    name - name of the element to be unlocked
    
    Returns:
    
    true on success, false if the element could not be unlocked
    
    Throws:
    
    IOException - if any file operation fails
  - unlock
```
boolean unlock(String name,
               boolean permissive)
        throws IOException
```
    Unlock an element.
    
    Parameters:
    
    name - name of the element to be unlocked
    
    permissive - work in permissive mode
    
    Returns:
    
    true on success, false if the element could not be unlocked
    
    Throws:
    
    IOException - if any file operation fails
  - remove
```
void remove(String name)
     throws IOException
```
    Remove a locked element from the queue.
    
    Parameters:
    
    name - name of the element to be removed
    
    Throws:
    
    IOException - if any file operation fails
  - count
```
int count()
```
    Return the number of elements in the queue.
    Locked elements are counted but temporary elements are not.
    
    Returns:
    
    number of elements in the queue
  - purge
```
void purge()
    throws IOException
```
    Purge the queue by removing unused intermediate directories, removing too old temporary elements and unlocking too old locked elements (aka staled locks); note: this can take a long time on queues with many elements.
    It uses default value for maxTemp and maxLock
    
    Throws:
    
    IOException - if any file operation fails
  - purge
```
void purge(int maxLock)
    throws IOException
```
    Purge the queue by removing unused intermediate directories, removing too old temporary elements and unlocking too old locked elements (aka staled locks); note: this can take a long time on queues with many elements.
    
    Parameters:
    
    maxLock - maximum time for a locked element (in seconds); if set to 0, locked elements will not be unlocked; if set to null, the object's default value will be used
    
    Throws:
    
    IOException - if any file operation fails
  - purge
```
void purge(int maxLock,
           int maxTemp)
    throws IOException
```
    Purge the queue by removing unused intermediate directories, removing too old temporary elements and unlocking too old locked elements (aka staled locks); note: this can take a long time on queues with many elements.
    
    Parameters:
    
    maxLock - maximum time for a locked element (in seconds); if set to 0, locked elements will not be unlocked; if set to null, the object's default value will be used
    
    maxTemp - maximum time for a temporary element (in seconds); if set to 0, temporary elements will not be removed if set to null, the object's default value will be used
    
    Throws:
    
    IOException - if any file operation fails

Interface Queue

Description

Terminology

Locking

Security

Method Summary

Methods inherited from interface java.lang.Iterable

Method Detail

getQueuePath

getId

add

add

addPath

get

getAsByteArray

getPath

lock

lock

unlock

unlock

remove

count

purge

purge

purge