messif.buckets.storage.impl
Class DiskStorage<T>

java.lang.Object
  extended by messif.buckets.storage.impl.DiskStorage<T>
Type Parameters:
T - the class of objects stored in this storage
All Implemented Interfaces:
java.io.Serializable, Addible<T>, Index<T>, Lockable, ModifiableIndex<T>, LongStorage<T>, LongStorageIndexed<T>, Storage<T>, StorageIndexed<T>

public class DiskStorage<T>
extends java.lang.Object
implements LongStorageIndexed<T>, Lockable, java.io.Serializable

Disk based storage. The objects in this storage are stored in a file in the order of insertion. The address is the position within the file. Objects are serialized using the provided BinarySerializator.

See Also:
Serialized Form

Field Summary
protected  boolean bufferDirect
          Allocate the buffers for read/write operations as direct
protected  int bufferSize
          Buffer sizes for read/write operations
protected  int deletedFragments
          The number of deleted objects - the file space fragmentation is the ratio between this and objectCount
protected  java.io.File file
          The file with data
protected  java.nio.channels.FileChannel fileChannel
          The channel on the file with data
static java.lang.String FILENAME_PREFIX
          The prefix for auto-generated filenames
static java.lang.String FILENAME_SUFFIX
          The suffix for auto-generated filenames
protected  long fileOccupation
          The number of bytes currently stored in the file (excluding headers)
protected static int FLAG_CLOSED
          Header flag constant for indication whether the file was correctly closed
protected  java.lang.ref.WeakReference<BufferInputStream> inputStream
          Stream for reading data
protected  long maximalLength
          The maximal length of the file
protected  boolean modified
          Flag whether the file is modified
protected  java.lang.Thread modifiedThread
          Finalizer thread that writes a modified header
protected  int objectCount
          The number of objects currently stored in the file
protected  FileChannelOutputStream outputStream
          Stream for writing data
protected  BinarySerializator serializator
          Serializator responsible for storing (and restoring) binary objects in the file
protected  long startPosition
          The position in the file where this storage starts (the real data starts at startPosition + headerSize)
protected  java.lang.Class<? extends T> storedObjectsClass
          Class of objects that the this storage works with
 
Constructor Summary
DiskStorage(java.lang.Class<? extends T> storedObjectsClass, java.io.File file, boolean readonly, int bufferSize, boolean bufferDirect, boolean memoryMap, long startPosition, long maximalLength, BinarySerializator serializator)
          Creates a new DiskStreamStorage instance.
DiskStorage(DiskStorage<? extends T> copyAttributesDiskStorage, java.io.File file)
          Creates a new DiskStreamStorage instance.
 
Method Summary
 boolean add(T object)
          Adds the specified object to this instance.
static
<E> DiskStorage<E>
castToDiskStorage(java.lang.Class<E> storageObjectsClass, java.lang.Object object)
          Cast the provided object to DiskStorage with generics typing.
protected  boolean closeFileChannel()
          Close the associated file channel if this storage is no longer references from any index.
static
<T> DiskStorage<T>
create(java.lang.Class<T> storedObjectsClass, java.util.Map<java.lang.String,java.lang.Object> parameters)
          Creates a new disk storage.
 void destroy()
          Destroy this index.
 void finalize()
          Finalize this index.
 void flush(boolean syncPhysical)
          Flushes this storage and forces any buffered data to be written out.
 java.io.File getFile()
          Returns the file where the data of this storage are stored.
protected  BufferInputStream getInputStream(long position)
          Retrieves an input stream for this storage's file.
 java.lang.Class<? extends T> getStoredObjectsClass()
          Returns the class of objects that the this storage works with.
 boolean isModified()
          Returns true if the storage was modified since last open/flush.
 Lock lock(boolean blocking)
          Acquires a lock on this object.
protected  java.nio.channels.FileChannel openFileChannel(java.io.File file, boolean readonly)
          Opens the file channel on file and reads the header.
protected  BufferInputStream openInputStream(java.nio.channels.FileChannel fileChannel)
          Create input stream on the specified channel.
protected  FileChannelOutputStream openOutputStream()
          Create output stream over the current file channel.
 T read(long position)
          Reads the object stored at the specified address in this storage.
protected  void readHeader(java.nio.channels.FileChannel fileChannel, long position)
          Read header information from the file.
protected  void reconstructHeader(java.nio.channels.FileChannel fileChannel, long position)
          Reconstruct header information by seeking through the whole file.
 void remove(long position)
          Removes the object stored at the specified address in this storage.
protected  void remove(long position, int objectSize)
          Removes object with size objectSize at position position.
 LongStorageSearch<T> search()
          Returns a search for all objects in this index.
<C> LongStorageSearch<T>
search(IndexComparator<? super C,? super T> comparator, C key)
          Returns a search for objects in this index that have any of the specified keys.
<C> LongStorageSearch<T>
search(IndexComparator<? super C,? super T> comparator, C from, C to)
          Returns a search for objects in this index that are within the specified key-range.
<C> LongStorageSearch<T>
search(IndexComparator<? super C,? super T> comparator, java.util.Collection<? extends C> keys)
          Returns a search for objects in this index that have any of the specified keys.
 int size()
          Returns the number of objects stored in this storage.
 LongAddress<T> store(T object)
          Stores an object in this storage.
protected  void writeHeader(java.nio.channels.FileChannel fileChannel, long position, int flags)
          Write header information to the file.
 
Methods inherited from class java.lang.Object
clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

FILENAME_PREFIX

public static final java.lang.String FILENAME_PREFIX
The prefix for auto-generated filenames

See Also:
Constant Field Values

FILENAME_SUFFIX

public static final java.lang.String FILENAME_SUFFIX
The suffix for auto-generated filenames

See Also:
Constant Field Values

FLAG_CLOSED

protected static final int FLAG_CLOSED
Header flag constant for indication whether the file was correctly closed

See Also:
Constant Field Values

bufferSize

protected final int bufferSize
Buffer sizes for read/write operations


bufferDirect

protected final boolean bufferDirect
Allocate the buffers for read/write operations as direct


objectCount

protected transient int objectCount
The number of objects currently stored in the file


fileOccupation

protected transient long fileOccupation
The number of bytes currently stored in the file (excluding headers)


deletedFragments

protected transient int deletedFragments
The number of deleted objects - the file space fragmentation is the ratio between this and objectCount


file

protected final java.io.File file
The file with data


fileChannel

protected final transient java.nio.channels.FileChannel fileChannel
The channel on the file with data


startPosition

protected final long startPosition
The position in the file where this storage starts (the real data starts at startPosition + headerSize)


maximalLength

protected final long maximalLength
The maximal length of the file


serializator

protected final BinarySerializator serializator
Serializator responsible for storing (and restoring) binary objects in the file


storedObjectsClass

protected final java.lang.Class<? extends T> storedObjectsClass
Class of objects that the this storage works with


inputStream

protected transient java.lang.ref.WeakReference<BufferInputStream> inputStream
Stream for reading data


outputStream

protected final transient FileChannelOutputStream outputStream
Stream for writing data


modified

protected transient boolean modified
Flag whether the file is modified


modifiedThread

protected transient java.lang.Thread modifiedThread
Finalizer thread that writes a modified header

Constructor Detail

DiskStorage

public DiskStorage(java.lang.Class<? extends T> storedObjectsClass,
                   java.io.File file,
                   boolean readonly,
                   int bufferSize,
                   boolean bufferDirect,
                   boolean memoryMap,
                   long startPosition,
                   long maximalLength,
                   BinarySerializator serializator)
            throws java.io.IOException
Creates a new DiskStreamStorage instance.

Parameters:
storedObjectsClass - the class of objects that the new storage will work with
file - the file in which to create the bucket
readonly - if true, the storage will be opened in read-only mode (e.g. the store method will throw an exception)
bufferSize - the size of the buffer used for reading/writing
bufferDirect - the bucket is either direct (true) or array-backed (false)
memoryMap - flag whether to use memory-mapped I/O
startPosition - the position in the file where this storage starts
maximalLength - the maximal length of the file
serializator - the object responsible for storing (and restoring) binary objects
Throws:
java.io.IOException - if there was an error opening the bucket file

DiskStorage

public DiskStorage(DiskStorage<? extends T> copyAttributesDiskStorage,
                   java.io.File file)
            throws java.io.IOException
Creates a new DiskStreamStorage instance. All parameters are copied from the provided copyAttributesDiskStorage except for the file name. The bucket is always opened in read-write mode.

Parameters:
copyAttributesDiskStorage - the disk storage from which to copy parameters
file - the file in which to create the bucket
Throws:
java.io.IOException - if there was an error opening the bucket file
Method Detail

closeFileChannel

protected boolean closeFileChannel()
                            throws java.io.IOException
Close the associated file channel if this storage is no longer references from any index.

Returns:
true if the file channel was closed
Throws:
java.io.IOException - if there was a problem closing the file channel

finalize

public void finalize()
              throws java.lang.Throwable
Description copied from interface: ModifiableIndex
Finalize this index. All transient resources associated with this index are released. After this method is called, the store and retrieval methods' behavior is unpredictable.

Specified by:
finalize in interface ModifiableIndex<T>
Specified by:
finalize in interface Storage<T>
Overrides:
finalize in class java.lang.Object
Throws:
java.lang.Throwable - if there was an error while cleaning

destroy

public void destroy()
             throws java.lang.Throwable
Description copied from interface: ModifiableIndex
Destroy this index. This method releases all resources (transient and persistent) associated with this index. After this method is called, the store and retrieval methods' behavior is unpredictable.

Specified by:
destroy in interface ModifiableIndex<T>
Specified by:
destroy in interface Storage<T>
Throws:
java.lang.Throwable - if there was an error while cleaning

create

public static <T> DiskStorage<T> create(java.lang.Class<T> storedObjectsClass,
                                        java.util.Map<java.lang.String,java.lang.Object> parameters)
                             throws java.io.IOException,
                                    java.lang.InstantiationException
Creates a new disk storage. The additional parameters are specified in the parameters map with the following recognized key names:

Type Parameters:
T - the class of objects that the new storage will work with
Parameters:
storedObjectsClass - the class of objects that the new storage will work with
parameters - list of named parameters (see above)
Returns:
a new disk storage instance
Throws:
java.io.IOException - if something goes wrong when working with the filesystem
java.lang.InstantiationException - if the parameters specified are invalid (non existent directory, null values, etc.)

castToDiskStorage

public static <E> DiskStorage<E> castToDiskStorage(java.lang.Class<E> storageObjectsClass,
                                                   java.lang.Object object)
                                        throws java.lang.ClassCastException
Cast the provided object to DiskStorage with generics typing. The objects stored in the storage must be of the same type as the storageObjectsClass.

Type Parameters:
E - the class of objects stored in the storage
Parameters:
storageObjectsClass - the class of objects stored in the storage
object - the storage instance
Returns:
the generics-typed DiskStorage object
Throws:
java.lang.ClassCastException - if passed object is not a DiskStorage or the storage objects are incompatible

writeHeader

protected void writeHeader(java.nio.channels.FileChannel fileChannel,
                           long position,
                           int flags)
                    throws java.io.IOException
Write header information to the file. The objectCount, fileOccupation and deletedFragments are stored. The write is two-phase, with synchronized flag for opened/closed file.

Parameters:
fileChannel - the file channel to write the header to
position - the position in the file channel to write the header to
flags - the flags accompanied with the bucket; currently only "opened/closed" flag is used to resolve validity of the header
Throws:
java.io.IOException - if something goes wrong when working with the filesystem

readHeader

protected void readHeader(java.nio.channels.FileChannel fileChannel,
                          long position)
                   throws java.io.IOException
Read header information from the file. The objectCount, fileOccupation and deletedFragments variables are replaced by the values from the header.

Parameters:
fileChannel - the file channel to read the header from
position - the position in the file channel to read the header from
Throws:
java.io.IOException - if something goes wrong when working with the filesystem

reconstructHeader

protected void reconstructHeader(java.nio.channels.FileChannel fileChannel,
                                 long position)
                          throws java.io.IOException
Reconstruct header information by seeking through the whole file. The objectCount, fileOccupation and deletedFragments variables are replaced by the reconstructed values.

Parameters:
fileChannel - the file channel to read the header from
position - the position in the file channel to read the header from
Throws:
java.io.IOException - if something goes wrong when working with the filesystem

openInputStream

protected BufferInputStream openInputStream(java.nio.channels.FileChannel fileChannel)
                                     throws java.io.IOException
Create input stream on the specified channel.

Parameters:
fileChannel - the file channel
Returns:
the created input stream
Throws:
java.io.IOException - if something goes wrong when working with the filesystem

getInputStream

protected BufferInputStream getInputStream(long position)
                                    throws java.io.IOException
Retrieves an input stream for this storage's file.

Parameters:
position - the position on which to set the input stream
Returns:
the prepared input stream
Throws:
java.io.IOException - if something goes wrong when working with the filesystem

openOutputStream

protected FileChannelOutputStream openOutputStream()
                                            throws java.io.IOException
Create output stream over the current file channel.

Returns:
the created output stream
Throws:
java.io.IOException - if something goes wrong when working with the filesystem

openFileChannel

protected java.nio.channels.FileChannel openFileChannel(java.io.File file,
                                                        boolean readonly)
                                                 throws java.io.IOException
Opens the file channel on file and reads the header.

Parameters:
file - the file to open the channel on
readonly - if true, the channel will be opened in read-only mode
Returns:
the new file channel
Throws:
java.io.IOException - if something goes wrong when working with the filesystem

lock

public Lock lock(boolean blocking)
          throws java.lang.IllegalStateException
Description copied from interface: Lockable
Acquires a lock on this object. If the locking was successful, a Lock object is obtained and the lock holds until the Lock.unlock() method is called. Otherwise, this method blocks until the lock can be acquired or null is returned if a non-blocking call was required.

Specified by:
lock in interface Lockable
Parameters:
blocking - if true, this method will block until the lock is obtained
Returns:
a lock on this object or null
Throws:
java.lang.IllegalStateException - if the lock cannot be obtained

getStoredObjectsClass

public java.lang.Class<? extends T> getStoredObjectsClass()
Returns the class of objects that the this storage works with.

Returns:
the class of objects that the this storage works with

getFile

public java.io.File getFile()
Returns the file where the data of this storage are stored.

Returns:
the file where the data of this storage are stored

size

public int size()
Returns the number of objects stored in this storage.

Specified by:
size in interface Index<T>
Returns:
the number of objects stored in this storage

isModified

public boolean isModified()
Returns true if the storage was modified since last open/flush.

Returns:
true if the storage was modified since last open/flush

flush

public void flush(boolean syncPhysical)
           throws java.io.IOException
Flushes this storage and forces any buffered data to be written out.

Parameters:
syncPhysical - if true then also the file is flushed to be sure the data are really written to disk
Throws:
java.io.IOException - if there was an I/O error

store

public LongAddress<T> store(T object)
                     throws BucketStorageException
Description copied from interface: Storage
Stores an object in this storage. The address returned by this call can be used to retrieve or remove the object.

Specified by:
store in interface LongStorage<T>
Specified by:
store in interface Storage<T>
Parameters:
object - the object to store
Returns:
the address where the object has been stored
Throws:
BucketStorageException - if there was an error writing the data

remove

public void remove(long position)
            throws BucketStorageException
Description copied from interface: LongStorage
Removes the object stored at the specified address in this storage. This operation is optional and need not be implemented.

Specified by:
remove in interface LongStorage<T>
Parameters:
position - the address of the object to remove
Throws:
BucketStorageException - if there was an error deleting an object

remove

protected void remove(long position,
                      int objectSize)
               throws BucketStorageException
Removes object with size objectSize at position position.

Parameters:
position - the absolute position in the file
objectSize - the number of bytes to remove
Throws:
BucketStorageException - if there was an error writing to the file

read

public T read(long position)
       throws BucketStorageException
Description copied from interface: LongStorage
Reads the object stored at the specified address in this storage.

Specified by:
read in interface LongStorage<T>
Parameters:
position - the address of the object to read
Returns:
the object retrieved
Throws:
BucketStorageException - if there was an error reading the data

add

public boolean add(T object)
            throws BucketStorageException
Description copied from interface: Addible
Adds the specified object to this instance.

Specified by:
add in interface Addible<T>
Parameters:
object - the object to be added
Returns:
true if the addition was successful
Throws:
BucketStorageException - if there was an error adding the object

search

public LongStorageSearch<T> search()
                            throws java.lang.IllegalStateException
Description copied from interface: Index
Returns a search for all objects in this index.

Specified by:
search in interface Index<T>
Specified by:
search in interface ModifiableIndex<T>
Specified by:
search in interface LongStorageIndexed<T>
Specified by:
search in interface StorageIndexed<T>
Returns:
a search for all objects in this index
Throws:
java.lang.IllegalStateException - if there was an error initializing the search on this index

search

public <C> LongStorageSearch<T> search(IndexComparator<? super C,? super T> comparator,
                                       java.util.Collection<? extends C> keys)
                            throws java.lang.IllegalStateException
Description copied from interface: Index
Returns a search for objects in this index that have any of the specified keys. The equality is checked exclusively by using the specified comparator, thus key need not necessarily be of the same class as the objects stored in this index and also consistency with equals is not required.

Note that objects are not necessarily returned in the order defined by the comparator

Specified by:
search in interface Index<T>
Specified by:
search in interface ModifiableIndex<T>
Specified by:
search in interface LongStorageIndexed<T>
Specified by:
search in interface StorageIndexed<T>
Type Parameters:
C - the type of the keys used by the search
Parameters:
comparator - compares the keys with the stored objects
keys - the keys to search for (at least one key must be given)
Returns:
a search for objects in this index
Throws:
java.lang.IllegalStateException - if there was an error initializing the search on this index

search

public <C> LongStorageSearch<T> search(IndexComparator<? super C,? super T> comparator,
                                       C key)
                            throws java.lang.IllegalStateException
Description copied from interface: Index
Returns a search for objects in this index that have any of the specified keys. The equality is checked exclusively by using the specified comparator, thus key need not necessarily be of the same class as the objects stored in this index and also consistency with equals is not required.

Note that objects are not necessarily returned in the order defined by the comparator

Specified by:
search in interface Index<T>
Specified by:
search in interface ModifiableIndex<T>
Specified by:
search in interface LongStorageIndexed<T>
Specified by:
search in interface StorageIndexed<T>
Type Parameters:
C - the type of the key used by the search
Parameters:
comparator - compares the key with the stored objects
key - the key to search for
Returns:
a search for objects in this index
Throws:
java.lang.IllegalStateException - if there was an error initializing the search on this index

search

public <C> LongStorageSearch<T> search(IndexComparator<? super C,? super T> comparator,
                                       C from,
                                       C to)
                            throws java.lang.IllegalStateException
Description copied from interface: Index
Returns a search for objects in this index that are within the specified key-range. The key boundaries [from, to] need not necessarily be of the same class as the objects stored in this index, however, the comparator must be able to compare the boundaries and the internal objects.

Note that objects are not returned in the order defined by the comparator

Specified by:
search in interface Index<T>
Specified by:
search in interface ModifiableIndex<T>
Specified by:
search in interface LongStorageIndexed<T>
Specified by:
search in interface StorageIndexed<T>
Type Parameters:
C - the type the boundaries used by the search
Parameters:
comparator - compares the boundaries [from, to] with the stored objects
from - the lower bound on the searched objects, i.e. objects greater or equal are returned
to - the upper bound on the searched objects, i.e. objects smaller or equal are returned
Returns:
a search for objects in this index
Throws:
java.lang.IllegalStateException - if there was an error initializing the search on this index