com.google.appengine.tools.mapreduce.inputs

Class LevelDbInputReader

java.lang.Object

com.google.appengine.tools.mapreduce.InputReader<ByteBuffer>

com.google.appengine.tools.mapreduce.inputs.LevelDbInputReader

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

GoogleCloudStorageLevelDbInputReader

public abstract class LevelDbInputReader
extends InputReader<ByteBuffer>

Reads LevelDB formatted input. (Which is produced by LevelDbOutputWriter) If you want to read about the format it is here: "https://github.com/google/leveldb/" This implementation deviates from the specification above, in that it allows blocks to be zero padded regardless of how much data is in the block, rather than only if the block is within 6 bytes of full. Data is read in as needed, so the only state required to reconstruct this class when it is serialized is the offset. In the event that corrupt data is encountered a CorruptDataException is thrown. If this occurs, do not continue to attempt to read. Behavior is not guaranteed. For internal use only. User code cannot safely depend on this class.

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor and Description
	LevelDbInputReader()
protected	LevelDbInputReader(int blockSize)

Method Summary

Methods

Modifier and Type	Method and Description
void	beginShard() Performs setup at the beginning of the shard.
void	beginSlice() Prepares the InputReader for reading, after possibly having gone through serialization and deserialization.
abstract ReadableByteChannel	createReadableByteChannel()
void	endShard() Calls close on the underlying stream.
protected long	getBytesRead()
ByteBuffer	next() Returns the next input value, or throws NoSuchElementException when there is no more input data.

Methods inherited from class com.google.appengine.tools.mapreduce.InputReader

close, endSlice, estimateMemoryRequirement, getContext, getProgress, open, setContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LevelDbInputReader

public LevelDbInputReader()

LevelDbInputReader

protected LevelDbInputReader(int blockSize)

Method Detail

createReadableByteChannel

public abstract ReadableByteChannel createReadableByteChannel()

Returns:

A Serializable ReadableByteChannel from which data may be read.

getBytesRead

protected long getBytesRead()

beginShard

public void beginShard()

Description copied from class: InputReader

Performs setup at the beginning of the shard. This method is invoked before the first call to InputReader.beginSlice(). It will not be invoked again unless the shard restarts. When a shard is restarted, this method is invoked and the input should be read from the beginning.

Overrides:

beginShard in class InputReader<ByteBuffer>

endShard

public void endShard()
              throws IOException

Calls close on the underlying stream.

Overrides:

endShard in class InputReader<ByteBuffer>

Throws:

IOException - in the event of failure

beginSlice

public void beginSlice()

Description copied from class: InputReader

Prepares the InputReader for reading, after possibly having gone through serialization and deserialization.

Overrides:

beginSlice in class InputReader<ByteBuffer>

next

public ByteBuffer next()
                throws IOException,
                       NoSuchElementException

Returns the next input value, or throws NoSuchElementException when there is no more input data. This is done rather than providing a hasNext() function, to allow the implementation to deal with streams and remain serializable, even if the item being read is not.

Specified by:

next in class InputReader<ByteBuffer>

Throws:

IOException

NoSuchElementException