com.google.appengine.tools.mapreduce.outputs

Class SliceSegmentingOutputWriter<O,WriterT extends OutputWriter<O>>

java.lang.Object

com.google.appengine.tools.mapreduce.OutputWriter<O>

com.google.appengine.tools.mapreduce.outputs.SliceSegmentingOutputWriter<O,WriterT>

Type Parameters:

O - type of values accepted by this output

WriterT - type of the output writer being written to

All Implemented Interfaces:

Serializable

public abstract class SliceSegmentingOutputWriter<O,WriterT extends OutputWriter<O>>
extends OutputWriter<O>

Segments data by using a separate writer for each slice. This is used by the Sort output so that there can be multiple sorted files per reducer.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SliceSegmentingOutputWriter()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	allowSliceRetry()
void	beginShard() Will be called once before any calls to write.
void	beginSlice() Creates a new writer.
protected abstract WriterT	createWriter(int sliceNumber) Creates a new writer.
void	endSlice() closes the current writer.
abstract long	estimateMemoryRequirement() Returns the estimated memory that will be used by this writer in bytes.
void	write(O value) Writes a value to the output.

Methods inherited from class com.google.appengine.tools.mapreduce.OutputWriter

cleanup, close, endShard, getContext, open, setContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SliceSegmentingOutputWriter

public SliceSegmentingOutputWriter()

Method Detail

beginShard

public void beginShard()

Description copied from class: OutputWriter

Will be called once before any calls to write. Prepares the writer for processing, after possibly having gone through serialization and deserialization.

Overrides:

beginShard in class OutputWriter<O>

beginSlice

public void beginSlice()
                throws IOException

Creates a new writer.

Overrides:

beginSlice in class OutputWriter<O>

Throws:

IOException - in the event of failure

createWriter

protected abstract WriterT createWriter(int sliceNumber)

Creates a new writer. This is called once per slice

endSlice

public void endSlice()
              throws IOException

closes the current writer.

Overrides:

endSlice in class OutputWriter<O>

Throws:

IOException - in the event of failure

write

public void write(O value)
           throws IOException

Description copied from class: OutputWriter

Writes a value to the output.

Specified by:

write in class OutputWriter<O>

Throws:

IOException

allowSliceRetry

public boolean allowSliceRetry()

Overrides:

allowSliceRetry in class OutputWriter<O>

estimateMemoryRequirement

public abstract long estimateMemoryRequirement()

Description copied from class: OutputWriter

Returns the estimated memory that will be used by this writer in bytes. (This is normally just set to the size of the buffers used by the implementation)

Overrides:

estimateMemoryRequirement in class OutputWriter<O>