com.google.appengine.tools.mapreduce.outputs

Class ItemSegmentingOutputWriter<O>

java.lang.Object

com.google.appengine.tools.mapreduce.OutputWriter<O>

com.google.appengine.tools.mapreduce.outputs.ForwardingOutputWriter<O>

com.google.appengine.tools.mapreduce.outputs.ItemSegmentingOutputWriter<O>

Type Parameters:

O - the type of the written values.

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

SizeSegmentingOutputWriter

public abstract class ItemSegmentingOutputWriter<O>
extends ForwardingOutputWriter<O>

Segments data by using a separate writer each time shouldSegment(O) returns true. This is used by the Merge output in the event that there are too many files to merge in one pass.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ItemSegmentingOutputWriter()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	allowSliceRetry()
void	beginShard() Will be called once before any calls to write.
protected abstract OutputWriter<O>	createNextWriter(int fileNum)
abstract long	estimateMemoryRequirement() Returns the estimated memory that will be used by this writer in bytes.
ShardContext	getContext() Returns the current context, or null if none.
protected OutputWriter<O>	getDelegate()
void	setContext(ShardContext context) Used internally to sets the context to be used for the processing that follows.
protected abstract boolean	shouldSegment(O value)
void	write(O value) Writes a value to the output.

Methods inherited from class com.google.appengine.tools.mapreduce.outputs.ForwardingOutputWriter

beginSlice, endShard, endSlice

Methods inherited from class com.google.appengine.tools.mapreduce.OutputWriter

cleanup, close, open

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ItemSegmentingOutputWriter

public ItemSegmentingOutputWriter()

Method Detail

beginShard

public void beginShard()
                throws IOException

Description copied from class: OutputWriter

Will be called once before any calls to write. Prepares the writer for processing, after possibly having gone through serialization and deserialization.

Overrides:

beginShard in class ForwardingOutputWriter<O>

Throws:

IOException - in the event of failure

write

public void write(O value)
           throws IOException

Description copied from class: OutputWriter

Writes a value to the output.

Specified by:

write in class OutputWriter<O>

Throws:

IOException

shouldSegment

protected abstract boolean shouldSegment(O value)

createNextWriter

protected abstract OutputWriter<O> createNextWriter(int fileNum)

getDelegate

protected OutputWriter<O> getDelegate()

Specified by:

getDelegate in class ForwardingOutputWriter<O>

estimateMemoryRequirement

public abstract long estimateMemoryRequirement()

Description copied from class: OutputWriter

Returns the estimated memory that will be used by this writer in bytes. (This is normally just set to the size of the buffers used by the implementation)

Overrides:

estimateMemoryRequirement in class ForwardingOutputWriter<O>

allowSliceRetry

public boolean allowSliceRetry()

Overrides:

allowSliceRetry in class ForwardingOutputWriter<O>

setContext

public void setContext(ShardContext context)

Description copied from class: OutputWriter

Used internally to sets the context to be used for the processing that follows.

Overrides:

setContext in class ForwardingOutputWriter<O>

getContext

public ShardContext getContext()

Description copied from class: OutputWriter

Returns the current context, or null if none.

Overrides:

getContext in class ForwardingOutputWriter<O>