com.google.appengine.tools.mapreduce

Class OutputWriter<O>

java.lang.Object

com.google.appengine.tools.mapreduce.OutputWriter<O>

Type Parameters:

O - type of values accepted by this output

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

BlobFileOutputWriter, DatastoreOutput.DatastoreOutputWriter, ForwardingOutputWriter, GoogleCloudStorageFileOutputWriter, InMemoryOutputWriter, ShardingOutputWriter, SliceSegmentingOutputWriter

public abstract class OutputWriter<O>
extends Object
implements Serializable

Writes key-value pairs.

Created by Output to write input for a given shard. Writers output for a shard as a number of slices, where the slicing is up to the caller.

beginShard() is called before any calls to beginSlice() or write(O) to setup the writer.

Each slice is written by calling beginSlice(), then write(O) any number of times, then endSlice(). Before the first slice, in between two slices, or after the final slice, the OutputWriter may go through serialization and deserialization.

At the end of the final slice, endShard() will be called after endSlice().

If a slice is aborted, there is no guarantee whether endSlice() will be called; however, if it is not called, the OutputWriter will not be serialized. If the slice is retried later, the OutputWriter serialized after the previous slice will be deserialized again.

Author:

ohler@google.com (Christian Ohler)

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
OutputWriter()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	allowSliceRetry()
void	beginShard() Will be called once before any calls to write.
void	beginSlice() Prepares the writer for writing after possibly having gone through serialization or deserialization.
void	cleanup() called after shards completes for any necessary cleanup.
void	close() Deprecated. Override endShard instead.
void	endShard() Called when no more output will be written to this writer.
void	endSlice() Prepares the writer for possible serialization.
long	estimateMemoryRequirement() Returns the estimated memory that will be used by this writer in bytes.
ShardContext	getContext() Returns the current context, or null if none.
void	open() Deprecated. Override beginShard instead.
void	setContext(ShardContext context) Used internally to sets the context to be used for the processing that follows.
abstract void	write(O value) Writes a value to the output.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OutputWriter

public OutputWriter()

Method Detail

setContext

public void setContext(ShardContext context)

Used internally to sets the context to be used for the processing that follows.

getContext

public ShardContext getContext()

Returns the current context, or null if none.

beginSlice

public void beginSlice()
                throws IOException

Prepares the writer for writing after possibly having gone through serialization or deserialization.

Throws:

IOException - in the event of failure

write

public abstract void write(O value)
                    throws IOException

Writes a value to the output.

Throws:

IOException

endSlice

public void endSlice()
              throws IOException

Prepares the writer for possible serialization.

Throws:

IOException - in the event of failure

open

@Deprecated
public void open()
          throws IOException

Deprecated. Override beginShard instead.

Throws:

IOException - in the event of failure

beginShard

public void beginShard()
                throws IOException

Will be called once before any calls to write. Prepares the writer for processing, after possibly having gone through serialization and deserialization.

Throws:

IOException - in the event of failure

estimateMemoryRequirement

public long estimateMemoryRequirement()

Returns the estimated memory that will be used by this writer in bytes. (This is normally just set to the size of the buffers used by the implementation)

close

@Deprecated
public void close()
           throws IOException

Deprecated. Override endShard instead.

Throws:

IOException - in the event of failure

endShard

public void endShard()
              throws IOException

Called when no more output will be written to this writer.

Throws:

IOException

allowSliceRetry

public boolean allowSliceRetry()

cleanup

public void cleanup()

called after shards completes for any necessary cleanup.