com.google.appengine.tools.mapreduce

Class Output<O,R>

java.lang.Object

com.google.appengine.tools.mapreduce.Output<O,R>

Type Parameters:

O - type of values accepted by this output

R - type returned by finish(java.util.Collection<? extends com.google.appengine.tools.mapreduce.OutputWriter<O>>)

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

BigQueryGoogleCloudStorageStoreOutput, BlobFileOutput, DatastoreOutput, GoogleCloudStorageFileOutput, GoogleCloudStorageLevelDbOutput, GoogleCloudStorageMapOutput, GoogleCloudStorageMergeOutput, GoogleCloudStorageSortOutput, InMemoryOutput, LevelDbOutput, MarshallingOutput, NoOutput, SizeSegmentedGoogleCloudStorageFileOutput, StringOutput

public abstract class Output<O,R>
extends Object
implements Serializable

Defines where output should go. Shards it into OutputWriters and may produce a result to be returned in MapReduceResult (see finish(java.util.Collection<? extends com.google.appengine.tools.mapreduce.OutputWriter<O>>)).

This class is really an interface that might be evolving. In order to avoid breaking users when we change the interface, we made it an abstract class.

Author:

ohler@google.com (Christian Ohler)

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Output()

Method Summary

Methods

Modifier and Type	Method and Description
abstract List<? extends OutputWriter<O>>	createWriters(int numShards) Returns a list of writers, one for each shard, for this output.
abstract R	finish(Collection<? extends OutputWriter<O>> writers) Returns a result to be made available through MapReduceResult.getOutputResult().
Context	getContext() Returns the current context, or null if none.
void	setContext(Context context) Used internally to sets the context to be used for the processing that follows.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Output

public Output()

Method Detail

setContext

public void setContext(Context context)

Used internally to sets the context to be used for the processing that follows.

getContext

public Context getContext()

Returns the current context, or null if none.

createWriters

public abstract List<? extends OutputWriter<O>> createWriters(int numShards)

Returns a list of writers, one for each shard, for this output.

Parameters:

numShards - The number of shards which should be equal to the number of OutputWriters returned.

finish

public abstract R finish(Collection<? extends OutputWriter<O>> writers)
                  throws IOException

Returns a result to be made available through MapReduceResult.getOutputResult().

This method allows the Output to inspect the final state of its OutputWriters to produce a final result object. For example, if the OutputWriters produce blobs in blobstore, finish could collect and return a list of blob IDs.

Should return null if no such result makes sense for this kind of output.

Called after all OutputWriters have been closed (with OutputWriter.close()). It is possible for this method to be called more than once with the same writers collection. It is also possible for the job to fail after this is invoked.

The writers argument will contain the same writers that createWriters(int) returned in the same order. Writers may be serialized and deserialized multiple times. typically, getWriter will have been called in a different JVM.

Throws:

IOException