com.google.appengine.tools.mapreduce.impl

Class MapShardTask<I,K,V>

java.lang.Object

com.google.appengine.tools.mapreduce.impl.WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

com.google.appengine.tools.mapreduce.impl.MapShardTask<I,K,V>

Type Parameters:

I - type of input values consumed by this mapper

K - type of intermediate keys produced by this mapper

V - type of intermediate values produced by this mapper

All Implemented Interfaces:

IncrementalTaskWithContext, IncrementalTask, Serializable

public class MapShardTask<I,K,V>
extends WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

Author:

ohler@google.com (Christian Ohler)

See Also:

Serialized Form

Field Summary

Fields inherited from class com.google.appengine.tools.mapreduce.impl.WorkerShardTask

claimedMemory, LIMITER

Constructor Summary

Constructors

Constructor and Description
MapShardTask(String mrJobId, int shardNumber, int shardCount, InputReader<I> in, Mapper<I,K,V> mapper, OutputWriter<KeyValue<K,V>> out, long millisPerSlice)

Method Summary

Methods

Modifier and Type	Method and Description
boolean	allowSliceRetry(boolean abandon)
protected void	callWorker(I input)
protected long	estimateMemoryRequirement()
protected String	formatLastWorkItem(I item)
InputReader<I>	getInputReader()
OutputWriter<KeyValue<K,V>>	getOutputWriter()
protected Worker<MapperContext<K,V>>	getWorker()
void	jobCompleted(Status status) A job completed callback to allow resource cleanup and compaction of the finalized state.
protected boolean	shouldCheckpoint(long timeElapsed)

Methods inherited from class com.google.appengine.tools.mapreduce.impl.WorkerShardTask

abbrev, cleanup, getContext, isDone, prepare, run, setFinalized, toString, wasFinalized

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MapShardTask

public MapShardTask(String mrJobId,
            int shardNumber,
            int shardCount,
            InputReader<I> in,
            Mapper<I,K,V> mapper,
            OutputWriter<KeyValue<K,V>> out,
            long millisPerSlice)

Method Detail

callWorker

protected void callWorker(I input)

Specified by:

callWorker in class WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

formatLastWorkItem

protected String formatLastWorkItem(I item)

Specified by:

formatLastWorkItem in class WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

shouldCheckpoint

protected boolean shouldCheckpoint(long timeElapsed)

Specified by:

shouldCheckpoint in class WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

Returns:

true iff a checkpoint should be performed.

estimateMemoryRequirement

protected long estimateMemoryRequirement()

Specified by:

estimateMemoryRequirement in class WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

getWorker

protected Worker<MapperContext<K,V>> getWorker()

Specified by:

getWorker in class WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

getOutputWriter

public OutputWriter<KeyValue<K,V>> getOutputWriter()

Specified by:

getOutputWriter in class WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

getInputReader

public InputReader<I> getInputReader()

Specified by:

getInputReader in class WorkerShardTask<I,KeyValue<K,V>,MapperContext<K,V>>

allowSliceRetry

public boolean allowSliceRetry(boolean abandon)

Parameters:

abandon - true if a retry is due to an abandoned lock.

Returns:

true if a slice retry after failure are permitted.

jobCompleted

public void jobCompleted(Status status)

Description copied from interface: IncrementalTask

A job completed callback to allow resource cleanup and compaction of the finalized state.

Parameters:

status - the status of the job