com.google.appengine.tools.mapreduce

Class MapReduceSettings.Builder

java.lang.Object

com.google.appengine.tools.mapreduce.MapReduceSettings.Builder

Enclosing class:

MapReduceSettings

public static class MapReduceSettings.Builder
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
protected String	backend
protected String	baseUrl
protected int	maxShardRetries
protected int	maxSliceRetries
protected int	millisPerSlice
protected String	module
protected String	workerQueueName

Constructor Summary

Constructors

Constructor and Description
MapReduceSettings.Builder()
MapReduceSettings.Builder(MapReduceSettings settings)
MapReduceSettings.Builder(MapSettings settings)

Method Summary

Methods

Modifier and Type	Method and Description
MapReduceSettings	build()
protected MapReduceSettings.Builder	self()
B	setBackend(String backend) Deprecated. Use modules instead.
B	setBaseUrl(String baseUrl) Sets the base URL that will be used for all requests related to this job.
MapReduceSettings.Builder	setBucketName(String bucketName) Sets the GCS bucket that will be used for temporary files.
MapReduceSettings.Builder	setMapFanout(int mapFanout) The maximum number of files the map stage will write to at the same time.
B	setMaxShardRetries(int maxShardRetries) The number of times a Shard can fail before it gives up and fails the whole job.
B	setMaxSliceRetries(int maxSliceRetries) The number of times a Slice can fail before triggering a shard retry.
MapReduceSettings.Builder	setMaxSortMemory(Long maxMemory) The maximum memory the sort stage should allocate (in bytes).
MapReduceSettings.Builder	setMergeFanin(int mergeFanin) Number of files the merge stage will read at the same time.
B	setMillisPerSlice(int millisPerSlice) Sets how long a worker will process items before endSlice is called and progress is checkpointed to datastore.
B	setModule(String module) Specifies the Module that the job will run on.
MapReduceSettings.Builder	setSortBatchPerEmitBytes(int sortBatchPerEmitBytes) Size (in bytes) of items to batch together in the output of the sort.
MapReduceSettings.Builder	setSortReadTimeMillis(int sortReadTimeMillis) The maximum length of time sort should spend reading input before it starts sorting it and writing it out.
B	setWorkerQueueName(String workerQueueName) Sets the TaskQueue that will be used to queue the job's tasks.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

baseUrl

protected String baseUrl

module

protected String module

backend

protected String backend

workerQueueName

protected String workerQueueName

millisPerSlice

protected int millisPerSlice

maxShardRetries

protected int maxShardRetries

maxSliceRetries

protected int maxSliceRetries

Constructor Detail

MapReduceSettings.Builder

public MapReduceSettings.Builder()

MapReduceSettings.Builder

public MapReduceSettings.Builder(MapReduceSettings settings)

MapReduceSettings.Builder

public MapReduceSettings.Builder(MapSettings settings)

Method Detail

self

protected MapReduceSettings.Builder self()

setBucketName

public MapReduceSettings.Builder setBucketName(String bucketName)

Sets the GCS bucket that will be used for temporary files. If this is not set or null the app's default bucket will be used.

setMapFanout

public MapReduceSettings.Builder setMapFanout(int mapFanout)

The maximum number of files the map stage will write to at the same time. A higher number may increase the speed of the job at the expense of more memory used during the map and sort phases and more intermediate files created. Using the default is recommended.

setMaxSortMemory

public MapReduceSettings.Builder setMaxSortMemory(Long maxMemory)

The maximum memory the sort stage should allocate (in bytes). This is used to lower the amount of memory it will use. Regardless of this setting it will not exhaust available memory. Null or unset will use the default (no maximum) Using the default is recommended.

setSortReadTimeMillis

public MapReduceSettings.Builder setSortReadTimeMillis(int sortReadTimeMillis)

The maximum length of time sort should spend reading input before it starts sorting it and writing it out. Using the default is recommended.

setSortBatchPerEmitBytes

public MapReduceSettings.Builder setSortBatchPerEmitBytes(int sortBatchPerEmitBytes)

Size (in bytes) of items to batch together in the output of the sort. (A higher value saves storage cost, but needs to be small enough to not impact memory use.) Using the default is recommended.

setMergeFanin

public MapReduceSettings.Builder setMergeFanin(int mergeFanin)

Number of files the merge stage will read at the same time. A higher number can increase the speed of the job at the expense of requiring more memory in the merge stage. Using the default is recommended.

build

public MapReduceSettings build()

setBaseUrl

public B setBaseUrl(String baseUrl)

Sets the base URL that will be used for all requests related to this job. Defaults to

setModule

public B setModule(String module)

Specifies the Module that the job will run on. If this is not set or null, it will run on the current module.

setBackend

@Deprecated
public B setBackend(String backend)

Deprecated. Use modules instead.

setWorkerQueueName

public B setWorkerQueueName(String workerQueueName)

Sets the TaskQueue that will be used to queue the job's tasks.

setMillisPerSlice

public B setMillisPerSlice(int millisPerSlice)

Sets how long a worker will process items before endSlice is called and progress is checkpointed to datastore.

setMaxShardRetries

public B setMaxShardRetries(int maxShardRetries)

The number of times a Shard can fail before it gives up and fails the whole job.

setMaxSliceRetries

public B setMaxSliceRetries(int maxSliceRetries)

The number of times a Slice can fail before triggering a shard retry.