com.google.appengine.tools.mapreduce.impl

Class GoogleCloudStorageReduceInput<K,V>

java.lang.Object

com.google.appengine.tools.mapreduce.Input<KeyValue<K,Iterator<V>>>

com.google.appengine.tools.mapreduce.impl.GoogleCloudStorageReduceInput<K,V>

Type Parameters:

K - type of intermediate keys

V - type of intermediate values

All Implemented Interfaces:

Serializable

public class GoogleCloudStorageReduceInput<K,V>
extends Input<KeyValue<K,Iterator<V>>>

Defines the way the data is read in by the reducer. This consists of a number of files in GCS where the content is sorted KeyValue of K and a list of V written LevelDb format. KeyValuesMarshaller to unmarshall the individual records. To maintain the sorted order a MergingReader is used.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
GoogleCloudStorageReduceInput(FilesByShard files, Marshaller<K> keyMarshaller, Marshaller<V> valueMarshaller)

Method Summary

Methods

Modifier and Type	Method and Description
List<? extends InputReader<KeyValue<K,Iterator<V>>>>	createReaders() Returns a list of readers for this input.

Methods inherited from class com.google.appengine.tools.mapreduce.Input

getContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GoogleCloudStorageReduceInput

public GoogleCloudStorageReduceInput(FilesByShard files,
                             Marshaller<K> keyMarshaller,
                             Marshaller<V> valueMarshaller)

Method Detail

createReaders

public List<? extends InputReader<KeyValue<K,Iterator<V>>>> createReaders()

Description copied from class: Input

Returns a list of readers for this input. It is the Input's responsibility to determine an appropriate number of readers to split into. This could be specified by the user or determined algorithmically. The number of input readers returned determines the number of map shards.

Specified by:

createReaders in class Input<KeyValue<K,Iterator<V>>>