Modifier and Type | Method and Description |
---|---|
static <K,V> KeyValue<K,V> |
KeyValue.of(K k,
V v) |
Modifier and Type | Method and Description |
---|---|
static <K,V> Marshaller<KeyValue<K,V>> |
Marshallers.getKeyValueMarshaller(Marshaller<K> keyMarshaller,
Marshaller<V> valueMarshaller)
Returns a
Marshaller for key-value pairs based on keyMarshaller and
valueMarshaller . |
static <K,V> Marshaller<KeyValue<K,? extends Iterable<V>>> |
Marshallers.getKeyValuesMarshaller(Marshaller<K> keyMarshaller,
Marshaller<V> valueMarshaller)
Returns a
Marshaller for key-values pairs based on keyMarshaller and
valueMarshaller . |
Modifier and Type | Method and Description |
---|---|
KeyValue<K,V> |
KeyValueMarshaller.fromBytes(ByteBuffer input) |
KeyValue<K,Iterable<V>> |
KeyValuesMarshaller.fromBytes(ByteBuffer input) |
Modifier and Type | Method and Description |
---|---|
protected void |
ReduceShardTask.callWorker(KeyValue<K,Iterator<V>> input) |
protected String |
ReduceShardTask.formatLastWorkItem(KeyValue<K,Iterator<V>> item) |
ByteBuffer |
KeyValuesMarshaller.toBytes(KeyValue<K,? extends Iterable<V>> keyValues) |
ByteBuffer |
KeyValueMarshaller.toBytes(KeyValue<K,V> keyValues) |
Modifier and Type | Method and Description |
---|---|
FilesByShard |
GoogleCloudStorageSortOutput.finish(Collection<? extends OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>>> writers) |
FilesByShard |
GoogleCloudStorageMergeOutput.finish(Collection<? extends OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>>> writers) |
FilesByShard |
GoogleCloudStorageMapOutput.finish(Collection<? extends OutputWriter<KeyValue<K,V>>> outputWriters) |
Constructor and Description |
---|
MapShardTask(String mrJobId,
int shardNumber,
int shardCount,
InputReader<I> in,
Mapper<I,K,V> mapper,
OutputWriter<KeyValue<K,V>> out,
long millisPerSlice) |
ReduceShardTask(String mrJobId,
int shardNumber,
int shardCount,
InputReader<KeyValue<K,Iterator<V>>> in,
Reducer<K,V,O> reducer,
OutputWriter<O> out,
long millisPerSlice) |
Modifier and Type | Method and Description |
---|---|
InputReader<KeyValue<ByteBuffer,Iterator<ByteBuffer>>> |
MergeShardTask.getInputReader() |
InputReader<KeyValue<ByteBuffer,ByteBuffer>> |
SortShardTask.getInputReader() |
OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>> |
MergeShardTask.getOutputWriter() |
OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>> |
SortShardTask.getOutputWriter() |
Modifier and Type | Method and Description |
---|---|
protected void |
SortShardTask.callWorker(KeyValue<ByteBuffer,ByteBuffer> input) |
protected void |
MergeShardTask.callWorker(KeyValue<ByteBuffer,Iterator<ByteBuffer>> input) |
protected String |
SortShardTask.formatLastWorkItem(KeyValue<ByteBuffer,ByteBuffer> item) |
protected String |
MergeShardTask.formatLastWorkItem(KeyValue<ByteBuffer,Iterator<ByteBuffer>> item) |
Constructor and Description |
---|
MergeShardTask(String mrJobId,
int shardNumber,
int shardCount,
InputReader<KeyValue<ByteBuffer,Iterator<ByteBuffer>>> in,
OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>> out,
int sortReadTimeMillis) |
MergeShardTask(String mrJobId,
int shardNumber,
int shardCount,
InputReader<KeyValue<ByteBuffer,Iterator<ByteBuffer>>> in,
OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>> out,
int sortReadTimeMillis) |
SortShardTask(String mrJobId,
int shardNumber,
int shardCount,
InputReader<KeyValue<ByteBuffer,ByteBuffer>> in,
SortWorker worker,
OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>> out,
int sortReadTimeMillis) |
SortShardTask(String mrJobId,
int shardNumber,
int shardCount,
InputReader<KeyValue<ByteBuffer,ByteBuffer>> in,
SortWorker worker,
OutputWriter<KeyValue<ByteBuffer,List<ByteBuffer>>> out,
int sortReadTimeMillis) |
Modifier and Type | Method and Description |
---|---|
void |
IdentityMapper.map(KeyValue<K,V> input) |
Modifier and Type | Class and Description |
---|---|
class |
ShardingOutputWriter<K,V,WriterT extends OutputWriter<KeyValue<K,V>>>
Writes
KeyValue s out to a number of output writers each corresponding to a different
shard A sharder and an Output object are provided in the constructor. |
Modifier and Type | Method and Description |
---|---|
void |
ShardingOutputWriter.write(KeyValue<K,V> value) |
Copyright © 2015 Google. All rights reserved.