com.google.appengine.tools.mapreduce.outputs

Class GoogleCloudStorageFileOutputWriter

java.lang.Object

com.google.appengine.tools.mapreduce.OutputWriter<ByteBuffer>

com.google.appengine.tools.mapreduce.outputs.GoogleCloudStorageFileOutputWriter

All Implemented Interfaces:

Serializable

public class GoogleCloudStorageFileOutputWriter
extends OutputWriter<ByteBuffer>

An OutputWriter that writes bytes to a GCS file that it creates. Produces a single file output (usually on a per-shard basis). This format does not insert any separator characters, so it by default cannot be read back with the CloudStorageLineInputReader.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static long	MEMORY_REQUIRED
static long	MEMORY_REQUIRED_WITHOUT_SLICE_RETRY

Constructor Summary

Constructors

Constructor and Description
GoogleCloudStorageFileOutputWriter(com.google.appengine.tools.cloudstorage.GcsFilename file, String mimeType)
GoogleCloudStorageFileOutputWriter(com.google.appengine.tools.cloudstorage.GcsFilename file, String mimeType, boolean supportSliceRetries)

Method Summary

Methods

Modifier and Type	Method and Description
void	beginShard() Will be called once before any calls to write.
void	beginSlice() Prepares the writer for writing after possibly having gone through serialization or deserialization.
void	cleanup() called after shards completes for any necessary cleanup.
void	endShard() Called when no more output will be written to this writer.
void	endSlice() Prepares the writer for possible serialization.
long	estimateMemoryRequirement() Returns the estimated memory that will be used by this writer in bytes.
com.google.appengine.tools.cloudstorage.GcsFilename	getFile()
String	toString()
void	write(ByteBuffer bytes) Writes a value to the output.

Methods inherited from class com.google.appengine.tools.mapreduce.OutputWriter

allowSliceRetry, close, getContext, open, setContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MEMORY_REQUIRED_WITHOUT_SLICE_RETRY

public static final long MEMORY_REQUIRED_WITHOUT_SLICE_RETRY

See Also:

Constant Field Values

MEMORY_REQUIRED

public static final long MEMORY_REQUIRED

See Also:

Constant Field Values

Constructor Detail

GoogleCloudStorageFileOutputWriter

public GoogleCloudStorageFileOutputWriter(com.google.appengine.tools.cloudstorage.GcsFilename file,
                                  String mimeType)

GoogleCloudStorageFileOutputWriter

public GoogleCloudStorageFileOutputWriter(com.google.appengine.tools.cloudstorage.GcsFilename file,
                                  String mimeType,
                                  boolean supportSliceRetries)

Method Detail

cleanup

public void cleanup()

Description copied from class: OutputWriter

called after shards completes for any necessary cleanup.

Overrides:

cleanup in class OutputWriter<ByteBuffer>

beginShard

public void beginShard()
                throws IOException

Description copied from class: OutputWriter

Will be called once before any calls to write. Prepares the writer for processing, after possibly having gone through serialization and deserialization.

Overrides:

beginShard in class OutputWriter<ByteBuffer>

Throws:

IOException - in the event of failure

beginSlice

public void beginSlice()
                throws IOException

Description copied from class: OutputWriter

Prepares the writer for writing after possibly having gone through serialization or deserialization.

Overrides:

beginSlice in class OutputWriter<ByteBuffer>

Throws:

IOException - in the event of failure

write

public void write(ByteBuffer bytes)
           throws IOException

Description copied from class: OutputWriter

Writes a value to the output.

Specified by:

write in class OutputWriter<ByteBuffer>

Throws:

IOException

endSlice

public void endSlice()
              throws IOException

Description copied from class: OutputWriter

Prepares the writer for possible serialization.

Overrides:

endSlice in class OutputWriter<ByteBuffer>

Throws:

IOException - in the event of failure

endShard

public void endShard()
              throws IOException

Description copied from class: OutputWriter

Called when no more output will be written to this writer.

Overrides:

endShard in class OutputWriter<ByteBuffer>

Throws:

IOException

getFile

public com.google.appengine.tools.cloudstorage.GcsFilename getFile()

toString

public String toString()

Overrides:

toString in class Object

estimateMemoryRequirement

public long estimateMemoryRequirement()

Description copied from class: OutputWriter

Returns the estimated memory that will be used by this writer in bytes. (This is normally just set to the size of the buffers used by the implementation)

Overrides:

estimateMemoryRequirement in class OutputWriter<ByteBuffer>