com.google.appengine.tools.mapreduce.bigqueryjobs

Class BigQueryLoadGoogleCloudStorageFilesJob

java.lang.Object

com.google.appengine.tools.pipeline.Job<T>

com.google.appengine.tools.pipeline.Job1<List<BigQueryLoadJobReference>,BigQueryStoreResult<GoogleCloudStorageFileSet>>

com.google.appengine.tools.mapreduce.bigqueryjobs.BigQueryLoadGoogleCloudStorageFilesJob

All Implemented Interfaces:

Serializable

public final class BigQueryLoadGoogleCloudStorageFilesJob
extends com.google.appengine.tools.pipeline.Job1<List<BigQueryLoadJobReference>,BigQueryStoreResult<GoogleCloudStorageFileSet>>

A pipeline job that loads files stored in Google Cloud Storage into a bigquery table.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
BigQueryLoadGoogleCloudStorageFilesJob(String dataset, String tableName, String projectId)

Method Summary

Methods

Modifier and Type	Method and Description
com.google.appengine.tools.pipeline.Value<BigQueryStoreResult<GoogleCloudStorageFileSet>>	handleException(Throwable t)
com.google.appengine.tools.pipeline.Value<List<BigQueryLoadJobReference>>	run(BigQueryStoreResult<GoogleCloudStorageFileSet> bigQueryStoreResult) Divides the files into bundles having size less than or equal to the maximum size allowed per bigquery load Job and then starts separate pipeline jobs for each of the bundles.

Methods inherited from class com.google.appengine.tools.pipeline.Job

backOffFactor, backOffSeconds, futureCall, futureCall, futureCall, futureCall, futureCall, futureCall, futureCall, futureCallUnchecked, futureList, getJobDisplayName, getJobKey, getOnBackend, getOnModule, getOnQueue, getPipelineKey, getStatusConsoleUrl, immediate, maxAttempts, newDelayedValue, newPromise, newPromise, onBackend, onModule, onQueue, setStatusConsoleUrl, statusConsoleUrl, waitFor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigQueryLoadGoogleCloudStorageFilesJob

public BigQueryLoadGoogleCloudStorageFilesJob(String dataset,
                                      String tableName,
                                      String projectId)

Parameters:

dataset - the name of the bigquery dataset.

tableName - name of the bigquery table to load data.

projectId - bigquery project Id.

Method Detail

run

public com.google.appengine.tools.pipeline.Value<List<BigQueryLoadJobReference>> run(BigQueryStoreResult<GoogleCloudStorageFileSet> bigQueryStoreResult)
                                                                              throws Exception

Divides the files into bundles having size less than or equal to the maximum size allowed per bigquery load Job and then starts separate pipeline jobs for each of the bundles. Returns a list of JobReferences.

Specified by:

run in class com.google.appengine.tools.pipeline.Job1<List<BigQueryLoadJobReference>,BigQueryStoreResult<GoogleCloudStorageFileSet>>

Throws:

Exception

handleException

public com.google.appengine.tools.pipeline.Value<BigQueryStoreResult<GoogleCloudStorageFileSet>> handleException(Throwable t)
                                                                                                          throws Throwable

Throws:

Throwable