Data subcommand

This subcommand will perform data migration in either the POC or the minimal downtime mode. This subcommand requires users to pass the session file (which contains schema mapping) generated by either the schema subcommand or web UI.

The command below assumes that the open-source version of SMT is being used. For the CLI reference of the gCloud version of SMT, please refer here.

Table of contents
  1. NAME
  2. SYNOPSIS
  3. DESCRIPTION
  4. EXAMPLES
  5. REQUIRED FLAGS
  6. OPTIONAL FLAGS

NAME 

./spanner-migration-tool data - migrate data from a source database to
    Cloud Spanner given a schema

SYNOPSIS 

./spanner-migration-tool data --session=SESSION --source=SOURCE
    [--dry-run] [--log-level=LOG_LEVEL] [--prefix=PREFIX]
    [--skip-foreign-keys] [--source-profile=SOURCE_PROFILE]
    [--target=TARGET] [--target-profile=TARGET_PROFILE]
    [--write-limit=WRITE_LIMIT] [--project=PROJECT] [GCLOUD_WIDE_FLAG ...]

DESCRIPTION 

Migrate data from a source database to Cloud Spanner given a
schema.

EXAMPLES 

To copy data to Cloud Spanner given a session file and a PG dump file:

    $ ./spanner-migration-tool data --session=./session.json \
        --source=postgresql --target-profile='instance=spanner-instance' < ~/cart.pg_dump
        

To run a minimal downtime data migration:

    $ ./spanner-migration-tool data --session=./session.json \
        --source=MySQL \
        --source-profile='host=host,port=3306,user=user,password=pwd,dbName=db,streamingCfg=streaming.json' \
        --target-profile='project=spanner-project,instance=spanner-instance' --project=migration-project

To run a minimal downtime data migration for a sharded source:

    $ ./spanner-migration-tool data --session=./session.json \
        --source=MySQL \
        --source-profile='config=shard-config.json' \
        --target-profile='project=spanner-project,instance=spanner-instance' --project=migration-project

For sharded migrations, you can refer to the example source-config.json passed in the source-profile option here.

REQUIRED FLAGS 

 --session=SESSION
    Specifies the file that you restore session state from. This file can be generated using the [schema](/spanner-migration-tool/cli/schema.html) sub command.

 --source=SOURCE
    Flag for specifying source database (e.g., PostgreSQL, MySQL,
    DynamoDB).

OPTIONAL FLAGS

Detailed description of optional flags can be found here.

 --dry-run
    Flag for generating DDL and schema conversion report without creating a
    Cloud Spanner database.

 --log-level=LOG_LEVEL
    To configure the log level for the execution (INFO, VERBOSE).

 --prefix=PREFIX
    File prefix for generated files. Details on generated files can be found [here](/spanner-migration-tool/reports.html#file-descriptions)

 --skip-foreign-keys
    Skip creating foreign keys after data migration is complete.

 --source-profile=SOURCE_PROFILE
    Flag for specifying connection profile for source database (e.g.,
    "file=<path>,format=dump").

 --target=TARGET
    Specifies the target database, defaults to Spanner (accepted values:
    Spanner) (default "Spanner").

 --target-profile=TARGET_PROFILE
    Flag for specifying connection profile for target database (e.g.,
    "dialect=postgresql").

 --write-limit=WRITE_LIMIT
    Number of parallel writers to Cloud Spanner during bulk data migrations
    (default 40).

 --project=PROJECT
    Flag for specifying the name of the Google Cloud Project in which the Spanner migration tool
    can create resources required for migration. If the project is not specified, Spanner migration 
    tool will try to fetch the configured project in the gCloud CLI.

 --dataflow-template=DATAFLOW_TEMPLATE
    The google cloud storage path of the minimal downtime migration
    template to use to run the migration job. Default value is the latest dataflow template.