Generate google and google-beta providers #

This quickstart guides you through setting up your development environment, making a change to magic-modules, generating provider changes to the google and google-beta Terraform providers, and running tests related to the change.

Before you begin #

1. Install the gcloud CLI.
2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don’t already have a project to use for testing changes to the Terraform providers, create a project instead of selecting an existing poject. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector
3. Make sure that billing is enabled for your Google Cloud project. Learn how to check if billing is enabled on a project.

Set up your development environment #

If you are familiar with Docker or Podman, you may want to use the experimental container-based environment instead of this section.

1. Install git

2. Install rbenv, ensuring you follow both steps 1 and 2.

3. Use rbenv to install ruby 3.1.0

   rbenv install 3.1.0
   

4. Install go

5. Add the following values to your environment settings such as .bashrc:

   # Add GOPATH variable for convenience
   export GOPATH=$(go env GOPATH)
   # Add Go binaries to PATH
   export PATH=$PATH:$(go env GOPATH)/bin
   

6. Install goimports

   go install golang.org/x/tools/cmd/goimports@latest
   

7. Install terraform

8. Clone the magic-modules repository

   cd ~
   git clone https://github.com/GoogleCloudPlatform/magic-modules.git
   

9. Run the following command from the root of your cloned magic-modules repository.

   cd magic-modules
   ./scripts/doctor
   

   Expected output if everything is installed properly:

   Check for ruby in path...
      found!
   Check for go in path...
      found!
   Check for goimports in path...
      found!
   Check for git in path...
      found!
   Check for terraform in path...
      found!
   Check for make in path...
      found!
   

Generate a provider change #

1. In your cloned magic-modules repository, edit mmv1/products/pubsub/Topic.yaml to change the description for the schemaSettings field:

   - !ruby/object:Api::Type::NestedObject
     name: 'schemaSettings'
     description: |
       UPDATED_DESCRIPTION    
   

2. Clone the google and google-beta provider repositories with the following commands:

   git clone https://github.com/hashicorp/terraform-provider-google.git $GOPATH/src/github.com/hashicorp/terraform-provider-google
   git clone https://github.com/hashicorp/terraform-provider-google-beta.git $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta
   

3. Generate changes for the google provider

   make provider VERSION=ga OUTPUT_PATH="$GOPATH/src/github.com/hashicorp/terraform-provider-google" PRODUCT=pubsub
   

4. Generate changes for the google-beta provider

   make provider VERSION=beta OUTPUT_PATH="$GOPATH/src/github.com/hashicorp/terraform-provider-google-beta" PRODUCT=pubsub
   

5. Confirm that the expected changes were generated

   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google
   git diff -U0
   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta
   git diff -U0
   

   In both cases, the changes should include:

   diff --git a/google-beta/resource_pubsub_topic.go b/google-beta/resource_pubsub_topic.go
   --- a/google-beta/resource_pubsub_topic.go
   +++ b/google-beta/resource_pubsub_topic.go
   @@ -115 +115 @@ and is not a valid configuration.`,
   -                               Description: `Settings for validating messages published against a schema.`,
   +                               Description: `UPDATED_DESCRIPTION`,
   diff --git a/website/docs/r/pubsub_topic.html.markdown b/website/docs/r/pubsub_topic.html.markdown
   --- a/website/docs/r/pubsub_topic.html.markdown
   +++ b/website/docs/r/pubsub_topic.html.markdown
   @@ -146 +146 @@ The following arguments are supported:
   -  Settings for validating messages published against a schema.
   +  UPDATED_DESCRIPTION
   

   Note: There may be additional changes present due to specifying a PRODUCT= value or due to the magic-modules repository being out of sync with the provider repositories. This is okay as long as tests in the following section pass.

Test changes #

1. Set up application default credentials for Terraform

   gcloud auth application-default login
   export GOOGLE_USE_DEFAULT_CREDENTIALS=true
   

2. Set required environment variables

   export GOOGLE_PROJECT=PROJECT_ID
   export GOOGLE_REGION=us-central1
   export GOOGLE_ZONE=us-central1-a
   

   Replace PROJECT_ID with the ID of your Google Cloud project.

3. Enable required APIs

   gcloud config set project $GOOGLE_PROJECT
   gcloud services enable pubsub.googleapis.com
   gcloud services enable cloudkms.googleapis.com
   

4. Run all linters

   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google
   make lint
   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta
   make lint
   

5. Run all unit tests

   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google
   make test
   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta
   make test
   

6. Run acceptance tests for Pub/Sub Topic

   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google
   make testacc TEST=./google/services/pubsub TESTARGS='-run=TestAccPubsubTopic_'
   cd $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta
   make testacc TEST=./google-beta/services/pubsub TESTARGS='-run=TestAccPubsubTopic_'
   

Troubleshoot #

Too many open files #

If you are getting “Too many open files” ulimit needs to be raised.

ulimit -n 8192

Cleanup #

1. Optional: Revoke credentials from the gcloud CLI.

gcloud auth revoke

What’s next #

• Learn about Magic Modules
• Learn about the contribution process
• Learn about make commands