Add resource tests#
This page describes how to add tests to a new resource in the google or google-beta Terraform provider.
The providers have two basic types of tests:
- Unit tests: test specific functions thoroughly. Unit tests do not interact with GCP APIs.
- Acceptance tests (aka VCR tests, or create and update tests): test that resources interact as expected with the APIs. Acceptance tests interact with GCP APIs, but should only test the provider’s behavior in constructing the API requests and parsing the responses.
Acceptance tests are also called “VCR tests” because they use go-vcr to record and play back HTTP requests. This allows tests to run more quickly on PRs because the resources don’t actually need to be created, updated, or destroyed by the live API.
For more information about testing, see the official Terraform documentation.
Before you begin#
- Determine whether your resources is using MMv1 generation or handwritten.
- If you are not adding tests to an in-progress PR, ensure that your
magic-modules,terraform-provider-google, andterraform-provider-google-betarepositories are up to date.cd ~/magic-modules git checkout main && git clean -f . && git checkout -- . && git pull cd $GOPATH/src/github.com/hashicorp/terraform-provider-google git checkout main && git clean -f . && git checkout -- . && git pull cd $GOPATH/src/github.com/hashicorp/terraform-provider-google-beta git checkout main && git clean -f . && git checkout -- . && git pull
Add unit tests#
A unit test verifies functionality that is not related to interactions with the API, such as diff suppress functions, validation functions, CustomizeDiff functions, and so on.
Unit tests should be added to the appropriate folder in magic-modules/mmv1/third_party/terraform/services in the file called resource_PRODUCT_RESOURCE_test.go. (You may need to create this file if it does not already exist. Replace PRODUCT with the product name and RESOURCE with the resource name; it should match the name of the generated resource file.)
Unit tests should be named like TestFunctionName - for example, TestDiskImageDiffSuppress would contain tests for the DiskImageDiffSuppress function.
Example:
func TestSignatureAlgorithmDiffSuppress(t *testing.T) {
cases := map[string]struct {
Old, New string
ExpectDiffSuppress bool
}{
"ECDSA_P256 equivalent": {
Old: "ECDSA_P256_SHA256",
New: "EC_SIGN_P256_SHA256",
ExpectDiffSuppress: true,
},
// Additional cases excluded for brevity
}
for tn, tc := range cases {
if signatureAlgorithmDiffSuppress("signature_algorithm", tc.Old, tc.New, nil) != tc.ExpectDiffSuppress {
t.Errorf("bad: %s, %q => %q expect DiffSuppress to return %t", tn, tc.Old, tc.New, tc.ExpectDiffSuppress)
}
}
}Add an acceptance test#
An acceptance test verifies that a resource can be created, updated, and destroyed successfully.
Note: All resources should have a “basic” test covering the minimal required fields. Additional tests should be added to cover all fields, with updatable fields being covered by an update step. Updatable fields are fields that can be updated without recreating the entire resource; that is, they are not marked
immutablein MMv1 orForceNewin handwritten code
Steps#
Add an entry to your
RESOURCE_NAME.yamlfile’ssampleslist. Each sample can contain multiple steps. The first step will generate acreatetest, and any subsequent steps will generateupdatetests. For a comprehensive reference, see MMv1 sample reference ↗.When defining variables for your steps, follow these guidelines:
- Use
resource_id_varsfor resource identifiers (like names or IDs) that need to be unique. Values automatically receive atf-testprefix and random suffix, unless they contain an underscore_, in which case they receive atf_testprefix and random suffix. If a resource name doesn’t support hyphens-or underscores_, usetest_vars_overridesinstead. For non-identifier variables, usevars. - Use
varsonly for fields that vary between steps (for example, to test the update functionality of specific fields). - Hardcode all other values directly in the
.tf.tmplconfiguration file. Don’t usevarsfor values that remain constant across all steps.
samples: # name is used to generate the test name. - name: "pubsub_topic_update" # primary_resource_id will be used for the Terraform resource id in the configuration file. primary_resource_id: "default" # min_version can be set at the top level if it applies to all steps. min_version: beta steps: # The first step defines the initial create configuration. # Step name: Matches the template file name by default (for example, pubsub_topic_minimal.tf.tmpl). Use config_path to override this. - name: "pubsub_topic_minimal" # resource_id_vars contains key/value pairs to inject into the configuration file. # These can be referenced as a key inside `{{$.ResourceIdVars}}`. resource_id_vars: resource_name: "example-resource" network_name: "example-network" # test_vars_overrides contains literal overrides for variables in tests. test_vars_overrides: network_name: 'servicenetworking.BootstrapSharedServiceNetworkingConnection(t, "pubsub-topic-network-config")' # Subsequent steps define update configurations. - name: "pubsub_topic_full" resource_id_vars: resource_name: "example-resource" network_name: "example-network" test_vars_overrides: network_name: 'servicenetworking.BootstrapSharedServiceNetworkingConnection(t, "pubsub-topic-network-config")' # vars should ONLY be used for fields that vary between steps. # Fields that stay constant across steps should be hardcoded in the .tf.tmpl file. vars: display_name: "Display Name" - name: "pubsub_topic_full" resource_id_vars: resource_name: "example-resource" network_name: "example-network" test_vars_overrides: network_name: 'servicenetworking.BootstrapSharedServiceNetworkingConnection(t, "pubsub-topic-network-config")' vars: display_name: "Updated Display Name" # The new value for the updatable field- Use
Create one or more
.tf.tmplfiles inmmv1/templates/terraform/samples/services/SERVICE_NAME/. The file names should match the step name (for example,pubsub_topic_minimal.tf.tmplandpubsub_topic_full.tf.tmpl).In those files, write the Terraform configuration for your test steps. This should include all required dependencies.
pubsub_topic_minimal.tf.tmpl:
resource "google_pubsub_topic" "{{.PrimaryResourceId}}" {
name = "{{index $.ResourceIdVars "resource_name"}}"
network = google_compute_network.network.name
labels = {
env = "test"
}
}
resource "google_compute_network" "network" {
name = "{{index $.ResourceIdVars "network_name"}}"
auto_create_subnetworks = false
routing_mode = "REGIONAL"
}pubsub_topic_full.tf.tmpl: (This file added the display_name variable).
resource "google_pubsub_topic" "{{.PrimaryResourceId}}" {
name = "{{index $.ResourceIdVars "resource_name"}}"
# This is an example of a field whose value changes between steps
# to test update functionality of display_name
display_name = "{{index $.Vars "display_name"}}"
# The rest of the fields can be baked in the configuration directly
message_retention_duration = "86600s"
labels = {
env = "test"
}
network = google_compute_network.network.name
}
resource "google_compute_network" "network" {
name = "{{index $.ResourceIdVars "network_name"}}"
auto_create_subnetworks = false
routing_mode = "REGIONAL"
}- For beta-only resources or features:
- If the resource or the whole test is beta-only
- Add
provider = google-betato every resource in the file. - Add
min_version: betaat the top sample level
- Add
- If only a single test step is beta-only
- Add
min_version: betaat the individual step level
- Add
- If the resource or the whole test is beta-only
This workflow is an alternative for when the recommended
samplesgenerator framework is insufficient (for example, when you require a customCheckFunctionor other complex test assertions).
An update test ensures that updatable fields can be changed without recreating the entire resource. All updatable fields must be covered by at least one update test (often a single update step covering all fields is sufficient).
- Generate the beta provider.
- From the beta provider, copy and paste the generated
*_generated_test.gofile into the appropriate service folder insidemagic-modules/mmv1/third_party/terraform/servicesas a new file call*_test.go. - Using an editor of your choice, delete the
*DestroyProducerfunction, and all but one test. The remaining test should be the “full” test, or if there is no “full” test, the “basic” test. This will be the starting point for your new update test. - Modify the
TestAcc*test function to support updates.- Change the suffix of the test function to
_update. - Copy the 2
TestStepblocks and paste them immediately after, so that there are 4 total test steps. - Change the suffix of the first
Configvalue to_full(or_basic). - Change the suffix of the second
Configvalue to_update. - Add
ConfigPlanChecksto the update step of the test to ensure the resource is updated in-place. - The resulting test function would look similar to this:
import "github.com/hashicorp/terraform-plugin-testing/plancheck" func TestAccPubsubTopic_update(t *testing.T) { ... acctest.VcrTest(t, resource.TestCase{ ... Steps: []resource.TestStep{ { Config: testAccPubsubTopic_full(...), }, { ... }, { Config: testAccPubsubTopic_update(...), ConfigPlanChecks: resource.ConfigPlanChecks{ PreApply: []plancheck.PlanCheck{ plancheck.ExpectResourceAction("google_pubsub_topic.foo", plancheck.ResourceActionUpdate), }, }, }, { ... }, }, }) } - Change the suffix of the test function to
- Modify the
testAcc*Terraform template function to support updates.- Copy the template function and paste it immediately after so that there are 2 template functions.
- Change the suffix of the first template function to
_full(or_basic). - Change the suffix of the second template function to
_update. - The resulting template functions would look similar to this:
func testAccPubsubTopic_full(...) string { ... } func testAccPubsubTopic_update(...) string { ... } - Modify the test as needed.
- Replace all occurrences of
github.com/hashicorp/terraform-provider-google-beta/google-betawithgithub.com/hashicorp/terraform-provider-google/google - Modify the template function ending in
_updateso that updatable fields are changed or removed. This may require additions to thecontextmap in the test function. - Remove the comments at the top of the file.
- If beta-only fields are being tested, do the following:
- Change the file suffix to
.go.tmpl - Wrap each beta-only test in a separate version guard:
{{- if ne $.TargetVersionName "ga" -}}...{{- else }}...{{- end }} - In each beta-only test, ensure that the TestCase sets
ProtoV5ProviderFactories: acctest.ProtoV5ProviderBetaFactories(t) - In each beta-only test, ensure that all Terraform resources in all configs have
provider = google-betaset
- Change the file suffix to
- Replace all occurrences of
Bootstrap API resources#
Most acceptance tests run in a the default org and default test project, which means that they can conflict for quota, resource namespaces, and control over shared resources. You can work around these limitations with “bootstrapped” resources.
CryptoKeys#
There are a few functions provided for bootstrapping CryptoKeys, depending on your needs.
BootstrapKMSKeyWithPurposeInLocationAndName(t *testing.T, purpose, locationID, keyShortName string)BootstrapKMSKeyWithPurposeInLocation(t *testing.T, purpose, locationID string)- Uses a default key name based on the purpose.
BootstrapKMSKeyWithPurpose(t *testing.T, purpose string)- Uses
globallocation and a key name based on the purpose.
- Uses
BootstrapKMSKeyInLocation(t *testing.T, locationID string)- Uses
ENCRYPT_DECRYPTfor the purpose and the corresponding key name.
- Uses
BootstrapKMSKey(t *testing.T)- Uses
globallocation,ENCRYPT_DECRYPTfor the purpose, and the corresponding key name for that purpose.
- Uses
Example usage:
samples:
- name: service_resource_basic
primary_resource_id: example
steps:
- name: service_resource_basic
resource_id_vars:
kms_key_name: 'kms-key'
test_vars_overrides:
kms_key_name: 'kms.BootstrapKMSKey(t).CryptoKey.Name'import (
"github.com/hashicorp/terraform-provider-google/google/services/kms"
)
func TestAccProductResource_update(t *testing.T) {
t.Parallel()
context := map[string]interface{}{
"kms": kms.BootstrapKMSKey(t).CryptoKey.Name,
// other variables
}
// rest of test
}IAM resources#
Specify member/role pairs that should always exist. {project_number} will be replaced with the default project’s project number. {organization_id} will be replaced with the “target” test organization’s ID – we don’t modify IAM in the main test org to avoid accidentally locking ourselves out.
Permissions attached to resources created in a test should instead be provisioned with standard terraform resources.
Example usage:
# Project-level IAM
samples:
- name: service_resource_basic
primary_resource_id: example
bootstrap_iam:
- member: "serviceAccount:service-{project_number}@gcp-sa-healthcare.iam.gserviceaccount.com"
role: "roles/bigquery.dataEditor"
steps:
- name: service_resource_basic
config_path: samples/basic.tf.tmpl# Org-level IAM
samples:
- name: service_resource_basic
primary_resource_id: example
bootstrap_iam:
- member: "serviceAccount:service-org-{organization_id}@gcp-sa-osconfig.iam.gserviceaccount.com"
role: "roles/osconfig.serviceAgent"
steps:
- name: service_resource_basic
test_env_vars:
org_id: ORG_TARGET # Resolves to envvar.GetTestOrgTargetFromEnv in tests// Project-level IAM
import (
"github.com/hashicorp/terraform-provider-google/google/services/resourcemanager"
)
func TestAccProductResource_update(t *testing.T) {
t.Parallel()
resourcemanager.BootstrapIamMembers(t, []resourcemanager.IamMember{
{
Member: "serviceAccount:service-{project_number}@gcp-sa-pubsub.iam.gserviceaccount.com",
Role: "roles/cloudkms.cryptoKeyEncrypterDecrypter",
},
})
// rest of test
}// Org-level IAM
import (
"github.com/hashicorp/terraform-provider-google/google/envvar"
"github.com/hashicorp/terraform-provider-google/google/services/resourcemanager"
)
func TestAccProductResource_update(t *testing.T) {
t.Parallel()
resourcemanager.BootstrapIamMembers(t, []resourcemanager.IamMember{
{
Member: "serviceAccount:service-org-{organization_id}@gcp-sa-osconfig.iam.gserviceaccount.com",
Role: "roles/osconfig.serviceAgent",
},
})
context := map[string]string{
"org_id": envvar.GetTestOrgTargetFromEnv(t),
}
// rest of test
}Networks#
Bootstrapping networks can be useful for two reasons:
- Resources like
google_service_networking_connectionuse a consumer network and create a complementing tenant network which we don’t control. These tenant networks never get cleaned up and they can accumulate to the point where a limit is reached for the organization. By reusing a consumer network across test runs, we can reduce the number of tenant networks that are needed. (Googlers: See b/146351146 for more context.) - Bootstrap networks used in tests (gke clusters, dataproc clusters…) to limit traffic to the default network (preventing conflicts).
When creating a bootstrapped network in a test, you can specify an identifier. Note that if the network is being used for a google_service_networking_connection, you should use an identifier unique to the test to avoid race conditions where multiple tests attempt to modify the connection at once.
You can also bootstrap one or more subnetworks within a bootstrapped network if necessary, to avoid subnetwork-level quotas and race conditions.
Example usage:
samples:
- name: service_resource_basic
primary_resource_id: example
steps:
- name: service_resource_basic
resource_id_vars:
network_name: 'default'
subnetwork_name: 'default'
test_vars_overrides:
network_name: 'tpgcompute.BootstrapSharedTestNetwork(t, "network-identifier")'
subnetwork_name: 'tpgcompute.BootstrapSubnet(t, "subnet-identifier", tpgcompute.BootstrapSharedTestNetwork(t, "network-identifier"))'import (
tpgcompute "github.com/hashicorp/terraform-provider-google/google/services/compute"
)
func TestAccProductResource_update(t *testing.T) {
t.Parallel()
networkName :=
subnetName :=
context := map[string]interface{}{
"network_name": tpgcompute.BootstrapSharedTestNetwork(t, "network-identifier"),
"subnetwork_name": tpgcompute.BootstrapSubnet(t, "subnet-identifier", tpgcompute.BootstrapSharedTestNetwork(t, "network-identifier")),
// other variables
}
// rest of test
}Create test projects#
If bootstrapping doesn’t work or isn’t an option for some reason, you can also work around project quota issues or test project-global resources by creating a new test project. You will also need to enable any necessary APIs and wait for their enablement to propagate.
import (
"testing"
"github.com/hashicorp/terraform-plugin-testing/helper/resource"
"github.com/hashicorp/terraform-provider-google/google/acctest"
"github.com/hashicorp/terraform-provider-google/google/envvar"
)
func TestAccProductResourceName_update(t *testing.T) {
t.Parallel()
context := map[string]interface{}{
"random_suffix": acctest.RandString(t, 10),
"billing_account": envvar.GetTestBillingAccountFromEnv(t),
"org_id": envvar.GetTestOrgFromEnv(t),
}
acctest.VcrTest(t, resource.TestCase{
// ...
// Add ExternalProviders so you can use `time_sleep`
ExternalProviders: map[string]resource.ExternalProvider{
"time": {},
},
Steps: []resource.TestStep{
{
testAccProductResourceName_update1(context),
},
// ...
},
})
}
func testAccProductResourceName_update1(context map[string]interface{}) string {
return accest.Nprintf(`
// Set up a test project
resource "google_project" "project" {
project_id = "tf-test%{random_suffix}"
name = "tf-test%{random_suffix}"
org_id = "%{org_id}"
billing_account = "%{billing_account}"
deletion_policy = "DELETE"
}
// Enable APIs in a deterministic order to avoid inconsistent VCR recordings
resource "google_project_service" "servicenetworking" {
project = google_project.project.project_id
service = "servicenetworking.googleapis.com"
}
resource "google_project_service" "compute" {
project = google_project.project.project_id
service = "compute.googleapis.com"
depends_on = [google_project_service.servicenetworking]
}
// wait for API enablement
resource "time_sleep" "wait_120_seconds" {
create_duration = "120s"
depends_on = [google_project_service.compute]
}
resource "google_product_resource" "example" {
// ...
depends_on = [time_sleep.wait_120_seconds]
}
`, context)
}Skip tests in VCR replaying mode#
Acceptance tests are run in VCR replaying mode on PRs (using pre-recorded HTTP requests and responses) to reduce the time it takes to present results to contributors. However, not all resources or tests are possible to run in replaying mode. Incompatible tests should be skipped during VCR replaying mode. They will still run in our nightly test suite.
Skipping acceptance tests that are generated from example files can be achieved by adding skip_vcr: true in the example’s YAML:
samples:
- name: 'bigtable_app_profile_anycluster'
...
# bigtable instance does not use the shared HTTP client, this test creates an instance
skip_vcr: true
steps:
- name: service_resource_basicIf you skip a test in VCR mode, include a code comment explaining the reason for skipping (for example, a link to a GitHub issue.)
Skipping acceptance tests that are handwritten can be achieved by adding acctest.SkipIfVcr(t) at the start of the test:
func TestAccPubsubTopic_update(t *testing.T) {
acctest.SkipIfVcr(t) // See: https://github.com/hashicorp/terraform-provider-google/issues/9999
acctest.VcrTest(t, resource.TestCase{ ... })
}If you skip a test in VCR mode, include a code comment explaining the reason for skipping (for example, a link to a GitHub issue.)
Time-based skips#
Acceptance tests can be marked to be skipped until a certain future date, such as the projected date of a launch or rollout. This should generally be added after coordinating with your reviewer to capture a successful test run such as through a local run with an allowlisted project, against staging, etc.
Please include a comment with context where the skip is defined.
Skipping acceptance tests that are generated from example files can be achieved by adding skip_func: acctest.SkipTestUntil(t, "YYYY-MM-DD") in the example’s YAML:
samples:
- name: 'compute_address_basic'
...
skip_func: acctest.SkipTestUntil(t, "2026-01-31") # waiting for rollout
steps:
- name: service_resource_basicSkipping acceptance tests that are handwritten can be achieved by adding acctest.SkipTestUntil(t, "YYYY-MM-DD") at the start of the test:
func TestAccPubsubTopic_update(t *testing.T) {
acctest.SkipTestUntil(t, "2026-01-31") // b/1234567890
acctest.VcrTest(t, resource.TestCase{ ... })
}Reasons that tests are skipped in VCR replaying mode#
| Problem | How to fix/Other info | Skip in VCR replaying? |
|---|---|---|
Incorrect or insufficient data is present in VCR recordings to replay tests. Tests will fail with Requested interaction not found errors during REPLAYING mode | Make sure that you’re not introducing randomness into the test, such as by unnecessarily using the random provider to set a resource’s name. | If you cannot avoid this issue you should skip the test, but try to ensure that it cannot be fixed first. |
Bigtable acceptance tests aren’t working in VCR mode. Requested interaction not found errors are seen during Bigtable tests run in REPLAYING mode | Currently the provider uses a separate client than the rest of the provider to interact with the Bigtable API. As HTTP traffic to the Bigtable API doesn’t go via the shared client it cannot be recorded in RECORDING mode. | Skip the test in VCR for Bigtable. |
| Using multiple provider aliases doesn’t work in VCR. You may have two instances of the google provider in the test config but one of them doesn’t seem to be using its provider arguments - for example, using the wrong default project. | See this GitHub issue: https://github.com/hashicorp/terraform-provider-google/issues/20019 . The problem is that, due to how the VCR system works, one provider instance will be configured and the other will be forced to reuse the first instance’s configuration, despite them being given different provider arguments. | Skip the test in VCR is using aliases is unavoidable. |
Using multiple versions of the google/google-beta provider in a single test isn’t working in VCR. Unexpected test failures may occur during tests in REPLAYING mode where ExternalProviders is used to pull in past versions of the google/google-beta provider. | When ExternalProviders is used to pulling in other versions of the provider, any HTTP traffic through the external provider will not be recorded. If the HTTP traffic produces an unexpected result or returns an API error then the test will fail in REPLAYING mode. | Skip the test in VCR when testing the current provider behaviour versus previous released versions. |
Some additional things to bear in mind are that VCR tests in REPLAYING mode will still interact with GCP APIs somewhat. For example:
- When the provider is configured it will use credentials to obtain access tokens from GCP
- Some acceptance tests use bootstrapping functions that ensure long-lived resources are present in a testing project before the provider is tested.
These tests can still run in VCR replaying mode; however, REPLAYING mode can’t be used as a way to completely avoid HTTP traffic generally or with GCP APIs.