Fix a permadiff

Fix a permadiff #

Permadiffs are an extremely common class of errors that users experience. They manifest as diffs at plan time on fields that a user has not modified in their configuration. They can also show up as test failures with the error message: “After applying this test step, the plan was not empty.”

In a general sense, permadiffs are caused by the API returning a different value for the field than what the user sent, which causes Terraform to try to re-send the same request, which gets the same response, which continues to result in the user seeing a diff. In general, APIs that return exactly what the user sent are more friendly for Terraform or other declarative tooling. However, many GCP APIs normalize inputs, have server-side defaults that are returned to the user, do not return all the fields set on a resource, or return data in a different format in some other way.

This page outlines best practices for working around various types of permadiffs in the google and google-beta providers.

API returns default value for unset field #

For new fields, if possible, set a client-side default that matches the API default. This will prevent the diff and will allow users to accurately see what the end state will be if the field is not set in their configuration. A client-side default should only be used if the API sets the same default value in all cases and the default value will be stable over time. Changing a client-side default is a breaking change.

default_value: DEFAULT_VALUE

In the providers, this will be converted to:

"field": {
    // ...
    Default: "DEFAULT_VALUE",
}

See SDKv2 Schema Behaviors - Default ↗ for more information.

"field": {
    // ...
    Default: "DEFAULT_VALUE",
}

See SDKv2 Schema Behaviors - Default ↗ for more information.

For existing fields (or new fields that are not eligible for a client-side default), mark the field as having an API-side default. If the field is not set (or is set to an “empty” value such as zero, false, or an empty string) the provider will treat the most recent value returned by the API as the value for the field, and will send that value for the field on subsequent requests. The field will show as (known after apply) in plans and it will not be possible for the user to explicitly set the field to an “empty” value.

default_from_api: true

In the providers, this will be converted to:

"field": {
    // ...
    Optional: true,
    Computed: true,
}

See SDKv2 Schema Behaviors - Optional ↗ and SDKv2 Schema Behaviors - Computed ↗ for more information.

"field": {
    // ...
    Optional: true,
    Computed: true,
}

See SDKv2 Schema Behaviors - Optional ↗ and SDKv2 Schema Behaviors - Computed ↗ for more information.

API returns an empty value if default value is sent #

Use a flattener to store the default value in state if the response has an empty (or unset) value.

Use the standard default_if_empty flattener.

custom_flatten: 'templates/terraform/custom_flatten/default_if_empty.tmpl'
func flattenResourceNameFieldName(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
    if v == nil || tpgresource.IsEmptyValue(reflect.ValueOf(v)) {
        return "DEFAULT_VALUE"
    }
    // Any other necessary logic goes here.
    return v
}

API normalizes a value #

In cases where the API normalizes and returns a value in a simple, predictable way (such as capitalizing the value) add a diff suppress function for the field to suppress the diff.

The tpgresource package in each provider supplies diff suppress functions for the following common cases:

  • tpgresource.CaseDiffSuppress: Suppress diffs from capitalization differences between the user’s configuration and the API.
  • tpgresource.DurationDiffSuppress: Suppress diffs from duration format differences such as “60.0s” vs “60s”. This is necessary for Duration API fields.
  • tpgresource.ProjectNumberDiffSuppress: Suppress diffs caused by the provider sending a project ID and the API returning a project number.
# Use a built-in function
diff_suppress_func: 'tpgresource.CaseDiffSuppress'

# Reference a resource-specific function
diff_suppress_func: 'resourceNameFieldNameDiffSuppress'

Define resource-specific functions in a custom_code.constants file.

func resourceNameFieldNameDiffSuppress(_, old, new string, _ *schema.ResourceData) bool {
    // Separate function for easier unit testing
    return resourceNameFieldNameDiffSuppressLogic(old, new)
}

func resourceNameFieldNameDiffSuppressLogic(old, new) bool {
    // Diff suppression logic. Returns true if the diff should be suppressed - that is, if the
    // old and new values should be considered "the same".
}

See SDKv2 Schema Behaviors - DiffSuppressFunc ↗ for more information.

Define resource-specific functions in your service package, for example at the top of the related resource file.

func resourceNameFieldNameDiffSuppress(_, old, new string, _ *schema.ResourceData) bool {
    // Separate function for easier unit testing
    return resourceNameFieldNameDiffSuppressLogic(old, new)
}

func resourceNameFieldNameDiffSuppressLogic(old, new) bool {
    // Diff suppression logic. Returns true if the diff should be suppressed - that is, if the
    // old and new values should be considered "the same".
}

Reference diff suppress functions from the field definition.

"field": {
    // ...
    DiffSuppressFunc: resourceNameFieldNameDiffSuppress,
}

See SDKv2 Schema Behaviors - DiffSuppressFunc ↗ for more information.

API field that is never included in the response #

This is common for fields that store credentials or similar information. Such fields should also be marked as sensitive.

In the flattener for the field, return the value of the field in the user’s configuration.

On top-level fields, this can be done with:

ignore_read: true

For nested fields, ignore_read is not currently supported, so this must be implemented with a custom flattener. You will also need to add the field to ignore_read_extra on any examples that are used to generate tests; this will cause tests to ignore the field when checking that the values in the API match the user’s configuration.

func flatten{{$.GetPrefix}}{{$.TitlelizeProperty}}(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
    // We want to ignore read on this field, but cannot because it is nested
    return d.Get("path.0.to.0.nested.0.field")
}
examples:
   # example configuration
   ignore_read_extra:
     - "path.0.to.0.nested.0.field"

Use d.Get to set the flattened value to be the same as the user-configured value (instead of a value from the API).

func flattenParentField(d *schema.ResourceData, disk *compute.AttachedDisk, config *transport_tpg.Config) []map[string]interface{} {
    result := map[string]interface{}{
        "nested_field": d.Get("path.0.to.0.parent_field.0.nested_field")
    }
    return []map[string]interface{}{result}
}

In tests, add the field to ImportStateVerifyIgnore on any relevant import steps.

{
    ResourceName:            "google_product_resource.default",
    ImportState:             true,
    ImportStateVerify:       true,
    ImportStateVerifyIgnore: []string{""path.0.to.0.parent_field.0.nested_field"},
},

API returns a list in a different order than was sent #

For an Array of unique string values (or nested objects with unique string identifiers), use the SortStringsByConfigOrder or SortMapsByConfigOrder helper functions to sort the API response to match the order in the user’s configuration. This will also simplify diffs if new values are added or removed. Imported resources will not have access to a configuration, so the field will be sorted alphabetically. This means that tests for the resource need to ignore the field’s import behavior via ignore_read_extra (for MMv1 examples) or ImportStateVerifyIgnore (for handwritten tests).

Add a custom flattener for the field.

func flatten{{$.GetPrefix}}{{$.TitlelizeProperty}}(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
    configValue := d.Get("path.0.to.0.parent_field.0.nested_field").([]string)

    sorted, err := tpgresource.SortStringsByConfigOrder(configValue, v.([]string))
    if err != nil {
        log.Printf("[ERROR] Could not sort API response value: %s", err)
        return v
    }

    return sorted.(interface{})
}

Define resource-specific functions in your service package, for example at the top of the related resource file.

func flattenResourceNameFieldName(v interface{}, d *schema.ResourceData, config *transport_tpg.Config) interface{} {
    configValue := d.Get("path.0.to.0.parent_field.0.nested_field").([]string)

    sorted, err := tpgresource.SortStringsByConfigOrder(configValue, v.([]string))
    if err != nil {
        log.Printf("[ERROR] Could not sort API response value: %s", err)
        return v
    }

    return sorted.(interface{})
}

For other Array fields, convert the field to a Set – this is a breaking change and can only happen in a major release.