Migrate Compute Engine Persistent Disks to Hyperdisk¶
Fourth-generation (Gen4) Compute Engine machine series, such as c4, c4a,
c4d, m4, n4, and x4, and select third-generation (Gen3) machine series,
such as c3d, h3, and z3, support only Google Cloud Hyperdisk storage
volumes and require Non-Volatile Memory Express (NVMe) storage drivers and
Google Virtual NIC (gVNIC) network interfaces. While transitional Gen3 machine
series such as c3 and m3 support both Persistent Disk and Hyperdisk,
existing second-generation (Gen2) Virtual Machine (VM) instances that use
Persistent Disk types such as pd-standard, pd-balanced, pd-ssd, or
pd-extreme cannot change their machine type in place to a Gen4 series while
Persistent Disks remain attached.
This project provides a reference architecture, a Terraform lab environment, and two Agent Skills that assess Gen2 VMs and Persistent Disks and execute a low-downtime two-stage snapshot migration to Gen4 Compute Engine instances backed by Hyperdisk volumes.
Architecture¶
The migration workflow uses a two-stage snapshot pipeline to reduce the maintenance window to the time required to capture incremental block changes, create Hyperdisk volumes from those snapshots, and boot the target Gen4 instance.
flowchart TB
subgraph SourceEnv ["Source environment (Gen2)"]
Gen2VM["Gen2 Compute Engine VM (n2-standard-2)"]
SourceBootPD["Boot Persistent Disk (pd-balanced)"]
SourceDataPD["Data Persistent Disks (pd-ssd, pd-balanced, pd-standard)"]
SnapPolicy["Regional snapshot schedule"]
PDSizing["Disk type, sizeGb, and per-instance baseline formulas"]
end
subgraph AgentSkills ["Agent Skills (.agents/skills)"]
AssessSkill["assess-pd-to-hyperdisk-gce (read-only assessment)"]
MigrateSkill["migrate-pd-to-hyperdisk-gce (two-stage cutover)"]
end
subgraph SnapshotPipeline ["Two-stage snapshot pipeline"]
Stage1Snap["Stage 1 online baseline snapshots (*-s1-timestamp)"]
Stage2Snap["Stage 2 incremental cutover snapshots (*-s2-timestamp)"]
end
subgraph TargetEnv ["Target environment (Gen4)"]
TargetBootHD["Boot Hyperdisk Balanced volume"]
TargetDataHD["Right-sized data Hyperdisk volumes"]
Gen4VM["Gen4 Compute Engine VM (n4-standard-2 with NVMe and gVNIC)"]
end
Gen2VM --> SourceBootPD
Gen2VM --> SourceDataPD
SnapPolicy --> SourceDataPD
SourceBootPD --> PDSizing
SourceDataPD --> PDSizing
PDSizing --> AssessSkill
SourceBootPD --> AssessSkill
SourceDataPD --> AssessSkill
AssessSkill --> MigrateSkill
MigrateSkill --> Stage1Snap
SourceBootPD --> Stage1Snap
SourceDataPD --> Stage1Snap
Stage1Snap --> Stage2Snap
Stage2Snap --> TargetBootHD
Stage2Snap --> TargetDataHD
TargetBootHD --> Gen4VM
TargetDataHD --> Gen4VM
SnapPolicy --> TargetDataHD
Components¶
- Terraform lab environment (
terraform/): Provisions an isolated Virtual Private Cloud (VPC) network and subnetwork with Private Google Access enabled, Identity-Aware Proxy (IAP) Secure Shell (SSH) firewall rules, a least-privilege VM service account, a regional snapshot schedule, and two Gen2 (n2-standard-2) Debian 12 instances: a single-data-disk instance (pd-ssd) and a three-data-disk instance with transactional (pd-ssd), write-ahead log (WAL) (pd-balanced), and throughput-spool (pd-standard) disks. - Assessment skill (
.agents/skills/assess-pd-to-hyperdisk-gce/): Performs a read-only discovery of a single Compute Engine instance or a multi-VM migration wave. It evaluates Gen4 compatibility gates, calculates effective source Persistent Disk Input/Output Operations Per Second (IOPS) and throughput baselines from disk type, capacity (sizeGb), and per-instance sharing rules, checks regional Hyperdisk and virtual CPU (vCPU) quotas, and outputs a structured readiness report with right-sizedhyperdisk-balanced,hyperdisk-extreme, orhyperdisk-throughputtargets. - Migration skill (
.agents/skills/migrate-pd-to-hyperdisk-gce/): Executes the two-stage cutover workflow with explicit user approval gates before any state-changing operation:- Pre-cutover capture and guest verification: Exports source VM and disk
JavaScript Object Notation (JSON) manifests, checks Resource Manager tag
bindings, and verifies over SSH that
/etc/fstabuses filesystem Universally Unique Identifiers (UUIDs) or labels rather than/dev/sd*device names and that the guest kernel includes NVMe (nvme) and gVNIC (gve) drivers. - Stage 1 (online baseline snapshots): Creates baseline snapshots of every attached Persistent Disk while the Gen2 VM remains running and serves traffic.
- Stage 2 (maintenance-window cutover snapshots): Stops the Gen2 VM after explicit user confirmation and creates incremental snapshots that transfer only blocks modified since Stage 1.
- Target provisioning and verification: Creates target Hyperdisk volumes
with the assessed IOPS and throughput limits, transfers regional snapshot
schedules, boots the target Gen4 VM with NVMe and gVNIC, and verifies
mounts,
google-*udev symlinks, and workload integrity checks while keeping the stopped Gen2 VM and Persistent Disks intact for rollback.
- Pre-cutover capture and guest verification: Exports source VM and disk
JavaScript Object Notation (JSON) manifests, checks Resource Manager tag
bindings, and verifies over SSH that
Requirements¶
To deploy the lab environment and run the Agent Skills, you need:
- A Google Cloud project with billing enabled.
- An account granted the following Identity and Access Management (IAM) roles on
the project:
- Compute Instance Admin (v1) (
roles/compute.instanceAdmin.v1) - Compute Storage Admin (
roles/compute.storageAdmin) - Compute Network Admin (
roles/compute.networkAdmin) - Compute Security Admin (
roles/compute.securityAdmin) - Service Account Admin (
roles/iam.serviceAccountAdmin) - Service Account User (
roles/iam.serviceAccountUser) - Project IAM Admin (
roles/resourcemanager.projectIamAdmin) - Browser (
roles/browser) or Viewer (roles/viewer) - Cloud Quotas Viewer (
roles/cloudquotas.viewer) - Service Usage Admin (
roles/serviceusage.serviceUsageAdmin) - Tag Viewer (
roles/resourcemanager.tagViewer) and Tag User (roles/resourcemanager.tagUser) - IAP-secured Tunnel User (
roles/iap.tunnelResourceAccessor)
- Compute Instance Admin (v1) (
- The
Google Cloud command-line interface (CLI)
(
gcloud, with thegcloud alphacomponent installed orcurlfor the Representational State Transfer (REST) quota fallback),jq, andcurl. - Terraform installed.
- An AI coding agent CLI that supports the Agent Skills specification, such as Gemini CLI or Antigravity CLI.
Prepare the environment¶
-
Set your project, region, and zone environment variables and configure the
gcloudCLI: -
Clone the repository and change the working directory:
Deploy the lab environment with Terraform¶
If you already have Gen2 Compute Engine instances with Persistent Disks that you want to assess or migrate, skip to Assess Persistent Disks and Gen2 VMs.
To provision the sample Gen2 testbed:
-
Initialize Terraform in the
terraformdirectory: -
Apply the Terraform configuration to create both the single-disk Gen2 VM and the multi-disk Gen2 VM:
-
Inspect the provisioned VM and Persistent Disk names from the Terraform outputs:
The startup script formats the attached Persistent Disks, mounts them by filesystem UUID with the
nofailoption in/etc/fstab, and writes a reference dataset and Secure Hash Algorithm 256-bit (SHA-256) manifest (checksums.sha256) on each mounted data volume so the migration skill can verify data integrity before and after cutover.
Assess Persistent Disks and Gen2 VMs¶
Start your agent CLI from the projects/migrate-persistent-disks-to-hyperdisks
directory so it discovers the skills under .agents/skills/.
-
To assess every Compute Engine VM and Persistent Disk in the project, give your agent the following prompt:
-
To assess a single Gen2 instance in a specific zone, give your agent the following prompt:
-
Review the generated assessment report, which includes:
- Disk inventory covering disk types such as
pd-balanced,pd-ssd, andpd-standard, attached snapshot schedules, encryption type, and architecture. - Compatibility and blocker status for the target Gen4 machine series, including NVMe and gVNIC support, regional zone availability, and per-VM Hyperdisk attachment limits.
- Source Persistent Disk effective IOPS and throughput baselines computed
from disk type, capacity (
sizeGb), and per-instance sharing rules, along with right-sized target Hyperdisk types, IOPS, and throughput values. - Regional quota headroom and pre-filled
gcloudcommands for the two-stage cutover.
- Disk inventory covering disk types such as
Migrate Persistent Disks to Hyperdisk¶
After reviewing the assessment report, use the migrate-pd-to-hyperdisk-gce
skill to execute the two-stage snapshot migration.
-
Prompt your agent with the source Gen2 VM name, target Gen4 VM name, target machine type, and assessed Hyperdisk parameters:
-
Review and approve each gate as the skill progresses:
- Step 1 (Pre-cutover manifests and guest checks): The skill exports
JSON manifests of the source VM and disks and runs a read-only SSH check
to confirm
/etc/fstabUUID entries,nvmeandgvekernel drivers, and pre-cutoverchecksums.sha256verification. - Step 2 (Stage 1 online baseline snapshots): After your approval, the
skill creates baseline snapshots (
*-s1-<timestamp>) of the boot and data Persistent Disks while the Gen2 VM continues running. - Step 3 (Stage 2 maintenance-window stop and incremental snapshots):
When you confirm the maintenance window, the skill stops the source Gen2
VM (
TERMINATED) and captures incremental snapshots (*-s2-<timestamp>) containing only blocks modified since Stage 1. - Step 4 (Target Hyperdisk creation and schedule transfer): The skill
provisions the target
hyperdisk-balanced,hyperdisk-extreme, orhyperdisk-throughputvolumes from the Stage 2 snapshots and transfers any regional snapshot schedules from the source Persistent Disks to the new Hyperdisk volumes. - Step 5 (Target Gen4 VM creation and verification): The skill creates
the target VM with
--network-interface="nic-type=GVNIC", waits forgoogle-guest-agentto publish the SSH host keys to Compute Engine guest attributes (hostkeys/), and runs a post-cutover check over IAP SSH to confirm/dev/nvme*n*devices,/dev/disk/by-id/google-*symlinks,gvnicdriver binding, mounted filesystems, andchecksums.sha256integrity.
- Step 1 (Pre-cutover manifests and guest checks): The skill exports
JSON manifests of the source VM and disks and runs a read-only SSH check
to confirm
Clean up the environment¶
To remove all resources created in the lab environment and avoid ongoing charges, destroy the Terraform configuration:
terraform -chdir=terraform destroy \
-var="project_id=${PROJECT_ID}" \
-var="region=${REGION}" \
-var="zone=${ZONE}"
During terraform destroy, the terraform_data.migration_cleanup destroy-time
provisioner runs terraform/scripts/cleanup_migrated_resources.sh to delete any
migrated Gen4 VMs, reserved internal Internet Protocol (IP) addresses, Hyperdisk
volumes, and snapshots matching the lab's naming prefix before Terraform
destroys the regional snapshot schedule and VPC network.