How to Create a VM Instance with a GPU?
Written By
Ananaya Kandpal
Reading time: 4 minutes
This document offers an overview of the steps needed to create a Computer Engine instance with attached graphics processing units (GPUs). You can use GPUs to increase specific workloads, such as machine learning and data processing.
You can also use some GPU machine types on the AI Hypercomputer. AI Hypercomputer is a supercomputing system that is optimized to support your AI and ML workloads. This option is required for creating a densely distributed, performance-optimized infrastructure that has integrations for Google Kubernetes Engine and Slurm schedulers.
VMs with connected GPUs are well-suited for operations that require substantial computational resources. For example, machine learning, scientific simulations, and video rendering. This post is going to explain how to create a VM instance with a GPU
Step 1: Select Your Cloud Provider: Choose a cloud service provider that delivers GPU-enabled VM instances. Here are the trusted cloud providers:
-
Net2Secure Cloud
-
Amazon Web Services
-
Google Cloud Platform
-
Microsoft Azure
Step 2: Log in to Your Console: The first step is to open your preferred web browser and log in to the AWS Management Console.
Step 3: Direct to the EC2 Dashboard: After logging in, go to the “Services” tab located in the main window. Still under Compute, select EC2 as it’s an acronym for Elastic Compute Cloud.
Step 4: Launch a New Instance: Go to your EC2 Dashboard. Click the & Launch Instance; button to begin creating a new virtual machine.
Step 5: Choose an Amazon Machine Image (AMI): Pick an AMI that supports GPU acceleration. Look for images that already include NVIDIA drivers or are created specifically for GPU workloads. You can find these in the AWS Marketplace or use Amazon’s official GPU-ready AMIs.
Step 6: Select an Instance Type: Choose a GPU-supported instance type. On AWS, these usually start with “p” or “g” (such as p3.2xlarge or g4dn.xlarge). Select the one that fits your performance needs and budget.
Step 7: Configure Instance Details
Set up the key configuration settings:
-
Number of instances
-
VPC and subnet
-
Networking options
-
IAM role (if needed)
-
Shutdown behavior
-
Termination protection (recommended)
These settings help define how your instance behaves inside your cloud environment.
Step 8: Add Storage: Choose the size and type of storage for your VM. For GPU-related workloads, SSD storage is recommended because it provides better speed and performance.
Step 9: Add Tags (Optional): Add tags to organize your instance. Tags are helpful when working in a team or managing many resources.
Step 10: Configure Security Groups: Set up a security group to control incoming and outgoing traffic. At a minimum, allow SSH access (port 22) from your IP address so you can connect to your instance later.
Step 11: Review and Launch: Review all your settings carefully. If everything is correct, click & Launch.
Step 12: Create or Choose a Key Pair: You can either create a new key pair or use an existing one. This key pair is required for secure SSH access. Download the .pem file and keep it safe.
Step 13: Launch the Instance: Click “Launch Instances” to start the creation process. AWS will now begin setting up your GPU-enabled VM.
Step 14: Wait for Initialization: Your instance will take a few minutes to initialize. You can track its progress from the EC2 Dashboard.
Step 15: Connect to the Instance: Once the status shows Running, connect to your VM:
Select the instance in the EC2 Dashboard
Click “Connect”
Follow the SSH instructions using your key pair.
-
Select the instance in the EC2 Dashboard
-
Click “Connect”
-
Follow the SSH instructions using your key pair.
Step 16: Verify GPU Attachment: After connecting, check whether your GPU is detected:
Run: nvidia-smi
This command should display information about your attached NVIDIA GPU.
Step 17: Install Required Software: Install any additional software needed for your project, for example, CUDA Toolkit, TensorFlow, PyTorch, or other ML or data processing tools.
Step 18: Configure Your Environment: Set up your working environment, upload your data, and adjust settings based on your GPU workload.
Troubleshooting Tips
-
If nvidia-smi doesn’t work, confirm that you selected a GPU-enabled instance type and an AMI that supports GPU drivers.
-
If you can’t connect via SSH, check instance status checks in the EC2 Dashboard.
-
Look at system logs for any startup errors.
Conclusion
By applying these steps, you can easily create a GPU-enabled VM instance to manage heavy workloads, like ML training, scientific computing, or video rendering. Moreover, make sure to monitor your instance usage and costs, and follow security best practices to keep your data secure.
