How to Deploy a Deep Learning Environment (PyTorch & CUDA) on an Ubuntu GPU Dedicated Server

Prepare Ubuntu Environment

• System Readiness

  Before installing any deep learning libraries, you must ensure your Ubuntu server (22.04 LTS or 24.04 LTS) is fully updated and has the necessary build tools.

Update and Install Tools

1. Run Update Commands

   Connect to your server via SSH and run the following commands to refresh repositories and install dependencies:

   bash Copy

   
   sudo apt update && sudo apt upgrade -y
   sudo apt install build-essential python3-dev python3-pip python3-venv software-properties-common -y
                                           

Driver Configuration

• Avoid Open-Source Drivers

  Open-source drivers (like nouveau) cannot be used for deep learning. You must install the official NVIDIA proprietary drivers.

Installation Process

1. Identify and Install Recommended Drivers

   bash Copy

   
   # Check recommended drivers
   ubuntu-drivers devices
   
   # Auto-install the driver
   sudo ubuntu-drivers autoinstall
   
   # Reboot to apply changes
   sudo reboot
   
   # Verify after reboot
   nvidia-smi
                                           

   nvidia-smi should display your GPU model and VRAM usage.

System-Wide CUDA

• Why install CUDA?

  Installing the system-wide CUDA toolkit is required if you plan on compiling custom CUDA extensions or using frameworks like DeepSpeed.

Installation Steps

1. Install from Repositories

   bash Copy

   
   sudo apt install nvidia-cuda-toolkit -y
   nvcc --version
                                           

Virtual Environments

• Best Practices

  Never install PyTorch globally. Always use a virtual environment to prevent dependency conflicts.

Setup Workspace

1. Create and Activate

   bash Copy

   
   mkdir ~/ai-workspace
   cd ~/ai-workspace
   python3 -m venv pytorch-env
   source pytorch-env/bin/activate
                                           

   source pytorch-env/bin/activate will change your terminal prompt.

PIP Installation

• PyTorch Binaries

  We will use the official PyTorch PIP index to ensure we get the version compiled with CUDA 12.1.

Run Installation

1. Install Libraries

   bash Copy

   
   pip install torch torchvision torchaudio
                                           

Hardware Communication

• Check GPU Accessibility

  The final step is to prove that PyTorch can successfully "talk" to your NVIDIA GPU.

Python Test Script

1. Run Verification Code

   python Copy

   
   import torch
   # Check if CUDA is available
   print("CUDA Available: ", torch.cuda.is_available())
   
   # Get the name of the GPU
   if torch.cuda.is_available():
       print("GPU Model: ", torch.cuda.get_device_name(0))
                                           

   Expected output: CUDA Available: True