Deploying a High-Performance Vector Database (Milvus or Qdrant) on NVMe Dedicated Servers for RAG

For a production RAG pipeline processing millions of vectors, we recommend a bare-metal server with at least 64GB of RAM and dedicated NVMe storage. In this guide, we are using Ubuntu 24.04 LTS.

First, connect to your server via SSH and ensure the system is fully updated:

sudo apt update && sudo apt upgrade -y
                                

Milvus (and most modern vector databases) are best deployed as containerized microservices. This ensures all dependencies (like etcd and MinIO, which Milvus uses internally) are perfectly isolated.

Install Docker:

sudo apt install docker.io -y
sudo systemctl enable --now docker
                                

Install Docker Compose (the plugin used to manage multi-container applications):

sudo apt install docker-compose-v2 -y
                                

(Verify the installation by running docker compose version).

To get the performance benefits of your iDatam server, you must ensure Docker writes the vector data directly to your NVMe drive, not the standard OS drive (if they are separate).

Assuming your NVMe drive is formatted and mounted at /mnt/nvme-data (refer to our MinIO tutorial for formatting instructions), create a dedicated directory for Milvus:

sudo mkdir -p /mnt/nvme-data/milvus/volumes
                                

We will deploy the Milvus Standalone version, which is perfect for a single, high-powered dedicated server.

Create a directory for your Milvus project and download the official docker-compose.yml file:

mkdir ~/milvus-deploy
cd ~/milvus-deploy
wget https://github.com/milvus-io/milvus/releases/download/v2.4.0/milvus-standalone-docker-compose.yml -O docker-compose.yml
                                

(Note: Always check the official Milvus documentation for the latest release version).

By default, the downloaded docker-compose.yml file will save data in the current directory. We need to edit it to point to our high-speed NVMe mount.

Open the file:

nano docker-compose.yml
                                

Locate the volumes section under the etcd, minio, and standalone services. Change the local path from ./volumes/... to your NVMe path /mnt/nvme-data/milvus/volumes/....

For example, modify the minio service volumes:

minio:
  image: minio/minio:RELEASE.2023-03-20T20-16-18Z
  environment:
    MINIO_ACCESS_KEY: minioadmin
    MINIO_SECRET_KEY: minioadmin
  ports:
    - "9001:9001"
    - "9000:9000"
  volumes:
    - /mnt/nvme-data/milvus/volumes/minio:/minio_data
  command: minio server /minio_data --console-address ":9001"
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
    interval: 30s
    timeout: 20s
    retries: 3
                                

(Make similar updates to the etcd and standalone volume mappings).

With the storage paths optimized, start the Milvus cluster in detached mode:

sudo docker compose up -d
                                

Docker will pull the necessary images (Milvus, etcd, MinIO) and start the services. This may take a few minutes depending on your network speed (which, on an iDatam server, will be incredibly fast).

Verify that all containers are running cleanly:

sudo docker compose ps
                                

You should see all three containers with a status of Up.

Managing vectors via the command line or API is standard for applications, but having a visual dashboard is crucial for debugging your RAG pipeline. Attu is the official GUI for Milvus.

We can run Attu as a lightweight Docker container alongside Milvus:

sudo docker run -p 8000:3000 -e MILVUS_URL=10.0.0.11:19530 zilliz/attu:latest
                                

(Replace 10.0.0.11 with your server's actual IP address).

Open your web browser and navigate to http://your_server_ip:8000. You will be greeted by the Attu login screen. Click "Connect" (using the default Milvus port 19530), and you can now visually inspect your vector collections, monitor memory usage, and run manual similarity searches.

You have successfully deployed a production-ready vector database. Your RAG pipeline is now capable of ingesting millions of documents and returning context to your LLM in milliseconds.

The speed of AI is entirely dependent on the speed of data retrieval. Don't build a brilliant generative AI application only to host its brain on a slow, shared VPS.

To ensure your similarity searches execute with zero hypervisor latency, deploy your vector databases on iDatam’s Storage Dedicated Servers featuring enterprise PCIe Gen 5 NVMe arrays. Own your infrastructure, secure your data, and deliver answers instantly.