The Best Way to Get Deepseek-R1: By Using an LXC in Proxmox

Josef_Founder · February 3, 2025, 3:11am

Supplemental for - Local And Distributed AI Redux (Proxmox)

Local AI (Deepseek-R1) Using Open WebUi Within a Proxmox LXC Container

You might think it’s wise to install an advanced LLM chat bot with unknown allegiances outside of a controlled container. I don’t. Enter ‘The Best Way to Get Deepseek-R1: By Using an LXC in Proxmox.’ Did I mention you can scale containers using Kubernetes and/or a fabric of resources; or even a swarm of chat bots? Spiffy. I digress…

Prerequisites:

Proxmox Installed - Installation - Proxmox Virtual Environment

Minimum System Requirements:

A computer or node with 8GB RAM, 32GB Hard Drive, 4 Processor Cores, and a second computer to access Open WebUI from inside browser (Chrome, Brave, etc.).

Recommend:

16GB RAM minimum, 100GB Hard Drive, 8 Processor Cores minimum, your gaming friend’s finest graphics card ~2018 or newer with at least 8GB of RAM, and a second computer to access Open WebUI from inside browser (Chrome, Brave, etc.).

Installation

From: Proxmox VE Helper-Scripts (scroll down)

In a Proxmox Host’s shell, we need to command the host to create our LXC:

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/openwebui.sh)"

You can select default settings for ease of use, but you will save yourself some problems in the future as enumerated below if you set yourself up for success by performing a manual installation instead.

Specifically, give your LXC a static IP address (e.g. 192.168.0.14/24) so it’s easy for you to find later, set the root password yourself so it’s easier to find later, give Open WebUI about 100GB instead of 16GB hard disk space, give yourself as much RAM as possible instead of the default 4GB. Give yourself minimum 8 processor cores, more is better. (We will follow up later on with instructions for graphics card passthrough which dramatically improves performance).

Hop On Your Other Computer

Now that it’s installed navigate in web browser to the address provided by the installer, or the web address you had provided as the static IP address (192.168.0.14:8080). Yes, you need the port 8080 at the end of your URL.

After that, we need to download our model of choice for this example, ‘deepseek-r1’. The following guide was helpful, but skip to the next sentence for the simplest method.
https://dev.to/pavanbelagatti/run-deepseek-r1-locally-for-free-in-just-3-minutes-1e82

Simpler method: use the search box inside of the web-based WebUI, search for ‘deepseek-r1’. Scroll down for screenshots. But first…

As before, for the default of 16GB HDD, 8GB RAM is not sufficient for this download. You will need to shut down your LXC, resize the disk (try 100GB for starts), boot back up. You do this by clicking on the LXC container that was created by the installer script, then click resources, click memory, click edit, change your RAM size. Then click Root Disk → Volume Action → Resize to also increase your hard drive space.

Now navigate to your LXC in browser including port 8080 (e.g. http://192.168.0.14:8080), search for ‘deepseek-r1,’ and click ‘Pull “deepseek-r1” from Ollama.com’ to download. It’s 4.7GB. We already increased our RAM prior.

A Common Problem?

I had the following error - 500: Ollama: 500, message=‘Internal Server Error’, url=‘http://0.0.0.0:11434/api/chat’

I read this was resolved if you simply update Ollama by running the install script again in the LXC’s shell. This didn’t fix the issue for me, but this is a nice command to have in your toolbelt:

curl -fsSL https://ollama.com/install.sh | sh

shutdown

To actually resolve the issue, I went to container->resources->memory->edit and increased my memory to greater than 8GB to resolve the error message preemptively.

Now click start (inside Proxmox) to boot the LXC with the shiny new resources.

Next hop back on the other computer, navigate again to our static IP 192.168.0.14:8080 and the error should be resolved.

CONGRATULATIONS!

So we are off to the races. Next I will brush on adding our graphics card(s) or coprocessors to significantly improve performance.

Josef_Founder · February 3, 2025, 6:16pm

What are you doing, HAL?

(source: Hal 9000 Gif GIFs | Tenor)

So I mentioned over on my other guide the dangers of trusting this Deepseek-R1 LLM. If it seems too good to be true, it probably is. This is suspicious. Now it could be me, granted. But I thought the reader might like to know about this potential vulnerability. Whenever I am conversing with the LLM at present, my pihole becomes completely clogged by appearances. Though the processing power in Proxmox dashboard is not bogged, the UI is clearly throwing red flags and feels the need to rate limit my laptop. The kicked client is my own laptop, not the LLM’s container. Interesting. Now I’ve encountered a similar error before but that was a year ago and I don’t recall exactly what was causing that, it could just be me, but I view the following as a red flag. Why is it only rate limiting my laptop during conversation? We need an LLM that we can trust, and I just don’t. That’s sort of why myself and others are working towards a more open source and less pre-programmed instance. Here’s the screenshot:

Josef_Founder · February 9, 2025, 7:40pm

This might help with adding NGINX:

Something incomplete which I am toying with right now:

And here’s a repo I have for Tor + Unbound + Pihole + DNSSEC:

Or More Cloudflare-centric si tu préfères:

Josef_Founder · February 17, 2025, 11:26pm

Graphics Card Passthrough

*heretofore is rough draft at this time, won’t you kindly report anything I might have missed? Thank you.

THIS IS THE MOST IMPORTANT LINK FOR PASSING PERMISSIONS:
GitHub - H3rz3n/proxmox-lxc-unprivileged-gpu-passthrough: A small guide to help user correctly passthrough their GPUs to an unprivileged LXC container

https://bbs.archlinux.org/viewtopic.php?id=292371

github.com/ollama/ollama

docs/linux.md

main

# Linux

## Install

To install Ollama, run the following command:

```shell
curl -fsSL https://ollama.com/install.sh | sh
```

## Manual install

> [!NOTE]
> If you are upgrading from a prior version, you should remove the old libraries with `sudo rm -rf /usr/lib/ollama` first.

Download and extract the package:

```shell
curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
sudo tar -C /usr -xzf ollama-linux-amd64.tgz

This file has been truncated. show original

GPU passthrough to an LXC in Proxmox is not as straightforward as doing it for a VM because LXCs share the host kernel. However, it is possible by binding the GPU to the container using lxc.cgroup settings and configuring the necessary drivers.

Here’s a step-by-step guide to passing through your GPU to your LXC container in Proxmox:

1. Enable IOMMU on the Proxmox Host

First, make sure IOMMU is enabled on your Proxmox server.

Check if IOMMU is enabled:

dmesg | grep -e DMAR -e IOMMU

If you don’t see output indicating IOMMU is enabled, proceed to enable it.

Edit GRUB Configuration:

nano /etc/default/grub

Find the line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet"

Modify it based on your CPU vendor:

For Intel:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

For AMD:

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

Save the file and update GRUB:

update-grub
reboot

2. Check GPU and Bind It to the Host

After rebooting, check if your GPU is recognized:

lspci -nnk | grep -i -A3 vga

If you see something like:

makefile

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [xxxx] (rev a1)

Make note of the PCI ID (01:00.0 in this case).

3. Configure the GPU for LXC

Since LXCs share the kernel, we cannot use PCI passthrough like in a VM, but we can pass through the GPU devices.

3.1. Identify GPU Devices

Run:

ls -l /dev/dri

You should see something like:

/dev/dri/card0
/dev/dri/renderD128

These are the devices your container needs access to.

3.2. Modify the LXC Configuration

Edit your LXC container’s config file:

nano /etc/pve/lxc/100.conf

(Add the following lines)

ini

lxc.cgroup2.devices.allow = c 226:* rwm
lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir

(Replace 100 with your actual LXC ID)

If you’re using an NVIDIA GPU, also add:

ini

lxc.cgroup2.devices.allow = c 195:* rwm
lxc.mount.entry = /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry = /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry = /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry = /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file

Save and exit.

4. Install GPU Drivers in the LXC

Start the container:

bash

pct start 100

Inside the container, install the required drivers.

For NVIDIA GPUs:

apt install -y nvidia-driver nvidia-cuda-toolkit

For AMD GPUs (ROCm support):

apt install -y mesa-utils rocm-opencl-runtime

Verify the GPU is accessible in the container:

For NVIDIA:

nvidia-smi

For AMD:

clinfo

5. Configure Open WebUI & Deepseek-R1

If you’re running Open WebUI or DeepSeek-R1, ensure it detects the GPU.

Inside the LXC, set the environment variable:

export CUDA_VISIBLE_DEVICES=0

Then, start DeepSeek-R1 with GPU acceleration:

deepseek-r1 --device cuda

For Open WebUI: Modify its config file to ensure it uses the GPU.

Final Notes

If the container doesn’t detect the GPU, check if the /dev/dri devices are accessible.
Make sure your host Proxmox kernel has the necessary GPU modules loaded (modprobe nvidia or modprobe amdgpu).
If you’re using an NVIDIA GPU, ensure the NVIDIA Container Toolkit is installed for compatibility.

Some useful drivers I found over at:

"From what I’ve gathered:
For Nvidia you need drivers: ollama/docs/linux.md at main · ollama/ollama · GitHub
For AMD you need additional package and drivers: ollama/docs/linux.md at main · ollama/ollama · GitHub ollama/docs/linux.md at main · ollama/ollama · GitHub
For Intel you need to build Ollama source code from scratch with intel-basekit and have Intel drivers (this script covers it: Proxmox/install/ollama-install.sh at main · tteck/Proxmox · GitHub) "

Josef_Founder · February 17, 2025, 11:33pm

Alternative Method: GPU Passthrough

Setting up GPU passthrough in Proxmox to provide GPU support to an LXC container running Open Web UI with DeepSeek-R1 involves several steps. However, it’s important to note that GPU passthrough to LXC containers can be complex and may not always yield optimal performance. Some users have found that using virtual machines (VMs) for GPU passthrough offers a more straightforward and reliable setup.

forum.proxmox.com

Hardware Requirements:

DeepSeek-R1 models have significant VRAM requirements. For instance, the DeepSeek-R1-Distill-Qwen-1.5B model requires approximately 3.9 GB of VRAM, with an NVIDIA RTX 3060 12GB or higher recommended. Larger models, like the DeepSeek-R1-Distill-Qwen-14B, require around 36 GB of VRAM, necessitating a multi-GPU setup (e.g., NVIDIA RTX 4090 x2).

apxml.com

Setting Up GPU Passthrough to an LXC Container:

If you choose to proceed with GPU passthrough to an LXC container, follow these general steps:

Prepare the Proxmox Host:

Enable IOMMU:
- Edit /etc/default/grub and modify the GRUB_CMDLINE_LINUX_DEFAULT line to include intel_iommu=on for Intel CPUs or amd_iommu=on for AMD CPUs.
- Update GRUB:

sql

update-grub

* Reboot the host.

Load VFIO Modules:
- Add the following lines to /etc/modules:

nginx

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

* Identify the GPU's device IDs using `lspci` and create a VFIO configuration in `/etc/modprobe.d/`.

Install NVIDIA Drivers:
- Install the appropriate NVIDIA drivers on the Proxmox host to ensure proper GPU functionality.

Configure the LXC Container:

Create a Privileged LXC Container:
- GPU passthrough typically requires privileged containers.
Modify Container Configuration:
- Edit the container’s configuration file located at /etc/pve/lxc/<container_id>.conf and add:

bash

lxc.cgroup.devices.allow = c 195:* rwm
lxc.mount.entry = /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry = /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry = /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file

* Ensure these device files exist on the host.

Install NVIDIA Drivers Inside the Container:
- This may involve binding the host’s driver libraries into the container or installing drivers directly within the container.

Alternative Approach:

Due to the complexities and potential issues with LXC GPU passthrough, consider using a VM for this purpose. VMs often provide better support and stability for GPU passthrough configurations. Users have reported that CUDA on a VM with GPU passthrough works seamlessly on Proxmox, making it a more pragmatic choice for GPU-intensive workloads.

forum.proxmox.com

Additional Resources:

For a visual walkthrough of setting up GPU passthrough in Proxmox LXC containers, you may find the following video helpful:

Please ensure you have proper backups and understand the potential risks before modifying system configurations. Consult the Proxmox documentation and community forums for detailed guidance tailored to your specific setup.

Josef_Founder · February 18, 2025, 8:59pm

Josef_Founder · February 19, 2025, 3:26am

A script I made to automate GPU Passthrough:

UPDATE: JUST FOLLOW THIS LINK, PLEASE

sudo nano gpu.sh

Paste the below script (gpu.sh) into our nano file

Give gpu.sh permission to do its thing:

chmod +x gpu.sh

Replace LXC_ID with your Container Number (e.g. Proxmox defaults to 100)

sudo bash gpu.sh <LXC_ID>

gpu.sh

#!/bin/bash

set -e

LXC_ID=$1  # LXC container ID passed as an argument
if [ -z "$LXC_ID" ]; then
    echo "Usage: $0 <LXC_ID>"
    exit 1
fi

# Enable IOMMU
if grep -q "GenuineIntel" /proc/cpuinfo; then
    echo "Detected Intel CPU. Enabling Intel IOMMU."
    IOMMU_PARAM="intel_iommu=on iommu=pt"
elif grep -q "AuthenticAMD" /proc/cpuinfo; then
    echo "Detected AMD CPU. Enabling AMD IOMMU."
    IOMMU_PARAM="amd_iommu=on iommu=pt"
else
    echo "Unknown CPU vendor. Exiting."
    exit 1
fi

sed -i "s/GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT=\"quiet $IOMMU_PARAM\"/" /etc/default/grub
proxmox-boot-tool refresh
echo "IOMMU enabled. Reboot required."

# Detect GPU
GPU_TYPE="unknown"
if lspci | grep -i nvidia; then
    GPU_TYPE="nvidia"
elif lspci | grep -i amd; then
    GPU_TYPE="amd"
elif lspci | grep -i 'Intel Corporation' | grep -i 'VGA'; then
    GPU_TYPE="intel"
fi

echo "Detected GPU: $GPU_TYPE"

# Configure LXC container for GPU passthrough
LXC_CONF="/etc/pve/lxc/$LXC_ID.conf"

if [ ! -f "$LXC_CONF" ]; then
    echo "LXC container config not found: $LXC_CONF"
    exit 1
fi

echo "Configuring LXC GPU passthrough..."
cat <<EOL >> $LXC_CONF
lxc.cgroup2.devices.allow = c 226:* rwm
lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
EOL

if [ "$GPU_TYPE" == "nvidia" ]; then
    cat <<EOL >> $LXC_CONF
lxc.cgroup2.devices.allow = c 195:* rwm
lxc.mount.entry = /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry = /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry = /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry = /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
EOL
fi

# Restart LXC container
pct stop $LXC_ID
pct start $LXC_ID

# Install drivers inside LXC
echo "Installing GPU drivers in LXC container..."
if [ "$GPU_TYPE" == "nvidia" ]; then
    pct exec $LXC_ID -- bash -c "apt update && apt install -y nvidia-driver nvidia-cuda-toolkit"
elif [ "$GPU_TYPE" == "amd" ]; then
    pct exec $LXC_ID -- bash -c "
        apt update && apt install -y wget && \
        wget https://repo.radeon.com/amdgpu-install/6.0/ubuntu/jammy/amdgpu-install_6.0.60000-1_all.deb && \
        dpkg -i amdgpu-install_6.0.60000-1_all.deb && \
        amdgpu-install --usecase=rocm"
elif [ "$GPU_TYPE" == "intel" ]; then
    pct exec $LXC_ID -- bash -c "
        apt update && apt install -y intel-media-va-driver && \
        echo 'Intel GPUs require building Ollama from source with intel-basekit. Use the script from: https://github.com/tteck/Proxmox/blob/main/install/ollama-install.sh'"
fi

# Verify GPU availability inside LXC
if [ "$GPU_TYPE" == "nvidia" ]; then
    pct exec $LXC_ID -- nvidia-smi
elif [ "$GPU_TYPE" == "amd" ]; then
    pct exec $LXC_ID -- clinfo
elif [ "$GPU_TYPE" == "intel" ]; then
    pct exec $LXC_ID -- vainfo
fi

echo "GPU passthrough setup complete for LXC ID $LXC_ID. A reboot is recommended."

Now we are ready to set up Open WebUI, itself:

To enable your graphics card(s) in Open WebUI, you need to configure the Ollama backend (if using Ollama for LLM inference) or CUDA/ROCM for models that support GPU acceleration.

1. Verify GPU Availability

Before configuring Open WebUI, ensure your GPU is accessible inside the LXC container:

For NVIDIA GPUs:

Run:

bash

nvidia-smi

If your GPU appears in the output, it’s recognized.

For AMD GPUs (ROCM-based):

Run:

bash

clinfo

or

bash

rocminfo

For Intel GPUs:

Run:

bash

vainfo

2. Configure Ollama for GPU Acceleration

If you’re using Ollama in Open WebUI, you need to ensure GPU acceleration is enabled.

For NVIDIA GPUs (CUDA)

Enable CUDA support in Ollama:

bash

OLLAMA_CUDA=1 ollama run llama3

Or set it permanently:

bash

echo "export OLLAMA_CUDA=1" >> ~/.bashrc
source ~/.bashrc

For AMD GPUs (ROCM)

Enable ROCm support:

bash

OLLAMA_ROCM=1 ollama run llama3

Or set it permanently:

bash

echo "export OLLAMA_ROCM=1" >> ~/.bashrc
source ~/.bashrc

3. Update Open WebUI Settings

Open Open WebUI in your browser.
Go to Settings (gear icon in the UI).
Look for Backend Settings.
Ensure that Ollama (or the relevant LLM engine) is selected.
If there’s an option for GPU Acceleration, enable it.

4. Restart Open WebUI & Test GPU Usage

Restart Open WebUI to apply changes:

bash

systemctl restart open-webui

Then, check if the GPU is being utilized:

bash

nvidia-smi  # For NVIDIA
clinfo       # For AMD
vainfo       # For Intel

Alright, OK, I’ll just automate this for you…

run the following inside of the LXC container:

sudo nano gpuLXC.sh

chmod +x gpuLXC.sh

sudo ./gpuLXC.sh

gpuLXC.sh:

#!/bin/bash

set -e

# Ensure running as root
if [ "$EUID" -ne 0 ]; then
    echo "Please run as root or with sudo"
    exit 1
fi

# Update package lists
apt update

# Install necessary packages including lspci
apt install -y git python3 python3-pip wget clinfo vainfo pciutils

# Clone or update Open WebUI repository
if [ -d "open-webui" ]; then
    echo "Updating existing open-webui repository..."
    cd open-webui
    git fetch --all
    git reset --hard origin/main
    git pull
else
    echo "Cloning open-webui repository..."
    git clone --depth 1 --branch main https://github.com/open-webui/open-webui.git
    cd open-webui
fi

# Install Python dependencies
if [ -f "requirements.txt" ]; then
    pip3 install -r requirements.txt
else
    echo "No requirements.txt found. Installing common dependencies..."
    pip3 install fastapi uvicorn python-multipart jinja2 aiofiles
    echo "Please check if additional dependencies are needed for your specific setup."
fi

# Detect GPUs and install drivers
GPU_TYPES=()
if lspci | grep -i nvidia; then
    GPU_TYPES+=("nvidia")
    apt install -y nvidia-driver nvidia-cuda-toolkit
    pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
fi

if lspci | grep -i amd; then
    GPU_TYPES+=("amd")
    echo "Attempting to install AMD GPU drivers..."

    # Temporarily suppress errors from dkms and initramfs-tools
    export DEBIAN_FRONTEND=noninteractive
    apt -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y amdgpu-dkms amdgpu || {
        echo "Failed to install amdgpu-dkms with standard settings. Trying to resolve initramfs update issues..."
        
        # Force update of initramfs with different compression
        update-initramfs -u -k all -c -z lz4 || {
            echo "Failed to update initramfs with lz4 compression. Attempting with gzip..."
            update-initramfs -u -k all -c -z gzip || {
                echo "Failed to update initramfs. Manual intervention might be needed."
            }
        }

        # Try installing again
        apt -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install -y amdgpu-dkms amdgpu || {
            echo "Installation of AMD GPU drivers failed. Please check logs for manual fixes."
        }
    }
    pip3 install torch torchvision torchaudio rocm
fi

if lspci | grep -i 'Intel Corporation' | grep -i 'VGA'; then
    GPU_TYPES+=("intel")
    apt install -y intel-media-va-driver
    pip3 install openvino
fi

# Configure GPU environment for Python
cat <<EOL > gpu_config.py
import os

# Set CUDA_VISIBLE_DEVICES if NVIDIA GPU is present
if 'nvidia' in os.environ.get('GPU_TYPES', '').split(','):
    os.environ['CUDA_VISIBLE_DEVICES'] = 'all'

# For AMD and Intel, we'll use specific environment variables
os.environ['GPU_TYPES'] = '${GPU_TYPES[*]}'

# Additional setup for AMD or Intel might be required here

if 'amd' in os.environ.get('GPU_TYPES', '').split(','):
    os.environ['ROCM_PATH'] = '/opt/rocm'
    
if 'intel' in os.environ.get('GPU_TYPES', '').split(','):
    os.environ['INTEL_OPENVINO_DIR'] = '/opt/intel/openvino'

EOL

# Adjust Open WebUI's main script or config to include GPU settings
# Here's an example; adapt as necessary based on Open WebUI's actual structure
if [ -f "run.py" ]; then
    sed -i '/import os/a import gpu_config' run.py
else
    echo "Could not find 'run.py'. Please manually configure GPU usage in the appropriate script or config file."
fi

# Run Open WebUI (this is a placeholder; replace with actual run command)
if [ -f "run.py" ]; then
    echo "Starting Open WebUI with GPU support..."
    python3 run.py  # Note: This is a placeholder. Use the actual command to start Open WebUI
else
    echo "No run.py found. Please use the correct command to start Open WebUI."
fi

# Verify GPU usage
echo "Verifying GPU usage:"
for gpu_type in "${GPU_TYPES[@]}"; do
    case $gpu_type in
        "nvidia")
            nvidia-smi || echo "NVIDIA GPU detected but 'nvidia-smi' failed to run."
            ;;
        "amd")
            clinfo || echo "AMD GPU detected but 'clinfo' failed to run."
            ;;
        "intel")
            vainfo || echo "Intel GPU detected but 'vainfo' failed to run."
            ;;
    esac
done

echo "Open WebUI setup with GPU support is complete. Check the application interface for model performance."

Josef_Founder · February 19, 2025, 9:23pm

My /etc/pve/lxc/CONTAINERNUMBER.conf config initially looked like so when I finally got it to work:

arch: amd64
cores: 8
hostname: openwebui
memory: 16000
net0: name=eth0,bridge=vmbr0,hwaddr=YOURPRIVATEMACADDRESS,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: CephPool:vm-100-disk-0,size=100G
swap: 512
tags: proxmox-helper-scripts
unprivileged: 0
lxc.apparmor.profile: unconfined
lxc.cap.drop:
lxc.cgroup.devices.allow: c 226:* rwm
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/kfd dev/kfd none bind,optional,create=file
#lxc.idmap: u 0 100000 65536
#lxc.idmap: g 0 100000 65536
#lxc.idmap: u 100000 0 1
#lxc.idmap: g 100000 0 1
#lxc.idmap: g 100001 44 1

Some cool helper commands:

ls -l /dev/dri

glxinfo | grep "OpenGL"

dmesg | grep amdgpu
ls -lah /dev/dri

apt install -y rocm-dev rocm-utils

rocminfo
clinfo
dmesg | grep amdgpu

Run the following inside your Proxmox host (not inside the LXC container):

lspci -nnk | grep -A3 VGA

This should confirm that the GPU is recognized and assigned the correct driver.

Check Disk UUIDs
Run the following command to list available disk UUIDs:

lsblk -f

If 'UUIDis missing but another UUID exists, update/etc/kernel/proxmox-boot-uuids` to reflect the correct one.

Verify IOMMU Status
After rebooting, check if IOMMU is enabled by running:

dmesg | grep -i iommu

Check GPU Availability for Passthrough
If your goal is to pass through a GPU, list available GPUs with:

lspci -nnk | grep -iA3 vga

This will help confirm if the GPU is properly detected and available for passthrough.:

Ready, set, go:

OLLAMA_ROCM=1 ollama run llama3