PixInsight GPU acceleration for AMD

I am thrilled to share that GPU accelerated tensorflow plugins on PixInsight now work with AMD 7000 series (76000,7800,7900xtx) GPUs. It took a little bit to figure out the build process and successfully get a library. I hope this simple guide can help get you up and running and enjoying GPU acceleration with StarNet++ and other apps.

This guide has been updated to reflect that WSL2 GPU Pass through support works on Windows and Ubuntu with latest ROCm release. This will allow windows users to run PixInsight from WSL2 and enjoy GPU Pass through acceleration. I hope we see continued development for native Windows tensorflow, but for now, this will allow you to run the Linux version almost like it’s native.

Requirements

• Ubuntu 22.0.4
• PixInsight (Latest version)
• AMD GPU (AMD 7000 series)
• GPU enabled scripts to verify (Starnet++)

Optional: WSL2 ROcm

These steps are only needed if you run WSL2 Ubuntu inside Windows. Only tested on 7000 series GPUs.

• Install latest Radeon Driver 24.10.21.01 for WSL2
• Reboot
• Jump to the Install ROCM Windows WSL2 section.

Install rocm – Ubuntu no WSL2

These steps are for bare metal / VM running Ubuntu 22.04

sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"
sudo usermod -a -G render,video $LOGNAME # Adding current user to Video, Render groups. See prerequisites.
sudo apt update
wget https://repo.radeon.com/amdgpu-install/6.1.3/ubuntu/jammy/amdgpu-install_6.1.60103-1_all.deb
sudo apt install ./amdgpu-install_6.1.60103-1_all.deb
amdgpu-install --usecase=rocm,dkms
echo "Please reboot system for all settings to take effect."

Be sure to reboot as the above script says. This process will build kernel drivers and also set up permissions for render/video so the GPU can access the kernel drivers. GPU Acceleration in PixInsight with tensorflow based apps will NOT work until you restart.

Note: If you are having issues with PixInsight failing for QT crashes on WSL2 Ubuntu, please be sure to install the following packages and their dependencies.

sudo apt-get install libqt5core5a libqt5svg5 libqt5webenginecore5 libqt5webengine5 libqt5webenginewidgets5 libqt5x11extras5

Install ROCM – Windows WSL2

These steps are for WSL2 on Windows 10/11 running Ubuntu 22.04

sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"
sudo apt update
wget https://repo.radeon.com/amdgpu-install/6.1.3/ubuntu/jammy/amdgpu-install_6.1.60103-1_all.deb
sudo apt install ./amdgpu-install_6.1.60103-1_all.deb
amdgpu-install -y --usecase=wsl,rocm --no-dkms
echo "Please restart WSL2 session for these to take effect."

Download libtensorflow for amd

I compiled this on Ubuntu 22.0.4 against ROCm 6.1.3 using kernel 6.6. Please make sure you are on a kernel 6.6 (or if 6.8 is available when you read this, use kernel 6.8)

Install into pixinsight

Backup your original libtensorflow in PixInsight:

cd /opt/PixInsight/bin/lib
mkdir bak
mv libtensorflow* bak

Extract the download file to /usr/local from the path where you downloaded it too.

sudo tar -C /usr/local -xzf /path/to/Downloads/libtensorflow217.tar.gz

Run linker to build OS library

sudo ldconfig /usr/local/lib

Exit/Restart PixInsight and try StarNet or run StarNet CLI. PixInsight won’t load this library if you don’t restart after running ldconfig.

Update mesa & Drivers

PixInsight needs updated MESA drivers on WSL2 and standard Ubuntu Installs.

sudo add-apt-repository ppa:kisak/kisak-mesa
sudo apt update
sudo apt upgrade

Once this is complete, then you will want to reboot as it may have updated the kernel blobs for your graphics driver as well.

GPU accelerated Starnet++

Now let’s verify that StarNet will use the AMD GPU. StarNet includes a libtensorflow.so in its directory that you will need to replace with my AMD ones if you run the CLI version, but if you run this tool in PixInsight it should use the one we installed in your ld path.

bymiller@byron-X570:~/Downloads/StarNetv2CLI_linux$ sh run_starnet.sh 
2024-05-12 21:01:54.012227: E external/local_xla/xla/stream_executor/plugin_registry.cc:90] Invalid plugin kind specified: DNN
2024-05-12 21:01:54.033232: E external/local_xla/xla/stream_executor/plugin_registry.cc:90] Invalid plugin kind specified: BLAS
Reading input image... Done!
Bits per sample: 16
Samples per pixel: 3
Height: 712
Width: 1048
Restoring neural network checkpoint... Done!
2024-05-12 21:01:54.384899: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-12 21:01:54.436196: I external/local_xla/xla/stream_executor/rocm/rocm_gpu_executor.cc:812] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-05-12 21:01:54.458272: I external/local_xla/xla/stream_executor/rocm/rocm_gpu_executor.cc:812] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-05-12 21:01:54.458309: I external/local_xla/xla/stream_executor/rocm/rocm_gpu_executor.cc:812] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-05-12 21:01:54.458427: I external/local_xla/xla/stream_executor/rocm/rocm_gpu_executor.cc:812] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-05-12 21:01:54.458460: I external/local_xla/xla/stream_executor/rocm/rocm_gpu_executor.cc:812] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-05-12 21:01:54.458493: I external/local_xla/xla/stream_executor/rocm/rocm_gpu_executor.cc:812] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2024-05-12 21:01:54.458511: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1929] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 23512 MB memory:  -> device: 0, name: Radeon RX 7900 XTX, pci bus id: 0000:0a:00.0
Total number of tiles: 15
2024-05-12 21:01:54.798149: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:388] MLIR V1 optimization pass is not enabled
100% finished

Done!

Hooray! AMD Users Rejoice at beautiful GPU accelerated StartNet++ (and other PixInsight tools that are GPU enabled)

Radeontop (Bare metal/VM only)

You can install radeontop and run it in a shell while you run starnet or other Tensorflow jobs and see the GPU load spike.

Note: This only works on VM and bare metal. GPU pass through on WSL2 does not show up as a linux device as radeontop and nvtop expect. You can use Windows task manager to view your GPU resources.

sudo apt install radeontop

View GPU load on Windows

From Windows machine, you can right click on your start windows icon, select task manager and then click performance and select GPU. When you run a PixInsight process that uses GPU, you can see it spike here in Windows.

Here you can see the memory and Computer spike when running Noise Xterminator. This process would usually take 20 minutes or longer on full frame masters but now finishes in seconds.

AMD GPU Accelerated RC-Astro

I’m happy to report that the amazing rc-astro tools all work with this set up as well.

StarXTerminator – Remove stars or create star mask.

NoiseXTerminator – AI Noise Removal.

BlurXTerminator – Deconvolution.

You can download the trial version of these from: https://www.rc-astro.com/pixinsight-installation-instructions/

Comments

Please leave a comment below if this works or doesn’t work for you. I have another tensorflow library built on 2.13 that may work with older AMD GPUs if this one doesn’t work for you.

May your skies be clear.