I recently noticed that a configuration of three GPUs in Thunderbolt 3 enclosures connected to a Thunderbolt 4 hub that is connected to a Thunderbolt 4 port of a PC with a 12th generation Intel CPU works.
I also confirmed that a total of 6 GPUs will enumerate after connecting a second Thunderbolt 4 hub with 3 more GPUs to the laptop’s other TB4 port.
Here are actual and relative completion times using three GTX 1060 GPUs in this configuration to Optix render the Classroom image in Blender https://download.blender.org/demo/test/classroom.zip
|# of GTX 1060||seconds||relative to 1 GPU|
Current OS and drivers seem to do a good job sorting out PCI memory address assignments after a cold boot or restart. Hot plug and unplug worked with the AMD driver but has not consistently worked with the
Daisy chaining hubs off of hubs did not enumerate though.
Here is the equipment I tried.
- Acer Swift 3 SF314-512 laptop, Windows 11 22H2 Build 22621.1413
- ASUS Z690 motherboard + ThunderboltEX4 card, Ubuntu 22.04 linux 5.15-60
- 4 Nvidia GTX 1060 6GB, 1 AMD RX 6900 XT, 1 RX 5700 XT
- Breakaway Box 750 enclosures and Thunderbolt 4 hubs from Sonnet Technologies
I used Sonnet devices because I have a business relationship with them. I imagine equivalent products from other vendors would work too.
I wonder if 12 eGPUs would enumerate with the Frameworks laptop that has four Thunderbolt 4 ports.
I would describe this niche use case as follows.
- Run software that uses multiple GPUs such as Blender, MATLAB from Mathworks, Mathematica from Wolfram Research and perhaps machine learning tools like PyTorch.
- Run software locally because access to remote servers with GPUs is unavailable.
- Use a laptop
- Have a computer with Thunderbolt 4 ports
- Have workloads that are not significantly affected by the limited PCIe bus bandwidth between CPU and GPUs.
- Have space and budget for eGPU enclosures and hubs.
I am not sure a configuration like this meets anyone’s needs. I just wanted to share that it exists.