For the complete 140-node DGX SuperPOD, the three-layer switches all use 40-port NVIDIA QM8790 switches. Every 20 DGX A100s in the cluster form a SU, and there are 8 Leaf switches in each SU. The design is rail optimized through both the leaf and spine levels—each InfiniBand HCA on a DGX A100 system is connected to its fat tree topology. This rail-optimized network architecture is of great help in improving the performance of deep learning training.