{"id":11253,"date":"2026-09-28T10:01:28","date_gmt":"2026-09-28T09:01:28","guid":{"rendered":"https:\/\/yfconnectivity.com\/?p=11253"},"modified":"2026-09-28T10:05:12","modified_gmt":"2026-09-28T09:05:12","slug":"gpu-networking-guide","status":"publish","type":"post","link":"https:\/\/yfconnectivity.com\/ar\/gpu-networking-guide\/","title":{"rendered":"\u0634\u0631\u062d \u0634\u0628\u0643\u0627\u062a GPU: \u0643\u064a\u0641 \u062a\u0631\u0628\u0637 \u0639\u0646\u0627\u0642\u064a\u062f \u0627\u0644\u0630\u0643\u0627\u0621 \u0627\u0644\u0627\u0635\u0637\u0646\u0627\u0639\u064a \u0622\u0644\u0627\u0641 \u0648\u062d\u062f\u0627\u062a GPU"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"11253\" class=\"elementor elementor-11253\" data-elementor-post-type=\"post\">\n\t\t\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-9241746 e-con-full e-flex e-con e-parent\" data-id=\"9241746\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-eda93d4 elementor-toc--content-ellipsis elementor-widget__width-initial elementor-hidden-desktop elementor-hidden-tablet elementor-toc--minimized-on-tablet elementor-widget elementor-widget-table-of-contents\" data-id=\"eda93d4\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;no_headings_message&quot;:&quot;No headings were found on this page.&quot;,&quot;container&quot;:&quot;.main-content&quot;,&quot;min_height&quot;:{&quot;unit&quot;:&quot;px&quot;,&quot;size&quot;:0,&quot;sizes&quot;:[]},&quot;headings_by_tags&quot;:[&quot;h2&quot;,&quot;h3&quot;,&quot;h4&quot;,&quot;h5&quot;,&quot;h6&quot;],&quot;marker_view&quot;:&quot;numbers&quot;,&quot;minimize_box&quot;:&quot;yes&quot;,&quot;minimized_on&quot;:&quot;tablet&quot;,&quot;hierarchical_view&quot;:&quot;yes&quot;,&quot;min_height_tablet&quot;:{&quot;unit&quot;:&quot;px&quot;,&quot;size&quot;:&quot;&quot;,&quot;sizes&quot;:[]},&quot;min_height_mobile&quot;:{&quot;unit&quot;:&quot;px&quot;,&quot;size&quot;:&quot;&quot;,&quot;sizes&quot;:[]}}\" data-widget_type=\"table-of-contents.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<div class=\"elementor-toc__header\">\n\t\t\t\t\t\t<h4 class=\"elementor-toc__header-title\">\n\t\t\t\tTABLE OF CONTENTS\t\t\t<\/h4>\n\t\t\t\t\t\t\t\t\t\t<div class=\"elementor-toc__toggle-button elementor-toc__toggle-button--expand\" role=\"button\" tabindex=\"0\" aria-controls=\"elementor-toc__eda93d4\" aria-expanded=\"true\" aria-label=\"Open table of contents\"><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-chevron-down\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M207.029 381.476L12.686 187.132c-9.373-9.373-9.373-24.569 0-33.941l22.667-22.667c9.357-9.357 24.522-9.375 33.901-.04L224 284.505l154.745-154.021c9.379-9.335 24.544-9.317 33.901.04l22.667 22.667c9.373 9.373 9.373 24.569 0 33.941L240.971 381.476c-9.373 9.372-24.569 9.372-33.942 0z\"><\/path><\/svg><\/div>\n\t\t\t\t<div class=\"elementor-toc__toggle-button elementor-toc__toggle-button--collapse\" role=\"button\" tabindex=\"0\" aria-controls=\"elementor-toc__eda93d4\" aria-expanded=\"true\" aria-label=\"Close table of contents\"><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-chevron-up\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M240.971 130.524l194.343 194.343c9.373 9.373 9.373 24.569 0 33.941l-22.667 22.667c-9.357 9.357-24.522 9.375-33.901.04L224 227.495 69.255 381.516c-9.379 9.335-24.544 9.317-33.901-.04l-22.667-22.667c-9.373-9.373-9.373-24.569 0-33.941L207.03 130.525c9.372-9.373 24.568-9.373 33.941-.001z\"><\/path><\/svg><\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<div id=\"elementor-toc__eda93d4\" class=\"elementor-toc__body\">\n\t\t\t<div class=\"elementor-toc__spinner-container\">\n\t\t\t\t<svg class=\"elementor-toc__spinner eicon-animation-spin e-font-icon-svg e-eicon-loading\" aria-hidden=\"true\" viewBox=\"0 0 1000 1000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M500 975V858C696 858 858 696 858 500S696 142 500 142 142 304 142 500H25C25 237 238 25 500 25S975 237 975 500 763 975 500 975Z\"><\/path><\/svg>\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-289e275 elementor-widget elementor-widget-text-editor\" data-id=\"289e275\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>GPU networking is the set of links, switches and protocols that let GPUs exchange data with each other, both inside a server and across a cluster. In AI training, this network matters as much as the GPUs themselves. GPUs must synchronize their results many times during every training step, so one slow or failing link can hold up the whole job.<\/p><p>We will walk through GPU networking from the architecture down to the physical layer: how GPUs talk inside a server, how clusters are wired together, and which cables and fiber connections carry the traffic.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-327b0cb elementor-widget elementor-widget-image\" data-id=\"327b0cb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-cluster-fiber-cabling-racks-1024x768.webp\" class=\"attachment-large size-large wp-image-11257\" alt=\"GPU networking illustration: a GPU server rack linked to a switch rack by MPO trunk cables on an overhead cable tray\" srcset=\"https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-cluster-fiber-cabling-racks-1024x768.webp 1024w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-cluster-fiber-cabling-racks-300x225.webp 300w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-cluster-fiber-cabling-racks-768x576.webp 768w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-cluster-fiber-cabling-racks-16x12.webp 16w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-cluster-fiber-cabling-racks-600x450.webp 600w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-cluster-fiber-cabling-racks.webp 1200w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-77e2f82 elementor-widget elementor-widget-heading\" data-id=\"77e2f82\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Why GPU Clusters Need a Different Network<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c057bb9 elementor-widget elementor-widget-text-editor\" data-id=\"c057bb9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>A traditional data center network moves many independent flows: web requests, database queries, file transfers. An AI training cluster behaves differently.<\/p><p>When a model is trained across many GPUs, each GPU computes part of the work and then shares results with the others through <strong>collective operations<\/strong>. NVIDIA&#8217;s NCCL library, for example, provides AllReduce, AllGather, ReduceScatter and AlltoAll. In an AllReduce, data from every GPU is combined (for example summed), and the result is delivered back to all of them. In Meta&#8217;s words, this synchronization &#8220;requires the transfer of large amounts of data among the GPUs within milliseconds, repeated over multiple iterations until the model converges.&#8221;<\/p><p>Two consequences follow:<\/p><ul><li><strong>The slowest link sets the pace.<\/strong> A collective finishes only when every participant has finished. One congested path or a link that keeps retraining slows every GPU in the job.<\/li><li><strong>Predictable latency matters as much as raw bandwidth.<\/strong> A network with high average throughput but unpredictable delays still wastes GPU time.<\/li><\/ul><p>Failures matter too. During a 54-day period of Llama 3 405B pre-training, Meta recorded 466 job interruptions. Of these, 47 were planned and 419 were unexpected. &#8220;Network Switch\/Cable&#8221; accounted for 35 of the unexpected interruptions (8.4%), and NICs for another 7 (1.7%). Faulty GPUs and GPU memory were larger causes, and the team still achieved more than 90% effective training time. We are not saying networks are the main problem. We are saying that at this scale, cables and switches fail often enough that you have to plan for it.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-15b6d62 elementor-widget elementor-widget-heading\" data-id=\"15b6d62\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Scale-Up vs. Scale-Out<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7bb4d91 elementor-widget elementor-widget-text-editor\" data-id=\"7bb4d91\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>GPU networking has two layers that are often confused.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-68339dc elementor-widget elementor-widget-text-editor\" data-id=\"68339dc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<table><thead><tr><th>\u00a0<\/th><th>Scale-up<\/th><th>Scale-out<\/th><\/tr><\/thead><tbody><tr><td>Scope<\/td><td>GPUs inside a server or rack-scale domain<\/td><td>GPUs in different servers and racks<\/td><\/tr><tr><td>Typical technology<\/td><td>NVLink and NVSwitch<\/td><td>InfiniBand or Ethernet (RoCE)<\/td><\/tr><tr><td>Bandwidth example<\/td><td>900 GB\/s per GPU (NVLink, Hopper); 1,800 GB\/s (Blackwell)<\/td><td>Up to 400 Gb\/s per NIC (ConnectX-7); 800 Gb\/s (ConnectX-8)<\/td><\/tr><tr><td>Physical media<\/td><td>Board traces, NVLink cables and backplanes<\/td><td>Copper cables, active cables, optical transceivers and fiber<\/td><\/tr><\/tbody><\/table>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ccd833a elementor-widget elementor-widget-text-editor\" data-id=\"ccd833a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Take care with the units: NVLink figures are in gigabytes per second, while network ports are rated in gigabits per second. The two layers do different jobs and don&#8217;t compete. Scale-up links the GPUs of one node or rack tightly. Scale-out lets thousands of those nodes work as one cluster.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-959fd64 elementor-widget elementor-widget-image\" data-id=\"959fd64\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1024\" height=\"507\" src=\"https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-scale-up-vs-scale-out-1024x507.webp\" class=\"attachment-large size-large wp-image-11258\" alt=\"Diagram of scale-up and scale-out in a GPU cluster: eight GPUs joined by NVSwitch inside one server, and servers joined to leaf and spine switches over fiber\" srcset=\"https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-scale-up-vs-scale-out-1024x507.webp 1024w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-scale-up-vs-scale-out-300x149.webp 300w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-scale-up-vs-scale-out-768x380.webp 768w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-scale-up-vs-scale-out-18x9.webp 18w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-scale-up-vs-scale-out-600x297.webp 600w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-scale-up-vs-scale-out.webp 1200w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-665b7bc elementor-widget elementor-widget-heading\" data-id=\"665b7bc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">The Four Networks Inside an AI Data Center<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-19df466 elementor-widget elementor-widget-text-editor\" data-id=\"19df466\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>An AI cluster usually runs several separate networks. NVIDIA&#8217;s DGX SuperPOD reference architecture for H100 systems defines four:<\/p><ol><li><strong>Compute (back-end) fabric.<\/strong> This carries GPU-to-GPU traffic between nodes. Each DGX H100 system has eight ConnectX-7 adapters for this network, at up to 400 Gb\/s each, exposed through four OSFP ports. That works out to one network adapter per GPU. Meta&#8217;s Grand Teton training servers follow the same 1:1 mapping between GPUs and NICs.<\/li><li><strong>Storage fabric.<\/strong> This feeds training data and saves checkpoints. The SuperPOD design states that I\/O per node &#8220;must exceed 40 GBps.&#8221;<\/li><li><strong>In-band management.<\/strong> This connects the services that run the cluster, such as job scheduling, code repositories and data sources. In DGX H100, storage and in-band traffic use two dual-port ConnectX-7 cards.<\/li><li><strong>Out-of-band management.<\/strong> This reaches each server&#8217;s BMC for power, monitoring and recovery. On DGX H100 it is a 1 GbE RJ45 port, kept separate from the other networks.<\/li><\/ol><p>Keeping these networks apart isolates faults and keeps training traffic predictable. For cabling, it means each GPU server may need eight or more high-speed optical links for compute alone, plus separate storage and management connections.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a8c9f28 elementor-widget elementor-widget-heading\" data-id=\"a8c9f28\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Back-End Fabric Design: Topology, Rails and Lossless Transport<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dd436d5 elementor-widget elementor-widget-heading\" data-id=\"dd436d5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Leaf-spine and rail-optimized design<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-aa41d42 elementor-widget elementor-widget-text-editor\" data-id=\"aa41d42\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Most GPU back-end fabrics use a leaf-spine (Clos) topology. GPU clusters add a refinement called <strong>rail optimization<\/strong>, or rail alignment. In an eight-GPU server, GPU 1 connects to one leaf switch, GPU 2 to another, and so on. The set of all &#8220;GPU 1&#8221; ports across a group of servers forms a rail.<\/p><p>In the DGX H100 SuperPOD, each scalable unit (SU) of 32 nodes is rail-aligned. As NVIDIA puts it, traffic on a rail &#8220;is always one hop away from the other 31 nodes in a SU,&#8221; while traffic between rails or SUs &#8220;traverses the spine layer.&#8221; The full reference design connects 127 nodes. Rail alignment keeps most collective traffic on short, predictable paths. It works best when the job scheduler is aware of the topology.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-07c9375 elementor-widget elementor-widget-heading\" data-id=\"07c9375\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">InfiniBand vs. RoCE<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8e27b91 elementor-widget elementor-widget-text-editor\" data-id=\"8e27b91\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The scale-out fabric typically runs one of two RDMA transports:<\/p><ul><li><strong>InfiniBand<\/strong> is purpose-built for high-performance computing. It has built-in congestion control and adaptive routing, and it has long been the default in NVIDIA reference designs.<\/li><li><strong>RoCE (RDMA over Converged Ethernet)<\/strong> runs RDMA on Ethernet switches. Meta runs RoCE at large scale. It has shared designs for clusters of up to 32,000 GPUs, and a large Llama 3 variant was trained on 16,000 GPUs on a 24,000-GPU RoCE cluster.<\/li><\/ul><p>RoCE needs careful tuning. Meta reported using DSCP-based priority flow control (PFC) with a single lossless queue. It first tried DCQCN congestion control, then moved congestion management into the collective library itself. &#8220;Lossless&#8221; in this context is a design goal that depends on end-to-end configuration, not a guarantee that packets never drop.<\/p><p>Ethernet for AI is still developing. The Ultra Ethernet Consortium released its UEC Specification 1.0 on June 11, 2025, covering the stack from NICs and switches to optics and cables. It is a new specification, not yet a replacement for either option above.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c1f194a elementor-widget elementor-widget-heading\" data-id=\"c1f194a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">From Fabric to Physical Links: Cables, Optics and Connectors<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bd13814 elementor-widget elementor-widget-text-editor\" data-id=\"bd13814\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>This is where the architecture diagram meets the cable tray, and where we see many AI builds run into trouble.<\/p><p><strong>Link types.<\/strong> GPU clusters mix several media. Passive copper (DAC) and active copper (ACC or AEC) cables serve very short runs inside a rack or to an adjacent rack. Active optical cables (AOC) cover longer fixed runs. For links between rows and rooms, and for any link you want to re-patch later, clusters use pluggable optical transceivers with structured fiber cabling. Reach limits depend on the exact product, so check each vendor&#8217;s specification rather than assuming a rule of thumb.<\/p><p><strong>Let us make that concrete.<\/strong> On NVIDIA&#8217;s air-cooled 400G InfiniBand and Ethernet switches, each OSFP cage holds two 400 Gb\/s ports. The matching 800G twin-port transceiver contains two DR4 engines and has two MPO-12\/APC single-mode connectors, one per 400G port, each using eight fibers (four transmit, four receive). NVIDIA gives a maximum reach of 500 m and notes that its 100 m cable length &#8220;assumes two optical patch panels in the link.&#8221; On the server side, each 400G ConnectX-7 port takes a matching DR4 optic. The result is a large number of MPO-12\/APC single-mode links. For how DR4 and other 400G optics map to fiber, see our <a href=\"\/400g-network-guide\/\">400G network guide<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-33027b5 elementor-widget elementor-widget-image\" data-id=\"33027b5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1024\" height=\"426\" src=\"https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-400g-link-mpo-patch-panels-1024x426.webp\" class=\"attachment-large size-large wp-image-11259\" alt=\"\" srcset=\"https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-400g-link-mpo-patch-panels-1024x426.webp 1024w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-400g-link-mpo-patch-panels-300x125.webp 300w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-400g-link-mpo-patch-panels-768x319.webp 768w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-400g-link-mpo-patch-panels-18x7.webp 18w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-400g-link-mpo-patch-panels-600x250.webp 600w, https:\/\/yfconnectivity.com\/wp-content\/uploads\/2026\/09\/gpu-400g-link-mpo-patch-panels.webp 1200w\" sizes=\"(max-width: 1024px) 100vw, 1024px\">\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d3b3e13 elementor-widget elementor-widget-text-editor\" data-id=\"d3b3e13\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><strong>Scale is the real challenge.<\/strong> Multiply eight compute links per server by hundreds of servers, add spine uplinks, storage and management, and the fiber count rises quickly. There is no reliable industry figure for &#8220;fiber per GPU,&#8221; because it depends on topology. Count it from your design: nodes \u00d7 ports per node \u00d7 fibers per port, plus trunks, patch panels and spares. The practical problems are density, cable routing, labeling, and keeping thousands of MPO end faces clean.<\/p><p>This is where cabling quality becomes cluster performance. Low-loss MPO\/MTP cords leave room in the loss budget for patch panels. Consistent polarity and correct pinning prevent links that simply stay dark. Clean, inspected connectors cut down on the link errors and flaps that slow collectives.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-489b29c elementor-widget elementor-widget-heading\" data-id=\"489b29c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Common GPU Network Problems and How to Catch Them Early<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-62a9d5b elementor-widget elementor-widget-text-editor\" data-id=\"62a9d5b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Here are the issues we see most often, working from the physical layer up:<\/p><ol><li><strong>Links that flap or show rising error counters.<\/strong> Start at the physical layer. Inspect and clean both end faces, reseat the optic, and check the fiber path and polarity. Don&#8217;t clear the counters until you have recorded them. Rising FEC-corrected errors do not always mean a dirty connector, but it is the cheapest cause to rule out.<\/li><li><strong>Some lanes or ports stay down.<\/strong> Check port mode and breakout configuration, pinned\/unpinned mating, polarity method, and APC\/UPC mismatch.<\/li><li><strong>Throughput lower than expected with no obvious fault.<\/strong> Look at traffic balance across rails and spine links, congestion settings and job placement. Engineers on forums report training runs losing throughput with no errors on the ML side. Treat these as anecdotes rather than statistics, but they show that the network is worth checking first.<\/li><li><strong>Problems inside the server.<\/strong> AMD&#8217;s cluster troubleshooting guide advises checking &#8220;PCIe link status on all the devices in the CPU-to-NIC packet datapath.&#8221; A degraded PCIe link can look exactly like a network problem.<\/li><li><strong>Hard-to-reproduce hangs.<\/strong> Collective-level tracing helps. Meta&#8217;s Llama 3 team used NCCL flight recording to capture collective metadata and timing when jobs stalled.<\/li><\/ol><p><strong>Before go-live,<\/strong> test insertion loss on every link, verify polarity, label both ends of every cord, and record baseline error counters. Future troubleshooting will depend on that baseline.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8b61b2c elementor-widget elementor-widget-heading\" data-id=\"8b61b2c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Passive Connectivity for GPU Clusters from YingFeng<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d25f8d4 elementor-widget elementor-widget-text-editor\" data-id=\"d25f8d4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>AI clusters stand or fall on thousands of small, repetitive connections. YingFeng has 25+ years of fiber optic manufacturing experience and makes the passive parts of those links:<\/p><ul><li><strong><a href=\"\/mpo-patch-cables\/\">MPO\/MTP patch cords and trunks<\/a><\/strong> in 8-, 12-, 16- and 24-fiber counts, including low-loss versions, produced in a cleanroom.<\/li><li><strong><a href=\"\/fiber-optic-patch-cords\/\">LC and other duplex patch cords<\/a><\/strong> for storage, management and FR4-type links.<\/li><li><strong><a href=\"\/fiber-optic-connectors\/\">Connector kits and components<\/a><\/strong> for assemblers building their own AI cabling, with custom tooling available.<\/li><\/ul><p>Share your cluster size and transceiver list, and we will help you build a matching patch cord BOM. OEM\/ODM service is available and samples are provided on request. See also our <a href=\"\/data-center-fiber-optics\/\">data center fiber optics solutions<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2dfbcb7 elementor-widget elementor-widget-heading\" data-id=\"2dfbcb7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"> FAQ<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7b20380 elementor-widget elementor-widget-n-accordion\" data-id=\"7b20380\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;default_state&quot;:&quot;expanded&quot;,&quot;max_items_expended&quot;:&quot;one&quot;,&quot;n_accordion_animation_duration&quot;:{&quot;unit&quot;:&quot;ms&quot;,&quot;size&quot;:400,&quot;sizes&quot;:[]}}\" data-widget_type=\"nested-accordion.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"e-n-accordion\" aria-label=\"Accordion. Open links with Enter or Space, close with Escape, and navigate with Arrow Keys\">\n\t\t\t\t\t\t<details id=\"e-n-accordion-item-1290\" class=\"e-n-accordion-item\" open>\n\t\t\t\t<summary class=\"e-n-accordion-item-title\" data-accordion-index=\"1\" tabindex=\"0\" aria-expanded=\"true\" aria-controls=\"e-n-accordion-item-1290\" >\n\t\t\t\t\t<span class='e-n-accordion-item-title-header'><div class=\"e-n-accordion-item-title-text\"> What is GPU networking? <\/div><\/span>\n\t\t\t\t\t\t\t<span class='e-n-accordion-item-title-icon'>\n\t\t\t<span class='e-opened' ><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t\t<span class='e-closed'><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t<\/span>\n\n\t\t\t\t\t\t<\/summary>\n\t\t\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" role=\"region\" aria-labelledby=\"e-n-accordion-item-1290\" class=\"elementor-element elementor-element-f72436e e-con-full e-flex e-con e-child\" data-id=\"f72436e\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-82b8dfd elementor-widget elementor-widget-text-editor\" data-id=\"82b8dfd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>GPU networking is the interconnect that lets GPUs exchange data, both inside a server (NVLink\/NVSwitch) and between servers (InfiniBand or Ethernet\/RoCE). In AI training it carries the collective operations that keep all GPUs synchronized.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/details>\n\t\t\t\t\t\t<details id=\"e-n-accordion-item-1291\" class=\"e-n-accordion-item\" >\n\t\t\t\t<summary class=\"e-n-accordion-item-title\" data-accordion-index=\"2\" tabindex=\"-1\" aria-expanded=\"false\" aria-controls=\"e-n-accordion-item-1291\" >\n\t\t\t\t\t<span class='e-n-accordion-item-title-header'><div class=\"e-n-accordion-item-title-text\"> What is the difference between scale-up and scale-out? <\/div><\/span>\n\t\t\t\t\t\t\t<span class='e-n-accordion-item-title-icon'>\n\t\t\t<span class='e-opened' ><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t\t<span class='e-closed'><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t<\/span>\n\n\t\t\t\t\t\t<\/summary>\n\t\t\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" role=\"region\" aria-labelledby=\"e-n-accordion-item-1291\" class=\"elementor-element elementor-element-18fd5bf e-flex e-con-boxed e-con e-child\" data-id=\"18fd5bf\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-800e3f6 elementor-widget elementor-widget-text-editor\" data-id=\"800e3f6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Scale-up connects GPUs within a server or rack-scale domain with very high bandwidth, such as 900 GB\/s per GPU with Hopper NVLink. Scale-out connects many servers over a network fabric, typically at 400\u2013800 Gb\/s per network adapter.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/details>\n\t\t\t\t\t\t<details id=\"e-n-accordion-item-1292\" class=\"e-n-accordion-item\" >\n\t\t\t\t<summary class=\"e-n-accordion-item-title\" data-accordion-index=\"3\" tabindex=\"-1\" aria-expanded=\"false\" aria-controls=\"e-n-accordion-item-1292\" >\n\t\t\t\t\t<span class='e-n-accordion-item-title-header'><div class=\"e-n-accordion-item-title-text\"> Is InfiniBand or Ethernet better for GPU clusters? <\/div><\/span>\n\t\t\t\t\t\t\t<span class='e-n-accordion-item-title-icon'>\n\t\t\t<span class='e-opened' ><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t\t<span class='e-closed'><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t<\/span>\n\n\t\t\t\t\t\t<\/summary>\n\t\t\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" role=\"region\" aria-labelledby=\"e-n-accordion-item-1292\" class=\"elementor-element elementor-element-687fda6 e-con-full e-flex e-con e-child\" data-id=\"687fda6\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6074ae8 elementor-widget elementor-widget-text-editor\" data-id=\"6074ae8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Both are used at large scale. InfiniBand offers mature built-in congestion control. RoCE on Ethernet has been proven at large scale by operators such as Meta but needs careful tuning. The right choice depends on scale, existing skills and supplier strategy.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/details>\n\t\t\t\t\t\t<details id=\"e-n-accordion-item-1293\" class=\"e-n-accordion-item\" >\n\t\t\t\t<summary class=\"e-n-accordion-item-title\" data-accordion-index=\"4\" tabindex=\"-1\" aria-expanded=\"false\" aria-controls=\"e-n-accordion-item-1293\" >\n\t\t\t\t\t<span class='e-n-accordion-item-title-header'><div class=\"e-n-accordion-item-title-text\"> What is a rail-optimized network? <\/div><\/span>\n\t\t\t\t\t\t\t<span class='e-n-accordion-item-title-icon'>\n\t\t\t<span class='e-opened' ><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t\t<span class='e-closed'><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t<\/span>\n\n\t\t\t\t\t\t<\/summary>\n\t\t\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" role=\"region\" aria-labelledby=\"e-n-accordion-item-1293\" class=\"elementor-element elementor-element-0238114 e-flex e-con-boxed e-con e-child\" data-id=\"0238114\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-57eaad5 elementor-widget elementor-widget-text-editor\" data-id=\"57eaad5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>In a rail-optimized network, the same-numbered GPU in each server connects to the same leaf switch. In NVIDIA&#8217;s DGX H100 SuperPOD, this keeps traffic within a 32-node unit one hop away on each rail.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/details>\n\t\t\t\t\t\t<details id=\"e-n-accordion-item-1294\" class=\"e-n-accordion-item\" >\n\t\t\t\t<summary class=\"e-n-accordion-item-title\" data-accordion-index=\"5\" tabindex=\"-1\" aria-expanded=\"false\" aria-controls=\"e-n-accordion-item-1294\" >\n\t\t\t\t\t<span class='e-n-accordion-item-title-header'><div class=\"e-n-accordion-item-title-text\"> What cables connect GPU servers to switches? <\/div><\/span>\n\t\t\t\t\t\t\t<span class='e-n-accordion-item-title-icon'>\n\t\t\t<span class='e-opened' ><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t\t<span class='e-closed'><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span>\n\t\t<\/span>\n\n\t\t\t\t\t\t<\/summary>\n\t\t\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" role=\"region\" aria-labelledby=\"e-n-accordion-item-1294\" class=\"elementor-element elementor-element-c10dfd5 e-flex e-con-boxed e-con e-child\" data-id=\"c10dfd5\" data-element_type=\"container\" data-e-type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-855ab0c elementor-widget elementor-widget-text-editor\" data-id=\"855ab0c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>It depends on distance. Copper DAC or active copper cables serve very short in-rack runs. For longer links, clusters use optical transceivers, often 400G DR4 with MPO-12\/APC single-mode patch cords, or active optical cables.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/details>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1134529 elementor-widget elementor-widget-heading\" data-id=\"1134529\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">External references<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-42ed4ab elementor-widget elementor-widget-text-editor\" data-id=\"42ed4ab\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul><li>NVIDIA DGX SuperPOD H100 reference architecture, network fabrics: <a href=\"https:\/\/docs.nvidia.com\/dgx-superpod\/reference-architecture-scalable-infrastructure-h100\/latest\/network-fabrics.html\" target=\"_blank\" rel=\"noopener\">https:\/\/docs.nvidia.com\/dgx-superpod\/reference-architecture-scalable-infrastructure-h100\/latest\/network-fabrics.html<\/a><\/li><li>NVIDIA NCCL collective operations: <a href=\"https:\/\/docs.nvidia.com\/deeplearning\/nccl\/user-guide\/docs\/usage\/collectives.html\" target=\"_blank\" rel=\"noopener\">https:\/\/docs.nvidia.com\/deeplearning\/nccl\/user-guide\/docs\/usage\/collectives.html<\/a><\/li><li>Meta, &#8220;RDMA over Ethernet for Distributed AI Training at Meta Scale&#8221; (SIGCOMM 2024)<\/li><li>&#8220;The Llama 3 Herd of Models,&#8221; arXiv:2407.21783<\/li><li>Ultra Ethernet Consortium, Specification 1.0 announcement (June 11, 2025)<\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>\u0643\u064a\u0641\u064a\u0629 \u062a\u0648\u0635\u064a\u0644 \u0645\u062c\u0645\u0648\u0639\u0627\u062a \u0648\u062d\u062f\u0627\u062a \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (GPU)\u060c \u0628\u062f\u0621\u064b\u0627 \u0645\u0646 \u062a\u0642\u0646\u064a\u0629 NVLink \u062f\u0627\u062e\u0644 \u0627\u0644\u062e\u0627\u062f\u0645 \u0648\u0635\u0648\u0644\u0627\u064b \u0625\u0644\u0649 \u0634\u0628\u0643\u0627\u062a InfiniBand \u0623\u0648 RoCE \u0639\u0628\u0631 \u0642\u0627\u0639\u0629 \u0627\u0644\u0628\u064a\u0627\u0646\u0627\u062a\u060c \u0648\u0648\u0635\u0644\u0627\u062a \u0627\u0644\u0623\u0644\u064a\u0627\u0641 \u0627\u0644\u0636\u0648\u0626\u064a\u0629 MPO\u060c \u0648\u0646\u0642\u0627\u0637 \u0627\u0644\u0641\u0634\u0644\u060c \u0648\u0627\u0644\u0639\u0645\u0644\u064a\u0627\u062a \u0627\u0644\u0641\u062d\u0635 \u0627\u0644\u062a\u064a \u062a\u0636\u0645\u0646 \u0627\u0633\u062a\u0645\u0631\u0627\u0631 \u0639\u0645\u0644 \u0622\u0644\u0627\u0641 \u0648\u062d\u062f\u0627\u062a \u0645\u0639\u0627\u0644\u062c\u0629 \u0627\u0644\u0631\u0633\u0648\u0645\u0627\u062a (GPU).<\/p>","protected":false},"author":1,"featured_media":11257,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_gspb_post_css":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[26],"tags":[],"class_list":["post-11253","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-professional-insights"],"acf":[],"_links":{"self":[{"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/posts\/11253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/comments?post=11253"}],"version-history":[{"count":10,"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/posts\/11253\/revisions"}],"predecessor-version":[{"id":11266,"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/posts\/11253\/revisions\/11266"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/media\/11257"}],"wp:attachment":[{"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/media?parent=11253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/categories?post=11253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/yfconnectivity.com\/ar\/wp-json\/wp\/v2\/tags?post=11253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}