Data Center Lab · Reference Designs · Network
Sheet N-111-EN: Two BlueField-3 ports, one to each side's leaf; unused CVS ports reach the border panel
Each tray BlueField-3 puts one 400G port on an A-side NS leaf and one on a B-side NS leaf, each side having 30 leaves (LG01 through LG30) at 54 / 28 down/up (1.929:1) and 7 CVS, and unused CVS ports reach EXT-IF-NS-A and EXT-IF-NS-B via the border panel, beyond which note 1 assumes no upstream equipment.
Design Development — Public Edition P1: a reference design for study and evaluation, not a construction issue, not sealed by an Engineer of Record.
Scroll or pinch to zoom, drag to pan, click a highlighted device for its explanation. Vector drawing — it stays sharp at any magnification.
← N-101-EN · All network sheets · N-121-EN →
What this sheet shows
N-111-EN draws the north–south (DPU) path as two mirrored sides. The line under the top border states that each tray BlueField-3 (B3240) has two 400G ports, one to the A-side north–south leaf and one to the B-side north–south leaf, and that each NS leaf is 54 / 28 down/up (1.929:1) to 7 CVS per side. The same line says unused CVS ports reach the external interface via a border patch panel. On the A side the leaves sit in NETA-CUxx, U2–U5. The drawing shows CS-A-1 through CS-A-7, each marked SN5610 in CVS-A, then a row LG01 through LG30, then a bar reading: GPU cabinets, each NS leaf serves 3 cabinets × 18 trays = 54 BF3 ports. The caption under that bar gives each NS leaf 54 BF3 downlinks and uplinks to each of 7 CVS at 4 × 400G, total 28. The B side repeats the chain for NETB-CUxx, U2–U5: CS-B-1 through CS-B-7, each marked SN5610 in CVS-B, its own LG01 through LG30, and the same bar and caption. The A side ends at EXT-IF-NS-A and the B side at EXT-IF-NS-B, each labeled an external north–south interface. The two sides are drawn as separate CVS, separate leaf rows, separate border panels and separate interfaces. The BF3 ports note is the sentence that puts one tray on both sides: one B3240 per tray, two ports to different sides. The Per side column gives 1,620 × 400G, 30 leaves at 54 / 28 (1.929:1), 7 CVS and 336 Tb/s. The leaf note gives 46 unused (not reserved) ports per NS leaf, the CVS note gives 56 unused per side and 112 in total, and the callout gives 8 unused (not reserved) ports per CVS, as 30 leaves each sending 4 × 400G to one CVS make the 120 used ports per CVS.
How the counts meet
The per-side table and the drawing state the same port counts. BF3 ports are 1,620 × 400G per side, and the note says one B3240 per tray, with the two ports going to different sides, so 1,620 is this side's ports. The row is labeled LG01 through LG30, and the NS leaves (SN5610) row counts 30 at down/up 54 / 28 (1.929:1), with 46 unused (not reserved) ports per NS leaf. The bar under the leaves defines the 54: each NS leaf serves 3 cabinets × 18 trays = 54 BF3 ports. The caption defines the 28: uplinks to each of 7 CVS at 4 × 400G, total 28. Note 3 says those per-leaf uplinks come from the reference port table. Thirty leaves at 54 BF3 ports are the table's 1,620, and thirty leaves each sending 4 × 400G to one CVS are the table's 120 ports used. The CVS row lists 7, with 120 ports used and 8 unused (not reserved). The callout states the 8 as ports per CVS, and writes 8 ports per CVS × 7 = 56 × 400G per side, via the border MPO panel. The table gives 56 unused per side and 112 in total. NS leaf → CVS capacity is 336 Tb/s per side, basis 30 × 28 × 400G. The ratio 1.929:1 is printed under the border and again in this table, and the 46 unused ports in that same leaf row are marked not reserved.
What is left open at the border
The callout under each external interface states the unused CVS pool size, and the number of unused CVS ports to be assigned is a configuration item. Under EXT-IF-NS-A the callout reads: unused (not reserved) 8 ports per CVS × 7 = 56 × 400G per side, via border MPO panel BP-A (4U, in CVS-A). Under EXT-IF-NS-B the panel is BP-B (4U, in CVS-B). Note 2 states the limit on that panel, in the sheet's words: border-panel port positions are passive connection points, not additional switch-port capacity. The same note names the configuration items of the deploying organisation: required external bandwidth, the number of unused CVS ports to be assigned, and module type. Inside that item it names the sharing. Border use and the CVS-connected storage options SA-1/SA-2 share the 56 unused CVS ports per side. The distributed option SA-3 would use unused NS leaf ports instead. The note refers that choice to supplement Section IS2. The table's figure for those leaf ports is 46 unused (not reserved) per NS leaf. Note 1 sets the boundary past the dashed box: upstream routers, carriers, interface rates and optical modules beyond EXT-IF-NS-A/B are defined by the deploying organisation, and no upstream equipment is assumed in this design. Note 3 adds that the uplinks per NS leaf to each CVS come from the reference port table.
Status
The title block issues the drawing as Public Edition P2, number N-111-EN, sheet 7 of 30, dated 2026-10-05, scale NTS, units mm. The sheet title is North–south (DPU) topology to external interface. The status line reads: reference design for study and evaluation — not for construction. Notes 1 and 2 are the open items, and they are written on the sheet. Beyond EXT-IF-NS-A/B, upstream routers, carriers, interface rates and optical modules are defined by the deploying organisation, and no upstream equipment is assumed. Required external bandwidth, the number of unused CVS ports to be assigned, and module type are configuration items of that organisation. SA-1/SA-2 share the 56 unused CVS ports per side, and SA-3 would use unused NS leaf ports instead, with the choice referred to supplement Section IS2. The sheet is part of the Public Edition P2 network reference design for the GPU hall reference.
Sheet guide — the devices on this drawing
BlueField-3 (B3240) — one 400G port to each side
The line under the border states that each tray BlueField-3 (B3240) has two 400G ports, one to the A-side and one to the B-side north–south leaf. It states the NS leaf as 54 / 28 down/up (1.929:1) to 7 CVS per side, and it says unused CVS ports reach the external interface via a border patch panel.
CS-A-1 — SN5610 in CVS-A
CS-A-1 is the first of the A-side CVS, CS-A-1 through CS-A-7, and each box is marked SN5610 in CVS-A. The side heading places the NS leaves in NETA-CUxx, U2–U5. Lines from this row run up to EXT-IF-NS-A and down across the leaf row LG01 through LG30.
EXT-IF-NS-A — external north–south interface
The dashed box is labeled EXT-IF-NS-A, external north–south interface. The callout under it reads: unused (not reserved) 8 ports per CVS × 7 = 56 × 400G per side, via border MPO panel BP-A (4U, in CVS-A). Note 2 says those border-panel positions are passive connection points, not additional switch-port capacity.
LG01–LG15 — A-side NS leaves
LG01 through LG15 are the left portion of the A-side row, which the drawing continues through LG30. The per-side table counts 30 NS leaves (SN5610). The bar under the row says each NS leaf serves 3 cabinets × 18 trays = 54 BF3 ports, and the caption adds 54 BF3 downlinks with uplinks to each of 7 CVS at 4 × 400G, total 28.
CS-B-1 — SN5610 in CVS-B
CS-B-1 is the first of the B-side CVS, CS-B-1 through CS-B-7, each marked SN5610 in CVS-B. The side heading places these NS leaves in NETB-CUxx, U2–U5. The B side repeats the A-side chain: the same 30 leaf labels, the same 54 BF3 downlinks, and uplinks of 4 × 400G to each of 7 CVS, total 28.
EXT-IF-NS-B — external north–south interface
The dashed box is labeled EXT-IF-NS-B, external north–south interface. Its callout matches the A side: unused (not reserved) 8 ports per CVS × 7 = 56 × 400G per side, via border MPO panel BP-B (4U, in CVS-B). Note 1 says no upstream equipment is assumed beyond EXT-IF-NS-A/B.
LG01–LG15 — B-side NS leaves
LG01 through LG15 begin the B-side leaf row, drawn under the CS-B boxes and continued through LG30. The bar beneath reads that each NS leaf serves 3 cabinets × 18 trays = 54 BF3 ports. The table's per-side count of 30 NS leaves and 1,620 × 400G BF3 ports applies to this side as it does to the A side.
Per-side table — BF3 ports, leaves, CVS, capacity
Per side, BF3 ports are 1,620 × 400G, noted as one B3240 per tray with two ports to different sides. NS leaves (SN5610) are 30 at down/up 54 / 28 (1.929:1), with 46 unused (not reserved) ports per leaf, and CVS (SN5610) are 7 per side, with 120 ports used and 8 unused (not reserved) per CVS, 56 unused per side and 112 in total. NS leaf to CVS capacity is 336 Tb/s, basis 30 × 28 × 400G, and the CVS note calls border, storage or other use a configuration item.
Notes 1–3 — upstream boundary and unused ports
Note 1 states that upstream routers, carriers, interface rates and optical modules beyond EXT-IF-NS-A/B are defined by the deploying organisation, and that no upstream equipment is assumed in this design. Note 2 makes required external bandwidth, the number of unused CVS ports to be assigned, and module type a configuration item: SA-1/SA-2 share the 56 unused CVS ports per side, SA-3 would use unused NS leaf ports instead, and the note points to supplement Section IS2. Border-panel positions are passive connection points, not additional switch-port capacity, and note 3 takes the uplinks per NS leaf to each CVS from the reference port table.
N-111-EN — title block, sheet 7 of 30
The title block gives the sheet title, North–south (DPU) topology to external interface, at scale NTS, and the number N-111-EN, sheet 7 of 30. The edition cell reads Public Edition P2, dated 2026-10-05, units mm. The status line reads: reference design for study and evaluation — not for construction.
Get the full drawing set
This sheet is one page of the GPU Hall Network Reference Design — Public Edition P2 (75 pages, 1.6 MB) — a direct PDF download with a published SHA-256 checksum, no registration. The complete coordinated set is on the downloads page. Background guides: data center one-line diagrams · GB300 NVL72 facility requirements · liquid cooling and CDU design.
Questions about any device or parameter on this sheet: inquiry@kkdatasvc.com or the contact page.