100G txha nraub qaum- nplooj ntaub yog ib txoj hauv kev zoo tshaj plaws los txuas 25G servers, 100G uplinks, cia pawg, thiab sab hnub tuaj- sab hnub poob- hnyav ua haujlwm hauv lub chaw khaws ntaub ntawv niaj hnub. Qhov kev thov rov hais dua ntawm QSFP28 yog nws qhov yooj yim: ib qho chaw nres nkoj tuaj yeem nqa ib txwm 100G txuas lossis tawg mus rau plaub qhov kev sib txuas ntawm 25G server, yog li ib qho kev hloov pauv tuaj yeem ua haujlwm rau ob qho tib si ntawm ntug kev thiab cov ntaub ntaub.
Cov keyboards ceev yog qhov yooj yim. Ib tug 100G tsim nyob los yog tuag nyob rau hauv cov kev txiav txim siab ua ntej qhov kev txiav txim yuav: yuav ua li cas txhua qhov chaw nres nkoj muab faib, dab tsi lub oversubscription ratio zoo li nyob rau hauv ib txwm thiab tsis ua hauj lwm tej yam kev mob, uas optics phim lub tiag tiag cable khiav, ntau npaum li cas kub cov optics ntxiv, thiab seb cov ntaub yuav loj hlob mus rau 400G yam tsis muaj forklift upgrade.
Cov lus qhia no yog tus neeg muag khoom- siv cov phiaj xwm nruab nrab rau cov pab pawg hauv network thiab cov txheej txheem. Cov duab hauv qab no ua raws li tam sim no IEEE 802.3 Ethernet specifications thiab qhov cuam tshuam ntawm qhov muag pom ntau- qhov kev pom zoo, tab sis txhua qhov hloov pauv thiab transceiver muaj nws tus kheej cov ntaub ntawv, yog li paub meej cov lej tseeb rau cov khoom siv koj yuav.
Yuav ua li cas nyeem cov piv txwv hauv phau ntawv qhia no.Tshwj tsis yog hais tias lwm yam, lawv xav tias ib leeg -homed servers nrog ib 25G NIC txhua, 48 tus tswv chaw nres nkoj ib nplooj, 100G nplooj -rau- qaum uplinks, ib tug tag nrho mesh nyob rau hauv uas txhua nplooj txuas mus rau txhua tus txha nraub qaum, thiab kev kho yuam kev xa mus rau qhov twg optics xav tau. Dual-homing, ceev NICs, lossis cov chaw nres nkoj sib txawv yuav hloov txhua tus lej tom qab.
Dab tsi yog 100G Kab laug sab -Leaf Network?
Tus txha nraub qaum- nplooj yog ob- cov ntaub ntawv chaw nruab nrab ntawm cov ntaub ntawv tsim los ntawm cov nplooj hloov pauv thiab cov nqaj qaum. Nplooj hloov zaum saum toj kawg nkaus ntawm txhua lub khib thiab muab cov neeg rau zaub mov-fab cov chaw nres nkoj ntxiv rau uplinks rau qaum. Cov pob txha caj qaum ua rau lub siab - nrawm nrawm. Txhua nplooj txuas mus rau txhua tus txha nraub qaum, yog li kev tsheb khiav ntawm racks txav nplooj mus rau qaum mus rau nplooj raws qhov sib npaug - txoj kev ntev.
Tus tsim yog nrov vim nws xa:
- Predictable, sib npaug txoj kev ntev ntawm ob racks
- Kev them nyiaj yug haiv neeg rau hnyav sab hnub tuaj - tsheb khiav sab hnub poob
- Tag nrho cov uplinks nquag siv los ntawm ECMP es tsis yog thaiv los ntawm kev ncua ntoo
- Yooj yim kab rov tav scaling - ntxiv nplooj rau cov chaw nres nkoj, ntxiv spines rau lub peev xwm
Hauv 100G ntaub, nplooj -rau- qaum txuas khiav ntawm 100G, thaum lub server-fab cov chaw nres nkoj khiav ntawm 10G, 25G, 50G, lossis 100G nyob ntawm qhov ua haujlwm. Niaj hnub no, 25G nkag nrog 100G uplinks yog qhov kev lag luam feem ntau ua ke.
Physical Design vs Logical Design
"Network tsim" npog ob txheej uas yooj yim rau conflate. Phau ntawv qhia no tsom ntsoov rau lub cev thiab lub peev xwm txheej - chaw nres nkoj, optics, oversubscription, cabling - vim hais tias yog yam koj cog lus rau thaum koj yuav khoom kho vajtse. Tab sis cov txheej txheem txheej txheem txiav txim siab li cas cov ntaub xa mus rau kev khiav tsheb, thiab nws ua rau ntau qhov kev xaiv ntawm lub cev.
Ntawm lub cev sab xis zaum hloov thiab xaiv chaw nres nkoj, NIC ceev, oversubscription, optics, cabling, hwj chim, thiab cua txias. Ntawm qhov laj thawj sab zaum ECMP thauj khoom- sib npaug ntawm cov uplinks; ib qho kev sib tshooj xws li VXLAN nrog BGP EVPN tswj lub dav hlau rau ntau- tus neeg xauj Layer 2 thiab Txheej 3 hla ib txoj kab hauv qab; dual-homing nrog MLAG lossis MC-LAG thiab LACP ntawm ntug kev nkag; thiab tsis ua haujlwm- qhov loj me. Rau RDMA fabrics koj kuj yuav tsum engineers ze rau -tsis muaj network, npog hauv qab no. Teeb tsa cov qauv kev xav thaum ntxov, vim tias nws cuam tshuam rau kev suav uplink, pes tsawg tus spines koj xav tau rau ECMP dav, thiab seb cov nplooj raug xa mus ua MLAG khub.
Kauj Ruam 1 - Txhais Server Ceev thiab Ua Haujlwm
Pib nrog kev ua haujlwm, tsis yog optics. Ib pawg virtualization dav dav, cov ntaub ntim khoom, thiab AI kev cob qhia pod muaj qhov xav tau sib txawv heev, thiab tus qauv tsim raws li kev khiav tsheb.
25G servers nrog 100G uplinks
Rau feem ntau ntawm kev lag luam thiab ntiag tug- huab cua ib puag ncig, 25G nkag nrog 100G nplooj- mus rau- txha nraub qaum yog qhov chaw qab zib: dhia loj tshaj 10G thaum khaws NIC, cable, thiab hloov cov nqi tsim nyog. Ib qho kev tsim ua khub 25G downlinks, 100G uplinks, thiab 2: 1 rau 3: 1 piv rau cov kev suav dav dav, nrog qis dua kev tso npe tshwj tseg rau kev khaws cia thiab latency - qib siab. Nws haum rau virtualization, huab ntiag tug, web tiers, thiab feem ntau ntawm cov chaw lag luam cov ntaub ntawv.
Native 100G rau kev cia, AI, thiab HPC
Qee qhov kev ua haujlwm xav tau ib txwm muaj 100G rau tus neeg rau zaub mov: faib thiab NVMe-o cia, AI thiab tshuab- kev kawm, HPC, loj - ntsuas ntsuas, thiab qis -latency RDMA. Ntawm no oversubscription yuav tsum qis - feem ntau tsis yog - thaiv lossis ze rau nws - vim tias cov qauv tsheb yog qhov teeb meem, tsis yog qhov ntim xwb.
AI, HPC, and RDMA workloads generate dense, synchronized, all-to-all east-west traffic: many nodes transmit to many nodes at the same moment, so the statistical smoothing that saves you on a virtualization fabric no longer applies. RDMA dhau Converged Ethernet (RoCE) ntxiv qhov txwv thib ob, vim tias nws xav tias yuav nyob ze - tsis muaj ntaub, uas nyob rau hauv kev xyaum txhais tau hais tias Qhov Tseem Ceeb Flow Control (PFC) thiab Cov Lus Ceeb Toom Tsis Txaus Ntseeg (ECN) tau kho qhov kawg kom kawg. Ib daim ntaub uas tso cov thav duab hauv qab congestion yuav saib RoCE kev ua haujlwm tawg, yog li cov pawg no feem ntau ua rau ntawm 1: 1 nrog ceev faj tsis zoo thiab congestion configuration.
Kauj ruam 2 - Yuav ua li cas xam cov nplooj thiab txha nraub qaum hloov chaw nres nkoj rau 100G Npuag
Kev npaj chaw nres nkoj pib ntawm nplooj, tsis yog qaum. Ua haujlwm sab nraud los ntawm cov servers:
- Suav cov server-faus ports rau ib lub khib.
- Txiav txim siab seb txhua tus neeg yog haiv neeg 25G, haiv neeg 100G, lossis ib txoj kab tawg.
- Txuag QSFP28 chaw nres nkoj rau qaum uplinks.
- Ntxiv cov chaw nres nkoj rau kev loj hlob, rov ua dua, kuaj, thiab hloov.
- Recalculate oversubscription tom qab breakout yog muab, tsis ua ntej.
Suav server-faus ports
Rau txhua lub khib, pin down server count, NIC speed, NICs per server, single- or dual-homed, and need spares. Ib lub khib ntawm 48 servers nrog ib 25G NIC txhua tus xav tau 48 lub chaw nres nkoj. Dual-tsev cov servers mus rau nplooj nplooj thiab qhov chaw nkag nkag suav hla cov khub ob npaug.
Khaws cov chaw nres nkoj uplink, thiab saib ob - suav
Tom qab cov chaw nres nkoj host, khaws QSFP28 chaw nres nkoj rau qaum. Qhov no yog qhov uas feem ntau yuam kev nkaum: yog tias tib QSFP28 cov chaw nres nkoj siv rau 4x25G breakout, lawv tsis muaj nyob rau hauv uplinks. Qhov kev npaj ua yuam kev loj tshaj plaws yog tsis suav 100G uplinks, tab sis overestimating lub uplink ports tshuav ib zaug breakout tau noj rau hauv lawv. Muab qhov kev txiav tawm ua ntej tus lej oversubscription, lossis qhov piv txwv koj suav yog ntawv tseeb.
Ib qho piv txwv ua haujlwm pab. Siv ib qho 1U nplooj ntoo nrog 48 SFP28 cov chaw nres nkoj thiab 8 QSFP28 chaw nres nkoj:
| Chaw nres nkoj pawg | Lub luag haujlwm | Muaj peev xwm |
|---|---|---|
| 48 x 25G (SFP28) | Ib leeg -homed server nkag | 1,200G |
| 6 x 100G (QSFP28) | Rov qab uplinks | 600G |
| 2 x 100G (QSFP28) | Reserved: loj hlob, cia, los yog seem | - |
Nrog rau 6 lub uplinks nqa 1,200G ntawm kev nkag mus, cov nplooj khiav ntawm 2: 1, thiab ob qhov chaw nres nkoj QSFP28 nyob hauv cia. Muab txhua qhov chaw nres nkoj ib zaug, qhia meej lub luag haujlwm ntawm daim ntawv nthuav dav ua ntej koj loj dua lwm yam.
Cia lub peev xwm seem
Tsis txhob haus txhua qhov chaw nres nkoj rau hnub ib. Khaws lub taub hau rau cov servers tshiab, cov txha nraub qaum ntxiv, txuas mus kuaj ib ntus, ua tsis tiav -port swaps, saib xyuas cov kais dej, thiab kev tsiv teb tsaws chaw. Lub peev xwm me me uas tsis siv tau yog pheej yig dua li kev tsim kho dua tshiab.
Kauj Ruam 3 - Xam cov Oversubscription, suav nrog N-1
Oversubscription piv rau tag nrho cov neeg rau zaub mov -fab bandwidth ntawm ib nplooj nrog nws tag nrho uplink bandwidth rau qaum:
Oversubscription ratio=tag nrho downlink bandwidth / tag nrho uplink bandwidth
Rau cov nplooj saum toj no, 48 x 25G=1,200G nqes thiab 6 x 100G=600G nce, muab 1,200 / 600=2:1. Qhov ntawd txhais tau hais tias ob zaug ntau npaum li qhov theoretical nkag bandwidth li uplink bandwidth - feem ntau zoo rau kev suav dav dav, qhov twg cov servers tsis tshua muaj kev sib kis ntawm kab tus nqi ib zaug, tab sis qhov kev txwv tiag tiag rau kev cia, AI, HPC, thiab RDMA.
Nco ntsoov xyuas cov ntaub ntawv N-1
Ib daim ntaub tuaj yeem saib kev noj qab haus huv hauv kev ua haujlwm ib txwm thiab choke thaum ua tsis tiav. Xav txog ib nplooj ntoos nrog yim 100G uplinks kis tusyees hla plaub qaum - ob leeg ib leeg, 800G tag nrho, yog li 1,200G ntawm kev nkag muab 1.5: 1. Poob ib tus txha nraub qaum thiab nplooj poob ob qhov uplinks rau 600G, thawb qhov piv rau 2: 1 rau lub sijhawm ntawm qhov hluav taws xob. Yog tias koj lub hom phiaj yog "tsis muaj qhov phem tshaj 2: 1 txawm tias tsis ua haujlwm," koj yuav tsum pib ze 1.5: 1. Xam ob qhov piv txwv thiab N-1 piv tom qab poob ib tus txha nraub qaum lossis uplink; tus naj npawb thib ob yog tus uas tom thaum kho.
Kev npaj ntau yam los ntawm kev ua haujlwm
Tsis muaj qhov sib piv thoob ntiaj teb, yog li kho cov hauv qab no raws li kev npaj ntau yam, tsis yog qauv, thiab siv tau rau kev ntsuas tsheb uas koj tuaj yeem ua tau:
| Ua haujlwm | Tsim kev taw qhia |
|---|---|
| AI / HPC / RDMA | 1: 1 los yog ze tsis yog - thaiv |
| Muab faib cia | 1: 1 rau 2: 1 |
| General virtualization | 2:1a 3: 1 |
| Web/application tiers | 3: 1 los yog siab dua yog tias kev khiav tsheb tuaj yeem kwv yees tau |
| Dev/test | Tus nqi -kev ua tau zoo piv tau txais |
Ntawm kev hloov kho tshiab, tshuaj xyuas kev siv uplink tam sim no, ncov thiab sab hnub tuaj -west qauv, cia ntws, thiab cov qhov rais thaub qab ua ntej ua rau qhov piv.
Kauj ruam 4 - Xaiv QSFP28 Optics thiab Cables
QSFP28 100G interfaces yog tus qauv tsim los ntawm IEEE 802.3 - tus802.3bm uantxiv 100GBASE-SR4, nrog rau ib qho -mode LR4 PHY. Xaiv optics los ntawm kev ncua deb, hom fiber ntau, txuas, lub zog, thiab hloov kev sib raug zoo, thiab tiv thaiv kev ua haujlwm ntev tshaj plaws: ncav cuag koj tsis xav tau feem ntau txhais tau tias tus nqi thiab lub zog koj tsis xav tau. Sib piv cov module mus rau qhov kev khiav nrog ib tug sensible margin.
DAC thiab AOC rau luv luv server txuas
Rau hauv - khib thiab nyob ib sab - kev sib txuas khib, QSFP28 ncaj qha - txuas tooj liab (DAC) thiab cov kab hluav taws xob ua haujlwm (AOC) yog qhov ua tau zoo. Passive DAC haum rau luv luv hops - ob peb meters - ntawm tus nqi qis tshaj plaws thiab lub zog, thaum AOC txuas mus txog thiab sib dua thiab hloov tau yooj yim dua qhov twg tooj liab ua teeb meem. Rau 25G nkag mus, QSFP28-rau-4x SFP28 breakout DAC lossis AOC yog qhov tshwm sim thaum lub hloov pauv txhawb kev tawg.
100GBASE-SR4 rau luv luv multimode uplinks
SR4 nqa 100G dhauyim fibers ntawm parallel multimodesiv MPO/MTP connector, uas ua rau nws tus nqi -kev xaiv zoo rau nplooj ntawv luv luv-rau- qaum khiav hauv ib kab. Nws ncav cuag nyob ntawm qib fiber ntau - kwv yees li 70 m ntawm OM3 thiab 100 m ntawm OM4 - yog li nws them nyiaj kom paub qhov ncav cuag koj tuaj yeem xav tau los ntawmOM3, OM4, thiab OM5 multimode fiber ntauhauv koj pem teb. Qhov kev txwv tseem ceeb yog kev sib txuas ntawm cov kab txuas txuas: MPO patching thiab polarity yuav tsum tau ua ua ntej.
CWDM4 lossis FR rau ib zaug - hom khiav mus txog 2 km
Rau inter-kab, inter-chav, lossis inter-chaw txuas, ib leeg- hom optics xws li CWDM4 lossis FR yog qhov zoo dua. Cov100G CWDM4 MSAtxhais tau hais tias 2 km ncav cuag tshaj ib khub ntawm ib zaug - hom fibers nrog duplex LC connector thiab FEC. Vim hais tias lawv siv duplex fiber ntau es tsis txhob sib npaug ntawm MPO, CWDM4 thiab FR optics feem ntau poob rau hauv ib qho - hom cog ntau huv dua SR4 - thiab tshaj qhov kev ncua deb ntawm qhov kev xaiv ntawmOS1 thiab OS2 ib leeg -mode fiberpib teeb meem rau koj cov peev nyiaj poob. luv dua ib zaug - hom variants xws li DR npog kwv yees li 500 m qhov twg yog txhua yam koj xav tau.
100GBASE-LR4 rau tsev kawm ntawv thiab DCI
LR4 yog qhov ntev -kev xaiv ncav cuag, nqa 100Gtxog li 10 km dhau duplex ib zaug - hom fiber ntaurau lub tsev kawm ntawv, tsev - mus rau - tsev, lossis cov ntaub ntawv -center- kev sib txuas sib txuas. Siv nws tsuas yog qhov kev ncua deb tiag tiag hu rau nws; ntev - ncav cuag optics ntawm luv luv intra-cov ntaub ntawv-center hops tsuas yog ntxiv tus nqi, lub zog, thiab cua sov yam tsis tau txhim kho cov ntaub.
QSFP28 100G Optics Sib piv
Lub rooj qhia txog qhov twg txhua qhov kev xaiv haum. Kho qhov ncav cuag raws li cov qauv kev npaj, thiab paub meej tias tus lej, qib fiber ntau, thiab FEC xav tau ntawm txhua daim ntawv teev cov ntaub ntawv.
| Kev xaiv | Media / fiber ntau | Txuas | Hom ncav cuag | Qhov twg nws haum |
|---|---|---|---|---|
| QSFP28 DAC (passive tooj liab) | Twinax tooj | Kev koom ua ke | ~1–3 m | Hauv -rack server lossis nplooj - mus rau - nplooj |
| QSFP28 AOC | Multimode (integrated) | Kev koom ua ke | ~ mus txog 30 m | Nyob ib sab -rack servers, txuas luv luv |
| 100GBASE-SR4 | Parallel multimode, 8 fibers (OM3 / OM4) | MPO/MTP | ~ 70 m OM3 / 100 m OM4 | Luv luv hauv - kab nplooj - mus rau - qaum |
| 100 G CWDM4 | Duplex ib leeg - hom | LC | mus txog ~ 2 km | Inter-kab / inter-Hall uplinks |
| 100GBASE-FR / DR | Duplex ib leeg - hom | LC | ~ 500 m (DR) to ~ 2 km (FR) | Nruab nrab ib zaug - hom khiav |
| 100GBASE-LR4 | Duplex ib leeg - hom | LC | mus txog ~ 10 km | Tsev kawm ntawv / tsev - mus rau -building / DCI |
Cov piv txwv ua haujlwm: Cov ntaub me me, nruab nrab, thiab loj
Cov no yog cov qauv kev npaj ua kom yooj yim, tsis yog cov qauv tsim. Tus txha caj qaum feem ntau yog xaiv los faib cov uplinks tusyees thiab teeb ECMP dav: ob lub nraub qaum yog qhov ua tau yam tsawg kawg nkaus rau kev rov ua dua, plaub muab qhov zoo dua N-1 granularity thiab zoo dua thauj khoom sib kis, thiab yim haum rau cov ntaub loj. Nplooj suav suav nrog cov chaw nres nkoj server koj xav tau.
Cov ntaub me me
- 8 nplooj hloov
- 2 qaum hloov
- 48 x 25G server ports ib nplooj
- 4 x 100G uplinks ib nplooj
- 384 ib leeg -homed 25G server chaw nres nkoj
Ib nplooj: 1,200G down, 400G nce, yog li 3: 1. Ua haujlwm tau rau kev suav dav dav, tab sis nruj rau hnyav cia lossis AI. Ntxiv uplinks los yog luas nkag mus rau ib nplooj yog tias koj xav tau tus nqi qis dua.
Nruab nrab ntaub
- 16 nplooj ntawv hloov
- 4 qaum hloov
- 48 x 25G server ports ib nplooj
- 6 x 100G uplinks ib nplooj
- 768 ib leeg -homed 25G server chaw nres nkoj
Ib nplooj: 1,200G down, 600G nce, yog li 2: 1. Ib qho khoom sib npaug rau virtualization thiab kev ua haujlwm ntawm kev lag luam, thiab plaub tus txha nraub qaum kis ECMP zoo dua li ob.
Cov ntaub loj
- 32 nplooj ntawv hloov
- 8 txhav hloov
- 48 x 25G server ports ib nplooj
- 8 x 100G uplinks ib nplooj
- 1,536 ib leeg -homed 25G server chaw nres nkoj
Ib nplooj: 1,200G down, 800G nce, yog li 1.5: 1. Ntau uplink headroom, tab sis ntau optics, fiber ntau, nqi, hwj chim, thiab cabling los tswj. Ntawm qhov ntsuas no, cov ntaub ntawv yog ib feem ntawm kev tsim: daim ntawv lo, daim ntawv qhia chaw nres nkoj, polarity, spare optics, airflow, thiab saib xyuas txhua yam yuav tsum tau npaj ua ntej kev teeb tsa.
QSFP28 Breakout Planning (100G txog 4x25G)
Breakout yog qhov tseem ceeb tshaj plaws, thiab feem ntau nkag siab yuam kev, ib feem ntawm QSFP28 tsim. Qhov twg qhov hloov pauv, cable, thiab teeb tsa tso cai rau nws, ib qho QSFP28 chaw nres nkoj faib ua plaub 25G SFP28 txuas, txuas plaub 25G servers los ntawm ib qho chaw 100G. Nws khwv tau nws qhov chaw thaum koj xav tau siab 25G ceev, muaj ntau QSFP28 chaw nres nkoj, xav kom tus nqi qis rau ib qho kev sib txuas ntawm cov neeg rau zaub mov, lossis tab tom tsim kev hloov pauv 25G / 100G ntaub, siv QSFP28-to-4x SFP28 DAC, AOC, lossisMTP / MPO breakout cablesnyob ntawm qhov deb.
Qhov kev ntes yog tias breakout siv QSFP28 chaw nres nkoj. Yog tias 32-chaw nres nkoj QSFP28 hloov pauv hloov 16 chaw nres nkoj rau 4x25G breakout, cov 16 chaw nres nkoj txhawb nqa 64 servers - tab sis tsuas yog 16 QSFP28 chaw nres nkoj tseem nyob rau uplinks, cia, sib txuas, thiab khoom seem. Txoj cai ntawm tus ntiv tes xoo yog suav cov chaw nres nkoj tawg ua ntej, tom qab ntawd suav qhov tseem tshuav rau uplinks.
Ua ntej koj cog lus, lees paub ob peb yam, thiab txiav txim siab ntxov seb txhua qhov kev khiav yuav tsum yog ib qholub cev los yog ib qho kev sib sau ua ke:
- Cov chaw nres nkoj twg txhawb kev tawg, thiab puas muaj chaw nres nkoj- txwv kev pab pawg?
- Puas yog ua kom breakout lov tes taw cov chaw nres nkoj uas nyob ib sab?
- Puas yog qhov hloov pauv kev ua haujlwm txhawb nqa hom koj xav tau?
- DAC, AOC, lossis breakout optics rau txhua qhov khiav?
- Puas yog tag nrho plaub txoj kev xav tau tam sim no, lossis tsuas yog tom qab ntawd?
- Yuav ua li cas breakout cuam tshuam rau yav tom ntej tsiv mus rau haiv neeg 100G servers?
Fais fab, cua txias, thiab tswj cable
Ib 100G ntaub ua ntau tshaj bandwidth - nws ua kom sov, airflow load, thiab cable ceev. Kev siv nyiaj siv hluav taws xob yuav tsum them rau qhov hloov chassis thiab cov kiv cua, QSFP28 optical modules (thiab DAC lossis AOC qhov twg siv), cov khoom siv tsis tu ncua, khib- theem peev xwm, thiab kev loj hlob. Cov cua txias yuav tsum suav rau qhov kub- thiab txias- txoj kab kev teeb tsa, zoo ib yam pem hauv ntej- mus rau- rov qab los yog rov qab- mus rau - pem hauv ntej airflow, blanking panels, cable obstruction, ambient kub, thiab module- ntsuas kub, vim hais tias ib tug txha nraub qaum nrog cov khoom siv hluav taws xob tiag tiag.
Cabling scales ceev: 16 nplooj mus rau 4 spines twb 64 nplooj -rau- qaum txuas, txhua qhov yuav tsum tau sau npe, taug kev, kuaj, thiab sau ntawv. Ib daim ntaub puv -mesh yog yooj yim dua los tsim thiab tswj nrog ua ntej- xausMPO / MTP lub cev cabletshaj nrog teb- txiav fiber ntau. Cov pab pawg yuav tsum tau txiav txim siab sib txuas thiab polarity conventions pem hauv ntej; tuskev sib txawv ntawm MTP thiab MPOtsim nyog lees paub ua ntej koj xaj. Cov ntaub ntawv sloppy tsis raug nqi dab tsi rau hnub ib thiab muaj txiaj ntsig zoo thaum thawj zaug hluav taws xob.
Tsim kom muaj kev hloov kho 400G
Tsim cov ntaub nrog ib txoj hauv kev txhim kho tiag tiag. Koj tsis tas yuav 400G txhua qhov chaw hauv ib hnub, tab sis koj yuav tsum zam cov kev xaiv uas ua rau muaj kev mob siab rau tom qab. Pib xav txog 400G kev npaj txhij thaum tus txha nraub qaum yog twb loaded hnyav heev, thaum ntxiv 100G spines tau txais kev txaus ntshai, thaum ECMP txoj kev suav tau ze rau lub platform txwv, lossis thaum AI, cia, lossis sab hnub tuaj -kev loj hlob sab hnub poob nrawm dua.
Lub tswv yim ib txwm yog txhawm rau txhim kho tus txha nraub qaum ua ntej: nplooj khaws lawv 100G uplinks thaum lub siab dua - lub peev xwm qaum - siv cov chaw nres nkoj xws liQSFP-DD- ntxiv headroom, feem ntau nrog 400G cov chaw nres nkoj tawg rau hauv 4x100G rov qab mus rau cov nplooj uas twb muaj lawm. Lub dav trajectory yog teem los ntawm kev lag luam: lubEthernet Alliance txoj hauv kevtam sim no khiav los ntawm 400G, 800G, thiab tshaj, feem ntau tsav los ntawm AI. Thaum koj ntsuas cov keyboards, xyuas tias lub platform txhawb qhov nrawm, kho qhov muag, hom kev sib cais, thiab software nta ib theem kev hloov kho yuav xav tau.
Thaum 100G txha nraub qaum-Neeg tsim tsis yog qhov kev xaiv zoo
Qhov kev tsim no tsis yog universal, thiab ob peb rooj plaub hu rau lwm yam. Ib tug puv tes ntawm servers nyob rau hauv ib los yog ob lub racks tsis tshua pom zoo ib tug tag nrho qaum- nplooj tsim, qhov twg ib khub ntawm redundant keyboards yog yooj yim thiab pheej yig dua. Cov kev cob qhia AI loj heev tuaj yeem thawb dhau qhov kev nkag mus rau 100G thiab 100G txha nraub qaum ntaub ua haujlwm zoo, tsaws ntawm 400G lossis 800G ntaub - lossis txawm tias lub koom haum InfiniBand - txij thaum pib. Thiab yog tias ze li ntawm tag nrho cov tsheb khiav yog sab qaum teb- sab qab teb mus rau lub rooj vag es tsis yog sab hnub tuaj -sab hnub poob ntawm racks, sab hnub tuaj - sab hnub poob zoo ntawm qaum- nplooj tsis muaj teeb meem tsawg, yog li cov topology yuav tsum muaj kev ncaj ncees rau kev loj hlob thiab kev ua haujlwm ntau dua li qhov xav tau. Sib piv cov architecture rau kev khiav tsheb thiab cov nplai, tsis yog lwm txoj hauv kev.
Common 100G Kab laug sab-Leaf Design mistakes
- Suav QSFP28 chaw nres nkoj ob zaug.Ib qhov chaw nres nkoj yog 4x25G breakout lossis 100G uplink, tsis yog ob qho tib si. Muab txhua qhov chaw nres nkoj ib lub luag haujlwm.
- Xaiv optics los ntawm qhov siab tshaj plaws.Kev ncav cuag ntev ntxiv tus nqi thiab lub zog; phim optics rau qhov tseeb fiber ntau nrug thiab hom.
- Tsis quav ntsej N-1.Txheeb xyuas qhov sib piv thaum lub sijhawm ua haujlwm ib txwm thiab tom qab poob tus txha nraub qaum.
- Tsis nco qab optical fais fab thiab cua sov.Ib tus txha nraub qaum tag nrho ntawm QSFP28 modules yog lub thermal load tiag tiag, yog li suav nrog optics hauv lub zog thiab ua lej lej.
- Kev kho kab mob raws li kev xav tom qab.Routing, labeling, polarity, thiab cov ntaub ntawv muaj nyob rau hauv tus tsim, tsis yog lub installation.
- Tsim tsuas yog rau hnub no lub server ceev.Yog tias 25G nkag mus yuav hloov mus rau 100G, tawm hauv chav rau haiv neeg 100G lossis 400G qaum.
FAQ
Q: Dab tsi yog qhov zoo tshaj plaws oversubscription piv rau 100G qaum- nplooj network?
A: Tsis muaj ib qho piv txwv zoo tshaj plaws. Rau kev suav dav dav, 2: 1 lossis 3: 1 feem ntau siv tau. Rau kev khaws cia, AI, HPC, lossis RDMA cov haujlwm ua haujlwm, siv 1: 1 lossis qis dua - tus qauv tsim tawm qhov twg los tau, thiab siv tau rau kev ntsuas kev khiav tsheb.
Q: Kuv puas yuav tsum siv QSFP28 SR4 lossis CWDM4 rau nplooj ntawv -rau- qaum txuas?
A: Siv SR4 rau luv luv multimode khiav qhov twg MPO / MTP cabling muaj. Siv CWDM4 los yog ib qho zoo sib xws - hom optic thaum qhov kev ncua deb ntev dua los yog thaum lub duplex LC ib leeg - hom cog yog nyiam, txog li 2 km.
Q: QSFP28 puas tuaj yeem tawg rau hauv 4x25G?
A: Yog, ntau QSFP28 platforms txhawb 4x25G breakout, tab sis kev txhawb nqa nyob ntawm tus qauv hloov, pawg chaw nres nkoj, kev khiav hauj lwm, thiab hom cable. Nco ntsoov xyuas qhov hloov pauv kev sib raug zoo matrix ua ntej tsim ib ncig ntawm kev tawg.
Q: Puas yog 100G qaum- nplooj tseem tsim nyog tam sim no uas muaj 400G?
A: Yog lawm, rau feem ntau ntawm kev lag luam thiab huab ib puag ncig nrog 25G lossis 100G server nkag tau. 400G khwv tau nws cov nqi siab dua thaum muaj peev xwm uplink, AI tsheb, lossis loj - nplai sab hnub tuaj-west bandwidth ua pov thawj rau nws.
Q: Kuv xav tau pes tsawg tus txha nraub qaum hloov?
A: Yam tsawg kawg yog ob rau redundancy. Cov ntaub loj dua feem ntau siv plaub lossis ntau dua rau kev faib tawm ECMP zoo dua thiab muaj peev xwm nce ntxiv. Tus lej raug yog nyob ntawm nplooj suav, uplink ceev, oversubscription target, thiab platform txwv.
Q: Dab tsi yog qhov yuam kev ntawm ib qho kev tsim ntau tshaj plaws?
A: Port miscounting. Pab pawg npaj uplinks ua ntej thiab tom qab pom tias cov kab hluav taws xob tawg tau siv QSFP28 cov chaw nres nkoj uas lawv xav kom siv rau qaum. Muab cov chaw nres nkoj tawg ua ntej ua tiav qhov peev xwm uplink.
Xaus
Ib qho zoo 100G txha nraub qaum-nplooj tsim yog cov txiaj ntsig ntawm kev txiav txim siab ua ntej cov khoom siv tuaj txog: txhais cov haujlwm ua haujlwm, suav cov chaw nres nkoj kom raug, suav cov kev tshaj tawm nyob rau hauv ob qho tib si thiab tsis ua haujlwm, xaiv optics los ntawm kev ncua deb, npaj kev txiav txim siab txhob txwm tshaj tawm, pob nyiaj siv rau lub zog thiab cua txias, thiab tawm hauv chav rau 400G. Rau feem ntau cov chaw lag luam cov ntaub ntawv, 25G nkag nrog 100G QSFP28 uplinks tseem yog qhov muaj txiaj ntsig zoo ntawm kev ua tau zoo, nqi, thiab nplai, thaum khaws cia, AI, thiab HPC tsuas yog hu rau qis dua thiab siv tau nruj dua. Txoj hauv kev txhim khu kev qha tsis hloov pauv: tsim los ntawm cov neeg rau zaub mov sab nraud, ua pov thawj lej raws li qhov qub thiab N-1, thiab sau txhua qhov txuas ua ntej xa mus.
