Next Generation Optical Transport for IP Network EvolutionDrew Perkinsdperkins@infinera.com408-572-5308

2NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

Agenda

� Optical transport requirements for future IP networks� Ethernet services� Super-λ services� 10 GbE, 100 GbE and TbE� Reconfigurable IP Router Bypass

� Modulation approaches for higher capacity

� Photonic integration

3NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

10 GbE More Economic Than 10G and 40G POS

� Economics (First-in and ongoing CapEx and OpEx) is foremost concern� It’s the economics, stupid!

� Study of CONUS IP and transport network costs shows 10 GbE most cost effective IP link technology

� Future optical transport network should be Ethernet-based

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.14

90.

209

0.29

80.

417

0.59

60.

849

1.19

21.

684

2.38

43.

367

4.76

86.

759.

536

13.4

8519

.072

26.9

6938

.145

IP Traffic Volume (Tb/s)

IP N

etw

ork

Cos

t/Tra

ffic

Vol

ume 10GbE

OC-768 POS

OC-192 POS

Mixed OC-192/768 POS

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

0.849

1.192

1.684

2.384

3.367

4.768 6.7

5

9.536

13.48

5

19.07

2

26.96

9

38.14

5

4NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

IP Now Requires Super-λ (Multi-wavelength) Links

� IP links once used sub-λ SONET/SDH circuits

� Capacity growth of IP links has out-paced capacity growth of a single λ

� By late 1990s IP links required full 2.5G and then 10G λs

� IP now requires super-λ links (composite links, LAGs, etc.)

5NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

The Future Belongs to Tb/s Links!

� Carriers deployed Nx10 Gb/s links several years ago� N has now surpassed hardware limits of 8-32 in some networks

� Some carriers now deploying Nx40 Gb/s router links� Is this like putting out a 5-alarm fire with a garden hose?

� Current IP growth rates, if sustained, will require IP link capacity to scale beyond 1 Tb/s by 2010

6NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

Carrier/Network Operator Input to IEEE

� Other views on timing of 100GbE:� “100GigE Needed for Broadband Customer Aggregation urgently in the

core by 2009 and across the board by 2011”, Jason Weil, Cox Communications, IEEE HSSG April, ’07

� “2009 timing: Will be a very uncomfortable wait”, Donn Lee, Google, IEEE HSSG March ‘07

� “Bundles of 8 means that we will need 100 Gbps ye 2008 / beginning 2009”, Ad Bresser, KPN IEEE HSSG May ‘07

� “Work needs to begin on whatever follows 100G as soon as possible”,Ted Seely, Sprint Nextel, IEEE HSSG March ‘07

Source: Level3, OFC 2007 100GbE Workshop

YE10 projected scale

7NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

IP Network Economics Study:No IP Router Bypass Links� IP links connected between adjacent routers

� IP core links carried over WDM network

� No IP router bypass links� End-to-end IP demands switched hop-by-hop

� Increasing inefficiency & cost as traffic volume scales

IP over WDM Core Network Without IP Router Bypass L inks

8NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

IP Network Economics Study:With IP Router Bypass Links � Reduce end-to-end IP demands transiting through multiple

router hops

� Increase number of direct router-to-router core links

� IP router bypass in WDM layer

� Minimize use of high-cost router ports and capacity

IP over WDM Core Network with IP Router Bypass

9NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

IP Router Bypass Yields Dramatic Cost Savings: 10 GbE� Network cost savings rises with network scale

� Network cost rises after last point shown

� Sparse mesh least costly

Relative IP Network Costs (10 GbE Ports)

60%

70%

80%

90%

100%

0 5 10 15 20 25 50 75 100 114

Number of IP Bypass Links

0.298Tb/s

0.849Tb/s

2.384Tb/s

6.75Tb/s

19.072Tb/s

38.145Tb/s

Relative WDM Network Costs (10 GbE Ports)

80%

82%

84%

86%

88%

90%

92%

94%

96%

98%

100%

0 5 10 15 20 25 50 75 100 114

Number of IP Bypass Links

0.298Tb/s

0.849Tb/s

2.384Tb/s

6.75Tb/s

19.072Tb/s

38.145Tb/s

18-node IP core network modeled18x17=306 possible bypass links

10NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

IP Router Bypass Yields Dramatic Cost Savings: 40G POS� 40G POS more costly than 10 GbE (see above)

� 40G defers benefits of IP router bypass� 4x demand required for each bypass link

� Optical transport network must be rapidly reconfigurable to maximize IP router bypass benefits� Adjust for route changes, changing traffic sources, etc.

Relative IP Network Costs (40G POS Ports)

Number of IP Router Bypass Links

Relative WDM Network Costs (40G POS Ports)

Number of IP Router Bypass Links

60%

65%

70%

75%

80%

85%

90%

95%

100%

0 5 10 15 20 25 50 75 100

1.192Tb/s

2.384Tb/s

4.768Tb/s 9.536Tb/s

19.072Tb/s

38.145Tb/s

80%

82%

84%

86%

88%

90%

92%

94%

96%

98%

100%

0 5 10 15 20 25 50 75 100

1.192Tb/s

2.384Tb/s

4.768Tb/s

9.536Tb/s

19.072Tb/s

38.145Tb/s

11NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

All-Optical ROADMs

� All-optical wavelength switching using filters, ROADM, WSS, etc.

� OEO only for local add/drop

� No sub-wavelength add/drop

� No wavelength interchange

� No digital PM or OAMP

� Wavelength contention and blocking� Up to 30-40% incremental OEO for l-

conversion = hidden CapEx premium

� Service limited by wavelength path� Optical reach� Number of (R)OADM nodes passed� Fiber characteristics

All-opticalpass-through

“express” traffic

Add/Drop “local” traffic

O-E

-O

O-E

-O

O-E

-O

O-E

-O

λ1 …λN

λ1 …λN

λi, λj

O-E

-O

λ1 …λN

New service blocking due to wavelength contention

End-end wavelength service

Economic gain limited by all-optical ROADMs impleme ntation

12NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

Digital ROADMFull Reconfigurability at Every Node

� Use (analog) photonics for what it does best: WDM transmission

� Use (digital) electronics for everything else

� Digital add/drop, switching, grooming, PM and protection…

� …at every node

� Unconstrained digital add/drop

� Any service at any node

� End-end service delivery independent of physical path

� Robust digital PM and protection

� Digital OAMP & managementIntegrated P

hotonics

Integrated Photonics

• Sub-λ add/drop • Digital switching• Signal regeneration• PM & Error correction• Digital Protection• Digital OAMP

Digital Electronics& Software

Truly unconstrained reconfigurable optical networki ngTruly unconstrained reconfigurable optical networki ng

13NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

How Will Capacity Be Scaled In The Future?

Time Division Multiplexing(ie: Baud Rate)

10 Mbps

100 Mbps

1 Gbps

10 Gbps

100 Gbps

Space Division Multiplexing(ie: Parallel Optics)

14

128

Modulation(ie: Bits per Hz)

1 (e.g. NRZ)

2 (e.g. PAM-4, (D)QPSK)

4 (e.g. QAM-16)

8 (e.g. QAM-256)

Wavelength Division Multiplexing

(i.e. λs)1 2 4 6 8 10

DWDMCWDM

WDM proven to reach Tb/s level

14NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

It’s Really a Question of Economics

� 40G per λ TDM: Service Provider experience thus far� 4x bandwidth increase, but » 4x cost increase� Not 2.5x as historically experienced and now expected/wanted� cost(1x 40 Gb/s) » cost(4x 10 Gb/s)

� 100G per λ Modulation: Data released so far� Complicated, power-hungry electronics (21W per 40G!)� Will require significant integration to yield acceptable costs� Will it cost < 10x 10 GbE?� Equipment vendor(s) predicting 100 GbE may cost ~2.5x 40G POS

� cost(2.5x 40 Gb/s) » cost(10x 10 Gb/s)

� 10x 10G per λ DWDM: Equipment deployed today� Integrated via PICs and shipping since 2004� Prediction: cost(10x 10 Gb/s λs) will likely remain « cost(1x 100

Gb/s λs) for a long time

15NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

Multi-Tb/s Line Capacity and Tb/s Line Cards Will Require Massive Integration� 10 – 100x advancements in integration required to meet

density, power, reliability and cost requirements� E.g. 100G line cards will not set the hurdle; better be thinking 1T!

1.6 Tb/s 10’s Tb/s

Higher Line Rates(PIC Capabilities)

Beyond C-band(Enabled by PIC SOA)

Dem

ux

Mux

SOA array

PIC based SOA amplification1.6Tb/s PICS (40ch x 40G)

-60

-50

-40

-30

-20

-10

0

Nor

mal

ized

Out

put P

ower

(dB

)

1.5651.5601.5551.550

Wavelength (µm)

Ena

blin

g T

echn

olog

ies

Example Capacities

16NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

Summary

� Ethernet will rule the day economically

� Super-λ implementations have become and will remain the norm

� Optical transport network must be flexible to enable greatest economic savings from IP router bypass

� Multi-Tb/s line systems and Tb/s line cards required

� Massive integration of optical, OEO and electronic components required

Thanks!

Drew Perkinsdperkins@infinera.com408-572-5308

18NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

100G LH Transponder 2 (Rx)Router 2

Router 1

100 GbE Over DWDM Transport –10x 10Gb/s Example (1 of Several Possibilities)

100G LH Transponder 1 (Tx)

100G SR Transponder (Tx)

100 GbE MAC ASIC

10x 10 Gb/s FEC ASIC

10:4 Gearbox

10x

10G

100G SR Transponder (Rx)

4:10 Gearbox

10x

10G

SMF 1

25Gx4λ4x 25 Gb/s MD & EML

Arrays

4x 25 Gb/s

Detector & TIALA Arrays

4:1

WD

M

Mux

4:1 WD

M

Dem

ux

4x

25G

4x

25G

4x

25G

4x

25G

100G SR Transponder (Rx)

10x 10G FEC ASIC

100G SR Transponder (Tx)

100 GbE MAC ASIC

10:4 Gearbox

10x

10G4:10

Gearbox10x

10G

Optical Transport Network

1

25Gx4λ4x 25 Gb/s MD & EML

Arrays

4x 25 Gb/s

Detector & TIALA Arrays

4:1

WD

M

Mux

4:1 WD

M

Dem

ux

4x

25G

4x

25G

4x

25G

4x

25G

10x 10 Gb/s MD & EML

Arrays

10x 10 Gb/s

Detector & TIALA Arrays

10:1 W

DM

M

ux10:1

W

DM

D

emux

10x

10G

4x

25G

10x

10G

10x

10G

SMF

19NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

IP over WDM Analysis

� Develop base-line representative model of core IP & optical networks� Ensure realistic and accurate modeling� Realistic set of equipment options considered: router ports, WDM layer,

etc.� Realistic assumptions on market pricing: router ports, transponders,

ROADMs, etc.� Realistic optical layer design: regeneration, wavelength blocking etc.

� Investigate impact of network architecture changes to IP layer:� IP designs that reduce router interface count can greatly reduce overall

network cost.� IP design has impact on architecture and cost of WDM layer� Degree of router bypass on core links� IP core link (trunk) and router scaling� IP port evolution: 10G vs. 40G and 100G

� Investigate various optical transport options� Legacy & next-gen WDM technology options� Impact of WDM layer functions: optical bypass, restoration, reconfiguration,

etc.� Impact of IP design on transport layer (ie: fewer IP trunk interfaces versus

longer optical spans)

20NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

IP Network Cost Study Methodology

� Develop IP traffic model: generic A-Z city-city demand set

� Generate IP demands: collector and core

� Pick IP core nodes

� Generate IP core links

� Determine router-to-router link capacity

� Generate cost comparisons

� Sensitivity analysis� Sensitivity on IP traffic volume: 1x to 256x initial traffic� Sensitivity to degree of IP router bypass

21NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

Fiber Topology and A-Z Demands

Network Fiber Topology IP Network A-Z Demands

22NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

Core and Regional Links

IP Core Links IP Regional Links

23NANOG 41, October 15, 2007 Drew Perkins, Next Generation Optical Transport for IP Network Evolution © 2007 Infinera Corporation

“Alien” WavelengthsPractical Considerations & Operational Implications

� Interconnect IP routers to WDM transport via short-reach optics� Clear demarcation between client signal and

transport layer� Full end-end optimization, control &

management of end-end service� “Best-in breed” IP and transport systems

WD

M

WD

M

Optical Transport Network

IP Network

Optical Transport Network

IP NetworkSR

SR

SR

SR

Native Wavelengths

“Alien” Wavelengths

� Integrated ITU-grid WDM optics on router input directly to WDM line system� No end-end control/management plane� Design optical link budgets to worse-case� Loss of end-end turn-up automation� Complex or no inter-network protection� Loss of PM and fault sectionalization

capabilities� Complex service activation� Minimal CapEx savings – if any – from

transponder reduction offset by design tradeoffs and operational complexity

Ven

dor

AV

endo

r B

Ven

dor

AV

endo

r B