Columbia University

Getting I2 to the desktop

Alan Crosswell alan@columbia.edu

May 2001

NYSERTech

1

Outline

A little about Columbia Wide area Internet[2] connections Metro area inter-campus network Main campus network Gigabit Ethernet experience Some network apps (mcast, video, etc.)

May 2001

NYSERTech

2

Outline

A little about Columbia Wide area Internet[2] connections Metro area inter-campus network Main campus network Gigabit Ethernet experience Some network apps (mcast, video, etc.)

May 2001

NYSERTech

3

Columbia People

40K people: 5,700 CU undergrads 2,500 non-degree 12,000 grad students 3,000 Barnard, TC, UTS students 7,700 CU faculty 7,600 staff

May 2001

NYSERTech

4

Columbia Places

May 2001

NYSERTech

5

Columbia Places

Morningside Heights campus - 116/Bway

May 2001

NYSERTech

6

Columbia Places

Morningside Heights campus - 116/Bway Health Sciences campus - 168/Bway

May 2001

NYSERTech

7

Columbia Places

Morningside Heights campus - 116/Bway Health Sciences campus - 168/Bway Lamont-Doherty Earth Observatory - Palisades

May 2001

NYSERTech

8

Columbia Places

Morningside Heights campus - 116/Bway Health Sciences campus - 168/Bway Lamont-Doherty Earth Observatory - Palisades several smaller facilities

May 2001

NYSERTech

9

Outline

A little about Columbia Wide area Internet[2] connections Metro area inter-campus network Main campus network Gigabit Ethernet experience Some network apps (mcast, video, etc.)

May 2001

NYSERTech

10

Internet & Internet2 too

OC-3 Commodity Internet (Applied Theory) currently 150 Mbps outbound/60 Mbps inbound OC-3 NYSERNet ATM vBNS OC-3 out of NYC POP Abilene OC-3 POS out of NYC POP Abilene OC-3 POS out of Buffalo POP 100baseFX RCN Private Peering Various T1’s

May 2001

NYSERTech

11

Outline

A little about Columbia Wide area Internet[2] connections Metro area inter-campus network Main campus network Gigabit Ethernet experience Some network apps (mcast, video, etc.)

May 2001

NYSERTech

12

Metro Network: Lamont-Doherty

DS-3 leased line ATM T-1 leased line backup

May 2001

NYSERTech

13

Metro Network: Health Sciences

2 x 100 Mbps Ethernet microwave (23 GHz) DS-3 ATM microwave See also: Cornell Univ looking at dark fiber.

May 2001

NYSERTech

14

Outline

A little about Columbia Wide area Internet[2] connections Metro area inter-campus network Main campus network Gigabit Ethernet experience Some network apps (mcast, video, etc.)

May 2001

NYSERTech

15

Morningside Heights area

10 Mbps Ethernet microwave to an off-campus residence. 11 Mbps IEEE 802.11b to two residences. 100 Mbps 5.8GHz hop. 9 T1s to near-campus buildings.

May 2001

NYSERTech

16

Morningside Heights “ISP”

736 V.90 dialups 5,700 student and faculty apartments in 143 nearby buildings

May 2001

NYSERTech

17

Campus Telco

Campus and Neighborhood POTS, CATV, and: ISDN (phasing out) DSL (~50-100 users) copper to switch room own DSL hub RCN Cable modems in two buildings.

May 2001

NYSERTech

18

10baseT Ethernet

20,000 Ethernet ports 4,400 10baseT pillows in 15 undergrad halls 500 10baseT in 20 frats Mostly cat 3 wiring.

May 2001

NYSERTech

19

100baseT Ethernet

Lab of 72 Ultra-10s Central server farm Rewired SEAS, CS w/1500 cat5 jacks (3 buildings). All renovations & new construction are cat 5+ (~6 bldgs)

May 2001

NYSERTech

20

Separately administered School & Dept Nets

Some gigE uplinked Business/Law (3com) 100baseFX or FDDI (pending gigE): Law, Teachers College, Barnard College Bringing old dept-run nets back into fold.

May 2001

NYSERTech

21

Outline

A little about Columbia Wide area Internet[2] connections Metro area inter-campus network Main campus network Gigabit Ethernet experience Some network apps (mcast, video, etc.)

May 2001

NYSERTech

22

GigE Core Network

Centrally funded to upgrade all 10 Mbps shared ports to 10 Mbps switched with a gigabit Ethernet core. 4 Year $7.7M project (in final year) Replaced Cabletron and Xyplex shared hubs and FDDI backbone (99% done). Started as an ATM core. And stopped.

May 2001

NYSERTech

23

GigE Core Network

Installation started summer 99. Core network. All Residence Halls. VIP academic & admin bldgs.

May 2001

NYSERTech

24

GigE Core Network

Recently completed: Reverted early ATM OC-3 LANE bldgs to gigE. Converted school/dept LAN uplinks from 10 Mbps Ethernet to gigE or 100 Mbps.

May 2001

NYSERTech

25

A Building Hub

Catalyst 55xx, 4006, 29xx, 35xx Singly gigE uplinked to core switch/router New construction and cat5 renovations puts one of these typically every 3rd floor. No config changes needed by users. One or more VLANs with 802.1q trunk

May 2001

NYSERTech

26

Edge Router

Edge Router

May 2001

NYSERTech

27

Edge Router

Edge Router

May 2001

NYSERTech

28

Campus bldgs: Old copper

Cat 3: Unshielded twisted pair copper Was installed for phone system. Small FDFs. Vertical risers. BDF.

May 2001

NYSERTech

29

Campus bldgs: Old copper

Cat 3: Unshielded twisted pair copper Was installed for phone system. Small FDFs. Vertical risers. BDF.

May 2001

NYSERTech

30

Campus bldgs: shared Ethernet

Cat 3 phone wiring was also good for: Apple LocalTalk IBM Token Ring (4M) Ethernet 10baseT Basement hubs were squirreled away in closets or hallways. Shared 10 Mbps bandwidth for bldg.

May 2001

NYSERTech

31

Campus bldgs: shared Ethernet

Cat 3 phone wiring was also good for: Apple LocalTalk IBM Token Ring (4M) Ethernet 10baseT Basement hubs were squirreled away in closets or hallways. Shared 10 Mbps bandwidth for bldg.

May 2001

NYSERTech

32

Campus bldgs: New copper

Cat 5e: Unshielded twisted pair copper New construction and renovation standard. Max. 100 meter run. Max. 1 floor vertical. 100 Mbps. RJ45 patch bays in floor closets.

May 2001

NYSERTech

33

Campus bldgs: switched Ether

Ethernet switches in floor closets. 10/100 Mbps dedicated per jack. Gigabit Ethernet (1000 Mbps) fiber uplink per switch.

May 2001

NYSERTech

34

Campus bldgs: switched Ether

Ethernet switches in floor closets. 10/100 Mbps dedicated per jack. Gigabit Ethernet (1000 Mbps) fiber uplink per switch.

May 2001

NYSERTech

35

Campus bldgs: switched Ether

Ethernet switches in floor closets. 10/100 Mbps dedicated per jack. Gigabit Ethernet (1000 Mbps) fiber uplink per switch. Standard closet size accomodates 2 racks.

May 2001

NYSERTech

36

Horizontal Wiring

TIA-568 Commercial Building Standard for Telecommunications Cabling. 300 ft. maximum distance to closet. No more than one floor run vertically. Lucent Systimax category 5e+ plenum cable.

May 2001

NYSERTech

37

Outlet Placement

TIA-569 Commercial Building Standard for Telecommunications Pathways & Spaces. Min. 1 outlex box/faceplate per workstation. Min. 1 outlet box/faceplate per 100 sq. ft. T568B jack pinout. 4 jacks per outlet (2 data, 2 phone).

May 2001

NYSERTech

38

Closets

TIA-569 Commercial Building Standard for Telecommunications Pathways & Spaces. Standard says minimum 1 closet PER FLOOR. We cheap out w/1 per 3 floors.

Serving Area

Closet Size

10,000 sq. ft.

10 x 11

8,000 sq. ft.

10 x 9

5,000 sq. ft.

10 x 7

May 2001

NYSERTech

39

Closets

Bare minimum 4’Dx6’Wx7’H interior clearance. Double louvered doors opening out. Interior plywood walls. No other utilities!

May 2001

NYSERTech

40

Closets

Electrical, Enviro. Two 20A dedicated single NEMA 5-20R. Utility outlet(s), lighting. Ventilation 75° F 12,000 BTUH

May 2001

NYSERTech

41

Closets

Cable terminations EIA 19” relay racks. RJ45 patch bay horizontal jack wire. Telephone riser cross-connections. Network optical fiber risers.

May 2001

NYSERTech

42

Closets

Electronics Network switch. UPS.

May 2001

NYSERTech

43

A Core Router

Cisco Catalyst 6509 Redundant MSFC/2 router w/Supervisor II MSM->MSFC->MSFC/2 Concentrates several building hubs with gigE uplinks. Dual gigE uplinks to dual core L2 6509s Dual Power supplies w/dual UPS. Some generator backup (working on more).

May 2001

NYSERTech

44

A Core Switch

Catalyst 6509 Two of them at opposite sides of campus One’s currently on generator power Eight gigE uplinks from the eight switch/routers We might put routers in these too.

May 2001

NYSERTech

45

Older Network & Routers

FDDI backbone ATM OC-3 cloud Only thing left is two WAN DS-3 links. 7507s for WAN, FDDI, ATM Dual 100baseFX into core (for now) Temporarily gigE (due to bug in port channel support between 7507/6509). Various older routers for T1s (IOS 9:-)

May 2001

NYSERTech

46

Routing Protocols

Interior routing: EIGRP IGRP (incl. ESnet T1) RIP (w/Fore equip @Lamont) Would like to stop being Cisco zombies…. Exterior routing: BGP vBNS, Abilene NYSERNet schools & routers Applied Theory, Uunet, RCN

May 2001

NYSERTech

47

Conversion “Issues”

Had to convert right-angle 25pr telco connectors to straight-in! Had to convert 15A outlets to 20A. Long vs. short-haul GBICs. Expensive MM to SM conditioning cables. Pulled a lot of new SM fiber. Fiber testing: Plug and Pray.

May 2001

NYSERTech

48

Conversion “Issues”

MSMs vs. MSFCs vs. MSFC/2s MSM: limited ACLs; No Appletalk routing. MSFC: too little RAM to run iBGP. DoS caused by unrestrained fast hosts on a fast network (Anime servers, etc.). Need to RTFM and turn on QoS RSN. So far only doing fair queueing. cost of Smartnet service contract.

May 2001

NYSERTech

49

Outline

A little about Columbia Wide area Internet[2] connections Metro area inter-campus network Main campus network Gigabit Ethernet experience Some network apps (mcast, video, etc.)

May 2001

NYSERTech

50

Some Applications

Video Conferencing Voice over IP Multicast Wireless 802.11b

May 2001

NYSERTech

51

Conferencing

Dual-campus classes (Biomedical eng.) Had used ATT/Lucent EMMI mJPEG via dedicated DS-3 ATM microwave link Replacing with IP-based MPEG-2 (or bag of subway tokens). Users expect high quality video (full motion, full frame rate).

May 2001

NYSERTech

52

Conferencing Toys

H.323 codecs + ISDN H.320 gateway Litton CAMvision2 MPEG2

May 2001

NYSERTech

53

H.323 conferencing

Participated in Megaconf I NYSERNet NYC conferencing host site. Member of Videnet (www.cavner.org) Zydacron Z340, Z360 & COMstation. Radvision ISDN (H.320) gateway. Tandberg Codec 5000/RADvision VIU-323 Get a Polycom! (upgrade ISDN model)

May 2001

NYSERTech

54

MPEG2 conferencing

Litton CAMVision 2 Full D1 720x480 video 48 kHz 384 kbps stereo audio 15.4 Mbps Optivision, Minerva, VBRICK

May 2001

NYSERTech

55

Voice over IP

Have a few Cisco phones. Telecom dept is deploying Siemens phones on a very limited basis. SIP (Session Initiation Protocol) is big here in the CS dept. Switching our Cisco phones over to SIP cuz it’s cool and doesn’t require NT.

May 2001

NYSERTech

56

Multicast Backbone

CGMP/IGMP snooping on Catalysts Cisco/Borg Collective at work. PIM sparse mode. Peering w/Abilene, VBNS, NYSERNet, Cornell. RP is on I2 router (nn2k-gw). Seeing some bugs w/6509s.

May 2001

NYSERTech

57

Multicast Backbone

A few conferences so far (earthscape.org) IPTV 2.0 and Osprey-100 card (H.261) In parallel with RealProducer G2. Acquired 3.0 and an Optibase MPEG2 D1. Use 12/8/00 for music event (MPEG1).

May 2001

NYSERTech

58

Multicast Backbone

Would rather be using vic and vat/rat on a real operating system. Many things that use multicast but that don’t work with each other (Real, IPTV, videocharger, Litton, etc.) Need to learn a lot about video production.

May 2001

NYSERTech

59

Wireless 802

11b

Playing with CS/EE depts using Orinoco (Wavelan): Robots, wearable UIs. Central outdoor campus spaces covered. Putting them in lounges, libraries, classrooms, student center, and so on. Roof-to-roof applications for backhoe protection and near-campus buildings.

May 2001

NYSERTech

60

Campus bldgs: Wireless LANs

IEEE 802.11b deployed on small scale today. Not as fast as wired Ethernet. Applications: Outdoors. Conference rooms. Lecture rooms.

May 2001

NYSERTech

61

May, 2001

Columbia University Academic Information Systems

62

www

columbia.edu/acis/networks

May 2001

NYSERTech

63