The computer network which forms the backbone of the Center of Computer Science infrastructure is composed of three types of high performance Cisco switches. When designing the network our main goal was to facilitate access to the high throughput capabilities offered by the 10Gbit Ethernet standard. For this reason we have chosen to install MultiMode OC3/XG fiber optics with MT-RJ XG 50/125/900 sockets. These connections will handle communications between all major distribution points as well as between the devices located in the building’s central server room (Fig. 1).
Fig. 1. Schematic layout of the fiber-optic infrastructure in the CCS building
Non-core connections between distribution points and within individual rooms will use symmetric copper wiring (PiMF 1200MHz cat. 7A – doubly shielded S/FTP cables with individually shielded pairs and an additional common shield; core diameter of 22AWG). The system (see Fig. 2) can accommodate various types of clients by means of exchangeable hardware interfaces: BNC, RJ45, ARJ45, ISO cat. 7A (TerraConn), CATV type F and others (adjusted for impedance).
Rys2. Schematic layout of PiMF wiring in the CCS building
From a logical perspective, the core network consists of three key distribution points linked by fiber-optic cables. Together, these points represent the network backbone, which can be extended with additional, outlying distribution and access points. All core components have built-in hardware redundancy (Active/Active redundant switches and backup wiring) and are configured in such a way as to minimize the delay in switching over to backup channels in case of a failure.
LAN network topology and selection of Cisco communications devices follow state-of-the-art industry standards. The network itself is divided into three layers, differing with respect to the features they offer and the types of switches used in each layer:
Core layer – the primary backbone to which all other devices are attached. Its purpose is to provide fast switching capabilities. It relies on Cisco Catalyst 6506-E VSS devices. Core switching functionality is based on the MPLS technology which provides many unique features, such as virtual routing process instances, separation, advance reservation of resources, traffic tunneling etc. As uninterrupted operation of these devices is of crucial importance for the entire networking infrastructure, hardware redundancy had to be ensured. This was effected by installing additional physical links between key backbone nodes and by utilizing dual switches.
Fig. 3. Core layer interconnect
Distribution layer – this layer is responsible for access network routing (where each access network maps to a virtual routing instance in the core layer). It also handles security aspects by enforcing communication pathway policies and provides QoS guarantees. It operates on Cisco Catalyst 3750X switches (24- and 48-port versions), including support for the PoE+ technology. At distribution points where more than one switch is required the system provides data stacking (multiple physical devices aggregated into a single logical device which offers the required number of network interfaces) and power stacking (protection against the failure of any single power supply within the stack).
Fig. 4. Distribution layer interconnect
Access layer – this layer handles user access by exposing endpoints for individual clients, servers, access points and other network components. It handles separation of communications and enforces aspects of security related to user authentication and authorization. It is based on Cisco Catalyst 2960S switches (24- and 48-port versions, with optional PoE+ support). Similarly to the distribution layer, stacking is used wherever required.
Fig. 5. Access layer interconnect
DataCenter layer – this is a separate networking infrastructure built with devices which support convergent networks. It enables high-performance data switching (10Gbit Ethernet) as well as enterprise SAN communications (FiberChannel protocol). It is integrated with the server-room network in the CCS building. Devices used in this layer can also support hardware-assisted routing. The hardware pool consists of a convergent NEXUS 5548UP switch with a set of 12 so-called “fabric extenders” – physically distributed switches which, together with the main NEXUS switch, constitute a single logical component with common configuration parameters.
The list below describes the main devices and technologies used to implement the CCS LAN infrastructure:
Catalyst 6500 Virtual Switching System 1440 switch
Catalyst 6500 switches are the flagship Cisco product in the area of high-performance, scalable, reliable and highly configurable switching. The latest of these is the Catalyst 6500 switch with Supervisor VS-S720-10G technology, providing Virtual Switching System (VSS) capabilities.
Fig. 6. Cisco Catalyst 6500 switch
Any pair of Catalyst 6500 switches with VS-S720-10G supervisors can be linked to form an integrated VSS 1440 system. In this mode each device functions as a modular chassis, connected to its partner by a 10Gbps full-duplex interconnect. The network sees both switches as a single logical device with a common management interface and a common routing instance (no need for FHRP and Spanning Tree protocols). The resulting virtual system provides 1440 Mbps of bandwidth and can support up to 84 10Gbit Ethernet ports.
In addition, Catalyst 6500 devices support hardware-assisted IPv4 and IPv6 routing, as well as MPLS switching. The following dynamic routing protocols are supported: RIP, OSPF, BGP-4, IS-IS, PIM-DM, PIM-SM/SSM, IGMP, BGP-MP (with multicast support), MSDP and Anycast RP. Furthermore, IPv6 support is provided for RIPng, OSPFv3, IS-IS for IPv6, BGP for IPv6, PIM-SM/SSM and MLD. MPLS pathing and signaling comes with support for OSPF-TE, RSVP-TE, LDP and MPLS FRR protocols.
Catalyst 3750X switch
The Catalyst 3750X family comprises high-performance layer 3 switching devices which provide 24 or 48 interfaces in the 10/100/1000 (RJ-45) Ethernet standard. These switches are capable of powering input devices on all 48 ports in PoE or PoE+ configurations. Additionally, two 10Gbit Ethernet (SFP+) ports are provided, each of which can be optionally replaced with four 1Gbit Ethernet (SPF) sockets.
Fig. 7. Cisco Catalyst 3750X switches
A Catalyst 3750X switch can process 64-byte packets at a rate of 101Mpps. It contains a 160Gbps switching matrix and can support up to 6000 MAC addresses, 8000 routing paths and 1000 IGMP groups. 3750X series switches can also be linked to form a stack, facilitating virtualization of hardware resources and providing power redundancy for each physical device.
Catalyst 3750X switches support L3 routing (IPv4 and IPv6) with RIP, static routing, PIM, EIGRP (or other equivalent protocol), Hot Standby Routing (HSRP), BGP RIPng and OSPFv3. They handle the following QoS aspects of networking: support for at least four separate traffic queues, CIR per-port bandwidth throttling (with restrictions automatically enforced on the basis of data derived from layers 2-4), support for up to 64 policers for each GE port (with a global limit of 500 ACE), support for IEEE 802.1p CoS and DSCP, SRR, WTD, IEEE 802.1q (minimum of 1000VLANs and 4000 VLAN IDs) and capacity for internal cross-VLAN routing. In terms of security the switch supports user/port authorization via the 802.1x protocol with the ability to create VLAN subnets, ACL access list, guest VLANs for users not having access to 802.1x, packet filtering in layers 2-4, Private VLANs, port security support, IP source guard, dynamic ARP inspection, Spanning Tree Root guard and Bridge Protocol Data Unit (BDPU) guard.
Catalyst 2960S switch
Catalyst series 2960S switches provide 24 or 48 10/100/1000 (RJ-45) Ethernet ports. They support up to 24 PoE devices or 12 PoE+ client devices (providing a minimum of 7W of power to each device). Additionally, two 10Gbit Ethernet or 1000Base-X (-SX, -LX/LH, -ZX) uplink ports are provided, capable of operating in SFP+, SFP or equivalent modes.
Fig. 8. Cisco Catalyst 2960S switches
A Catalyst 2960S switch can process 64-byte packets at a rate of 101.2 Mpps. It contains a 60Gbps switching matrix and handles up to 8000 MAC addresses with 255 IGMP groups. 2960S series switches can also be linked to form a stack (with a 20Gbps interconnect). They handle the following QoS aspects of networking: support for 4 separate packet queues, CIR per-port bandwidth throttling (with a 1Mbps granularity), support for IEEE 802.1p CoS and DSCP, support for SRR and up to 64 filtering rules (policers) attached to each Gigabit Ethernet port. In terms of security the switch supports the 802.1q protocol with the ability to set up 255 separate VLAN subnets, access authorization protocols (administrative access and 802.1x), user/port authorization via 802.1x with support for custom attributes such as target VLANs, packet filtering on layers 2-4 and hardware support for ACL access lists.
Nexus 5548UP switch
This device can process 64-byte packets at a rate of 700Mpps and carries a 960Gbps switching matrix. It can also accept extension cards which provide additional ports (beyond the 32 built-in limit) in the 1/10Gbit Ethernet and FC/FCoE SFP+ standards. The switch is shipped with a module which can optionally support 16 8/4/2/1Gbps Fibre Channel or 1/10Gbit Ethernet FCoE clients. The entire network infrastructure in the CCS building will be connected to 12 external expansion modules with 48 100/1000BASE-T sockets and 4 10Gbit Ethernet SFP+ sockets each, with a 40Gbps inbuilt switching matrix (enabling 1.2:1 oversubscription).
Fig. 9. Cisco NEXUS 5548UP switch
The NEXUS 5548UP switch provides layer 3 features (static routing, RIPv2, OSPF, VRRP, IGMPv2/3, PIMv2, URPF, BGP) and the following layer 2 features: IEEE 802.1q VLAN (support for up to 4096 VLANs), IEEE 802.1w, IEEE 802.1s (up to 64 instances), Spanning Tree PortFast, Spanning Tree Root guard, Spanning Tree Bridge Assurance, IEEE 802.3ad, support for Jumbo frames (up to 9216 bytes) on all IEEE 802.3x ports, storm control (unicast, multicast, broadcast) and private VLANs. Additionally, it provides support for QoS (ACL layer 2, 3, 4 with Weighted Round-Robin classification) and security features – standard and extended internal ACLs, MAC-based or protocol-based layer 2 ACLs, layer 3/4 ACLs (IPv4, IPv6, ICMP, TCP, UDP), VLAN-based ACLs (VACLs) and port-based ACLs (PACLs).