Cisco/Insieme 40G Coming in Close to 10G

Premier Mellanox Selections: best-selling Mellanox Switches, Network Cards, and Cables

Preface: Spin-in developed transceiver for 30% premium over 10G; leaf, spine switches mix merchant, custom ASICs; ACI fabric based on VXLAN.

As the November 6 launch date gets closer, more details are emerging on Cisco/Insieme’s Nexus 9000 and Application Centric Infrastructure (ACI) introduction. Yes, merchant silicon will play a key role but the ACI fabric will be powered by a custom Insieme ASIC, sources say.CiscoInsieme 40G Coming in Close to 10G

Merchant silicon will be used for basic Layer 2 switching, Layer 3 routing and VXLAN overlay, while the Insieme ASIC will be used for buffering, virtual output queuing and VXLAN routing. ACI will be founded on VXLAN, with Insieme hooks in the VXLAN header for additional application identification and traffic classification. Within each VXLAN, traffic can be classified and policies applied to millions of applications, sources say.

ACI is based on a VXLAN flooding and learning underlay. The fabric will take in Layer 2 frames at the edge, send it across the spine as a VXLAN, and terminate it at the destination edge as a Layer 2 frame again.

The Nexus 9000 switches will include both modular spine and fixed-configuration leaf devices that run in either standalone mode – vanilla Layer2/3 based on Broadcom’s Trident II silicon — or ACI mode. The modular switches are the eight-slot Nexus 9508, and the four-slot 9504 and 16-slot 9516.

Line cards for these switches include 36x40G with Broadcom silicon for non-ACI standalone applications; 36x40G, 48x10G SFP+ and 4x40G, and 48×1/10GBase-T and 4x40G with a mix of Broadcom and Insieme silicon for standalone or ACI fabric leaf applications; and 36x40G wire-rate cards with Insieme-only chips for Layer 3-only ACI fabric spine applications.

The fixed configuration switches are the Nexus 9396PQ and 93128TX. The 9396PQ is a 2RU, 48-port 10G SFP+ switch that includes a 12-port 40G uplink module, and is designed for standalone or ACI leaf use. The 93128TX is a 3RU, 96-port 1/10GBase-T switch with a 12-port 40G uplink module also intended for standalone or ACI leaf use.

Insieme also developed a 40G transceiver designed to bring 40G pricing down to just 30% above 10G pricing, sources said. This is intended to make the Cisco/Insieme switches price competitive with Arista Networks’ gear. Currently, 40G pricing is roughly 4X 10G pricing, and 2012 ASPs on 10G SFP+ were $1,229 for modular ports and $335 for purpose built ports, according to Dell’Oro Group.

A six-port 40G uplink module for the 9300 line may come after first customer ship, sources say, as well as 96-port SFP and 48-port 10GBase-T varieties. Also in the pipeline might be a fixed configuration fabric spine switch for smaller environments, and a next-generation Unified Computing System (UCS) fabric interconnect with VM-FEX, FibreChannel and FibreChannel-over-Ethernet based on the Insieme developments, sources say.

Users looking to migrate from standalone to ACI mode might be disrupted, sources say. It could require recabling, deploying new spine switches, rebooting all leaves with new software, and reapplying the network configuration, they say.

The Nexus 9000 will run a stripped-down version of NX-OS with standalone and ACI fabric images. Standalone will include OpenFlow, onePK after first customer ship, and an OpenStack Quantum plug-in; ACI will feature “centralized” programmability that does not include onePK or OpenFlow. OpenFlow support in standalone mode implies support for Cisco’s Extensible Network Controller (XNC) and the controller being developed by the OpenDaylight Project.

The Insieme NX-OS will not include features such as LISP, MPLS or Overlay Transport Virtualization, sources say, which could keep the Nexus 7000 around for a while, especially for interconnecting multiple data centers.

The Insieme controller is not based on XNC. It is similar the UCS Manager for Cisco’s UCS server/switching/virtualization/storage access system, though designed specifically for the network, sources say. It provides an object-oriented representation of the network accessible through either a GUI or CLI, and it runs on Cisco UCS C-Series rack mount servers. There are plans, though, to integrate the controller into the spine switches, sources say.

The controller has an extensible API with hooks for compute and storage, and application profiles will include endpoint groups – 802.1q, VLAN, source MAC, NVGRE and VXLAN IDs – contracts (ACL, firewall), and service chaining (load balancing, VPN, WAN acceleration). Application services can be virtual or physical in ACI, sources said, and can be readily removed if an application profile is removed.

The impact of ACI on existing Cisco Nexus products could be significant. Its VXLAN foundation could obsolete Cisco’s five-year-old Fabricpath and recently-announced Dynamic Fabric Automation, yet users needing those features and LISP, OTV and MPLS will continue to require the Nexus 7000/7700 with F3 capabilities.

There are no plans for ACI to support TRILL or Nexus Virtual Device Contexts at this time either, sources say, but fabric extenders like the Nexus 2200 line will be supported.

Users might also be confused with the positioning of the Nexus 9000 and Cisco’s Nexus 5000 and 6000, and other 10G/40G products like the just announced Nexus 3132Q and 3172PQ. Cisco has started dropping prices on products to clear inventory for the arrival of the Nexus 9000 sources say, with Nexus 6000/2000 bundles coming down to Nexus 5000/2000 levels, and the Nexus 3064 costing half of its initial price.

And even though Cisco CEO John Chambers intimated that Insieme would develop a converged switch/compute/storage platform, compute and storage will be integrated through the Insieme controller API, with object definitions coming from storage vendors and UCS Manager, sources said. WHIPTAIL flash memory appears targeted, for now, at nearline storage for desktop virtualization.

The Nexus 9508 will be available later this year, sources said. The 9504 and 9516 will be available in March 2014. The ACI fabric will be available in April 2014, and the smaller fixed form factor fabric spine is targeted for October 2014.

Original News by Jim Duffy from

More Related Cisco News:

Cisco Released Wave 2-Ready 802.11ac Access Point and Catalyst 3650

Cisco Security Advisory: Multiple Vulnerabilities in Cisco ASA Software

Cisco Patches Flaw in Security Appliances, Switches, Routers

Cisco Nexus 3100, Ready to Support VMware NSX?

Cisco IOS Updates Fix Eight Denial of Service Vulnerabilities

Cisco: Internet of Everything Router is Revealed

Share This Post

Post Comment