Not that I blame Brad, there is no vendor that has TRILL implemented and there is also no white paper on TRILL of any value, since no one seems to understand it too well. The only write up on TRILL is the IETF document.

Your experience proves my point. So long Nortel!

Excellent postings!

I was reading on Fabric Path and realized that the F series card is capable of 320Gbps, but the current fabric (46Gbps x 5) is 230Gbps per slot. So for every Nexus 7018 considering the 230Gbps that would be 368 10GE interfaces which can be split in half for uplink versus end point connection for no over-subscription theoretically.

Is there a cabling methodology that can be used for Fabric Path to not have to traverse the chassis fabric to offer 320Gbps per line card. The reason why i ask is in the following document:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/data_sheet_c78-605622.html

[
230 Gbps in each direction (460 Gbps full duplex) distributed across five Cisco Nexus 7000 46Gbps/Slot Fabric Modules fabric modules
320-Gbps switching capacity, per module, in meshed architectures
]

What is this meshed architecture?

Is it like 1 port on each F series line card vertically forming a port channel to each spine while forming ISIS adjacencies horizontally?

This may be off topic, but since it was related to TRILL and Fabric Path…

Thanks Again for sharing a lot of interesting topics!

Rob

Sorry, I’m a bit late at replying at your reply of my earlier post…
I am quite familiar with the Nortel SMLT offering and have spent a considerable time evaluating SMLT vs vPC (and VSS; though this is not MCEC).
As Alex has pointed out already, there was no concept of software forwarding (slowpath) on any of the Nortel platforms which implemented SMLT. The software populates the hardware forwarding records (MAC tales, ARP entries, IP routes and multicast S.G records) but traffic is ultimately always switched in hardware.
In the original ERS8600 implementation software was also called upon to modify those hardware forwarding records following a link or node failure in the MCEC Cluster but still the solution consistently delivered sub-second recovery times (except in scaled environments where the tables were very large).

Now the example you give in your reply to Alex, below, is not correct:

>”Simple example: A server is attached via SMLT to two Nortel switches. The server sends a broadcast packet. Nortel switch #1 receives it, sends it to every other port in that VLAN including Nortel switch #2. Because the hardware on Nortel switch #2 is oblivious to the SMLT topology, it forwards (in hardware) the broadcast packet back to the Server that originated it and out to the upstream network again.”

This is not true, the hardware forwarding records are programmed in a way that is consistent with the active SMLT topology; traffic arriving on the IST (equivalent to vPC peer link) simply cannot be switched or flooded out of an active SMLT link (equivalent to vPC link) (where active means the corresponding SMLT was active on the IST-peer switch).

>”Nortel SMLT implementations are notorious for creating duplicate packets.”

Really? That’s news to me. If that were true, SMLT would be unusable.

Still, in comparing SMLT to vPC you are in fact comparing older Nortel platforms (ERS8600) with the Nexus and that is hardly a fair comparison.
You should raise your sights at the latest incarnation of SMLT in the Avaya VSP9000 platform, where MAC learning is hardware assisted and the hardware is now able to re-route traffic following link or node failures in the MCEC cluster giving sub 50ms recovery times, even in scaled environments. I have seen this in a recent bakeoff.

You were also very keen on hearing why I felt SMLT still had the edge over vPC, and these are the main points I have:
- On vPC, routing protocol peerings over a vPC VLAN (which is also carried over the vPC peer link) are not supported (even if using the vPC peer-gateway feature which is equivalent to Nortel’s RSMLT). This is because you might end up with traffic arriving on the vPC peer link and vPC is unable to IP route traffic from the vPC peer link over to an active vPC link. Nortel only had this issue on their lower end SMLT platform (ERS5500) but never on the higher end platform (ERS8600) where typically SMLT Clusters were connected back to back with OSPF enabled on a single OSPF VLAN (RSMLT was required on that vlan) allowing IP routing between SMLT clusters (Cisco can only do this with VSS today).
- vPC has issues with single attached devices in a vPC VLAN; this practice is discouraged by Cisco. Was never an issue on SMLT.
- IP multicast support over vPC is still not complete (PIM-SSM is not supported) and there is no distribution of IP Multicast streams across the vPC peers (only one of the vPC peers, the Primary, is selected to forward all streams); failover times for IP Multicast are not sub-second. With SMLT, both PIM-SM and PIM-SSM are supported and multicast streams are distributed across both nodes forming the MCEC cluster; failover times are the same as for unicast traffic. On SMLT you can even have a single logical PIM-RP running on both nodes forming the cluster (in a kind of anycast RP mode, though MSDP is not required) which ensures no PIM-SM reconvergence times in case of RP failure. Re-designing a protocol such as PIM to operate over MCEC is very complex; vPC still has some catching up to do.

So I have to insist! Cisco is NOT the only vendor to successfully engineer MCEC.

About Trill/802.1aq. Yes, you were right on both scores.
The ieee standard has 2 separate flavours to it ;SPBV which does not have any MAC-in-MAC capabilities and SPBM which uses 802.1ah MAC-in-MAC encapsulation. SPBV is a bit of a compromise for hardware which is not MAC-in-MAC capable; to date I have not heard any vendor implementing SPBV; the real interest is in SPBM which can be compared to TRILL.
The recent Nanog event last October did pitch TRILL against 802.1aq and I have found all the info I was looking for there.
It is interesting that vindicating Ms Perlman is a recurring argument from the TRILL camp. I don’t know exactly why she and the IEEE disagreed, but clearly TRILL’s design goal of providing massive mutipathing requires a TTL field for loop mitigation which in turn requires a new encapsulation.
Yet that must have been incompatible with the IEEE’s desire to preserve prior IEEE standards such as 802.1ah (MACinMAC) and 802.1ag (OA&M) which relies on 802.1ah enacpsulation.

As far as I can tell, the two standards are substantially similar in what they achieve and the differences represent different trade offs. The biggest difference is in TRILL greater multipathing flexibility which it pays for with a new encapsulation (which requires new chipsets) and lack of OA&M, which in turn are SPBM’s greatest strengths.

Incidentally, apparently Cisco’s FastPath implementation uses a different encapsulation than that defined by TRILL (Nexus needs to re-spin chipsets to do TRILL?) so initially it is more of a proprietary implementation anyway.

Anyhow, I think we just have to wait and see how these variants play out in the market.

Best regards
John Scaglietti

802.1aq’s mac-in-mac mode called SPBM was first born in my lab in its proprietary form which was Nortel’s PLSB. It was designed by about 5 individuals and went through a number of flavors before the design settled and we shipped it in the years leading up to Nortel’s bankruptcy. There are a number of live networks running it and one has 60+ nodes. There are DC and metro deployments. There was never any question what data path to use, mac-in-mac was in our hardware and thorougly tested top to bottom, certainly we did not consider use of something new that had no OA&M (and still doesn’t) and no I-SID (and still doesn’t). We certainly did not need to learn about encapsulation from anbody else .. come on get serious .. mac-in-mac was preceeded by a proprietary format that was somewhat similar OEL2. The first standard version of IEEE 802.1aq was based on only S-tag encapsulation and no link state (but had shortest paths) and we brought the mac-in-mac mode and link state proposals as proposed solutions to the IEEE. The work was based on many years of real live deployment experience with a proprietary and then a standard encapsulation in our metro products.

Anyway speaking of 802.1aq SPBM we just did interoperability tests on real honnest to goodness switches .. what a concept. If anybody wants to see the actual network tested (37 nodes, 5 physical, 2 real vendors) the screen shots and CLI commands executed, network diagrams etc. etc. have a look below. This is all real deal, line rate forwarding, real ASIC/NPU implementations, Equal Cost paths with .1ag OA&M all up and running.

http://www.ieee802.org/1/files/public/docs2010/aq-ashwood-interop1-1110-v02.pdf

A second round will be done in the early new year with many more physical nodes.

The MAC learning process on Nortel and other switches might be in software, but that is not the case with Cisco switches. Cisco switches have always performed hardware MAC learning, which is far more robust. Obviously the data plane on a Nortel switch forwards in hardware just like any other switch, but the hardware is oblivious to the logical topology SMLT was building on top of it.

Simple example: A server is attached via SMLT to two Nortel switches. The server sends a broadcast packet. Nortel switch #1 receives it, sends it to every other port in that VLAN including Nortel switch #2. Because the hardware on Nortel switch #2 is oblivious to the SMLT topology, it forwards (in hardware) the broadcast packet back to the Server that originated it and out to the upstream network again. Nortel SMLT implementations are notorious for creating duplicate packets. Normally this goes unnoticed in a small network, but as the network grows lots of strange performance problems begin to surface.