This question about Cisco UCS traffic steering comes from a reader named Geoff:

There is one aspect of UCS which I have been trying to understand and have not found a good explanation. Perhaps you can point me in the right direction and maybe if it is of general interest it might form the basis for one of your great articles.

As far as I understand it the ports from the virtualization adaptors of different blades are interleaved to the fabric switches, e.g., all “A” ports from even blades go to the left fabric switch (via the fabric extender), but the “A” ports from odd blades go the the right fabric switch. Similarly for the “B” ports. This seems to result in a active/active switch configuration with traffic evenly spread on both switches but how does an odd blade talk to an even blade?? The two Nexus fabric switches are separate with no VPC link between them (as far as I know). Does this blade to blade traffic have to take place via the Nexus 7000. I hope not! Probably the answer is that I have misunderstood something.

My answer:

Every dual port adapter is hard wired the same way. One port is hard wired to Fabric A, the other port is hard wired to Fabric B. This is the case regardless of slot placement.

For every vNIC you create, you can decide whether it uses Fabric A or Fabric B, and if you want failover. The OS sees the vNIC, not the physical adapter. In theory you could have all your vNICs on all the blades configured to use just Fabric A, with Fabric B used only for failover. In this case, all Layer 2 blade to blade traffic would be locally handled by the Fabric A 6100. Any Layer 3 traffic would need to be handled by an upstream L3 switch (Nexus 7000 / Cat 6500).

In most cases customers will use both Fabrics by steering some vNICs out of Fabric A, others out of Fabric B. Should any vNIC on Fabric A need to send traffic to a vNIC on Fabric B on the same VLAN (Layer 2), Yes, that traffic would need to traverse an upstream switch to get from Fabric A to Fabric B. If you are aware ahead of time of a group of servers that will send a lot of Layer 2 traffic between each other, you could certainly set up their vNIC profiles such that they will communicate on the same Fabric. It wouldn’t matter what slot you placed each server because you are steering traffic via the vNIC settings, not via the hardwires.

Make sense?