Comments for BRAD HEDLUND .com http://bradhedlund.com Studies in Data Center Networking, Virtualization, Computing Sat, 04 Feb 2012 19:38:16 +0000 hourly 1 http://wordpress.org/?v=3.2.1 Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Anton N http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8632 Anton N Sat, 04 Feb 2012 19:38:16 +0000 http://bradhedlund.com/?p=3482#comment-8632 Hi Brad, nice reading. Could you please clarify term "2.5:1 oversubscription"? If all leaves\spines switches are L3, Can a leaf box use simultaneously 16 10G uplinks to route traffic to other leaf box? The case when all server of a leaf want to talk with all server of another single leaf. If I see it right, oversubscription is more "statistical" term than "real" in this example. Thanks in advance! Hi Brad, nice reading.
Could you please clarify term “2.5:1 oversubscription”? If all leaves\spines switches are L3, Can a leaf box use simultaneously 16 10G uplinks to route traffic to other leaf box? The case when all server of a leaf want to talk with all server of another single leaf. If I see it right, oversubscription is more “statistical” term than “real” in this example.
Thanks in advance!

]]> Comment on How to Calculate TCP throughput for long distance WAN links by Guru http://bradhedlund.com/2008/12/19/how-to-calculate-tcp-throughput-for-long-distance-links/comment-page-2/#comment-8623 Guru Fri, 03 Feb 2012 13:39:55 +0000 http://www.bradhedlund.com/?p=74#comment-8623 Nice post.. referred it in my post http://www.consultguru.me/post/2012/02/02/SQL-Server-Replication-Project-–-2-(Network).aspx Guru Nice post.. referred it in my post

http://www.consultguru.me/post/2012/02/02/SQL-Server-Replication-Project-–-2-(Network).aspx

Guru

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Brad Hedlund http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8609 Brad Hedlund Wed, 01 Feb 2012 00:43:53 +0000 http://bradhedlund.com/?p=3482#comment-8609 Yep, in the Z9000 there will be a small difference in latency depending on the ingress/egress port pairs. That's not any different than a chassis switch having lower latency on the linecard vs. between linecards. When you're building a fabric that can scale, this is generally acceptable. As for bandwidth, the Z9000 is line rate on all ports. Yep, in the Z9000 there will be a small difference in latency depending on the ingress/egress port pairs. That’s not any different than a chassis switch having lower latency on the linecard vs. between linecards. When you’re building a fabric that can scale, this is generally acceptable. As for bandwidth, the Z9000 is line rate on all ports.

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Alex http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8608 Alex Wed, 01 Feb 2012 00:29:29 +0000 http://bradhedlund.com/?p=3482#comment-8608 Brad, are you able to deliver uniform latency and bandwidth across the fabric with a Z9000 as the Spine switch. Is the Z9000 not a trident network in a box, meaning port-to-port latency will not be consistent accross all 32/128 ports of each spine switch. Brad, are you able to deliver uniform latency and bandwidth across the fabric with a Z9000 as the Spine switch. Is the Z9000 not a trident network in a box, meaning port-to-port latency will not be consistent accross all 32/128 ports of each spine switch.

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Brad Hedlund http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8591 Brad Hedlund Mon, 30 Jan 2012 12:11:59 +0000 http://bradhedlund.com/?p=3482#comment-8591 John, Why does the "L3 hop out of this fabric" need to be connected at the Spine? What's wrong with connecting routers for external connectivity to a Leaf? John,
Why does the “L3 hop out of this fabric” need to be connected at the Spine? What’s wrong with connecting routers for external connectivity to a Leaf?

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by John G. http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8590 John G. Mon, 30 Jan 2012 04:53:50 +0000 http://bradhedlund.com/?p=3482#comment-8590 If we zoom out a bit on the 40G uplink scenario specifically, we also have constraints for the L3 hop out of this fabric and this is even more limiting. If we zoom out a bit on the 40G uplink scenario specifically, we also have constraints for the L3 hop out of this fabric and this is even more limiting.

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Mark http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8587 Mark Sun, 29 Jan 2012 22:10:00 +0000 http://bradhedlund.com/?p=3482#comment-8587 Got it. Thanks for the explanation. Got it. Thanks for the explanation.

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Brad Hedlund http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8585 Brad Hedlund Sun, 29 Jan 2012 18:54:04 +0000 http://bradhedlund.com/?p=3482#comment-8585 Hi Mark, A fabric should provide more than just connectivity for all hosts, it should go a step further in providing uniform latency and bandwidth across the fabric from any host to any host. This enables the flexibility to place workloads anywhere in the fabric without concern for network performance. That's one difference between a "fabric", and a "network". Your design is more like a "network". Yes, it provides connectivity. But it doesn't provide the uniform latency and bandwidth properties of a real "fabric". Each Leaf is only connected to 4 of the 16 Spines. As a result, you'll have some some hosts that will need to make several Leaf-Spine-Leaf hops to communicate, whereas other hosts will only be one Spine hop away. Non-uniform latency. Similarly, you'll have some rack pairs that can only communicate through one Leaf uplink, while other rack pairs can communicate with more than one Leaf uplink. Non-uniform bandwidth. As a result, application performance will vary depending on where workloads are placed in the network, complicating the provisioning model and partitioning the resources. To provide the uniform bandwidth and latency people will expect from your fabric, <strong>make sure your Leafs are connected to all Spines</strong> with the same amount of bandwidth. Hi Mark,
A fabric should provide more than just connectivity for all hosts, it should go a step further in providing uniform latency and bandwidth across the fabric from any host to any host. This enables the flexibility to place workloads anywhere in the fabric without concern for network performance. That’s one difference between a “fabric”, and a “network”.

Your design is more like a “network”. Yes, it provides connectivity. But it doesn’t provide the uniform latency and bandwidth properties of a real “fabric”.

Each Leaf is only connected to 4 of the 16 Spines. As a result, you’ll have some some hosts that will need to make several Leaf-Spine-Leaf hops to communicate, whereas other hosts will only be one Spine hop away. Non-uniform latency.

Similarly, you’ll have some rack pairs that can only communicate through one Leaf uplink, while other rack pairs can communicate with more than one Leaf uplink. Non-uniform bandwidth.

As a result, application performance will vary depending on where workloads are placed in the network, complicating the provisioning model and partitioning the resources.

To provide the uniform bandwidth and latency people will expect from your fabric, make sure your Leafs are connected to all Spines with the same amount of bandwidth.

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Mark http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8584 Mark Sun, 29 Jan 2012 15:07:00 +0000 http://bradhedlund.com/?p=3482#comment-8584 Thank you Brad for the article. I am confused on one point. I think I am missing something, because it appears to me that you can also have the scaling benefits without using 10gb/breakout between the leaf and the spine. I am thinking if you have 16 spines and 128 leafs, you would still be able to scale to a guest count of 5120. My math: 16 spines with 32 40gb ports = 512 40gb ports available for leafs. 512 40gb ports available for leafs divided by 4 40gb ports on each leaf = 128 leafs. 40 guests per leaf times 128 leafs = 5120 guests. Help me find what I'm missing here please :) Thank you Brad for the article. I am confused on one point. I think I am missing something, because it appears to me that you can also have the scaling benefits without using 10gb/breakout between the leaf and the spine. I am thinking if you have 16 spines and 128 leafs, you would still be able to scale to a guest count of 5120. My math:

16 spines with 32 40gb ports = 512 40gb ports available for leafs.
512 40gb ports available for leafs divided by 4 40gb ports on each leaf = 128 leafs.
40 guests per leaf times 128 leafs = 5120 guests.

Help me find what I’m missing here please :)

]]> Comment on Construct a Leaf Spine design with 40G or 10G? An observation in scaling the fabric. by Jason Edelman http://bradhedlund.com/2012/01/25/construct-a-leaf-spine-design-with-40g-or-10g-an-observation-in-scaling-the-fabric/comment-page-1/#comment-8582 Jason Edelman Sat, 28 Jan 2012 16:21:32 +0000 http://bradhedlund.com/?p=3482#comment-8582 You did state it in your post...thanks for the re-clarification. Jason You did state it in your post…thanks for the re-clarification.

Jason

]]>