Does 10GE Hadoop make any sense? And if so, how might you design the cluster? Let’s discuss some rationale for and against 10gig Hadoop and then look at some potential network setups for 10G clusters. If you need a quick into or refresher, read this post on the basics of Hadoop clusters and the network, [...]
Hadoop network design challenge
November 5, 2011 By 31 Comments
I had a bit of fun recently working on a hypothetical network design for a large scale Hadoop implementation. A friend of mine mentioned he was responding to a challenging RFP, and when I asked him more about it out of curiosity he sent me these requirements: (4) Containers In each Container: (25) Racks (64) [...]
Understanding Hadoop Clusters and the Network
September 10, 2011 By 25 Comments
This article is Part 1 in series that will take a closer look at the architecture and methods of a Hadoop cluster, and how it relates to the network and server infrastructure. The content presented here is largely based on academic work and conversations I’ve had with customers running real production clusters. If you run production [...]
Distributed systems trickle down into Enterprise IT
August 19, 2011 By 9 Comments
In my new role at Cisco I’ve had the opportunity to observe and study something happening that is (in my opinion) truly significant and mind blowing. The IT data center landscape as we know it is on the precipice of a major upheaval. I’m not talking about virtualization, and cloud, and all that other stuff that’s been [...]
Recent Comments