I was recently surprised to find that MySpace had open sourced a distributed “MapReduce Framework” called Qizmt (http://qizmt.myspace.com/). From the site’s description:
MySpace Qizmt [kiz-mit] is a mapreduce framework for both developing and executing distributed computing applications on large clusters of Windows servers.
This has been a topic that I’ve been interested in for a while so I’m glad to see that someone has been making progress in this space for the .NET world. Dryad remains an interesting prospect which is apparently even seen production use in Microsoft’s ad service, however it’s clear that it hasn’t been “productionized” yet.
One interesting aspect of Qizmt is that it seems they paid a lot of attention to making it easy to deploy. In my opinion, the ability for a developer to easily get started with a one machine install is a key enabler for the uptake of any new technology. Hopefully with more competition for technologies such as this, we will see some cool options become available to us as developers for high performance computing.