Skip to content

Hortonworks Data Platform for Windows and HDInsight News

2013 February 26
by Brian Mitchell

Microsoft and Hortonworks recently announced the release of the beta for Hortonworks Data Platform (HDP) for Windows.   In short, Hadoop on Windows.  The 100% fully compatible Apache Hadoop distribution that runs on Windows servers.  This means Hadoop users can port your current Hadoop applications from a Linux environment to Windows Server today.  Hortonworks has a very good webpage dedicated to HDP on Windows.

You can download HDP 1.1 for Windows here.

Is this the HDInsight Server that we’ve heard so much about?

No, HDP for Windows is the foundation for HDInsight Server that will be available at a future date from Microsoft.   HDInsight Server is an on-prem solution that has full integration with other Microsoft enterprise tools such as Active Directory and System Center for deployment and provisioning.  Hortonworks Data Platform for Windows is ideal for those who have some experience with Hadoop and want to deploy on Windows today.

Download HDInsight Server Developer Preview here.

How is that different from HDInsight Service?

HDInsight Service is Microsoft’s cloud offering for organizations that want lower upfront costs and elasticity in their Big Data solution.

All three solutions are 100% Apache Hadoop compatible.  So you can build on HDP today, move it to HDInsight Server tomorrow, and move the solution to the cloud on HDInsight Service when you are ready.

Try out HDInsight Service here.

Why have three solutions?

Some people like to build their own solutions from scratch and they can use HDP.  Some want to eliminate most complexity but still want some control over their environment and they will likely use HDInsight Server on premises.   Finally, some could care less about the technology and just want a system that will get them the results they are looking for; these users will likely choose HDInsight Service on Azure.

I’m an occasional cyclist and you can pretty much put most cyclists into three camps:

  1. Those who build their bike from scratch.  They buy a frame from one vendor, handlebars from another, wheels from another.  They spend hours mulling over the decisions of which part is a couple grams lighter than another.  They love spending the time in the garage assembling their masterpiece.

  2. Those who buy a really nice carbon bike but make several modifications such as new carbon wheels and a specific crank set.  They spend some time in the shop adjusting everything to get it just right for them and their style.  They don’t spend near as much time as those from camp#1 do in preparing their ride, but they don’t take the decisions lightly either.

3. Those who go buy a Trek Madone, adjust the seat, and go ride.

At the end of the day, all three bikes are very comparable. When you are riding in a group it doesn’t really matter which one of the three camps you fall into.  We are all doing the same thing, peddling about 100 rpm’s, hydrating regularly, drafting as much as possible, and finally giving it our all in the last few miles. How your bike was constructed has little bearing on the ride because let’s face it that factory-built Madone is a damn good bike, but your comfort level with how your ride was built affects your mindset and performance.  Do you want to spend lots of time building and riding your bike or do you just want to ride?

Whether you choose to build your own solution today with Hortonworks Data Platform for Windows, wait for HDInsight Server to use MS tools deploy a Hadoop solution, or use HDInsight Service in Azure to spin up a cluster in 10 minutes, you will be running the same MapReduce jobs.  Some of you will want to build a specialized solution that is perfect for your organization and others will just want to code cool solutions and don’t care as much about the platform that gets it done. Microsoft provides choice because people naturally demand it.

2 Responses leave one →
  1. manish permalink
    March 6, 2013

    Hello Brian,
    Thank you very much for your posts. I am a SQL developer /DBA with very limited JAVA skills. I was scared of diving into Hadoop because it needs some programming skills in JAVA. I wanted to know that is HDinsight has reduced the JAVA involvement? Is Querying HDinsight very much like T-SQL?

  2. Brian Mitchell permalink*
    March 7, 2013

    HDInsight is Hadoop and you can program Map/Reduce code against it using Java just like any other Hadoop installation. HDInsight is also designed so that you can use C# and JavaScript, amongst other languages. If you want to use a SQL like language to program against HDInsight, look into Hive. Hive uses a SQL like language known as HiveQL. HDInsight does have a Hive console built right into the product. Hive is great when you have a specific query you want to run against your data that is stored in Hadoop/HDInsight.

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS