Polybase and SQL Server PDW V2
I’m very belated on posting about PASS 2012. The PASS summit had some very exciting announcements about the next release of SQL Server Parallel Data Warehouse. I was at a customer site during the week of PASS and then took some vacation around the holidays. I just got around to watching the videos and wanted to share them here.
Starting at 1 hour 10 minutes in the Day 1 Keynote with Ted Kummert, Ted discusses PDW V2 and Christian Kleinerman demos key new V2 features. This demo includes the addition of xVelocity ColumnStore into SQL Server PDW. After that, Ted introduces Polybase which allows us to query data in Hadoop using TSQL within SQL Server PDW.
I would recommend watching the Day 3 Spotlight Session with David DeWitt next as he deep dives into Polybase and its value proposition versus using Sqoop. After you have watched the video, you can download his slides here: http://gsl.azurewebsites.net/Projects/Polybase.aspx
Finally, the Day 2 Keynote with Quentin Clark includes a great demo of Polybase starting at 1 hour 15 minutes into the video.
My thoughts are that the xVelocity Columnstore integration is going to provide immediate benefits to existing customers looking forward to improvements in query times. Christian was able to show 10x improvements in query times using ColumnStore vs. traditional clustered indexes. My guess is that we will be able to see even greater improvements in query times for some query mixes. In the long run as we continue to integrate traditional RDMS systems such as PDW with Hadoop systems, Polybase looks to be an exciting addition to the product. Immediate thoughts of mine are around being able to store Petabytes of data in Hadoop and only bring the data we need into PDW to answer analytical questions that drive our Business Intelligence tools. Conversely, this may be a great solution to offloading data from PDW for DR purposes by pushing data from SQL Server PDW to Hadoop. As we get closer to release time, I’ll post more about Polybase and its place in the ecosystem.