Data Lakes with Michael Rys
Data Lakes are growing up, and you want one! While at Ignite in Atlanta, Carl and Richard sat down with Michael Rys to talk about Azure Data Lakes - a place to store your data "as is" so that you can easily query and organize the data for further analysis. Michael discusses the problems of data warehouses, with their Extract-Transform-Load (ETL) processes that manipulate the data into a particular shape for the warehouse - and make it harder to ask new questions of the data. Leave the data as it is in the data lake and then build mechanism to extract on demand for the various data marts you have. The conversation turns to USQL (U as in Universal) and HDInsights (Hadoop) as different ways to extract data from the Data Lake for analysis. Lots of choices!
Guests:
Michael Rys
Michael Rys has been doing data processing and query languages since the 1980s. Among other things he has been representing Microsoft on the XQuery and SQL design committees and has taken SQL Server beyond relational with XML, Geospatial and Semantic Search. Currently he is working on Big Data query languages such as SCOPE and U-SQL when he is not enjoying time with his family under water, on the ski slopes, or at autocross.
Links:
- Biggest DDoS Attack Ever http://arstechnica.com/security/2016/09/botnet-of-145k-cameras-reportedly-deliver-internets-biggest-ddos-ever/
- Krebs on Security DDoS Post https://krebsonsecurity.com/2016/09/krebsonsecurity-hit-with-record-ddos/
- Azure Data Lake https://azure.microsoft.com/en-us/solutions/data-lake/
- USQL Tutorial https://azure.microsoft.com/en-us/documentation/articles/data-lake-analytics-u-sql-get-started/
- Azure HDInsight https://azure.microsoft.com/en-us/services/hdinsight/
- Microsoft Research Dryad https://www.microsoft.com/en-us/research/project/dryad/
- Azure Data Factory https://azure.microsoft.com/en-us/services/data-factory/