U-SQL Learning Resources (SQLBits 2016)

U-SQL Learning Resources (SQLBits 2016 ADL/U-SQL Pre-Conference)

Data & Analytics

Michael Rys
Principal Program Manager, Big Data @ Microsoft
@MikeDoesBigData, {mrys, usql}@microsoft.com
U-SQL Additional Features and Learnings

Additional
resources
• Tools:
• http://aka.ms/adltoolsVS
• Blogs, videos and community page:
• http://usql.io (Link to Github with code samples)
• http://blogs.msdn.com/b/visualstudio/
• http://azure.microsoft.com/en-us/blog/topics/big-data/
• https://channel9.msdn.com/Search?term=U-SQL#ch9Search
• Documentation and articles and slides:
• http://aka.ms/usql_reference
• https://azure.microsoft.com/en-us/documentation/services/data-lake-analytics/
• https://msdn.microsoft.com/en-us/magazine/mt614251
• http://www.slideshare.net/MichaelRys
• ADL forums and feedback
• http://aka.ms/adlfeedback
• https://social.msdn.microsoft.com/Forums/azure/en-US/home?forum=AzureDataLake
• http://stackoverflow.com/questions/tagged/u-sql

Unifies natively SQL’s declarativity and C#’s extensibility
Unifies querying structured and unstructured
Unifies local and remote queries
Increase productivity and agility from Day 1 forward for
YOU!
Sign up for an Azure Data Lake account and join the Public Preview
http://www.azure.com/datalake and give us your feedback via
http://aka.ms/adlfeedback or at http://aka.ms/u-sql-survey!

What's hot

U-SQL Reading & Writing Files (SQLBits 2016)

U-SQL User-Defined Operators (UDOs) (SQLBits 2016)

Tuning and Optimizing U-SQL Queries (SQLPASS 2016)

The Road to U-SQL: Experiences in Language Design (SQL Konferenz 2017 Keynote)

U-SQL Meta Data Catalog (SQLBits 2016)

More and more customers who are looking to modernize analytics needs are exploring the data lake approach in Azure. Typically, they are most challenged by a bewildering array of poorly integrated technologies and a variety of data formats, data types not all of which are conveniently handled by existing ETL technologies. In this session, we’ll explore the basic shape of a modern ETL pipeline through the lens of Azure Data Lake. We will explore how this pipeline can scale from one to thousands of nodes at a moment’s notice to respond to business needs, how its extensibility model allows pipelines to simultaneously integrate procedural code written in .NET languages or even Python and R, how that same extensibility model allows pipelines to deal with a variety of formats such as CSV, XML, JSON, Images, or any enterprise-specific document format, and finally explore how the next generation of ETL scenarios are enabled though the integration of Intelligence in the data layer in the form of built-in Cognitive capabilities.

Modernizing ETL with Azure Data Lake: Hyperscale, multi-format, multi-platfor...

Survey of the Microsoft Azure Data Landscape

Dive Into Azure Data Lake - PASS 2017

Tips & Tricks SQL in the City Seattle 2014

Cloud architectural patterns and Microsoft Azure tools

Pushkar Chivate

A lap around microsofts business intelligence platform

Microsoft TechNet - Belgium and Luxembourg

Database Schema Management & Deployment using SQL Server Data Tools (SSDT)

SQL PASS BAC - 60 reporting tips in 60 minutes

Adf and ala design c sharp corner toronto chapter feb 2019 meetup nik shahriar

Nilesh Shah

DMann-SQLDeveloper4Reporting

David Mann

Not only SQL - Database Choices

Lynn Langit

Azure DocumentDB 101

Machine Learning Data Lineage with MLflow and Delta Lake

Databricks

24 Hour of PASS: Taking SQL Server into the Beyond Relational Realm

Apache Spark in Industry

Dorian Beganovic

What's hot (20)

U-SQL Reading & Writing Files (SQLBits 2016)

U-SQL User-Defined Operators (UDOs) (SQLBits 2016)

Tuning and Optimizing U-SQL Queries (SQLPASS 2016)

The Road to U-SQL: Experiences in Language Design (SQL Konferenz 2017 Keynote)

U-SQL Meta Data Catalog (SQLBits 2016)

Modernizing ETL with Azure Data Lake: Hyperscale, multi-format, multi-platfor...

Survey of the Microsoft Azure Data Landscape

Dive Into Azure Data Lake - PASS 2017

Tips & Tricks SQL in the City Seattle 2014

Cloud architectural patterns and Microsoft Azure tools

A lap around microsofts business intelligence platform

Database Schema Management & Deployment using SQL Server Data Tools (SSDT)

SQL PASS BAC - 60 reporting tips in 60 minutes

Adf and ala design c sharp corner toronto chapter feb 2019 meetup nik shahriar

DMann-SQLDeveloper4Reporting

Not only SQL - Database Choices

Azure DocumentDB 101

Machine Learning Data Lineage with MLflow and Delta Lake

24 Hour of PASS: Taking SQL Server into the Beyond Relational Realm

Apache Spark in Industry

Viewers also liked

Killer Scenarios with Data Lake in Azure with U-SQL

U-SQL Query Execution and Performance Tuning

U-SQL Query Execution and Performance Basics (SQLBits 2016)

U-SQL Partitioned Data and Tables (SQLBits 2016)

U-SQL - Azure Data Lake Analytics for Developers

Azure Data Lake Intro (SQLBits 2016)

U-SQL Does SQL (SQLBits 2016)

U-SQL is the query language for big data analytics on the Azure Data Lake platform. This session will explore the unification of SQL and C# in this new query language, examples of combining data from external sources such as Azure SQL Database and Blob storage with Azure Data Lake store, creating and referencing assemblies, job submission and tools. The ADL platform will also be compared and contrasted to the HDInsight/Hadoop platform.

Hands-On with U-SQL and Azure Data Lake Analytics (ADLA)

Jason L Brugger

Big data is the new big thing where storing the data is the easy part. Gaining insights in your pile of data is something different. Based on a data dump of the well-known StackExchange websites, we will store & analyse 150+ GB of data with Azure Data Lake Store & Analytics to gain some insights about their users. After that we will use Power BI to give an at a glance overview of our learnings. If you are a developer that is interested in big data, this is your time to shine! We will use our existing SQL & C# skills to analyse everything without having to worry about running clusters.

Analyzing StackExchange data with Azure Data Lake

BizTalk360

Viewers also liked (9)

Killer Scenarios with Data Lake in Azure with U-SQL

U-SQL Query Execution and Performance Tuning

U-SQL Query Execution and Performance Basics (SQLBits 2016)

U-SQL Partitioned Data and Tables (SQLBits 2016)

U-SQL - Azure Data Lake Analytics for Developers

Azure Data Lake Intro (SQLBits 2016)

U-SQL Does SQL (SQLBits 2016)

Hands-On with U-SQL and Azure Data Lake Analytics (ADLA)

Analyzing StackExchange data with Azure Data Lake

Similar to U-SQL Learning Resources (SQLBits 2016)

December 2020 Microsoft 365 Need to Know Webinar

CIAOPS Need to Know Office 365 Webinar - December 2017

DBA Advanced - Hiram Fleitas - SQL ML / AI

Hiram Fleitas León

December 2022 Microsoft 365 Need to Know Webinar

September 2018 Office 365 Need to Know Webinar

Power BI February update with Recent Cool features

Vishal Pawar

December 2022 CIAOPS Need to Know Webinar

Microsoft 365

Suhail Jamaldeen

August 2020 Microsoft 365 Need to Know Webinar

In this session, Davide will explore Azure ML from the inside out. After a gentle approach on Machine Learning, we’ll see the Microsoft offering in this field and all the feature it offers, creating a simple yet 100% complete Machine Learning solution. We’ll start from something simple and then we’ll also move to some more complex topics, such as the integration with R and Python, IPython Notebook until the Web Service publishing and usage, so that we can integrate the created ML solution with batch process or even use it in real time with LOB application. All of this sound cool to you, yeah? Well it is, since with ML you can really give that “something more” to your customers or employees that will help you to make the difference. Guaranteed at 98.75%!

Azure ML: from basic to integration with custom applications

Davide Mauri

March 2022 CIAOPS Need to Know Webinar

- TITLE: Using Machine Learning and Python in SQL Server To Predict The Sentiment Speaker: Fleitas, Hiram - ABSTRACT: In this session, I'm very excited to show you from start to finish how to use Machine Learning to predict a sentiment in real-time with SQL Server (On-Premise). - AGENDA: 1. Add ML Features 2. Grant Access 3. Config 4. Install Pre-Trained & Open Source ML Models (DNN) 5. Code in Python and T-SQL 6. Python Profiling 7. Real-time scoring 8. Review Sentiment Results 9. Resources

Migrando aplicações para o Windows Azure

Lucas A. Romão

Microsoft Azure - News - October 2014

Daniel Toomey

淺談 Startup 公司的軟體開發流程 v2

Wen-Tien Chang

Agile startup company management and operation

Jiang Zhu

Microsoft Graph community call-November 2018

Microsoft 365 Developer

Microsoft Azure News - Aug 2016

Daniel Toomey

(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...

Hiram Fleitas León

January 2021 Microsoft 365 Need to Know Webinar

Microsoft Azure and Microsoft 365 - How Will They Help You

Suhail Jamaldeen

Similar to U-SQL Learning Resources (SQLBits 2016) (20)

December 2020 Microsoft 365 Need to Know Webinar

CIAOPS Need to Know Office 365 Webinar - December 2017

DBA Advanced - Hiram Fleitas - SQL ML / AI

December 2022 Microsoft 365 Need to Know Webinar

September 2018 Office 365 Need to Know Webinar

Power BI February update with Recent Cool features

December 2022 CIAOPS Need to Know Webinar

Microsoft 365

August 2020 Microsoft 365 Need to Know Webinar

Azure ML: from basic to integration with custom applications

March 2022 CIAOPS Need to Know Webinar

Migrando aplicações para o Windows Azure

Microsoft Azure - News - October 2014

淺談 Startup 公司的軟體開發流程 v2

Agile startup company management and operation

Microsoft Graph community call-November 2018

Microsoft Azure News - Aug 2016

(BI Advanced) Hiram Fleitas - SQL Server Machine Learning Predict Sentiment O...

January 2021 Microsoft 365 Need to Know Webinar

Microsoft Azure and Microsoft 365 - How Will They Help You

More from Michael Rys

Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...

Big Data Processing with .NET and Spark (SQLBits 2020)

Presentation by James Baker and myself on Running cost effective big data workloads with Azure Synapse and Azure Datalake Storage (ADLS) at Microsoft Ignite 2020. Covers Modern Data warehouse architecture supported by Azure Synapse, integration benefits with ADLS and some features that reduce cost such as Query Acceleration, integration of Spark and SQL processing with integrated meta data and .NET For Apache Spark support.

Running cost effective big data workloads with Azure Synapse and ADLS (MS Ign...

The presentation discusses how to migrate expensive open source big data workloads to Azure and leverage latest compute and storage innovations within Azure Synapse with Azure Data Lake Storage to develop a powerful and cost effective analytics solutions. It shows how you can bring your .NET expertise with .NET for Apache Spark to bear and how the shared meta data experience in Synapse makes it easy to create a table in Spark and query it from T-SQL.

Running cost effective big data workloads with Azure Synapse and Azure Data L...

Big Data Processing with Spark and .NET - Microsoft Ignite 2019

Bringing the Power and Familiarity of .NET, C# and F# to Big Data Processing ...

Building data pipelines for modern data warehouse with Apache® Spark™ and .NE...

When processing TB and PB of data, running your Big Data queries at scale and having them perform at peak is essential. In this session, we show you some state-of-the art tools on how to analyze U-SQL job performances and we discuss in-depth best practices on designing your data layout both for files and tables and writing performing and scalable queries using U-SQL. You will learn how to analyze performance and scale bottlenecks and will learn several tips on how to make your big data processing scripts both faster and scale better.

Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...

Big data processing increasingly needs to address not just querying big data but needs to apply domain specific algorithms to large amounts of data at scale. This ranges from developing and applying machine learning models to custom, domain specific processing of images, texts, etc. Often the domain experts and programmers have a favorite language that they use to implement their algorithms such as Python, R, C#, etc. Microsoft Azure Data Lake Analytics service is making it easy for customers to bring their domain expertise and their favorite languages to address their big data processing needs. In this session, I will showcase how you can bring your Python, R, and .NET code and apply it at scale using U-SQL.

Bring your code to explore the Azure Data Lake: Execute your .NET/Python/R co...

From theory to implementation - follow the steps of implementing an end-to-end analytics solution illustrated with some best practices and examples in Azure Data Lake. During this full training day we will share the architecture patterns, tooling, learnings and tips and tricks for building such services on Azure Data Lake. We take you through some anti-patterns and best practices on data loading and organization, give you hands-on time and the ability to develop some of your own U-SQL scripts to process your data and discuss the pros and cons of files versus tables. This were the slides presented at the SQLBits 2018 Training Day on Feb 21, 2018.

Best practices on Building a Big Data Analytics Solution (SQLBits 2018 Traini...

When analyzing big data, you often have to process data at scale that is not rectangular in nature and you would like to scale out your existing programs and cognitive algorithms to analyze your data. To address this need and make it easy for the programmer to add her domain specific code, U-SQL includes a rich extensibility model that allows you to process any kind of data, ranging from CSV files over JSON and XML to image files and add your own custom operators. In this presentation, we will provide some examples on how to use U-SQL to process interesting data formats with custom extractors and functions, including JSON, images, use U-SQL’s cognitive library and finally show how U-SQL allows you to invoke custom code written in Python and R. Slides for SQL Saturday 635, Vancouver BC presentation, Vancouver BC. Aug 2017.

U-SQL Killer Scenarios: Custom Processing, Big Cognition, Image and JSON Proc...

Data Lakes have become a new tool in building modern data warehouse architectures. In this presentation we will introduce Microsoft's Azure Data Lake offering and its new big data processing language called U-SQL that makes Big Data Processing easy by combining the declarativity of SQL with the extensibility of C#. We will give you an initial introduction to U-SQL by explaining why we introduced U-SQL and showing with an example of how to analyze some tweet data with U-SQL and its extensibility capabilities and take you on an introductory tour of U-SQL that is geared towards existing SQL users. slides for SQL Saturday 635, Vancouver BC, Aug 2017

Introduction to Azure Data Lake and U-SQL for SQL users (SQL Saturday 635)

U-SQL Killer Scenarios: Taming the Data Science Monster with U-SQL and Big Co...