This document summarizes a presentation on Azure Data Factory (ADF) given by Cathrine Wilhelmsen. The presentation provided an overview of ADF and how it compares to SQL Server Integration Services (SSIS). It demonstrated how to lift and shift existing SSIS packages to ADF and how to map data flows between different data stores using ADF data flows. The presentation concluded with lessons learned and encouraged attendees to ask any remaining questions.

1. Cathrine Wilhelmsen, Senior BI Consultant, Inmeta
Moderated By: Giuliana Grecco
Pipelines and Packages:
Introduction to
Azure Data Factory

2. If you require assistance
during the session, type
your inquiry into the
question pane on the right
side.
Maximize your screen with
the zoom button on the
top of the presentation
window.
Please fill in the short
evaluation following the
session. It will appear in
your web browser.
Technical Assistance

3. Thank you to our Presenting Sponsors

4. PASS’ flagship event
November 5-8
Seattle, Washington
Free 1-day local
training events
Local user groups
around the world
Online special interest
user groups
Business analytics
training
Get involved
Explore
everything
PASS has
to offer
Free Online Resources
Newsletters
PASS.org

5. Cathrine Wilhelmsen
Senior BI Consultant, Inmeta
Work Things
Senior BI Consultant and Microsoft Data
Platform Tech Lead in Inmeta, Norway.
Specialties: ADF, SSIS, Biml and T-SQL
Geeky Things
Fun Things
Active in the PASS community as a speaker,
blogger and chronic volunteer.
Likes: coffee, chocolate and cat gifs :)
/cathrinewilhelmsen
@cathrinew
cathrinew.net
hi@cathrinew.net

6. Cathrine Wilhelmsen, Senior BI Consultant, Inmeta
Moderated By: Giuliana Grecco
Pipelines and Packages:
Introduction to
Azure Data Factory

7. Past Learnings
and Future Visions

8. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
10 years ago…

9. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
SSIS
SQL Server Integration Services

10. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)

11. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Then…

12. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
ADF v1
Azure Data Factory Version 1

13. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)

14. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)

15. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Today…

16. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
ADF v2
Azure Data Factory Version 2

17. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)

18. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Why?
How?
What?

19. Azure
Data Factory

20. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Azure Data Factory

21. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Data Movement and Integration
Azure Data Factory
Pipelines
Activities
Datasets
Data Flows Triggers
Integration RuntimesLinked Services
Templates

22. DEMO
Azure
Data Factory

23. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Wait…
I already have
thousands of SSIS
packages!

24. SSIS
Lift and Shift

25. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Why Lift and Shift SSIS?

26. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
1.
2.
3.
4.
How to Lift and Shift SSIS

27. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
SSIS Integration Runtime

28. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
https://sqlbits.com/Sessions/Event18/Deeper_Integration
_and_New_Transformation_for_SSIS_in_ADF
SSIS Integration Runtime Deep Dive

29. DEMO
SSIS
Lift and Shift

30. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
SSIS Lift and Shift: Lessons Learned
like all VMs

31. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
≈
≈
≈
≈
≈
≈
ADF vs SSIS

32. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Pipeline
Linked Service
Source
Sink
Activity
Data Flow
≈
≈
≈
≈
≈
≈
Package
Connection Manager
Source
Destination
Control Flow Task
Data Flow
ADF vs SSIS

33. Mapping
Data Flows

34. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Mapping Data Flows

35. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Mapping Data Flows Example

36. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Kamil Nowinski's Cheat Sheet
github.com/SQLPlayer/CheatSheets/blob/master/ADFDF-Cheat-Sheet-sqlplayer.pdf

37. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Schema Drift

38. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Schema Drift in SSIS

39. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Schema Drift in ADF

40. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Oh no!

41. DEMO
Mapping
Data Flows

42. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Lessons Learned

43. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net)
Good luck!
Thank you :)

44. Questions?

45. Thank you for attending!
@sqlpass
#sqlpass
@PASScommunity
Learn more from Cathrine Wilhelmsen
@cathrinew hi@cathrinew.net