Presentation delivered by Sajith C P, Integration Architect at the 2017 Global Integration Bootcamp, Bangalore. https://www.biztalk360.com/gib2017-india/#speakers[inline]/7/ In this session the speaker talked about ‘on-premises data gateway’ as a secure centralized gateway that can be used for accessing on premise data from various Azure Services. He took a deep dive on how it works, how to install and various methods to troubleshoot connectivity. He concluded the session with few demos of its use in Azure Logic App, Microsoft Flow, Power Apps and Power BI.

1. PRESENTS
MICROSOFT GTSC, Bengaluru March 25, 2017
Powered by Brought to you by
Sajith C P
Technical Architect – BizTalk, Azure App Services
Using on-premises data gateway to leverage on
premise data for hybrid scenarios

2. Powered by Brought to you by
• Sajith C P
• Integration Architect with close to 11 years of IT experience
primarily in BizTalk and Azure
• Personal Blog : https://absorbedinbiztalkthoughts.wordpress.com
2
Who am I?
@sajith_cp
https://in.linkedin.com/in/sajith-c-p-nair-86766649
sajith.cp.1729@gmail.com

3. Powered by Brought to you by
• Introduction to On-premises data gateway
• Understand multiple versions of data gateways
• Deep Dive – How does the gateway works?
• Things to consider before installing the gateway
• Troubleshooting On-premises data gateway – Tools of the trade,
Performance etc.
• Disaster Recovery and Gateway Restore
• Supported data sources
• Tips and Best Practices
3
Session Objectives and Key Takeaways

4. Powered by Brought to you by
In Computing terms, a gateway is a piece of device that connects two different networks. In
a similar fashion an on-premises data gateway,
Acts as a bridge between your cloud services and on-premises data source
Helps to access on premise data sources without having to deploy any custom code or
infrastructure
Enables secure access without any major change in network configuration
Provides a centralized mechanism to enterprise admins for greater control and visibility
Used by Logic Apps, Microsoft Flow, Power Apps, Power BI and Azure Analysis Services
(Preview)
4
Introducing On-Premises Data Gateway

5. Powered by Brought to you by
Microsoft created data gateways as a way to connect your cloud services to on premise. The confusion
is that there are number of different gateways for different azure services and some of them were
renamed multiple times.
Data Management Gateway
• Used previously by Power BI
• Currently used by Azure Data Factory and Azure Machine Learning
• The original or classic version
On-premises data gateway
• Formerly called as ‘Enterprise Gateway’ in Power BI
• Multiple users can share and reuse this gateway
• Used now by Logic Apps, Microsoft Flow, Power Apps, Power BI and recently by Azure Analysis
Services (preview)
• Supports schedule refresh and live query in Power BI
• Central monitoring and control
5
Data Gateway Confusion : Multiple Versions

6. Powered by Brought to you by
Personal Gateway
• Used only by ‘Power BI’
• Can be used by individuals without any admin configuration (cannot be shared between users)
• Supports only schedule refresh/refresh now
• No central monitoring/control
On-premises hybrid connection manager
• Used by Azure App Service (Web App, Mobile App and API Apps)
• Formerly used by Logic Apps
• Uses Azure BizTalk Services and Service Bus
• Included in Azure BizTalk Services
Future Roadmap of Data Gateways ?
• Could be to provide a unified gateway allowing reusability across all cloud services
Vote for this request for unified gateway at user feedback forum -
https://feedback.azure.com/forums/556165-azure-analysis-services/suggestions/16813687-create-a-
data-gateway-manager-to-be-used-for-all-c 6
Data Gateway Confusion : Multiple Versions..

7. Powered by Brought to you by
• Download and Install
• Sign in with your work account (e.g.,
nancy@contoso.onmicrosoft.com)
• Set up a new gateway, or migrate, restore, or take
over an existing gateway
• Configure connections (Power Apps, Flow), data
source (Power BI), connectors (Logic Apps)
• Consume from different services
Installs as,
• A ‘Windows Service’
• PBIEgwService (On-premises data
gateway service)
• A ‘Gateway Configuration Wizard’
• Managed by the cloud
• Azure Portal (Logic Apps)
• Office Portal (Flow, Power Apps, Power BI)
• REST API, Azure PowerShell 7
On-premises data gateway : Deep Dive

8. Powered by Brought to you by
Creds
How does the gateway works ?
DB

9. Powered by Brought to you by
Requirements
Minimum:
• . 64-bit version of Windows 7 or Windows Server 2008 R2 (or later)
• .NET 4.5 Framework
Recommended:
• 8 Core CPU
• 8 GB Memory
• 64-bit version of Windows Server 2012 R2 (or later)
Related Considerations
• Install the gateway on a local machine preferably over a wired network. You can't install the gateway
on a domain controller.
• Install the gateway on a computer that does not turn off, go to sleep, or doesn't connect to the
Internet.
• Use a work or school email address to register the gateway in Azure, so that you can associate the
on-premises data gateway with your Azure Active Directory-based account. Live account, like
@outlook.com does not work. 9
Things to consider before installing

10. Powered by Brought to you by
Configure Firewall
• Check if your gateway machine is behind a web proxy/firewall
• Gateway runs as a windows service and creates outbound connection to Azure Service Bus
• Communicates on outbound ports and no inbound ports required
10
Things to consider before installing..
Open Outbound Ports
in your firewall :
TCP 443 (default), 5671,
5672, 9350 through
9354
Whitelist the IP ranges
of your datacenter in
your firewall

11. Powered by Brought to you by
Configure Proxy
If you are behind a web proxy and if it allows outbound connection on port 80/443,
Restart the service after making the change.
Things to consider before installing..
This will change ‘ServiceBusSystemConnectivityModeString’ settings from ‘AutoDetect’ to ‘Https’
Location of the file - C:/Program Files/On-premises data gateway/Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config

12. Powered by Brought to you by
Authentication to Proxy Server : Using gateway configurator
• The default service account is the Service SID and not an authenticated domain user.
• You may change the service account of the gateway to allow proper authentication with your proxy.
12
Things to consider before installing..
• It will automatically restart the service.
• It will force you to register a new gateway or
restore the gateway. This will allow the new
service account to be able to decrypt stored
credentials for data sources.

13. Powered by Brought to you by
Authentication to Proxy Server : Manual change
 Change the service logon to a domain account manually
 Restart the gateway service after making the change
 Run the On-premises Data Gateway configurator. You can select the windows start button and
search for On-premises Data Gateway
 Sign in again with your work account
 Restore the current gateway using your recovery key. This will allow the new service account to be
able to decrypt stored credentials for data sources.
13
Things to consider before installing..

14. Powered by Brought to you by
demoHow to install on-premises data gateway?

15. Powered by Brought to you by
Connectivity
15
Troubleshooting and Tools of the Trade
• If TcpTestSucceeded is ‘False’ or you get a timeout then you are blocked by a firewall.
• The gateway uses a combination of FQDN and IP Address to establish an outbound connection.
Consider whitelisting IP ranges.
Update to latest version
• Microsoft comes up with an updated version almost every month
• A lot of issues can surface when the gateway is out of date. As a best practice, make sure to install the
latest version or if you have a version already installed then update.
• Update will always be an in place upgrade

16. Powered by Brought to you by
16
Troubleshooting and Tools of the Trade..
Installation Issues
Check Installer Logs -> %localappdata%TempOn-
premises_data_gateway_*.log
Configuration Issues
Check Configuration Logs ->
%localappdata%Microsofton-premises data
gatewayGatewayConfigurator*.log
Gateway Service Logs
C:UsersPBIEgwServiceAppDataLocalMicrosofton-
premises data gatewayGateway*.log
Export
Logs

17. Powered by Brought to you by
Event Logs
The On-premises data gateway service event
logs are present under Application and Services
Logs.
17
Troubleshooting and Tools of the Trade..
Refresh History in case of Power BI
Fiddler Trace
Monitor http traffic using Fiddler which is a free
tool from Telerik. This may show errors and
other information.

18. Powered by Brought to you by
Troubleshooting Articles
https://powerbi.Microsoft.com/en-us/documentation/powerbi-gateway-onprem-tshoot/
and many more..
Reach out to community or raise support with Microsoft
• For Power BI, you can use the communities site or you can create a support ticket.
• For PowerApps, you can use the communities site or you can create a support ticket.
• For Microsoft Flow, you can use the communities site of you can create a support ticket.
• For Logic Apps, you can submit a support ticket through the Azure portal.
18
Troubleshooting and Tools of the Trade..

19. Powered by Brought to you by
Performance Counters
• Use PerfMon to access counters specific to On-premises data
gateway
• General Grouping – ADO.NET, ADOMD, OLEDB, Mashup
• Check for,
 Open Connection Executed
 Open Connection Failed
 Query Executed
 Query Failed
 Items in Connection Pool
• There is no baseline. Monitor and identify your baseline during
load to understand when to trigger an alert.
• Other key counters are,
 Process Memory
 Overall system memory
 CPU utilization
19
Troubleshooting performance

20. Powered by Brought to you by
Additional logging to output queries and their timings
20
Troubleshooting performance..
• This will enable the EmitQueryTraces value from False to True in the C:Program FilesOn-premises
data gateway/Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config
• This will change the TraceVerbosity value from 4 to 5 in the C:Program FilesOn-premises data
gateway/ Microsoft.PowerBI.DataMovement.Pipeline.Diagnostics.dll.config file. Changing this setting will
begin to log verbose entries to the gateway log. This includes entries that show duration.
• Search for ‘FireActivityCompletedSuccessfullyEvent’ entry with the duration.

21. Powered by Brought to you by
• Gateway admin can use the recovery key to restore a gateway to a different machine. Gateway is
tied up with the id used to register and not to the machine where it is installed.
• Once a gateway is restored , all data sources and credentials will continue to work through the new
gateway.
• Restored gateway will have the same name, so no need to re-publish
21
Disaster Recovery and Restore

22. Powered by Brought to you by
Logic Apps
• BizTalk Server, DB2, File System, Informix,, MQ, Oracle DB, SAP (Application and Message Server),
SQL Server, SharePoint
Power Apps & Flow
• DB2, File System, SQL Server, SharePoint, Oracle Database, Informix
Power BI
• Analysis Services, SQL, SAP HANA, Oracle, Teradata, File, Folder, SharePoint List, Web, OData, DB2,
MySQL, Sybase, SAP BW, Informix, ODBC
22
Supported data sources

23. Powered by Brought to you by
Logic Apps
Sign up for azure subscription and start using on-premises data gateway. Start with Rs 13,300 credit
for one month
Power Apps
Access to On-premises data gateway is available to Power Apps for Office 365 and Dynamics 365,
PowerApps Plan 1 and Power Apps Plan 2 users. Free trial to PowerApps plan 1 and plan 2 for 90 days.
Flow
Access to On-premises data gateway is available to Flow for Office 365 and Dynamics 365, Flow Plan 1
and Flow Apps Plan 2. Free trial to Flow plan 1 and plan 2 for 90 days.
Power BI
On-premise data gateway is available only for Pro Power BI users and not included with Power BI Free.
60 day trial available.
23
Licensing

24. Powered by Brought to you by
• Monthly updates : Always update to the latest version
• Keep it always on and credentials up to date
• Keep the restore key in a safe place
• Install the gateway as close to the data source as possible to reduce latency. If you can install on the
data source itself it will improve performance (Not always possible)
• Consider data centers. For example, if your Power BI service uses the West US datacenter, and you
have SQL Server hosted in an Azure VM, your Azure VM should be in the West US too. This
proximity minimizes latency and avoids egress charges on the Azure VM.
• Ensure good network bandwidth. Again depends on the volume of data being exchanged. Use
http://azurespeedtest.azurewebsites.net/ to gauge your latency.
• Data gateway must be in the same LOCATION as the Logic App in order to appear in the Gateway
drop-down list of the connector - although it can be in a different resource group.
24
Tips and Best Practices

25. Powered by Brought to you by
• Matching your Power BI Desktop connections with a gateway. The connection string configured in
the Power BI Desktop file needs to match a data source already configured in the Power BI service.
When a match is found, it will use the gateway that the data source belongs to.
• Side by side installation of gateways. You can have a personal, on-premises and data management
gateway installed on the same machine side by side.
• Power BI data sources can only be used by Power BI. The connections that you set up in Power Apps
can be used by Flow and Vice versa. Logic app connectors can be used by Logic apps alone.
• Azure management portal does not provide an option to delete gateway. This can be done only
using REST API or Azure PowerShell at the moment.
25
Tips and Best Practices..

26. Powered by Brought to you by
demoUsing on-premises data gateway with Logic Apps

27. Powered by Brought to you by
demoUsing on-premises data gateway with Flow and Power Apps

28. Powered by Brought to you by
demoUsing on-premises data gateway with Power BI

29. Powered by Brought to you by
resources

30. Powered by Brought to you by
• Download - https://www.microsoft.com/en-
us/download/details.aspx?id=53127https://powerbi.microsoft.com/en-us/documentation/powerbi-
gateway-onprem/
• Gateway in depth - https://powerbi.microsoft.com/en-us/documentation/powerbi-gateway-
onprem-indepth/
• Troubleshoot - https://powerbi.microsoft.com/en-us/documentation/powerbi-gateway-onprem-
tshoot/
• https://powerbi.microsoft.com/en-us/documentation/powerbi-admin-troubleshooting-power-bi-
personal-gateway/
• Hybrid Connection Manager - https://docs.microsoft.com/en-us/azure/app-service-web/web-
sites-hybrid-connection-get-started
• Flow - https://flow.microsoft.com
• Power BI - https://powerbi.microsoft.com
• Power Apps - https://web.powerapps.com
• Azure Management Portal - https://portal.azure.com
• Logic App Connectors - https://docs.microsoft.com/en-us/azure/connectors/apis-list
30
Resources

31. Powered by Brought to you by
• Introduction to On-premises data gateway
• Understand multiple versions of data gateways
• Deep Dive – How does the gateway works?
• Things to consider before installing the gateway
• Troubleshooting On-premises data gateway – Tools of the trade,
Performance etc.
• Disaster Recovery and Gateway Restore
• Supported data sources
• Tips and Best Practices
31
In Review : Session Objectives and Key
Takeaways

32. Powered by Brought to you by
questions