Presented at the CHPC conference in Johannesburg, South Africa, on December 5, 2019.

1. Globus: A Data Management Platform
for Collaborative Research
Vas Vasiliadis
vas@uchicago.edu
CHPC National Conference
December 5, 2019

2. Globus is …
a non-profit service
developed and
operated by

3. Our mission is to…
increase the efficiency and
effectiveness of data-driven
science and scholarship
through sustainable software

4. Development is funded by...
U.S. DEPARTMENT OF
ENERGY

5. …and operations are supported by subscribers

6. 6460
active shared
endpoints
120
subscribers
768PB
moved
25519
active personal
endpoints
101 billion
files processed
1720
active server
endpoints
80
2.9PB
largest single
transfer to date
99.9%
availability
862
identity providers
1960
most shared
endpoints
at a single
institution 122,000
registered users
Broad and growing adoption…
countries
with Globus
endpoints

7. What can Globus do
for researchers?

8. 8
Research Computing HPC
Desktop Workstations
Mass Storage Instruments
Personal Resources
National Resources
Unify data access across storage tiers…
Cloud Storage

9. Public / private cloud stores
Campus
storage
Project
repositories,
replication stores
Public repositories
…simplify sharing with collaborators…

10. Analysis
store
Next-Gen Sequencer
MRI
Advanced Light Source
Personal system
Remote visualization
Light Sheet Microscope
High-durability,
low-cost archive
…help researchers manage instrument data…
Cryo-EM

11. …and enable building data-centric applications
11

12. Common Use Cases

13. Fast, reliable file transfer …from any to any system
User-initiated,
or automated
transfer request
1
Instrument,
Lab server
Compute
Facility
Globus transfers files
reliably, securely
2
Globally accessible
multi-tenant service
• Fire-and-forget transfers
• Optimized speed
• Assured reliability
• Unified view of storage
• Browser, REST API, CLI
Optional
notifications
3

14. Globus hybrid SaaS architecture

15. Secure data sharing …from any storage
Collaborator logs into Globus
and accesses shared files;
no local account required;
download via Globus2
On-prem or public
cloud storage
Select files to share,
select user or group,
and set access
permissions
1Globally accessible
multi-tenant service
Globus controls access to
shared files on existing storage;
no staging of data required
Laptop, server,
compute facility

16. Sharing via Globus
User managed ”overlay”
permissions stored on
Globus service

17. Automated instrument data egress
Cryo EM
Lightsheet
Sequencer
ALS/APS
….
Local system
download
Remote analysis,
visualization
Local
policy
store
--/cohort045
--/cohort096
--/cohort127

18. Repository data distribution
Bulk data
transfer
2
Search, request
data of interest
1
2
Browser
based
download
Globally accessible
multi-tenant service
2

19. --/run123/output (r)
Output data staged
with access control
2
Data staging for compute
Compute service
3
User accesses,
downloads results
--/run123/input (rw)
1
Input data
upload

20. …makes your
storage system a
Globus endpoint

21. Planned/In development
Storage connectors - globus.org/connectors

22. Globus Connect Personal
• Installers do not require admin access
• Zero configuration; auto updating
• Handles NATs

23. Demonstration

24. Globus sustainability model
• Free Tier (transfer only)
• Standard Subscription
– File sharing
– HTTPS support
– Management console, usage reports
– Application integrated support
– Priority support
• Branded Web Site
• Premium Storage Connectors
• Alternate Identity Provider
• High Assurance Subscriptions
24

25. Standard Globus security features
• Access Control
– Identities provided and managed by institution
– Institution controls all access policies
– Globus is identity broker; no access to/storage of user credentials
• Data remain at institutions, not stored by Globus
• Integrity checks of transferred data
• High availability and redundancy
• Encryption of user files and Globus control data

26. High Assurance features for PHI, CUI
• Additional authentication assurance
– Reauthentication after specified time period
– Authenticates with the specific identity within session
• Isolation of applications
– Authentication context is per application, per session
• Enforced encryption of data in transit
• Local audit logging

27. Secure operations
• Intrusion detection and prevention
• Encryption
• Logging
• Secure remote access, access control
• Uniform configuration management and change control
• AWS best practices for securing operating environment:
VPCs, security groups, identity and access management
• Comply with HIPAA, NIST SP800-171, NIST SP800-53

28. Demonstration

29. globus.org
docs.globus.org
support@globus.org
outreach@globus.org
globus.org/mailing-lists