From idea to production in a day – Leveraging Azure ML and Streamlit to build...
Pacific Research Platform Supporting Earth Sciences
1. “Pacific Research Platform
Supporting Earth Sciences”
Briefing to The Quilt Visit
to Calit2’s Qualcomm Institute
University of California, San Diego
February 8, 2017
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
http://lsmarr.calit2.net
1
2. The Pacific Research Platform’s Second Year:
a Working End-to-End Science-Driven DMZ-Connector
NSF CC*DNI Grant
$5M 10/2015-10/2020
PI: Larry Smarr, UC San Diego Calit2
Co-Pis:
• Camille Crittenden, UC Berkeley CITRIS
• Tom DeFanti, UC San Diego Calit2
• Philip Papadopoulos, UC San Diego SDSC
• Frank Wuerthwein, UC San Diego Physics SDSC
Science Teams:
• Visualization and Virtual Reality
• Biomedical
• Earth Sciences
• Particle Physics
• Astronomy and Astrophysics
• Cryo-EM
• Deep Learning & Robotics
• High-Performance Wireless
3. PRP Continues to Expand Rapidly While Increasing Connectivity:
One Year of Progress – 12 Sites to 20 Sites
January 29, 2016 December 15, 2016
4. PRP Workshop Held in Collaboration with UC-Wide Research IT
May 1, 2016
• 45 Attendees
– Ten UC Campuses
– Presentations from
– Prof. Mike Norman, Dept. of Physics, UC San Diego
– Dr. Frank McKenna, Chief Information Officer, PEER, UC Berkeley
– Prof. Ben Porter, Acting Director, Phoebe A. Hearst Museum of Anthropology, UC Berkeley
5. PRP Will Link the Laboratories of
the Pacific Earthquake Engineering Research Center
http://peer.berkeley.edu/
8. Dan Cayan
USGS Water Resources Discipline
Scripps Institution of Oceanography, UC San Diego
much support from Mary Tyree, Mike Dettinger, Guido Franco and other colleagues
NCAR Upgrading to 10Gbps Link Over Westnet
from Wyoming and Boulder to CENIC/PRP
Sponsors:
California Energy Commission
NOAA RISA program
California DWR, DOE, NSF
Planning for climate change in California
substantial shifts on top of already high climate variability
UCSD Campus Climate Researchers Need to Download
Results from NCAR Remote Supercomputer Simulations
to Make Regional Climate Change Forecasts
9. average summer
afternoon temperature
average summer
afternoon temperature
Downscaling Supercomputer Climate Simulations
To Provide High Res Predictions for California Over Next 50 Years
9
Source: Hugo Hidalgo, Tapash Das, Mike Dettinger
10. The Future of Supercomputing Will Blend Traditional HPC and Data Analytics
Integrating Non-von Neumann Architectures
“High Performance Computing Will Evolve
Towards a Hybrid Model,
Integrating Emerging Non-von Neumann Architectures,
with Huge Potential in Pattern Recognition,
Streaming Data Analysis,
and Unpredictable New Applications.”
Horst Simon, Deputy Director,
U.S. Department of Energy’s
Lawrence Berkeley National Laboratory
11. Calit2’s Qualcomm Institute Has Established a Pattern Recognition Lab
For Machine Learning on non-von Neumann Processors
“On the drawing board are collections of 64, 256, 1024, and 4096
chips.
‘It’s only limited by money, not imagination,’ Modha says.”
Source: Dr. Dharmendra Modha
Founding Director, IBM Cognitive Computing Group
August 8, 2014
UCSD ECE Professor Ken Kreutz-Delgado Brings
the IBM TrueNorth Chip
to Start Calit2’s Qualcomm Institute
Pattern Recognition Laboratory
September 16, 2015
12. New Brain-Inspired Non-von Neumann Processors Are Emerging:
KnuEdge Has Provided Processor to Calit2’s PRL
www.tomshardware.com/news/knuedge-announces-knuverse-and-knupath,31981.html
www.calit2.net/newsroom/release.php?id=2704
“KnuEdge and Calit2
have worked together
since the early days of
the KnuEdge LambdaFabric
processor, when key
personnel and technology
from UC San Diego
provided the genesis for
the first processor design.”
www.calit2.net/newsroom/release.php?id=2726
June 6, 2016
13. Building a Cognitive Hardware and Software Ecosystem
On the Pacific Research Platform
• Working With 30 CSE Machine Learning Researchers
– Goal is 320 game GPUs in 32-40 FIONAs at 10 PRP campuses
– PRP Couples FIONAs with GPUs into a Condor-Managed Cloud
• PRP Access to Emerging Processors
– IBM TrueNorth, KnuEdge, FPGA, and Qualcomm Snapdragon
• Software Including a Wide Range of Open ML Algorithms
• Metrics for Performance of Processors and Algorithms
Source: Tom DeFanti, Calit2
Multiple Proposals Under Review
14. Our Support:
• US National Science Foundation (NSF) awards CNS 0821155 and
CNS-1338192, CNS-1456638, ACI-1540112, and ACI-1541349
• University of California Office of the President CIO
• UCSD Chancellor’s Integrated Digital Infrastructure Program
• UCSD Next Generation Networking initiative
• Calit2 and Calit2 Qualcomm Institute
• CENIC, PacificWave and StarLight
• DOE ESnet