The document discusses measuring high performance computing (HPC) systems by assessing performance, costs, and value. Measuring performance involves using benchmarks to evaluate system speed and code scaling. Measuring costs is complex and includes factors like hardware, facilities, staffing, software, and more. Measuring value is difficult and subjective, going beyond simple return on investment calculations. Overall, measuring HPC can provide important insights if done comprehensively and effectively.
Ensuring Technical Readiness For Copilot in Microsoft 365
Measuring HPC: Performance, Cost, & Value
1. Experts in numerical software and
High Performance Computing
Measuring HPC:
Performance, Cost & Value
IDC HPC User Forum
Austin, TX
6-8 September 2016
Andrew Jones
Vice-President Strategic HPC Services
3. 3High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Instead, use your time to
optimize benefit from those decisions make sure the next decision is right
Weak idea
Don’t waste your valuable time justifying past decisions
“Justify the investments made”
4. 4High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Optimize usage to maximize
science/engineering impact and business value
Where possible, “fairness” innovation vs. production?
Understand how the current HPC capability is used
6. 6High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Use data on value delivered by
current HPC capability
so future investments can be
made with confidence
at the right scale, type and
timing for maximum value
7. 7High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Make best use of current capability
Make best decisions for the future capability
9. 9High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Benchmarks
“How fast is my
code?”
“How fast is this
supercomputer?”
“Which
system/processor
is faster?”
“How does my
code scale?”
11. 11High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Drive value for money + technical confidence
Guide
procurement
planning
Support
procurement
evaluation
Acceptance
testing
12. 12High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
0
2
4
6
8
10
12
14
A B C D E F G
Performance
0
2
4
6
8
10
12
14
A B C D E F G
Performance
13. 13High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
How well is the HPC facility/service run?
How well is it used?
How fair is it?
Cost, business impact, … [in next section]
17. 17High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
How effectively do teams work together?
Productivity and happiness of users, service staff, etc.
Organizational structures
Governance
Etc.
18. 18High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
… but also affects everyone
Performance becomes a discipline in its own right …
Lots of things to measure!
Not just code speed
20. 20High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Computer Storage Network Staff Electricity
Cooling Facility costs Security Time (!) Resilience
Measurement &
Reporting
Financing effects Insurance
Irrecoverable
taxes
Procurement
project
Application
porting
Software licenses
User training &
support
Documentation Commissioning
Warranty,
support,
maintenance
Decommissioning
Availability /
throughput
Data migration … and more …
21. 21High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
?
Which costs
can you
control?
Which costs
does your
budget bear?
How are you
incentivized
against other
costs?
How will costs
change over
time?
22. 22High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Total Cost of Ownership
(TCO)
• What are all the costs
involved in owning and
operating this HPC service?
• Most important to HPC
owners/managers
• It costs money even if not
used (always)
Total Cost of Solution
(TCS)
• What are all the costs I will
incur using this HPC service
for a specific project?
• Most important to HPC
users
• It costs nothing if not used
(sometimes)
23. 23High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Much more complicated than just price of hardware …
Which costs can be controlled?
TCO vs. TCS
27. 27High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
ROI is only part of the answer
Value to the business might be binary:
project/product not possible without HPC
Determining value can be hard & sometimes subjective
29. 29High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Measuring HPC can be very powerful
for the business and the user when done well
Measuring performance, costs and value
are all complex and can be hard to get right
Measurement is only the start …
effective reporting can be even more important!
NAG HPC consulting can help you
30. 30High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Experts in High Performance Computing,
Algorithms and Numerical Software Engineering
www.nag.com | blog.nag.com | @NAGtalk
@hpcnotes
Please come to our tutorial at SC16 in Austin this November:
Essential HPC Finance Practice – TCO, cost recovery models, etc.