Presentation given at RAMS 2013 Three-phase inverters are physically large, complex and expensive elements of major solar power generation systems. The inverter converts DC power created by the photovoltaic (PV) panels to AC power suitable for adding to the power grid. The inverters’ reliability testing is a complex task and relies on reliability block diagrams (RBD), vendor and field data, plus selecting accelerated life tests (ALT) based on critical elements of the product. This paper illustrates a case study that developed an RBD, used field and vendor data, and includes the design and use of two ALTs. The result is a working framework or model that provides a reasonable estimate of the expected lifetime performance of the inverter. While any project similar to this, is always a work in progress, the examination of the decisions and inputs for the model proves valuable for the continued improvement of the model and resulting life predictions. This project provides an excellent real life example of reliability estimation having a multitude of constraints including: sample size, test duration, and field data, thus having to rely on all sources of available data starting from field and vendor data to theoretical component reliability calculations, ALT plan execution, failure analysis, and finally summarizing the results using RBD to estimate product expected lifetime. At the time of writing this paper, based on completion of system level ALT, an availability of 99.97% is valid over a 10 year period according to southern Ontario weather as the main installation base. This will be revisited once subsystem ALT is completed.

1. PRESENTATION ON
DFR ROI: Calculating ROI
When Implementing a DFR
Program
BY
Mike Silverman, CRE, Ops A La Carte LLC

2. Agenda
• Background/Introduction 5 min
• Reliability Program Assessment 15 min
• DFR ROI Calculator 15 min
• Example 10 min
• Summary 5 min
• Questions 10 min
© Ops A La Carte LLC 2012

3. Background
• Last year we presented a paper on Design for
Reliability (DFR), reviewing the benefits of a
good DFR program and included some of the
essential building blocks of DfR along with
pointing out some erroneous practices that
people today are using today.
© Ops A La Carte LLC 2012

4. Background (cont’d)
• We discussed a good DFR Program having the
following attributes:
– Setting Goals at the beginning of the program and
then developing a plan to meet the goals.
– Having the reliability goals being driven by the
design team with the reliability team acting as
mentors.
– Providing metrics so that you have checkpoints on
where you are against your goals.
– Writing a Reliability Plan (not only a test plan) to
drive your program.
© Ops A La Carte LLC 2012

5. Background (cont’d)
• A Good DFR Program must choose the best
tools from each area of the product life cycle
– Identify
– Design
– Analyze
– Verify
– Validate
– Monitor and Control
• The DFR Program must then integrate the tools
together effectively
© Ops A La Carte LLC 2012

6. Introduction
• Today we will calculateathe Return on
developed to
discuss method we have
Investment (ROI) from a Design for
Reliability (DFR) program, also known as
the DFR ROI.
 The DFR ROI is the cornerstone of our
new Ounce of Prevention Solution
This is the OPS in Ops A La Carte!
© Ops A La Carte LLC 2012

7. Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012

8. Calculating Improvement in
Warranty
• To calculate your Improved Warranty Rate,
you will need to determine both your
current Reliability Maturity Level as well as
your desired Reliability Maturity Level.
• To determine these, you will need to
conduct a Reliability Program Assessment
(RPA).
© Ops A La Carte LLC 2012

9. RELIABILITY
PROGRAM
ASSESSMENT
© Ops A La Carte LLC 2012

10. Reliability Program Assessment
© Ops A La Carte LLC 2012

11. Assessment Motivation
• Identify systemic changes that impact
reliability
– Tie into culture and product
– Both enjoy benefits
• Provides roadmap for activities that
achieve results
– Matching of capabilities and expectations
– Cooperative approach
• Save Money (ROI)
© Ops A La Carte LLC 2012

12. When to perform an
Assessment?
• Entering into a new market
• Product reliability is below target level
• New personnel
• New technology
• Product design > 50% different than
previous.
© Ops A La Carte LLC 2012

13. Steps Involved in an
Assessment
• Selecting People to Survey
• Selecting Survey Topics (from IEEE1624)
• Develop a Scoring System
• Results and Meaning
• Data Analysis/Reliability Maturity Matrix
(our new Matrix is aligned with IEEE1624)
© Ops A La Carte LLC 2012

14. Selecting People to Survey
• Hardware Engineers
• Software Engineers
• Test Engineers
• Manufacturing Engineers
• Quality/Reliability Engineers
• Program Management
• Procurement
• others?
© Ops A La Carte LLC 2012

15. Survey Topics
1. Do you have a Reliability Plan? Reliability Goals?
2. Is reliability an integral part of strategic business planning?
3. Do reliability requirements exist for all engineering projects?
4. Is there a reliability plan for each engineering project?
5. What training relevant to reliability is provided to employees?
6. What reliability analysis is conducted?
7. Are design rules used effectively to avoid known failures?
8. How effective is testing?
9. How are suppliers managed
10. How is parts selection managed?
11. How is parts supply managed?
12. How effective is failure data tracking and analysis?
© Ops A La Carte LLC 2012

16. Develop a Scoring System
Scoring:
4 = 100%, top priority, always done
3 = >75%, use normally, expected
2 = 25% - 75%, variable use
1 = <25%, only occasional use
0 = not done or discontinued
- = not visible, no comment
© Ops A La Carte LLC 2012

17. Example Survey Topic/Score
To what extent is FMEA used?
 Design Engineer Response: Used only as a
troubleshooting tool
 Score = 1

Manufacturing Engineer Response: Commonly used on
critical design elements
 Score = 3
 Reliability Engineer Response: Always used on all
products
 Score = 4
•Results: Score 2.6
•Comment: Clearly a disconnect between
reliability and design engineering – indicative
of a problem with the tool.
© Ops A La Carte LLC 2012

18. Results & Meaning
• Looking for trends, gaps in process, skill
mismatches, over analysis, under analysis.
• Looking for differences across the
organization, pockets of excellence, areas
with good results
• Process provides snapshot of current system
• No one tool make an entire reliability
program. The tools need to match the needs
of the products and the culture.
• Check step is critical before moving to
recommendation around improvement plan
© Ops A La Carte LLC 2012

19. Summary of Past Assessments
• What Companies Are
Weak at
– Goal setting/Planning
– Repair/warranty invisible
– Lessons learned capture
– Single owner of product
reliability
– Multiple defect tracking
systems
– Reliability Integration
– Statistics
© Ops A La Carte LLC 2012

20. Reliability Maturity Matrix
• 5 levels of maturity
• Based on IEEE 1624: “IEEE Standard for
Organizational Reliability Capability”
• Similar to Crosby’s Quality Maturity
• On the following page is a matrix based on
Crosby’s as an example.
• Read across each row and find the statement
that seems most true for your organization.
• The center of mass of the levels is the
organization’s overall level.
© Ops A La Carte LLC 2012

21. Reliability Maturity Matrix
ATTRIBUTES Stage 1: Stage 2: Awakening Stage 3: Stage 4: Wisdom Stage 5: Certainty
Uncertainty Enlightenment
1.1 Understanding No comprehension of Recognizing that reliability Still learning more about reliability Participating. Understand absolutes of Consider reliability management an essential
reliability as a management may be of value but not management. Becoming supportive reliability management. Recognize their part of company system.
& Attitude management tool. Tend willing to provide money or time to and helpful. personal role in continuing emphasis.
to blame engineering make it happen.
for ‘reliability problems’
1.2 Status Reliability is hidden in A stronger reliability leader appointed, Reliability manager reports to top Reliability manager is an officer of Reliability manager is on board of directors.
1. Management
manufacturing or yet main emphasis is still on an audit management, with role in management company; effective status reporting and Prevention is main concern. Reliability is a
engineering of initial product functionality. of division. preventive action. Involved with consumer thought leader.
departments. Reliability Reliability testing still not performed. affairs.
testing probably not
done. Emphasis on
initial product
functionality.
1.3 Measured Cost Not done other than Direct warranty expenses only Warranty, corrective action materials Customer and lifecycle unreliability costs Lifecycle cost reduction done through product
anecdotally and engineering costs monitored determined and tracked reliability improvements
of Unreliability
2.1 Requirement Informal or nonexistent Basic requirements based on Requirements include environment and Plans are tailored for each project and Contingency planning occurs. Decisions
2. Product
customer requirements or standards. use profiles. Some apportionment. projected risks. Use of distributions for based on business or market considerations.
& Planning
Reqts.
Plans have required activities. Plans have more details with regular environmental and use conditions. Part of strategic business plan.
reviews.
2.2 Training & Informally available to Select individuals trained in concepts Training for engineering community for Reliability and statistics courses tailored New technologies and reliability tools tracked
some, if requested and data analysis. Available training key reliability related processes. for design and manufacturing engineers. and training adjusted to accommodate.
Development for design engineers Managers trained on reliability and Senior managers trained on reliability Reliability training actively supported by top
lifecycle impact. impact on business. management.
3.1 Analysis Nonexistent or solely Point estimates and reliance on Formal use of FMEA. Field data analysis Predictions are expressed as distributions Lifecycle cost considered during design.
based on handbook parts count methods. Basic of similar products used to adjust and include confidence limits. Stress and damage models created and
3. Engineering
manufacturing issues identification and listing of failure predictions. Design changes cause re- Environmental and use conditions used for used. Extensive risk analysis for new
modes and impact evaluation of product reliability simulation and testing. technologies.
3.2 Testing Primarily functional Generic test plan exists with reliability Detailed reliability test plan with sample Accelerated tests and supporting models Test results used to update component stress
testing only to meet customer or size and confidence limits. Results used used. Testing to failure or destruct limits and damage models. New technologies
standards specifications for design changes & vendor evals. conducted characterized.
3.3 Supply Chain Selection based on Approved vendor list maintained. Assessments and audit results used to Vendor selection includes analysis of Changes in environment, use profile, or
function and price Audits based on issues or with critical update AVL. Field data and failure vendor’s reliability data. Suppliers conduct design, trigger vendor reliability assessment.
Management parts. Qualification primarily based on analysis related to specific vendors. assessments and audit of their suppliers. Component parameters and reliability
vendor datasheets. monitored for stability
4.1 Failure Data Failures during function Pareto analysis of field return and Root cause analysis used to update Focus is on failure mechanisms. Failure Customer satisfaction relationship to product
testing may be internal testing. Failure analysis relies AVL and prediction models. Summary distribution models updated based on failures is understood. Use of prognostic
Tracking & addressed on vendor support. of analysis results disseminated. failure data methods to forestall failure.
Analysis
4.2 Validation & Informal and based on Basic verification that plans are Supplier agreements around reliability Internal reviews of reliability processes Reliability predictions match observed field
4. Feedback
individuals rather than followed. Field failure data regularly monitored. Failure modes regularly and tools. Failure mechanisms regularly reliability.
Verification
Process
process reported. monitored. monitored and used to update models and
test methods
4.3 Reliability Nonexistent or informal Design and process change followed. Effectiveness of corrective actions Identified failure mechanisms addressed New technologies evaluated and adopted to
Corrective action process includes tracked over time. Identified failure in all products. Advanced modeling improve reliability. Design rules updated
Improvements internal and vendor engagement. modes addressed in other product. techniques explored and adopted. Formal based on field failure analysis.
Improvement opportunities identified as and effective lessons learned process
environment and use profiles change. exists.
5. Prevailing “We don’t know why we “Is it absolutely necessary to always “Through commitment and reliability “Failure prevention is a routine part of our “We know why we do not have problems with
have problems with have problems with reliability?” improvement we are identifying and operation.” reliability.”
Sentiment reliability” resolving our problems.”

22. Reliability Maturity Matrix
(cont’d)
• Lets look at one row to get a better understanding
ATTRIBUT Stage 1: Stage 2: Stage 3: Stage 4: Stage 5:
E Uncertainty Awakening Enlighten- Wisdom Certainty
ment
1.1 Under- No Recognizing Still learning Participating. Consider
standing comprehen- that reliability more about Understand reliability
& sion of management reliability absolutes of manage-
Attitude reliability as a may be of management reliability ment an
management value but not Becoming management essential
tool. Tend to willing to supportive Recognize part of
blame provide and helpful. their company
engineering money or personal role system.
for ‘reliability time to make in continuing
problems’ it happen. emphasis.
© Ops A La Carte LLC 2012

23. Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012

24. Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012

25. Warranty vs. Maturity by
Industry
% of Revenue
SECTOR Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Automotive Manufacturer 9.2 4.6 2.3 1.15 0.58
Auto Parts Suppliers
Aerospace and Marine
Computers
Telecom 4.8 2.4 1.2 0.6 0.3
Semiconductor
Consumer Electronics
Medical and Scientific 2.8 1.4 0.7 0.35 0.18
Data Storage
PC Peripherals
Appliances and HVAC
Homebuilders
Building Materials
Power Generation
Some of above data courtesy of
Eric Arnum, Warranty Week
© Ops A La Carte LLC 2012

26. Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012

27. Calculating the Cost of Repair
• When calculating, include the following
processes:
– Call center processes (even if you outsource)
– Support organization processes (even if you outsource
this function)
• On-site repair (service engineers/authorized service suppliers)
– Supply chain processes
• Spare parts management (usually as an overhead)
• Spare parts logistics
– Manufacturing/factory resources expensed against above
processes (usually as overhead expenses)
• Return parts testing process
– Reliability/Quality engineering resources
© Ops A La Carte LLC 2012

28. Calculating the Cost of Repair
• Note that our Cost of Repair model is
conservative because it does not include
– Time Value of Money
– Cost to Customer (survey showed cost of failure to your customer is
about 2-3X your costs).
– Preventive Maintenance
– Software support
– Training
– Disposal cost (future)
Information provided by Alan Wood “LCC Present to IEEE SCV Aug
03”
© Ops A La Carte LLC 2012

29. Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012

30. Cost of New Reliability Program
• Extra engineering support for tools
• Consulting/test lab time for extra tests
• Test equipment rental
• Cost of units under test
• Cost of troubleshooting/failure analysis
• Cost of product changes/ECNs
• Training
© Ops A La Carte LLC 2012

31. Factors Involved in Calculating
DFR ROI
• Improved Warranty Rate (derived from
your Reliability Maturity Level)
• Current Warranty Rate
• Cost per Repair
• Cost of New Reliability Program
• Savings from Losing Fewer Customers
• Volume
© Ops A La Carte LLC 2012

32. Savings from Losing
Fewer Customers
• A. Calculate # of customers lost over the past year
due to reliability/quality issues
• B. Calculate # of customers lost over the past year for
unexplained reasons
– If Stage 1, Multiply by 80%
– If Stage 2, Multiply by 60%
– If Stage 3, Multiply by 40%
– If Stage 4, Multiply by 20%
– If Stage 5, Multiply by 0%
• Calculate Number of Fewer Customers Lost
• Determine average amount of revenue per customer
• Multiply Difference in # of customers lost x revenue
per customer
© Ops A La Carte LLC 2012

33. DFR ROI
CALCULATION
© Ops A La Carte LLC 2012

34. DFR ROI Calculation
1. Perform Assessment/Determine Maturity Level
2. Determine maturity level desired
3. Determine your industry sector
4. Subtract Warranty % between two levels
5. Calculate Average Cost of Repair
6. Calculate Cost of New Reliability Program
7. Calculate Savings from Fewer Lost Customers
8. Determine Volume of Shipments
9. Calculate Repair Cost Savings
10.Calculate ROI
© Ops A La Carte LLC 2012

35. ROI EXAMPLE:
MEDICAL PRODUCT
© Ops A La Carte LLC 2012

36. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
© Ops A La Carte LLC 2012

37. Reliability Maturity Matrix - start
ATTRIBUTES Stage 1: Stage 2: Awakening Stage 3: Stage 4: Wisdom Stage 5: Certainty
Uncertainty Enlightenment
1.1 Understanding No comprehension of Recognizing that reliability Still learning more about reliability Participating. Understand absolutes of Consider reliability management an essential
reliability as a management may be of value but not management. Becoming supportive reliability management. Recognize their part of company system.
& Attitude management tool. Tend willing to provide money or time to and helpful. personal role in continuing emphasis.
to blame engineering make it happen.
for ‘reliability problems’
1.2 Status Reliability is hidden in A stronger reliability leader appointed, Reliability manager reports to top Reliability manager is an officer of Reliability manager is on board of directors.
1. Management
manufacturing or yet main emphasis is still on an audit management, with role in management company; effective status reporting and Prevention is main concern. Reliability is a
engineering of initial product functionality. of division. preventive action. Involved with consumer thought leader.
departments. Reliability Reliability testing still not performed. affairs.
testing probably not
done. Emphasis on
initial product
functionality.
1.3 Measured Cost Not done other than Direct warranty expenses only Warranty, corrective action materials Customer and lifecycle unreliability costs Lifecycle cost reduction done through product
anecdotally and engineering costs monitored determined and tracked reliability improvements
of Unreliability
2.1 Requirement Informal or nonexistent Basic requirements based on Requirements include environment and Plans are tailored for each project and Contingency planning occurs. Decisions
2. Product
customer requirements or standards. use profiles. Some apportionment. projected risks. Use of distributions for based on business or market considerations.
& Planning
Reqts.
Plans have required activities. Plans have more details with regular environmental and use conditions. Part of strategic business plan.
reviews.
2.2 Training & Informally available to Select individuals trained in concepts Training for engineering community for Reliability and statistics courses tailored New technologies and reliability tools tracked
some, if requested and data analysis. Available training key reliability related processes. for design and manufacturing engineers. and training adjusted to accommodate.
Development for design engineers Managers trained on reliability and Senior managers trained on reliability Reliability training actively supported by top
lifecycle impact. impact on business. management.
3.1 Analysis Nonexistent or solely Point estimates and reliance on Formal use of FMEA. Field data analysis Predictions are expressed as distributions Lifecycle cost considered during design.
based on handbook parts count methods. Basic of similar products used to adjust and include confidence limits. Stress and damage models created and
3. Engineering
manufacturing issues identification and listing of failure predictions. Design changes cause re- Environmental and use conditions used for used. Extensive risk analysis for new
modes and impact evaluation of product reliability simulation and testing. technologies.
3.2 Testing Primarily functional Generic test plan exists with reliability Detailed reliability test plan with sample Accelerated tests and supporting models Test results used to update component stress
testing only to meet customer or size and confidence limits. Results used used. Testing to failure or destruct limits and damage models. New technologies
standards specifications for design changes & vendor evals. conducted characterized.
3.3 Supply Chain Selection based on Approved vendor list maintained. Assessments and audit results used to Vendor selection includes analysis of Changes in environment, use profile, or
function and price Audits based on issues or with critical update AVL. Field data and failure vendor’s reliability data. Suppliers conduct design, trigger vendor reliability assessment.
Management parts. Qualification primarily based on analysis related to specific vendors. assessments and audit of their suppliers. Component parameters and reliability
vendor datasheets. monitored for stability
4.1 Failure Data Failures during function Pareto analysis of field return and Root cause analysis used to update Focus is on failure mechanisms. Failure Customer satisfaction relationship to product
testing may be internal testing. Failure analysis relies AVL and prediction models. Summary distribution models updated based on failures is understood. Use of prognostic
Tracking & addressed on vendor support. of analysis results disseminated. failure data methods to forestall failure.
Analysis
4.2 Validation & Informal and based on Basic verification that plans are Supplier agreements around reliability Internal reviews of reliability processes Reliability predictions match observed field
4. Feedback
individuals rather than followed. Field failure data regularly monitored. Failure modes regularly and tools. Failure mechanisms regularly reliability.
Verification
Process
process reported. monitored. monitored and used to update models and
test methods
4.3 Reliability Nonexistent or informal Design and process change followed. Effectiveness of corrective actions Identified failure mechanisms addressed New technologies evaluated and adopted to
Corrective action process includes tracked over time. Identified failure in all products. Advanced modeling improve reliability. Design rules updated
Improvements internal and vendor engagement. modes addressed in other product. techniques explored and adopted. Formal based on field failure analysis.
Improvement opportunities identified as and effective lessons learned process
environment and use profiles change. exists.
5. Prevailing “We don’t know why we “Is it absolutely necessary to always “Through commitment and reliability “Failure prevention is a routine part of our “We know why we do not have problems with
have problems with have problems with reliability?” improvement we are identifying and operation.” reliability.”
Sentiment reliability” resolving our problems.”

38. Reliability Maturity Matrix - start
ATTRIBUTES Stage 1: Stage 2: Awakening Stage 3: Stage 4: Wisdom Stage 5: Certainty
Uncertainty Enlightenment
1.1 Understanding No comprehension of Recognizing that reliability Still learning more about reliability Participating. Understand absolutes of Consider reliability management an essential
reliability as a management may be of value but not management. Becoming supportive reliability management. Recognize their part of company system.
& Attitude management tool. Tend willing to provide money or time to and helpful. personal role in continuing emphasis.
to blame engineering make it happen.
for ‘reliability problems’
1.2 Status Reliability is hidden in A stronger reliability leader appointed, Reliability manager reports to top Reliability manager is an officer of Reliability manager is on board of directors.
1. Management
manufacturing or yet main emphasis is still on an audit management, with role in management company; effective status reporting and Prevention is main concern. Reliability is a
engineering of initial product functionality. of division. preventive action. Involved with consumer thought leader.
departments. Reliability Reliability testing still not performed. affairs.
testing probably not
done. Emphasis on
initial product
functionality.
1.3 Measured Cost Not done other than Direct warranty expenses only Warranty, corrective action materials Customer and lifecycle unreliability costs Lifecycle cost reduction done through product
anecdotally and engineering costs monitored determined and tracked reliability improvements
of Unreliability
2.1 Requirement Informal or nonexistent Basic requirements based on Requirements include environment and Plans are tailored for each project and Contingency planning occurs. Decisions
2. Product
customer requirements or standards. use profiles. Some apportionment. projected risks. Use of distributions for based on business or market considerations.
& Planning
Reqts.
Plans have required activities. Plans have more details with regular environmental and use conditions. Part of strategic business plan.
reviews.
2.2 Training & Informally available to Select individuals trained in concepts Training for engineering community for Reliability and statistics courses tailored New technologies and reliability tools tracked
some, if requested and data analysis. Available training key reliability related processes. for design and manufacturing engineers. and training adjusted to accommodate.
Development for design engineers Managers trained on reliability and Senior managers trained on reliability Reliability training actively supported by top
lifecycle impact. impact on business. management.
3.1 Analysis Nonexistent or solely Point estimates and reliance on Formal use of FMEA. Field data analysis Predictions are expressed as distributions Lifecycle cost considered during design.
based on handbook parts count methods. Basic of similar products used to adjust and include confidence limits. Stress and damage models created and
3. Engineering
manufacturing issues identification and listing of failure predictions. Design changes cause re- Environmental and use conditions used for used. Extensive risk analysis for new
modes and impact evaluation of product reliability simulation and testing. technologies.
3.2 Testing Primarily functional Generic test plan exists with reliability Detailed reliability test plan with sample Accelerated tests and supporting models Test results used to update component stress
testing only to meet customer or size and confidence limits. Results used used. Testing to failure or destruct limits and damage models. New technologies
standards specifications for design changes & vendor evals. conducted characterized.
3.3 Supply Chain Selection based on Approved vendor list maintained. Assessments and audit results used to Vendor selection includes analysis of Changes in environment, use profile, or
function and price Audits based on issues or with critical update AVL. Field data and failure vendor’s reliability data. Suppliers conduct design, trigger vendor reliability assessment.
Management parts. Qualification primarily based on analysis related to specific vendors. assessments and audit of their suppliers. Component parameters and reliability
vendor datasheets. monitored for stability
4.1 Failure Data Failures during function Pareto analysis of field return and Root cause analysis used to update Focus is on failure mechanisms. Failure Customer satisfaction relationship to product
testing may be internal testing. Failure analysis relies AVL and prediction models. Summary distribution models updated based on failures is understood. Use of prognostic
Tracking & addressed on vendor support. of analysis results disseminated. failure data methods to forestall failure.
Analysis
4.2 Validation & Informal and based on Basic verification that plans are Supplier agreements around reliability Internal reviews of reliability processes Reliability predictions match observed field
4. Feedback
individuals rather than followed. Field failure data regularly monitored. Failure modes regularly and tools. Failure mechanisms regularly reliability.
Verification
Process
process reported. monitored. monitored and used to update models and
test methods
4.3 Reliability Nonexistent or informal Design and process change followed. Effectiveness of corrective actions Identified failure mechanisms addressed New technologies evaluated and adopted to
Corrective action process includes tracked over time. Identified failure in all products. Advanced modeling improve reliability. Design rules updated
Improvements internal and vendor engagement. modes addressed in other product. techniques explored and adopted. Formal based on field failure analysis.
Improvement opportunities identified as and effective lessons learned process
environment and use profiles change. exists.
5. Prevailing “We don’t know why we “Is it absolutely necessary to always “Through commitment and reliability “Failure prevention is a routine part of our “We know why we do not have problems with
have problems with have problems with reliability?” improvement we are identifying and operation.” reliability.”
Sentiment reliability” resolving our problems.”

39. Reliability Maturity Matrix - desired
ATTRIBUTES Stage 1: Stage 2: Awakening Stage 3: Stage 4: Wisdom Stage 5: Certainty
Uncertainty Enlightenment
1.1 Understanding No comprehension of Recognizing that reliability Still learning more about reliability Participating. Understand absolutes of Consider reliability management an essential
reliability as a management may be of value but not management. Becoming supportive reliability management. Recognize their part of company system.
& Attitude management tool. Tend willing to provide money or time to and helpful. personal role in continuing emphasis.
to blame engineering make it happen.
for ‘reliability problems’
1.2 Status Reliability is hidden in A stronger reliability leader appointed, Reliability manager reports to top Reliability manager is an officer of Reliability manager is on board of directors.
1. Management
manufacturing or yet main emphasis is still on an audit management, with role in management company; effective status reporting and Prevention is main concern. Reliability is a
engineering of initial product functionality. of division. preventive action. Involved with consumer thought leader.
departments. Reliability Reliability testing still not performed. affairs.
testing probably not
done. Emphasis on
initial product
functionality.
1.3 Measured Cost Not done other than Direct warranty expenses only Warranty, corrective action materials Customer and lifecycle unreliability costs Lifecycle cost reduction done through product
anecdotally and engineering costs monitored determined and tracked reliability improvements
of Unreliability
2.1 Requirement Informal or nonexistent Basic requirements based on Requirements include environment and Plans are tailored for each project and Contingency planning occurs. Decisions
2. Product
customer requirements or standards. use profiles. Some apportionment. projected risks. Use of distributions for based on business or market considerations.
& Planning
Reqts.
Plans have required activities. Plans have more details with regular environmental and use conditions. Part of strategic business plan.
reviews.
2.2 Training & Informally available to Select individuals trained in concepts Training for engineering community for Reliability and statistics courses tailored New technologies and reliability tools tracked
some, if requested and data analysis. Available training key reliability related processes. for design and manufacturing engineers. and training adjusted to accommodate.
Development for design engineers Managers trained on reliability and Senior managers trained on reliability Reliability training actively supported by top
lifecycle impact. impact on business. management.
3.1 Analysis Nonexistent or solely Point estimates and reliance on Formal use of FMEA. Field data analysis Predictions are expressed as distributions Lifecycle cost considered during design.
based on handbook parts count methods. Basic of similar products used to adjust and include confidence limits. Stress and damage models created and
3. Engineering
manufacturing issues identification and listing of failure predictions. Design changes cause re- Environmental and use conditions used for used. Extensive risk analysis for new
modes and impact evaluation of product reliability simulation and testing. technologies.
3.2 Testing Primarily functional Generic test plan exists with reliability Detailed reliability test plan with sample Accelerated tests and supporting models Test results used to update component stress
testing only to meet customer or size and confidence limits. Results used used. Testing to failure or destruct limits and damage models. New technologies
standards specifications for design changes & vendor evals. conducted characterized.
3.3 Supply Chain Selection based on Approved vendor list maintained. Assessments and audit results used to Vendor selection includes analysis of Changes in environment, use profile, or
function and price Audits based on issues or with critical update AVL. Field data and failure vendor’s reliability data. Suppliers conduct design, trigger vendor reliability assessment.
Management parts. Qualification primarily based on analysis related to specific vendors. assessments and audit of their suppliers. Component parameters and reliability
vendor datasheets. monitored for stability
4.1 Failure Data Failures during function Pareto analysis of field return and Root cause analysis used to update Focus is on failure mechanisms. Failure Customer satisfaction relationship to product
testing may be internal testing. Failure analysis relies AVL and prediction models. Summary distribution models updated based on failures is understood. Use of prognostic
Tracking & addressed on vendor support. of analysis results disseminated. failure data methods to forestall failure.
Analysis
4.2 Validation & Informal and based on Basic verification that plans are Supplier agreements around reliability Internal reviews of reliability processes Reliability predictions match observed field
4. Feedback
individuals rather than followed. Field failure data regularly monitored. Failure modes regularly and tools. Failure mechanisms regularly reliability.
Verification
Process
process reported. monitored. monitored and used to update models and
test methods
4.3 Reliability Nonexistent or informal Design and process change followed. Effectiveness of corrective actions Identified failure mechanisms addressed New technologies evaluated and adopted to
Corrective action process includes tracked over time. Identified failure in all products. Advanced modeling improve reliability. Design rules updated
Improvements internal and vendor engagement. modes addressed in other product. techniques explored and adopted. Formal based on field failure analysis.
Improvement opportunities identified as and effective lessons learned process
environment and use profiles change. exists.
5. Prevailing “We don’t know why we “Is it absolutely necessary to always “Through commitment and reliability “Failure prevention is a routine part of our “We know why we do not have problems with
have problems with have problems with reliability?” improvement we are identifying and operation.” reliability.”
Sentiment reliability” resolving our problems.”

40. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Subtract Warranty % between two levels
© Ops A La Carte LLC 2012

41. Warranty vs. Maturity by Industry
% of Revenue
SECTOR Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
Automotive Manufacturer 9.2 4.6 2.3 1.15 0.58
Auto Parts Suppliers
Aerospace and Marine
Computers
Telecom 4.8 2.4 1.2 0.6 0.3
Semiconductor
Consumer Electronics
Medical and Scientific 2.8 1.4 0.7 0.35 0.18
Data Storage
PC Peripherals
Appliances and HVAC
Homebuilders
Building Materials
Power Generation
© Ops A La Carte LLC 2012

42. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Subtract Warranty % between two levels
– Warranty % Level 2 = 1.4%
– Warranty % Level 3 = 0.7%
– Difference = 0.7%
© Ops A La Carte LLC 2012

43. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Warranty % Difference = 0.7%
5. Calculate Average Cost of Repair
© Ops A La Carte LLC 2012

44. Calculating the Cost of Repair
• Call center processes = $200
• Support organization processes (even if you
outsource this function) = $200
• Supply chain processes = $1000
• Manufacturing/factory resources expensed
against above processes = $300
• Reliability/Quality engineering resources = $300
TOTAL = $2K
© Ops A La Carte LLC 2012

45. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Warranty % Difference = 0.7%
5. Average Cost of Repair = $2K
6. Calculate Cost of New Reliability Program
© Ops A La Carte LLC 2012

46. Cost of New Reliability Program
• Extra engineering support for tools = $10K
• Consulting/test lab time for extra tests = $10K
• Test equipment rental = $5K
• Cost of units under test = $10K
• Cost of troubleshooting/failure analysis = $10K
• Cost of product changes/ECNs = $20K
• Training = $10K
TOTAL = $75K
© Ops A La Carte LLC 2012

47. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Warranty % Difference = 0.7%
5. Average Cost of Repair = $2K
6. Cost of New Reliability Program = $75K
7. Savings from Fewer Lost Customers
© Ops A La Carte LLC 2012

48. Savings from Losing
Fewer Customers
• A. Calculate # of customers lost over the past year
due to reliability/quality issues = 4
• B. Calculate # of customers lost over the past year for
unexplained reasons = 10
– If Stage 2 = Multiply by 60%
– If Stage 3 = Multiply by 40%
• Calculate Number of Fewer Customers Lost
– Stage 2 = 3 + (10*0.6) = 9
– Stage 3 = 3 + (10*0.4) = 7
– Difference = 2
• Average amount of revenue per customer = $100K
• Multiply Difference in # of customers lost x revenue
per customer
$200K savings from fewer lost customers
© Ops A La Carte LLC 2012

49. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Warranty % Difference = 0.7%
5. Average Cost of Repair = $2K
6. Cost of New Reliability Program = $75K
7. Savings from Fewer Lost Customers = $200K
8. Volume of Shipments = 5,000/year
© Ops A La Carte LLC 2012

50. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Warranty % Difference = 0.7%
5. Average Cost of Repair = $2K
6. Cost of New Reliability Program = $75K
7. Savings from Fewer Lost Customers = $200K
8. Volume of Shipments = 5,000/year
9. Repair Cost Savings = Volume * Warranty ∆ *
Cost of Repair = 5000*0.7%*$2K = $70K
© Ops A La Carte LLC 2012

51. DFR ROI Calculation
1. Starting maturity level = 2
2. Desired maturity level = 3
3. Industry Sector = Medical and Scientific
4. Warranty % Difference = 0.7%
5. Average Cost of Repair = $2K
6. Cost of New Reliability Program = $75K
7. Savings from Fewer Lost Customers = $200K
8. Volume of Shipments = 5,000/year
9. Repair Cost Savings = $70K
10.Calculate ROI
© Ops A La Carte LLC 2012

52. Calculating ROI
ROI = Repair Savings
+ Fewer Lost Customers
- Reliability Prog Cost
ROI = $70K + $200K - $75K
ROI = $195K / year
© Ops A La Carte LLC 2012

53. Conclusion
• Today we have shown you a powerful new
tool, the DFR ROI Calculator.
• With this tool, you can
– Justify using new techniques
– Quantify how much improvement you will make
– Balance cost savings with improvements
© Ops A La Carte LLC 2012

54. Q&A
© Ops A La Carte LLC 2012

55. Contact Information
Ops A La Carte, LLC
Mike Silverman
Managing Partner
(408) 472-3889
mikes@opsalacarte.com
www.opsalacarte.com
© Ops A La Carte LLC 2012