An overview of how to take a helicopter view of calibration programs and implement a step change in performance.

1. Key Steps in Transforming a
Calibration Program
John Cummins, CPIP

2. Agenda
• Maintenance & Reliability
• The Perception of Calibration
• Presentation Purpose
• Presentation Format
• Classifications
• Calibration Frequency Analysis
• Work Load Levelling
• Work Practices
• End-to-end Paperless Calibration Execution
• The 8 Steps in Complete Calibration Transformation
• Summary

3. Preventativ
e
Maintenanc
e
Corrective
Maintenance
Predictive
Maintenanc
e
Site
Maintenance
Program
OPR / UDt
Review
FMECA
Equipment
Performanc
e Reviews
MTBF
Studies
MaintenanceStrategy
ReliabilityStrategy

4. Increased Equipment Availability
Increased Production Capability
Increased Output and Profit
For these reasons, Maintenance and Reliability
programs are treated as a key driver of the overall
business goals.

5. What about Calibration?
How is it viewed within your
organization?

6. We do it
because the
regulations say
we have to
It’s a nuisance…
calibration
means
equipment
downtime!
We contract
that out, not my
concern.
Just part of the
day to day
running of the
facility

7. So What?
People don’t know/care about Calibration…

8. Most of all… it’s a missed opportunity

9. Increased Equipment Availability
Increased Production Capability
Increased Output and Profit
Let’s look at this earlier slide again…
What if we applied the same rigour to Calibrations as we do to our
Maintenance and Reliability efforts?
What would it look like?
We’ll come back to this later…

10. • The aim of this presentation is to show you – based on case
studies and my personal experience – the impact that a
transformational calibration effort can have on your business.
• The steps outlined in this presentation are by no means a
complete list. However, I have found that they are the key
components in achieving a step-change in performance.
• Think of the steps as a menu that you can choose from, or
personalize depending on the particular environment of your
company or facility.
• As engineering leaders within your organizations, you have the
ability to ask the right questions and identify what changes will
have the biggest impact.
• Most importantly, I want people to realize that calibration
management is a BUSINESS PROCESS, and significant rewards
can be obtained for treating it as such…
Purpose

11. Format
Key Metric
Case Studies
Impact
Lessons Learned

12. Classifications

13. Classifications – Critical %
• The first key step I will talk about is Instrument Classification review, and
the key metric in this regard is the overall % of “Critical” instrumentation.
Do you
know
your
Critical %
Yes No
Is it
good
bad?
Don’t know…
Good
Are
you
sure?

14. Classifications: Case Studies

15. Classification Variation Case Study 1
0% 10% 20% 30% 40% 50% 60% 70% 80%
USA 2
Puerto Rico 1
Puerto Rico 2
Puerto Rico 3
Canada
USA 1
72%
56.7%
51.4%
35%
34%
33%
% "Critical" Instruments – Different Sites Within Same Network
1
2
3
Commonality between
sites was a clear
classification decision
process, tied to
CPPs/CQAs

16. Classification Variation Case Study 2
0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%
Solid Oral Dose
Solid Oral Dose Post Optimization
Sterile Fill/Finish
Solid Oral Dose Pre Optimization
Medical Devices
30.45%
39.89%
46.12%
61.34%
95.67%
% Critical Instruments – 3 Different Companies / Locations

17. Classifications – The CRA Process
• Some of you may know this as the CAT (Criticality Assessment Team)
process… and may have experienced the perils of a subjective decision
making process.
Do you have a
clear decision
making
process
Yes No
Don’t know…

18. Case Study – Subjectivity in the CRA Process
Initiator
Engineering
Production
AutomationValidation
Safety
Quality
This
instrument
is not
Critical
But its on a
batch
record!
Well its not
listed as a
critical
alarm!
The
measured
parameter
is not a CPP
I agree with
production
It’s critical!

19. Classification Changes

20. Impact of Classification Changes
• In the Classification Variation case study, we looked at a site
that had a Critical Instrument percentage of 61.34%. This was
associated with an annual calibration figure of roughly 12,000.
• After implementing a revised approach to classifications and
calibration frequency, they had 5,844 calibrations per year, and
reduced their Critical items to just 39.89% .
• This was a great achievement for the facility in terms of
reducing the calibration workload… but who else is affected by
this drastic reduction in Critical instrumentation?

21. Investigation Process
• On average, 4.5% of calibrations would experience an out of
tolerance (OOT) event each year.
• An investigation was required for any Critical instrument that
was found to be OOT. This process involved multi disciplinary
teams and several stages.
Initial reporting of event and
closure of calibration activity:
2 hours x 1 person = 2 hours
Root Cause Analysis carried
out, involving the calibration
technician, production SMEs,
Validation and Quality.
2 hours x 4 people = 8 hours
Writing up the investigation
based on the RCA
8 hours x 1 person = 8 hours
Reviewing and approving the
investigation report.
1.5 hours x 4 people = 6 hours

22. Investigation Process – Impact of Change
Pre-Optimization
7,320 Critical Calibrations
329 Investigations
7,896 Man Hours
Post-Optimization
1,753 Critical Calibrations
79 Investigations
1,896 Man Hours
A reduction of 6,000 man hours a year, or 3 x FTEs

23. Classifications

24. Lessons Learned - Classifications
• You must understand your critical instrument % figure, and
how this figure stands up against your competition (internal /
external).
• You must have a CLEAR process for determining your
classification path.
• Classifications should be tied to tangible site knowledge (e.g.
Critical assets tied to CPPs/CQAs).
• Reduce your CRA approval (i.e. your criticality assessment
team) to only those who need to have an input.
• Look outside the calibration department to see what other
areas are impacted by the calibration business process. You
can underestimate the full value of a transformation effort.

25. Calibration Frequencies

26. Calibration Frequencies
• Let’s look at the simple questions we should ask when we talk about
Calibration Frequencies.
Do I ever
change my
calibration
frequencies?
Yes No
Is it based on
Instrument
Performance
No
Yes
Is it
planned
?

27. Calibration Frequency:
Case Study

28. Calibration Frequency: Case Study
Scope was
determined
Analysis
Methodology
Developed
Change
Control
Approved
Results
Implemented

29. Calibration Frequencies

30. Impact of Calibration Frequency Changes
Calibration FTE Headcount reduced by
25%.
Annual vendor spend reduced by 33%
A 50% reduction in annual calibrations
was achieved for the site.

31. Impact of Calibration Frequency Changes
• During an audit by the IMB (now HPRA), a portion of
the change control was called into question.
• As part of the risk assessment, instruments had been
“grouped” together (i.e. assessed by instrument
make/model).
• The auditor felt that more scrutiny should have been
applied at the instrument level.
• A fully automated QRA was developed which
specifically dealt with each instrument individually
based on its risk to the process, and interval “caps”
were applied based on this risk.

32. Calibration Frequencies

33. • When implementing a calibration frequency change
effort, apply your analysis at the instrument level.
• Use the analysis method as a tool for suggesting
change rather than enforcing it. Allow SMEs with
extensive process knowledge to have an input on the
final changes that are implemented.
• Despite the complexity of this type of project, do not
be discouraged from taking it on… it can have the
biggest impact to your annual workload and overall
budget.
Lessons Learned – Calibration Frequencies

34. Work Load Levelling

35. Do I routinely
review my
work load
distribution?
Yes No
Is it fit for
purpose based
on my resource
availability?
No
Yes
Work Load Levelling
• Here is the basic questions you should ask yourself on Work Load Levelling
activity.

36. Work Load Levelling
Case Study

37. 500
600
700
800
900
1000
1100
January February March April May June July August September October November December
CalibrationsPerYear
Before Work Load Levelling Activity
Case Study – Workload Levelling

38. 500
550
600
650
700
750
800
850
900
950
1000
January February March April May June July August September October November December
CalibrationsPerYear
After Work Load Levelling Activity
Case Study – Workload Levelling

39. Work Load Levelling

40. Impact of Work Load Levelling
• Much more satisfied workforce, denoted by an
improvement in the annual employee survey from
63% to 87%.
• Reduction in associated Overtime and Time-in-Lieu
costs, equating to over €120,000 per annum.
• The department now had the ability for each team
member to take on individual improvement projects,
which showed savings / cost avoidances of over
€90,000 within the first 3 months of implementation.

41. Work Load Levelling

42. • Ability to only pull work forward (without the need
for site change controls).
• PM Activity and the potential to “de-sync”, resulting
in equipment downtime being more frequently
requested.
• Consideration for shutdown periods, holidays etc.
• Vendor activity / Service Level Agreements (most
will require significant advanced notice if required
calibration dates change).
• Biggest change in this effort was to team morale! No
more “panic” months or “idle” months. Ability to
take holidays with no major impact to team.
Lessons Learned – Work Load Levelling

43. Work Practices

44. • “Work Practices” is an all-encompassing term that can cover a wide
variety of metrics.
• In essence, it means identifying and removing obstacles that prevent
you / your team to operating in the most efficient manner possible.
• YOU must identify the Work Practices most important to you , your
department, your facility or your company.
• Some examples of work practices are:
 Training methodologies and programs
 Work arrangements (shifts, flexi-time etc.)
 Work Conditions (adequate facilities, hardware, tools etc.)
 Continuous Improvement Project Programs
 Reward and Recognition Programs
• For the purposes of this presentation, I have chosen to discuss one
small example of how changing a minor work practice had a positive
effect on the future state of a calibration department.
Important Note on Work Practices

45. Work Practices
Case Study

46. Work Practices Case Study
Temperature
SOP-101 Calibration of
Temperature Gauges
SOP-102 Calibration of
Temperature
Transmitters
SOP-103 Calibration of
Temperature Switches
SOP-106 Calibration of
Temperature Loops
SOP-104 Calibration of
Temperature Via
Handheld Ref Probe
SOP-105 Calibration of
Temperature Via
Electronic Simulation
6 SOPs for One Measurement Type, each requiring biennial review…
Also 6 learning items in a calibration technician's learning plan.

47. Work Practices Case Study
Temperature
SOP-101 Calibration of Temperature
Section 1: Calibration of Temperature Gauges
Section 2: Calibration of Temperature Transmitters
Section 3: Calibration of Temperature Switches
Section 4: Calibration of Temperature Via Handheld Ref Probe
Section 5: Calibration of Temperature Via Electronic Simulation
Section 6: Calibration of Temperature Loops

48. So What?

49. Work Practices

50. • When the process was repeated for all
calibration types:
 65 SOPs for Calibration were reduced to
24, reducing the required biennial reviews
by 63%
 The former biennial review process
involved 195 signatures. Only 72 were
required now.
 Learning time was reduced by 20.5 man
hours per technician
Impact of Change in Work Practices

51. Work Practices

52. • Do not overlook what may seem like a minor / trivial
change. Small changes can sometimes have a big
impact…
• Nobody knows your day-to-day processes (and the
flaws in them) better than the people who have to
follow them. Seek their advice and ask them what
would they improve if they had the chance.
• Reward ideation – have a program whereby
Continuous Improvement efforts driven from the
team are championed and celebrated.
Lessons Learned – Work Practices

53. End-to-end Paperless
Calibration Management

54. Do I utilize a
fit-for-
purpose
CCMS
Yes No
Do I need paper
to support my
CCMS business
process
Yes
No
End-to-end Calibration Management
• Follow the flowchart to find out if you are truly “paperless”.

55. End-to-end Paperless
Calibration Management
Case Study

56. Case Study: Paper-based CRA
Receive TOP handover
Add items to CRA template
Print populated CRA template
Review printed CRA with all
stakeholders (6)
Make required changes
Re-review CRA
Sign CRA
Populate System of Record
• 6 people involved in the
process.
• Involved “walking” the
list multiple times
before it would be
signed.
• Prone to disagreements
and errors.
• Laborious process to get
calibrated assets live on
the system of record
and to manage
thereafter.
Execute Calibrations / Manage
record on paper

57. Case Study: Paperless CRA
Receive TOP handover
Review items Populated
on CCMS
Make required changes
electronically
Approve Changes
Populate System of
Record (CCMS)
• 3 people involve in the process
(mirroring ISPE approach)
• No “walking” of list involved.
All changes made and
approved electronically.
• Clear decision path for
classifications removed
subjectivity.
• Much more efficient system for
getting calibrated assets into
the system.
• Removal of all paper from the
process after instrument is
“live”
Once instrument is live,
manage paperlessly
throughout life-cycle

58. Paperless Calibration
Management

59. • Administrative Cost reductions
• Removal of instrument detail change forms.
• Removal of Calibration Request Forms.
• All events reported electronically.
• Overdue reports contained directly on the
system.
• Always audit-ready.
• Planning/Scheduling made easier with no
requirement to print, distribute, execute,
collect and file paper records.
Impact: Paperless Calibration Management

60. Paperless Calibration
Management

61.  The target condition (end-to-end paperless) means COMPLETELY
paperless.
 This is difficult to achieve, as some aspects of the calibration
program (e.g. vendor calibrations) will almost always have to be
based on paper record review and retention.
 However, ensure you look at what other paper elements of your
business process you could eliminate such as:
 Shipping forms
 Event forms
 Investigation Forms
 Calibration Labels
 Project Certificates
 Criticality Risk Assessment Process
Lessons Learned – Paperless Cal Management

62. Evaluate
Current
State
Set Targets
and Wins
Draft Your
Plan
Classifications
Calibration
Frequency
Analysis
Work Load
Levelling
Work
Practices
End-to-end
paperless
calibration
execution
8 Steps in Complete Calibration Transformation

63. Optimized Classifications
Frequencies based on performance
Evenly Distributed Workload
Work Practices that Enhance Business Operations
Paperless Calibration Management
After you implement a calibration transformation program, you
now have key performance indicators that reflect a fully optimized
business.

64. Summary
• The aim of this presentation was to show you the impact that a
transformational calibration effort can have on your business.
• Having seen the results from the case studies, what would the
impact be if you could do the same at your facility?
• I hope that you will now ask the questions outlined in this
presentations within your own organization, and that you
endeavour to embark on your own calibration transformation
journey.

65. John Cummins, CPIP
jcummins@CalOpEx.com