SlideShare a Scribd company logo
1 of 37
Download to read offline
Metrics and KPIs for
Localization
Agenda
• Definitions — what are indicators?
• A brief case for metrics and KPIs — why they matter?
• A brief case against KPIs — reasons to be cautious
• Practical Localization Metrics 101
• Practical Localization Metrics 200
• Program Dashboards
• Developing Localization Metrics
• Localization Performance
Indicators 300+
Definitions1
Data, Metrics and Indicators
Data is a collection of facts and statistics collected
together for reference or analysis
A metric is a collection of the same type of data
“You can’t pick your data, but you must pick your
metrics!”
An indicator provides evidence that a certain
condition exists or certain results have or have not
occurred.
KRIs, PIs, and KPIs
1. Key result indicators (KRIs) tell you how
you have done in a perspective.
2. Performance indicators (PIs) tell you
what to do.
3. Key Performance indicators (KPIs) tell
you what to do to increase performance
dramatically.
Why You Need KPIs
“What gets measured gets done” – Attributed to many*
“Without a standard, there is no logical basis for making a decision or
taking action.” - Joseph Juran
• Some benefits of KPIs
• KPIs shift a group to proactive mode
• Healthier partner relationships
• Pay-for-performance becomes easier to
implement (or even possible)
• Scalability, consistency, normalcy
• Move away from “constant crisis” style
• Ability to focus teams on essential goals
KPIs in Action: British Airways
In the 1980s turned the company around by
focusing on just one KPI:
• Financial – Operating margin
• Customers – Recommendation
• Operations – Punctuality
• Colleagues – Colleague involvement
Actions:
- Doubled cleaning crew
- Prioritized refueling
- Personal call policy
Results:
- Lower fuel costs
- Lowered airport surcharges
- Reduced lost baggage
- Improved customer satisfaction
- Improved vendor relationships
- More satisfied employees
A Case Against KPIs
“Not everything that counts can be
counted, and not everything that
can be counted counts.”
Albert Einstein
“People will do what management
inspects, not necessarily what
management expects.”
Dean Spitzer
Practical Localization Metrics 1012
The Best Place to Start…
Common localization metrics
• Percentage of on-time deliveries
• Data points: hand-off date, requested delivery date, approved delivery date, actual
delivery date
• Number of linguistic or technical quality errors
• Data points: quality framework and measurement methodology
• Pass/fail rates and First pass yield
• Data points: pass criteria, pass/fail data
• Quantified quality tiers
• Average throughput rate
• Weighted words, hand-offs, pages per business day
• Fully-loaded cost per word
Start with Data
1. Evaluate the data you have. What are the gaps?
2. Tracking sheets and invoicing data is a good place to start
3. An existing TMS will typically have good sources of raw data to export
4. Assign a clear owner for data integrity
5. Normalized data is a MUST
6. Excel is good for modeling and getting started,
but it stops being effective quickly
7. SharePoint has decent SQL back-end support. Consider
customizing the user fields by task or station
8. Access and SQL are good next steps, but require more
specialized skills to configure
9. More advanced systems exist for more mature programs
Percentage of On-time Deliveries
Calculation:
∑ total deliveries that are on-time or early / total deliveries
Best value:
Paces production, focuses team on meeting deadlines
Variables to consider:
Project complexity is an issue
Ignoring holidays often means you get the B-team
You should plan to track 4 data points
MBO opportunity:
Reward meeting target OTD goals (e.g. 98% or higher = +2% month)
Penalize late deliveries (e.g. <90% = -2%)
Average Language Quality Scores
Calculation:
∑ total weighted errors / total word count
Best value:
Pushes client and suppliers to define clear quality expectations
Brings common linguistic issues into focus — easier to address. Practical actions.
Variables to consider:
Track translators by ID (not name), use RCA, arbitration & valid statistics
Measure samples and get feedback back to translators in a timely manner
MBO opportunity:
Bonus for keeping average scores above a target (e.g. 95%)
Issue warning letter for fails. 3 Fails on the same language
results in an escalation.
Pass / Fail Rate and First Pass Yield
Calculation:
∑ total passed tests / total tests; ∑ total accepted jobs / total jobs
Best value:
Incentivizes consistency
Variables to consider:
Track translators by ID (not name)
Translators should not be penalized for bad source or instructions
Use RCA, arbitration & valid statistics to resolve issues
MBO opportunity:
Bonus for keeping pass rate above a target (e.g. 95%)
Issue warning letter for fails. 3 Fails on the same language
results in an escalation.
End-User Escalations / Resolution Rate
Calculation:
∑ total escalations / total products or deliveries per region
Best value:
This is one of several customer-centric metrics that is a “reality check”
to traditional quality tracking. How much does quality matter?
Variables to consider:
Important to understand root causes
Distinguish between I18n, L10n and source failures
Often difficult to measure and normalize
MBO opportunity:
Bonus for keeping pass rate below a target (e.g. .1%)
Throughput
Calculation:
∑ weighted word count / (network days – holidays)
Best value:
Helps understand performance under range of operating conditions
Encourages a comprehensive view of program capacity
Measures scalability and program resiliency
Better anticipate bottlenecks and plan capacity
Variables to consider:
Measure by type of work (complexity) and priority
MBO opportunity:
Helpful to build a business case for investments to reduce TAT
Practical Localization Metrics 2003
Quantified Quality Tiers
Process:
Define quality process and error thresholds per threshold
Best value:
Creates opportunity for different workflows and pricing
Allocates QC resources more effectively
Variables to consider:
Consider testing to qualify and monitor individual translators
It is possible to set this up without additional pricing tiers
In-line review, self-certification documentation and tracking
Prioritization
Process:
Create clear process SLA guidelines for priorities (at least 2)
Best value:
Creates opportunity for different workflows and pricing
Allocates QC resources more effectively
More likely to get the urgent things done first,
first come first served
By default everything has same priority and so is first
Theory of Constraints
• Add buffers to production to optimize the performance of
the most critical resources — steady work flow eliminates
exception management model
• Ideal for reigning in large chaotic workflows
• Set prioritization to allow for buffering
• Set minimum capacity expectations at the maximum
number of high-priority projects you expect to
require (e.g. 2k words per day)
Fully-Loaded Price Per Word
Calculation:
∑ total outsourced cost/total number of words per project
Best value:
Total program efficiency including development
Helps identify inefficient production, especially if
the average is much higher than the new word rate
Variables to consider:
Larger new projects will bump the average up
Higher leverage brings the average down. Track by
project type
MBO opportunity:
Can’t be used as an MBO for localization teams only, but
it can support content development and tech initiatives
Localization Dashboards4
Effective Localization Dashboards…
• …contain only the highest-priority indicators
• …are updated regularly
• …are validated regularly
Tips:
• Consider designating a data entry and/or
analyst to manage the data
• Careful tracking discipline is not a critical path
task unless you make it one
• Keep it simple
Effective Localization Dashboards
• Schedule
• % on-time delivery (the KPI should be interpreted as
“as agreed” rather than “as requested.” The former
impacts the production line, the latter is nice to have.
• Quality
• Pass/Fail target periodic performance (yes/no).
The data and metrics can be there, but on a
dashboard you just want to know if OK or not.
• Cost
• Could contain periodic costs to date, e.g.
against a PO or budget, but most localization
teams do not have much control over costs.
Example:
Mozilla
Notice the priorities:
• Missing translations
• Errors
• Deadline
• Priority
• % complete
Example: Basic Localization Dashboard
Area KPI Name Target Compliance Actual
Linguistic Language quality pass ≥ 95% 97.8%
Project management
On-time deliveries ≥ 90% for traditional waterfall 99.1%
Leadership of project management Feedback 4.0
Engineering
Technical accuracy of deliveries ≥ 95% 98.3%
Production handback issues < 5/month 2.3
Finance
Timeliness of invoicing 100% 100%
Invoicing correctness 0% reinvoicing or corrections Pass
Developing Localization Metrics5
A Phased Approach to Developing KPIs
• Phase 1: All models require starting with good data
• If you don’t have reliable and consistent sources of
data, nothing else is possible
• Set up the basics and start tracking them: delivery
data, priority quality, and any classifications you need
• Phase 2. Establish your quality measurement system
• Without solid quality data, MBO goals are not possible
• “Intuitive quality management” is worse than the
default “if I don’t hear about it the quality is fine”
management plan
• Phase 3: Start setting goals
• Patterns will emerge quickly that will drive
focused actions almost immediately
• Use these to normalize data and metrics
• Clearly and consistently communicate goals
and expectations
• Phase 4: MBOs
• Once solid data and processes have been set,
MBOs are possible
• Use the framework to rework pricing and terms
• Use a combination of sticks and carrots
A Phased Approach to Developing KPIs
Some Additional Thoughts on Metrics
• It’s helpful to manage at the job level. Hand-offs and projects are merely
containers for jobs.
• A job is a deliverable (e.g. German UI file)
• Aggregating data to a dashboard is OK, but the data should
always be stored as jobs
• Focus on data integrity
• Dates should be dates, numbers should be numbers. Any data
point used in a metric needs to have a name, type, range,
money should be money and it should have a validation
method (to detect errors).
• Data is not free — Don’t track what you can’t use
• Often data is collected based on the “that might be interesting” policy
• Use ISO!
Localization Performance
Indicators 300+6
Localization Business Metrics 300+
Overall cost of ownership
• Globalization (G11n) headcount
• Localization (L10n) headcount
• Outsourced vs. in-house G11n costs
• G11n costs (outright)
• L10n costs (outright)
• L10n cost against L10n product revenue
• Outsourced L10n per word cost
• In-house L10n per word cost
• Internationalization (I18n) resources
• Internal and external linguist cost
ROI (return on investment)
• L10n product ROI
• Tiered market definitions
Parking Lot: Potential future metrics
• International web site revenue
• Click-through rates, and how that translates
into dollars
• Customer support — % offered in local vs.
source language
• Cost of localizing web sites, web presence
(download center, online store, etc.)
• Partial vs. full product localization
Additional Localization
Performance Indicators
• Presales metrics
• Business metrics for globalization
• After-sales metrics
Presales Metrics
• International market growth rate
• Relative market share
• Return on investment (ROI) in localization
• Return on localization assets
• Customer online engagement
• Number of qualified international leads
• Cost per qualified international lead
• Online share of voice
• Brand awareness
• Aided, unaided
Business Metrics for Globalization
• Total localization headcount
• Total localization headcount relative to revenues
• Total localization costs
• Total localization costs relative to revenues
• Fully-loaded average cost per word
• Average cost of outsourced word
• Time to market
• Opportunity cost of a delayed release
After-Sales Metrics
• Support call deflection
• First contact resolution
• Brand recognition by region
• Localized product sentiment analysis
• Customer satisfaction
• Loyalty metric such as Net Promoter Score (NPS)
• Praise-to-complaint ratio
• Track rework, warranties, recalls
Localization Metrics and KPIs

More Related Content

What's hot

Email Communication
Email CommunicationEmail Communication
Email Communicationjjs1981
 
Bank credit officer kpi
Bank credit officer kpiBank credit officer kpi
Bank credit officer kpipoexarsa
 
Why mba presentation
Why mba presentationWhy mba presentation
Why mba presentationRavi kurakula
 
Safety (Security) Training
Safety (Security) TrainingSafety (Security) Training
Safety (Security) Trainingworksteadc
 
Presentation on "Email writing skills"
Presentation on "Email writing skills"Presentation on "Email writing skills"
Presentation on "Email writing skills"Azhar Hussain
 
Training & Development - Validation & Implementation of Training
Training & Development - Validation & Implementation of TrainingTraining & Development - Validation & Implementation of Training
Training & Development - Validation & Implementation of TrainingM R Jhalawad
 
Guidelines for CV Preparation (NATO)
Guidelines for CV Preparation (NATO)Guidelines for CV Preparation (NATO)
Guidelines for CV Preparation (NATO)Tara Nordick
 
Key Performance Indicators
Key Performance IndicatorsKey Performance Indicators
Key Performance IndicatorsCIToolkit
 
MDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKER
MDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKERMDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKER
MDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKERMustafa Bello
 
Training material development guide
Training material development guideTraining material development guide
Training material development guideInstansi
 
Training evaluation
Training evaluationTraining evaluation
Training evaluationNancy Raj
 
Ethiopia TVET at a glance, March 2016latest.ppt
Ethiopia  TVET at a glance, March  2016latest.pptEthiopia  TVET at a glance, March  2016latest.ppt
Ethiopia TVET at a glance, March 2016latest.pptBakalcha Bari
 
KPI for Human Resource Management
KPI for Human Resource ManagementKPI for Human Resource Management
KPI for Human Resource ManagementSCS universal
 
Hr manager kpi
Hr manager kpiHr manager kpi
Hr manager kpizewfurita
 
5 essential tips for business email etiquette.ppt
5 essential tips for business email etiquette.ppt5 essential tips for business email etiquette.ppt
5 essential tips for business email etiquette.pptSherin Thomas
 

What's hot (20)

Email Communication
Email CommunicationEmail Communication
Email Communication
 
Bank credit officer kpi
Bank credit officer kpiBank credit officer kpi
Bank credit officer kpi
 
Why mba presentation
Why mba presentationWhy mba presentation
Why mba presentation
 
Safety (Security) Training
Safety (Security) TrainingSafety (Security) Training
Safety (Security) Training
 
Presentation on "Email writing skills"
Presentation on "Email writing skills"Presentation on "Email writing skills"
Presentation on "Email writing skills"
 
Training & Development - Validation & Implementation of Training
Training & Development - Validation & Implementation of TrainingTraining & Development - Validation & Implementation of Training
Training & Development - Validation & Implementation of Training
 
Guidelines for CV Preparation (NATO)
Guidelines for CV Preparation (NATO)Guidelines for CV Preparation (NATO)
Guidelines for CV Preparation (NATO)
 
Email Etiquette for Business
Email Etiquette for BusinessEmail Etiquette for Business
Email Etiquette for Business
 
Key Performance Indicators
Key Performance IndicatorsKey Performance Indicators
Key Performance Indicators
 
MDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKER
MDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKERMDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKER
MDSSL HUBO QUESTIONNAIRE 88 CHEMICAL TANKER
 
Training material development guide
Training material development guideTraining material development guide
Training material development guide
 
Service life cycle
Service life cycleService life cycle
Service life cycle
 
Training evaluation
Training evaluationTraining evaluation
Training evaluation
 
Ethiopia TVET at a glance, March 2016latest.ppt
Ethiopia  TVET at a glance, March  2016latest.pptEthiopia  TVET at a glance, March  2016latest.ppt
Ethiopia TVET at a glance, March 2016latest.ppt
 
KPI for Human Resource Management
KPI for Human Resource ManagementKPI for Human Resource Management
KPI for Human Resource Management
 
Training ppt
Training pptTraining ppt
Training ppt
 
Safety kpi examples
Safety kpi examplesSafety kpi examples
Safety kpi examples
 
Fire drill report
Fire drill reportFire drill report
Fire drill report
 
Hr manager kpi
Hr manager kpiHr manager kpi
Hr manager kpi
 
5 essential tips for business email etiquette.ppt
5 essential tips for business email etiquette.ppt5 essential tips for business email etiquette.ppt
5 essential tips for business email etiquette.ppt
 

Viewers also liked

Localization Project Management
Localization Project ManagementLocalization Project Management
Localization Project Managementbarakdanin
 
Localization Project Management
Localization Project ManagementLocalization Project Management
Localization Project ManagementMaria Asnes
 
Cursos Agile Think - Framework Scrum - 1/4
Cursos Agile Think - Framework Scrum - 1/4Cursos Agile Think - Framework Scrum - 1/4
Cursos Agile Think - Framework Scrum - 1/4André Vidal
 
Translation, Localization And Globalization Zhu Xianchao
Translation, Localization And Globalization Zhu XianchaoTranslation, Localization And Globalization Zhu Xianchao
Translation, Localization And Globalization Zhu Xianchaozhuxianchao
 
Lean Agile Metrics And KPIs
Lean Agile Metrics And KPIsLean Agile Metrics And KPIs
Lean Agile Metrics And KPIsYuval Yeret
 
How localization can double your revenues
How localization can double your revenuesHow localization can double your revenues
How localization can double your revenuesRWS Moravia
 
проектное управление
проектное управлениепроектное управление
проектное управлениеplupinovich
 
International trade manager kpi
International trade manager kpiInternational trade manager kpi
International trade manager kpiricosjom
 
Localization and globalization in c#
Localization and globalization in c#Localization and globalization in c#
Localization and globalization in c#PaYal Umraliya
 
ActionFlow Social Media Scorecard Template 2.0
ActionFlow Social Media Scorecard Template 2.0ActionFlow Social Media Scorecard Template 2.0
ActionFlow Social Media Scorecard Template 2.0ActionFlow
 
Understanding Social Media Metrics and KPIs by Bansi Patel
Understanding Social Media Metrics and KPIs by Bansi PatelUnderstanding Social Media Metrics and KPIs by Bansi Patel
Understanding Social Media Metrics and KPIs by Bansi PatelBansi Patel
 
Проектные офисы. Сервисная модель и организация.
Проектные офисы. Сервисная модель и организация.Проектные офисы. Сервисная модель и организация.
Проектные офисы. Сервисная модель и организация.ProjectPractice2013
 
Отчетность и аналитика в проектной деятельности
Отчетность и аналитика в проектной деятельностиОтчетность и аналитика в проектной деятельности
Отчетность и аналитика в проектной деятельностиProjectPractice2013
 
DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS
DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS
DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS Natasha Byrne
 
Testing as a Managed Service using SLAs and KPIs
Testing as a Managed Service using SLAs and KPIsTesting as a Managed Service using SLAs and KPIs
Testing as a Managed Service using SLAs and KPIsProlifics
 
Vision goal csf kpi
Vision goal csf kpiVision goal csf kpi
Vision goal csf kpiGlen Alleman
 

Viewers also liked (20)

Localization Project Management
Localization Project ManagementLocalization Project Management
Localization Project Management
 
Localization Project Management
Localization Project ManagementLocalization Project Management
Localization Project Management
 
Cursos Agile Think - Framework Scrum - 1/4
Cursos Agile Think - Framework Scrum - 1/4Cursos Agile Think - Framework Scrum - 1/4
Cursos Agile Think - Framework Scrum - 1/4
 
Translation, Localization And Globalization Zhu Xianchao
Translation, Localization And Globalization Zhu XianchaoTranslation, Localization And Globalization Zhu Xianchao
Translation, Localization And Globalization Zhu Xianchao
 
Lean Agile Metrics And KPIs
Lean Agile Metrics And KPIsLean Agile Metrics And KPIs
Lean Agile Metrics And KPIs
 
How localization can double your revenues
How localization can double your revenuesHow localization can double your revenues
How localization can double your revenues
 
Проектное управление
Проектное управлениеПроектное управление
Проектное управление
 
проектное управление
проектное управлениепроектное управление
проектное управление
 
Best Practices in Translation and Localization of E-learning Courses
Best Practices in Translation and Localization of E-learning CoursesBest Practices in Translation and Localization of E-learning Courses
Best Practices in Translation and Localization of E-learning Courses
 
International trade manager kpi
International trade manager kpiInternational trade manager kpi
International trade manager kpi
 
Localization and globalization in c#
Localization and globalization in c#Localization and globalization in c#
Localization and globalization in c#
 
Localization framework
Localization frameworkLocalization framework
Localization framework
 
Проектный офис
Проектный офисПроектный офис
Проектный офис
 
ActionFlow Social Media Scorecard Template 2.0
ActionFlow Social Media Scorecard Template 2.0ActionFlow Social Media Scorecard Template 2.0
ActionFlow Social Media Scorecard Template 2.0
 
Understanding Social Media Metrics and KPIs by Bansi Patel
Understanding Social Media Metrics and KPIs by Bansi PatelUnderstanding Social Media Metrics and KPIs by Bansi Patel
Understanding Social Media Metrics and KPIs by Bansi Patel
 
Проектные офисы. Сервисная модель и организация.
Проектные офисы. Сервисная модель и организация.Проектные офисы. Сервисная модель и организация.
Проектные офисы. Сервисная модель и организация.
 
Отчетность и аналитика в проектной деятельности
Отчетность и аналитика в проектной деятельностиОтчетность и аналитика в проектной деятельности
Отчетность и аналитика в проектной деятельности
 
DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS
DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS
DIGITAL KPIS EVOLUTION AND MEASUREMENT TOOLS
 
Testing as a Managed Service using SLAs and KPIs
Testing as a Managed Service using SLAs and KPIsTesting as a Managed Service using SLAs and KPIs
Testing as a Managed Service using SLAs and KPIs
 
Vision goal csf kpi
Vision goal csf kpiVision goal csf kpi
Vision goal csf kpi
 

Similar to Localization Metrics and KPIs

Jason uyderv pmi 2 16 12
Jason uyderv pmi 2 16 12Jason uyderv pmi 2 16 12
Jason uyderv pmi 2 16 12Jason Uyder
 
Measuring Success in the Lean IT World
Measuring Success in the Lean IT WorldMeasuring Success in the Lean IT World
Measuring Success in the Lean IT WorldLean IT Association
 
Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...
Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...
Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...Perficient, Inc.
 
Software Quality Dashboard Benchmarking Study
Software Quality Dashboard Benchmarking StudySoftware Quality Dashboard Benchmarking Study
Software Quality Dashboard Benchmarking StudyJohn Carter
 
Daniel Breston - DevOps metrics that matter
Daniel Breston - DevOps metrics that matterDaniel Breston - DevOps metrics that matter
Daniel Breston - DevOps metrics that matteritSMF UK
 
Measure everything - but make NPS the Key
Measure everything - but make NPS the Key Measure everything - but make NPS the Key
Measure everything - but make NPS the Key Andy Kucharski
 
Stephen "Steve" Muzzy Memphis Schools Presentation
Stephen "Steve" Muzzy Memphis Schools PresentationStephen "Steve" Muzzy Memphis Schools Presentation
Stephen "Steve" Muzzy Memphis Schools Presentationsteve muzzy
 
04/21/2011 Meeting - Auditing with Data Analytics
04/21/2011 Meeting - Auditing with Data Analytics04/21/2011 Meeting - Auditing with Data Analytics
04/21/2011 Meeting - Auditing with Data Analyticsacfesj
 
Launching Successful Applications
Launching Successful ApplicationsLaunching Successful Applications
Launching Successful ApplicationsChris Chew
 
Game Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational ExcellenceGame Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational Excellencekushshah
 
it_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptx
it_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptxit_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptx
it_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptxAbdulelah Aljabri
 
BPM (Business Process Management) Introduction
BPM (Business Process Management) IntroductionBPM (Business Process Management) Introduction
BPM (Business Process Management) IntroductionIntegrify
 
SQL Saturday STL 2016 Presentation
SQL Saturday STL 2016 PresentationSQL Saturday STL 2016 Presentation
SQL Saturday STL 2016 PresentationMatthew W. Bowers
 
Case study businessone (en) 1.0
Case study  businessone (en) 1.0Case study  businessone (en) 1.0
Case study businessone (en) 1.0BIcasestudy
 

Similar to Localization Metrics and KPIs (20)

Jason uyderv pmi 2 16 12
Jason uyderv pmi 2 16 12Jason uyderv pmi 2 16 12
Jason uyderv pmi 2 16 12
 
Benchmarking
BenchmarkingBenchmarking
Benchmarking
 
Measuring Success in the Lean IT World
Measuring Success in the Lean IT WorldMeasuring Success in the Lean IT World
Measuring Success in the Lean IT World
 
Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...
Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...
Move from Business Intelligence to Advanced Analytics by Integrating IBM SPSS...
 
June2008 Hdi Quality Presentation
June2008   Hdi Quality PresentationJune2008   Hdi Quality Presentation
June2008 Hdi Quality Presentation
 
Software Quality Dashboard Benchmarking Study
Software Quality Dashboard Benchmarking StudySoftware Quality Dashboard Benchmarking Study
Software Quality Dashboard Benchmarking Study
 
Daniel Breston - DevOps metrics that matter
Daniel Breston - DevOps metrics that matterDaniel Breston - DevOps metrics that matter
Daniel Breston - DevOps metrics that matter
 
PQF Overview
PQF OverviewPQF Overview
PQF Overview
 
Measure everything - but make NPS the Key
Measure everything - but make NPS the Key Measure everything - but make NPS the Key
Measure everything - but make NPS the Key
 
chapter 7.ppt
chapter 7.pptchapter 7.ppt
chapter 7.ppt
 
Stephen "Steve" Muzzy Memphis Schools Presentation
Stephen "Steve" Muzzy Memphis Schools PresentationStephen "Steve" Muzzy Memphis Schools Presentation
Stephen "Steve" Muzzy Memphis Schools Presentation
 
04/21/2011 Meeting - Auditing with Data Analytics
04/21/2011 Meeting - Auditing with Data Analytics04/21/2011 Meeting - Auditing with Data Analytics
04/21/2011 Meeting - Auditing with Data Analytics
 
Launching Successful Applications
Launching Successful ApplicationsLaunching Successful Applications
Launching Successful Applications
 
Game Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational ExcellenceGame Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational Excellence
 
it_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptx
it_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptxit_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptx
it_Define_Service_Desk_Metrics_That_Matter_Storyboard.pptx
 
Business process reengineering
Business process reengineeringBusiness process reengineering
Business process reengineering
 
BPM (Business Process Management) Introduction
BPM (Business Process Management) IntroductionBPM (Business Process Management) Introduction
BPM (Business Process Management) Introduction
 
SQL Saturday STL 2016 Presentation
SQL Saturday STL 2016 PresentationSQL Saturday STL 2016 Presentation
SQL Saturday STL 2016 Presentation
 
Case study businessone (en) 1.0
Case study  businessone (en) 1.0Case study  businessone (en) 1.0
Case study businessone (en) 1.0
 
Case Study- BusinessOne
Case Study- BusinessOneCase Study- BusinessOne
Case Study- BusinessOne
 

Recently uploaded

Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)tazeenaila12
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toumarfarooquejamali32
 
ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...
ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...
ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...ISONIKELtd
 
Entrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizationsEntrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizationsP&CO
 
Chapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptx
Chapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptxChapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptx
Chapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptxesiyasmengesha
 
Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Lviv Startup Club
 
A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.mcshagufta46
 
To Create Your Own Wig Online To Create Your Own Wig Online
To Create Your Own Wig Online  To Create Your Own Wig OnlineTo Create Your Own Wig Online  To Create Your Own Wig Online
To Create Your Own Wig Online To Create Your Own Wig Onlinelng ths
 
Fabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and FestivalsFabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and FestivalsWristbands Ireland
 
Scrum Events & How to run them effectively
Scrum Events & How to run them effectivelyScrum Events & How to run them effectively
Scrum Events & How to run them effectivelyMarianna Nakou
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...Brian Solis
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Winbusinessin
 
7movierulz.uk
7movierulz.uk7movierulz.uk
7movierulz.ukaroemirsr
 
Trauma Training Service for First Responders
Trauma Training Service for First RespondersTrauma Training Service for First Responders
Trauma Training Service for First RespondersBPOQe
 
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 Building Your Personal Brand on LinkedIn - Expert Planet-  2024 Building Your Personal Brand on LinkedIn - Expert Planet-  2024
Building Your Personal Brand on LinkedIn - Expert Planet- 2024Stephan Koning
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfAnhNguyen97152
 
PDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdfPDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdfHajeJanKamps
 
MoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor PresentationMoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor Presentationbaron83
 
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...IMARC Group
 
MC Heights construction company in Jhang
MC Heights construction company in JhangMC Heights construction company in Jhang
MC Heights construction company in Jhangmcgroupjeya
 

Recently uploaded (20)

Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
Harvard Business Review.pptx | Navigating Labor Unrest (March-April 2024)
 
Lecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb toLecture_6.pptx English speaking easyb to
Lecture_6.pptx English speaking easyb to
 
ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...
ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...
ISONIKE Ltd Accreditation for the Conformity Assessment and Certification of ...
 
Entrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizationsEntrepreneurship & organisations: influences and organizations
Entrepreneurship & organisations: influences and organizations
 
Chapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptx
Chapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptxChapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptx
Chapter_Five_The_Rural_Development_Policies_and_Strategy_of_Ethiopia.pptx
 
Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)
 
A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.
 
To Create Your Own Wig Online To Create Your Own Wig Online
To Create Your Own Wig Online  To Create Your Own Wig OnlineTo Create Your Own Wig Online  To Create Your Own Wig Online
To Create Your Own Wig Online To Create Your Own Wig Online
 
Fabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and FestivalsFabric RFID Wristbands in Ireland for Events and Festivals
Fabric RFID Wristbands in Ireland for Events and Festivals
 
Scrum Events & How to run them effectively
Scrum Events & How to run them effectivelyScrum Events & How to run them effectively
Scrum Events & How to run them effectively
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024
 
7movierulz.uk
7movierulz.uk7movierulz.uk
7movierulz.uk
 
Trauma Training Service for First Responders
Trauma Training Service for First RespondersTrauma Training Service for First Responders
Trauma Training Service for First Responders
 
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 Building Your Personal Brand on LinkedIn - Expert Planet-  2024 Building Your Personal Brand on LinkedIn - Expert Planet-  2024
Building Your Personal Brand on LinkedIn - Expert Planet- 2024
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
 
PDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdfPDT 88 - 4 million seed - Seed - Protecto.pdf
PDT 88 - 4 million seed - Seed - Protecto.pdf
 
MoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor PresentationMoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor Presentation
 
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
Boat Trailers Market PPT: Growth, Outlook, Demand, Keyplayer Analysis and Opp...
 
MC Heights construction company in Jhang
MC Heights construction company in JhangMC Heights construction company in Jhang
MC Heights construction company in Jhang
 

Localization Metrics and KPIs

  • 1. Metrics and KPIs for Localization
  • 2. Agenda • Definitions — what are indicators? • A brief case for metrics and KPIs — why they matter? • A brief case against KPIs — reasons to be cautious • Practical Localization Metrics 101 • Practical Localization Metrics 200 • Program Dashboards • Developing Localization Metrics • Localization Performance Indicators 300+
  • 4. Data, Metrics and Indicators Data is a collection of facts and statistics collected together for reference or analysis A metric is a collection of the same type of data “You can’t pick your data, but you must pick your metrics!” An indicator provides evidence that a certain condition exists or certain results have or have not occurred.
  • 5. KRIs, PIs, and KPIs 1. Key result indicators (KRIs) tell you how you have done in a perspective. 2. Performance indicators (PIs) tell you what to do. 3. Key Performance indicators (KPIs) tell you what to do to increase performance dramatically.
  • 6. Why You Need KPIs “What gets measured gets done” – Attributed to many* “Without a standard, there is no logical basis for making a decision or taking action.” - Joseph Juran • Some benefits of KPIs • KPIs shift a group to proactive mode • Healthier partner relationships • Pay-for-performance becomes easier to implement (or even possible) • Scalability, consistency, normalcy • Move away from “constant crisis” style • Ability to focus teams on essential goals
  • 7. KPIs in Action: British Airways In the 1980s turned the company around by focusing on just one KPI: • Financial – Operating margin • Customers – Recommendation • Operations – Punctuality • Colleagues – Colleague involvement Actions: - Doubled cleaning crew - Prioritized refueling - Personal call policy Results: - Lower fuel costs - Lowered airport surcharges - Reduced lost baggage - Improved customer satisfaction - Improved vendor relationships - More satisfied employees
  • 8. A Case Against KPIs “Not everything that counts can be counted, and not everything that can be counted counts.” Albert Einstein “People will do what management inspects, not necessarily what management expects.” Dean Spitzer
  • 10. The Best Place to Start… Common localization metrics • Percentage of on-time deliveries • Data points: hand-off date, requested delivery date, approved delivery date, actual delivery date • Number of linguistic or technical quality errors • Data points: quality framework and measurement methodology • Pass/fail rates and First pass yield • Data points: pass criteria, pass/fail data • Quantified quality tiers • Average throughput rate • Weighted words, hand-offs, pages per business day • Fully-loaded cost per word
  • 11. Start with Data 1. Evaluate the data you have. What are the gaps? 2. Tracking sheets and invoicing data is a good place to start 3. An existing TMS will typically have good sources of raw data to export 4. Assign a clear owner for data integrity 5. Normalized data is a MUST 6. Excel is good for modeling and getting started, but it stops being effective quickly 7. SharePoint has decent SQL back-end support. Consider customizing the user fields by task or station 8. Access and SQL are good next steps, but require more specialized skills to configure 9. More advanced systems exist for more mature programs
  • 12. Percentage of On-time Deliveries Calculation: ∑ total deliveries that are on-time or early / total deliveries Best value: Paces production, focuses team on meeting deadlines Variables to consider: Project complexity is an issue Ignoring holidays often means you get the B-team You should plan to track 4 data points MBO opportunity: Reward meeting target OTD goals (e.g. 98% or higher = +2% month) Penalize late deliveries (e.g. <90% = -2%)
  • 13. Average Language Quality Scores Calculation: ∑ total weighted errors / total word count Best value: Pushes client and suppliers to define clear quality expectations Brings common linguistic issues into focus — easier to address. Practical actions. Variables to consider: Track translators by ID (not name), use RCA, arbitration & valid statistics Measure samples and get feedback back to translators in a timely manner MBO opportunity: Bonus for keeping average scores above a target (e.g. 95%) Issue warning letter for fails. 3 Fails on the same language results in an escalation.
  • 14. Pass / Fail Rate and First Pass Yield Calculation: ∑ total passed tests / total tests; ∑ total accepted jobs / total jobs Best value: Incentivizes consistency Variables to consider: Track translators by ID (not name) Translators should not be penalized for bad source or instructions Use RCA, arbitration & valid statistics to resolve issues MBO opportunity: Bonus for keeping pass rate above a target (e.g. 95%) Issue warning letter for fails. 3 Fails on the same language results in an escalation.
  • 15. End-User Escalations / Resolution Rate Calculation: ∑ total escalations / total products or deliveries per region Best value: This is one of several customer-centric metrics that is a “reality check” to traditional quality tracking. How much does quality matter? Variables to consider: Important to understand root causes Distinguish between I18n, L10n and source failures Often difficult to measure and normalize MBO opportunity: Bonus for keeping pass rate below a target (e.g. .1%)
  • 16. Throughput Calculation: ∑ weighted word count / (network days – holidays) Best value: Helps understand performance under range of operating conditions Encourages a comprehensive view of program capacity Measures scalability and program resiliency Better anticipate bottlenecks and plan capacity Variables to consider: Measure by type of work (complexity) and priority MBO opportunity: Helpful to build a business case for investments to reduce TAT
  • 18. Quantified Quality Tiers Process: Define quality process and error thresholds per threshold Best value: Creates opportunity for different workflows and pricing Allocates QC resources more effectively Variables to consider: Consider testing to qualify and monitor individual translators It is possible to set this up without additional pricing tiers In-line review, self-certification documentation and tracking
  • 19. Prioritization Process: Create clear process SLA guidelines for priorities (at least 2) Best value: Creates opportunity for different workflows and pricing Allocates QC resources more effectively More likely to get the urgent things done first, first come first served By default everything has same priority and so is first
  • 20. Theory of Constraints • Add buffers to production to optimize the performance of the most critical resources — steady work flow eliminates exception management model • Ideal for reigning in large chaotic workflows • Set prioritization to allow for buffering • Set minimum capacity expectations at the maximum number of high-priority projects you expect to require (e.g. 2k words per day)
  • 21. Fully-Loaded Price Per Word Calculation: ∑ total outsourced cost/total number of words per project Best value: Total program efficiency including development Helps identify inefficient production, especially if the average is much higher than the new word rate Variables to consider: Larger new projects will bump the average up Higher leverage brings the average down. Track by project type MBO opportunity: Can’t be used as an MBO for localization teams only, but it can support content development and tech initiatives
  • 23. Effective Localization Dashboards… • …contain only the highest-priority indicators • …are updated regularly • …are validated regularly Tips: • Consider designating a data entry and/or analyst to manage the data • Careful tracking discipline is not a critical path task unless you make it one • Keep it simple
  • 24. Effective Localization Dashboards • Schedule • % on-time delivery (the KPI should be interpreted as “as agreed” rather than “as requested.” The former impacts the production line, the latter is nice to have. • Quality • Pass/Fail target periodic performance (yes/no). The data and metrics can be there, but on a dashboard you just want to know if OK or not. • Cost • Could contain periodic costs to date, e.g. against a PO or budget, but most localization teams do not have much control over costs.
  • 25. Example: Mozilla Notice the priorities: • Missing translations • Errors • Deadline • Priority • % complete
  • 26. Example: Basic Localization Dashboard Area KPI Name Target Compliance Actual Linguistic Language quality pass ≥ 95% 97.8% Project management On-time deliveries ≥ 90% for traditional waterfall 99.1% Leadership of project management Feedback 4.0 Engineering Technical accuracy of deliveries ≥ 95% 98.3% Production handback issues < 5/month 2.3 Finance Timeliness of invoicing 100% 100% Invoicing correctness 0% reinvoicing or corrections Pass
  • 28. A Phased Approach to Developing KPIs • Phase 1: All models require starting with good data • If you don’t have reliable and consistent sources of data, nothing else is possible • Set up the basics and start tracking them: delivery data, priority quality, and any classifications you need • Phase 2. Establish your quality measurement system • Without solid quality data, MBO goals are not possible • “Intuitive quality management” is worse than the default “if I don’t hear about it the quality is fine” management plan
  • 29. • Phase 3: Start setting goals • Patterns will emerge quickly that will drive focused actions almost immediately • Use these to normalize data and metrics • Clearly and consistently communicate goals and expectations • Phase 4: MBOs • Once solid data and processes have been set, MBOs are possible • Use the framework to rework pricing and terms • Use a combination of sticks and carrots A Phased Approach to Developing KPIs
  • 30. Some Additional Thoughts on Metrics • It’s helpful to manage at the job level. Hand-offs and projects are merely containers for jobs. • A job is a deliverable (e.g. German UI file) • Aggregating data to a dashboard is OK, but the data should always be stored as jobs • Focus on data integrity • Dates should be dates, numbers should be numbers. Any data point used in a metric needs to have a name, type, range, money should be money and it should have a validation method (to detect errors). • Data is not free — Don’t track what you can’t use • Often data is collected based on the “that might be interesting” policy • Use ISO!
  • 32. Localization Business Metrics 300+ Overall cost of ownership • Globalization (G11n) headcount • Localization (L10n) headcount • Outsourced vs. in-house G11n costs • G11n costs (outright) • L10n costs (outright) • L10n cost against L10n product revenue • Outsourced L10n per word cost • In-house L10n per word cost • Internationalization (I18n) resources • Internal and external linguist cost ROI (return on investment) • L10n product ROI • Tiered market definitions Parking Lot: Potential future metrics • International web site revenue • Click-through rates, and how that translates into dollars • Customer support — % offered in local vs. source language • Cost of localizing web sites, web presence (download center, online store, etc.) • Partial vs. full product localization
  • 33. Additional Localization Performance Indicators • Presales metrics • Business metrics for globalization • After-sales metrics
  • 34. Presales Metrics • International market growth rate • Relative market share • Return on investment (ROI) in localization • Return on localization assets • Customer online engagement • Number of qualified international leads • Cost per qualified international lead • Online share of voice • Brand awareness • Aided, unaided
  • 35. Business Metrics for Globalization • Total localization headcount • Total localization headcount relative to revenues • Total localization costs • Total localization costs relative to revenues • Fully-loaded average cost per word • Average cost of outsourced word • Time to market • Opportunity cost of a delayed release
  • 36. After-Sales Metrics • Support call deflection • First contact resolution • Brand recognition by region • Localized product sentiment analysis • Customer satisfaction • Loyalty metric such as Net Promoter Score (NPS) • Praise-to-complaint ratio • Track rework, warranties, recalls

Editor's Notes

  1. Lots of people get this wrong. Many people think if KPIs as any metrics. A Key Performance Indicator (KPI) is a measurable value that demonstrates how effectively a company is achieving key business objectives. Organizations use KPIs to evaluate their success at reaching targets. Parmenter, D. (2010). Key performance indicators: Developing, implementing, and using winning KPIs, second edition (2nd ed.). Hoboken: John Wiley & Sons http://www.hfrp.org/publications-resources/browse-our-publications/indicators-definition-and-use-in-a-results-based-accountability-system
  2. “What gets measured gets done.” It has been attributed to Peter Drucker, Tom Peters, Edwards Deming, Lord Kelvin and others. https://christophelecoent.files.wordpress.com/2012/11/common-sense-and-bureaucracy.jpg
  3. The theory of constraints (TOC) is an overall management philosophy introduced by Eliyahu M. Goldratt in his 1984 book titled The Goal, that is geared to help organizations continually achieve their goals.[1] Goldratt adapted the concept to project management with his book Critical Chain, published 1997.
  4. Tim Hussey, Locworld Oct 16, 2003