DAS Slides: Data Quality Best Practices

Copyright Global Data Strategy, Ltd. 2019
Data Quality Best Practices
Donna Burbank & Nigel Turner
Global Data Strategy, Ltd.
August 22nd 2019
Twitter Event hashtag: #DAStrategies

Global Data Strategy, Ltd. 2019
Donna Burbank
2
Donna is a recognised industry expert in
information management with over 20 years
of experience in data strategy, information
management, data modeling, metadata
management, and enterprise architecture.
Her background is multi-faceted across
consulting, product development, product
management, brand strategy, marketing,
and business leadership.
She is currently the Managing Director at
Global Data Strategy, Ltd., an international
information management consulting
company that specializes in the alignment of
business drivers with data-centric
technology. In past roles, she has served in
key brand strategy and product
management roles at CA Technologies and
Embarcadero Technologies for several of the
leading data management products in the
market.
As an active contributor to the data
management community, she is a long time
DAMA International member, Past President
and Advisor to the DAMA Rocky Mountain
chapter, a contributor to the DMBOK 2.0,
and was recently awarded the Excellence in
Data Management Award from DAMA
International in 2016.
Donna is also an analyst at the Boulder BI
Train Trust (BBBT) where she provides advice
and gains insight on the latest BI and
Analytics software in the market. She was on
several review committees for the Object
Management Group’s for key information
management and process modeling
notations.
She has worked with dozens of Fortune 500
companies worldwide in the Americas,
Europe, Asia, and Africa and speaks regularly
at industry conferences. She has co-
authored two books: Data Modeling for the
Business and Data Modeling Made Simple
with ERwin Data Modeler and is a regular
contributor to industry publications. She can
be reached at
donna.burbank@globaldatastrategy.com
Donna is based in Boulder, Colorado, USA.
Follow on Twitter @donnaburbank
Twitter Event hashtag: #DAStrategies

Nigel Turner
Nigel Turner has worked in Information
Management (IM) and related areas for
over 20 years. This experience has
embraced Data Governance,
Information Strategy, Data Quality, Data
Governance, Master Data Management,
& Business Intelligence.
He spent much of his career in British
Telecommunications Group (BT) where
he led a series of enterprise wide IM &
data governance initiatives.
After leaving BT in 2010 Nigel became
VP of Information Management Strategy
at Harte Hanks Trillium Software, a
leading global provider of Data Quality
& Data Governance tools and
consultancy. Here he engaged with over
150 customer organizations from all
parts of the globe.
Currently Principal Consultant for EMEA
at Global Data Strategy, Ltd, he has been
a principal consultant at such firms as
FromHereOn and IPL, where he has led
Data Governance engagement with
customers such as First Great Western.
Nigel is a well known thought leader in
Information Management and has
presented at many international
conferences. He also works part time at
Cardiff University, where he is setting up
a Student Software Enterprise company.
In addition he has also been a part time
Associate Lecturer at the UK Open
University where he taught Systems &
Management.
Nigel is very active in professional Data
Management organizations and is Vice-
Chair Of the UK Data Management
Association (DAMA). He was the joint
winner of DAMA International’s 2015
Community Award for the work he
initiated and led in setting up a
mentoring scheme in the UK where
experienced DAMA professionals coach
and support newer data management
professionals.
Nigel is based in Cardiff, Wales, UK.
Follow on Twitter @NigelTurner8
Today’s hashtag: # DAStrategies
3

DATAVERSITY Data Architecture Strategies
• January 24 - on demand Emerging Trends in Data Architecture – What’s the Next Big Thing?
• February 18 - on demand Building a Data Strategy - Practical Steps for Aligning with Business Goals
• March 28 - on demand Data Modeling at the Environment Agency of England - Case Study
• April 25 - on demand Data Governance - Combining Data Management with Organizational Change
• May 23 - on demand Master Data Management - Aligning Data, Process, and Governance
• June 27 - on demand Enterprise Architecture vs. Data Architecture
• July 25 - on demand Metadata Management: Technical Architecture & Business Techniques
• August 22 Data Quality Best Practices (w/ guest Nigel Turner)
• Sept 26 Self Service BI & Analytics: Architecting for Collaboration
• October 24 Data Modeling Best Practices: Business and Technical Approaches
• December 3 Building a Future-State Data Architecture Plan: Where to Begin?
4
This Year’s Lineup

Today’s Topic
Tackling Data Quality problems requires more than a series of tactical, one-off improvement
projects.
By their nature, many Data Quality problems extend across and often beyond an organization.
Addressing these issues requires a holistic architectural approach combining people, process, and
technology.
5

Data Quality is Part of a Larger Enterprise Landscape
6
Successful Data Quality Improvement Requires Many Inter-related Disciplines
“Top-Down” alignment with
business priorities
“Bottom-Up” management &
inventory of data sources
Managing the people, process,
policies & culture around data
Coordinating & integrating
disparate data sources
Leveraging & managing data for
strategic advantage

Agenda
7
What we’ll cover today
What is data quality and why does it matter
What can happen when data quality goes wrong
The continuing impact of poor data quality
Traditional approaches to tackling data quality
Holistic approaches to tackling data quality
Q & A

DataQuality–aSimpleDefinition
Data that is demonstrably
fit for purpose
8
Meets defined business needs for:
• Accuracy
• Completeness
• Reliability
• Accessibility
• Timeliness
8

Poor Data Quality:
Overall Impact on Companies & Organizations
ECONOMIC:
REVENUES, COSTS, PROFITS
LAW & REGULATION
BRAND, REPUTATION &
CUSTOMER LOYALTY
9

Recent Data Quality Bloopers – Online Retail
• Lens normally retails at circa $13,000
(around £10,700)
• At start of Prime Day, price quoted by
Amazon was $94.98 (£78)
• Error spotted and price adjusted to
$9,498 (£7,816)
• Hundreds purchased at $94.98 (£78)
• Amazon honored the deal at a potential
loss of $7,738 (£6,370) per lens
10
Canon EF 800mm f/5.6L IS
Telephoto Lens

Recent Data Quality Bloopers – Healthcare
• Man entered hospital for a cystoscopy
procedure
• Unfortunately his name very similar to
another patient awaiting surgery
• When he came around, he found that he
had been confused with the other patient
• Given a circumcision instead of a
cystoscopy
• One of a string of similar errors at the
hospital so major investigation
undertaken
11

Data Quality – Real Stakeholder Feedback 2019
12
“We should be checking the
data before we enter it on the
system but we don’t have the
time”
“There is a lack of appreciation
of what happens to data from
the front end to the back end”
“There is no
accountability for
bad quality data”
“Our customers say ‘you’re
embarrassing yourselves –
the maths are wrong!’”
“We’ve got lots of data,
but it’s hard to connect it”
“If we could just know
that we can trust what
we are looking at”
“A lot of time (spent) fixing
things & not enough time
to be creative about the
future”
“We need to be Partner-
ready, but every partner
meeting discusses the
challenges with data”
“We cannot achieve the
growth we need if we
cannot sort out the
data.”
“We only know if we have a data
problem if a customer contacts
us…… [too much marketing activity
is done] in blind faith… I press ‘send’
and hope for the best”
“Our systems don’t talk to
each other”

The Industry Impact of Poor Data Quality – The Evidence
On average, half of all
organisations believe
at least 26% of their data is
inaccurate
(Source: BARC 2019)
On average, poor data
quality costs companies
between 15-25% of revenue
(Source: MIT Sloan 2017)
The US economy loses
$3.1 trillion a year
because poor data quality
(Source: IBM 2016)
Poor quality customer data is
costing UK companies an average
of 6% of their annual revenues
(Source: Royal Mail Data Services 2017)
13
13

Why Does Poor Data Quality Persist?
• The data world has become more complex and
diffuse
• The world changes, and data models the world
• Not recognising that poor data quality is a
business problem, not an IT problem
• People will make mistakes with data
• Conflict or absence of common data definitions
& metadata context
• The data Newton’s Cradle
• Lack of accountability for improving data
14

The Data World Becoming More Complex….
15

B2C Data Volatility – UK facts
3.2 million people move
house each year
821,600 babies are born
each year
568,800 people die
each year
126,400 get divorced 517,800 immigrants
come from overseas
352,100 leave the UK
16

B2B Data Volatility – UK facts
Average decay of UK B2B contact
database is 2% per month
There are 4.9 million businesses
in the UK
489,636 new companies start up
every year
30% of people change email
addresses each year
17

It’s NOT an IT Problem…It’s a Business Problem
• Human error
• No data accountability
• Poor training
• Internal politics
• Denial
• Data capture & user design failures
• Multiple data silos
• Interface errors
• Poor data architecture & design
• Poor business process design
• Process failures
• Flawed goal setting
• No agreed data standards
18

Lack of Governance & Accountability for Data
19
In many organizations, nobody is formally
responsible for data and its improvement…
… therefore bad data never gets
systematically fixed
“If we are all supposed to be responsible, no
one is responsible and nothing changes”
(Quote from senior GDS client – Professional Services Organisation 2019)

Turning the
Data Quality
tanker around
20

Turning the Data Quality Oil Tanker Around: Key ‘Must Dos’
21
• Implement a business led, structured Data Governance framework &
organization to ensure clear priorities for data quality improvement
• Develop reusable data improvement approaches that can be applied
across the organization and able to tackle traditional & new data
challenges
• Automate data quality improvement – invest in a data quality toolkit to
support reusable approaches
• Link data quality to Data Architecture – design improvements into data
design & implementation
• Build a business case for every data quality improvement initiative so
that the focus is on highest value work
• Manage each initiative via a Data Improvement Plan

Applying a Structured Data Governance Framework
22
22
Organization &
People
Process &
Workflows
Data Management &
Measures
Culture &
Communication
Vision & Strategy
Tools & Technology
Business Goals &
Objectives
Data Issues &
Challenges

Evolution of Data Quality Since 2000…
TIMELINE
BATCH REAL TIME / ONLINE
REACTIVE PROACTIVE
IT DRIVEN BUSINESS DRIVEN
PLATFORM SPECIFIC ENTERPRISE WIDE
DATA CLEANSE DATA RE-ENGINEERING
MANUAL AUTOMATED
OPERATIONAL FOCUS REPORTING / ANALYTICS FOCUS
IN-PLATFORM DATA QUALITY DATA QUALITY AS A SERVICE
SILOED FOCUS PART OF HOLISTIC DM CHANGE
23

Traditional Approaches to Data Quality
• Inspect data sources and highlight data deficiencies, omissions
and duplication
• Develop data standards and common data definitions
• Build business rules to enforce and police data standards
• Automate data cleanse and enhancement projects, using the
business rules defined
• Embed the standards & rules into both batch and real-time
environments to keep the data clean
• Produce Data Quality KPIs and measures to monitor ongoing
quality and track trends
24
Will continue to have great value

The New Age of Data Quality
• Increased focus on real time data validation and improvement at the point of data
creation, ingestion or use
• Automated digitized processes and self-service will fail if the data is not fit for purpose
• Data validation and improvement must be done in real time on large data volumes & varieties
• ‘After the fact’ data cleanse and improvement is now too late
• Opportunity to exploit IoT and AI to develop self-checking data quality capabilities
• Business users need more control over the creation & management of business rules
• They need the ability to create business rules dynamically when preparing data
• Different data users will potentially require the application of varying business rules
• The paradigm where business rules are created and held centrally by IT is obsolete
• End user self-service data quality functionality is essential
• Data preparation & formatting
• Data parsing & cleansing
• Data enhancement & enrichment
• Toolsets must support a wider variety of platforms & data types
• Legacy and Big Data environments (e.g. Data Warehouses and Data Lakes)
• Real-time and batch
• Structured / semi-structured / unstructured data types – via Data Profiling, Data Preparation &
Metadata tagging
25
New approaches for data quality in digital organisations

Finding the Right Balance
26
Human
Automated
Resolve at Source Resolve via Post-Processing
• Business Process Change
• Policies & Procedures
• Governance Steering Committees
• End User Training
• Industry Advisory Councils
• Data definition & glossary
• Data Quality Working Groups
• Application-Driven Data Entry & Workflow
• Application-level data validation
• Database-level data validation & integrity
(data models)
• Data Quality tool validation at source
• Data Cleansing Tools and/or SQL
• ETL (Data Warehouse)
• Data Stewardship
• “Conscious Disregard”
Proactive Business Management Reactive Business Management
Proactive Technical Management Reactive Technical Management
• Data Audit & Dashboards
• External Data Sources

Repeatable
Approaches:
need for an
integrated
toolset
Problem
Ticketing
Workflow
Data Glossaries
/ Metadata
Repositories
Data Modelling
Data Analysis /
Profiling
Data
Preparation
Data Quality
Re-engineering
Data Analytics
& Data Mining
Data Reporting
&
Dashboarding
27

Data
Architecture
Data Quality
Data
Governance
Data Quality, Data Governance & Data Architecture –
the Virtuous Circle
Provides the
structure to deliver
Drives the
need for
Scopes & helps
prioritize
DATA ARCHITECTURE DATA GOVERNANCE DATA QUALITY
Model entity & attribute
relationships
Overarching strategic
framework
Data profiling baselines
current state
Scope & prioritize key data Help identify data
stakeholders
Raise awareness of data
quality issues in source data
Develop business led
business rules
Assign owners & stewards to
lead data quality efforts
Data cleanse, enrichment &
maintenance
Models provide
communication tools
Ensure data quality
alignment with changing
business needs
Automate business rule
enforcement through DQ
tools
First step in defining data
standards & KPIs
Creates vehicle for cross-
organization collaboration
Metrics to provide factual
foundation for action
Links business rules >
definitions > physical data
designs
Leads business case
development & realization
Helps build the business case
for action
28

What is a Data Improvement Plan?
A Data Improvement Plan (DIP) is a formal plan to
specify and manage improvements to a specified data
domain and / or data problem
29
The benefits of a Data Improvement Plan are:
• sets out goals and expectations for data improvement
• acts as a focal point for all data improvement activities
• prioritises improvement activities
• can be used to track improvements and communicate
successes
• can evolve to align with the changing needs of the
business
• data domain DIPs can be rolled up to form the core of a
company wide Data Improvement Programme

Creating a Data Improvement Plan
30
INVESTIGATE
ORGANISE
PRIORITISE
IMPROVE
STEP 1
STEP 2
STEP 3
STEP 4
INDICATES
ITERATION
• Define data domain and its uses (Processes and Functions)
• Identify data stakeholders (Creators, Modifiers, Consumers)
• Engage with stakeholders (e.g. interviews, workshops, documents)
• Identify key data fields (i.e. what data really matters)
• Baseline data (Systems & Quality) & Data Governance maturity
• Identify data problems & impact
• Set up Data Domain Working Group (Business, IT and key stakeholders)
• Create a log of data problems, opportunities and business impact
• Initially identify and define potential improvement initiatives (People / Process /
IT)
• Prioritise data problems
• Define improvement projects
• Create improvement team(s) (from Working Group & others)
• Produce Motivation Model & business case(s) for action
• Finalise initial Data Improvement Plan for Steering Group endorsement
• Launch improvement initiatives
• Set KPIs and success measures
• Perform root cause analysis and propose & evaluate changes
• Design and implement improvements (People / Process / IT)
• Produce improvement plans, monitor progress and measure data improvements
• Log benefits, publicise successes, and identify lessons learnt

DATA
DOMAIN
BENEFIT TYPE DESCRIPTION EXPECTED REVENUE INCREASE / COST SAVING
Year 1 Year 2 Year 3
CUSTOMER COST
REDUCTION
BONUS ABUSE REDUCTION 125,000 £125,000 £125,000
COST
REDUCTION
EMAIL M/K COST REDN £10,000 £10,000 £10,000
COST
REDUCTION
REDUCTION
IN 3RD PARTY ADDRESS CLEANSE
£50,000 £50,000 £50,000
SALES RISK
AVOIDANCE
AUTOMATED
REGULATORY
REPORTS
£20,000 £20,000 £20,000
REVENUE INCREASE CROSS-SELLING OPPS 50,000 50,000 50,000
TOTAL £255,000 £255,000 £255,000
Business Case: Example of Benefit Analysis
31
31

Over $800M from Data Quality Improvement
32
• BT Group plc - a British multinational telecommunications
holding company which had the following challenges:
• Business Agility to deliver new products and services
• Regulatory Compliance to meet industry demands in a
heavily-regulated industry
• Customer Satisfaction through positive customer
interaction.
• Unfortunately, these efforts were being hampered by poor
data quality:
• Poor Supplier & Customer data hampered self-service
interactions
• Inaccurate inventory led to increased capital
expenditure
• Billing errors caused customer dissatisfaction
• To improve Data Quality, BT embarked on an enterprise-wide
data improvement journey which included both technology
and culture change.
The Challenge
• As a result, BT achieved in aggregate more than $800
million in quantified benefits including:
• Capital Cost Avoidance: Through improving accuracy of
inventory data, BT optimized equipment inventory and
reduced inventory costs.
• Improved Revenue Assurance: By improving Billing Data
accuracy, revenue loss decreased from more than 15% to
less than 1%.
• Productivity Gains in B2B Processes: By resolving data
quality issues, BT was able to automate customer and
supplier interactions, and reduce cycle times and manual
effort.
The Result
A Focus on Data Quality Yields Big Benefits for BT 1
1 Gartner, Publication Number G00138085, 24 March 2006

Summary
• Addressing data quality issues requires a holistic architectural approach combining people,
process, and technology.
• Data quality is complex because businesses and organizations are complex
• Data governance helps orchestrate the people, processes and organizational structure required
to improve data quality
• Data architecture provides business and technical alignment to implement data quality business
rules into core systems
• Build quantifiable data improvement plans to show demonstrable ROI
33

DATAVERSITY Data Architecture Strategies
• January 24 Emerging Trends in Data Architecture – What’s the Next Big Thing?
• February 18 Building a Data Strategy - Practical Steps for Aligning with Business Goals
• March 28 Data Modeling at the Environment Agency of England - Case Study (w/ guest Becky Russell from the EA)
• April 25 Data Governance - Combining Data Management with Organizational Change (w/ guest Nigel Turner)
• May 23 Master Data Management - Aligning Data, Process, and Governance
• June 27 Enterprise Architecture vs. Data Architecture
• July 25 Metadata Management: Technical Architecture & Business Techniques
• August 22 Data Quality Best Practices (w/ guest Nigel Turner)
• Sept 26 Self Service BI & Analytics: Architecting for Collaboration
• October 24 Data Modeling Best Practices: Business and Technical Approaches
• December 3 Building a Future-State Data Architecture Plan: Where to Begin?
34
Join Us Next Month

About Global Data Strategy, Ltd
• Global Data Strategy is an international information management consulting company that
specializes in the alignment of business drivers with data-centric technology.
• Our passion is data, and helping organizations enrich their business opportunities through data and
information.
• Our core values center around providing solutions that are:
• Business-Driven: We put the needs of your business first, before we look at any technology solution.
• Clear & Relevant: We provide clear explanations using real-world examples.
• Customized & Right-Sized: Our implementations are based on the unique needs of your organization’s
size, corporate culture, and geography.
• High Quality & Technically Precise: We pride ourselves in excellence of execution, with years of
technical expertise in the industry.
35
Data-Driven Business Transformation
Business Strategy
Aligned With
Data Strategy
Visit www.globaldatastrategy.com for more information

Questions?
36
• Thoughts? Ideas?

DAS Slides: Data Quality Best Practices

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to DAS Slides: Data Quality Best Practices

Similar to DAS Slides: Data Quality Best Practices (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

DAS Slides: Data Quality Best Practices