A Strategic Approach to Improving Data Quality

Maybe if we plan for
better data, we’ll
actually produce
better data…

A Strategic Approach to Data Quality: A Dataversity Webinar
Laura Sebastian-Coleman, Ph.D., IQCP
October 15, 2013

Goals & Agenda
• Goals
– Get you to think strategically about data quality improvement
– Provide concrete suggestions about actions you can take to assess your
organization’s readiness to adopt a strategic approach

• Agenda
– Brief introduction
– An important disclaimer
– Review definitions and some assumptions and about how following concepts
are connected: Strategy, Data, Data Quality, Data Quality Assessment, Data
Quality Strategy
– Review the 12 Directives of Data Quality Strategy
– Discuss the actions you can take to assess your organization’s readiness to
adopt a strategic approach to improving data quality

– Comments, questions, discussion

Confidential property of Optum. Do not distribute or reproduce without express permission from Optum.

2

About Optum
• Optum is a leading information and technology-enabled health services
business dedicated to helping make the health system work better for
everyone.
• With more than 35,000 people worldwide, Optum delivers intelligent,
integrated solutions that modernize the health system and help to
improve overall population health.
• Optum solutions and services are used at nearly every point in the
health care system, from provider selection to diagnosis and treatment,
and from network management, administration and payments to the
innovation of better medications, therapies and procedures.
• Optum clients and partners include those who promote wellness, treat
patients, pay for care, conduct research and develop, manage and
deliver medications.
• With them, Optum is helping to improve the delivery, quality and cost
effectiveness of health care.

3

About me
• 10+ years experience in data quality in the health care industry.
• Working on the IT side of things, in data warehousing
• My thinking about data quality and data governance has been
influenced by the demands of data warehousing
• Author of Measuring Data Quality for Ongoing Improvement : A Data
Quality Assessment Framework (2013).
• Have also worked in banking, manufacturing, distribution, commercial
insurance, and academia.
• This combination of experiences has influenced my understanding of
data, quality, and measurement.
• It has also led me to conclude that most organizations do not think
strategically about data.
– Data is still treated more as a by-product of processes than as a product
itself.
– Most organizations assume data from different processes should fit together,
and they feel pain when that assumption turns out to be incorrect.

4

Disclaimer!
• None of the 12 Directives is new. And none of them should be surprising.
• They are synthesized from thought leaders in Data Quality and Data
Governance – and from thought leaders in product quality who preceded them.
Many of them will be familiar and, I hope, obvious. All are interconnected.
• What is new is a set of concrete steps you can take to assess your
organization’s readiness to move forward with an appropriate sub-set of the
directives.
• The directives are also based on the assumption that most of us face at least
these common challenges:
– Organizational systems have evolved over time

through a series of undocumented compromises.
– Data is created in disparate places within the organization
(silos) and it does not always fit together in expected ways.
– Critical knowledge about data is stored in the heads of

Subject Matter Experts and is not directly accessible to
data consumers.


5

Strategy
• Military origin of the word: strategia , the Greek word for generalship,
was first used to describe military operations and movement.
– Strategy is planning for a set of engagements (battles and other military
interventions) that will achieve the overall goal of a war.
– Tactics describe how each of these engagements will be carried out.

– Tactical “success” that does not contribute to strategy—“winning the battle
but losing the war”—is not success at all.

• What does this tell us about any strategy?
– Strategy is intentional:

– To be strategic is to plan for success by thinking in terms of time and space.
– What do you want to accomplish?
– Where do you want to be within a defined time period?
– How do you get there?


6

Strategy
• The purpose of a business or an organizational strategy is to align
work efforts with long-term goals and to plan for achieving overall
goals.
• Strategy requires:
– A clear vision & mission—where the organization wants to go

– An understanding of current state—where it is now
– Tactics—how it will move from where it is to where it wants to be

• Strategy provides criteria to set priorities and to make decisions when
conflicting needs arise between or within teams.
– Of course, for strategy to be effective, people must actually use these
criteria.


7

Conceptual Relation between Strategic Planning,
Tactical Execution, and Strategic Success


8

Warning!
• Strategy is exciting and motivating!
• People like to know where they are going and how to get there.
Sometimes they even try to do everything at once.

• Unfortunately, that does not usually work.

• Strategy is about planning. So, to repeat, a plan requires defining:
– Where the organization wants to go
– Where it is now
– How it will move from where it is to where it wants to be

• Data Strategy requires defining these goals in relation to the role that data
plays within your organization.

9

Data
• The New Oxford American Dictionary (NOAD) defines data as “facts and
statistics collected together for reference or analysis.”
• ASQ defines data as “A set of collected facts.” ASQ identifies two kinds of
numerical data: “measured or variable data … and counted or attribute data.”
• ISO defines data as “re-interpretable representation of information in a
formalized manner suitable for communication, interpretation, or processing”
(ISO 11179).
• I define data as: abstract representations of selected characteristics of realworld objects, events, and concepts, expressed and understood through
explicitly definable conventions related to their meaning, collection, and
storage.
• Observations:
– Data tries to tell the truth about the world (“facts”)
– Data is formal – it has a shape
– Data is created through human choices, so to understand data’s “truth” you need to
understand the choices that influence its shape
– That is, you need to understand how data effects – brings about – its representation of
the world
– Today, data almost always means electronically stored data


10

Data Quality
• Data Quality / Quality of Data:
– The level of quality of data represents the degree to which data meets the
expectations of data consumers, based on their intended use of the data.
– Because data also serves a semiotic function (it serves as a sign of
something other than itself), data quality is also directly related to the
perception of how well data effects (brings about) this representation.

• Observations:
– High-quality data meets expectations for use and for representational
effectiveness to a greater degree than low-quality data.
– Assessing the quality of data requires understanding those expectations and
determining the degree to which the data meets them.
– To understand the quality of data, you need to understand where it comes
from and how it works
•
•
•
•

The concepts the data represents
The processes that created data
The systems through which the data is created
The known and potential uses of the data


11

Assessment
• Assessment is the process of evaluating or estimating the nature,
ability, or quality of a thing.
• Like measurement, assessment requires comparison.
• But importantly! Assessment implies drawing a conclusion about—evaluating—the
object of the assessment, whereas measurement does not always imply doing so.

• Assessing organizational readiness for a strategic approach to data quality
means assessing your organization’s culture:
• Taking a hard, objective look at how people work together, how they manage change,
resolve conflict, reward success, etc.
• Making general observations about what works well and what does not work well in
your organization can help you identify obstacles and opportunities to improving data
quality.
• Quantifying characteristics that help you explain changes that need to take place.


12

What is a Data Strategy?
• The concept of strategy can be applied to many facets of an
organization: Financial strategy, HR strategy, Product Strategy,
Technology strategy, etc.
• “Strategies” for different parts of an organization need to be aligned. At
the very least, they should not contradict each other, but ideally they
should support each other.
• Strategy is future-oriented, but strategic success depends on knowing
your starting point: Assessing current state in order to remove
obstacles to success and create the conditions for success.
• Translated to Data Strategy, this means
– Recognizing how data supports your organization’s overall mission
– Applying techniques from other areas of quality improvement to data (e.g.,
problem definition, process analysis, measurement, root cause analysis)

• So that …..
– Actions can be defined to create an organizational commitment to producing
high quality data.


13

Strategic Alignment
• Data Strategy – an organization’s plan for improving the quality of data
it produces and uses, in order to meet current and future organizational
goals
• Should align with the organization’s overall mission and its
– Infrastructure strategy – a plan for building ensuring the organization’s
technology infrastructure is sound and positioned to meet future needs.
– Technology strategy – a plan for ensuring that investment in technology is
coordinated across parts of the organization, that systems can “talk” to each
other.
– Data Governance strategy – the organization’s approach to establishing
decision rights and accountabilities for information-related processes*
– Data Stewardship strategy – its plan for ensuring ongoing engagement of
business and technology staff in the care and management of organizational
data assets.
* Gwen Thomas, The Data Governance Institute

14

Without a Strategy… Any road will take you where?
Alice came to a fork in the road.
Alice: “Would you tell me, please,
which way I ought to go from here?”
The Cheshire Cat: “That depends a
good deal on where you want to get
to.”
Alice: “I don't much care where.”

The Cheshire Cat: “Then it doesn't
much matter which way you go.”
Alice: “So long as I get somewhere.

The Cheshire Cat: “Oh, you're sure
to do that, if only you walk long
enough.”

15

The 12 Directives
• Component pieces of a strategic approach to data quality that can be
applied within any organization – they support each other.
• Not a process – these are not “steps” to data quality strategy. They are
ways of orienting your organization toward demanding higher quality data.
• Where you start depends on your organization & your role within it. So we
will discuss assessing organizational readiness to move forward in these
areas:
– Directive 1: Obtain Management Commitment to Data Quality
– Directive 2: Treat Data as an Asset
– Directive 3: Apply Resources to Focus on Quality
– Directive 4: Build Explicit Knowledge of Data

– Directive 5: Treat Data as a Product of Processes which can be Measured and Improved
– Directive 6: Recognize Quality is Defined by Data Consumers
– Directive 7: Address the Root Causes of Data Problems
– Directive 8: Measure Data Quality, Monitor Critical Data
– Directive 9: Hold Data Producers Accountable for the Quality their Data

– Directive 10: Provide Data Consumers with the Knowledge the Require for Data Use
– Directive 11: Data will continue to evolve – Plan for evolution
– Directive 12: Data Quality goes Beyond the Data – Build a Culture Focused on Data Quality


16

The 12 Directives: Set One: Recognize the importance of
data to the organization’s mission
• Directive 1: Obtain Management Commitment to Data Quality
• Current State Assessment Goal: To understand how management generally respond to
strategic needs in order to identify obstacles to and opportunities for developing an
organizational commitment to data quality improvement.
• Assess how your organization’s management works – review business plans, identify
obstacles to success, talk with and formally survey the management team.
• Plan to communicate on an ongoing basis. Management commitment must be cultivated
over time. Plan to talk their language: Success stories, cost/benefit analysis (CBA),
results of other assessments.
• Directly relate business uses of data to the organization’s mission statement.

• Directive 2: Treat Data as an Asset
• Current State Assessment Goal : To understand how the organization currently describes
the value of data and to turn the organization toward recognizing the value of data in
monetary terms.
• Review operational and project budgets to define the current investment in data
management and the cost of data issues to put the value of data in monetary terms.
• Identify best case and worst case scenarios for one strategic data set to quantify the
benefits of improvement and the risks of poor quality data.
• As a pilot project, pick one improvement opportunity define its CBA directly in relation to
the organization’s mission.


17

The 12 Directives: Set One: Recognize the importance of
data to the organization’s mission
• Directive 3: Apply Resources to Focus on Quality
• Current State Assessment Goal: To understand the organization’s readiness to
formally engage a team in the tactical execution of data quality improvement.
• Assess how the organization currently responds to data quality issues – review help
desk tickets, incident reports, break-fix projects.
• Survey teams to identify activities they currently engage in that support data quality.
Identify areas of redundant activity. Translate these to CBA.
• As with Directive 2, review operational and project budgets to define the current
investment in data management and the cost of data issues to put the value of data
in monetary terms.

• Directive 4: Build explicit knowledge of data
• Current State Assessment Goal: To understand the organization’s existing
knowledge-sharing practices in order to identify improvements that support
improvement of data quality.
• Identify existing processes for the creation and management of metadata, training
materials, and system documentation. Produce and inventory and identify gaps in
these areas.
• Survey teams about how they find answers to questions about data.

18

The 12 Directives: Set Two: Apply concepts related to
manufacturing physical goods to data
• Directive 5: Treat Data as a Product of Processes which can be
Measured and Improved
• Current State Assessment Goal: To understand how your organization currently
thinks about data in order to move the organization toward an understanding of
data as a product so that product quality improvement practices succeed.
• Survey business and IT stakeholders about the condition of data.
• Catalog existing documentation of the data chain; identify gaps in knowledge (see
Directive 4).
• Identify processes which are well defined and measured and find out how they
became so. These can serve as examples of the impact of improvements.

• Directive 6: Recognize Quality is Defined by Data Consumers
• Current State Assessment Goals: To identify data consumers who can provide
ongoing input about the quality of the organization’s data and how to improve it. And
to identify which data is most critical or at risk within the organization.
• Survey data consumers about their needs, perceptions and concerns.
• Review access logs and reports to quantify which data elements are used most
frequently. Review helpdesk tickets, issue logs, and break-fix projects to identify
recurrent quality issues.
• Findings can serve as input to pilot improvement or measurement projects.

19

The 12 Directives: Set Two: Apply concepts related to
manufacturing physical goods to data
• Directive 7: Address the Root Causes of Data Problems
• Current State Assessment Goal: To understand the organization’s cultural approach
to problem solving in general in order to identify obstacles and opportunities for
advocating for root cause remediation.
• Identify (or diagram) the existing processes for remediation of data issues; including
how issues are prioritized and how remediation is funded.
• Identify examples of data issues rooted in one system that impact the use of data in
a different system.
• Perform a CBA on at least one example to quantify the benefits of root cause
remediation.

• Directive 8: Measure Data Quality, Monitor Critical Data
• Current State Assessment Goal: To understand the organization’s current practices
for data quality measurement and to identify options for implementing a set of pilot
measurements.
• Identify a set of critical or at risk data elements; focus on data with recurrent issues.
• Apply DQAF (data quality assessment framework) measurement concepts to
quantify issues relate to this data and produce a CBA.
• Propose a pilot for ongoing monitoring.


20

The 12 Directives: Set Three: Build a culture of quality
that can carry out strategic data management
• Directive 9: Hold Data Producers Accountable for the Quality their Data
(and knowledge about that data)
• Current State Assessment Goal: To understand the organization’s general approach to
accountability in order to identify opportunities for creating greater accountability for
producing high quality data.
• Identify existing mechanisms for communication up and down the data chain. Assess
obstacles to communication, gaps in communication, and instances where
communication has worked well.
• Survey data producers for ideas on how they can improve their own processes.
• Incorporate data quality goals in to performance evaluations, service level agreements,
and other formal mechanisms for accountability.

• Directive 10: Provide Data Consumers with the Knowledge the Require for
Data Use
• Current State Assessment Goal: To understand how well informed data consumers are
about the production and meaning of data and to identify ways that they educate
themselves about data in order to identify opportunities for improvement.
• Use same assessments as under directive 4 and directive 6. Focus on how data
consumers leverage or create explicit knowledge of data.
• Identify processes that encourage data consumers to actively learn about data rather
than passively accept data in its current condition.
• Survey data consumers on how to improve their ability to understand and use data.


21

The 12 Directives: Set Three: Build a culture of quality
that can carry out strategic data management
• Directive 11: Data will continue to evolve – Plan for evolution
• Current State Assessment Goal: To understand the organization’s overall ability to
respond to emerging business opportunities and to identify ways to manage data so
that the organization can respond in the most agile ways possible.
• Identify major trends in your industry. How will your organization obtain or produce
the data needed to meet them?

• Identify risks within existing business and technical processes that could become
obstacles to future evolution. Use these examples to elicit suggestions about
approaches to effective planning.

• Directive 12: Data Quality goes Beyond the Data – Build a Culture
Focused on Data Quality
• Current State Assessment Goal: To identify and understand the effectiveness of
existing organizational structures that support improved data quality.
• Identify organizational components that are directly charged with data
management, data governance, or data quality, and survey the people involved
about what is working and what could be done better.


22

Current State Assessment Deliverables
• If you assess organizational readiness for 1 or all 12 of these
directives, you will have a set of findings and opinions and examples
that describe where your organization is and where it could go:
– Critical data; data that is less important.
– Processes that are working well; processes that are not working well.

– Insight on how your organization works:
• Behaviors that will enable improved quality
• Behaviors that will get in the way of improving quality

• This information is input to your formulation of a set of tactical actions
that can move your organization toward its strategic goal of improved
data quality.
– A set of prioritized recommendations
– Proposal for how to implement them


23


24

Classics of Information Quality
• English, Larry P. (1999). Improving Data Warehouse and Business Information Quality. Indianapolis,
IN: Wiley.
• English, Larry P. (2009) Information Quality Applied. Indianapolis, IN: Wiley.
• Loshin, David. (2001). Enterprise Knowledge Management: The Data Quality Approach. Boston, MA:
Morgan Kaufmann.
• Loshin, David. (2011). The Practitioner’s Guide to Data Quality Improvement. Boston, MA: Morgan
Kaufmann.
• Maydanchik, Arkady. (2007). Data Quality Assessment. Bradley Beach, NJ: Technics Publications,
LLC.
• McGilvray, Danette. (2008). Executing Data Quality Projects: Ten Steps to Quality Data and Trusted
Information.™ Boston, MA: Morgan Kaufmann.
• Mosely, Mark, Brackett, Michael, Early, Susan, & Henderson, Deborah (eds.). (2009). The Data
Management Body of Knowledge (DAMA-DMBOK Guide). Bradley Beach, NJ: Technics Publications,
LLC.
• Olson, Jack. (2003). Data Quality: The Accuracy Dimension. Boston, MA: Morgan Kaufmann.
• Redman, Thomas C. (2008). Data Driven: Profiting from Your Most Important Business Asset. Boston,
MA: Harvard Business Press.
• Redman, Thomas C. (2001). Data Quality: The Field Guide. Boston, MA: Digital Press.
• Redman, Thomas C. (1996). Data Quality for the Information Age. Boston, MA Artech House.
• Wang, Richard. (1998, February). A Product Perspective on Total Data Quality Management.
Communications of the AMC. 58-65.
• Wang, Richard and Strong, Diane. (1996, Spring). Beyond Accuracy: What Data Quality Means to
Customers. Journal of Management Information Systems. 5-33.


25

Recommended Reading – Thinkin’ differently about data
• Chisholm, Malcolm D. (2010). Definitions in Information Management: A Guide to
the Fundamental Semantic Metadata. Canada: Design Media.
• Chisholm, Malcolm D. (2012-08-16) Data Quality is Not Fitness for Use. Information
Management. http://www.information-management.com/news/data-quality-is-notfitness-for-use-10023022-1.html
• Crease, Robert P. (2011). World in the Balance: The Historic Quest for an Absolute
System of Measurement. New York: W. W. Norton Company.
• Derman, Emanuel. (2011). Models. Behaving. Badly.: Why Confusing Illusion With
Reality can lead to Disaster on Wall Street and in Life. New York: Free Press.
• Gould, Stephen Jay. (1996). The Mismeasure of Man. New York, NY: Norton.
• Ivanov, Kristo. (1972). Quality-Control of Information: On the Concept of Accuracy
of Information in Data-Banks and in Management Information Systems. Stockholm,
Sweden: The Royal Institute of Technology and the University of Stockholm
Sweden.
• Kent, William. (2000). Data and Reality. Bloomington, IN: 1st Books Library.
• Taleb, Nassim Nicholas. (2007). The Black Swan: The Impact of the Highly
Improbable. New York, NY: Random House.
• Tufte, Edward R. (1983). The Visual Display of Quantitative Information. Cheshire,
CT: Graphics Press.
• West, Matthew. (2011). Developing High Quality Data Models. Boston, MA: Morgan
Kaufmann.


26

Thank You

Laura Sebastian-Coleman, Ph.D., IQCP
Optum Insight
Laura.Sebastian-Coleman@optum.com

A Strategic Approach to Improving Data Quality

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to A Strategic Approach to Improving Data Quality

Similar to A Strategic Approach to Improving Data Quality (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

A Strategic Approach to Improving Data Quality