Many User Experience (UX) practitioners face organizational barriers that limit their ability to influence product decisions. Unfortunately, there is little concrete knowledge about how to systematically overcome these barriers to optimize UX work and foster a stronger organizational UX culture. This paper introduces the concept of User Experience Capacity-Building (UXCB) to describe the process of building, strengthening, and sustaining effective UX practices throughout an organization. Through an integrated literature review of relevant HCI and capacity-building research, this paper defines UXCB and proposes a conceptual model that outlines the conditions, strategies, and outcomes that define a UXCB initiative. Five areas of future research are presented that aim to deepen our understanding of UXCB as both a practice and an area of scholarship.

1. User Experience (UX)
Capacity-Building:
A Conceptual Model and
Research Agenda
Craig M. MacDonald
Associate Professor
Pratt Institute, School of Information
June 26, 2019

2. Background

3. UX in industry
The UX industry has experienced rapid growth over the last
decade, driven largely by the creation of in-house UX
teams.
But, many organizations misunderstand UX and/or don’t
devote enough resources to UX, which makes it difficult to
create or sustain a user-centered culture.
• (Rosenbaum, Rohn, & Humburg, 2000; Law, Van Shaik, & Roto, 2014;
Winter, Ronkko, & Rissanen, 2014; Stone, Bentley, & Shebanek, 2016)

4. Organizational challenges to UX
Too much emphasis on persuasion and compromise
Barriers to effective collaboration
Difficulty securing budget/resources for UX
Lack of buy-in or support from executives
Too many organizational inefficiencies
Resistance or hostility towards UX
Hard to navigate organizational cultures
• (Ardito, Buono, Caivano, Costabile, & Lanzilotti, 2014; Farrell & Nielsen, 2014;
Wale-Kolade, 2015; Gray 2016; MacDonald, 2017; Treder, Cao, & Ho, 2017;
Bruun, Larusdottir, Nielsen, Nielsen, & Persson, 2018)

5. UX Capacity-Building (UXCB)
UXCB can be a solution to these challenges.
Notably, UXCB is not new; rather, it is proposed as an
organizing concept that can tie together past and future
HCI research related to the identification, development,
and evaluation of strategies to grow and sustain a healthy
UX culture.

6. Inspiration: ECB
Evaluation Capacity-Building (ECB) started in the 1980s as
a way to help non-profit and government organizations
improve their ability to evaluate the quality of programs
and services
ECB is “the intentional work to continuously create and
sustain overall organizational processes that make quality
evaluation and its uses routine”
• (Stockdill et al., 2002)

7. ECB vs. EC
ECB researchers draw a distinction between:
the activities that define an organization’s evaluation capacity; and
the activities used to strengthen or sustain that capacity.
This distinction implies that:
(1) evaluation capacity is not a static construct; and
(2) there are techniques specifically designed to build evaluation
capacity.
So, ECB is itself a practice and field of study with its own
structural elements, themes, knowledge, and competencies
• (Baizerman et al., 2002)

8. UX Capacity-Building
Drawing inspiration from ECB, UXCB is proposed as:
The intentional work to continuously create and sustain
overall organizational processes that make quality UX
work routine.

9. UXCB vs. UXC
Like ECB, we can draw a distinction between:
the activities that define an organization’s UX capacity; and
the activities used to strengthen or sustain that capacity.
Like ECB, this distinction implies that:
(1) UX capacity is not a static construct, and
(2) there are techniques specifically designed to build UX capacity.
Like ECB, this suggests UXCB can also be a practice and field of
study with its own structural elements, themes, knowledge,
and competencies

10. Related Work

11. Related conceptual frameworks
Flow of Competence (Gray, Toombs, & Gross, 2015)
Describes how individual designers’ perceptions and use of UX
competencies influences how they work within and/or build their
org’s UX culture.
SC5 Design Strategy Framework (Liikkanen, 2016)
A “roadmap” for UX transformation with six dimensions.
Organizational Competence (Furniss, Curzon, & Blandford, 2018)
Identified six areas that form an “integrated web” of competence and
determines the success of a product.
Strategic Usability Assessment (Kieffer & Vanderdonkct, 2016)
Assessment instrument to determine how well usability evaluations
are used to achieve business goals.

12. UX Maturity Models
Define a linear path from an initial/beginning stage to the
highest stage of organizational maturity.
Usually 5-7 clearly defined stages, with brief description of
characteristics of each stage.
Some examples:
Earthy’s Organizational Human Centeredness Scale (1998)
Schaffer and Lahiri’s Usability Maturity Model (2004)
Chapman and Plewes’ UX Maturity Model (2014)

13. Another UX Maturity Model
Source: https://www.nngroup.com/articles/ux-maturity-stages-1-4/
Image: https://uxdesign.cc/a-framework-for-measuring-design-maturity-8fdb578e82c

14. Yet Another UX Maturity Model
Source: https://articles.uie.com/beyond_ux_tipping_point/Jared Spool, 2014

15. And more and more models

16. Useful, to a point
The many different models of UX maturity are informative
and can be inspirational for some organizations.
But:
Unclear where these models come from and how they were
developed, so their validity, reliability, and generalizability is
questionable.
Not much evidence that maturity models actually work as a
strategic tool for organizations.
Little guidance on how to move between stages.
• (Lacerda & von Wangenheim, 2018; Sauro, Johnson, & Meenan, 2017;
Uskarci & Demirors, 2017)

17. Methodology

18. Integrative Literature Review (ILR)
ILR is a specific type of research that “reviews, critiques,
and synthesizes representative literature on a topic…such
that new frameworks and perspectives on the topic are
generated.” (Torraco, 2005, p. 356)
Especially valuable when exploring less mature research
areas because it brings together divergent streams of
research into a single conceptual framework.

19. Step 1: Locate Literature
Goal was to be representative rather than exhaustive, so
search strategy was purposive rather than systematic.
Searched Google Scholar with broad keywords (e.g.,
“organizational UX” or “UX industry”)
Backward- and forward-searching
Targeted searching of ACM Digital Library, ScienceDirect, and
Taylor & Francis
Total (not exhaustive) sample: 88 articles and books
(Vom Brocke, et al., 2005)

20. Step 2: Analyze literature
A concept matrix was used to assist with analysis and
synthesis.
Begin with a set of concepts (topics, theories, dimensions, etc.).
Review each article to identify one or more concepts relevant to
it; some concepts can change and new concepts can emerge
during the analysis.
(Vom Brocke, et al., 2005; Webster & Watson, 2002)

21. Beginning set of concepts
Integrated ECB Framework
(Labin, et al., 2012; Labin, 2014)

22. Results of analysis
A purposive sample of 51 articles from the initial sample was
selected for further analysis.
Each article was reviewed by three researchers to identify concepts
relevant to that article.
Some concepts changed and new concepts emerged along the way.
7 articles were deemed not relevant to UXCB and were excluded.
The resulting classification scheme created a concept-centric
organizing framework for the remaining articles.
The final concept matrix can be viewed at bit.ly/uxcbmatrix-dis19

23. UXCB Conceptual Model
I. Conditions
(Why)
II. Strategies
(What & How)
III. Outcomes
(Results)
Buy-in and Support
Organizational Needs
Goals
Activities
- Training workshops
- Technical assistance
- Hiring
- Team building/structuring
- Events
- Broadcasting
- Guides, tools, frameworks
Content
Implementation
Resources
- Time
- Infrastructure
- Materials
- Budget
Individual
- Attitudes
- Knowledge, skills,
behaviors
Product
Organizational
- UX practices/processes
- UX culture
- Non-UX measures

24. UXCB Case Studies

25. Case Study: User First (Yahoo)
Conditions (why)
The User First program was initiated with direct support from the
company’s CEO (buy-in & support)
There was a fragmentation of company’s UX efforts, with dozens
of product teams all operating differently (organizational needs)
To achieve the primary objective of promoting “user first mentality
at all levels of company” (p. 823), leadership established a
company-wide goal that all employees would participate in a
user-facing activity at least once per quarter (goals)
(Stone, Bentley, White, & Shebanek, 2016)

26. Case Study: User First (Yahoo)
Strategy (what and how)
Consisted of multiple activities, including:
• Creation of a centralized UX Research & Accessibility team (team
structuring)
• Articulation of company-wide vision and expectations for UX research
(guides, tools, and frameworks)
• Held a series of User Nights where employees were paired with users
for conversations and observations (events)
• Recorded and livestreamed every user research session (broadcasting)
• Hosted company-wide brown bag lunches to discuss UX research; got
UXRA team involved in new employee orientation; wrote intranet
articles about UX research; published papers at internal conferences
(broadcasting)
(Stone, Bentley, White, & Shebanek, 2016)

27. Case Study: User First (Yahoo)
Strategy (what and how) – cont’d
UX research was the primary focus of all UXCB activities (content)
All activities required significant planning, time investment, and a
dedicated budget (implementation; resources)
• Renovated the usability lab to be more inviting and comfortable for
observers;
• Invested in hardware and software for livestreaming;
• Organized several User Nights
(Stone, Bentley, White, & Shebanek, 2016)

28. Case Study: User First (Yahoo)
Outcomes (results)
There was an increased percentage of employees participating in
a User First activity, with over 50% participation in a recent
quarter (individual-knowledge, skills, behaviors)
The UXRA team saw an increased awareness of the role and
value of UX research (organizational-UX culture)
The UXRA team was asked to do research at more stages of
product development and adopted new practices more closely
tailored to the company’s development cycle and product areas
(organizational-UX practices/processes)
The program also resulted in feature, performance, and reliability
improvements to many different products (product)
(Stone, Bentley, White, & Shebanek, 2016)

29. Conditions
Backed by top-level
executives (buy-in)
Product teams didn’t
utilize enough user
feedback (needs)
Wanted to increase
empathy among
engineers, program
and product
managers (goals)
Strategies
3-hour training
workshops with
product teams
(workshop)
8 months; 1500+
employees
Covered user research
basics (content)
Created sign-up
website, videos,
training materials,
scripts, and debrief
questions (resources)
Case Study: Pokerface (Google)
Outcomes
Increased the perceived
value of user
research and using
research results to
improve products
(attitudes)
Started culture change
to increase the
amount of user
research (practices)
(Liu, Sosik, & Singh, 2018)

30. Discussion
A UXCB Research Agenda

31. Limitations (1)
The model was developed through an analysis of UXCB
activities reported primarily in academic HCI literature.
But, this literature is biased towards formalized and highly
structured approaches to UXCB.
Both case studies represented the high-end of the UXCB
spectrum: large-scale UXCB efforts at multi-national
companies that already had the support of key corporate
leaders.

32. Limitations (2)
The UXCB model was informed by a well-established ECB model
and reflects a variety of perspectives and approaches to
UXCB.
But, few of the articles reviewed addressed every element.
Few articles discussed the budget or other resources required to
conduct UXCB.
Few articles mentioned conducting a needs assessment, either
formally or informally, prior to launching a UXCB initiative.
Most articles relied on anecdotal data for evidence of UXCB impact.
Nearly all were implemented in corporate settings (many in the
software development industry).

33. #1: Exploring UXCB Strategies
Not all UXCB programs need to be as large-scale, structured, or
formal as the ones presented here.
UXCB efforts should vary greatly in terms of breadth and scope
because by nature they are context-dependent
Future research should explore other UXCB strategies,
especially less formal or “lean” methods that can be
implemented by smaller teams or in organizations with
resource or time constraints
Is executive buy-in necessary prior to starting? If not, can buy-in
become an outcome rather than a pre-condition?
What other activities can be classified as UXCB?

34. #2: Applying UXCB in Different Contexts
Not all UXCB efforts that are successful in software
development companies will be successful in other contexts.
Future research should apply and evaluate UXCB strategies in a
variety of different industries and sectors and with
organizations of different sizes and with different levels of
resources
These efforts can help to:
• Validate and extend the conceptual model
• Identify new implementation challenges
• Uncover new success factors
• Develop new impact measures
• Demonstrate new models for successful UXCB initiatives

35. #3: Measuring UXCB Impact
Effective capacity-building often requires resources; how can we
be sure that it’s worth the investment?
Future research should focus on developing and validating
different ways of evaluating the short-and long-term impacts
of UXCB
What are reliable ways to measure the three levels of UXCB impact?
Is it possible to identify a stronger link between UXCB activities and
product-specific changes?
Are there other positive outcomes of UXCB?
This research can provide organizations with persuasive
evidence that UXCB is worth investing in.

36. #4: Assessing UX Capacity
For it to be successful, it is critical to fully understand the
organizational culture prior to developing or implementing a
UXCB strategy.
Future research should be aimed at identifying, describing, and
measuring the dimensions of UX capacity so that practitioners
can better determine where to focus their UXCB efforts.
Could there be an assessment tool to help organizations identify their
biggest areas of need?
Are the dimensions of UX capacity generalizable to organizations of
different sizes and types?
Is it possible to use both formal/systematic and lightweight/lean
assessment methods?

37. #5: Teaching UXCB
UXCB requires skills that are distinct from the traditional
toolset of UX professionals; can these skills be taught?
Future research can focus on exploring how UXCB can be
taught at different levels:
How can aspiring and junior UX professionals engage in effective
UXCB?
How can mid-level and senior UX professionals best leverage their
position and authority?
How can managers better support UXCB efforts?

38. In Summary
UXCB has the potential to be a robust and flexible approach
to helping UX professionals make the structural, cultural,
and procedural changes necessary to increase the impact
of their work.
A secondary benefit: it can add clarity by helping UX professionals
better distinguish between their day-to-day UX work and the
activities they undertake to build their organization’s UX capacity
Work remains to be done to transform UXCB into a mature
area of research and practice.
And this work is currently underway!

39. Thanks!
Q&A
Craig M. MacDonald
Associate Professor | Pratt Institute, School of Information
cmacdona@pratt.edu