This half talk, half exercise, explains how to conduct impactful team dialogues and grow design culture at organizations through implementing best practices learned from IBM Studios DC. Talk was shared at UX Camp DC, 1/12/19
4. di·a·loguecri·tique
To build and improve a culture of productive feedback
we must reimagine what it means to “critique”.
Nicole Coumes | | UX Camp DC 2019WHY | | Value of Critique
6. – Merriam-Webster
: an act of criticizing
especially : a critical
estimate or discussion
–Collins Dictionary
: to analyze
and evaluate (a
subject, literary
work, etc.); criticize
– Cambridge Dictionary
: to give an opinion or
judgment about a piece
or work, book, film, etc.
cri·tique
WHAT | | “Defining Critique” Nicole Coumes | | UX Camp DC 2019
7. – Merriam-Webster
: an act of criticizing
especially : a critical
estimate or discussion
–Collins Dictionary
: to analyze
and evaluate (a
subject, literary
work, etc.); criticize
– Cambridge Dictionary
: to give an opinion or
judgment about a piece
or work, book, film, etc.
cri·tique
WHAT | | “Defining Critique” Nicole Coumes | | UX Camp DC 2019
8. WHAT | | Perceptions of Critique
Tedious
One-sided
Unhelpful
Lack of Trust
Not collaborative
Nicole Coumes | | UX Camp DC 2019
9. Visual Design
• What : Students/designers “pin-up” in-progress work and make
recommendations on improvements. Various styles of this exist.
• Goals: Strengthen craft of designers by getting feedback from
peers.
• Outcomes: Designers leave with a concrete list of things to
adjust in their design.
(Art School) Design Critique
WHAT | | Feedback Types by Industry Nicole Coumes | | UX Camp DC 2019
10. Development
• What : A team meeting that's held at the
end of a sprint that solicits feedback on
what went well and what can be
improved on.
• Goals : Identify areas of a process so a
team can improve iteratively
• Outcomes : Due to iterative feedback
the team completes work quicker.
• What : A quality assurance activity in
which developers review another’s
source code.
• Goals : Ensure best code quality and
identify/record found defects.
• Outcomes : Regular review mitigates risk
and promote knowledge share through
debugging.
Retrospectives (Agile) Code (peer) Reviews
WHAT | | Feedback Types by Industry Nicole Coumes | | UX Camp DC 2019
11. UX
• What : Method to evaluate a product by
testing a series of tasks with users.
• Goals : Gather actionable input by
subjects to improve a product.
• Outcomes: Regular testing results in risk
mitigation and improves the usability of
the product.
• What : Practice of comparing two
versions of a design to see which is
more effective.
• Goals : Settle quandaries between the
best approach for a design.
• Outcomes: A/B testing is done
frequently and results in improved
performance.
User Testing A/B Testing
WHAT | | Feedback Types by Industry Nicole Coumes | | UX Camp DC 2019
12. In order to achieve best case
outcomes, we need to adjust what it
means to crit.
Nicole Coumes | | UX Camp DC 2019WHAT | | “Defining Critique”
13. : a discussion between two
or more people or groups,
especially one directed
towards exploration of a
particular subject or
resolution of a problem.
: an act of criticizing
especially : a critical
estimate or discussion.
di·a·loguecri·tique
Nicole Coumes | | UX Camp DC 2019WHAT | | “Defining Critique”
15. Use Case: IBM Studios, DC
HOW | | IBM Use Case Nicole Coumes | | UX Camp DC 2019
16. Setting The Scene
Get buy-in when needed
to start, but don’t ask for
permission
1
Need expressed from group
in All-Hands meeting.
2 weeks later I started our
first daily critique series.
HOW | | IBM Use Case Nicole Coumes | | UX Camp DC 2019
17. HOW | | IBM Use Case
Logistics
2
Set a regular cadence
with your team—automate
when you can!
Establish a code of
conduct
Automated Slackbot &
calendar hold with logistics
were set up as a reminders.
3
Nielsen Norman Group
19. Practice
In order to encourage
participation, often team
members nominated each
other to share work.
6
7
Share work before it’s
“done”
Rotate (or Volun-tell)
presenters to build trust &
comfort among group
HOW | | IBM Use Case Nicole Coumes | | UX Camp DC 2019
20. Reflect & Iterate
Conduct retrospectives
for continuous
improvement
8
Feedback: reduce # of
critiques per week, shift daily
time by 30 min, & change
video software.
Conduct quarterly
retrospectives.
HOW | | IBM Use Case Nicole Coumes | | UX Camp DC 2019
21. In Q1 daily
critiques went from
0 to 2 avg. / wk
In Q2 daily
critiques went from
2 to 3 avg. / wk
(+ more remote & self-organized critiques)
HOW | | IBM Use Case Nicole Coumes | | UX Camp DC 2019
22. Best Practices
Get buy-in when needed to start, but
don’t ask for permission
Set a regular cadence with your
team—automate when you can!
Establish a code of conduct
Assign clear roles
HOW | | Best Practices
Invite a diverse range of participants
Share work before it’s “done”
Rotate (or Volun-tell) presenters to
build trust & comfort among group
Conduct retrospectives for
continuous improvement
1
2
3
4
5
6
7
8
Nicole Coumes | | UX Camp DC 2019
23. • Be kind
• Be candid & specific with feedback
• Affirm what works
• Ask probing questions
• Provide suggestions, not dead-ends
HOW | | Best Practices
Feedback Guidance
Nicole Coumes | | UX Camp DC 2019
24. • Task-master & time-keeper.
• Responsible for admin tasks.
• Prompts both presenters and participants when
needed.
Facilitator (1) :
Someone who was appointed by the group
before the critique.
HOW | | Best Practices
Roles
Nicole Coumes | | UX Camp DC 2019
25. • Defines the scope of feedback desired from the
group.
• Coordinates with facilitator to schedule.
• Explains context of designs in succinct manner
and ties work to goals.
Presenter(s) (1-2) :
Creator(s) of work who wish to receive feedback.
HOW | | Best Practices
Roles
Nicole Coumes | | UX Camp DC 2019
26. • Provides honest and thoughtful feedback.
• Challenges presenters & asks questions.
• Reviews material prior to session (if applicable).
Designers, strategists, developers ect; can be
part of project or greater design team (non-client).
HOW | | Best Practices
Roles
Participants (3-7) :
Nicole Coumes | | UX Camp DC 2019
27. Design dialogs can take on many
forms and should be adapted to the
problem at hand.
Nicole Coumes | | UX Camp DC 2019HOW | | Best Practices
28. • Most common method.
• Designer(s) define their problem and present a
portion of their work to the group for feedback.
• Feedback can be both narrow or focused.
• Participants and presenters exchange a
dialogue that results in actionable feedback.
HOW | | Types of Critique
Conversation
Nicole Coumes | | UX Camp DC 2019
29. Sample Scenario
Prompt: Presenter shows in-
progress screens for a project
and asks for feedback
specific to the colors in the
design system.
HOW | | Types of Critique Nicole Coumes | | UX Camp DC 2019
30. • Presenter(s) share a portion of their work and
allow each participant to rotate through and
offer (often written) feedback.
• Round robin may include a dialogue
afterward to discuss feedback.
• Highly interactive.
• Activity lends self well to visual artifacts.
HOW | | Types of Critique
Round Robin
Nicole Coumes | | UX Camp DC 2019
31. Sample Scenario
Prompt: Presenters have
participants rotate through a
set of personas in order to
ensure accuracy and
improve fidelity.
HOW | | Types of Critique Nicole Coumes | | UX Camp DC 2019
32. • Presenter(s) share a portion of their work with
minimal instruction and allow participants to test
their product/artifact.
• Typically task-based, with prompts from the
presenter.
• Often a test can be a prelude to more rigorous
usability testing.
• Feedback is derived from observation and/or
debrief at the end of the session.
HOW | | Types of Critique
Test
Nicole Coumes | | UX Camp DC 2019
33. Sample Scenario
Prompt: A research team is in
the process of developing a
prototype to test with users
but want to simulate their test
before they travel. They run a
test to determine the best set
up and see if they run into
any technical errors.
HOW | | Types of Critique Nicole Coumes | | UX Camp DC 2019
34. • A combination of any or all of the
methods (conversation, round-robin, test).
• This often happens organically over the
course of a critique.
HOW | | Types of Critique
Hybrid
Nicole Coumes | | UX Camp DC 2019
35. Sample Scenario
Method: Test + Round Robin
HOW | | Types of Critique
Prompt: Presenters had
participants rotate through a
set of draft design principles
and had them guess which
principle was which, while
leaving edits on the definitions.
Nicole Coumes | | UX Camp DC 2019
37. Exercise Run Down
• 5 min to create, 5 min to critique, 5 min to reflect
• Form Groups of 3-4
• Roles: 1 Presenter, 1 Facilitator, 1-2 Participants
PRACTICE | | Overview Nicole Coumes | | UX Camp DC 2019
38. Task: Design a better way for UX Camp
conference speakers to sign up for time slots
and share their talks with participants.
Presenters
For the sake of this exercise, please create something visual your group can react to.
PRACTICE | | Overview Nicole Coumes | | UX Camp DC 2019
39. Task: Come up with an introduction for your
presenter that sets clear expectations for the
critique (feel free to review ground rules).
Facilitators
PRACTICE | | Overview Nicole Coumes | | UX Camp DC 2019
40. Task: Create a short list of probing questions to
ask the presenters relating to their goals.
Participants
PRACTICE | | Overview Nicole Coumes | | UX Camp DC 2019
42. Reflect!
What went well?
What could have been improved?
What do we want to try in the future?
PRACTICE | | Retrospective Nicole Coumes | | UX Camp DC 2019