Techniques for More Efficient WCAG Website Testing - CSUN 2020

Techniques for
more efficient
WCAG website testing
Bill Tyler
March 11, 2020
#bespokewcagchecklists

© 2020 Optum, Inc. All rights reserved. Confidential property of Optum. Do not distribute or reproduce without express permission from Optum. 2
Meet the speaker
My Super Power
• UX developer & designer experience back to 1984
Education & Experience
• Full-time accessibility professional since 2013
First exposure & experiences go back to 2002
• W3C Invited Expert on Role-based Accessibility
Related Presentation
Introducing ARRM: A Framework to Fight Accessibility
Apathy
With Denis Boudreau (remote), Sean Kelly, Jenn Chadwick
Tomorrow @ 8:00AM, Platinum 3-4 (not Grand D)
Bill Tyler
Principal Digital
Accessibility Engineer
btyler@optum.com
@billtyler

Approach & techniques used at UnitedHealth Group for WCAG testing
• Optum Technology Accessibility Center of Excellence (A11y CoE)
• 6+ yrs. of ongoing accessibility testing research & analysis
• Applied to 200+ UnitedHealth Group (UHG) products

Accessibility
Center of Excellence
A brief history
4

0
15
30
45
Staff size before, during & after take-off
11
31

Growth and change over past 6+ years
• Primary practice in UnitedHealth Group (UHG), Optum and UnitedHealthcare (UHC)
• Started December 2013 with 2 people (including me!)
• Through internal mergers and restructuring grew to 10 people (including managers)
• Summer 2016 (Take-off): Complete assessments on 200+ products
− Increased 300%: 11 to 31 over 6 months
• Starting 2018 transitioned to consulting, governance, separate UHC digital accessibility
program (DAP)
• Today: Maturing of Optum A11y CoE (40) and UHC DAP (5)
But still growing!

WCAG 2.1 summary
WCAG: Web Content Accessibility Guidelines from the W3C
4 (POUR) Principles
• Perceivable – 4 guidelines
• Operable – 5 guidelines
• Understandable – 3 guidelines
• Robust – 1 guideline
78 success criteria across 3 levels
• A – 30 criteria
• AA – 20 criteria
• AAA – 28 criteria

WCAG 2.1 conformance
Target: WCAG AA
• WCAG 2.0 (2008)
− 38 (A & AA) criteria
− Current US Section 508 (and many others, including most existing legal cases)
• WCAG 2.1 (2018)
− 50 (A & AA) criteria
− Started in EU (EN 301 549) some organizations such as University of Minnesota
− UHG / ACoE Standard as of January 1, 2019

WCAG is great for learning and organizing…
…but ”one size does not fit all.”

WCAG off the rack is not really useful for testing
Principles – single-word abstractions
• Great for organizing, but not for testing
Guidelines – goals
• Elaborate on principles, but do not add much detail
Success Criteria – testable requirements!
• Targets to be met
• Basis for conformance
• But the text is still not checklist or checkpoints
− Understanding documents includes situations, techniques, failures, and more

What did we do?
Bespoke checklists
Custom tailor WCAG to our needs

Bespoke
Adjective [BRITISH]
made for a particular customer or user.
"a bespoke suit"

Why bespoke (custom tailored) checklists? – Before take-off
No budget, but lots of time
• First couple years we had the luxury of time to prepare
− We knew what was coming
• With no budget for off-the-shelf tools we had to build our own checklist
• Excel was the tool of choice
− Standard issue
− Basic database operations
− Sorting
− Filtering
− Import/Export

Why bespoke (or custom) checklists? – After take-off
Little budget, no time, and nothing fit properly
• Had to build and adapt on the fly using the tools at hand
• Rapid ramp up (from ~3 to 200+ products) left little time for new tools
− Time needed to hire people,
− Not evaluate, buy, deploy and start using new tools
• Still limited budget, especially for size of organization
• Off-the-shelf tools were not viable
− Incomplete automation and coverage
− Concerns about fitting and integration into the future
− UHG’s size and number of products were at a scale beyond most tools

Bespoke checklist benefits
• Tailored – explicitly to enterprise needs
• Integrated – into existing corporate tools and processes
− Import from automated testing tool
− Export to Governance Risk Compliance (GRC) application
• Optimized – for changing accessibility program requirements
• Control – multibillion dollar companies don’t like risk

Downsides? Many
• Time to build
• Cost and resources
• Support and maintenance
Not for everyone, but the lessons and techniques can still be useful

Bespoke checkpoints
Custom tailored testing of success criteria

Checklist v1.0: One checkpoint per success criterion
Too small
• Obvious, logical mapping for 38 checkpoints (WCAG 2.0)
• 1:1 relationship ensured full coverage for WCAG conformance (and VPATs)
• Didn’t have many projects so it seemed acceptable initially
Problem: WCAG’s variable success criteria complexity
• Many criteria are much more detailed than others
• SC1.1.1 Non-Text Content ≠ SC3.1.1 Language of Page
• Often too broad, covering multiple different cases
• SC1.3.1 Info and Relationships: covers forms, labels, tables, groupings, lists, etc.
• Often difficult to describe effectively to other roles (developers, designers, authors)

Checklist v2.0 – One checkpoint for each sub-case (in success criteria)
Too large and cumbersome
• Understanding documents for each criteria provide the real details useful for testing
• One for each situation – standard WCAG building blocks
• Add other sub-cases such from success criterion text as needed (requirements, options)
• Added our best practices and priorities
• Checkpoint potential count of 113, but started with 84 at take-off
Problems: Duplication, overlap, confusion, extra work
• Examples
− SC1.1.1 Non-Text Content went from 1 to 11 checkpoints
− Multiple references to required attributes
• Example: SC1.3.1 Info and Relationships, SC3.3.2 Labels or Instructions

Checklist v3.0: Reviewed, revised, rewritten, reduced
Custom tailored to our needs
• Reviewed all 84 checkpoints
• Revised and rewritten for clarity based on experience of growing team
• Merged checkpoints to remove duplication and overlap
• Removed checkpoints mapped into new versions for continuity
− Still at least one per success criterion to ensure WCAG AA conformance testing
• Reduced count by 22 from 84 to 62 (26%)
− WCAG 2.1 (2019) added 15 making current count 76

Bespoke groupings
Custom tailored principles and guidelines

WCAG principles and guidelines are for accessibility professionals
They do not work well for product testing
• Accessibility topics are not familiar to business owners, designers, authors, developers
− Some terminology is
• Particularly obscure (time-based media)
• Vague or not actionable (adaptable, robust, compatible)
• Difficult to find
− Do not align with roles or features (forms, layout, presentation, semantics)
− Some topics span success criteria, guidelines and principles

Custom groupings
Group checkpoints for your audience and their needs
• Size to audience needs
− Fewer, simpler groupings for stakeholders (8 categories)
− More detailed groupings for testers (10-11 test groups)
• Organize by subjects
− Functional: keyboard operation, forms, tables
− Role: presentation, content and semantics, programming
• Structure to meet needs of product types
− Emphasize key functions (error prevention and handling)

Custom groupings
Benefits
• Use groups appropriate for testing and reporting needs
− Test using groups optimized for assessments
− Create stakeholder reports using more meaningful categories
• Filter by topic (display all the table-related or image checkpoints)
− Easier search – no need to memorize guidelines or criteria
• Filtering allows easier “batch setting” of values for videos, forms, tables

Custom groupings
Test groups for accessibility engineers (WCAG 2.1)
1. Keyboard / Input
2. Mobile
3. Navigation
4. Forms
5. Tables
6. Images
7. Presentation
8. Semantics
9. Content
10. Code
11. Time-Based Media (TBM)

Bespoke sort orderings
Custom tailored numbering and sequencing

WCAG order is a jumble
Principles, guidelines, criteria order are good for learning, but…
• WCAG order is inefficient for testing
− SC1.1.1 Non-text Content  GL1.2 “Video”  SC1.3.1 Info and Relationships
 SC1.3.2 Meaningful Sequence  SC1.3.3 Sensory Characteristics
• Must switch testing modes and techniques or constantly jump back and forth
− SC1.3.1 Info and Relationships  GL3.3 Input Assistance
• Critical issues come later
− SC2.1.1 Keyboard Operation is the 21st criterion of WCAG 2.1, A & AA only
− WCAG 2.0 order wasn’t great, WCAG 2.1 made it worse
− New criteria added after originals, starting over with level A

Custom sort orders
Create a logical sequence for needs and efficiency
• Prioritize group order
− Common, critical groups first (such as keyboard, navigation, forms)
− Less critical later (time-based media)
• Start with showstoppers to confirm testing can proceed
− Example: focus  keyboard operation  keyboard trap
• Sort checkpoints within groups to follow more efficient testing protocol
− Example: content images  images of text  complex images  decorative images

Checkpoint numbering tips
Encoding IDs
• Mandatory IDs – row numbers become meaningless after sorting
• Unique IDs for each checkpoint – duplicates will get out of order after multiple sorting
• Extra digits – for consistent sorting of multiple checkpoints to a criterion or sub-case
• Base on WCAG – provides quick reliable default WCAG order
• Leading zeroes – for GL1.4 – 1.4.01 or 1.401
− If not, they never sort into to the expected WCAG order
• 1.4.1
• 1.4.10 …
• 1.4.13
• 1.4.2

Bespoke severity levels
Custom tailored level indicators

A, AA and AAA don’t work well for testing
WCAG AA isn’t very meaningful for prioritizing checkpoints
It’s really all one group (all A & AA)
• AAA (usually) not included
• Does anyone target Level A conformance?
A and AA are too broad for processes like Agile
• Severity scales typically have four levels
− Example: MoSCoW method – Must / Should / Could / Won’t
• Which are showstoppers? blockers? major or more minor barriers to use?
− SC2.1.2 No Keyboard Trap = SC4.1.1 Parsing? Both are Level A

Custom severity levels
Sample levels
• Critical Showstoppers or blockers to site access, operation and testing
• High Could be very challenging to overcome, impossible for most
• Medium May be very frustrating or time consuming for all
• Low Might have significant impact for many
Benefits
• Not forced to WCAG levels (Example: raise SC2.4.7 Visible Focus from AA to Critical)
• Align to product and project needs (like minimum viable or accessible product)
• Write so they cannot be optional (must include only)
• Align and reference by WCAG levels as needed

Other bespoke details
Additional custom tailoring

WCAG options
WCAG version
• v2.0 – only
• v2.1 – complete (2.0 and 2.1)
• v2.1 – supplement (new 2.1 checkpoints only)
WCAG level indicators
• A – only
• AA – only
• AA – conformance (A & AA)
• AAA – if applicable (A & AA & AAA)

WCAG options (cont.)
The Four POUR Principles
• P – Perceivable
• O – Operable
• U – Understandable
• R – Robust
Could include Guidelines
• 1.1 Text Alternatives
• 1.2 Time-based Media
• 1.3 Adaptable
• …

Custom checkpoints can be distributed by skill
Optimizing work to screen reader and visual testers
Custom checkpoints can be tailored and grouped based upon skills
• Screen Reader Tester / SRT AT expert using screen reader
• Visual Assistant / VA User working relying on vision, not AT
Filtering allows clear, consistent work assignments
• SRT Screen Reader Tester preferred
• EitherScreen Reader Tester or Visual Assistant can perform equally well
• Dual Both should test (AT experience very different from visual)
• VA Visual Assistant preferred (usually for efficiency)

Map checkpoints to non-WCAG standards
Section 508 (1998 or “pre-refresh”)
• § 1194.21 Software Applications and Operating Systems
− 12 criteria (a) – (l)
• § 1194.22 Web-based Intranet and Internet Information and Applications
− 16 criteria (a) - (p)
• § 1194.24 Video or Multimedia Products
− 5 criteria (a) – (e)
And possibly transition to future standards like AG (WCAG 3.0)

Other information for reports
• Additional options can be added for checkpoints and reports to communicate benefits
Example: “Affects users who are…” [blind, low vision, deaf, cognitively impaired, etc.]
• Accessibility tools for testing
List of tools to test checkpoint – useful for developers and designers
References links to standards and resources
• Roles
List of roles that may own the issue or should be consulted about the checkpoint
− Example: color contrast issues (SC1.4.2) likely belong to visual designer

Final fitting
Putting it all together
1

You can and must support all WCAG
• Success criteria
• Guidelines
• Principles
• Conformance
• Reporting (VPAT)
How you get there is up to you.

Custom tailor WCAG to your needs with bespoke checklists
• Success Criteria  Checkpoints
− Create as many as appropriate with minimum of 1 for each criterion
− Structure using Understand Documents (leveraging situations, options, best practices, etc.)
• Principles and Guidelines  Groupings
− Tailor and size them to needs (and vocabulary) of target audiences
• Numbering  Sort order
− Encode IDs so they provide consistent, helpful, efficient sequences for testing
• Level Indicators  Severity levels
− Prioritize to needs of testing and delivered product
• Other details as needed

Thank you.
Bill Tyler
Principal Digital Accessibility Engineer
btyler@optum.com
@billtyler

Techniques for More Efficient WCAG Website Testing - CSUN 2020

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Techniques for More Efficient WCAG Website Testing - CSUN 2020