Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing More Ethical and Unbiased Experiences - Abstractions

Presented at Abstractions, Pittsburgh, PA
Karen Bachmann and Carol Smith, August 23, 2019
Humans are biased, and sadly, we are not always able to filter our deeply ingrained biases. UX designers and researchers have long understood this, but as we watch major technology companies make significant mistakes with regard to ethics and bias, the cost of not accounting for bias and ethics is becoming more evident and widely known.

Even knowing what pitfalls exist, we still miss opportunities for doing good as a result of our own human biases obscuring our vision. We need tools to explore and challenge our biases in a productive way to deliver better outcomes. We need a set of shared values within teams and, ultimately, across the industry to promote our common responsibility to deliver the greatest benefit while causing the least amount of harm. How can we work together to intensify the focus on ethical design? In this session, we’ll share ways you can empower yourself and your teams to do the right thing for people.

  • Be the first to comment

Designing More Ethical and Unbiased Experiences - Abstractions

  1. 1. Designing more ethical and unbiased experiences This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License except where noted otherwise. Karen Bachmann Carol Smith @KarenBachmann @Carologic
  2. 2. #AbstractionsCon @KarenBachmann @Carologic What do we need most Awareness, ourselves and our colleagues Exposure to other perspectives Connection with our own humanity
  3. 3. “ #AbstractionsCon @KarenBachmann @Carologic “AND ETHICS IS SUCH A NEGATIVE SUBJECT.” Many people believe that embracing ethics would limit their options, their opportunities, their very ability to succeed in business. John C. Maxwell, Ethics 101 (CEO considering ethics as a topic for a sales meeting)
  4. 4. #AbstractionsCon @KarenBachmann @Carologic Empathy
  5. 5. #AbstractionsCon @KarenBachmann @Carologic Empathy vs. Ethics?
  6. 6. #AbstractionsCon @KarenBachmann @Carologic A progression for doing good LawsNorms/CodesEthics Values/morals Empathy Individual Society
  7. 7. #AbstractionsCon @KarenBachmann @Carologic Ethics
  8. 8. #AbstractionsCon @KarenBachmann @Carologic Ethics Based on well-founded standards of right and wrong Standard of expected behavior that guides the correct course of action What is Ethics? By Manuel Velasquez, Claire Andre, Thomas Shanks, S.J., and Michael J. Meyer. Markkula Center for Applied Ethics
  9. 9. #AbstractionsCon @KarenBachmann @Carologic What is Ethics? By Manuel Velasquez, Claire Andre, Thomas Shanks, S.J., and Michael J. Meyer. Markkula Center for Applied Ethics Ethics in terms of… Rights Obligations Benefits to society Fairness Specific virtues
  10. 10. #AbstractionsCon @KarenBachmann @Carologic Does the Trolley Problem Have a Problem? What if your answer to an absurd hypothetical question had no bearing on how you behaved in real life? By Daniel Engber. June 18, 2018. Not the Trolley Problem
  11. 11. #AbstractionsCon @KarenBachmann @Carologic No absolute right or wrong or even catastrophic failure – but a cost to be weighed Taking a broader perspective to include extended communities, the environment, and other “stakeholders” Resource: HBR articles that explore real, nuanced situations MIT Moral Machine Project Complex problems need sophisticated tools
  12. 12. #AbstractionsCon @KarenBachmann @Carologic ● Pace of change ○ Avoid creating fear, uncertainty, doubt, anger, and unrest ● Counter perceptions in the industry ○ Unethical behavior, arrogance, and lack of empathy ● Consider ramifications ○ Don’t jump into the new and the cool too quickly ● Understand nuances of behavior and habits Ethics is needed in tech
  13. 13. #AbstractionsCon @KarenBachmann @Carologic Addressing ethics is for the entire industry Product owners QA testers Developers BAs Data scientists Project managers UX researchers Designers
  14. 14. #AbstractionsCon @KarenBachmann @Carologic ● Consider human need and benefit fully in our work ○ Consider full human context over time and social ecosystem ○ Mitigate the thrill of the “new” ● Counter ○ Emphasis of technology over humanity ○ Primacy of "data" over people ● Raise visibility of actual people, rather than getting lost in data What can tech professionals do? Ethics offer a pragmatic path to supporting these goals and the conversations around them
  15. 15. “ #AbstractionsCon @KarenBachmann @Carologic ETHICS DOESN’T HAVE TO BE COMPLICATED When you know better, you do better. Betty Easterly It’s about being aware and informed
  16. 16. #AbstractionsCon @KarenBachmann @Carologic Bias
  17. 17. #AbstractionsCon @KarenBachmann @Carologic Content Design Data Solutions ...Are Biased
  18. 18. #AbstractionsCon @KarenBachmann @Carologic Due to… Social class Resource availability Education Race, gender, sexuality Culture, theology, tradition More…
  19. 19. #AbstractionsCon @KarenBachmann @Carologic So many biases!
  20. 20. #AbstractionsCon @KarenBachmann @Carologic The right humans are required to reduce bias
  21. 21. #AbstractionsCon @KarenBachmann @Carologic Bias?
  22. 22. #AbstractionsCon @KarenBachmann @Carologic Need to talk about biases in non-threatening, productive ways
  23. 23. #AbstractionsCon @KarenBachmann @Carologic Bias: A Mental Shortcut Part of being human Kahneman’s System 1 Not inherently bad, may be misapplied Implicit = invisible Not necessarily in sync with our conscious beliefs Can be managed and changed
  24. 24. #AbstractionsCon @KarenBachmann @Carologic Start with building awareness and knowing ourselves
  25. 25. #AbstractionsCon @KarenBachmann @Carologic Know thyself Start with understanding your own bias Don’t fall into the illusion of complete objectivity Don’t be afraid to acknowledge bias
  26. 26. #AbstractionsCon @KarenBachmann @Carologic Scenarios
  27. 27. #AbstractionsCon @KarenBachmann @Carologic Assumption That Math Reduces Bias Amazon’s computer models trained to vet applicants - observed patterns in resumes - most came from men Taught itself that male candidates preferable Penalized resumes that included the word “women’s,” as in “women’s chess club captain” Downgraded graduates of two all-women’s colleges
  28. 28. #AbstractionsCon @KarenBachmann @Carologic Embedding Existing Bad Behaviors Both online and face-to-face mortgage lenders charge higher interest rates to black and Latino borrowers Costing homebuyers up to half a billion USD more in interest every year than white borrowers with comparable credit scores “Even if the people writing the algorithms intend to create a fair system, their programming is having a disparate impact on minority borrowers — in other words, discriminating...”
  29. 29. #AbstractionsCon @KarenBachmann @Carologic Dehumanizing thru Math
  30. 30. #AbstractionsCon @KarenBachmann @Carologic Not Anticipating Bad Behavior
  31. 31. #AbstractionsCon @KarenBachmann @Carologic Lacking Plan for Mitigation
  32. 32. #AbstractionsCon @KarenBachmann @Carologic Be Intentional Make it your business to keep people safe
  33. 33. #AbstractionsCon @KarenBachmann @Carologic “Ensure humans can unplug the machines” – Grady Booch, TEDTalk Grady Booch, Scientist, philosopher, IBM’er
  34. 34. #AbstractionsCon @KarenBachmann @Carologic Benefits
  35. 35. #AbstractionsCon @KarenBachmann @Carologic vs. Making ethics a success goal
  36. 36. #AbstractionsCon @KarenBachmann @Carologic Easy to find examples of unethical behavior
  37. 37. #AbstractionsCon @KarenBachmann @Carologic ● Focus on value ● Measure success gained, not penalties avoided ● Recognize opportunity and growth, not constraint and stagnation ● Avoid negativity fatigue caused by only focusing on shocking failures The “carrot” more compelling than “stick” ● Think of ethics on a larger scale ● More than "treat your participants well" ● Avoid focusing on getting “caught” because it encourages only the minimum
  38. 38. #AbstractionsCon @KarenBachmann @Carologic ● It reduces business liability. ● It helps employees make good decisions. ● It assures high-quality customer service. ● It prevents costly administrative errors and rework. ● It consistently grows the bottom line. Support: National Ethics Association
  39. 39. #AbstractionsCon @KarenBachmann @Carologic Companies with good ethical policies earn: ● Marketing advantages over their competitors. Customers readily invest in the companies through shares and also want to establish long lasting business relations with the company. ● The performance of employees improves with good ethical policies present in a company. Morale is high and employees feel obligated to put in their all to continue to make it a success. ● Reputation management: A bad reputation is created by unethical behaviour which will eventually lead to a scandal. A scandal will result in falling stock prices, anxiety, and low morale among employees as well as government and public scrutiny and inquests. ● Legal and financial incentives: Companies known for their high ethical standards and education of employees on ethical policies are provided with strong legal and financial incentives by regulatory bodies. Support: SixSigma Online
  40. 40. #AbstractionsCon @KarenBachmann @Carologic “ A better motivation for inspiring ethical behavior is the benefits it provides, both personally and financially. Besides feeling good about doing the right thing, principled business conduct can also be profitable. Ethical behavior is good business. Four key benefits: ● Easier accounting – real numbers and honest account is easier than maintaining falsified records ● Better branding – creates trust with employees, suppliers and customers ● Improved bottom line – retain customers, earn high ratings, and gain referrals ● Better health – Reduce stress and feel better about your business Support: Steve Parrish in Forbes business/#654f4a9e7687
  41. 41. “ #AbstractionsCon @KarenBachmann @Carologic QUOTING JAMES BURKE If you invested $30,000 in a composite of the Dow Jones thirty years ago, it would be worth $134,000 today. If you had put that $30,000 into these [socially and ethically responsible] firms - $2,000 into each of the fifteen [in the study] – it would now be worth over $1 million. John C. Maxwell, Ethics 101 Chairman of Johnson and Johnson
  42. 42. #AbstractionsCon @KarenBachmann @Carologic Resources and Tools
  43. 43. “ #AbstractionsCon @KarenBachmann @Carologic Tools for ethical design If it's not ethical, it cannot be beautiful. Yves Behar How can we apply our understanding of ethics?
  44. 44. #AbstractionsCon @KarenBachmann @Carologic ● A commitment to doing the right thing ● Clear standards ● Organizational values put into action (by leaders and employees) ● Accountability ● Open communications up, across and down the organization Attributes of ethical organization cultures Steve Priest in “Ethics Exchange: Culture (Part Two): Characteristics of Ethical Organizational Cultures”
  45. 45. #AbstractionsCon @KarenBachmann @Carologic ● Understand your values before a crisis ● Establish what it means to do good, not merely avoid doing harm ● Consider context as well as tactics ● Choose your work as carefully as possible Know your ethical boundaries
  46. 46. #AbstractionsCon @KarenBachmann @Carologic Know your limitations: Karen’s IAT results on disability
  47. 47. “ #AbstractionsCon @KarenBachmann @Carologic ETHICAL ALIGNMENT - CODES? Be conscious and aware of what you’re endorsing with your time, not only for the welfare of others but for your own sense of self. The work you choose to take on defines you. Stephen Anderson Making sure we are in the right place
  48. 48. #AbstractionsCon @KarenBachmann @Carologic ● Support and bolster your colleagues’ ethical practice ○ Share case studies and techniques ○ Adhere to and promote ethical practices ○ Amplify successes ● Protect integrity of industry best practices by setting a standard of ethical design practices as the norm Promote professional boundaries
  49. 49. “ #AbstractionsCon @KarenBachmann @Carologic CONSIDER ETHICS WHEN DESIGNING NEW TECHNOLOGIES What is needed is strong, anticipatory guidance by those who intersect the technology, health and ethics worlds to determine how we develop and deploy technologies that deliver the greatest societal benefits. Christie and Yach, TechCrunch Reacting to Facebook’s dilemma with fake news
  50. 50. #AbstractionsCon @KarenBachmann @Carologic Code of ethics WIKIPEDIA “PROFESSIONAL ETHICS” Most professionals have internally enforced codes of practice that members of the profession must follow to prevent exploitation of the client and to preserve the integrity of the profession.
  51. 51. #AbstractionsCon @KarenBachmann @Carologic Code of Conduct / Ethics What do you value? What lines won’t you cross? Track progress?
  52. 52. #AbstractionsCon @KarenBachmann @Carologic ● Technical education including ethics in the curriculum ● Ethical design methodologies, heuristics, guidelines, practices ● Increasing awareness and ongoing discussions Do we have the tools?
  53. 53. #AbstractionsCon @KarenBachmann @Carologic ● New thinking: Cradle-to-cradle or closed loop thinking ● Biomimicry, nature as inspiration ● Focus on human and ecosystem benefit and betterment ● Focus on design over time and evolving scenarios Rethinkinghow we do our work
  54. 54. #AbstractionsCon @KarenBachmann @Carologic ● Montreal Declaration for Responsible AI ● Ethical OS ● Comparison of various tools (Principled AI: Mapping Consensus and Divergence in Ethical and Rights-Based Approaches) ● Organizational questions (see appendix) Ethical Tools
  55. 55. #AbstractionsCon @KarenBachmann @Carologic Most tools start with a set of questions ● Who? ○ Directly affected ○ Indirectly affected ● What is one benefit gained by someone in each group? ○ Initially ○ After using the product over time ● What is one harm those same people might experience? ○ Initially ○ After using the product over time ● Can the harm be mitigated? Does it require a trade-off against a benefit?
  56. 56. #AbstractionsCon @KarenBachmann @Carologic Introspective Detection Tools ● Changing decision making approach ○ Pre-mortems ○ Worst case scenario ○ Multiple estimations -> designs ○ Seek outside view ● Implicit Association Test ● Flip it to test it ● Invite examination and input from people who are not like “us” ● Appoint a Devil’s Diversity Advocate ● Slow down and reflect
  57. 57. #AbstractionsCon @KarenBachmann @Carologic The challenge is less the lack of tools, but the will to use them
  58. 58. #AbstractionsCon @KarenBachmann @Carologic Continue the conversation... LinkedIn: CarolJSmith KarenBachmann-UXResearcher Twitter: @Carologic @karenbachmann Slideshare: carologic karenbachmann Toward ethical, transparent and fair AI/ML: a critical reading list By Eirini Malliaraki, Feb 19 via tweet from @robmccargow critical-reading-list-d950e70a70ea
  59. 59. #AbstractionsCon @KarenBachmann @Carologic AppendixAdditional techniques and thoughts to continue the conversation in your organization
  60. 60. #AbstractionsCon @KarenBachmann @Carologic Go beyond the common, simplistic Example: Instead of a just tech code of ethics… consider a users’ bill of rights!
  61. 61. #AbstractionsCon @KarenBachmann @Carologic Example: Computer User’s Manifesto ● The user is always right. If there is a problem with the use of the system, the system is the problem, not the user. ● The user has the right to easily install software and hardware systems. ● The user has the right to a system that performs exactly as promised. ● The user has the right to easy-to-use instructions for understanding and utilizing a system to achieve desired goals. ● The user has the right to be in control of the system and to be able to get the system to respond to a request for attention. ● The user has the right to a system that provides clear, understandable, and accurate information regarding the task it is performing and the progress toward completion. ● The user has the right to be clearly informed about all system requirements for successfully using software or hardware. ● The user has the right to know the limits of the system’s capabilities. ● The user has the right to communicate with the technology provider and receive a thoughtful and helpful response when raising concerns. ● The user should be the master of software and hardware technology, not vice-versa. Products should be natural and intuitive to use. By Dr. Claire-Marie Karat, as shared by Theo Mandel
  62. 62. #AbstractionsCon @KarenBachmann @Carologic Design principle User has a right … Digestibility Not to work too hard to accomplish their goals Digestibility To have their goals and their time respected Clarity To understand exactly what they are getting Clarity To be respected Trust To be informed and in control Familiarity To be oriented from the moment they start using a product Familiarity To have their prior experience and knowledge respected Delight To have an enjoyable experience Delight To have an effortless (or appropriate effort) experience Delight To have the UI disappear into the context of their work Example: Design principles as user rights Based on 5 simple UX principles from a blog post from InVision
  63. 63. #AbstractionsCon @KarenBachmann @Carologic Mindfulness is a journey, not a destination
  64. 64. #AbstractionsCon @KarenBachmann @Carologic Discussions with the team ● Provide tools for mindfulness and self-awareness, build team awareness ● Acknowledge your own biases - be open to learn more ● Establish clear goals to understand biases ○ Why it matters to the team and to your work ● Establish communication ground rules ● Use ability to see biases of others with empathy and compassion ● Use these tools to approach the difficult conversations - but don’t shy away from them!
  65. 65. #AbstractionsCon @KarenBachmann @Carologic Integrating the tools for awareness ● Make the detection tools part of the design process – not once and done ● Define goals and checks around identified biases ○ Set objectives, check objectives ○ Key decision points, to avoid trouble ● Watch for emergent “excuses” for regressing to System 1 thinking
  66. 66. #AbstractionsCon @KarenBachmann @Carologic Using frameworks, guidelines, toolkits ● Cultural journey, not a formulaic process ● Tools can help with framing discussions initially, but not the endpoint ● Examples ○ Ethical OS ○ Big company’s ethical guidelines and programs ○ Google starting AI initiative and then ending it shortly after
  67. 67. #AbstractionsCon @KarenBachmann @Carologic Enabling communication ● Establish a culture of questioning ○ Listen for discord and regression ○ Bias buddies ○ Use technological aids where possible ● Communicate shared understanding continuously ● Be kind to each other
  68. 68. #AbstractionsCon @KarenBachmann @Carologic Organizational Questions ● Is there value in ○ defining a set of ethical guidance for employees? ○ providing guidance on techniques for identifying unintended negative consequences of work? ● Would different departments/products need different guidance? ● How might we ○ reflect our cultural norms in a code of ethics? ○ ensure we are helping people? ○ track progress? ● How can we describe the “lines” our work won’t cross? ● Would we include ○ mitigation strategies? ○ considerations of social consequences such as job displacement? ● Should we build on existing codes, guidance and best practices?
  69. 69. #AbstractionsCon @KarenBachmann @Carologic References
  70. 70. #AbstractionsCon @KarenBachmann @Carologic References, Page 1 AI Now Institute "After a Year of Tech Scandals, Our 10 Recommendations for AI. Let’s begin with better regulation, protecting workers, and applying “truth in advertising” rules to AI" tech-scandals-our-10-recommendations-for-ai-95b3b2c5e5 Angwin, Julia; Larson, Jeff; Mattu, Surya; and Kirchner, Lauren. "Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks.“ ProPublica. 23 May 2016. Baer, Mark B. “The Connection Between Empathy Toward Others and Ethics,” Psychology Today. others-and-ethics Benson, Buster. “Cognitive bias cheat sheet.” 2016: 55a472476b18 Beshears, J. Frederick, S. and Gino, F. “Test Yourself: Are You Being Tricked by Intuition?” Harvard Business Review. 2015: Booch, Grady. “Ensure humans can unplug the machines.” TED@IBM. November 2016. Bowles, Cennydd. “Ethics in the AI Age,” presented at IxDA 2017. Bowles, Cennydd. Future Ethics. 2018. NowNext Press.
  71. 71. #AbstractionsCon @KarenBachmann @Carologic References, Page 2 Bradberry, T. “13 Cognitive Biases That Really Screw Things Up For You.” 2018. cognitive-biases-really-screw-things-up-you-dr-travis-bradberry/ Campbell-Dollaghan, Kelsey. “The Year Dark Patterns Won.” Co.Design. 2016. Christie, Gillian and Yach, Derek. ”Consider ethics when designing new technologies.” 2016. "Data, Ethics, and AI: Practical activities for data scientists and other designers." IDEO Medium article. Cutler, Adam; Pribić, Milena; and Humphrey, Lawrence. "Everyday Ethics for Artificial Intelligence: a Practical Guide for Designers and Developers." IBM Medium Article. intelligence-75e173a9d8e8 Dastin, Jeffrey. "Amazon scraps secret AI recruiting tool that showed bias against women" Reuters. 9 October 2018. showed-bias-against-women-idUSKCN1MK08G Dempsey, Samantha and Taylor, Ciara. Designer’s Oath. DeNisco, Alison. “3 guiding principles for ethical AI, from IBM CEO Ginni Rometty.” Tech Republic. 17 January 2017.
  72. 72. #AbstractionsCon @KarenBachmann @Carologic References, Page 3 Engber, Daniel. "Does the Trolley Problem Have a Problem? What if your answer to an absurd hypothetical question had no bearing on how you behaved in real life?" 18 June 2018. Foot, Philippa. Original paper introducing the Trolley Problem: "The Problem of Abortion and the Doctrine of the Double Effect" Oxford Review, no. 5, 1967. Funchess, M. “Implicit Bias -- how it effects us and how we push through.” TEDxFlourCity. 2014. Gray, D. Liminal Thinking: Create the change you want by changing the way you think. 2016. IDEO. "Data, Ethics, and AI: Practical activities for data scientists and other designers." IDEO Medium article. Internet Encyclopedia of Philosophy, “Empathy and Sympathy in Ethics.” Hall, Erika. “The 9 Rules of Design Research.” 2018. 1a273fdd1d3b Halvorson, H.G. “How To Recognize (And Overcome) Your Unconscious Biases In Hiring.” Fast Company. 2015. Judd, S. “Superfan.” 2018: Kahneman, Daniel. Thinking, Fast and Slow. 2013. Farrar, Straus and Giroux.
  73. 73. #AbstractionsCon @KarenBachmann @Carologic References, Page 4 Keereepart, T. “3 design principles to help us overcome everyday bias.” TEDxPortland. 2016. Knight, Will. "Why does Beijing suddenly care about AI ethics?" MIT Technology Review. Lawrence-Lightfoot, S. and Hoffmann Davis, J. The Art and Science of Portraiture. 1997. Madrigal, A.C. . “Disposable America.” The Atlantic. 2018. Maxwell, John C. Ethics 101: What Every Leader Needs to know. 2003. Center Street. McIntosh, P. "White Privilege: Unpacking the Invisible Knapsack." Peace and Freedom Magazine. 1989. Microsoft. “Keeping an Eye on AI with Dr. Kate Crawford.” Microsoft Research Podcast. 2018. crawford/?OCID=msr_podcast_kcrawford_tw Pressner, K. ”Are you biased? I am.” TEDxBasel. 2016. Price, Rob. “Microsoft is deleting its AI chatbot's incredibly racist tweets” Business Insider UK. 24 March 2016.
  74. 74. #AbstractionsCon @KarenBachmann @Carologic References, Page 5 Project Implicit. 2011: Public Affairs. "Mortgage algorithms perpetuate racial bias in lending, study finds." UC Berkeley, Berkeley News. 13 November 2018. finds/ Ratcliff, C. “12 ways to fight bias in your team” Synopsys of #uxchat on Twitter, hosted by K. Bachmann. 2018. Smith, S.E. “Why philosophy is so important in science education.” Quartz. 2017: philosophy-is-so-important-in-science-education/ Soll, J.B., Milkman, K.L. and Payne, J.W. “Outsmart your own biases.” Harvard Business Review. 2015. #uxchat on Twitter “Embedding Ethics into Design,” hosted by What Users Do. 2018. Velasquez, Manuel; Andre, Claire; Shanks, S.J. Thomas; and Meyer, Michael J. "What is Ethics?" Markkula Center for Applied Ethics. Vincent, James. "Google ‘fixed’ its racist algorithm by removing gorillas from its image-labeling tech. Nearly three years after the company was called out, it hasn’t gone beyond a quick workaround." The Verge. 12 January 2018.
  75. 75. #AbstractionsCon @KarenBachmann @Carologic References, Page 6 Wachter-Boettcher, Sara and Meyer, Eric. Design for Real Life. 2016. Watcher-Boettcher, Sara. Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech. 2017. @sara_ann_marie. “the line from "look at our cool new facial-recognition app" to "wow this thing only works for white people" is a fucking millimeter long and clearly marked. why can't a single tech co seem to be able to see it?” 2018. Wendt, Thomas. "Empathy as Faux Ethics.” 2017.