Talk given at #CraftConf in Budapest in 2019. Video coming soon!

1. Andreessen’s Corollary
Ethical Dilemmas in Software Engineering
CTO
bryan@joyent.com
Bryan Cantrill
@bcantrill

2. Ethics in software engineering, ca. 1997

3. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

4. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

5. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

6. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

7. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

8. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

9. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

10. Ethics in software engineering, ca. 1997
— Gottbaum et al., “Software Engineering Code of Ethics” (1997)

11. Ethics in software engineering, ca. 1997
• There is northing wrong per se with the 1990s code of software
ethics — it just represents a precambrian era of software
• Notably, the guiding context surrounding the code of ethics itself
remains timeless…
• …but the code itself is quaint, and serves primarily to remind
how much software has changed in the last two decades
• With the rise of ubiquitous internet in the late 1990s came the
first real foreshocks of the ethical dilemmas to come…

12. 1999: Napster

13. 2003: Friendster

14. 2003: Friendster
Source: Gary Rivlin, “Wallflower at the Web Party” (New York Times, October 15th, 2006)

15. 2011: Andreessen’s prophesy
— Marc Andreessen, “Why Software Is Eating The World” (2011)

16. Software after Andreessen
• As software has indeed come into broader domains, the
internet-era challenges have multiplied and compounded — and
with increasingly serious ramifications
• There have been many clear ethical transgressions, common to
any era with a frenzied rush for mammon…
• …but much more common are true ethical dilemmas, laden
with complexity and ambiguity
• These are not entirely unrelated! Those least burdened by
ethics seem most likely to find themselves on the ethical
frontier, facing the greatest dilemmas

17. 2012: Facebook emotional manipulation
Source: Kashmir Hill, “Facebook Manipulated 689,003 Users’ Emotions for Science” (Forbes, June 28th, 2014)

18. 2012: Facebook emotional manipulation
Source: Kramer et al., “Experimental evidence of massive scale emotional contagion through social networks”

19. 2013: Zenefits “Macro”
Source: Findings of fact in SEC Administrative Proceedings against Parker Conrad et al. (file 3-18263)

20. 2014: Uber Greyball
Source: Greyball Audit Report, Portland Bureau of Transportation, April, 2017

21. 2014: Uber Greyball
Source: letter from Uber to Portland City Attorney, as quoted in Greyball Audit Report, Portland Bureau of Transportation, April, 2017

22. 2015: Google Photos mis-labelling
Source: https://twitter.com/jackyalcine/status/615329515909156865

23. 2016: First Tesla “Autopilot” fatality
Source: National Transportation Safety Board Highway Accident Report (NTSB/HAR-17/02)

24. 2016: First Tesla “Autopilot” fatality
Source: National Transportation Safety Board Highway Accident Report (NTSB/HAR-17/02)

25. 2017: Facebook and Anti-Rohingya violence
Source: Report of the independent international fact-finding mission on Myanmar, United Nations Human Rights Council

26. 2017: Facebook and Anti-Rohingya violence

27. 2018: Joyent provides infrastructure to Gab

28. 2019: 737 MAX MCAS and JT610/ET302 crashes
Source: Aircraft Accident Investigation Bureau Preliminary Report, Ethiopian Airlines Group, B737-8 ET-AVJ, March 10, 2019

29. Source: Boeing Company, April 17th 2019 MCAS update
2019: 737 MAX MCAS and JT610/ET302 crashes

30. Software in the post-Andreessen world
• This has been but a tiny sampling of the ethical dilemmas faced
by software in the post-Andreessen age
• It is clear that what is right for software is not necessarily right
for society: we must address our ethical dilemmas directly!
• Software is on the cusp of yet broader societal impact — and at
a time when our society is increasingly divided and fractured
• We have a greater burden to society than ever before —
how can we process these dilemmas?

31. Post-Andreessen ethics

32. Software in the post-Andreessen world
• Finished in the summer of 2018, the ACM’s new Code of Ethics
and Professional Conduct has arrived at the right time
• A radical overhaul of the 1990s-era code of ethics, it is much
more principles based, e.g.:
• Contribute to society and to human well-being
• Avoid harm
• Be honest and trustworthy
• Entire code is at https://ethics.acm.org

33. Beyond a Code of Ethics
• The ACM has gone beyond merely providing a Code of Ethics
by kicking off its Integrity Project
• Includes case studies and an “Ask an Ethicist” feature
• The ACM’s efforts show that to face ethical dilemmas, we need
to be able to ask tough questions
• These will often not have crisp answers — but the resulting
discussion is most likely to yield behavior consistent with the
code of ethics
• Organizations must initiate and support this discussion!

34. Andreessen’s Corollary
• Organizations that support frank discussion fo ethical dilemmas
will enjoy ethical differentiation
• They will attract like-minded individuals who can help
perpetuate a culture of consideration of ethical consequences
• We must recognize that as software’s footprint continues to
grow, out ethical dilemmas will only expand
• We must brace ourselves for future dilemmas
• Ethical quandary is Andreessen’s corollary — and it is our
collective responsibility to address It directly!