1) The document discusses designing scientific research practices and tools for truth, scale, and sustainability. It argues current systems are designed for friction rather than collaboration and progress. 2) It notes a perception crisis where up to 70% of research cannot be reproduced, representing wasted money. Shifting practice requires a multi-faceted approach including open tools, standards, incentives and recognition to foster reuse. 3) The document calls for further adoption of "web-enabled science" through access to content, data, code and materials with rewards for openness and collaboration. It discusses rethinking professional development to lower barriers to entry and foster sustainable practitioner communities.

1. kaitlin thaney
@kaythaney ; @mozillascience
WSSSPE / 16 nov 2014
designing for truth, scale
and sustainability

2. doing good is part of our code

3. help researchers use the
power of the open web to
change science’s future.

4. (0)

7. power, performance, scale

8. our current systems are
designed to create
friction.
despite original intentions.

9. current state of science
articles
data
patents

10. some have a firehose
articles
data
patents

11. traditions last not because they are
excellent, but because influential
people are averse to change and
because of the sheer burdens of
transition to a better state ...
“
“
Cass Sunstein

12. downside of output-driven
recognition systems

13. “There’s greater reward,
and more temptation to
bend the rules.”
- David Resnik, bioethicist

14. let’s look at an example

15. 2004-2010
$350 million spent
$60m+ for “management”
70+ tools created

16. (1)

17. the culture of the specialist
is changing.
lowering barriers to entry.

18. - access to content, data, code, materials.
- emergence of “web-native” tools.
- rewards for openness, interop, collaboration, sharing.
- push for ROI, reuse, recomputability, transparency.
“web-enabled research”

19. what do we mean by
“open research”?
community technology practices
collaborative interoperable open review
participatory discoverable
data
management
recognition open tools sharing / reuse
mentorship
designed for
reuse
documentation /
versioning

20. we’re facing a
perception crisis.
at the sacrifice of scientific progress.

21. Source: Wolkovich et al. GCB 2012.

22. “... up to 70 percent of research from academic labs cannot be
reproduced, representing an enormous waste of money and effort.”
- Elizabeth Iorns, Science Exchange

23. http://xkcd.com/1445/

24. instill best
(digital,
reproducible)
practice
“research hygiene”

25. (2)

26. shifting practice takes a
multi-faceted approach.
designing for scale.

27. research social capital capacity
infrastructure layers for
efficient, reproducible research
open tools
standards
best practices
research objects
scientific software
repositories
incentives
recognition / P&T
interdisciplinarity
collaboration
community dialogue
training
mentorship
professional dev
new policies
recognition
stakeholders: universities, researchers,
tool dev, funders, publishers ...

29. our systems need to
talk to one another.
focus on environments that foster reuse, progress.

31. code as a research object
what’s needed to reuse ?
http://bit.ly/mozfiggit

35. (community driven)
metadata for software discovery: JSON-LD
http://bit.ly/mozfiggit

36. http://softwarediscoveryindex.org/report/

38. Instead of cancer driving the development
of technology, it was the development of
technology that drove caBIG moving into
position where this technology could be
adopted by individuals who were
interested in cancer.
- Andrea Califano
Columbia University
“
“

39. [Their] approach to fulfilling [their] mission
was upside down.
- Andrea Califano
Columbia University
“
“

40. (3)

41. our practices are
limiting us.
how to further adoption of
open, web-enabled science?

42. “web-enabled science”
- access to content, data, code, materials.
- emergence of “web-native” tools.
- rewards for openness, interop, collaboration, sharing.
- push for ROI, reuse, recomputability, transparency.

43. “web-enabled science”
what’s missing?
- access to content, data, code, materials.
- emergence of “web-native” tools.
- rewards for openness, interop, collaboration, sharing.
- push for ROI, reuse, recomputability, transparency.

44. current activity:
235+ instructors
(60+, training)
4000+ learners

46. rethinking
“professional development”

47. lowering (not exacerbating)
barriers to entry

48. fostering a (sustainable)
community of practitioners

49. (4)

50. we need to create safe
spaces for our software.
designing with sustainability in mind.

51. “nested sustainability”
KTucker, CC-BY-SA-3.0

53. We are used to billion-dollar software,
and it’s not what we can afford.
I am worried that unless we rein in our
expectations, we will do this experiment
again and we will get the same result ...
- Joe Gray
Oregon Health + Science University Center
“
“

54. 1. design with the
audience, not individual,
in mind.
(be careful of the snowflake syndrome.)

55. 2. design to unlock latent
potential of our systems.
(and keep in mind engagement.)

56. 3. rethink how we reward
researchers and support
roles.

57. 4. be mindful of jargon/
semantics traps.

58. we’re here to help.
teach, contribute, learn.
http://mozillascience.org
sciencelab@mozillafoundation.org

59. kaitlin@mozillafoundation.org
@kaythaney ; @mozillascience
special thanks: