Presented at Node Summit 2017. Video at https://vimeo.com/230142234

1. Platform as a reflection of values
Joyent, node.js, and beyond
CTO
bryan@joyent.com
Bryan Cantrill
@bcantrill

2. Platform and values
• Software platform decisions are big, important decisions with
lasting consequences
• One doesn’t merely select component infrastructure for where it
is, but also where it’s going — and what it represents
• In the post-open source world of software infrastructure, this is
reflected and guided by a community’s values
• Beyond right tool for the job, it is the right values for the job…
• …and then the right software for the values

3. Some platform values
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

4. Platform core values
• All of these values are important — but they are in tension
• Platforms aspire to balance all of them — but for every platform
some small subset represents its core values
• These core values attract a like-minded community — and
become self-reinforcing…

5. Platform core values: Seventh Edition Unix
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

6. Platform core values: OpenBSD
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

7. Platform values: K
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

8. Platform values: Awk
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

9. Platform values: ZFS
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

10. Platform values: DTrace
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

11. Platform values: node.js circa 2010
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

12. Joyent’s core values (historically):
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

13. Joyent’s core values (now):
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

14. Growing core values
• While the core values can mutate over time, it usually isn’t with
the same people; core values often track people
• Ideally, a community is able to grow its core values by attracting
and welcoming people with those new values
• In 2010, there was reason for optimism for node.js, as it was
enjoying an influx of technologists bearing the scars from failed
experiments in Twisted Python, Event Machine, C, etc.

15. The node.js demographic circa 2010

16. Growing core values: Debuggability
• As we at Joyent built our own production infrastructure on
node.js, we began to need to assert some of our own values…
• It started with debuggability as we added:
• DTrace probes in node.js core (2010)
• DTrace probes in arbitrary node.js (2011)
• MDB support (postmortem!) for node.js with mdb_v8 (2011)
• DTrace ustack() helper for node.js (2011)

17. Growing core values: Rigor
• As we hit performance problems in production, we made heavy
use of the DTrace ustack() helper for profiling
• The need for rigor in understanding this output inspired us to
invent flame graphs (2012) to visualize node.js stacks:

18. Growing core values: Maintainability
• As we had more and more services in node.js, we found the
need for a unified logging library…
• We developed Bunyan in 2012, reflecting not only our values of
maintainability but also debuggability (DTrace probes built in!)
• Bunyan also reflects our value of stability: we needed
abstractions that we could build upon!

19. Growing core values: Robustness
• We found that the de facto modules in several critical areas
didn’t entirely reflect our own values — especially around
robustness and debuggability
• For building REST-based services we developed Restify (2011),
with an emphasis on RFC compliance and debuggability
• For constructing robust asynchronous pipelines, we developed
vasync (2012), with a focus on error handling and debuggability

20. Growing core values: Robustness
• An early problem — and one with many dynamic languages —
is increasing memory usage over time
• It’s easy to errantly hold a reference to an entire object graph!
• We added “::findjsobjects” to mdb_v8 (2012) and have used to
find many thorny production problems
• We later added the ability to iterate over all functions with
“::jsfunctions” (!) — and then the ability to actually print the
variables in a closure with “::jsclosure” (!!)

21. Growing core values: Stability
• We maintained quite a few binary add-ons — and immediately
saw the need for stability in the interface
• Stability does not mean intransigence or stasis!
• Stability means developing the right abstractions that can
survive future change on both sides of the interface boundary
• C++ can make this difficult task much more challenging…
• We developed v8plus (2012), a C interface for add-ons that we
continue to use in lieu of NAN (or N-API)

22. Growing core values: Resiliency
• Our experience trying to incorporate resiliency in our node.js-
based systems led us to taxonomize node.js errors
• The resulting document quickly became authoritative; searching
“node.js error handling” + “I’m feeling lucky” still yields it:

23. The challenge of changing values
• Just because we believed in production-oriented values like
debuggability, robustness, and stability doesn’t mean these
values were widely shared!
• Values aren’t necessarily contagious!
• The node.js community broadly identified more closely with
growth-oriented values like approachability, expressiveness,
and velocity
• By 2014, these values were increasingly coming into conflict,
and a fracture was (in retrospect) inevitable…

24. Farewell Node.js

25. Farewell Node.js, cont.

26. Farewell to TJ
• TJ Holowaychuk was the most prolific node module contributor;
if nothing else, probably wrote more node.js than anyone else
• TJ wasn’t the first; there had been foreshocks:

27. The challenge of large communities
• The departures highlight a challenge with larger communities…
• When communities are large, values can easily conflict or
become diluted
• Smaller communities identify more strongly with their values —
but larger communities are more likely to fracture over them
• Sometimes this is a fork, but more often, people just leave
• Sometimes the values change because people leave, but more
often people leave because the values change (or fail to!)

28. Divergent values
• We erred in either not recognizing or not articulating that the
io.js fork in late 2014 represented divergent values
• Even to the degree that we recognized it and articulated it, not
clear that it mattered: the problem was the divergence itself
• The problem was even more basic at root: we held aspirations
for the values of node.js that it didn’t hold for itself
• The values of node.js are (and probably have long been) the
values of JavaScript itself…

29. Platform values: node.js circa 2015
• Approachability
• Availability
• Compatibility
• Composability
• Debuggability
• Expressiveness
• Extensibility
• Interoperability
• Integrity
• Maintainability
• Measurability
• Operability
• Performance
• Portability
• Resiliency
• Rigor
• Robustness
• Safety
• Security
• Simplicity
• Stability
• Thoroughness
• Transparency
• Velocity

30. node.js
• The values for JavaScript/node.js and our values didn’t align —
and it was clear that they weren’t going to
• We transitioned node.js to a foundation in 2015
• But many in the community putatively believed in our values (or
wanted to), so we remained engaged
• This hasn’t been easy, and one issue in particular highlights the
challenge in accommodating divergent values…

31. Promises
• Promises are a different way of asynchronous programming,
dating back to Argus, a CLU derivative from the mid-1980s:
• node.js was originally implemented in terms of promises, but
removed them entirely in February 2010 (0.2!)

32. Promises
• Because of their use in broader JavaScript, demand grew to
reintroduce promise support in node.js…
• But JavaScript promises conflate operational errors with
programmatic errors in that they both become exceptions
• Unlike thrown exceptions absent promises, we cannot know if a
thrown exception in a promise is going to be later caught or not
• This makes it challenging to abort a process on, say, a type
error with enough information to debug it post-mortem
• Promises make writing robust software more difficult

33. Promises
• Promises embody the divergence of values in modern node.js:
approachability, compatibility and expressiveness in tension
with robustness, debuggability, and simplicity
• If node.js must do whatever JavaScript does in the name of
compatibility, is node.js permitted its own values?
• This conundrum was on vivid display in the util.promisify() PR:
https://github.com/nodejs/node/pull/12442

34. https://github.com/nodejs/node/pull/12442

35. https://github.com/nodejs/node/pull/12442

36. Promises
• This is not an issue of someone being right or wrong!
• These are two different value systems — both admirable and
reasonable — coming into conflict
• This issue was put to a vote: 9 in favor, 1 against, 4 abstained
• But elections do not resolve differences in values
• Democracy does not necessarily imply leadership!

37. The challenge for Joyent
• We at Joyent are resigned to the divergence in values
• This doesn’t mean we’re abandoning node.js…
• …but it does mean that we are actively considering other
platforms when making future software platform decisions
• In the coming year(s), you will certainly see us in the Go
community — and perhaps Rust or others
• These communities aren’t necessarily a perfect reflection of our
values either — but we feel better prepared for any divergence

38. The challenge for node.js
• node.js should define what its values are — and aren’t
• Growth may be an objective, but growth is not a value!
• Claptrap around values that node.js doesn’t in fact have only
plants the seeds of disappointment and discord
• node.js should be upfront that compatibility with JavaScript will
trump robustness, debuggability, stability, etc.
• We remain hopeful that node.js will be a big enough tent to not
run so counter to our values that we abandon it entirely!

39. Platform as a reflection of values
• When picking a platform, values matter
• Get consensus on values before selecting a platform!
• With values in hand, the challenge becomes: how does one
determine the values of a community?
• How a community deals with an open issue reflects much on its
values around robustness, thoroughness, rigor, etc.
• It’s a balance between your values and that of a community…
• …that may require you to create your own platform and attract
your own community bound by like values!