1. Universal Userland
Why do we need to be different just like everyone else?
Engineering
seanc@joyent.com
Sean Chittenden
@SeanChittenden
2. Getting Started
• Packer - packer.io
• Image building utility for Vagrant (VMware, Virtualbox, bhyve), AWS, Triton, GCE, Azure
• An appreciation for version controlled software
• Pick your poison: git, hg, fossil
2
3. Agenda
1. Why Part 1
2. Bootstrap an image using Packer
3. Iterate using go-coreutils
4. Why Part 2
5. Iterate using rust-coreutils
6. Parting Thoughts
3
4. Why? Part 1: What is an OS?
• Is it a kernel?
• Is it a userland?
• Is it the applications?
• Is it a release?
• Is it a distribution?
• Is it an ethos and shared set of values?
• Is it a community?
4
5. What is an OS, Kernel Edition
• CDNs? Yes. "The network stack is everything."
• Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door."
• Embedded systems? Yes. "Userland is never installed."
• Hosting Providers? Yes. "Application containment."
• Network Security Deployments? Yes. "Hardened network stack."
• Users? If it's stable, "what's a kernel?"
• Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged."
• Developers? "That's why we're here (or to support the kernel)."
• Management? Maybe.
5
6. What is an OS, Kernel Edition
6
• CDNs? Yes. "The network stack is everything."
• Appliance vendors? Yes. "The license is everything. Userland doesn't make it out the door."
• Embedded systems? Yes. "Userland is never installed."
• Hosting Providers? Yes. "Application containment."
• Network Security Deployments? Yes. "Hardened network stack."
• Users? If it's stable, "what's a kernel?"
• Operators? Yes. "This is my primary toolbox and choice of toolboxes is how I avoid getting paged."
• Developers? "That's why we're here (or to support the kernel)."
• Management? Maybe.
YES
7. What is an OS, Userland Edition
7
• CDNs? Partially.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? Yes. "Familiarity for customers is important."
• Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code."
• Users? "I like my ls(1) and top(1)*."
* if [ "`uname -s`" = "SunOS" ]; then alias top="prstat"; fi
8. What is an OS, Userland Edition
8
• CDNs? Partially.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? Yes. "Familiarity for customers is important."
• Network Security Deployments? Yes. "Most tools run are OSS. Plus audited code."
• Users? "I like my ls(1) and top(1)."
• Operators? Yes. "This is how I deploy, maintain, investigate, and manipulate the kernel."
• Developers? It depends on who you ask. "Team effort!"
• Management? "Productivity? Effectiveness?"
12. What is an OS, Applications Edition
12
• CDNs? Partially. "We use nginx and/or varnish, so yeah."
• Appliance vendors? No. "Everything that matters is BSDL or proprietary already."
• Embedded systems? No. "What's a userland application?"
• Hosting Providers? Yes. "Customers can't be bothered to build 3rd party apps."
• Network Security Deployments? Yes. "Wireshark is an app, but tcpdump is kernel, right?"
• Users? "Give me X11, emacs/vi, and a webserver. Or give me death."
• Operators? Yes. This is how I deploy, maintain, investigate, and manipulate the kernel.
• Developers? "Well duh, how else would someone install Chromium?"
• Management? "Productivity? Effectiveness?"
14. What is an OS, Release Engineering Edition
14
• Is an OS the release and stability guarantee?
• CDNs? No.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? Yes. "IPU and upgrades are important: every customer is a snowflake."
• Network Security Deployments? No.
• Users? Yes. "IPU and upgrades on my laptop is important."
• Operators? Depends: legacy thinking, yes. Modern: not really.
• Developers? "We need to protect our users from themselves, so yes."
• Management? "Productivity?"
15. What is an OS, Distribution Edition
15
• Is an OS the distribution?
• CDNs? No.
• Appliance vendors? No.
• Embedded systems? No.
• Hosting Providers? No.
• Network Security Deployments? No.
• Users? Yes. "IPU and upgrades on my laptop is important."
• Operators? Depends: legacy thinking, yes. Modern: not really.
• Developers? "We need to protect our users from themselves, so yes."
• Management? "We don't have to do that work anymore, right?"
16. Bootstrap an Image
16
$ git clone -q https://github.com/sean-/BSDCan-2017.git
$ cd BSDCan-2017
$ make install-cfgt
go get -u github.com/sean-/cfgt
$ which packer envchain
/Users/seanc/go/bin/packer
/opt/local/bin/envchain
$ make packer-build TEMPLATE=freebsd-userland-go.json EXTRA_ARGS=-on-error=abort
cfgt -i freebsd-userland-go.json5 -o freebsd-userland-go.json
envchain tf_triton_test
packer build
-var "image_name=my-freebsd-image"
-var "image_version=1.0.0"
-on-error=abort
freebsd-userland-go.json
triton output will be in this color.
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Waiting for SSH to become available...
==> triton: Connected to SSH!
==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363
triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
17. Bootstrap an Image
17
triton output will be in this color.
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Waiting for SSH to become available...
==> triton: Connected to SSH!
==> triton: Provisioning with shell script: /var/folders/w0/tf58slqj3rs5mklbnvp5k7vh0000gp/T/packer-shell175541363
triton: FreeBSD freebsd-userland-go-builder 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0: Wed Feb 22 06:12:04 UTC 2017
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
triton: Checking integrity... done (0 conflicting)
[snip]
21. go-coreutils: Initial thoughts?
21
triton:
triton: Rebuilding locate database:
triton: flag provided but not defined: -128
triton: Usage of head:
triton: -bytes int
triton: output the first K bytes
triton: -c int
triton: output the first K bytes
triton: -help
triton:
triton: Usage: head [OPTION]... [FILE]...
triton:
triton: Print the first 10 lines of each FILE to standard output. With more than one FILE, precede each with a header
giving the file name. With no FILE, or when FILE is -, read standard input.
triton:
triton:
triton: Mandatory arguments to long options are mandatory for short options too.
triton:
triton: -help display this help and exit
triton: -version output version information and exit
triton:
triton: -c, --bytes=K
triton: output the first K bytes; or use -n +K to output starting with the Kth byte.
22. go-coreutils: Initial thoughts?
22
triton: -n, -lines=K
triton: output the first K lines; or use -n +K to output starting with
triton: the Kth
triton:
triton: -q, -quiet, -silent
triton: never output headers giving file names
triton:
triton: -lines int
triton: output the first K lines (default 10)
triton: -n int
triton: output the first K lines (default 10)
triton: -qnever output headers giving file names
triton: -quiet
triton: never output headers giving file names
triton: -silent
triton: never output headers giving file names
triton: -version
triton:
triton: head (go-coreutils) 0.1
triton:
triton: Copyright (C) 2014, The GO-Coreutils Developers.
triton: This program comes with ABSOLUTELY NO WARRANTY; for deprintTails see
triton: LICENSE. This is free software, and you are welcome to redistribute
triton: it under certain conditions in LICENSE.
triton:
23. go-coreutils: Initial thoughts?
23
• Needs to bake more
• Probably not going anywhere
• GPLv3ed code
• Suspicion: No commercial incentive or expectation of quality
25. Science: A lens through which we learn
• Whereas research has shown that to a considerable extent children learn by imitating the
behavior of adults, especially those they are dependent upon; and the use of corporal
punishment by adults having authority over children is likely to train children to use physical
violence to control behavior rather than rational persuasion, education, and intelligent forms
of both positive and negative reinforcement;
• Whereas research has shown that the effective use of punishment in eliminating
undesirable behavior requires precision in timing, duration, intensity, and specificity, as well
as considerable sophistication in controlling a variety of relevant environmental and
cognitive factors, such that punishment administered in institutional settings, without
attention to all these factors, is likely to instill hostility, rage, and a sense of powerlessness
without reducing the undesirable behavior;
• Therefore, be it resolved that the American Psychological Association opposes the use of
corporal punishment in schools, juvenile facilities, child care nurseries, and all other
institutions, public or private, where children are cared for or educated (Conger, 1975).
American Psychological Association
http://www.apa.org/about/policy/corporal-punishment.aspx
25
40. Software Usability?
40
Time
----------------
Time tolerance +/- 6 months, depending on which book/article you read; if it
was the announcement in Usenet or if it was available as tape.
[44B] McKusick, Marshall Kirk, Keith Bostic, Michael J Karels,
and John Quarterman. The Design and Implementation of
the 4.4BSD Operating System.
[APL] Apple website [http://www.apple.com/macosx/]
[BSDI] Berkeley Software Design, Inc.
[DFB] DragonFlyBSD Project, The.
[DOC] README, COPYRIGHT on tape.
[FBD] FreeBSD Project, The.
[KB] Keith Bostic. BSD2.10 available from Usenix. comp.unix.sources,
Volume 11, Info 4, April, 1987.
[KKK] Mike Karels, Kirk McKusick, and Keith Bostic. tahoe announcement.
comp.bugs.4bsd.ucb-fixes, June 15, 1988.
[KSJ] Michael J. Karels, Carl F. Smith, and William F. Jolitz.
Changes in the Kernel in 2.9BSD. Second Berkeley Software
Distribution UNIX Version 2.9, July, 1983.
[NBD] NetBSD Project, The.
[OBD] OpenBSD Project, The.
[QCU] Salus, Peter H. A quarter century of UNIX.
[SMS] Steven M. Schultz. 2.11BSD, UNIX for the PDP-11.
[TUHS] The Unix Historical Society. http://minnie.tuhs.org/Unix_History/.
[USE] Usenet announcement.
[WRS] Wind River Systems, Inc.
[dmr] Dennis Ritchie, via E-Mail
Multics 1965
UNIX Summer 1969
DEC PDP-7
First Edition 1971-11-03 [QCU]
DEC PDP-11/20, Assembler
Second Edition 1972-06-12 [QCU]
1965
46. Why? Part 2: Why the quagmire?
• Pluralistic Base Existed
• Ended up with a base that is only made up of self-bootstrapped bits
• Deprecated everything not C or sh(1)
• Deorbiting base to ports
• Why are base utilities limited to C and sh(1) again if we're building base with
poudriere(1)?
46
47. Automation to the Rescue
• Building releases image is now "easy" by comparison
• Debian GNU/kFreeBSD was a big effort
• Gentoo FreeBSD was a big effort
• Rust/FreeBSD? Pretty trivial
47
48. Bootstrap an Image
48
$ make packer-build TEMPLATE=freebsd-userland-rust.json EXTRA_ARGS=-on-error=abort
cfgt -i freebsd-userland-rust.json5 -o freebsd-userland-rust.json
envchain tf_triton_test
packer build
-var "image_name=my-freebsd-image"
-var "image_version=1.0.0"
-on-error=abort
freebsd-userland-rust.json
triton output will be in this color.
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Creating source machine...
==> triton: Waiting for source machine to become available...
==> triton: Waiting for SSH to become available...
==> triton: Connected to SSH!
[snip]
triton: gmake: Entering directory '/root/coreutils'
triton: cargo build --features "arch base32 base64 basename cat chgrp chmod chown chroot cksum comm cp cut dircolors
dirname du echo env expand expr factor false fmt fold groups hashsum head hostid hostname id install kill link ln logname
mkdir mkfifo mknod mktemp more mv nice nl nohup nproc od paste pathchk pinky printenv printf ptx pwd readlink realpath
relpath rm rmdir seq shred shuf sleep sort split stdbuf sum sync tac tail tee test timeout touch tr true truncate tsort
tty uname unexpand uniq unlink uptime users wc whoami yes" --release --no-default-features
triton: Updating registry `https://github.com/rust-lang/crates.io-index`
triton: Updating git repository `https://github.com/rust-lang/libc.git`
triton: Updating registry `https://github.com/rust-lang/crates.io-index`
51. Bootstrap an Image
51
triton:
triton: Rebuilding locate database:
$ echo It worked
ls(1), stat(1), and who(1) don't work and install(1) is missing options. Plus more WIP problems.
52. Parting Thoughts
• Swapping out syscall(2) tables is somehow okay with us
• Do we package each binary?
• Can we version binaries? We do shared objects and APIs
• Squatting on the filesystem is not versioning or an API
• What history? Nuke and pave. No instance up older than 3 years.
• Need tight coupling? Depend on a specific package.
• How can we embrace plurality (read: R&D)
• SmartOS and OmniOS uses different userlands, all slightly different
• What history? Nuke and pave. No instance up older than 3 years.
52
53. Parting Thoughts
• Writing BSD licensed software gives you access to the guild of like-minded
commercially motivated individuals who require:
• quality
• debugability
• observability
• performance
• shipability
53
54. Parting Thoughts
• Exporting an OSes base utilities takes excessive effort as the modern
programming languages have shown
• Embrace the future
• If we could start over knowing we had 10M LoC and >1K contributors, what and
how would we build an OS?
• If the ideal is different than what we're doing now, let's begin to plan how to get to
an ideal situation
Post Talk Comment: "This is like introducing the metric system to the UNIX countries
of the world."
54
This resonated with me