1. v
Open Network Install Environment (ONIE)
Webinar
Curt Brune, Member of Technical Staff
July 16, 2014
2. Webinar Agenda
ONIE Solves a Real Problem
ONIE Design Approach
ONIE Adoption
ONIE Technical Deep Dive
2cumulusnetworks.com
3. Vision
10+ years ago the compute side of the datacenter dramatically changed and
enabled high scale applications, but that change never made it to networking.
It’s time for networking without constraints.
Mission
To enable high capacity networks that are easy to deploy and affordable helping
customers realize the full promise of the software-defined data center.
Cumulus®
Linux®
Cumulus Linux is the first full-featured Linux OS for datacenter networking.
About Cumulus Networks
3cumulusnetworks.com
5. ONIE – Open Network Install Environment
§ Modern, efficient installer
§ Network OS installer
• Discovery, transfer, and execution
• Implemented through Linux kernel with BusyBox (OpenWRT-like)
§ Industry support
§ Enables open network hardware ecosystem
§ Open source
§ Well-documented
• http://www.onie.org/
cumulusnetworks.com 5
6. Challenges Facing Bare Metal Switching
§ Confluence of Difficulties
§ Each hardware platform is unique
§ Data center operators want to automate provisioning
§ End users want choice of HW and SW
§ Similar to server industry moving from various
hardware platforms (SUN, SGI, x86)
§ PXE was OK, but could be much more modern
cumulusnetworks.com 6
7. Create an Ecosystem for Bare Metal Switches
Bare Metal
Hardware
Platform
Operating
System A
Bare Metal
Hardware
Platform
Operating
System B
Bare Metal
Hardware
Platform
Operating
System C
• OS Vendors wrote their own
platform dependent OS installer
and integration layer
• Hardware Vendors maintained
separate SKUs for each
combination of OS and platform
• End Users could not change the
OS, it was an appliance
Before ONIE
The Appliance Model
Platform
Dependent
Platform
Dependent
Platform
Dependent
7cumulusnetworks.com
8. Create an Ecosystem for Bare Metal Switches cont.
Bare Metal
Hardware
Platform
ONIE
Operating
System A
Bare Metal
Hardware
Platform
Operating
System B
Bare Metal
Hardware
Platform
Operating
System C
• Hardware Vendors maintain a
single ONIE SKU for each hardware
platform. Opens up the
distribution channels
• OS Vendors target a common
installer format. Speeds time to
market
• End Users can pick and choose
among a variety of Operating
Systems. Automated mega-scale
OS and hardware provisioning is
now possible
ONIE Smooths out the Rough Edges
ONIE ONIE
8cumulusnetworks.com
9. ONIE Innovation
§ A New Approach
§ More modern than PXE
§ Pre-installed on the switch by the hardware vendor
§ A mini operating system based on Linux
§ Could have enhanced BIOS or U-Boot, but Linux is way
more capable: device drivers, networking, applications
§ Support IPv6, HTTP/S, DHCP and existing TFTP infra
§ Designed for Automation
§ Additional meta-data during network installs
cumulusnetworks.com 9
10. ONIE Adoption
§ Time line
§ Late 2012, brainstormed initial features
§ Early 2013, evangelized with HW partners
§ May 2013, first public demo at OCP MIT Workshop
§ Summer 2013, first products available from multiple
vendors
§ Summer 2013, project incubated by OCP
§ June 2014, project fully adopted by OCP
cumulusnetworks.com 10
13. § Standardize ONIE Implementations …
§ UTSA OCP Certification Laboratory
§ Collaborate on test plans
§ Help develop test suites
§ Help HW vendors certify their ONIE
implementation
§ Benefit OS vendors
§ Improve the quality of the ONIE project
OCP – ONIE Certification Program
13cumulusnetworks.com
15. Technical Deep Dive
§ How Does ONIE Work?
§ Leverages the Linux kernel and BusyBox
§ A few state machines to manage network interfaces
and image discovery methods
§ Shell scripting is the programming language
15cumulusnetworks.com
16. ONIE – Anatomy of a Network Switch
( Management Interfaces ) ( Data Plane )
CPU
SoC
DRAM
Boot
Flash
Mass
Storage
Switching
ASIC
Serial
Console
Ethernet
Mgmt Port
10Gb
Port
40Gb
Port…
10Gb
Port
40Gb
Port
…
PCIe
16cumulusnetworks.com
17. ONIE – Uses Management Interfaces
( Management Interfaces ) ( Data Plane )
CPU
SoC
DRAM
Boot
Flash
Mass
Storage
Switching
ASIC
Serial
Console
Ethernet
Mgmt Port
10Gb
Port
40Gb
Port…
10Gb
Port
40Gb
Port
…
PCIe
17cumulusnetworks.com
18. ONIE – Bare Metal Install – First Time Boot Up
Boot Loader
(HW Vendor Supplied)
ONIE
(HW Vendor Supplied)
Installer
(OS Vendor)
Boot Loader
• Low Level boot loader, configures CPU complex
• Loads and boots ONIE
ONIE
• Linux Kernel with Busybox
• Configures management Ethernet interface
• Locates and executes an OS installer
• Provides tools and environment for installer
OS Installer
• Available from network or USB
• Linux executable
• Installs vendor OS into mass storage
Network OS
(OS Vendor Supplied)
Fetches
Installs
18cumulusnetworks.com
19. ONIE – Subsequent Reboots – Vendor’s OS is Already Installed
Boot Loader
(HW Vendor Supplied)
ONIE
(HW Vendor Supplied)
Boot Loader
• Low Level boot loader, configures CPU complex
• Loads and boots OS vendor’s installed OS
Network OS
• Configures Switching ASIC
• Runs Network Protocols
• Provides CLI
Network OS
(OS Vendor Supplied)
ONIE
• Still exists, but is not used
• Available for uninstall / re-install operations
19cumulusnetworks.com
20. ONIE – Network OS Installer Discovery and Install Behavior
Configure Network
Interface
Locate Installer
Run Installer
• Uses DHCPv4, DHCPv6
• Configures Ethernet interface for IPv4 / IPv6
• Configures DNS and hostname
• Determines the location of an installer executable
• Examines local file systems, e.g. USB flash drives
• Uses DHCP options, DNS Service Discovery, Multicast
DNS and IPv6 Neighbors
• Downloads installer via URL
• Passes various environment variables to
installer
• Launches installer
20cumulusnetworks.com
21. ONIE – Discovery Waterfalls
§ Management interface
§ IPv6 network discovery
§ DHCPv4
§ Static IP address
cumulusnetworks.com 21
1 § OS installation source
§ Local USB
§ HTTP
§ FTP
§ TFTP
2
§ File name search
§ onie-‐installer-‐<arch>
§ onie-‐installer-‐<vendor>
§ onie-‐installer
26. ONIE – Other Behaviors
§ Provides a mechanism for a network OS to invoke
the above behaviors.
Reinstall Remove currently installed OS and return to the
“out of box” provisioning state
Uninstall Completely wipe out everything, except ONIE
Rescue Reboot box into ONIE for repair, debug and
forensics
Update Install a new ONIE version
Diag Run HW Vendor’s diag (optional)
26cumulusnetworks.com
27. ONIE – Ongoing Development
§ Support Multiple CPU Architectures
§ PowerPC – Ready Today
§ x86 – Ready Today
§ x86 Virtual Machine
§ Planning for ARM and MIPS
§ Maintain ONIE behaviors across architectures
27cumulusnetworks.com
