Linux kernel status in RISC-V (July 2018) presented at RISC-V Chennai workshop https://content.riscv.org/wp-content/uploads/2018/07/0930-19.07.18-Atish-Patra-Western-Digital.pdf

1. Linux Kernel on RISC-V:
Where do we stand ?
Atish Patra, Principal R&D Engineer
Damien Le Moal, Director, System Software Group
7/19/2018©2018 Western Digital Corporation or its affiliates. All rights reserved.

2. • Software ecosystem status overview
– Development toolchain
– Linux distributions
– Linux kernel
• What’s next ?
• Our learnings
• Contribution process
• Summary
Overview
7/19/2018 2©2018 Western Digital Corporation or its affiliates. All rights reserved.

3. Why Linux ?
Linux
96%
Others
4%
TOP 1M WEBSITES
7/19/2018 3©2018 Western Digital Corporation or its affiliates. All rights reserved.
Linux
50%
Others
50%
X86 SERVER
Linux
72%
Others
28%
IOT DEVICES
Android
77%
Others
23%
MOBILE DEVICES

4. Software Ecosystem Overview
7/19/2018 4©2018 Western Digital Corporation or its affiliates. All rights reserved.
User Space
Glibc, core libraries
Linux Kernel
RISCV CPU
Core Device drivers (CPU management, interrupt
management, ...)
Toolchainsupportand
optimization
CPU extensions device drivers
Bootloader(BBL)
Supervisor Binary Interface (SBI)
Memory Management (MMU, IOMMU)
Linux
Distribution
Applications

5. Jan 18
Glibc
Software Ecosystem Growth
7/19/2018 5©2018 Western Digital Corporation or its affiliates. All rights reserved.
March 17
Binutils
May 17
GCC
Nov 17
4.15 Linux
Kernel
March 18
QEMU
Feb 18
Debian
port start
March 18
Fedora bootstrap
done

6. • Almost Complete Development Environment
– GCC 7.3.1
– Perl 5.26
– Python 2 and 3
– Git
– Systemd
– LLVM
Development toolchain
7/19/2018 6©2018 Western Digital Corporation or its affiliates. All rights reserved.

7. • 1st Linux distro available on RISC-V
• Fedora 29 available
• The koji build farm
– Contains 3 boards + 6 QEMU VMs
– 3000+ builds a week, 500+ a day
– The best was 85+% success rate for a week
• 22347 number of packages are built
Linux Distributions: Fedora
7/19/2018 7©2018 Western Digital Corporation or its affiliates. All rights reserved.

8. • More than 80% Debian packages are built for RISC-V
• Higher than Itanium!!
• More than 9k of ~13k arch-dependent packages built
Linux Distributions: Debian
7/19/2018 8©2018 Western Digital Corporation or its affiliates. All rights reserved.
March 2018 June 2018

9. Demonstration (Thanks to Palmer)
7/19/2018 9©2018 Western Digital Corporation or its affiliates. All rights reserved.
• So what Can I do on a RISC-V
desktop ?
– Browsing
– 3D gaming
– Chat/Tweet
– Crypto currency mining ???

10. • What is SBI ?
– OS interface for Supervisor Execution Environment(SEE)
– Equivalent to microcode implementation
– Implemented by Berkely Boot Loader (bbl)
– Provides cleaner interface for Supervisor OS (i.e. Linux)
• Interface functions
– Set Clock events
– Send/Clear Inter Processor Interrupts(IPI)
– Serial Console Get/Update
– Remote Memory barrier (aka fence)
Kernel Status: Supervisor Binary Interface (SBI)
7/19/2018 10©2018 Western Digital Corporation or its affiliates. All rights reserved.

11. Kernel Status: Core Features
7/19/2018 11©2018 Western Digital Corporation or its affiliates. All rights reserved.
• UMP/SMP Support
• Interrupt Controller
• Clock driver
• Virtual memory support
• Serial Driver
• System call/trap handling
• Kernel module support
• CPU hotplug support

12. Kernel Status: RISC-V Boot Process
7/19/2018 12©2018 Western Digital Corporation or its affiliates. All rights reserved.
• No dedicated boot CPU
• A lottery based mechanism to select the boot CPU
• Non-boot CPUs boot in random order
• ~100 lines of assembly instructions
• All device initialization depends on device-tree
• Device tree is read from ROM & passed to kernel by BBL

13. Kernel Status: Interrupt Management
7/19/2018 13©2018 Western Digital Corporation or its affiliates. All rights reserved.

14. • One timer per hart
• Per CPU based clock event
• Clockevents are initialized via CPU hotplug during SMP bring up
• One shot timer
• Every clock event reprograms next one via SBI call
• All the timers across all the harts are synchronized within one tick of
each other
• Tickless kernel (NOHZ) support is also enabled
Kernel Status: Timer Management
7/19/2018 14©2018 Western Digital Corporation or its affiliates. All rights reserved.

15. Kernel Status: CPU Hotplug
7/19/2018 15©2018 Western Digital Corporation or its affiliates. All rights reserved.
Offline
Online
CPU hotplug core
CPU hotplug core
subsy1 subsy2 subsy3 subsy4 ….
User space Kernel
CPU offline
CPU online
__cpu_disabl
e
__cpu_die
__cpu_up
Arch
WAIT on
WFI
Issue an IPI
boot
cpu

16. Kernel Status: CPU Hotplug
7/19/2018 16©2018 Western Digital Corporation or its affiliates. All rights reserved.

17. • Ftrace
– Normal trace
– Function graph tracing
– Function profiling
– Filtering
– No kprobes
• Perf
– Instructions count
– Cycle count
• GDB port is in progress
Kernel Status: Tracing/Debugging
7/19/2018 17©2018 Western Digital Corporation or its affiliates. All rights reserved.

18. • High performance Java support
– OpenJDK port by J extension group going on
– Jikes RVM going on. Not yet public
• In progress
– Rust
– GO, Clang
• JavaScript
– V8
– Node.js
• Others
– Dart, Ruby
– GTK, OpenCV
What’s next – User space
7/19/2018 18©2018 Western Digital Corporation or its affiliates. All rights reserved.

19. • Independent boot loader(uboot or coreboot)
• SBI extension for power management/virtualization
• CPU topology
• Multi-level interrupt controller through irq domains (in progress)
• CPU power management
• Context tracking for NO_HZ_FULL
• Precise IRQ time accounting
• IRQ stats update for Timer/IPI interrupt
What’s next – Core Linux Kernel
7/19/2018 19©2018 Western Digital Corporation or its affiliates. All rights reserved.

20. • Specification required
– IOMMU
– Performance Monitor counters
– Virtualization support
• Not useful at this time due to lack of hardware
– Numa
– Memory hotplug
– Huge pages
What’s next – Core Linux Kernel
7/19/2018 20©2018 Western Digital Corporation or its affiliates. All rights reserved.

21. • Kexec
• Kdump
• Lockdep
• Kasan
• Kprobes
• kernel function-return probes
• KGDB
• BPF
What’s next – Tracing/Debugging
7/19/2018 21©2018 Western Digital Corporation or its affiliates. All rights reserved.

22. • QEMU verification is very handy
• NFS & chroot in QEMU brings up the Fedora/Debian without any costly
hardware!!
• Remote gdb debugging possible in QEMU
• Print device tree in bbl
• Use Ftrace to analyze kernel issues
• Print to debug kernel
• Dumping dmesg in QEMU
• Inspecting registers in QEMU
Our Learnings
7/19/2018 22©2018 Western Digital Corporation or its affiliates. All rights reserved.

23. • RISC-V port available in Mainline Linux since v4.15
• The RISC-V development tree
– kernel.org/palmer/riscv-linux.git
– master: a copy of Linus’ master, up to the latest RC.
– for-linus : RISC-V branch that’s pulled into Linus’ master branch
– for-next : RISC-V branch that’s pulled into linux-next
– riscv-all : RISC-V integration branch
• Patches should be sent to
– linux-riscv@lists.infradead.org
• IRC Chatrooms on freenode
– #linux-riscv
– #riscv
Contribution Process
7/19/2018 23©2018 Western Digital Corporation or its affiliates. All rights reserved.

24. Q&A
atish.patra@wdc.com
7/19/2018 24©2018 Western Digital Corporation or its affiliates. All rights reserved.
Atish Patra

25. 7/19/2018©2018 Western Digital Corporation or its affiliates. All rights reserved. 25