It describes the difference between ARMv7 and ARMv8, how to port Linux and Debian on ARM64 development board and how to write ARM64-safe code.

1. Embedded Linux/ Debian
with ARM64 Platform
Harder Better Faster Stronger
SZ LIN
szlin@debian.org
1
Date: 2016/11/17

2. 2
SZLIN (林上智)
➔ Debian Developer
➔ Industrial Grade Linux Distribution
➔ Blog: https://szlin.me
About Me

3. 3
Trends
That’s really what ARM has to look at when
defining a new architecture. That is the nature of
our business, we need to look a long way forward,
and plan.
Why ARMv8 instead of using ARMv7+ LPAE [11]

4. 4

5. 5
ARMv7 -> ARMv8 [2]
32-bit ARMv7-A
compatibility
Cryptography support
Dual Instruction set
architecture
Improved support for
virtualization

6. 6
AArch32 (compat) Support [8]
Support ARMv7 Linux EABI for compat tasks
• Different set of system calls (unistd32.h)
• Compat user structures
• No SWP instruction, no unaligned LDM/STM
access
Supports both ARM and Thumb-2 32-bit user
tasks
Supports 32-bit ptrace
Address space limited to 4GB
Emulated vectors page
• ARM Linux EABI expects helper routines in the
vectors page accessible by user tasks

7. 7
AArch32 (compat) Support
arch/arm64/kernel/ptrace.c
arch/arm64/include/asm/unistd32.h
arch/arm64/include/asm/unistd.h

8. 8
ARMv8-A Designed for Efficiency [2]
Enhancement Why it Matters
64-bit architecture Efficient access to large datasets
Increased number and size of general
purpose registers
Gains in performance and code
efficiency
Double the number and size of NEON
registers
Enhanced capacity of multimedia
engine
Cryptography support Over 10x software encryption
performance New security models for
consumer and enterprise

9. 9
ARMv8-A Family [3]
Core Instruction cycle DMIPS/MHz
Processor
Architecture
big.LITTLE role
Cortex-A32 In-order execution 2.0 ~ 2.1 32 Unknown
Cortex-A35 In-order execution 1.78 64 little
Cortex-A53
In-order execution 2.24 64
little
Cortex-A57 Out-of-order
execution
4.6 64
big
Cortex-A72 Out-of-order
execution
4.72 64
big
Cortex-A73 Out-of-order
execution
4.8 64 big

10. 10
The ARM Registers Comparison [4]
AArch32 AArch64 Remarks
32-bit General purpose registers r0 - r14 w0 - w30
aliased to lower half of each
64-bit register in AArch64
64-bit General purpose registers - x0 - x30
Program counter r15 aka. pc PC
not directly accessible in
AArch64
Zero register -
x31 / w31 aka. xzr /
wzr
in most instructions, except
as a base register for address
generation
Stack pointer r13 SP
encoded as x31 on address
generation
Procedure call link register r14 x30

11. 11
ARMv7
name
AArch64/ARMv8
name
Remarks
Monitor mode EL3
highest exception level,
mostly for firmware
HYP mode EL2
exception level for
hypervisors like Xen (or
parts of KVM)
SVC mode EL1
the Linux kernel is
running in this
USR mode EL0 for unprivileged userland
The ARM Execution Mode [4][8]

12. 12
Memory Layout [15]
AARCH64
It allows up to 4 levels of translation
tables with a 4KB page size and up to
3 levels with a 64KB page size.

13. 13
Memory Layout [9][15]
32 bit address space:
• 4GB of virtual addresses
• 1MPages
• Each entry is 4 bytes (a 32 bit physical
address)
• Page size : 4kb

14. 14
Memory Layout [14][15]
AArch64 Linux uses either 3 levels or 4 levels of
translation tables with the 4KB page
configuration, allowing 39-bit (512GB) or 48-
bit (256TB) virtual addresses, respectively, for
both user and kernel.
64 bit address space
• 16 exabytes(1018) of virtual address space
• 4PetaPages (1015)
• Entry is 8 bytes
• Page size : 4kb/ 64kb

15. 15
Memory Layout [14][15]

16. 16
Memory Layout [14]
With 64KB pages, only 2 levels of translation
tables, allowing 42-bit (4TB) virtual address,
are used but the memory layout is the same.

17. 17
The ARM64 with Linux Kernel [5]
First patch in Kernel 3.7
 Without real hardware
First ARM64 SoC supported in Kernel 3.11
 2013/6
Second ARM64 SoC supported in Kernel 3.18
 2014/12
Different assembly code/ ABI between ARM32/64
AArch64 is the name for the 64-bit ARM architecture

18. 18
The ARM64 with Linux Kernel [5]
 Device tree
 arch/arm64/boot/dts/<vendor>/
 Compressed Kernel
 arch/arm64/boot/Image.gz
 No more “arch/arm64/mach-<soc>” [7]
 No more C files, all SoC features should be handled by drivers
 Or it can go “drivers/soc” if it cannot fit an existing drivers
 No more uImage [7]
 Incompatible with multiplatform kernels
 Switch to bootz command in bootloader

19. 19
The ARM64 with Debian [6]
First release in Debian 8
AARCH64 Toolchain in Debian
 Package: crossbuild-essential-arm64
 aarch64-linux-gnu-gcc
Build ARM64 Debian root filesytem via debootstrap

20. Hands on experience
Build Debian on AARCH64
20

21. 21
The QorIQ® LS1043A reference design board [13]

22. 22
The QorIQ® LS1043A reference design board [13]

23. 23
Cross Tool chain in Debian

24. 24
Build Kernel 4.1…with Error
root@szlin:/home/szlin/work/Kernel/moxa_linux/ls1043a# make -j30
CHK include/config/kernel.release
CHK include/generated/uapi/linux/version.h
HOSTCC scripts/kallsyms
CC scripts/mod/empty.o
CC scripts/mod/devicetable-offsets.s
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/modpost.o
HOSTCC scripts/mod/sumversion.o
In file included from include/linux/compiler.h:54:0,
from include/uapi/linux/stddef.h:1,
from include/linux/stddef.h:4,
from ./include/uapi/linux/posix_types.h:4,
from include/uapi/linux/types.h:13,
from include/linux/types.h:5,
from include/linux/mod_devicetable.h:11,
from scripts/mod/devicetable-offsets.c:2:
include/linux/compiler-gcc.h:121:30: fatal error: linux/compiler-gcc6.h: No such file or directory
#include gcc_header(__GNUC__)
^

25. 25
Build Kernel 4.1 – Patch with GCC6 issue

26. 26
Build Kernel 4.1 – LZ4 is not existed in ARM 64
LZ4 is existed in ARM Kernel 4.1
But not in ARM 64… it needs patch !

27. 27
Build uImage – Failed in ARM64 [17]
> Bringing in uImage as a target on ARM might have
been a mistake, since
> now people want to bring in new targets for whate
ver boot format they
> happen to want. So it's a good idea to keep arm64 f
rom going down the
> same path and stick to the native, raw, formats.
mkimage -A arm64 -O linux -T kernel -C gzip -a 0x80080000 -e 0x80080000 -
n 4.1.0-ltsi-rt+ -d arch/arm64/boot/Image.gz uImage
No, there should be no uImage target on
arm64. Newer u-boots can
already boot an ARM zImage (through the
bootz command), and should be
enhanced as needed to boot a vmlinux if
needed.
However, we can still make uImage manually

28. 28
Configure system
account
Configure pure
root filesystem
4321
Have fun with Debian
Steps to Build Your Debian ARM
Pre-download pure
root filesystem

29. 29
Configure system
account
Configure pure
root filesystem
4321
Have fun with Debian
Steps to Build Your Debian ARM
Pre-download pure
root filesystem

30. 30
Pre-download pure root filesystem
1. Build pure Debian root filesystem via “debootstrap” command
debootstrap --arch=arm64 --foreign --keyring /usr/share/keyrings/debian-archive-keyring.gpg --
exclude=debfoster jessie *DIRECTORY*
Note:“man debootstrap” to get further information
2. Download ARM64 “sudo” package
 https://packages.debian.org/jessie/sudo
3. Put “sudo” package into *DIRECTORY*/var/cache/apt/archives/
DEMO: https://github.com/szlin/Pure_Debian_aarch64.git

31. 31
Configure system
account
Configure pure
root filesystem
4321
Have fun with Debian
Steps to Build Your Debian ARM
Pre-download pure
root filesystem

32. 32
Linux Kernel
and device tree
Debian root filesystem
Tiny root filesystem
Non-volatile memory
setenv bootargs "console=ttyS0,115200
root=/dev/mmcblk0p3 rw rootdelay=5
earlycon=uart8250,0x21c0500,115200"
fatload mmc 0 0xa0000000 uImage
fatload mmc 0 0x90000000 fsl-ls1043a-rdb.dtb
bootm 0xa0000000 - 0x90000000
Set root = tiny root filesystem in first boot
up

33. 33
Boot up the device via manual
command

34. 34
ln -sf /dev/null /dev/tty3
ln -sf /dev/null /dev/tty4
ln -sf /dev/null /dev/tty2
Mount Debian partition and chroot to debian

35. 35
Start to configure Debian rootfilesystem in second stage
Unnecessary files will removed afterwards

36. 36
Configure system
account
Configure pure
root filesystem
4321
Have fun with Debian
Steps to Build Your Debian ARM
Pre-download pure
root filesystem

37. 37
Create “sudo” account
DEMO: https://github.com/szlin/Debian_aarch64.git

38. 38
Configure system
account
Configure pure
root filesystem
4321
Have fun with Debian
Steps to Build Your Debian ARM
Pre-download pure
root filesystem

39. 39
Linux Kernel
and device tree
Debian root filesystem
Tiny root filesystem
Non-volatile memory
setenv bootargs "console=ttyS0,115200
root=/dev/mmcblk0p2 rw rootdelay=5
earlycon=uart8250,0x21c0500,115200"
fatload mmc 0 0xa0000000 uImage
fatload mmc 0 0x90000000 fsl-ls1043a-rdb.dtb
bootm 0xa0000000 - 0x90000000
Set root = Debian root filesystem in
bootloader

40. 40
Login and check “sudo” mechanism

41. 41
ARM 64 Programming

42. 42
ARM 64 Programming [16]
What is the different between 32 and 64 bits in C

43. 43
ARM 64 Programming [16]
Pre-defined macros relating to ARMv8 compilation

44. 44
ARM 64 Programming [16]
Bit manipulation operations
Any issue?

45. 45
ARM 64 Programming [16]
Bit manipulation operations
“1” has int type…

46. 46
ARM 64 Programming [16]
Indexes
Any issue?

47. 47
ARM 64 Programming [16]
Indexes
size_t: 64 bit
unsigned int: 32 bit
Infinite loop

48. 48
ARM 64 Programming [16]
Pointers and ints
Is p = q?

49. 49
ARM 64 Programming [16]
Pointers and ints
1. For ARM32
 p==q
2. For ARM64
 p!=q

50. 50
ARM 64 Programming [16]
Structure padding
What is the size
of structure in C ?

51. 51
ARM 64 Programming [16]
Structure padding
1. For ARM32
int a : 4
long b: 4
int c: 4
Total = 12 bytes
2. For ARM64
int a : 4
long b: 8
int c: 4
Total = 20 bytes

52. 52
ARM 64 Programming [16]
Structure padding
1. For ARM64
int a : 4
long b: 8
int c: 4
Total = 20 bytes
For double-word aligned, it needs to introduce four
bytes of padding between the end of the first ”int” and
the ”long”

53. 53
Thank you

54. 54
References
1. AArch64 planning
https://www.einval.com/~steve/talks/Debconf12-aarch64
2. ARMv8-A
http://media.corporate-ir.net/media_files/IROL/19/197211/ARMv8-
A%20IR%20webcast%2024_03_2014.pdf
3. Comparison of ARMv8-A cores
https://en.wikipedia.org/wiki/Comparison_of_ARMv8-A_cores
4. Arm64
http://linux-sunxi.org/Arm64
5. ARM64 SoC checklist
http://events.linuxfoundation.org/sites/events/files/slides/clement-arm64-soc-checklist.pdf
6. ARM64Port
https://wiki.debian.org/Arm64Port
7. AArch64 kernel image decompression
http://lists.infradead.org/pipermail/linux-arm-kernel/2014-January/225354.html
8. Linux on AArch64 ARM 64-bit Architecture
https://events.linuxfoundation.org/images/stories/pdf/lcna_co2012_marinas.pdf
9. Migrating code from ARM to ARM64
https://www.linuxplumbersconf.org/2014/ocw/system/presentations/2343/original/08%20-
%20Migrating%20code%20from%20ARM%20to%20ARM64.pdf

55. 55
References
10. ARM64 vs ARM32 -- What's different for Linux programmers
http://edn.com/design/systems-design/4440662/ARM64-vs-ARM32-What-s-different-
for-Linux-programmers
11. ARMv8
https://www.arm.com/files/downloads/ARMv8_white_paper_v5.pdf
12. Debian
https://www.debian.org
13. LS1043a – RDB
http://www.nxp.com/products/microcontrollers-and-processors/arm-processors/qoriq-
arm-processors/qoriq-ls1043a-reference-design-board:LS1043A-RDB
14. Memory Layout on AArch64 Linux
https://www.kernel.org/doc/Documentation/arm64/memory.txt
15. Virtual memory
https://cseweb.ucsd.edu/classes/wi11/cse141/Slides/19_VirtualMemory.key.pdf

56. 56
References
16. Porting to 64-bit ARM
http://malideveloper.arm.com/downloads/Porting%20to%20ARM%2064-bit.pdf
17. AArch64 kernel image decompression
http://lists.infradead.org/pipermail/linux-arm-kernel/2014-January/225354.html
18. arm64: enable more compressed Image formats
https://patchwork.kernel.org/patch/6810841/
19. Fix-the-compile-issue-under-gcc6
https://raw.githubusercontent.com/Freescale/meta-freescale-
3rdparty/master/recipes-kernel/linux/linux-variscite-4.1.15/Fix-the-compile-issue-
under-gcc6.patch